Weekend Project VR/CV/RC-Car: Part 1

Another weekend project, this time a bit bigger and counting with my son participation. Five year old boy is already eager to drive this toy.

The Idea

Been a race enthusiast and eager to create something challenging while doing something with my son. Looking around the house found lots of equipment not used at the moment and then the idea popped in my head: Lets mix and match all this:

Drive a powerful RC car in FPV mode with OpenCV to help avoid obstacles. Using a VR device to drive remotely.

Example Image: While in VR you will see the real environment

Why doing this?

In the market we can find few toys with the same purpose, Mario Kart, FPV Drones and others. This article is about DIY from scratch, add mods and have fun while doing it. Remember this is a weekend coding session!

Challenges

• Shaky camera feed, the car is fast and tough, this means one stiff suspension. For initial tests no dampening will be used. But it already sounds we need to add something to help with vibration (a different type of support for the iPhone inside the car).
• Delay, racing is all about reaction. Getting the image from Camera, transmit over the air, get into the VR. All those steps sound like too many milliseconds wasted. We can face problems with a car like this. Maybe some other solution drones use for it.

What will be used for this project

• Samsung S7 for Camera and 4G connection
• Oculus Quest 1
• Arduino Wifi or Bluetooth
• Old 1/10 scale Serpent S411 with 10.5T Motor
• WebRTC to transmit data/video/audio

Equipment to use, Oculus Quest, Serpert S411, Arduino Uno

Initial Test

How the phone will be placed inside the car? Will it fit, will it work as expected? Some cutout on the car body will be required, the original plastic is not fully transparent. See the next image:

iPhone 11Pro inside the car

Next Steps

A lot of testing, measuring, cutting was done to check if the phone could actually be used in this scenario. It sounds feasible and now time to get into details. The start point will be to transmit data from phone to the VR session.

References

Check here for the progress: https://github.com/jeffscm/driving

Links used in this project:

• https://assetstore.unity.com/packages/tools/integration/opencv-for-unity-21088
• https://assetstore.unity.com/packages/tools/integration/oculus-integration-82022
• https://assetstore.unity.com/packages/tools/network/webrtc-video-chat-68030

And feel free to ask/suggest ideas!

Part 2: Phone to VR connection (soon)

Part 3: Arduino Setup and Let's Race!!! (near future)