DeepRacer: How to master physical racing?
As developers gear up for re:Invent 2024, they again face the unique challenges of physical racing. What are the obstacles? Let's have a look...
Published Nov 19, 2024
Last Modified Nov 30, 2024
In this article I will look at what makes physical DeepRacer racing - a real car on a real track - different to that of the virtual world - a model in a simulated 3d environment. We will cover the basics, the differences virtual vs. physical, and what steps I have taken to get a deeper understanding of the challenge.
The DeepRacer League is wrapping up. In a few weeks 32 racers will face off in Las Vegas for one last time. This year the qualification has been all-virtual, so the transition from virtual to physical racing will be a challenge.
DeepRacer relies on the racer training a model within the simulator, a 3D environment built around ROS and Gazebo, originally built on AWS RoboMaker.
The trained model is subsequently utilized for either virtual or physical races. The model comprises of a convolutional neural network (CNN) and an action space translating class labels into speed and throttle movement. In the basic scenario involving a single camera only, a 160x120 pixels, 8-bit grayscale image is captured 15 times a second, passed through the neural network and the action with the highest weight/probability is executed.
The small piece of AI-magic is that during model evaluation ("racing") there is no context; each image is processed independently of the image before it, and without knowledge of the state of the car itself. If you process the images in reverse order the results remain the same!
The virtual worlds are 3D worlds created in Gazebo, and the software is written in Python and C++ using ROS as the framework. The 3D simulation is fairly 'flat', with basic textures and surfaces. There is little or no reflections or shine, and the environment is as visually clean as you make it. Input images are captured 15 times per second.
Within this world a small car is simulated. Compared to a real car the model is really basic and lacks quite a few of the things that makes a real car work: There is no suspension, the tires are rigid cylinders, there is no Ackermann steering, and there are no differentials. It is almost surprising that this car can drive at all. On the positive side the camera is 'perfect'; irrespective of lighting conditions you get crisp clear pictures with no motion blur.
A typical virtual car drives at speeds between 0.5 m/s and 4.0 m/s, depending on the shape of the track. If you go too fast, it will often oversteer and spin out of the turn due to the relatively low grip.
The real world is less perfect - sim-to-real gap #1 is around visual noise created by light, reflections (e.g. if track is printed on reflective material) and background noise (e.g. if the barriers around the track are too low, and the car sees people and objects in the back). Input images are captured 30 times per second.
The car itself - based on the readily available WLToys A979 - has all the things the model car doesn't; proper tires, suspension and differential. One problem is that the car is heavy - around 1.5kg - and the placement of some components causes the center of gravity to be very high. This causes a sim-to-real gap #2: Roll and pitch during corners at high speeds cause the camera to rotate, confusing the neural network as the 'horizon' moves.
Gap #3 comes from motion blur when the light is too dim - the blur can cause the dashed centerline to look like a solid line, making it hard to distinguish the centerline from the solid inner and outer lines.
The steering geometry, the differentials, the lack of engineering precision of the A979, and the corresponding difficulty in calibrating it, causes gap #4. Even if the model wants to go straight the car still pulls left or right, needing constant correction to stay on track. You can best see this as the car is often unable to drive down the straights in a straight line.
The original DeepRacer, without modifications, has a smaller speed range of around 2 m/s. It has a better grip but suffers from the above mentioned roll movements. If you go too fast, it will understeer and potentially roll over. Since 2023 the AWS pit-crews operate their fleets of DeepRacers with shock spacers to stiffen the suspension, reduce the roll, and increase the max effective speed.
Looking at the sim-to-real gaps there are four questions what we want to explore:
- How can we train the model to better handle the real world? This covers altering the simulator to close some of the gaps, combined with adapting reward function, action space and training methodology to make better use of this simulator.
- How can we better evaluate what the car does, and why? In the virtual world we can perform log analysis to investigate, in the real world this has not yet been possible.
- How can we evaluate our newly trained models? A standard DeepRacer track is, with its size of 8m x 6m, prohibitively big. Is it possible to downscale the track to fit at home?
- Will a modified car perform better? Pimp my DeepRacer with better shocks? Add ball bearings and shims to improve steering precision? Or build a new lighter car based on a Raspberry Pi?
To answer the questions, some solutions are required to support the experiments. The following will assume you are using Deepracer-for-Cloud to run the training locally or in an EC2 instance. We will not go into the details but provide references that will enable the reader to try things out on their own.
The first thing to look at was how I could alter the simulator. The simulator code is available, and tweaking does not require too many skills. We can choose to alter the car and the physics of the world, or adjust the visual environment.
Changing the environments means altering the 3D world. It can be done through altering the features in a pre-existing track, in ways such as adding/removing track parts like lines, changing lighting, adding background features (walls, buildings), swapping out textures and so on. Making changes to the world will require building a new docker image, which can take quite some time - but luckily there are ways to speed that up. Going a step further it is also possible to make the world programmatically (command line or code) alterable during run-time.
The starting point are the track COLLADA (
.dae) files found in the meshes folder. You can import it into Blender, make your changes, and export the file again. Note that lights and camera positions from Blender are not considered by Gazebo. To alter the lightning conditions you will have to alter the .world file in the worlds - the files are XML files in sdformat.
The competition cars owned by AWS are off-limits, so the objective of tuning the car in the simulator is to make it behave in ways more similar to the real one. Trained Neural Networks have an embedded expectation of what will happen next; it learned that by taking a specific action, it would get a turn of a given radius. If the simulator car steers more or less than the physical one in a given situation, things become unpredictable...
Lack of Ackermann steering, no differentials, but wheels that can deflect up to 30 degrees - the real ones only go to a bit more than 20 degrees 'outwards', and less than that 'inwards' - my experience is that the real car surprisingly enough still has a shorter turning radius than the virtual one...
The car models are found in the urdf folder. There are three different cars, relating to the different version of physics which you configure in your actions space (
model_metadata.json). Today only the deepracer (v3/v4 physics) and deepracer_kinematics (v5 physics) models are relevant. There are variant models for single camera, stereo camera, both with and without the LIDAR.Each physics version is different - the big question is if that has an impact on the behaviour of the physical car.
- Version 3: Steering and throttle is managed via PID controller, making speed and steering changes smooth (and slow). The simulation environment runs at all times (also during image processing and inference), leading to a higher latency between image capture and action taking effect.
- Version 4: Steering and throttle is managed via PID controller, but the world is put on hold during inference, reducing the latency.
- Version 5: Steering and throttle is managed via position and velocity controller, and the world is put on hold during inference, eliminating latency almost fully. (This is very unnatural; the car can take alternating 30 degree left/right turns and will go almost straight ahead.)
The PID controller for v3/v4 can be changed in the racecar control file, by changing the P, I and D values you can tune how fast or how slow the car accelerates and steers.
Additionally you can tune the friction. In our simulator friction is defined for the wheels, and not the surfaces it drives on. The values (called mu and mu2) are found in racecar.gazebo; increasing them (once per tyre!) will allow the car to drive faster without spinning.
Finally I implemented an experimental version of the Ackermann steering geometry including differentials. Why? When turning, a car's wheels follow two circles with the same center point, the inner one is having a smaller radius than the outer one. In short, the inner wheels will have to steer more (larger curvature), but rotate slower (smaller circumference) than the outer wheels.
The initial work on creating an altered software stack for the original DeepRacer started back in 2022. The first experiments included operating the DeepRacer with an R/C controller, capturing the camera images and IMU data to create an in-car video. There was a lot to learn about ROS2, including creating a custom node for publishing IMU sensor data and capturing and creating videos on the fly. During the Berlin Summit in 2022 I also got to give my pimped car a spin on the track!
In the context of physical racing the motivation for customizing the car software is to obtain more information; "what does the car do, and why". Watching the video, you can clearly see the rolling movement in the turns, and the blurring of certain parts of the image discussed earlier.
The work triggered a need to alter several of the open source DeepRacer packages, and included work like optimizing the performance from camera to inference through compressing images and enabling GPU and Compute Stick acceleration of the inference. In the end this turned into several scripts pulling together all the changes to the different nodes and creating an upgraded software package that could be installed on an original DeepRacer car.
The work evolved, and a logging mechanism using ROS Bag allowed us to analyze not only pictures, but also the actions that the car took. Using the deepracer-viz library of Jochem Lugtenburg, a fellow DeepRacer community leader, I could add a GradCam overlay on the video feed, which gives a better understanding of what is going on.
The outcome of this has now evolved into the community DeepRacer Custom Car repository, which will allow anyone to upgrade their DeepRacer with an improved software with two simple commands, and without having to compile the modules themselves!
Benefits are:
- Performance improvement through using compressed image transport for main processing pipeline
- Inference using OpenVINO with Intel GPU (original DeepRacer), OpenVino with Myriad Neural Compute Stick (NCS2), or TensorFlow Lite
- Model Optimizer caching, speeding up switching of models
- Capture in-car camera and inference results to a ROS Bag for logfile analysis
- UI tweaks and fixes
- Support for Raspberry Pi4 - enabling us to create the DeepRacer Pi!
Capturing data is great, but you need a way to test it all - bringing models trained in a customized environment onto a track to see what works and what does not...
The question turned out to be: How hard is it to make a track that has the same 'design' as the official tracks, but that takes up less space than the 8m x 6m of the re:Invent 2018 track? After re:Invent 2023 I started to investigate. The goal was to create a custom track that would fit in my Garage with a theoretical max space 5.5m x 4.5m. The track should be printable on vinyl as well as available in the Simulator for virtual testing.
After some trial and error, it proved to be quite straight forward, even if it requires multiple steps, starting in a Jupyter Notebook, moving into a vector drawing program (Inkscape) and finalizing in Blender (to create the simulator meshes).
The Trapezoid track is a good example on how to create a brand new track - the notebook starts with 8 points in an array, and builds out the track step by step, adding outer line, center line and color.
In the end I chose to print a narrower version of Trapezoid - Trapezoid Narrow - to fit also behind my garage, with dimensions of 5.20m x 2.85m including the green borders around the track. I printed it on PVC with 500 g / m², and the comparatively heavy material was a good choice. It prevents folds and wrinkles, and generally ensures that the track stays in place even when you walk on it.
Around the track I added a boundary of mesh PVC; mounted on some 20x20cm Alu poles. Not entirely a success, as the light shone through, and there was a need for a lining of black fleece.
re:Invent is a few weeks out. Experiments are still running, and as I need to fight my way through the Wildcard race this is not the time to spill the beans. Let's just say that things are not always as straight forward as expected. I suspect that two-wrongs-make-a-right more than once in the DeepRacer, so fixing one half of something...
As a teaser of what is going on, I end with the latest iteration of the in-car video, showing a DeepRacer Pi doing laps in the garage. And do check back after re:Invent for the big reveal!