Learning to Drive Smoothly in Minutes

This is a version of araffin's reinforcement learning approach using deep deterministic policy gradient (DDPG) with a variational encoder (VAE) adopted and trained on BeamNG.research.

Install and use trained AI with DriveBuild

0. Install the dependencies pip install -r requirements.txt
1. Most probably Shapely is going to make troubles during the installation. If you use Anaconda, you can install Shapely by executing conda install shapely in your conda environment. Otherwise, please, follow the instructions from this post.
2. Import the DDPGAI class from ai.py (root directory)
3. Instantiate an object and execute the .start(sid, vid) method

from ai import DDPGAI
from client.aiExchangeMessages_pb2 import SimulationID, VehicleID

ai = DDPGAI()

sid = SimulationID()
sid.sid = "some simulation id"

vid = VehicleID()
vid.vid = "ego"

ai.start(sid, vid)

For best results a minimum controller frequency of 20 Hz should be used. The optimum controller frequency is 50 Hz.

Modes

There are two pretrained variational encoders and models which were trained on roads generated by AsFault. However, different environments were used:

1. Pure smallgrid level (the environment does not contain anything but the road)
2. Smallgrid level filled with grass

For both environments the same road materials were used: 0. Asphalt: a_asphalt_01_a

1. Mid line marking: line_yellow
2. Left and right line markings: line_white

Pure smallgrid	Smallgrid filled with grass
LEVEL_NAME = Level.PURE	LEVEL_NAME = Level.GRASS

By default a VAE and an agent trained in the grass environment are used. In config.py it is defined as LEVEL_NAME = Level.GRASS. Change it to LEVEL_NAME = Level.PURE in order to use the VAE and the agent which were trained in the pure smallgrid environment. Make sure to use the correct mode, otherwise you may encounter some weird behavior. (Note: the agent which was trained in the pure smallgrid environment performs much worse than the ones trained in the grass environment)

Camera properties

During the training the front camera was used with these properties:

1. Position: (0, 1.4, 1.8)
2. Direction: (0, 1, -0.23)
3. Field of view (FOV): 120
4. Resolution (Width, Height): (160, 120)

Credits

Related Paper: "Learning to Drive in a Day".

• r7vme Author of the original implementation
• Wayve.ai for idea and inspiration.
• Tawn Kramer for Donkey simulator and Donkey Gym.
• Stable-Baselines for DDPG/SAC and PPO implementations.
• RL Baselines Zoo for training/enjoy scripts.
• S-RL Toolbox for the data loader
• Racing robot for the teleoperation
• World Models Experiments for VAE implementation.