huangyh98/drivehack

DriveHack is a research add-on for testing self-driving car software. It works on top of another project called DriveStudio, which reconstructs realistic 3D driving scenes from real Waymo self-driving car data using a rendering technique called 3D Gaussian Splatting, often shortened to 3DGS. DriveHack takes one of these reconstructed scenes and drops a fully textured running character into it, letting the character move along a path you choose. You draw the character's path in a browser based 3D editor by clicking points inside the reconstructed scene. The tool then computes a walking or running pace that matches the length of the path and the length of the output video, so the character's feet do not slide or overshoot as it moves. Other vehicles, pedestrians, and cyclists already present in the scene keep moving too, and the system checks for collisions between them and the added character over time. The character is blended into the scene with matching lighting, so its shading follows the sun and sky conditions already present in the reconstruction, and a pixel by pixel depth check makes sure the character is correctly hidden behind or in front of scene objects as it moves through them. The main purpose described in the README is generating test scenarios for self-driving car software: placing a person unexpectedly in the path of a vehicle inside a realistic scene, at scale, to see how a driving system responds. Output videos combine five camera angles matching a real Waymo vehicle's camera layout along with a bird's eye view map in a single grid. Setting this up is involved. You first need a working DriveStudio installation with a scene already trained from Waymo data, which itself needs a capable Nvidia GPU, specific CUDA and PyTorch versions, and roughly fifty gigabytes of disk space per scene, plus several hours to train. Only after that is DriveHack installed on top, adding its own scripts, a five camera configuration file, and a downloaded character animation file. The project also supports baking custom characters from Mixamo using Blender. DriveHack is released under the MIT license.