caliperai-ai/caliperai-gt

This project is an open source platform for labeling the sensor data that self driving cars collect, specifically LiDAR point clouds, which are 3D scans made of laser sensors, alongside regular camera images from the same vehicle. The point of the platform is to let human reviewers mark up both types of data together, so a 3D shape drawn around a car in the point cloud lines up automatically with the same car seen in the camera image. It supports several kinds of labeling: 3D boxes around objects in the point cloud that can be rotated in any direction, 2D boxes, lines, and points drawn directly on camera images, and even labeling every individual point in a 3D scan by what object it belongs to. There is also an option to use an AI model called SAM2 to help speed up 2D image labeling by clicking on an object, and to carry that selection into 3D. Beyond single frame labeling, the platform can track the same object as it moves across many frames of a driving scene. Work is organized into a structure of campaigns, datasets, scenes, and tasks, and it includes a formal review process where annotation work moves through stages like quality assurance and customer quality assurance before being marked accepted. It supports role based permissions for admins, project managers, annotators, and reviewers, and can optionally connect to company login systems like Google or Okta. Behind the scenes it runs on a Python backend, a PostgreSQL database, a Redis cache, and MinIO for storing files, with a React and TypeScript frontend using Three.js for the 3D views. The whole stack is meant to be run using Docker Compose, and the README walks through cloning the repository, setting a handful of required settings like a secret key and database password, starting the containers, and creating an admin account through a script. To use it with your own data, you import a folder or zip file containing LiDAR scans, camera images, and calibration information describing how the sensors relate to each other, or you can even upload a plain video file and let the platform pull frames out of it automatically.