myrobotproject/dtof-lidar-hm-ld1-gesture-recognition

This project builds a real time pipeline that turns depth data from a HM-LD1 DToF LiDAR camera into hand landmark detection, meaning it can track the position of a hand and its fingers using depth sensing instead of a normal color camera. It is aimed at things like contactless control for robots or smart home devices. The pipeline works in a fixed sequence: it reads small 40 by 30 depth frames from the camera, removes any depth values farther than a chosen distance (one meter by default), converts the remaining depth values into a false color image, upsamples that image to a larger 384 by 288 resolution, and then feeds it into a hand landmark detector. By default that detector is Google's MediaPipe HandLandmarker, which outputs 21 points describing the shape and position of a detected hand. Results can be viewed live with an on screen visualization or saved to a JSONL file for later use. The hand landmark detection step is kept separate from the rest of the code, so someone could swap in a different detection model by writing a small adapter class and registering it, without needing to touch the camera reading or image processing steps. Setup involves installing some system packages, running a provided script to create a Python virtual environment, downloading the MediaPipe model, and building a small C++ tool that streams frames from the camera using its official SDK, which must be obtained separately and placed next to this repository. The project has been tested on Ubuntu 22.04 and on WSL2, and includes instructions for attaching a USB camera to a WSL2 environment from Windows. Because MediaPipe was trained on ordinary color photos rather than depth based pseudocolor images, the README notes that detection quality depends on keeping the hand's depth image looking hand shaped, for example by avoiding background clutter and highly reflective or very dark surfaces. The project's own code is released under the MIT License, though the camera SDK and any third party models or packages it depends on keep their own separate licenses.