stray128/egobench

egoBench is a benchmark project that compares several existing hand-tracking systems to see how accurately each one can figure out where a person's hand actually is in real-world space, using video recorded from a camera worn on the body, the kind of view you would get from smart glasses. It tests three systems, called MediaPipe, HaMeR, and WiLoR, against real sensor data collected by Apple's Vision Pro headset across 112 different everyday tasks like folding paper or typing on a keyboard. The project's main finding is that these hand-tracking systems are good at figuring out the shape of a hand, meaning its pose and finger positions, but bad at figuring out exactly where that hand sits in space relative to the camera. Adding a well known geometry technique called PnP, which uses known facts about how the camera itself works, fixes most of that placement error, cutting it by more than three and a half times in the best case, without any extra training. A second key point the project makes is about licensing. Some of the more accurate hand-tracking models cannot legally be used in a commercial product because of restrictive licenses, while others can. The benchmark reports which models are actually safe to build a business on, not just which ones score best on paper, treating that legal question as just as important as the accuracy numbers. The authors are careful to point out what this project is not: it is not a new hand-tracking model, and it does not invent a new way of measuring accuracy. It is a straightforward, repeatable test of tools that people already use, run against real sensor measurements rather than against another model's guesses, which the README argues is a more honest way to check accuracy. To reproduce the results yourself, you need Python 3.12, several machine learning libraries, and a 16 gigabyte dataset of test video downloaded directly from Apple. The project then runs a scoring script and produces charts and spreadsheets summarizing how each method performed, task by task. This is written in Python and aimed at researchers and engineers building products, such as robots that learn from watching human hands, who need to know which hand-tracking tool is both accurate and legally usable.