jiaxiwangau-star/gesto-engine

Gesto Engine is a hand and body gesture recognition tool that runs entirely in a web browser, using a person's webcam. Instead of a developer hard coding rules for what each gesture looks like, the system works by recording a few short samples of a gesture and then using a nearest neighbor comparison, a simple form of machine learning, to recognize that same gesture later. Whatever gestures a person trains it on are the ones it learns to recognize, and each recognized gesture can be linked to a chosen image or emoji that pops up on screen. This engine was extracted from a separate project called gestomeme.com so it could be reused as its own standalone module. To use it, someone runs a small local web server, opens the page in a browser over https or on localhost, since browsers require a secure context for camera access, and allows camera permission. From there they pick or upload an image or emoji, name a gesture, and record a short sample of themselves performing it, repeating for multiple gestures, and recording a neutral pose as well so the system does not misfire when no gesture is being made. All of this training data is stored locally in the browser rather than sent anywhere. For developers, the project also exposes a small JavaScript API so the recognition engine can be embedded into another project, with functions to start the camera and recognition loop, record a new gesture sample, list current gestures, remove one, or export the training data as a backup file. Under the hood it uses MediaPipe, a widely used library from Google for detecting body and hand landmarks from video, to extract key points that are turned into a feature vector, which the nearest neighbor method then classifies. Privacy is a stated design goal: everything runs in the browser, camera video is never uploaded or stored, and the only saved data is the local training samples. The project uses the PolyForm Noncommercial license, which allows free personal, educational, and research use with attribution, but explicitly forbids commercial use, including resale or use inside a paid product, unless a separate commercial license is obtained from the author.