explaingit

facebookresearch/stmg

Analysis updated 2026-07-17 · repo last pushed 2024-05-15

★ 14Audience · researcherComplexity · 4/5DormantSetup · moderate

TLDR

A Meta Quest demo that recognizes thumb swipe and tap gestures for controller-free VR/AR input, based on published research.

Mindmap

mindmap
  root((stmg))
    What it does
      Recognizes thumb gestures
      Controller-free input
      Tracks hand skeleton
    Tech stack
      Machine learning
      Meta Quest
      Hand tracking
    Use cases
      Navigate VR menus
      Grab objects by gesture
      Research gesture UX
    Audience
      VR developers
      HCI researchers
      Meta Quest builders

Code map

Detail Auto

An interactive map of this repo's files and how they connect — its source is parsed live in your browser. Click Visualize to build it.

filefunction / class

What do people build with it?

USE CASE 1

Let VR app users swipe or tap their thumb to navigate menus without a controller.

USE CASE 2

Experiment with controller-free hand-based interfaces on Meta Quest headsets.

USE CASE 3

Study and build on the gesture recognition approach published at CHI 2024.

USE CASE 4

Prototype interactions like grabbing objects using thumb swipe gestures.

What is it built with?

Meta QuestHand TrackingMachine Learning

How does it compare?

facebookresearch/stmg0c33/agentic-ai0xbebis/hyperpay
Stars141414
Language—PythonTypeScript
Last pushed2024-05-15——
MaintenanceDormant——
Setup difficultymoderatehardhard
Complexity4/54/55/5
Audienceresearcherdeveloperdeveloper

Figures from each repo's GitHub metadata at analysis time.

How do you get it running?

Difficulty · moderate Time to first run · 1h+

Requires enabling hand tracking on Quest and installing via developer tools, only recognizes swipes and taps, not pinches.

No license information is provided in the explanation.

Copy-paste prompts

Prompt 1
Help me set up this thumb gesture demo on my Meta Quest headset.
Prompt 2
Explain how the hand skeleton tracking feeds into the gesture recognition model.
Prompt 3
Show me how to trigger an app action when a thumb swipe gesture is detected.
Prompt 4
What are the limitations of this gesture model compared to the one in the CHI 2024 paper?

Frequently asked questions

What is stmg?

A Meta Quest demo that recognizes thumb swipe and tap gestures for controller-free VR/AR input, based on published research.

Is stmg actively maintained?

Dormant — no commits in 2+ years (last push 2024-05-15).

What license does stmg use?

No license information is provided in the explanation.

How hard is stmg to set up?

Setup difficulty is rated moderate, with roughly 1h+ to a first successful run.

Who is stmg for?

Mainly researcher.

Open on GitHub → Explain another repo

This repo across BitVibe Labs

Verify against the repo before relying on details.

Alternatives & similar repos