yeemachine/kalidokit

★ 5,638TypeScriptAudience · developerComplexity · 3/5Setup · moderate

Mindmap

mindmap
  root((repo))
    What it does
      Motion conversion
      Avatar animation
      Real-time tracking
    Inputs
      Face landmarks
      Body landmarks
      Hand landmarks
    Outputs
      VRM 3D avatars
      Live2D 2D avatars
      Rotation values
    Tech stack
      TypeScript
      MediaPipe
      TensorFlow.js

mindmap root((repo)) What it does Motion conversion Avatar animation Real-time tracking Inputs Face landmarks Body landmarks Hand landmarks Outputs VRM 3D avatars Live2D 2D avatars Rotation values Tech stack TypeScript MediaPipe TensorFlow.js

Click or tap to explore — scroll the page freely

Things people build with this

USE CASE 1

Build a vtuber-style avatar that mirrors your face and body movements from a webcam in real time

USE CASE 2

Animate a 3D VRM character using head rotation and facial expression values from MediaPipe face tracking

USE CASE 3

Create a 2D anime-style Live2D character that reacts to your hand and body gestures captured by camera

Tech stack

TypeScriptJavaScriptMediaPipeTensorFlow.jsThree.jsPixi.js

Getting it running

Difficulty · moderate Time to first run · 1h+

Officially deprecated, functionality is being integrated into MediaPipe directly. New projects should use MediaPipe instead of this library.

License not specified in the explanation.

In plain English

KalidoKit is a JavaScript library that takes face, body, and hand tracking data from a camera feed and converts it into the values needed to animate a 3D or 2D avatar in real time. The result is a vtuber-style experience where movements of the person in front of the camera are reflected in an on-screen character. The library does not do the tracking itself. That part is handled by separate AI models from Google's MediaPipe or TensorFlow.js, which detect where your face, eyes, hands, and body are in each video frame and return a set of coordinate points. KalidoKit takes those coordinate points and calculates the rotation angles and blending values needed to move an avatar's head, arms, fingers, and facial expressions accordingly. It is organized into three classes: Face, Pose, and Hand. Each takes the relevant set of landmark points as input and returns the motion values for that part of the body. There are also smoothing utilities to reduce jitter in things like eye blinks. The library is compatible with two types of avatar formats: VRM (3D models often used in VR and social apps) and Live2D (a format common in anime-style 2D character animation). The README includes ready-to-remix starter templates on Glitch for both formats, using Three.js for the 3D case and Pixi.js for the 2D case. Note from the repository: KalidoKit is officially deprecated. Its functionality is being folded directly into MediaPipe, so new projects should look there instead of using this library. The library is available on npm and via CDN.

Copy-paste prompts

Prompt 1

How do I set up KalidoKit with MediaPipe face tracking to animate a VRM 3D avatar in the browser using Three.js? Start from the Glitch starter template.

Prompt 2

I want to use KalidoKit to drive a Live2D character with my webcam. How do I connect the MediaPipe hand landmarks to the KalidoKit Hand class and pass the output to Pixi.js?

Prompt 3

KalidoKit is deprecated. What is the recommended way to achieve the same face and body tracking to avatar animation result using MediaPipe directly?

Prompt 4

How do I use the smoothing utilities in KalidoKit to reduce jitter in eye blink detection when the camera feed is noisy?

Prompt 5

How do I combine KalidoKit Face, Pose, and Hand classes together to drive a full-body avatar from a single webcam feed?

Open on GitHub → Explain another repo

← yeemachine on gitmyhub — every repo by this author, as a profile.

Verify against the repo before relying on details.