yangchris11/samurai

★ 7,065PythonAudience · researcherComplexity · 4/5Setup · hard

Mindmap

mindmap
  root((repo))
    What it does
      Video object tracking
      Frame by frame outlines
      Zero-shot no training
    How it works
      Built on SAM 2
      Kalman filter motion
      Memory-aware tracking
    Inputs and outputs
      Video file input
      Bounding box text file
      Segmentation mask output
    Benchmarks
      LaSOT GOT-10k
      OTB100 evaluation
      Academic research use

mindmap root((repo)) What it does Video object tracking Frame by frame outlines Zero-shot no training How it works Built on SAM 2 Kalman filter motion Memory-aware tracking Inputs and outputs Video file input Bounding box text file Segmentation mask output Benchmarks LaSOT GOT-10k OTB100 evaluation Academic research use

Click or tap to explore — scroll the page freely

Things people build with this

USE CASE 1

Track a specific player in a sports video by marking them in the first frame and letting the tool follow them automatically through the rest of the clip.

USE CASE 2

Extract frame-by-frame segmentation masks of a moving object for use in a video editing or analysis pipeline.

USE CASE 3

Evaluate object tracking accuracy against standard academic benchmarks like LaSOT, GOT-10k, and OTB100 using the included scripts.

Tech stack

PythonPyTorchSAM 2CUDA

Getting it running

Difficulty · hard Time to first run · 1h+

Requires a compatible GPU with CUDA and Python 3.10 or higher, SAM 2 model weights must be downloaded separately before running.

In plain English

SAMURAI is a Python research tool from the University of Washington that can follow and outline a specific object as it moves through a video, frame by frame. You point it at an object in the first frame by drawing a bounding box around it, and the system tracks that object through the rest of the clip, drawing a detailed outline around it at each step. It does not need to be trained on your particular video or object type: it works immediately on whatever you give it, which is why the authors call it a zero-shot method. The system is built on top of SAM 2, a model released by Meta that can segment and identify objects in images and videos. SAMURAI adds a motion-aware memory layer on top of SAM 2 that helps it stay locked onto an object even when the object moves quickly, gets partially blocked, or changes appearance as it moves. The motion estimation uses a classical technique called a Kalman filter, which predicts where a moving object is likely to be next based on its past positions and speeds. To use it, you need a computer with a compatible GPU and Python 3.10 or higher. After installing the required libraries and downloading the model weights, you can run a demo script on any video file or folder of image frames by specifying the video path and a text file that gives the initial bounding box coordinates. The repository also includes scripts for evaluating performance against standard academic tracking benchmarks such as LaSOT, GOT-10k, and OTB100. Streaming or live webcam input is not currently supported. The tool is designed for pre-recorded video files. The code was produced as part of a research paper and is primarily intended for academic use. It is built on top of SAM 2, which is developed by Meta, and the repository includes citation instructions for both the original SAM 2 paper and the SAMURAI paper.

Copy-paste prompts

Prompt 1

I have a video file and want to track a specific object using SAMURAI. Write the command-line call with the correct arguments for the video path and the initial bounding box text file.

Prompt 2

I installed SAMURAI and downloaded the SAM 2 model weights. Walk me through running the demo script on a sample video to verify the setup works correctly.

Prompt 3

I want to evaluate SAMURAI tracking performance on the LaSOT benchmark. Show me the steps to prepare the dataset and run the evaluation script.

Prompt 4

Explain how the Kalman filter motion prediction in SAMURAI helps it stay locked on a fast-moving object compared to using SAM 2 without SAMURAI.

Open on GitHub → Explain another repo

← yangchris11 on gitmyhub — every repo by this author, as a profile.

Verify against the repo before relying on details.