fabio-rovai/vjepa2-mlx

This project takes Meta's V-JEPA 2, an AI model that turns video clips into a compact set of numbers describing what is happening in them, and rewrites it to run natively on Apple's MLX framework instead of the usual PyTorch framework. MLX is built to run efficiently on the graphics chip inside Apple Silicon Macs, so this port lets an M-series Mac process video and extract these features without needing PyTorch installed at all. At the time this was written, no version of V-JEPA 2 built for MLX existed anywhere, so every available copy of the model required PyTorch. This repository loads Meta's official model weights directly and reproduces the same math inside MLX. The author did not just assume the port worked, they wrote a separate script that runs the same video clip through both the original PyTorch version and this MLX version and compares the output numbers directly. On the tested checkpoint, the two outputs matched almost perfectly, differing only by tiny rounding level amounts you would expect from running the same calculation on different hardware. Using it involves installing a handful of Python packages, then loading one of Meta's official model sizes by name, which downloads its weights automatically. You then pass in a video clip as a stack of frames and get back either detailed per patch features or one averaged summary vector for the whole clip. On an Apple M-series chip, processing a short clip through the largest tested size takes about a fifth of a second. This project only covers the part of V-JEPA 2 that turns video into these feature numbers. It does not include the additional pieces Meta's original project uses for tasks like classification. The code for this port is released under the MIT license, though the downloaded model weights themselves follow Meta's own separate license terms.