petermercell/megaflowtracker-for-nuke

This project is a plugin for Nuke, a professional compositing application used in film and television visual effects work. It connects Nuke to a separate AI system called MegaFlow, which analyzes a video shot and figures out how every point in the frame moves across every frame. That kind of dense motion data is useful for tasks like making a digital element stick to a moving surface, correcting camera shake, or replacing the content of a screen that appears in the footage. The workflow is split into two parts. First, you run a Python script that feeds your video footage through the MegaFlow model on your graphics card. This step is slow and computationally expensive, but you only have to do it once per shot. The results are saved to a compact file on disk. Second, you open Nuke, load the plugin, point it at that saved file, and click on the spots in the frame you want to track. The plugin reads motion data for those specific points from the file and instantly generates Nuke tracking nodes with fully animated position data. This second step requires no graphics card and runs in seconds. The plugin can produce two types of Nuke nodes. A Tracker4 node follows up to four individual points. A CornerPin2D node, which requires exactly four points, maps how a four-cornered area in the frame deforms over time, useful for inserting replacement imagery onto a flat surface like a poster or a phone screen. Installation requires having MegaFlow already set up on your machine, which in turn needs a CUDA-capable graphics card for the solve step. The plugin gizmo itself is a single file that you can paste directly into Nuke or add to your permanent Nuke menu. The project is open source under the Apache 2.0 license.