lxfater/video-to-doc-stepfun

This tool takes a screen-recording or tutorial video and automatically converts it into a step-by-step written guide with screenshots embedded. The output is a Markdown document and a PDF, both structured around the actions visible in the video. The default workflow uses subtitles to drive the process. A local Whisper model listens to the audio and produces a transcript with timestamps. An AI model from StepFun (step-3.7-flash) reads those subtitles, figures out which actions are happening and when, and picks the best frame to capture as a screenshot for each step. The tool then uses ffmpeg to pull those frames out of the video. For steps the AI is less sure about, it sends the screenshot back to the AI for a second look and a corrected description. You can also turn on a mode where the full video is uploaded to the AI and it watches the screen directly, rather than working from audio alone. That approach costs more in API calls but can catch steps that have no spoken narration. A web-search option is available to supplement the final document with additional context from the internet. The project includes a small web application so you can do everything through a browser. You upload a video, watch the processing stages in real time, then edit the draft document in a split-screen editor (Markdown on the left, preview on the right) before exporting the PDF. The backend is FastAPI, the frontend is React, and they communicate via a server-sent-events stream for live progress updates. Setup requires Python, ffmpeg installed on your system, and an API key from the StepFun platform. The configuration is minimal: copy the example environment file, paste in your key, and run the script. The README is in Chinese, but the code and configuration options are straightforward. All outputs land in a folder named after the original video file.