huangserva/musical-mv-storyboard

Musical MV Storyboard is a Python toolkit and workflow framework for producing AI-assisted music videos, using a song as the master timeline that drives all visual decisions. The central idea is that everything, shot choices, lip-sync moments, voice direction, and editing cuts, must be planned around the music's structure rather than assembled arbitrarily. The workflow starts by analyzing the song: a script transcribes lyrics with Whisper, identifies beat timing, detects energy peaks and climax windows, then builds a shot plan that maps musical phrases to visual types (close-ups, B-roll, group dance shots, atmospheric frames). For lip-sync sequences, the toolkit requires you to define phrase boundaries from complete vocal phrases before generating any video clips, preventing the common mistake of cutting reference audio at arbitrary time windows. Voice direction gets a dedicated step: you write a plan, audition a short sample, approve a template, then generate the full track. Scripts validate that all these gates are passed before video generation or final delivery. Video clips are generated via Seedance (an AI video generation service referenced in the README), then assembled using ffmpeg into a final edit with an EDL (edit decision list) that records all cut points and lip-sync offsets. A preview HTML page is auto-maintained as a director review board. You would use this if you are building an AI-generated music video, particularly one with live-action style performance footage, lip-sync, or narrative structure, and you want a disciplined, gate-driven process rather than ad-hoc clip generation. The toolkit is written in Python and designed to be driven by an AI coding agent.