gnipbao/story-to-handdrawn-video

This repository takes Chinese story text or a set of ordered hand-drawn images and converts them into a vertical hand-drawn diary-comic animation. The output is a silent video file, designed so that voiceover and music can be added later in post-production. The animations feature handwritten-style captions, a left-to-right reveal that progresses from text to a black-and-white sketch to a color illustration, and an optional page-curl flip transition in the bottom-right corner. The project has two parts. The first is a renderer, built on a tool called Remotion, which handles the actual work of splitting stories into scenes, applying animations, and producing the final video. The second part is an "agent skill," a package you install into a coding assistant like Codex, Claude Code, or Kimi Code. Once installed, you control the renderer using plain natural language instead of running scripts yourself. You simply tell the assistant what story or images you want to use, and it handles sentence splitting, storyboarding, image generation, and rendering automatically. To get started, you need Node.js 20 or higher, Python 3.10 or higher, and FFmpeg installed on your computer. You also need Google Chrome or a compatible browser. After downloading the repository and installing its dependencies, you copy the skill package into your assistant's skills directory and point it at the renderer project folder. From there, you can paste a story or provide image paths, and the assistant takes over. The default workflow uses Codex Image2 to generate images, with an optional OpenAI API path if you explicitly request it. You can generate a quick preview at a lower resolution before committing to a full render. Final videos are 1080 by 1440 pixels, while previews are 720 by 960. The project includes a bundled Chinese brush-style font called Ma Shan Zheng, released under the SIL Open Font License. The entire repository is available under the MIT license, and the author welcomes contributions, particularly to the core renderer logic and skill contract.