gokunwu/cm_omni

cm_omni is a training codebase for building multimodal AI models from scratch. "Multimodal" means the model can handle more than one type of data, in this case working with both text and audio (the README describes a Thinker-Talker architecture). Rather than starting from an existing pre-trained model and fine-tuning it, this codebase is designed for training from random initialization, which the README describes as a "native" multimodal training stack. Training is organized as a three-stage pipeline: text pretraining, then omni pretraining, then a talker stage. You run each stage in sequence, validating after each before moving to the next. The repo includes scripts to download and convert open training data, apply that data to the stage configurations, and launch each training stage from the command line. The codebase also includes practical infrastructure for industrial use: data governance, evaluation gates, manifests, and readiness checks. It is written in Python and licensed under Apache-2.0. Documentation lives in internal docs folders with a training runbook and contributing guide. The repository also carries a public roadmap and a contributing guide, with some issues labeled good first issue for newcomers who want to start contributing. A continuous integration workflow, defined under the .github/workflows folder, runs on each change to check the codebase automatically. Most of the project, including its code, configuration files, scripts, tests, and documentation, lives together under a single cm_omni subdirectory, which keeps the repository layout simple even though the training pipeline itself runs in three separate stages.