nmm-roadmap/awesome-nmm-list

This repository is not software you install or run. It is a curated reading list, sometimes called an awesome list, that tracks research papers and models in a specific area of artificial intelligence called native multimodal modeling. Multimodal means an AI system that can work with more than one type of input at once, such as text, images, video, and audio, rather than being limited to just text. The list focuses specifically on models built from the ground up to handle multiple types of input together inside a single unified system, as opposed to older approaches that bolt separate specialized components together afterward, which the maintainers describe as blind to raw signals from the other modalities. The repository organizes the field into three categories: models that take mixed inputs and produce text output, models that take mixed inputs and produce content in a specific format like video or speech, and models that can both understand and generate across multiple modalities in either direction. For each category, the list links out to dozens of individual research projects and models from companies and labs such as Meta, Tencent, ByteDance, Alibaba's Qwen team, and various universities, giving each one a short label describing its architectural approach along with links to its GitHub page, research paper, or project website. There is also a companion academic paper referenced at the top that lays out the roadmap this list is organized around. Because this is a reference document rather than working code, there is nothing to build or run. Its value is as a starting point for anyone trying to understand the current landscape of multimodal AI research, or looking for links to specific papers and open-source implementations without having to search for them individually. The maintainers describe it as actively maintained and invite others to submit additions for any relevant work that is missing. It is released under the MIT license.