facebookresearch/seamless_communication

Seamless Communication is a collection of AI translation models released by Meta's research team. The models are designed to translate spoken and written language across roughly 100 languages, with the goal of making translated speech sound more like a natural human conversation rather than a robotic reading. The core model is called SeamlessM4T. It can take speech or text as input and produce speech or text as output. That means it handles tasks like converting spoken Spanish to written English, or reading an English sentence aloud in French. A second version of this model was released with improvements to translation quality and speed. Building on that foundation, SeamlessExpressive focuses on preserving how someone sounds when their speech is translated. Things like the pace of speaking and natural pauses are carried through to the translated version, rather than being flattened into a monotone output. The goal is to preserve personal speaking style across the language barrier. SeamlessStreaming handles translation in real time. Instead of waiting for a speaker to finish a sentence before translating, it processes and outputs translation as the speech arrives, which is useful for live conversations or broadcasts. The unified Seamless model combines the expressive and streaming capabilities into a single system. All models are available through the repository with command-line tools for running translations. Demos are hosted online and on Hugging Face, and a tutorial notebook from a 2023 research conference walks through the full suite of models. The models are also available through the Hugging Face Transformers library for easier integration. The full README is longer than what was shown.