nineninesix-ai/gepard-vllm

Gepard-vLLM is a text-to-speech system built for real-time spoken conversations. Instead of turning a full block of text into audio all at once, it generates speech as text arrives, streaming the sound out chunk by chunk so a conversation feels natural rather than laggy. The name comes from the German word for cheetah, a nod to how fast the model responds. Under the hood it combines a large language model with an audio codec, and both run inside the same GPU process using a system called vLLM, which is built for serving language models efficiently. One notable feature is voice cloning: you can give the system a short reference audio clip and it will create a new voice matching that sample. These voices are saved in a Postgres database, so they are not lost when the server restarts and can be reused across multiple copies of the service running at once. The project exposes its features through an API that matches the format used by a service called Cartesia. Because of this, it can be plugged directly into voice assistant frameworks such as Pipecat and LiveKit Agents without writing custom integration code, just by pointing them at this server instead of Cartesia's own servers. Running this project requires serious hardware: a modern NVIDIA GPU from the Hopper or Blackwell generation, with a large amount of video memory, plus a specific version of CUDA, the software NVIDIA GPUs use for this kind of workload. The README describes two ways to run it: directly on a machine with the right setup, or through Docker, a tool that packages the software and its dependencies together. The Docker image includes everything except the actual AI model, which downloads automatically the first time the server starts. This is a specialized, infrastructure heavy project best suited to teams already building real time voice applications who need low latency speech generation and are prepared to manage GPU servers plus a database for storing cloned voices.