amichail-1/orbination-whisper-ai

Orbination Whisper AI is a compressed, deployable version of OpenAI's Whisper speech-to-text model. The original Whisper large-v3-turbo model weighs 1.6 GB and requires Python to run. This project shrinks it to 368 MB and packages it as a single executable built in Go, with no Python required at runtime. The compression uses a technique called quantization: instead of storing model weights as full 32-bit or 16-bit numbers, they are stored as 3-bit numbers, which dramatically reduces file size. The challenge with this level of compression is that accuracy usually drops significantly. The project addresses this with a training method called Q3_K-matched quantization-aware training, where the exact compression algorithm is run inside the training process itself. This means the model learns to compensate for the quantization error before it is ever exported, so the deployed accuracy matches what was measured during training. On real speech data across English, Spanish, French, and Greek, the 368 MB version achieves accuracy close to the full 1.6 GB model. Greek is where the improvement over naive compression is most visible: a standard post-training 3-bit compression reached a 28.5% word error rate on the hardest content, while this training approach brings it down to 14.8%. The runtime is a Go application that wraps whisper.cpp (a C library for running Whisper models efficiently). It supports running on CPU, NVIDIA GPU, Apple Silicon, Vulkan, and ROCm hardware. A hybrid scheduling system splits work between GPU and CPU so the GPU handles normal load and the CPU assists during bursts. Decoding uses beam search, which reduces repetition errors common with simpler greedy decoding. You can run it as a command-line tool by passing an audio file and a language code, or start it as an HTTP server with inference, stats, and health endpoints. Pre-built binaries for Windows, macOS, and Linux are attached to the GitHub releases page, so most users do not need to build from source. The project is MIT licensed.