gauravvij/parakeet-optimization

This project speeds up an existing speech to text AI model called Parakeet, made by Nvidia, so that it converts spoken audio into written text much faster when it runs on an ordinary computer processor instead of a specialized graphics chip. The author took the standard version of Parakeet and applied a technique called quantization, which shrinks the numbers the model uses internally so it needs less computing power to run, while trying to keep its accuracy about the same. There is already an official shrunken version from Nvidia's Hugging Face page, but this project uses a more careful, tailored version of that shrinking process on the part of the model that listens to and interprets audio, called the encoder. The measured result is that this project's version runs noticeably faster than Nvidia's official shrunken version, roughly fifty percent faster on a server style processor and about thirty percent faster on an Apple laptop chip, while producing the same or very nearly the same transcribed text on the test recordings used. The README is upfront that this is not a new AI design, just a careful, well tested application of a known shrinking technique to this specific model, with the real contribution being the testing process and the ready to use faster version. To use it, you set up a Python environment, download the provided faster model files alongside Nvidia's original ones, and run included scripts to transcribe an audio file or to benchmark how fast the model runs. Long audio files need to be split into smaller chunks first, since feeding in a very long recording all at once can use a huge amount of computer memory and cause it to fail. The project includes settings for that chunking process. The testing so far uses a small set of English audio clips, so the speed gains have only been confirmed in a limited way rather than across many languages or accents.