drumih/turbo-fieldfare

TurboFieldfare is a tool for running a large AI language model on an ordinary Apple laptop instead of a powerful server. The model it runs, called Gemma 4 26B-A4B, has 26 billion parameters, which normally requires far more memory than a typical Mac has. TurboFieldfare gets around this by keeping only a small core of the model in memory at all times and streaming the rest from the computer's storage as needed, so it can run using roughly 2 gigabytes of RAM, even on Macs with only 8 gigabytes total. The project is written in Swift and uses Apple's Metal graphics technology directly, rather than relying on existing tools like MLX or llama.cpp. It comes as several pieces: a native Mac app for chatting with the model, a command line tool, a background service that manages the model and graphics hardware, a local server that mimics OpenAI's chat API for other programs to use, and an installer that downloads and prepares the model files. To use it, someone would clone the code, build it using Swift's build tools, and run the resulting app. On first launch, the app downloads about 15 gigabytes of model data and repacks it into its own format before it can generate text. After that, users can type prompts and get responses, with settings to control creativity and randomness of the output. The project only works on Apple Silicon Macs (M-series chips) running a fairly recent version of macOS, and it does not support older Intel Macs. It currently handles text only, not images or audio. The included benchmarks show generation speeds ranging from about 5 tokens per second on an older 8 gigabyte MacBook Air up to over 30 tokens per second on a newer, more powerful Mac. The project is licensed under Apache 2.0, a permissive license that allows free use, modification, and commercial use.