trinity-cloud/tess-server

Tess Server is a tool that turns a Mac with Apple Silicon into a machine for running large AI models entirely on your own computer. It focuses on models in the 300-billion-parameter range, which are large enough that running them locally has traditionally been impractical. The software provides a local API that matches the OpenAI format, so existing applications can connect to it without modification. Nothing leaves your machine: no cloud calls, no per-token costs, no telemetry, no automatic updates. Installation is a single npm package with no external dependencies. You do not need Homebrew, Python, or a separate engine. After installing, you run a doctor command to check your setup, then start the server. The software finds model files on your internal and attached storage, verifies them against known profiles, and serves them with preconfigured settings. You bring your own model files, which you obtain separately under their respective licenses. The project reports performance benchmarks against stock llama.cpp, a widely used local inference engine, on an M4 Max with 128 gigabytes of unified memory. One model decodes at 29.47 tokens per second versus 7.56 for stock, a 3.90x speedup. Another model runs at 97.66 tokens per second. The README also publishes results where stock is faster, with the stated reasoning that a benchmark reporting only wins is an advertisement. All raw data is available in JSON and CSV files. Two of the six supported model profiles do not load in stock llama.cpp at all, because the architecture support exists only in the Tess Server engine. The software also supports very long context windows, up to 196,608 tokens for one model and qualified to 256K or 512K for others. Four profiles ship with speculative decoding turned on by default, where a faster draft model proposes tokens that the main model then verifies for correctness. Privacy is a design principle, not just a claim. The server binds only to the local machine, never to your network. There is no logging of prompts or generated text, and model files are checksum-verified before launch. The software does not request administrator privileges or change system settings. It is described as an inference server, not a security boundary, and the README advises only loading models you trust. The full README is longer than what was shown.