guilt/tinytot

TinyToT is a local AI style server that answers questions, does reasoning and math, writes code, works with documents, and holds multi turn conversations, all without downloading a large AI model, without needing a graphics card, and without any internet connection. It plugs in as a drop in replacement for Ollama, a popular tool for running AI models locally, and is also compatible with an interface style called Hermes. The project's central idea is that normal large language models mix two different jobs together: memorizing huge numbers of facts, and combining those facts into a coherent answer. TinyToT deliberately separates these. Facts are stored as plain text markdown files that you can read and edit yourself, and finding the relevant ones uses a classic text search technique called TF-IDF combined with a ranking method called BM25, rather than a trained AI model. Only the final step of putting a fluent answer together would genuinely benefit from a small amount of AI, and the author argues that step needs a fairly small model, not a huge one. Beyond factual questions, TinyToT includes a rule based calculator that can solve arithmetic, algebra, geometry, percentages, unit conversions, and simple logic puzzles exactly, without guessing. It also handles common writing and coding tasks such as rewriting text, extracting information, brainstorming ideas, debugging code snippets, and generating small starter projects in several programming languages, all driven by configuration files rather than hardcoded logic. It can read and explain Python code structure, work with documents like PDFs and Word files, fetch and summarize web pages, and handle images, audio, and video files using local tools. It also supports conversations in twenty four languages. The README reports detailed benchmark scores across many of these tasks, mostly showing perfect or near perfect results on the project's own test sets. To try it, you install it with pip and run a single command to start the local server, then send it requests the same way you would talk to an Ollama server.