karpathy/llm-council

LLM Council is a local web application that lets you submit a question to multiple AI language models at once and get a combined, reviewed answer. Instead of asking one AI and taking its response at face value, this tool sends your question to several models simultaneously, has each model review and rank the others' responses anonymously, and then has a designated "Chairman" model compile everything into a single final answer. The workflow has three stages: first, all models independently answer your question, second, each model reviews the others' answers without knowing which model produced which response, third, the Chairman model synthesizes the best elements into a final response. You can inspect each model's individual answer in a tab view alongside the final synthesis. The project uses OpenRouter, a service that provides access to many AI models through a single API key, so you configure which models to include by editing a config file. The backend is built with Python and the frontend with a JavaScript framework. The author describes it as a "vibe coded" Saturday project built for personal exploration and does not plan to maintain or extend it. Setup requires installing dependencies for both the backend and frontend, then starting each with separate commands.