nhclimber87/llm-serve-dashboard

LLM Serve Dashboard is a live monitoring page for anyone running their own local AI models on a home or office machine using tools like llama.cpp or vLLM. It shows one screen with everything you would want to check at a glance: GPU usage, temperature, and power, how many tokens per second each model is generating, how full the model's memory context is, and general system stats like CPU and RAM. The whole thing is deliberately simple. There is no framework, no build step, and nothing to install beyond Python itself, which most machines already have. The dashboard is a single HTML file you can open directly in a browser, and a small Python script reads data from your GPU driver and from each model server's built in statistics page, then serves that data locally for the dashboard to display. Nothing is sent anywhere outside your own machine. It automatically finds which port your model server is running on, so if you restart or move a model, the dashboard still finds it. You can also add extra rows for other machines or smaller models you are running elsewhere on your network, and there is an optional panel that can show a model's live reasoning text if you set that up separately. The author also built a second version of this same dashboard in Rust, called fleet-tap, aimed at people who want more accurate real time speed numbers and are comfortable with it sitting directly in the path of their traffic. This Python version here is meant for a quicker, lower commitment setup: good for keeping an eye on one machine without extra software to compile or maintain. This tool is for developers and hobbyists who are already running their own local AI models and want a simple way to watch how their hardware and models are performing without adding another complicated piece of software.