tonyd2wild/2wild-coding-agent-latency-monitor

Coding Agent Latency Monitor is a small dashboard for testing how fast an AI model server responds when many coding agent requests hit it at the same time. It works with any endpoint that speaks the OpenAI-compatible API, such as vLLM, SGLang, llama.cpp server, or TGI, and it needs nothing beyond Python itself to run. The tool fires a chosen number of parallel streaming requests, up to 32 at once, and shows live numbers for each one: time to first token, tokens per second, total end to end time, and token count. Because the requests are launched from the server side and multiplexed back through one connection, the tool avoids the browser's usual limit on how many connections it can hold open at once. There is also a real stop button and a kill all button that actually cut every open connection to the model server, so nothing keeps running in the background after you stop a test. Alongside the per-run numbers, a hardware strip shows metrics pulled from vLLM, and can optionally show real GPU usage, memory, and temperature over SSH if you set up a small config file listing your machines. When a run finishes or is stopped, the dashboard calculates peak and average throughput, per-stream highs and lows, total tokens, and average timing, then offers a one-click copy button so you can share those results elsewhere. Every finished run is also saved automatically to a local file, and a Past Runs tab lets you search and filter through your history with a chart for each run. Two extra modes exist for fun: a Matrix-style visual theme, and an experimental Art Mode where multiple agents each draw one slice of the same picture, which doubles as a rough way to compare how well different sized models can follow shared instructions. Setup involves copying an example config file, editing it to point at your model endpoint, and running a single Python script that starts a local web server. The project only talks to the endpoint you configure and does not send data anywhere else. It is licensed under MIT, so it can be freely used, modified, and shared.</content>