aeon-7/aeon-bench-pod

This project lets you test how good an open AI language model actually is by running a full benchmark on your own computer, then cryptographically signing the result so nobody can fake the score afterward. The pitch is that AI companies publish impressive numbers about their models, and this tool lets anyone independently check those claims on their own hardware, with a result that other people can reproduce and verify. It ships as a single Docker container with a web dashboard. You point it at any open model on Hugging Face, and it pulls the model, checks that the downloaded weights match what was published, starts the model server, and then runs it through a battery of tests covering general quality, real world speed, coding ability across three separate testing setups, and optionally vision, audio, and video. Once a full run finishes, the result is signed with a private key tied to your device and can be submitted to a public leaderboard at aeon-bench.com, where anyone can see the exact settings used and repeat the test themselves. The dashboard tries to make the technical parts approachable: it detects your hardware automatically and offers a one click recommended setup, explains every tuning option in plain language along with its tradeoffs, and groups leaderboard results by hardware class so you can see the best model and settings for a machine like yours. On Apple Silicon Macs, since the container cannot access the GPU directly, the tool detects this and hands you the exact command to run the model outside the container while still recording and verifying the result the same way. There is also a built in tool server so an AI coding assistant can be asked to set up and run the whole benchmark on your behalf. A quick test run can be started for private checking, but only a complete run through the full pipeline can be submitted to the public leaderboard. The project targets people with GPUs or Apple Silicon machines who want to independently verify model performance claims, and stars sit at 21 with the code mostly written in Python.