tonyd2wild/hy3-295b-nvfp4-mtp-2x-dgx-spark

This repository is a detailed recipe for running a very large AI language model, Tencent's Hunyuan 3, on a specific pair of NVIDIA computers called DGX Spark. The model has 295 billion parameters but only activates a smaller portion of them at once, and it has been compressed down in size so it can fit and run efficiently on this particular hardware, using two machines working together over a fast network connection. The README explains that the author got the model running with a speed-boosting technique built into the model itself, which predicts extra words ahead of time to make generation faster. Their key finding is that some of the official recommended settings for this technique actually hurt performance on this specific hardware, while different settings they measured directly worked better, reaching about 21.8 words per second for a single conversation and nearly 60 words per second when six conversations run at once. The repository includes shell scripts to download and place the model weights on both machines, launch the serving software, and verify it is working correctly, along with configuration files for tuning settings like memory usage and context length, meaning how much text the model can consider at once. It also documents, in detail, several specific technical problems the author ran into while setting this up and how each one was solved, so that someone repeating the setup does not have to rediscover the same fixes. This project is aimed squarely at people who already work with serving large AI models on specialized hardware. It assumes familiarity with concepts like GPU memory, model quantization, and distributed serving, and is best understood as a documented, benchmarked configuration rather than a general-purpose application.