tonyd2wild/glm5.2-2bit-2-dgx-spark--21.5tok-s

This repository is a field report from someone who managed to run a massive AI language model, GLM-5.2, which has 753 billion parameters, on a pair of NVIDIA DGX Spark computers connected together. Getting a model this large to run at all normally needs far more hardware than two machines, so the author shrank the model's memory footprint by compressing most of its weights down to 2 bits per number, a technique called quantization, while keeping certain sensitive parts of the model at higher precision so the output stays coherent rather than turning into gibberish. The repository itself is not the core engineering work. That credit goes to an existing open source fork and a set of pre-processed model files that others built and published. What this repo adds is the practical experience of getting that existing recipe to actually work on a real two machine setup, plus the small patches and scripts the author wrote to fix problems along the way. The README documents measured speeds around 14 to 15 tokens generated per second on tasks like writing code, solving math problems, and writing prose, with later improvements reaching over 21 tokens per second at peak. It also documents ten separate problems the author ran into while setting this up, such as the two machines having different home folder paths that broke file lookups, missing environment variables, a bug in how a fallback path handled a certain kernel setup, and, most notably, output that looked fine at a glance but was actually gibberish because one required file, containing a small but essential piece of the model, had not been downloaded. The author closes with three lessons for anyone attempting something similar: keep file paths consistent across machines, verify that every needed model file downloaded successfully before assuming garbled output means something else is broken, and check specific markers in the startup logs rather than only watching for outright errors. This project is aimed at people already working with large scale AI model deployment on specialized hardware. It has 30 stars and does not state a license.