0xsero/glm52-hybrid-sm120-deploy

This is a small deployment script, not an application in itself. It packages up the steps needed to run a specific large language model, GLM-5.2, on a very specific and expensive piece of hardware: a server with four RTX PRO 6000 Blackwell GPUs. It is meant for someone who already owns or rents that exact hardware setup and wants to get the model serving requests with one command instead of configuring everything by hand. Running the included deploy.sh script checks that the four RTX PRO 6000 GPUs are present, generates a configuration file, downloads the model checkpoint to the local machine if it is not already there, and then starts everything using Docker Compose. The script is written to only use those four specific GPUs, so if the same machine also has a different, unrelated GPU installed, that other GPU is deliberately left out. The README lists the exact technical settings this recipe has been validated with, including a particular container image, a large context window, compressed key-value cache format, a form of splitting the model across GPUs, speculative decoding to speed up generation, and specialized sparse attention and mixture of experts kernels tuned for this hardware. The service listens on port 8000 once running. After deployment, a separate validate.sh script is included to check that everything came up correctly. This project assumes deep familiarity with GPU serving infrastructure and Docker. It is not something a beginner or non-technical reader would use directly, since it has no purpose without the specific four-GPU hardware it targets.