anthonymq/qwen3.6-35b-a3b-ud-q8_k_xl_dgx-spark-recipe

This repository is a small collection of bash scripts for starting and stopping a local AI model server. It does not contain any AI model itself. Instead, it wraps llama-server, a program from the llama.cpp project, and configures it to run a specific large language model called Qwen3.6-35B-A3B. The scripts are built for powerful machines with 96 to 128 gigabytes of graphics card memory, such as an NVIDIA DGX Spark, since this particular model and its very large context window need that much room. The start script checks whether the required model files already exist, and if not, it downloads them automatically from Hugging Face, a site that hosts AI models, resuming the download if it gets interrupted. It also looks for the llama-server program itself in several common locations, refuses to start a second copy if one is already running, and waits until the server reports it is healthy before saying it is ready. A matching stop script shuts the server down cleanly. Both scripts keep the server running in the background even if you close your terminal window. Once running, the server exposes an interface compatible with OpenAI's API format, so other tools that already know how to talk to that kind of API can send it chat messages, ask it to list available models, or check on its health, all over a local web address. Using this requires some technical setup beyond just running the scripts. You need to separately build the llama-server program yourself from source code with graphics card support enabled, since the repository does not include a ready made copy of it. You will also want a Hugging Face account token to avoid download slowdowns, and around 40 gigabytes of free disk space for the model file alone. This is a specialized tool for people who already run large AI models on their own hardware, not a general purpose application. The repository does not state a license for this code.