r0b0tlab/nvidia-qwen-3.6-27b-sm121-nvfp4

This project is a ready-made setup for running a specific large AI language model, NVIDIA's Qwen3.6-27B, on a particular type of NVIDIA hardware called the GB10, also known as DGX Spark. It packages the model together with a matching build of vLLM, a program that serves AI models so other software can send them requests and get responses back. The README is written for people who already work with AI model serving infrastructure, so it assumes familiarity with GPUs, model formats, and performance tuning. In plain terms, the project takes a quantized version of the model, meaning a version compressed to run faster and use less memory, and configures it with specific settings for context length, batch size, and caching so it runs reliably on this hardware. The setup deliberately avoids an experimental memory-caching approach and instead uses a more proven one for the version people are meant to actually run in production, while keeping the experimental version in a separate file for reference. The repository ships as a Docker image, a self-contained package that can be downloaded and run without manually installing all the underlying software. Users pull the image, run it with GPU access enabled, and it starts serving the model over a local port that other applications can send requests to. There is also an option to install it through a separate tool called sparkrun using a recipe file included in the repository. The README includes detailed performance measurements, such as how many tokens per second the model can generate at different levels of simultaneous requests, response latency, and accuracy on a math reasoning test. It also lists a series of automated checks meant to confirm the build behaves as expected before it is trusted. Instructions are provided to rebuild the Docker image from source if someone wants to reproduce the exact setup themselves. Credit is given to the underlying open source projects the setup depends on. The scripts and documentation in this repository are released under the MIT license, though the underlying model weights are not included and follow their own separate license from NVIDIA.