reederey87/laguna-s-2.1-dgx-spark

This repository is a set of scripts and documentation for running a specific large language model, Laguna S 2.1, on a single NVIDIA DGX Spark machine, a small dedicated AI computer. It uses a serving engine called vLLM to handle the requests, and it is designed to run entirely without administrator privileges, using its own isolated Python environment instead of touching the system's installed software. The author found that the default settings recommended for this hardware actually chose a less efficient configuration due to a quirk in how vLLM measures the DGX Spark's shared memory, and fixing that setting measurably reduced the time before the model starts responding, at a small cost in how much conversation history it can hold in memory at once. The README includes detailed measured numbers for response speed and startup time, gathered by actually running the setup rather than estimating them. The scripts handle the full lifecycle: a one time installer that downloads the roughly 74 gigabytes of model weights and sets up dependencies, a set of startup checks that refuse to launch if something looks wrong, a smoke test that runs several checks to confirm the model is working correctly, and a watchdog that checks every five minutes whether the model server has stopped responding and restarts it if needed. Everything is designed to run as a background system service that starts automatically and keeps itself healthy. The expected setup is a Mac controlling the DGX Spark machine over a remote terminal connection. A working example command sequence copies the repository over, runs the installer, starts the server, and runs the acceptance tests. The scripts and documentation in this repository are released under the MIT license, but the AI model itself has a separate license that must be accepted directly on Hugging Face before downloading it.