miaai-lab/deepseek-v4-flash-dual-dgx-spark-1m-context

This repository provides scripts and configuration files for running a large AI language model called DeepSeek V4 Flash across two high-performance NVIDIA computers called DGX Spark nodes. The setup is designed for people who already have access to this specialized hardware and want to run the model themselves rather than relying on a cloud service. DeepSeek V4 Flash is a type of model called a Mixture-of-Experts, which means different parts of the model activate depending on what the input asks for, making it more efficient than older designs. The main headline feature in this deployment is the context length: the server is configured to accept up to one million tokens at once, which is roughly equivalent to a very long book or many hours of conversation. Most AI services limit you to a fraction of that. The two machines run the model in parallel using a technique called tensor parallelism, where the model's weights are split across the two GPUs and both do work simultaneously. They communicate through an InfiniBand connection, which is a high-speed cable type used in data centers. The setup also uses FP8 KV-cache, a memory compression technique that lets the hardware store more information about the current conversation without running out of GPU memory. Setup involves cloning the repository on both machines, copying and editing a configuration file with the IP addresses and network interface names for your specific cluster, and then running a start script from the head node. The start script remotely starts the container on the second machine via SSH and then starts its own, with the two containers connecting automatically. The whole process can take up to 20 minutes on first run because the model weights need to download. Once running, the server exposes an API at port 8000 on both nodes that is compatible with the same format used by OpenAI, so any software already written to talk to ChatGPT can talk to this server instead with minimal changes.