drowzeys/keys-concurrency-patch-for-dspark-deepseek-v4-flash

This project is a small patch for people running the DeepSeek V4 Flash AI model on a specific piece of NVIDIA hardware called the DGX Spark, using a speed up technique named DSpark. Out of the box, that setup only allows one request to be answered at a time, meaning if several people ask the AI a question at once, their requests get processed one after another instead of together. The patch fixes two problems that caused this limit. First, the system was tracking each in progress request by its position in a batch, but as requests finish and new ones take their place, that position can point to the wrong request, corrupting its data. The patch instead tracks each request by a stable ID so it never gets mixed up with another one. Second, the code that manages memory for these requests assumed every request in a batch was exactly the same shape, which broke when handling both new and continuing requests together. The patch handles requests of different shapes without that assumption. With the patch applied, the author reports that a single request still runs at the same speed as before, and multiple requests running side by side reach meaningfully higher combined throughput, with no errors observed across their tests. The author also confirms that results stay identical whether a request runs alone or alongside others. Applying the patch requires already having the DSpark setup for DeepSeek V4 running, since this only adds a patch file on top of that existing installation, followed by rebuilding and restarting the service. It has been tested on the Flash version of the model, the author expects it to also work on the Pro version but has not yet confirmed this. This is a technical infrastructure project for people already operating their own AI model servers, not an end user application. It builds on other open source projects and is licensed under Apache-2.0.