graph-com/kveraser

KVEraser is the code release accompanying a research paper titled "KVEraser: Learning to Steer KV Cache for Efficient Localized Context Erasing," which the README says was an oral presentation at the ICML 2026 MemFM workshop. Large language models often keep a running memory of everything they have read so far, called a KV cache, so they do not have to reprocess earlier text from scratch each time they generate a new word. The paper's title suggests KVEraser is a learned method for selectively removing, or erasing, specific parts of that memory efficiently, rather than having to recompute the whole cache when some context needs to be forgotten or replaced, though the README itself does not explain the method in plain language and points readers to the paper on arXiv for the technical details. The README is mostly a set of setup and command-line instructions rather than a walkthrough of how the tool works or why someone would use it. Setup involves creating a Python 3.11 environment with conda, activating it, and installing dependencies from a requirements file, then logging into Weights and Biases for experiment tracking. Training is run through a train.py script that in the example command targets the Qwen3-8B language model and saves checkpoints to a specified output directory while logging to a named Weights and Biases project. Two evaluation scripts are also included. One, evaluate_kveraser.py, runs the KVEraser method itself against a trained checkpoint on a specified model. The other, evaluate_recompute.py, runs what the README calls an exact recompute baseline, presumably a comparison method that recomputes context exactly rather than using the learned erasing approach, to serve as a point of comparison in the paper's experiments. Both examples in the README are configured to run on a single GPU. The repository includes a formal citation entry for the associated paper, authored by Mufei Li, Shikun Liu, Dongqi Fu, Haoyu Wang, Yinglong Xia, Hong Li, Hong Yan, and Pan Li, published as a 2026 arXiv preprint. No license file or license section is mentioned in the README.