unkcenter/ok-dmd

OK-DMD is a Python tool that helps large language models handle very long conversations without losing track of important earlier information. When an AI model reads a long document or chat, it keeps a running memory called a KV-cache. Because that memory is limited, older parts often get deleted to make room for new ones, and standard methods decide what to delete based only on recent attention, which can accidentally throw away something important if the topic briefly changes and then comes back later. This project takes a different approach. It treats the flow of a conversation like a river, using a mathematical method called Koopman Operator Theory to spot the steady underlying "current" of a topic instead of just watching moment to moment shifts. That lets it tell the difference between information that matters for the whole document and passing side comments that can be safely forgotten. The README gives an example: if a document about astronomy is briefly interrupted by an unrelated lasagna recipe, older methods can wipe out the astronomy memory to make room, while OK-DMD is designed to protect it. The project includes a benchmark script that reproduces this exact scenario and reports how well OK-DMD preserves memory compared to a common alternative method called H2O, showing improved results under a tight memory budget. The README is honest about a limitation: if something is mentioned only once and never brought up again, OK-DMD may treat it as unimportant and remove it, so it recommends pairing it with other filtering techniques in real production systems. Installation is done by cloning the repository and installing it locally with pip. The project is licensed under Apache 2.0.