mulabpku/in-parameter-learning

This repository is the home of a position paper from Peking University, MIT, and Tencent Youtu Lab called In-Parameter Learning: Why Lifelong AI Systems Need More Than Longer Context. The repository itself is mainly a PDF of the paper plus a few figures, so the content here is an argument rather than running code. The paper starts from a practical observation: AI agents are now expected to work with the same user, tools, and surroundings for long stretches of time, but the models behind them stay frozen after training. Anything new the agent picks up, whether that is fresh facts, user corrections, or shifts in behaviour, is currently handled outside the model, mostly by stuffing more text into the prompt or by external memory tools. The authors call this in-context learning, or ICL. Their position is that in-context tricks are not enough for systems that need to keep learning over a lifetime. They put forward a complementary idea they call In-Parameter Learning, or IPL, which means letting a deployed model continue to update its own weights, carefully and safely, while it is in use. The hybrid they recommend uses ICL for short-lived, reversible, immediate context, and IPL for the durable, cumulative knowledge a system should actually keep. The arguments they list are straightforward. Even generous estimates of a lifetime of experience are orders of magnitude beyond a one million token context window. Scaling context length further runs into hard limits in compute, training data, and model architecture. Pulling new information into the weights, by contrast, raises the ceiling on what the model can do, supports better generalisation, and lowers the cost of inference because the knowledge no longer has to be re-fed every time. The README also shows a chart of publication trends from 2018 to 2025 from OpenAlex, where long context modelling has become a major research direction while continual in-parameter learning is still small. There is a link to the PDF of the paper and a long author list with the senior authors Xing Sun and Muhan Zhang and a contact email at Peking University.