gadda00/agentreplay

AgentReplay is a Python library that sits between an AI agent and everything unpredictable it talks to, such as the AI model API, external tools, the network, and the clock. Its main promise is that once you record one run of your agent, you can replay that exact run again and again, byte for byte identical, without ever calling the AI model again. This matters because AI agents can behave differently each time they run, which makes debugging and testing them difficult. The README explains that existing tools for watching agent behavior let you see what happened, but do not guarantee that running the same scenario again produces the same result, because their time travel features restart live execution rather than replaying a fixed recording. AgentReplay borrows an idea from deterministic replay tools used in other areas of software, such as HTTP mocking libraries, where every unpredictable input is recorded exactly during a real run, then played back later instead of being generated again. Because of this, the library unlocks two extra abilities. One is counterfactual testing, where you can edit a single recorded step, such as making a tool return an error instead of its real result, and then replay forward to see how the agent would have reacted differently. The other is that every recorded failure or important run can become a permanent, free regression test in continuous integration, since replaying it never costs another model call. The library works in four modes: recording a real run, replaying a recorded run with no live calls, a hybrid mode that replays until something differs from the recording and then falls back to live calls, and a live mode that behaves as if AgentReplay were not installed at all. The agent's own code does not need to know which mode it is running in. It installs through pip, with optional extras for OpenAI, Anthropic, LangGraph, CrewAI, and AutoGen, and it includes a command line tool for inspecting, listing, replaying, comparing, and mutating recorded runs.