envision-research/streamma

StreamMA is a research project from HKUST, Zhejiang University, and Alibaba that explores how to make multi-agent AI reasoning faster by having agents communicate through streaming rather than waiting for a previous agent to fully finish before the next one starts. In a typical multi-agent pipeline, Agent A finishes generating its entire response, then passes it to Agent B, which waits until Agent B is done before Agent C can begin. StreamMA changes this by feeding Agent B a stream of partial output from Agent A as it is generated. This means Agent B can begin building up its internal context while Agent A is still writing, which lets Agent C start sooner as well. The result is a reduction in total wall-clock time compared to a strictly sequential pipeline. The library is defined as a Python package. Users specify the pipeline topology as a configuration dictionary where each agent has a system prompt and a list of next agents to send output to. The topology can be a simple chain, a tree where one agent fans out to multiple, or a graph with additional shortcut connections. A built-in logger records per-agent token counts, how much of the input was served from the model's key-value cache (which indicates how effectively prior context was reused), and a text-based timeline showing which agents were processing or idle at each point. In an example run shown in the README, a three-agent chain achieved a 1.74x speedup over the sequential wall time, with Agent C hitting a 54% KV cache hit rate because it received a large portion of its input from earlier streaming segments. The project is accompanied by a paper on arXiv (arXiv:2606.05158). Installation requires the openai Python package and a compatible API endpoint. No license is stated in the README.