openhft/chronicle-queue

Chronicle Queue is a Java library for storing and reading messages at extremely high speeds. It is designed for financial trading systems, telemetry pipelines, and any application where thousands or millions of events need to be recorded and processed every second with very low delay. Unlike traditional message brokers, Chronicle Queue has no separate server process to deploy: it runs entirely within your Java application and persists everything directly to disk. The core idea is off-heap storage. Most Java messaging systems hold messages in memory managed by the Java runtime, which means the garbage collector occasionally pauses the program to clean up. Chronicle Queue bypasses this by writing to memory-mapped files on disk, which the operating system manages directly. Your program never stops for garbage collection, which is critical when you need consistent microsecond-level response times. Multiple writers can add messages to the same queue, and multiple readers can consume from it concurrently without locking. Readers can also seek to any point in the queue, not just read from the end. Messages can vary in size and type. The library supports appending new messages, tailing (reading in order as they arrive), and random-access seeks. For systems that span multiple machines, Chronicle Queue supports replication across a network with latencies under 10 microseconds for inter-machine message passing. There is a C++ version available as well, and the two versions can exchange messages, allowing mixed Java and C++ applications to share a queue. Chronicle Queue is an enterprise-grade component aimed at Java developers building performance-critical systems, particularly in finance and high-frequency data processing.