tylertreat/boomfilters

Boom Filters is a collection of tools for managing data that never stops flowing. Imagine you have a firehose of information, like website clicks, sensor readings, or financial transactions, and you need to quickly check if you have seen a specific item before, count how many times something has happened, or estimate how many unique items exist. This project gives developers a toolbox to do all of that efficiently without needing massive amounts of memory. Traditional tools for tracking data require you to know how much data you will have upfront. If you guess too small, the tool eventually breaks down and starts falsely reporting that it has seen everything. The structures in this project solve that problem by accepting tiny trade-offs. Instead of giving perfect answers, they provide answers that are accurate within a small, predictable margin. For example, a tool might occasionally say it saw an item when it actually did not, but you can configure exactly how often that happens. A developer building a live dashboard might use this to deduplicate incoming events so the same action is not counted twice. If they need to know roughly how many unique users visited a site today, they can use a tool here that estimates the total using a fraction of the memory normally required. Another tool can track the most frequent items in a stream, like the top trending hashtags, or estimate how often specific errors occur in a system. The appropriate tool depends on whether the developer needs to add and remove items, or if they just need a fast, read-only summary. What makes these tools notable is their focus on unbounded streams. Rather than holding an entire dataset in memory, they use clever math to continuously evict old information or summarize data on the fly. This allows applications to process real-time data continuously without eventually crashing from running out of memory. Each tool makes a specific trade-off between memory usage, speed, and accuracy, allowing developers to pick the exact balance that fits their application.