apache/incubator-heron

Heron is a system for processing large streams of real-time data. It was originally built by Twitter to handle the enormous volume of events their platform generates every second, things like tweets, clicks, and user activity that need to be processed and analyzed as they happen rather than in batches hours later. The project was eventually donated to the Apache Software Foundation, where it entered the incubator program, which is how open-source projects graduate into full Apache projects. The "incubator-heron" name reflects that stage of the process. Heron is positioned as an improvement over an earlier stream processing system called Apache Storm, which Twitter had also used before building Heron. The README describes it as having a wide array of architectural improvements over its predecessor, though the short README does not go into detail about what those improvements are. The full documentation lives on a separate website at heron.apache.org. To run Heron you need Java 11, Python 3.6, and a build tool called Bazel. The project communicates through mailing lists and a Slack workspace, and there was a regular meetup group in the San Francisco Bay Area. Academic papers and blog posts from Twitter about the original design are linked from the README for those who want to understand the technical rationale behind the system. The README for this repository is quite sparse and serves mainly as a pointer to external documentation rather than a self-contained guide.