neochaotic/leoflow

Leoflow is a workflow orchestrator: a tool that schedules and runs automated jobs in a defined sequence. Think of it as a planner that says "run task A, then run task B with A's output, then run task C every night at 5 AM." It is written in Go and positioned as a faster, less painful replacement for Apache Airflow, which is one of the most widely deployed tools of this type but is known for being slow and resource-intensive. The README spells out five specific problems with Airflow that Leoflow aims to fix: the scheduler is slow (3 to 10 seconds between task decisions versus under 200 milliseconds in Leoflow), the sensor system chokes above 500 concurrent monitors (Leoflow claims 100,000 or more), DAG definition files are re-parsed on every scheduler loop wasting CPU, long-running worker processes accumulate memory leaks, and different workflows cannot use conflicting library versions in the same environment. Leoflow's design choices address these by building the control layer in Go instead of Python, running every task in its own short-lived container that is created and then discarded, compiling each workflow definition into a fixed artifact (a JSON file plus a container image) that does not need to be re-read on every cycle, and giving each workflow its own isolated container image with its own dependencies. Developers define a workflow using a short YAML config file and a Python script. Running a single CLI command builds the container, pushes it to an image registry, and registers the workflow with the control plane. Crucially, the project uses the unmodified Apache Airflow web interface, so teams familiar with Airflow's dashboard do not need to learn a new tool. The project is pre-alpha and not yet production-ready. The core scheduling engine, task execution, data passing between tasks, and log streaming are working. The Airflow UI integration and a Helm chart for easy Kubernetes installation are described as the next steps on the roadmap.