mateusamp2119/iris-lakehouse

Iris Lakehouse is a data engine with a command line tool, written in Go, for running data pipelines and tracking exactly where every piece of data came from. The project describes itself as a more capable version of a cron job scheduler, rebuilt as a daemon that runs recurring tasks called pipelines. Each pipeline is a folder containing one script, written in any programming language, plus a small declaration file that says what the pipeline reads, writes, and depends on. The core feature is data lineage: every write to a table is recorded inside the same database transaction as the write itself, tagged with the run, the program, and the declaration that produced it. This history is sealed into a tamper evident journal signed with ed25519 cryptography, so a command like iris data provenance can tell you exactly where any row in any table came from, at any point in its history. All of this state, including pipeline definitions and the lineage journal, is stored in a PostgreSQL database, which Iris can either manage for you automatically or connect to an existing Postgres 16 or newer cluster. There are no other external services required, and the tool is designed to run on anything from a single small server up to a multi instance setup built for high availability. Installing Iris is a single shell command that downloads a ready to run binary, with no other dependencies needed. Developers with Go 1.25 or newer installed can also install it directly through Go's own install command, or clone the repository and build it from source. A snapshot channel is also available for people who want the newest, untested code from the development branch rather than the stable release. Once installed, the command line tool lets you apply pipeline and schema declarations, queue pipeline runs, view a live graph of running and completed runs, inspect logs, and manage failed runs through a dedicated retry list. It also exposes a read only web API, protected by access tokens, so other programs can query the current state of pipelines, runs, and the underlying data tables. This is aimed at people building or maintaining data infrastructure who need trustworthy answers about where their data came from, rather than casual users, the README assumes familiarity with databases, command line tools, and running background services.