b-erdem/belay

Belay is a background job engine for the Elixir programming language that runs entirely on Postgres, without needing Redis or a separate workflow server. Its main idea is that a job is made of steps, and each step's result is saved once it succeeds. If a job fails partway through and gets retried, Belay does not redo the whole thing from scratch, it picks up from the first step that did not finish, replaying the already completed steps almost instantly from what it calls the journal. This matters most for jobs that mix expensive work such as calls to a paid API with long waits such as a human approval, or work that fans out across many machines. A classic job queue would retry everything from the start if any part failed, which can mean paying for the same API call twice. Belay's step based recovery avoids that. It also supports a workflow made of multiple steps arranged as a graph, waiting for a signal from a person before continuing, and canceling or steering a job while it runs. Because many of these jobs call paid AI models, Belay has built in spend tracking and budgets. You can set a dollar limit on a single job so it stops once that limit is crossed, set a shared spend limit across many queues so a burst of usage cannot outrun a dashboard, and correct the estimated cost inside a step once you know the real usage from the API response. The project ships with an embedded dashboard you can run inside your own application to watch jobs and workflows live, including a graph view of how a workflow's steps and any spawned child jobs relate to each other. It only depends on Postgrex, Jason, and telemetry, and does not require Ecto. It is distributed under the Apache 2.0 license and is installed as a regular dependency in an Elixir project's mix.exs file, plus a small amount of configuration for queues, storage, and any scheduled cron jobs.