rashad-ahmed-imtiaz/databricks-finops-accelerator

This project is a tool for organizations that use Databricks and want to understand where their cloud spending is going. It reads Databricks own System Tables, which track billing, compute usage, and job activity, and turns that raw data into organized tables and dashboard-ready views that platform teams, data architects, finance teams, and business stakeholders can actually use. Rather than just answering how much was spent, it aims to answer what drove the spend, which team or project owns it, what needs review, and how confident the tool is in each answer. It builds views covering daily cost trends, which workloads cost the most, compute utilization, job reliability and failures, tagging quality, and a list of optimization opportunities ranked with supporting evidence. Importantly, it only produces recommendations for humans to review. It does not automatically resize, stop, delete, or otherwise change any Databricks jobs or clusters, and it does not use any AI or external services to generate its suggestions. The tool is deployed using Databricks Asset Bundles and runs on serverless compute, so there is no manual cluster setup required. It reads several official Databricks system tables covering billing and job activity, with one required table for basic billing and several optional tables that add more detail when available. If an optional table is missing, the tool marks affected data as limited rather than failing outright, though it does require the core billing table to run at all. Setting it up involves installing dependencies with Poetry, configuring a Databricks CLI profile, validating and deploying the bundle, and then running the included job, which creates all the output tables and views in a configurable catalog and schema. Default settings like the spend lookback window, currency, and pricing assumptions live in a configuration file that can be edited before deployment. The recommended rollout is to test locally, deploy to a development workspace first, confirm the results, and only then deploy to production.