ssanjaychandra123/databricks-engineering-patterns

This repository is a free reference collection of 350 patterns for engineers who work with Azure Databricks, a cloud data platform built on top of Apache Spark. The patterns are organized into seven PDF books, each covering a different area of the platform. Each pattern follows a consistent structure: it names a common wrong assumption, explains what is actually happening under the hood, and describes what to do about it. The idea is that knowing the right mental model for how something behaves saves time spent guessing or debugging. The format is practical rather than theoretical. The seven books cover clusters and compute, the Delta Lake storage format, workflows and job orchestration, streaming data processing with a feature called Auto Loader, Unity Catalog (the platform's data governance and permission system), Databricks SQL and its query accelerator called Photon, and platform cost architecture. The PDFs are all downloadable directly from the repository. The author describes the collection as useful for three situations: preparing for a job interview about Databricks, debugging a problem in a production environment, and learning the platform for the first time. The repository has an open issue tracker for corrections, since the platform changes frequently.