explaingit

carlopi/ducklake2

Analysis updated 2026-07-18 · repo last pushed 2025-09-29

C++Audience · dataComplexity · 3/5QuietLicenseSetup · moderate

TLDR

A DuckDB extension that turns Parquet files into a queryable SQL data lake, adding time travel, schema evolution, and change tracking on top.

Mindmap

mindmap
  root((repo))
    What it does
      Manages Parquet files
      Tracks metadata catalog
      Enables time travel
    Tech stack
      C++
      DuckDB
      Parquet
      SQL
    Use cases
      Build a data lake
      Query historical data
      Track row changes
    Audience
      Data engineers
      Startup founders

Code map

Detail Auto

An interactive map of this repo's files and how they connect — its source is parsed live in your browser. Click Visualize to build it.

filefunction / class

What do people build with it?

USE CASE 1

Attach a DuckLake database with a SQL command to build a lightweight data lake without cloud warehouse costs.

USE CASE 2

Query data as it existed at an earlier point in time for audits or to recover accidentally deleted rows.

USE CASE 3

Add new columns to a table via schema evolution without breaking existing queries.

USE CASE 4

Track exactly which rows were inserted, updated, or deleted to sync data to other systems.

What is it built with?

C++DuckDBSQLParquet

How does it compare?

carlopi/ducklake2achanana/mavsdkalange/llama.cpp
Stars0
LanguageC++C++C++
Last pushed2025-09-292024-05-20
MaintenanceQuietDormant
Setup difficultymoderatemoderatemoderate
Complexity3/54/54/5
Audiencedatadeveloperdeveloper

Figures from each repo's GitHub metadata at analysis time.

How do you get it running?

Difficulty · moderate Time to first run · 30min

Still early-stage (version 0.1), best for experimentation rather than production use.

Open source and free to use, per the explanation, though the specific license terms aren't stated.

In plain English

DuckLake is an extension for DuckDB (a lightweight SQL database engine) that lets you build and manage a data lake, essentially a centralized storage system for your data. Instead of scattering files across your computer or cloud storage, DuckLake organizes everything in one structured format, storing metadata (information about your data) in a catalog and the actual data in Parquet files (a compressed, efficient file format). The big benefit: you can treat your data lake like a normal database, querying and updating it with standard SQL commands. Here's how it works in practice. You attach a DuckLake database using a simple SQL command, point it to where you want to store files, and then create tables and run queries just like you would in any SQL database. When you insert or update data, DuckLake automatically manages the Parquet files and metadata catalog in the background. You don't need to think about file management, the system handles it for you. What makes DuckLake stand out is a few built-in features that data teams find valuable. Time travel lets you query your data as it existed at an earlier point in time, useful for audits or accidentally-deleted data recovery. Schema evolution means you can add new columns to a table without breaking existing code. Change data feeds track exactly which rows were inserted, updated, or deleted, which is essential if you're syncing data to other systems. These features are typically only available in enterprise data platforms, DuckLake brings them into an open, SQL-based format that anyone can use. This project would appeal to data engineers, analysts, or startup founders building data pipelines who want a simple alternative to expensive cloud data warehouses. It's also useful for anyone who currently manages data across multiple CSV or Parquet files and wants a lightweight, queryable catalog system. Since it's still in early development (version 0.1), it's best suited for experimentation rather than critical production systems, but it's open source and free to use.

Copy-paste prompts

Prompt 1
Show me the SQL commands to attach a DuckLake database and create my first table backed by Parquet files.
Prompt 2
Using DuckLake's time travel feature, help me write a query that recovers a row I accidentally deleted yesterday.
Prompt 3
Explain how DuckLake's change data feed works so I can sync updates from my data lake to another system.
Prompt 4
Help me design a lightweight data pipeline using DuckLake instead of a full cloud data warehouse for my startup.
Prompt 5
Walk me through DuckLake's schema evolution so I can add a new column to an existing table without breaking my app.

Frequently asked questions

What is ducklake2?

A DuckDB extension that turns Parquet files into a queryable SQL data lake, adding time travel, schema evolution, and change tracking on top.

What language is ducklake2 written in?

Mainly C++. The stack also includes C++, DuckDB, SQL.

Is ducklake2 actively maintained?

Quiet — no commits in 6-12 months (last push 2025-09-29).

What license does ducklake2 use?

Open source and free to use, per the explanation, though the specific license terms aren't stated.

How hard is ducklake2 to set up?

Setup difficulty is rated moderate, with roughly 30min to a first successful run.

Who is ducklake2 for?

Mainly data.

Open on GitHub → Explain another repo

This repo across BitVibe Labs

Verify against the repo before relying on details.