restsend/flowdb

FlowDB is a storage engine designed specifically for time-series data, meaning data that is recorded with timestamps, such as sensor readings, application metrics, or event logs. It is written in Rust and built for high write and read throughput. The author benchmarked it against RocksDB, a widely used storage library from Meta, and reports that FlowDB is about twice as fast at writing and more than twelve times faster at looking up individual records in their test conditions. The way FlowDB stores data is based on a design pattern called an LSM-tree, which is common in databases that need to handle many writes quickly. Incoming data is first written to a log on disk (so it is not lost if the program crashes), then held in memory, and eventually sorted and written to larger files on disk. FlowDB also uses bloom filters, which are a way of quickly ruling out a lookup without reading the whole file, and two compression methods: a fast one when data is first written and a more space-efficient one during background consolidation. FlowDB can be used in two ways. The first is as an embedded library inside a Rust program, where the developer calls it directly in code. The second is as a standalone server that accepts data over HTTP or UDP and responds to queries over HTTP. The server mode includes an admin dashboard, Prometheus-compatible metrics, and API key authentication. Queries support filtering by a key prefix, a key range, a time range, or a combination of these. Common operations like deleting a range of keys at once, setting automatic expiry times for records, and running periodic garbage collection to reclaim space are all supported. Configuration is handled through a TOML file or command-line flags, and all the key settings such as cache size, compaction behavior, and flush frequency are adjustable. The project is early stage (version 0.1) and the license is not stated in the README. The source code is written in Rust.