bmschmidt/quadfeather

Quadfeather helps you organize large spreadsheets of location-based data into a fast, web-friendly format. Instead of keeping millions of rows in a single CSV file, it breaks them into a nested tree structure where each piece contains around 65,000 rows. This lets you grab just the data you need without loading everything at once, perfect if you're building a map or dashboard that shows different regions at different zoom levels. The tool works by taking your CSV or feather file (a columnar data format) and recursively dividing it into quadrants based on geographic coordinates. Each quadrant becomes its own feather file, which is a highly compressed format that's fast to read. The tree structure means you can ask for "give me all points in this corner of the map" and get back only the relevant files, rather than scanning through an entire dataset. The README notes this is still a work in progress, so expect some rough edges. You'd use this if you're building a geospatial visualization tool, a mapping application, or any system that queries large datasets by location. For example, if you have a dataset of 10 million restaurant locations and you want a web interface where users can pan and zoom to see nearby options, Quadfeather transforms that into a structure where loading the data for a single city block is nearly instant. The tool includes a test command-line interface to get started quickly. The project trades some complexity (you need to think about which columns to keep, and there's JavaScript integration to figure out) for significant performance gains in how fast you can serve location-based queries over the internet. The README hints at future plans like supporting 3D data and other ways to partition the tree, but as written, it focuses on the core quadtree approach for 2D geographic data.