dathere/qsv

qsv is a command line toolkit written in Rust for working with tabular data, mainly CSV files but also Excel and a handful of other formats. It bundles together dozens of small commands that each do one job well, things like counting rows, removing duplicates, joining files, sorting, converting between formats, formatting dates, and validating data against rules, and the project markets these commands as composable and very fast, something you would notice when working with large datasets that spreadsheet software struggles to open. The commands cover the full range of what people call data wrangling. Some handle simple mechanical tasks, like dropping headers, concatenating files, or computing file hashes for integrity checks. Others are more advanced, such as an apply command that runs string, date, math, and currency transformations on a column and can even detect things like sentiment, profanity, or language using built in natural language processing functions. There is also a describegpt command that can generate a data dictionary or answer questions about a CSV file using any OpenAI compatible language model, including local models run through tools like Ollama, Jan, or LM Studio, for people who do not want to send their data to a cloud service. The count command is highlighted as an example of the project's speed focus, reporting eleven million rows counted from a fifteen gigabyte file in under twelve seconds without any index, and near instantly once an index exists. An optional polars feature adds a multithreaded, memory mapped CSV reader for even faster counts and support for extra formats like Arrow, Avro, and Parquet. qsv can be tried directly in a browser at qsv.dathere.com without installing anything, or installed as a command line binary for local use. It targets data engineers, analysts, and anyone doing FAIR data work who is comfortable in a terminal and needs a fast, scriptable alternative to spreadsheet tools or heavier data pipeline frameworks. The full README is longer than what was shown.