google/brotli

Brotli is a compression algorithm developed at Google. Compression algorithms take a file, find patterns and repetitions, and rewrite the data in a shorter form that can be expanded back later without losing anything. The README says Brotli mixes a modern version of an older technique called LZ77 with Huffman coding and second-order context modeling, all of which are standard ideas in the compression field. The practical claim in the README is that Brotli runs at speeds similar to deflate, the algorithm behind gzip and zip files, but produces smaller output. The Brotli data format is published as an open standard called RFC 7932, so anyone can write their own encoder or decoder against the same spec. The project is released under the MIT license. There is a warning in the README worth noting for builders: Brotli is a stream format. It does not include checksums or a record of the original size. That means the compressed bytes will decode without error even if someone modifies parts of them, so applications that need to detect tampering have to add their own checks on top. Installation is covered for several environments. On Debian-based Linux you can run apt install brotli, and on macOS you can run brew install brotli. For C and C++ users there are build instructions using vcpkg, Bazel, and CMake. There is also a Python module that you install with pip install brotli, with a separate readme inside the python directory for development details. The README ends with pointers for getting help and a list of benchmarks, including the Squash Compression Benchmark and the Large Text Compression Benchmark, where Brotli can be compared with other algorithms. It also lists third-party ports the authors do not maintain, including an independent decoder by Mark Adler, a JavaScript port, a Haxe port that outputs JavaScript, PHP, Python, Java and C# code, a 7Zip plugin, and a Dart wrapper. These ports let projects in many languages read and write the Brotli format.