lemire/fastconstmap

Fastconstmap is a Python library that gives you a very fast, read-only lookup table for mapping text strings to numbers. Think of it like a Python dictionary, except it is built once and then frozen, you cannot add or change entries after creation, but in exchange it is roughly twice as fast to look up values and uses about thirteen times less memory than a regular Python dict. The library comes in two flavors. The simpler one, called ConstMap, uses around nine bytes of memory per entry and answers lookups with a single hash calculation plus three array reads. The trade-off is that if you look up a key that was never in the map, you get back garbage data with no warning. The second flavor, VerifiedConstMap, doubles the memory usage to about eighteen bytes per key, but returns None or raises a KeyError when you ask for a missing key, behaving more like a familiar Python dictionary. Both versions can be saved to a file or a block of raw bytes, and loaded back later without rebuilding. For programs that run multiple worker processes at once, the library supports placing the map in shared memory, a chunk of RAM shared across processes, so that millions of lookups can happen simultaneously without each process needing its own copy of the data. You would reach for this library when you have a fixed set of string-to-number mappings that you build once at startup and look up millions of times, such as tokenizing text, mapping category names to integer IDs, or any high-throughput classification task. It is written in C and installed via pip. The full README is longer than what was provided.