doitintl/bigtable-follower-testing

This project is a testing playground for figuring out how to efficiently look up who a user is following, but only show the ones who are currently online. Think of a Twitch-like app where you follow several streamers: when you open your following list, you want to see which of those streamers are live right now, without having to check each one individually. The data is stored in Google BigTable, a database designed for very large-scale applications. Each user's ID is converted into a hashed string for even data distribution. A record's key looks like "user_id:following:streamer_id," creating a clear link between a viewer and each person they follow. The project explores several approaches to querying this data, ranging from filtering on the database side to pulling records and filtering on the application side. The most complete script also demonstrates toggling a streamer's status between online and offline across all of their followers' records. This would be useful for engineers building social platforms or live-streaming services where relationship data is massive and needs to be filtered by real-time presence. For example, a platform with millions of users following thousands of streamers needs a way to quickly serve a viewer's personalized list of who is currently live. The project tests which query strategies perform well enough to handle that scale. What stands out is the honest, experimental nature of the repo. The author openly notes which approaches performed poorly, such as regex-based filtering on the database side and multiprocessing on the client side, neither of which are recommended. The project compares using a direct BigTable client versus a compatibility layer called HappyBase, and ultimately leans toward a specific row-key lookup approach that matches an existing production architecture. It reads like an engineer's working notebook for benchmarking real tradeoffs.