qihoo360/mongosync

mongosync is a tool for copying data between two MongoDB databases. It was built by the database and infrastructure team at Qihoo 360 to move data from one MongoDB server to another, whether you are doing a full copy, keeping a destination database continuously updated with changes, or just archiving a record of what changed. The tool supports three main modes. The first is a straightforward data clone, which copies your existing records from a source database to a destination. The second goes further by also replaying the source database's "oplog" onto the destination, meaning any new writes, updates, or deletes happening on the source are applied to the destination in real time. The third mode simply copies the oplog itself without applying the changes, which is useful if you want to keep an archive of database activity without modifying your destination data. This would be useful for a team migrating a database to new hardware, creating a live backup that stays in sync, or setting up a separate database for analytics that mirrors production data. You can configure exactly what gets copied by specifying particular databases or collections, and you can even filter records so only certain data is transferred. It supports authentication on both sides, so you can securely connect to protected servers. In terms of performance, the team tested it by moving about 36 million records (roughly 1 GB of data) in just over six minutes, achieving around 96,000 operations per second. They note that the main bottleneck is how fast the destination database can write, so using multiple connections can speed things up if your target server can handle the load. The project is written in C++ and must be compiled before use, though the README notes it should not be compiled with C++11 support due to errors that arise.