gilbertchen/duplicacy

Duplicacy is a command-line backup tool designed to copy your files to cloud storage in a way that saves space by avoiding storing the same data twice. When you back up your files, Duplicacy splits them into chunks and only uploads chunks that have not been stored before. If two different computers both back up similar files, the duplicate chunks only need to be stored once. This process is called deduplication. What sets Duplicacy apart from other backup tools is how it handles multiple computers backing up to the same storage location at the same time. Most other tools rely on a locking system, where one computer has to wait for another to finish before it can access the shared storage. Locks can fail in unpredictable ways over cloud storage, where network delays and outages are common. Duplicacy instead uses an approach its author calls lock-free deduplication: each chunk is saved as a separate file named after its own content fingerprint, so multiple computers can write to the same storage simultaneously without interfering with each other. The design is described in a paper published in IEEE Transactions on Cloud Computing. Because every backup is a complete snapshot rather than a chain of incremental changes, you can delete any individual backup without breaking the others. There is no requirement for periodic full backups to reset a chain. Each backup stands on its own. Duplicacy supports a wide range of storage destinations, including local disk, SFTP servers, Amazon S3, Google Cloud Storage, Microsoft Azure, Backblaze B2, Google Drive, OneDrive, Dropbox, Wasabi, DigitalOcean Spaces, and several others. Benchmark results in the README show it compares favorably in speed against other open-source backup tools when backing up the Linux source code. The tool runs on Windows, macOS, and Linux via the command line. A separate web-based graphical interface is available at duplicacy.com for users who prefer not to use the terminal. A VMware-specific edition called Vertical Backup is also available for backing up virtual machine files. The command-line version in this repository is open-source, written in Go.