zhaolabs-sjtu/baidu-netdisk-transfer

This project is a Linux command line tool for moving large files directly between a Linux server and Baidu Netdisk, a popular Chinese cloud storage service. It was built by researchers working with large biology and genomics data files, where a single sequencing run can produce hundreds of gigabytes that need to move between cloud storage and a lab server. Normally, moving a file from Baidu Netdisk to a Linux server means downloading it to your own computer first, then uploading it again to the server, which wastes time and disk space and is limited by home internet speed. This tool skips that middle step. It runs directly on the server and transfers files in one hop, using the data center's own internet connection, which the project reports reaching around 40 megabytes per second in real tests, including a 343 gigabyte transfer across 32 files. Under the hood it combines two existing command line tools: bypy, which talks to the Baidu Netdisk API, and aria2c, a multi threaded download manager, to speed up downloads and resume them automatically if a connection drops. Every download can be checked against an MD5 checksum afterward to confirm the file was not corrupted. Uploads work the same way but are limited to 8 megabytes per file because of a restriction in Baidu's own upload API, so larger uploads still need the Baidu desktop app or website. Setup takes about thirty seconds and does not need administrator or root access. Both tools install into the user's own home folder rather than touching shared system files, so it will not interfere with other software already running on a shared server. After installing, a one time login step authorizes the tool with a Baidu account, and after that transfers can run unattended in the background using nohup. The repository also includes a beginner focused documentation set, a Word manual, an Excel command reference, and a slide deck walking through setup and use, aimed at helping new lab members get started without prior command line experience. The project is released under the MIT license.