hartator/wayback-machine-downloader

Wayback Machine Downloader is a Ruby command-line tool that retrieves a full copy of a website from the Internet Archive, which stores snapshots of websites going back decades. You give it a web address and it downloads every file it can find for that site, recreating the original directory structure on your computer. The files saved are the originals, not reformatted versions, so links and URLs work the same way as they did on the live site. Installation requires Ruby and a single gem install command. Basic use is equally simple: run the command with the website URL and it places all downloaded files inside a websites/ folder named after the domain. By default, it grabs the most recent version of each file the archive holds. Several optional flags give you control over what gets downloaded. You can narrow the download to files captured within a specific time window using from and to timestamps, which appear in any Wayback Machine URL. You can also restrict downloads to files matching a string or a regular expression, or exclude files by the same method. Downloading a single page rather than a whole site is possible with the exact-url flag. Performance options include concurrency, which lets you download multiple files simultaneously to speed things up, and a snapshot-pages setting that controls how many pages of archive history the tool searches through. A list-only mode prints the files and their timestamps in JSON format without downloading anything, useful for inspecting what the archive holds before committing to a full download. The tool also runs inside Docker if you prefer not to install Ruby directly. By default it skips error pages and redirects, but an option exists to include those too. The README is practical and example-driven, covering each flag with a concrete command.