Friday, August 16, 2019

Wayback Machine Downloader

Internet Archive's Wayback Machine is a gift to the world. For quick checks you just enter the URL and you get the archived version going years back.

A whole little cottage industry seems to have been formed around the Wayback Machine. They offer you whole-site download and conversions for the low price of $5 or $15 or $45 or however much they can convince you their service is worth.

Among these busy bees, the free Ruby based Wayback Machine Downloader is a little gem.

You just install it then run

wayback_machine_downloader -c 10 -s

and you get everything! Total cost: $0.

Installing the actual gem on macOS as a non-admin user seems to have contradicting info online. There's a `gem install --local` command but it doesn't seem to do what one expects -- installing in the home folder of the current user.

What did the trick for me was:

gem install -i ~/.gem/ruby/2.3.0/ wayback_machine_downloader

and this after I manually downloaded the proper .gem file from

Some were even recommending to add a http (versus the default https) source to gem but that seemed foolish and even gem itself complained about using http in 2019.

Whatever road you pick with downloading from Wayback Machine, remember all the work the Internet Archive is doing for all this to be available to you and donate to them.

No comments:

Global ecosystem report 2020: Java text editors and IDEs

The JChoice Global Ecosystem Report 2020 is looking good: Apache NetBeans at 50% of the free Java tools, Eclipse at 45% and vi / Emacs a...