When we did an initial search for books similar to the Book of Mormon, we needed a large set of books to search through. Luckily, archive.org has an open and generous policy that allows anyone (even us!) to download books using the archive.org API. Using this API, we downloaded English books published from the year 1500 AD to the year 1830 AD (about 130,000 of them).
The script we used initially is here, but reader beware! It was a quick and dirty solution that needed some improvements, which led us to create Archivist–a ruby gem that can be easily installed in a ruby environment and used to search for archive.org books, then download them.
As described by Archivist’s readme file, you can use it like so:
require 'archivist' # Create an Archivist client: client = Archivist::Client.new # Search for the books you're interested in: books = client.search(:start_year => 1500, :end_year => 1510) # Download them: books.each do |book| puts book.download end
Note that this assumes you’ve installed “archivist” by listing it in your Gemfile like so:
gem 'archivist', github: "wordtreefoundation/archivist"