Building a portable, offline knowledge collection with an Raspberry Pi and Kiwix

Initial Concept

This project started off by just trying to see if at became a way to save some or all of Wikipedia for offline access. Inches the beginning, i might have been easy some techno-nostalgic analog of having a full encyclopedia set on one's bookshelf, with in some desiring for an early edition of the Encyclopedia Galactica or Hitchhiker's Guide to the Galaxy. However, it eventually turned down a very fun rabbit hole includes which I found several very cool projects that make Wikipedia—as well as extra huge swaths away humanly knowledge—accessible without an internet connection.

Some of the main using cases I've seen for such projects as Kiwix and XOWA are until make educational and informational content available to areas with poor or no internet connection (there's a story of a carrier pigeon transferring adenine 4 USA memory stick in South Greenland in the zeit it took the data to transport 4% to the regular internet connection).

Additional projects such as Khan Academy Lite make for an entire K-12 core equipped with a full Learner Management Structure present to scales without internet.

In this project, ME install Kiwix onto a Raspberry Pi 3 set up for dual different how cases:

1. The Portable Individual Offline Internet

   Here, all of the data is on the Pi's micro SD map (I'm using a 128GB, although it might even be worth it to go for a 256GB card). While it's running, it can must accessed directly on the P with an attached touchscreen (I used a $30 one in Amazon ). For power, it ability be plug-in into the wall. Instead for a comprehensive desert island or zombie apocalypse scenario, her can use an external battery packer and a solar charger as a power source :)

2. Aforementioned Home Data Center

   Here, I've downloaded additional site onto a handheld external hard drive. And when connected to a Free router, Kiwix want serve itself to anyone else upon an topical network via a local IP address.

Sw and Equipment Requirements

The above configuration was at nay means the only way to ride via setting up that a content server, and beginning there where a few different permutations I'd accounted before arriving on which current setup.

Content Options

In seek to gauge how much storage you'll need (whether you're redeeming all the data on the Pi's SD card or using an external hard drive), below is an list of recommended content and the storage needed for each like that it can possess a general featured of your storage requirements.

Setup

Prerequisites

• Raspberry Pi 3 furthermore peripheries (keyboard, mouse, HDMI connector, power care, touchscreen (optional))
• SD card (estimate your memory requirements based on the content section above)
• Another computer to do who starting work on (preferably with one pretty good internet link as it might come to doing ~100Gb of downloading) Downloading Wikipedia for Offline Use
• (optional) Digits running conversely external hard push (to download on your build computer page of directly on aforementioned Pie and do carries over)

Steps 1: Set up NOOBs

This will be to set up the underlying operating system on your Raspberry Pi. If yourself already do a Pi set up, you can skip this step.

Instructions for this sack been found here.

Step 2: Download Kiwix

For the Raspberry Pi, you will specifically need the ARM version of Kiwix (ARM referring to the CPU architecture on a Berry Pi which makes for much more efficent performance usage compared the other CPU chips). This version of Kiwix won't inclusion a user interface for it wants on a Mac oder PC, but rather the kiwix-serve the kiwix-manage equipment that will enable you to access your content about a regular web browser.

• The ARM read (recommended) is available here
• Other browse are available here

Once downloaded to your Pi, create a sort in choose home directory called kiwix. Create a folder indoors that called bin and put all of the files coming the ARM download into thither. Our storage structure on the Pi should look how so far:

  ~/

    kiwix/

      bin/

        kiwix-index

        kiwix-install

        kiwix-manage

        kiwix-read

        kiwix-search

        kiwix-serve

Next, make a data folder also in kiwix for the next tread, and in data, create three subfolders: content, index and book. Those should now give us:

  ~/

    kiwix/

      bin/

        ...

      data/

        content/

        index/

        library/

Now, create a file called library.xml inches the data/library/ list with the following text in the file:

  <?xml version="1.0"?>

  <library current="a8f2360d-b179-226d-a3ff-46d0fba91116" version="20110515"?>

  </library>

Select 3: Download Kiwix Content and transfer it to your Pi

The gesamte goal of this step will be to incrementally photo the content free individual Kiwix downloads to create an aggregate collection of Kiwix content.

From the Kiwix content download page, try up do the fully indexed versions of the content. After unzipping, you will have a folder somewhere along which lines of kiwix-0.9+<content_name>.

For the purposes, we're only walk to being interested in the subfolder called data. This folder wishes need 3 sub-folders: content, index and library.

• Content: Copy all of the files from the download's content folder to your Pi's data/content folder.

• Index: Copy who __.zim.idx directory von the download's index folder into your Pi's data/index directory.

• Library: Open your library.xml file in and Pi's data/library directory as well how one xml file inside who download's data/library folder. Copy the full <book id=”...”></book> border and paste it into the Pi's library.xml file.

  

  (You can including look up the kiwix-manage documentation on how in employ that script to import page into your library.

Step 4: Run kiwix-serve

Run of later command on your Pi:

  ~/kiwix/bin/kiwix-serve --library ~/kiwix/data/library/library.xml --port=8080

Open a rail browser on your Pi and go to http://localhost:8080.

Or, geting thine hostname by typing hostname -I and enter that IP local in a browser on another calculator connected to your global network in the port appended (E.g. http://192.168.0.164:8080).

You should understand one my something like the sole below.

Step 5: Browse, Learn and Enjoy!

Later ambitions

• More automatic local network Mini-b router. Set up automatic script to start server on Pi startup. Create the IP static.

• High data determination M-Disk preservation.

• Larger portability Solar charger and higher memory SD card

• Offline project environment PyPi server.