I’ve recently turned a spare Raspberry Pi into a usenet crawler for downloading and unpacking .nzb files. If you’re not too sure what they are, this guide probably isn’t for you…
What you need
- Raspberry Pi 2 (or better)
- microSD (at least 16GB)
- Additional storage (USB/HDD/NAS)
- Internet Access
- Subscription to a Usenet provider
- Newsgroups Indexer (optional)
I would recommend using a 32GB microSD in the Pi. You will need at least 4GB for the installation of the OS, so if you were to download and unpack a 120 minute movie in 1080P (another 12GB), you’ll find your 16GB card just isn’t up to the task.
The additional storage is for storing your downloads after they have been unpacked. I have SABnzbd move files to a network share after they have been unpacked. You could do the same or opt for a USB drive instead.
It is worth noting that this guide will work on earlier models of the Pi (such as the model A+ and B+), but there is a significant impact on performance; I could only achieve half the download speeds with the B+. Any board that uses separate buses for ethernet and USB (such as the RPi4) is ideal.
Step 1: Get the Pi running
First of all, download and install the latest OS (e.g. Raspbian) to your microSD card. There are plenty of guides on the internet for achieving this, so I wont bother repeating the process here.
Step 2: Install SABnzbd
Once your up and running, you will need to download several bits of software. The first is SABnzbd, which handles the downloading of .nzb files. This is quite a demanding bit of software, which is why I found the single core models just weren’t up to the task.
Open up Terminal on your RPi and type the following (followed by return):
sudo apt-get install sabnzbdplus
You will probably get a warning during the install telling you it is aborting due to a configuration problem. Ignore this, it will still install fine and we will configure everything later on. Once it has finished the installation, type the following to run SABnzbd:
sabnzbdplus --server 0.0.0.0
You can access the SABnzbd interface remotely by typing the command below. Obviously change the IP address in the example to the IP address of your Raspberry Pi.
You can now enter your credentials for your usenet service provider.
You can also start sab with the command:
Step 3: Install Unrar-nonfree
The files you download with SABnzbd will be .rar and neither Raspbian or SABnzbd know how to unpack them. To fix this, we need to download Unrar-nonfree, which despite what the name suggests, is actually free. You need to add a new location to your source list:
sudo nano /etc/apt/sources.list
deb-src http://archive.raspbian.org/raspbian wheezy main contrib non-free rpi
Save and quit nano Ctrl x and overwrite when asked Y. Then update your sources with the following command:
sudo apt-get update
Now we need to make a directory for Unrar-nonfree to work in:
mkdir ~/unrar-nonfree && cd ~/unrar-nonfree
Download Unrar-nonfree’s dependencies and then build them:
sudo apt-get build-dep unrar-nonfree
sudo apt-get source -b unrar-nonfree
Install the package (changing the numbers if your version of Unrar-nonfree is different):
sudo dpkg -i unrar_4.1.4-1_armhf.deb
Test the installation by seeing if Unrar-nonfree is available in Terminal:
Clean up the mess you’ve made:
cd && rm -r ~/unrar-nonfree
Step 4: Get SABnzbd talking to Unrar-nonfree
Restart SABnzbdplus and ensure the unrar error is no longer showing.
Let’s create some directories to save our .nzb’s into. The commands below should all be on one line :
sudo mkdir /media/USBHDD1/shares/SABnzbd/downloading
sudo mkdir /media/USBHDD1/shares/SABnzbd/completed
sudo mkdir /media/USBHDD1/shares/SABnzbd/watch
sudo mkdir /media/USBHDD1/shares/SABnzbd/watch/nzb-backup
sudo mkdir /media/USBHDD1/shares/SABnzbd/scripts
Change the directories in the SABnzbd UI to your new directories:
At this point we are almost done. The final step is to let SABnzbd start on boot:
sudo nano /etc/default/sabnzbdplus
Change the USER=; to the user you chose when you installed SABnzbd earlier. Now let’s start SABnzbd as a service:
sudo service sabnzbdplus start
If running SABnzbd as a service fails, ensure it was not already running in full before you issue the command.
Step 5b: Mounting a Drive
I plan to have SAB download and unpack my files on its SD card. It will then move the completed download to a folder called nzb on my NAS (which has the IP address 192.168.2.5
In order for SAB to move files to that folder, I need to mount the folder on my Pi. Mounting the folder basically gives it a location on the Pi’s directory tree, just like any other folder on the SD card. In terminal:
sudo mkdir /mnt/nzb
This makes a directory called nzb within the mnt directory. We now need to install cifs utilities, which lets the Pi use the network drive.
sudo apt-get install cifs-utils
Once that is installed, we will add the mount command to fstab, which is a configuration file that deals with mounting options each time the Pi reboots.
sudo nano /etc/fstab
At the bottom of the text file, enter the information below:
//192.168.2.5/nzb /mnt/nzb cifs username=daniel,password=banana,ui=1000,gid=1000,iocharset=utf8 0 0
You will need to change some of those details e.g. username, password and network address to match your particular set-up. When you are done, close the file and confirm the changes:
Now we can test the mounting process manually using Terminal:
sudo mount -a
We can now test whether the mount worked. Before we issue any farther commands, manually copy some small files into the network folder we just mounted using another computer. When you are done, we will have a look in the mount location for the new files:
Hopefully the filename(s) will show up after the list (ls) command. If nothing shows up, something has gone wrong with the mount. I would suggest you look over your details in all the steps above to see where the issue might be.
If those new files do show up, the mount has worked. You can now reboot the Pi and run those last two commands again once you are logged back in.
When you are logged back in, there is a chance the re-mount was not completed successfully. One possible explanation for this is that the mount command we placed in fstab ran before the network connection was ready. We can see if that is the case by changing our current directory back to home:
Then try running the mount command manually (again): sudo mount -a
Go back to the nzb directory and list the files to see if they show up. If they do, we know the network connection was the issue. We can create a small file to run that mount command automatically. Thanks to JolJols on StackExchange for the script.
sudo nano /etc/init.d/mountscript
This created a new file called mountscript in that location and opens it with nano ready to edit. Add the following text to the file:
#! /bin/sh # /etc/init.d/mountscript ### BEGIN INIT INFO # Provides: Mounting of all drives # Required-Start: $remote_fs $syslog # Required-Stop: $remote_fs $syslog # Default-Start: 2 3 4 5 # Default-Stop: 0 1 6 # Short-Description: Simple script to mount drives at boot # Description: Added 30jan 2014 ### END INIT INFO #Now mount all drives sudo mount -a echo "All drives mounted"
Now Ctrl X to exit and press Y then return to save. Make script executable:
sudo chmod 755 /etc/init.d/mountscript
Test starting the script:
sudo /etc/init.d/mountscript start
Register script to be run at start-up:
sudo update-rc.d mountscript defaults
Reboot the RPi and see if the NAS drive is mounted:
Step 6: Tweaking
According to howtogeek using SSL can slow download speeds by up to 50% and will add extra demand on the processor when unpacking. It is up to you whether you disable SSL. If you disable SSL, your ISP can see that you’re connecting to Usenet and can work out what you’re downloading. With SSL enabled, they can only see what you are connecting to.
Pausing the downloads during post-processing and when downloading an encrypted rar will also help relieve processor stress on your RPi. These options can be found in the SABnzbd UI.
Hopefully you are completely functional! Drop a .nzb into your watched folder and see if SABnzbd deals with it correctly.
I have found that my B+ model struggles to cope when download speeds go any faster than 2.3Mbps. If I leave the download speed uncapped, it will go faster than 2.3Mbps for a few seconds before starting back at 0. I assume the hardware cannot cope with the data rate, so I have capped Sabnzbd’s download speed at 2300Kbps and everything seems stable. I’d be interested to know if people achieve faster speeds on higher models.