Backblaze pod, somebody set us up the RAID

Just for fun, here are some short notes on setting up the old backblaze pod. This backblaze pod is the older chassis, but with the new SuperMicro X9SCL-F motherboard, Intel i3-2120 and 16GB of ram. It is using the older controllers (not the new RAID controller). The reason this is a franken-pod and not a new one is that the new backblaze units (hardware RAID) use a backplane that requires a new pod, different power supply and the hardware raid, which were pushing the project over budget.  Although there are online instructions for building a pod, I like mine on one page.

In a nutshell, install Debian on the main disk. Note for this I used the netinstall with non-free drivers. Personally, I had to remove the SATA controllers to do this as grub was getting confused and the rescue was not working, so this was simpler. Near the end when it asks for packages, I just installed SSH server and unchecked the rest. There is no need for desktop services/etc here. Finish that up, shutdown, install controllers, boot up. Now lets configure this box o’ drives.
Install a few tools we need first:

apt-get install hdparm mdadm parted xfsprogs lvm2

You will need to partition all of the disks. I leave writing a small one line script for parted as an exercise to the reader, but by hand it would look like:

That would be done for all 45 drives (not the boot disk, of course).
Next issue: The most important issue with the backblaze is knowing where each drive is located in the chassis. The backblaze/45 drives people do it by choosing which SATA cables go to which controllers, which works with the exact controllers and motherboard they have. If you use a different one it may not match. If someone else wires up the cables (as in my case) it will not match. The way I present it here takes about 15 minutes with the machine open in front of you, but when you are done you know where the drives are. When you built it you should have noted each drive and its position. We arbitrarily choose one end to start at and call that space 1 and the last space 45 and then write down the serial number of each drive (they are on the drive) and the associated position. After that is done, in Debian, get the serial number associated with the device name by doing:

for each device. Again, it is left as an exercise to script that out for all of your devices. Write down the device number next to the drive in your list. This will come in very handy when a drive fails.
Now we just need to make the arrays. Since we do not have stacks of backblaze units all backing each other up, we use RAID 6 here and not RAID 5. Every third disk is chosen to more evenly distribute the drives over the controllers and the backplane, as shown in the whiteboard example below.

Create the raid arrays with the mdadm tool you installed previously. You will note in the commands below that the first drive is not /dev/sda. That is because Debian took it to be the USB stick upon installation. /dev/sdau is the 500G boot drive. It comes in last as it is on the motherboard SATA Controller and Debian is starting with the PCIe SATA Controller drives first.

The output from cat /proc/mdstat should show them all there, but read-only and pending. You will have to force them to get a move on.

Now you should see them all in a state of active and resyncing:

At this point we should not forget to put the array into mdadm so that it will come online at boot.

When mdadm returns with the output of the three arrays, simply add that to the end of /etc/mdadm/mdadm.conf (right under the line that says: # definition of existing MD arrays).
You can do anything you want with the drives now, but personally, I like to wait until they are done syncing before hammering them with data. Ready? Great, our game plan is to initialize them for use with LVM, create the volume groups using all three RAID arrays then format it and put it to use.

Ok, so we initialized the three RAID arrays with pvcreate and then created a single volume group of all three arrays with vgcreate and then displayed the volume group with vgdisplay. This volume group has the name backblaze2. You can name yours fred, or whatever you like. Let us now create the logical volume and then format it. Look carefully at the output of vgdisplay backblaze2. Note there is a line that says: "Free PE / Size"

You will want to take the number that corresponds to the “Free PE” and use that in the next command. In my case it was 34337853. The name after -n is the name of the volume group. You can use fred-backup or whatever you like.

So, if that all looks good, let us go ahead and format it with xfs and mount it on /mnt/data.

At this point if it all looks good, you can add an entry to fstab that looks like the following to auto mount the filesystem at boot:

You should now be good to go with the drive portion of the backblaze. Next up, how to install Amanda on the pod.