Because this NAS machine runs on a small PC having only 2 SATA slots, but also for security reasons, I've installed this OpenBSD setup on a readonly USB stick.

1) Let's first talk about NAS's security.

On most of the cases, any NAS contains common documents, but, often, here and there, some of those documents are personal and must not be read by external persons.

So the basic idea is to encrypt fully the disk where data reside.

Moreover, to my eyes, the encryption has an extra added value: protect my backups.
Indeed, since a disk's crash is always possible, I duplicate the disk and store this spare disk physically in an another place. Knowing that all data are encrypted, it's easier for me to store this 2nd disk: at my friend's house, at my office desk, ...
As you may know, it's important that the backup disk does not remain in the same building as the main one. Indeed, in case of fire in the building, you could lose your both disks.



2) Setup OpenBSD on a readonly system is also an element of security. Indeed, in some specific situation where I do not need the NAS, I just poweroff the machine and simply unplug the USB stick. Having the possibility to remove the OS so easily is, to my eyes, also an important aspect of security.

Hardware

As for my other machines, I've selected a small board having an Intel Atom embedded. It does not consume lot of power and are easy to find on the market. Moreover OpenBSD runs easily on such hardware.

The only point of attention is to find a board where the ethernet port is compatible with OpenBSD.

In my specific case, I'm using a D2500HN with 4GB of DDR3. The ethernet port is "Intel 82574L" recognized by the em driver.

The 2 disks are Western Digital sata disk of 1TB. (As mentioned the system runs with only 1 disk)

Note that, despite software raid mirroring provided by OpenBSD works perfectly, my idea was to put the 2nd disk is a safe place for business continuity. Thus I synchronizing the 2 disks manually when needed and I place immediately the 2nd disk in a safe place.

Software

Step 1: Let's encrypt the disk

To create an encrypted disk, I refer you to the explanations available on the OpenBSD web site: Softraid Crypto

In short you have to:

1. Remove all exiting layouts and create one OpenBSD slice on your targeted disk (if sd2 is the targeted disk) perform the following command: fdisk -iy /dev/sd2
2. With the command disklabel you have to define 1 partition with a "FS type" called "RAID", let's call it "i:"
3. Create an encrypted partition with the following command: bioctl -c C -l /dev/sd2i softraid0

The command will request you 2x a passphrase. For security reasons, please do not use too short ones.

After that, OpenBSD answers you to which virtual disk it corresponds.

softraid0: CRYPTO volume attached as sd3

In this case OpenBSD has create a disk called sd3.
Now create a partition on this disk and format it:

• create a partition with disklabel: /dev/sd3i.
• finally format it with the following command: newfs /dev/rsd3i.

For easier management I suggest you to note the disk's name:

~#disklabel sd0  | grep duid 
duid: 8fbf08f1b85e8f65

That's it !!??. Your disk sd3i is the encrypted one.

Step 2: mount your encrypted disk at boot

Now, we have to make sure that your disk is correctly recognized at each reboot.
For that purpose, I've added the following in my /etc/rc.local file:

bioctl -c C -l 8fbf08f1b85e8f65.i -p /root/nas softraid0 > /tmp/tt    #decrypt my disk by using the passphrase stored in /root/nas
device=` sed -n -e '/CRYPTO/ s/.* //p' /tmp/tt`                     #check the feedback of OpenBSD to see which virtual disk has been created 
rm /tmp/tt
mount -o noatime,softdep /dev/${device}i /mnt/sd1                     #try the mount
if [ $? -gt 0 ]; then                                                 #in case of failure, we initiate a filesystem check. 
 fsck -y /dev/${device}i
 mount -o noatime,softdep /dev/${device}i /mnt/sd1   
fi
sync

Some details:

• Please replace the DUID (in my case it's 8fbf08f1b85e8f65) with the one for your disk.
• Assure a consistency with your disk creation (in my case I'm using "i:").
• Finally adapt the mount point to your needs (in my case it's /mnt/sd1).

As you see I'm using the method where the "passphrase" is stored in a file. For sure this file in owned only by root in readonly mode: chmod 600 /root/nas
OpenBSD offers you an another option which consist to store a "key" (required to decrypt the filesystem) onto an USB stick.

I've not used this option because the bios is sometimes missing the USB stick where this key resides.
This problem is not linked to OpenBSD itself, but to the bios or to the USB key I'm using for this key.
In any cases this is to put the "key" on a USB stick; which is, in fact, my case since the whole OS with the "key" resides on this USB stick.

Step 3: Define my NFS rules

I'll not talk about SMB, because, in my case only NFS is used (my machines are OpenBSD and one is Linux).
For that respect, I adapt the /etc/exports file as following:

/mnt/sd1/personal_files -maproot=root -alldirs -network=192.168.3.0 -mask=255.255.255.0
/mnt/sd1/share -maproot=root -network=192.168.3.0 -mask=255.255.255.0

To explain it:

• /mnt/sd1/personal_files are storage places for every users on my network. This is a kind of $HOME directory where they can store their own files. Every user is owner of his own repository and cannot see the other ones. For that purpose is important to keep consistent the user's UID on each machines installed.
• mnt/sd1/share is a shared drive available to everyone (except me) in readonly. This is where I store all photos and videos interesting enough for everyone on my internal network.

Since we are talking about NFS, here after some words to explain how to define NFS on the client machines.

On the laptop machines, I've used the AutoMountDaemon present in OpenBSD. But this solution does not satisfy me. I had too much NFS's stale situations because, by definition a laptop, can see different networks during the day. And such situation is not tolerated by NFS.
As solution, I'm currently using a kind of "button" on my desktop. When I push on it, it toggle mount/umount my nfs filesystem. I just have to not forget to push on it when I plan to move away from my network.

If someone has ideas to easily manage NFS mount on a laptop, you can explain your setup in the comments box here bellow.

On the NFS's clients, the best parameters I've found are:

~$ grep nfs /etc/fstab
obsd-nas:/mnt/sd1/personal_files /net/nas nfs rw,bg,nodev,nosuid,soft,intr,-r=65536,-w=65536,-U 0 0
obsd-nas:/mnt/sd1/share /net/share nfs ro,bg,nodev,nosuid,soft,intr,-r=65536,-U 0 0

In a future post, I'll explain how I've implemented a simple "time machine" on top of this NFS server.

Step 4: Backups

Once every 3 months or when I've uploaded lot of files, I install my spare disk in the NAS box (plug the sata cable) and mount it (bioctl + mount) on the following mount point: /mnt/sd0.
Once everything sounds correct on the spare disk (correctly decrypted, no filesystem check requested, ...) I start my backup process.

As a first stage, my backup methodology was based on the command dd. But I'm recently moved to pax which offers much more functionality

The commands I'm doing are:

# rm -fr /mnt/sd0/share
# cd /mnt/sd1
# pax -rw -pe ./share /mnt/sd0

Please note that for a 600GB of data is takes 29hours to perform the task. The biggest bottle neck is coming from the CPU. Indeed, because disks are encrypted, he has to perform several computations for each data's block. Anyhow, this process is totally transparent for the users.

For your info, in my case:

• /dev/sd0 is my USB stick with OpenBSD
• /dev/sd1 is the encrypted disk
• /dev/sd2 is the decrypted disk I mount it to /mnt/sd1. This is my main disk
• /dev/sd3 is the spare disk encrypted
• /dev/sd4 is the decrypted spare disk.

Shutdown

I order to correctly shutdown your machine, we have to umount and detach, via the command bioctl, correctly the encrypted disk.

For such task, I've added the following into the /etc/rc.shutdown file:

for i in `mount | grep -v mfs | grep -v " / " | cut -d' ' -f1`
do
  logger "rc.shutdown: umount:$i"
  umount -f $i
  sleep 5
  sync
  logger "rc.shutdown: bioctl -d `echo $i | cut -d '/' -f 3 | cut -d 'i' -f1`"
  bioctl -d `echo $i | cut -d '/' -f 3 | cut -d 'i' -f1`
done

Lessons learned

After +2 years, this NAS machine runs like a charm.

I had 2 or 3 power cut (without UPS) during this period, and, in every cases, the NAS restarted as it should, with filesystem check when needed.
The Filesystem check takes some long minutes because I have 1TB disks, but no need to perform specific interventions. Thus the NAS can restart even if I'm not physically present

One of the question I often hear is "why did you not buy a solution like synology, ... ?". My answer is mainly because of the flexibility of OpenBSD. I've not yet explain it, but on my NAS I have several other features:

• I'm using a simple "time machine" to store my files (It's here)
• I'm running a DNLA server,
• thanks to mpd and sndiod I'm using it as "jukebox" (It's here),
• and last but not least, I'm much more confident to the security's rules in OpenBSD than anything else.