Using FreeBSD, ZFS, and IPv6 on Azure

I need an external VM that can connect to my home lab IPv6-only network as well as serve a small .NET website. For any server, I tend to look first to any of the BSDs, and it turns out there's a good FreeBSD option on Azure that includes ZFS as the root partition (zroot) as well as IPv6. Also, FreeBSD 14.x has good support for .NET 8.

Here are some notes how I use an inexpensive Azure VM running FreeBSD for various tasks.

Creating an Azure FreeBSD VM with a ZFS root

When creating a VM in Azure, you can choose several operating systems in Marketplace including FreeBSD. In my case I chose FreeBSD 14.1-RELEASE (ZFS) on x64. There is also an ARM64 option, but I want to run .NET 8 workloads and this is not yet fully supported on that platform. I chose a B-series size with 1gb. This is more than enough RAM for my use case.

Choose to only use an SSH key to access the VM. I also enabled a System Assigned Identity since my VM needs read access to a Keyvault secret and a storage account. Otherwise, it's fine to accept the defaults for remaining properties.

Once your shiny new FreeBSD VM is up and running and you access via SSH, the first thing I recommend is changing the default SSH server port. Edit /etc/ssh/sshd_config and change Port from 22 to some other higher port number. This minimizes random scans from showing up in /var/log/auth.log. Now reboot your VM and add a custom Network Security Group (Microsoft.Network/networkSecurityGroups) inbound port rule that matches the new port you chose. You can also change the existing SSH rule from Allow to Deny (or simply delete it).

Adding a public IPv6 address

By default, Azure will create an IPv4-only VM. For dual-stack, what you need to do is add a new IP configuration to your network interface that has an IPv6 address. The first step is to add an IPv6 address space to your Virtual Network (Microsoft.Network/virtualNetworks aka VNET) address spaces. I added 2404:f800:0:1::/64.

Next, edit your default VNET subnet and enable IPv6 addresses. Once set, you can then edit the subnet settings again and add a public IPv6 address. I chose a dynamic IPv6 address and it has been stable.

Configuring FreeBSD for IPv6

Now that the VM configuration has IPv6 set, we next need to configure our FreeBSD VM to receive an IPv6 address from Azure. An Azure VM first needs an internal IPv6 address provided via the address space we added to our VNET address spaces and IPv6 IP configuration. We first need to run sudo pkg install dual-dhclient.

Once you have dual-dhclient installed, add the following to /etc/rc.conf:

ipv6_activate_all_interfaces="YES"
ifconfig_hn0_ipv6="SYNCDHCP accept_rtadv"
dhclient_program="/usr/local/sbin/dual-dhclient"

Now reboot and when you log back in do an ifconfig hn0 and you'll see the internal IPv6 we configured, e.g. inet6 2404:f800:0:1::4 prefixlen 128. The VM now has a local IP that gets you to the VNET's public facing IPv6 that you previously configured in the VNET subnet. Do a simple test:

ping6 -c 3 freebsd.org
PING(56=40+8+8 bytes) 2404:f800:0:1::4 --> 2610:1c1:1:606c::50:15
16 bytes from 2610:1c1:1:606c::50:15, icmp_seq=0 hlim=43 time=69.095 ms
16 bytes from 2610:1c1:1:606c::50:15, icmp_seq=1 hlim=43 time=68.517 ms
16 bytes from 2610:1c1:1:606c::50:15, icmp_seq=2 hlim=43 time=68.514 ms

--- freebsd.org ping statistics ---
3 packets transmitted, 3 packets received, 0.0% packet loss

Cool, we have a dual-stack VM. I can now create IPv6 connections from my Azure VM to my home lab, mainly for specific cases to run my external website.

FreeBSD with .NET and ZFS

FreeBSD has support for .NET which is how I was able to have this option for my VM. When I wrote this, I was running FreeBSD 14.1 and using .NET 8. Microsoft's .NET has been open source and supported on Linux for many years now. It's great to see the FreeBSD community and Microsoft work together to make .NET development available for FreeBSD.

I also really appreciate having ZFS on my VM. I can now create ZFS snapshots and do simple backups.

Adding an SSH tunnel to connect to the FreeBSD VM

My home firewall (OpenBSD) passes in an IPv6 port to a FreeBSD jail that runs a hardened sshd daemon on my IPv6-only home lab VLAN. When I'm away, my laptop usually has IPv6 via my tethered mobile phone and I can connect directly via IPv6. I can also use an IPv6 SSH tunnel for things like connecting remotely to Home Assistant or RDP'ing into my home Windows client. When I can't get IPv6 when away, I need a way to bridge from IPv4 to IPv6.

What I'll show is how I add an SSH reverse tunnel from my home lab SSH server to my FreeBSD VM. I created an A record for my domain (e.g. subdomain.idatum.net) that has the public IPv4 for the FreeBSD VM. I don't need to add an AAAA record, since the point of this is an IPv4 to IPv6 bridge. With NAT64/DNS64 on my IPv6-only VLAN, my SSH server uses an IPv6 with the DNS64 prefix I have in my OpenBSD router /var/unbound/etc/unbound.conf:

# DNS64
module-config: "dns64 iterator"
dns64-prefix: 64:ff9b::/96

And on the same OpenBSD router, /etc/pf.conf:

# NAT64
pass in inet6 from any to 64:ff9b::/96 af-to inet from ($int_if)

I add my home SSH server's client public SSH key to my FreeBSD VM's ~/.ssh/authorized_keys. This let's the SSH server connect using SSH key auth.

I wrote a script that sets up the reverse SSH tunnel, running as the same user that has the SSH client key configured on the VM, something like this:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16

#!/bin/sh
# Open local port on FreeBSD VM to connect to this home SSH server.
host=my-freebsd-vm-dns-name.net
home_lab_ssh_port=22
vm_ssh_port=5001
vm_local_port=5002

while [ true ]
do
    ssh -NT -R $vm_local_port:localhost:$home_lab_ssh_port \
        -o ExitOnForwardFailure=yes \
        -o ServerAliveInterval=2 \
        -o ServerAliveCountMax=2 \
        -6 -p $vm_ssh_port azureuser@$host
    sleep 1
done

host is the DNS record that has my VM's IPv4 address. home_lab_ssh_port is the SSH port to access my home lab. vm_ssh_port is the port we configured earlier in the VM's NSG rules and sshd_config. vm_local_port is the port available at localhost on my VM.

Once the SSH reverse tunnel is set up, I can log in from my FreeBSD VM to my home SSH server. From a remote client, I use something like ssh -t -p 5001 azureuser@my-freebsd-vm-dns-name.net ssh -p 5002 my_home_username@localhost to log into the home SSH server directly. This connects to my FreeBSD VM and creates a terminal session via ssh with my home SSH server. Another way is to use the -J SSH jump host option. The advantage of this is you can use key auth with your local public key in your home SSH authorized_keys, something like this: ssh -J azureuser@my-freebsd-vm-dns-name.net:5001 -p 5002 my_home_username@localhost. Either way, I'm in via IPv4!

Conclusion

I'm happy with my Azure FreeBSD VM. When I'm remote and don't have IPv6, I can tunnel through my inexpensive VM and get access to my IPv6-only home lab. And I can also host my simple website I use for checking on my weather station. ZFS gives me the flexibility of dataset snapshots.

There is only trust from my FreeBSD VM. The FreeBSD VM is not trusted by my home server -- only by creating the SSH reverse tunnel can the VM access my home lab SSH server. My home lab SSH server is running in a FreeBSD jail with stict configuration including SSH key only authorization. Also, I allow list my VM's public IPV6 address in pf.conf for HTTPS connections and block all others -- only my external website can connect for content.

I'll find other uses for my FreeBSD VM running on Azure. Perhaps I'll start experimenting with sending ZFS snapshots from my home ZFS server for an additional backup.

Yes, I could have used a Linux VM on any cloud provider. I chose Azure and it delivers a solid FreeBSD VM with that familiar BSD experience.

Cheers to the BSDs!