Starting a homelab - Part 05 - Virtualise Your Firewall

Video of the whole process is available on Youtube:

Today we will continue our journey with building a homelab. Unfortunately, I was too busy in the past year and there's been a lot of changes to my own homelab environment as well.

One of the things that has changed is my own server infrastructure. For the past few years I've been working with Dell servers, and although they are great (and are still in use), they are no longer part of my lab. Instead, I switched to a couple of 12th generation Intel NUCs, a Beelink mini PC and another Minisforum mini PC on the way - and to be completely honest, I haven't noticed any downsides yet.

All of my VMs are regularly backed up and I even have backup of backups by using a backup NAS for my primary NAS. It's not off-site, but it does give me some peace of mind when it comes to redundancy. I have no illusion that mini PCs/NUCs are not server-grade equipment, but they are easily replaceable and what is most important - they are extremely fast, compared to most servers I've had in the past; I don't have enough money to buy new equipment, so most of it is already quite behind by the time it becomes affordable. By using mini PCs, I can use pretty much up-2-date equipment, and since I don't care about performance, I'm pretty happy with my setup.

With that said, let's continue with the homelab setup. First, let's take a look at the diagram of the setup we will be working with:

This is a little bit different from the setup that we discussed here. Let's explain the what this is - we will start with the ISP router.

ISP router will provide you with routed or switched ports on it, depending who your provider is. That ultimately doesn't matter, as long as we use a managed L2 switch between the ISP router and our equipment. Why is this important? Because, if we want to use VLANs in our network, we will need to use a managed L2 switch, as we will need to use its management interface (`GUI` or CLI) to create them.

So what are VLANs? It's a way to divide a physical network into multiple logical networks. Think of VLANs as separate virtual networks that exist within a single physical network.

Since VLANs exist on L2, and IP addressing exists on L3, we usually associate a VLAN with a subnet. In our scenario, we will associate VLANs 200 (Business), 210 (DMZ) and 220 (Home) with subnets 10.198.128.0/24, 10.198.129.0/24 and 10.198.130.0/24, respectively.

If we are extending our WAN interface to a virtual firewall, we will need another VLAN for WAN, but we will not associate a subnet with it, as it is already configured on the ISP device. This is usually the subnet that covers IP addresses that are assigned to you when you connect to the ISP device directly. In our scenario, we will be working with a double-NAT scenario, as this requires least initial configuration, but if you want to host services directly on the Internet, this may not be the best approach and you may want to look into bridging your ISP device, if your ISP allows it. If not, there are other options on how to achieve this and we will cover that in future articles.

So, with that said, in this scenario, we will operate with the following VLANs:

• 2 (Management),
• 601 (WAN),
• 200 (Business),
• 210 (DMZ), and
• 220 (Home)

Please note that you may want to look into VLANs a little bit more, as it is very easy to get lost in them if you don't know what you're doing. The worst case scenario is that you get locked out of the device you're managing - in that case, you will need to reset it to factory defaults to get access to it again.

For that reason, I suggest connecting to the L2 switch either via console or a dedicated management interface. That allows you to make changes on data ports without getting locked out. Console port is for CLI access only so you must be familiar with the syntax that your device uses to make any changes.

There are two basic port types when it comes to VLANs - tagged (trunk) and untagged (access)mode. Think of it this way - an access port is a two-way highway that, for some reason (reason being that we configured it that way), only allows certain types of cars to be used. Basically, it says something like - I only allow cars with Ontario license plates, and only those cars are allowed on this highway. Everything else is discarded on those ports. So, how do you extend this highway? Well, you configure other ports with the same VLAN ID!

Keep in mind that end devices usually don't know how to read tagged frames that are received on trunk ports and ports facing those devices should, in general, be configured in access mode.

Once two ports are configured with the same VLAN ID, devices connected to those ports will be able to communicate with each other, under the condition that they have IP addresses configured in the same network.

Another thing to keep in mind is that devices in different VLANs cannot communicate with each other, even if they have IP addresses configured in the same network (not the best idea in general).

With that behind us, let's explain the concept of a trunk (tagged) port. Opposite to an access port, a trunk port will allow all VLANs to traverse it. If we use our previous analogy of a highway, this is a two-way highway where everyone is welcome by default.

We use trunk ports to interconnect switches, servers, firewalls and routers. In most cases we use them to interconnect switches, but in our scenario, we will actually use a trunk port to connect our physical switch to our ESXi server.

So, why do we want to use trunk port to connect to our server? And didn't I just say a few paragraphs above that end devices (and servers are considered end devices from a network perspective) are usually connected with access ports? Well, yes, I did. But I also said that end devices usually don't know how to read tagged frames. That is not the case with ESXi, and also not the case with modern network cards. It's basically a combo of both hardware and software. You may have hardware that is capable of reading tagged frames, but OS doesn't like that. Or vice-versa, you have an OS that is willing to read tagged frames, but your network card doesn't like it. Luckily, that is not the case with ESXi and modern network cards.

By extending our VLANs into our virtual environment, we can use same VLANs that exist in our physical environment. This is also extremely important if we want to virtualise our firewall, as this is the only way physical devices will be able to get routed through our virtual firewall.

There are two ways that you can connect your ESXi server to your physical network - you can use a single physical interface to connect it to your switch, but since we want to use the same VLANs on the virtual side as well, we need to employ native VLAN to achieve this:

interface GigabitEthernetX/0/X
 description ESXi trunk
 switchport trunk native vlan 2
 switchport mode trunk
end

This will configure the switch port as a trunk, allow all VLANs to pass through it and, finally, configure a native VLAN (ID 2) for management purposes.

Another way to do it is to connect the ESXi server with 2 cables to the physical switch (you need two network interfaces on your ESXi server). In that scenario, one port would be used for ESXi management and would be assigned to VLAN 2 in access mode, while the second one would be configured similarly as previous example, just without native VLAN:

interface GigabitEthernetX/0/X
 description ESXi MGMT
 switchport mode access
 switchport access vlan 2
end

interface GigabitEthernetX/0/X
 description ESXi trunk
 switchport mode trunk
end

One thing to remember is that, without a working DHCP server, all addresses in all VLANs would have to be set manually. By deploying this firewall we will resolve this issue, but before doing it, the only way is to use static IP addresses.

Now, once all of this is behind us and we have all the interfaces configured as required, it's time to install our firewall.

We will use pfSense for this, but any virtual firewall will do. If you want to go really open-source, you may want to look into OPNsense instead. The logic is the same, but the interface (GUI)is different.

If you followed my instructions on how to install ESXi, I mentioned that you could create a trunk port group inside of the ESXi as well - that allows you to assign it to VMs and then allow VM to read tagged frames as well. As I mentioned there, that would allow us to use a single interface and create subinterfaces on the firewall, but that will be topic for some other post.

For now, we will use port groups for specific VLANs and assign as many interfaces as necessary to the firewall. In our case, we will start with 3 of them - WAN interface (VLAN 601 in my case), Business interface (VLAN 200) and DMZ interface (VLAN 210). We will add another one in the future (Home, VLAN 220), but it is not necessary for now.

This is how my VM settings look like:

Whole configuration video can be found here. I apologize for a blurry video, seems that I messed some OBS settings and only realised when the video was done so it was too late to repeat the whole process.

Once we prepared the VM, it's time to start it up and open a console window.

We will accept the EULA and continue. Next screen will ask us to choose what we want to do - we will select Install and press OK to continue.

As part of the install process, we need to choose what type of partitions we will use. Since this is running in a VM, just leave it at default - Auto (ZFS) and press OK to continue.

Next we will confirm the configuration options, select Install and press Select to continue.

We will choose Stripe, as we don't need redundancy in a virtual machine. If you have a more complex ESXi environment, you may want to add additional disks to your pfSense install, and then you would be able to choose different options, but in our case we will just use Stripe and take care of the backups some other way.

Finally, we will confirm the selection by choosing the disk and press YES to finalise the install.

Once the VM restarts, we are greeted with the following screen:

It may be too quick to notice with some other messages popping up, but first question it will ask you is if you want to setup VLANs.

As I mentioned earlier, if you were to use a single interface in ESXi to assign it to your pfSense VM, you could use a trunk port group inside of ESXi. In that case, you would be extending the trunk all the way from your physical infrastructure to the pfSense VM. In that case, you would need to create VLANs first in pfSense, as it would need to know what the VLAN IDs are to successfully read them.

Since we didn't take that approach, and we actually used untagged port groups and assigned each network interface of the VM to a separate untagged port group, we can skip this step. In one of the following posts I will show you how to use VLAN tagging to get the same functionality, but with a single port on the pfSense VM. For now, we will skip this step as we have our interfaces assigned with VLANs.

In the following picture, next steps are shown - we need to choose our WAN interface (vmx0), our LAN interface - or in our case it will be called Business (vmx1) and finally, our DMZ interface - in the initial setup it is called OPT1 interface (vmx2). We will change all of this naming when we get to the GUI of pfSense.

Next, choose option 2) to reconfigure IP addresses on our interfaces. We will leave WAN interface with DHCP, but we definitely want to change our LAN interface address. Just to clarify, again, our addressing is as follows:

• WAN: DHCP
• Business: 10.198.128.0/24
• DMZ: 10.198.129.0/24
• Home: 10.198.130.0/24

Once we have changed the IP address of the LAN interface, it should be accessible from within the Business VLAN. There are two options on how to do that - one is if you have physical environment, and connect your computer to a port that is part of VLAN 200, and the other one is to have a VM inside of ESXi that has port group of VLAN 200 associated with the network interface. Since I'm doing this whole lab from inside of a virtual environment, I will choose second option, but you can choose physical computer as well, as long as it is in VLAN 200.

If everything is OK, you should be greeted with this screen in your browser:

Initial credentials are admin/pfsense.

By logging into it, an initial startup wizard is started.

In the previous screenshot, note that I disabled Override DNS option as I want to make sure that specific DNS servers are used.

Since we already configured interfaces in CLI, we will leave most of the configuration as it is. Addressing of the OPT1 (DMZ) interface will be done after the initial setup wizard is complete.

Finally, we will change the admin password and once that is done, firewall will reload:

Once the firewall is reloaded, it will present you with the System Update and if everything is OK, status should be Up to date.

Next, we will change names of our interfaces - click on Interfaces and select LAN. Under description, change it to Business or whatever you want to name your interface.

One thing that needs to be changed on all of the interfaces is this:

Since we are working exclusively with private addressing scheme, we do NOT want to block any private networks, so disable both of these options. Since our firewall is behind an ISP router that provides private addressing scheme, there is a zero chance that any bogon networks will ever reach our firewall, so it is safe to disable the Block bogon networks option as well.

Leave the rest of the configuration unchanged and click on Save at the bottom of the screen.

Don't forget to apply the changes!

Last thing to do with interfaces is to rename the DMZ interface (from OPT1 to DMZ) and assign an IP address to it:

With interface configuration out of the way, we just need to make sure that our firewall and NAT rules are in place:

By default, pfSense will create necessary firewall rules for our LAN (Business) interface, so there's nothing left to do there.

For the DMZ, there are no rules in place, so we will create a basic allow any any rule. This will get us started, but we will modify this in the next parts of the homelab.

We will create a new firewall rule for DMZ interface:

Since we want to log our traffic, we also enable Log packets that are handled by this rule. Make sure that you save and apply the rule!

Finally, we need to check our outbound NAT rules:

We will change it to Hybrid Outbound NAT, as this will allow us to create manual NAT rules in the future, but for now, everything is as it should be - second rule in the list is the one that will allow us access to the Internet since it identifies all the networks configured on the firewall as source and for all destinations it uses firewall WAN address as its source IP.

Please note that, at the moment, there is no functional DNS server inside of our network. For that reason, we can actually ping to the outside, but we won't be able to open any web pages without either changing our DHCP server settings on the firewall to include DNS servers IP addresses :

By default, pfSense enabled the DHCP server, but it does not include DNS server in DHCP offers - if you don't want to wait until the next post that will deal with the DNS servers, you can easily modify your pfSense to include DNS servers in DHCP offers; go to Services -> DHCP Server and choose Business interface; scroll down to Servers section and populate DNS servers fields with your preferred DNS servers. Save the settings and apply them if needed, and then reconnect your computer and VM and it should now have DNS servers in your IP settings as well.

And that is it. We now have a fully functional firewall that can route traffic between all configured VLANs. In the next part, we will deploy our own DNS server as it is one of the essential parts of a fully functioning homelab environment.

Simple DNS setup is not that difficult to configure, but a misconfiguration in DNS is one of the most common root causes in network issues. Good understanding of how DNS operates is crucial in troubleshooting any networking issues. There's a reason why most of the network admins use the phrase "It's Always DNS", meaning that most of the networking issues are usually caused by a misconfigured DNS server :)