(Jan 2023) For my router, including DNS (BIND9) and DHCP (ISC DHCP) I am using a Supermicro SYS-E200-9B that comes with a Supermicro motherboard X11SBA-LN4F. I purchased this in 2016 and got functional in 2017, whilst waiting for NFTables to run all required features on Ubuntu. The X11SBA-LN4F has an Intel Pentium N3700 system with 4 x Intel i210-AT GbE LAN. I got with maximum 8GB RAM and 120GB mSata HD. Sadly the mSata HD was a Chinese branded unit that failed after 3 years operation. I replaced it with an old Samsung 256GB 860 SSD that I had on hand. I also took the opportunity to change the router from Ubuntu to Debian at this time. The N3700 CPU had reasonable performance at the time and includes AES instruction, which a number of common lower priced options at the time did not, e.g. J1900 CPU. The AES CPU instruction helps improve encryption performance significantly, handy for SSL / VPN. The unit is still performing well now. including the 10 year old Samsung SSD. I run the following software on it, all bare metal:
I would consider to try setting up a VM and Docker on this machine, however I suspect it maybe under powered for this. I would want Docker to be on a VM as I do not like the amount of IPtables configuration it does on its host. This would interfere with my NFTables router firewall configuration, if on the same host.
I looked at the various options for the router hardware, written in 2016.
tldr;
I don't see the point installing a 64bit OS on systems with less than 4GB of RAM. A 32bit OS can only natively access up 4 GB RAM, but should give better compromise with such limited RAM.
Router key features:
Operate reliably 24 hours per day, 7 days a week
Low power operation, power cost money
Headless Remote access
Hardware suitable for purpose:
At least 2 NICs (preferably native, not USB based)
Sufficient CPU power not to limit primary performance
Correct CPU options, e.g. AES
No graphical user interface environment install (although individual applications could have web interface)
Connectivity to upstream IPS provided internet
Firewall
-
DCHP
VPN for use as secure gateway to allow private access from public internet
The following key services that define the router:
Assumptions and Limitations
I would like to experiment with
Docker really does some work on the firewall using iptables. For this reason I decided to setup a virtual machine environment, Linux QEMU/KVM
IPMI KVM Display Problems
Acronyms can be painful. IPMI = Intelligent Platform Management Interface, KVM = Keyboard video and mouse, BMC = Baseboard management controller.
The remote KVM and IPMI, BMC are not used often, however they negate the need for the use of separate keyboards and monitors to set up and maintain these machines and allow true convenient headless set up, maintenance and operation. Normally an SSH terminal is all that is required, however a BMC with KVM allows full on/off/reset control and remote access to GRUB and terminal that SSH does not provide until after the base machine is running correctly.
The Pentium N3700 comes with a built-in graphics adaptor. On the headless BMC system the built-in graphics adapter is not required and can interfere with the BMC graphic adapter. The best solution is to turn off the Intel integrated graphics device (IGD), which is enabled by default. The graphics then defaults to the BMC adaptor. The IGD can be turned of from the BIOS motherboard options (In this case under Advanced-Chipset Configuration-North Bridge-Intel IGD Configuration). The terminal also seems to default to 1024×768 resolution, so no additional work is required for this. The 18.04 Server loader also had a problem with existing drive partitions, so I needed to manually remove all existing partitions using fdisk, from 18.04 install terminal.
My home server already in service over 5 years (as of 2017) has a Supermicro motherboard with Intel Atom C2750 CPU A1SAi-2750F also with IPMI, BMC & KVM and did not display this problem. This makes sense as the Atom C2750 CPU does not have a internal graphic capacity, so the only graphics capacity was on the BMC video controller. The Ubuntu drivers defaulted to this basic BMC graphics display system. (This is now my backup server and my main server is a
I now have a new server with the newer Supermicro motherboard with Intel Atom C3000 series CPU, also the 8 core version. (It was hard to justify the extra cost for the 12 or 16 core versions and I had no other hardware for the 10GB/s Ethernet option). The link to 8 core Supermicro motherboard with embedded 4 x 1GBe LAN A2SDi-8C+-HLN4F. This server is now running as my primary.
Forcing Display option at boot in Ubuntu
Note this method did not work in Ubuntu 18.04 amd64 server edition
Basically after setting up Ubuntu 16.04 amd64 server edition on the router hardware I noticed a problem with the IPMI KVM terminal display. During the Ubuntu start-up the KVM screen would just go blank. However login into a SSH session on the main board NIC was working normally. After a bit of head scratching and investigation I worked out the problem to be related to the design of Intel N3700 with the built graphics processor that was conflicting with the BMC graphics processor built into the motherboard a Supermicro X11SBA-LN4F in the also Supermicro SYS-E200-9B.
So the solution is to ensure that Ubuntu does not load any “special” main board (Celeron N3700) CPU graphic drivers. For Debian and Ubuntu this is done by setting the “nomodeset” option into the grub bootloader. This can be done by editing the grub bootloader during boot up, a one off solution and by making permanent by editing the grub configuration file. The reliablesite.net give a good explanation in their article How to set 'nomodeset' into the grub bootloader. At the grub menu hit the arrow key to select, select the default option, 1st line. Press the 'e' key to edit. Add the the 'nomodeset' option to the end of the line starting with 'linux'. Hit 'F10' to proceed with modified boot. For the permanent solution basically edit etc/default/grub, adding nomodeset such that GRUB_CMDLINE_LINUX_DEFAULT=“nomodeset” and then execute “sudo update-grub”. Note that the “quiet splash” options in GRUB_CMDLINE_LINUX_DEFAULT should be removed to allow all boot information to be seen.
Controlling BMC Terminal Resolution in Ubuntu
Note this method was not tested in Ubuntu 18.04 amd64 server edition
The BMC terminal screen seems to default to 640×480 resolution. To improve consider the following. Add 'GRUB_GFXPAYLOAD_LINUX=1024×768' to the etc/default/grub file. There are an number of other possible options, the default 640×480 and 800×600 are too small and the 1280×1024 and 1600×1200 option too big. To check the options at the grub menu type 'c' to input the command 'vbeinfo' to list the available grub video modes. You can also specify colour depth, e.g. 1024x768x24, but if this is not correct it totally ignores your parameter and goes to default. As I did not care about colour depth, I just used the resolution only, which seems more reliable. You can also similarly increase the Grub menu screen by adding 'GRUB_GFXMODE=1024×768' to the etc/default/grub file. This help make use of the grub menu easier. Always run 'sudo update-grub' to make the modified grub file the current boot one. An on-line reference ChangeTTYResolution.
Router Ethernet Hardware Consideration
The X11SBA-LN4F hardware comes with 4 dedicated NIC controllers. NIC0 is on a dedicated PCIe lane, whereas NIC1 to 3 use a multiplexer to share another PCIe lane. The PCIe lane with the 3 shared NIC controllers have enough bandwidth to handle maximum combined throughput of the 3 NICs, however the multiplex does add a minor processing delay, although better than an additional external switch. I suspect this probably does not have significant affect on final performance.
I plan to dedicate NIC0 to the WAN and bridge NICs 1-3 to the LAN. Also the bridged LAN network will used for the main server and its VMs with dedicated IP addresses on the LAN. The main NFTables based router will run on bare metal and a number of VMs used for DNS, DHCP, VPN and logger.