Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Both sides previous revisionPrevious revision
Next revision		Previous revision
linux_router:hardware [2023-04-16 Sun wk15 08:43] – [Router Hardware] baumkplinux_router:hardware [2024-02-24 Sat wk08 12:12] (current) – [Router Hardware] baumkp
Line 11: Line 11:
 ++++tldr;| ++++tldr;|
   *A small ARM based machine, e.g. Raspberry Pi 3. (The current RPi looks much more capable.) However these machines are generally limited in a number of way, including by definition not x86 based. Many do not have more than one NIC and the NIC are often not full Gigabit. (To be fair this hardware may be sufficient in most cases, as most homes do not have better than 100Mb/s internet connections, and in general much slower.) The main upside is that they are small, low power and relatively cheap. Those with only one NIC need to be setup with USB NIC adaptors, that further complicates setup, performance and reliability. Although better spec'ed machines, e.g. with multiple gigabit NICs, start getting more pricey too. I suppose you get what you pay for....   *A small ARM based machine, e.g. Raspberry Pi 3. (The current RPi looks much more capable.) However these machines are generally limited in a number of way, including by definition not x86 based. Many do not have more than one NIC and the NIC are often not full Gigabit. (To be fair this hardware may be sufficient in most cases, as most homes do not have better than 100Mb/s internet connections, and in general much slower.) The main upside is that they are small, low power and relatively cheap. Those with only one NIC need to be setup with USB NIC adaptors, that further complicates setup, performance and reliability. Although better spec'ed machines, e.g. with multiple gigabit NICs, start getting more pricey too. I suppose you get what you pay for....
-  *The Raspberry Pi 4 looks like a much better option than earlier versions for a home router. Still has the complexity of only native 1 NIC, but that is full 1Gbe and there are 2 USB 3 port to allow another full 1Gbe NIC off USB.+  *The Raspberry Pi 4 & 5 looks like a much better option than earlier versions for a home router. Still has the complexity of only native 1 NIC, but that is full 1Gbe and there are 2 USB 3 port to allow another full 1Gbe NIC off USB.
   *An older x86 based machine. The main downside to these is poor power consumption and large size, even an old server tends to use more than 30W at the wall, or greater than $60/year power. Also the board I had only had one built in NIC, so I would need a PCIe NIC card. There is also the issue of reliability and performance for the older hardware, although it is probably good enough in this respect. That all being said if one is strapped for cash this may be a good way to start as the upfront cost would be smallest, if not zero.   *An older x86 based machine. The main downside to these is poor power consumption and large size, even an old server tends to use more than 30W at the wall, or greater than $60/year power. Also the board I had only had one built in NIC, so I would need a PCIe NIC card. There is also the issue of reliability and performance for the older hardware, although it is probably good enough in this respect. That all being said if one is strapped for cash this may be a good way to start as the upfront cost would be smallest, if not zero.
   *At the moment, 2016, there are a lot of Intel Celeron J1900 based units with 4 NICs around. The J1900 is an older CPU, 4 cores, 2.0-2.42 GHz. Also in many cases the NIC hardware is older, particularly on the cheaper units, so care must be taken if you want to ensure more up to date hardware. These machines are a good option, low power (~8 - 10W), small size. They come with 2 SATA ports and mini PCI-E slots. By the time you fit them out they cost out USD250 - 350, with 4-8GB RAM and 120GB mSata drive. The cheaper options are as noted above usually with older NIC hardware and lower memory and HD size and can be had at even lower prices.   *At the moment, 2016, there are a lot of Intel Celeron J1900 based units with 4 NICs around. The J1900 is an older CPU, 4 cores, 2.0-2.42 GHz. Also in many cases the NIC hardware is older, particularly on the cheaper units, so care must be taken if you want to ensure more up to date hardware. These machines are a good option, low power (~8 - 10W), small size. They come with 2 SATA ports and mini PCI-E slots. By the time you fit them out they cost out USD250 - 350, with 4-8GB RAM and 120GB mSata drive. The cheaper options are as noted above usually with older NIC hardware and lower memory and HD size and can be had at even lower prices.
-  *I decided to get a Supermicro [[https://www.supermicro.com/products/system/Mini-ITX/SYS-E200-9B.cfm|SYS-E200-9B]] that comes with a Supermicro motherboard [[https://www.supermicro.com/products/motherboard/X11/X11SBA-LN4F.cfm|X11SBA-LN4F]], an Intel Pentium N3700 system with 4 x Intel i210-AT GbE LAN, from [[https://mitxpc.com/products/sys-e200-9b|Mitxpc]]. I got with maximum 8GB RAM and 120GB mSata HD. The N3700 CPU is more modern than the J1900 and includes AES instruction that the J1900 does not have. The AES CPU instruction helps improve encryption performance significantly, handy for SSL / VPN. Otherwise the overall performance is similar (4 cores at 1.6-2.4GHz) and power slightly lower than the J1900. (The Intel LAN controllers are also the more modern ones). This unit also comes with a dedicated IPMI LAN Port, allowing full remote KVM operation on the network. A downside of the IPMI is that it uses another 3.5W of power (1W power 24/7 costs $2.29/year @ $0.25/kWhr, so 3.5W IPMI costs $7.67/yr extra for power over the main units 9W at $19.71/year). The upside is that the unit can be remotely off-site operated, with configuration options for auto on at power up and heart-beat with auto reset. (My home server is also a Supermicro based unit with dedicated IPMI LAN Port and has given me a good 5 years of service to date.) Downside is mainly the price, USD490 + delivery, as these units are not sold locally I purchase in USA and had it mailed at USD75. In any case this hardware should allow for a router with great performance for some years to come. Again you get what you paid for.....+  *I decided to get a Supermicro [[https://www.supermicro.com/products/system/Mini-ITX/SYS-E200-9B.cfm|SYS-E200-9B]] that comes with a Supermicro motherboard [[https://www.supermicro.com/products/motherboard/X11/X11SBA-LN4F.cfm|X11SBA-LN4F]], an Intel Pentium N3700 system with 4 x Intel i210-AT GbE LAN, from [[https://mitxpc.com/products/sys-e200-9b|Mitxpc]]. I got with maximum 8GB RAM and 120GB mSata HD. The N3700 CPU is more modern than the J1900 and includes AES instruction that the J1900 does not have. The AES CPU instruction helps improve encryption performance significantly, handy for SSL / VPN. Otherwise the overall performance is similar (4 cores at 1.6-2.4GHz) and power slightly lower than the J1900. (The Intel LAN controllers are also the more modern ones). This unit also comes with a dedicated IPMI LAN Port, allowing full remote KVM operation on the network. A downside of the IPMI is that it uses another 3.5W of power (1W power 24/7 costs $2.29/year @ $0.25/kWhr, so 3.5W IPMI costs $7.67/yr extra for power over the main units 9W at $19.71/year). The upside is that the unit can be remotely off-site operated, with configuration options for auto on at power up and heart-beat with auto reset. (My home server is also a Supermicro based unit with dedicated IPMI LAN Port and has given me a good 5 years of service to date.) Downside is mainly the price, USD490 + delivery, as these units are not sold locally I purchase in USA and had it mailed at USD75. In any case this hardware should allow for a router with great performance for some years to come. Again you get what you paid for..... So some 7 years later I am having problems with the BMC on this unit, it is very unreliable now and requires the entire computer is reset, which in many aspect defeats the purpose of having it. The main unit otherwise works, but it is now much more difficult to use headlessly. The main unit is still go enough for my home internet which can be provided up to 1000Mb/s, however is usually much lower than this upstream....
  
 <fs smaller> I don't see the point installing a 64bit OS on systems with less than 4GB of RAM. A 32bit OS can only natively access up 4 GB RAM, but should give better compromise with such limited RAM.</fs> <fs smaller> I don't see the point installing a 64bit OS on systems with less than 4GB of RAM. A 32bit OS can only natively access up 4 GB RAM, but should give better compromise with such limited RAM.</fs>
 ++++ ++++
  
-=====Docker on Router=====+====VM / Docker on Router==== 
 +===Progress=== 
 +As of 2023/01 I setup a VM manager (Libvirt/qemu/KVM) on the router and loaded Docker on it.  It is slow but does seem to work. 
 +Next: 
 +  *ISC Kea DHCP in Docker (currently ISC DHCP in bare metal) 
 +  *ISC Bind 9 in Docker (currently ISC Bind 9 in bare metal) 
 +  *Wireguard VPN in Docker (currently Wireguard VPN in bare metal) 
 + 
 +===Router key features=== 
 +  - Operate reliably 24 hours per day, 7 days a week 
 +  - Low power operation, power cost money 
 +  - Headless Remote access, with separate BMC NIC (this could be integrated or external KVM, e.g. [[https://pikvm.org/|PiKVM]]) 
 +  - Hardware suitable for purpose: 
 +    - At least 2 NICs (1 WAN plus 1 or more LAN, quality native type NICs, not USB based), 4+ NICs preferable. 
 +    - NICs to be 1 GB/s type minimum, although as of 2023 2.5GB/s NIC would now be minimum specification 
 +    - Sufficient CPU power not to limit primary performance 
 +    - Correct CPU options, e.g. AES, [[https://www.intel.com/content/www/us/en/virtualization/virtualization-technology/intel-virtualization-technology.html|virtualization]] (VT-x, and as of 2023 VT-d). 
 +  - No graphical user interface environment install (although individual applications could have web interface) 
 +  - Connectivity to upstream IPS provided internet 
 +  - Firewall 
 +  - DNS 
 +  - DCHP 
 +  - VPN for use as secure gateway to allow private access from public internet 
 +The following key services define the router: 
 +  *network services (bare metal) 
 +  *ISP Internet connectivity (bare metal) 
 +  *main firewall (bare metal) 
 +  *DNS 
 +  *DHCP 
 +  *VPN (for secure public access to LAN) 
 + 
 +===Assumptions and Limitations=== 
 +  *Low power means low CPU resources, hence care with applications that require significant or otherwise unnecessary resources. 
 +  *Some services on bare metal to ensure reliable performance 
 +  *This machine is much slower than usual hardware, and this is noticeable on interface usage, even no graphical. 
 +  *The network and related services performance must NOT limit performance on upstream IP connectivity to greater than 100Mb/s and preferably only limit as speed get close to NIC's 1 Gb/s hardware speed.  (At the moment my internet connection is via VSDL and is limited to about 65Mb/s down and 16MB/s up and this hardware and setup seem to be performing well.) 
 + 
 +Docker really does some work on the firewall using iptables.  For this reason I decided to setup a virtual machine (VM) environment, Linux QEMU/KVM/Libvirt based. VM's seem to impact the firewall / network setup less adversely than Docker. The use of the VM isolates the Docker firewall machinations from the bare metal.  
 + 
 +===Why not Proxmox=== 
 +++++tldr;| 
 +  *I have not used to date, this is I have no experience with Proxmox 
 +  *I already have a lot of experience on run Debian, libvirt/qemu/kvm, which is what Proxmox seems to be built on 
 +  *Proxmox seems to need to be installed on bare metal.  I am not so sure this would work well with my bare metal firewall feature requirements 
 +++++
 ====Specific issues with use of headless X11SBA-LN4F hardware==== ====Specific issues with use of headless X11SBA-LN4F hardware====
 ++++IPMI KVM Display Problems| ++++IPMI KVM Display Problems|