Trace: • KPTree Torrent VM Setup

KPTree Torrent VM Setup

← Back  
 Next →

This is an old revision of the document!

← Back  
Start page  
 Next →
← prev page  
 next page →

KPTree Torrent VM Setup

Deluge Headless setup

The main reference used is Havetheknowhow.com, specifically for the deluge headless setup, Havetheknowhow.com - How to install Deluge Headless. The Deluge support page is also a root source of knowledge Deluge systemd. Whatbox Deluge also has some interesting dialog.

If installing on a virtual machine (VM) it is important to also have set up the NFS to allow access to the main storage. Also, consider aligning the appropriate file system user and groups between the VM server and VM guests. (Check the local deluge directory actually exists for the user as otherwise the deluge and deluge web daemons will crash and not load. (For the Ubuntu Network Filing System, NFS, see Havetheknowhow.com - How to configure NFS Version 4, that contains configuration information for both the VM server and clients.

Unfortunately, Havetheknowhow.com does not seem to cover the alignment of user and groups between the main OS and VMs. I create a basic template machine and manual update the group and user id numbers to align, a use the basic information given in Linux: Changing UIDs and GIDs for a user. The process is tedious and takes a bit of care to complete, but once setup properly allows better operation between the server and virtual machines.

I use a dedicated VM guest for Deluge with a VPN set up on this VM. I have been using the VPN BTGuard for a few years now without any significant problems, save for BTGuard changing the IP address of their servers on occasion without informing end users, this affects the firewall software as noted below.

Recent changes not fully updated:

  • I no longer use BTGuard, they were expensive and seem technically outdated. (There is a section on this below.)
  • I am using Debian as the Media VM now. (I believe I will try to move fully to Debian for my server in the future instead of Ubuntu. As noted elsewhere I am now using Manjaro (XFCE) on my main desktop now instead of MSWindows. Manjaro is based upon Archlinux, not Debian or Redhat and seems a good desk top choice. I am certainly happy with it after using over 6 months. I am still forced to use Windows 10 and MS corporate software at work.)

deluge systemd setup

Setting up deluged and and deluge-web as systemd services is described in deluge docs How to create systemd services for Linux

We want to delay the start of the deluged.service to after the related drives are mounted.

First we need to check the units on on the machine. Use sudo systemctl list-units and look for the relevant .mounts . In mycase: 

  -.mount               loaded active mounted   Root Mount
  mnt-deluge.mount      loaded active mounted   /mnt/deluge
  mnt-disk2.mount       loaded active mounted   /mnt/disk2

Similarly, in the same listing look for the relevant .services: 

  deluge-web.service    loaded active running   Deluge Bittorrent Client Web Interface
  deluged.service       loaded active running   Deluge Bittorrent Client Daemon

systemd-analyze – Find System Boot-up Performance Statistics in Linux

OpenVPN with PrivateVPN

I decided to go with PrivateVPN that seems to get reasonably good reviews at the time of change over (2020-04-18) and my old VPN provided (BTGuard) was getting very poor reviews, was up for renewal, and was becoming comparatively expensive. The following is a list of resources to help with Linux Ubuntu CLI setup:

The PrivateVPN OpenVPN Linux CLI setup and associated download script worked well and was similar to the BTGuard setup, with the following issues easily found:

  • Once the script was run PrivateVPN recommended using sudo privatvpn to run. The setup script creates a systemctl profile: openvpn@privatvpn.service. So the standard systemctl commands made much more sense to use! e.g. sudo systemctl start openvpn@privatvpn.service with other standard systemctl operators: stop, status, enable, disable, etc. Similarly to get a better look at current log status: journalctl -u openvpn@privatvpn.service -xe.
  • The description suggested editing /etc/privatevpn.conf to change the VPN servers, however the correct file was /etc/openvpn/privatevpn.conf.
  • The remote server server in the downloaded script no longer was correct and had to be changed to one of the listed one in PrivateVPN Server List. The listed servers all look to now end with the domain .host.
  • The PrivatVPN tunnel host IP address is different from the end IP address, unlike BTGuard which were the same. The host IP address must be used in the NFTables firewall to prevent VPN leakage.
  • I use https://www.findip-address.com/ to help find the corresponding ip address of the server.
  • I then allow the tunnel VPN to pinhold these addresses only in my nftables setup, sudo vim /etc/nftables.conf. Basically the nftables setup firewalls the system to only allow wan through the tunnel, except the pinhole address required to setup the tunnel.
  • wget http://ipinfo.io/ip -qO - to check my current external ip address.
  • Unfortunately PrivatVPN changes their server ip address and does not seem to tell their end users.

OpenVPN setup with BTGuard

I decided to move away from BTGuard after many years of service. Primary reasons: 1. They were becoming more expense (in part of currency variation with USD) and 2. Their services seemed not to be very comprehensive and up to date. Basically not the best value for money.

BTGuard Setup Notes

Some Handy Commands to Check Configuration

  • To start the vpn service: sudo systemctl start openvpn@privatvpn.service, was openvpn@btguard.service
  • To check the end of the journal file for the vpn specific comments from the last boot: journalctl -u openvpn@privatvpn.service -b 0 -xe
  • To check current external IP address: wget http://ipinfo.io/ip -qO -
  • To list current tables in nftables: sudo nft list table ip firewall, not inet, specifically ip, as ip6 is disabled.
  • To start nftables: sudo systemctl start nftables
  • To check the journal for comments on nftables since the last boot: journalctl -u nftables.service -b 0
  • Other systemctl directives:
    • enable : to start the systemctl service at boot
    • disable : to not start the systemctl service at boot
    • start : to start the systemctl service
    • stop : to stop the systemctl service
    • status : to provide the status information of the systemctl service.

Some of the last journal log lines are shown too, to see all the journal log use the journalctl command. Some journalctl directives:

  • -u service_unit only list for service_unit
  • -b n Only list boot n, no number or 0 for last boot and 1 for previous and 2 for 2 previous etc.
  • -xe List end of file
  • -f Live continuous tail listing, until keyboard ^c

NFTables Configuration

NFTables supersedes IPTables, although IPTables is still most common in use. I have taken the effort to learn both IPTables and NFTables and prefer the syntax of NFTables.

A problem with the OpenVPN system is should it fail then the IP connectivity falls back to the standard connection with all the VPN benefits lost. To diminish this risk, I have set up the Deluge VM with a NFtables based firewall to effectively stop external internet connection upon any failure of the VPN, that is only the VPN connectivity is allowed. I have learned from the original IPTables based configuration below. With the ease of use of NFTables I have been able to improve and optimise the firewall.

To install nftables: “sudo apt update” followed by sudo apt install nftables

If the nftables repository can not be found in Ubuntu 18.04 they will need to be added: sudo vim /etc/apt/sources.list and add the following repositories: 

  deb http://archive.ubuntu.com/ubuntu/ bionic universe
  deb http://archive.ubuntu.com/ubuntu/ bionic-updates universe
  deb http://security.ubuntu.com/ubuntu bionic-security universe

Sample NFTables configuration script for PrivatVPN

sudo vim /etc/nftables.conf

nftables.conf script

  • Select the server from PrivateVPN Server List.
  • The host IP address can be found using the following commands: host au-mel.pvdata.host or ping au-mel.pvdata.host.
  • Edit the vpn (host name) and firewall (host ip address) configuration files, respectively: sudo vim /etc/openvpn/privatvpn.conf & sudo vim /etc/nftables.conf.
  • Restart the respective services: sudo systemctl restart openvpn@privatvpn.service & sudo systemctl restart nftables
  • Remember the host IP address and end point IP address are not the same. To check end point IP use: wget http://ipinfo.io/ip -qO -

Sample NFTables configuration script for BTGuard

sudo vim /etc/nftables.conf

nftables.conf script

The use of the counter directive allows the usage to be checked. I commented out lines that were not being used.
To list the firewall: sudo nft list table ip firewall

IPTables Configuration

A problem with this protection is should the OpenVPN system fail then the VPN protection will fail. To diminish this risk, I have set up the Deluge VM with an IPtables based firewall to effectively stop external internet connection upon any failure of the VPN. (The reference for the iptables filter script was AirVPN - Prevent leaks with Linux & iptables )

Sample IPTables configuration script

BTGuard_iptable.sh:

TL;DR;

  • Remember if creating the script from scratch to make executable use: sudo chmod a+x BTGuard_iptable.sh
  • Use the route, ifconfig, ip a and ip route commands to help check configuration of your network. (As noted below, particularly check the eth0 / tun0 references match.)
  • The network configuration file can be found at /etc/network/interfaces
  • If iptables is not installed run sudo apt install iptables
  • Ensure the network reference such as “eth0” is consistent, particularly with Ubuntu 16.04 VM which seems to rename this reference, see Changing Network Interfaces name Ubuntu 16.04 for further information on this matter.
  • The outgoing packet IP addresses, including subnet masks, must match those specified by BTGuard.
  • To get the external IP address use: wget http://ipinfo.io/ip -qO -
  • To see the current openvpn status use: sudo systemctl status openvpn@btguard.service or journalctl -u openvpn@btguard.service -b

How to make IPTables configuration permanent / restore on boot

Reference How to save rules of the iptables? and Ubuntu Netfilter manual

  • Install iptables persistent sudo apt install iptables-persistent
  • To save current IPTable configuration use: sudo netfilter-persistent save or journalctl -u openvpn@btguard.service -f, where options are; -u is followed by the service and the -f give continuous output, use cont-c to breakout.

Kodi Setup

I have been using WD Media players around the house the past 8 years or so. Originally with individual 2.5“HD and then connecting to my home server. These units are very simple to use and setup and have played media files well. Circa 2016 I notice that some media files will not play anymore. Upon investigating the WD (Western Digital) web site I notice that these units look to be no longer sold and support is limited, perhaps already end of life. Earlier investigation of these unit indicate the internal hardware is quite limited, particularly by current standards, but as already stated good enough as a general media player.

I notice a few years ago the the web site Have the know how recommended the Kodi Overview, in fact even states, ”Kodi: The holy grail of media streamer front-ends“. I will not go over the benefits and history of Kodi, just look at the Kodi home page and Have the Know How web pages.

Kodi can be loaded on to many different types of hardware and operating systems, it even has a Linux based OS design for it, OpenELEC. Specifically it does run on MS Windows and most flavours of Linux.

Kodi Server MariaDB (MySQL) Setup

I have been using MariaDB instead of MySQL, as MariaDB is true opensource software and MySQL belongs to a large corporation. MariaDB has a few “quirks” to setup comparied to MySQL which is more commonly described.

Make sure that MySQL is NOT installed, if so uninstall it.

MariaDB can be installed by sudo apt install mariadb-server. The latest version can be downloaded from instructions from MariaDB Foundation Downloads Setting up MariaDB Repositories.

To setup secure root access:

  • sudo mysql -u root
  • use mysql;
  • update user set plugin='' where User='root';
  • flush privileges;
  • \q
  • mysql_secure_installation
  • Add password, and then Y for all other questions

Check status of mariadb sudo systemctl status mysql

Next move MySQL Data Directory

Check current location of MySQL data directory: mysql -u root -p and then at mysql> select @@datadir;. It should be /var/lib/mysql. Exit mysql prompt with \q.

Stop MySQL : sudo systemctl stop mysql

My the mysql data directory to your preferred location: sudo mv /var/lib/mysql /mnt/shared/kodi

Next create a symlink pointing to the new location: sudo ln -s /mnt/shared/kodi/mysql /var/lib/mysql. Another option instead of using a symlink is to adjust the mysql.cnf file datadir parameter, i.e. change datadir=/var/lib/mysql to datadir=/mnt/shared/kodi/mysql. Again most the documentation on mysql.cnf is for mysql not mariadb, which has its own quirks. See the notes on adjusting bind-address below to clarify.

The current version of mariadb does not use apparmor, so no need to adjust for this!

The mysql network access needs to be configured to all full LAN access, default access is localhost only. The mysql.cnf configuration file needs to be updated. Mariadb handles this differently than MySQL. The main configuration file /etc/mysql/mariadb.cnf points to other directories with configuration files. Checking these directories shows the mariadb server parameters are located in /etc/mysql/mariadb.conf.d/50-server.cnf, to edit: sudo vim /etc/mysql/mariadb.conf.d/50-server.cnf. Find the parameter bind-address = 127.0.0.1 and change to the server IP address bind-address = 192.168.1.19.

Restart mysql: sudo systemctl restart mysql, check all is running well: sudo systemctl status mysql, and then check listening address on mysql port: sudo lsof -i -P | grep :3306

Next setup the mysql kodi database access:

  • mysql -u root -p
  • create user 'kodi' identified by 'kodi';
  • grant all on *.* to 'kodi';
  • show grants for 'kodi';
  • \q

Kodi Headless Server Setup

Old to be deleted

Docker Setup - Preliminary

Docker seems to be a reasonably recent variation based upon quite old concepts with modern aspects thrown in. The documentation seems a bit sparse. I found it difficult to find a good balanced reference. A good reference is of course the Docker Website itself: Get Started, Part 1: Orientation and setup. A good reference (pdf) was from Anthony Blair of Universit'e de Rennes, Docker Tutorial. Many of the Docker tutorials seem to focus on features that I am not interest at this time, such as Docker swarms and not enough on basic usage.

Some handy docker commands and related:

  • docker exec -it <name> /bin/bash to get terminal inside container <name>
  • sudo netstat -nlp
  • docker rm <name>, to remove docker <name> container, container must be stopped docker stop <name>
  • docker ps, to list running docker containers, docker ps -a to list all docker containers, running and stopped
  • To remove an unused docker images, docker rmi <Repository>, list all images with docker images -a
  • docker logs <name>, to see docker <name> log file, and docker logs -f <name>, to see docker <name> log file online output
  • docker port <name> to see docker <name> port mappings to host
Start container at boot:

It is assumed that the container has been previously created/run and is available to be started at boot.

sudo vim /etc/systemd/system/docker-kodi-headless.service 

[Unit]
Description=kodi-headless container
Requires=docker.service
After=docker.service

[Service]
Restart=always
ExecStart=/usr/bin/docker start -a kodi-headless
ExecStop=/usr/bin/docker stop -t 2 kodi-headless

[Install]
WantedBy=default.target
  • Get external IP address wget http://ipinfo.io/ip -qO -
  • Save the current netfilter (IPTables) configuration sudo netfilter-persistent save
  • To check the current network hardware configuration ip a
  • System command to start(/stop /status) the OpenVPN program sudo systemctl start openvpn@btguard.service
  • To run script with IPTABLES configuration (netfilter) sudo /mnt/shared/scripts/BTGuard_iptable_ens3.sh
  • To label list current IPTABLES configuration, verbose sudo iptables -L -v or simple sudo iptables -S
  • To edit BTGuard openVPN configuration file sudo vim /etc/openvpn/btguard.conf
  • To flush the current IPtables configuration sudo iptables -F
  • To check log on OpenVPN with BTGuard (/VPN /open) grep btguard /var/log/syslog
  • To check running process with open for openvpn(/del, for deluge) ps -A | grep open
  • To start (/stop /enable) the deluge daemon sudo systemctl start deluged
  • To start (/stop /enable) the deluge web interface daemon sudo systemctl start deluge-web
  • Systemd common commands (start / stop / restart / status) (enable / disable for boot control)
← prev page  
Start page  
 next page →
/app/www/public/data/attic/home_server/media_vm_setup.1621671040.txt.gz · Last modified: 2023-04-30 Sun wk17 17:44
CC Attribution-Share Alike 4.0 International Except where otherwise noted, content on this wiki is licensed under the following license: CC Attribution-Share Alike 4.0 International