KPTree - Email Server Setup

• sudo apt install netplan.io
• sudo mv /etc/network/interfaces /etc/network/interfaces.old
• sudo /etc/netplan/network.yaml

  • network:
       version: 2
       renderer: networkd
       ethernets:
            enp1s0:
              addresses: [192.168.1.16/24]
              gateway4: 192.168.1.1
              nameservers:
                  addresses: [192.168.1.1, 1.1.1.1]

• sudo apt install nfs-common
  • The base client configurations options are given in: sudo vim /etc/default/nfs-common. However the current base options meet our needs and do not need to be modified.
    • NEED_STATD=no :(default) we do not need to change the standard port number used
    • NEED_GSSD=no :(default) we do not use kerebos security
    • NEED_IDMAPD=no:(default) we do not need to map gid and uid with names
    • To apply configuration changes sudo systemctl restart nfs-client.target
  • Mounting the nfs client manually:
    • sudo mount -t nfs -o rw,vers=4 192.168.1.10:/export/storage /mnt/storage to mount just the specific directory. The mount directory must already exist.
    • sudo mount -t nfs -o rw,vers=4 192.168.1.10:/export /mnt to mount all the defined exports. The mount directory must already exist, not necessarily the sub-directories.
    • sudo unmount /mnt/storage to unmount
  • Check status of clients:
    • findmnt to list file system in tree format or findmnt -l in flat list.
    • df -h to list all mounted drives
    • lsblk to list all drive
• To allow automatic boot of nfs clients: sudo vim /etc/fstab
  • 192.168.1.10:/export /mnt nfs4 bg 0 0 this will mount all the drive defined on the server as sub-directories under
  • 192.168.1.10:/export/shared /mnt/shared nfs4 bg 0 0 to mount a single sub-directory

My first email server was setup on an Ubuntu 14.04 virtual machine on my home server. It was based upon the setup recipe given in the blog “Beware Here Be Musings”, Installing a Mailserver on Ubuntu 18.04 LTS, Part 1 and Part 2. I got the mailserver to run well after a couple of attempts. It is simple and performs well and a good starting point to learn from. Limitations include: the database is a semi-manual setup, and addition of virtual mail users must be performed manually. There is also no webmail setup instructions which generally requires integration with a web server and the associated setup. The blogging.dragon.org.uk mail setup guide has not been updated for Ubuntu 20.04 as of Jan 2022, it looks like this blog also has no significant activity in the past 3 years.

Most recipes I found are limited in some ways. They assume use of a commercial remote server, whereas I have setup on a homeserver on a virtual machine. Although adjustments to run on my home sever virtual machine were relatively straight forwards. Few note the external setup requirements, DNS, MX, SPF and reverse DNS, which must be performed to get operating, and ensure best operating practices, a notable exception being Linode's Running a Mail Server.

A more comprehensive recipe I found was from Ex Ratione - A Mailserver on Ubuntu 18.04: Postfix, Dovecot, MySQL, Postfixadmin, Roundcube. An Apache webserver is installed for webmail and Postfix admin. If you look at the Exratione archives there are earlier Ubuntu version setup recipes. Sadly the last post on Exratione website is the Unbuntu 18.04 mailserver and as of over 3 years after 18.04 this is now outdated.

I am slowly moving away from Ubuntu to Debian for my various home servers. I find that Ubuntu seems to add a lot of bloat to their server distros compared to Debian. I find that I do not need these extra Ubuntu features, whatever they maybe. Perhaps the only benefit if Ubuntu is that it has a quicker LTS update cycle and longer LTS support period than Debian, however ever this is only marginal at best. In general for my home sever applications reliability and stability are more important than the latest and greatest. I am preferentially using Manjaro Linux for my home desktop, instead of Windows. Manjaro is a rolling release.

• ISPmail on Debian Buster – your mail server workshop This is a key source*
• Run your own mail server with Postfix and Dovecot
• How to set up a mail server in Ubuntu or Debian
• How to set up a mail server on a GNU / Linux system.

*Sadly many of older reference resources are no longer directly available, but many can found on wayback machine

Basically follow the setup given in A Mailserver on Ubuntu 18.04: Postfix, Dovecot, MySQL, Postfixadmin, Roundcube, with the following comments:

• Ignore the section “Using Amazon Web Services”
• For “Use of example.com and mail.example.com” use kptree.net and mail.kptree.net instead
• For “Fire up an Ubuntu 18.04 AWS Instance with a Suitable Security Group” prefer the following instead
  • Setup an Ubuntu 18.04 VM with 4 CPUs and 4G of RAM, no/disabled SWAP drive (Expect 2 CPUs and 2 GB RAM sufficient based upon text)
  • My home router needed to have the port forwarding setup to redirect inbound Internet TCP traffic to the local VM IP address for the nominated ports.
  • I never open VM SSH access ports to the Internet…..
  • The Internet ports are only redirected later in the build, when necessary for testing / implementation
• For “Now Build a LAMP Web Server” I manually install the various LAMP install components, as I prefer to use Mariadb instead of MySQL
• For “Configure MySQL” with Mariadb refer to SQL_MODE:

sudo mysql -u root -p
SET GLOBAL sql_mode = 'ONLY_FULL_GROUP_BY,STRICT_TRANS_TABLES,ERROR_FOR_DIVISION_BY_ZERO,NO_AUTO_CREATE_USER,NO_ENGINE_SUBSTITUTION';
SET SESSION sql_mode = 'ONLY_FULL_GROUP_BY,STRICT_TRANS_TABLES,ERROR_FOR_DIVISION_BY_ZERO,NO_AUTO_CREATE_USER,NO_ENGINE_SUBSTITUTION';
SELECT @@SQL_MODE, @@GLOBAL.SQL_MODE; 

• For “Install Postfix Admin 3.2 and the MySQL Schema” the latest version of is postfixadmin-3.2, as of 2019-12-01, so no change…
• Problems with running https://mail.example.com/postfixadmin/setup.php failed with debug error relating to index to long. Basically indexes are restricted to 1000byte length. So the index with char(255) is ok with char length byte, however modern databases seem to default to utf8mb4 (4 bytes length) or UTF (3 bytes length) to allow for international character sets. Refer to the following links:
  • Sourceforge Postfix Admin #379 Setup fails on MySQL with non-Latin1 as default charset.
  • Converting Character sets in MySQL to UTF8
  • Code to convert default character to allow setup to work:
    • Login to MySQL: sudo mysql -u root -p
    • Use the mail database use mail; (show database; to confirm available databases)
    • Confirm current character sets in use for this database: “SHOW VARIABLES LIKE 'char%';”
    • To convert the default database character set: ALTER DATABASE mail COLLATE=latin1_swedish_ci;
• For the Configure Dovecot Section
  • The Dovecot configuration file /etc/dovecot/dovecot-sql.conf.ext
    • My vmail user uid has historically been 6004. The mail uid and gid are both 8 on default Ubuntu (Linux?) distributions.
    • I propose to install the working mail directory on an NFS drive to help limit the VM drive size, /mnt/shared/vmail.
  • Follow change directions for the Dovecot configuration file /etc/dovecot/conf.d/10-auth.conf
  • The Dovecot configuration file /etc/dovecot/10-mail.conf
    • I propose to install the working mail directory on an NFS drive to help limit the VM drive size, /mnt/shared/vmail.
    • My vmail user uid has historically been 6004. The mail uid and gid are both 8 on default Ubuntu (Linux?) distributions.
  • Note that /etc/amavis/conf.d/15-content_filter_mode directives were back-slashed as follows:
    • @bypass_virus_checks_maps = (
    • \%bypass_virus_checks, \@bypass_virus_checks_acl, \$bypass_virus_checks_re);
    • @bypass_spam_checks_maps = (
    • \%bypass_spam_checks, \@bypass_spam_checks_acl, \$bypass_spam_checks_re);
  • Configuration file /etc/default/spamassassin not changed as directed, as on systemd use sudo systemctl enable spamassassin.service instead. CRON=0 changed to CRON=1.
  • Attempting to run “sudo freshclam” came up with error. I had to disable the clamav daemons to allow the update to work.
    • To find clamav related running daemons sudo systemctl list-units -all | grep clam
    • Stop the daemons sudo systemctl stop clamav-daemon and any other found daemons
    • sudo freshclam does not functions without error
    • Restart clamav sudo systemctl start clamav-daemon, however a sudo reboot may be in order….
• For the Configure Postfix Section no major changes (other than ensuring consistency with configuration site specifics, as above)
• Restart Everything, and Test the Server - check /var/log/mail.log & /var/log/mail.err
• Log showed error [postfix/smtpd[15073]: warning: SASL: Connect to private/auth failed: No such file or directory] and [postfix/smtpd[15073]: fatal: no SASL authentication mechanisms]. Upon restarting dovecot and then postfix, this error went away. (Restarting Dovecot initialises the file /var/spool/postfix/private/auth as per configuration /etc/dovecot/conf.d/10-master.conf after which this Postfix error is resolved.)
• Added compatibility_level = 2 to /etc/postfix/main.cf concerning log message “Postfix is running with backwards-compatible default settings”

• sudo certbot certonly --webroot --dry-run --webroot-path /var/www/mail.kptree.net -d mail.kptree.net
• sudo vim /etc/apache2/sites-available/mail.kptree.net-https.conf to edit the apache site configuration file (https)
• sudo vim /etc/apache2/sites-available/mail.kptree.net-http.conf to edit the apache site configuration file (http)
• sudo apachectl configtest to test an apach2 configuration
• sudo systemctl reload apache2 to restart apache2 and enable any configuration changes, could also use restart instead of reload to test (dry) run the certbot renewal process. Cerbot only allow 5 new certificates for a domain per week.
• sudo certbot renew –dry-run for certificate renewal test
• sudo certbot renew for certificate renewal run

Use sudo apache2 -S to get current status

• Using Certbot –post-hook to configure Let’s Encrypt Certs for use by multiple services
• Apache HTTP Server Project:
  • Apache Module mod_alias
  • <Location> Directive
  • <Directory> Directive

IMAP Client Settings
   Email address: admin@example.com
   Server Type: IMAP
   Server: mail.example.com
   Port: 143
   Username: admin@example.com
   Encryption method: STARTTLS  (not SSL/TLS)
    Tick, Server requires authorisation
   Authentication: Password
POP3 Client Settings - not used any more
SMTP Client Settings
   Server: mail.example.com
   Port: 567
   TICK, Server requires authentication
   Encryption Method: TLS (start after connecting) (or STARTTLS)
   Username: admin@example.com
   Auth Type: Login 

Gmail
  IMAP Client Settings
    server address: imap.gmail.com
    user name: Your full Gmail address (e.g. me@gmail.com)
    password: Your Gmail password
    With Gmail 2-step authentication enabled, use an application-specific Gmail password.
    IMAP port : 993
    IMAP TLS/SSL required: yes
  POP3 Client Settings
    server address: pop.gmail.com
    user name: Your full Gmail address (e.g. me@gmail.com)
    password: Your Gmail password
    With Gmail 2-step authentication enabled, use an application-specific Gmail password.
    IMAP port : 995
    IMAP TLS/SSL required: yes
  SMTP Client Settings
    server address: smtp.gmail.com
    user name: Your full Gmail address (e.g. me@gmail.com)
    password: Your Gmail password
    With Gmail 2-step authentication enabled, use an application-specific Gmail password.
    SMTP port (TLS): 587
    SMTP TLS/SSL required: yes

MSOutlook / Hotmail
  IMAP Client Settings
    server address: imap-mail.outlook.com
    user name: Your full email address (e.g. me@hotmail.com)
    password: Your hotmail password
    IMAP port : 993
    IMAP TLS/SSL required: yes
  POP3 Client Settings
    server address: pop-mail.outlook.com
    user name: Your full email address (e.g. me@hotmail.com)
    password: Your hotmail password
    IMAP port : 995
    IMAP TLS/SSL required: yes
  SMTP Client Settings
    server address: smtp-mail.outlook.com
    user name: Your full email address (e.g. me@hotmail.com)
    password: Your hotmail password
    SMTP port (TLS): 587
    SMTP port (SSL): 465
    SMTP TLS/SSL required: yes (STARTTLS) 

I attempted to use the instructions from Exratione A Mailserver on Ubuntu 18.04: Postfix, Dovecot, MySQL, Postfixadmin, Roundcube to install Roundcube. However the package database configuration failed. So I instead followed the instructions given in LinOxide How to Install Roundcube Webmail on Ubuntu 18.04. These instructions assume an already working LAMP stack and running email server and down load the latest install files directly from RoundCube, i.e. do not use the Unbuntu apt package system.

Some tips on the LinOxide instructions:

• As of writing (2017-01-04) the latest version of Roundcube is 1.2.3
• My website and mail server are already setup only to use https, so:
  • The IMAP host chosen to perform login is ssl://sub1.example.com and port 993 (standard IMAP secure port)
  • Similarly, The SMTP server host is ssl:sub1.example.com and port 25 (not 465). There is no SMTP username and password, just select the “Use the current IMAP username and password for SMTP authentication.
  • Personally I do not understand why you would want to use a webmail client to Gmail, Outlook, etc., as these all have there own webmail clients.
• Instead of deleting the installer directory, disable access in apache2:

create new file: /etc/apache2/sites-available/roundcube.conf and add following:

  <Directory /var/www/html/webmail>
    Options +FollowSymLinks
    AllowOverride All
    AllowOverrideList None
    <RequireAll>
      Require all granted
    </RequireAll>
  </Directory>

  <Directory /var/www/html/webmail/installer>
    Options +FollowSymLinks
    AllowOverride None
    AllowOverrideList None
    <RequireAll>
      Require all denied
    </RequireAll>
  </Directory>

To enable site configuration: sudo a2ensite roundcube.conf

• Note to function correctly the ”/var/www/html/webmail/.htaccess“ file should have the <IfModule mod_php5.c> changed to <IfModule mod_php7.c>

Some additional followups:

• GitHub Roundcube Plugin Resources
• Roundcube Plugins The Official Plugin Repository

In order to direct default traffic to the webmail page I added directive Redirectmatch ^/$ https://mail.kptree.net/webmail to the Apaches site configuration file /etc/apache2/sites-available/sub1.example.com-ssl.conf.

Attempt to follow ExRatione - A Mailserver on Ubuntu 12.04: Postfix, Dovecot, MySQL 19) Install Horde 5 for Webmail, full copy with update to get working on current Ubuntu 16.04 install. However upon following this procedure it look like it is quite outdated in many ways. Upon further investigation I found this installation description that looks more up to date and functional, Howtoforge Install Horde 5 Webmail For ISPConfig On Debian Wheezy Through PEAR. The Horde site instructions Installing Horde Framework 5 and Installing Horde Groupware Webmail Edition 5 states “These are very terse instructions how to install Horde Groupware Webmail Edition and its prerequisites on a LAMP system. They are addressed to experienced administrators who know exactly what they are doing.”. A last key reference is the Horde Horde Administrator's FAQ.

To summarise my feeling on Horde, basically it is too complicated to setup and use. It looks to be beyond the needs of a simple home server. At the time of writing (2017-01-03) the current version of Horde is not optimised for PHP7, which is standard with Ubuntu 16.04, which leads to further complications with the install. Exratione in subsequent later version of its installations instructions went with Roundcube for webmail interface. Furthermore some of the key features I was looking for, such as shared calendar and contact resources are included in Nextcloud. Whilst I managed to get Horde webmail running, there are still a number of issues with it that I have not been able to resolve. At this time I have decided not to proceed with using Horde. These instructions are therefore incomplete.

1. Install as many of the needed packages as possible through apt install:

  <code>
  "sudo apt update"
  "sudo apt install php7.0-dev php-pear php7.0-tidy php-imagick" (no php5-sasl, php-auth-sasl?)
  "sudo apt install php-geoip geoip-bin" (packages geoip-database and libgeoip1 included in php-geoip install)
  "sudo apt install php-xml-serializer php-memcache php7.0-soap php7.0-intl"
  "sudo apt install libidn11-dev libmagickwand-dev imagemagick" (libmagick++4?)
  "sudo apt install libsasl2-dev php-ssh2 php-http-webdav-server" (no libssh2-php, php-ssh2?, no libphp-jpgraph)</code>

2. Next update the PECL and Pear package managers and install the remaining required packages:

    sudo pecl channel-update pear.php.net
    sudo pear channel-update pear.php.net
    sudo pecl install lzf - with messages:
        configuration option "php_ini" is not set to php.ini location
        You should add "extension=lzf.so" to php.ini
    Check the Pear website for latest package versions and versions that suit you at pear Packages
    sudo pear install --alldeps channel://pear.php.net/Date_Holidays-0.21.8
    sudo pear install --alldeps channel://pear.php.net/Date_Holidays_UNO-0.1.3
    sudo pear install --alldeps channel://pear.php.net/Date_Holidays_Australia-0.2.2
    sudo pear install --alldeps channel://pear.php.net/Numbers_Words-0.18.1
    sudo pear install --alldeps channel://pear.php.net/Text_CAPTCHA-1.0.2
    sudo pear install --alldeps channel://pear.php.net/Console_GetoptPlus
    sudo pear install --alldeps channel://pear.php.net/HTTP_Request2
    sudo pear install --nodeps channel://pear.php.net/MDB2_Driver_mysql

3. Next up is installing the Horde components. Start with these commands:

    sudo pear channel-discover pear.horde.org
    sudo pear install horde/Horde_role
    sudo pear run-scripts horde/Horde_role

4. At this point, you will be prompted to enter the “Filesystem location for the base Horde application” - so enter the full path to your web root without a trailing slash, i.e. /var/www/html/horde.

5. Next start the installation process with the command below. This will take a while to run to completion:

6. sudo pear install -a -B –force horde/webmail

The –force is required to allow the use of php7, otherwise this will error out the install.

Use “pear config-show -c horde” to show where the key Horde php related files are on the system.