{{tag>linux docker mail mailserver Alpine dovecot postfix sqlite s6 s6-rc}} ======Docker mailserver====== This mailserver setup follows Workaround's [[https://workaround.org/ispmail-bookworm/|SPmail guide for Debian 12 “Bookworm”]]. Key changes are that instead of installing on Debian 11 virtual machine1, with a Maria mysql database2, this setup is for installation on latest Alpine linux Docker image with s6-rc init using sqlite database. //As this follows Workaround's [[https://workaround.org/ispmail-bookworm/|SPmail guide for Debian 12 “Bookworm”]], significant amounts of text have been copied and generally modified from there. **I hereby credit Workaround's author Christoph Haas.** Furthermore Christoph's guide is very descriptive and should be referenced to get a better understanding of how to put together a mailserver.// **The notes here are my current working attempt to get an Alpine s6-rc Docker implementation of Postfix and Dovecot, with sqlite based mail server functional and are currently incomplete.** - Use of virtual machines is much more common these days than base metal for applications. However Workarounds Debian email server could be loaded on base metal. - The database requirements for a small mailserver with a few dozen domains, with each domain having hundreds of emails and aliases is well within the capacity of the sqlite database. The use of a full multi user server / client relational database is not necessary, particularly for a Docker based server implementation. See [[https://www.digitalocean.com/community/tutorials/sqlite-vs-mysql-vs-postgresql-a-comparison-of-relational-database-management-systems|SQLite vs MySQL vs PostgreSQL: A Comparison Of Relational Database Management Systems]] =====Dockerfile===== I go annoyed with the messy UID and GID and found this reference to attempt to standardise upon. //Sadly there seems to be no comprehensive standard!// * Distribution List of Ports (information only) * Gentoo [[https://wiki.gentoo.org/wiki/Project:Quality_Assurance/UID_GID_Assignment|Project:Quality Assurance/UID GID Assignment]] * Archlinux [[https://wiki.archlinux.org/title/DeveloperWiki:UID_/_GID_Database|DeveloperWiki:UID / GID Database]] * Red Hat [[https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/5/html/deployment_guide/s1-users-groups-standard-users|37.3. Standard Users]] ====References==== *Docker *[[https://docs.docker.com/engine/reference/builder/#volume|Dockerfile reference]] *[[https://docs.docker.com/storage/volumes/|Volumes]] *[[https://docs.docker.com/storage/bind-mounts/|Bind mounts]] *[[https://stackoverflow.com/questions/41935435/understanding-volume-instruction-in-dockerfile|Understanding "VOLUME" instruction in DockerFile]] *[[https://superuser.com/questions/1167696/install-whereis-command-on-alpine-linux|Install whereis command on alpine linux]] =====packages===== ====working==== Required for server: *sqlite => ''/usr/bin/sqlite3'' *postfix => ''/usr/sbin/postfix'' *postfix-sqlite ''postfix-sqlite.so'' plugin? *#dovecot => #main dovecot included in dovecot-sqlite *dovecot-sqlite => ''/usr/sbin/dovecot'' *#dovecot-pop3 => #Only if need pop3 connectivit2y *dovecot-imapd => plugin? *dovecot-lmtpd => plugin? *dovecot-pigeonhole-plugin => plugin? *apache2 => ''/usr/sbin/httpd'' *php => ''/usr/bin/php'' -> ''/usr/bin/php81'' So the standard php package currently links to php81 at this time of writing. *Adminer is downloaded separately ''adminer'' is a single php file program to administer database programs including sqlite *rspamd => ''/usr/sbin/rspamd'' The email spam removing daemon *swaks => ''/user/bin/swaks'' Seems to only be in testing repository, use following to load ''%%swaks --repository=http://dl-cdn.alpinelinux.org/alpine/edge/testing/%%'' *mutt => ''/usr/bin/mutt'' *jq => ''/usr/bin/jq'' To assist with working in container: *util-linux => adds some addition utilities to Alpine, e.g. ''whereis'' This is not required for the final package and can be commented out of the Dockerfile. *pwgen => ''pwgen'' generates encrypted passwords *vim => because it is nicer to use than ''vi'' *less => because sometimes it is nicer to use than ''cat'' (it looks like util-linux includes the ''more'' command. ''less'' seems more full featured and what I am use to. It looks like ''less'' will not work properly unless ''export TERM=rxvt'' is used. Hmmm... unimpressed ====alias==== I could not get the alias command to work in Alpine shell. I tried /etc/profile and /etc/profile.d to no avail. So the following seems to meet my needs: *create file ''ll.sh'' *#!/bin/sh exec ls -la "$@" *chmod +x ll.sh to allow to to be an executable. *Copy the ll.sh file to /bin/ll or link it ''/bin/ln -s /app/scripts/ll.sh /bin/ll'' =====sqlite===== ====sqlite commands==== *''sqlite3 /app/mailserver.db'' to start sqlite and create or open mailserver.db. *''.open /app/mailserver.db'' to open mailserver.db *''.tables'' to list tables in a database *To list tables from sqlite_schema:SELECT name FROM sqlite_schema WHERE type ='table' AND name NOT LIKE 'sqlite_%'; *''SELECT * FROM table_name;'' to query all data from a table. ====virtual_domains==== This table just holds the list of domains that you will use as virtual_mailbox_domains in Postfix. ^Column ^Purpose^ |id |A unique number identifying each row. It is added by the database automatically.| |name |The name of the domain you want to receive email for.| This SQL statement creates a table like that: ===sql code=== CREATE TABLE IF NOT EXISTS `virtual_domains` ( `id` int(11) NOT NULL auto_increment, `name` varchar(50) NOT NULL, PRIMARY KEY (`id`) ) ENGINE=InnoDB DEFAULT CHARSET=utf8; ===sqlite code=== CREATE TABLE IF NOT EXISTS 'virtual_domains' ( id INTEGER PRIMARY KEY, name TEXT NOT NULL UNIQUE ); ''DROP TABLE IF EXISTS virtual_domains;'' will delete the table. ====virtual_users==== The next table contains information about your users. Every mail account takes up one row. ^Column ^Purpose^ |id |A unique number identifying each row. It is added by the database automatically. | |domain_id |Contains the number of the domain’s id in the virtual_domains table. This is called a foreign key. A “delete cascade” makes sure that if a domain is deleted that all user accounts in that domain are also deleted to avoid orphaned rows. | |email |The email address of the mail account. | |password |The hashed password of the mail account. It is prepended by the password scheme. By default it is {BLF-CRYPT} also known as bcrypt which is considered very secure. Previous ISPmail guides used {SHA256-CRYPT} or even older crypt schemes. Prepending the password field the hashing algorithm in curly brackets allows you to have different kinds of hashes. So you can easily migrate your old passwords without locking out users. Users with older schemes should get a new password if possible to increase security. | |quota |The number of bytes that this mailbox can store. You can use this value to limit how much space a mailbox can take up. The default value is 0 which means that there is no limit. | This is the appropriate SQL query to create that table: ===sql code=== CREATE TABLE IF NOT EXISTS `virtual_users` ( `id` int(11) NOT NULL auto_increment, `domain_id` int(11) NOT NULL, `email` varchar(100) NOT NULL, `password` varchar(150) NOT NULL, `quota` bigint(11) NOT NULL DEFAULT 0, PRIMARY KEY (`id`), UNIQUE KEY `email` (`email`), FOREIGN KEY (domain_id) REFERENCES virtual_domains(id) ON DELETE CASCADE ) ENGINE=InnoDB DEFAULT CHARSET=utf8; ===sqlite code=== CREATE TABLE IF NOT EXISTS `virtual_users` ( `id` INTEGER PRIMARY KEY, `domain_id` INTEGER NOT NULL, `email` TEXT NOT NULL UNIQUE, `password` TEXT NOT NULL, `quota` INTEGER NOT NULL DEFAULT 0, FOREIGN KEY (domain_id) REFERENCES virtual_domains(id) ON DELETE CASCADE ); ====virtual_aliases==== The last table contains forwardings from an email address to other email addresses. ^Field ^Purpose ^ |id |A unique number identifying each row. It is added by the database automatically. | |domain_id |Contains the number of the domain’s id in the virtual_domains table again. | |source |The email address that the email was actually sent to. In case of catch-all addresses (that accept any address in a domain) the source looks like “@example.org”. | |destination |The email address that the email should instead be sent to. | This is the required SQL query you need to run: ===sql=== CREATE TABLE IF NOT EXISTS `main.virtual_aliases` ( `id` int(11) NOT NULL auto_increment, `domain_id` int(11) NOT NULL, `source` varchar(100) NOT NULL, `destination` varchar(100) NOT NULL, PRIMARY KEY (`id`), FOREIGN KEY (domain_id) REFERENCES virtual_domains(id) ON DELETE CASCADE ) ENGINE=InnoDB DEFAULT CHARSET=utf8; ===sqlite=== CREATE TABLE IF NOT EXISTS `virtual_aliases` ( `id` INTEGER PRIMARY KEY, `domain_id` INTEGER NOT NULL, `source` TEXT NOT NULL, `destination` TEXT NOT NULL, FOREIGN KEY (domain_id) REFERENCES virtual_domains(id) ON DELETE CASCADE ); ====sqlite example test data==== The following test dat acan be used to test the data returns from postfix. REPLACE INTO virtual_domains (id,name) VALUES ('1','example.org'); REPLACE INTO virtual_users (id,domain_id,password,email) VALUES ('1', '1', '{BLF-CRYPT}$2y$05$.WedBCNZiwxY1CG3aleIleu6lYjup2CIg0BP4M4YCZsO204Czz07W', 'john@example.org'); REPLACE INTO virtual_aliases (id,domain_id,source,destination) VALUES ('1', '1', 'jack@example.org', 'john@example.org'); This sample data should be deleted before using the mailserver live. DELETE FROM mailserver.virtual_domains WHERE name='example.org'; ====references==== *sqlitetutorial.net [[https://www.sqlitetutorial.net/|SQLite Tutorial]] *sqlitetutorial.net [[https://www.sqlitetutorial.net/sqlite-cheat-sheet/|SQLite Cheat Sheet]] *tutorialspoint.com [[https://www.tutorialspoint.com/sqlite/index.htm|SQLite Tutorial]] *database.guide [[https://database.guide/category/dbms/rdbms/sqlite/|sqlite]] *[[https://stackoverflow.com/questions/8246649/why-cant-you-use-sqlite-rowid-as-a-primary-key|Why can't you use SQLite ROWID as a Primary key?]] =====mariadb===== *Used linuxserver.io mariadb image *''docker exec -it mail_db /bin/bash'' *See [[https://mariadb.com/kb/en/configuring-mariadb-for-remote-client-access/|Configuring MariaDB for Remote Client Access]] *Un-comment ''bind-address=0.0.0.0'' near end of config file in ''custom.cnf'' ( ''.config_sql/custom.cnf'') *''grant select on mailserver.* to 'mailserver'@'172.%' identified by 'PASSWORD';'' The Docker internal network is on 172. *php modules required do not seem to be defined anywhere, I used the following ++php modules| php-fpm php-session #php-opcache #php-gd #php-zlib #php-curl #php-bz2 ?php-mysqli ?php-mysqlnd #php-zip php-bcmath ++ I am not sure if all these modules are required. Those with # are not required for Adminer. ISPMailAdmin requires bcmath. Adminer seemed to work without mysqli and mysqlnd. ====mariadb sql commands==== *''SELECT User, Host FROM mysql.user;'' this shows all the user(s) and their host(s) *''grant select on mailserver.* to 'mailserver'@'172.16.0.0/255.240.0.0' identified by 'password.x.y.z.';'' This creates the user ''mailserver'', assigns ''select'' level privileges and password. //When I used CIDR '172.16.0.0/12' Mariadb did not seem to allow access where as subnet mask, '172.16.0.0/255.240.0.0' did function. Strange....// *''grant all privileges on mailserver.* to 'mailadmin'@'172.16.0.0/255.240.0.0' identified by 'password.w.x.y.';'' This creates the user ''mailadmin'', assigns all privileges and password. *''GRANT ALL PRIVILEGES ON mailserver.* TO 'mailadmin'@'172.16.0.0/255.240.0.0';'' This grants ''all'' privileges on database mailserver to user mailadmin at defined host. *''REVOKE ALL PRIVILEGES, grant option from 'mailserver'@'172.%';'' This revokes all privileges from user mailserver'@'172.%. *''SELECT User, Host FROM mysql.user WHERE Host <> 'localhost';'' Filters the user / host list. *''drop user 'mailserver'@'172.16.0.0/255.240.0.0';'' This drops the user. *''show grants for 'mailserver'@'172.16.0.0/255.240.0.0';'' This shows the privileges for the specified user. Many of the problems that I had trying to get Adminer and ISPMailAdmin operation related to access privileges on the Mariadb. I should have read and understood the ISPMail [[https://workaround.org/ispmail-bookworm/prepare-the-database/|Preparing the database]] section more carefully. Some keypoints: *The mailadmin user was linked to localhost and granted all privileges *In mariadb localhost uses the socket file ''/var/run/mysqld/mysqld.sock'' This tends to be more secure than the TCP connection and the ISPmail uses this user for database administrative control. *The mailserver user was linked to 127.0.0.0 and granted select privilege only *In mariadb an IP host uses TCP to communicate. In ISPmail Postfix is set to use this interface, effectively read only (select privilege only). * I am using in Docker with multiple separate containers. * As these Docker containers are operated on a common Linux server the socket file is effectively one a common machine and could be shared across multiple containers. I have successfully performed this with separate Nginx / PHP-FPM (fastCGI) containers sharing a common socket file. * Because the database is running in a separate container to the Nginx web server there is no common localhost or 127.0.0.1 available to share. * Docker internal networking uses 172.16.0.0/12 (subnet mask 255.240.0.0), supposedly recent version of mysql support this notation ([[https://stackoverflow.com/questions/11742963/how-to-grant-remote-access-to-mysql-for-a-whole-subnet#38389851|How to grant remote access to MySQL for a whole subnet?]]). Hence, grant [select|all] on mailserver.* to 'mailserver'@'172.16.0.0/12' identified by 'PASSWORD'; * I need to consider the following users *Local mail_db container administrative access, continue to use 'mailadmin'@'localhost'. *Docker stack container administrative access to adjust database with suitable table edit/create privilege, perhaps, 'mailstackadmin'@''172.16.0.0/12'. *Docker stack container postfix select only access to adjust database with suitable table edit/create privilege, 'mailserver'@''172.16.0.0/12'. ====adminer==== I will setup DNS and Traefik for this to be ''mailsql.local.kptree.net''. This will only be accessible on the LAN. ====ISPmail Admin==== I will setup DNS and Traefik for this to be ''mailadmin.local.kptree.net''. This will only be accessible on the LAN. Admin user is mailserver with associated password. =====adminer===== **phpMyAdmin** is a web based mysql management interface. **adminer** is a web based single php file database manager, that is suitable for many type of databases, including **sqlite** and being only a single file is easier to implement. Copy the latest version of adminer to ''.config/adminer'' *[[https://www.adminer.org/|Adminer Database management in a single PHP file]] *[[https://kinsta.com/blog/adminer/|How to Use Adminer to Manage Databases Easily with a Single PHP File]] To test: 192.168.1.14:8910 database: SQLite 3 user: password: standard database: /app/mailserver.db There is another program call [[https://www.sqlitestudio.pl/|SQLiteStudio]]. This look looks like a full GUI program, that will probably not be suitable for installation on a Docker mailserver. Noted. Adminer needs a few php modules to run, session and pdo_sqlite, apk packages: php$phpverx-session, php$phpverx-pdo_sqlite, php$phpverx-sqlite3. Also Adminer does not like working with Sqlite without forcing some kind of password protect. Hence the Adminer plugin module and password-less plugin need to be used. It seems that Apache2 php runs more efficiently if the php-fpm module is used and setup. See Alpine [[https://wiki.alpinelinux.org/wiki/Apache_with_php-fpm|Apache with php-fpm]] =====postfix===== ''/etc/postfix'' / # postconf mail_version mail_version = 3.7.4 ====Making Postfix get its information from the sqlite database==== ====virtual_mailbox_domains==== A mapping in Postfix is just a table that contains a left-hand side (LHS) and a right-hand side (RHS). To make Postfix get information about virtual domains from the database we need to create a ‘cf’ file (configuration file). Start by creating a file called /etc/postfix/sqlite-virtual-mailbox-domains.cf for the virtual_mailbox_domains mapping that contains: dbpath = /app/mailserver.db query = SELECT 1 FROM virtual_domains WHERE name='%s' Imagine that Postfix receives an email for somebody@example.org and wants to find out if example.org is a virtual mailbox domain. It will run the above SQL query and replace ‘%s’ by ‘example.org’. If it finds such a row in the virtual_domains table it will return a ‘1’. Actually it does not matter what exactly is returns as long as there is a result. Now you need to make Postfix use this database mapping: ''postconf virtual_mailbox_domains=sqlite:/etc/postfix/sqlite-virtual-mailbox-domains.cf'' The “postconf” command conveniently adds configuration lines to your /etc/postfix/main.cf file. It also activates the new setting instantly so you do not have to reload the Postfix process. The test data you created earlier added the domain “example.org” as one of your mailbox domains. Let’s ask Postfix if it recognizes that domain: ''postmap -q example.org sqlite:/etc/postfix/sqlite-virtual-mailbox-domains.cf'' You should get ‘1’ as a result. That means your first mapping is working. Feel free to try that with other domains after the -q in that line. You should not get a response ====virtual_mailbox_maps==== The virtual_mailbox_maps which is mapping email addresses (left-hand side) to the location of the user’s mailbox on your hard disk (right-hand side). Postfix has a built-in transport service called “virtual” that can receive the email and store it into the recipient’s email directory. But we will not make Postfix save the email to disk. We will delegate that to Dovecot as it allows us better control. All that Postfix needs to know is whether an email address belongs to a valid mailbox. That simplifies things a bit because we just need the left-hand side of the mapping. Similar to the above virtual_domains mapping you need an SQL query that searches for an email address and returns “1” if it is found. To accomplish that please create another configuration file at ''/etc/postfix/sqlite-virtual-mailbox-maps.cf'': dbpath = /app/mailserver.db query = SELECT 1 FROM virtual_users WHERE email='%s' Tell Postfix that this mapping file is supposed to be used for the virtual_mailbox_maps mapping: ''postconf virtual_mailbox_maps=sqlite:/etc/postfix/sqlite-virtual-mailbox-maps.cf'' Test if Postfix is happy with this mapping by asking it where the mailbox directory of our john@example.org user would be: ''postmap -q john@example.org sqlite:/etc/postfix/sqlite-virtual-mailbox-maps.cf'' Again you should get “1” back which means that john@example.org is an existing virtual mailbox user on your server. Very good. Later when we deal with the Dovecot configuration we will also use the password field but Postfix does not need it right here. ====virtual_alias_maps==== The virtual_alias_maps mapping is used for forwarding emails from one email address to one or more others. In the database multiple targets are achieved by using multiple rows. Create another “.cf” file at ''/etc/postfix/sqlite-virtual-alias-maps.cf'': dbpath = /app/mailserver.db query = SELECT destination FROM virtual_aliases WHERE source='%s' Make Postfix use this database mapping: ''postconf virtual_alias_maps=sqlite:/etc/postfix/sqlite-virtual-alias-maps.cf'' Test if the mapping file works as expected: ''postmap -q jack@example.org sqlite:/etc/postfix/sqlite-virtual-alias-maps.cf'' You should see the expected destination: john@example.org So if Postfix receives an email for jack@example.org it will redirect it to john@example.org. ====postfix start/stop==== Basic postfix control are: *''postfix start'' to start postfix in background *''postfix start-fg'' to start postfix in the foreground *''postfix stop'' to stop postfix *''postfix reload'' to reload configuration *''pstfix status'' to return current postfix operating status It looks a shell script is used to control Postfix, in Alpine is is located ++here|/usr/libexec/postfix/postfix-script++ ===systemd=== The systemd service script is a hoot!, it seems to do nothing meaningful. ++++/lib/systemd/system/postfix.service| [Unit] Description=Postfix Mail Transport Agent Conflicts=sendmail.service exim4.service ConditionPathExists=/etc/postfix/main.cf [Service] Type=oneshot RemainAfterExit=yes ExecStart=/bin/true ExecReload=/bin/true [Install] WantedBy=multi-user.target ++++ Look like postfix is started from ''/etc/init.d/postfix''. This script notes the following options, "Usage: /etc/init.d/postfix {start|stop|restart|reload|flush|check|abort|force-reload|status}" ===Alpine=== * ''/usr/libexec/postfix/postfix-script'' * ''/usr/sbin/postfix'' postfix library modules * ''/etc/postfix'' configuration files ===s6 setup=== The s6 rc run file: ++++run|#!/command/execlineb -P fdmove -c 2 1 /usr/sbin/postfix start-fg ++++ ====Postfix logging==== Alpine posfix would seem to be setup to use postlogd, as master.cf has the following line already configured: ''%%postlog unix-dgram n - n - 1 postlogd%%''. Hence the following does not need to be used: ''%%/bin/echo 'postlog unix-dgram n - n - 1 postlogd' >> '/etc/postfix/master.cf'%%'' ====/etc/postfix/aliases==== I get an error when recreating the container; "error: open database /etc/postfix/aliases.lmdb: No such file or directory" The postfix command recreates the missing/corupt aliases.lmdb file; ''newaliases''.I added this to my Docker container startup script. This solved the problem, but not sure if this is the right way to do this. ====Postfix References==== *Postfix: *[[http://www.postfix.org/docs.html|Postfix Howtos and FAQs]] *[[http://www.postfix.org/postconf.5.html|Postfix Configuration Parameters]] *[[http://www.postfix.org/SQLITE_README.html|Postfix SQLite Howto]] *Postfix logging references *[[https://www.postfix.org/MAILLOG_README.html|Postfix logging to file or stdout]] *[[https://www.postfix.org/postlogd.8.html|POSTLOGD(8) - Postfix internal log server]] *Other Postfix resources: *Gentoo Linux: [[https://wiki.gentoo.org/wiki/Complete_Virtual_Mail_Server|Complete Virtual Mail Server]] *[[https://www.cyberciti.biz/faq/linux-unix-start-stop-restart-postfix/|Start / Stop / Restart Postfix Mail Server]] *[[https://kerneltalks.com/linux/how-to-start-stop-and-reload-postfix/|How to start, stop and reload postfix]] *[[https://serverfault.com/questions/885828/is-there-any-way-to-run-postfix-in-foreground|Is there any way to run Postfix in foreground?]] This is interesting as it infers that postfix was originally reliant upon syslog. =====dovecot===== ''/etc/dovecot/conf.d'' / # dovecot --version 2.3.20 (80a5ac675d) ====dovecot database configuration and testing==== ===database setup: /etc/dovecot/dovecot-sql.conf.ext=== See Dovecot howto on sqlite setup [[http://rob0.nodns4.us/howto/3-dovecot|Dovecot configuration sqlite]], linked from Dovecot [[https://wiki.dovecot.org/HowTo|HOWTOs / Examples / Tutorials]] You will find this file well documented although all configuration directives are commented out. Add these lines at the bottom of the file: driver = sqlite connect = /app/mailserver.db user_query = SELECT email AS user, \ '*:bytes=' || quota AS quota_rule, \ '/var/vmail/%d/%n' AS home, \ 5000 AS uid, 5000 AS gid \ FROM virtual_users WHERE email='%u' password_query = SELECT password FROM virtual_users WHERE email='%u' iterate_query = SELECT email AS user FROM virtual_users Notes: *The Dovecot file uses the ''\'' at the end of each line to indicate the command continues on to the next line. Sqlite does not allow this. *The mysql concat command syntax is || for sqlite *connect = is the full path to the sqlite mailserver database ===Dovecot database testing=== ==Testing directly within sqlite3:== *Open the sqlite3 database at the command line sqlite3 /app/mailserver.db *user_query: SELECT email AS USER, '*:bytes=' || quota AS quota_rule, '/var/vmail/%d/%n' AS home, 5000 AS uid, 5000 AS gid FROM virtual_users WHERE email='john@example.org'; * output example on success: john@example.org|*:bytes=0|/var/vmail/%d/%n|5000|5000 *password_query: SELECT password FROM virtual_users WHERE email='john@example.org'; *Output on success {BLF-CRYPT}$2y$05$.WedBCNZiwxY1CG3aleIleu6lYjup2CIg0BP4M4YCZsO204Czz07W *iterate_query: SELECT email AS user FROM virtual_users; *Output: john@example.org ==Testing with openssl== ''openssl s_client -servername mail.kptree.net -connect mail.kptree.net:pop3s'' ==Testing with doveadmin== //doveadm auth test// ''doveadm auth test john@example.org'', you will be prompted for password. A successful response looks like: passdb: john@example.org auth succeeded extra fields: user=john@example.org //doveadm user// ''doveadm user john@example.org'' will give output: field value uid 5000 gid 5000 home /var/vmail/example.org/john mail maildir:~/Maildir quota_rule *:bytes=0 whereas ''doveadm user *example.org'' will give a list of users: john@example.org ====dovecot start / stop==== ===systemd=== The debian 10 systemctl dovecot.service: ++++/lib/systemd/system/dovecot.service| # This file is part of Dovecot # # DO NOT CUSTOMIZE THIS FILE, INSTEAD # create the file: # `/etc/systemd/system/dovecot.service.d/service.conf'. # or copy this as # `/etc/systemd/system/dovecot.service` and edit then # and put your changes there [Unit] Description=Dovecot IMAP/POP3 email server Documentation=man:dovecot(1) Documentation=http://wiki2.dovecot.org/ After=local-fs.target network-online.target [Service] Type=simple ExecStart=/usr/sbin/dovecot -F PIDFile=/var/run/dovecot/master.pid ExecReload=/usr/bin/doveadm reload ExecStop=/usr/bin/doveadm stop PrivateTmp=true NonBlocking=yes ProtectSystem=full ProtectHome=no PrivateDevices=true # You can add environment variables with e.g.: #Environment='CORE_OUTOFMEM=1' # If you have trouble with `Too many open files', increase LimitNOFILE=65535 # If you want to allow the Dovecot services to produce core dumps, use: #LimitCORE=infinity [Install] WantedBy=multi-user.target ++++ * ''/usr/sbin/dovecot'' to start in background * [-F] to start in foreground * [-c ] * [reload] forces Dovecot to reload configuration * [stop] shutsdown Dovecot and all child processes * [-a] dump configuration settings and exit //(it is a long list!)// * [--help] * [--version] * [--build-options] show build options and exit ====dovecot testing with mutt==== Workaround suggests the following command to be used to test: ''%%mutt -f imaps://john@example.org@webmail.example.org%%'' The webmail.example.org simple made no sense to me and did not function with error ''Could not find the host "webmail.example.org"''. As I am creating this in Docker and separately taking the certificates from Traefik, this mailserver simple is not linked in a anyway with a webserver! The webservers for the database access and webmail are totally separate containers. The "simple" solution was to use "localhost" from within the mailserver docker container, e.g. ''%%mutt -f imaps://john@example.org@localhost%%''. ====dovecot logging==== ====Dovecot References==== *Doc Dovecot: *[[https://doc.dovecot.org/configuration_manual/dovecot_ssl_configuration/|Dovecot SSL configuration]] *[[https://doc.dovecot.org/configuration_manual/howto/simple_virtual_install/#simple-virtual-install-smtp-auth|Simple Virtual User Installation]] *[[https://doc.dovecot.org/configuration_manual/namespace/|Namespaces]] *[[https://wiki.dovecot.org/HowTo|HOWTOs / Examples / Tutorials]] *[[https://doc.dovecot.org/settings/plugin/|Plugin Settings]] *[[https://wiki2.dovecot.org/LDA/Postfix|Dovecot LDA with Postfix]] *[[https://doc.dovecot.org/configuration_manual/mail_location/|Mail Location Settings]] *[[https://doc.dovecot.org/admin_manual/doveadm_mailbox_commands/|Doveadm Mailbox Commands]] *[[https://wiki.dovecot.org/Tools/Doveadm|ToolsDoveadm]] *[[https://doc.dovecot.org/configuration_manual/authentication/password_schemes/|Password Schemes]] *[[https://wiki1.dovecot.org/AuthDatabase/SQL|AuthDatabase / SQL]] *[[https://doc.dovecot.org/configuration_manual/authentication/sql/|SQL]] *[[https://doc.dovecot.org/admin_manual/logging/|Dovecot Logging]] *[[https://doc.dovecot.org/configuration_manual/quota_plugin/|Quota Plugin]] *Other: *[[http://rob0.nodns4.us/|Postfix, DNS and general Linux system administration and consulting]] *[[http://rob0.nodns4.us/howto/3-dovecot|F. Dovecot configuration]] *[[https://serverfault.com/questions/803533/dovecot-storing-mails-in-var-mail-but-mail-location-set-to-maildir-var-mail-v|Dovecot storing mails in /var/mail, but mail_location set to maildir:/var/mail/vhosts/%d/%n]] *[[https://manpages.org/dovecot|man dovecot]] =====Certificates SSL/TSL===== Early on, before 2015 there were not many free SSL certificate providers. I used StartSSL for a free certificate. They would purchased by a company that managed to get their certificate deregister...... So StartSSL basically became non-usable circa 2017. Fourtunately a better solution came about a year or 2 earlier called LetsEncrypt. This could be used with certbot to get free certificates and eventually free wildcard certificates. Then came Traefik which handled certificates =====apache2===== Alpine apk apache2 distribution seems to follow the Red Hat setup style. The daemon is httpd instead of apache2. So I need to learn a new setup. Keep features: * http is the main program, ''/usr/sbin/http'' * ''httpd -S'' list the current defined * ''httpd -M'' list the current defined modules * ''httpd -k restart'' to restart, also stop to kill, graceful to gracefully stop * main configuration files located at ''/etc/apache2'', including httpd.conf ====apache2.4 remote php-fpm proxy configuration==== There are a number of ways to configure apache 2.4 to access remote server, see Apache module [[https://httpd.apache.org/docs/2.4/mod/mod_proxy_fcgi.html|mod_proxy_fcgi]] for details. I used the following commands. *''DocumentRoot "/var/www/roundcube"'' this is not really necessary, however it removes a warning message. *''%%ProxyPassMatch "\.php$" "fcgi://roundcubemail:9000/var/www/html" enablereuse=on%%'' basically redirects all *.php requests to the fcgi server. * The first part of the match, ''"\.php$"'' basically indicates that all *.php call should be directed to the proxy * The second part ''%%"fcgi://roundcubemail:9000/var/www/html" enablereuse=on%%'' is the proxy call (fcgi:) to IP/TCP ''roundcubemail:9000''. The DNS resolves roundcubemail to the local IP address at port 9000. The /var/www/html provides the access path on the remote server. ++++vim .config/apache2/mail.kptree.net-443.conf| ServerAdmin admin@kptree.net # ServerName mail.kptree.net ServerName localhost DocumentRoot "/var/www/roundcube" # proxy-fcgi-pathinfo ProxyPassMatch "\.php$" "fcgi://roundcubemail:9000/var/www/html" enablereuse=on # # SetHandler "proxy:fcgi://roundcubemail:9001/var/www/html" # # # ProxyPass fcgi://roundcubemail:9001 enablereuse=on # Require all granted # Options Indexes FollowSymLinks # Allow from all # AllowOverride all # # ProxyPass "/var/www/html" "fcgi://roundcubemail:9001" enablereuse=on # # Options Indexes FollowSymLinks ExecCGI # AllowOverride All # Require all granted # # # DirectoryIndex index.php # ++++ ====apache2.4 local php-fpm configuration==== ++++vim .config/apache2/mail.local.kptree.net-8080.conf| Listen 8080 ServerName mail.local.kptree.net #ServerAlias ServerAdmin webmaster@localhost DocumentRoot /var/www/local #Some optimisation and security directives Header always set Strict-Transport-Security "max-age=15552000; includeSubDomains;" Header set X-Frame-Options "SAMEORIGIN" Header set X-Content-Type-Options "nosniff" #Location of log if specific for this virtual host is required #ErrorLog ${APACHE_LOG_DIR}/error.log #CustomLog ${APACHE_LOG_DIR}/access.log combined Options Indexes FollowSymLinks AllowOverride None Require all granted DirectoryIndex index.php ++++ =====nginx and php-fpm===== I though I should give nginx a try as I was having linted success with Apache and remote php-fpm. The Apline package locations as of php version 8.2 are as follows. This may vary with version and also definitely varies with different distributions and their package managers. * Main php-fpm configuration file is located at ''/etc/php82/php-fpm.d/www.conf'' * Unfortunately the php version number changes with each setup, so for php version 7.4 the directory would be ''/etc/php74/php-fpm.d/www.conf'' * Main nginx configuration file is located at ''/etc/nginx/http.d/default.conf'' * http://wiki.nginx.org/Pitfalls * https://wiki.nginx.org/QuickStart * https://wiki.nginx.org/Configuration ===Reference=== *[[https://levelup.gitconnected.com/containerizing-nginx-php-fpm-on-alpine-linux-953430ea6dbc|Containerizing nginx + PHP FPM on Alpine Linux]] *[[https://github.com/johnathanesanders/docker-nginx-fpm|johnathanesanders/docker-nginx-fpm]] *Stackoverflow *[[https://stackoverflow.com/questions/64137225/how-to-setup-a-single-nginx-server-with-multiple-php-fpm-docker-containers|how to setup a single nginx server with multiple php-fpm docker containers]] *[[https://stackoverflow.com/questions/29905953/how-to-correctly-link-php-fpm-and-nginx-docker-containers?rq=4|How to correctly link php-fpm and Nginx Docker containers?]] *[[https://stackoverflow.com/questions/49754286/multiple-images-one-dockerfile|Multiple images, one Dockerfile]] *[[https://medium.com/@shrikeh/setting-up-nginx-and-php-fpm-in-docker-with-unix-sockets-6fdfbdc19f91|Setting up nginx and PHP-FPM in Docker with Unix Sockets]] *[[https://www.librebyte.net/en/systems-deployment/how-to-install-php-php-fpm-in-alpine-linux/|How to install PHP, PHP-FPM in Alpine Linux?]] *[[https://serversforhackers.com/c/php-fpm-configuration-the-listen-directive|PHP-FPM: Configuration the Listen Directive]] =====php and php-fpm===== I decided to setup php-fpm [fpm = FastCGI Process Manager, and CGI = Common Gateway Interface, reputedly more performant than the builtin Apache php module. //Amazing an acronym in acronym, some people are so smart they are dumb!//]. I basically followed Alpine Linux instructions [[https://wiki.alpinelinux.org/wiki/Apache_with_php-fpm|Apache with php-fpm]] php-fpm acts like a php server and can be setup to allow remote connections from clients, usually web pages. The server is normally setup to listen on port 9000. Additional servers can be setup using other ports. This allows servers per app and different server php configuration, e.g. amount of memory and number of process, etc. This is performed by defining additional php-fpm server configuration pools that are differentiated by ip address and port, or socket file. If the client and servers are on the same machine they can communicate via local host and port or via socket file. If remotely hosted they communicate via ip address and port number. The pool definition only allows referencing via discrete ip address and port numbers, where as the web browsers can use name resolution of ip addresses adding flexibility. ====Alpine Docker php-fpm==== An annoying problem is the php-fpm version is carried on through the file system. The Alpine Docker stable version at writing is 8.1 (called 81). So the following 3 locations need to be considered when updating. This also means that the latest version can not simply be used. *Dockerfile ''ARG PHP_VERSION=82'' *Main configuration script ''pre_start_script.sh'', ''PHP_VERSION=82'' *s6-rc.d php-fpm longrun start script at s6-rc.d/php-fpm/run, the command ''/usr/sbin/php-fpm82 -F'' needs to be adjusted manually with version. *Need to create directory in log, ''sudo mkdir .config/log/php82'' As php-fpm automatically runs in daemon mode, when using s6-rc init as a longrun it needs to be run in foreground mode, ''/usr/sbin/php-fpm82 -F'', as per the -F option. In daemon mode, s6-rc thinks php-fpm has failed and attempts to restart. As the original program is running, the subsequent repeated attempts to run, give the noted repeated error message. //(I believe there is a method in s6-rc to stop after a defined number of failed attempts. I have not looked into this yet.)// *To list installed php modules ''php -m'' *A comprehensive php information list is provided with ''php -i'' ====php-fpm pool==== *To check php-fpm defined pools, where xx is the version, ''php-fpm82 -tt'' *To list php-fpm information, ''php-fpm81 -i'', note this is a text version of the the php -> phpinfo(); *''php-fpm82 -h'' to list all options *[[https://serversforhackers.com/s/managing-php-fpm|Managing PHP-FPM]] Learning how php-fpm works will make optimizing your server a breeze! See how to secure and optimize php-fpm here. *[[https://serversforhackers.com/c/apache-and-php-fpm|Apache and PHP-FPM]] Learn to hook Apache up to PHP-FPM using Apache's proxy modules. ====references==== *[[https://stackoverflow.com/questions/38035726/how-do-i-run-apache-2-on-alpine-in-docker|How do I run Apache 2 on Alpine in Docker?]] *[[https://opensource.com/article/18/2/apache-web-server-configuration|How to configure an Apache web server]] *[[https://opensource.com/article/18/3/configuring-multiple-web-sites-apache|How to configure multiple websites with Apache web server]] *[[https://www.tecmint.com/check-apache-modules-enabled/|How to Check Which Apache Modules are Enabled/Loaded in Linux]] *[[https://www.tecmint.com/list-enabled-virtual-hosts-in-apache-web-server/|How to List All Virtual Hosts in Apache Web Server]] *[[https://wiki.alpinelinux.org/wiki/Apache_with_php-fpm|Apache with php-fpm]] *[[https://www.php.net/manual/en/install.fpm.configuration.php|PHP-FPM Configuration]] *[[https://cwiki.apache.org/confluence/display/httpd/PHP-FPM|PHP-FPM]] *[[https://dev.to/joetancy/php-fpm-with-apache2-2mk0|PHP-FPM with Apache2]] *[[https://cwiki.apache.org/confluence/display/httpd/PHP-FPM|PHP-FPM]] *[[https://blog.joshwalsh.me/docker-nginx-php-fpm/|https://blog.joshwalsh.me/docker-nginx-php-fpm/]] *[[https://github.com/JoshuaWalsh/docker-nginx-for-php-fpm/blob/master/nginx-for-php-fpm/default.conf|https://github.com/JoshuaWalsh/docker-nginx-for-php-fpm/tree/master]] *[[https://www.php.net/manual/en/install.fpm.php#122457|FastCGI Process Manager (FPM)]] =====Roundcube WebMail===== I decided to use the [[https://hub.docker.com/r/roundcube/roundcubemail|Roundcube Docker]] official image and followed the instructions to setup. As usual I decided to go with the Alpine Linux image option. *[[https://github.com/roundcube/roundcubemail-docker/blob/master/examples/docker-compose-fpm-alpine.yaml|roundcubemail-docker/examples/docker-compose-fpm-alpine.yaml]] *[[https://github.com/roundcube/roundcubemail-docker/blob/master/tests/nginx-default.conf|roundcubemail-docker/tests/nginx-default.conf]] *[[https://github.com/roundcube/roundcubemail-docker/blob/master/tests/docker-compose.test-fpm-postgres.yml|roundcubemail-docker/tests/docker-compose.test-fpm-postgres.yml]] My final Docker compose file, ''docker-compose.yml'' ++++file| version: '3.9' services: roundcubemail: image: roundcube/roundcubemail:latest-fpm-alpine container_name: roundcubemail # restart: unless-stopped depends_on: - roundcubedb # links: # - roundcubedb networks: proxy: ports: - 9000:9000 volumes: - ./.config/www:/var/www/html environment: - ROUNDCUBEMAIL_DB_TYPE=pgsql - ROUNDCUBEMAIL_DB_HOST=roundcubedb # same as pgsql container name - ROUNDCUBEMAIL_DB_NAME=roundcube # same as pgsql POSTGRES_DB env name - ROUNDCUBEMAIL_DB_USER=roundcube # same as pgsql POSTGRES_USER env name - ROUNDCUBEMAIL_DB_PASSWORD=roundcube # same as pgsql POSTGRES_PASSWORD env name - ROUNDCUBEMAIL_SKIN=elastic - ROUNDCUBEMAIL_DEFAULT_HOST=tls://mail.kptree.net - ROUNDCUBEMAIL_SMTP_SERVER=tls://mail.kptree.net - ROUNDCUBEMAIL_DEFAULT_PORT=143 - ROUNDCUBEMAIL_SMTP_PORT=587 - ROUNDCUBEMAIL_UPLOAD_MAX_FILESIZE=50M roundcubedb: image: postgres:alpine container_name: roundcubedb # restart: unless-stopped networks: proxy: ports: - 5432:5432 volumes: - ./.config/db/postgres:/var/lib/postgresql/data environment: - POSTGRES_DB=roundcube - POSTGRES_USER=roundcube - POSTGRES_PASSWORD=roundcube roundcubenginx: image: nginx:alpine container_name: roundcubenginx # restart: unless-stopped networks: proxy: ports: - 9008:80 # If you need SSL connection # - '443:443' depends_on: - roundcubemail # links: # - roundcubemail volumes: - ./.config/www:/var/www/html - ./.config/nginx/templates:/etc/nginx/templates - ./.config/nginx/nginx-default.conf:/etc/nginx/conf.d/default.conf # Provide a custom nginx conf # - ./nginx/nginx.conf:/etc/nginx/nginx.conf:ro # If you need SSL connection, you can provide your own certificates # - ./certs:/etc/letsencrypt # - ./certs-data:/data/letsencrypt environment: - NGINX_HOST=localhost # set your local domain or your live domain - NGINX_PHP_CGI=roundcubemail:9000 # same as roundcubemail container name ### Optional: add a full mail server stack to use with Roundcube like https://hub.docker.com/r/tvial/docker-mailserver # mailserver: # image: tvial/docker-mailserver:latest # hostname: mail.example.org # ... # for more options see https://github.com/tomav/docker-mailserver#examples networks: proxy: external: true ++++ See: *[[https://docs.docker.com/compose/compose-file/05-services/#depends_on|Docker compose: depends_on]] Depends upon defines start-up and shutdown dependencies on containers in a stack. *[[https://docs.docker.com/compose/compose-file/05-services/#links|Docker compose: links]] Links is not specifically required in compose version 3. services connected to a shared network can communicate. docker compose ++++file structure| -- ./ (based docker compose directory) ├── docker-compose.yml └── .config ├── db ├── postgres .... (directory location of Postgres database files, created by Roundcube if empty) ├── nginx | ├── nginx-default.conf | └── templates (directory) └── www (directory location of Roundcube files, created by Roundcube if empty) ++++ Some links: *[[https://workaround.org/ispmail-bookworm/webmail-using-roundcube/|Webmail using Roundcube]] *[[https://plugins.roundcube.net/#/|Roundcube plugins]] *dockerhub *[[https://hub.docker.com/_/postgres|postgres]] *[[https://hub.docker.com/r/roundcube/roundcubemail|roundcubemail]] =====Traefik Routing===== Looks like Traefik can not handle routing of STARTTLS. At least as of 2023-12-02. My LAN DNS points to the one common IP address for the mail server. DNS does not differentiate the services via port numbers. As noted I can not use Traefik to perform this routing if the STARTTLS protocol is used. I could possibly use my router, but this is getting too complex. To allow operation of a Docker Web browser based mail client with my existing VM mail server I used a slightly different URL to help with routing, e.g. webmail.kptree.net versus mail.kptree.net for mail server. The webmail.kptree.net successfully used Traefik router and successfully operated with the mailserver on mail.kptree.net. ====References==== *[[https://roundcube.net/|Roundcube]] *[[https://hub.docker.com/r/roundcube/roundcubemail/|Docker roundcube/roundcubemail]] *[[https://github.com/roundcube/roundcubemail-docker|Roundcube Docker Git Page]] *[[https://wiki.alpinelinux.org/wiki/Roundcube|Alpine Roundcube]] seems to be based upon Lighttpd web server and postgres database with manual download of Roundcube. =====dmarc===== [[https://www.mailercheck.com/articles/how-to-read-a-dmarc-report-and-actually-understand-it|How to read a DMARC report—and actually understand it!]] =====rspamd===== rspamd -u rspamd -g rspamd Help from rspamd: ++++rspamd -h| Usage: rspamd [OPTION*] - run rspamd daemon Summary: Rspamd daemon version 3.7.4 Help Options: -h, --help Show help options Application Options: -f, --no-fork Do not daemonize main process -c, --config Specify config file(s) -u, --user User to run rspamd as -g, --group Group to run rspamd as -p, --pid Path to pidfile -d, --debug Force debug output -i, --insecure Ignore running workers as privileged users (insecure) -v, --version Show version and exit --var Redefine/define environment variable -T, --skip-template Do not apply Jinja templates --lua-env Load lua environment from the specified files ++++ [[https://rspamd.com/doc/quickstart.html|Rspamd quick start]] =====redis===== *Load the Alpine package ''redis'' *To start on command line ''redis-server /etc/redis.conf &'' *The redis configuration file ''/etc/redis.conf'' is installed with the redis package. *The default conf log file is ''/var/log/redis/redis.log''. Use ''mkdir /var/log/redis'' to make directory if required. *To stop on command line ''redis-cli shutdown'' *Redis complains if 'vm.overcommit_memory' is not equal to 1. *This is not set in the container (Docker does have sysctl directive, but not all kernal parameters are support, including this one.) *This can be set in the host ''sudo sysctl -w vm.overcommit_memory=1'' Some reference links: *Redis *[[https://redis.io/docs/install/install-redis/|Install Redis]] *[[https://redis.io/docs/get-started/faq/#background-saving-fails-with-a-fork-error-on-linux|Redis FAQ]] *[[https://ourcodeworld.com/articles/read/2083/how-to-remove-redis-warning-on-docker-memory-overcommit-must-be-enabled|How to remove Redis warning on Docker: Memory overcommit must be enabled ]] *[[https://docs.docker.com/engine/reference/commandline/run/#sysctl|Docker sysctl]] ====nftables==== rspamd requires netfilter chains to functions. So a nftables needs to be available and a basic input chain setup to function. See [[https://wiki.kptree.net/doku.php?id=docker_notes:docker#dns_and_nftable_iptables_netfilter_within_containers|netfilter use within containers]] that describe why care must be taken no to interfere with existing netfilter nat chains required for container DNS function. =====References===== *Workaround.org [[https://workaround.org/ispmail/bullseye/|ISPmail guide for Debian 11 “Bullseye”]] *Gentoo [[https://wiki.gentoo.org/wiki/Complete_Virtual_Mail_Server|Complete Virtual Mail Server]] *[[https://kinsta.com/blog/adminer/|How to Use Adminer to Manage Databases Easily with a Single PHP File]] *[[https://techoverflow.net/2021/07/18/how-to-export-certificates-from-traefik-certificate-store/|How to export certificates from Traefik certificate store]] python script *[[https://r4uch.com/export-traefik-certificates/|Export Traefik Certificates]] using bash script with ''jq'' program *Docker documentation specific *Docker [[https://github.com/compose-spec/compose-spec/blob/master/build.md|The Compose Specification - Build support]] *Alpine *[[https://pkgs.alpinelinux.org/packages|Alpine Packages]] *[[https://wiki.alpinelinux.org/wiki/Repositories|Repositories]] It is not recommended to mix repositories. *[[https://wiki.alpinelinux.org/wiki/Protecting_your_email_server_with_Alpine|Protecting your email server with Alpine]] *[[https://wiki.alpinelinux.org/wiki/Hosting_Web/Email_services_on_Alpine|Hosting Web/Email services on Alpine]] *Dovecot *[[https://doc.dovecot.org/admin_manual/pigeonhole_managesieve_server/|Pigeonhole ManageSieve Server]] *Misc *Digitalocean [[https://www.digitalocean.com/community/tutorials/how-to-configure-a-mail-server-using-postfix-dovecot-mysql-and-spamassassin?fbclid=IwAR2VEJt2HfQrZAJJ40buTSoPHwxTshKEIISLsE1nX4M5RRq0oO-PV_EbHew#step-5-configure-spamassassin|How To Configure a Mail Server Using Postfix, Dovecot, MySQL, and SpamAssassin]] *Codehouse [[https://codehouse.digfish.org/how-to-configure-an-e-mail-distribution-service-using-postfix-courier-imap-with-sqlite-as-the-credentials-storage-endpoint/|How to configure an E-mail distribution service using Postfix + Courier IMAP with SQLite as the credentials storage endpoint]] <- docker_notes:docker-deluge|Back ^ docker_notes:index|Start page ^ docker_notes:docker-dns|Next ->