Reverse Proxy Server [KPTree Wiki]

This page is read only. You can view the source, but not change it. Ask your administrator if you think this is wrong.
{{tag>linux docker traefik "reverse proxy" proxy openssl ssl certificate portainer cloudsec}}
======Reverse Proxy Server======
I seem to have gotten the Traefik reverse proxy working according to Techno Tim [[https://docs.technotim.live/posts/traefik-portainer-ssl/|Put Wildcard Certificates and SSL on EVERYTHING]] ([[https://github.com/techno-tim/techno-tim.github.io/tree/master/reference_files/traefik-portainer-ssl|github reference_files for traefik-portainer-ssl]]) 

Below is a basic description of the process that aligns with my configuration files. I do this for 2 reasons, both allowing me independence. 
  - Sometimes the source information or link are; changed, lost or removed. 
  - These note reference my current specific installation.

=====Proxy network to connect them all=====
These containers all talk via a docker bridge network named proxy, ''docker network create proxy''
=====Traefik=====
<code bash [enable_line_numbers="true"]>
cd /home/docker_store
sudo mkdir traefik
sudo chown baumkp:baumkp traefik
cd traefik
mkdir data
cd data
touch acme.json
chmod 600 acme.json
touch traefik.yml
cd ..</code>
My traefik.yml locatation: ''/home/docker_store/traefik/data/traefik.yml''. The current TechnoTim one [[https://github.com/techno-tim/techno-tim.github.io/tree/master/reference_files/traefik-portainer-ssl/traefik|here]].\\

====create docker network====
<code bash [enable_line_numbers="true"]>docker network create proxy</code>
<code bash [enable_line_numbers="true"]>touch docker-compose.yml
touch provider.env</code>
My docker-compose.yml location: ''/home/docker_store/traefik/docker-compose.yml''. The current TechnoTim one [[https://github.com/techno-tim/techno-tim.github.io/tree/master/reference_files/traefik-portainer-ssl/traefik|here.]]\\ 
//<fc #ff0000><fs small>**Note** my docker compose file has some changes from the TechnoTim one, in particular the use of the Godaddy DNS chanlenge API instead of the the Cloudflare one used by TechnoTim.</fs></fc>//\\
 \\
====Generate and Install Godaddy DNS Challenge Data====
Godaddy changed their policies circa April 2024 that basically does not give small users access to their developers API system.  Sadly and unprofessionally they did this without informing users of the policy.  I only found out when my proxy server issued messages of certificate update failure, as the API DNS challenge stopped working.  I checked the internet and it was indicated that Godaddy had changed their policies, however I found it difficult to believe that Godaddy would change their policy without contacting me, after all I am a paying customer!  After raising a ticket on the matter, I was sent an email explaining the policy change.  I immediately moved my domain DNS server to Hurricane Electronics.  I needed to change the DNS verification process to suit that used by Hurricane Electronics.  

My domain is still registered via Godaddy, I expect that I will look at moving to another registry when the registration comes due.  I do not wish to support Godaddy going forward with my business. <color #ff7f27>**Godaddy SUCKS!**</color>

++++Old, tl;dr;|
Sadly Godaddy does not make it as transparent as it should be to access their DNS challenge API. Perhaps because they are focused on their commercial certificate product. It is accessed from their developer portal [[https://developer.godaddy.com/|Godaddy Developer Portal]], from here the API keys can be made.  These keys then need to be copied into ''/home/docker_store/traefik/data/provider.env'':
<code [enable_line_numbers="true">GODADDY_API_KEY=[Your API_KEY key from Godaddy API]
GODADDY_API_SECRET=[Your API_SECRET key from Godaddy API]</code>
++++
====Generate and install Basic Authentication Password====
<code bash [enable_line_numbers="true"]>sudo apt update
sudo apt install apache2-utils</code>
<code bash [enable_line_numbers="true"]>echo $(htpasswd -nb "<USER>" "<PASSWORD>") | sed -e s/\\$/\\$\\$/g</code>
NOTE: Replace <USER> with your username and <PASSWORD> with your password to be hashed.

Paste the output in your docker-compose.yml in line (traefik.http.middlewares.traefik-auth.basicauth.users=<USER>:<HASHED-PASSWORD>)
 \\
 \\
<code bash [enable_line_numbers="true"]>cd data
touch config.yml</code>
<code bash [enable_line_numbers="true"]>docker-compose up -d</code>
=====Portainer=====
  *[[https://www.portainer.io/|portainer]]
<code bash [enable_line_numbers="true"]>cd /home/docker_store
sudo mkdir portainer
sudo chown baumkp:baumkp portainer
cd portainer
touch docker-compose.yml
mkdir data</code>
My docker-compose.yml location: ''/home/docker_store/portainer/docker-compose.yml''. The current TechnoTim one [[https://github.com/techno-tim/techno-tim.github.io/tree/master/reference_files/traefik-portainer-ssl/portainer|here.]]\\
<code bash [enable_line_numbers="true"]>docker-compose up -d</code>

=====Traefik Routes Config=====
<code bash [enable_line_numbers="true"]>cd /home/docker_store/traefik/data
nvim config.yml</code>
I have broken down the Traefik router dynamic configuration file, My config.yml into 2 configuration files. One for the http/https specific router configuration and the other for the tcp router configuration, I do not use any UDP router configurations to date. I placed these files in the sub-directory: ''/home/docker_store/traefik/data/config''. Traefik has been setup to look at all configuration files in this sub-directory and to dynamically update changes on the run. The current TechnoTim one [[https://github.com/techno-tim/techno-tim.github.io/tree/master/reference_files/traefik-portainer-ssl/traefik|here.]], also look at **Portainer's** instructions here: [[https://docs.portainer.io/advanced/reverse-proxy/traefik|Deploying Portainer behind Traefik Proxy]]\\
<code bash [enable_line_numbers="true"]>docker-compose up -d --force-recreate</code>\\
Folder Structure:
<code>./traefik
├── data
│   ├── acme.json         | This is the Lets Encrypt RSA key file downloaded by Traefik
│   ├── config
|   |   ├── http.yml      | This is the dynamic configuration file for http (want to separate into 2 smaller files, basic and main services)
|   |   └── tcp.yml       | This is the dynamic configuration file for tcp (not using at the moment, starttls is not supported by Traefik at this time)
│   ├── provider.env      | This has the key file for DNS wildcard challenge on LetsEncrypt 
│   ├── traefik.yml       | This is the main traefik static configuration file 
│   └── traefik.log       | This is the main traefik log file (permanent, but does not show up on the Docker error log)
└── docker-compose.yml</code>
=====whitelisting=====
The Traefik middleware ipWhitelist only allows the define ip address(es) to be forwarded.  All other address will have 403 forbidden returned.
++++ipWhitelist|
    <code>default-whitelist:|
      ipWhiteList:
        sourceRange:
        - "10.0.0.0/8"
        - "192.168.0.0/16"
        - "172.16.0.0/12"</code>
++++

=====BasicAuth=====
For any internal service I expose to the public internet that are either not full services with own password, e.g. dokuwiki, nextcloud and mail server, but I do not want general public access I would like to add basic password protection.  This is built into the web server applications such as Apache and presumably Nginx, but Traefik also has some functionality.

The middleware [[https://doc.traefik.io/traefik/middlewares/http/basicauth/|BasicAuth]] seems to define this functionality.  If I setup Gotify, that does not have an iOS client I can then use a public access webpage with password protection to check notifications.  Unfortunately this is not active, in that it does not actively alert of new messages that presumably an app would do, but would probably meet my needs.

=====SSL Services=====
For TCP and HTTPS services behind the Traefik router that require TLS the Traefik router must be specified to pass through the TLS, that is not terminate the SSL connection.

=====Entrypoints=====
The Standard entry point port normally defined are HTTP (port 80 and perhaps 8080) and HTTPS (port 443).  If you are using other services then additional entry points need to be defined, as required for each service / port.  Mail servers are an example that requires use of specialised TCP entrypoints. Often these entry points also needed to be passed to the server without handling (termination) of SSL connections. //Do not forget to expose the used ports in the Docker / Docker Compose file.//

====References====
  *[[https://doc.traefik.io/traefik/routing/routers/#passthrough|Router passthrough]]
  *[[https://blog.alexanderhopgood.com/traefik/letsencrypt/2020/07/18/traefik-tls-passthrough|Traefik and TLS Passthrough]]
  *[[https://community.traefik.io/t/enable-and-configure-file-provider-within-a-docker-compose-defined-traefik-instance/9732|Enable and configure file provider within a docker-compose defined Traefik instance]]

====References====
  *Traefik
    * [[https://hub.docker.com/_/traefik|Traefix]]
    * [[https://doc.traefik.io/traefik/https/acme/|traefik proxy & Lets Encrypt]]
    * Smarthome Beginner [[https://www.smarthomebeginner.com/traefik-docker-compose-guide-2022/|Ultimate Traefik Docker Compose Guide [2022] with LetsEncrypt]]
    * Christian Lempa [[https://github.com/ChristianLempa/boilerplates/tree/main/docker-compose/traefik|boilerplates/docker-compose/traefik/]]
    * Techno Tim [[https://github.com/techno-tim/techno-tim.github.io/blob/master/reference_files/traefik-portainer-ssl/traefik/docker-compose.yml|
techno-tim.github.io/reference_files/traefik-portainer-ssl/traefik/docker-compose.yml]] / [[https://docs.technotim.live/posts/traefik-portainer-ssl/|Put Wildcard Certificates and SSL on EVERYTHING]]
    *[[https://docs.portainer.io/v/2.15/admin/environments/add/docker|Add a Docker Standalone environment]]
    * [[https://github.com/traefik/traefik/issues/6686| (Traefik v2.2) Unable to obtain ACME certificate with DNS challenge using Go Daddy]]
    * [[https://stackoverflow.com/questions/61234489/cannot-get-wildcard-certificate-with-traefik-v2-and-godaddy|Cannot get wildcard certificate with traefik v2 and godaddy]]
    * [[https://forums.docker.com/t/traefik-acme-with-godaddy-as-provider/56743|Traefik - ACME with GoDaddy as provider]]

  *Traefik whitelists

  *Nginx Proxy Manager
    * Nginxproxymanager.com [[https://nginxproxymanager.com/advanced-config/#best-practice-use-a-docker-network|Best Practice: Use a Docker network]]

====ssl certificates====
  *''openssl x509 -in (path to certificate and certificate filename) -text -noout''
  *''openssl s_client -connect localhost:443 2>/dev/null | openssl x509 -noout -dates''
  *[[https://www.techrepublic.com/article/how-to-utilize-openssl-in-linux-to-check-ssl-certificate-details/|How to utilize openssl in Linux to check SSL certificate details]]

====Export Traefik certificates====

  *[[https://r4uch.com/export-traefik-certificates/|Export Traefik Certificates]]
  *Need to install the jq package
<code bash>#!/bin/bash

# Requirements: you will need to install jq and maybe openssl

# creates a directory for all of your certificates
mkdir -p certificates/

# reads the acme.json file, please put this file in the same directory as your script
json=$(cat acme.json)

export_cer_key () {
    echo $json | jq -r '.[].Certificates[] | select(.domain.main == "'$1'") | .certificate' | base64 -d > certificates/$1.cer
    echo $json | jq -r '.[].Certificates[] | select(.domain.main == "'$1'") | .key' | base64 -d > certificates/$1.key
}

export_pfx () {
        openssl pkcs12 -export -out certificates/$domain.pfx -inkey certificates/$domain.key -in certificates/$domain.cer -passout pass: 
}

read -p "Do you want to export as .pfx file as well [y]?" REPLY

# iterates through all of your domains
for domain in $(echo $json | jq -r '.[].Certificates[].domain.main')
do
    if [[ $REPLY =~ ^[Yy]$ ]]
    then
        export_cer_key "$domain"
        export_pfx "$domain"
    else
        export_cer_key "$domain"
    fi
done</code>
There is also [[https://techoverflow.net/2021/07/18/how-to-export-certificates-from-traefik-certificate-store/|How to export certificates from Traefik certificate store]] in python.

=====Cloudsec=====
AS of writing (June 2023) Crowdsec is still relatively new.  The documentation is not that easy to follow.  Most the Youtube crowd info uses Traefik to with Crowdsec.  I could also consider using nftable that I use as main ingress fire wall [[https://doc.crowdsec.net/docs/bouncers/firewall/|Firewall Bouncer]].

References:
  *[[https://www.crowdsec.net/|Crowdsec home]]
  *[[https://docs.technotim.live/posts/crowdsec-traefik/|Open Source & Collaborative Security with CrowdSec and Traefik - CrowdSec & Traefik Tutorial]]
  *[[https://docs.ibracorp.io/crowdsec/|ibracorp.io Crowdsec]]
  *Smarthome Beginner
    *[[https://www.smarthomebeginner.com/crowdsec-docker-compose-1-fw-bouncer/|Crowdsec Docker Compose Guide Part 1: Powerful IPS with Firewall Bouncer]]
    *[[https://www.smarthomebeginner.com/crowdsec-cloudflare-bouncer/|CrowdSec Docker Part 2: Improved IPS with Cloudflare Bouncer]]
    *[[https://www.smarthomebeginner.com/crowdsec-traefik-bouncer/|CrowdSec Docker Part 3: Traefik Bouncer for Additional Security]
    *[[https://www.smarthomebeginner.com/crowdsec-multiserver-docker/|CrowdSec Multiserver Docker (Part 4): For Ultimate Protection]]
    *[[https://www.smarthomebeginner.com/cloudflare-settings-for-traefik-docker/|Cloudflare Settings for Traefik Docker: DDNS, CNAMEs, & Tweaks]]
    *[[https://www.smarthomebeginner.com/traefik-docker-security-best-practices/|20 Docker Security Best Practices – Hardening Traefik Docker Stack]]
Crowdsec Bouncer Traefik plugin
  *[[https://plugins.traefik.io/plugins/6335346ca4caa9ddeffda116/crowdsec-bouncer-traefik-plugin|Crowdsec Bouncer Traefik Plugin]]


<- docker_notes:init|Back ^ docker_notes:index|Start page ^ docker_notes:docker-dokuwiki|Next ->