Infrastructure Mint System

A short introduction into the infrastructure of Mint System.


Early on we wanted to be able to self-host web application.

ERP-Systems are operating system for companies

  • They have to be hosted reliably
  • Companies want know where their data is

Infrastructure as Code:

  • Every system / server can be rebuilt from code
  • The deployment of configuration must be automated

Run apps with Docker containers:

  • Manage containers not apps
  • Every decent web app provides Docker images


We don't want to manage hardware. We use these providers:

  • Hetzner
  • Exoscale
  • Infomaniak
  • Ungleich

And service partners.


Everything is done with Ansible. There is a role for that: in new window

  • Install and configure the OS
  • Manage firewalls and access rights
  • Deploy Docker containers
  • Manage backups and cron jobs
  • Build Wireguard networks


We provide a managed server service.

Every customer has a server. No shared environments.

Every application is a hosting offer.


For things that cannot be automated there are scripts: in new window

Ansible roles rely on these helper scripts. Helper scripts work independent of Ansible.


To monitor servers, containers and applications we use Prometheus/Grafana.

Prometheus provides "Exporters" for many applications. Data collection and visualization is simple.


Backups are defined in Ansible and are done using restic.

  • In Ansible inventory a backup set is defined
  • Ansible creates a cron job to run the backup
  • The backup job uses helper scripts to create local backups
  • The local backups are snapshoted with restic to the backup server
  • All backup data on the backup server is mirrored to an S3 bucket


On every server there is Nginx-instance running.

The Nginx config is generated by Ansible: in new window

Some roles require specific configs: in new window

Certificates are managed with certbot and let's encrypt.


We a running Keycloak and integrate it using OAuth/OpenID Connect.

It is possible to manage all Keycloak config in Ansible.


We try to apply best practices:

  • Update our applications / Docker images
  • Linux server patching
  • Basics: fail2ban, ssh pubkey, named users

Nginx WAF with OWASP has been tried. It was too much effort to train.


Configuration drift

One an Ansible role or a config is update we have to apply this change to all hosts. This rarely done.

Proxy config not integrated

The proxy config is not provided by the Ansible role. It would be better if the application provides the config.

Backup verification

We cannot be sure that all backups are working. So far they did.