docker-mailserver: [BUG] new server: fail2ban and opendkim errors: cpu hogged

Miscellaneous first checks

  • I checked that all ports are open and not blocked by my ISP / hosting provider.
  • I know that SSL errors are likely the result of a wrong setup on the user side and not caused by DMS itself. I’m confident my setup is correct.

Affected Component(s)

fail2ban server fails

What happened and when does this occur?

mailserver | 2023-02-11 23:18:48,020 DEBG ‘fail2ban’ stderr output: mailserver | 2023-02-11 23:18:48,018 fail2ban [710]: ERROR Could not start server. Maybe an old socket file is still present. Try to remove /var/run/fail2ban/fail2ban.sock. If you used fail2ban-client to start the server, adding the -x option will do it mailserver | mailserver | 2023-02-11 23:18:53,037 DEBG ‘fail2ban’ stderr output: mailserver | /usr/local/bin/fail2ban-wrapper.sh: line 27: /var/run/fail2ban/fail2ban.pid: No such file or directory mailserver | mailserver | 2023-02-11 23:18:53,039 DEBG fd 51 closed, stopped monitoring <POutputDispatcher at 139749306469920 for <Subprocess at 139749306469872 with name fail2ban in state RUNNING> (stdout)> mailserver | 2023-02-11 23:18:53,039 DEBG ‘fail2ban’ stderr output: mailserver | /usr/local/bin/fail2ban-wrapper.sh: line 27: kill: `': not a pid or valid job spec mailserver |

What did you expect to happen?

Hi, I am setting up a new server from scratch

A few problems:

  1. In the environment I disabled dkim, but it still starts and crashes because a file does not exist
  2. Worse, fail2ban server does not start properly. After a while there are 15 server processes eating all cpu

How do we replicate the issue?

  1. start a new server in docker
  2. use my config I think , I disabled opendkim and clamav and spamassasin and enabled rspamd
  3. see fail2ban fail? …

Also I noticed although I have disabled opendkim it still starts it up anyway

DMS version

v11.3.0

What operating system is DMS running on?

Linux

Which operating system version?

vmware photon 4

What instruction set architecture is DMS running on?

AMD64 / x86_64

What container orchestration tool are you using?

Docker

docker-compose.yml

services:
  mailserver:
    image: docker.io/mailserver/docker-mailserver:latest
    container_name: mailserver
    # If the FQDN for your mail-server is only two labels (eg: example.com),
    # you can assign this entirely to `hostname` and remove `domainname`.
    hostname: mail
    domainname: fanscees.net
    env_file: mailserver.env
    # More information about the mail-server ports:
    # https://docker-mailserver.github.io/docker-mailserver/edge/config/security/understanding-the-ports/
    # To avoid conflicts with yaml base-60 float, DO NOT remove the quotation marks.
    ports:
      - "25:25"    # SMTP  (explicit TLS => STARTTLS)
      - "143:143"  # IMAP4 (explicit TLS => STARTTLS)
      - "465:465"  # ESMTP (implicit TLS)
      - "587:587"  # ESMTP (explicit TLS => STARTTLS)
      - "993:993"  # IMAP4 (implicit TLS)
      - "995:995"  # pop3 (implicit TLS)
      - "110:110"  # pop3 (implicit TLS)
    volumes:
      - etc:/etc
      - maildata:/var/mail/
      - mailstate:/var/mail-state/
      - log:/var/log/
      - Tconfig:/tmp/docker-mailserver/
      - /etc/localtime:/etc/localtime:ro
      - ssl:/tmp/ssl
    restart: always
    stop_grace_period: 1m
    environment:
     # If you need SSL connection, you can provide your own certificates
     - SSL_TYPE=manual
     - SSL_CERT_PATH=/tmp/ssl/fullchain.pem
     - SSL_KEY_PATH=/tmp/ssl/privkey.pem
    cap_add:
      - NET_ADMIN
     healthcheck:
      test: "ss --listening --tcp | grep -P 'LISTEN.+:smtp' || exit 1"
      timeout: 3s
      retries: 0

volumes:
  maildata:
  mailstate:
  Tconfig:
  log:
  ssl:
  etc:

Relevant log output

mailserver  | ACCOUNT_PROVISIONER='FILE'
mailserver  | AMAVIS_LOGLEVEL='2'
mailserver  | CLAMAV_MESSAGE_SIZE_LIMIT='25M'
mailserver  | DEFAULT_RELAY_HOST=''
mailserver  | DOVECOT_INET_PROTOCOLS='ipv4'
mailserver  | DOVECOT_MAILBOX_FORMAT='maildir'
mailserver  | DOVECOT_TLS='no'
mailserver  | ENABLE_AMAVIS='0'
mailserver  | ENABLE_CLAMAV='0'
mailserver  | ENABLE_DNSBL='1'
mailserver  | ENABLE_FAIL2BAN='1'
mailserver  | ENABLE_FETCHMAIL='0'
mailserver  | ENABLE_MANAGESIEVE='0'
mailserver  | ENABLE_POP3='1'
mailserver  | ENABLE_POSTGREY='1'
mailserver  | ENABLE_QUOTAS='0'
mailserver  | ENABLE_SASLAUTHD='0'
mailserver  | ENABLE_SPAMASSASSIN='0'
mailserver  | ENABLE_SPAMASSASSIN_KAM='1'
mailserver  | ENABLE_SRS='0'
mailserver  | ENABLE_UPDATE_CHECK='1'
mailserver  | FAIL2BAN_BLOCKTYPE='drop'
mailserver  | FETCHMAIL_PARALLEL='0'
mailserver  | FETCHMAIL_POLL='300'
mailserver  | LOGROTATE_INTERVAL='weekly'
mailserver  | LOGWATCH_INTERVAL='none'
mailserver  | LOGWATCH_RECIPIENT='hanscees@hanscees.com'
mailserver  | LOGWATCH_SENDER='mailserver-report@mail.hanscees.net'
mailserver  | LOG_LEVEL='debug'
mailserver  | MOVE_SPAM_TO_JUNK='0'
mailserver  | NETWORK_INTERFACE='eth0'
mailserver  | ONE_DIR='1'
mailserver  | OVERRIDE_HOSTNAME='mail.hanscees.net'
mailserver  | PERMIT_DOCKER='none'
mailserver  | PFLOGSUMM_RECIPIENT='hanscees@hanscees.com'
mailserver  | PFLOGSUMM_SENDER='mailreport@hanscees.com'
mailserver  | PFLOGSUMM_TRIGGER='logrotate'
mailserver  | POSTFIX_INET_PROTOCOLS='ipv4'
mailserver  | POSTFIX_MAILBOX_SIZE_LIMIT='0'
mailserver  | POSTFIX_MESSAGE_SIZE_LIMIT='80240000'
mailserver  | POSTGREY_AUTO_WHITELIST_CLIENTS='3'
mailserver  | POSTGREY_DELAY='15'
mailserver  | POSTGREY_MAX_AGE='35'
mailserver  | POSTGREY_TEXT='Delayed by Postgrey'
mailserver  | POSTMASTER_ADDRESS='hanscees@hanscees.com'
mailserver  | POSTSCREEN_ACTION='enforce'
mailserver  | RELAY_HOST=''
mailserver  | REPORT_RECIPIENT='hanscees@hanscees.com'
mailserver  | REPORT_SENDER='mailserver-report@mail.hanscees.net'
mailserver  | SA_KILL='10.0'
mailserver  | SA_SPAM_SUBJECT='***SPAM*****'
mailserver  | SA_TAG2='6.31'
mailserver  | SA_TAG='2.0'
mailserver  | SMTP_ONLY='0'
mailserver  | SPAMASSASSIN_SPAM_TO_INBOX='1'
mailserver  | SPOOF_PROTECTION='1'
mailserver  | SRS_SENDER_CLASSES='envelope_sender'
mailserver  | SSL_TYPE='manual'
mailserver  | SUPERVISOR_LOGLEVEL='debug'
mailserver  | TLS_LEVEL='modern'
mailserver  | TZ='Europe/Amsterdam'
mailserver  | UPDATE_CHECK_INTERVAL='1d'
mailserver  | VIRUSMAILS_DELETE_DELAY='7'
mailserver  | 2023-02-11 22:30:14,701 DEBG 'mailserver' stdout output:
mailserver  | ACCOUNT_PROVISIONER='FILE'
mailserver  | AMAVIS_LOGLEVEL='2'
mailserver  | CLAMAV_MESSAGE_SIZE_LIMIT='25M'
mailserver  | DEFAULT_RELAY_HOST=''
mailserver  | DOVECOT_INET_PROTOCOLS='ipv4'
mailserver  | DOVECOT_MAILBOX_FORMAT='maildir'
mailserver  | DOVECOT_TLS='no'
mailserver  | ENABLE_AMAVIS='0'
mailserver  | ENABLE_CLAMAV='0'
mailserver  | ENABLE_DNSBL='1'
mailserver  | ENABLE_FAIL2BAN='1'
mailserver  | ENABLE_FETCHMAIL='0'
mailserver  | ENABLE_MANAGESIEVE='0'
mailserver  | ENABLE_POP3='1'
mailserver  | ENABLE_POSTGREY='1'
mailserver  | ENABLE_QUOTAS='0'
mailserver  | ENABLE_SASLAUTHD='0'
mailserver  | ENABLE_SPAMASSASSIN='0'
mailserver  | ENABLE_SPAMASSASSIN_KAM='1'
mailserver  | ENABLE_SRS='0'
mailserver  | ENABLE_UPDATE_CHECK='1'
mailserver  | FAIL2BAN_BLOCKTYPE='drop'
mailserver  | FETCHMAIL_PARALLEL='0'
mailserver  | FETCHMAIL_POLL='300'
mailserver  | LOGROTATE_INTERVAL='weekly'
mailserver  | LOGWATCH_INTERVAL='none'
mailserver  | LOGWATCH_RECIPIENT='hanscees@fanscees.com'
mailserver  | LOGWATCH_SENDER='mailserver-report@mail.fanscees.net'
mailserver  | LOG_LEVEL='debug'
mailserver  | MOVE_SPAM_TO_JUNK='0'
mailserver  | NETWORK_INTERFACE='eth0'
mailserver  | ONE_DIR='1'
mailserver  | OVERRIDE_HOSTNAME='mail.fanscees.net'
mailserver  | PERMIT_DOCKER='none'
mailserver  | PFLOGSUMM_RECIPIENT='hanscees@fanscees.com'
mailserver  | PFLOGSUMM_SENDER='mailreport@fanscees.com'
mailserver  | PFLOGSUMM_TRIGGER='logrotate'
mailserver  | POSTFIX_INET_PROTOCOLS='ipv4'
mailserver  | POSTFIX_MAILBOX_SIZE_LIMIT='0'
mailserver  | POSTFIX_MESSAGE_SIZE_LIMIT='80240000'
mailserver  | POSTGREY_AUTO_WHITELIST_CLIENTS='3'
mailserver  | POSTGREY_DELAY='15'
mailserver  | POSTGREY_MAX_AGE='35'
mailserver  | POSTGREY_TEXT='Delayed by Postgrey'
mailserver  | POSTMASTER_ADDRESS='hanscees@fanscees.com'
mailserver  | POSTSCREEN_ACTION='enforce'
mailserver  | RELAY_HOST=''
mailserver  | REPORT_RECIPIENT='hanscees@fanscees.com'
mailserver  | REPORT_SENDER='mailserver-report@mail.fanscees.net'
mailserver  | SA_KILL='10.0'
mailserver  | SA_SPAM_SUBJECT='***SPAM*****'
mailserver  | SA_TAG2='6.31'
mailserver  | SA_TAG='2.0'
mailserver  | SMTP_ONLY='0'
mailserver  | SPAMASSASSIN_SPAM_TO_INBOX='1'
mailserver  | SPOOF_PROTECTION='1'
mailserver  | SRS_SENDER_CLASSES='envelope_sender'
mailserver  | SSL_TYPE='manual'
mailserver  | SUPERVISOR_LOGLEVEL='debug'
mailserver  | TLS_LEVEL='modern'
mailserver  | TZ='Europe/Amsterdam'
mailserver  | UPDATE_CHECK_INTERVAL='1d'
mailserver  | VIRUSMAILS_DELETE_DELAY='7'

Other relevant information

I have no idea why this happens. Two different processes complain about a missing file opendkim and fail2ban. That is odd.

Only volume I do not throw away is my email addresses data, so when docker image is destroyed the postfix email addresses remain in the docker volume.

What level of experience do you have with Docker and mail servers?

  • I am inexperienced with docker
  • I am rather experienced with docker
  • I am inexperienced with mail servers
  • I am rather experienced with mail servers
  • I am uncomfortable with the CLI
  • I am rather comfortable with the CLI

Code of conduct

Improvements to this form?

No response

About this issue

  • Original URL
  • State: closed
  • Created a year ago
  • Comments: 17 (17 by maintainers)

Most upvoted comments

I am running the :edge thing now and it does not hog the cpu so it looks good 😃

Still wrestling a bit with the iptables and nftables rules on the vmware host the docker is running on here. Got that running, but later on its critical the fail2ban indeed bans IP’s effectively. Looking good for now, more tomorrow.

+1
hanscees on Feb 13, 2023

Probably a good idea to add a note or two. I’d go for something similar to the second proposal. I will provide an update after my current PRs are merged (or @polarathene will, he’s our docs guy 😄)

+1
georglauterbach on Feb 14, 2023

I have now red the readme page and I still don’t understand howto find the right version if I want to use the stable version (presumably most users will want to use that right?). I know it is logical to you, since you all live here. But my recommendation would be:

  1. make the documentation on your readme page suitable for the stable version. not for the edge version
  2. second best if you think thats nonsense (and most people who are in a project for a while don’t see what new users see ) at least add a warning on your landing page https://github.com/docker-mailserver/docker-mailserver Below the heading " get the tools" state The commands below will download the edge branch, if you want the stable branch, see the documentation.
+1
hanscees on Feb 13, 2023

OpenDKIM I believe runs by default until the new release is made (with changes in :edge). If it hasn’t been given all the config files, it repeatedly crashes and supervisord keeps restarting it. That bug has been fixed (but also only in :edge).

Fail2Ban in :edge has been upgraded to a newer version that won’t run into the problem you experienced either. It should cleanup that socket file itself now and not end up running so many copies as the command to run it with supervisord has been changed / improved.

Sorry about those issues, we should hopefully have the new breaking release v12 out this month 👍

If you’d like to test the latest changes use the :edge image, that follows our master branch and will be updated each time (where you’ll have to pull the image again explicitly to update it).

Also, I am fairly sure I recognize your username as someone who contributed a script to the Github Wiki before we migrated the docs back in early 2021? You’re probably aware by now, but there was a breaking change made back in 10.2 that impacted your script that interacted with internal-only cert files 😅 (breaking change allowed in this sense since it was internal information, not something to be publicly relied on)

This is the script I remembered. If you still use that or the other ones for DMS, you may want to review them.

+1
polarathene on Feb 12, 2023

I see where the issue is 😃 Please read the README more carefully; it states:

If you have issues, read the full README and the documentation for your version (default is edge) first before opening an issue.

and our tagging convention also states that :latest is the latest release, not the latest container image.

You’re running :latest, which is 11.3.X. You need :edge though when trying to disable OpenDKIM. Please try changing the image tag, then we’ll see what happens.

Make sure to use the documentation for the image tag you’re using. Our documentation has a “drop-down” for adjusting the version.

+1
georglauterbach on Feb 12, 2023
Read more comments on GitHub
← docker-mailserver: [BUG] More than 20s delay before postfix server responds
docker-mailserver: [BUG] opendmarc rejects mails that should be passed (SPF) →