docker-mailserver: [NOT A BUG] Fresh 10.2.0 container crashes while booting: Failed to remove ClamAV configuration
Miscellaneous first checks
- I checked that all ports are open and not blocked by my ISP / hosting provider.
- I know that SSL errors are likely the result of a wrong setup on the user side and not caused by DMS itself. I’m confident my setup is correct.
Affected Component(s)
ClamAV
What happened and when does this occur?
When running a fresh container (no old mounted folders or files), the container crashes with the error:
[[ INF ]] Fixing Amavis permissions
[[ TASKS ]] Cleaning up disabled ClamAV
rm: cannot remove '/etc/logrotate.d/clamav-*': No such file or directory
rm: cannot remove '/etc/cron.d/clamav-freshclam': No such file or directory
[ FATAL ] Failed to remove ClamAV configuration
[ ERROR ] Shutting down..
### What did you expect to happen?
```Markdown
The container should start normally
How do we replicate the issue?
Just run the container using the settings below.
DMS version
v10.2.0 (edge)
How much RAM is available to DMS explicitly?
less than 2GB
How many CPU cores are available?
less than 1 Core
Is DMS running in a virtualized environment?
… a virtual private server (VPS) (with virtual CPU cores)
What operating system is DMS running on?
Linux
What instruction set architecture is DMS running on?
x86_64 / AMD64
I/O - Persistent memory
What container orchestration tool are you using?
Docker Compose
Docker version
Docker version 20.10.8, build 3967b7d
Docker Compose version
docker-compose version 1.29.2, build 5becea4c
The output of uname -a
Linux srv02 5.4.0-84-generic #94-Ubuntu SMP Thu Aug 26 20:27:37 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux
Important environment variables
DMS_DEBUG=1
POSTMASTER_ADDRESS=postmaster@MYDOMAIN.com
SSL_TYPE=manual
SSL_CERT_PATH=/etc/ssl/pki/mail.MYDOMAIN.com.crt
SSL_KEY_PATH=/etc/ssl/pki/mail.MYDOMAIN.com.key
ENABLE_LDAP=1
LDAP_START_TLS=yes
LDAP_SERVER_HOST=dc1.internal.MYDOMAIN.com
LDAP_SEARCH_BASE=dc=internal,dc=MYDOMAIN,dc=com
LDAP_BIND_DN=cn=Administrator,cn=users,dc=internal,dc=MYDOMAIN,dc=com
LDAP_BIND_PW=Ch@ngeMe
# Specifies how ldap should be asked for users.
LDAP_QUERY_FILTER_USER=(&(objectclass=person)(mail=%s))
# Specifies how ldap should be asked for groups.
LDAP_QUERY_FILTER_GROUP=(|)
# Specifies how ldap should be asked for aliases.
LDAP_QUERY_FILTER_ALIAS=(|)
# Specifies how ldap should be asked for domains.
LDAP_QUERY_FILTER_DOMAIN=(mail=*@%s)
DOVECOT_TLS=yes
DOVECOT_USER_FILTER=(&(objectclass=person)(sAMAccountName=%n))
DOVECOT_USER_ATTRS==uid=%{ldap:uidNumber},=gid=6000,=home=/var/mail/%Ln,=mail=maildir:~/Maildir
DOVECOT_PASS_ATTRS=sAMAccountName=user,userPassword=password
DOVECOT_AUTH_BIND=yes
ENABLE_SASLAUTHD=1
SASLAUTHD_MECHANISMS=ldap
SASLAUTHD_LDAP_SERVER=dc1.internal.MYDOMAIN.com
SASLAUTHD_LDAP_BIND_DN=cn=Administrator,cn=users,dc=internal,dc=MYDOMAIN,dc=com
SASLAUTHD_LDAP_SEARCH_BASE=dc=internal,dc=MYDOMAIN,dc=com
SASLAUTHD_LDAP_FILTER=(&(sAMAccountName=%U)(objectClass=person))
SASLAUTHD_LDAP_START_TLS=yes
SASLAUTHD_LDAP_TLS_CHECK_PEER=yes
SASLAUTHD_LDAP_TLS_CACERT_FILE=/etc/ssl/self-signed-pki/pki/ca.crt
Relevant log output
[[ INF ]] Fixing Amavis permissions
[[ TASKS ]] Cleaning up disabled ClamAV
rm: cannot remove '/etc/logrotate.d/clamav-*': No such file or directory
rm: cannot remove '/etc/cron.d/clamav-freshclam': No such file or directory
[ FATAL ] Failed to remove ClamAV configuration
[ ERROR ] Shutting down..
Other relevant information
**(do NOT miss number 2 below)**
1. Find the **Docker-Compose** file below:
version: '3.9'
services:
mailserver:
container_name: mailserver
image: docker.io/mailserver/docker-mailserver:edge # TODO change the TAG to 10.2.0 when released.
hostname: mail
domainname: ${PUBLIC_DOMAIN_NAME}
# dns: 1.1.1.1
# To avoid conflicts with yaml base-60 float, DO NOT remove the quotation marks.
# More information about the mailserver ports:
# https://docker-mailserver.github.io/docker-mailserver/edge/config/security/understanding-the-ports/
networks:
- internal01
ports:
- "25:25"
- "143:143"
- "587:587"
- "993:993"
volumes:
- mailserver-data:/var/mail/
- mailserver-state:/var/mail-state/
- mailserver-logs:/var/log/mail/
- mailserver-config:/tmp/docker-mailserver/
- /etc/localtime:/etc/localtime:ro
- publicly-trusted-pki:/etc/ssl/pki/:ro
restart: always
stop_grace_period: 1m
cap_add: [ "NET_ADMIN", "SYS_PTRACE" ]
env_file:
- mailserver/mailserver.env
environment:
# The following variables will be passed from the .env (if exist) to override the variables in mailserver.env.
- DMS_DEBUG
- POSTMASTER_ADDRESS
- SSL_TYPE
- SSL_CERT_PATH
- SSL_KEY_PATH
- ENABLE_LDAP
- LDAP_START_TLS
- LDAP_SERVER_HOST
- LDAP_SEARCH_BASE
- LDAP_BIND_DN
- LDAP_BIND_PW
- LDAP_QUERY_FILTER_USER
- LDAP_QUERY_FILTER_GROUP
- LDAP_QUERY_FILTER_ALIAS
- LDAP_QUERY_FILTER_DOMAIN
- DOVECOT_TLS
- DOVECOT_USER_FILTER
- DOVECOT_USER_ATTRS
- DOVECOT_PASS_ATTRS
- DOVECOT_AUTH_BIND
- ENABLE_SASLAUTHD
- SASLAUTHD_MECHANISMS
- SASLAUTHD_LDAP_SERVER
- SASLAUTHD_LDAP_BIND_DN
- SASLAUTHD_LDAP_SEARCH_BASE
- SASLAUTHD_LDAP_FILTER
- SASLAUTHD_LDAP_START_TLS
- SASLAUTHD_LDAP_TLS_CHECK_PEER
- SASLAUTHD_LDAP_TLS_CACERT_FILE
volumes:
mailserver-data:
mailserver-state:
mailserver-logs:
mailserver-config:
internal01 network and the volume of the public key are defined in another docker-compose.yml file. But this should not be an issue.
2. Make sure the container does not validate the CA in the ldap.conf file, as is seems that there is an issue here (the certificate seems to be refused even if it is valid). In simple words, execute the following on the host and this will do the job:
docker exec mailserver sh -c 'echo "TLS_REQCERT never" > /etc/ldap/ldap.conf'
What level of experience do you have with Docker and mail servers?
Trust me, I’m a (computer) engineer! [expert]
Code of conduct
- I have read this project’s Code of Conduct and I agree
- I have read the README and the documentation and I searched the issue tracker but could not find a solution
Improvements to this form?
No response
About this issue
- Original URL
- State: closed
- Created 3 years ago
- Comments: 44 (44 by maintainers)
Fair point. We should improve this. I agree that the error handling in this case is a but harsh.
@casperklein shall we emit a warning now, instead of logging a fatality?
I think Casper will address it as he worked on the previous PR that introduced this change. If not I’ll take care of it within the next day or so.
There doesn’t appear to be any reason to aggressively shutdown the container for those checks, the value is still there if using warning logs instead.
Thank you for raising the issue, it was helpful and it should be taken care of before the official release 😃
I’m not sure how you’re starting your containers, but here is the two ways to do this and avoid this problem:
docker run --rmdocker-compose up --force-recreateAnd then add any extra options you normally have. The
dockerexample will remove when container is stopped, whiledocker-composewill always recreate the containers.And if you later have something that would be a silent error because you didn’t create a fresh container?
I think it’s fair though. If other maintainers agree that it would be better to emit a warning instead. I don’t think we have a need to explicitly shut the container down for this error handling.
You’d need a compelling case for it I think. As I explained in my last comment, it can be easy to accidentally get different behaviour that we can’t workaround, so it’s better to detect that early on.
If you really want to avoid this, you can override the error handling here, just remove the content on each line from
||onwards. You can either build custom Dockerfile, or volume mount over it in the container.Your previous workflow (docker start/stop) might work, but it’s not best practice. With the new version, we introduced a stricter error checking. So these files couldn’t be deleted in the old version as well, but no error was shown.
Think of containers like the were toilet papers (should only be used once 😄). As a thumb rule, containers shouldn’t be stopped and started again.
Okay. Sorry. I didn’t know that 😞 I thought you would mistake
.envfile withmailserver.envTIL that there is another crazy way to provide container variables 😄
I am not sure if I understand you correctly, so please correct me if I did not get your point correctly.
Restarting the container (or at least the service that uses the certificate) is necessary in most cases (I am not sure what about postfix), but in general you would restart the services after renewing the certificates. That is what all other services (and docker containers) do. Actually I know no docker container that requires removing and recreating the container as a better solution than restarting. Of course if the container is stateless is good. I am just trying to say that nor restarting nor recreating should be better the other, they both should be good, imo.
@polarathene sorry I did not mean
ignoreliterally. I just meant to log them as warnings without crashing. This is gonna be the best.If it is possible please try to get it in this version, because this is directly related to a change done it it.
Thank you all for this discussion!
@polarathene @casperklein thank you for the explanation! At least I understand the whole thing now.
To be honest, I see it as a bad behavior, you don’t know what can happen with your server or container, and it is always good if your container can just restart and recover itself. I know it is good that you know exactly that something went wrong, but you may have that as a very emphasized log entry, but to just crash the container for such minor error is not something I would see, imho.
Can’t we have the ability to configure this behavior as environment variable to
ignoresuch not-really-errors (I mean to log them as warnings)?So something like this:
or
@casperklein, recycling everything is always good 😄
This can happen for example, if you use
docker-compose upand then hitting CTRL-C. This will just shutdown but not delete/destroy the container.@MohammedNoureldin
The error occurs, when you do not fully shutdown (destroy) the container. On next start, the files are already gone because they were deleted previously.
In any case, you must run
docker-compose down.In doubt, check if the container is really gone with
docker ps -a.If you see something like this:
then issue
docker-compose down. Otherwise, THIS “already used” container will be used bydocker-compose upagain.And now it works:
You can add
env_filefor a custom env file, but you’ll still find vars passed throughenvironmentoverride it.Good news: I can replicate your issue. Bad news: You probably didn’t do as adviced 😉