plugins: mail/rspamd: Daemon sometimes doesn't start after reboot

Hi,

this is the third time rspamd didn’t start after a reboot. @fabianfrz did you experienced this too?

Seems to be something with the pid file:

2018-01-04 09:46:07 #86582(controller) rspamd_controller_on_terminate: closing rrd file: /var/db/rspamd/rspamd.rrd
2018-01-04 09:46:07 #84334(main) <ac2e90>; main; rspamd_term_handler: catch termination signal, waiting for children
2018-01-04 09:46:07 #86438(rspamd_proxy) <ac2e90>; main; rspamd_worker_term_handler: terminating after receiving signal Terminated
2018-01-04 09:46:07 #84334(main) <ac2e90>; main; wait_for_workers: hs_helper process 87334 terminated softly
2018-01-04 09:46:07 #84334(main) <ac2e90>; main; wait_for_workers: rspamd_proxy process 86438 terminated softly
2018-01-04 09:46:07 #84334(main) <ac2e90>; main; wait_for_workers: normal process 86929 terminated softly
2018-01-04 09:46:07 #84334(main) <ac2e90>; main; wait_for_workers: normal process 87155 terminated softly
2018-01-04 09:46:07 #84334(main) <ac2e90>; main; wait_for_workers: controller process 86582 terminated softly
2018-01-04 09:46:07 #84334(main) <ac2e90>; main; main: terminating...
2018-01-04 09:47:03 #41567(main) rspamd_regexp_library_init: pcre is compiled with JIT for x86 64bit (little endian + unaligned)
2018-01-04 09:47:03 #41567(main) <849ijo>; cfg; chartable_module_config: init internal chartable module
2018-01-04 09:47:03 #41567(main) <849ijo>; cfg; dkim_module_config: init internal dkim module
2018-01-04 09:47:03 #41567(main) <849ijo>; cfg; spf_module_config: init internal spf module
2018-01-04 09:47:03 #41567(main) <w4bd4g>; map; rspamd_map_add_from_ucl: added map from ucl
2018-01-04 09:47:03 #41567(main) <ttqirq>; map; rspamd_map_add_from_ucl: added map from ucl
2018-01-04 09:47:03 #41567(main) <849ijo>; cfg; surbl_module_config: init internal surbls module, 16 uribl rules loaded
2018-01-04 09:47:03 #41567(main) <849ijo>; cfg; regexp_module_config: init internal regexp module, 113 regexp rules and 0 lua rules are loaded
2018-01-04 09:47:03 #41567(main) <849ijo>; cfg; fuzzy_parse_rule: added fuzzy rule FUZZY_UNKNOWN, key: ef43ae80cc8d, shingles_key: ef43ae80cc8d, algorithm: mum
2018-01-04 09:47:03 #41567(main) <849ijo>; cfg; fuzzy_check_module_config: init internal fuzzy_check module, 1 rules loaded
2018-01-04 09:47:03 #41567(main) <849ijo>; cfg; rspamd_init_lua_filters: init lua module antivirus
2018-01-04 09:47:03 #41567(main) <849ijo>; cfg; rspamd_init_lua_filters: init lua module arc
2018-01-04 09:47:03 #41567(main) <849ijo>; cfg; rspamd_init_lua_filters: init lua module asn
2018-01-04 09:47:03 #41567(main) <849ijo>; cfg; rspamd_config_is_module_enabled: lua module bayes_expiry is enabled but has not been configured
2018-01-04 09:47:03 #41567(main) <849ijo>; cfg; rspamd_config_is_module_enabled: bayes_expiry disabling unconfigured lua module
2018-01-04 09:47:03 #41567(main) <849ijo>; lua; clickhouse.lua:639: no servers are specified, disabling module
2018-01-04 09:47:03 #41567(main) <849ijo>; cfg; rspamd_init_lua_filters: init lua module clickhouse
2018-01-04 09:47:03 #41567(main) lua; dcc.lua:142: DCC module not configured
2018-01-04 09:47:03 #41567(main) <849ijo>; cfg; rspamd_init_lua_filters: init lua module dcc
2018-01-04 09:47:03 #41567(main) <849ijo>; cfg; rspamd_init_lua_filters: init lua module dkim_signing
2018-01-04 09:47:03 #41567(main) <849ijo>; cfg; rspamd_init_lua_filters: init lua module dmarc
2018-01-04 09:47:03 #41567(main) <849ijo>; cfg; rspamd_config_is_module_enabled: lua module dynamic_conf is enabled but has not been configured
2018-01-04 09:47:03 #41567(main) <849ijo>; cfg; rspamd_config_is_module_enabled: dynamic_conf disabling unconfigured lua module
2018-01-04 09:47:03 #41567(main) <849ijo>; lua; emails.lua:174: add emails rule ebl.msbl.org
2018-01-04 09:47:03 #41567(main) <849ijo>; lua; emails.lua:174: add emails rule email.rspamd.com
2018-01-04 09:47:03 #41567(main) <849ijo>; cfg; rspamd_init_lua_filters: init lua module emails
2018-01-04 09:47:03 #41567(main) <849ijo>; cfg; rspamd_config_is_module_enabled: lua module fann_classifier is enabled but has not been configured
2018-01-04 09:47:03 #41567(main) <849ijo>; cfg; rspamd_config_is_module_enabled: fann_classifier disabling unconfigured lua module
2018-01-04 09:47:03 #41567(main) <849ijo>; lua; fann_redis.lua:864: Module is unconfigured
2018-01-04 09:47:03 #41567(main) <849ijo>; cfg; rspamd_init_lua_filters: init lua module fann_redis
2018-01-04 09:47:03 #41567(main) <849ijo>; cfg; rspamd_config_is_module_enabled: lua module fann_scores is enabled but has not been configured
2018-01-04 09:47:03 #41567(main) <849ijo>; cfg; rspamd_config_is_module_enabled: fann_scores disabling unconfigured lua module
2018-01-04 09:47:03 #41567(main) <849ijo>; cfg; rspamd_init_lua_filters: init lua module force_actions
2018-01-04 09:47:03 #41567(main) <849ijo>; cfg; rspamd_init_lua_filters: init lua module forged_recipients
2018-01-04 09:47:03 #41567(main) <849ijo>; cfg; rspamd_config_is_module_enabled: lua module fuzzy_collect is enabled but has not been configured
2018-01-04 09:47:03 #41567(main) <849ijo>; cfg; rspamd_config_is_module_enabled: fuzzy_collect disabling unconfigured lua module
2018-01-04 09:47:03 #41567(main) <849ijo>; cfg; rspamd_map_parse_backend: cannot open file '/usr/local/etc/rspamd/local.d/greylist-whitelist-domains.inc': Permission denied
2018-01-04 09:47:03 #41567(main) <849ijo>; cfg; rspamd_map_add_from_ucl: map has no urls to be loaded: empty list
2018-01-04 09:47:03 #41567(main) <849ijo>; lua; greylist.lua:428: no servers are specified, disabling module
2018-01-04 09:47:03 #41567(main) <849ijo>; cfg; rspamd_init_lua_filters: init lua module greylist
2018-01-04 09:47:03 #41567(main) <4dego7>; map; rspamd_map_add_from_ucl: added map from ucl
2018-01-04 09:47:03 #41567(main) <4dego7>; map; rspamd_map_add_from_ucl: added map from ucl
2018-01-04 09:47:03 #41567(main) <4dego7>; map; rspamd_map_add_from_ucl: added map from ucl
2018-01-04 09:47:03 #41567(main) <4dego7>; map; rspamd_map_add_from_ucl: added map from ucl
2018-01-04 09:47:03 #41567(main) <849ijo>; cfg; rspamd_init_lua_filters: init lua module hfilter
2018-01-04 09:47:03 #41567(main) <849ijo>; lua; history_redis.lua:218: no servers are specified, disabling module
2018-01-04 09:47:03 #41567(main) <849ijo>; cfg; rspamd_init_lua_filters: init lua module history_redis
2018-01-04 09:47:03 #41567(main) <849ijo>; lua; ip_score.lua:359: no servers are specified
2018-01-04 09:47:03 #41567(main) <849ijo>; cfg; rspamd_init_lua_filters: init lua module ip_score
2018-01-04 09:47:03 #41567(main) <849ijo>; cfg; rspamd_init_lua_filters: init lua module maillist
2018-01-04 09:47:03 #41567(main) <849ijo>; cfg; rspamd_init_lua_filters: init lua module metadata_exporter
2018-01-04 09:47:03 #41567(main) <849ijo>; cfg; rspamd_init_lua_filters: init lua module metric_exporter
2018-01-04 09:47:03 #41567(main) <849ijo>; cfg; rspamd_map_parse_backend: cannot open file '/usr/local/etc/rspamd/local.d/mid.inc': Permission denied
2018-01-04 09:47:03 #41567(main) <x4rkzq>; map; rspamd_map_add_from_ucl: added map from ucl
2018-01-04 09:47:03 #41567(main) <849ijo>; cfg; rspamd_init_lua_filters: init lua module mid
2018-01-04 09:47:03 #41567(main) <849ijo>; cfg; rspamd_init_lua_filters: init lua module milter_headers
2018-01-04 09:47:03 #41567(main) <849ijo>; cfg; rspamd_map_parse_backend: map '/var/db/rspamd/mime_types.inc.local' is not found, but it can be loaded automatically later
2018-01-04 09:47:03 #41567(main) <g9tfd6>; map; rspamd_map_add_from_ucl: added map from ucl
2018-01-04 09:47:03 #41567(main) <849ijo>; cfg; rspamd_init_lua_filters: init lua module mime_types
2018-01-04 09:47:03 #41567(main) <qwxppg>; map; rspamd_map_add: added map https://rspamd.com/freemail/free.txt.zst
2018-01-04 09:47:03 #41567(main) <qwxppg>; map; rspamd_map_add: added map https://rspamd.com/freemail/free.txt.zst
2018-01-04 09:47:03 #41567(main) <xa6qqy>; map; rspamd_map_add: added map https://rspamd.com/freemail/disposable.txt.zst
2018-01-04 09:47:03 #41567(main) <849ijo>; lua; multimap.lua:860: reuse url for DISPOSABLE_CC: "https://rspamd.com/freemail/disposable.txt.zst"
2018-01-04 09:47:03 #41567(main) <849ijo>; lua; multimap.lua:860: reuse url for DISPOSABLE_TO: "https://rspamd.com/freemail/disposable.txt.zst"
2018-01-04 09:47:03 #41567(main) <849ijo>; lua; multimap.lua:860: reuse url for DISPOSABLE_FROM: "https://rspamd.com/freemail/disposable.txt.zst"
2018-01-04 09:47:03 #41567(main) <849ijo>; lua; multimap.lua:860: reuse url for FREEMAIL_REPLYTO: "https://rspamd.com/freemail/free.txt.zst"
2018-01-04 09:47:03 #41567(main) <qwxppg>; map; rspamd_map_add: added map https://rspamd.com/freemail/free.txt.zst
2018-01-04 09:47:03 #41567(main) <qwxppg>; map; rspamd_map_add: added map https://rspamd.com/freemail/free.txt.zst
2018-01-04 09:47:03 #41567(main) <xa6qqy>; map; rspamd_map_add: added map https://rspamd.com/freemail/disposable.txt.zst
2018-01-04 09:47:03 #41567(main) <849ijo>; lua; multimap.lua:860: reuse url for FREEMAIL_CC: "https://rspamd.com/freemail/free.txt.zst"
2018-01-04 09:47:03 #41567(main) <xa6qqy>; map; rspamd_map_add: added map https://rspamd.com/freemail/disposable.txt.zst
2018-01-04 09:47:03 #41567(main) <849ijo>; cfg; rspamd_init_lua_filters: init lua module multimap
2018-01-04 09:47:03 #41567(main) <849ijo>; cfg; rspamd_config_is_module_enabled: lua module mx_check is disabled in the configuration
2018-01-04 09:47:03 #41567(main) <849ijo>; cfg; rspamd_init_lua_filters: init lua module once_received
2018-01-04 09:47:03 #41567(main) <uy18in>; map; rspamd_map_add: added map /usr/local/etc/rspamd/redirectors.inc
2018-01-04 09:47:03 #41567(main) <849ijo>; cfg; rspamd_map_parse_backend: cannot open file '/usr/local/etc/rspamd/local.d/redirectors.inc': Permission denied
2018-01-04 09:47:03 #41567(main) <849ijo>; lua; phishing.lua:239: cannot add map: /usr/local/etc/rspamd/local.d/redirectors.inc for symbol: LOCAL_REDIRECTOR_FALSE
2018-01-04 09:47:03 #41567(main) <849ijo>; cfg; rspamd_init_lua_filters: init lua module phishing
2018-01-04 09:47:03 #41567(main) <849ijo>; lua; ratelimit.lua:628: enabled rate buckets: []
2018-01-04 09:47:03 #41567(main) <849ijo>; lua; ratelimit.lua:681: no servers are specified, disabling module
2018-01-04 09:47:03 #41567(main) <849ijo>; cfg; rspamd_init_lua_filters: init lua module ratelimit
2018-01-04 09:47:03 #41567(main) <849ijo>; cfg; rspamd_init_lua_filters: init lua module rbl
2018-01-04 09:47:03 #41567(main) <849ijo>; lua; replies.lua:127: no servers are specified, disabling module
2018-01-04 09:47:03 #41567(main) <849ijo>; cfg; rspamd_init_lua_filters: init lua module replies
2018-01-04 09:47:03 #41567(main) <849ijo>; cfg; rspamd_init_lua_filters: init lua module rspamd_update
2018-01-04 09:47:03 #41567(main) <849ijo>; cfg; rspamd_init_lua_filters: init lua module settings
2018-01-04 09:47:03 #41567(main) <849ijo>; cfg; rspamd_init_lua_filters: init lua module spamassassin
2018-01-04 09:47:03 #41567(main) <849ijo>; cfg; rspamd_config_is_module_enabled: lua module spamtrap is disabled in the configuration
2018-01-04 09:47:03 #41567(main) <849ijo>; lua; trie.lua:165: no tries defined
2018-01-04 09:47:03 #41567(main) <849ijo>; cfg; rspamd_init_lua_filters: init lua module trie
2018-01-04 09:47:03 #41567(main) <849ijo>; lua; url_redirector.lua:273: no servers are specified, disabling module
2018-01-04 09:47:03 #41567(main) <849ijo>; cfg; rspamd_init_lua_filters: init lua module url_redirector
2018-01-04 09:47:03 #41567(main) <849ijo>; cfg; rspamd_config_is_module_enabled: lua module url_reputation is disabled in the configuration
2018-01-04 09:47:03 #41567(main) <849ijo>; cfg; rspamd_config_is_module_enabled: lua module url_tags is disabled in the configuration
2018-01-04 09:47:03 #41567(main) <849ijo>; cfg; rspamd_map_parse_backend: map '/var/db/rspamd/dmarc_whitelist.inc.local' is not found, but it can be loaded automatically later
2018-01-04 09:47:03 #41567(main) <87rxcr>; map; rspamd_map_add_from_ucl: added map from ucl
2018-01-04 09:47:03 #41567(main) <849ijo>; cfg; rspamd_map_parse_backend: map '/var/db/rspamd/spf_dkim_whitelist.inc.local' is not found, but it can be loaded automatically later
2018-01-04 09:47:03 #41567(main) <4b9dnq>; map; rspamd_map_add_from_ucl: added map from ucl
2018-01-04 09:47:03 #41567(main) <849ijo>; cfg; rspamd_map_parse_backend: map '/usr/local/etc/rspamd/dkim_whitelist.inc' is not found, but it can be loaded automatically later
2018-01-04 09:47:03 #41567(main) <849ijo>; cfg; rspamd_map_parse_backend: map '/var/db/rspamd/dkim_whitelist.inc.local' is not found, but it can be loaded automatically later
2018-01-04 09:47:03 #41567(main) <xqwnbp>; map; rspamd_map_add_from_ucl: added map from ucl
2018-01-04 09:47:03 #41567(main) <849ijo>; cfg; rspamd_map_parse_backend: map '/usr/local/etc/rspamd/spf_whitelist.inc' is not found, but it can be loaded automatically later
2018-01-04 09:47:03 #41567(main) <849ijo>; cfg; rspamd_map_parse_backend: map '/var/db/rspamd/spf_whitelist.inc.local' is not found, but it can be loaded automatically later
2018-01-04 09:47:03 #41567(main) <iwso89>; map; rspamd_map_add_from_ucl: added map from ucl
2018-01-04 09:47:03 #41567(main) <849ijo>; cfg; rspamd_init_lua_filters: init lua module whitelist
2018-01-04 09:47:03 #41567(main) <849ijo>; cfg; rspamd_map_parse_backend: map '/var/db/rspamd/rspamd_dynamic' is not found, but it can be loaded automatically later
2018-01-04 09:47:03 #41567(main) <ykjbdn>; map; rspamd_map_add: added map /var/db/rspamd/rspamd_dynamic
2018-01-04 09:47:03 #41567(main) <5462bf>; re_cache; rspamd_re_cache_init: loaded hyperscan engine witch cpu tune 'generic' and features ''
2018-01-04 09:47:03 #41567(main) <1b9028>; main; main: rspamd 1.6.5 is starting, build id: release
2018-01-04 09:47:03 #41567(main) <1b9028>; main; main: cpu features: avx, sse2, sse3, ssse3, sse4.1, sse4.2, rdrand
2018-01-04 09:47:03 #41567(main) <1b9028>; main; main: cryptobox configuration: curve25519(avx), chacha20(avx), poly1305(avx), siphash(generic), blake2(avx), base64(ref)
2018-01-04 09:47:03 #41567(main) <1b9028>; main; main: cannot write pid file /var/run/rspamd/rspamd.pid

About this issue

Original URL
State: closed
Created 6 years ago
Comments: 34 (28 by maintainers)

Commits related to this issue

HBSD+OPNsense: Set hardening.pax.disallow_map32bit.status=1 by default LuaJIT creates 32-bit memory maps in a 64-bit execution environment, which is disabled by default in HardenedBSD. We never porte... — committed to opnsense/src by lattera 6 years ago
HBSD+OPNsense: Set hardening.pax.disallow_map32bit.status=1 by default LuaJIT creates 32-bit memory maps in a 64-bit execution environment, which is disabled by default in HardenedBSD. We never porte... — committed to opnsense/src by lattera 6 years ago
rc: add workaround for rspamd clashing with 32 bit mmap disallow PR: https://github.com/opnsense/plugins/issues/466 — committed to opnsense/core by fichtner 6 years ago
rc: add workaround for rspamd clashing with 32 bit mmap disallow PR: https://github.com/opnsense/plugins/issues/466 (cherry picked from commit 90d8ef6cf2fa3a7fd6a2e80982854a119549159a) — committed to opnsense/core by fichtner 6 years ago
HBSD+OPNsense: Set hardening.pax.disallow_map32bit.status=1 by default LuaJIT creates 32-bit memory maps in a 64-bit execution environment, which is disabled by default in HardenedBSD. We never porte... — committed to opnsense/src by lattera 6 years ago
HBSD+OPNsense: Set hardening.pax.disallow_map32bit.status=1 by default LuaJIT creates 32-bit memory maps in a 64-bit execution environment, which is disabled by default in HardenedBSD. We never porte... — committed to opnsense/src by lattera 6 years ago

Most upvoted comments

I found the issue: rspamd calls mmap(MAP_32BIT), which was allowed in the 17.x kernel, but disallowed now in the 18.1 kernel. The output from truss is here: https://gist.github.com/135d5b5420ddca3966180bb2061f4394

Until I come up with a permanent fix, I’m going to drop the feature that disallows calls to mmap with the MAP_32BIT flag set. From the perspective of 17.x to 18.x, we’re not losing anything by dropping the feature.

This will give me a good excuse to finally integrate exploit mitigation toggling directly into pkg. Note that this particular feature (the pkg integration) will take a while to implement (maybe for 18.7).

lattera on Jan 12, 2018