salt: salt-master 2016.11.4 crashes on CentOS 7 with error "RSA key format is not supported"
Description of Issue/Question
Within the past day, package updates were applied to a CentOS 7 salt master server, upgrading salt-master and salt-minion to 2016.11.4, and installing python2-pycryptodomex (presumably as a dependency). The salt-master on this server now crashes immediately with the error “RSA key format is not supported.” No configs have been altered on this master recently. It was running 2016.11.3, and was stable for months. Now it is dead.
Setup
I copied the latest /etc/salt/master.rpmnew
config file to /etc/salt/master
and edited the file_roots section. I made no other changes. This is a very “vanilla” installation of Salt.
Steps to Reproduce Issue
Starting point: CentOS 7 host with Salt 2016.11.3 running
- Run
yum update
- The following four updates (nothing else) were applied on April 25:
Apr 25 05:49:21 Installed: python2-pycryptodomex.x86_64 3.4.3-1.el7
Apr 25 05:49:25 Updated: salt.noarch 2016.11.4-1.el7
Apr 25 05:49:25 Updated: salt-minion.noarch 2016.11.4-1.el7
Apr 25 05:49:32 Updated: salt-master.noarch 2016.11.4-1.el
- Upon restart, the salt-master service crashes with the following error message:
Apr 26 02:13:55 ueb-salt-01 salt-master[2845]: Traceback (most recent call last):
Apr 26 02:13:55 ueb-salt-01 salt-master[2845]: File "/usr/bin/salt-master", line 22, in <module>
Apr 26 02:13:55 ueb-salt-01 salt-master[2845]: salt_master()
Apr 26 02:13:55 ueb-salt-01 salt-master[2845]: File "/usr/lib/python2.7/site-packages/salt/scripts.py", line 90, in salt_master
Apr 26 02:13:55 ueb-salt-01 salt-master[2845]: master.start()
Apr 26 02:13:55 ueb-salt-01 salt-master[2845]: File "/usr/lib/python2.7/site-packages/salt/cli/daemons.py", line 204, in start
Apr 26 02:13:55 ueb-salt-01 salt-master[2845]: super(Master, self).start()
Apr 26 02:13:55 ueb-salt-01 salt-master[2845]: File "/usr/lib/python2.7/site-packages/salt/utils/parsers.py", line 947, in start
Apr 26 02:13:55 ueb-salt-01 salt-master[2845]: self.prepare()
Apr 26 02:13:55 ueb-salt-01 salt-master[2845]: File "/usr/lib/python2.7/site-packages/salt/cli/daemons.py", line 185, in prepare
Apr 26 02:13:55 ueb-salt-01 salt-master[2845]: self.master = salt.master.Master(self.config)
Apr 26 02:13:55 ueb-salt-01 salt-master[2845]: File "/usr/lib/python2.7/site-packages/salt/master.py", line 399, in __init__
Apr 26 02:13:55 ueb-salt-01 salt-master[2845]: SMaster.__init__(self, opts)
Apr 26 02:13:55 ueb-salt-01 salt-master[2845]: File "/usr/lib/python2.7/site-packages/salt/master.py", line 119, in __init__
Apr 26 02:13:55 ueb-salt-01 salt-master[2845]: self.master_key = salt.crypt.MasterKeys(self.opts)
Apr 26 02:13:55 ueb-salt-01 salt-master[2845]: File "/usr/lib/python2.7/site-packages/salt/crypt.py", line 233, in __init__
Apr 26 02:13:55 ueb-salt-01 salt-master[2845]: self.key = self.__get_keys()
Apr 26 02:13:55 ueb-salt-01 salt-master[2845]: File "/usr/lib/python2.7/site-packages/salt/crypt.py", line 285, in __get_keys
Apr 26 02:13:55 ueb-salt-01 salt-master[2845]: key = RSA.importKey(f.read())
Apr 26 02:13:55 ueb-salt-01 salt-master[2845]: File "/usr/lib64/python2.7/site-packages/Cryptodome/PublicKey/RSA.py", line 739, in import_key
Apr 26 02:13:55 ueb-salt-01 salt-master[2845]: return _import_keyDER(der, passphrase)
Apr 26 02:13:55 ueb-salt-01 salt-master[2845]: File "/usr/lib64/python2.7/site-packages/Cryptodome/PublicKey/RSA.py", line 683, in _import_keyDER
Apr 26 02:13:55 ueb-salt-01 salt-master[2845]: raise ValueError("RSA key format is not supported")
Apr 26 02:13:55 ueb-salt-01 salt-master[2845]: ValueError: RSA key format is not supported
Apr 26 02:13:55 ueb-salt-01 systemd[1]: salt-master.service: main process exited, code=exited, status=1/FAILURE
Apr 26 02:13:55 ueb-salt-01 systemd[1]: Failed to start The Salt Master Server.
-- Subject: Unit salt-master.service has failed
-- Defined-By: systemd
-- Support: http://lists.freedesktop.org/mailman/listinfo/systemd-devel
--
-- Unit salt-master.service has failed.
--
-- The result is failed.
Versions Report
Salt Version:
Salt: 2016.11.4
Dependency Versions:
cffi: Not Installed
cherrypy: Not Installed
dateutil: Not Installed
docker-py: Not Installed
gitdb: Not Installed
gitpython: Not Installed
ioflo: Not Installed
Jinja2: 2.7.2
libgit2: Not Installed
libnacl: Not Installed
M2Crypto: Not Installed
Mako: Not Installed
msgpack-pure: Not Installed
msgpack-python: 0.4.8
mysql-python: Not Installed
pycparser: Not Installed
pycrypto: 2.6.1
pycryptodome: 3.4.3
pygit2: Not Installed
Python: 2.7.5 (default, Nov 6 2016, 00:28:07)
python-gnupg: Not Installed
PyYAML: 3.11
PyZMQ: 15.3.0
RAET: Not Installed
smmap: Not Installed
timelib: Not Installed
Tornado: 4.2.1
ZMQ: 4.1.4
System Versions:
dist: centos 7.3.1611 Core
machine: x86_64
release: 3.10.0-514.16.1.el7.x86_64
system: Linux
version: CentOS Linux 7.3.1611 Core
About this issue
- Original URL
- State: closed
- Created 7 years ago
- Reactions: 3
- Comments: 24 (11 by maintainers)
Hi,
I just had a similar problem with a salt-minion on CentOS 7. For me it worked if I deinstall the package python2-pycryptodomex.x86_64. It is a install dependency of salt. So you have to deinstall it with
rpm -e --nodeps python2-pycryptodomex.x86_64
The following error was there before I deinstalled the package (the minion did run perfectly with 2016.11.3):
Just for completeness this is my versions report:
The remaining mystery is why my master key generated less than one year ago is breaking pycryptodome. I will save a copy of my key, re-key, and then do some analysis on the old master.