salt: 2018.3 regression - unable to create Caller in a runner
Description of Issue/Question
Runners which create salt.client.Caller throw exception listed below. This stops us from upgrading to 2018.3 release.
Steps to Reproduce Issue
On a CentOS 7.4 minimally installed VM, do the following -
# curl -o bootstrap-salt.sh -L https://bootstrap.saltstack.com
# sudo sh bootstrap-salt.sh -M git 2018.3
Make sure that following returns true -
[root@base2 ~]# salt-call test.ping
local:
True
Now create a test runner with the following - cat /srv/_runners/test_runner.py
# -*- coding: utf-8 -*-
'''
test_runner.py - implementation of test runner api
Custom salt runner module
'''
import logging
import salt.client
__virtualname__ = 'test_runner'
#pylint: disable=bad-indentation, line-too-long, invalid-name
def __virtual__():
return __virtualname__
def _get_logger():
'''
Get current logger.
'''
logger = logging.getLogger(__name__)
return logger
def ping():
'''
Verify presence and installation of the module
CLI Example:
salt-run test_runner.ping
'''
local = salt.client.Caller()
_get_logger().info('%s pinged', __name__)
return True
Running (after configuring runners etc.) the salt-run test_runner.ping returns the following exception -
2018-04-05 12:41:06,259 [tornado.application:611 ][ERROR ][9823] Exception in callback <functools.partial object at 0x2800ec0>
Traceback (most recent call last):
File "/usr/lib64/python2.7/site-packages/tornado/ioloop.py", line 591, in _run_callback
ret = callback()
File "/usr/lib64/python2.7/site-packages/tornado/stack_context.py", line 342, in wrapped
raise_exc_info(exc)
File "/usr/lib64/python2.7/site-packages/tornado/stack_context.py", line 313, in wrapped
ret = fn(*args, **kwargs)
File "/usr/lib64/python2.7/site-packages/tornado/ioloop.py", line 597, in <lambda>
self.add_future(ret, lambda f: f.result())
File "/usr/lib64/python2.7/site-packages/tornado/concurrent.py", line 214, in result
raise_exc_info(self._exc_info)
File "/usr/lib64/python2.7/site-packages/tornado/gen.py", line 230, in wrapper
yielded = next(result)
File "/usr/lib/python2.7/site-packages/salt/crypt.py", line 595, in _authenticate
io_loop=self.io_loop)
File "/usr/lib/python2.7/site-packages/salt/transport/client.py", line 108, in factory
return salt.transport.zeromq.AsyncZeroMQReqChannel(opts, **kwargs)
File "/usr/lib/python2.7/site-packages/salt/transport/zeromq.py", line 133, in __new__
obj.__singleton_init__(opts, **kwargs)
File "/usr/lib/python2.7/site-packages/salt/transport/zeromq.py", line 197, in __singleton_init__
kwargs={'io_loop': self._io_loop})
File "/usr/lib/python2.7/site-packages/salt/transport/zeromq.py", line 927, in __init__
super(AsyncReqMessageClientPool, self).__init__(AsyncReqMessageClient, opts, args=args, kwargs=kwargs)
File "/usr/lib/python2.7/site-packages/salt/transport/__init__.py", line 71, in __init__
self.message_clients = [tgt(*args, **kwargs) for _ in range(sock_pool_size)]
File "/usr/lib/python2.7/site-packages/salt/transport/zeromq.py", line 974, in __init__
self._init_socket()
File "/usr/lib/python2.7/site-packages/salt/transport/zeromq.py", line 1028, in _init_socket
self.stream = zmq.eventloop.zmqstream.ZMQStream(self.socket, io_loop=self.io_loop)
File "/usr/lib64/python2.7/site-packages/zmq/eventloop/zmqstream.py", line 114, in __init__
self._init_io_state()
File "/usr/lib64/python2.7/site-packages/zmq/eventloop/zmqstream.py", line 535, in _init_io_state
self.io_loop.add_handler(self.socket, self._handle_events, self._state)
File "/usr/lib64/python2.7/site-packages/tornado/ioloop.py", line 703, in add_handler
self._impl.register(fd, events | self.ERROR)
TypeError: argument must be an int, or have a fileno() method.
If I manually update the pyzmq using pip to 17.0.0 version, this exception disappears.
Observation - By default repo.saltstack.com is supplying python-zmq version 15.3, should this be updated?
Versions Report
Salt Version:
Salt: 2018.3.0-n/a-a064a3e
Dependency Versions:
cffi: Not Installed
cherrypy: Not Installed
dateutil: Not Installed
docker-py: Not Installed
gitdb: Not Installed
gitpython: Not Installed
ioflo: Not Installed
Jinja2: 2.7.2
libgit2: Not Installed
libnacl: Not Installed
M2Crypto: Not Installed
Mako: Not Installed
msgpack-pure: Not Installed
msgpack-python: 0.5.1
mysql-python: Not Installed
pycparser: Not Installed
pycrypto: 2.6.1
pycryptodome: Not Installed
pygit2: Not Installed
Python: 2.7.5 (default, Aug 4 2017, 00:39:18)
python-gnupg: Not Installed
PyYAML: 3.11
PyZMQ: 15.3.0
RAET: Not Installed
smmap: Not Installed
timelib: Not Installed
Tornado: 4.2.1
ZMQ: 4.1.4
System Versions:
dist: centos 7.4.1708 Core
locale: UTF-8
machine: x86_64
release: 3.10.0-693.17.1.el7.x86_64
system: Linux
version: CentOS Linux 7.4.1708 Core
About this issue
- Original URL
- State: closed
- Created 6 years ago
- Comments: 34 (30 by maintainers)
@akbhat - for what it’s worth, I used pip to upgrade pyzmq to version 17 (as recommended by @isbm on April 12), and found this to be a suitable workaround for the time being:
Yuck… Sorry for this, guys. It was pretty heavily tested with 14 and then 17. Part of this because at SUSE we had 14 and then upgraded it to the latest in upcoming SLE 15 and Leap. And so I had naїve trust to ZMQ’s claim it is different only after 16, but till 16 it is same as 13. But seems I have shouldn’t trust that… Also strange that this popped up only now. No Beta testers around?
If you are unable to wait, just upgrade your PyZMQ to 17 and libzmq (or upgrade to Leap over CentOS, then it will work flawlessly out of the box — Salt is default there now, hehe 😛 ). It has anyway quite dramatic bugfixes to ZMQ you want to have.
Jokes aside, in the meanwhile I will try to fix this for you.