pulsar: python functions cannot find python3 interpreter
Describe the bug python functions do not start if the system has “python3” but not “python” (which is the case for modern Ubuntu systems - “python” is python2 and is not installed by default)
To Reproduce
Write a simple function func1.py
from pulsar import Function
class FirstFunction(Function):
def process(self, item, context):
log = context.get_logger()
log.info("Got %r with properties %r" % (item, context.get_message_properties()))
Deploy it:
apache-pulsar-2.4.1/bin/pulsar-admin functions create --name womble --inputs my-topic \
--py /home/ubuntu/func1.py --classname FirstFunction --log-topic pulsar-log
(side note: have to give absolute path to function, pulsar-admin changes the working directory)
Check its status:
$ apache-pulsar-2.4.1/bin/pulsar-admin functions status --name womble
{
"numInstances" : 1,
"numRunning" : 0,
"instances" : [ {
"instanceId" : 0,
"status" : {
"running" : false,
"error" : "Cannot run program \"python\": error=2, No such file or directory",
"numRestarts" : 16,
"numReceived" : 0,
"numSuccessfullyProcessed" : 0,
"numUserExceptions" : 0,
"latestUserExceptions" : [ ],
"numSystemExceptions" : 0,
"latestSystemExceptions" : [ ],
"averageLatency" : 0.0,
"lastInvocationTime" : 0,
"workerId" : "c-standalone-fw-ldex-pulsar.int.example.net-8080"
}
} ]
}
Note error: Cannot run program “python”: error=2, No such file or directory
Expected behavior I would expect the function to deploy.
Screenshots N/A
Desktop (please complete the following information): N/A
Additional context This may be the same problem as #5369 - however in that case, it’s the shebang line in pulsar’s own python code, whereas this one is my own function which is not starting.
I cannot find it stated in Pulsar documentation whether Pulsar supports python2, python3 or both. However python2 is very shortly going to be end-of-life.
Workarounds
Install python2 and write my code in python2. I really don’t want to do that, with python2 being end-of-life in 2 months’ time.
Symlink python to python3. However in Ubuntu systems, symlinking python to python3 has the potential to cause all sorts of problems. Many applications and scripts depend on python
meaning python2
(in which case they will depend on the python-XXX
packages rather than python3-XXX
). More content here.
Ideally pulsar would try python3 and fallback to python. If pulsar were to be hard-coded to run “python3” instead of “python” that would also be fine IMO, given the impending EOL of python2.
About this issue
- Original URL
- State: closed
- Created 5 years ago
- Comments: 17 (17 by maintainers)
How about adding a setting to broker.conf / standalone.conf? e.g.
However I notice at least one place where searching in
$PATH
is currently used:So I suggest:
python_interpreter=python
$PATH
as todayIf you don’t want a conf setting, then an environment variable would work for me too (
PYTHON=/usr/bin/python3
)