doit: 'get_var' fails while the multiprocess execution if any task uses delayed creation.
@schettino72 , hey! ๐
So, i found strange behaviour while trying to implement some stuff.
#! /usr/bin/doit -f
# -*- coding: utf-8 -*-
from doit import get_var
from doit import create_after
get_var( 'A', None )
def task_foo() :
return {
'actions': [ 'echo foo' ],
'task_dep': [ 'bar' ],
}
@create_after( executed = 'baz' )
def task_bar() :
for i in range( 10 ) :
yield {
'name': 'bar_{}'.format( i ),
'actions': [ 'echo bar_{}'.format( i ) ]
}
def task_baz() :
for i in range( 10 ) :
yield {
'name': 'baz_{}'.format( i ),
'actions': [ 'echo baz_{}'.format( i ) ]
}
This code works fine while being executed in single-process mode. But it fails in multi process mode.
/root>GetVar.py -n 2
Traceback (most recent call last):
File "<string>", line 1, in <module>
File "/py3.4/lib/multiprocessing/spawn.py", line 106, in spawn_main
exitcode = _main(fd)
File "/py3.4/lib/multiprocessing/spawn.py", line 116, in _main
self = pickle.load(from_parent)
File "/root/GetVar.py", line 7, in <module>
get_var( 'A', None )
File "/py3.4/lib/site-packages/doit/doit_cmd.py", line 35, in get_var
return _CMDLINE_VARS.get(name, default)
AttributeError: 'NoneType' object has no attribute 'get'
Traceback (most recent call last):
File "<string>", line 1, in <module>
File "/py3.4/lib/multiprocessing/spawn.py", line 106, in spawn_main
exitcode = _main(fd)
File "/py3.4/lib/multiprocessing/spawn.py", line 116, in _main
self = pickle.load(from_parent)
File "/root/GetVar.py", line 7, in <module>
get_var( 'A', None )
File "/py3.4/lib/site-packages/doit/doit_cmd.py", line 35, in get_var
return _CMDLINE_VARS.get(name, default)
AttributeError: 'NoneType' object has no attribute 'get'
But if i remove the create_after
decorator, all became works fine even in multi process mode. So, i found it ambiguous. I had seen no notifications in docs about such behaviour.
Can you confirm this is the doit error or not? Currently i use the last (0.30.0) version of doit.
About this issue
- Original URL
- State: closed
- Created 7 years ago
- Comments: 15 (9 by maintainers)
Commits related to this issue
- fix #164, add to CHANGES and extra source code comment. — committed to pydoit/doit by schettino72 6 years ago
Ah, yes, that makes sense, thanks ๐ See the PR. Not sure if this is the optimal way to test itโฆ
The OPโs test file works now with multiple processes.