st2: Workflows with action policies can deadlock
Given the attached pack (useful for meditation), the system deadlocks when the action _dispatcher_pool (50 entries) gets filled with workflows, and all workflow actions are subsequently held in the _work_buffer queue indefinitely.
One possible solution would be to limit the number of workflows in the _dispatcher_pool to one less than half the total entries, in order to leave room for action execution.
Deadlock pack
This pack contains actions for testing policy deadlock and other behaviors
Actions
delay2- a two second delaydelay20- a twenty second delaydelay2test- a workflow with delay2 followed by delay20
Policies
delay20- limit concurrency to 1delay2test- limit concurrency to 10
Testing
Launch multiple instances of the workflow:
for i inseq 1 100; do st2 action execute deadlock.delay2test; done
Without the delay2test policy, the actions deadlock
About this issue
- Original URL
- State: closed
- Created 8 years ago
- Comments: 22 (21 by maintainers)
@bri365 Thanks for the feedback.
I will also wait on @lakshmi-kannan and others to chime in and if they agree with my proposed approach, I will go ahead and implement it 😃