dolphinscheduler: [Bug] [TaskExecutionRunnable] Sub workflow task always repeat run in master-server failover
Search before asking
- I had searched in the issues and found no similar issues.
What happened
Normally, FailoverCoordinator.getFailoverWorkflowsForMaster() finds all workflows that need failover. When the sub-workflow task’s TaskExecutionRunnable.failover() method is called, takeOverTaskFromExecutor() now returns false if the task is a logic task. This results in the creation of a new sub-workflow task instance and the publication of its TaskStartLifecycleEvent, causing the sub-workflow to run again during the failover process.
so i think it would be better to check the sub-workflow instance properly(by dao or server communicate) and take it over, instead of creating a whole new task instance.
What you expected to happen
Take over sub-workflow task if the sub-workflow instance is in good status.
How to reproduce
execute a workflow with sub-workflow, restart the master-server, query the database
Anything else
No response
Version
dev
Are you willing to submit PR?
- Yes I am willing to submit a PR!
Code of Conduct
- I agree to follow this project’s Code of Conduct
About this issue
- Original URL
- State: closed
- Created 2 years ago
- Comments: 18 (14 by maintainers)
Commits related to this issue
- Fix #16767 — committed to reele/dolphinscheduler by reele 2 years ago
- Fix #16767 — committed to reele/dolphinscheduler by reele 2 years ago
- Fix #16767 — committed to reele/dolphinscheduler by reele 2 years ago
it’s indeed a simpler approach - my method impacted architecture design, i tried.
Also, during failover, the master directly starts sub-workflows, which creates a split from SubWorkflowLogicTask’s processing flow, i think it would be better to have SubWorkflowLogicTask fully handle triggering fault-tolerant sub-workflows.
@reele Hi, the subworkflow take-over logic is under
SubWorkflowLogicTask, once we failover a subworkflow task, will generate a new logic task instance, the new task instance will contains the originSubWorkflowLogicTaskRuntimeContextand then can track the origin sub workflow instance.I will test this.