argo-workflows: Dependencies and Enhanced Depends is broken if you call a task 'split'

Pre-requisites

  • I have double-checked my configuration
  • I can confirm the issues exists when I tested with :latest
  • I’d like to contribute the fix myself (see contributing guide)

What happened/what you expected to happen?

If you happen to call a task split, you’ll break anything that depends on it.

This is broken:

apiVersion: argoproj.io/v1alpha1
kind: Workflow
metadata:
  generateName: broken-
spec:
  entrypoint: main
  templates:
    - name: main
      dag:
        tasks:
          - name: split
            template: foo
          - name: map
            template: foo
            depends: split

    - name: foo
      container:
        image: alpine
        command:
          - sh
          - -c
          - |
            echo "foo"

This is the same workflow, but with split renamed to bar. It works fine:

apiVersion: argoproj.io/v1alpha1
kind: Workflow
metadata:
  generateName: working-
spec:
  entrypoint: main
  templates:
    - name: main
      dag:
        tasks:
          - name: bar
            template: foo
          - name: map
            template: foo
            depends: bar

    - name: foo
      container:
        image: alpine
        command:
          - sh
          - -c
          - |
            echo "foo"

The resulting error in the UI is

unable to evaluate expression '(split.Succeeded || split.Skipped || split.Daemoned)': unable to evaluate expression '(split.Succeeded || split.Skipped || split.Daemoned)': type func(...interface {}) (interface {}, error)[string] is undefined (1:8) | (split.Succeeded || split.Skipped || split.Daemoned) | .......^

This used to work on 3.4.11 and earlier. I’m even convinced it worked on both 3.5.0 release candidates.

Version

v3.5.0

Paste a small workflow that reproduces the issue. We must be able to run the workflow; don’t enter a workflows that uses private images.

See above

Logs from the workflow controller

time="2023-10-19T07:11:28.513Z" level=warning msg="was unable to obtain the node for wat-fpdsp-2364469670, taskName map"                                                                                  time="2023-10-19T07:11:28.513Z" level=info msg="node wat-fpdsp-4039980688 phase Running -> Succeeded" namespace=ci workflow=wat-fpdsp
time="2023-10-19T07:11:28.513Z" level=info msg="node wat-fpdsp-4039980688 finished: 2023-10-19 07:11:28.513582048 +0000 UTC" namespace=ci workflow=wat-fpdsp                                              time="2023-10-19T07:11:28.513Z" level=warning msg="was unable to obtain the node for wat-fpdsp-2364469670, taskName map"
time="2023-10-19T07:11:28.513Z" level=info msg="Skipped node wat-fpdsp-2364469670 initialized Error (message: unable to evaluate expression '(split.Succeeded || split.Skipped || split.Daemoned)': unabletime="2023-10-19T07:11:28.513Z" level=info msg="Outbound nodes of wat-fpdsp-2263525054 set to [wat-fpdsp-2364469670]" namespace=ci workflow=wat-fpdsp
time="2023-10-19T07:11:28.513Z" level=info msg="node wat-fpdsp-2263525054 phase Running -> Error" namespace=ci workflow=wat-fpdsp                                                                         time="2023-10-19T07:11:28.513Z" level=info msg="node wat-fpdsp-2263525054 finished: 2023-10-19 07:11:28.513781073 +0000 UTC" namespace=ci workflow=wat-fpdsp
time="2023-10-19T07:11:28.513Z" level=info msg="Retry Policy: OnError (onFailed: false, onError true)" namespace=ci workflow=wat-fpdsp                                                                    time="2023-10-19T07:11:28.513Z" level=info msg="1 child nodes of wat-fpdsp failed. Trying again..." namespace=ci workflow=wat-fpdsp
time="2023-10-19T07:11:28.513Z" level=info msg="was unable to obtain node for , letting display name to be nodeName" namespace=ci workflow=wat-fpdsp                                                      time="2023-10-19T07:11:28.513Z" level=info msg="DAG node wat-fpdsp-2733151291 initialized Running" namespace=ci workflow=wat-fpdsp
time="2023-10-19T07:11:28.513Z" level=warning msg="was unable to obtain the node for wat-fpdsp-2348667151, taskName map"                                                                                  time="2023-10-19T07:11:28.513Z" level=warning msg="was unable to obtain the node for wat-fpdsp-942064205, taskName split"
time="2023-10-19T07:11:28.514Z" level=warning msg="was unable to obtain the node for wat-fpdsp-942064205, taskName split"                                                                                 time="2023-10-19T07:11:28.514Z" level=info msg="All of node wat-fpdsp(1).split dependencies [] completed" namespace=ci workflow=wat-fpdsp
time="2023-10-19T07:11:28.514Z" level=warning msg="Node was nil, will be initialized as type Skipped" namespace=ci workflow=wat-fpdsp                                                                     time="2023-10-19T07:11:28.514Z" level=info msg="Retry node wat-fpdsp-942064205 initialized Running" namespace=ci workflow=wat-fpdsp
time="2023-10-19T07:11:28.514Z" level=info msg="Pod node wat-fpdsp-2693710620 initialized Pending" namespace=ci workflow=wat-fpdsp                                                                        time="2023-10-19T07:11:28.537Z" level=info msg="Created pod: wat-fpdsp(1).split(0) (wat-fpdsp-foo-2693710620)" namespace=ci workflow=wat-fpdsp
time="2023-10-19T07:11:28.537Z" level=warning msg="was unable to obtain the node for wat-fpdsp-2348667151, taskName map"                                                                                  time="2023-10-19T07:11:28.537Z" level=warning msg="was unable to obtain the node for wat-fpdsp-2348667151, taskName map"
time="2023-10-19T07:11:28.537Z" level=info msg="TaskSet Reconciliation" namespace=ci workflow=wat-fpdsp                                                                                                   time="2023-10-19T07:11:28.537Z" level=info msg=reconcileAgentPod namespace=ci workflow=wat-fpdsp
time="2023-10-19T07:11:28.537Z" level=info msg="Workflow to be dehydrated" Workflow Size=3285                                                                                                             time="2023-10-19T07:11:28.575Z" level=info msg="Workflow update successful" namespace=ci phase=Running resourceVersion=175536869 workflow=wat-fpdsp
time="2023-10-19T07:11:31.534Z" level=info msg="Processing workflow" namespace=ci workflow=wat-fpdsp                                                                                                      time="2023-10-19T07:11:31.535Z" level=info msg="Task-result reconciliation" namespace=ci numObjs=0 workflow=wat-fpdsp
time="2023-10-19T07:11:31.535Z" level=info msg="node changed" namespace=ci new.message=PodInitializing new.phase=Pending new.progress=0/1 nodeID=wat-fpdsp-2693710620 old.message= old.phase=Pending old.ptime="2023-10-19T07:11:31.535Z" level=info msg="node unchanged" namespace=ci nodeID=wat-fpdsp-2072654003 workflow=wat-fpdsp
time="2023-10-19T07:11:31.535Z" level=warning msg="was unable to obtain the node for wat-fpdsp-2348667151, taskName map"                                                                                  time="2023-10-19T07:11:31.535Z" level=warning msg="was unable to obtain the node for wat-fpdsp-2348667151, taskName map"
time="2023-10-19T07:11:31.535Z" level=warning msg="was unable to obtain the node for wat-fpdsp-2348667151, taskName map"                                                                                  time="2023-10-19T07:11:31.535Z" level=info msg="TaskSet Reconciliation" namespace=ci workflow=wat-fpdsp
time="2023-10-19T07:11:31.535Z" level=info msg=reconcileAgentPod namespace=ci workflow=wat-fpdsp                                                                                                          time="2023-10-19T07:11:31.535Z" level=info msg="Workflow to be dehydrated" Workflow Size=3343
time="2023-10-19T07:11:31.555Z" level=info msg="Workflow update successful" namespace=ci phase=Running resourceVersion=175536902 workflow=wat-fpdsp                                                       time="2023-10-19T07:11:34.536Z" level=info msg="Processing workflow" namespace=ci workflow=wat-fpdsp
time="2023-10-19T07:11:34.537Z" level=info msg="Task-result reconciliation" namespace=ci numObjs=0 workflow=wat-fpdsp                                                                                     time="2023-10-19T07:11:34.537Z" level=info msg="node changed" namespace=ci new.message= new.phase=Running new.progress=0/1 nodeID=wat-fpdsp-2693710620 old.message=PodInitializing old.phase=Pending old.p
time="2023-10-19T07:11:34.537Z" level=info msg="node unchanged" namespace=ci nodeID=wat-fpdsp-2072654003 workflow=wat-fpdsp                                                                               time="2023-10-19T07:11:34.538Z" level=warning msg="was unable to obtain the node for wat-fpdsp-2348667151, taskName map"
time="2023-10-19T07:11:34.538Z" level=warning msg="was unable to obtain the node for wat-fpdsp-2348667151, taskName map"                                                                                  time="2023-10-19T07:11:34.538Z" level=warning msg="was unable to obtain the node for wat-fpdsp-2348667151, taskName map"
time="2023-10-19T07:11:34.538Z" level=info msg="TaskSet Reconciliation" namespace=ci workflow=wat-fpdsp                                                                                                   time="2023-10-19T07:11:34.538Z" level=info msg=reconcileAgentPod namespace=ci workflow=wat-fpdsp
time="2023-10-19T07:11:34.538Z" level=info msg="Workflow to be dehydrated" Workflow Size=3332                                                                                                             time="2023-10-19T07:11:34.543Z" level=info msg="cleaning up pod" action=terminateContainers key=ci/wat-fpdsp-foo-2693710620/terminateContainers
time="2023-10-19T07:11:34.553Z" level=info msg="Workflow update successful" namespace=ci phase=Running resourceVersion=175536924 workflow=wat-fpdsp                                                       time="2023-10-19T07:11:37.619Z" level=info msg="Processing workflow" namespace=ci workflow=wat-fpdsp
time="2023-10-19T07:11:37.619Z" level=info msg="Task-result reconciliation" namespace=ci numObjs=0 workflow=wat-fpdsp                                                                                     time="2023-10-19T07:11:37.620Z" level=info msg="node changed" namespace=ci new.message= new.phase=Succeeded new.progress=0/1 nodeID=wat-fpdsp-2693710620 old.message= old.phase=Running old.progress=0/1 w
time="2023-10-19T07:11:37.620Z" level=info msg="node unchanged" namespace=ci nodeID=wat-fpdsp-2072654003 workflow=wat-fpdsp                                                                               time="2023-10-19T07:11:37.620Z" level=warning msg="was unable to obtain the node for wat-fpdsp-2348667151, taskName map"
time="2023-10-19T07:11:37.620Z" level=info msg="node wat-fpdsp-942064205 phase Running -> Succeeded" namespace=ci workflow=wat-fpdsp                                                                      time="2023-10-19T07:11:37.620Z" level=info msg="node wat-fpdsp-942064205 finished: 2023-10-19 07:11:37.62036697 +0000 UTC" namespace=ci workflow=wat-fpdsp
time="2023-10-19T07:11:37.620Z" level=warning msg="was unable to obtain the node for wat-fpdsp-2348667151, taskName map"                                                                                  time="2023-10-19T07:11:37.620Z" level=info msg="Skipped node wat-fpdsp-2348667151 initialized Error (message: unable to evaluate expression '(split.Succeeded || split.Skipped || split.Daemoned)': unable
time="2023-10-19T07:11:37.620Z" level=info msg="Outbound nodes of wat-fpdsp-2733151291 set to [wat-fpdsp-2348667151]" namespace=ci workflow=wat-fpdsp                                                     time="2023-10-19T07:11:37.620Z" level=info msg="node wat-fpdsp-2733151291 phase Running -> Error" namespace=ci workflow=wat-fpdsp
time="2023-10-19T07:11:37.620Z" level=info msg="node wat-fpdsp-2733151291 finished: 2023-10-19 07:11:37.620538442 +0000 UTC" namespace=ci workflow=wat-fpdsp                                              time="2023-10-19T07:11:37.620Z" level=info msg="Retry Policy: OnError (onFailed: false, onError true)" namespace=ci workflow=wat-fpdsp
time="2023-10-19T07:11:37.620Z" level=info msg="2 child nodes of wat-fpdsp failed. Trying again..." namespace=ci workflow=wat-fpdsp                                                                       time="2023-10-19T07:11:37.620Z" level=info msg="was unable to obtain node for , letting display name to be nodeName" namespace=ci workflow=wat-fpdsp
time="2023-10-19T07:11:37.620Z" level=info msg="DAG node wat-fpdsp-49276704 initialized Running" namespace=ci workflow=wat-fpdsp                                                                          time="2023-10-19T07:11:37.620Z" level=warning msg="was unable to obtain the node for wat-fpdsp-911175476, taskName map"
time="2023-10-19T07:11:37.620Z" level=warning msg="was unable to obtain the node for wat-fpdsp-4145173754, taskName split"                                                                                time="2023-10-19T07:11:37.620Z" level=warning msg="was unable to obtain the node for wat-fpdsp-4145173754, taskName split"
time="2023-10-19T07:11:37.620Z" level=info msg="All of node wat-fpdsp(2).split dependencies [] completed" namespace=ci workflow=wat-fpdsp                                                                 time="2023-10-19T07:11:37.620Z" level=warning msg="Node was nil, will be initialized as type Skipped" namespace=ci workflow=wat-fpdsp
time="2023-10-19T07:11:37.620Z" level=info msg="Retry node wat-fpdsp-4145173754 initialized Running" namespace=ci workflow=wat-fpdsp                                                                      time="2023-10-19T07:11:37.620Z" level=info msg="Pod node wat-fpdsp-3447821985 initialized Pending" namespace=ci workflow=wat-fpdsp
time="2023-10-19T07:11:37.655Z" level=info msg="Created pod: wat-fpdsp(2).split(0) (wat-fpdsp-foo-3447821985)" namespace=ci workflow=wat-fpdsp                                                            time="2023-10-19T07:11:37.655Z" level=warning msg="was unable to obtain the node for wat-fpdsp-911175476, taskName map"
time="2023-10-19T07:11:37.655Z" level=warning msg="was unable to obtain the node for wat-fpdsp-911175476, taskName map"                                                                                   time="2023-10-19T07:11:37.656Z" level=info msg="TaskSet Reconciliation" namespace=ci workflow=wat-fpdsp
time="2023-10-19T07:11:37.656Z" level=info msg=reconcileAgentPod namespace=ci workflow=wat-fpdsp                                                                                                          time="2023-10-19T07:11:37.656Z" level=info msg="Workflow to be dehydrated" Workflow Size=4503
time="2023-10-19T07:11:37.676Z" level=info msg="Workflow update successful" namespace=ci phase=Running resourceVersion=175536948 workflow=wat-fpdsp

Logs from in your workflow’s wait container

none relevant

About this issue

  • Original URL
  • State: open
  • Created 8 months ago
  • Reactions: 1
  • Comments: 15 (13 by maintainers)

Most upvoted comments

If that’s the case that 3.4.x worked with Argo variables overriding built-in functions, then it’s possibly a regression in expr: that built-ins now have priority over added context variables?

It also breaks if you call a task map