pipeline: "Finally" not work as expected

Expected Behavior

When task failed, the finally tasks should be trigger immediately.

Actual Behavior

In parallel case, finally tasks not trigger only all parallel tasks done. See the code snippet:

      - name: job-activity-job-1
        runAfter:
          - job-activity-fake-run-cmd
        taskSpec:
          results:
            - name: err-source
              description: indicates exit code for fake command
          steps:
            - name: run
              image: ubuntu
              script: |
                #!/bin/sh
                sleep 60
                printf "job 1 found issue" | tee /tekton/results/err-source
                echa "job-1 start to run..."
      - name: job-activity-job-2
        runAfter:
          - job-activity-fake-run-cmd
        taskSpec:
          results:
            - name: err-source
              description: indicates exit code for fake command
          steps:
            - name: run
              image: ubuntu
              script: |
                #!/bin/sh
                printf "job 2 found issue" | tee /tekton/results/err-source
                echa "job-2 start to run..."
    finally:
      - name: exception-handler
        taskRef:
          apiVersion: custom.tekton.dev/v1alpha1
          kind: Exception
          name: exception
        params:
        - name: pipelinerun_name
          value: guarded-pr

See, the job-activity-job-1 and job-activity-job-2 will be run as parallel and job-activity-job-2 will failed immediate(wrong command:echa) and job-activity-job-1 will failed after 60s.

The finally is triggered exactly after job-activity-job-1 failed.

Steps to Reproduce the Problem

Additional Info

  • Kubernetes version:

    Output of kubectl version:

    (paste your output here)
    
  • Tekton Pipeline version:

    Output of tkn version or kubectl get pods -n tekton-pipelines -l app=tekton-pipelines-controller -o=jsonpath='{.items[0].metadata.labels.version}'

See the code: https://github.com/tektoncd/pipeline/blob/f764c3ba94294d5675b2d5490240336bd66c1972/pkg/reconciler/pipelinerun/resources/pipelinerunstate.go#L387-L398

Seems if one taskrun is not “done” (skip, success,failed), the finally will not be trigger, and it’s also depends on the order of taskruns, I think this not make sense.

About this issue

  • Original URL
  • State: closed
  • Created 4 years ago
  • Comments: 15 (9 by maintainers)

Most upvoted comments

cc @jerop @pritidesai

I think it is by design, we wait for all tasks to finish before running the finally. If multiple tasks started concurrently and one fail, we usually let the other ones finish (we do not cancel it), we just “stop” scheduling new ones.

When task failed, the finally tasks should be trigger immediately

Is it documented like that ? Because as said above, the “guarantee” we make is the finally tasks will execute at the end, no matter what failed during the “normal tasks”, I don’t think there is anything on “trigger immediatly”.

Thanks @pritidesai I guess not, since I concern the time point to trigger the Finally, not the behavior when Finally trigger. but thank you all the same.