pipeline: unclear why pipeline task is stuck Pending / tekton entrypoint behaving different

Expected Behavior

Expected task to run, fail, or be refused as invalid when added originally

While writing this report it finally failed

Some detail as to why this happened and maybe where to look to debug this would be great. I tried looking for the pod, describing the pipelinerun, I had a little look at the tekton-pipeline-controler’s logs but gave up quite quick as they’re pretty hard to read in the terminal

Actual Behavior

Pipeline sat with an incomplete pending task, no pod for the task, pipeline run also doesn’t show a pod for ages

Steps to Reproduce the Problem

  1. Create KIND cluster
  2. Deploy operator (I was using 0.22.0-1)
  3. Deploy config all
  4. Apply task below + tkn hub install -> tkn hub install task git-clone golang-build golang-test
  5. Apply pipeline below (modify FILLTHISIN parts)
  6. Apply PVC below
  7. Run pipeline using tkn

Additional Info

  • Kubernetes version:

    Output of kubectl version:

λ kubectl version
Client Version: version.Info{Major:"1", Minor:"20", GitVersion:"v1.20.5", GitCommit:"6b1d87acf3c8253c123756b9e61dac642678305f", GitTreeState:"archive", BuildDate:"1980-01-01T00:00:00Z", GoVersion:"go1.16.3", Compiler:"gc", Platform:"linux/amd64"}
Server Version: version.Info{Major:"1", Minor:"20", GitVersion:"v1.20.2", GitCommit:"faecb196815e248d3ecfb03c680a4507229c2a56", GitTreeState:"clean", BuildDate:"2021-01-21T01:11:42Z", GoVersion:"go1.15.5", Compiler:"gc", Platform:"linux/amd64"}
  • Tekton Pipeline version:

    Output of tkn version or kubectl get pods -n tekton-pipelines -l app=tekton-pipelines-controller -o=jsonpath='{.items[0].metadata.labels.version}'

λ tkn version
Client version: 0.17.2
Pipeline version: v0.21.0
Triggers version: v0.11.2
Dashboard version: v0.14.0

From within the taskrun section of the pipeline run for ages:

    sample-pipeline-run-z48h8-hello-dt4v9:
      pipelineTaskName: hello
      status:
        podName: ""

Then finally updated to:

    sample-pipeline-run-z48h8-hello-dt4v9:
      pipelineTaskName: hello
      status:
        completionTime: "2021-04-27T15:52:31Z"
        conditions:
        - lastTransitionTime: "2021-04-27T15:52:31Z"
          message: |
            "step-hello" exited with code 2 (image: "docker.io/library/busybox@sha256:ae39a6f5c07297d7ab64dbd4f82c77c874cc6a94cea29fdec309d0992574b4f7"); for logs run: kubectl -n default logs sample-pipeline-run-z48h8-hello-dt4v9-pod-zqqqn -c step-hello
          reason: Failed
          status: "False"
          type: Succeeded
        podName: sample-pipeline-run-z48h8-hello-dt4v9-pod-zqqqn
        startTime: "2021-04-27T15:34:55Z"
        steps:
        - container: step-hello
          imageID: docker.io/library/busybox@sha256:ae39a6f5c07297d7ab64dbd4f82c77c874cc6a94cea29fdec309d0992574b4f7
          name: hello
          terminated:
            containerID: containerd://9073b8fea524187e38007c865d4f3bc09a8690b434a4f3d81af87add9760c3bb
            exitCode: 2
            finishedAt: "2021-04-27T15:52:30Z"
            reason: Error
            startedAt: "2021-04-27T15:52:30Z"
        taskSpec:
          description: hello world
          steps:
          - args:
            - echo
            - hi
            image: busybox
            name: hello
            resources: {}

PipelineRun’s TaskRun:

Name Task Created Duration
sample-pipeline-run-z48h8-hello-dt4v9 hello-world 19 minutes ago 17 minutes 36 seconds
hello-world-run-rhtmt hello-world 16 minutes ago 14 minutes 34 seconds

Task:

---
apiVersion: tekton.dev/v1beta1
kind: Task
metadata:
  name: hello-world
  labels:
    app.kubernetes.io/version: "0.2"
  annotations:
    tekton.dev/pipelines.minVersion: "0.17.0"
    tekton.dev/tags: cli
    tekton.dev/displayName: "Hello World"
spec:
  description: >-
    hello world
  steps:
    - name: hello
      image: busybox
      args:
        - echo
        - hi

Equivelent job that runs fine

apiVersion: batch/v1
kind: Job
metadata:
  name: hello
spec:
  template:
    spec:
      containers:
        - name: hello
          image: busybox
          args: ["echo", "hello"]
      restartPolicy: Never

λ kubectl logs hello-r99h9
hello

docker run busybox echo hi runs fine

Pipeline:

---
apiVersion: tekton.dev/v1beta1
kind: Pipeline
metadata:
  name: sample-pipeline
spec:
  workspaces:
    - name: source
      description: The workspace containing the source code which is to be build, pushed and deployed
  tasks:
    - name: clone
      taskRef:
        name: git-clone
      workspaces:
        - name: output
          workspace: source
      params:
        - name: url
          value: FILLTHISIN
        - name: deleteExisting
          value: "true"
    - name: build
      taskRef:
        name: golang-build
      workspaces:
        - name: source
          workspace: source
      params:
        - name: package
          value: FILLTHISIN
      runAfter:
        - clone
    - name: test
      taskRef:
        name: golang-test
      workspaces:
        - name: source
          workspace: source
      params:
        - name: package
          value: FILLTHISIN
      runAfter:
        - build
    - name: hello
      taskRef:
        name: hello-world
      runAfter:
        - build

pvc:

apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: pipeline-source-pvc
spec:
  storageClassName: standard
  accessModes:
    - ReadWriteOnce
  resources:
    requests:
      storage: 1Gi

About this issue

  • Original URL
  • State: closed
  • Created 3 years ago
  • Reactions: 1
  • Comments: 17 (3 by maintainers)

Most upvoted comments

Yeah, very bizarre. Thanks for the additional data here, I will try to find some time next week to see if this is a kind-specific issue.