pipeline: Cannot clone private Git repository as an input PipelineResource in tekton task

Expected Behavior

Clone the private Git repository using the supplied secret.

Actual Behavior

Fails with a misleading error.

Steps to Reproduce the Problem

As documented in https://github.com/tektoncd/pipeline/blob/master/docs/resources.md I have configured a private GitHub repository as a PipelineResource for a task, and have created the relevant secret as well:

---
apiVersion: v1
kind: Secret
metadata:
  name: github-secrets
type: Opaque
data:
  token: github_personal_access_token_secret # in base64 encoded form
---
apiVersion: tekton.dev/v1alpha1
kind: PipelineResource
metadata:
  name: my-repo-git
spec:
  type: git
  params:
    - name: revision
      value: master
    - name: url
      value: https://github.my-company.com/my-team/my-repo.git
  secrets:
    - fieldName: authToken
      secretName: github-secrets
      secretKey: token

Now when I am using the above PipelineResource as an input to a task:

apiVersion: tekton.dev/v1alpha1
kind: Task
metadata:
  name: my-task
spec:
  inputs:
    resources:
      - name: my-repo-git
        type: git
  steps:
    - name: print-info
      image: image-registry.openshift-image-registry.svc:5000/default/my-task-runner-image:latest
      imagePullPolicy: Always
      command: ["/bin/sh"]
      args:
        - "-c"
        - >
          echo "List the contents of /workspace, expecting to find the Git repository in there" &&
          ls -R /workspace

I get the following error:

[test : git-source-my-repo-git-qhsvq] {"level":"warn","ts":1580370880.0396245,"logger":"fallback-logger","caller":"logging/config.go:69","msg":"Fetch GitHub commit ID from kodata failed: \"KO_DATA_PATH\" does not exist or is empty"}
[test : git-source-my-repo-git-qhsvq] {"level":"error","ts":1580370880.653762,"logger":"fallback-logger","caller":"git/git.go:40","msg":"Error running git [fetch --depth=1 --recurse-submodules=yes origin master]: exit status 128\nfatal: could not read Username for 'https://github.my-company.com': No such device or address\n","stacktrace":"github.com/tektoncd/pipeline/pkg/git.run\n\t/go/src/github.com/tektoncd/pipeline/pkg/git/git.go:40\ngithub.com/tektoncd/pipeline/pkg/git.Fetch\n\t/go/src/github.com/tektoncd/pipeline/pkg/git/git.go:91\nmain.main\n\t/go/src/github.com/tektoncd/pipeline/cmd/git-init/main.go:39\nruntime.main\n\t/usr/local/go/src/runtime/proc.go:198"}
[test : git-source-my-repo-git-qhsvq] {"level":"error","ts":1580370881.232096,"logger":"fallback-logger","caller":"git/git.go:40","msg":"Error running git [pull --recurse-submodules=yes origin]: exit status 1\nfatal: could not read Username for 'https://github.my-company.com': No such device or address\n","stacktrace":"github.com/tektoncd/pipeline/pkg/git.run\n\t/go/src/github.com/tektoncd/pipeline/pkg/git/git.go:40\ngithub.com/tektoncd/pipeline/pkg/git.Fetch\n\t/go/src/github.com/tektoncd/pipeline/pkg/git/git.go:94\nmain.main\n\t/go/src/github.com/tektoncd/pipeline/cmd/git-init/main.go:39\nruntime.main\n\t/usr/local/go/src/runtime/proc.go:198"}
[test : git-source-my-repo-git-qhsvq] {"level":"warn","ts":1580370881.2322857,"logger":"fallback-logger","caller":"git/git.go:95","msg":"Failed to pull origin : exit status 1"}
[test : git-source-my-repo-git-qhsvq] {"level":"error","ts":1580370881.2347631,"logger":"fallback-logger","caller":"git/git.go:40","msg":"Error running git [checkout master]: exit status 1\nerror: pathspec 'master' did not match any file(s) known to git.\n","stacktrace":"github.com/tektoncd/pipeline/pkg/git.run\n\t/go/src/github.com/tektoncd/pipeline/pkg/git/git.go:40\ngithub.com/tektoncd/pipeline/pkg/git.Fetch\n\t/go/src/github.com/tektoncd/pipeline/pkg/git/git.go:97\nmain.main\n\t/go/src/github.com/tektoncd/pipeline/cmd/git-init/main.go:39\nruntime.main\n\t/usr/local/go/src/runtime/proc.go:198"}
[test : git-source-my-repo-git-qhsvq] {"level":"fatal","ts":1580370881.2348266,"logger":"fallback-logger","caller":"git-init/main.go:40","msg":"Error fetching git repository: exit status 1","stacktrace":"main.main\n\t/go/src/github.com/tektoncd/pipeline/cmd/git-init/main.go:40\nruntime.main\n\t/usr/local/go/src/runtime/proc.go:198"}

The error could not read Username for 'https://github.my-company.com': No such device or address indicates that it looks for a username. The sourceSecret in build configs has different format:

apiVersion: v1
kind: Secret
metadata:
  name: github-secrets
type: kubernetes.io/basic-auth
data:
  username: my-github-username
  password: github_personal_access_token_secret # in base64 encoded form

However, even when trying the above secret type I still get the same error.

How do I configure the secret for a private repository?

Additional Info

Related (though different) issues:

  1. https://github.com/tektoncd/pipeline/issues/1603
  2. https://github.com/tektoncd/pipeline/issues/1375

About this issue

  • Original URL
  • State: closed
  • Created 4 years ago
  • Reactions: 1
  • Comments: 17 (1 by maintainers)

Most upvoted comments

Hi @g000444555, thanks for the issue.

The way to use secret in tekton is a bit different than usual, see auth.md.

  • Your secret need to be annotated with tekton.dev/git-0: https://github.my-company.com
  • You need to add that secret to a serviceaccount
  • You need to run you Task or Pipeline with that serviceAccount (using serviceAccountName)

/kind question

That’s a very good answer. Thank you very much.

That’s a very good answer. Thank you very much.

It worked well. Thank you for this answer

@sbwsg @CoderPoet Does github.com that is used with ssh-auth qualify for “name” https://github? It sound like it could be caused by wrong protocol. I would expect that with http protocol it looks only for basic-auth key. If that’s the case how can you dynamically rewrite protocol?

"msg":"Error running git [fetch --recurse-submodules=yes --depth=1 origin master]: exit status 128\nfatal: could not read Username for 'https://github.company.com': No such device or address\n"

Edit: Wrong protocol was indeed the problem at least in my case. Error with the https protocol and ssh-auth suggest wrong protocol this can be fixed in the github case by using value: $(body.repository.ssh_url) instead of value:$(body.repository.html_url)

@zertan Yes it should work with any git repo. But you might find the git-clone task from the catalog more useful: https://github.com/tektoncd/catalog/tree/master/git

The git pipelineresource can be difficult to inspect/debug when there are problems fetching from a repo.