Yes, that’s specifically for actions triggered by a PR. On push it should be using the tip of the pushed branch.

Just an FYI that dvc exp remove -g ... will probably not work out of the box in codespaces, the default github access token generated for a codespace does not have delete permissions. (This can be verified by trying git push --delete refs/exps/... in the codespace)

I’m assuming it’s possible to configure codespaces to use a specific token, in which case exp remove -g origin would work as long as the token had the appropriate delete permissions.

@dberenbaum I’m not sure that this workflow will actually do what you are intending though. If you run an experiment in a github actions PR with the default actions/checkout workflow, HEAD in the pull request is the merge commit generated by github (refs/pull/###/merge), and not the actual tip of the PR branch (refs/pull/###/head).

So the resulting experiment will be derived from the pull request merge commit (and not from some commit on your original main branch).

To illustrate this, the linked run from my example-get-started fork creates an experiment derived from 716deb4 (the PR merge commit). The tip of my PR ci-push-test branch is actually 66610db

$ dvc exp list origin -A
716deb4:
        pmrowla-test

Even if I pull the experiment and then do exp show -A I will not be able to see the experiment since it does not actually belong to any of the branches in my repo (the PR merge commit generated by github is essentially a dangling merge commit that does not belong to any branch). To actually view the experiment in exp show, you would have to git checkout 716deb4 (detached-checkout the merge commit) and then do exp show from the detached checkout.

If you want the experiment to derive from the tip of the PR branch, users need to explicitly tell github to not use the merge ref in their GHA workflow

- uses: actions/checkout@v3
  with:
    ref: ${{ github.event.pull_request.head.sha }}

(see https://github.com/actions/checkout#checkout-pull-request-head-commit-instead-of-merge-commit)

This is actually supported in dulwich, there’s a bug in how we were calling dulwich send-pack and will be fixed in the next scmrepo release. (see https://github.com/pmrowla/example-get-started/actions/runs/4676585105/jobs/8283071376)

It looks like git push does work (https://github.com/pmrowla/example-get-started/actions/runs/4673849990/jobs/8277481487), so this is probably due to dulwich’s shallow ref support being incomplete. I’ll start looking into it but it may be non-trivial to fix (shallow repo support is complicated, there’s a reason libgit2 still hasn’t implemented any shallow support at all)

In the meantime the suggested workflow for using exp push in CI should be to just use fetch-depth: 0

@pmrowla When do you expect users need to set the fetch depth? I thought it was no longer needed?

It’s not needed for exp run and exp apply.

git push (and exp push) depends on git history that may or may not be available in shallow repos. I haven’t tested to see if it’s actually possible to manually git push exp refs from an actions/checkout GHA workflow (with the default shallow fetch-depth: 1), but in general any time you need to git push anything in GHA I’m pretty sure the recommendation is to use fetch-depth: 0.

I’m assuming it’s possible to configure codespaces to use a specific token, in which case exp remove -g origin would work as long as the token had the appropriate delete permissions.

Yes, the example-repos-dev is correctly configured so exp remove -g and exp push both work:

https://github.com/iterative/example-repos-dev/blob/dcfd76301af427f9cc33f7e3e6248fa68669d16b/.devcontainer.json#L20-L22

I am currently debugging as I am not sure if it is something different, but exp push is also failing for me inside GitHub Codespaces