semantic-release: Dry-run for previewing release does nothing

Current behavior

Dry-run does nothing and instead reports something like: This run was triggered by a pull request and therefore a new version won't be published. or [2:59:20 PM] [semantic-release] › ℹ This test run was triggered on the branch refs/pull/3/merge, while semantic-release is configured to only publish from main, therefore a new version won’t be published.

Expected behavior

Dry-run runs the normal steps and plugins like with a release but does not commit anything or release artifacts.

Environment

About this issue

  • Original URL
  • State: open
  • Created 3 years ago
  • Reactions: 7
  • Comments: 16 (1 by maintainers)

Most upvoted comments

The most annoying thing is that it ends up returning refs/pull/35/merge regardless if I let it run in GITHUB (my CI) or in GIT (doing unset GITHUB_ACTIONS)

Github env: https://github.com/semantic-release/env-ci/blob/b39d3440157a17e89fc964876f4b5eb38df991d5/services/github.js#L39

Git env: https://github.com/semantic-release/env-ci/blob/b39d3440157a17e89fc964876f4b5eb38df991d5/lib/git.js#L11

Which would be ok only if semantic-release wasn’t checking THAT against real branch names: which will never match

https://github.com/semantic-release/semantic-release/blob/1beeb84c9032a22d455fc21b9a736efbcfde6b5d/index.js#L40-L41

https://github.com/semantic-release/semantic-release/blob/1beeb84c9032a22d455fc21b9a736efbcfde6b5d/index.js#L68-L69

Any updates from semantic-release contributors on how to make this work?

I believe all messages ending with ...therefore a new version won’t be published. should be followed by, on a new line…

  • the new version would have remained as x.y.z (configuration is working AND commits were analyzed)
  • the new version would have bumped to x.y.z. (configuration is working AND commits were analyzed)
  • the new version could not be determined. (configuration is invalid OR unable to analyze commits)

Or, for simplicity

  • the version could have been (x.y.z | UNDETERMINED).

Currently, errors irrelevant in dev environments such as ENOGHTOKEN and non-error rules as as “The local branch main is behind the remote one, therefore a new version won’t be published.” prevent users from troubleshooting configuration errors and improving existing configurations without PUSHing to remote…which, when checking single-line changes, can be annoying or even lead to undesired changes by the CI/CD environment. The latter can be resolved by first adjusting the CI/CD’s invokation of semantic-release to also use --dry-run, but then you need at least two more commits (and/or changes in a web interface) for diagnostics that could have been done without committing or pushing.

GitTools’ GitVersion CLI has comparable commit analysis and version generation capabilities. It can work offline, while out-of-sync with remote, and with uncommitted changes.

GitTools' GitVersion CLI output
> gitversion
{
  "Major": 2,
  "Minor": 3,
  "Patch": 0,
  "PreReleaseTag": "feat-AvanloniaUI.253",
  "PreReleaseTagWithDash": "-feat-AvanloniaUI.253",
  "PreReleaseLabel": "feat-AvanloniaUI",
  "PreReleaseLabelWithDash": "-feat-AvanloniaUI",
  "PreReleaseNumber": 253,
  "WeightedPreReleaseNumber": 253,
  "BuildMetaData": null,
  "BuildMetaDataPadded": "",
  "FullBuildMetaData": "Branch.feat-AvanloniaUI.Sha.4cbbd44a302bf6c156b76212df7c68e0502ac298",
  "MajorMinorPatch": "2.3.0",
  "SemVer": "2.3.0-feat-AvanloniaUI.253",
  "LegacySemVer": "2.3.0-feat-AvanloniaUI253",
  "LegacySemVerPadded": "2.3.0-feat-AvanloniaUI0253",
  "AssemblySemVer": "2.3.0.0",
  "AssemblySemFileVer": "2.3.0.0",
  "FullSemVer": "2.3.0-feat-AvanloniaUI.253",
  "InformationalVersion": "2.3.0-feat-AvanloniaUI.253+Branch.feat-AvanloniaUI.Sha.4cbbd44a302bf6c156b76212df7c68e0502ac298",
  "BranchName": "feat/AvanloniaUI",
  "EscapedBranchName": "feat-AvanloniaUI",
  "Sha": "4cbbd44a302bf6c156b76212df7c68e0502ac298",
  "ShortSha": "4cbbd44",
  "NuGetVersionV2": "2.3.0-feat-avanloniaui0253",
  "NuGetVersion": "2.3.0-feat-avanloniaui0253",
  "NuGetPreReleaseTagV2": "feat-avanloniaui0253",
  "NuGetPreReleaseTag": "feat-avanloniaui0253",
  "VersionSourceSha": "5c973c9326cc7412bd47f743a2e1ab82cf4995f2",
  "CommitsSinceVersionSource": 253,
  "CommitsSinceVersionSourcePadded": "0253",
  "UncommittedChanges": 14,
  "CommitDate": "2023-07-17"
}

Ok, I found a workaround. Not sure if it’s ideal, but it’s working if anyone is interested

  1. We need to checkout to the actual PR branch
  2. We need unset “GITHUB_ACTIONS” from env so that env-ci will infer the branch name from git instead of GH Actions data
      - name: Checkout branch
        run: git checkout $GITHUB_HEAD_REF
      - name: Preview semantic-release
        run: unset GITHUB_ACTIONS && npx semantic-release --dry-run --no-ci

If you’re running in a CI other than GitHub actions, I suggest you look at its corresponding detect criteria so that you can force it to false and get force semantic-release to use git

@seedy I create an action for this. I combine it with create-or-update-comment, in a way that my yml file looks like this:

name: Integration

on: pull_request

jobs:
  preview_release_notes:
    runs-on: ubuntu-latest
    name: Preview release notes
    steps:
      - name: Checkout repository
        uses: actions/checkout@v2
        with:
          fetch-depth: 0
      - name: Setup Node.js
        uses: actions/setup-node@v2
        with:
          node-version: 12
      - name: Cache node_modules
        uses: actions/cache@v2
        with:
          path: ~/.npm
          key: ${{ runner.os }}-node-${{ hashFiles('**/package-lock.json') }}
          restore-keys: ${{ runner.os }}-node-
      - name: Install dependencies
        run: npm ci --ignore-scripts
        env:
          NODE_AUTH_TOKEN: ${{ secrets.GH_TOKEN }}
      - name: Generate release notes preview
        id: build-release-notes
        uses: guilhermetod/semantic-release-notes-preview@v1.0.0
      - name: Comment release notes preview
        if: steps.build-release-notes.outputs.releaseNotes
        uses: peter-evans/create-or-update-comment@v1
        with:
          issue-number: ${{ github.event.number }}
          body: |
            ## 👋 Hey there!
            Thank you for you contribution. Below is a preview of the release notes if your PR gets merged.
            Please, make sure it includes all your significant changes with descriptive messages.
            Keep in mind that release notes are automatically generated from the commit messages according to [conventional commits](https://www.conventionalcommits.org/).
            If you feel like you need to change anything, please take a look into [this guide](https://github.com/angular/angular/blob/master/CONTRIBUTING.md#addressing-review-feedback) to learn how you can fixup your commits.

            ---

            ${{ steps.build-release-notes.outputs.releaseNotes }}

I hope this helps you!

Edit: here’s the marketplace link

@Avivbens your implementation also uses ${{ github.head_ref }} so suffers from the same issues I described above. If that’s not a problem for your use case then great! I do like your addition of the PR comment logic though so will be adopting that, thank you!

Hi all 👋

I’ve created a nice GitHub action workflow, with no additional configuration needed 👍

Here are the workflow results for chore commit:

CleanShot 2024-01-19 at 22 40 23@2x

Once you committed additional stuff, it will remove the last report and create a new one. Here are the results for fix commit:

CleanShot 2024-01-19 at 22 41 17@2x

The workflow:

name: Semantic Release Check 📝
on:
    pull_request:
        branches:
            - master

permissions: write-all

jobs:
    PR-checks:
        runs-on: self-hosted
        name: Semantic Release Check 📝
        steps:
            - name: 📀 Checkout
              uses: actions/checkout@v3
              with:
                  fetch-depth: 0

            - name: 🖥️ Setup Node
              uses: actions/setup-node@v3
              with:
                  node-version: 20

            - name: 🔗 Install Dependencies
              shell: bash
              run: |
                  npm ci --no-fund --no-audit --ignore-scripts

            - name: 🔬 Check semantic versioning
              id: semantic-release
              run: |
                  GITHUB_REF=${{ github.head_ref }}
                  npx semantic-release --no-ci --dry-run --plugins @semantic-release/commit-analyzer,@semantic-release/release-notes-generator --branches ${{ github.head_ref }} > output.txt
                  OUTPUT=$(cat output.txt | base64 -w 0)
                  echo "::set-output name=releaseNote::$OUTPUT"

            - name: 📝 Report semantic versioning
              uses: actions/github-script@v3
              if: ${{ steps.semantic-release.outputs.releaseNote != '' }}
              with:
                  github-token: ${{ secrets.GITHUB_TOKEN }}
                  script: |
                      // build release note
                      const semanticReleaseOutput = Buffer.from('${{ steps.semantic-release.outputs.releaseNote }}', 'base64').toString('utf8');
                      const semanticReleaseLogMatch = /^[[0-9:\sAMPM]+\]\s\[semantic-release\].*$/;
                      const lines = semanticReleaseOutput.split('\n');
                      const lastSemanticReleaseLogIndex = [...lines]
                          .reverse()
                          .findIndex((line) => line.match(semanticReleaseLogMatch));

                      const releaseNoteIndex = lines.length - lastSemanticReleaseLogIndex;
                      const releaseNote = lines.slice(releaseNoteIndex);

                      let res = releaseNote.join('\n');
                      if (!releaseNote.length || !res) {
                          res = '### No release note would be generated.';
                      }

                      const SEMANTIC_RELEASE_BODY_HEADER = '## 📝 Semantic Release Report';
                      const body = [SEMANTIC_RELEASE_BODY_HEADER, res].join('\n');

                      // get last comment
                      const comments = await github.issues.listComments({
                          issue_number: context.issue.number,
                          owner: context.repo.owner,
                          repo: context.repo.repo
                      });

                      // find comments to delete
                      const commentsToDelete = comments.data.filter((comment) =>
                          comment.body.startsWith(SEMANTIC_RELEASE_BODY_HEADER)
                      );

                      // delete comments
                      const prms = commentsToDelete.map((comment) =>
                          github.issues.deleteComment({
                              comment_id: comment.id,
                              owner: context.repo.owner,
                              repo: context.repo.repo
                          })
                      );

                      await Promise.all(prms);

                      // create new comment for release note
                      github.issues.createComment({
                          issue_number: context.issue.number,
                          owner: context.repo.owner,
                          repo: context.repo.repo,
                          body
                      });

Enjoy 🥳

CC: @TJM @seedy @OliverEhrhardt @XC- @BinToss

@TJM if you also pass through the --no-ci flag, semantic-release won’t do the CI environment checks and so will skip the pull request check. This command should do the full dry run:

npx semantic-release --dry-run --no-ci --branches master,$CI_COMMIT_BRANCH