changesets: SyntaxError: Unexpected end of JSON input when trying to publish packages

Yep, already pushed out a change like this a couple of hours ago 😉 :

https://github.com/atlassian/changesets/commit/89a70f9d6702740c8b4c7903faefb8c7286bff20

Need to self-review and retest this before publishing a PR.

Thank you also for other comments - i will take a deeper look into everything you have mentioned

For me, I had to add the following to my package.json’s:

  "publishConfig": {
    "access": "public"
  }

Answer from: https://stackoverflow.com/a/53478078

I had the same issue earlier today and it happened for two reasons.

Reason One

There was a package that was set to "private": true in the package.json file. This was unintentional. I attempted to fix this mistake in a recent release by updating the package.json. The problem is that the package already had multiple CHANGELOG.md entries and as a result, changesets attempts to update the package with a PUT rather than publish with a POST request. Since the package doesn’t actually exist on the registry this causes an error and npm doesn’t respond with json.

Fix: Make sure there is only one entry in your changelog file before publishing a package for the first time.

Reason Two

A package I was trying to publish seemed available, but it actually causes an error response from npm when trying to publish. The naming is too similar to another package that already exists. When an error happens the npm publish command doesn’t return json and the json cannot be parsed.

Fix: Rename the package and retry.

Suggestions

I think the main problem is that changesets assumes that npm publish will always return valid json which is not the case. As a result the actual error response is hidden and all we see is an invalid json message.

https://github.com/atlassian/changesets/blob/fe8db7500f81caea9064f8bec02bcb77e0fd8fce/packages/cli/src/commands/publish/npm-utils.ts#L180

My suggestion is that this jsonParse call should be wrapped in a try catch and when failing it should log the original reponse from npm. This way it will be much easier to identify what the problem is.

Updated: Check for stderr before stdout so errors can be diagnosed more easily.

https://github.com/atlassian/changesets/blob/fe8db7500f81caea9064f8bec02bcb77e0fd8fce/packages/cli/src/commands/publish/npm-utils.ts#L169-L175

pnpm does not support the --json flag at all.

@rossng where did u get this info from? do u know of any pnpm + other stuff combination that fails to use --json with pnpm? i was just testing it and it seems to work. pnpm simply forwards almost all arguments to the npm (and it uses npm’s CLI under the hood for publishing) so in a sense this should “just work”. I’ve tested this manually with pnpm v6 and verified the code for v3, v4, v5, and v6 - they all do roughly the same thing when it comes to this args forwarding to npm publish.

There are news? Now I have the same error…

@abdonrd I’m trying to figure out the proper solution for most of the similar publishing issues to fix this once and for all. Stay tuned

Had this same issue and was able to fix it by downgrading from npm7 to latest npm@6

I managed to get the github action to work using this setup.

name: Release

on:
  push:
    branches:
      - main

concurrency: ${{ github.workflow }}-${{ github.ref }}

env:
  GITHUB_TOKEN: ${{ secrets.GH_RELEASE_TOKEN }}
  NPM_TOKEN: ${{ secrets.GH_RELEASE_TOKEN }}
  NODE_AUTH_TOKEN: ${{ secrets.GH_RELEASE_TOKEN }}

jobs:
  release:
    name: Release
    runs-on: ubuntu-latest
    steps:
      - name: Checkout Repo
        uses: actions/checkout@v3

      - name: Setup Node.js 18.x
        uses: actions/setup-node@v3
        with:
          node-version: 18.x
          cache: 'npm'
          token: ${{ secrets.GH_RELEASE_TOKEN }}
          registry-url: 'https://npm.pkg.github.com'

      - name: Install Dependencies
        run: npm ci

      - name: Create Release Pull Request or Publish to npm
        id: changesets
        uses: changesets/action@v1
        with:
          publish: npm run release

with this patch

diff --git a/node_modules/@changesets/cli/dist/cli.cjs.dev.js b/node_modules/@changesets/cli/dist/cli.cjs.dev.js
index f771824..1577813 100644
--- a/node_modules/@changesets/cli/dist/cli.cjs.dev.js
+++ b/node_modules/@changesets/cli/dist/cli.cjs.dev.js
@@ -727,7 +727,8 @@ async function internalPublish(pkgName, opts, twoFactorState) {
     npm_config_registry: getCorrectRegistry()
   };
   let {
-    stdout
+    stdout,
+    stderr,
   } = await spawn__default['default'](publishTool.name, ["publish", opts.cwd, "--json", ...publishFlags], {
     env: Object.assign({}, process.env, envOverride)
   }); // New error handling. NPM's --json option is included alongside the `prepublish and
@@ -735,7 +736,7 @@ async function internalPublish(pkgName, opts, twoFactorState) {
   // some struggles
   // Note that both pre and post publish hooks are printed before the json out, so this works.
 
-  let json = jsonParse(stdout.toString().replace(/[^{]*/, ""));
+  let json = jsonParse(stdout.toString().replace(/[^{]*/, "") || stderr.toString().replace(/^npm ERR.*$/gm, '').replace(/[^{]*/, ""));
 
   if (json.error) {
     // The first case is no 2fa provided, the second is when the 2fa is wrong (timeout or wrong words)

I got this output:

🦋  error an error occurred while publishing cem-plugin-async-function: ENEEDAUTH This command requires you to be logged in.
🦋  error You need to authorize this machine using `npm adduser`

yarn run v1.22.10
$ yarn changeset publish
$ /Users/bennyp/Developer/cem-plugins/node_modules/.bin/changeset publish
🦋  info npm info cem-plugin-async-function
🦋  info npm info cem-plugin-copy
🦋  info npm info cem-plugin-jsdoc-example
🦋  info npm info cem-plugin-module-file-extensions
🦋  info npm info cem-plugin-readonly
🦋  info npm info cem-plugin-type-descriptions-markdown
🦋  warn Received 404 for npm info "cem-plugin-async-function"
🦋  warn Received 404 for npm info "cem-plugin-module-file-extensions"
🦋  warn Received 404 for npm info "cem-plugin-copy"
🦋  warn Received 404 for npm info "cem-plugin-type-descriptions-markdown"
🦋  warn Received 404 for npm info "cem-plugin-readonly"
🦋  warn Received 404 for npm info "cem-plugin-jsdoc-example"
🦋  info cem-plugin-async-function is being published because our local version (0.0.1) has not been published on npm
🦋  info cem-plugin-copy is being published because our local version (0.0.1) has not been published on npm
🦋  info cem-plugin-jsdoc-example is being published because our local version (0.0.1) has not been published on npm
🦋  info cem-plugin-module-file-extensions is being published because our local version (0.0.1) has not been published on npm
🦋  info cem-plugin-readonly is being published because our local version (0.0.1) has not been published on npm
🦋  info cem-plugin-type-descriptions-markdown is being published because our local version (0.0.1) has not been published on npm
🦋  info Publishing "cem-plugin-async-function" at "0.0.1"
🦋  info Publishing "cem-plugin-copy" at "0.0.1"
🦋  info Publishing "cem-plugin-jsdoc-example" at "0.0.1"
🦋  info Publishing "cem-plugin-module-file-extensions" at "0.0.1"
🦋  info Publishing "cem-plugin-readonly" at "0.0.1"
🦋  info Publishing "cem-plugin-type-descriptions-markdown" at "0.0.1"
🦋  error an error occurred while publishing cem-plugin-async-function: ENEEDAUTH This command requires you to be logged in.
🦋  error You need to authorize this machine using `npm adduser`
🦋  error an error occurred while publishing cem-plugin-jsdoc-example: ENEEDAUTH This command requires you to be logged in.
🦋  error You need to authorize this machine using `npm adduser`
🦋  error an error occurred while publishing cem-plugin-copy: ENEEDAUTH This command requires you to be logged in.
🦋  error You need to authorize this machine using `npm adduser`
🦋  error an error occurred while publishing cem-plugin-module-file-extensions: ENEEDAUTH This command requires you to be logged in.
🦋  error You need to authorize this machine using `npm adduser`
🦋  error an error occurred while publishing cem-plugin-readonly: ENEEDAUTH This command requires you to be logged in.
🦋  error You need to authorize this machine using `npm adduser`
🦋  error an error occurred while publishing cem-plugin-type-descriptions-markdown: ENEEDAUTH This command requires you to be logged in.
🦋  error You need to authorize this machine using `npm adduser`
🦋  error packages failed to publish:
🦋  cem-plugin-async-function@0.0.1
🦋  cem-plugin-copy@0.0.1
🦋  cem-plugin-jsdoc-example@0.0.1
🦋  cem-plugin-module-file-extensions@0.0.1
🦋  cem-plugin-readonly@0.0.1
🦋  cem-plugin-type-descriptions-markdown@0.0.1
error Command failed with exit code 1.

which was a fair sight better than Unexpected end of JSON input.

This is using an NPM “automation token”

I was eventually able to publish by removing the “publishConfig” blocks from my package.jsons

@zkochan I just did my first update using pnpm publish -r --tag next and it worked like a charm. Thanks for the tip.

I also like how you’re managing releases for the pnpm project. I take it you’re using changesets to generate the package specific changelogs and then you’re manually adding your own top level changelog and github release. This strategy makes a lot of sense and I think I’ll adopt it. With 80+ packages in remirror, creating unfiltered package releases has become too noisy.

@ifiokjr you may just use pnpm to publish the bumped versions. Just run pnpm publish -r.

I’m having the exact same issue with pnpm

The failing build is here https://github.com/remirror/remirror/pull/251/checks?check_run_id=838235175 The commit causing the issue is here: https://github.com/remirror/remirror/tree/7f4153e0b08ae709dbdd28b09d6ebd3809e9a825 Config: https://github.com/remirror/remirror/blob/7f4153e0b08ae709dbdd28b09d6ebd3809e9a825/support/root/.changeset/config.json Prerelease config: https://github.com/remirror/remirror/blob/7f4153e0b08ae709dbdd28b09d6ebd3809e9a825/support/root/.changeset/pre.json

Node: 12 pnpm: 5.2.6

The annoying thing is that I also can’t run the release command from my own machine. It just freezes at the same point with no error displayed.