lerna: lerna publish --canary publishes broken packages

lerna publish --canary only looks at HEAD to determine what packages changed. This means that a package can be published without correct dependencies if those dependencies were changed in an earlier commit. I believe that --canary only operating on the HEAD commit is intentional, but I’m not sure how to achieve the “nightly release” behavior that this flag is supposed to provide.

I read through a few issues about the --canary and this may be a duplicate but I decided to create a new issue to share my use case and describe the problems im running into.

Expected Behavior

Every package published by lerna publish --canary should have dependencies that align with the state of the repo when the publish was run.

Current Behavior

Since lerna publish --canary only looks at the HEAD for changes it can publish broken packages. For an example, see the “Steps to Reproduce” section

Possible Solution

One solution of this could be to add --changed to lerna publish like https://github.com/lerna/lerna/issues/962 suggested

As a workaround I could use --force-publish "*" or try to replicate the --since flag using --force-publish and lerna list --since. I could also stop using the --canary flag but then I would lose its ability to automatically increment the -alpha.X suffix.

Steps to Reproduce (for bugs)

Do the following in a monorepo with package-a and package-b where package-a depends on package-b

Commit a new feature to package-b and run lerna publish --canary. lerna releases package-a@1.0.0-alpha.0 and package-b@1.0.0-alpha.0. package-a will depend on package-b@1.0.0-alpha.0 as expected.
Commit a new feature to package-a and run lerna publish --canary. lerna releases package-a@1.0.0-alpha.1 that depends on package-b@1.0.0. package-a will be broken if it uses a feature introduced in a previous commit and therefore is only available in package-b’s canary

lerna.json

I was using a pretty basic config:

{
  "packages": [
    "packages/*"
  ],
  "npmClient": "npm",
  "version": "1.0.1"
}

Context

I have a monorepo containing a few apps and libraries that are used by the apps. I want to run lerna publish --canary in CI for each commit to the master branch and a lot of the apps end up getting broken canaries as they are not using the correct versions of the libraries.

Your Environment

Executable	Version
`lerna --version`	3.13.4
`npm --version`	6.4.1
`yarn --version`	1.13.0
`node --version`	10.15.1

OS	Version
macOS Mojave	10.14.4

About this issue

Original URL
State: open
Created 5 years ago
Reactions: 9
Comments: 15 (4 by maintainers)

Commits related to this issue

Docs: Adds `--force-publish` to publish options I didn't see [this option](https://github.com/lerna/lerna/issues/2060#issuecomment-619403848) documented anywhere and it proved very helpful for my use... — committed to kendallstrautman/lerna by kendallstrautman 3 years ago
Docs: Adds `--force-publish` to publish options I didn't see [this option](https://github.com/lerna/lerna/issues/2060#issuecomment-619403848) documented anywhere and it proved very helpful for my use... — committed to kendallstrautman/lerna by kendallstrautman 3 years ago
Fix publish version number never increasing Lerna uses '+${SHA}' in the package version but GitHub Packages ignores everything after the '+', which means the package version is always the same T... — committed to bugsnag/bugsnag-js-performance by imjoehaines a year ago
Fix publish version number never increasing Lerna uses '+${SHA}' in the package version but GitHub Packages ignores everything after the '+', which means the package version is always the same T... — committed to bugsnag/bugsnag-js-performance by imjoehaines a year ago

Most upvoted comments

@NMinhNguyen This is sorely needed.

Right now we’re using a hack to workaround this, but it’s unpleasant:

lerna publish --canary --preid next.$(git rev-parse --short HEAD)

Produces:

my-package@0.0.1-next.604b79a7.4

Where the trailing .4 is the remainder of the (broken) canary versioning.

jakewhelan on Jul 14, 2020

@dimaShin The simple workaround is to just pass the --force-publish flag to lerna publish --canary which will make it publish a canary for every package in your monorepo.

If for some reason you don’t want to publish all those extra packages you could also write a script that gets the intersection of lerna changed --json (packages changed since last publish) and lerna list --since $GIT_PREVIOUS_SUCCESSFUL_COMMIT^ --include-filtered-dependencies --json (uses jenkins specific var to get packages changed in commits you added to master). This would give you the minimal set of packages to pass to the --force-publish argument.

bimbiltu on Jan 4, 2021

Yes, true, the spec is only speaking to precedence there. In my manual testing, I found that attempting to publish version 1.2.3-alpha.0+12345 will always result in a version of 1.2.3-alpha.0 being available from the registry. Of course, this was done after i had shipped it… ;_;

@evocateur would you accept a PR that removes the +${sha} part or changes it to .${sha} maybe? https://github.com/lerna/lerna/blob/3367257cabe1540a3b9468acbfa0d01ba391077d/commands/publish/index.js#L365

NMinhNguyen on May 20, 2020

Yeah, that definitely does seem broken. Part of the problem with “nightly” releases, at least as Lerna attempts them, is that CI often uses a shallow clone, which means in many cases the tags from the previous release literally aren’t there. There are workarounds, of course.

At a higher level, I think part of the problem with --canary is that it never had high-enough test coverage to illustrate the intent without ambiguity. I attempted to clear up some of these ambiguities earlier in the 3.x series, but I suspect I just made them “break harder” in use cases like this.

Theoretically, we have “better” coverage now. That being said, I’m positive there are big chunks missing. Awhile back I was hoping that semantic-release could replace lerna publish --canary, but that never really went anywhere once the complexity of lerna versioning was investigated. So I’m pretty open to a lerna release command that could encapsulate this better, but I have a vanishingly small amount of free time available to devote to maintenance (to say nothing of new feature development).

evocateur on Apr 29, 2019

Yes, true, the spec is only speaking to precedence there. In my manual testing, I found that attempting to publish version 1.2.3-alpha.0+12345 will always result in a version of 1.2.3-alpha.0 being available from the registry. Of course, this was done after i had shipped it… ;_;

The version specifiers passed to npm commands are generally cleaned up by semver’s valid() method before further processing, which yields the same behavior as my ad-hoc registry tests.

evocateur on Dec 31, 2019

@erykpiast Yeah, in retrospect using the + to append commit_hash was a mistake, as even the public registry completely ignores it (it’s allowed to, per the spec).

evocateur on Dec 31, 2019

Great to see lerna is getting a new maintainer!

I’ve opened a PR with a reproducer for this https://github.com/lerna/repro/pull/2

bimbiltu on Jun 11, 2022

The spec seems to say that anything after the plus does not count towards precedence. However, I would think that if someone types out the full version string, it should find that version exactly.

Is the problem that we cannot pass raw version strings to npm, because it always interprets them as a “version range” or “version matcher” or whatever it’s called? Does using an equal sign help?

On Tue, Dec 31, 2019, 12:55 PM Daniel Stockman notifications@github.com wrote:

@erykpiast https://github.com/erykpiast Yeah, in retrospect using the + to append commit_hash was a mistake, as even the public registry completely ignores it (it’s allowed to, per the spec https://semver.org/#spec-item-10).

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/lerna/lerna/issues/2060?email_source=notifications&email_token=AAC35OESLYCMX4YA3Y3HZPTQ3OBPVA5CNFSM4HJFUDSKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEH4P56Q#issuecomment-569966330, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAC35OCU75YGVWYPEDGESUTQ3OBPVANCNFSM4HJFUDSA .

cspotcode on Dec 31, 2019