cargo: A way for users to bulk upgrade across incompatible versions

Problem

A lot of incompatible upgrades have a small enough of breakages that a “upgrade all of my incompatible version requirements” would make it easier to go through this process.

This is currently exposed in the cargo upgrade command

Proposed Solution

Tasks

Deferred

  • Modifying version requirements for compatible upgrades

See https://github.com/rust-lang/cargo/issues/12425#issuecomment-1659453818

Previously, https://internals.rust-lang.org/t/feedback-on-cargo-upgrade-to-prepare-it-for-merging/17101/141

Unresolved questions

  • Flag name
    • --breaking to focus on semver (as this is limited to ^)
    • --incompatible applies to all version req operators and doesn’t have a good short flag (-i is generally assumed to be “interactive”)
    • Dart calls it --major-versions though we have the issue of “absolute major” (x.0.0) and "relative major (also 0.x.0, 0.0.x).
    • pnpm calls it --latest because of a “latest” tag

Notes

Related

About this issue

  • Original URL
  • State: open
  • Created 3 years ago
  • Reactions: 24
  • Comments: 43 (29 by maintainers)

Commits related to this issue

Most upvoted comments

Having been fairly involved in this discussion via the internals thread, I’m happy to see this move forward in a direction that I wholeheartedly support. @epage thanks for pushing this forward!

And just to have this all in the same place, from https://github.com/rust-lang/cargo/issues/14178#issuecomment-2275791649:

I am not sure about if 2.0.0-beta.1 to 3.0.0-beta.1 should works

Perhaps that’s better as 2.0.0-beta.1 to 3.0.0? That way you have to opt into each pre-release series, but you never “skip over” a chance to return to stable?

So this upgrading between pre-releases only happens when all of the pre-releases have that same MAJOR, MINOR, and PATCH versions, and otherwise it looks for a newer stable version to upgrade to?

Personally I don’t know if we took the right approach to pre-releases in #14250 — I left more of a comment on that issue (#14178 (comment)), but my overall proposal is:

  1. If version is currently stable / non-pre-release, then ignore pre-releases
  2. If version is currently a pre-release, upgrade it as normal — including upgrading to the stable version!

This, what I would consider a bug, showed up with derive-more when cargo upgrade --breaking first failed to upgrade from 1.0.0-beta.6 to 1.0.0-beta.7, and again (more egregiously) from 1.0.0-beta.7 to 1.0.0.

Let me know if I’m misguided here / if you’d like me to try to submit a PR!

Isn’t this the last checkbox of https://github.com/rust-lang/cargo/issues/12425#issuecomment-2186198258?

From what I understand of the current semantics, they are what they should be.

--breaking is a “force” flag to update a dependency when a ^ operator would otherwise prevent the update.

At this time, updating through pre-releases is not a breaking change, so --breaking does not apply (see #6016, #2222, #3263). If you work around that with =, then --breaking still does not apply.

To update pre-releases that use ^, then we need the feature that is left out of this Issue, a way to bump the minimum of version reqs to what is in the lockfile (see #10498).

`“^2.0.0-beta.1” when “^2.0.0-beta.2” is available

cargo update --breaking should not change compatible version requirements

"^2.0.0-beta.1" when `“^2.0.0” is available

cargo update --breaking should not change compatible version requirements

`“^2.0.0-beta.1” when “^3.0.0-beta.1”

We need to clarify the semantics for how to handle this case

Should it also upgrade from 1.0.0-alpha.1 to 1.0.0-beta.1? How about from 1.0.0-beta.1 to 1.0.0-pre.1? And But what if 1.0.0-pre.1 is older than 1.0.0-beta.1?

I reckon we should try to stick to semver’s rules here (https://semver.org/#semantic-versioning-specification-semver):

Precedence for two pre-release versions with the same major, minor, and patch version MUST be determined by comparing each dot separated identifier from left to right until a difference is found as follows:

  1. Identifiers consisting of only digits are compared numerically.

  2. Identifiers with letters or hyphens are compared lexically in ASCII sort order.

  3. Numeric identifiers always have lower precedence than non-numeric identifiers.

  4. A larger set of pre-release fields has a higher precedence than a smaller set, if all of the preceding identifiers are equal.

Example: 1.0.0-alpha < 1.0.0-alpha.1 < 1.0.0-alpha.beta < 1.0.0-beta < 1.0.0-beta.2 < 1.0.0-beta.11 < 1.0.0-rc.1 < 1.0.0.

Personally I don’t know if we took the right approach to pre-releases in #14250 — I left more of a comment on that issue (https://github.com/rust-lang/cargo/issues/14178#issuecomment-2275688972), but my overall proposal is:

  1. If version is currently stable / non-pre-release, then ignore pre-releases
  2. If version is currently a pre-release, upgrade it as normal — including upgrading to the stable version!

This, what I would consider a bug, showed up with derive-more when cargo upgrade --breaking first failed to upgrade from 1.0.0-beta.6 to 1.0.0-beta.7, and again (more egregiously) from 1.0.0-beta.7 to 1.0.0.

Let me know if I’m misguided here / if you’d like me to try to submit a PR!

The non-breaking update does not fail, but outputs something like this:

[UPDATING] `dummy-registry` index
[LOCKING] 0 packages to latest compatible versions
[NOTE] pass `--verbose` to see 3 unchanged dependencies behind latest

The breaking update currently only outputs this:

[UPDATING] `dummy-registry` index

So maybe the breaking one should add:

[UPGRADING] 0 packages to latest compatible versions

as well as a feature to list the other packages using --verbose?

@VorpalBlade I just ran into the same issue as you, seems like updating non-breaking versions (https://github.com/rust-lang/cargo/issues/12425#issuecomment-2211987113) is now being tracked here: https://github.com/rust-lang/cargo/issues/10498.

Yes, I agree the logic should be:

  • Use the highest precedence release (i.e. not prerelease) that has higher precendence. If there are none, continue.
  • If the version is a prerelease, use the highest precedence prerelease that has higher precedence and is within the same release (i.e. same major, minor, and patch). If there are none, continue.
  • Keep the version as is.

(Note that even though I’m writing that, that’s not what I would like to use. I would like something like #14372. But it could be a separate flag.)

And would agree with @djc that moving to a higher version is better than taking into account age — though really there should probably be a lint for anyone trying to push a lower prerelease version than is already published (if there isn’t already?)

I would say yes. If you decide to go with a prerelease, it usually means you want the latest thing. Also, I remember having read somewhere that prereleases are incompatible between each other, so --breaking should bump to the latest because each version is incompatible.

I think all of those should be in scope for --breaking.

These are alternatives I had considered that help give an idea of what I mean by fitting into cargo.

cargo update always modifies Cargo.toml

  • This would be a breaking change
  • This would get in the way of people intentionally keeping separate versions from version requirements

For my use case of cargo upgrade (which cargo update --breaking has been advertised as a replacement of) this would be the mode I want. I’m fine with it being an optional flag to do so, but currently cargo update --breaking is a step back functionality wise (performance is of course better though).

Quoting myself from https://github.com/rust-lang/cargo/issues/14204 (apparently @epage prefers feedback in this issue instead):

I was led to believe that cargo +nightly -Zunstable-options update --breaking should be a drop-in replacement for cargo upgrade -i. But it isn’t. Specifically it doesn’t seem to upgrade Cargo.toml unless there is a breaking change.

I use cargo upgrade to upgrade to new minor versions in Cargo.toml too. There are several reasons for this:

  • As I’m on the latest version when developing, I can’t be sure I’m not relying on some new API that didn’t exist in the version that is declared in Cargo.toml. While I believe there is some “min-version” flag to test with, I prefer to just force everything the latest. (No I don’t care about LTS distros like Debian etc, not one bit, nor about people who are not on the most recent stable Rust. My MSRV is N.)
  • If there are any security issues in a dependency I’d also prefer to force the latest version there too.

As such it seems I will probably have to port cargo-upgrade from cargo-edit to sparse registry myself at some point. When and if I have time. As cargo update --breaking just doesn’t cut it for my use case.

That is covered in https://github.com/rust-lang/cargo/issues/12425#issuecomment-1659453818

Somewhere between deferred and rejected (speaking for myself): Support in cargo update for writing to the manifest for non-breaking changes, like bulk compatible upgrades of version requirements (ie a -save flag) which was one of our lower priority workflows. A --save flag is more about updating versions for your dependents, which while important for having valid lower-bounds on version requirements, doesn’t fit with the existing model of cargo update. Maybe in the future we can find a way to express this in cargo update that fits with how it works or maybe another command can take on this role. We just aren’t wanting to distract our efforts for handling most of the use cases to handle this one

We cover those for cargo update --breaking but I’m not seeing those covered for cargo update --breaking [SPEC]. We only test just-foo shared ws which are all candidates for upgrading. I’m concerned about the semantics of a user explicitly naming a package that --breaking doesn’t apply to.

It might also be of interest to have a mixed workspace, where one package depends on renamed without a rename while another depends on it with a rename. This would need to be a separate test as this wouldn’t error but the other case could potentially (depending on what semantics we want to give it)

. Also, I suppose we shouldn’t do a full ops::update_lockfile(), but only update the dependencies that got modified by ops::update_manifests().

Yes, I believe the feature as specified says we shouldn’t do compatible updates. We probably want the semantics of cargo check after editing Cargo.toml in case a requirement change causes a cascade of compatible updates.

The other problem with modifying the manifests before anything else is that we will leave the workspace in a half-updated state on failure.

My assumption was that we’d edit the Dependencys for workspace members (maybe all path sources?) and on success, we’d find all Cargo.toml all dependency blocks, including workspace.dependencies, and edit them then.

FYI on zulip I brought up the idea of a pedantic machine-applicable (ie --fixable) lint to flag imprecise dependencies.

Can --incompatible be used in conjunction with --package? Just recall https://github.com/rust-lang/cargo/issues/11974 and not sure about the implication of cargo update --incompat -p <pkgid>.

Yes, we’d update the version requirement, if an incompatible version exists, and then run the normal code.