rustup: "Can't find crate for `rustc_*`" when cross-compiling a crate with `rustc_private` on

Problem

In our project, we are using rustc components. When building for a x86_64 target it’s fine. However, when setting the target to i686-unknown-linux-gnu, cargo (rustc) starts complaining about components not existing.

I think this is a problem due to components installation through rustup. It doesn’t look to be installing them for the specified target.

The initial install is done through the following command.

$> rustup toolchain install nightly-2023-01-26 --target i686-unknown-linux-gnu --component rust-src rustc-dev llvm-tools-preview 
info: syncing channel updates for 'nightly-2023-01-26-x86_64-unknown-linux-gnu'
info: latest update on 2023-01-26, rust version 1.69.0-nightly (c18a5e8a5 2023-01-25)
info: downloading component 'cargo'
info: downloading component 'clippy'
info: downloading component 'llvm-tools'
info: downloading component 'rust-docs'
info: downloading component 'rust-src'
info: downloading component 'rust-std' for 'i686-unknown-linux-gnu'
info: downloading component 'rust-std'
info: downloading component 'rustc'
 68.3 MiB /  68.3 MiB (100 %)  45.8 MiB/s in  1s ETA:  0s
info: downloading component 'rustc-dev'
111.1 MiB / 111.1 MiB (100 %)  44.5 MiB/s in  2s ETA:  0s
info: downloading component 'rustfmt'
info: installing component 'cargo'
info: installing component 'clippy'
info: installing component 'llvm-tools'
 32.8 MiB /  32.8 MiB (100 %)  23.6 MiB/s in  1s ETA:  0s
info: installing component 'rust-docs'
 19.4 MiB /  19.4 MiB (100 %)  17.7 MiB/s in  1s ETA:  0s
info: installing component 'rust-src'
info: installing component 'rust-std' for 'i686-unknown-linux-gnu'
 28.1 MiB /  28.1 MiB (100 %)  20.7 MiB/s in  1s ETA:  0s
info: installing component 'rust-std'
 30.1 MiB /  30.1 MiB (100 %)  20.5 MiB/s in  1s ETA:  0s
info: installing component 'rustc'
 68.3 MiB /  68.3 MiB (100 %)  23.2 MiB/s in  2s ETA:  0s
info: installing component 'rustc-dev'
111.1 MiB / 111.1 MiB (100 %)  21.6 MiB/s in  5s ETA:  0s
info: installing component 'rustfmt'

  nightly-2023-01-26-x86_64-unknown-linux-gnu installed - rustc 1.69.0-nightly (c18a5e8a5 2023-01-25)

However, during building rustup complaints as follows.

$> cargo build --target=i686-unknown-linux-gnu
...
error[E0463]: can't find crate for `rustc_middle`
  --> compiler/src/lib.rs:19:1
   |
19 | extern crate rustc_middle;
   | ^^^^^^^^^^^^^^^^^^^^^^^^^^ can't find crate
   |
   = help: maybe you need to install the missing components with: `rustup component add rust-src rustc-dev llvm-tools-preview`

Trying to install components again raises a conflict issue.

$> rustup component add --target i686-unknown-linux-gnu rust-src rustc-dev llvm-tools-preview  
info: component 'rust-src' is up to date
info: downloading component 'rustc-dev' for 'i686-unknown-linux-gnu'
info: installing component 'rustc-dev' for 'i686-unknown-linux-gnu'
128.4 MiB / 128.4 MiB (100 %)  21.6 MiB/s in  5s ETA:  0s
info: rolling back changes
error: failed to install component: 'rustc-dev-i686-unknown-linux-gnu', detected conflict: 'lib/rustlib/rustc-src/rust/Cargo.lock'

Originally, we are using rust-toolchain.toml to define the components and targets. In both cases, these issues exist. Also tried installing it side by side the x86_64 target and the same issue exists.

Steps

  1. Create a project using rustc components.
  2. Create a rust-toolchain.toml to override the toolchain and define components and targets as follows.
[toolchain]
channel = "nightly-2023-01-26"
components = ["rust-src", "rustc-dev", "llvm-tools-preview"]
targets = ["i686-unknown-linux-gnu"]
  1. Build using cargo build --target=i686-unknown-linux-gnu.

Possible Solution(s)

No response

Notes

No response

Rustup version

rustup 1.25.2 (17db695f1 2023-02-01)

Installed toolchains

Default host: x86_64-unknown-linux-gnu
rustup home:  ...

installed toolchains
--------------------

stable-x86_64-unknown-linux-gnu
nightly-2023-01-26-x86_64-unknown-linux-gnu
nightly-x86_64-unknown-linux-gnu (default)

installed targets for active toolchain
--------------------------------------

i686-unknown-linux-gnu
x86_64-unknown-linux-gnu

active toolchain
----------------

nightly-2023-01-26-x86_64-unknown-linux-gnu (overridden by '.../rust-toolchain.toml')
rustc 1.69.0-nightly (c18a5e8a5 2023-01-25)

About this issue

  • Original URL
  • State: closed
  • Created 4 years ago
  • Reactions: 2
  • Comments: 17 (9 by maintainers)

Commits related to this issue

Most upvoted comments

It’d definitely be nice if a follow-up to https://github.com/rust-lang/rust/issues/70838 can be filed. If it’s not a valid use case to cross compile with rustc-dev, I think the release team should simply ban it from appearing on the server. _https://github.com/rust-lang/rustup/issues/3255#issuecomment-2430572528_

Yeah, I think it would make sense to omit rustc-dev-aarch64-unknown-linux-gnu from the rustup manifest for hosts other than aarch64-unknown-linux-gnu. _https://github.com/rust-lang/rust/issues/132728#issuecomment-2462950628_

Following the previous comments, I’m going to close this issue in favor of https://github.com/rust-lang/rust/issues/132728, as this indeed looks more like a release-related problem.

Thanks for testing it out. I made an issue in the main Rust repo: https://github.com/rust-lang/rust/issues/132728

Thanks, great, so here’s the minimal reproducible repo: https://github.com/tombh/nightly_cross_minimal_repro

Pasting main.rs:

#![feature(rustc_private)]

extern crate rustc_ast;
extern crate rustc_driver;
extern crate rustc_macros;

fn main() {
    println!("{}", rustc_ast::node_id::CRATE_NODE_ID)
}

The build commands are here: https://github.com/tombh/nightly_cross_minimal_repro/blob/main/.github/workflows/ci.yml

Pasting the most relevant part:

rustup toolchain install nightly-2024-09-30
rustup default nightly-2023-09-30
rustup target add aarch64-unknown-linux-gnu
rustup component add rust-src rustc-dev-aarch64-unknown-linux-gnu llvm-tools
...
cargo -v build --locked --release --target aarch64-unknown-linux-gnu

And here is the failing build: https://github.com/tombh/nightly_cross_minimal_repro/actions/runs/11463471346/job/31897390824#step:5:9

Also, just out of interest, here is a failing build with yesterday’s nightly (nightly-2024-10-21), it fails differently, it can’t find derive_where: https://github.com/tombh/nightly_cross_minimal_repro/actions/runs/11463505291/job/31897508604#step:5:9

I can also reproduce the build errors locally on my aarch64 machine using: cargo build --release --target x86_64-unknown-linux-gnu.

If you or anybody else can also reproduce this then I’ll report it on https://github.com/rust-lang/rust too.

Thanks for looking into this!

So if I’m understanding the original bug report the issue arises before, and indeed can be reproduced without, the failed to install component error. Copying from the parent comment, this is the error that both @momvart and I are seeing:

$> cargo build --target=i686-unknown-linux-gnu
...
error[E0463]: can't find crate for `rustc_middle`
  --> compiler/src/lib.rs:19:1
   |
19 | extern crate rustc_middle;
   | ^^^^^^^^^^^^^^^^^^^^^^^^^^ can't find crate
   |
   = help: maybe you need to install the missing components with: `rustup component add rust-src rustc-dev llvm-tools-preview`

In other words, this is how I believe the error can be stated: there appears to be no way in which rust-src can be installed such that cross-compilation works on nightly Rust.

I agree that the subsequent error (ie failed to install component) is both an understandable next step (heeding the advice of the hint) and a reasonable response from rustup (either there is a further double-install bug, or there is a conflict with rust-toolchain.toml). But this is not the error in question.

https://github.com/Rust-GPU/rust-gpu is the original, underlying repo. The thing to note is that the core of the project is a custom codegen backend, which is what necessitates the rust-src, rustc-dev, llvm-tools components. And because it’s still in beta, they’ve never tried to release any packages, and so have never ventured into cross-compiling.

Ah yes, that was a typo in aarch63, but it was just a debugging line. I removed it and still get the same error: https://github.com/tombh/rust-gpu-cli/actions/runs/11386877927/job/31680155871#step:5:373