cargo: Duplicate artifact tracking issue.

Documenting Servo produces lots of warnings like:

warning: output filename collision.
The lib target `parking_lot` in package `parking_lot v0.8.0` has the same output filename as the lib target `parking_lot` in package `parking_lot v0.7.1`.
Colliding filename is: /repo/target/doc/parking_lot/index.html
The targets should have unique names.
Consider changing their names to be unique or compiling them separately.
This may become a hard error in the future; see <https://github.com/rust-lang/cargo/issues/6313>.

This simply becoming a hard error is not acceptable. Having multiple crates with the same name in the dependency graph is a use-case that Cargo needs to support. Even this warning arguably should not exist: this situation is normal, it’s up to Cargo and rustdoc together to figure something out.

If you are going to make something a terminating error, could you please make sure that the message sets out the steps that we programmers should follow to fix the error. Otherwise it is like “You have been bad. Compilation prohibited. Goodbye.” and then you get all these issues and forum posts.

Collisions are almost certainly bad behavior that will cause problems, so rejecting it is probably the right thing to do.

In this type of reasoning, please consider: who’s behavior is bad? Who is responsible to fix it? Warnings and errors are only appropriate when the end user is responsible.

Even if the current output is arguably wrong/incomplete, erroring would be a regression.

For example, Windows and macOS will probably never have case-sensitive filesystems by default

And yet that’s exactly what should happen if this is to be solved cleanly. They caused the problem, likely before they knew case-insensitive FSs were a major PITA. Any fix other than that is nothing more than a bandaid, and thus solves it only partially, for the project doing the fixing.

We can reword the warning to convey “this is a bug”

@ehuss Yes, rewording would be good. As a user of cargo doc when I see this:

The targets should have unique names.
Consider changing their names to be unique or compiling them separately.
This may become a hard error in the future; see <https://github.com/rust-lang/cargo/issues/6313>.

This message is very strongly suggesting that I did something wrong and I should consider changing the names of “targets”. (What are targets in this context? Do I even have any control over them?)

But there is nothing wrong with having multiple versions of the same crate in a dependency graph. Cargo was designed from the very beginning to support that scenario.

So it is not users who are doing something wrong, but the historical design of rustdoc (or is it cargo doc?) who incorrectly assumes that crate names are unique and can be used as-is as directory names.

For an eventual fix, maybe cargo doc could detect this situation and add a version number the directory names in that case. cargo vendor already does this.

Until then, yes then emitting a message to warn users of the data loss is better the silently overwriting. But blaming users (with a “we’ll break your stuff” threat!) is very much not the right response.

As described here I have a crate in which I’d like to produce both a dylib and a cdylib file.

This seems impossible at the moment. Specifically, Cargo writes both dylibs to the same filename i.e. it seems to write one and then it seems to overwrite it with the other (can’t tell you which is which without digging into the final file though). If it is possible to have Cargo produce both cdylib and dylib, I’ve yet to find out how, as adding crate-type = ["cdylib", "dylib"] to the [lib] section of the crate gives a compile warning leading to this issue.

In https://github.com/libp2p/rust-libp2p/pull/4248, we’ve hit what I think is a more common usecase than what is described in the current PR description:

PDB collisions on Windows. If the package has a binary and a dylib library of the same name, then the .pdb file for each target will have the same name. This is easy to hit if you have a proc-macro package with a binary.

We have a crate that has a binary that functions as a web-server and a library target that is compiled to WASM and served by said web-server as an app. On windows, this results in a PDB filename collision.

Similar to @espindola, I have a crate that has

[lib]
crate-type = ["lib", "dylib"]

and when running cargo test --all from my workspace, I get the collision, but going to each dependant crate and running cargo test works fine. What is the standard operating procedure for something like this? It doesn’t seem like that unusual of a use-case.

A situation where we can hit this issue is when using resolver version 2 and a crate with different features in build-dependencies and dependencies:

[workspace]
resolver = "2"

[package]
name = "api"
version = "0.1.0"

[build-dependencies]
swc_common = { path = "./swc_common", default-features = true }

[dependencies]
swc_common = { path = "./swc_common", default-features = false }

and swc_common has

[lib]
crate-type = ["lib", "dylib"]

I think that at least for my original use case, which is to produce native libraries compatible with c for ffi, https://github.com/rust-lang/cargo/issues/10083 may have solved it. I no longer need to restructure my code into leaf nodes (which was only possible when using cdylibs anyway) because I can pass --crate-type=cdylib to cargo rustc

As @SimonSapin suggested, I solved this by moving any cdylibs that the crates produce to “leaf nodes” of the dependency graph. In my case, instead of creating simple trivial helper crates that produce cdylibs, I simply stopped emitting cdylibs for transitive dependencies - the only cdylib that gets produced is in one FFI crate.

As a work-around for this case, could you afford to make the dylib a separate Cargo package? It would then have a separate name, avoiding collisions.

My program depends on tungstenite 0.17.2 and redis 0.21.5. They depend on sha-1 and sha1 respectively.

warning: output filename collision.
The lib target `sha1` in package `sha1 v0.6.1` has the same output filename as the lib target `sha1` in package `sha-1 v0.10.0`.
Colliding filename is: /home/emerald/discord/collection_watcher/target/doc/sha1/index.html
The targets should have unique names.
This is a known bug where multiple crates with the same name use
the same path; see <https://github.com/rust-lang/cargo/issues/6313>.

I have a (possibly) different problem with this issue. It seems to be caused by a library that is built as a cdylib and which is also required in a build script. A minimal example can found here: cargo_collision.zip Building this workspace produces the duplicate artifact warning (but only when building in release mode).

In my real setup, I also get a linking error, where the symbols from the communication library cannot be found (Again, only in Release builds, so these two issues seem to be connected). Both issues started appearing once I added the communication lib as a build dependency. This also happened once with the example given above, but I’m not able to reproduce it right now.

Does anyone have an idea what’s causing these problems?

Some behavior around this may have changed, causing a failed to remove file error of a duplicated bin target name, as seen in this crater log. That’s the only failure like this I see. (rust-lang/rust#87749)

I’m currently not treating this as a bug, since I think these warnings have been around for awhile, but wanted to note it somewhere.

There are two copies of fst in your dependency graph. There is the root fst, and then the one from crates.io (from various dependencies, such as fst-bin and regex-automata). They both output to the doc/fst/ directory. cargo tree --workspace -i https://github.com/rust-lang/crates.io-index#fst:0.4.6 will show where the crates.io copy comes from.

I’m not sure what you are running cargo doc for, but some options are to use --no-deps, or to patch the crates.io copies like this in the root Cargo.toml:

[patch.crates-io]
fst = {path = "."}

@ehuss Thanks, that seems to work-around the error, but I still get this warning:

warning: output filename collision.
The lib target `fst` in package `fst v0.4.6` has the same output filename as the lib target `fst` in package `fst v0.4.6 (/home/andrew/rust/fst)`.
Colliding filename is: /home/andrew/rust/fst/target/doc/fst/index.html
The targets should have unique names.
This is a known bug where multiple crates with the same name use
the same path; see <https://github.com/rust-lang/cargo/issues/6313>.

Reproduction:

$ git clone -b ag/doc-false https://github.com/BurntSushi/fst
$ cd fst
$ cargo doc --all

Version info:

$ rustc --version
rustc 1.54.0-nightly (c79419af0 2021-06-04)
$ cargo --version
cargo 1.54.0-nightly (0cecbd673 2021-06-01)

@jjpe could you clarify how the root issue might be fixed otherwise? For example, Windows and macOS will probably never have case-sensitive filesystems by default, so it’s not clear how we could avoid this issue on those platforms.

Case-sensitive filenames also cause problems for other tools, like committing cargo doc output to git from a case-sensitive filesystem and later trying to access it from case-sensitive filesystem. This is the problem we’re currently hitting and will probably have to workaround by post-processing cargo doc output.

would be great if to create unique filenames to avoid the case-insensitive collision or skip these somehow.

I disagree. As annoying as it is, files need to stay human findable IMO. Having a filename that incorporates a hash of some kind (the most straightforward way to do this) makes that effectively impossible. The real issue is case-insensitive filesystems and a solution needs to be found to tackle the issue at the root rather than just applying hierarchies of patches to try to fix the collateral damage of that root issue.

I am curious is there a way to select which one of the colliding packages should be used in the doc gen in case of being in a secondary dependency especially if they are two different versions of the same package?

The logs seem to suggest both packages got documented. But only the first one seems to be there. Is there something more to it?

@Lokathor OK, I see what is happening. It is a consequence of having cdylib and panic="abort" with an integration test and a binary. For reasons, cdylibs don’t have a unique hash added to their filename. In this case, the “thorium” library needs to be built twice (once with panic=“abort” for main.rs, and once without for the test which doesn’t allow panic=“abort”), and both of those end up with the same filename.

I can’t think of an easy workaround for you, unfortunately.

I’m not sure what the best approach here is. Perhaps dylibs could be placed in unique directory names. Also, maybe Cargo could be more conservative what it builds for tests, since it is only linking against one lib type.

I’m trying to produce rlib, staticlib, and cdylib just to make sure that all modes are buildable (because I’ve had some tricky build problems once or twice already) and I’m getting a warning from cargo test that told me to come here.

Also, cargo test fails to run if I don’t run cargo build first because it tries to link to target/debug/deps/my_lib.dll and fails to find it.