tarpaulin: Compile error: linking with `cc` failed

Okay took me quite a few hours fixing and testing everything, but I fixed it. Let me explain thing for other people that have similar issues. For initial problems I had see: https://github.com/xd009642/tarpaulin/issues/517#issuecomment-874089885 But I’ll summarize below as well.

After checking the issue out again I stumbled on this problem again:

It works locally but not on CI. Also tried running docker container locally but could not reproduce it. Although CI consistently fails. I first hit (I think) and infinite loop because my build hit the limit of 60min (usually take about 16min to run CI). But did not get any errors from this.

Note that before testing I upgraded to cargo-tarpaulin v0.18.2

My CI got stuck while compiling/linking (at end of compiling) but returned no errors. Locally it seemed to work just fine.

After testing I found out that the CI docker container was running out of RAM. This is the reason the compiler (mainly the linker) got stuck. Because when compiling a lot of test modules in parallel it starts multiple linking jobs. Each linking job was using about 2 GB or memory (don’t know why, but it did, this was with GNU ld). So 2 GB*8 = 16GB, this was to much for the CI. I switched to the lld (LLVM linker) to see if this fixed the issue, it did change some thing and I fixed some issues with this in the past.

To switch to lld I used: .cargo/config.toml:

[target.x86_64-unknown-linux-gnu]
rustflags = [
    "-C", "link-arg=-fuse-ld=lld",
]

Add this file to the root of your project. So:

my_project
├─ target/..
├─ src or workspace folders
├─ Cargo.toml
└─ .cargo
   └─ cargo.toml

This might solve or create new problems. But could be a way out if you are stuck.

Switching back to ld linker did solve the Broken pipe (os error 32) with cc in this case (I know this did not solve issue in the past, but it did now). I also started the coverage test in a machine with more RAM and after some other fixes I got it working again. 🎉

In my case, my lib referencing an external library, while cargo build succeeds, cargo tarpaulin fails.

My lib.rs:

#[repr(C)]
pub struct PyObject {
    pub ob_refcnt: i32,
}

extern "C" {
    #[cfg_attr(PyPy, link_name = "_PyPy_NoneStruct")]
    static mut _Py_NoneStruct: PyObject;
}

pub unsafe fn Py_None() -> *mut PyObject {
    &mut _Py_NoneStruct
}

root@task20679-78cfbd5754-xlv8z:/home/test/abc# cargo tarpaulin --verbose
Sep 30 17:38:49.502 DEBUG cargo_tarpaulin: set up logging
Sep 30 17:38:49.502  INFO cargo_tarpaulin::config: Creating config
Sep 30 17:38:49.534  INFO cargo_tarpaulin: Running Tarpaulin
Sep 30 17:38:49.534  INFO cargo_tarpaulin: Building project
Sep 30 17:38:49.534  INFO cargo_tarpaulin::cargo: Cleaning project
   Compiling abc v0.1.0 (/home/test/abc)
     Running `rustc --crate-name abc --edition=2018 src/lib.rs --error-format=json --json=diagnostic-rendered-ansi --emit=dep-info,link -C embed-bitcode=no -C debuginfo=2 --test -C metadata=3a4832eb8b7395b3 -C extra-filename=-3a4832eb8b7395b3 --out-dir /home/test/abc/target/debug/deps -C incremental=/home/test/abc/target/debug/incremental -L dependency=/home/test/abc/target/debug/deps -C debuginfo=2 --cfg=tarpaulin -C link-dead-code`
error: Broken pipe (os error 32)
error: could not compile `abc`

Caused by:
  process didn't exit successfully: `rustc --crate-name abc --edition=2018 src/lib.rs --error-format=json --json=diagnostic-rendered-ansi --emit=dep-info,link -C embed-bitcode=no -C debuginfo=2 --test -C metadata=3a4832eb8b7395b3 -C extra-filename=-3a4832eb8b7395b3 --out-dir /home/test/abc/target/debug/deps -C incremental=/home/test/abc/target/debug/incremental -L dependency=/home/test/abc/target/debug/deps -C debuginfo=2 --cfg=tarpaulin -C link-dead-code` (exit status: 1)
Sep 30 17:38:50.165 ERROR cargo_tarpaulin: Failed to compile tests! Error: abc: linking with `cc` failed: exit status: 1
Error: "Failed to compile tests! Error: abc: linking with `cc` failed: exit status: 1"
root@task20679-78cfbd5754-xlv8z:/home/test/abc#

Running command rustc --crate-name adc ... reports error:

undefined reference to `_Py_NoneStruct'

Summary for solving this (and maybe other) issues: These fixes can be used independently. (are in rough order of how best to check)

1. Clear caches / do a clean build cargo clean https://github.com/xd009642/tarpaulin/issues/517#issuecomment-681194362
2. Update Tarpaulin, make sure you are running the latest version (there might be an alpha out depending on when you read this) cargo install cargo-tarpaulin to auto update or cargo install cargo-tarpaulin --version 0.18.0-alpha3 to force a particular version.
3. Make sure you have enough RAM (especially if you are running in CI) https://github.com/xd009642/tarpaulin/issues/517#issuecomment-923429731
4. Switch to stable or nightly toolchain when possible cargo +nightly tarpaulin
5. Run with --verbose flag set ( fixed in #787)
6. Try running less jobs concurrently, use --jobs 2 flag (pick a number less then amount of CPU cores/virtual threads the machine has)
7. Make sure you are not missing libraries https://github.com/xd009642/tarpaulin/issues/517#issuecomment-912070823
8. Switch linker ld (GNU, default) or lld (LLVM, see this )
9. Give grcov a try, this might surface errors that might fix Tarpaulin too. (see below for more info)
10. … (there might be more ways to solve this)

export CARGO_INCREMENTAL=0
export RUSTFLAGS="-Zinstrument-coverage -Zprofile -Ccodegen-units=1 -Copt-level=0 -Clink-dead-code -Coverflow-checks=off -Zpanic_abort_tests -Cpanic=abort"
export LLVM_PROFILE_FILE="myapp-%p-%m.profraw"
export RUSTDOCFLAGS="-Cpanic=abort"
export RUSTUP_TOOLCHAIN="nightly"
cargo clean
cargo +nightly build -vv --jobs 4
cargo +nightly test -vv --jobs 4
# or use `-t html` instead
grcov . -s . --binary-path ./target/debug/ -t lcov --branch --ignore-not-existing -o ./lcov.info

# Upload the `lcov.info` file for coverage data ( codecov for example )
./codecov.sh -f ./lcov.info -e CI=true

Little shout-out to @xd009642 his appearance in Rustacean Station Podcast episode 037. 😃 Check it out if you want to know more about Tarpaulin and its future. Looking forward to LLVM update. 😉

@ArekPiekarz

@orhun Unfortunately I already had libxkbcommon installed when I tried to run cargo tarpaulin.

My point was: in my case “something” was missing and cargo-tarpaulin was failing due to that. I used grcov and it revealed the error. Your issue looks the same.

Hmm, interesting. This might be my problem too, but have to test. I got the problem after adding more tests, but might have enabled a feature or something that needed an other lib. This might also explain why it works in my PC but not in CI pipeline.

Thanks for sharing! Will have a look and see if that fixes it.

For a number of people hopefully once this is closed (cargo PR done by alex) and the next cargo is released this issue should go away https://github.com/rust-lang/cargo/issues/9220