risingwave: bug(ci): multi-version tikv-jemalloc-sys break docker pipeline

Describe the bug

RisingWave got 2 version of tikv-jemalloc-sys built in docker pipeline. And it breaks CI.

cp ./target/release/build/tikv-jemalloc-sys-*/out/build/bin/jeprof /risingwave/bin/
image

Error message/log

<img width="1055" alt="image" src="https://github.com/risingwavelabs/risingwave/assets/22407295/21c68b59-7d1b-47a8-82a9-df8f30f9bb03">

About this issue

  • Original URL
  • State: closed
  • Created 10 months ago
  • Comments: 18 (18 by maintainers)

Commits related to this issue

Most upvoted comments

I found one copy is release build, while another is debug build 🤯

image

This doesn’t seem to be caused by [build-dependencies]

In my cases (#12625), tikv-jemalloc-sys was produced under [dependencies] but it still compiles both for release and debug, exactly as https://github.com/risingwavelabs/risingwave/issues/12315#issuecomment-1722358681 shows.

@xxchan Seems we can finally close this issue via #12333 ?

By default, cargo hakari produces the minimal set of features that can be unified across all possible platforms. However, in practice, most developers on a codebase use one of a few platforms. cargo hakari can run specific queries for a few platforms, producing better results for them.

Alternatively, this also exclude it from hakari

diff --git a/.config/hakari.toml b/.config/hakari.toml
index e998f5fea..9cdfbb3b0 100644
--- a/.config/hakari.toml
+++ b/.config/hakari.toml
@@ -35,6 +35,4 @@ third-party = [
     { name = "criterion" },
     { name = "console" },
     { name = "similar" },
-    # FYI: https://github.com/risingwavelabs/risingwave/issues/12315
-    { name = "tikv-jemalloc-sys", git = "https://github.com/risingwavelabs/jemallocator.git", rev = "64a2d9" },
 ]
diff --git a/src/compute/Cargo.toml b/src/compute/Cargo.toml
index 24d12151c..6e3f8674a 100644
--- a/src/compute/Cargo.toml
+++ b/src/compute/Cargo.toml
@@ -39,7 +39,6 @@ risingwave_storage = { workspace = true }
 risingwave_stream = { workspace = true }
 serde = { version = "1", features = ["derive"] }
 serde_json = "1"
-tikv-jemalloc-ctl = { git = "https://github.com/risingwavelabs/jemallocator.git", rev = "64a2d9" }
 tokio = { version = "0.2", package = "madsim-tokio", features = [
     "rt",
     "rt-multi-thread",
@@ -54,6 +53,10 @@ tonic = { workspace = true }
 tower = { version = "0.4", features = ["util", "load-shed"] }
 tracing = "0.1"
 
+[target.'cfg(unix)'.dev-dependencies]
+tikv-jemalloc-ctl = { git = "https://github.com/risingwavelabs/jemallocator.git", rev = "64a2d9" }
+
+
 [target.'cfg(not(madsim))'.dependencies]
 workspace-hack = { path = "../workspace-hack" }
 
diff --git a/src/tests/simulation/Cargo.toml b/src/tests/simulation/Cargo.toml
index 82992b8b0..01c20b172 100644
--- a/src/tests/simulation/Cargo.toml
+++ b/src/tests/simulation/Cargo.toml
@@ -46,11 +46,14 @@ serde_derive = "1.0.188"
 serde_json = "1.0.107"
 sqllogictest = "0.15.3"
 tempfile = "3"
-tikv-jemallocator = { workspace = true }
 tokio = { version = "0.2.23", package = "madsim-tokio" }
 tokio-postgres = "0.7"
 tracing = "0.1"
 tracing-subscriber = { version = "0.3", features = ["env-filter"] }
 
+[target.'cfg(unix)'.dev-dependencies]
+tikv-jemallocator = { workspace = true }
+
+
 [lints]
 workspace = true

The duplicated versions are with and without feature unprefixed_malloc_on_supported_platforms, which is target.'cfg(unix)'.dependencies.

I guess it’s because the build-dependency (introduced by hakari) doesn’t contain this feature.

We can see cargo does not unify features for dependencies and build-dependencies

[dependencies]
rand = { version = "0.8.5", default-features = false }

[build-dependencies]
rand = { version = "0.8.5", default-features = false, features = ["serde"] }


❯ cargo tree -p rand -e features -i              
rand v0.8.5
└── tmp v0.1.0 (/private/tmp/tmp)
    └── tmp feature "default" (command-line)

rand v0.8.5
└── rand feature "serde"
    [build-dependencies]
    └── tmp v0.1.0 (/private/tmp/tmp) (*)