risingwave: performance regression (for debug & single binary) of backtrace capturing after bumping toolchain on macOS

Describe the bug

I found risingwave becomes significantly slow on my mac. e.g., e2e_test/batch/basic/array.slt.part takes 30+ seconds. After git bisect, I found the regression happens after https://github.com/risingwavelabs/risingwave/pull/6025.

To Reproduce

No response

Expected behavior

No response

Additional context

~~@skyzh can’t reproduce. CI also works fine. Need someone else to confirm it.~~ Confirmed it happens on debug & single binary mode.

https://github.com/rust-lang/rust/issues/104388

About this issue

Original URL
State: closed
Created 2 years ago
Comments: 24 (24 by maintainers)

Most upvoted comments

TImeout is a separate issue. reopen.

skyzh on Nov 15, 2022

Closing this issue in favor of #4077 and #6205.

The slowness is acknowledged by the doc of Backtrace::capture, and our take on it would be to avoid unnecessary backtrace capture via #4077.

The segfault is worse and tracked by #6205.

xiangjinwu on Nov 14, 2022

cc. @BowenXiao1999

fuyufjh on Nov 2, 2022

skyzh on Nov 1, 2022

Shall we file a bug report to rust-lang? (Though I believe this is more of an LLVM bug / LLVM upgrade)

skyzh on Nov 1, 2022

In multi-binary:

backtrace captured in 1ms
backtrace captured in 0ms
backtrace captured in 0ms
backtrace captured in 0ms
backtrace captured in 0ms
backtrace captured in 0ms
backtrace captured in 0ms
backtrace captured in 0ms
backtrace captured in 0ms
backtrace captured in 0ms

In single-binary:

backtrace captured in 393ms
backtrace captured in 369ms
backtrace captured in 370ms
backtrace captured in 365ms
backtrace captured in 362ms

Probably a bug with the Rust compiler, but I would still recommend not returning so many errors.

skyzh on Oct 31, 2022

<del>single-binary doesn’t work well, while multi-binary looks good. Is there anything that might go wrong with the madsim update? cc @wangrunji0408 </del> there should be no problem with madsim

skyzh on Nov 1, 2022