swift-package-manager: Linking with lld stopped working on linux/Android since lld 13 and this repo enabled `--gc-sections`

Description

After dead-stripping was enabled in #4135, various people reported problems with using lld, though it wouldn’t always show up. It appears to be some change with lld 13, as lld 12 works fine, perhaps the commit @drodriguez linked (I’ll try reverting that commit and report back later).

Obviously, this is really a regression in how Swift interacts with lld 13 and 14, but I’m reporting it here first as we’ll probably want to amend SPM 5.7 quickly before the underlying lld issue is figured out. @keith, maybe we should disable stripping on ELF if the linker is lld for now?

@3405691582, can you reproduce on OpenBSD?

Expected behavior

Linking to work fine with lld, as it does with gold.

Actual behavior

Several linker errors like this show up instead:

ld.lld: error: undefined hidden symbol: __start_swift5_protocols
>>> referenced by SwiftRT-ELF.cpp
>>>               /home/butta/swift-DEVELOPMENT-SNAPSHOT-2022-07-25-a-ubuntu20.04/usr/lib/swift/linux/x86_64/swiftrt.o (swift_image_constructor())

Steps to reproduce

  1. Clone swift-argument-parser
  2. Remove a file because of some unrelated bug: rm Sources/ArgumentParserTestHelpers/TestHelpers.swift
  3. Build the package in release mode with lld: ../swift-DEVELOPMENT-SNAPSHOT-2022-07-25-a-ubuntu20.04/usr/bin/swift build -v -c release -Xswiftc -use-ld=lld

Swift Package Manager version/commit hash

latest Jul. 25 trunk snapshot and 5.7 Jul. 23 snapshot

Swift & OS version (output of swift --version && uname -a)

Swift 5.8 and 5.7 on Ubuntu 20.04 x86_64

About this issue

  • Original URL
  • State: closed
  • Created 2 years ago
  • Reactions: 1
  • Comments: 23 (12 by maintainers)

Commits related to this issue

Most upvoted comments

Opened apple/swift#60406 with my Android test info, let’s take discussion of this underlying lld issue there.

For reference this post also covers many pieces of this change https://maskray.me/blog/2021-01-31-metadata-sections-comdat-and-shf-link-order

For LinkerSections/function_sections.swift I investigated and the problem with that test specifically is it’s validating the map file format from ld, which lld does not replicate. The behavior appears to still be “passing” if you adapt it to the lld output.

Do you happen to have any of the example failures around still?

Sadly no. I am recovering --gc-sections internally and I am building again. It is from a completely different branch, but I don’t think it is very interesting to know if it was really failing with a branch from back in March.

It would be good to know what else failed for you and why, hopefully not a result of this fix.

I do not think it was a result of the fix. I think when I tried, the fix was very good, but not enough. Hopefully it will all be green and that’s the fix.

Here’s a fix for this, it’s not fully complete yet because the problem is the R directive I’m adding here wasn’t supported until clang 13+, and binutils 2.36, so I think I need to conditionalize this somehow based on those versions. https://github.com/apple/swift/pull/60357

Can you test with your real world example to make sure this works there too?