swift-protobuf: Linking against static library that uses SwiftProtobuf causes linker errors

Hello! I’m seeing a pretty weird linker error related to SwiftProtobuf symbols. I’m going to describe the situation as much as possible.

We build a static library that contains some SwiftProtobuf generated code. This library is built for library distribution, since we want to support multiple Xcode and Swift versions.
When we integrate this library in a consumer, that builda SwiftProtobuf from source (we had problems with distributing SwiftProtobuf as a binary built with library distribution in the past since it would cause some weird Objective-C runtime data corruption), the compilation works fine.
When the app gets to linking stage, it throws the following errors because some symbols included in the prebuilt static library aren’t found in SwiftProtobuf. We are using the same version everywhere, so I’m pretty sure it’s not a version mismatch kind of error.

Undefined symbols for architecture x86_64:
  "_$s13SwiftProtobuf19_ProtoNameProvidingP17_protobuf_nameMapAA01_dH0VvgZTq", referenced from:
  "_$s13SwiftProtobuf26_MessageImplementationBaseP29_protobuf_generated_isEqualTo5otherSbx_tFTq", referenced from:
  "_$s13SwiftProtobuf7DecoderP14decodeMapField9fieldType5valueyAA01_bE0Vyqd__qd_0_Gm_SDy04BaseH0Qyd__AJQyd_0_GztKAA0e3KeyH0Rd__AA0e5ValueH0Rd_0_r0_lFTj", 
  "_$s13SwiftProtobuf7DecoderP15nextFieldNumberSiSgyKFTj", referenced from:
  "_$s13SwiftProtobuf7DecoderP24decodeSingularInt32Field5valueys0F0VSgz_tKFTj", referenced from:
  "_$s13SwiftProtobuf7DecoderP24decodeSingularInt64Field5valueys0F0VSgz_tKFTj", referenced from:
  "_$s13SwiftProtobuf7DecoderP25decodeSingularStringField5valueySSSgz_tKFTj", referenced from:
  "_$s13SwiftProtobuf7MessageP05protoC4NameSSvgZTq", referenced from:
  "_$s13SwiftProtobuf7MessageP06decodeC07decoderyqd__z_tKAA7DecoderRd__lFTq", referenced from:
  "_$s13SwiftProtobuf7MessageP13isInitializedSbvgTq", referenced from:
  "_$s13SwiftProtobuf7MessageP13unknownFieldsAA14UnknownStorageVvMTq", referenced from:
  "_$s13SwiftProtobuf7MessageP13unknownFieldsAA14UnknownStorageVvgTq", referenced from:
  "_$s13SwiftProtobuf7MessageP13unknownFieldsAA14UnknownStorageVvsTq", referenced from:
  "_$s13SwiftProtobuf7MessageP4hash4intoys6HasherVz_tFTq", referenced from:
  "_$s13SwiftProtobuf7MessageP8traverse7visitoryqd__z_tKAA7VisitorRd__lFTq", referenced from:
  "_$s13SwiftProtobuf7MessageP9isEqualTo7messageSbAaB_p_tFTq", referenced from:
  "_$s13SwiftProtobuf7MessagePxycfCTq", referenced from:
  "_$s13SwiftProtobuf7VisitorP13visitMapField9fieldType5value0G6NumberyAA01_bE0Vyqd__qd_0_Gm_SDy04BaseH0Qyd__AKQyd_0_GSitKAA0e3KeyH0Rd__AA0e5ValueH0Rd_0_r0_lFTj", "_$s13SwiftProtobuf7VisitorP23visitSingularInt32Field5value11fieldNumberys0F0V_SitKFTj", referenced from:
  "_$s13SwiftProtobuf7VisitorP23visitSingularInt64Field5value11fieldNumberys0F0V_SitKFTj", referenced from:
  "_$s13SwiftProtobuf7VisitorP24visitSingularStringField5value11fieldNumberySS_SitKFTj", referenced from:
  "_$s13SwiftProtobuf8_NameMapV0C11DescriptionO4sameyAEs12StaticStringV_tcAEmFWC", referenced from:
  "_$s13SwiftProtobuf8_NameMapV0C11DescriptionO8standardyAEs12StaticStringV_tcAEmFWC", referenced from:
ld: symbol(s) not found for architecture x86_64

(I redacted the from references since they contain internal code, but these symbols are referenced from the static library built with library distribution turned on)

SwiftProtobuf is built as a static library in our project, and I noticed that building it with SwiftProtobuf with library distribution in our project would fix the issue, but given the other issue raised above, we can’t do that.

Do you have any idea how what the problem could be here? I’m pretty sure someone has used SwiftProtobuf in a Swift SDK before, so this problem should have surfaced. Why would building some Swift proto generated code for library distribution include some symbols that are not present when linking it to a SwiftProtobuf which is not built for library distribution.

what OS you are developing on (Linux or macOS, including the version) macOS 10.15.
for macOS, what v7ersion of Xcode you are using (xcodebuild -version), for Linux, what version of Swift (swift --version) Xcode 12.0 but reproducible on 12.2.
what version of Swift is your code set to compile with (i.e. from project settings, etc.) Swift 5.2
what branch/tag of SwiftProtobuf you are using (1.0.0, etc.) 1.12.0 but can reproduce with 1.14.0 as well.

Thanks for the help!

About this issue

Original URL
State: open
Created 4 years ago
Reactions: 2
Comments: 34 (14 by maintainers)

Most upvoted comments

I am experiencing also multiple linking issues with the latest version of Firebase 8.1.0 that added SwiftProtobuf as a dependency. The SwiftProtobuf version is 1.17.0:

❌ Undefined symbols for architecture x86_64
❌   "method descriptor for SwiftProtobuf.Message.init() -> A", referenced from:
...

I managed to reproduce it in a sample project. The sample project integrates SwiftProtobuf in different ways, some of them work well some of them dont. The dependency tree is App -> Framework (static) -> FirebaseMLModelDownloader (static) -> SwiftProtobuf.

To run the sample project install xcodegen, xcbeautify and carthage. Then do:

git clone https://github.com/acecilia/SwiftProtobufLinkageIssue.git
cd SwiftProtobufLinkageIssue
make carthage_bootstrap # To build SwiftProtobuf as xcframeworks. This will take a couple of minutes

I tested multiple integration alternatives for SwiftProtobuf:

✅ Integrate the SwiftProtobuf binary distributed by Firebase, which is a static xcframework. This does not raise any errors and the app builds and runs well. It is possible to verify this by building the app: use the command make firebase_swift_protobuf
💥 Integrate the SwiftProtobuf binary built with carthage from apple/swift-protobuf, as a static xcframework. This results in linking errors. It is possible to verify this by building the app: use the command make static_swift_protobuf
✅ Integrate the SwiftProtobuf binary built with carthage from apple/swift-protobuf, as a static xcframework with library distribution enabled. This does not raise any errors and the app builds and runs well. It is possible to verify this by building the app: use the command make static_distribution_swift_protobuf
💥 Integrate the SwiftProtobuf binary built with carthage from apple/swift-protobuf, as a dynamic xcframework. This results in linking errors. It is possible to verify this by building the app: use the command make dynamic_swift_protobuf

TLDR:

✅ Both theSwiftProtobuf xcframework distributed by Firebase and the static one build with library distribution work well. Firebase probably builds SwiftProtobuf for library distribution, thus the initial question by @BalestraPatrick holds:

Why would building some Swift proto generated code for library distribution include some symbols that are not present when linking it to a SwiftProtobuf which is not built for library distribution.

💥 Building SwiftProtobuf with Carthage using the recommended way (without library distribution) leads to linking errors when building it as static or dynamic xcframework

So far I could not figure out the root cause of the linking issues, but it is clear that they only happen when BUILD_LIBRARY_FOR_DISTRIBUTION = NO

acecilia on Jun 5, 2021

There might be, but those distributions would need to hide the presence of SwiftProtobuf. Swift Protobuf does not support itself being built in library evolution mode, so it cannot be present in the public ABI of any framework that is built in library evolution mode. That means it can only ever be imported @_implementationOnly, and its types can never be public.

I face a similar issue. I develop an SDK that at the present time uses Protobuf as an internal implementation detail. We ship this framework both as a dynamic .xcframework as well as a static .xcframework for our consumers that then build their own SDK which encapsulates ours. Using the @_implementationOnly import approach has worked fine and we happily have been moving along.

I now face the situation where I need to make one of the Protobuf generated objects from this SDK public and accessible to the consumers of our SDK. Flipping the first bit pubic, of course begins the cascade where the the @_implemenationOnly imports fail. Reverting those then causes the warning:

Module 'SwiftProtobuf' was not compiled with library evolution support; using it means binary compatibility for 'MySDK' can't be guaranteed

I guess the main question I am asking here is… what is the reason that

Swift Protobuf does not support itself being built in library evolution mode

Is there anything that can be done to make that support possible?

paynerc on Jan 14, 2022

There might be, but those distributions would need to hide the presence of SwiftProtobuf. Swift Protobuf does not support itself being built in library evolution mode, so it cannot be present in the public ABI of any framework that is built in library evolution mode. That means it can only ever be imported @_implementationOnly, and its types can never be public.

Lukasa on Dec 29, 2020

I’m trying to ship one binary Swift framework that uses SwiftProtobuf. I think this should be a pretty common use case. It’s hard for me to believe that there’s isn’t a single SDK out there that uses SwiftProtobuf and is built for distribution as a XCFramework?

BalestraPatrick on Dec 29, 2020