electron-builder: Hardened runtime causes app to crash on open. (Signed and notarized)
-
Versions:
electron:5.0.6electron-builder:21.0.11electron-notarize:0.1.1electron-webpack:2.7.4Working on:MacOS Catalina 10.15 Beta 3 (19A501i)Xcode:Xcode 11 beta 3 -
What I’m trying to do: Sign and notarize an electron, web-packed, react desktop application for distribution outside the mac store.
-
Problem and exact case of error:
- Build the app unsigned, unnotarized, no hardened runtime: runs ✅
- Build the app signed, no hardened runtime: runs ✅
- Build the app signed, hardened runtime: (build and sign successful) error below when opening app ❌
- Build the app signed, hardened runtime, notarize: (build, sign and notatrize successful) error below when opening app ❌
- In both the error cases I’ve run: 1/ Verify code signing ✅
test.app: valid on disk
test.app: satisfies its Designated Requirement
2/ Verify code notarization ✅
test.app: accepted
source=Notarized Developer ID
origin=Developer ID Application: XXX, Inc. (XXXXXXXXXX)
- Entitlements for mac
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
<key>com.apple.security.cs.allow-unsigned-executable-memory</key>
<true/>
</dict>
-
Target: darwin
-
Error dump
Process: Test [7467]
Path: /Users/USER/Documents/*/Test.app/Contents/MacOS/Test
Identifier: ai.XXXX.desktop
Version: 0.0.4 (0.0.4)
Code Type: X86-64 (Native)
Parent Process: ??? [1]
Responsible: Test [7467]
User ID: 501
Date/Time: 2019-07-10 16:44:52.073 -0400
OS Version: Mac OS X 10.15 (19A501i)
Time Awake Since Boot: 27000 seconds
Time Since Wake: 11000 seconds
System Integrity Protection: enabled
Crashed Thread: 0 Dispatch queue: com.apple.main-thread
Exception Type: EXC_BAD_ACCESS (Code Signature Invalid)
Exception Codes: 0x0000000000000032, 0x00001a9c2b202040
Exception Note: EXC_CORPSE_NOTIFY
Termination Reason: Namespace CODESIGNING, Code 0x2
...
Logical CPU: 6
Error Code: 0x00000015 (invalid protections for user instruction write)
Trap Number: 14
About this issue
- Original URL
- State: closed
- Created 5 years ago
- Reactions: 6
- Comments: 149 (15 by maintainers)
@UdaraJay Alright, I’ve finally got it working. The trick was two fold.
First:
Set both entitlements and entitlementsInherit in your mac build settings. Here’s mine:
Then, in my entitlements file, I stripped it down to only what I need.
com.apple.security.cs.allow-unsigned-executable-memory is really the key one here. Others mentioned adding another one, but I was back to the signing crash when I did that.
Finally, the app went blank after hardened runtime code sign because my entitlements miss a key-value pair:
Spent couples of days trying every possible solution…hope this would help someone like me.
Awesome! Im happy to report we’ve solved all the build problems that we’ve been experiencing. Thank you @johannesjo, this was the issue driving us up a wall 😂
entitlements for mac:
mac entitlementsInherit is the same as regular mac entitlements
MAS entitlements:
MAS entitlementsInherit:
Solved! We now have the MAC (not MAS) hardened runtime working on Apple macOS. I hope this review might help others:
The Apple notarising process was spewing out false positives, so there was officially no problem, until our application died when running on a Mac, and created a dump file. Then it was difficult for Apple support to work out what the problem was.
Sometime on or before 10th April 2020, Apple (finally) did some upgrading work on their notarising process, an Apple Tech Support chap told me. It meant that we finally got some meaningful error messages, although over a couple of days (9th / 10th) the messages altered. Each run seemed to wipe out the previous ‘https://osxapps-ssl.itunes.apple.com/…’ logs and made them show the latest results as though they are their own?!
The Apple logs eventually had the error “Embedded entitlements are invalid: syntax error near line 1”. Their docs claim this means the entitlements file is in a binary format. In order to help the support engineer by providing the binary to which the entitlements had been applied, we had to create a new stand alone Azure (macOS) pipeline since you can’t export artefacts from their releases pipelines. And this new pipeline worked like a charm! Notarising passed, and the software opened correctly on the Mac.
It seems as though passing the code over from the main Azure (Windows) pipeline to the releases (macOS) pipeline somehow damaged the entitlements.plist file? There is a nagging suspicion that in 6 months time, Apple will suddenly say that there is no problem.
One curious anomaly, is that although we’re using the exact same Azure pipeline agent specification as before, 10.15, Taccy previously though that the software had been build on macOS 10.13 and now thinks it is 10.15. Or perhaps that is a reflection on the version used by Apple itself to notarise?
So the problem is solved. One of the things we’ve found incredibly frustrating is that Apple do not provide engineer tools to work out what is going on, hence the use of Taccy et al. Frankly, their ecosystem support is nowhere near as good as Microsoft.
Our final entitlements file:
What did help:
Making sure the path to the entitlements file was relative ‘./…’ Removing comments from the plist file might have helped (?)
What didn’t fix it for us:
Endlessly changing the entitlements Downgrading the Electron version to 6.x or 7.x
Hope this is of some use!
Further info:
You need at least the following entitlements:
Further info from https://medium.com/@TwitterArchiveEraser/notarize-electron-apps-7a5f988406db
Make sure to try out starting the app after this process, if some entitlements are missing, the app might be broken after notarization.
Note that publishing Electron apps to the Windows 10S store (as well as Mac App Store on Catalina, if I understand correctly) requires keeping native NodeJS plugins outside of
app.asar, due to the security policy (i.e. signed executables / dynamic libraries).I have been using this
electron-builderdirective to ensure that all native modules (regardless of their “owner” NPM package) are deployed into theapp.asar.unpackedfolder as standalone files (which would otherwise be extracted into temporary folders at runtime, thus the security breach):entitlements-sandbox.plist
entitlements-inherit.plist
I am unsure if the last two “security.cs” properties are required. I remember reading in docs that the “security.inherit” property must only exist as a sibling to app-sandbox… but, here we are… 😄 When I submit my next maintenance build I’ll start paring these back to remove any non-essential properties.
Alright y’all, my app (Label LIVE) was finally approved by Apple. 🍻
I just got an email notification that my app was finally accepted. I’ll post details soon, but it was not notarized and is not using hardened runtime.
Same issue. No fix but I did some tests with debug on (meaning
DEBUG=electron-osx-sign)read-onlykey andread-writekey to this entitlement file, it still seems to work: Extract of the log:sandboxkey, the entitlement become corrupt: Extract of the log:Got it to build and notarized. Have to use the default entitlement for both main and inherit.
Electron-builder: 22.7.0 Traget: mac hardenedRuntime = true.
Any other setting causes code signature invalid error.
Looks like building for mas is a whole new can of worms.
Alright folks, I finally cracked the situation wide open and learned what was happening. I haven’t submitted to MAS yet, but the “developer cannot be verified” error is gone on Catalina. 🙌
TLDR: Ran
stringson dynamically created executable and realized it was FSEvents native module. Reinstalled fs-jetpack to correct app/package.json.I solved it by opening a developer ticket with Apple. A great engineer walked me through what they saw. They asked, “Can you explain to me who is creating (or downloading) this
.7NB4X9P2DR.com.semireg.LabelLIVE.x38El1Mach-O 64-bit bundle x86_64 file?”No, I couldn’t… but I started looking in the temp folder and found a whole mess of those randomly created files.
I searched my application bundle for a file of similar size or exact md5 and came up empty. The file is definitely being dynamically created…
I ran strings on it, and doh…
This is a native node module related to filesystem events.
When I first started my project last year I used https://github.com/electron-react-boilerplate/electron-react-boilerplate which, at the time, used a 2 package.json system where native modules are installed to app/package.json, and all other non-native modules are installed to the top-level package.json (TLP).
If you accidentally add a native module to the TLP it will probably work in development, but come packaging you are in for cryptic errors.
This one takes the cake. I had
fs-jetpackinstalled in my top-level package.json, causing the FSEvents’ Mach-O 64-bit bundle x86_64 file to be dynamically extracted from the ASAR. This causes grief with 10.15, and thus the “developer cannot be verified” error.I uninstalled
fs-jetpackfrom the top-level, re-installed into app/package.json, added the appropriate declarations to my top-level package.json (asarUnpack) and now the mas-dev build runs on Catalina without an error.As mentioned above, our app was just accepted in the Mac App Store as working on Catalina. In the past, we have never used hardened runtime or notarized the
masbuild of our app. That was reserved for thedmgbuild distributed on our website. That turned out to still be true here. As previously highlighted, trying to use hardened runtime withmasbuilds results in a blank white renderer page for whatever reason.In the end, the problem that was causing the “____ can’t be opened because its integrity cannot be verified.” followed by the
dlopenerror was that we needed toasarUnpackthe native node modules that we were using. In our case, this wasdesktop-idleandkeytar. Once I properly added that, the app passed review.For reference, we have been tracking this issue on our repo here: https://github.com/bitwarden/desktop/issues/320
The final working pieces of our build can be found here:
https://github.com/bitwarden/desktop/blob/master/package.json#L63 https://github.com/bitwarden/desktop/blob/master/scripts/after-sign.js https://github.com/bitwarden/desktop/blob/master/resources/entitlements.mas.plist https://github.com/bitwarden/desktop/blob/master/resources/entitlements.mas.inherit.plist
Hope this helps!
@matteotomasoni ha yeah we are building https://yac.chat. So definitely needs audio.
I had the same fear as you and I think ultimately just got lucky. That’s also why the inherit property is so important. I’m not sure who at Apple thinks it’s normal to just crash an app if the entitlements are wrong though.
Hi all!
Two questions:
My MAS app builds and runs fine if I don’t add this to the entitlements.plist
com.apple.security.app-sandboxIs this this required or not? Can I submit the app to App Store without it?
The MAS builds seems to generate an executable AND an installer (PKG). However, the latter uses default icons, text etc and I don’t see any options in electron builder docs to set these. I mean, the options available for “PKG” builds are not available for MAS. Can customize the generated pkg installer somehow?
We’ve discovered that if in doubt, reset the perms that your application is asking for by running this sort of thing on the Mac you’re testing out installations on:
tccutil reset All com.YourCompanyName.YourAppName
This just resets all the requests your app has made to the OS. After doing this, our application behaved a lot more normally and the OS asked for perms to access different folders as we’d expect. Unfortunately, we’re not sure why. But using this was a massive improvement.
@akashnimare Yes, correct. “App-specific Passwords” you want to generate one and use that as the applepassword and ur email as the appleId
I finally got it working. My debugging efforts were seriously confused by this bug: https://github.com/electron/electron/issues/9985
So take care to comment out
app.makeSingleInstance()in case your trying to debug this.Yes thanks again @semireg, like @klen above we also passed review today thanks to your leads.
The problem for us was
electron-spellcheckerand including it inasarUnpackwas part of the fix, although we also had to includenode_modules/@felixrieseberg/spellcheckerthere as well.Tips for anyone else on this same issue:
hardenedRuntime: false, and had aprovisioningProfiledefinedELECTRON_BUILDER_CONFIGenv var, theasarUnpackin our package.json was ignored so we had to define it in the config-specific JSON file. Check the generated builder-effective-config.yaml to see what options EB is really using. (It may be a bug that the configs aren’t merged)find /private/var/folders -name [cryptic filename from warning]strings [full path to resulting file]mdfind "[unique string]" -onlyin ./app/node_modules@semireg man you saved me! Your investigation totally makes sense. I was struggling with notarization for a few days and it’s completely wrong. Today I’ve passed the review.
I’m able to confirm you don’t need notarization for MAS. Ensure that if you have native modules you have to unpack them from asar. If you sign your application the “developer cant be verified” problem on Catalina related only to dynamically unpacking modules.
The other thing that has been helpful for me is this app that exposes a lot more info about your build https://brockerhoff.net/RB/AppCheckerLite/
I’ve made more progress. Getting closer… some interesting tidbits here.
I decided to redo my search of the blank/white screen issue by focusing more on sandboxing causing the issue (vs. hardened runtime). That led to this comment where the recommendation is to remove entitlementsInherit. https://github.com/electron/electron-osx-sign/issues/151#issuecomment-331096044
I removed entitlementsInherit and something strange happened inside the auto-generated entitlements file. It added both my personal developer team ID “LX” and my MAS team ID “7N” (again, these are abbreviated).
Note, to view/edit the newly generated entitlements I ran:
When I saw this, the first thing I did was try to run the app as-is. It wouldn’t launch. To get it to launch I still needed to remove the team-identifier key/string and resign:
But once I did this… wow, it works. Sandboxed and hardened runtime. 😮
And best of all it seems to run fine on Catalina.
There’s still some work to do. I haven’t submitted to MAS, yet… but I’m hopeful.
i think its simply in the way the custom entitlements file gets accessed or read. because i too have the same problem as @UdaraJay. defaults are fine, customs are not. but i have no problems when i go to where the default file is located, and modify that one with what i need
WOW! Thanks for this thread. Totally saved me. Couldn’t figure out why
navigator.mediaDevices.getUserMedia()wasn’t working after bundling for production. I used a custom entitlements file and it works just great! Thanks!@UdaraJay I installed the latest Xcode beta today actually in order to get notarization working. So 11 beta 3. I’m on OSX 10.14.5. I have this running in Travis CI as well.
Update: Tested on xcode 10.2 and same issue
is this the fix? https://github.com/electron-userland/electron-builder/pull/3926
This was an exceptionally helpful post ! Thank you all for the in-depth comments.
In the end, keeping my entitlements as per instructed here: https://medium.com/@TwitterArchiveEraser/notarize-electron-apps-7a5f988406db … and adding one extra
com.apple.security.cs.allow-dyld-environment-variablesdid the trick!Thanks guys! I was starting to go bald already.
This is what was happening for me in a project completely unrelated to Electron: I had some binaries (go-flutter-desktop actually) and they loaded a
dylibbundled along with them in a.dmgwhich caused the.appto crash on start when signed for hardened runtime.It transpired this action requires Allow Unsigned Executable Memory Entitlement during the codesign operation with
options=runtimeenabled. Without this entitlement the app just crashed on start with no intelligible error message.Thanks to you lot I figured which entitlement was missing by trial and error.
@johannesjo amazing! thank you, the singleApp fix allowed me to build a mas-dev application and have it launch! This was the problem haha but now the app just shows blank white screen… Time to debug this problem now 😂
@semireg Congrats! You’re using electron 4 right? Would be cool if you could share the config that did the trick with the rest of us! 😃
@patrickmichalina I checked and no we aren’t using UDP. The network entitlements predate my time on the project, so I’m not sure of exactly what circumstance caused them to be added. Sorry I can’t be of more help on that one.
@Jocs The notarizing tool logged a warning:
The executable does not have the hardened runtime enabledwhenhardenedRuntimewas set tofalsebut the notarization still finished successful.@johannesjo
Running the
.appafter that works.@johannesjo, can you edit your comment to include what OS you’re trying to install on and what electron/electron-builder version you’re using?
@alibosworth, thank you for the note. If I understand your situation, it makes me feel better that you and I are in the same relative place.
My app is using the latest Electron 4, and like I mentioned above, our self-distributed .zip build (“mac”) is not affected by the “macOS cannot verify that this app is free from malware” error. That leads me to believe we’re dealing with a bug in Catalina or electron-builder.
In case it helps anyone, I am 99% sure that you don’t need to notarize for a MAS submission. We recently got our self-distributed .zip build working on Catalina via electron-builder 21.2.0 + Electron 5.0.3 + electron-notarize 0.1.1. We are still struggling with a new-to-the-app-store MAS build, though we have gotten it to the point that it doesn’t crash, but it does give the “macOS cannot verify that this app is free from malware” warning on Catalina.
To avoid MAS builds crashing we had to:
masspecifically and letmacbe set to true`)The other thing that was a revelation for us, is that you CAN test your PKG built via
target: mas(not mas-dev) by “installing” it viasudo installer -store -pkg path/to/your.pkg -target /after build. Right now it works great on 10.14.6. I have a hunch that Catalina requires a build from Electron 6 and not 5, and I am concerned that there hasn’t been an electron-builder release in 3 months.This is all the info they had for me. Now I really don’t know where to go from here
@james-criscuolo As far as I know, notarization is not needed for mas builds. At least I have never notarized them.
I’ve seen a lot of other issues surrounding this, but never this one, or the others that have @semireg has been posting in. I’ve been banging away at this for close to 2 weeks, with 5 or 6 failed app store submissions, but day 1 had a DMG version working in Catalina, electron 6/hardened runtime/notarized. I don’t think I’m any further along than anyone in this thread, but figured I’d post some of my findings to see if anybody sees anything intriguing:
I went back to electron 5 for the time being. Electron 6 adds a login helper (and other helpers) that just seems to be another entitlements hurdle to jump, plus you need to manually adjust your node_module with this accepted PR.
The binary plist thing seemed like a surefire issue, but as far as I could tell it was a red herring. Switching to
electron-packagergot rid of binary plists, but no errors have been different.I’ve switched from
electron-buildertoelectron-packager,electron-osx-notarize, andelectron-osx-flat, at least for now. I don’t think this has gotten me any further, but I at least feel like I have a bit more control (I probably don’t).All the blog posts and “working” solutions I’ve found always end up being for direct download situations. One point of contention I found is that all directions online, and even
electron-packager’s built-in notarize, will only notarize the.appfile and not the.pkgfile. At this point, I’m allowing it to notarize that, then converting to pkg, then notarizing THAT as well. Love the significantly longer built process notarization causes, not to mention the notarization system had an outage last week. Only my latest build had this last point in it, and it was just rejected about 15 minutes ago with thedlopenissue, which I’ve been seeing this whole time. I had been seeing a “you didn’t notarize” issue prior to this, so it was at least a step in the right direction.I guess my next step is to disable hardenedRuntime, as you’ve mentioned above, and try again. My last two submissions took 50 hours and 24 hours to get rejected, so I may as well start the long wait.
Edit: Also throwing my only native module (
keytar) into asar unpacked, so with these two changes I’m already feeling far too optimistic.I submitted a build with
"hardenedRuntime": falseearlier today along with some of theasarUnpacksuggestions that @semireg posted earlier. Waiting for apple review.Ok, more info on this. If you google “bplist00” you get information on Apple’s binary format. I understand that plists can be either XML or binary. I’m running electron-builder manually using:
DEBUG=electron-osx-sign npx electron-builder --mac mas-devWhat’s interesting is that, when
<key>com.apple.security.app-sandbox</key>is defined, the debug output containsEntitlements file updatedwith a path such as:Inspecting this file reveals it’s in Apple’s binary format.
If you
catthis file, the output is identical to what you see with:codesign -d --entitlements :- path/to/My App…You can easily convert it after the fact (not that this does any good):
Which will yield:
And
cating this file looks like what you’d expect. An entitlements plist.My hunch is that electron-builder is wrongly code signing using a binary version of the entitlements.
@UdaraJay tried that just now, removed the properties “entitlements” and “entitlementsInherit” from the package json under the mac key, still the exact same error
also, running "codesign -d --ent :- “Appname.app/Contents/MacOS/Appname” doesnt give back anything if its supposed to set default entitlements what are the defaults?
update: nevermind, the defaults are being set.
in the end, my build is loading with the default entitlements. the other issue i had was that i had set
mac: {type: "development"}@UdaraJay entitlements got rid of the sign error but yeah my UI crashes on start now