conscrypt: Spike in "Trust anchor for certification path not found " errors
I’ve been getting a lot of reports from Crashlytics in the past week about the following issue affecting ~10% of my user base. I’ve never seen this issue myself, and it never happens in our dev server environment.
I ran the ssllabs.com and digicert.com analysis and there were no issues.
We do use cert pinning through Android.
It’s happening on Android 7-10 and all the big devices and manufacturers.
network_security_config.xml
<?xml version="1.0" encoding="utf-8"?>
<network-security-config>
<domain-config>
<domain includeSubdomains="true">mydomain.com</domain>
<pin-set>
<!--
NOTE recommendation is to have a second certificate to pin just in case the
main one needs to be invalidated
-->
<pin digest="SHA-256">the_hash</pin>
</pin-set>
</domain-config>
</network-security-config>
Crash:
Caused by java.security.cert.CertPathValidatorException: Trust anchor for certification path not found.
at com.android.org.conscrypt.TrustManagerImpl.checkTrustedRecursive(TrustManagerImpl.java:654)
at com.android.org.conscrypt.TrustManagerImpl.checkTrusted(TrustManagerImpl.java:499)
at com.android.org.conscrypt.TrustManagerImpl.checkTrusted(TrustManagerImpl.java:422)
at com.android.org.conscrypt.TrustManagerImpl.getTrustedChainForServer(TrustManagerImpl.java:343)
at android.security.net.config.NetworkSecurityTrustManager.checkServerTrusted(NetworkSecurityTrustManager.java:94)
at android.security.net.config.RootTrustManager.checkServerTrusted(RootTrustManager.java:88)
at com.android.org.conscrypt.Platform.checkServerTrusted(Platform.java:208)
at com.android.org.conscrypt.ConscryptFileDescriptorSocket.verifyCertificateChain(ConscryptFileDescriptorSocket.java:426)
at com.android.org.conscrypt.NativeCrypto.SSL_do_handshake(NativeCrypto.java)
at com.android.org.conscrypt.NativeSsl.doHandshake(NativeSsl.java:383)
at com.android.org.conscrypt.ConscryptFileDescriptorSocket.startHandshake(ConscryptFileDescriptorSocket.java:231)
at okhttp3.internal.connection.RealConnection.connectTls(RealConnection.java:367)
at okhttp3.internal.connection.RealConnection.establishProtocol(RealConnection.java:325)
at okhttp3.internal.connection.ExchangeFinder.findConnection(ExchangeFinder.java:197)
at okhttp3.internal.connection.RealConnection.connect(RealConnection.java:197)
at okhttp3.internal.connection.ExchangeFinder.findConnection(ExchangeFinder.java:249)
at okhttp3.internal.connection.ExchangeFinder.findHealthyConnection(ExchangeFinder.java:108)
at okhttp3.internal.connection.ExchangeFinder.find(ExchangeFinder.java:76)
at okhttp3.internal.connection.RealCall.initExchange$okhttp(RealCall.java:245)
at okhttp3.internal.connection.ConnectInterceptor.intercept(ConnectInterceptor.java:32)
at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:100)
at okhttp3.internal.cache.CacheInterceptor.intercept(CacheInterceptor.java:96)
at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:100)
at okhttp3.internal.http.BridgeInterceptor.intercept(BridgeInterceptor.java:83)
at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:100)
at okhttp3.internal.http.RetryAndFollowUpInterceptor.intercept(RetryAndFollowUpInterceptor.java:76)
at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:100)
at okhttp3.logging.HttpLoggingInterceptor.intercept(HttpLoggingInterceptor.java:219)
at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:100)
at co.twenty.api.CustomPropertyInterceptor.intercept(CustomPropertyInterceptor.java:49)
at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:100)
at co.twenty.api.DefaultParamsInterceptor.intercept(DefaultParamsInterceptor.java:44)
at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:100)
at okhttp3.internal.connection.RealCall.getResponseWithInterceptorChain$okhttp(RealCall.java:197)
at okhttp3.internal.connection.RealCall$AsyncCall.run(RealCall.java:502)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1167)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:641)
at java.lang.Thread.run(Thread.java:764)
About this issue
- Original URL
- State: closed
- Created 4 years ago
- Comments: 35 (15 by maintainers)
I think that particular case is a wash for us (we’re not building for their intranet).
I’d say it’s very likely that this is the main cause of the errors that we’re seeing, since it spiked around the time we partnered with that campus.
We recently released a version with this debugging in, and got our first hit!
Looks like the university network is MITM us (I’m assuming it’s a configuration thing, and not intentional because we partner with them, unless there’s a bad actor somewhere).
I’ll keep the debugging in for a few versions and see if anything else interesting pops up.
Hmm, none of the root certificates in Android 10 have expired recently (most recent was May 30) so the cross-signed intermediate theory isn’t looking good.
There’s also proxies/middleboxes but for those to suddenly affect 10% of your population seems weird.
That said, such breakages do happen… I recently investigated an issue where a noticeable proportion of TLS connections to Android OTA servers were failing that turned out to be a misconfigured traffic shaping middlebox on a telco’s network which was sending its own self-signed certificate instead of the target server’s.