okhttp: IllegalArgumentException for the url in the Cache Entry

The following error is one of our main crashes right now, around 25% of our crashes.

Fatal Exception: java.lang.IllegalArgumentException: Expected URL scheme 'http' or 'https' but no colon was found
       at okhttp3.HttpUrl$Builder.parse$okhttp(HttpUrl.java:1260)
       at okhttp3.HttpUrl$Companion.get(HttpUrl.java:1632)
       at okhttp3.Request$Builder.url(Request.java:184)
       at okhttp3.Cache$Entry.response(Cache.java:641)
       at okhttp3.Cache.get$okhttp(Cache.java:183)
       at okhttp3.internal.cache.CacheInterceptor.intercept(CacheInterceptor.java:47)
       at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:109)
       at okhttp3.internal.http.BridgeInterceptor.intercept(BridgeInterceptor.java:83)
       at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:109)
       at okhttp3.internal.http.RetryAndFollowUpInterceptor.intercept(RetryAndFollowUpInterceptor.java:76)
       at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:109)
       at de.stocard.dagger.ProvidedDependenciesModule$provideHttpClient$$inlined$-addInterceptor$1.intercept(ProvidedDependenciesModule.java:1083)
       at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:109)
       at okhttp3.internal.connection.RealCall.getResponseWithInterceptorChain$okhttp(RealCall.java:201)
       at okhttp3.internal.connection.RealCall$AsyncCall.run(RealCall.java:517)
       at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1167)
       at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:641)
       at java.lang.Thread.run(Thread.java:919)

The same error was reported https://github.com/square/okhttp/issues/4322 and https://github.com/square/okhttp/issues/4918

I would like to reopen the discussion and maybe together we can find the root cause and a solution.

During the last week we were building a custom version based on the 4.9.0 to report the url contained in the Cache Entry. We modified the following line with the code below and we also modified the cache path directory to avoid possible old corrupted files.

throw IllegalArgumentException("Expected URL scheme 'http' or 'https' but no colon was found for the url input $input")

We are getting in that case the corrupted url stored in the cache file.

For example, with the following retrofit configuration:

val clientBuilder = OkHttpClient.Builder()

clientBuilder.protocols(listOf(Protocol.HTTP_1_1))

clientBuilder.addInterceptor { chain ->
    val originalRequest = chain.request()
    val requestWithUserAgent = originalRequest.newBuilder()
            .header("User-Agent", "Example/1.0.0 HTTPClient Android").build()
    chain.proceed(requestWithUserAgent)
}

val certificatePinner = CertificatePinner.Builder()
certificatePinner.add("*.example.com", "sha256/public-key-1")
clientBuilder.certificatePinner(certificatePinner.build())

val cacheDir = File(context.cacheDir, "new_caching_directory")
val cache = Cache(cacheDir, (16 * 1024 * 1024).toLong())
clientBuilder.cache(cache)

val okhttpClient = clientBuilder.build()

val retrofit = Retrofit.Builder()
                .baseUrl("https://www.example.com/")
                .client(okHttpClient)
                .build()

val exampleApi = retrofit.create(ExampleApi::class.java)

interface ExampleApi {
    @GET("users")
    fun getUsers(): Single<List<User>>
}

We are getting in the error stack traces the following urls:

Fatal Exception: java.lang.IllegalArgumentException: Expected URL scheme 'http' or 'https' but no colon was found for the url input 5����e?��뷘b���@&-����1�W�ہ�n��?�!y6�j٬�6N�S��https://www.example.com/users
Fatal Exception: java.lang.IllegalArgumentException: Expected URL scheme 'http' or 'https' but no colon was found for the url input 5T�k;=�fEr��,��k�H��2�C�A*�����s�������.�^https://www.example.com/users
Fatal Exception: java.lang.IllegalArgumentException: Expected URL scheme 'http' or 'https' but no colon was found for the url input ://www.example.com/users

The url stored in the cache file, as you can see, can contain some corrupted text or the https is removed from the url.

Could you have a look? I can also investigate further or give you more data if needed.

Regards

About this issue

  • Original URL
  • State: closed
  • Created 4 years ago
  • Reactions: 56
  • Comments: 42 (5 by maintainers)

Commits related to this issue

Most upvoted comments

In change log for Version 5.0.0-alpha.2 I see

Fix: Fail fast when the cache is corrupted.

Is that fix for this issue? If yes how long we will wait stable 5.0.0 version? And is it possible to fix it in version 4.9.x?=)

I can confirm that we have the same issue, see it on clients with cache. And it’s not an issue of request itself, we have a debug and request URL is correct, so after digging into sources, it may happen only if a cache entry has invalid value for URL

Never be able to reproduce it, only on prod. We have not so many such cases, 10-30 per day maximum, but still it in top 10 our crashes

Yep:

Fix: Recover gracefully when a cached response is corrupted on disk.

Any corrections? @yschimke

I see the backport was just merged https://github.com/square/okhttp/pull/6940 to 4.9.x. Can we expect this to be released shortly? Starting to see this happening and would love to just update the library to solve but can utilize the workaround in the meantime.

Thanks @yschimke for improving the logging in #6495 for the next release. In the meantime what could we do to progress on this issue? If the issue is detected on the reading side, maybe we could try to detect it even earlier, meaning at the writing side ? Maybe HttpUrl.url is invalid…

I can confirm that with the newest okhttp alpha version 5.0.0-alpha.12, it’s fixed - https://square.github.io/okhttp/changelogs/changelog/#version-500-alpha12 .

Please check the change log.

any corrections? it happens only in production when using “minifyEnabled true” for obfuscation, and “-keep class okhttp3.**{*;}” didn’t help to avoid this issue - Do any idea?

The fix (#6495 and #6940) makes it much more difficult to attempt to recover from this state using something like @danh32 's CacheDeborkifier since OkHttp is catching the exception internally.

Of course a recovery mechanism built into OkHttp (#3251) would be great, but in the meantime is there another way to attempt to recover?

I suppose we could install a Logger that listens for the new “cache corruption” log, but that feels worse than the CacheDeborkifier’s approach of an interceptor that catches the old exception.

For some context - of the many endpoints my app hits, only ~2 are actually cacheable and we hit those two very frequently. The cache being broken for them can mean hundreds of additional calls. I have no problem just deleting the entire cache right now if we detect it is corrupted 😆

@yschimke we included some logs when the cache object is instantiated. After checking few logs of the affected users I could see that the cache is instantiated only once.

Can we do something else to help?

Edit: by the way @yschimke we are using dagger to instantiate the cache and we are using the @Singleton annotation.

@brunoescalona not sure, I’m brainstorming what we can do in Version+1, to get information to resolve by Version+2. And wondering if we should ask for reports and also make it not fatal until then.

Yes I’d like to get to root cause on this one too!

@brunoescalona I don’t see a cache configured in your example. Is that happening elsewhere? Is it possible to provide a minimal test or code sample that you can reliably reproduce with?