okhttp: Problems with DNS resolution

I’ve had this happen to a good 10% of my devices, affecting ~1000 users. However, I cannot seem to reproduce this issue. It only happens when connected over Wifi, and multiple users have confirmed me they weren’t behind any captive portals or anything. They could access the url normally when typing it into a browser. There isn’t any good indicator this is a vendor-specific issue either. My statistics show a majority of samsung devices, grafik but then again, it works flawlessly on my samsung test device. I don’t know what to make of it. Here’s the stack trace:

java.net.UnknownHostException
Unable to resolve host "<valid hostname>": No address associated with hostname
java.net.InetAddress.lookupHostByName (InetAddress.java:394)
java.net.InetAddress.getAllByNameImpl (InetAddress.java:236)
java.net.InetAddress.getAllByName (InetAddress.java:214)
arrow_right
okhttp3.Dns$1.lookup (Dns.java:40)
okhttp3.internal.connection.RouteSelector.resetNextInetSocketAddress (RouteSelector.java:185)
okhttp3.internal.connection.RouteSelector.nextProxy (RouteSelector.java:149)
okhttp3.internal.connection.RouteSelector.next (RouteSelector.java:84)
okhttp3.internal.connection.StreamAllocation.findConnection (StreamAllocation.java:214)
okhttp3.internal.connection.StreamAllocation.findHealthyConnection (StreamAllocation.java:135)
okhttp3.internal.connection.StreamAllocation.newStream (StreamAllocation.java:114)
okhttp3.internal.connection.ConnectInterceptor.intercept (ConnectInterceptor.java:42)
okhttp3.internal.http.RealInterceptorChain.proceed (RealInterceptorChain.java:147)
okhttp3.internal.http.RealInterceptorChain.proceed (RealInterceptorChain.java:121)
okhttp3.internal.cache.CacheInterceptor.intercept (CacheInterceptor.java:93)
okhttp3.internal.http.RealInterceptorChain.proceed (RealInterceptorChain.java:147)
okhttp3.internal.http.RealInterceptorChain.proceed (RealInterceptorChain.java:121)
okhttp3.internal.http.BridgeInterceptor.intercept (BridgeInterceptor.java:93)
okhttp3.internal.http.RealInterceptorChain.proceed (RealInterceptorChain.java:147)
okhttp3.internal.http.RetryAndFollowUpInterceptor.intercept (RetryAndFollowUpInterceptor.java:126)
okhttp3.internal.http.RealInterceptorChain.proceed (RealInterceptorChain.java:147)
okhttp3.internal.http.RealInterceptorChain.proceed (RealInterceptorChain.java:121)
okhttp3.RealCall.getResponseWithInterceptorChain (RealCall.java:200)
okhttp3.RealCall.execute (RealCall.java:77)
retrofit2.OkHttpCall.execute (OkHttpCall.java:180)
retrofit2.adapter.rxjava2.CallExecuteObservable.subscribeActual (CallExecuteObservable.java:41)
io.reactivex.Observable.subscribe (Observable.java:10151)
retrofit2.adapter.rxjava2.BodyObservable.subscribeActual (BodyObservable.java:34)
io.reactivex.Observable.subscribe (Observable.java:10151)
io.reactivex.internal.operators.observable.ObservableSingleSingle.subscribeActual (ObservableSingleSingle.java:35)
io.reactivex.Single.subscribe (Single.java:2517)
io.reactivex.internal.operators.single.SingleSubscribeOn$SubscribeOnObserver.run (SingleSubscribeOn.java:89)
io.reactivex.Scheduler$1.run (Scheduler.java:131)
io.reactivex.internal.schedulers.ScheduledRunnable.run (ScheduledRunnable.java:59)
io.reactivex.internal.schedulers.ScheduledRunnable.call (ScheduledRunnable.java:51)

If you need more information, don’t hesitate to ask.

About this issue

  • Original URL
  • State: closed
  • Created 9 years ago
  • Reactions: 34
  • Comments: 40

Most upvoted comments

check your device is opened wifi?

Hi everyone, was anyone able to figure out a solution for this? We are facing same issues with our app

I’m not sure if this is a generic networking error. I always saw this a lot on fabric but I wasn’t able to debug, but now something weird just happened with a user. I have two apps (lite version and pro version), exactly same codebase and exactly same API hostname. The user has the two apps installed on Samsung device, but, the pro version is returning java.net.UnknownHostException Unable to resolve host "<valid hostname>": No address associated with hostname and the lite one is working normally.

How’s that possible?

You can try using an implementation of the DNS interface that falls back to Google DNS.

Android has this problem but iOS works always fine.

As seen in https://github.com/square/okhttp/issues/4789#issuecomment-1925394802

Android caches false results for 2 seconds, so if your app made a request in doze mode or similar, or the connection was down. Then it will fail until the Android cache is cleared after 2 seconds.

why this isse is closed?

Even though we can’t really reproduce, is there at least a way to prevent these “crashes” from showing up on Firebase? Also do these “crashes” happen for you guys in the background or while the app is running?

Update2: I changed the DNS resolver to system default again and tried with different URLs: http://bot.whatismyipaddress.com/ and GET https://checkip.dyndns.org/ and GET https://ipinfo.io/ip :

                    targeturl = "https://ipinfo.io/ip";
                    response = _makeSimpleHttpRequest(targeturl, "GET", "",true); 

The same issue occurs with checkip.dyndns.org which is also hosted on cloudflare. However the ipinfo.io works and is hosted on Google Domains according to Whois-Lookup! I Suppose there is an issue that cloudflare is simply blocking native calls from android devices, aka the admin of the website is to strict. Since there is also an issue in DNS resolving these cloudflare domains it could be some kind of active DDOS-Protection mechanisms filtering calls that look “to basic”? I think this is more an webserver admin issue we need to work around…

Hey there. This seems to be a good aproach!

After two hours i got this:

public void dostuff(){
                    ...
                    targeturl = "http://bot.whatismyipaddress.com/";
                    response = _makeSimpleHttpRequest(targeturl, "GET", "",true); 
                    ...
}
 public static String _makeSimpleHttpRequest(String targetUrl, String methodGetPostX, String postBody, boolean useSelfImplementedDNSwithFallbackTopdomain) throws MalformedURLException, IOException, NotImplementedException{
String responseline = "";
        String temp = "";
        HttpURLConnection httpConn = null;
        InputStream inputStream = null;
        InputStreamReader inputStreamReader = null;
        BufferedReader bufferedReader = null;
        OutputStream outputStream = null;
        OutputStreamWriter outputStreamWriter = null;
        BufferedWriter bufferedWriter = null;

        try {
            java.net.URL url = null;
            if(!useSelfImplementedDNSwithFallbackTopdomain){
                url = new java.net.URL(targetUrl); // system default DNS resolver
            }
            else{
                String baseDomainName = _getDomainFromFullURL(targetUrl);
                String targetUrlReplacedDomainWithIP = targetUrl.replace(baseDomainName,_makeDnsLookupForDomainOrFallbackTopdomain(baseDomainName));
                url = new java.net.URL(targetUrlReplacedDomainWithIP); 
            }
            httpConn = (HttpURLConnection)url.openConnection(); // HERE THE ERROR OCCURED PREVIOUSLY, NOW THE IP GOT RESOLVED CORRECTLY BEFOREHAND, BUT SSL ERROR FOROBVIOUS REASON SINCE THE DOMAIN WAS REPLACED BY THE IP MANUALLY
}

public static String _getDomainFromFullURL(String url) throws URISyntaxException {
        URI uri = new URI(url);
        String domain = uri.getHost();
        return domain;
}

public static String _getRightSideOfStringSplitByFirstDot_TopDomainOfSubDomain(String stringWithDots_SubDomain){
        String[] splitByDot = stringWithDots_SubDomain.split("\\.");
        String topDomain="";
        for(int i = 1; i< splitByDot.length; i++){
            topDomain += splitByDot[i];
            if(i<splitByDot.length-1){
                topDomain += ".";
            }
        }
        return topDomain;
}
    
public static String _makeDnsLookupForDomainOrFallbackTopdomain(String domainName) throws UnknownHostException {
        String ipAddr = "";
        int i = 0;
        try{
            System.out.println("dns probe with fallback to top-domain ["+i+"]: " + domainName +" = ...");
            ipAddr = _makeDnsLookupForDomain(domainName);
            System.out.println("dns probe with fallback to top-domain ["+i+"]: " + domainName +" = " + ipAddr);
        } catch (UnknownHostException e){
            while(domainName.contains(".") && !domainName.endsWith(".") && !domainName.startsWith(".")){
                i++;
                domainName = _getRightSideOfStringSplitByFirstDot_TopDomainOfSubDomain(domainName);
                System.out.println("dns probe with fallback to top-domain ["+i+"]: " + domainName +" = ...");
                try{
                    ipAddr = _makeDnsLookupForDomain(domainName);
                    System.out.println("dns probe with fallback to top-domain ["+i+"]: " + domainName +" = " + ipAddr);
                    break;
                } catch (UnknownHostException e2){
                    //
                    throw e2;
                }
            }
        }
        return ipAddr;
}

public static String _makeDnsLookupForDomain(String domainName) throws UnknownHostException {
        // http://www.avajava.com/tutorials/lessons/how-do-i-use-a-host-name-to-look-up-an-ip-address.html
        InetAddress[] inetAddressArray = InetAddress.getAllByName(domainName/*"www.google.com"*/);
        //
        int selectThisEntryNumber = 0;
        if(inetAddressArray.length==1){
            selectThisEntryNumber = 0;
        }
        else if(inetAddressArray.length>1){
            Random random = new Random();
            int max = inetAddressArray.length-1;
            int min = 0;
            selectThisEntryNumber = random.nextInt(max - min + 1) + min; // number between max and min including both
        }
        else{
            // 0
            throw new UnknownHostException("Retrieved 0 addresses for this host: " + domainName);
        }
        String ipAddressString = inetAddressArray[selectThisEntryNumber].getHostAddress();
        return ipAddressString;
}   

Now the flow is: GET https://bot.whatismyipaddress.com/ goes into my own DNS resolver which gets rid of the subdomain if UnknownHost-Exception occurs:

dns probe with fallback to top-domain [0]: bot.whatismyipaddress.com = ...
dns probe with fallback to top-domain [1]: whatismyipaddress.com = ...
dns probe with fallback to top-domain [1]: whatismyipaddress.com = 104.16.155.36

Now the call would be transformed to: GET https://104.16.155.36/

However this throws SSL exception for obvious reasons (we just replaced the host and the certificate doesn’t match to a floating ip!).

Now i am sure this was the problem for my issue. Since removing SSL Checks is not recommended do you guys have another idea how to make this work?

Do you have any contextual information

  • Device manufacturers
  • Android OS version
  • Background state
  • Wifi/Cell?
  • Proxy enabled?

Airplane mode fails this way also, but then would affect all apps. If you have a crash reporter app, it would be interesting to see what else correlates with the errors…

Thanks for the reference, will research further. We know users experiencing this issue aren’t on airplane mode from talking to them

After surveying a few of the users, it appears the user ONLY happens on wifi. Our app works fine on cellular data.

More details on our issue

~80% of our app is in React Native, on v0.62.2. We got this issue after upgrading from 0.61.5 to 0.62.2, but that might be unrelated. We also have a native part of our app where we directly use retrofit and okhttp. We use retrofit 2.8.0 so that sets our okhttp to v3.14.7. In an attempt to fix, we have pinned retrofit to 2.6.4 so its version for okhttp is 3.12.0, which matches what RN also expects. We haven’t seen reports of any other users getting this issue upgrading to RN’s 0.62.2, so we thought this might be due to our version mismatch of okhttp.

Finally, the only adjustment we really make to okhttp in our app:

      NetworkingModule.setCustomClientBuilder {
        it.retryOnConnectionFailure(false)
      }

This is because we were seeing that requests to multipart uploads were queuing up for retry automatically on failure. Later we would sometimes see these uploads get retried, and it would overwhelm our services sometimes with hundreds of uploads with the same image.

Here are a few of our issues we have seen coming in through our monitoring.

java.net.UnknownHostExceptionForwardingOs.java:58
java.net.UnknownHostException Unable to resolve host "s3-us-west-2.amazonaws.com": No address associated with hostname 
    Inet6AddressImpl.java:141 java.net.Inet6AddressImpl.lookupHostByName
    Inet6AddressImpl.java:90 java.net.Inet6AddressImpl.lookupAllHostAddr
    InetAddress.java:787 java.net.InetAddress.getAllByName
    Dns.java:39 okhttp3.Dns$-CC.lambda$static$0
    Unknown okhttp3.-$$Lambda$Dns$mTkNcZf2K4euny3_jks6Cac6Az0.lookup
    RouteSelector.java:171 okhttp3.internal.connection.RouteSelector.resetNextInetSocketAddress
    RouteSelector.java:135 okhttp3.internal.connection.RouteSelector.nextProxy
    RouteSelector.java:84 okhttp3.internal.connection.RouteSelector.next
    ExchangeFinder.java:187 okhttp3.internal.connection.ExchangeFinder.findConnection
    ExchangeFinder.java:108 okhttp3.internal.connection.ExchangeFinder.findHealthyConnection
    ExchangeFinder.java:88 okhttp3.internal.connection.ExchangeFinder.find
    Transmitter.java:169 okhttp3.internal.connection.Transmitter.newExchange
    ConnectInterceptor.java:41 okhttp3.internal.connection.ConnectInterceptor.intercept
java.net.UnknownHostException Unable to resolve host "s3-us-west-2.amazonaws.com": No address associated with hostname 
    Inet6AddressImpl.java:126 java.net.Inet6AddressImpl.lookupHostByName
    Inet6AddressImpl.java:105 java.net.Inet6AddressImpl.lookupAllHostAddr
    InetAddress.java:1154 java.net.InetAddress.getAllByName
    Dns.java:39 okhttp3.Dns$-CC.lambda$static$0
    Unknown okhttp3.-$$Lambda$Dns$mTkNcZf2K4euny3_jks6Cac6Az0.lookup
    RouteSelector.java:171 okhttp3.internal.connection.RouteSelector.resetNextInetSocketAddress
    RouteSelector.java:135 okhttp3.internal.connection.RouteSelector.nextProxy
    RouteSelector.java:84 okhttp3.internal.connection.RouteSelector.next
    ExchangeFinder.java:187 okhttp3.internal.connection.ExchangeFinder.findConnection
    ExchangeFinder.java:108 okhttp3.internal.connection.ExchangeFinder.findHealthyConnection
    ExchangeFinder.java:88 okhttp3.internal.connection.ExchangeFinder.find
    Transmitter.java:169 okhttp3.internal.connection.Transmitter.newExchange
    ConnectInterceptor.java:41 okhttp3.internal.connection.ConnectInterceptor.intercept
    RealInterceptorChain.java:142 okhttp3.internal.http.RealInterceptorChain.proceed
    RealInterceptorChain.java:117 okhttp3.internal.http.RealInterceptorChain.proceed
    CacheInterceptor.java:94 okhttp3.internal.cache.CacheInterceptor.intercept
java.lang.reflect.UndeclaredThrowableExceptionNoServiceActivity
java.lang.reflect.UndeclaredThrowableException
    Unknown:-1 $Proxy31.downloadAsset
    AssetManager.kt:112 com.openfit.openfit.app.AssetManager.downloadAndWrite
    AssetManager.kt:92 com.openfit.openfit.app.AssetManager.downloadAsset
    AssetManager.kt:48 com.openfit.openfit.app.AssetManager$downloadSessionAssets$$inlined$repeat$lambda$1.invokeSuspend
    ContinuationImpl.kt:33 kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith
    Dispatched.kt:241 kotlinx.coroutines.DispatchedTask.run
    CoroutineScheduler.kt:594 kotlinx.coroutines.scheduling.CoroutineScheduler.runSafely
    CoroutineScheduler.kt:60 kotlinx.coroutines.scheduling.CoroutineScheduler.access$runSafely
    CoroutineScheduler.kt:740 kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.run

Caused by: java.net.UnknownHostException Unable to resolve host "s3-us-west-2.amazonaws.com": No address associated with hostname 

FWIW adb enhanced is useful for testing similar OS modes https://github.com/ashishb/adb-enhanced