googleads-mobile-unity: Huge number of ANRs in Unity: Input dispatching timed out...

[REQUIRED] Step 1: Describe your environment

  • Unity version: 2020.3.18f1
  • Google Mobile Ads Unity plugin version: v6.1.0, v6.1.1
  • Platform: Android
  • Platform OS version: Android 6 - 10
  • Any specific devices issue occurs on: BQru BQ-4028UP!, Lenovo Tab M7, Nokia 5, ZTE Blade L130, Redmi 9C NFC, HOMTOM S16, Huawei 荣耀9青春版
  • Mediation ad networks used, and their versions: None

[REQUIRED] Step 2: Describe the problem

In past year Admob team introduced a bug into its Unity SDK which causes a huge number of ANRs for many developers. You can find lots of reports on support forum, but the most recent topic is https://groups.google.com/g/google-admob-ads-sdk/c/6LfTaInDGtU.

The issue seems to be a deadlock related to the init of the WebView. Initially Admob team ignored and and attributed the problem to a bug in a specific Android’s WebView bug, and as they said issue should have recede as people upgrade to newer WebView. Turns out it’s not the case, and we see more and more ANRs in a newer versions of Android. What even more suspicious these ANRs come in waves - few days there is none of them, and then huge wave which pushes apps to a bad behaviour threshold.

This problem is super critical, since i ruins many apps and business, because Google Play severely punishes apps for even slight increase in ANRs rate.

So far Admob team was handling this issue in a disgraceful way: ignoring it initially, and “investigating” it for about 2-3 months already with no result.

There is two types of ANRs related to the same issue:

1. Broadcast of Intent { act=android.intent.action.SCREEN_OFF flg=0x58200010 }

No stack trace

2. Input dispatching timed out (Waiting to send non-key event because the touched window has not finished processing certain input events that were delivered to it over 500.0ms ago. Wait queue length: 9. Wait queue head age: 14852.2ms.)
com.unity3d.player.UnityPlayerActivity

  #00  pc 00000000000706b4  /system/lib64/libc.so (nanosleep+8)
  #00  pc 000000000045b150  /data/app/com.app.id-3lXBPjkSmjrQATT-XgPpFg==/lib/arm64/libil2cpp.so (???)
  #00  pc 0000000000467d58  /data/app/com.app.id-3lXBPjkSmjrQATT-XgPpFg==/lib/arm64/libil2cpp.so (???)
  #00  pc 000000000047a010  /data/app/com.app.id-3lXBPjkSmjrQATT-XgPpFg==/lib/arm64/libil2cpp.so (???)
  #00  pc 000000000043bb3c  /data/app/com.app.id-3lXBPjkSmjrQATT-XgPpFg==/lib/arm64/libil2cpp.so (???)
  #00  pc 00000000002d42c0  /data/app/com.app.id-3lXBPjkSmjrQATT-XgPpFg==/lib/arm64/libunity.so (???)
  #00  pc 000000000017ad04  /data/app/com.app.id-3lXBPjkSmjrQATT-XgPpFg==/lib/arm64/libunity.so (???)
  #00  pc 000000000017ac1c  /data/app/com.app.id-3lXBPjkSmjrQATT-XgPpFg==/lib/arm64/libunity.so (???)
  at com.unity3d.player.ReflectionHelper.nativeProxyInvoke (Native method)
  at com.unity3d.player.ReflectionHelper.a (unavailable)
  at com.unity3d.player.ReflectionHelper$1.invoke (unavailable)
  at java.lang.reflect.Proxy.invoke (Proxy.java:1006)
  at com.google.android.gms.ads.initialization.OnInitializationCompleteListener.onInitializationComplete
  at com.google.android.gms.internal.ads.zzbdx.zzb
  at com.google.android.gms.internal.ads.zzbnp.zzbK
  at com.google.android.gms.internal.ads.zzhr.onTransact
  at android.os.Binder.transact (Binder.java:675)
  at zf.bb (com.google.android.gms.policy_ads_fdr_dynamite@213806100@213806100057.397311113.397311113:2)
  at com.google.android.gms.ads.internal.initialization.f.e (com.google.android.gms.policy_ads_fdr_dynamite@213806100@213806100057.397311113.397311113)
  at com.google.android.gms.ads.nonagon.initialization.c.run (com.google.android.gms.policy_ads_fdr_dynamite@213806100@213806100057.397311113.397311113:2)
  at android.os.Handler.handleCallback (Handler.java:907)
  at android.os.Handler.dispatchMessage (Handler.java:105)
  at ato.a (com.google.android.gms.policy_ads_fdr_dynamite@213806100@213806100057.397311113.397311113)
  at com.google.android.gms.ads.internal.util.f.a (com.google.android.gms.policy_ads_fdr_dynamite@213806100@213806100057.397311113.397311113:1)
  at ato.dispatchMessage (com.google.android.gms.policy_ads_fdr_dynamite@213806100@213806100057.397311113.397311113)
  at android.os.Looper.loop (Looper.java:216)
  at android.app.ActivityThread.main (ActivityThread.java:7625)
  at java.lang.reflect.Method.invoke (Native method)
  at com.android.internal.os.RuntimeInit$MethodAndArgsCaller.run (RuntimeInit.java:524)
  at com.android.internal.os.ZygoteInit.main (ZygoteInit.java:987)

About this issue

  • Original URL
  • State: closed
  • Created 3 years ago
  • Reactions: 23
  • Comments: 120 (2 by maintainers)

Most upvoted comments

As soon as we started receiving more of these ANRs without changing anything, we literally started losing tens of thousands of dollars every month due to being penalized by ANR rates. Despite big spend campaings, we’re not getting any organic users at all anymore. Our games are very lean, they have absolutely no issues with the exception of AdMob.

Steady 60fps, but as soon as we start loading ads, framerate drops considerably. So far we’re loading ads one by one, but availability becomes low, we’re losing ARPDAU, and many times this isn’t even enough. Especially on low end devices and especially on android 8.1

What are we supposed to do now, @google-admin? Just start losing money because you’re too lazy to fix your damn code?

History has never been kind to companies that get to have a monopoly and then mistreat their customers.

Hello there, what is the status of this issue? Just to mention few numbers, by 2020 Goggle Ad Network (Google revenues from advertising outside of its IP’s) were $23B. More or less 30%-40% of them are from Admob and probably something like 30% might come from this plugin. So, aproximately $2B revenues were generated from this plugin itself.

Most of the problems are affecting other plugins such as the C++ and/or Java ones, or even other libraries and services such as Firebase. But lets focus on this one, $2B per year, not a big deal for a company such Google but actually it is if we look at it with perspective. The strenght of Advertising is in the capacity to deliver targeted and valuable customers (own ecosystem + external one), if they begin to lose the trust of the advertisers from a publisher perspective they will be getting less data so they will become weaker and the performance of the whole ecosystem will begin to fall. This plugin is targetting mainly Games (Where Admob has a pretty dominant situation but getting weaker due to these problems) where customers spend most of their time when on mobile (+ social networks -> none of them google owned or even showing Goggle Ads). This is a really risky situation for you guys, it has been months of failures and delays.

Hello

@NVentimiglia What is the state of this issue? Why it’s not marked as P1 if so many people report it?

The same issue has drastically lowered our rankings. It’s ridiculous to get penalized from Google for a problem caused by Google…

Please prioritize this issue

We’re facing the same issue and our ranking in Google Play has been lowered as a result of that. This is a critical issue.

It’s unthinkable how such issues can be overlooked for so long. Countless companies are giving you all the info you need to fix it, and then some. We’ve lost unimaginable amounts of money due to being penalized by google for THEIR OWN MISTAKES. How can you sleep at night knowing you’re ruining other people’s hard work?

Hey @NVentimiglia

What is the state of this issue? Why do you ignore it?

hi everyone! what is the state of the issue? @NVentimiglia maybe you have some information?

What is the state of this issue?

Is there any workaround? Same problem, almost a year has passed.

We’ve switched to IronSource for now. The crashes and ANR went down dramatically.

@NVentimiglia could you please prioritize the issue? There a lot of evidence that issue has a massive behaviour. All of us just waiting the update and during this drammatic waiting we are getting tons of ANR which we just can’t avoid, we even got a lot of feedback from our players which can’t play because of this issue and all of this caused by Admob Unity plugin. Your latest reply looks like nobody from your team really took enough care to estimate how this problem is important and you continue asking so regular questions instead of give us the sure that your team is going to fix this. Please solve the issue.

Indeed having at least some update would be nice.

Our game has been impacted by a ton of ANR and it’s heavily affecting our revenue because of the lack of new players.

If Google devs think this might not be the cause, at least you could say it or participate in the conversation

Any ETA on this would be great news.

Following this conversation and some others makes it clear that this bug has a deeper root cause tied to the Android ads plugin and that has been going on for longer than A YEAR!

Hi, We are getting the same ANR’s We are looking to downgrade. Do we know exactly which versions are affected and which don’t? Is 6.0.0 affected?

Edit: Is version 5+ affected? I’m looking for a safe and tested version to downgrade to. Can anyone enlighten me?

Same problem

Here’s a flamegraph of all the stacktraces from ANR Exits we’ve found in our game. Again, there’s many more ANRs (over 1000) - but these are just ANR Exits

If this doesn’t help, idk what does 😛 flamegraph

Hi Everyone,

Thanks for raising awareness of this issue. We are looking at this issue, and I will do what I can to escalate the issue.

I think it’s time to work through https://groups.google.com/g/google-admob-ads-sdk/search?q=ANR and see when this all started. Also it seems as if https://bugs.chromium.org/p/chromium/issues/detail?id=1197928#c6 may be related - with the webview team being very attentive and believing the issue is resolved.

Also I noticed somebody else reduced ANRs by NOT initializing interstitial and reward ads in the same frame.

Please, Google, at least a feedback.

On Sun, 19 Dec 2021 at 23:48, Rizwan31840 @.***> wrote:

We are facing this bug since updating from googlemobileads version 5 to 6 and so on. this bug has bring our games to the floor, im afraid further delay in removing bug gonna destroy our years long progress in playstore!

— Reply to this email directly, view it on GitHub https://github.com/googleads/googleads-mobile-unity/issues/1825#issuecomment-997475724, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAURNYLAZYKOMLJA5B62QK3URZOM3ANCNFSM5GSCF2LQ . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.

You are receiving this because you commented.Message ID: @.***>

Still not fixed as of 02.03.22

Here’s some more data:

By ANR type : image

Android 7 and 12 not affected? Or too few sessions image

Affected devices: image

Vivo 1906 with 10.77% and realme c11 with 17.26% ANR rate seem to be high contenders to reproduce this issue. I hope this is not based on some country ads.

Chromebooks and Tablets higher ANR rate : image

Before this update we had a very low ANR rate - maybe around 0.45%.

We can also see that the app only locks up once, since our Multiple ANR rate is low : image

@NVentimiglia can you speak to your boss and refer this issue to address it?

We are facing this bug since updating from googlemobileads version 5 to 6 and so on. this bug has bring our games to the floor, im afraid further delay in removing bug gonna destroy our years long progress in playstore!

I guess we can close this topic for now because right now if you’re still working with ad mediation + almost adapter from them, you will always face this trouble because of various reasons (Android Webview, coding logic, library conflict,…).

Btw, I think almost ANR comes from ads that are using the Android Webview library (which handles it). As you can see from the image below, after Google updates Android Webview on Aug 2, our ANR + Crash Rate increased a lot, however, they updated the fix bug version on Sep 19 ==> Our ANR + Crash Rate came back to normal.

And after checking, Google just roll a 25% new version of Android Webview on Sep 27, I hope this will fix all conflict ads from the network. image

Btw, you just need to carefully check your apps in deeper to optimize every frame and try to make it run smoothly. You can read this article for more information about reducing the ANR (just for ref, not help so much in almost case that are using Unity): https://medium.com/okcredit/how-we-reduced-our-anr-by-three-times-d9ae0b41ad94

Thanks

It went from an average of 0.47% ANRs to 0.42% ANRs so it’s definitely a slight improvement, but the main ANR still persists, and we still will have to occasionally exclude devices that expose too many ANRs

So as far as I can see we now have a release 7.0.2 with SDK 21.0.0, I will test this new version and see how the ANRs will develop in the following days

Hey all, it looks like i managed to find a solution for my case. Our ANR rate was around 1-1.2% daily. We switched to ironsource mediation from Fyber mediation, kept all ad providers (AdMob, Applovin, Meta and Unity Ads), after that ANR dropped a little bit to around 0.8-1% daily. Then I added in firebase audience that collects players from devices like Xiaomi, Oppo, Realme, Vivo and all other devices that cause most of the ANR-s in our app and for them I turned off ironsource mediation initialization through firebase remote config. The first day I did this, ANR dropped to 0.35% and next day it was 0.42%. Will let you know how it goes in next few days. You don’t have to switch to ironsource mediation, you can try this solution with any. Just investigate what brand of devices are causing the most ANRs. Regarding revenue drop, it’s small but its there. The thing is that most of players who use those devices which caused ANRs for us are from countries where eCPM is lower.

Hope this helps! Next solution is to block certain device models from watching ads instead of the whole brand.

Edit: Problem with this approach is that in order for user to become part of of audience it takes up to 24 hours. So new installs from these devices can still cause ANR-s. But players from these devices that are playing everyday will be in audience and they wont get ad mediation initialization. If you can find a away to detect users from those devices instantly, I think it will have even better results.

SDK 21 is available in unityplugin 7.0.1?

On Thu, May 26, 2022 at 4:54 PM adiohaion @.***> wrote:

This may be the solution (still in beta) https://developers.google.com/admob/android/optimize-initialization?hl=en

— Reply to this email directly, view it on GitHub https://github.com/googleads/googleads-mobile-unity/issues/1825#issuecomment-1138461108, or unsubscribe https://github.com/notifications/unsubscribe-auth/AQZ756DY5OGGC3WCEJSOWELVL5Q7TANCNFSM5GSCF2LQ . You are receiving this because you commented.Message ID: @.***>

@NVentimiglia Any update on the situation? ANR Rate gets worse and worse every day

Google should hire more devs to fix this issue ASAP

So, Admob team promised the fix in early February, it’s almost mid of March and we still having tons of Admob caused ANRs. @NVentimiglia please report on the state of the issue, have a decency and respect towards the community

UPDATE: @alex-blastproofgames @mastef @mastef @genin43 @zhouzhuli 100% Crashes were fixed by upgrading to Unity 2020.3.22f1 and most of ANR were fixed too.

@djcvijic

As far as i understand, you should not do this: frame1

       
             this.interstitial = new InterstitialAd(Id);


                AdRequest request = new AdRequest.Builder().Build();
          
                this.interstitial.LoadAd(request);

frame1

            
                this.rewardedAd = new RewardedAd(Id);

      
                AdRequest request = new AdRequest.Builder().Build();
         
                this.rewardedAd.LoadAd(request);

I think it’s time to work through https://groups.google.com/g/google-admob-ads-sdk/search?q=ANR and see when this all started. Also it seems as if https://bugs.chromium.org/p/chromium/issues/detail?id=1197928#c6 may be related - with the webview team being very attentive and believing the issue is resolved.

Also I noticed somebody else reduced ANRs by NOT initializing interstitial and reward ads in the same frame.

Also I noticed somebody else reduced ANRs by NOT initializing interstitial and reward ads in the same frame.

It’s an interesting theory. I’ll try it in my game by initializing the Iron Source SDK with AdMob Adapter in IEnumerator with some delay.

I think that, at least, Google should confirm this problem in order that the users can move on replacing the library with something else.

Regards

On Wed, 5 Jan 2022 at 14:13, Michal Oláh @.***> wrote:

@mastef https://github.com/mastef @zhouzhuli https://github.com/zhouzhuli Same here. A lot of new ANRs starting from 22.12.2021 (the curve looks almost identical)

Unity 2020.3.0f1 IronSource 7.1.12.1 IronSource Google (AdMob and Ad Manager) 4.3.33

— Reply to this email directly, view it on GitHub https://github.com/googleads/googleads-mobile-unity/issues/1825#issuecomment-1005674775, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAURNYKH6FHPOVXGZ7T7OC3UUQ7YLANCNFSM5GSCF2LQ . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.

You are receiving this because you commented.Message ID: @.***>

@mastef @zhouzhuli Same here. A lot of new ANRs starting from 22.12.2021 (the curve looks almost identical)

Unity 2020.3.0f1 IronSource 7.1.12.1 IronSource Google (AdMob and Ad Manager) 4.3.33

@mastef I also encountered the same problem recently. I just updated the game version and updated the admob library, but there are a lot of ANRs, the ANR curve is the same as yours, do you know what caused it?

Unity 2019.1.1f1 Google Mobile Ads Unity Plugin v5.4.0

I have same problem at : Unity version: 2020.3.13f1 Google Mobile Ads Unity plugin version: v6.0.1 Platform: Android Platform OS version: API level 21 - 30 Mediation ad networks used, and their versions: None

We recently updated the app with the same conditions. Does the problem only happen with newly updated apps?

@denis-ftc-denisov indeed we are mediating AdMob via IS. We still get crashes and ANRs, but it’s still a marked improvement.

@Cptweirdo are you using Admob as ad source in IronSource’s mediation? We switched to IS and do use AdMob source, this improved crash and ANR rate but did not fix it completely.