fastlane: Google Api Error: Unauthorized - Request is missing required authentication credential when trying to do a Google Play release

New Issue Checklist

Issue Description

We have a problem that started 2 days ago with publishing Google Play releases. When running the release_playstore lane that use supply Google APIs return Google Api Error: Unauthorized - Request is missing required authentication credential. We have checked the service account key and it is valid and does not have an expiry date. Can someone experienced take a look at this problem. The assumption is Google made changes to the way API requests need to be done.

Command executed
lane :release_playstore do
  identifier = CredentialsManager::AppfileConfig.try_fetch_value(:app_identifier)
  supply(
     package_name: identifier,
     metadata_path: "fastlane/metadata/android/#{identifier}",
)
end
Complete output when running fastlane, including the stack trace and command used
| release_status                     | completed                                                          |
| track                              | beta                                                               |
| json_key                           | /home/vsts/work/_temp/fastlane-serviceaccount.json                 |
| aab                                | /home/vsts/work/1/xxx-Android-Production/xxx-release.aab |
| skip_upload_apk                    | false                                                              |
| skip_upload_aab                    | false                                                              |
| skip_upload_metadata               | false                                                              |
| skip_upload_changelogs             | false                                                              |
| skip_upload_images                 | false                                                              |
| skip_upload_screenshots            | false                                                              |
| track_promote_release_status       | completed                                                          |
| validate_only                      | false                                                              |
| check_superseded_tracks            | false                                                              |
| timeout                            | 300                                                                |
| deactivate_on_promote              | true                                                               |
| changes_not_sent_for_review        | false                                                              |
| rescue_changes_not_sent_for_review | true                                                               |
| ack_bundle_installation_warning    | false                                                              |
+------------------------------------+--------------------------------------------------------------------+

[09:28:36]: Preparing aab at path '/home/vsts/work/1/golive-Android-Production/xxx-release.aab' for upload...
[09:29:02]: Updating track 'beta'...
[09:29:03]: Preparing to upload for language 'de-DE'...
[09:29:04]: Updating changelog for '14551' and language 'de-DE' to default changelog...
[09:29:04]: Preparing to upload for language 'en-US'...
[09:29:04]: Uploading image file fastlane/metadata/android/xxx/en-US/images/featureGraphic.png...
[09:29:07]: Uploading image file fastlane/metadata/android/xxx/en-US/images/icon.png...
[09:29:10]: Uploading screenshot fastlane/metadata/android/xxx/en-US/images/phoneScreenshots/1_en-US.png...
+---------------+---------------------------+
|               Lane Context                |
+---------------+---------------------------+
| ENVIRONMENT   | xxx                    |
| PLATFORM_NAME | android                   |
| LANE_NAME     | android release_playstore |
+---------------+---------------------------+
[09:29:12]: Google Api Error: Unauthorized - Request is missing required authentication credential. Expected OAuth 2 access token, login cookie or other valid authentication credential. See https://developers.google.com/identity/sign-in/web/devconsole-project.

+------+---------------+-------------+
|          fastlane summary          |
+------+---------------+-------------+
| Step | Action        | Time (in s) |
+------+---------------+-------------+
| 1    | opt_out_usage | 0           |
| đź’Ą   | supply        | 37          |
+------+---------------+-------------+
[09:29:12]: fastlane finished with errors

[!] Google Api Error: Unauthorized - Request is missing required authentication credential. Expected OAuth 2 access token, login cookie or other valid authentication credential. See https://developers.google.com/identity/sign-in/web/devconsole-project.
 [REPLACE THIS WITH YOUR INFORMATION] 

Environment

 [REPLACE THIS WITH YOUR INFORMATION] 

About this issue

  • Original URL
  • State: closed
  • Created 10 months ago
  • Reactions: 47
  • Comments: 37 (4 by maintainers)

Commits related to this issue

Most upvoted comments

We shipped fastlane 2.215.1 this weekend, which includes my patch to workaround this Google API issue.

Once you update to fastlane 2.215.1, set the SUPPLY_UPLOAD_MAX_RETRIES env var to an integer to auto-retry each randomly failed Google API error.

You can do this either in your fastalane/.env file (if you use one), or inside your Fastfile using ENV['SUPPLY_UPLOAD_MAX_RETRIES']='5' for example.

Alternatively, you can also pass that value on each call to fastlane (especially since this Google API random error is an issue on Google’s side that will hopefully get resolved by Google soon-ish) via SUPPLY_UPLOAD_MAX_RETRIES=5 fastlane your_lane_that_calls_upload_to_play_store

We just released fastlane 2.215.0 which contain some (opt-in) changes which optimize Google API calls during screenshots upload (sync_image_upload: true). That should help significantly reduce the number of requests done during skip_upload_screenshots: false and thus make it less likely to hit this random API error.

That sync_image_upload: option was not initially developed in the context of this issue — but rather in an unrelated goal of making supply not uploading screenshots that are already present in Google Play — and that won’t fully solve the problem (which is on Google’s side)… but reducing the likeliness of hitting the random issues with Google API like the one we see in this issue will still be a nice side-effect of that new opt-in option 🙂

Try setting the root_url:

upload_to_play_store(
    root_url: "https://androidpublisher.googleapis.com/",
    ...
)

2.215.1 + SUPPLY_UPLOAD_MAX_RETRIES did the trick for us! Thank you !

Someone on the Android DevRel team at Google contacted me 5h ago saying they were made aware of this and were monitoring this GitHub issue, and that the Play team had identified an issue which may have contributed to this issue and believe they have now fixed this. 🤞

They added:

It isn’t confirmed that the issue that they identified and fixed was indeed the cause of the issue Fastlane has been experiencing, but we’d be interested to know whether users of Fastlane are now seeing an improvement, with no more failures, and no more need for retries?

So feel free to comment here to indicate if from now on your upload_to_play_store calls don’t even need retrying anymore and the issue is fixed, or if you’re still experiencing issues even after their patch.

Like @AliSoftware, I just wrote a script to repeatedly keep trying the deployment and it eventually goes through. It’s a problem on Google’s end.

Lots of folks are (were?) having the same issue here: https://github.com/fastlane/fastlane/issues/21091

As @anh1979 mentioned, after updating Fastlane version, we added the SUPPLY_UPLOAD_MAX_RETRIES to our Jenkins pipeline and it just worked… Even saw the Retring... message in the logs

fl-21507

Try setting the root_url:

upload_to_play_store(
    root_url: "https://androidpublisher.googleapis.com/",
    ...
)

Worked as a charm! Thank you

2.215.1 + SUPPLY_UPLOAD_MAX_RETRIES did the trick for us! Thank you !

Can you share a code snippet of how you set SUPPLY_UPLOAD_MAX_RETRIES?

Unfortunately, setting the root_url does not solve the issue… This seems to be a timing issue like the lease on the authorization token is not long enough. The deployment does not always fail in the same spot and network speed seems to play apart as well.

The only way I have been able to get an app uploaded with upload_to_play_store is by disabling the image and screenshots. I’ve even tried uploading them separately but it always seems to fail at some point uploading screenshots.

upload_to_play_store(
    root_url: "https://androidpublisher.googleapis.com/",
     skip_upload_images: true,
     skip_upload_screenshots: true,
    ...
)

I’m pretty sure the issue is on Google’s end, because the request failure is completely random, sometimes it passes sometimes it doesn’t.

When you’re doing only one apk upload request to the Google API it can be enough to retry a couple of times until it passes, but if you need to upload new screenshots for example, that requires many Google API calls (one per image upload), so there is a way higher chance that one of them will fail along the many ones that are sent.


In the meantime—i.e. until Google fixes it on their end—I was able to apply a quick-and-dirty patch to my local copy of the fastlane gem:

  1. Open vendor/bundle/ruby/2.7.0/gems/fastlane-2.214.0/supply/lib/supply/client.rb in your favorite code editor
    • Adjust the path to wherever your local fastlane gem is located (e.g. if you don’t use bundler or don’t use vendor/bundle as your BUNDLE_PATH for your bundler config like we do)
    • Adjust the -2.214.0 in that path according to which fastlane version your project is using Apply the following changes to the call_google_api method
  2. Apply the following changes to the def call_google_api method:
     def call_google_api
+      tries_left ||= 5
       yield if block_given?
     rescue Google::Apis::Error => e
       error = begin
@@ -92,7 +93,13 @@ module Supply
         message = e.body
       end
 
-      UI.user_error!("Google Api Error: #{e.message} - #{message}")
+      if tries_left.positive?
+        UI.error("Google Api Error: #{e.message} - #{message} - Retrying...")
+        tries_left -= 1
+        retry
+      else
+        UI.user_error!("Google Api Error: #{e.message} - #{message}")
+      end
     end

Obviously this is a temporary workaround and dirty patch, that is only aimed to get you out of a bind if you need to call upload_to_play_store today and can’t wait for Google to fix the issue on their end hopefully soon. Once you’re done with it, you should ideally revert that patch (e.g. remove vendor/bundle then re-install the original fastlane gem code from scratch using bundle install again).


Note about why not making a PR to fastlane with that patch

I thought about submitting a PR to fastlane but I think this would need a nicer implementation than my dirty patch above — especially adding a max_retries option to Supply::Options that you could then pass as a parameter to upload_to_play_store, etc.

Also I’m pretty sure this is a very exceptional case that we should probably not officially cover for in fastlane’s end but in GoogleApis’ end, as the Google::Apis gem that supply uses internally already implements a retry mechanism.

Indeed, the reason why this mechanism from the Google:Apis gem doesn’t retry in the case we’re seeing here is because Google::Apis::AuthorizationError is not one of the codes for which they retry; which makes sense because if the error was really about the request not including an OAuth token in the headers or had incorrect credentials, there would be no point in retrying the exact same request (still without valid authentication) over and over anyway. Which kinda confirms that this being a random issue is really on Google servers’ side.

+1 works after retry