fastlane: Rate limit has been exceeded for: itunes-connect|nonauth_url_spamControl|

I am also getting this error - Message: Rate limit has been exceeded for: itunes-connect|nonauth_url_spamControl|<my_ip_address>

I am trying to get a list of all users and their permissions using Spaceship

all_users = Spaceship::Tunes::Members.all
user = all_users[0] # for example
account_permissions = {}
account_permissions[:name] = teamname
account_permissions[:islive] = !user.not_accepted_invitation
account_permissions[:permissions] = user.roles
account_permissions[:apps] = user.has_all_apps ? ["all"] : user.selected_apps.map { |a| a.name if !a.nil? }
account_permissions[:lastupdated] = Time.now.strftime("%Y-%m-%d")

My code gets the details for first 15 or so users, and fails after with the above error message. Was able to find this information - https://stackoverflow.com/a/41377650/1946418; but not sure how true that is. The number seems too low?

Let me know if you need more information, please!

This is my Fastlane’s version - fastlane 2.113.0

About this issue

Original URL
State: closed
Created 5 years ago
Comments: 19 (18 by maintainers)

Most upvoted comments

Makes sense! Didn’t think of it from ‘attacking’ perspective 😃

I will look into this over the weekend (have to get used to Charles as well)!

scorpion35 on Jan 23, 2019

Got it. Usually 429 error code comes with the time limit we can try after, but Apple doesn’t seem to include it. I have implemented incremental sleep on my side, not a perfect solution, but is getting the job done!

This is my code for reference,

total_tries = 0
max_tries = 5 # Comes from a config file

begin
  get_all_users()
  $logger.info("")
  $logger.info("Script completed.")
rescue Exception => ex
  $logger.error("Message: #{ex.message}")
  $logger.error("Callstack: ")
  ex.backtrace.each do |cs| 
    $logger.error("\t\t#{cs}")
  end

  if ex.message.include?("nonauth_url_spamControl")
    total_tries = total_tries + 1 
    sleep(total_tries * 60) # Incrementally sleep, to give enough gap between each try.
    retry if total_tries <= max_tries # Retry blocked by Apple
  end # if ex.message.include?("nonauth_url_spamControl")

end # begin

If it fails even after max_tries has been reached, then 🤷‍♂️

Not sure if it’s possible, but can we request Apple to include a time limit in the header when they block the calls? This way we can implement the above sleeping solution in Fastlane itself?

Also, do you have any update on my PR, please? I am currently referencing my script to local fastlane changes, would have to move it to Jenkins build farm soon. Would appreciate if it’s merged soon.

scorpion35 on Jan 18, 2019

We unfortunately don’t have many data points on these rate limiting problems yet, as not many people did report their information in an issue here, so we also don’t have a good picture about the limits and possible reasons.

My absolute guess is that it is multi bucket based, so they have a counter for the last x min, 2x min and 10x min for example. If any of those hit a limit, you are temporarily blocked.

(We also have no idea yet if continuing requests after you get rate limited increases the limit, or if it would be a valid decision to keep “testing” every z seconds if you are unblocked again.)

(We also have no idea how long the rate limiting is in effect, so we also can’t implement a hard “do nothing for x minutes until the rate limiting is removed”)

janpio on Jan 18, 2019

Was able to test your suggestion @janpio and submitted a PR with changes - https://github.com/fastlane/fastlane/pull/14084

Note I am still running into this error. It’s very random when this block is applied on Apple’s side. I get the error on 150th user one time and on 86th user another time. However, the PR does optimize GET requests sent very nicely. Only one GET request per user with the fix!

I did change code in my script as well to -

user_apps = user.selected_apps
fresh_account_permissions[:apps] = user_apps.length == 0 ? ["all"] : user_apps.map { |a| a.name if !a.nil? }

Not calling user.has_all_apps anymore, as it triggers another GET request, and making Apple block my script even sooner.

I will implement a smart sleeping solution as suggested by Josh as well.

Please take a look at the PR and let me know if you need anything else from me!

scorpion35 on Jan 17, 2019

Copy that 😃

scorpion35 on Jan 17, 2019