fastlane: pilot and deliver both fail to upload binary file to Apple using

New Issue Checklist

Issue Description

On Monday morning, our app delivery process broke. We were on fastlane 2.152.0 at the time. We updated to 2.157.2 but encountered another bug that breaks the app builds entirely (https://github.com/fastlane/fastlane/issues/17122), so we then downgraded to 2.156.0

We are encountering this issue when attempting to upload apps using pilot or deliver:

10:36:34 [10:36:33]: [Transporter Error Output]: An error occurred while processing the http request for the webDAV upload.

10:36:34 [10:36:33]: [Transporter Error Output]: An exception has occurred: Authorization Required

10:36:35 [10:36:35]: Transporter transfer failed.
10:36:35 [10:36:35]: 
10:36:35 [10:36:35]: An error occurred while processing the http request for the webDAV upload.
10:36:35 An exception has occurred: Authorization Required

We had this variable setup since 2018, including when it suddenly broke: DELIVER_ITMSTRANSPORTER_ADDITIONAL_UPLOAD_PARAMETERS=-t DAV

We also tried the following variations:

- DELIVER_ITMSTRANSPORTER_ADDITIONAL_UPLOAD_PARAMETERS=-t Aspera
- DELIVER_ITMSTRANSPORTER_ADDITIONAL_UPLOAD_PARAMETERS=-t Signiant
- DELIVER_ITMSTRANSPORTER_ADDITIONAL_UPLOAD_PARAMETERS=-t Aspera,Signiant,DAV
- DELIVER_ITMSTRANSPORTER_ADDITIONAL_UPLOAD_PARAMETERS=<blank>

Each time we attempted the build, the resulting error was always the same for our account. However, we have successfully used this same system/environment to build the same app code (branded differently) for other Apple Developer accounts and delivered it successfully. We’ve also verified there are no prompts in the Apple dev portal such as accepting new terms, etc (as the build account and the account holder). The same build account is enabled as an Admin across multiple Apple Dev accounts and it’s hit or miss as to if we can push apps or not.

Environment

🚫 fastlane environment 🚫

Stack

Key Value
OS 10.15.3
Ruby 2.4.4
Bundler? true
Git git version 2.21.1 (Apple Git-122.3)
Installation Source ~/.rvm/gems/ruby-2.4.4@xxx_wlma_ios/bin/fastlane
Host Mac OS X 10.15.3 (19D76)
Ruby Lib Dir ~/.rvm/rubies/ruby-2.4.4/lib
OpenSSL Version OpenSSL 1.1.1d 10 Sep 2019
Is contained false
Is homebrew false
Is installed via Fabric.app false
Xcode Path /Applications/Xcode.app/Contents/Developer/
Xcode Version 11.3.1

System Locale

Variable Value
LANG en_US.UTF-8
LC_ALL en_US.UTF-8
LANGUAGE

fastlane files:

`./fastlane/Fastfile`
# This file contains the fastlane.tools configuration
# You can find the documentation at https://docs.fastlane.tools
#
# For a list of all available actions, check out
#
#     https://docs.fastlane.tools/actions
#
# For a list of all available plugins, check out
#
#     https://docs.fastlane.tools/plugins/available-plugins
#

# Uncomment the line if you want fastlane to automatically update itself
# update_fastlane

fastlane_require 'date'
fastlane_require 'active_support/all'


xxx_app_email = "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxx"


default_platform(:ios)

platform :ios do
  before_all do |lane, options|
    load_env_vars
    ENV['KEYCHAIN_PASSWORD'] = "xxxxxxxxxxxxx"
    ENV['KEYCHAIN_NAME'] = "xxxxxxxxxxxxxxxxxxx"

    pods_up_to_date = system("cd ..; diff Podfile.lock Pods/Manifest.lock > /dev/null")
    cocoapods unless pods_up_to_date

    create_build_keychain

    unless ['clean','show_help', 'update_plists', 'update_info_plist', 'update_config_plist', 'copy_customer_assets_into_project'].include? lane.to_s
      check_for_mobile_config_data(options)
      check_for_team_name(options)
      check_for_org_id(options)
    end
  end

  after_all do |lane|
	clear_build_keychain
  end

  error do |lane, exception, options|
    revert_signing_identity
    clear_build_keychain
  end

  desc "creates the keychain"
  lane :create_build_keychain do
    create_keychain(
      name: ENV['KEYCHAIN_NAME'],
      default_keychain: false,
      password: ENV['KEYCHAIN_PASSWORD'],
      unlock: true,
      timeout: false,
      lock_when_sleeps: true
    )
  end

  desc "deletes the created keychain"
  lane :clear_build_keychain do
    delete_keychain(name: ENV['KEYCHAIN_NAME'])
  end

  desc "Clean project build files"
  lane :clean do
    xcclean(scheme:'XXX')
  end

  desc "Show help"
  lane :show_help do |options|
    UI.header "Example of generating a release build for a customer in Fastlane if you don't have a document.xml handy..."
    UI.important 'fastlane customer_release team_name:"XXX Systems, LLC" org_id:NCC1701D app_name:"LeanPup Wash"'
    UI.header "Print out all custom lanes configured"
    UI.important 'fastlane lanes'
  end

  desc "For managing certificates and creating app within Developer and iTunes Connect Portals"
  lane :setup_new_app do |options|
    check_for_app_name(options)

    create_app(options)

    pull_all_signing_certs
  end

  desc "Builds a local Debug version"
  lane :local_beta do |options|
    match(
      type: 'appstore',
      git_branch: @match_git_org_branch,
      app_identifier: @bundle_id,
      keychain_name: ENV['KEYCHAIN_NAME'], 
      keychain_password: ENV['KEYCHAIN_PASSWORD'],
    )

    set_signing_identity

    gym(export_method: "app-store", clean: true)

    revert_signing_identity
  end

  desc "Builds & Deploys debug version to Customer's iTC Testflight"
  lane :customer_beta do |options|
    #ensure_git_status_clean(show_uncommitted_changes: true)

    # Make sure we have all signing certs and such ready to go!
    setup_new_app(options)

    match(
      type: 'appstore',
      git_branch: @match_git_org_branch,
      app_identifier: @bundle_id,
      keychain_name: ENV['KEYCHAIN_NAME'], 
      keychain_password: ENV['KEYCHAIN_PASSWORD'],
    )

    set_provisioning_profile

    set_signing_identity

    code_version = ENV['VERSION']
    custom_build = get_custom_build_number

    increment_version_number(
      version_number: code_version # Set a specific version number
    )

    increment_build_number(
      build_number: custom_build # Set a specific build number
    )

    formatted_version_name = format_version_name(code_version, custom_build)

    UI.important "iOS app version set to '#{formatted_version_name}'"

    write_version_file(formatted_version_name)

    gym(export_method: "app-store", clean: true)

    revert_signing_identity

    pilot(
      skip_waiting_for_build_processing: true,
      skip_submission: true,
      distribute_external: false,
      app_identifier: @bundle_id,
    )
  end

  desc "Builds & Deploys release version to Customer's iTC Store"
  lane :customer_release do |options|
    check_for_app_name(options)

    #ensure_git_branch(branch: 'master')
    #ensure_git_status_clean(show_uncommitted_changes: true)

    # Make sure we have all signing certs and such ready to go!
    setup_new_app(options)

    match(
      type: 'appstore',
      git_branch: @match_git_org_branch,
      app_identifier: @bundle_id,
      keychain_name: ENV['KEYCHAIN_NAME'], 
      keychain_password: ENV['KEYCHAIN_PASSWORD'],
    )

    set_provisioning_profile

    set_signing_identity

    code_version = ENV['VERSION']
    custom_build = get_custom_build_number

    #switch_to_git_tag_branch(code_version)

    increment_version_number(
      version_number: code_version # Set a specific version number
    )

    increment_build_number(
      build_number: custom_build # Set a specific build number
    )

    formatted_version_name = format_version_name(code_version, custom_build)
    write_version_file(formatted_version_name)

    gym(export_method: "app-store", clean: true)

    revert_signing_identity

    deliver(
      app_identifier: @bundle_id,
       keywords: {
        'en-US' => @app_team_name.scan(/\w+/)
      },  # Breaking up team_name on word boundaries for now... Creates keyword hash instead of array.
      release_notes: "",
      languages: "en-US",
      price_tier: 0,
      name: {
        'en-US' => @app_name
      }, # Sets app name as a hash instead of string.
      support_url: "",
      marketing_url: "",
      privacy_url: "",
      copyright: "#{Time.now.year} #{ENV['FASTLANE_ITC_TEAM_NAME']}",
      primary_category: "Lifestyle",
      app_review_information: {
        first_name: "xxxxxxxxxxx",
        last_name: "xxxxxxxxxxxxx",
        phone_number: "",
        email_address: "xxxxxxxxxxxxxxxxxx",
        notes: "",
      },
      submit_for_review: false,
      automatic_release: true,
      skip_metadata: true,  # Will not look on drive for metadata, but will still upload screenshots
      force: true, # Will not require verifying the upload via the HTML file since we are in non-interactive mode
    )
  end

  desc "Use XML file to update Info.plist & Config.plist with Customer's Preferences"
  lane :update_plists do |options|
    update_ios_info_plist(options)
    update_ios_config_plist(options)
  end

  private_lane :update_ios_info_plist do |options|

    unless options[:xml_path]
      UI.user_error! "xml_path: parameter missing. We need to know the Mobile Application's Customer Configuration!"
    end

    update_wlma_info_plist_from_config(
      xml_path: options[:xml_path],
      plist_path: 'XXX/Info.plist',
    )
  end

  private_lane :update_ios_config_plist do |options|

    unless options[:xml_path]
      UI.user_error! "xml_path: parameter missing. We need to know the Mobile Application's Customer Configuration!"
    end

    update_wlma_config_plist_from_config(
      xml_path: options[:xml_path],
      plist_path: 'XXX/Config/Config.plist',
    )
  end

  desc "Copy XML and Image assets into the project directory. Generally used in CI but can be tested locally"
  lane :copy_customer_assets_into_project do |options|

    unless options[:asset_path]
      UI.user_error! "asset_path: parameter missing. We need to know the Mobile Application's Customer Configuration Filepath!"
    end

    UI.important "asset_path set to: #{options[:asset_path]}"

    copy_ios_app_assets_into_project(
      asset_path: options[:asset_path],
    )
  end

  private_lane :set_provisioning_profile do
    # Manually determine the correct name of the Match certificate
    # in order to set the automatic signing of the bundle
    ENV["PROVISIONING_PROFILE_TARGET1"] = ENV["sigh_#{@bundle_id}_appstore"]
  end

  private_lane :set_signing_identity do

    team_id = ENV["sigh_#{@bundle_id}_appstore_team-id"]
    UI.important "Setting team_id in project to: #{team_id}"

    automatic_code_signing(
      path: "XXX.xcodeproj",
      use_automatic_signing: false,
      team_id: ENV["sigh_#{@bundle_id}_appstore_team-id"],
    )

    ENV['DEVELOPMENT_TEAM'] = ENV["sigh_#{@bundle_id}_appstore_team-id"]
    ENV['PROVISIONING_PROFILE_SPECIFIER'] = "match AppStore #{@bundle_id}"

    update_project_team(
      path: "XXX.xcodeproj",
      teamid: team_id,
    )

    update_project_provisioning(
      xcodeproj: "XXX.xcodeproj",
      profile: ENV["sigh_#{@bundle_id}_appstore_profile-path"],
      build_configuration: "Release",
    )

    update_app_identifier(
      xcodeproj: "XXX.xcodeproj", # Optional path to xcodeproj, will use the first .xcodeproj if not set
      plist_path: "XXX/Info.plist", # Path to info plist file, relative to xcodeproj
      app_identifier: @bundle_id # The App Identifier
    )

  end

  private_lane :revert_signing_identity do
    sh('cd ../ && git checkout -- XXX.xcodeproj')
    sh('cd ../ && git checkout -- XXX/')
  end

  private_lane :create_app do |options|
    # Fool Fastlane into thinking we're running on CI so that we get failures
    # if Customer has not properly registered xxxxxxxxxxxxxxxxxx to their Dev & iTC teams
    # regardless of running on Macos build slave or local dev machine.
    # We really do want this as turnkey as possible!
    ENV['JENKINS_HOME'] = 'Y'

    check_for_app_name(options)

    produce(
      username: xxx_app_email,
      app_identifier: @bundle_id,
      app_name: @app_name,
      app_version: "1.0",
      company_name: ENV['FASTLANE_ITC_TEAM_NAME'],
      language: 'English',
    )

    ENV.delete('JENKINS_HOME')
  end

  private_lane :pull_all_signing_certs do |options|
    register_devices(
      devices: {
        'XXX Developer iPhone' => 'abda7ad86e1e647d2578c79d1f895a62fcd6ceac'
      },
      team_name: @app_team_name
    )

    match(
      type: 'development',
      git_branch: @match_git_org_branch,
      app_identifier: @bundle_id,
      keychain_name: ENV['KEYCHAIN_NAME'], 
      keychain_password: ENV['KEYCHAIN_PASSWORD'],
    )

    match(
      type: 'appstore',
      git_branch: @match_git_org_branch,
      app_identifier: @bundle_id,
      keychain_name: ENV['KEYCHAIN_NAME'], 
      keychain_password: ENV['KEYCHAIN_PASSWORD'],
    )
  end

  def generate_bundle_id(org_id)
    return "com.carwash#{org_id}"
  end

  def switch_to_git_tag_branch(code_version)
    sh("git fetch --all --tags")
    sh("git checkout v#{code_version}")
  end

  def get_custom_build_number
    current_year = Date.today.year()
    day_of_year = Date.today.yday()

    unless ENV['BUILD_NUMBER']
      return "#{current_year}.#{day_of_year}.1"
    end

    "#{current_year}.#{day_of_year}.#{ENV['BUILD_NUMBER']}"
  end

  def format_version_name(version, build)
    "#{version} (#{build})"
  end
end


def check_for_mobile_config_data(options)
  path_to_load = './document.xml'

  if (options[:xml_path])
    path_to_load = options[:xml_path]
  end

  if (options[:asset_path])
    path_to_load = options[:asset_path] + '/Config/document.xml'
  end

  @config_data = get_mobile_app_config_data_from_xml(xml_path: path_to_load)
end

def check_for_app_name(options)
  unless @app_name
    @app_name ||= options[:app_name]
    @app_name ||= ENV['WLMA_MOBILE_APP_NAME']
    @app_name ||= @config_data.application_name

    if (options[:wlma_env])
      UI.important "Updating App Name for Mobile Connect Environment: #{options[:wlma_env]}"
      @app_name = @app_name + " " + options[:wlma_env].titleize
      UI.important "New App Name is: #{@app_name}"
    end
  end
end

def check_for_team_name(options)
  unless @app_team_name
    @app_team_name ||= options[:team_name]
    @app_team_name ||= ENV['WLMA_IOS_TEAM_NAME']

    if (@config_data)
      @app_team_name ||= @config_data.ios_team_name
    end

    team_name(@app_team_name)
    itc_team_name(@app_team_name)
  end
end

def check_for_org_id(options)
  unless @org_id
    @org_id ||= options[:org_id]
    @org_id ||= ENV['WLMA_ORG_ID']
    @org_id ||= @config_data.organization_id

    @match_git_org_branch = "org-#{@org_id}"

    @bundle_id = @config_data.application_id || generate_bundle_id(@org_id)

    if (options[:wlma_env])
      UI.important "Updating Bundle ID for Mobile Connect Environment: #{options[:wlma_env]}"
      @bundle_id = @bundle_id + "." + options[:wlma_env].downcase
      UI.important "New Bundle ID is: #{@bundle_id}"
    end
  end
end

def write_version_file(version)
  File.write('../version.tmp', version)
end

`./fastlane/Appfile`
# app_identifier("[[APP_IDENTIFIER]]") # The bundle identifier of your app
# apple_id("[[APPLE_ID]]") # Your Apple email address
apple_id("xxxxxxxxxxxxxxxxxxxxxxxxx")

# For more information about the Appfile, see:
#     https://docs.fastlane.tools/advanced/#appfile

fastlane gems

Gem Version Update-Status
fastlane 2.156.0 🚫 Update available

Loaded fastlane plugins:

No plugins Loaded

Loaded gems
Gem Version
did_you_mean 1.1.0
executable-hooks 1.6.0
bundler-unload 1.0.2
rubygems-bundler 1.4.5
bundler 1.16.6
rake 13.0.1
CFPropertyList 3.0.2
concurrent-ruby 1.1.5
i18n 0.9.5
minitest 5.11.3
thread_safe 0.3.6
tzinfo 1.2.5
activesupport 4.2.11.1
public_suffix 4.0.6
addressable 2.7.0
httpclient 2.8.3
json 2.3.1
algoliasearch 1.27.0
atomos 0.1.3
aws-eventstream 1.1.0
aws-partitions 1.363.0
aws-sigv4 1.2.2
jmespath 1.4.0
aws-sdk-core 3.105.0
aws-sdk-kms 1.37.0
aws-sdk-s3 1.79.1
babosa 1.0.3
claide 1.0.3
fuzzy_match 2.0.4
nap 1.1.0
cocoapods-core 1.8.0
cocoapods-deintegrate 1.0.4
cocoapods-downloader 1.2.2
cocoapods-plugins 1.0.0
cocoapods-search 1.0.0
cocoapods-stats 1.1.0
netrc 0.11.0
cocoapods-trunk 1.4.0
cocoapods-try 1.1.0
colored2 3.1.2
escape 0.0.4
fourflusher 2.3.1
gh_inspector 1.1.3
molinillo 0.6.6
ruby-macho 1.4.0
nanaimo 0.3.0
xcodeproj 1.18.0
cocoapods 1.8.0
coderay 1.1.2
colored 1.2
highline 1.7.10
commander-fastlane 4.4.6
declarative 0.0.20
declarative-option 0.1.0
digest-crc 0.6.1
unf_ext 0.0.7.7
unf 0.1.4
domain_name 0.5.20190701
dotenv 2.7.6
emoji_regex 3.0.0
excon 0.76.0
multipart-post 2.0.0
faraday 1.0.1
http-cookie 1.0.3
faraday-cookie_jar 0.0.7
faraday_middleware 1.0.0
fastimage 2.2.0
jwt 2.2.2
memoist 0.16.2
multi_json 1.15.0
os 1.1.1
signet 0.14.0
googleauth 0.13.1
mini_mime 1.0.2
uber 0.1.0
representable 3.0.4
retriable 3.1.2
google-api-client 0.38.0
google-cloud-env 1.3.3
google-cloud-errors 1.0.1
google-cloud-core 1.5.0
google-cloud-storage 1.28.0
mini_magick 4.10.1
plist 3.5.0
rubyzip 2.3.0
security 0.1.3
naturally 2.2.0
simctl 1.6.8
slack-notifier 2.3.2
terminal-notifier 2.0.0
unicode-display_width 1.7.0
terminal-table 1.8.0
tty-screen 0.8.1
tty-cursor 0.7.1
tty-spinner 0.9.3
word_wrap 1.0.0
rouge 2.0.7
xcpretty 0.3.0
xcpretty-travis-formatter 1.0.0
method_source 0.9.0
ox 2.11.0
pry 0.12.2

generated on: 2020-09-02

About this issue

  • Original URL
  • State: closed
  • Created 4 years ago
  • Comments: 16 (8 by maintainers)

Most upvoted comments

Believe it or not, I think I had a similar issue about 2 years ago @CovertLeopard 😥 And the same solution fixed it for me 😓

Just Apple ¯\_(ツ)_/¯

Thanks for reporting back! 💪

I know this is closed, but I wanted to follow up in case anybody else had issues. Apple support was completely useless. They never once gave any sort of update other than “The team looking into this is still investigating, but rest assured we will send you an update as soon as we get more info from them.”

Ultimately, there was an issue with the link between the build account we had been using for years and our root Apple developer account. This happened out of nowhere in a matter of a few hours. The build account showed as an admin on the team and could even log into the console and change things on the root account (certs and whatnot), but it couldn’t upload. Ultimately we had to remove the build account from our Apple dev team and then add it again (with the same permissions). For whatever reason, that fixed the issue for us.