fastlane: [2.107.0] Uploading metadata to iTunes Connect error: source sequence is illegal/malformed utf-8

This is a re-entry of issue #13125, which shouldn’t have been auto-closed since it was never resolved.

New Issue Checklist

Updated fastlane to the latest version
I read the Contribution Guidelines
I read docs.fastlane.tools
I searched for existing GitHub issues

Issue Description

I’m getting an error while running fastlane deliver

/Users/vsts/.gem-cache/gems/fastlane-2.107.0/spaceship/lib/spaceship/base.rb:66:in `to_json’: [!] source sequence is illegal/malformed utf-8 (JSON::GeneratorError)

Complete output when running fastlane, including the stack trace and command used

fastlane deliver --force -u *** -a com.gosmp.smpmobile -i /Users/vsts/agent/2.141.1/work/r1/a/_iOS release/drop/output/iphoneos/Distribution/_XcodeTaskExport_SMPNew/SMPNew.ipa --skip_binary_upload true -m /Users/vsts/agent/2.141.1/work/r1/a/_iOS release/drop/AppStore/metadata --skip_screenshots true --submit_for_review true [21:26:24]: [33mGet started using a Gemfile for fastlane https://docs.fastlane.tools/getting-started/ios/setup/#use-a-gemfile[0m [21:26:25]: Sending anonymous analytics information [21:26:25]: Learn more at https://docs.fastlane.tools/#metrics [21:26:25]: No personal or sensitive data is sent. [21:26:25]: You can disable this by adding `opt_out_usage` at the top of your Fastfile [21:26:25]: Login to App Store Connect (***) [21:26:27]: Login successful

[21:26:28]: Making sure the latest version on App Store Connect matches ‘1.7.11026’ from the ipa file… [21:26:30]: Successfully set the version to ‘1.7.11026’ [21:26:31]: Loading ‘/Users/vsts/agent/2.141.1/work/r1/a/_iOS release/drop/AppStore/metadata/en-US/description.txt’… [21:26:31]: Loading ‘/Users/vsts/agent/2.141.1/work/r1/a/_iOS release/drop/AppStore/metadata/en-US/release_notes.txt’… [21:26:31]: Loading ‘/Users/vsts/agent/2.141.1/work/r1/a/_iOS release/drop/AppStore/metadata/en-US/support_url.txt’… [21:26:31]: Loading ‘/Users/vsts/agent/2.141.1/work/r1/a/_iOS release/drop/AppStore/metadata/copyright.txt’… [21:26:32]: Activating language en-US… [21:26:57]: Uploading metadata to App Store Connect

Looking for related GitHub issues on fastlane/fastlane…

➡️ Uploading metadata to iTunes Connect error: source sequence is illegal/malformed utf-8 https://github.com/fastlane/fastlane/issues/13125 [closed] 4 💬 5 weeks ago

➡️ Uploading metadata to iTunes Connect error: source sequence is illegal/malformed utf-8 https://github.com/fastlane/fastlane/issues/11961 [closed] 3 💬 08 Jun 2018

🔗 You can ⌘ + double-click on links to open them directly in your browser. /Users/vsts/.gem-cache/gems/fastlane-2.107.0/spaceship/lib/spaceship/base.rb:66:in to_json': [!] source sequence is illegal/malformed utf-8 (JSON::GeneratorError) from /Users/vsts/.gem-cache/gems/fastlane-2.107.0/spaceship/lib/spaceship/base.rb:66:in to_json’ from /Users/vsts/.gem-cache/gems/fastlane-2.107.0/spaceship/lib/spaceship/tunes/tunes_client.rb:432:in block (2 levels) in update_app_version!' from /usr/local/lib/ruby/gems/2.5.0/gems/faraday-0.15.3/lib/faraday/connection.rb:384:in block in run_request’ from /usr/local/lib/ruby/gems/2.5.0/gems/faraday-0.15.3/lib/faraday/connection.rb:398:in block in build_request' from /usr/local/lib/ruby/gems/2.5.0/gems/faraday-0.15.3/lib/faraday/request.rb:26:in block in create’ from /usr/local/lib/ruby/gems/2.5.0/gems/faraday-0.15.3/lib/faraday/request.rb:25:in tap' from /usr/local/lib/ruby/gems/2.5.0/gems/faraday-0.15.3/lib/faraday/request.rb:25:in create’ from /usr/local/lib/ruby/gems/2.5.0/gems/faraday-0.15.3/lib/faraday/connection.rb:394:in build_request' from /usr/local/lib/ruby/gems/2.5.0/gems/faraday-0.15.3/lib/faraday/connection.rb:379:in run_request’ from /usr/local/lib/ruby/gems/2.5.0/gems/faraday-0.15.3/lib/faraday/connection.rb:175:in post' from /Users/vsts/.gem-cache/gems/fastlane-2.107.0/spaceship/lib/spaceship/client.rb:701:in block in send_request’ from /Users/vsts/.gem-cache/gems/fastlane-2.107.0/spaceship/lib/spaceship/client.rb:533:in with_retry' from /Users/vsts/.gem-cache/gems/fastlane-2.107.0/spaceship/lib/spaceship/client.rb:700:in send_request’ from /Users/vsts/.gem-cache/gems/fastlane-2.107.0/spaceship/lib/spaceship/client.rb:586:in request' from /Users/vsts/.gem-cache/gems/fastlane-2.107.0/spaceship/lib/spaceship/tunes/tunes_client.rb:430:in block in update_app_version!’ from /Users/vsts/.gem-cache/gems/fastlane-2.107.0/spaceship/lib/spaceship/tunes/tunes_client.rb:1433:in with_tunes_retry' from /Users/vsts/.gem-cache/gems/fastlane-2.107.0/spaceship/lib/spaceship/tunes/tunes_client.rb:429:in update_app_version!’ from /Users/vsts/.gem-cache/gems/fastlane-2.107.0/spaceship/lib/spaceship/tunes/app_version.rb:403:in save!' from /Users/vsts/.gem-cache/gems/fastlane-2.107.0/deliver/lib/deliver/upload_metadata.rb:130:in upload’ from /Users/vsts/.gem-cache/gems/fastlane-2.107.0/deliver/lib/deliver/runner.rb:120:in upload_metadata' from /Users/vsts/.gem-cache/gems/fastlane-2.107.0/deliver/lib/deliver/runner.rb:38:in run’ from /Users/vsts/.gem-cache/gems/fastlane-2.107.0/deliver/lib/deliver/commands_generator.rb:79:in block (2 levels) in run' from /usr/local/lib/ruby/gems/2.5.0/gems/commander-fastlane-4.4.6/lib/commander/command.rb:178:in call’ from /usr/local/lib/ruby/gems/2.5.0/gems/commander-fastlane-4.4.6/lib/commander/command.rb:153:in run' from /usr/local/lib/ruby/gems/2.5.0/gems/commander-fastlane-4.4.6/lib/commander/runner.rb:476:in run_active_command’ from /Users/vsts/.gem-cache/gems/fastlane-2.107.0/fastlane_core/lib/fastlane_core/ui/fastlane_runner.rb:76:in run!' from /usr/local/lib/ruby/gems/2.5.0/gems/commander-fastlane-4.4.6/lib/commander/delegates.rb:15:in run!’ from /Users/vsts/.gem-cache/gems/fastlane-2.107.0/deliver/lib/deliver/commands_generator.rb:182:in run' from /Users/vsts/.gem-cache/gems/fastlane-2.107.0/deliver/lib/deliver/commands_generator.rb:17:in start’ from /Users/vsts/.gem-cache/gems/fastlane-2.107.0/fastlane/lib/fastlane/cli_tools_distributor.rb💯in take_off' from /Users/vsts/.gem-cache/gems/fastlane-2.107.0/bin/fastlane:23:in <top (required)>’ from /usr/local/bin/fastlane:23:in load' from /usr/local/bin/fastlane:23:in <main>’

Environment

We’re using a Visual Studio Team Services hosted build server. I can’t run fastlane env, but this is what Microsoft says is installed on the current machine image.

-- Installation software information ---------

ruby 2.5.1p57 (2018-03-29 revision 63029) [x86_64-darwin17] Python 2.7.10 Python 3.7.0 pip 18.0 from /usr/local/lib/python3.7/site-packages/pip (python 3.7) Node.js v6.14.4 NPM 3.10.10 Git: git version 2.19.0 Git LFS: git-lfs/2.5.2 (GitHub; darwin amd64; go 1.11) curl 7.54.0 (x86_64-apple-darwin17.0) libcurl/7.54.0 LibreSSL/2.0.20 zlib/1.2.11 nghttp2/1.24.0 GNU Wget 1.19.5 built on darwin17.5.0. Homebrew 1.7.6 xctool 0.3.4 Bundler version 1.16.1 CocoaPods 1.5.3 xcpretty 0.3.0 Nomad CLI 2.7.6 Nomad CLI ipa 0.14.3 Mobile Center CLI 0.12.0 App Center CLI 1.1.2 fastlane 2.105.2 Carthage 0.30.1 Gradle 4.10.2 PowerShell 6.1.0 Yarn 1.10.1 java 1.7 jdk: 1.7.0_80 java 1.8 jdk: 1.8.0_181 (default) java 9 jdk: 9.0.4 java 10 jdk: 10.0.2 java 11 jdk: 11 Apache Maven 3.5.4 (1edded0938998edf8bf061f1ceb3cfdeccf443fe; 2018-06-17T18:33:14Z) .NET SDK 1.0.1 1.0.4 1.1.10 1.1.4 1.1.5 1.1.7 1.1.8 1.1.9 2.0.0 2.0.3 2.1.100 2.1.101 2.1.102 2.1.103 2.1.104 2.1.105 2.1.2 2.1.200 2.1.201 2.1.202 2.1.300 2.1.301 2.1.302 2.1.4 2.1.400 2.1.401 2.1.402 azure-cli (2.0.46) Go 1.11 Subversion (SVN) 1.10.2 Miniconda 4.5.11 macOS toolcache Python 2.7.15 3.4.8 3.5.5 3.6.5 3.7.0

– OS X info ----------------------------------

sw_vers ProductName: Mac OS X ProductVersion: 10.13.6 BuildVersion: 17G65

system_profiler SPSoftwareDataType Software:

System Software Overview:

System Version: macOS 10.13.6 (17G65) Kernel Version: Darwin 17.7.0 Boot Volume: Macintosh HD Boot Mode: Normal Computer Name: Mac (215) User Name: vsts (vsts) Secure Virtual Memory: Enabled System Integrity Protection: Enabled Time since boot: 2:29

– Disk ---------------------------------------

Total size: 407162134528 bytes (379Gi) Free: 111746826240 bytes (104Gi)

About this issue

Original URL
State: closed
Created 6 years ago
Reactions: 1
Comments: 53 (13 by maintainers)

Commits related to this issue

spaceship: log when to_json fails and verbose? #13608 — committed to lacostej/fastlane by lacostej 5 years ago
[spaceship] Feature to troubleshoot to_json issues, related to #13608 (#14682) * spaceship: log when to_json fails and verbose? #13608 * Allow to see the output of the test when DEBUG is defined — committed to fastlane/fastlane by lacostej 5 years ago
[spaceship] Feature to troubleshoot to_json issues, related to #13608 (#14682) * spaceship: log when to_json fails and verbose? #13608 * Allow to see the output of the test when DEBUG is defined — committed to minuscorp/fastlane by lacostej 5 years ago

Most upvoted comments

After re-encoding all our metadata files to UTF-8, we no longer encounter this bug.

SEnglishSMP on May 1, 2020

I reproduced this issue, and resolved myself.

It looks like accidentally, the release_notes.txt files are encoded with something other than UTF-8, and the files just contains some characters that is could not be recognized.

After re-encoding to UTF-8, it went back to normal.

quzongyao on Mar 16, 2020

@janpio, here’s the output from fastlane along with the debug file from spaceship. I’ve replaced sensitive information with asterisks, but all of that was just normal, printable ASCII characters.

fastlane output.txt spaceship debug output.txt

SEnglishSMP on Jul 24, 2019

Haven’t yet pushed anything new to Apple review yet. I’ll update the ticket when we do.

SEnglishSMP on Jun 21, 2019

@lacostej @janpio Attached is the output from the new fastlane build showing the data that’s causing the error. I’ve replaced sensitive information with asterisks, but all of that was just normal, printable ASCII characters.

fastlane debug output.txt

SEnglishSMP on May 14, 2019

Still getting this with fastlane 2.117.1.

SEnglishSMP on Mar 11, 2019

If you encountered this problem after updating to 2.109.0:

PR fixing this issue is at #13825.

We are going to review and merge this as fast as possible and release 2.109.1.

Sorry for the trouble, I am ashamed (and confused) that I didn’t catch this while testing the new logging code. Shouldn’t have happened 🙇‍♀️

More information here: https://github.com/fastlane/fastlane/issues/13815

janpio on Dec 5, 2018

It seems like you have not included the output of fastlane env

To make it easier for us help you resolve this issue, please update the issue to include the output of fastlane env 👍

fastlane-bot on Dec 5, 2018