rules_go: Cannot successfully extract go_sdk because of unicode filename
What version of rules_go are you using?
0.25.0
What version of gazelle are you using?
0.22.2
What version of Bazel are you using?
3.7.1
Does this issue reproduce with the latest releases of all the above?
Yes
What operating system and processor architecture are you using?
docker container (via dazel
) on macOS Big Sur (uname -a
== Linux 45ddc4b6c7ee 4.19.121-linuxkit #1 SMP Tue Dec 1 17:50:32 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux
)
Any other potentially useful information about your toolchain?
We use a custom c++ toolchain to build against musl, but that doesn’t seem relevant to this issue. We’re using go 1.15.6, using this go_download_sdk
invocation:
go_download_sdk(
name = "go_sdk",
# version = "1.15.6",
sdks = {
"darwin_amd64": ("go_sdk-darwin.tar.gz", "940a73b45993a3bae5792cf324140dded34af97c548af4864d22fd6d49f3bd9f"),
"linux_amd64": ("go_sdk-linux.tar.gz", "3918e6cc85e7eaaa6f859f1bdbaac772e7a825b0eb423c63d3ae68b21f84b844"),
},
urls = ["https://doesnotexistipromise.local/{}"],
)
What did you do?
Ran dazel test
on a go target in the repo
What did you expect to see?
Some test output
What did you see instead?
Extracting Bazel installation...
Starting local Bazel server and connecting to it...
INFO: Repository go_sdk instantiated at:
/Volumes/Projects/repo/WORKSPACE:62:16: in <toplevel>
/Users/swsnider/.cache/bazel/_bazel_swsnider/external/io_bazel_rules_go/go/private/sdk.bzl:129:21: in go_download_sdk
Repository rule _go_download_sdk defined at:
/Users/swsnider/.cache/bazel/_bazel_swsnider/external/io_bazel_rules_go/go/private/sdk.bzl:116:35: in <toplevel>
ERROR: An error occurred during the fetch of repository 'go_sdk':
Traceback (most recent call last):
File "/Users/swsnider/.cache/bazel/_bazel_swsnider/external/io_bazel_rules_go/go/private/sdk.bzl", line 100, column 16, in _go_download_sdk_impl
_remote_sdk(ctx, [url.format(filename) for url in ctx.attr.urls], ctx.attr.strip_prefix, sha256)
File "/Users/swsnider/.cache/bazel/_bazel_swsnider/external/io_bazel_rules_go/go/private/sdk.bzl", line 180, column 29, in _remote_sdk
ctx.download_and_extract(
Error in download_and_extract: java.io.IOException: Error extracting /Users/swsnider/.cache/bazel/_bazel_swsnider/external/go_sdk/temp18079659004892177486/go_sdk-linux.tar.gz to /Users/swsnider/.cache/bazel/_bazel_swsnider/external/go_sdk/temp18079659004892177486: /Users/swsnider/.cache/bazel/_bazel_swsnider/external/go_sdk/test/fixedbugs/issue27836.dir/?foo.go (Input/output error)
ERROR: Analysis of target '[REDACTED]' failed; build aborted: java.io.IOException: Error extracting /Users/swsnider/.cache/bazel/_bazel_swsnider/external/go_sdk/temp18079659004892177486/go_sdk-linux.tar.gz to /Users/swsnider/.cache/bazel/_bazel_swsnider/external/go_sdk/temp18079659004892177486: /Users/swsnider/.cache/bazel/_bazel_swsnider/external/go_sdk/test/fixedbugs/issue27836.dir/?foo.go (Input/output error)
INFO: Elapsed time: 267.761s
INFO: 0 processes.
FAILED: Build did NOT complete successfully (33 packages loaded, 19 targets configured)
FAILED: Build did NOT complete successfully (33 packages loaded, 19 targets configured)
Fetching /Users/swsnider/.cache/bazel/_bazel_swsnider/external/go_sdk; Extracting /Users/swsnider/.cache/bazel/_bazel_swsnider/external/go_sdk/temp18079659004892177486/go_sdk-linux.tar.gz 223s
About this issue
- Original URL
- State: closed
- Created 4 years ago
- Reactions: 2
- Comments: 15 (10 by maintainers)
Commits related to this issue
- go_download_sdk: work around Bazel .tar.gz extraction bug Use 'tar' installed on the system to extract .tar.gz archives instead of Bazel's download_and_extract. Go has at least one test with an inval... — committed to jayconrod/rules_go by deleted user 3 years ago
- go_download_sdk: work around Bazel .tar.gz extraction bug Use 'tar' installed on the system to extract .tar.gz archives instead of Bazel's download_and_extract. Go has at least one test with an inval... — committed to jayconrod/rules_go by deleted user 3 years ago
- go_download_sdk: work around Bazel .tar.gz extraction bug (#2836) Use 'tar' installed on the system to extract .tar.gz archives instead of Bazel's download_and_extract. Go has at least one test with... — committed to bazelbuild/rules_go by deleted user 3 years ago
- upgrade rules_go to v0.27.0 (#16065) in the resent version of rules_go, the issue https://github.com/bazelbuild/rules_go/issues/2771 was fixed. It should address the bazel build issue on some Linux o... — committed to QIvan/envoy by QIvan 3 years ago
- upgrade rules_go to v0.27.0 (#16065) (#16083) in the resent version of rules_go, the issue https://github.com/bazelbuild/rules_go/issues/2771 was fixed. It should address the bazel build issue on s... — committed to envoyproxy/envoy by QIvan 3 years ago
- upgrade rules_go to v0.27.0 (#16065) (#16083) in the resent version of rules_go, the issue https://github.com/bazelbuild/rules_go/issues/2771 was fixed. It should address the bazel build issue on som... — committed to gokulnair/envoy by QIvan 3 years ago
- [airgapped] use `ctx.download_and_extract` over `ctx.download` As we learned when attempting to use `rules_rust` in an airgapped environment (see https://github.com/lowRISC/rules_rust/commit/3ea1eda5... — committed to lowRISC/rules_go by timothytrippel 2 years ago
- go_download_sdk: apply extraction workaround to zips on non-windows OSs The Go distribution contains at least one test file with an invalid unicode name. Bazel cannot extract the distribution archive... — committed to jayconrod/rules_go by jayconrod a year ago
- go_download_sdk: apply extraction workaround to zips on non-windows OSs The Go distribution contains at least one test file with an invalid unicode name. Bazel cannot extract the distribution archive... — committed to jayconrod/rules_go by jayconrod a year ago
- go_download_sdk: apply extraction workaround to zips on non-windows OSs (#3563) * go_download_sdk: apply extraction workaround to zips on non-windows OSs The Go distribution contains at least one ... — committed to bazelbuild/rules_go by jayconrod a year ago
- go_download_sdk: apply extraction workaround to zips on non-windows OSs (#3563) * go_download_sdk: apply extraction workaround to zips on non-windows OSs The Go distribution contains at least one ... — committed to tingilee/rules_go by jayconrod a year ago
- Fix handling of non-ASCII characters in archive entry file names When creating a `PathFragment` from a ZIP or TAR entry file name, the raw bytes of the name are now wrapped into a Latin-1 encoded Str... — committed to bazelbuild/bazel by fmeum 9 months ago
- Fix handling of non-ASCII characters in archive entry file names When creating a `PathFragment` from a ZIP or TAR entry file name, the raw bytes of the name are now wrapped into a Latin-1 encoded Str... — committed to fmeum/bazel by fmeum 9 months ago
- [6.4.0] Fix handling of non-ASCII characters in archive entry file names (#19765) When creating a `PathFragment` from a ZIP or TAR entry file name, the raw bytes of the name are now wrapped into a L... — committed to bazelbuild/bazel by fmeum 9 months ago
@jayconrod would you consider restoring the previous workaround until bazelbuild/bazel#12986 is resolved? Looks like that’s slated for maybe Q2 of this year but until then I’m not aware of a workaround on affected machines without changes to
rules_go
.Agreed it makes sense to restore this workaround. This should be backported to release-0.24 and release-0.25.
Mirroring above request, we’re having some engineers hit this issue as well