go: x/build: "fatal error: out of memory" on windows-arm64-11
#!watchflakes
post <- builder ~ `^windows-arm64.*` && `fatal error: .*(?:out of memory|cannot allocate memory|The paging file is too small|Insufficient system resources)`
greplogs --dashboard -md -l -e '(?ms)\Awindows-arm64.*^fatal error: out of memory' --since=2021-01-01
2022-02-04T14:02:15-25d2ab2-4afcc9f/windows-arm64-11
We may need to reconfigure the builder to either turn down the build/test parallelism or have more RAM available.
There is only one of these failures in the logs, but OTOH this builder has only ever run x/tools 12 times — so that’s an 8% failure rate for this repo so far. 😅
(attn @golang/release)
About this issue
- Original URL
- State: open
- Created 2 years ago
- Reactions: 2
- Comments: 31 (25 by maintainers)
Commits related to this issue
- dashboard: reduce GOMAXPROCS on Windows 11 ARM64 The Windows 11 ARM64 builder is experiencing occasional OOMs while building tests. Reducing GOMAXPROCS will reduce the go command's parallelism and ho... — committed to golang/build by heschi 2 years ago
- dashboard: add known issue for windows-arm64-11 For golang/go#52653. Updates golang/go#51019. Change-Id: Ie57f7b2c2b6d4c3cc4b5f5f886773dff2a36a61e Reviewed-on: https://go-review.googlesource.com/c/b... — committed to golang/build by bcmills 2 years ago
I’m no longer sure this is a builder issue. I’m facing some OOM errors like this one
runtime: VirtualAlloc of 274879168512 bytes failed with errno=1455while running a test that shouldn’t need 274GB to run… There could be a problem in the runtime, will give it a try.The first page of the build dashboard shows 4 windows-arm64-11 fails that are all memory related, reopening.
greplogs since 2022-06-08
greplogs --dashboard -md -l -e '(?ms)\Awindows-arm64.*^fatal error: .*(?:out of memory|cannot allocate memory)' --since=2022-06-082022-07-20T23:32:27-244c8b0/windows-arm64-11 2022-07-20T23:29:03-df38614/windows-arm64-11 2022-07-20T16:45:46-bb1749b/windows-arm64-11 2022-07-19T14:08:07-c8730f7-ae7340a/windows-arm64-11 2022-07-18T15:58:37-c0c1bbd/windows-arm64-11 2022-07-18T12:16:49-f839522-88a06f4/windows-arm64-11 2022-07-18T12:16:49-f839522-2aa473c/windows-arm64-11 2022-07-13T15:21:36-c27b92c-88a06f4/windows-arm64-11 2022-07-05T12:57:42-84e091e/windows-arm64-11 2022-07-04T14:31:39-ceda93e/windows-arm64-11 2022-07-01T14:13:36-93bf1fc-c847a2c/windows-arm64-11 2022-07-01T13:38:07-79fefdf-c847a2c/windows-arm64-11 2022-07-01T13:37:03-fa4babc-c847a2c/windows-arm64-11 2022-06-29T15:00:52-bd1783e/windows-arm64-11 2022-06-28T13:00:32-751cae8/windows-arm64-11 2022-06-25T19:07:01-4f45ec5/windows-arm64-11 2022-06-24T13:47:25-2994e99-d38f1d1/windows-arm64-11 2022-06-17T15:02:55-694bf12-6c25ba6/windows-arm64-11 2022-06-16T15:42:24-041035c-ecc268a/windows-arm64-11 2022-06-16T14:42:12-d097bc9-ecc268a/windows-arm64-11 2022-06-13T15:38:06-d27128b/windows-arm64-11 2022-06-09T15:15:48-840e99e/windows-arm64-11 2022-06-09T15:15:48-0fc6e7c-840e99e/windows-arm64-11 2022-06-09T15:02:08-1d19788-1a2ca95/windows-arm64-11Tracking the new Microsoft-provided
windows/arm64builder in #53541, which will hopefully fix this issue.