WSL: gclient sync ran on WebRTC repository fails with [Err: panic: failed to execve "python": cannot allocate memory
Issue
Unable to properly fetch WebRTC repository under WSL, because python invocation fails with memory allocation error. The steps to reproduce mostly follows this WebRTC manual https://webrtc.org/native-code/development/
Environment details:
>ver
Microsoft Windows [Version 10.0.17093.1000]
> lsb_release -a:
No LSB modules are available.
Distributor ID: Ubuntu
Description: Ubuntu 16.04.3 LTS
Release: 16.04
Codename: xenial
➜ webrtc cat /proc/meminfo
MemTotal: 33491872 kB
MemFree: 21947188 kB
Buffers: 34032 kB
Cached: 188576 kB
SwapCached: 0 kB
Active: 167556 kB
Inactive: 157876 kB
Active(anon): 103104 kB
Inactive(anon): 17440 kB
Active(file): 64452 kB
Inactive(file): 140436 kB
Unevictable: 0 kB
Mlocked: 0 kB
SwapTotal: 41938300 kB
SwapFree: 41925628 kB
Dirty: 0 kB
Writeback: 0 kB
AnonPages: 102824 kB
Mapped: 71404 kB
Shmem: 17720 kB
Slab: 13868 kB
SReclaimable: 6744 kB
SUnreclaim: 7124 kB
KernelStack: 2848 kB
PageTables: 2524 kB
NFS_Unstable: 0 kB
Bounce: 0 kB
WritebackTmp: 0 kB
CommitLimit: 515524 kB
Committed_AS: 3450064 kB
VmallocTotal: 122880 kB
VmallocUsed: 21296 kB
VmallocChunk: 66044 kB
HardwareCorrupted: 0 kB
AnonHugePages: 2048 kB
HugePages_Total: 0
HugePages_Free: 0
HugePages_Rsvd: 0
HugePages_Surp: 0
Hugepagesize: 2048 kB
DirectMap4k: 12280 kB
DirectMap4M: 897024 kB
Steps to reproduce
- Choose desired working (in my case it looks like this)
➜ googlesource.com pwd
/home/mstyura/Projects/googlesource.com
# in my case Projects directory is symlink to folder on NTFS drive
➜ ~ ls -l
total 0
lrwxrwxrwx 1 mstyura mstyura 15 Feb 17 21:24 Projects -> /mnt/e/Projects
#
➜ ~ mount -l
rootfs on / type lxfs (rw,noatime)
sysfs on /sys type sysfs (rw,nosuid,nodev,noexec,noatime)
proc on /proc type proc (rw,nosuid,nodev,noexec,noatime)
none on /dev type tmpfs (rw,noatime,mode=755)
devpts on /dev/pts type devpts (rw,nosuid,noexec,noatime,gid=5,mode=620)
none on /run type tmpfs (rw,nosuid,noexec,noatime,mode=755)
none on /run/lock type tmpfs (rw,nosuid,nodev,noexec,noatime)
none on /run/shm type tmpfs (rw,nosuid,nodev,noatime)
none on /run/user type tmpfs (rw,nosuid,nodev,noexec,noatime,mode=755)
binfmt_misc on /proc/sys/fs/binfmt_misc type binfmt_misc (rw,noatime)
C: on /mnt/c type drvfs (rw,noatime,uid=1000,gid=1000)
D: on /mnt/d type drvfs (rw,noatime,uid=1000,gid=1000)
E: on /mnt/e type drvfs (rw,noatime,uid=1000,gid=1000)
F: on /mnt/f type drvfs (rw,noatime,uid=1000,gid=1000)
- Download depot tools by
git clone https://chromium.googlesource.com/chromium/tools/depot_tools
- Add depot tools to path
export PATH=$(pwd)/depot_tools:$PATH
- Create directory for WebRTC repository
mkdir webrtc && cd webrtc
- Run downloading of webrtc repository (on my environment it took about ~1hour)
fetch webrtc
Expected behavior
fetch webrtc
should succeed, but it failed when repository hooks were run
Actual behavior
Since fetch webrtc failed when it was running hooks, the repository is already there and to reproduce the issue again and again I just need to run gclient sync inside downloaded repository
➜ webrtc pwd
/home/mstyura/Projects/googlesource.com/webrtc
➜ webrtc ls
src
➜ webrtc cd src
➜ src git:(7c72c10) gclient sync
#long output skipped
5> Failed to fetch file gs://chromium-webrtc-resources/d19fb2879d091324c699987ab280da82af200933 for src/resources/paris_qcif.yuv, skipping. [Err: panic: failed to execve "/home/mstyura/.vpython-root/9eab6c/bin/python": cannot allocate memory
#long output skipped
strace -ff gclient sync produced thousands of files, some of them has failures found by ag "cannot allocate memory"
Attaching this files which contains memory allocation error: https://www.dropbox.com/s/iv6idf660s7mhrx/webrtc.zip?dl=0
About this issue
- Original URL
- State: closed
- Created 9 years ago
- Reactions: 2
- Comments: 19
Just an update, this should be fixed in Windows 10 Insider Preview Build 18342 (as mentioned in https://github.com/Microsoft/WSL/issues/3800). This will be included in the Windows 10 19H1 update, the codename for the 2019
AprilMay update.I’ve reproduced the bug myself by calling
pthread_createand having that thread call one of theexecfunctions. Theexeccall fails and setserrno = ENOMEMfor bizarre reasons. This is what is being fixed.Also, another workaround besides https://github.com/Microsoft/WSL/issues/2959#issuecomment-445991253 is to just re-run the failing command in a loop. Because of the randomness of which Python thread calls
exec, it occasionally works:windows_version:Microsoft Windows [Version 10.0.16299.334] Have 32GB physical RAM. No LSB modules are available. Distributor ID: Ubuntu Description: Ubuntu 16.04.3 LTS Release: 16.04 Codename: xenial
typing command gclient sync --reset --with_branch_heads the terminal print Running hooks: 50% (27/54) luci-go_linux ________ running ‘/usr/bin/python src/third_party/depot_tools/download_from_google_storage.py --no_resume --no_auth --bucket chromium-luci -d src/tools/luci-go/linux64’ in ‘/mnt/f/android/chromium’ panic: failed to execve “/home/daniel/.vpython-root/758a9e/bin/python”: cannot allocate memory
free command show subsytem memory 3.2G~can i make the memory larger? daniel@DESKTOP-7K3VVG7:/mnt/f/android/chromium/src$ free -m total used free shared buff/cache available Mem: 32645 14634 17780 17 230 17873 Swap: 29738 307 29431