pex: /usr/bin/python3: can't find '__main__' module in 'blah.pex'

Okay, I recently had this problem as well and it turns out that python doesn’t know how to handle .pex files that are larger than 2gb in size and that gives you the incredibly useful “Can’t find main” error. So, check the size of your pex, if it is >2gb, it’s time to go on a dependency diet or try and find other ways to package/deploy your project

@dgkatz - finally looping back. was your issue related to the huge PEX issue @tentwelfths and @rom1504 encountered (>2GB PEX)? If so, I’d like to close this issue since @rom1504 confirms the --layout packed Pex option is a viable workaround.

@tentwelfths hopefully --layout packed gives you an escape hatch too when you simply can’t pare down dependencies.

Sorry to see this so late - thanks for the bump with data @tentwelfths. Perhaps you could try the next Pex release or Pex master which now support a --venv mode. If you build your PEX file using that flag, when run the PEX file will unpack itself into a virtual environment under ~/.pex/venvs and re-execute from there. The upshot is the PEX runs just like any other Python application and size limits on the PEX zip file, etc are all sidestepped.

@rom1504 Pex does already build in parallel using your number of cores by default (see --help for --jobs). The build process looks like:

1. Single subprocess: pip download (this performs the resolve and downloads wheels and sdists)
2. –jobs number of sub-processes: pip wheel (this builds any downloaded sdists into wheels)
3. –jobs number of sub-processes: pip install (this installs each wheel in its own directory) The only remaining non-parallelized aspect of the PEX build for packed layout is zipping up all the individual wheel install chroots from step 3. That is done in serial: https://github.com/pantsbuild/pex/blob/7a6e9a46c7e4fc67c6d3f1a0fc19d5b204d5ee81/pex/pex_builder.py#L696-L719

@tentwelfths I repro, although I get a different error message:

$ rm -rf big* && mkdir big && yes "#" | head -n 2000000000 > big/data.py && echo "import data; print(data.__file__)" > big/exe.py && ls -lh big/
total 3.8G
-rw-r--r-- 1 jsirois jsirois 3.8G Jan  7 11:17 data.py
-rw-r--r-- 1 jsirois jsirois   34 Jan  7 11:17 exe.py
$ pex -D big/ --entry-point exe -obig.pex && ls -lh big.pex
-rwxr-xr-x 1 jsirois jsirois 4.1M Jan  7 11:18 big.pex
$ ./big.pex 
Traceback (most recent call last):
  File "/home/jsirois/dev/pantsbuild/jsirois-pex/big.pex/.bootstrap/pex/pex.py", line 487, in execute
  File "/home/jsirois/dev/pantsbuild/jsirois-pex/big.pex/.bootstrap/pex/pex.py", line 404, in _wrap_coverage
  File "/home/jsirois/dev/pantsbuild/jsirois-pex/big.pex/.bootstrap/pex/pex.py", line 435, in _wrap_profiling
  File "/home/jsirois/dev/pantsbuild/jsirois-pex/big.pex/.bootstrap/pex/pex.py", line 543, in _execute
  File "/home/jsirois/dev/pantsbuild/jsirois-pex/big.pex/.bootstrap/pex/pex.py", line 645, in execute_entry
  File "/home/jsirois/dev/pantsbuild/jsirois-pex/big.pex/.bootstrap/pex/pex.py", line 653, in execute_module
  File "/usr/lib/python3.9/runpy.py", line 213, in run_module
    return _run_code(code, {}, init_globals, run_name, mod_spec)
  File "/usr/lib/python3.9/runpy.py", line 87, in _run_code
    exec(code, run_globals)
  File "/home/jsirois/dev/pantsbuild/jsirois-pex/big.pex/exe.py", line 1, in <module>
  File "<frozen importlib._bootstrap>", line 1007, in _find_and_load
  File "<frozen importlib._bootstrap>", line 982, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 925, in _find_spec
  File "<frozen importlib._bootstrap_external>", line 1349, in find_spec
  File "<frozen importlib._bootstrap_external>", line 1323, in _get_spec
  File "<frozen importlib._bootstrap_external>", line 1304, in _legacy_get_spec
  File "<frozen importlib._bootstrap>", line 423, in spec_from_loader
  File "<frozen importlib._bootstrap_external>", line 656, in spec_from_file_location
  File "<frozen zipimport>", line 191, in get_filename
  File "<frozen zipimport>", line 709, in _get_module_code
  File "<frozen zipimport>", line 560, in _get_data
OSError: zipimport: can't read data

But the --unzip remedy works. Here I simply use the runtime PEX_UNZIP (see pex --help-variables) equivalent instead of rebuilding the PEX file with --unzip. Slow 1st run when the initial unzip happens, fastish after that:

$ time PEX_UNZIP=1 ./big.pex 
/home/jsirois/.pex/unzipped_pexes/d5e0ee5a82eafd6fe49ccc04bc54f8ec86a7218c/data.py

real	0m58.322s
user	0m46.461s
sys	0m4.554s
$ time PEX_UNZIP=1 ./big.pex 
/home/jsirois/.pex/unzipped_pexes/d5e0ee5a82eafd6fe49ccc04bc54f8ec86a7218c/data.py

real	0m0.584s
user	0m0.448s
sys	0m0.034s

Specifically, --venv mode was added in #1153. Going to grab that PR link though reminded me the existing --unzip mode should provide the same remedy in this case. Perhaps you could also or alternatively try that?