bazel: Windows: shell script cannot be used as an 'executable' for an action

This is an attempt to fix command line limit problem reported in #3579. It’s usual approach to work with a response file. So I adapted the code in question from passing the whole 25k command, that was truncated on 8191 character directly to ctx.action(), that was going through the shell:

ctx.action(
    inputs = inputs,
    outputs = [war],
    mnemonic = 'WAR',
    command = '\n'.join(cmd),
    use_default_shell_env = True,
)

with an intermediate step, generated shell script and pass it as executable to ctx.action() instead:

  # Executable script to generate war archive
  gen_war = ctx.new_file(ctx.label.name + "_gen_war.sh")
  ctx.file_action(gen_war, '\n'.join(cmd), True)

   ctx.action(
     inputs = inputs,
     outputs = [war],
     mnemonic = 'WAR',
     executable = gen_war,
     use_default_shell_env = True,
   )

This works as expected on Linux, but is failing on Windows with:

CreateProcess(): %1 is not a valid Win32 application.

I would expect that Bazel does some heuristic here and guess that the passed file is a shell script, for example, by trying to parse the first file line and see whether or not the first line is a shebang or something and shell out instead of trying to pass previously generated shell script directly to create process.

About this issue

Original URL
State: closed
Created 7 years ago
Comments: 31 (31 by maintainers)

Commits related to this issue

Bazel: Replace ctx.action with actions#run / actions#run_shell ctx.action() is deprecated in favor of ctx.actions.run() / ctx.actions.run_shell(): [1]. See discussion in [2], why we need this. [1] h... — committed to GerritCodeReview/gerrit by davido 7 years ago
Fix for run_in_repo on Windows Replaced call to ctx.actions.run with ctx.actions.run_shell. Windows was trying to execute the script as a win32 binary, which was failing with the following error: `... — committed to csilvestrini/arcs by csilvestrini 5 years ago
Fix for run_in_repo on Windows (#3880) Replaced call to ctx.actions.run with ctx.actions.run_shell. Windows was trying to execute the script as a win32 binary, which was failing with the followin... — committed to PolymerLabs/arcs by csilvestrini 5 years ago
Skylark: ctx.actions.run_shell uses helper script If the shell command in ctx.actions.run_shell.command is longer than the platform's shell's limit, Bazel will dump the command to... — committed to luca-digrazia/DatasetCommitsDiffSearch by deleted user 2 years ago

Most upvoted comments

@laszlocsomor Patch set 4 of your gerrit change just worked (I noticed, it needs a rebase, as it wasn’t cleanly applied on recent master):

$  bazel --nomaster_bazelrc build --workspace_status_command=/C/users/davido/projects/gerrit/tools/workspace-status.cmd --verbose_failures -s :headless
____Loading complete.  Analyzing...
____Found 1 target...
____Building...
____[53 / 159] Writing script headless.run_shell_0.sh
____[277 / 278] WAR headless.war
>>>>>>>>> # //:headless [action 'WAR headless.war']
cd C:/msys64/tmp/_bazel_davido/_biltsqa/execroot/gerrit
  SET PATH=C:\msys64\usr\bin;C:\msys64\bin;C:\Python36;C:\msys64\usr\local\bin;C:\msys64\usr\bin;C:\msys64\usr\bin;C:\msys64\opt\bin;C:\Windows\System32;C:\Windows;C:\Windows\System32\Wbem;C:\Windows\System32\WindowsPowerShell\v1.0\;C:\msys64\usr\bin\site_perl;C:\msys64\usr\bin\vendor_perl;C:\msys64\usr\bin\core_perl
    SET TEMP=C:\Users\davido\AppData\Local\Temp
    SET TMP=C:\Users\davido\AppData\Local\Temp
  C:/msys64/usr/bin/bash.exe bazel-out/msvc_x64-fastbuild/genfiles/headless.run_shell_0.sh
____[277 / 278] Still waiting for 1 job to complete:
      Running (local):
        WAR headless.war, 10 s
Target //:headless up-to-date:
  C:/msys64/tmp/_bazel_davido/_biltsqa/execroot/gerrit/bazel-out/msvc_x64-fastbuild/bin/headless.war
____Elapsed time: 34,942s, Critical Path: 21,59s

I checked the content of generated file: C:/msys64/usr/bin/bash.exe bazel-out/msvc_x64-fastbuild/genfiles/headless.run_shell_0.sh and it looks sane: [1].

Thank you again for the quick fix!

[1] http://paste.openstack.org/show/619831

davido on Aug 29, 2017