pip: [BUG] stderr duplication failed

Just waiting for a response from @rayzchen (and anyone else) and then I should be able to cut a release soon after that.

Thanks @cbrnr and @carlkl for redoing the tests. Looking promising 🤞

I think you probably cannot generalize to all launchers for Windows. I just tried pip install mnelab and the launcher works (it didn’t prior to v22).

I can’t reproduce this - it might be something specific in @rayzchen 's environment. I’m using Python 3.9.7 on Windows 10.

C:\Users\Vinay\Projects\scratch\distlib\pip_10444> type setup.py                                                                   
from setuptools import setup                                                                                                       
                                                                                                                                   
setup(                                                                                                                             
    name="test",                                                                                                                   
    packages=["test"],                                                                                                             
    entry_points={                                                                                                                 
        "gui_scripts": [                                                                                                           
            "test-script=test:main"                                                                                                
        ]                                                                                                                          
    }                                                                                                                              
)                                                                                                                                  
C:\Users\Vinay\Projects\scratch\distlib\pip_10444> type test\__init__.py                                                           
def main():                                                                                                                        
    pass                                                                                                                           
C:\Users\Vinay\Projects\scratch\distlib\pip_10444> python3 -m venv env                                                             
                                                                                                                                   
C:\Users\Vinay\Projects\scratch\distlib\pip_10444> env\Scripts\python -m pip install -U pip wheel                                  
Requirement already satisfied: pip in c:\users\vinay\projects\scratch\distlib\pip_10444\env\lib\site-packages (21.2.3)             
Collecting pip                                                                                                                     
  Using cached pip-21.2.4-py3-none-any.whl (1.6 MB)                                                                                
Collecting wheel                                                                                                                   
  Using cached wheel-0.37.0-py2.py3-none-any.whl (35 kB)                                                                           
Installing collected packages: wheel, pip                                                                                          
  Attempting uninstall: pip                                                                                                        
    Found existing installation: pip 21.2.3                                                                                        
    Uninstalling pip-21.2.3:                                                                                                       
      Successfully uninstalled pip-21.2.3                                                                                          
Successfully installed pip-21.2.4 wheel-0.37.0                                                                                     
                                                                                                                                   
C:\Users\Vinay\Projects\scratch\distlib\pip_10444> env\Scripts\pip install .                                                       
Processing c:\users\vinay\projects\scratch\distlib\pip_10444                                                                       
  DEPRECATION: A future pip version will change local packages to be built in-place without first copying to a temporary directory.
 We recommend you use --use-feature=in-tree-build to test your packages with this new behavior before it becomes the default.      
   pip 21.3 will remove support for this functionality. You can find discussion regarding this at https://github.com/pypa/pip/issue
s/7555.                                                                                                                            
Building wheels for collected packages: test                                                                                       
  Building wheel for test (setup.py) ... done                                                                                      
  Created wheel for test: filename=test-0.0.0-py3-none-any.whl size=1357 sha256=59ca865995d18af94cae5d4bf40bc7643ffa08ff8fdcd29e10b
9a8c940967239                                                                                                                      
  Stored in directory: C:\Users\Vinay\AppData\Local\Temp\pip-ephem-wheel-cache-fir7qy3v\wheels\6f\a7\45\f9fb5d415c64f56b92837eabf4b
edd182318b9abca73407cd5                                                                                                            
Successfully built test                                                                                                            
Installing collected packages: test                                                                                                
Successfully installed test-0.0.0                                                                                                  
                                                                                                                                   
C:\Users\Vinay\Projects\scratch\distlib\pip_10444> dir env\Scripts\                                                                
 Volume in drive C has no label.                                                                                                   
 Volume Serial Number is 26B4-06F0                                                                                                 
                                                                                                                                   
 Directory of C:\Users\Vinay\Projects\scratch\distlib\pip_10444\env\Scripts                                                        
                                                                                                                                   
25/09/2021  06:49    <DIR>          .                                                                                              
25/09/2021  06:49    <DIR>          ..                                                                                             
25/09/2021  06:48             1,997 activate                                                                                       
25/09/2021  06:48               989 activate.bat                                                                                   
25/09/2021  06:48            19,408 Activate.ps1                                                                                   
25/09/2021  06:48               368 deactivate.bat                                                                                 
25/09/2021  06:49           106,383 pip.exe                                                                                        
25/09/2021  06:49           106,383 pip3.9.exe                                                                                     
25/09/2021  06:49           106,383 pip3.exe                                                                                       
25/09/2021  06:48           543,464 python.exe                                                                                     
25/09/2021  06:48           542,440 pythonw.exe                                                                                    
25/09/2021  06:49           100,222 test-script.exe                                                                                
25/09/2021  06:48           106,370 wheel.exe                                                                                      
              11 File(s)      1,634,407 bytes                                                                                      
               2 Dir(s)  11,182,489,600 bytes free                                                                                 
                                                                                                                                   
C:\Users\Vinay\Projects\scratch\distlib\pip_10444> env\Scripts\test-script.exe                                                     
                                                                                                                                   
C:\Users\Vinay\Projects\scratch\distlib\pip_10444>                                                                                 

And no error dialog box comes up. If I additionally install pywin32, change the test application to the following:

# setup.py
from setuptools import setup

setup(
    name="test",
    packages=["test"],
    entry_points={
        "gui_scripts": [
            "test-script=test:main",
            "hello=test:hello"
        ]
    }
)

# test\__init__.py
def main():
    pass
    
def hello():
    import win32api, win32con
    win32api.MessageBox (0, "Test application says 'Hello, world!'", 'Hello, world!', win32con.MB_OK)
    pass

and reinstall using env\Scripts\pip install . followed by running env\Scripts\hello, I get the expected message box:

Didn’t hear from @rayzchen, but distlib 0.3.5 has just been released on PyPI.

Thanks. everyone, for the feedback. I’ve commented on the PR by @mbikovitsky and am aiming to merge the changes soon.

Can you please quickly reiterate how I can test the patched launcher? Where do I need to copy the .exe files?

The linked zip file contains some .exe files - just copy them over the files in <venv>\Lib\site-packages\pip\_vendor\distlib for whichever pip you’re testing with. That directory should already have the .exe files released with whichever version of distlib is vendored into that specific pip installation.

I’m not sure that’ll work, because pip vendors distlib rather than having it as a normal dependency. I would clone the distlib repo, then copy the .exe files into the corresponding place in a pip installation. For example, if you create a venv at c:\Temp\foo then that would be the directory c:\Temp\foo\Lib\site-packages\pip\_vendor\distlib - you can overwrite the .exe files in there. Then you can try installing the above test application and invoking hello.exe, or other windowed application of your choice.

I think it would be useful to separate out scenarios that need to be considered by the GUI launcher from those that need to be considered by the console launcher. I’m wary of making things in the launcher too complex for the intended use case - which ISTM are nearly always console scripts. Even when the GUI launcher is used, I doubt whether the child application will consider all of these issues around shutdown as thoroughly as all this, if at all!

My testing has revealed that it works fine (all 3 handles are actually valid) when run in MSYS2 via MinTTY, but not via “Windows default console”