esp-idf: ESP-IDF Tools Installer v2.2 does not work due to two different issues (IDFGH-2670)

Environment

Windows 10 with latest updates applied. (Other environment items were not applicable.)

Problem Description

I cannot install ESP-IDF Tools and ESP-IDF using the official installer. I would like to start developing for the ESP32 but I cannot install the tools. I would prefer not to install the environment manually.

I encounter two issues:

Issue 1

Expected Behavior

ESP-IDF is copied to destination directory after extraction and setp continues.

Actual Behavior

Error message “Failed to copy ESP-IDF to the destination directory: E:\espidf\3.3” appears and setup halts.

Steps to reproduce

Download and run esp-idf-tools-setup-2.2.exe
Choose option to download ESP-IDF (tested with 3.3 release and 3.3 branch)

Debug Logs

log1.txt

Relevant part:

2020-02-10 19:00:44.329 -------- 2020-02-10 19:00:44.332 Moving C:\Users\r\AppData\Local\Temp\is-EG1TH.tmp\esp-idf-v3.3.1 to E:\espidf\3.3 2020-02-10 19:00:44.332 Message box (OK): Failed to copy ESP-IDF to the destination directory: E:\espidf\3.3 2020-02-10 19:01:53.031 User chose OK. 2020-02-10 19:01:53.031 Message box (Yes/No): Installation log has been created, it may contain more information about the problem. Display the installation log now? 2020-02-10 19:01:53.962 User chose Yes. 2020-02-10 19:01:53.989 CurStepChanged raised an exception.

Workaround

Go to the .espressif\releases directory, manually extract and copy esp-idf-v3.3.1.zip to ESP-IDF destination directory, and re-run installer.

Issue 2

Expected Behavior

ESP-IDF Tools are set-up.

Actual Behavior

Error message “Installation has failed with exit code 1” appears and setup halts.

Steps to reproduce

Download and run esp-idf-tools-setup-2.2.exe
Choose option to download ESP-IDF (tested with 3.3 release and 3.3 branch)
When installation fails, apply workaround from Issue 1
Re-run esp-idf-tools-setup-2.2.exe
Choose option to use the previously extracted ESP-IDF

Debug Logs

log2.txt

Relevant part:

2020-02-10 19:10:02.444 Installing Python environment:C:\Users\r\AppData\Local\Programs\Python\Python37\python.exe E:\espidf\3.3\tools\idf_tools.py install-python-env 2020-02-10 19:10:03.415 Done, exit code=1 2020-02-10 19:10:03.415 -------- 2020-02-10 19:10:03.415 Running command: C:\Users\r\AppData\Local\Programs\Python\Python37\python.exe E:\espidf\3.3\tools\idf_tools.py install-python-env usage: virtualenv [–version] [-v | -q] [–discovery {builtin}] [-p py] [–creator {builtin,cpython3-win,venv}] [–seeder {app-data,pip}] [–no-seed] [–activators comma_separated_list] [–clear] [–system-site-packages] [–copies] [–download | --no-download] [–extra-search-dir d [d …]] [–pip version] [–setuptools version] [–wheel version] [–no-pip] [–no-setuptools] [–no-wheel] [–clear-app-data] [–prompt prompt] [-h] [dest] virtualenv: error: unrecognized arguments: --no-site-packages Creating a new Python environment in E:\espidf.espressif\python_env\idf3.3_py3.7_env Traceback (most recent call last): File “E:\espidf\3.3\tools\idf_tools.py”, line 1349, in <module> main(sys.argv[1:]) File “E:\espidf\3.3\tools\idf_tools.py”, line 1345, in main action_func(args) File “E:\espidf\3.3\tools\idf_tools.py”, line 1156, in action_install_python_env stdout=sys.stdout, stderr=sys.stderr) File “C:\Users\r\AppData\Local\Programs\Python\Python37\lib\subprocess.py”, line 347, in check_call raise CalledProcessError(retcode, cmd) subprocess.CalledProcessError: Command ‘[‘C:\Users\r\AppData\Local\Programs\Python\Python37\python.exe’, ‘-m’, ‘virtualenv’, ‘–no-site-packages’, ‘E:\espidf\.espressif\python_env\idf3.3_py3.7_env’]’ returned non-zero exit status 2.

2020-02-10 19:10:03.415 -------- 2020-02-10 19:10:03.415 Message box (OK): Installation has failed with exit code 1 2020-02-10 19:10:21.217 User chose OK. 2020-02-10 19:10:21.235 Message box (Yes/No): Installation log has been created, it may contain more information about the problem. Display the installation log now? 2020-02-10 19:10:36.417 User chose Yes. 2020-02-10 19:10:36.460 CurStepChanged raised an exception.

About this issue

Original URL
State: closed
Created 4 years ago
Reactions: 3
Comments: 41 (14 by maintainers)

Commits related to this issue

tools: bump version of windows tools installer to v2.3 Includes the following fixes: - bc43d48e: fix Windows Defender checkbox https://github.com/espressif/esp-idf/issues/4225 - 7a18f02a: idf_too... — committed to espressif/esp-idf by igrr 4 years ago

Most upvoted comments

Thanks for reporting! The fix for Python environment installation is in review and will be released soon (along with a new version of the installer).

As a temporary workaround, you may downgrade virtualenv package as follows:

C:\Users\r\AppData\Local\Programs\Python\Python37\python.exe -m pip install --upgrade virtualenv==16.7.9

After this, retry the installation.

+27

igrr on Feb 10, 2020

As a temporary workaround, you may downgrade virtualenv package as follows:
C:\Users\r\AppData\Local\Programs\Python\Python37\python.exe -m pip install --upgrade virtualenv==16.7.9

Or remove --no-site-packages flag from virtualenv inside idf_tools.py. The flag has been deprecated for quite some time (6 years).

+22

pavestru on Feb 10, 2020

If you are experiencing this issue (on Windows), please try this version of the installer: https://dl.espressif.com/dl/esp-idf-tools-setup-2.3.exe

igrr on Feb 11, 2020

I tried the installer version listed above and it worked - the --no-site-packages problem disappeared, and the installer finished, creating the proper shortcuts in the Start Menu etc.

MitchBradley on Feb 11, 2020

Just for the record

diff --git a/tools/idf_tools.py b/tools/idf_tools.py
index 158b5a764..32a7c57a1 100755
--- a/tools/idf_tools.py
+++ b/tools/idf_tools.py
@@ -1180,7 +1180,7 @@ def action_install_python_env(args):
             subprocess.check_call([sys.executable, '-m', 'pip', 'install', '--user', 'virtualenv'],
                                   stdout=sys.stdout, stderr=sys.stderr)

-        subprocess.check_call([sys.executable, '-m', 'virtualenv', '--no-site-packages', idf_python_env_path],
+        subprocess.check_call([sys.executable, '-m', 'virtualenv', idf_python_env_path],
                               stdout=sys.stdout, stderr=sys.stderr)
     run_args = [virtualenv_python, '-m', 'pip', 'install', '--no-warn-script-location']
     requirements_txt = os.path.join(global_idf_path, 'requirements.txt')

zzeroo on Mar 4, 2020

@pavestru yes, that’s the fix which is going to be merged. For the users of the Windows tools installer the problem is that it’s not trivial to modify idf_tools.py since it is extracted by the installer (at least when installing v3.3.1). For other OSes, modifying idf_tools.py can also be used as a workaround for now.

igrr on Feb 10, 2020

@krzychb Thanks for the info. I was able to get the Windows 10 install and example project build working with v2.3.3 tools installer when I used the “install Python” option, instead of pointing to my existing miniconda Python 3.8.5 installation.

har-in-air on Jan 8, 2021

It seems like that latest version of virtualenv has removed the --no-site-packages flag and the created virtualenv doesn’t inherit system site-packages by default. Instead, the flag --system-site-packages is added to inherit system site-packages.

paulzhn on Feb 29, 2020

With https://dl.espressif.com/dl/esp-idf-tools-setup-2.3.exe installation was smooth. I used python 3.8, git 2.23.0, window .1. Example hello_world running OK.

Zeni241 on Feb 14, 2020

The installer also successfully installs the master branch (w/ Python 3.8), the 3.3.1 release version, but not the 4.0 branch version. Is 3.3.1 expected to work at all? I see the --no-sites-package fix hasn’t been backported yet, so I find this surprising.

Once the fix is backported to release/v3.3 and release/v4.0, these branches will work as well. The backport should land tomorrow, hopefully. For the v3.3.1 and v4.0 versions, the installer includes a workaround (uses a bundled version of idf_tools.py). This is because we can’t change the release tags…

Also, what is the difference between “release version” and “branch version”? I though the first downloaded the SDK as a full zip package and the second cloned it from the Github branch, but it seems to me that the second one downloads a zip package as well.

The difference between release branches and release versions is the difference between branches and tags. The installation flow for release branches is indeed to download the latest release tag for the branch (if one exists) and then use it as a reference repository when cloning the branch. Most of the time, this is the quickest way to download a release branch of IDF, and the final result is the same as if you would clone the branch from scratch.

igrr on Feb 11, 2020

Okay, I was able to reproduce the first issue as well. It happens because we are trying to use “RenameFile” function to move the extracted ESP-IDF directory into place, but this function doesn’t work when the destination is on a different drive. This will also be fixed shortly.

igrr on Feb 10, 2020