lektor: Installer broken with virtualenv 20.x

Anyone up for making a PR?

I’ll do it.

No reason except I am more familiar with wget. Here is curl version:

--- install.sh	2020-03-16 00:31:08.799577500 +0100
+++ /mnt/e/programs/lektor/install.sh	2020-03-16 00:28:00.551836200 +0100
@@ -36,7 +36,7 @@
 
     sys.stdin = open('/dev/tty', 'r')
 
-    VENV_URL = "https://pypi.python.org/pypi/virtualenv/json"
+    VENV_URL = "https://bootstrap.pypa.io/virtualenv.pyz"
     KNOWN_BINS = ['/usr/local/bin', '/opt/local/bin',
                   os.path.join(os.environ['HOME'], '.bin'),
                   os.path.join(os.environ['HOME'], '.local', 'bin')]
@@ -114,17 +114,14 @@
 
     def install(virtualenv_url, lib_dir, bin_dir):
         t = tempfile.mkdtemp()
-        Popen('curl -sf "%s" | tar -xzf - --strip-components=1' %
+        Popen('curl -O "%s"' %
               virtualenv_url, shell=True, cwd=t).wait()
 
         try:
             os.makedirs(lib_dir)
         except OSError:
             pass
-        try:  # virtualenv 16.1.0, 17+
-            check_output([sys.executable, './src/virtualenv.py', lib_dir], cwd=t)
-        except CalledProcessError:  # older virtualenv
-            Popen([sys.executable, './virtualenv.py', lib_dir], cwd=t).wait()
+        Popen([sys.executable, './virtualenv.pyz', lib_dir], cwd=t).wait()
         Popen([os.path.join(lib_dir, 'bin', 'pip'),
            'install', '--upgrade', 'Lektor']).wait()
         os.symlink(os.path.join(lib_dir, 'bin', 'lektor'),
@@ -155,14 +152,7 @@
 
         if prompt: get_confirmation()
 
-        for url in json.loads(urlopen(VENV_URL).read().decode('utf-8'))['urls']:
-            if url['python_version'] == 'source':
-                virtualenv = url['url']
-                break
-        else:
-            fail('Could not find virtualenv')
-
-        install(virtualenv, lib_dir, bin_dir)
+        install(VENV_URL, lib_dir, bin_dir)
 
         print('')
         print('All done!')

Here is simple urlretrieve version also but it does not show download progress (for that it will be more complicated to solve python2/3 comaptibility issues):

--- install.sh	2020-03-15 23:50:37.555242800 +0100
+++ /mnt/e/programs/lektor/install.sh	2020-03-15 23:47:40.352555700 +0100
@@ -26,9 +26,9 @@
     import shutil
     from subprocess import CalledProcessError, check_output, Popen
     try:
-        from urllib.request import urlopen
+        from urllib.request import urlopen, urlretrieve
     except ImportError:
-        from urllib import urlopen
+        from urllib import urlopen, urlretrieve
 
     PY2 = sys.version_info[0] == 2
     if PY2:
@@ -36,7 +36,7 @@
 
     sys.stdin = open('/dev/tty', 'r')
 
-    VENV_URL = "https://pypi.python.org/pypi/virtualenv/json"
+    VENV_URL = "https://bootstrap.pypa.io/virtualenv.pyz"
     KNOWN_BINS = ['/usr/local/bin', '/opt/local/bin',
                   os.path.join(os.environ['HOME'], '.bin'),
                   os.path.join(os.environ['HOME'], '.local', 'bin')]
@@ -113,18 +113,26 @@
         sys.exit(1)
 
     def install(virtualenv_url, lib_dir, bin_dir):
-        t = tempfile.mkdtemp()
-        Popen('curl -sf "%s" | tar -xzf - --strip-components=1' %
-              virtualenv_url, shell=True, cwd=t).wait()
+	print('')
+	print("Fetching virtualenv ...")
+	urlfile = urlretrieve(VENV_URL)
+	print('')
+	print("virtualenv fetched")
+	t, vefile =os.path.split(urlfile[0])
 
         try:
             os.makedirs(lib_dir)
         except OSError:
             pass
-        try:  # virtualenv 16.1.0, 17+
-            check_output([sys.executable, './src/virtualenv.py', lib_dir], cwd=t)
-        except CalledProcessError:  # older virtualenv
-            Popen([sys.executable, './virtualenv.py', lib_dir], cwd=t).wait()
+	print('')
+	print("Creating Lektor virtualenv ...")
+	print('')
+        Popen([sys.executable, vefile, lib_dir], cwd=t).wait()
+	print('')
+	print("virtualenv created")
+	print('')
+	print("installing Lektor ...")
+	print('')
         Popen([os.path.join(lib_dir, 'bin', 'pip'),
            'install', '--upgrade', 'Lektor']).wait()
         os.symlink(os.path.join(lib_dir, 'bin', 'lektor'),
@@ -155,14 +163,7 @@
 
         if prompt: get_confirmation()
 
-        for url in json.loads(urlopen(VENV_URL).read().decode('utf-8'))['urls']:
-            if url['python_version'] == 'source':
-                virtualenv = url['url']
-                break
-        else:
-            fail('Could not find virtualenv')
-
-        install(virtualenv, lib_dir, bin_dir)
+        install(VENV_URL, lib_dir, bin_dir)
 
         print('')
         print('All done!')

I just tested this with python2 so I am not sure is it working with python3

It doesn’t seem like we can pin virtualenv.pyz

IIRC virtualenv only ever blew up because they changed the location of the executable. That won’t be a problem with a zipapp.

I think it’s OK to switch to the zipapp and get the one for the current version and if that 404s try the generic one.

That’s two branches of code, then there would need to be that pinning logic and procedure… not worth it since the zipapp is the official bootstrap method and that’s the official URL. So yeah, I changed my opinion, sorry for the noise. I think we should go forward with the zipapp way and forget about legacy.

Later we can set up a scheduled CI job that tries to install using this method to check for regressions.

That would be nice.