modoboa-installer: installer fails because of "new" pip dependency resolver
Impacted versions
- Distribution: Ubuntu
- Codename: Focal (20.04.3)
- Arch: 64 Bits
- Database: PostgreSQL
Also tried with the following OSes, but stopped the installer after about 4~6 hours without any result:
- CentOS 7
- Debian 10
- Ubuntu 18.04 (latest version)
Steps to reproduce
Basic Ubuntu 20.04.3 server setup (or any of the above listes OSes), with SSH enabled. After setup is completed (XXXXX
is a real existing domain):
$ time sudo python3 run.py --debug XXXXX
[...]
INFO: This is taking longer than usual. You might need to provide the dependency resolver with stricter constraints to reduce runtime. If you want to abort this run, you can press Ctrl + C to do so. To improve how pip performs, tell us what happened here: https://pip.pypa.io/surveys/backtracking
Downloading modoboa-webmail-1.1.4.tar.gz (87 kB)
|████████████████████████████████| 87 kB 3.0 MB/s
Downloading modoboa-webmail-1.1.3.tar.gz (87 kB)
|████████████████████████████████| 87 kB 4.5 MB/s
Downloading modoboa-webmail-1.1.2.tar.gz (85 kB)
|████████████████████████████████| 85 kB 4.3 MB/s
Downloading modoboa-webmail-1.1.1.tar.gz (84 kB)
|████████████████████████████████| 84 kB 3.8 MB/s
ERROR: Exception:
Traceback (most recent call last):
File "/srv/modoboa/env/lib/python3.8/site-packages/pip/_internal/cli/base_command.py", line 173, in _main
status = self.run(options, args)
File "/srv/modoboa/env/lib/python3.8/site-packages/pip/_internal/cli/req_command.py", line 203, in wrapper
return func(self, options, args)
File "/srv/modoboa/env/lib/python3.8/site-packages/pip/_internal/commands/install.py", line 315, in run
requirement_set = resolver.resolve(
File "/srv/modoboa/env/lib/python3.8/site-packages/pip/_internal/resolution/resolvelib/resolver.py", line 94, in resolve
result = self._result = resolver.resolve(
File "/srv/modoboa/env/lib/python3.8/site-packages/pip/_vendor/resolvelib/resolvers.py", line 472, in resolve
state = resolution.resolve(requirements, max_rounds=max_rounds)
File "/srv/modoboa/env/lib/python3.8/site-packages/pip/_vendor/resolvelib/resolvers.py", line 383, in resolve
raise ResolutionTooDeep(max_rounds)
pip._vendor.resolvelib.resolvers.ResolutionTooDeep: 2000000
Traceback (most recent call last):
File "/srv/modoboa/env/bin/modoboa-admin.py", line 4, in <module>
from modoboa.core.commands import handle_command_line
File "/srv/modoboa/env/lib/python3.8/site-packages/modoboa/core/__init__.py", line 3, in <module>
from django.core.urlresolvers import reverse
File "/srv/modoboa/env/lib/python3.8/site-packages/django/core/urlresolvers.py", line 16, in <module>
from django.http import Http404
File "/srv/modoboa/env/lib/python3.8/site-packages/django/http/__init__.py", line 4, in <module>
from django.http.response import (
File "/srv/modoboa/env/lib/python3.8/site-packages/django/http/response.py", line 13, in <module>
from django.core.serializers.json import DjangoJSONEncoder
File "/srv/modoboa/env/lib/python3.8/site-packages/django/core/serializers/__init__.py", line 23, in <module>
from django.core.serializers.base import SerializerDoesNotExist
File "/srv/modoboa/env/lib/python3.8/site-packages/django/core/serializers/base.py", line 6, in <module>
from django.db import models
File "/srv/modoboa/env/lib/python3.8/site-packages/django/db/models/__init__.py", line 6, in <module>
from django.db.models.query import Q, QuerySet, Prefetch # NOQA
File "/srv/modoboa/env/lib/python3.8/site-packages/django/db/models/query.py", line 13, in <module>
from django.db.models.fields import AutoField, Empty, FieldDoesNotExist
File "/srv/modoboa/env/lib/python3.8/site-packages/django/db/models/fields/__init__.py", line 18, in <module>
from django import forms
File "/srv/modoboa/env/lib/python3.8/site-packages/django/forms/__init__.py", line 6, in <module>
from django.forms.fields import * # NOQA
File "/srv/modoboa/env/lib/python3.8/site-packages/django/forms/fields.py", line 18, in <module>
from django.forms.utils import from_current_timezone, to_current_timezone
File "/srv/modoboa/env/lib/python3.8/site-packages/django/forms/utils.py", line 15, in <module>
from django.utils.html import format_html, format_html_join, escape
File "/srv/modoboa/env/lib/python3.8/site-packages/django/utils/html.py", line 16, in <module>
from .html_parser import HTMLParser, HTMLParseError
File "/srv/modoboa/env/lib/python3.8/site-packages/django/utils/html_parser.py", line 12, in <module>
HTMLParseError = _html_parser.HTMLParseError
AttributeError: module 'html.parser' has no attribute 'HTMLParseError'
None
real 1562m4.217s
user 1557m54.697s
sys 1m13.077s
Especially this line:
pip._vendor.resolvelib.resolvers.ResolutionTooDeep: 2000000
Expected behavior
Installation finishes.
Notes
- I did this installation on a Debian 10 about 5 months ago, and it finished pretty fast.
- Currently this does not work on any of the tested OS versions, because
pip
is installed in the modoboa environment at the latest version which uses the new dependency resolver. - I tried setting
~modoboa/env/pip.conf
(aka$VIRTUAL_ENV/pip.conf
) with different values, but currently I didn’t find a working fix/hack to make the installer work. This idea was based on several “stackoverflow suggestions”. Especiallyuse-deprecated=legacy-resolver
did not work, actually it fails pretty fast.
About this issue
- Original URL
- State: closed
- Created 3 years ago
- Comments: 21 (5 by maintainers)
Just to let you know, I’ve pushed a fix for this issue. Problem is latest stable version of Modoboa (1.17) depends on old requirement versions which are not compatible with latest pip version anymore. If you update your installer copy, you can now use the --beta flag to install latest beta release instead, which has updated requirements.
Here are the steps I took to get it installed on Ubuntu 20.04:
Change to root:
~$ sudo -i
Download Modoboa Installer:
~# git clone https://github.com/modoboa/modoboa-installer
Install build dependencies: (The installer will keep crashing without: rustc, python3-feedparser & python3-lxml)
~# apt install build-essential python3-dev python3-feedparser python3-lxml libxml2-dev libxslt-dev libjpeg-dev librrd-dev rrdtool libffi-dev libssl-dev rustc
Installer will keep crashing if /tmp is noexec:
~# mount -t tmpfs -o remount,exec tmpfs /tmp
Run the Modoboa Installer to generate config file:
~# cd modoboa-installer
~# ./run.py --stop-after-configfile-check <your domain>
Edit config file:
~# nano installer.cfg
Change the following sections:
[certificate]
type = letsencrypt
[letsencrypt]
email = <your email>@<your domain>
[database]
engine = <postgres or mysql>
^S ^X
Install Modoboa: (it’s going to fail)
~# ./run.py --interactive --debug <your domain>
Once you start seeing errors hit
^C
, then fix the issues keeping it from installing:Install django-braces:
~# su - modoboa
$ bash
~$ source ./env/bin/activate
~$ pip install django-braces
~$ exit
$ exit
Copy feedparser & lxml to modoboa env:
~# cp -r /usr/lib/python3/dist-packages/feedparser* /srv/modoboa/env/lib/python3.8/site-packages/
~# cp -r /usr/lib/python3/dist-packages/lxml* /srv/modoboa/env/lib/python3.8/site-packages/
Re-install Modoboa:
~# ./run.py --interactive --debug <your domain>
You should see:
Success!
Remount /tmp noexec:
~# mount -t tmpfs -o remount,noexec tmpfs /tmp
Going to webmail site will generate server error, so let’s fix it:
~# nano /srv/modoboa/instance/instance/settings.py
Change the following line:
USE_TZ = false
^S ^X
Restart uWSGI:
~# /etc/init.d/uwsgi restart
Fix Postfix greylisting:
~# nano /etc/postfix/main.cf
Comment out the following:
postscreen_pipelining_enable = yes
postscreen_pipelining_action = enforce
postscreen_non_smtp_command_enable = yes
postscreen_non_smtp_command_action = enforce
postscreen_bare_newline_enable = yes
postscreen_bare_newline_action = enforce
^S ^X
Restart Postfix:
~# systemctl restart postfix
~# exit
Success!
@slayster @Mik-Cryo Seems like the problem related to
error in feedparser setup command: use_2to3 is invalid.
, is because the installingfeedparser
might or might not include a filed namedsgmllib.py
.More can be found on the package page: https://pypi.org/project/feedparser/5.2.1/
I simply copied one from another
feedparser
package that was installed withsgmllib.py
to./env/lib/python3.8/site-packages
, and it seems to work.