python-language-server: 'Too many open files' error coming from the server

Hi -

when I use pyls and pyls-mypy on a python file in Emacs everything works fine for a while. Documentation pops up, function signatures are shown inline, etc. Then, after perhaps a few dozen interactions I get the following (repeatable) error:

Error from the Language Server: jedi.api.environment.InvalidPythonEnvironment: Could not get version information for ’/Users/dkatz/.local/share/virtualenvs/lsp_example_project-cOuqdw3J/bin/python3.7’: OSError(24, ’Too many open files’) (Invalid Parameters) [16 times]

Once that error is raised, I see it every time the cursor hits any interesting piece of code. Note that I see this with two different LSP clients for Emacs (lsp-mode and eglot), so I suspect the problem is at the server.

I can trigger this behavior even with a completely trivial file like this:

import sys

by just moving the cursor on and off of sys several times (which I assume triggers communication with the server each time as it pops up the module information).

If I watch the number of python processes as a function of how I interact with the server I see something like this (taken from moving the cursor on and off of sys in the trivial file mentioned above):

event effect python processes
Open python file in Emacs 5
Put cursor on sys pop up module info 13
Move cursor off sys module info popup goes away 13
Put cursor back on sys pop up module info 18
Move cursor off sys module info goes away 19
Put cursor back on sys pop up module info 23
Move cursor off sys module info goes away 24
Put cursor back on sys pop up module info 28
Move cursor off sys module info goes away 29
Put cursor back on sys pop up module info 34
Move cursor off sys module info goes away 35
Put cursor back on sys pop up module info 39
Move cursor off sys module info goes away 40
Put cursor back on sys pop up module info 44
Move cursor off sys module info goes away 45
Put cursor back on sys pop up module info 49
Move cursor off sys module info goes away 50

and so on until I got the error with 81 python processes running. The number 81 seems to be pretty consistent for me - I get that number of processes with different files and different interactions (e.g. module info, function info, displaying function signatures, etc.)

Any ideas?

Thanks.

Dan



\

Setup:

OS: macOS 10.13.6

Python: 3.7.0 Python package installer: pipenv Python packages:

  • python-language-server 0.21.2 [all]
  • pyls-mypy 0.1.3

About this issue

  • Original URL
  • State: closed
  • Created 6 years ago
  • Reactions: 8
  • Comments: 18 (4 by maintainers)

Most upvoted comments

There seems to be a bug that still exists on the master branch that leads to an explosion in python subprocesses. This seems to happen when using pipenv to manage virtualenvs.

There is a MR for that here: https://github.com/davidhalter/jedi/pull/1308

Hopefully it will be fixed soon.

Okay, I’ve been able to narrow this down and hack together a solution for this on my end. I’m using Spacemacs with Pyenv-mode and pyenv + pyenv-virtualenv to automatically switch my Python version when I open files. I determined:

  1. Pyenv-mode doesn’t interface with pyenv-virtualenv to set VIRTUAL_ENV when you open a file in a virtualenv, but it will change the python version/location so everything else will look the same.
  2. Jedi spawns a bunch of processes when VIRTUAL_ENV isn’t set (davidhalter/jedi#1238)
  3. lsp-python will spawn a new language server/jedi process without VIRTUAL_ENV set, since it inherits emacs’ environment, not your shell environment

You can force pyenv-mode to set VIRTUAL_ENV with:

  (defun pyenv-venv-wrapper-act (&optional ARG PRED)
    (setenv "VIRTUAL_ENV" (shell-command-to-string "_pyenv_virtualenv_hook; echo -n $VIRTUAL_ENV")))
  (advice-add 'pyenv-mode-set :after 'pyenv-venv-wrapper-act)
  (defun pyenv-venv-wrapper-deact (&optional ARG PRED)
    (setenv "VIRTUAL_ENV"))
  (advice-add 'pyenv-mode-unset :after 'pyenv-venv-wrapper-deact)

Hopefully this helps anyone with the same problem.