spyder: Vaex process stuck in the Spyder IPython console (only)

Issue Report Checklist

Searched the issues page for similar reports
Read the relevant sections of the Spyder Troubleshooting Guide and followed its advice
Reproduced the issue after updating with conda update spyder (or pip, if not using Anaconda)
Could not reproduce inside jupyter qtconsole (if console-related)
Tried basic troubleshooting (if a bug/error)
- Restarted Spyder
- Reset preferences with spyder --reset (fixed the problem with inline graphics, see below)
- Reinstalled the latest version of Anaconda
- Tried the other applicable steps from the Troubleshooting Guide
Completed the Problem Description, Steps to Reproduce and Version sections below

Problem Description

Spyder IPython console process becomes stuck when computing a mean with vaex. Does not happen with other IDEs / Consoles. Somehow, it may be related to the output graphics mode of the Spyder iPython console (see below)

What steps reproduce the problem?

Minimal error :

import numpy as np
import vaex
df = vaex.from_dict({"E":np.random.rand(1000)})
print(df)
print("before EQ mean")
E_bins = df.mean("E", binby="E", shape=10)
print("after EQ mean")       # 👈 I never get to here

Script works well if

ran from an Ipython ( 7.22.0) console in the terminal
ran from Python in the terminal
ran from VS Code
ran from Jupyter Lab (3.0.14)
ran in Jupyter QtConsole

As pointed out by @minouHub , opening a new Spyder IPython console unlocks the process. If console has been “unlocked”, I can even re-run the script another time before it gets stuck again. (?)

Last thing : although no graphics are plot anywhere in this example, this problem appears when my iPython console has Qt5 output graphics. It does not appear with Inline graphics (tried both consistently 3 times)

Versions

spyder
spyder-kernels 2.0.5 py38haa95532_0 ipython 7.22.0 py38hd4e2768_0

on Windows.

Spyder installed through Anaconda

Spyder version: 5.0.5 py38haa95532_2
Python version: 3.8.8
Qt version: 5.9.7 vc14h73c81de_0
PyQt version: 1.9.0
Operating System name/version: Windows 10

About this issue

Original URL
State: closed
Created 3 years ago
Comments: 29 (13 by maintainers)

Commits related to this issue

add warning if using Spyder and Vaex see https://github.com/spyder-ide/spyder/issues/16183 — committed to erwanp/radis by erwanp 3 years ago

Most upvoted comments

Tested right now ; everything works properly with Vaex 4.8

Vaex 4.8 is officially released so we can consider this issue fixed.

Thank you @maartenbreddels @ccordoba12

erwanp on Mar 6, 2022

If you install the latest alpha, you can try this trick https://github.com/vaexio/vaex/pull/1546#issuecomment-935535045 to see if it does not get stuck (that might tell you that nest_asyncio is the problem).

maartenbreddels on Oct 18, 2021

Well, I think @maartenbreddels is the one that deserves all the credit here. Thanks for making Spyder a fully supported platform for Vaex!

ccordoba12 on Mar 6, 2022

That’s great news. I’m actually going to test if I can make that the default, and only set it to ‘nest’ when we need it (when using the jupyter widgets).

maartenbreddels on Oct 19, 2021

@ccordoba12 https://github.com/spyder-ide/spyder-kernels/pull/317

impact27 on Aug 30, 2021

Hey @maartenbreddels, thanks for showing up! That’s a very good point: when people activate the Qt5 Matplotlib backend, a Qt event loop is created in the console to handle interactive figures created by it. That’s not done by Spyder, but by IPython (we just run %matplotlib qt5 behind the scenes).

@erwanp, could you try running in a notebook

%matplotlib qt5

then creating a Vaex dataframe to see what happens there? My guess is that you should observe the same stuck behavior.

ccordoba12 on Aug 19, 2021

I could reproduce this issue on Windows, we will investigate it because it doesn’t happen on macOS or Linux. We will try to fix it for one of our next releases.

@impact27, when the console stays opened some errors start appearing,

[SpyderKernelApp] ERROR | Exception in message handler:
Traceback (most recent call last):
  File "C:\Users\andfoy-windows\spyder\external-deps\spyder-kernels\spyder_kernels\comms\frontendcomm.py", line 164, in poll_one
    asyncio.run(handler(out_stream, ident, msg))
  File "C:\Users\andfoy-windows\AppData\Local\Continuum\miniconda3\envs\spyder-dev\lib\site-packages\nest_asyncio.py", line 32, in run
    return loop.run_until_complete(future)
  File "C:\Users\andfoy-windows\AppData\Local\Continuum\miniconda3\envs\spyder-dev\lib\site-packages\nest_asyncio.py", line 60, in run_until_complete
    f = asyncio.ensure_future(future, loop=self)
  File "C:\Users\andfoy-windows\AppData\Local\Continuum\miniconda3\envs\spyder-dev\lib\asyncio\tasks.py", line 679, in ensure_future
    raise TypeError('An asyncio.Future, a coroutine or an awaitable is '
TypeError: An asyncio.Future, a coroutine or an awaitable is required

I don’t know if this is related to the kernel.

steff456 on Aug 11, 2021

Same problem with Spyder 5.1.1 and Python 3.8.8 64-bit | Qt 5.12.10 | PyQt5 5.12.3 | Windows 10

I also confirm that the process is stuck if using Qt5 graphics in iPython console; and not stuck if using Inline graphics (although no graphic is shown in the minimal code)

Same symptoms for other users : @MinouHub; @BlehMaks

erwanp on Aug 10, 2021

Hi @erwanp,

Can you upgrade your Spyder to 5.1.1 and try it again? I’m in the latest version and I’m not able to reproduce this issue.

steff456 on Aug 10, 2021