phy: numpy broken and empty interface on Linux

Linux Ubuntu-like with KDE

<ws>:~/.local/apps$ python --version
Python 3.10.6
<ws>:~/.local/apps$ qmake --version
QMake version 3.1
Using Qt version 5.15.3 in /usr/lib/x86_64-linux-gnu

Virtual environment

<ws>:~/.local/apps$ python -m venv phy
<ws>:~/.local/apps$ source phy/bin/activate
(phy) <ws>:~/.local/apps$ pip install phy --pre --upgrade
Collecting phy
  Using cached phy-2.0b5-py2.py3-none-any.whl
Collecting requests
  Using cached requests-2.29.0-py3-none-any.whl (62 kB)
Collecting scipy
  Using cached scipy-1.10.1-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (34.4 MB)
Collecting click
  Using cached click-8.1.3-py3-none-any.whl (96 kB)
Collecting qtconsole
  Using cached qtconsole-5.4.2-py3-none-any.whl (121 kB)
Collecting mtscomp
  Using cached mtscomp-1.0.2-py2.py3-none-any.whl (16 kB)
Collecting PyQtWebEngine
  Using cached PyQtWebEngine-5.15.6-cp37-abi3-manylinux1_x86_64.whl (230 kB)
Collecting matplotlib
  Using cached matplotlib-3.7.1-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (11.6 MB)
Collecting dask
  Using cached dask-2023.4.1-py3-none-any.whl (1.2 MB)
Collecting PyQt5
  Using cached PyQt5-5.15.9-cp37-abi3-manylinux_2_17_x86_64.whl (8.4 MB)
Collecting pillow
  Using cached Pillow-9.5.0-cp310-cp310-manylinux_2_28_x86_64.whl (3.4 MB)
Collecting tqdm
  Using cached tqdm-4.65.0-py3-none-any.whl (77 kB)
Collecting h5py
  Using cached h5py-3.8.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (4.6 MB)
Collecting numpy
  Using cached numpy-1.24.3-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (17.3 MB)
Collecting colorcet
  Using cached colorcet-3.0.1-py2.py3-none-any.whl (1.7 MB)
Collecting joblib
  Using cached joblib-1.2.0-py3-none-any.whl (297 kB)
Collecting pyopengl
  Using cached PyOpenGL-3.1.6-py3-none-any.whl (2.4 MB)
Collecting mkdocs
  Using cached mkdocs-1.4.2-py3-none-any.whl (3.7 MB)
Collecting phylib
  Using cached phylib-2.4.3-py2.py3-none-any.whl (80 kB)
Collecting cython
  Using cached Cython-3.0.0b2-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (3.6 MB)
Collecting pyct>=0.4.4
  Using cached pyct-0.5.0-py2.py3-none-any.whl (15 kB)
Collecting toolz>=0.10.0
  Using cached toolz-0.12.0-py3-none-any.whl (55 kB)
Collecting packaging>=20.0
  Using cached packaging-23.1-py3-none-any.whl (48 kB)
Collecting partd>=1.2.0
  Using cached partd-1.4.0-py3-none-any.whl (18 kB)
Collecting pyyaml>=5.3.1
  Using cached PyYAML-6.0-cp310-cp310-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_12_x86_64.manylinux2010_x86_64.whl (682 kB)
Collecting importlib-metadata>=4.13.0
  Using cached importlib_metadata-6.6.0-py3-none-any.whl (22 kB)
Collecting cloudpickle>=1.5.0
  Using cached cloudpickle-2.2.1-py3-none-any.whl (25 kB)
Collecting fsspec>=2021.09.0
  Using cached fsspec-2023.4.0-py3-none-any.whl (153 kB)
Collecting contourpy>=1.0.1
  Using cached contourpy-1.0.7-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (300 kB)
Collecting pyparsing>=2.3.1
  Using cached pyparsing-3.1.0b1-py3-none-any.whl (102 kB)
Collecting kiwisolver>=1.0.1
  Using cached kiwisolver-1.4.4-cp310-cp310-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (1.6 MB)
Collecting python-dateutil>=2.7
  Using cached python_dateutil-2.8.2-py2.py3-none-any.whl (247 kB)
Collecting cycler>=0.10
  Using cached cycler-0.11.0-py3-none-any.whl (6.4 kB)
Collecting fonttools>=4.22.0
  Using cached fonttools-4.39.3-py3-none-any.whl (1.0 MB)
Collecting markdown<3.4,>=3.2.1
  Using cached Markdown-3.3.7-py3-none-any.whl (97 kB)
Collecting ghp-import>=1.0
  Using cached ghp_import-2.1.0-py3-none-any.whl (11 kB)
Collecting mergedeep>=1.3.4
  Using cached mergedeep-1.3.4-py3-none-any.whl (6.4 kB)
Collecting jinja2>=2.11.1
  Using cached Jinja2-3.1.2-py3-none-any.whl (133 kB)
Collecting pyyaml-env-tag>=0.1
  Using cached pyyaml_env_tag-0.1-py3-none-any.whl (3.9 kB)
Collecting watchdog>=2.0
  Using cached watchdog-3.0.0-py3-none-manylinux2014_x86_64.whl (82 kB)
Collecting PyQt5-sip<13,>=12.11
  Using cached PyQt5_sip-12.12.1-cp310-cp310-manylinux_2_5_x86_64.manylinux1_x86_64.whl (360 kB)
Collecting PyQt5-Qt5>=5.15.2
  Using cached PyQt5_Qt5-5.15.2-py3-none-manylinux2014_xphy template-gui params.py 
15:45:39.332 [W] model:603            Unreferenced clusters found in templates (generally not a problem)
15:45:39.333 [W] model:625            Unreferenced clusters found in spike_clusters (generally not a problem)
15:45:39.337 [W] model:667            Skipping spike waveforms that do not exist, they will be extracted on the fly from the raw data as needed.
15:45:39.338 [W] model:55             133/4489 values are nan in /home/rth/Rhythm_FPGA-100.0-ks2.5/similar_templates.npy, replacing by zero.

86_64.whl (59.9 MB)
Collecting PyQtWebEngine-Qt5>=5.15.0
  Using cached PyQtWebEngine_Qt5-5.15.2-py3-none-manylinux2014_x86_64.whl (67.5 MB)
Collecting jupyter-core
  Using cached jupyter_core-5.3.0-py3-none-any.whl (93 kB)
Collecting jupyter-client>=4.1
  Using cached jupyter_client-8.2.0-py3-none-any.whl (103 kB)
Collecting ipython-genutils
  Using cached ipython_genutils-0.2.0-py2.py3-none-any.whl (26 kB)
Collecting pygments
  Using cached Pygments-2.15.1-py3-none-any.whl (1.1 MB)
Collecting qtpy>=2.0.1
  Using cached QtPy-2.3.1-py3-none-any.whl (84 kB)
Collecting traitlets!=5.2.1,!=5.2.2
  Using cached traitlets-5.9.0-py3-none-any.whl (117 kB)
Collecting ipykernel>=4.1
  Using cached ipykernel-6.22.0-py3-none-any.whl (149 kB)
Collecting pyzmq>=17.1
  Using cached pyzmq-25.0.2-cp310-cp310-manylinux_2_28_x86_64.whl (1.1 MB)
Collecting idna<4,>=2.5
  Using cached idna-3.4-py3-none-any.whl (61 kB)
Collecting urllib3<1.27,>=1.21.1
  Using cached urllib3-1.26.15-py2.py3-none-any.whl (140 kB)
Collecting charset-normalizer<4,>=2
  Using cached charset_normalizer-3.1.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (199 kB)
Collecting certifi>=2017.4.17
  Using cached certifi-2022.12.7-py3-none-any.whl (155 kB)
Collecting zipp>=0.5
  Using cached zipp-3.15.0-py3-none-any.whl (6.8 kB)
Collecting debugpy>=1.6.5
  Using cached debugpy-1.6.7-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (3.0 MB)
Collecting comm>=0.1.1
  Using cached comm-0.1.3-py3-none-any.whl (6.6 kB)
Collecting tornado>=6.1
  Using cached tornado-6.3.1-cp38-abi3-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (426 kB)
Collecting psutil
  Using cached psutil-5.9.5-cp36-abi3-manylinux_2_12_x86_64.manylinux2010_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (282 kB)
Collecting ipython>=7.23.1
  Using cached ipython-8.13.1-py3-none-any.whl (797 kB)
Collecting matplotlib-inline>=0.1
  Using cached matplotlib_inline-0.1.6-py3-none-any.whl (9.4 kB)
Collecting nest-asyncio
  Using cached nest_asyncio-1.5.6-py3-none-any.whl (5.2 kB)
Collecting MarkupSafe>=2.0
  Using cached MarkupSafe-2.1.2-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (25 kB)
Collecting platformdirs>=2.5
  Using cached platformdirs-3.5.0-py3-none-any.whl (15 kB)
Collecting locket
  Using cached locket-1.0.0-py2.py3-none-any.whl (4.4 kB)
Collecting param>=1.7.0
  Using cached param-1.13.0-py2.py3-none-any.whl (87 kB)
Collecting six>=1.5
  Using cached six-1.16.0-py2.py3-none-any.whl (11 kB)
Collecting backcall
  Using cached backcall-0.2.0-py2.py3-none-any.whl (11 kB)
Collecting prompt-toolkit!=3.0.37,<3.1.0,>=3.0.30
  Using cached prompt_toolkit-3.0.38-py3-none-any.whl (385 kB)
Collecting pickleshare
  Using cached pickleshare-0.7.5-py2.py3-none-any.whl (6.9 kB)
Collecting stack-data
  Using cached stack_data-0.6.2-py3-none-any.whl (24 kB)
Collecting pexpect>4.3
  Using cached pexpect-4.8.0-py2.py3-none-any.whl (59 kB)
Collecting decorator
  Using cached decorator-5.1.1-py3-none-any.whl (9.1 kB)
Collecting jedi>=0.16
  Using cached jedi-0.18.2-py2.py3-none-any.whl (1.6 MB)
Collecting parso<0.9.0,>=0.8.0
  Using cached parso-0.8.3-py2.py3-none-any.whl (100 kB)
Collecting ptyprocess>=0.5
  Using cached ptyprocess-0.7.0-py2.py3-none-any.whl (13 kB)
Collecting wcwidth
  Using cached wcwidth-0.2.6-py2.py3-none-any.whl (29 kB)
Collecting executing>=1.2.0
  Using cached executing-1.2.0-py2.py3-none-any.whl (24 kB)
Collecting pure-eval
  Using cached pure_eval-0.2.2-py3-none-any.whl (11 kB)
Collecting asttokens>=2.1.0
  Using cached asttokens-2.2.1-py2.py3-none-any.whl (26 kB)
Installing collected packages: wcwidth, PyQtWebEngine-Qt5, PyQt5-Qt5, pyopengl, pure-eval, ptyprocess, pickleshare, mtscomp, ipython-genutils, executing, backcall, zipp, watchdog, urllib3, traitlets, tqdm, tornado, toolz, six, pyzmq, pyyaml, PyQt5-sip, pyparsing, pygments, psutil, prompt-toolkit, platformdirs, pillow, pexpect, parso, param, packaging, numpy, nest-asyncio, mergedeep, MarkupSafe, markdown, locket, kiwisolver, joblib, idna, fsspec, fonttools, decorator, debugpy, cython, cycler, cloudpickle, click, charset-normalizer, certifi, scipy, requests, qtpy, pyyaml-env-tag, python-dateutil, PyQt5, pyct, partd, matplotlib-inline, jupyter-core, jinja2, jedi, importlib-metadata, h5py, contourpy, comm, asttokens, stack-data, PyQtWebEngine, matplotlib, jupyter-client, ghp-import, dask, colorcet, phylib, mkdocs, ipython, ipykernel, qtconsole, phy
Successfully installed MarkupSafe-2.1.2 PyQt5-5.15.9 PyQt5-Qt5-5.15.2 PyQt5-sip-12.12.1 PyQtWebEngine-5.15.6 PyQtWebEngine-Qt5-5.15.2 asttokens-2.2.1 backcall-0.2.0 certifi-2022.12.7 charset-normalizer-3.1.0 click-8.1.3 cloudpickle-2.2.1 colorcet-3.0.1 comm-0.1.3 contourpy-1.0.7 cycler-0.11.0 cython-3.0.0b2 dask-2023.4.1 debugpy-1.6.7 decorator-5.1.1 executing-1.2.0 fonttools-4.39.3 fsspec-2023.4.0 ghp-import-2.1.0 h5py-3.8.0 idna-3.4 importlib-metadata-6.6.0 ipykernel-6.22.0 ipython-8.13.1 ipython-genutils-0.2.0 jedi-0.18.2 jinja2-3.1.2 joblib-1.2.0 jupyter-client-8.2.0 jupyter-core-5.3.0 kiwisolver-1.4.4 locket-1.0.0 markdown-3.3.7 matplotlib-3.7.1 matplotlib-inline-0.1.6 mergedeep-1.3.4 mkdocs-1.4.2 mtscomp-1.0.2 nest-asyncio-1.5.6 numpy-1.24.3 packaging-23.1 param-1.13.0 parso-0.8.3 partd-1.4.0 pexpect-4.8.0 phy-2.0b5 phylib-2.4.3 pickleshare-0.7.5 pillow-9.5.0 platformdirs-3.5.0 prompt-toolkit-3.0.38 psutil-5.9.5 ptyprocess-0.7.0 pure-eval-0.2.2 pyct-0.5.0 pygments-2.15.1 pyopengl-3.1.6 pyparsing-3.1.0b1 python-dateutil-2.8.2 pyyaml-6.0 pyyaml-env-tag-0.1 pyzmq-25.0.2 qtconsole-5.4.2 qtpy-2.3.1 requests-2.29.0 scipy-1.10.1 six-1.16.0 stack-data-0.6.2 toolz-0.12.0 tornado-6.3.1 tqdm-4.65.0 traitlets-5.9.0 urllib3-1.26.15 watchdog-3.0.0 wcwidth-0.2.6 zipp-3.15.0

phy crashes because of fresh numpy

(phy) <ws>:~/.local/apps$ phy -h
/home/rth/.local/apps/phy/lib/python3.10/site-packages/phy/plot/gloo/variable.py:86: FutureWarning: In the future `np.bool` will be defined as the corresponding NumPy scalar.
  gl.GL_BOOL: (1, gl.GL_BOOL, np.bool),
Traceback (most recent call last):
  File "/home/rth/.local/apps/phy/bin/phy", line 5, in <module>
    from phy.apps import phycli
  File "/home/rth/.local/apps/phy/lib/python3.10/site-packages/phy/apps/__init__.py", line 25, in <module>
    from .base import (  # noqa
  File "/home/rth/.local/apps/phy/lib/python3.10/site-packages/phy/apps/base.py", line 26, in <module>
    from phy.cluster._utils import RotatingProperty
  File "/home/rth/.local/apps/phy/lib/python3.10/site-packages/phy/cluster/__init__.py", line 9, in <module>
    from .views import *  # noqa
  File "/home/rth/.local/apps/phy/lib/python3.10/site-packages/phy/cluster/views/__init__.py", line 10, in <module>
    from .base import ManualClusteringView  # noqa
  File "/home/rth/.local/apps/phy/lib/python3.10/site-packages/phy/cluster/views/base.py", line 21, in <module>
    from phy.plot import PlotCanvas, NDC, extend_bounds
  File "/home/rth/.local/apps/phy/lib/python3.10/site-packages/phy/plot/__init__.py", line 17, in <module>
    from .base import BaseVisual, GLSLInserter, BaseCanvas, BaseLayout
  File "/home/rth/.local/apps/phy/lib/python3.10/site-packages/phy/plot/base.py", line 20, in <module>
    from . import gloo
  File "/home/rth/.local/apps/phy/lib/python3.10/site-packages/phy/plot/gloo/__init__.py", line 10, in <module>
    from .program import Program
  File "/home/rth/.local/apps/phy/lib/python3.10/site-packages/phy/plot/gloo/program.py", line 18, in <module>
    from .variable import Uniform, Attribute
  File "/home/rth/.local/apps/phy/lib/python3.10/site-packages/phy/plot/gloo/variable.py", line 86, in <module>
    gl.GL_BOOL: (1, gl.GL_BOOL, np.bool),
  File "/home/rth/.local/apps/phy/lib/python3.10/site-packages/numpy/__init__.py", line 305, in __getattr__
    raise AttributeError(__former_attrs__[attr])
AttributeError: module 'numpy' has no attribute 'bool'.
`np.bool` was a deprecated alias for the builtin `bool`. To avoid this error in existing code, use `bool` by itself. Doing this will not modify any behavior and is safe. If you specifically wanted the numpy scalar type, use `np.bool_` here.
The aliases was originally deprecated in NumPy 1.20; for more details and guidance see the original release note at:
    https://numpy.org/devdocs/release/1.20.0-notes.html#deprecations. Did you mean: 'bool_'?

FIX for the numpy problem

(phy) <ws>:~/.local/apps$ pip install numpy==1.23.5
Collecting numpy==1.23.5
  Using cached numpy-1.23.5-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (17.1 MB)
Installing collected packages: numpy
  Attempting uninstall: numpy
    Found existing installation: numpy 1.24.3
    Uninstalling numpy-1.24.3:
      Successfully uninstalled numpy-1.24.3
Successfully installed numpy-1.23.5
(phy) <ws>:~/.local/apps$ phy -h
Usage: phy [OPTIONS] COMMAND [ARGS]...

  Interactive visualization and manual spike sorting of large-scale ephys
  data.

Options:
  --version   Show the version and exit.
  -h, --help  Show this message and exit.

Commands:
  alf-convert        Convert an ephys dataset into ALF.
  extract-waveforms  Extract spike waveforms.
  kwik-describe      Describe a Kwik file.
  kwik-gui           Launch the Kwik GUI on a Kwik file.
  template-describe  Describe a template file.
  template-gui       Launch the template GUI on a params.py file.
  trace-gui          Launch the trace GUI on a raw data file.

Trying KS 2.5 phy export

(phy) <ws>:~/.local/apps$ cd ~/Rhythm_FPGA-100.0-ks2.5/
(phy) <ws>:~/Rhythm_FPGA-100.0-ks2.5$ phy template-describe params.py 
15:44:42.226 [W] model:603            Unreferenced clusters found in templates (generally not a problem)
15:44:42.228 [W] model:625            Unreferenced clusters found in spike_clusters (generally not a problem)
15:44:42.232 [W] model:667            Skipping spike waveforms that do not exist, they will be extracted on the fly from the raw data as needed.
15:44:42.233 [W] model:55             133/4489 values are nan in /home/rth/Rhythm_FPGA-100.0-ks2.5/similar_templates.npy, replacing by zero.
Data files              /home/rth/Rhythm_FPGA-100.0-ks2.5/temp_wh.dat
Directory               /home/rth/Rhythm_FPGA-100.0-ks2.5
Duration                1014.6s
Sample rate             30.0 kHz
Data type               int16
# of channels           128
# of channels (raw)     128
# of templates          67
# of spikes             46,850
(phy) <ws>:~/Rhythm_FPGA-100.0-ks2.5$ phy template-gui params.py 
15:45:39.332 [W] model:603            Unreferenced clusters found in templates (generally not a problem)
15:45:39.333 [W] model:625            Unreferenced clusters found in spike_clusters (generally not a problem)
15:45:39.337 [W] model:667            Skipping spike waveforms that do not exist, they will be extracted on the fly from the raw data as needed.
15:45:39.338 [W] model:55             133/4489 values are nan in /home/rth/Rhythm_FPGA-100.0-ks2.5/similar_templates.npy, replacing by zero.

Screenshot_20230502_152610

Then try to close

Screenshot_20230502_152638 Screenshot_20230502_152645

About this issue

  • Original URL
  • State: closed
  • Created a year ago
  • Comments: 27 (10 by maintainers)

Most upvoted comments

IT WORKS!

@rat-h Definitely keep me in the loop if you figure it out. I want to switch over to PopOS eventually, but I’m stuck on windows for work.

@caniko Sorry wasn’t meaning to blame linux. I meant that getting the packages to play together can require some work so I was agreeing with you in that it could be a specific build issue. Some of the other issues lately have involved Ubuntu problems, so I was wondering if some of the recent updates to some subset of packages upstream at the Ubuntu level are causing issues.

I guess then that still leaves the question why linux is doing this wrong, but windows isn’t. Since this is at the wait/timeout block (or the traceback is wrong) there must be some issue there, which then causes the gui to freak out and try to reload with empty dataset. I think you figured out what is going wrong, but I honestly don’t know why it is going wrong. Like I said, I’m not super familiar with all the packaging behind linux distros, so I probably won’t be any help beyond this point. If you have a working windows computer and access to a server I guess you could do the meat of your analysis on your Linux machine, do phy on windows, and then do the final analysis back on your linux machine. Annoying, but at least it works.

it is installed! I use KDE and libqt5-dev is installed 😃

qmake --version
QMake version 3.1
Using Qt version 5.15.3 in /usr/lib/x86_64-linux-gnu

Otherwise, I would have lib error, not an empty GUI with all right subwindows and doc areas… All Qt apps are functioning correctly on this computer.

Also the same phy directory works fine in Windows. So it is truly Linux related problem.