phy: Assertion Error when opening spike sorted file

Hi,

I recently ran a recording through Kilosort2.5, where it reported to find spikes and good units (if few), but when I try to open it up in phy I get this error:

16:07:42.861 [W] model:545            Unreferenced clusters found in templates (generally not a problem)
16:07:42.938 [W] model:567            Unreferenced clusters found in spike_clusters (generally not a problem)
16:07:42.965 [W] model:54             4279/40000 values are nan in C:\Users\idavalik\Data\Neuropixels\Socal_track_day5_g0\Socal_track_day5_g0_imec0\similar_templates.npy, replacing by zero.
16:07:43.826 [E] __init__:62          An error has occurred (AssertionError):
Traceback (most recent call last):
  File "c:\users\idavalik\appdata\local\continuum\anaconda3\envs\phy2\lib\runpy.py", line 193, in _run_module_as_main
    "__main__", mod_spec)
  File "c:\users\idavalik\appdata\local\continuum\anaconda3\envs\phy2\lib\runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "C:\Users\idavalik\AppData\Local\Continuum\anaconda3\envs\phy2\Scripts\phy.exe\__main__.py", line 7, in <module>
    sys.exit(phycli())
  File "c:\users\idavalik\appdata\local\continuum\anaconda3\envs\phy2\lib\site-packages\click\core.py", line 829, in __call__
    return self.main(*args, **kwargs)
  File "c:\users\idavalik\appdata\local\continuum\anaconda3\envs\phy2\lib\site-packages\click\core.py", line 782, in main
    rv = self.invoke(ctx)
  File "c:\users\idavalik\appdata\local\continuum\anaconda3\envs\phy2\lib\site-packages\click\core.py", line 1259, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "c:\users\idavalik\appdata\local\continuum\anaconda3\envs\phy2\lib\site-packages\click\core.py", line 1066, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "c:\users\idavalik\appdata\local\continuum\anaconda3\envs\phy2\lib\site-packages\click\core.py", line 610, in invoke
    return callback(*args, **kwargs)
  File "c:\users\idavalik\appdata\local\continuum\anaconda3\envs\phy2\lib\site-packages\click\decorators.py", line 21, in new_func
    return f(get_current_context(), *args, **kwargs)
  File "c:\users\idavalik\appdata\local\continuum\anaconda3\envs\phy2\lib\site-packages\phy\apps\__init__.py", line 159, in cli_template_gui
    template_gui(params_path, **kwargs)
  File "c:\users\idavalik\appdata\local\continuum\anaconda3\envs\phy2\lib\site-packages\phy\apps\template\gui.py", line 199, in template_gui
    gui = controller.create_gui()
  File "c:\users\idavalik\appdata\local\continuum\anaconda3\envs\phy2\lib\site-packages\phy\apps\base.py", line 1638, in create_gui
    self.supervisor.attach(gui)
  File "c:\users\idavalik\appdata\local\continuum\anaconda3\envs\phy2\lib\site-packages\phy\cluster\supervisor.py", line 942, in attach
    gui=gui, sort=gui.state.get('ClusterView', {}).get('current_sort', None))
  File "c:\users\idavalik\appdata\local\continuum\anaconda3\envs\phy2\lib\site-packages\phy\cluster\supervisor.py", line 760, in _create_views
    gui, data=self.cluster_info, columns=self.columns, sort=sort)
  File "c:\users\idavalik\appdata\local\continuum\anaconda3\envs\phy2\lib\site-packages\phy\cluster\supervisor.py", line 916, in cluster_info
    return [self.get_cluster_info(cluster_id) for cluster_id in self.clustering.cluster_ids]
  File "c:\users\idavalik\appdata\local\continuum\anaconda3\envs\phy2\lib\site-packages\phy\cluster\supervisor.py", line 916, in <listcomp>
    return [self.get_cluster_info(cluster_id) for cluster_id in self.clustering.cluster_ids]
  File "c:\users\idavalik\appdata\local\continuum\anaconda3\envs\phy2\lib\site-packages\phy\cluster\supervisor.py", line 745, in get_cluster_info
    out[key] = func(cluster_id)
  File "c:\users\idavalik\appdata\local\continuum\anaconda3\envs\phy2\lib\site-packages\phy\apps\base.py", line 1150, in get_best_channel_label
    return self._get_channel_labels([self.get_best_channel(cluster_id)])[0]
  File "c:\users\idavalik\appdata\local\continuum\anaconda3\envs\phy2\lib\site-packages\phy\utils\context.py", line 154, in memcached
    out = f(*args, **kwargs)
  File "c:\users\idavalik\appdata\local\continuum\anaconda3\envs\phy2\lib\site-packages\phy\apps\base.py", line 1144, in get_best_channel
    channel_ids = self.get_best_channels(cluster_id)
  File "c:\users\idavalik\appdata\local\continuum\anaconda3\envs\phy2\lib\site-packages\phy\utils\context.py", line 154, in memcached
    out = f(*args, **kwargs)
  File "c:\users\idavalik\appdata\local\continuum\anaconda3\envs\phy2\lib\site-packages\phy\apps\template\gui.py", line 149, in get_best_channels
    template = self.model.get_template(template_id)
  File "c:\users\idavalik\appdata\local\continuum\anaconda3\envs\phy2\lib\site-packages\phylib\io\model.py", line 855, in get_template
    return self._get_template_dense(template_id, channel_ids=channel_ids)
  File "c:\users\idavalik\appdata\local\continuum\anaconda3\envs\phy2\lib\site-packages\phylib\io\model.py", line 798, in _get_template_dense
    channel_ids_, amplitude, best_channel = self._find_best_channels(template)
  File "c:\users\idavalik\appdata\local\continuum\anaconda3\envs\phy2\lib\site-packages\phylib\io\model.py", line 787, in _find_best_channels
    assert best_channel in channel_ids
AssertionError


The recording was very noisy and I fear/suspect some probe damage, but I have had no problems opening noisy files before.

About this issue

  • Original URL
  • State: open
  • Created 4 years ago
  • Comments: 43 (5 by maintainers)

Most upvoted comments

I pushed a tentative fix on phylib (master branch), empty templates in KS2.5 are NaN which phy doesn’t support for some reason – the test dataset I have seems to load correctly after replacing NaNs with zeros

I am experiencing this issue as well. I tried the workaround suggested by @JoseGuzman without success. @rossant, is there any update? I am also willing to share the recording I am using via dropbox if that would be helpful.

@torbenott I was able to resolve this issue by downgrading Kilosort to the 2.0 release. I saw that @Hankslab-ABG tried this too which resulted in some weird sorting results, but in my case, the auto-sorting done by Kilosort2 looks perfectly reasonable. For what it’s worth I am using the Neuropixels 1.0 probes.

I tried re-running after deleting any of the files generated by KS2.5 but I still get this error. I don’t know the exact cause of this error but I suspect it has something to do with the fact that in my dataset (and maybe @Idavr as well?), a few channels are disabled. Then assert best_channel in channel_ids fails because the best channel for one cluster is mis-assigned to one of the disabled channels. @marius10p do you think this is possible?

For some recordings I have been able to work around it by connecting all channels (including reference), but this issue is not consistent across all my recordings. In the cases where I am completely unable to work around it with KS2.5, I use KS2 without issues, but I haven’t found a consistent reason for these phy issues when spike-sorting with KS2.5.

Any updates on this?

For what it’s worth, I re-ran a couple of the datasets that wouldn’t open in Phy when using KS2.5. I used KS2 instead of 2.5 and was able to open them in Phy. However, what I found was that all of the clusters were based around very ill-defined waveforms that did not look spike-like at all. Has anyone else tried this? I know that some had mentioned noisy datasets might be behind this issue, so I’d be interested to see if anyone was able to get sensible-looking waveforms when re-running in KS2.

sure!

@rossant would a Google Drive link work for you?

I’m now also getting a similar error trying to run phy on a KS 2.5 sorted session where a reference channel on an NPx probe was disabled. Any insights since the last posting?

Sent.

Any file-sharing like Dropbox, at cyrille.rossant at gmail.com

Le sam. 14 nov. 2020 à 16:28, Idavr notifications@github.com a écrit :

@rossant https://github.com/rossant Appreciate it very much! I could send you one of mine to take a look at? How would you want it sent?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/cortex-lab/phy/issues/1051#issuecomment-727223292, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAO2GVZAV2SB3SGSA7YSR7LSP2O3FANCNFSM4TQZDDGA .

I’ve also tried re-running it multiple times, getting the same error. Since this happened after something funky happened with the channels during the actual recording (the above mentioned noise) I am unable to open any sessions done after this weirdness, even if the raw stream of the recording does not have the same kind of noise. There are still very - very - few good units being detected by KS2.5, so I can not say whether that is a confounding factor but when there has been no spikes or units previously I haven’t been able to run KS at all. A sudden channel mislabeling (for whatever reason) would make sense, but I am very interested to hear what Marius has to say on the subject.