lightkurve: Couldn't select pixels of TPFs in interact()

Hey ya’ll, yes Bryan is absolutely right, and the bug is clear— I used 2D arrays for a ColumnDataSource, that only supports 1D arrays. The fact that it worked at all is sort of a coincidence/miracle, and I thank Bryan for figuring that out, and also apologize for letting this issue fester and polluting your notifications. Sorry Bryan! We are grateful for all the awesome bokeh features that enabled this popular interact() dashboard.

The clear fix is to flatten and the arrays. I started a PR (linked) in that direction, but it needs some spot-checking of array shapes that is tricky to do without some numpy surgery. If any lightkurve users want to pair code with me for fun, tweet @ me: gully_ And yes to reiterate what previous folks have said, the temporary workaround is to use bokeh <2.3, but we emphasize that’s temporary while we address the root cause.

Update: a fix for this issue has been included in the Lightkurve v2.2.0 release.

@orionlee As I mentioned above, there are problems on the Lightkurve side that will have to be addressed:

This is not a 1d array. For all the “scatter” type glyphs (which is almost all glyphs. including rect) the CDS should contain only one-dimensional arrays. Perhaps something “worked” by accident previously, I’m not sure. But this has never been the expected data format. These arrays should be flattened.

I’m not aware of anything that should have changed with image selection. We don’t actually have many examples of image selection that I can think of, so perhaps something slipped through. Though the image hover example is functioning as expected and that essentially uses all the same internal machinery.

What would really help investigate is to isolate a minimal reproducer in a standalone script.

Edit: Also, if you haven’t looked already, the browser JavaScript console log may hold clues.

I tried in Bokeh version 2.2.3. And it’s working perfectly. Also, note that when I used Bokeh version 2.3.0, the interact() didn’t even show the default pipeline mask. In version 2.2.3, everything is working perfectly.

OK I printed out the actual data for the rect in the callback and I believe this is the problem:

{'xx': array([[447., 448., 449., 450., 451., 452., 453., 454., 455., 456., 457.],
       [447., 448., 449., 450., 451., 452., 453., 454., 455., 456., 457.],
       [447., 448., 449., 450., 451., 452., 453., 454., 455., 456., 457.],
       [447., 448., 449., 450., 451., 452., 453., 454., 455., 456., 457.],
       [447., 448., 449., 450., 451., 452., 453., 454., 455., 456., 457.],
       [447., 448., 449., 450., 451., 452., 453., 454., 455., 456., 457.],
       [447., 448., 449., 450., 451., 452., 453., 454., 455., 456., 457.],
       [447., 448., 449., 450., 451., 452., 453., 454., 455., 456., 457.],
       [447., 448., 449., 450., 451., 452., 453., 454., 455., 456., 457.],
       [447., 448., 449., 450., 451., 452., 453., 454., 455., 456., 457.],
       [447., 448., 449., 450., 451., 452., 453., 454., 455., 456., 457.]])

This is not a 1d array. For all the “scatter” type glyphs (which is almost all glyphs. including rect) the CDS should contain only one-dimensional arrays. Perhaps something worked by accident previously, I’m not sure. But this has never been the expected data format. These arrays should be flattened.

I’m not at all familiar with the specific subject matter, so not in a position to submit or evaluate the correctness of a fix for this (I also do not have the bandwidth to do so) but hopefully this points someone else in he right direction.

So lightkurve is doing looking to respond on changes to “indices”:

https://github.com/lightkurve/lightkurve/blob/d9f9ba1145cad12ddc2b2bef0413827fc7e4a09c/src/lightkurve/interact.py#L860

But Bokeh set image_indices for at least two years, if not forever:

https://github.com/bokeh/bokeh/blame/branch-2.4/bokehjs/src/lib/models/glyphs/image_base.ts#L192

So speaking candidly don’t understand how this could have ever worked at all.

Edit: Oh, those are rect, not an image? I was hoping (ish) to demonstrate some issue with tap selection on rect but this isolated example is functioning exactly as expected:

from bokeh.io import curdoc
from bokeh.plotting import figure, ColumnDataSource

source = ColumnDataSource(data=dict(
    x=[0,1,2] * 3,
    y=[0,0,0,1,1,1,2,2,2]
))

p = figure(tools="tap")
p.rect("x", "y", width=1, height=1, line_color="white", source=source)

def cb(attr, old, new):
    print(source.selected.indices)

source.selected.on_change("indices", cb)

curdoc().add_root(p)

Fascinating, I’m seeing a similar bug in Linux / Chrome / bokeh=2.3.0 / and lightkurve=2.0.3. I’m using TESScut TPFs. The TPF mask selection defaults to the bottom row, and pixel selection is offset/fickle. Now that I have it working on my machine I will attempt to get to the bottom of it and make a minimal reproducer in a standalone script. Importantly there is now circumstantial evidence that this only arises in bokeh=2.3.0. So if you are a lightkurve user seeing this issue, a workaround is to downgrade to bokeh<2.3

Fantastic to hear we have a way to reproduce the issue, that’s very helpful! 👍

We’ll want to have a look at the change log for bokeh 2.3.0 and try to understand why the behavior changed, and whether or not Lightkurve needs to be adapted.

I’m not a fan of changing the dependency to say bokeh<2.3.0 for reasons explained here, but we could specify bokeh != 2.3.0 if we were to ascertain that it is a temporary bug.

(I won’t have time to dig into this myself, so any and all help welcome!)

What browser are you using for Jupyter? As another workaround and debugging step, I would recommend seeing if the bug reproduces in other browsers. I found no bug in Firefox with bokeh version 2.2.3.

Update: Google Chrome browser in Windows behaves as expected also— no bug.

Hi @NaveenTime thanks for your interest in lightkurve especially my favorite tool .interact()! 🚀

My first instinct was that it could be a Windows issue, since Windows may not have as-extensive user testing, so I did a fresh install of lightkurve (2.0.4.dev) on Windows 10 and was not able to reproduce this erroneous interact pixel-selection bug— the pixels selected as intended.

I seem to recall having some strange selection problems with dual monitors a while ago. Are you using dual monitors?