vscode-jupyter: Jupyter plugin hides some custom kernelspecs when the executable defined in the argv cannot be found

Like @iakovn, myself (and actually multiple other physicists in my collaboration but I’m the one debugging it on our behalf) are hitting this. We’d like to be able to use VS Code’s remote explorer to interact with Jupyter on remote environments but given some requirements for the software that we need in the environment (we need to be able to see and interact with the CernVM File System) we need to mutate the default kernelspec generated from something like

(lcg-example) [23:01] login01.af.uchicago.edu:~ $ python -m ipykernel install --user --name="lcg-example" --display-name="Example LCG view + venv kernel"
Installed kernelspec lcg-example in /home/feickert/.local/share/jupyter/kernels/lcg-example
(lcg-example) [23:10] login01.af.uchicago.edu:~ $ cat ~/.local/share/jupyter/kernels/lcg-example/.bak.kernel.json
{
 "argv": [
  "/home/feickert/.venvs/lcg-example/bin/python",
  "-m",
  "ipykernel_launcher",
  "-f",
  "{connection_file}"
 ],
 "display_name": "Example LCG view + venv kernel",
 "language": "python",
 "metadata": {
  "debugger": true
 }
}(lcg-example) [23:10] login01.af.uchicago.edu:~ $ 

to something that would allow for the environment manipulation suggested by the Jupyter team in https://github.com/jupyterhub/jupyterhub/issues/847#issuecomment-260152425

(lcg-example) [23:11] login01.af.uchicago.edu:~ $ cat ~/.local/share/jupyter/kernels/lcg-example/kernel.json 
{
 "argv": [
  "/home/feickert/.local/share/jupyter/kernels/lcg-example/startup.sh",
  "-f",
  "{connection_file}"
 ],
 "display_name": "Example LCG view + venv kernel",
 "language": "python",
 "metadata": {
  "debugger": true
 }
}
(lcg-example) [23:11] login01.af.uchicago.edu:~ $ cat ~/.local/share/jupyter/kernels/lcg-example/startup.sh 
#!/usr/bin/env bash

export ATLAS_LOCAL_ROOT_BASE=/cvmfs/atlas.cern.ch/repo/ATLASLocalRootBase
# Allows for working with wrappers as well
source "${ATLAS_LOCAL_ROOT_BASE}/user/atlasLocalSetup.sh" --quiet || echo "~~~ERROR: setupATLAS failed!~~~"

lsetup 'views LCG_103 x86_64-centos7-gcc11-opt'

. ~/.venvs/lcg-example/bin/activate

exec python -m ipykernel_launcher $@
(lcg-example) [23:11] login01.af.uchicago.edu:~ $ 

Resolution on my side was to change “mycommand” into “/full/path/mycommand” and it worked!

@iakovn, can you elaborate on this? My experience with the valid kernelspec above is failure with VS Code failing to find the kernel at all.

@DonJayamanne while I appreciate that the VS Code team might not want to make things visible by default, can it be made possible so that there is at least some dev option to still give a full path to a valid kernel?

---

Environment:

• VS Code version: 1.79.2
• Jupyter Extension version (available under the Extensions sidebar): v2023.6.1101941928
• Python Extension version (available under the Extensions sidebar): v2023.12.0
• OS (Windows | Mac | Linux distro) and version: Linux Ubuntu 22.04 (remote server is CentOS7)
• Python and/or Anaconda version: CPython 3.9.12
• Type of virtual environment used (N/A | venv | virtualenv | conda | …): venv
• Jupyter server running: Remote

Ouput after enabling Jupyter: Show Output:

Visual Studio Code (1.80.0, ssh-remote, desktop)
Jupyter Extension Version: 2023.6.1101941928.
Python Extension Version: 2023.12.0.
Platform: linux (x64).
Workspace folder /home/feickert, Home = /home/feickert
23:30:52.840 [info] User belongs to experiment group 'FastKernelPicker'
23:30:53.125 [info] Start refreshing Kernel Picker (1689654653125)
23:30:53.135 [info] Using Pylance
23:30:54.641 [warn] No interpreter with path ~/.local/share/jupyter/kernels/lcg-example/startup.sh found in Python API, will convert Uri path to string as Id ~/.local/share/jupyter/kernels/lcg-example/startup.sh
23:30:54.645 [warn] No interpreter with path ~/example/.venv/bin/python found in Python API, will convert Uri path to string as Id ~/example/.venv/bin/python
23:30:55.194 [info] Process Execution: ~/.venvs/lcg-example/bin/python -m pip list
23:30:56.757 [warn] Kernel Spec for 'Local Test' (~/.local/share/jupyter/kernels/local-test-venv/kernel.json) hidden, as we cannot find a matching interpreter argv = '~/example/.venv/bin/python'
23:31:12.764 [warn] Kernel Spec for 'Example LCG view + venv kernel' (~/.local/share/jupyter/kernels/lcg-example/kernel.json) hidden, as we cannot find a matching interpreter argv = '~/.local/share/jupyter/kernels/lcg-example/startup.sh'
23:31:12.847 [info] End refreshing Kernel Picker (1689654653125)

---

ATLAS Experiment internal reference link: https://atlas-talk.sdcc.bnl.gov/t/running-jupyter-notebooks-within-vscode/445

oh sorry, we will still display these custom python kernel specs, and if it fails, its on the user, basically you have a bogus kernel spec, and we have no idea its bogus or not. In the issue you can see users have custom kernel.specs and just because we fail, doesn’t mean its invalid, the language can be python but they are not really pyton kernels, e.g. they can be launched through some custom executable.

So its correct to display these, thats whats expected.

Hi everyone, I’m sorry about this issue, I will review the current workflow next iteration. Perhaps what the Jupyter extension can do is,

• Not hide items that have paths such as /bin/bash or the like
• And if we do hide items (which have ambiguous paths such as python), then we display a more informative message in the logs and how it can be resolved.

Those are good fixes! I think that docs are still the higher prio (inform that some kernels may be hidden, point out dependency on python interpreter setting, recommend “show output” to see warnings, etc).

If you can spend more time, then a new setting to explicitly specify kernels (instead of autodetect + hide not needed) would be great. In my devcontainer I need to exclude 5 choices out of 6…

Hi everyone, I’m sorry about this issue, I will review the current workflow next iteration. Perhaps what the Jupyter extension can do is,

• Not hide items that have paths such as /bin/bash or the like
• And if we do hide items (which have ambiguous paths such as python), then we display a more informative message in the logs and how it can be resolved.

EDIT: Per discussion below it’s language set to be different from python (like “python-custom”) that does the trick

@iakovn Sorry one final question, from this do I correctly understand that python-custom" is just an arbitrary string choice and the only thing that matters for the kernel to get picked up is that "language" is not "python"? That is, "python-custom" isn’t something from IPython, correct?

That’s my understanding, yes. Seems that the plugin has some undocumented heuristics for “python” aiming to automatically handle venvs.

@DonJayamanne would be great if you explain the logic either here or in the docs.

changing “language” to “python-custom” should solve the problem for you as far as I understand.

Wow, indeed this works! Amazing and thank you!

Using /bin/bash as the first element in argv

I had previously run chmod +x on my startup script, but you’re right that explicitly putting this in the kernel is probably a better approach to be safe.

$ cat ~/.local/share/jupyter/kernels/lcg-example/kernel.json
{
 "argv": [
  "/bin/bash",
  "/home/feickert/.local/share/jupyter/kernels/lcg-example/startup.sh",
  "-f",
  "{connection_file}"
 ],
 "display_name": "Example LCG view + venv kernel",
 "language": "python-custom",
 "metadata": {
  "debugger": true
 }
}

Thanks for your help @iakovn.

@DonJayamanne Are there any official docs on using python-custom?

Using /bin/bash as the first element in argv and changing “language” to “python-custom” should solve the problem for you as far as I understand.

It feels weird that the field affects how kernel is getting detected, don’t you think?

Thanks, I agree. However we found that most users end up with just python in that path or pyspark or the like, and when using VS Code, such commands are not in the path (env variable) of VS Code, as a result these kernels cannot be launched from within VS Code.

Outside vscode users open terminals and have conda activated or the like, hence those executables are in the current path But go into vscode they fail, and thats why we hide them.

I understand that this doesn’t work for you, i just wanted you to understand why we decided to exclude them (to avoide users frolm running into issues where the kernel might not start)

I’ll discuss this with the team