vscode-jupyter: Jupyter cannot connect to a remote server using a custom ServerPicker
Applies To
- Notebooks (.ipynb files)
- Interactive Window and/or Cell Scripts (.py files with #%% markers)
What happened?
Hi there!
I created a custom VSCode extension in order to interact with my company’s remote JupyterHub clusters, based on the examples provided on your repository here. My extension defines, among other things, a custom Jupyter Server Picker which is then registered with the Jupyter extension.
The custom server picker is registered and functions properly since it is properly displayed when clicking the Select kernel button after activating my extension. The Server picker also seems to return proper IJupyterServerUri objects, which is confirmed by logs:
authorizationHeader: {Authorization: 'Bearer MYTOKEN'}
baseUrl: 'https://jh-cluster.mycompany.com/'
expiration: undefined
token: 'MYTOKEN'
However, the actual resulting connexion fails immediately with the following traceback (true sensitive values are edited out in all caps):
~/.myenv/bin/python ~/.vscode-oss/extensions/ms-toolsai.jupyter-2023.3.1000641008/pythonFiles/vscode_datascience_helpers/getServerInfo.py
info 15:55:44.004: Connecting local server kernel http://localhost:8888/
info 15:55:44.005: Creating server with url : http://localhost:8888/
info 15:55:44.006: Server started.
info 15:55:59.363: Creating server with url : https://jh-cluster.mycompany.com/
warn 15:55:59.409: DataScience Error [eo [Error]: Failed to connect to the remote Jupyter Server 'https://remote/'. Verify the server is running and reachable. (Not Found).
at lo.setJupyterURIToRemote (/Users/maximet/.vscode-oss/extensions/ms-toolsai.jupyter-2023.3.1000641008/out/extension.node.js:2:1679005)
at async /Users/maximet/.vscode-oss/extensions/ms-toolsai.jupyter-2023.3.1000641008/out/extension.node.js:24:216365] {
category: 'remotejupyterserverconnection',
url: 'https://remote/?id=NAME OF MY REMOTE SERVER PICKER&uriHandle=live',
serverId: 'HEX VALUE',
originalError: [t [Error]: Not Found
at t.create (/Users/maximet/.vscode-oss/extensions/ms-toolsai.jupyter-2023.3.1000641008/out/extension.node.js:2:68133)
at process.processTicksAndRejections (node:internal/process/task_queues:96:5)
at async Object.listRunning (/Users/maximet/.vscode-oss/extensions/ms-toolsai.jupyter-2023.3.1000641008/out/extension.node.js:2:55998)
at async mg.getRunningKernels (/Users/maximet/.vscode-oss/extensions/ms-toolsai.jupyter-2023.3.1000641008/out/extension.node.js:17:237122)
at async Promise.all (index 0)
at async Ti.validateRemoteConnection (/Users/maximet/.vscode-oss/extensions/ms-toolsai.jupyter-2023.3.1000641008/out/extension.node.js:2:1664096)
at async lo.setJupyterURIToRemote (/Users/maximet/.vscode-oss/extensions/ms-toolsai.jupyter-2023.3.1000641008/out/extension.node.js:2:1678698)
at async /Users/maximet/.vscode-oss/extensions/ms-toolsai.jupyter-2023.3.1000641008/out/extension.node.js:24:216365] {
response: L [Response] {
size: 0,
timeout: 0,
[Symbol(Body internals)]: [Object],
[Symbol(Response internals)]: [Object]
},
traceback: ''
},
baseUrl: 'https://remote/'
}
error 15:55:59.415: SessionManager:getKernelSpecs failure: [Error: Poll (@jupyterlab/services:KernelSpecManager#specs) is disposed.
at e.dispose (/Users/maximet/.vscode-oss/extensions/ms-toolsai.jupyter-2023.3.1000641008/out/extension.node.js:2:171555)
at u.dispose (/Users/maximet/.vscode-oss/extensions/ms-toolsai.jupyter-2023.3.1000641008/out/extension.node.js:2:62984)
at mg.dispose (/Users/maximet/.vscode-oss/extensions/ms-toolsai.jupyter-2023.3.1000641008/out/extension.node.js:17:236236)
at process.processTicksAndRejections (node:internal/process/task_queues:96:5)]
Which seems to indicate that the url provided by the server picker is not properly read or parsed.
Furthermore, manually registering the remote cluster using the https://jh-cluster.mycompany.com/token?=MYTOKEN functions perfectly well.
Any idea what is going on? Thanks!
VS Code Version
1.76.2
Jupyter Extension Version
v2023.3.1000641008
Jupyter logs
~/.myenv/bin/python ~/.vscode-oss/extensions/ms-toolsai.jupyter-2023.3.1000641008/pythonFiles/vscode_datascience_helpers/getServerInfo.py
info 15:55:44.004: Connecting local server kernel http://localhost:8888/
info 15:55:44.005: Creating server with url : http://localhost:8888/
info 15:55:44.006: Server started.
info 15:55:59.363: Creating server with url : https://jh-cluster.mycompany.com/
warn 15:55:59.409: DataScience Error [eo [Error]: Failed to connect to the remote Jupyter Server 'https://remote/'. Verify the server is running and reachable. (Not Found).
at lo.setJupyterURIToRemote (/Users/maximet/.vscode-oss/extensions/ms-toolsai.jupyter-2023.3.1000641008/out/extension.node.js:2:1679005)
at async /Users/maximet/.vscode-oss/extensions/ms-toolsai.jupyter-2023.3.1000641008/out/extension.node.js:24:216365] {
category: 'remotejupyterserverconnection',
url: 'https://remote/?id=NAME OF MY REMOTE SERVER PICKER&uriHandle=live',
serverId: 'HEX VALUE',
originalError: [t [Error]: Not Found
at t.create (/Users/maximet/.vscode-oss/extensions/ms-toolsai.jupyter-2023.3.1000641008/out/extension.node.js:2:68133)
at process.processTicksAndRejections (node:internal/process/task_queues:96:5)
at async Object.listRunning (/Users/maximet/.vscode-oss/extensions/ms-toolsai.jupyter-2023.3.1000641008/out/extension.node.js:2:55998)
at async mg.getRunningKernels (/Users/maximet/.vscode-oss/extensions/ms-toolsai.jupyter-2023.3.1000641008/out/extension.node.js:17:237122)
at async Promise.all (index 0)
at async Ti.validateRemoteConnection (/Users/maximet/.vscode-oss/extensions/ms-toolsai.jupyter-2023.3.1000641008/out/extension.node.js:2:1664096)
at async lo.setJupyterURIToRemote (/Users/maximet/.vscode-oss/extensions/ms-toolsai.jupyter-2023.3.1000641008/out/extension.node.js:2:1678698)
at async /Users/maximet/.vscode-oss/extensions/ms-toolsai.jupyter-2023.3.1000641008/out/extension.node.js:24:216365] {
response: L [Response] {
size: 0,
timeout: 0,
[Symbol(Body internals)]: [Object],
[Symbol(Response internals)]: [Object]
},
traceback: ''
},
baseUrl: 'https://remote/'
}
error 15:55:59.415: SessionManager:getKernelSpecs failure: [Error: Poll (@jupyterlab/services:KernelSpecManager#specs) is disposed.
at e.dispose (/Users/maximet/.vscode-oss/extensions/ms-toolsai.jupyter-2023.3.1000641008/out/extension.node.js:2:171555)
at u.dispose (/Users/maximet/.vscode-oss/extensions/ms-toolsai.jupyter-2023.3.1000641008/out/extension.node.js:2:62984)
at mg.dispose (/Users/maximet/.vscode-oss/extensions/ms-toolsai.jupyter-2023.3.1000641008/out/extension.node.js:17:236236)
at process.processTicksAndRejections (node:internal/process/task_queues:96:5)]
### Coding Language and Runtime Version
Python v3.8
### Language Extension Version (if applicable)
_No response_
### Anaconda Version (if applicable)
_No response_
### Running Jupyter locally or remotely?
N/A or Not sureAbout this issue
- Original URL
- State: closed
- Created a year ago
- Comments: 24 (14 by maintainers)
Just and update for everyone here, we’re working on stabilizing the API in the current iteration https://github.com/microsoft/vscode-jupyter/issues/13894 This iteration we will spend time reviewing the API
I will post the sample later once we have a stable API, this prevents issues with users using an old API and then having to switch again to another API. I’m sorry for the delay in getting this addressed and thank you for your patience
This should be addressed in the latest pre-release version, please let me know how that goes
A new API has been published and this new API will be available for use in the next version of the Jupyter exetnsion (to be published sometime this week).
Here’s a sample https://github.com/microsoft/vscode-extension-samples/tree/main/jupyter-server-provider-sample The JupyterHub extension also makes use of this same API https://github.com/microsoft/vscode-jupyter-hub
@brandonjbjelland are you still running into this issue with the current API. FYI, #13894 should end up getting completed next month, i.e. the API should be ready for use then. YOu are welcome to use it now and provide feedback, though I haven’t got a sample for anyone yet.
I’m sorry you are running into this I have a sample extension that I created to use this api, will publish that so you can use that for reference purposes Leaving this issue open for now
Found a few issues and have addressed them, however it will take a while to ship those fixes, will keep you posted. Here’s a PR with all of the changes and fixes https://github.com/microsoft/vscode-jupyter/pull/13588