vscode-python: Inside of a notebook, magic cells should not cause an error in pylance

I wanted to try and make the case that this issue should be treated more like a bug.

I see how this can be considered an enhancement (it is a new scenario which was not considered initially), but for a jupyter notebook beginner like myself, a red squiggle in a notebook indicates that I did something wrong, and a squiggle on the first character of a notebook indicates that perhaps I have a setup issue. Despite the error, the cell actually executed without a problem. IMHO, in general, it is problematic if a tool reports a squiggle and I have no possibility to get rid of it; this erodes my trust in the tool. This can also be frustrating for users who want to have a clean Problems panel.

Happy to read that this issue is being addressed - the wiggles are really annoying. As far as I understand, ipykernel adds three syntax additions that are relevant and should be addressed here.

1. line magics, eg. %pip. Those are by definition just single lines and can probably be handled by simply replacing % against # when the % is at the beginning of the line. I guess this would be a good start already, although this approach is not 100% exact yet – depending on the options available here, it could be improved by testing if the %… token is part of a literal and/or by testing if the name of the magic is known. However, in case of checking against magics names, I would appreciate an algorithm which can cope with arbitrary magics and would not only test for some well-known magics names.

2. cell magics., eg. %%writefile. Cell magics pass the entire content of the cell to the corresponding magic. The content can be any language. My suggestion here would be to just ignore the entire cell if it starts with %%, and if it is parsed at all, it should be parsed by the language server that belongs to the language of the magic but not by default by a Python language server.

3. System shell access, eg. !ls. In principle the same as line magics, with the only difference that the content of the line is executed as a shell command if it starts with a !.

Sometimes, ipykernel also expands variables, ie. it replaces variable tokens like $my_var with corresponding variable values defined outside of the cell. However, Python cells seem not to do this variable expansion, hence can likely be neglected in this case.

If this issue can be fixed, it would be awesome to include awareness of imports and functions in %run [bla.ipynb/.py]. I use shared notebooks with data loading helpers and imports for many types of analysis, and having half your cells get highlighted as undefined is no bueno.