vscode: Debug console: '\n' not shown as new line for evaluation returning a variablesReference

From a practical point of view, it’s vary hard to inspect the python data science structures from numpy, xarray, and pandas in VSCode at all at the moment. The structures are too complex for the Debuger’s variable inspector to really be that productive to use.

Python encourages for these things to produce formatted strings to be more human readable, which these libraries do. So even if it’s decided that it’s best that the debug console not allow this, there should be some way to get to this representation in a VSCode debug session. Have even tried to print(my_array) and switching to the terminal tab to try and get some clue as to the shape of these data structures.

This feature: https://github.com/microsoft/vscode-python/issues/5113 may end up solving this, but I don’t believe it is currently possible to use this in debug or would be available that soon.

Mainly feel that leaving it to the extension to decide, as mentioned above, so it can be done on a per-language basis would be great. And I think it does in python.

Also, it is a pain point in using VSCode with these data science libraries and debugging data structures, when all you are looking for is if a matrix has values in the diagonal. So if the opposition to this is somewhat notional, I think the user experience here may trump that.

Cheers.

Thank you very much for your feedback.

After discussing with @weinand we came to the following conclusion:

The VS Code frontend should not escape newlines. It is up to the debug extension to decide how the data should be presented and the debug extension can decide if a newline shuold be escaped or not. VS Code is agnostic with regards to the data it is showing, and the extensions are already adding quotes to strings and other to make the data more presentable.

The downside of this approach is that in the Variables view the multine values will from now on only show the first line of the value. However it is up to the debug extensions to change their responses to escape new lines. Due to that we plan to make this change start of next milestone so extensions are given enough time to adopt. Thus reopening.

We do not want to add a setting for this behavior as it does not feel right for the user to control this and adding a capability also requires adoption by extensions (a similar work as our escaping approach requires).

@weinand please add more details or correct if I captured something wrongly from our discussion fyi @connor4312 @roblourens

As a note, in the Python extension the user could already see inside as a structured object as you’re doing in node (so, there’s nothing to actually be fixed in the Python extension, that behavior already existed), and if it wasn’t a structured object then it’d already appear in multiple lines.

I think it’s more about expectations… when using an interactive console in Python users are used to see things printed (so, in the terminal using Python interactively if you wanted to see the \r\n\t of the numpy array you’d do a repr(arr) instead of just arr as just arr would show the new lines expanded)…

Now, this change empowers extensions to do as they please (if the node plugin wants, they can show the actual \n char and not put a new line there by doing a replace('\n', '\\n') if an evaluate request was done for the “repl” context for a structured object), whereas without this patch the behavior can’t be customized by the extension.

I personally prefer the current version instead of having to click through to see things in the terminal, but I also agree it’s pretty much a matter of personal taste.

I agree that Copy Value should have its own context. A year ago, Copy Value still used null as its context. Although it is not documented, at least it works. I have also requested that it be documented #69768.

As announced in my previous comment we have changed that VS Code debugger is no longer escaping whitespaces. Now it is up to the debugger to handle this on their side. I pushed this early in the milestone so we catch potential corner cases where this new strategy does not work and to give enough time for debuggers to adopt this.

fyi @roblourens @connor4312

What seems to be happening is that in the case where an expression is evaluated and returns a variablesReference, the variablesReference is shown the same way it’s shown in the variables view (where new lines are changed for a \n representation).

Some more info from the debugger backend:

Given the following Python code (not using numpy to make it simpler):

class Foo(object):

    def __repr__(self):
        return 'aa\nbb\ncc'
    __str__ = __repr__

foo = Foo()
print('add breakpoint here')

Pausing at the breakpoint and sending foo to the repl (by typing foo in the Debug Console) the request sent to the debugger is:

{
    "arguments": {
        "context": "repl",
        "expression": "foo",
        "format": {},
        "frameId": 22
    },
    "command": "evaluate",
    "seq": 1000000016,
    "type": "request"
}

and the response is:

{
    "type": "response", 
    "request_seq": 1000000016, 
    "success": true, 
    "command": "evaluate", 
    "body": {
        "result": "aa\nbb\ncc", 
        "variablesReference": 4, 
        "type": "Foo", 
        "presentationHint": {}
    }, 
    "seq": 42
}

So, you can see that the result has proper new lines and VSCode chose to show it in a single line and not expanded in the debug console (so, there’s not much to be done on the extension as it seems to be doing the proper thing).

So, this would be a request (and not a bug since it seems it’s deliberately doing the same thing done in the variables view) for VSCode to show new lines expanded in the debug console for a given variablesReference.

As a note, if print(foo) is done, the result is shown expanded – it’ll actually return a variablesReference to None in that case as that’s the return of print(foo) but then the representation will be printed as a string afterwards with proper new lines.