vscode: Global regex search with "Not matching character" doesn't match newline

Issue Type: Bug

I was trying to search for

Promise\.all\(\[[^\]]*await[^\]]*\]\)

but I had no way to turn on multiline support so that ^\] would match newlines too. So searching for below would not be found.

return await Promise.all([
  await this.someFunction1(),
  await this.someFunction2(),
]);

I later put in the below and was happy to find that it suddely worked, but knew that it would only find if await was on the first line after the bracket, and also if there was no chars other than newline space or tab.

Promise\.all\(\[[^\]]*[\n\r \t]await[^\]]*\]\)

But, I then started to find others that actually SHOULDN’T have worked… It was at this point, I realsed that ^\] was working as expected the first time, and was matching on newline chars… (Perfect, I removed my tab/space hack)… To find, it stopped working again…

It seems I need atleast 1 \n in the regex to make it work.

My quick fix was to add this at the very end as such

Promise\.all\(\[[^\]]*await[^\]]*\]\)\n{0}

But this is obviously a dirty hack and not an obvious one, be far better to test for Not Matching and turn multiline back on, similar to what ever test you are doing for searching for \n and turning on multiline

VS Code version: Code 1.35.0 (553cfb2c2205db5f15f3ee8395bbd5cf066d357d, 2019-06-04T01:17:12.481Z) OS version: Windows_NT x64 10.0.10240

System Info
Item Value
CPUs Intel® Xeon® CPU E5-1620 v3 @ 3.50GHz (8 x 3492)
GPU Status 2d_canvas: enabled
checker_imaging: disabled_off
flash_3d: enabled
flash_stage3d: enabled
flash_stage3d_baseline: enabled
gpu_compositing: enabled
multiple_raster_threads: enabled_on
native_gpu_memory_buffers: disabled_software
rasterization: enabled
surface_synchronization: enabled_on
video_decode: enabled
webgl: enabled
webgl2: enabled
Load (avg) undefined
Memory (System) 15.92GB (5.33GB free)
Process Argv
Screen Reader no
VM 0%
Extensions (19)
Extension Author (truncated) Version
markdown-preview-github-styles bie 0.1.6
npm-intellisense chr 1.3.0
ssh chr 0.0.4
vue-peek dar 1.0.2
vscode-eslint dba 1.9.0
vscode-ts-auto-return-type ebr 1.0.1
tslint eg2 1.0.43
RunOnSave eme 0.0.18
prettier-vscode esb 1.9.0
todo-tree Gru 0.0.134
node-module-intellisense lei 1.5.0
camelcasenavigation map 1.0.1
rainbow-csv mec 1.1.1
node-modules-resolve nau 1.0.2
uuid-generator net 0.0.4
vetur oct 0.21.0
gitconfig sid 2.0.0
open-in-browser tec 2.0.0
sort-lines Tyr 1.8.0

About this issue

  • Original URL
  • State: open
  • Created 5 years ago
  • Reactions: 9
  • Comments: 23 (4 by maintainers)

Most upvoted comments

I just use \n{0} a the end of all my searches… been working fine so far…

This heuristic of using \n is my alternative to actually having a button or checkbox to enable multiline mode.

IMO it’s much better to have slower search than to potentially cause bugs (among other things), e.g. when someone refactors code and the search doesn’t match all it should, in which case you better hope your CI catches it.

Nothing worse than a regex engine that doesn’t comply to spec in such a glaring way as redefining \s to exclude \n, which is pretty unintuitive. I’d at least show a warning message when the regex contains \s.

Also, having an option to opt-in for \s to include \n seems reasonable to me.

All I can do is what I think is right for the majority of users and hope that the rest can google their way to a solution. Because of the perf impact and possibility for unexpected behavior I think multiline search should always be an explicit opt in. My guess is that we would add a button for this some day but it will take careful UX thinking because we have too many options crammed into a small space already.

I explicitly asked it to match any char that was not a ] To the best of my knowledge \n and \r are not ] so therefore should have been matched. There is no visible indication anywhere that multiline is turned on or off. It is just some magic that goes on behind the scenes. Only way to get consistent results would be to add \n{0} to every regex I do.

You may feel that \s should match only spaces and tabs but can you guarantee that every single user of VSCode would (1) know that that is indeed what is going on, and (2) know how to bypass the limitation if they needed to?

It seems like you are just saying that “if a regex can match \n then the multiline flag should be turned on”. Isn’t this the same as having the multiline flag turned on all the time?

Anyway to me the whole point is that the majority of regexes are not intended to match multiple lines, it’s quite common to include \s when you really are looking for spaces and tabs within a line. And including \s should not blow up your search results by matching a bunch of \r and \n when you did not explicitly ask for it.

This heuristic of using \n is my alternative to actually having a button or checkbox to enable multiline mode.

Then 👍 Big thumbs up that [^s] should indeed turn that mode on… Ok… I am up to date and agreeing. Good job everyone… take a 5 minute break and enjoy your evening.

Wanted to create a new issue that \s does not match newlines, but found this.

@roblourens Yeah, currently multiline mode is only turned on when a literal newline is in the regex, for perf.

How about better newline-regex detection? Something like this:

const { assert } = require("chai");

// Just came up with this, thus requires more tests. Doesn't match some edge cases.
// Requires a variable-length negative-lookbehind support, which is supported by latest ES
const IS_MULTILINE = /(?<!(?<!\\)\[[^\]]*)(?<!\\)\\[ns]|(?<!\\)\[\^(.(?<!(?<!\\)\\[ns]))*(?<!\\)\]|(?<!\\)\[(?!\^)[^\]]*(?<!\\)\\[ns]/;

assert.notOk(IS_MULTILINE.test(/[^\n]/));
assert.notOk(IS_MULTILINE.test(/[^\s]/));
assert.notOk(IS_MULTILINE.test(/\[^a]/));
assert.notOk(IS_MULTILINE.test(/[\\s]/));

assert.ok(IS_MULTILINE.test(/[^\\s]/));
assert.ok(IS_MULTILINE.test(/[^\\n]/));
assert.ok(IS_MULTILINE.test(/[^s]/));
assert.ok(IS_MULTILINE.test(/[\s]/));
assert.ok(IS_MULTILINE.test(/\[\s/));
assert.ok(IS_MULTILINE.test(/[^\s]\s\]/));
assert.ok(IS_MULTILINE.test(/\[\s\]/));