tlaplus: SANY can't handle backticks in strings
Description
If SANY encounters a backtick ` inside quotes it will just read until the end of the file. Trying to escape the backtick with a backslash results in the error “Lexical error at line 37, column 87. Encountered: “`” (96), after : “\””
Expected Behavior
I expected SANY to not treat backticks as any sort of special character, or for them to be escaped using a backslash.
Actual Behavior
SANY produces the error “Lexical {error: EOF reached, possibly open comment starting around line 37”
Steps to Reproduce
Parse this with SANY:
---- MODULE Test ----
str == "foo ` bar"
====
Then parse this with SANY:
---- MODULE Test ----
str == "foo \` bar"
====
Steps Taken to Fix
N/A
Possible Fix
N/A
Your Environment
- TLC version: 3c7caa5 (Theano release)
- Operating System: Linux
About this issue
- Original URL
- State: open
- Created 4 years ago
- Comments: 16 (6 by maintainers)
Commits related to this issue
- Support unbalanced backticks in string literals (fixes #802) Signed-off-by: Max Moiseev <max@maxmoiseev.com> — committed to moiseev/tlaplus by moiseev 2 years ago
- Support unbalanced backticks in string literals (fixes #802) Signed-off-by: Max Moiseev <max@maxmoiseev.com> — committed to moiseev/tlaplus by moiseev 2 years ago
FWIW I am happy to modify the PR in either direction (add more tests or roll back the sexond commit). Will await The Decision.
The more I think about it, the more I think we shouldn’t support this feature. It only leads to confusion. Consider e.g.
which parses as a single operator
Tickequal to the textTo the extent that’s true, I am definitively against the weird backtick-apostrophe syntax. Pending review by the language committee I think we should accept @moiseev’s original fix (9e27080).
Woah!!! Good spot Calvin. That’s quite an interesting feature actually. Who knew TLA+ had multi line strings?
My first choice would be to merge #1190 and be done with this mess, especially since the official grammer doesn’t mention backticks.
…but there is actually a notable change here that I don’t think we’ve discussed: within a matched backtick-apostrophe block, SANY allows all characters, including quotes, newlines, etc.
That behavior makes me feel like the backticks are meant for a purpose.
Thanks for figuring out the TLATeX connection. After some testing I see what you mean. This feature was clearly intended to map backtick ` to
‘and apostrophe'to’(UK English-style inverted commas) in the generated LaTeX files. However, thetla2tex.TLAcommand handles the case of an unbalanced backtick quite robustly (I tested this using lots of random backticks & apostrophes), recovering and resuming parsing the file. So it’s strange that SANY errors out on it when nothing else in the language treats backticks and apostrophes as anything other than just another ASCII character.I think the desired behavior is to just allow arbitrary numbers of backticks and apostrophes inside of strings without requiring balancing.
An alternative solution is to keep the same behavior but improve error reporting, telling users that they need to balance backticks with apostrophes inside their string. I am uncertain about the value of this feature other than as a sort of linter check before translating the TLA+ specs to TeX (the translator will happily handle syntactically invalid TLA+, it seems to assume the specs will have been parsed with SANY beforehand).
My inclination is to choose the first option given that it doesn’t seem possible to crash the TeX translator by giving it any combination of unbalanced backticks and apostrophes. @lemmy and @calvin do you have any thoughts?
I decided to give this issue a shot. At the very least I have found why this is the problem and how to work around it (if one really really needs back-ticks inside string literals, make sure you pair the “opening” backtick with a “closing” single quote
"like ==> `this' <== ").Now the question really becomes: what is the desired behavior? Should unpaired backticks be supported at all? If not then it could certainly issue a better diagnostics instead of just failing to parse the file.