markdown: Compile error with fresh MiKTeX installation

Hello, I’ve recently ran into a problem that causes markdown to fail with the following error message:

...a/Local/Programs/MiKTeX/tex/luatex/markdown/markdown.lua:3237: bad argument #1 to 'open' (string expected, got nil)
stack traceback:
	[C]: in function 'io.open'
	...a/Local/Programs/MiKTeX/tex/luatex/markdown/markdown.lua:3237: in function <...a/Local/Programs/MiKTeX/tex/luatex/markdown/markdown.lua:3235>
	...a/Local/Programs/MiKTeX/tex/luatex/markdown/markdown.lua:3235: in main chunk
	[C]: in function 'require'
	[\directlua]:1: in main chunk.
\lua_now:e #1->\__lua_now:n {#1}
                                
l.8 \end{markdown}
                
The lua interpreter ran into a problem, so the
remainder of this lua chunk will be ignored.

Here are my steps to reproduce:

  1. Create a fresh Windows 11 VM

  2. Install the most recent version of MiKTeX

  3. Create file document.tex

    \documentclass{article}
    \usepackage{markdown}
    \begin{document}
    Hello World!
    
    \begin{markdown}
    _Hello,_ **Markdown**!
    \end{markdown}
    \end{document}
    
  4. Run lualatex --interaction=nonstopmode --shell-escape --synctex=1 document.tex

  5. The above error occurs.

In the resulting PDF, only the “Hello World” part is there, the “Hello Markdown” part is missing. I have attached the full log file: document.log

Since everything is a fresh install, all packages are up-to-date.

Interestingly, on a different system where I haven’t updated my MiKTeX packages in a while, I still have markdown version 2.23.0-0-g0b22f91 installed, and everything works fine there.

Please let me know if I can help you with any additional information.

About this issue

  • Original URL
  • State: closed
  • Created 2 years ago
  • Comments: 29 (20 by maintainers)

Most upvoted comments

Thanks for your analysis. I hoped that https://github.com/Witiko/markdown/pull/482 would also help here besides improving the speed.

So thank you very much, I think this highly intriguing issue is solved with your PR. 😃

Please note that https://github.com/Witiko/markdown/pull/482 is not yet finished, see the failing CI. For some reason, the parser does not match some punctuation symbols such as “€”, likely due to some obscure typo in my depth-first search algorithm.

Well, apart from https://github.com/MiKTeX/miktex/issues/1515 that is. But for that issue, at least there is a clear and easy workaround.

As also discussed in https://github.com/Witiko/markdown/pull/482, I would like to remove the dependency on UnicodeData.txt and precompile the parser into a Lua file markdown-unicode-data.lua that would be distributed together with the Markdown package. That way, we should be fully compatible with MikTeX.

Independently, I’d like to try and reproduce the issue with the MikTeX docker image for Linux to see if the issue only manifests on Windows.

Could there be something wrong with this file or how it is read under Windows?

I confirmed that the issue is with MikTeX under Windows:

$ docker run -it -e MIKTEX_UID=`id -u` miktex/miktex:basic
miktex@9da1b10ba72b:/miktex/work$ curl https://mirrors.ctan.org/macros/generic/unicode-data/UnicodeData.txt
<!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN">
<html><head>
<title>307 Temporary Redirect</title>
</head><body>
<h1>Temporary Redirect</h1>
<p>The document has moved <a href="https://mirrors.nic.cz/tex-archive/macros/generic/unicode-data/UnicodeData.txt">here</a>.</p>
</body></html>
miktex@9da1b10ba72b:/miktex/work$ curl https://mirrors.nic.cz/tex-archive/macros/generic/unicode-data/UnicodeData.txt > UnicodeData.txt
miktex@9da1b10ba72b:/miktex/work$ cat > example.md <<EOF
_Hello,_ **Markdown**!
EOF
miktex@9da1b10ba72b:/miktex/work$ texlua `kpsewhich markdown-cli.lua` example.md
\markdownRendererDocumentBegin
\markdownRendererEmphasis{Hello,} \markdownRendererStrongEmphasis{Markdown}!\markdownRendererDocumentEnd

I will now look into whether the issue is also present with TeX Live under Windows.

I dug a bit deeper into this. I disabled various parsers (by setting them to P(false)) and worked my way upwards the whole parser chain to see what causes the crash. And I think I found it.

image

The crash occurs in the check_unicode_type function. Specifically, in the line that reads if lpeg.match(parsers.punctuation[char_length], s, i+pos) then.

As you can see in the console output, line 4982 got hit, but neither did 4987 nor 4990.

Earlier in the code, the parsers.punctuation table is loaded from the UnicodeData.txt file. Could there be something wrong with this file or how it is read under Windows? I downloaded the file from https://ctan.joethei.xyz/macros/generic/unicode-data/UnicodeData.txt and double-checked that it is intact.

With the [plain] option, I got the same result:

image

I then ran the markdown-cli.lua script with the example.md file. Afer a few seconds, the script exited with code -1073741571 and produced no output.

2024-07-31 10_32_36-LaTeX Test VM 3 (Installed MiKTeX)  wird ausgeführt  - Oracle VM VirtualBox

I have tested it again on a fresh Windows 11 VM, with the same steps as in my initial comment, but with the additional UnicodeData.txt file. Unfortunately I ran into the same problem you described yesterday. I took this screenshot right after the process crashed.

2024-07-31 08_33_32-LaTeX Test VM 3 (Clean)  wird ausgeführt  - Oracle VM VirtualBox

I have attached the log file: document.log

I then tested it again on anther VM with twice the assigned memory (16 GB total), but with the same result. I’m fairly confident in saying that this is not an insufficient memory issue.

Please let me know if I can test additional things for you, I’m happy to help you investigate this problem.