eleventy: `addPreprocessor` doesn't work together with keyed `addExtension`
Operating system
macOS 14
Eleventy
3.0.0-alpha.19
Describe the bug
I’m trying to use addPreprocessor to transform the input file contents before handing it off to a template renderer.
This works for simple cases, but I’ve hit a wall when trying to use it with keyed addExtension (for aliasing an existing template language). Specifically, I’m trying to use key: "11ty.js" so that I can access this.defaultRenderer inside the compile function. What happens is 11ty tries to process this template too early, before it has been transformed by the logic in addPreprocessor, which means it fails when it encounters unsupported syntax.
[11ty] Problem writing Eleventy templates:
[11ty] Unexpected token (3:4) (via SyntaxError)
Reproduction steps
See stackblitz.
Note: I’m only using "js" to illustrate the problem more clearly. Using "jsx" or "tsx" also fails, but with a different “Unknown file extension” error (possibly related to https://github.com/11ty/eleventy/issues/3393#issuecomment-2276921293).
Note 2: A similar problem also happens in Deno (which natively supports JSX/TSX), except there is no error; the content in addPreprocessor is just empty. See minimal repro.
Expected behavior
addPreprocessor should execute before setting up a template alias (i.e. using addExtension with key). Both features should be usable together.
Reproduction URL
https://stackblitz.com/edit/github-qaud3e?file=eleventyJsxPlugin.js
Screenshots
No response
About this issue
- Original URL
- State: closed
- Created 2 years ago
- Comments: 15 (15 by maintainers)
Commits related to this issue
- Fixes #3433. — committed to 11ty/eleventy by zachleat 2 years ago
- Fix to allow #3433 to use custom compile functions with preprocessors — committed to 11ty/eleventy by zachleat 2 years ago
- Fix for virtual templates #3433 — committed to 11ty/eleventy by zachleat 2 years ago
Shipping with 3.0.0-alpha.21 and 3.0.0-beta.2
It is an off-the-books API but everyone uses it. I originally found it via Svelte’s repo I think? Pour one out for https://github.com/nodejs/node/issues/40098
Oh wow, TIL about
Module#_compile. This looks like an undocumented/private API, which does make me nervous but otherwise it’s pretty much exactly what I need. Between this andregister, I think I have a clearer path forward. Thank you!Regardless, I’ll have a deeper think about the additional changes made in this PR and slate it for 3.0 for now: https://github.com/11ty/eleventy/pull/3452 I do think we made forward progress there—thank you!
Hmm, we’re iterating towards something 😄
Let’s start with the Node behavior first:
I think there is some confusion about the
11ty.jsformat—it doesn’t accept strings nor execute string input. Adding a virtual template as an 11ty.js template is just passing in JavaScript. So I wouldn’t expect that thecontentpassed toaddPreprocessorto be a string (nor aliases viaaddExtension)—I would expect it to be JavaScript at that point.When you remove
key: '11ty.js'from your alias, it treats the template as plaintext and thenaddPreprocessorwill pass in a string.Worse, I wouldn’t expect returning a string from
addPreprocessorto execute a string of JavaScript either. We have a similar limitation executing arbitrary JS in the Render plugin https://www.11ty.dev/docs/plugins/render/#rendertemplateI think I would classify that as a bigger enhancement request (not a bug right now).
All that said, we do use the node-retrieve-globals package to implement the
jsfront matter type to execute arbitrary JavaScript https://github.com/11ty/eleventy/issues/2819 so there is a limited precedent for a path forward here.Finally, I would call out the method used by
tsxhttps://github.com/privatenumber/tsx (likely some magic using module.register https://nodejs.org/docs/latest/api/module.html#moduleregisterspecifier-parenturl-options but I haven’t dove all the way in to see) that might be a more ergonomic way to intercept and preprocess JavaScript. This is what we recommend on https://www.11ty.dev/docs/languages/typescript/Open to suggestions here (of course).