eleventy: `addPreprocessor` doesn't work together with keyed `addExtension`

Operating system

macOS 14

Eleventy

3.0.0-alpha.19

Describe the bug

I’m trying to use addPreprocessor to transform the input file contents before handing it off to a template renderer.

This works for simple cases, but I’ve hit a wall when trying to use it with keyed addExtension (for aliasing an existing template language). Specifically, I’m trying to use key: "11ty.js" so that I can access this.defaultRenderer inside the compile function. What happens is 11ty tries to process this template too early, before it has been transformed by the logic in addPreprocessor, which means it fails when it encounters unsupported syntax.

[11ty] Problem writing Eleventy templates:
[11ty] Unexpected token (3:4) (via SyntaxError)

Reproduction steps

See stackblitz.

Note: I’m only using "js" to illustrate the problem more clearly. Using "jsx" or "tsx" also fails, but with a different “Unknown file extension” error (possibly related to https://github.com/11ty/eleventy/issues/3393#issuecomment-2276921293).

Note 2: A similar problem also happens in Deno (which natively supports JSX/TSX), except there is no error; the content in addPreprocessor is just empty. See minimal repro.

Expected behavior

addPreprocessor should execute before setting up a template alias (i.e. using addExtension with key). Both features should be usable together.

Reproduction URL

https://stackblitz.com/edit/github-qaud3e?file=eleventyJsxPlugin.js

Screenshots

No response

About this issue

  • Original URL
  • State: closed
  • Created 2 years ago
  • Comments: 15 (15 by maintainers)

Commits related to this issue

Most upvoted comments

Shipping with 3.0.0-alpha.21 and 3.0.0-beta.2

It is an off-the-books API but everyone uses it. I originally found it via Svelte’s repo I think? Pour one out for https://github.com/nodejs/node/issues/40098

Oh wow, TIL about Module#_compile. This looks like an undocumented/private API, which does make me nervous but otherwise it’s pretty much exactly what I need. Between this and register, I think I have a clearer path forward. Thank you!

Regardless, I’ll have a deeper think about the additional changes made in this PR and slate it for 3.0 for now: https://github.com/11ty/eleventy/pull/3452 I do think we made forward progress there—thank you!

Hmm, we’re iterating towards something 😄

Let’s start with the Node behavior first:

I think there is some confusion about the 11ty.js format—it doesn’t accept strings nor execute string input. Adding a virtual template as an 11ty.js template is just passing in JavaScript. So I wouldn’t expect that the content passed to addPreprocessor to be a string (nor aliases via addExtension)—I would expect it to be JavaScript at that point.

When you remove key: '11ty.js' from your alias, it treats the template as plaintext and then addPreprocessor will pass in a string.

Worse, I wouldn’t expect returning a string from addPreprocessor to execute a string of JavaScript either. We have a similar limitation executing arbitrary JS in the Render plugin https://www.11ty.dev/docs/plugins/render/#rendertemplate

I think I would classify that as a bigger enhancement request (not a bug right now).

All that said, we do use the node-retrieve-globals package to implement the js front matter type to execute arbitrary JavaScript https://github.com/11ty/eleventy/issues/2819 so there is a limited precedent for a path forward here.

Finally, I would call out the method used by tsx https://github.com/privatenumber/tsx (likely some magic using module.register https://nodejs.org/docs/latest/api/module.html#moduleregisterspecifier-parenturl-options but I haven’t dove all the way in to see) that might be a more ergonomic way to intercept and preprocess JavaScript. This is what we recommend on https://www.11ty.dev/docs/languages/typescript/

Open to suggestions here (of course).