eslint: Allow processor API to be configurable and to formally be able to lint both a file and its blocks

Thanks for the feedback. Since you’re on board with the direction, that’s enough for me to continue sketching out a prototype implementation and write up an RFC. It’ll be easy to change the special value, so I’ll explore the tradeoffs of each option in the RFC and we can decide which way to go there.

I looked at standardizing the existing workaround in which preprocess() returns the full file text as the first “block” in the array. However, that mean would mean the processor and not the user determines when to further lint the file. In contrast, the processor API change released in ESLint v6 was all about giving users control over processors.

I’m now thinking the processor key should optionally accept an array of multiple processors to apply, and the special value "passthrough" would tell ESLint to lint the file as normal in addition to running any other processors:

const config = {
    files: ["*.js"],
    processor: [
        // Continue to lint this file as normal...
        "passthrough",
        // ...but also extract and lint any JSDoc blocks.
        "jsdoc/processor"
    ]
};

Yes, assuming there are no objections, I need to write an RFC that will allow linting a file and extracting its blocks with a processor and linting those too.

There’s already an unofficial way to do that mentioned in #14198, so the RFC may just pave that cow path and add make it configurable.

The second half of this is asking for processor configuration, but if the RFC unblocks the first half, I believe the second half can be achieved with parser options for the processor’s parser and may not need any further changes.

To adapt @btmills ’ reply https://github.com/eslint/eslint/issues/14745#issuecomment-870603657 accurate summary of the remaining needs, it would be to address two limitations of the processor API:

1. Adopt an officially-recommended way to run a file through a processor to lint its blocks while still linting the file itself. (This would be usable, e.g., by our jsdoc plugin, to lint the JavaScript within @example while linting the rest of the file.)

Need to determine whether to standardize the current informal return [text, …blocks]; approach (and document it as such) or define some other API.

1. Means to configure a processor’s behavior, supplying it config, e.g., for the jsdoc plugin to conditionally parse out <caption> or apply a fixed leading indent.

I think code block with custom filename + overrides maybe is all you want.

const jsdoc = {
  preprocess() {
    return [
      text,
      {
        filename: 'jsdoc-example' + getExtFromJsdoc(text),
      },
    ]
  },
}

{
  "overrides": [
    {
      "files": ["*.js", "*.ts"],
      "processor": "jsdoc/example" // a pretended value here
    },
    {
      "files": [
        "*.js/*_jsdoc-example.js",
        "*.ts/*_jsdoc-example.js",
        "*.js/*_jsdoc-example.ts"
      ],
      "rules": {
        // specific rules for examples in jsdoc only here
        // And other rules for `.js` and `.ts` will also be enabled for them
      }
    }
  ]
}

Moving to Waiting for RFC

This use case sounds very similar to #14198. Here, we want to lint a JS file and separately lint expressions from JSDoc comments. #14198 wants to lint a Markdown file and separately lint the code blocks within.

I’m not thrilled about the API proposed in #14198. Similarly, the API proposed in this issue has some downsides including performance as you’ve already mentioned.

Looking at these two use cases together gave me an idea:

eslint-plugin-jsdoc wants to extract chunks from JSDoc comments. That’s exactly what ESLint’s processor API is for. However, that API assumes that we lint either a file XOR the blocks from a processor. Because of that restriction, eslint-plugin-JSDoc cannot be written as a processor today. If it were, it would prevent linting the parent JS file.

What if we removed that limitation? If we could lint both parent.js and run a JSDoc processor on it to lint JSDoc expressions as parent.js/x_y.jsdoc, eslint-plugin-jsdoc wouldn’t need to call back into CLIEngine from within a rule anymore. That removes performance overhead, allows it to use the built-in config system for the JSDoc expressions, and probably simplifies the plugin/rule implementation considerably. It also makes #14198 a first-class supported use case.