typescript-eslint: Bug: `tsconfig.json` not respected in new files when multiple tsconfigs include the same directory with `projectService` enabled
Before You File a Bug Report Please Confirm You Have Done The Following…
- I have tried restarting my IDE and the issue persists.
- I have updated to the latest version of the packages.
- I have searched for related issues and found none that matched my issue.
- I have read the FAQ and my problem is not listed.
Issue Description
When EXPERIMENTAL_useProjectService: true and at least two tsconfig.jsons include a non-empty common directory, typescript-eslint will not respect tsconfig.json for new/renamed files until ESLint is reloaded.
Reproduction Repository Link
https://github.com/fire332/typescript-eslint-bug-repro/
Repro Steps
- clone the repo
pnpm install- open project in VSCode with the ESLint extension installed
- IMPORTANT: open
.prettierrc.jsoreslint.config.jsand wait for ESLint to initialize - create a new
.tsfile in./srcor./src-alt - problems reported will show:
This rule requires the `strictNullChecks` compiler option to be turned on to function correctly. eslint(@typescript-eslint/no-unnecessary-condition)
- run
Developer: Reload Windowfrom the command palette - check the file you created and note there’s no problems reported
- rename the file and note that the same problem will be reported again
Versions
| package | version |
|---|---|
typescript-eslint |
7.5.0 |
TypeScript |
5.4.3 |
ESLint |
8.57.0 |
node |
>=20.11.1 |
About this issue
- Original URL
- State: open
- Created 3 years ago
- Reactions: 6
- Comments: 19 (13 by maintainers)
If anyone wants to test, I’ve moved to this branch: https://github.com/higherorderfunctor/typescript-eslint/tree/patches2upstream. There are a lot of modifications, so I’ll work to get these up-streamed over several PRs. I’ve confirmed the
strictNullChecksissue is still fixed for the scenario I was able to reproduce. I’m still oneslint@8.57.0for reference as there is one plugin I use that doesn’t work withFlatCompatineslint@9(eslint-config-airbnb-base/whitespace).I’ve included the
typesandtypescript-estreebuilds in the repo if your package manager supports installing from git. Mypackage.jsonlooks like below withpnpm@9.1.x. The patch is for https://github.com/typescript-eslint/typescript-eslint/issues/9223 until those fixes make it into eslint-plugin-import-x.package.json
My RAM utilization is way down today… not sure if it is something I did, something upstream in the
rc-v8branch, or something in the latesttypescript@rc, but linting my whole project now is down to ~2min and RAM is slightly over 3G. I had my heap up to 9G a couple days ago or it would crash.If you do get OOM, the heap can be increased.
Known Issues
With the flat config and
extraFileExtensions, it is best to set it once if possible. I had differentprojectServicesettings between my front-end and back-end. Changing theextraFileExtensionsmid-lint causes tsserver to do a full project reload. I have guards in place to do it, but it will cause a performance hit. I originally only set it once and was wondering why I was having issue with.vuefiles intermittently. Was caused by whichprojectServicehappened to be loaded first.Additional Options
I’ve added two new options with their defaults below.
incrementalonly sends deltas to tsserver which I am hoping makes the LSP experience better.maximumOpenFileswill automatically close files with tsserver, which is nice on the CLI which has no mechanism to know when it is done with a file. The new options are why thetypespackage is needed if you enable typing on youreslint.config.js.Hmm. Interesting. I’ll take another look, thanks.
In case it’s useful, I’m only testing with v8 - as of yesterday,
typescript-eslint@8.0.0-alpha.25. We have some improvements to the project service on the v8 line that we don’t plan on backporting to v7.Yeah, these are a lot of open questions. I really really appreciate your offer to hustle but long-term I don’t think it’s a given that we’d want to try to handle these cases internally (https://github.com/typescript-eslint/typescript-eslint/issues/8835#issuecomment-2254489382). I’m going to go ahead and remove this from the v8 milestone.
That’s one of the things I’m very hopeful for with the project service!
@JoshuaKGoldberg I can probably get a minimum viable implementation done by Monday. Basically, is the file in the project service? If no, trigger the best matching watcher. Recheck and if still no add to default project or treat as an isolated file.
This will deal with new source files requiring the LSP to be restarted.
Some of the more complex decision making I don’t think I’d be able to have ready by release day. Examples:
I mentioned this in another thread, but I have a busy week this week in case someone else needs to pick up the PR. Dropping my notes in case anyone else does pick it up.
The working implementation is https://github.com/higherorderfunctor/typescript-eslint/tree/patches2upstream, the draft PR is not currently in a working state.
While I have a working implementation, it duplicates some of the existing logic in
getWatchProgramsForProjectsts.ts.saveWatchCallbacklooks likely to be a drop in candidate for my implementation.Exporting or proving an access function for
fileWatchCallbackTrackingMapandfolderWatchCallbackTrackingMapcould allow for the watches to be updated easily. I haven’t analyzed the paths tsserver requests the host to watch. I opted to use a trie so I could find the most specific path.getWatchProgramsForProjectsts.tsjust uses aMap, so maybe supporting partial matches is not even needed.hasTSConfigChangedis probably good to consider since changes totsconfig.jsonI don’t believe will reported to the plugin.maybeInvalidateProgramwhich calls the above has quite a bit of logic, but I am not sure all of it is needed for project services. I didn’t see a need to do all the caching that is done. Linting results were almost immediate by just walking the service, even on a larger project, and triggering the watcher for the most specific matching path.So I have a hobbled together set of what I hope are the start of fixes . One note if anyone clones, the
package.jsons were modified to use links for in-repo dependencies so I could add the build outputs to my larger work repository also using links but with pnpm.First, I track watches set by the
ProjectServicewith a new getWatchesForProjectService.ts. Will need to look through getWatchProgramsForProjects.ts to see if there is room to consolidate.This is used by createProjectService.ts to call the watch constructors via the host and useProgramFromProjectService.ts to trigger the watches when a file has not been seen by the
ProjectService. I trigger the most specific watch which fixed thestrictNullChecksfor me.I use
getScriptInfoandScriptInfo.editContentto deal with lint requests for files that are already open (e.g.,--fix). Some@stylisticrules crash on the AST for vue files without it. I have some commented out caches for open files and programs that produced similar issues. I was hoping to get away with not needing to re-open the file after a fix. There may be some room for improvements, but it would take some more research.Also noteworthy is that calling
service.setHostConfigurationtriggers a full project reload for files with irregular extensions likevue. It should probably be set once withProjectServiceduring construction instead of on every file. Linting a vue package in my work monorepo went from 3m30s to 20s. The whole monorepo I am testing on went from 16m in CI on server hardware to 3m40s on the mini pc I am currently using which is a much slower machine. I’m excited by those numbers and they produced the same results for both static analysis and running with--fix.I may have a memory leak as I had to increase the heap. Could also just be linting a large monorepo using only a default project withProjectServices without ever closing a file with tsserver. Again, some more research is needed to cut that memory usage down.edit: Been looking through the heap. I’m thinking it might be “okay”. We use a lot of complex types which is likely inflating the memory usage. The
SourceFileObjectcount is accurate. Most of the memory is taken up by NodeObjects which looks like some low level AST data structure. Should be good enough for an LSP and I’ll just have to bump the heap if I want the speed on a full CLI lint. I did put an LRU in to limit the number of opened files. Not really seeing a benefit on the CLI where it just churns through everything quickly.I ran into the same warning but with a different setup.
I don’t use
EXPERIMENTAL_useProjectService: truebut there may be a common root cause. If I use--fixI get the warning, if I don’t use--fixI don’t get that warning.I did some debugging. It looks like
typescript-estreecreates two different “programs” for the same file internally, because--fixuses multiple cycles:https://github.com/typescript-eslint/typescript-eslint/blob/63e53e28f68f84764ec632d502da3d33095f6a0d/packages/typescript-estree/src/parser.ts#L235-L253
The first program is created through
getProgramAndAST, the second one usingcreateIsolatedProgram.The first cycle uses
getProgramAndASTwhich in turn usescreateProgramFromConfigFileto load the correcttsconfig.jsonand pass the options tocreateProgram: https://github.com/typescript-eslint/typescript-eslint/blob/63e53e28f68f84764ec632d502da3d33095f6a0d/packages/typescript-estree/src/create-program/useProvidedPrograms.ts#L55-L89The second cycle uses
createIsolatedProgramwhich completely ignorestsconfig.jsonand uses pre-defined options: https://github.com/typescript-eslint/typescript-eslint/blob/63e53e28f68f84764ec632d502da3d33095f6a0d/packages/typescript-estree/src/create-program/createIsolatedProgram.ts#L64-L74That in turn explains the warning because
strictNullChecksis indeedfalsein that case.Since my problem is caused by the same file being compiled multiple times due to
--fix, it may as well be related to this issue where the same file is compiled multiple times.Did some log analysis to see if I could find the inflection point between the original file and a copied file as I am able to reproduce on a large repo. It did not work on a small example repo. The last line is where the difference occurs.
Original File
This log message happens here: https://github.com/microsoft/TypeScript/blob/27bcd4cb5a98bce46c9cdd749752703ead021a4b/src/server/editorServices.ts#L2462
Copied File
That log message is set here when setting up the watch: https://github.com/microsoft/TypeScript/blob/27bcd4cb5a98bce46c9cdd749752703ead021a4b/src/server/editorServices.ts#L2192
I haven’t had the time to do a full code level trace or fire up the debugger yet, but at a glance it looks like tsserver treats the new file as an orphan. Since I cannot reproduce on a small repo, it may suggest a race condition.
Wrapping up for the night I took a shot in the dark at changing this line in the plugin to use
getScriptInfoEnsuringProjectsUptoDateinstead ofgetScriptInfo: https://github.com/typescript-eslint/typescript-eslint/blob/c9a6dd9f3df4e3c2bfcc5dceb770b021709152ae/packages/typescript-estree/src/useProgramFromProjectService.ts#L73It didn’t fix the issue, but this week is also my first venture into the tsserver code, so I may not be using it correctly.
To reproduce the logging this is roughly the code I ended up using at https://github.com/typescript-eslint/typescript-eslint/blob/585423db7dda28c0c7b8efc05a8c244dde6ca92b/packages/typescript-estree/src/create-program/createProjectService.ts#L56
I am using neovim but with the vscode eslint LSP. I just started my editor with the
DEBUG=*environment variable set to generate the logs which end up in~/.local/state/nvim/lsp.log. This generates ALOT of data so I wouldn’t recommend leaving it on.