Dart-Code: Analyzer high memory consumption, "Too many files open" errors

BTW, Dart 1.25 dev has a MUCH better analysis server page. Parsing data there is much better.

nm…they’re private

Yeah; I keep thinking about “unhiding” them, because the editor giving squiggles is a bit misleading (it also bugs me that I see these warnings all the time, because I always have logging turned on). I’ll put an issue in for the next release.

@johnbland-wf - please don’t be a help vampire.

This is a free (as in beer) plugin, and nobody pays or sponsors @DanTup to work on this. If you have a team of dozens of developers affected, I’d consider either (a) sponsoring Danny (this is increasingly common in open source work), or (b) assigning someone to help Danny burn down issues you find relevant (i.e. contribute back).

You may feel it is harsh, but I’m letting you know how this is impacting us. We have a large swath of IntelliJ/WebStorm folks here and several switched to VS Code only to switch back after getting these errors. It seriously is a pain to have to deal with this numerous times throughout the day.

I moved to VS Code for this plugin. It just isn’t working out great at this point so I’ve spent all of this time explaining the issue, providing debug logs, potential solutions we’ve used successfully before, etc in hopes of getting the plugin updated.

Telling me things like my plugin is driving people away from Code is not the way to get me to implement the things you want; you need to be reasonable and you need to help me by answering my questions.

I’ve done that over and over again to only get Reload Window as the solution. This is impacting the entire OS…not just Code so it slows us down to have to close our dev environments (terminal, etc) then reopen them.

I care much more about delivering quality and being able to maintain and support Dart Code into the future than the number of features it has, so it’s critical to me that I understand why every feature exists. I really don’t feel I’m asking a lot in return for this free plugin.

Great. I’m not arguing you do stuff that isn’t useful. I just don’t know how else to explain this situation to you, though. I’ve provided several options, debugging, etc to no avail.

I’ve already agreed with this several times, and so is Restart Analysis Server but at least this workaround is free and already there and not something I had to implement and maintain.

I don’t want anyone having to reload their project (or restart the analysis server, or anything else like this). I gave it as an option because it sounds like you were asking for a feature that is functionally identical to it. If it’s not, then I need to understand why (otherwise, I would presumably implement it incorrectly).

I’m asking for a fix to this issue. Restarting the analyzer without having to reload my entire window was just a stopgap since reloading the window isn’t discoverable at all; see people leaving Code not knowing how to “fix” the issue.

Can you provide more details about why/when you’d use each and for what?

Restart Analysis Server (or Reanalyze) is an easy way to get the analyzer to restart for any reason not just extension issues. Sometimes it is outdated and didn’t catch the branch changes or whatever happened and it says “File X does not exist” when it clearly does. Reload Window is expensive. It restarts every single extension/process/etc when only one thing is the problem: analyzer. Reload Window is killing a fly with a shotgun.

Stop Analysis Server allows Code to be an empty text editor without analyzer. I keep 2 windows open between the projects I am working on – sometimes simultaneously – and I don’t care about analyzer data in the other. If I want to preserve memory but still see the files for reference as I shift things from one repo to another, being able to stop the analyzer could be quite useful in doing just that; especially seeing as the analyzer somehow runs into the GB of consumption per window at times.

Status, to start, would provide us the data necessary to debug this problem. Right now you’ve built a black box so devs won’t have to see under the hood. At times, we need to and this is one. The Atom one shows me when the analyzer is working or idle (footer in Code shows that), but it also shows us the number of contexts, how many files were analyzed, and the task queue so we know if something is pending.

Here is the status section of the panel I mentioned working and clearly letting me know what’s happening:

All of that information helps us debug issues when analyzer trips up unrelated to this plugin as well as plugin issues.

Also note it shows what dart sdk is being used, seeing as multiple can exist on your machine at once.

I’d argue the same for Restart Analysis Server. If Dart Code isn’t working correctly how would the average user know that the fix is to Restart Analysis Server? Many users don’t even know what the analysis server is and many of those that do don’t know which parts of Dart Code it powers. IMO the analysis server is an implementation detail that the average user shouldn’t need to worry about.

As noted above, reanalyzing your source can be highly useful. It is a targeted fix and most people absolutely should know exactly what Analyzer is if they’re writing Dart code. Ignoring, as a dev, or hiding it as an “implementation detail” is handicapping yourself/your users for the sake of them not seeing under the hood. For new devs, maybe that helps. For experienced devs, it hurts.

Maybe we’re different here, though. We built our own tools around the analyzer and make everyone learn about the analyzer plus fail our builds if it doesn’t pass so everyone is intimately aware of what it does even if they’re not familiar with the inner workings.

Also, you’re well aware people will look at the Dart extension for settings/commands so that is way more discoverable than an unrelated feature of Code.

My main concern with implementing a specific command is that it will make it look like I’m ok with people having to restart the analysis server when they have issues, and I’m not.

But the issue may not be the plugin. The issue could be in analyzer. Also, I’d say you’re tacitly saying “restarting is ok” seeing as that is what Reload Window is doing.

I would rather people came here and reported issues than quietly “fixed” the issues themselves, otherwise the real issue will never be fixed.

I did. I was told to reload the window. -_-

I do not, because you hadn’t told me. If that is the case, I would expect that restarting the analyzer will have the same issue, since it gets restarted when you reload the window? 😦

From the thread, I thought that was understood. After opening Code and using it for a short period of time, we run out of files. Reloading the window is the same as opening Code so it happens again and again and again.

I’m more than happy to try and resolve your issues; but you need to help me by answering my questions above. I have a growing list of requests and limited time so I’m not going to spend time on things that I don’t believe will actually solves issues or deliver value. So far I don’t understand the value of these features (or how they solve your problem).

As noted, I don’t care about the features as much as fixing the issue but when I’m instructed to just reload the window I take that as a need for better features so I suggested some. Feel free not to implement any of the features but the issue absolutely needs some attention.

---

Ultimately, I get this is a free extension. There is no need to impress upon me the virtues of open source/free…I’m well aware. I’ve spent 2-3 hours debugging/discussing this issue to help you fix this issue to help me continue using Code so I don’t have to switch back to Atom – which it isn’t all that bad, I just like Code.

All requests have been answered throughout this thread. You can see in the gif above there are 12,729 files analyzed. Something is amiss here. Maybe it is analyzer. Maybe it is the plugin. I can’t say. My hope is you can.

@johnbland-wf Please understand that I really want to resolve your issues (and implement any useful features) but I can’t implement something without fully understanding the problems.

This plugin is making people stop using VS Code.

I think this is a little harsh. For many people, Dart Code works great (I finally got to use it myself recently for some work on the analysis server and with the exception of a couple of minor issues that I raised cases for, it worked very well, including running tests with F5). I personally know of several people that have moved to VS Code because of this plugin.

Please remember that this is an open source project - I don’t work for Google, I don’t get paid to work on this, my company doesn’t benefit from this (we don’t use Dart) and working on Dart Code is actually the main reason I don’t actually get to write much Dart myself (because any time I have that I want to spend on something Dart-related is actually spent on this extension). I enjoy working on Dart Code because I learn lots of new stuff and I like to imagine it’s useful to at least some people. Telling me things like my plugin is driving people away from Code is not the way to get me to implement the things you want; you need to be reasonable and you need to help me by answering my questions. I apologise if things that are obvious to you are not to me, I’m trying my best to help you while trying to avoid Dart Code descending into a random collection of features asked for by different people that I don’t understand and have to maintain long-term. I care much more about delivering quality and being able to maintain and support Dart Code into the future than the number of features it has, so it’s critical to me that I understand why every feature exists. I really don’t feel I’m asking a lot in return for this free plugin.

It is not a duplicate solution. It isn’t even a solution, tbh, it is a workaround

I’ve already agreed with this several times, and so is Restart Analysis Server but at least this workaround is free and already there and not something I had to implement and maintain.

I don’t want anyone having to reload their project (or restart the analysis server, or anything else like this). I gave it as an option because it sounds like you were asking for a feature that is functionally identical to it. If it’s not, then I need to understand why (otherwise, I would presumably implement it incorrectly).

Look at the screenshot. You can stop the analyzer, start, reanalyze, view status, etc.

I’m afraid I haven’t written as much Dart as you so I don’t entirely understand the need for each of these things. Can you explain the value of each? To my (very limited) eyes:

• Restart Analysis Server - This is functionally the same as Reload Window as far as I can tell and a hack/workaround for something going bad
• Stop Analysis Server - this will break the huge majority of functionality in Dart Code - I don’t really understand why anyone would want to do this
• Status - We currently collect this info to include in error reports, but I’m not sure of the value of displaying it generally (we do show in the status bar when it’s analyzing)

Can you provide more details about why/when you’d use each and for what?

I’m not sure how else to show how poorly “Reload Window” is as a solution; seeing as no one without knowledge of it being the fix would realize it.

I’d argue the same for Restart Analysis Server. If Dart Code isn’t working correctly how would the average user know that the fix is to Restart Analysis Server? Many users don’t even know what the analysis server is and many of those that do don’t know which parts of Dart Code it powers. IMO the analysis server is an implementation detail that the average user shouldn’t need to worry about.

My main concern with implementing a specific command is that it will make it look like I’m ok with people having to restart the analysis server when they have issues, and I’m not. I would rather people came here and reported issues than quietly “fixed” the issues themselves, otherwise the real issue will never be fixed.

Also; if the issue is that the analysis server has crashed/not responding/whatevever - why should the user even need to run a command? Why can’t Dart Code detect it and fix it silently in the background?

A panel/command/otherwise to turn off or restart analyzer would be helpful.

Can you explain what you expect to happen if you “Turn off the analysis server”? The server is providing a significant chunk of the functionality in Dart Code (completion, errors, highlighting occurrences, go to definition, find references, etc.) - are you asking for a switch that would stop all of these from working? If so, which bits are left that you actually want (eg. that means you don’t want to just disable the extension? Are you using it just for the debugger?)

And you do realize shortly after Reload Window it does the exact same thing again, right?

I do not, because you hadn’t told me. If that is the case, I would expect that restarting the analyzer will have the same issue, since it gets restarted when you reload the window? 😦

I’m more than happy to try and resolve your issues; but you need to help me by answering my questions above. I have a growing list of requests and limited time so I’m not going to spend time on things that I don’t believe will actually solves issues or deliver value. So far I don’t understand the value of these features (or how they solve your problem).

I think the implication is that the Dart Analyzer shouldn’t NEED to be restarted, and that the underlying issue should be addressed instead. Also that the same process of restarting the analyzer is performed by reloading the window.

Those extra options might make the idea more palatable though. All the same, the best option seems to be just throwing in a few new commands to the palette.