cmder: [Bug] cmder loads very slowly under Windows

This also solves the problem for me 😃 Now it’s an instant startup. @d4k0 , good catch!!

Don’t know if it helps but starting Cmder I can see rotating cmd.exe and findstr.exe in the lower tab:

Happens on my Windows 10 Pro private PC and my Windows 11 Pro office PC. On the office PC the startup takes significant more time than on the Win 10 machine.

• Win 10: 10 sec
• Win 11: 30 sec

@d4k0 have you tried the latest build linked in this issue.

I tried it and it significantly reduces startup time (17s vs 5s). Great work!

Still haven’t found a way to adequately whitelist it but it seems the issue comes specifically from Eset’s “Deep Behavioral Inspection”, which is found under the HIPS section of advanced settings. Disabling that pretty much restores cmder’s speed for me. No restart is required when disabling/enabling that option, like when disabling the whole HIPS engine.

I can confirm that the cause is ESET’s “Deep Behavioral Inspection”. I installed ESET in a VM and then the startup time of Cmder Mini (latest build) was also suddenly 5s.

After disabling “Deep Behavioral Inspection” the startup time is about 0.7s.

But since this is not a permanent solution for me, I played around with the exceptions and probably found a solution. It is enough to add the path to cmd.exe (C:\Windows\System32\cmd.exe) under the exceptions for “Deep Behavioral Inspection”.

If I add ALSO the folder of Cmder there, the startup time does not improve further. If I ONLY add the folder of Cmder, the startup time improves only minimally (about 0.5s faster, about 4.5s in total).

The question now is: is the cause due to a Windows update (patch day was January 10) or was it due to the last ESET program update on January 16? I have been using ESET for a year now and never had any problems with it in conjunction with Cmder until mid-January. I’m just a little puzzled that the cmd.exe exception “solves” the problem. Maybe we should inform ESET support?

I guess we need to start asking specifically which antivirus are you running when people report slownes

Seems like a bit of a pattern here to me.

@d4k0 ah, a smoking gun 😎 yes a Jan 16 update of ESET fits the timeline with the first report here on Jan 18.

Yes, I would definitely recommend informing ESET support about this severe performance problem.

Also, a sidebar – it’s is an interesting point for troubleshooters to be aware of for future issues: protection suites may require multiple actions to turn them off to the point where they’re doing zero work. Until they’re doing zero work, they’re not really “turned off” enough to rule out whether they’re contributing to or causing the problem. Unfortunately, since each suite is different, it might take a bit of Q&A in each incident to identify which suite is in use and how to fully turn it off.

On Jan 24 @andrej-dyck said “I turned off my antivirus software; still slow.” I expect that test probably suffered the same wrinkle that occurred in @mtaki14 's test where pausing protection doesn’t fully turn off the ESET suite, and the Host-based Intrusion Prevention System remains on (and also a reboot might be necessarily when turning off ESET for it to fully take effect).

More improvements in #2825

• Added a new lean yet full featured optional %cmder_root%\vendor\user_init.optional.cmd
  • Usage documentation is in the file.
  • Basically an almost completely hard coded version of the %cmder_root%\vendor\init.bat that gives the user complete control if they want it.
  • See speed improvements from master -> more_speed_2 -> more_speed_2 with user_init.cmd below with /t only:

Downloads here

Yes, that’s interference from antivirus/etc.

There aren’t any CMD commands between those two echo commands, but there is the end ) after finishing the if defined git_locale ( ... ) block that was just evaluated with enabledelayedexpansion. That ) triggers a bunch of system API calls and file IO for opening the .bat file again and parsing a next command, and getting/setting the console title, and a bunch of other minutia that involve system APIs.

The antivirus/etc software is intercepting one or more of the APIs and decides that it wants to do some extra processing, and the extra processing takes a bunch of time.

I agree that it doesn’t make sense why that would be needed or take so long. But it’s a bug or malfunction in the antivirus/etc software – for whatever reason, it got confused into thinking it needed to be extra work.

You’d need to identify all the antivirus and DLP and corporate monitoring tools that are installed, and try to narrow down specifically which of them are going awry, and report the issue to the respective support team for whichever software(s) are involved. There’s unfortunately nothing Cmder or ConEmu can do about this, because it’s happening externally from them.

Here are some questions / ideas / observations based on reviewing the WPA ETL I received from @dk40.

Later tonight, I’ll capture a similar trace on my machine where processes are fast within ConEmu, and compare the performance traces to look for similarities and differences. Hopefully that will reveal some things to explore further.

Does the speed improve if you temporarily turn off ESET?

• The files ebehmoni.dll and ebehmonl.dll are loaded into each process.
• The files are part of the ESET anti-malware suite.
• ekrn.exe (the ESET Kernel) is the largest consumer of CPU time, and it correlates with each process start. That’s to be expected, but I can’t tell whether it’s interfering with the speed of execution.
• It’s also possible that ESET is suspicious about ConEmu injecting conemuhk.dll into each process. Turning off ConEmuHk injection might also be able to improve the speed, especially if the slowness is coming from ESET.

lsass.exe network operations

• There appears to be a steady stream of roughly 1 second long network Send requests from lsass.exe during the trace. They roughly align with just after each of the cmd.exe and findstr.exe etc processes starts. It almost looks as though some kind of authentication is happening that is correlated with starting processes.
• This looks surprising to me. There is correlation, but correlation does not imply causation.

Does the speed improve if Cmder is installed in C:\Cmder\?

• Cmder was installed into %USERPROFILE%\Downloads\cmder_mini\. The search indexer (and backup programs and other programs) watch for activity on files in the user’s Downloads and Documents directories and other user profile directories, and they may trigger additional work.
• I see no speed difference when running Cmder from d:\tmp\Cmder\ or %USERPROFILE%\Downloads\Cmder\, but I don’t have ESET installed.

Something looks potentially surprising with SearchIndexer and/or Prefetch

• SearchIndexer.exe appears to be getting woken repeatedly due to changes in Prefetch files cmd.exe-4a81b364.pf and findstr.exe-2e9c6fe2.pf files, and a few other prefetch files.
• I don’t understand why SearchIndexer is monitoring for file change notifications in C:\Windows\Prefetch\, but maybe that’s expected and normal?

Other observations

• SmartScreen.exe starts right before ConEmu starts. SmartScreen might be doing extra verifications, and might also do extra verifications on all programs started from within ConEmu. Have you verified for each individual .exe and .dll file in all subdirectories under cmder_mini, that each of them has been successfully Unblocked? I.e. that none of them show an “Unblock” button in their individual File Properties dialogs.
• PunkBuster (PnkBstrA.exe) appears to be doing a heartbeat of network IO, roughly once per second.
• GamingServices.exe seems to be doing some kind of work in the background as well, including some network IO.

@Crayder As stated above something changed in Windows builds that caused a less than optimal %cmder_root%\vendor\init.bat to become ridiculously slow for a subset of users. We have been unable to determine the root cause.

My code is not a fix it is a fundamental change in the way Cmder initializes cmd.exe sessions.

It is for ALL Cmder users regardless of how long it takes to initialize! All users will see an improvement in first launch speeds and second launch speeds are even faster as shown by the screenshots above.

Latest build here

@andrej-dyck See below after some modifications, PR #2819, using Cmder Mini and User installed git with /f:

Same here!

Cmder version: v1.3.21 (1.3.20.151)
Clink: v1.4.10.45c041
git: version 2.39.0.windows.1
ConEmu: 221218 [64] Stable

Operating system: Windows 11 Pro 22H2 (Build 22621.1105)

Right, Thanks. I checked for update within cmder but because it only checks for conemu updates, it fooled me into thinking I have up-to-date version of cmder too. Before writing down this comment I didn’t know that right way to check cmder versions is to look into its folder and see the Version file. May be there is another way but I don’t know about it.

The cmder I replaced (moved to another folder) was Version 1.3.19.1181. The cmder mini I am now using is Version 1.3.24.236.

That was the problem.

.19 had a bug that ran part of the git stuff in the foreground by accident.

Upgrading to .24 would solve the problem, because the fix was made between those versions.

In general, when an issue is encountered in a program, it’s handy to check if the latest version fixes it.

@chrisant996 @SMUsamaShah I am not certain the newest Cmder release includes background prompt refresh. Probably need to get a pre-rel!ase archive and try it.

With eset nod people were adding cmd.exe to solve the slowness. I doubt adding the cmder root folder will do much to speed things up.

Edit: I do not recommend doing the above. I’m just letting you know what others have done to temporarily work around overly aggressive DLP/Anti-Virus apps until they are fixed.

@daxgames That’s https://github.com/cmderdev/cmder/issues/2800, and it’s a bug in Cmder, not a Clink issue. I assume @DRSDavidSoft is busy and hasn’t gotten around to fixing it.

@pmsobrado The “Clink initialization has failed” message is a bug in Cmder. Apart from looking unsightly, everything is actually fine. The Cmder script misinterprets Clink’s response, mistakenly thinks Clinks failed when Clink is configured for autostart, and the Cmder script says a failure occurred, but actually everything is fine.

@pmsobrado you found another bug in this branch. I’ll fix that.

Same great result for me:

I’ve tried disabling ESET HIPs, and now it tooks me 0.16s to load! 😃

Wow @WickedSilver is right, disabling HIPS fixed the issue for me as well. I assumed that pausing protection stopped everything ESET related, I guess not then.

I am too currently trying to figure out how to whitelist it. Adding the whole cmder folder as a hips exclusion doesn’t seem to do the trick.

I can confirm that disabling HIPS (Host-based Intrusion Prevention System) in ESET fixed the slowness for me. Disabling HIPS does require a restart though. It might be best to add a rule to exclude cmder/conemu from ESET instead of disabling it altogether, but not sure how to do that at the moment.

Thanks for the reported observations @chrisant996!

If you disconnect ethernet and put the computer into Airplane Mode, does that improve the speed problem?

No change. With Eset disabled, Ethernet disconnected and computer in Flight Mode, cmder start-up time is still 5s+, same as before

@deniscuculic,

1. Cmder is not a terminal emulator.
2. It is a set of init scripts developed by the Cmder team that ties together 4 externally developed components.

• A terminal emulator called Conemu.
• Shell enhancements clink and clink completions.
• Git for Windows in the full version.
• Packaged together as a portable shell env called Cmder.

1. The EXTREME slowness reported in this thread started happening AFTER a specific Windows update to current and all previous Cmder releases. This indicates it is indeed an external issue.

I can run the Cmder init scripts in native Cmd.exe, VSCode Terminal, Tabby, Conemu, Windows Terminal, and Hyper.

Edit: Start All have relatively THE SAME speed results I mentioned above, less than 1 second, with full Cmder functionality. Even on an old underpowered VM on a laptop with a slow hdd. ConEmu is a bit slower by maybe .5-.75 second, so it is 1.x seconds on my VM. On my big fast workstation with NVMe SSD even ConEmu is .19 seconds. Edit: End

We are here to help, but we have done EVERYTHING we can since none of us are experiencing the problem and never have.

Instead of being so negative and trying to piss someone off, how about you tell us how it is performing for you?

Are you seeing extreme slowness reported above? Please provide launch times by adding /t to the task used to start Cmder as it provides the most accurate launch times.

If so, can you provide the traces asked for above by @chrisant996 that might point to an issue we can not see?

@deniscuculic, as stated above several times, this is not a Cmder problem.

That being said, this thread contains links to an unreleased build that removes some inefficiency in the Cmder init process.

I am routinely seeing load times of .19 seconds. That is pretty fast.

Prior to this build I routinely saw launch times of 1.7-2.5 seconds.

The problem is that none of the Cmder developers can reproduce the problem to troubleshoot it adequately.

@andrej-dyck regarding these:

My windows is not managed by any company (i.e., it’s a personal Notebook). I turned off my antivirus software; still slow.

Did you turn off Windows Defender? I ask in case “turned off my antivirus software” refers to AV software that you installed separately.

I have nothing in autostart

Do you mean OS autostart? Or do you mean CMD AutoRun? What does clink autorun show report? Has clink autorun uninstall been tried?

Maybe this list of conclusions and questions will help:

What’s Known So Far

• Old versions of Cmder exhibit the problem suddenly, as well.
  • Conclusion: This is not due to any recent change in Cmder or ConEmu.
• Multiple builds of Win10 and Win11 exhibit the problem, but only for some people.
  • Conclusion: This might be related to some OS change, but it isn’t happening universally for everyone with those OS versions.
• The slowness is evident in the invoking of programs such as findstr and piping their output.
  • Conclusion: The fact that “findstr” takes long enough to show up in the title suggests that the slowness is actually happening to findstr and the other invoked programs.

Pending Questions

• Is there a connection with Clink?
• Is there a connection with Windows Defender?
• Does slowness only occur during Cmder’s init.bat script, or are similar commands slow after that, as well?
• Does slowness only happen on the first init.bat invocation in a session? What happens if you set CMDER_CONFIGURED= and then run vendor\init.bat again?
• Does slowness happen when invoking Cmder’s init.bat script from a plain old cmd.exe without using ConEmu?

@usoliveira I am working on a way that is even faster for users who want to be in complete control and do not need all of Cmder ability to do auto configuration.

It is no good for the first time/inexperienced user but for someone seeing slowness due to some unknown external cause with full auto config it can be useful.

Users can control the config COMPLETELY using an optional hard coded script in %cmder_root%\config that still supports everything %cmder_root%\vendor\init.bat does. If present %cmder_root%\vendor\init.bat runs the optional script and exits skipping all the auto config that makes Cmder very flexible.

I have seen load times of less than 1 second using the same VM in the PR.

@chrisant996 sorry for not replying. Lot’s to do right now. I’m moved on to Windows Command Prompt + Clink + some Clink plugins. I saw that WPR comes with tools that are 2GB+. Seems very heavy and I have a fresh install. I’m not sure what information would be recorded and I don’t have the time right now to verify it. Sorry again.

Understood; no worries.

What it would record is a performance trace of all the APIs that are called by processes and the timings for them, so that they can be viewed/searched/analyzed in a timeline view (and other views). That would make it possible to identify specifically how the additional time is being spent.

P.S. I thought the WPR install can be customized to be smaller, but I don’t remember for sure. And like you, I have a lot going on and I’m not going to dig into that, specifically. 🙃

@daxgames @chrisant996 Some more information.

While I was writing an response, I started up Windows Sandbox, extracted cmder_mini, and it starts in 2sec.

However, notice the older Windows 10 version in the sandbox!

Furthermore, on my company notebook, which is managed, has an aggressive antivir and most of the same software, Cmder is quite quick (1-2sec). There, I have Windows 11 installed.

Can someone try downgrading to an older version of Cmder or ConEmu, and see if the problem goes away?

If downgrading ConEmu resolves the problem, then the issue is likely something in ConEmu, not Cmder.

If downgrading ConEmu doesn’t resolve the problem, but downgrading Cmder resolves the problem, then there would be a possibility that the issue might be due to some recent change in the Cmder scripts.

Same. I extracted cmder.zip on a fresh install of Win10 22H2, and it takes up to 50sec to start.

Hi, Same here , it takes 20 or more seconds with /f and close to 60 without /f . Also tried adding variable of git location , this improves startup for few seconds.

Elapsed Time: 0:0:22.87 (22.87s total)

Microsoft Windows [Version 10.0.19045.2251] - Windows 10 Pro 22H2 cmder 1.3.20.151