fd: 8.1.0 find the targets very slow in a large directory when using fzf. 8.0.0 works well.
Hi,
I am using fd as the default command of fzf. After updating to the latest version, I find it is very slow to find the target in a large directory (like home).
For FZF, I set export FZF_DEFAULT_COMMAND="fd --hidden --follow --exclude .git". Then I run fzf under my home directory. Next, I type a directory name which is right under home directory say .config, and it will take very very long to find ~/.config. It shows many files under ~/Library which I do not care about. I attached a screenshot below. You can see that ~/.config won’t show up even though it has scanned 1080073 entries.
Before updating fd and I cannot remember which version it was, it works very well. When I type .config, it will show up almost immediately. The results could be adjusted automatically based on which query I typed so that my target will show up right away. For example, when I typed .config, it would not show me all those Library/…/config which I did not care about, and instead it would show ~/.config. Very smart.
What version of fd are you using?
fd 8.1.0
Which operating system / distribution are you on?
macOS 10.15.4 (the latest)
Thank you very much!!
Update1:
To be clearer, I put two GIF below (this time I ran fzf under home and tried to search a directory, ~/gitrepos):
1). I didn’t set fzf default command, which means let fzf use the default find. When I started to type gitrepos, the targets appeared immediately.
2). I set fzf default command to fd --type f. This time when I started to type gitrepos, the targets wouldn’t appear until all entries were scanned.
Update2:
I just now deleted the latest version of fd and re-installed version 8.0.0 (with brew install https://raw.githubusercontent.com/Homebrew/homebrew-core/d874b06712ec20efd86f2fbf20e97aa2f24e9f5b/Formula/fd.rb). The original performance is back. So I am sure the latest version has a bug. Thank you!
About this issue
- Original URL
- State: open
- Created 4 years ago
- Reactions: 1
- Comments: 25 (7 by maintainers)
Thank you.
Below the output of hyperfine:
v8.1.0 takes a very long time compared to v8.0.0.
I first need to understand all the implications of this change. This comes as a surprise to me as well. I certainly agree that breadth-first sounds more useful for the typical
fduse case. I will keep you updated.I think I might have found the reason why this happens. I actually upgraded the
fddependencies without noticing that there was a new version of theignorecrate available. It comes with this pretty significant change: https://github.com/BurntSushi/ripgrep/pull/1554 (!)This would explain perfectly well why you previously saw low-depth results first (as the search was breadth first) while it takes almost until the end of the search now (with depth first traversal). The overall search time might also be influenced by this, as the scheduling of different threads might be affected by this (?).
1.03 x faster means: 3% faster. Not very conclusive. But the runs in the other two directories show a much larger gap.
Hi, Here is the result.
8.0.0 is faster (not sure whether it is accurate because I was watching YouTube while hyperfine was running 😛). And actually based on my observation when running fzf, both of the scanning speed for 8.0.0 and 8.1.0 are very fast. When we run fzf, we can see a number that shows how many entries have been scanned. The number increases very fast for both versions. My concern is that 8.1.0 can get the target immediately but 8.0.0 not.
Also, I ran this test under some different directories. 8.1.0 is slower. Screenshots attached below.
1). ~/gitrepos which has the repos I cloned from github
2). ~/Dropbox
Sure. Still running under my home folder. Just a moment please. 😃 v8.0.0 has finished. Now, v8.1.0 is running and it is much slower. I will post the screenshot once it finishes.
Ok, there seems to be a drastic regression. But you are running the command from
/tmp. Could you run it from your home folder?