eza: feat: `eza` is substantially slower than `ls` in large directories
As discussed here, eza is significantly slower than ls when listing files in large directories. This issue persists with the latest version (0.18.9).
Here’s a benchmark for a directory with ~50k files:
Benchmark 1: ls -1 dir | wc -l
Time (mean ± σ): 807.9 ms ± 219.8 ms [User: 102.2 ms, System: 20.4 ms]
Range (min … max): 597.6 ms … 1036.0 ms 3 runs
Benchmark 2: eza -1 dir | wc -l
Time (mean ± σ): 3.541 s ± 0.422 s [User: 0.203 s, System: 0.654 s]
Range (min … max): 3.074 s … 3.895 s 3 runs
Summary
ls -1 dir | wc -l ran
4.38 ± 1.30 times faster than eza -1 UHGV | wc -l
About this issue
- Original URL
- State: open
- Created 3 years ago
- Reactions: 4
- Comments: 15 (10 by maintainers)
I noticed a substantial slowdown as well.
Example (single sample):
It seems like the futex / clone3 syscalls take the longest.
Oh, one of the reasons there’s so much lock traffic through
futexcalls with-lis that eza depends on uzers to get UID -> name conversion but uzers uses a cache based on aRefCellso it’s incompatible withRwLock. So the formatting code thrashes theMutexthat eza’s concurrency approach requires.Not a bad idea, even as just a nice feature to replace
eza -l | head, which removes all styling. However, in terms of performance, it’s just a workaround, because the difference between eza and the other utils show that eza could do better. The write calls should be easy to improve. The statx are necessary to determine whether a file is executable, so it’s more of a design decision for the maintainers. It can still be improved easily in cases where the color is turned off though. If there’s demand for that I can pick up my PR again.