gitleaks: GitLeaks throwing "too many open files" error when running with the --no-git option
Describe the bug When running the “gitleaks” command against a git repository that’s locally checked out on Mac OS X, it throws a “too many open files” error:
gitleaks --path=. -v --no-git
ERRO[0000] open .git/objects/06/c465fef0df46ca67970792e4a3b31695bd6498: too many open files
INFO[0000] scan time: 485 milliseconds 934 microseconds
ERRO[0000] open .git/objects/06/c465fef0df46ca67970792e4a3b31695bd6498: too many open files
The reason for running with the “–no-git” option is that I would like to only scan the current revision of the repository (all the files, not just the ones referenced by the latest commit).
The error seems to be being thrown due to GitLeaks scanning the “.git” repo in particular, which has a lot of files.
Other things I’ve tried to work around the above with no success are:
- Running with “threads=1” to try and reduce the number of files that the program opens “gitleaks --path=. -v --threads=1 --no-git”
- Moving the “.git” directory out of the checked out repository temporarily
To Reproduce On a machine running OS X 11.4, clone a large-ish Git repository.
Install gitleaks, change directory into the repository and run:
gitleaks --path=. -v --no-git
Expected behavior I would expect that gitleaks scans the directory without throwing an error and identifies any secrets in files as per normal.
Screenshots N/A
Basic Info (please complete the following information):
- OS: Mac OS X 11.4
- Gitleaks Version: 7.5.0
Additional context The OS X system max open file limits are set to the following (after increasing):
$ launchctl limit maxfiles
maxfiles 65536 1048576
cc @zricethezav
About this issue
- Original URL
- State: closed
- Created 3 years ago
- Comments: 15 (1 by maintainers)
@srkiNZ84
Damn, true. Thanks for pointing that out. Gotta fix that
have upgraded, ran the command with “–no-git” that was consistently failing before and it is working now! Thank you so much! 🙂
@jboler @srkiNZ84 should be fixed in 7.6.0, feel free to reopen if you find it still broken
@jboler @srkiNZ84 found the issue, turns out I wasn’t limiting the number of goroutines spinning up 😕
Gonna throw a limiter on it like the other scans
Temporarily set
ulimitto a larger value in linux, such as:This worked for me.