gitleaks: GitLeaks throwing "too many open files" error when running with the --no-git option

Describe the bug When running the “gitleaks” command against a git repository that’s locally checked out on Mac OS X, it throws a “too many open files” error:

gitleaks --path=. -v --no-git
ERRO[0000] open .git/objects/06/c465fef0df46ca67970792e4a3b31695bd6498: too many open files
INFO[0000] scan time: 485 milliseconds 934 microseconds
ERRO[0000] open .git/objects/06/c465fef0df46ca67970792e4a3b31695bd6498: too many open files

The reason for running with the “–no-git” option is that I would like to only scan the current revision of the repository (all the files, not just the ones referenced by the latest commit).

The error seems to be being thrown due to GitLeaks scanning the “.git” repo in particular, which has a lot of files.

Other things I’ve tried to work around the above with no success are:

  • Running with “threads=1” to try and reduce the number of files that the program opens “gitleaks --path=. -v --threads=1 --no-git”
  • Moving the “.git” directory out of the checked out repository temporarily

To Reproduce On a machine running OS X 11.4, clone a large-ish Git repository.

Install gitleaks, change directory into the repository and run:

gitleaks --path=. -v --no-git

Expected behavior I would expect that gitleaks scans the directory without throwing an error and identifies any secrets in files as per normal.

Screenshots N/A

Basic Info (please complete the following information):

  • OS: Mac OS X 11.4
  • Gitleaks Version: 7.5.0

Additional context The OS X system max open file limits are set to the following (after increasing):

$ launchctl limit maxfiles
	maxfiles    65536          1048576

cc @zricethezav

About this issue

  • Original URL
  • State: closed
  • Created 3 years ago
  • Comments: 15 (1 by maintainers)

Most upvoted comments

@srkiNZ84

The error seems to be being thrown due to GitLeaks scanning the “.git” repo in particular, which has a lot of files.

Damn, true. Thanks for pointing that out. Gotta fix that

should be fixed in 7.6.0, feel free to reopen if you find it still broken

have upgraded, ran the command with “–no-git” that was consistently failing before and it is working now! Thank you so much! 🙂

@jboler @srkiNZ84 should be fixed in 7.6.0, feel free to reopen if you find it still broken

@jboler @srkiNZ84 found the issue, turns out I wasn’t limiting the number of goroutines spinning up 😕

Gonna throw a limiter on it like the other scans

Temporarily set ulimit to a larger value in linux, such as:

$ ulimit -n 524288

This worked for me.