lucene: jenkins dump file traversal exceptions ("no matches found within 10000")

Description

These stack traces at the end of jenkins runs are super annoying:

...
Archiving artifacts
hudson.FilePath$ValidateAntFileMask$1Cancel
        at hudson.FilePath$ValidateAntFileMask$1.isCaseSensitive(FilePath.java:3300)
        at org.apache.tools.ant.DirectoryScanner.lambda$isIncluded$3(DirectoryScanner.java:1374)
        at java.base/java.util.stream.MatchOps$1MatchSink.accept(MatchOps.java:90)
        at java.base/java.util.Spliterators$ArraySpliterator.tryAdvance(Spliterators.java:958)
...
        at hudson.FilePath$ValidateAntFileMask.hasMatch(FilePath.java:3313)
Caused: hudson.FilePath$FileMaskNoMatchesFoundException: no matches found within 10000

This is a weird path traversal limit in jenkins (to dodge circular link structure, I guess) that is configurable via system properties -

https://www.jenkins.io/doc/book/managing/system-properties/#hudson-filepath-validate_ant_file_mask_bound

I don’t think we can tweak it but we can change the default path traversal patterns to be more restrictive:

No artifacts found that match the file pattern "**/*.events,heapdumps/**,**/hs_err_pid*". Configuration error?

The *.events is no longer used (used to be emitted from the custom runner in ant). I don’t think we can limit the scanning pattern to not exceed the limit but I think we can add a test finalization task to gradle that will scan only those folders that can contains hs_err_pid* files and perhaps copy them over to one top-level directory, then add a much simpler pattern to include anything present in that folder?

Version and environment details

No response

About this issue

Original URL
State: open
Created 7 months ago
Comments: 23 (23 by maintainers)

Most upvoted comments

Apache Infra has bumped the default for us to 60k. Let’s see if these stack traces still show up. https://issues.apache.org/jira/browse/INFRA-25269

dweiss on Dec 29, 2023

On Policeman Jenkins the stack traces are gone and the message, too @ Main, Mac, Win

uschindler on Dec 11, 2023

Looks like you can’t pass sysprops to individual workers. There are different types of workers (I use shell script launcher as this works better with Virtualbox), but the default launchers with SSH or JNDI can’t pass properties, only env args.

uschindler on Dec 11, 2023

I though changing the pattern to lucene/**/build/hs_err_pid* would help but there’s way too many files/folders in there so it’ll eventually hit that threshold anyway. Unless we set the threshold to MAX_VALUE, which then bypasses the entire “early exit” logic:

https://github.com/jenkinsci/jenkins/blob/f9a777bc682963de4640303b2f28ac488f9b93ef/core/src/main/java/hudson/FilePath.java#L3295-L3299

Now I see that there is even a hardcoded timeout condition placed in there if traversal bound is provided. So strange.

dweiss on Dec 11, 2023