DependencyCheck: MavenCentral (504): Gateway Time-out
Describe the bug
While trying to scan a *.jar file using dependency-check in the most recent version (8.0.1) it tries to reach https://search.maven.org/solrsearch/select and gets a gateway timeout.
Version of dependency-check used 8.0.1 in the official docker image
Log file error.log
DEBUG - Searching Central url https://search.maven.org/solrsearch/select?q=1:2973d150c0dc1fefe998f834810d68f278ea58ec&wt=xml
2023-01-18 12:47:01,395 org.owasp.dependencycheck.analyzer.CentralAnalyzer:346
DEBUG - Could not connect to Central search (tries left: 6): Could not connect to MavenCentral (504): Gateway Time-out
To Reproduce
- Download
https://github.com/WebGoat/WebGoat/releases/download/7.1/webgoat-container-7.1-exec.jar - run
/usr/share/dependency-check/bin/dependency-check.sh -s webgoat-container-7.1-exec.jar -l /dev/stdout
Expected behavior It should finish the scan within some seconds or minutes.
About this issue
- Original URL
- State: open
- Created a year ago
- Reactions: 2
- Comments: 18 (1 by maintainers)
@ImSaSu PR #5849 is out now for supporting Nexus V3 in the NexusAnalyzer
Initial efforts have started, hope to spend some serious time to finish up the changes from next Thursday onward and expect that to result in a PR to add Nexusv3 as an alternative flavour of Sonatype Nexus support. At least the initial experiments make me optimistic in achieving a satisfiable result.
Regarding central search: the ‘new API’ does not have a public search API. The ‘new central search’ website uses private internal APIs.
It would be nice to have a parameter similar to “ossIndexRemoteErrorWarnOnly” (e.g., “mavenCentralRemoteErrorWarnOnly”), which can be set to true to avoid failing all the build due to this error.
Your remark got me to revisit Sonatype’s pages and looks like the search APIs allowing search by SHA are also available in Nexus 3 community. I intend to give it a spin on a local VM and see whether that’s correct and whether I can manage to build a Nexusv3Analyzer for it.
In any case the sharing of data directory is useful. Besides the CentralAnalyzer caches there’s also caching of the OSSIndexAnalyzer which can be beneficial to avoid hitting the OSSIndex rate-limits (unauthenticated is very restrictive in the limit, registered account might hit the higher rate-limits if you use it massively and don’t cache - https://ossindex.sonatype.org/doc/rest), NodeAudit results, RetireJS and the hostedSuppressions file.
@TheRealArlie changed it from bug to question, because there is not a bug in the application, but the application is suffering from the current availability issues of Maven Central’s search API. To my knowledge this is the first serious disturbance in Maven Central’s search availability and I hope they’ll be able to pinpoint the root-cause and put a resolution in place soon.
The reason that it is not recommended to disable it is that the DependencyCheck CLI has only limited information to go by (filename and some metadata in the zip). The chances of avoiding false-positive noise are significantly increased for libraries that are published in Maven Central (for which the pom.xml can be retrieved) as they allow to use the Maven coordinates (groupId, artifactId, version) in the evaluation of what libraries was scanned by DependencyCheck.
Disabling the CentralAnalyzer will increase the chance that libraries are matched to an incorrect ‘CPE’ (Common Product Enumeration) leading to vulnerabilities being reported that involve a different library (False positives). The CPE is the identifier for libraries in the NIST NVD (nvd.nist.gov) to link vulnerabilities(CVEs) to the vulnerable library versions.
For the time being disabling Central Analyzer seems the most workable approach. When Maven Central Search is back to stable you could consider re-enabling it to benefit from fewer false-positives.