DependencyCheck: Cannot complete initial NVD download - 503 status
Describe the bug The default configuration, with an API key, is either making requests too quickly, or not retrying enough, or both. It always eventually fails with a 503 error from NVD.
Version of dependency-check used Maven plugin 9.0.0
Log file
[INFO] --- dependency-check:9.0.0:check (default-cli) @ test ---
[INFO] Checking for updates
[INFO] NVD API has 171,358 records in this update
[INFO] Downloaded 20,000/171,358 (12%)
[ERROR] Error updating the NVD Data
org.owasp.dependencycheck.data.update.exception.UpdateException: Error updating the NVD Data
at org.owasp.dependencycheck.data.update.NvdApiDataSource.processApi (NvdApiDataSource.java:336)
at org.owasp.dependencycheck.data.update.NvdApiDataSource.update (NvdApiDataSource.java:110)
at org.owasp.dependencycheck.Engine.doUpdates (Engine.java:902)
at org.owasp.dependencycheck.Engine.initializeAndUpdateDatabase (Engine.java:707)
at org.owasp.dependencycheck.Engine.analyzeDependencies (Engine.java:633)
at org.owasp.dependencycheck.maven.BaseDependencyCheckMojo.runCheck (BaseDependencyCheckMojo.java:1936)
at org.owasp.dependencycheck.maven.BaseDependencyCheckMojo.execute (BaseDependencyCheckMojo.java:1119)
at org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo (DefaultBuildPluginManager.java:126)
at org.apache.maven.lifecycle.internal.MojoExecutor.doExecute2 (MojoExecutor.java:328)
at org.apache.maven.lifecycle.internal.MojoExecutor.doExecute (MojoExecutor.java:316)
at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:212)
at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:174)
at org.apache.maven.lifecycle.internal.MojoExecutor.access$000 (MojoExecutor.java:75)
at org.apache.maven.lifecycle.internal.MojoExecutor$1.run (MojoExecutor.java:162)
at org.apache.maven.plugin.DefaultMojosExecutionStrategy.execute (DefaultMojosExecutionStrategy.java:39)
at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:159)
at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject (LifecycleModuleBuilder.java:105)
at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject (LifecycleModuleBuilder.java:73)
at org.apache.maven.lifecycle.internal.builder.singlethreaded.SingleThreadedBuilder.build (SingleThreadedBuilder.java:53)
at org.apache.maven.lifecycle.internal.LifecycleStarter.execute (LifecycleStarter.java:118)
at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:261)
at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:173)
at org.apache.maven.DefaultMaven.execute (DefaultMaven.java:101)
at org.apache.maven.cli.MavenCli.execute (MavenCli.java:906)
at org.apache.maven.cli.MavenCli.doMain (MavenCli.java:283)
at org.apache.maven.cli.MavenCli.main (MavenCli.java:206)
at jdk.internal.reflect.NativeMethodAccessorImpl.invoke0 (Native Method)
at jdk.internal.reflect.NativeMethodAccessorImpl.invoke (NativeMethodAccessorImpl.java:62)
at jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke (DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke (Method.java:566)
at org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced (Launcher.java:283)
at org.codehaus.plexus.classworlds.launcher.Launcher.launch (Launcher.java:226)
at org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode (Launcher.java:407)
at org.codehaus.plexus.classworlds.launcher.Launcher.main (Launcher.java:348)
at jdk.internal.reflect.NativeMethodAccessorImpl.invoke0 (Native Method)
at jdk.internal.reflect.NativeMethodAccessorImpl.invoke (NativeMethodAccessorImpl.java:62)
at jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke (DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke (Method.java:566)
at org.apache.maven.wrapper.BootstrapMainStarter.start (BootstrapMainStarter.java:52)
at org.apache.maven.wrapper.WrapperExecutor.execute (WrapperExecutor.java:161)
at org.apache.maven.wrapper.MavenWrapperMain.main (MavenWrapperMain.java:73)
Caused by: io.github.jeremylong.openvulnerability.client.nvd.NvdApiException: NVD Returned Status Code: 503
at io.github.jeremylong.openvulnerability.client.nvd.NvdCveClient.next (NvdCveClient.java:327)
at org.owasp.dependencycheck.data.update.NvdApiDataSource.processApi (NvdApiDataSource.java:315)
at org.owasp.dependencycheck.data.update.NvdApiDataSource.update (NvdApiDataSource.java:110)
at org.owasp.dependencycheck.Engine.doUpdates (Engine.java:902)
at org.owasp.dependencycheck.Engine.initializeAndUpdateDatabase (Engine.java:707)
at org.owasp.dependencycheck.Engine.analyzeDependencies (Engine.java:633)
at org.owasp.dependencycheck.maven.BaseDependencyCheckMojo.runCheck (BaseDependencyCheckMojo.java:1936)
at org.owasp.dependencycheck.maven.BaseDependencyCheckMojo.execute (BaseDependencyCheckMojo.java:1119)
at org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo (DefaultBuildPluginManager.java:126)
at org.apache.maven.lifecycle.internal.MojoExecutor.doExecute2 (MojoExecutor.java:328)
at org.apache.maven.lifecycle.internal.MojoExecutor.doExecute (MojoExecutor.java:316)
at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:212)
at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:174)
at org.apache.maven.lifecycle.internal.MojoExecutor.access$000 (MojoExecutor.java:75)
at org.apache.maven.lifecycle.internal.MojoExecutor$1.run (MojoExecutor.java:162)
at org.apache.maven.plugin.DefaultMojosExecutionStrategy.execute (DefaultMojosExecutionStrategy.java:39)
at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:159)
at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject (LifecycleModuleBuilder.java:105)
at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject (LifecycleModuleBuilder.java:73)
at org.apache.maven.lifecycle.internal.builder.singlethreaded.SingleThreadedBuilder.build (SingleThreadedBuilder.java:53)
at org.apache.maven.lifecycle.internal.LifecycleStarter.execute (LifecycleStarter.java:118)
at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:261)
at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:173)
at org.apache.maven.DefaultMaven.execute (DefaultMaven.java:101)
at org.apache.maven.cli.MavenCli.execute (MavenCli.java:906)
at org.apache.maven.cli.MavenCli.doMain (MavenCli.java:283)
at org.apache.maven.cli.MavenCli.main (MavenCli.java:206)
at jdk.internal.reflect.NativeMethodAccessorImpl.invoke0 (Native Method)
at jdk.internal.reflect.NativeMethodAccessorImpl.invoke (NativeMethodAccessorImpl.java:62)
at jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke (DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke (Method.java:566)
at org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced (Launcher.java:283)
at org.codehaus.plexus.classworlds.launcher.Launcher.launch (Launcher.java:226)
at org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode (Launcher.java:407)
at org.codehaus.plexus.classworlds.launcher.Launcher.main (Launcher.java:348)
at jdk.internal.reflect.NativeMethodAccessorImpl.invoke0 (Native Method)
at jdk.internal.reflect.NativeMethodAccessorImpl.invoke (NativeMethodAccessorImpl.java:62)
at jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke (DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke (Method.java:566)
at org.apache.maven.wrapper.BootstrapMainStarter.start (BootstrapMainStarter.java:52)
at org.apache.maven.wrapper.WrapperExecutor.execute (WrapperExecutor.java:161)
at org.apache.maven.wrapper.MavenWrapperMain.main (MavenWrapperMain.java:73)
[INFO] Skipping RetireJS update since last update was within 24 hours.
[INFO] Skipping Hosted Suppressions file update since last update was within 2 hours.
[INFO] Skipping Known Exploited Vulnerabilities update check since last check was within 24 hours.
[WARNING] Unable to update 1 or more Cached Web DataSource, using local data instead. Results may not include recent vulnerabilities.
[ERROR] Unable to continue dependency-check analysis.
[ERROR] Fatal exception(s) analyzing Test
[INFO] Cache event queue destroyed: CacheEventQueue [listenerId=-135459835, cacheName=NODEAUDIT]
[INFO] Cache event queue destroyed: CacheEventQueue [listenerId=-135459835, cacheName=CENTRAL]
[INFO] Cache event queue destroyed: CacheEventQueue [listenerId=-135459835, cacheName=POM]
[ERROR] Region [NODEAUDIT] : Not alive and dispose was called, filename: NODEAUDIT
[ERROR] Region [CENTRAL] : Not alive and dispose was called, filename: CENTRAL
[ERROR] Region [POM] : Not alive and dispose was called, filename: POM
The debug log in this case is a nightmare as it logs every raw request and response, 16 bytes at a time, without synchronisation so they’re all misordered. I’m not going through that to sanitise my API key.
To Reproduce
mvn dependency-check:check
Expected behavior Successful completion of the download.
About this issue
- Original URL
- State: closed
- Created 7 months ago
- Reactions: 21
- Comments: 112 (27 by maintainers)
Just so everyone knows - it appears the NVD has put a temporary block on all requests that use a virtualMatchString of “cpe:2.3:a” - which is what ODC is using. I have a dialog going with maintainers of the API.
Folks, a whole lot of ‘me too’ without additional information probably isn’t really helpful. If there are server issues on NIST NVD side, it will affect all ODC variants in similar ways with 503s in the error messages.
Stay on 8.x until the server issues are resolved or suggested mitigations are discovered.
There’s a subscribe button if you want to see updates on an GitHub issue, no need to post messages.
Same for me.
@chadlwilson thanks!
While not ideal, updating the open-vulnerability-client#90 appears to allow a successful download of the NVD data from the API under the current load. It did take ~30 minutes to create the cache. I’ll be testing this directly in ODC and hopefully release a new version with a few fixes this weekend.
Isn’t working for me since two days. I got new API key and I get
They also seems to have closed th API Key deliverance.
Also - due to an unfortunate issue with H2, if anyone has succesfully created a db with 9.0.0 - it will likely need to be re-created with 9.0.1. I should be able to finish development and testing tomorrow morning.
Can confirm it now retries all the 503s, but then dies on a 504.
Fails with 9.0.1 after 20 minutes:
-> https://github.com/jeremylong/Open-Vulnerability-Project/pull/98
9.0.1 was just released. Due to an unfortunate issue with H2 and the gradle plugin - we had to revert the upgrade of H2. This means that anyone that successfully created an H2 database with 9.0.0 will likely need to purge and recreate the database with 9.0.1. The good news is that 9.0.1 now has retry logic built in and we’ve been able to successfully download the entire dataset from the NVD API.
Notes:
We are also getting the same error, we are using the dependency check in our azure devops pipeline via azure devops extension. https://marketplace.visualstudio.com/items?itemName=dependency-check.dependencycheck
task:
error:
[ERROR] Error updating the NVD Data org.owasp.dependencycheck.data.update.exception.UpdateException: Error updating the NVD Data at org.owasp.dependencycheck.data.update.NvdApiDataSource.processApi(NvdApiDataSource.java:336) at org.owasp.dependencycheck.data.update.NvdApiDataSource.update(NvdApiDataSource.java:110) at org.owasp.dependencycheck.Engine.doUpdates(Engine.java:902) at org.owasp.dependencycheck.Engine.initializeAndUpdateDatabase(Engine.java:707) at org.owasp.dependencycheck.Engine.analyzeDependencies(Engine.java:633) at org.owasp.dependencycheck.App.runScan(App.java:260) at org.owasp.dependencycheck.App.run(App.java:192) at org.owasp.dependencycheck.App.main(App.java:87) Caused by: io.github.jeremylong.openvulnerability.client.nvd.NvdApiException: NVD Returned Status Code: 503 at io.github.jeremylong.openvulnerability.client.nvd.NvdCveClient.next(NvdCveClient.java:327) at org.owasp.dependencycheck.data.update.NvdApiDataSource.processApi(NvdApiDataSource.java:315) … 7 common frames omitted
Yeah even with DT and API key I’m getting rate limited to no end.
If that is the outcome then it’d be a win for everyone 😆 DC adopting the NVD REST API must be one of the most brutal load tests you can get! 9.0 was only released today so I’m assuming the situation will not get better as folks proceed with upgrading.
yup… apparently the NVD API is under load… maybe they’ll delay the retirement of the data feeds…
Hello everyone, issue still occurs:
Thanks
@ferben version 8.x doesn’t use the NVD API but the NVD data feed. You can’t really compare them, they probably don’t even use the same servers.
I am getting below error while using Azure devops extension task[https://marketplace.visualstudio.com/items?itemName=dependency-check.dependencycheck]
[WARN] NVD API request failures are occurring; retrying request for the 5 time [INFO] NVD API has 171,546 records in this update [WARN] NVD API request failures are occurring; retrying request for the 5 time [WARN] NVD API request failures are occurring; retrying request for the 5 time [WARN] NVD API request failures are occurring; retrying request for the 5 time [WARN] NVD API request failures are occurring; retrying request for the 5 time [WARN] NVD API request failures are occurring; retrying request for the 6 time [WARN] NVD API request failures are occurring; retrying request for the 6 time [WARN] NVD API request failures are occurring; retrying request for the 6 time [WARN] NVD API request failures are occurring; retrying request for the 6 time [ERROR] Error updating the NVD Data org.owasp.dependencycheck.data.update.exception.UpdateException: Error updating the NVD Data at org.owasp.dependencycheck.data.update.NvdApiDataSource.processApi(NvdApiDataSource.java:338) at org.owasp.dependencycheck.data.update.NvdApiDataSource.update(NvdApiDataSource.java:110) at org.owasp.dependencycheck.Engine.doUpdates(Engine.java:904) at org.owasp.dependencycheck.Engine.initializeAndUpdateDatabase(Engine.java:709) at org.owasp.dependencycheck.Engine.analyzeDependencies(Engine.java:635) at org.owasp.dependencycheck.App.runScan(App.java:261) at org.owasp.dependencycheck.App.run(App.java:193) at org.owasp.dependencycheck.App.main(App.java:88) Caused by: io.github.jeremylong.openvulnerability.client.nvd.NvdApiException: NVD Returned Status Code: 502 at io.github.jeremylong.openvulnerability.client.nvd.NvdCveClient.next(NvdCveClient.java:346) at org.owasp.dependencycheck.data.update.NvdApiDataSource.processApi(NvdApiDataSource.java:317) … 7 common frames omitted
Due to the way common API gateways behave, generally I would say one should retry on all 5xx errors (other arguably than 500) - but especially 502 (bad gateway) and 504 (gateway timeout). Both tend to indicate ephemeral problems.
Currently the code only does 429 and 503 I believe.
Hi @jeremylong ,
Sorry, if I re-use this closed issue, but it’s not working for me after multiple times tried:
I saw the retry it’s works as expected, but I’m getting
504status code and it didn’t do any retry after that 😞@somera you don’t need to increase the delay - that won’t really help. The issue is that a retry is needed - which will be in 9.0.1.
Experiencing this issue too, posting to stay in the loop of updates
[Update] It’s working now for me with
9.0.2ThanksCan you elaborate as to why this was closed? I see no solution and am experiencing the same issue as mentioned in the ticket.
@Maxouwell I have just generated an API key, it seems they have enabled that back. Just FYI
The remaining issues should be resolved with #6151, I’m waiting for it to be merged. @jeremylong when are you planning to release the next version?
@Laurens-makel - there is no need to post to subscribe and get updates on an issue. On the right side of the issue screen there is a Notifications option with a subscribe/unsubscribe button.
Can you please clarify this: When I use --nvdApiKey option I use automatically NVD API calls → Understand. When I not use --nvdApiKey i use NVD API too? If yes, why -> dependency-check can work until NVD switch off Data feeds (NVD says 15 Dec, but if API servers is not stable they should postpone this deadline) without problems and after Data feeds would go offline users can switch to NVD API adding one option into command line.
OK, so I’ pray that NVD stabilize API servers availability ASAP.
The logs state an API key was NOT provided.
@ahammoudeh96 we do not maintain the Jenkins plugin. I would suggest opening a ticket on their repo.
I think it’s entirely luck/timing. I managed to get an initial DB population run completed a few hours ago with the default delays.
@jeremylong could you please upload your local cache somewhere? it would help me, and probably others if we just have to download the diff to update our local databases
or if anyone else who succeeded with maven could zip up
.m2\repository\org\owasp\dependency-check-data\9.0folder and make it available… this would be appreciatedThe only way I was able to get past the 503 errors was to add retry logic to the NvdCveClient. Ideally the NVD API would add a Retry-After header with the 503 response, but they don’t. https://github.com/jeremylong/Open-Vulnerability-Project/pull/88
We also experience the issue.
I am using version 8.4.3 wit api key and i also get the 503 https error
@jeremylong a couple of questions:
Given I’m using a shared (Postgres) database which was fully up-to-date before I switched from 8.4.3 to 9.0.0, how come Dependency-Check reports
NVD API has 171,457 records in this update- surely what I actually need to update is much much smaller than that?Might it be feasible for Dependency-Check to write to the DB in the case where it fails part way through consuming from the NVD API? That is, where a shared database is in use, why does run 2 still report the same number of records (171,457) as run 1, when (and I admit I’m guessing here) run 1 might have been able to download some updates before falling over in a heap due to the API unavailability?
@ferben the point is that the NVD data feed is being removed on December 15th.
Confirmed! Version 8.4.3 (without support of nvdApiKey) works, so there must be a issue in implementation #5978.
I am having the same issue with both the command line (v9.0.0) and Azure DevOps plug-in. I get the 503 errors when testing on the command line regardless of whether I have specified an API key or not.
Reverting to v8.4.0 completes an update check and allows the completion of a scan.
@chadlwilson Yep. Ran the test curl -H “Accept: application/json” -H “apiKey: ########-####-####-####-############” -v https://services.nvd.nist.gov/rest/json/cves/2.0\?cpeName\=cpe:2.3:o:microsoft:windows_10:1607:\*:\*:\*:\*:\*:\*:\*
The key was bad. Requested and new one, and the second one they sent me works. Looks like they have some issues to workout with this new API.
I’m getting404s now (while using an API key) - wonder if it’s related or something else bad in the API data or dep check’s use of it.Edit: seems this was an invalid API key somehow, see https://github.com/jeremylong/DependencyCheck/issues/6107#issuecomment-1823967207
Can you try increasing the delay? For the CLI it would be
--nvdApiDelay 16000?