DependencyCheck: New year bug

Describe the bug dependency-check cli is failing without the nvd metadata file for the new year

[ERROR] Unable to download meta file: https://nvd.nist.gov/feeds/json/cve/1.1/nvdcve-1.1-2020.meta; received 404 -- resource not found
org.owasp.dependencycheck.data.update.exception.UpdateException: Unable to download meta file: https://nvd.nist.gov/feeds/json/cve/1.1/nvdcve-1.1-2020.meta; received 404 -- resource not found
	at org.owasp.dependencycheck.data.update.NvdCveUpdater.getMetaFile(NvdCveUpdater.java:351)
	at org.owasp.dependencycheck.data.update.NvdCveUpdater.getUpdatesNeeded(NvdCveUpdater.java:398)
	at org.owasp.dependencycheck.data.update.NvdCveUpdater.update(NvdCveUpdater.java:122)
	at org.owasp.dependencycheck.Engine.doUpdates(Engine.java:922)
	at org.owasp.dependencycheck.Engine.initializeAndUpdateDatabase(Engine.java:723)
	at org.owasp.dependencycheck.Engine.analyzeDependencies(Engine.java:653)
	at org.owasp.dependencycheck.App.runScan(App.java:252)
	at org.owasp.dependencycheck.App.run(App.java:184)
	at org.owasp.dependencycheck.App.main(App.java:80)
Caused by: org.owasp.dependencycheck.utils.ResourceNotFoundException: Requested resource does not exists - received a 404
	at org.owasp.dependencycheck.utils.HttpResourceConnection.obtainConnection(HttpResourceConnection.java:204)
	at org.owasp.dependencycheck.utils.HttpResourceConnection.fetch(HttpResourceConnection.java:138)
	at org.owasp.dependencycheck.utils.Downloader.fetchContent(Downloader.java:110)
	at org.owasp.dependencycheck.data.update.NvdCveUpdater.getMetaFile(NvdCveUpdater.java:340)

** Version of dependency-check used ** The problem occurs using version 5.2.4 of the the cli (cli, gradle plugin, maven plugin, etc.)

** Log file ** When reporting errors, 99% of the time log file output is required. Please post the log file as a gist and provide a link in the new issue.

To Reproduce Steps to reproduce the behavior: Run a dependency check scan on Jan 1 of a new year.

Expected behavior Tool should continue without the metadata and cve feed for the new year.

Additional context Add any other context about the problem here.

About this issue

  • Original URL
  • State: closed
  • Created 5 years ago
  • Reactions: 69
  • Comments: 37 (5 by maintainers)

Commits related to this issue

Most upvoted comments

The files for 2020 have just been released: https://nvd.nist.gov/vuln/data-feeds#JSON_FEED That said: Everything should be working again for you. 😎

This is open source people, stop demanding a release, a fix has been commited, a release will be made when Jeremy is ready for it. You can compile it on your own if you need it now. If I was Jeremy I would not rush to release as more changes have been made then just this small fix. If you want to have a 5.2.4 with just this patch you can take the source code from my fork (or preferably make your own) https://github.com/vdotjansen/DependencyCheck/tree/5.2.4-patched

As I expected - the NVD 2020 files would be published before we could complete testing on the next release. There is one more important update - so we are going to hold off publishing 5.3.0 until the CPE-Parser PR#30 is merged and released.

Happy new year!

happy new year

The easiest, temporary fix seems to be setting the “–cveUrlBase” parameter (https://jeremylong.github.io/DependencyCheck/dependency-check-cli/arguments.html) to a hardcoded year until the fix/update has happened.

waiting too

hoping this will be fixed soon

The easiest temporary solution is to do nothing and wait for the updated code to publish to maven and/or NMD to update their data file. It requires no action, and a little bit of patience.

The second easiest temporary solution if you need a one-time action is to temporarily change your computer’s clock time to a date in 2019.

The third easiest temporary solution if you need automated CI actions is disabling this plugin until it’s updated or NVD updates the data feed.

The fourth easiest solution isn’t easy but involves using the nist-data-mirror plugin. I’m going to think about this one for a few days and expect the first solution to work first. 😉

facing similar issue, hope it will get fix as soon as possible.

waiting for updates

facing similar issue, waiting for its fix

Upvote for a patch release

Any ETA on when this will be released?

The files for 2020 have just been released: https://nvd.nist.gov/vuln/data-feeds#JSON_FEED That said: Everything should be working again for you. 😎

It’s working from me now many thanks max

Just out of curiosity: I cannot find a comparable new year’s problem in the closed issues. Has this never occurred? Did NIST always provide empty files for the next year early on and did not this time?

Because last year the resolution of the CVE DB was switched from XML to JSON feed which in turn required some rewrite of the download code. Thats why this issue popped up these days the first time. 😉

If you use mirrored data, you could temporarily duplicate your 2019 files with 2020 names.

https://nvd.nist.gov/vuln/data-feeds#JSON_FEED

^^ This does say meta file is unavailable for CVE 2020 feed.

Seems it would be a more robust solution to attempt year - 1 on failure.

image

@Mercious Thanks for that hint. Worked locally and with Travis CI and JitCi, although GitHub action Java CI had a problem inserting records in the internal database:

Caused by: org.h2.jdbc.JdbcSQLException: Unique index or primary key violation: "CONSTRAINT_INDEX_7 ON PUBLIC.VULNERABILITY(CVE) VALUES ('CVE-2019-5877', 7171)"; SQL statement:
INSERT INTO vulnerability (cve, description, cvssV2Score, cvssV2AccessVector, cvssV2AccessComplexity, cvssV2Authentication, cvssV2ConfidentialityImpact, cvssV2IntegrityImpact, cvssV2AvailabilityImpact, cvssV2Severity, cvssV3AttackVector, cvssV3AttackComplexity, cvssV3PrivilegesRequired, cvssV3UserInteraction, cvssV3Scope, cvssV3ConfidentialityImpact, cvssV3IntegrityImpact, cvssV3AvailabilityImpact, cvssV3BaseScore, cvssV3BaseSeverity) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?) [23505-196]
	at org.h2.message.DbException.getJdbcSQLException(DbException.java:345)
	at org.h2.message.DbException.get(DbException.java:179)
	at org.h2.message.DbException.get(DbException.java:155)
	at org.h2.index.BaseIndex.getDuplicateKeyException(BaseIndex.java:103)
	at org.h2.mvstore.db.MVSecondaryIndex.checkUnique(MVSecondaryIndex.java:231)
	at org.h2.mvstore.db.MVSecondaryIndex.add(MVSecondaryIndex.java:190)
	at org.h2.mvstore.db.MVTable.addRow(MVTable.java:705)
	at org.h2.command.dml.Insert.insertRows(Insert.java:156)
	at org.h2.command.dml.Insert.update(Insert.java:114)
	at org.h2.command.CommandContainer.update(CommandContainer.java:101)
	at org.h2.command.Command.executeUpdate(Command.java:260)
	at org.h2.jdbc.JdbcPreparedStatement.execute(JdbcPreparedStatement.java:207)
	at org.owasp.dependencycheck.data.nvdcve.CveDB.updateVulnerabilityInsertVulnerability(CveDB.java:1081)
	... 10 more
There was an error attempting to close the CveDB, see the log for more details.
Unable to update 1 or more Cached Web DataSource, using local data instead. Results may not include recent vulnerabilities.

Just out of curiosity: I cannot find a comparable new year’s problem in the closed issues. Has this never occurred? Did NIST always provide empty files for the next year early on and did not this time?

@Aaqib041 there are no config options that can bypass this, other than ‘skip’ and friends. Workarounds are changing your system clock, or establishing a mirror and duplicating 2019 files. Or, wait… I expect this will all be resolved within a day or less.

@dbwiddis Thank You

Yeah, changing the system clock to some date in 2019 works.