DependencyCheck: DatabaseException: Error updating 'CVE-2020-36569'

Raised a PR which could be a potential fix for the issue : https://github.com/jeremylong/DependencyCheck/pull/5221 Please review the PR.

Instead of arguing if one or the other approach is correct, acknowledge that a lot of people are impacted by this situation, that appears quite simple to solve, and there is even a PR for it. I suggest time is better spent fixing it…

To the very long list of people commenting your builds are broken – you should not be using this plugin to block your builds!

Think about it – if one of your dependencies is marked vulnerable, now you can’t fix any bugs with your software, or even test it, until you patch or ignore the CVE! It’s crazy.

You should be running this (or similar) on a regular basis, e.g. nightly, and take the output seriously. But making it block your builds is actually a massive security hole in your system, as you are (re)-discovering over and over, since bad actors (or negligent OSS fuzzers) can file wrong or spurious CVEs with common dependencies and block builds all over the world. You’re basically allowing strangers on the internet to block your internal processes whenever they want, for any or no reason.

Maybe it is worth to add a protection for such kind of problems in the future? Meaning, - skipping erroneous CVEs or at least not failing further db inserts?

I disagree. Not blocking the build would allow CVEs like log4shell in your software, with the risk of being automatically deployed. Over the past years wrong or spurious CVEs (at least for me) have not been a problem at all. Occasionally there were false positives but those cases can easily be solved with the suppression.xml

Same problem here, this bug breaks our build pipelines. So a fix asap is highly appreciated.

To the very long list of people commenting your builds are broken – you should not be using this plugin to block your builds!

Think about it – if one of your dependencies is marked vulnerable, now you can’t fix any bugs with your software, or even test it, until you patch or ignore the CVE! It’s crazy.

You should be running this (or similar) on a regular basis, e.g. nightly, and take the output seriously. But making it block your builds is actually a massive security hole in your system, as you are (re)-discovering over and over, since bad actors (or negligent OSS fuzzers) can file wrong or spurious CVEs with common dependencies and block builds all over the world. You’re basically allowing strangers on the internet to block your internal processes whenever they want, for any or no reason.

You seem to misunderstand the problem. It’s not that anyone attempts to block their builds by using dependecy check report. Maybe there are some people who do this, certainly not me or the majority of reporters.

It’s that the plugin breaks the build because internal DB is using much too restrictive limits on externally provided values, which is not the expected behavior.

@karstenspang @mlemmens I think you folks are right, the issue could be with CVE database, I have also reported an issue with https://cveform.mitre.org/

Is it possible to get the 7.4.4 within a gradle plugin?

Isn’t it good to have some automatic test before changing the CVE database and breaking thousands of projects all over the world?

I know it’s a question more to the direction of CVE, but anyway, this behaviour is to be avoided.

will this be released after merging? this currently breaks all our builds an we are wondering if we need to disable CVE scanning in general or if it is feasible to wait?

(and btw kudos @srini00088 for the speed of the fix, when I discovered the bug this morning the PR was already underway)

Such issues are very critical for enterprise companies. It started to made us thought should we use dependency-check now? How can we make sure that such issues won’t exist in future?

I think NVD is in error with the version string.

CVE-2020-36569:

Authentication is globally bypassed in github.com/nanobox-io/golang-nanoauth between v0.0.0-20160722212129-ac0cc4484ad4 and v0.0.0-20200131131040-063a3fb69896 if ListenAndServe is called with an empty token.

0.0.0-20160722212129-ac0cc4484ad4_before_v0.0.0-20200131131040-063a3fb69896 is a incorrect version.

The gradle plugin was also just pushed.

For all suffering from it: if you use a persistent folder across builds (you do, don’t you???) you can temporarily work around this issue by disabling the auto-update, which means dependency-check will only run against the already collected set of vulnerabilities and not discover new ones.

In any case I would recommend a setup where your build uses the CVE database as is (autoupdate disabled) and you run a dedicated scheduled job to perform the update of the CVE-database in the datadirectory from the NVD datastreams (update-only goal of maven-plugin / update Task of gradle plugin)

Running into this issue, can we please have an ETA on the fix?

@ericgoffster see the linked PR (#5221)… integration and a release is to be expected later today

And it’s not mitre, but NIST NVD team that needs to correct the entry. The CVE is correct, the linked CPE in NIST NVD is having an incorrect version stated (and it’s a bug in DependencyCheck to break on it while NIST NVD allows the large version value)

How many CVEs are fixed by checking the size of an input before doing something with it?

It’s that the plugin breaks the build because internal DB is using much too restrictive limits on externally provided values, which is not the expected behavior.

60 characters in that strange a limit to a version-bound that should contain a single unique version of a software component. I presume you’d rather have us claim 2GB of your harddrive for an insane 2GB version number?

To the very long list of people commenting your builds are broken – you should not be using this plugin to block your builds!

Think about it – if one of your dependencies is marked vulnerable, now you can’t fix any bugs with your software, or even test it, until you patch or ignore the CVE! It’s crazy.

You should be running this (or similar) on a regular basis, e.g. nightly, and take the output seriously. But making it block your builds is actually a massive security hole in your system, as you are (re)-discovering over and over, since bad actors (or negligent OSS fuzzers) can file wrong or spurious CVEs with common dependencies and block builds all over the world. You’re basically allowing strangers on the internet to block your internal processes whenever they want, for any or no reason.

To the very long list of people commenting your builds are broken – you should not be using this plugin to block your builds!

Think about it – if one of your dependencies is marked vulnerable, now you can’t fix any bugs with your software, or even test it, until you patch or ignore the CVE! It’s crazy.

You should be running this (or similar) on a regular basis, e.g. nightly, and take the output seriously. But making it block your builds is actually a massive security hole in your system, as you are (re)-discovering over and over, since bad actors (or negligent OSS fuzzers) can file wrong or spurious CVEs with common dependencies and block builds all over the world. You’re basically allowing strangers on the internet to block your internal processes whenever they want, for any or no reason.

Fyi, failBuildOnAnyVulnerability is by default false…the issue here happens before the scan

I am having the same issue here when upgrading to v 7.4.3

This is completely unrelated to versions. Any version will run into it when attempting to update with the latest additions to the NIST NVD

For Jenkins pipeline users I recommend to wrap the dependency check execution in a catchError block and have it in a separate stage. We have done that sometime ago as it’s not the first time a CVE DB update breaks. That way builds continue and only the stage for the dependency check is marked as failed.

stage('Analyze dependencies') {
  catchError(buildResult: 'UNSTABLE', catchInterruptions: false, message: 'OWASP dependency check failed', stageResult: 'FAILURE') {
    sh 'mvn ...' 
  }
}

@keithboone as noted in the PR… the ETA is ‘later today’. This is a small volunteer project of people that also have a day-job, so we do our best, but we cannot just ‘drop all work and work on a fix’ you’d have to resort to the paid security scanners for that. The tools all have a flag that skips the entire update, so if you have a local cached CVE database (which you should have, to avoid the constant recreation) you can avoid the issue by temporarily enabling the don’t update my CVE DB flag of the tool that you use.

i think this ticket can be closed

I think the problem with Gradle was solved, I’m doing analysis of different versions and it gives me successful results.

@Dhanxy Likely multiple versions work (as the database version is sufficiently compatible), but you will need 7.4.4 to run the upgrade-script and your safest bet would be to be in-sync on the version used). Not sure whether that’s easy to arrange within a Gradle environment. I’m more familiar with the maven environment where I advise (and practice) to use a property for the version in the pom.xml (set to a default value and updated to latest release during regular maintenance) which then gets a system- or user-wide override by that same property configured in the maven settings.xml, so that scans can use an updated version of the plugin without a need to first change the pom.xml (maven’s counterpart of the build.gradle)

What most want (including me) is an ETA for a fix. If you cannot provide it, can you tell us where to get it from?

I would also suggest that extending the column width is at best a temporary patch. I’d propose that what should happen is that a single incorrect row in the input shouldn’t cause dependency check to fail, instead, it should report a warning that the database is malformed.

In regard to blocking builds, many who aren’t using dependencycheck to block a build are still having build failures. Dependency check produces a report for our builds via mvn, a failed dependency check does NOT block our builds.
However, a failed maven plugin that generates an exception does block the build. Maven is reporting a build failure because the plugin generated an exception, not a failure because dependencycheck reported a vulnerability.

builds are failing due to [ERROR] org.owasp.dependencycheck.data.nvdcve.DatabaseException: Unable to retrieve id for new vulnerability for ‘CVE-2022-32923’ org.owasp.dependencycheck.data.update.exception.UpdateException: org.owasp.dependencycheck.data.nvdcve.DatabaseException: Unable to retrieve id for new vulnerability for ‘CVE-2022-32923’

FWIW 7.4.4 fixed the CLI issue for me - tytyty and have a rad weekend! 🍻

Looks like a corrupted H2 database to me at first sight. You could try a one-off build that uses org.owasp:dependency-check-maven:7.4.4:purge org.owasp:dependency-check-maven:7.4.4:aggregate to clear out the data and recreate it (and then revert back to just the aggregate goal) (I know maven-plugin of 7.4.4 can complete successfully as I’ve done successful updates of the CVEDB with maven in our CI infra)

Appreciate the quick turn around. Does the Jenkins Plugin (v5.2.0) cache the available versions to install?

will the CLI update itself or do I need to pull a fresh copy when it’s available?

It is available (linked on the releases), but you’ll have to download and unpack it yourself

Any idea when its going to hit the Maven Repo https://mvnrepository.com/artifact/org.owasp/dependency-check-maven

It’s already available: https://repo1.maven.org/maven2/org/owasp/dependency-check-maven/7.4.4/ mvnrepository.com and search.maven.org don’t index new artifacts immediately, but that doesn’t mean the new version isn’t ready to be downloaded.

The codebase was using v6.0.3 – are you saying that I did not need to upgrade it to pick up the new fix? I realized we’re pretty far behind the latest version and noted that v7.x.x would be a breaking change that requires the purge. I attempted to update the version locally first, and the build failed until I added the purge goal.

You need to upgrade, only 7.4.4 will work.

If codebase was still on v6.0.3 than ODC would IIRC provision a new data directory with a 7.x compatible database.

If that breaks on an unusable database a one-off purge should suffice

no need to purge when upgrading. Upgrading will just work out-of-the-box. The purging was for Dnu12 who wanted to run with a manually self-patched version of 7.4.3.

When you run an analysis with ODC in its default configuration it will patch the database and succesfully update.

edit: and to answer your question on how often: if you add it the plugin assumes that you want it so it will purge everytime that you run it and purge your data, so typically you only want to run a purge (if ever) as a one-off action to cleanup a corrupted data directory.

@jeremylong @aikebah et al – ty for your alacrity in resolving this issue. I’m sure everyone here will join me in saying we appreciate it and all’y’all for everything you’re doing 😃

To the very long list of people commenting your builds are broken – you should not be using this plugin to block your builds! Think about it – if one of your dependencies is marked vulnerable, now you can’t fix any bugs with your software, or even test it, until you patch or ignore the CVE! It’s crazy. You should be running this (or similar) on a regular basis, e.g. nightly, and take the output seriously. But making it block your builds is actually a massive security hole in your system, as you are (re)-discovering over and over, since bad actors (or negligent OSS fuzzers) can file wrong or spurious CVEs with common dependencies and block builds all over the world. You’re basically allowing strangers on the internet to block your internal processes whenever they want, for any or no reason.

To the very long list of people commenting your builds are broken – you should not be using this plugin to block your builds! Think about it – if one of your dependencies is marked vulnerable, now you can’t fix any bugs with your software, or even test it, until you patch or ignore the CVE! It’s crazy. You should be running this (or similar) on a regular basis, e.g. nightly, and take the output seriously. But making it block your builds is actually a massive security hole in your system, as you are (re)-discovering over and over, since bad actors (or negligent OSS fuzzers) can file wrong or spurious CVEs with common dependencies and block builds all over the world. You’re basically allowing strangers on the internet to block your internal processes whenever they want, for any or no reason.

Fyi, failBuildOnAnyVulnerability is by default false…the issue here happens before the scan

you should not be using this plugin to block your builds! - I think that is a choice, maybe you want to have that as you a conservative approach as you are very security aware and prefer to have no build that one with any possible security concern. Sure you need processes in place to quickly resolve false positives. We prefer to have a DevSecOps tightly integrated into our build so it is detected immediately. Yes we allow strangers on the internet to develop software we depend on (that is the nature of open source), and some bad things can happen so we have processes in place to manage that.

If you are suffering from this problem, it is not good and due to you not having the level of control you need. This is a bug and they happen.

Best regards to the hard working team that manage this plugin! Great job!

I increased column length to 90 in initialize.sql and upgrade_5.1.sql for columns “versionEndExcluding VARCHAR(90), versionEndIncluding VARCHAR(90),versionStartExcluding VARCHAR(90), versionStartIncluding VARCHAR(90)” and updated it in jars .But on running again getting same error .Running it via terminal .Any help on how to increase the length and run it via command line ? @aikebah

If you patched your local jars you should be able to purge the database and get it to work (but it will need purging as it will not upgrade your database, which is already upgraded to the expected version, so you need to force it to be recreated)

But how do I purge the database? Do we internally purge the old db and create new one on running ‘sh dependency-check.sh’.I also couldn’t find where h2 db is created .( I don’t have anything(such as data) inside .m2 repo )

For the CLI using your invocation that would be the command sequence sh dependencycheck.sh --purge to purge it followed by sh dependencycheck.sh --updateonly or a regular scan to trigger the recreation

By default the H2 database file gets created in userhome/.m2/repository/org/owasp/dependency-check-data/7.0 directory with name odc.mv.db We can access this database file using H2 console and run command to modify the column length. I followed below step:

1. Downloaded the H2 database zip from https://github.com/h2database/h2database/releases/download/version-2.1.214/h2-2022-06-13.zip
2. Extracted it and from bin folder ran command ‘java -cp h2-2.1.214.jar org.h2.tools.Shell’
3. It opened H2 shell
4. Entered below details in the shell: URL- jdbc:h2:userhome.m2\repository\org\owasp\dependency-check-data\7.0\odc Driver- org.h2.Driver User- dcuser Password- DC-Pass1337!
5. Now in sql prompr run command ‘ALTER TABLE software ALTER COLUMN versionEndExcluding SET DATA TYPE VARCHAR(100);’

NOTE: userhome is user’s home directory

I couldn’t find ‘dependency-check-data’ folder inside my user home .m2/repo . I ran ODC via command line.Is there any other place where we store the h2 db files.

I increased column length to 90 in initialize.sql and upgrade_5.1.sql for columns “versionEndExcluding VARCHAR(90), versionEndIncluding VARCHAR(90),versionStartExcluding VARCHAR(90), versionStartIncluding VARCHAR(90)” and updated it in jars .But on running again getting same error .Running it via terminal .Any help on how to increase the length and run it via command line ? @aikebah

If you patched your local jars you should be able to purge the database and get it to work (but it will need purging as it will not upgrade your database, which is already upgraded to the expected version, so you need to force it to be recreated)

I hit the same problem today.

Most of the information is here so I won’t repeat it I just want to add that I met this issue on 7.1.3 version then upgraded to latest 7.4.3 and the problem persists.

the same question, I’m using “dependency-check-gradle” … is there a way to say “ignore failed/crashed” plugin? or how to avoid in further this issue?

Short question: I’m using “dependency-check-maven” … is there a way to say “ignore failed/crashed” plugin?

We use this ${cve.check.skip} which allows us to turn off the plugin by using this Maven parameter -Dcve.check.skip=false. Add this as a skip tag in the configuration