DependencyCheck: Malformed \uxxxx encoding error on some python packages
Hello! Thank you very much for this tool! Please help to troubleshoot problem that i facing
Describe the bug On Some Python packages scanner returns error “[ERROR] Malformed \uxxxx encoding.” For example torchmetrics-0.7.1-py3-none-any.whl
Version of dependency-check used The problem occurs using version 6.4.1 of the cli
Log file ODC scan failed with 242.
Output: [WARN] dbPassword used on the command line, consider moving the password to a properties file using the key data.password and using the --propertyfile argument instead
[INFO]
Dependency-Check is an open source tool performing a best effort analysis of 3rd party dependencies; false positives and false negatives may exist in the analysis performed by the tool. Use of the tool and the reporting provided constitutes acceptance for use in an AS IS condition, and there are NO warranties, implied or otherwise, with regard to the analysis or its use. Any use of the tool and the reporting provided is at the user’s risk. In no event shall the copyright holder or OWASP be held liable for any damages whatsoever arising out of or in connection with the use of this tool, the analysis performed, or the resulting report.
About ODC: https://jeremylong.github.io/DependencyCheck/general/internals.html False Positives: https://jeremylong.github.io/DependencyCheck/general/suppression.html
💖 Sponsor: https://github.com/sponsors/jeremylong
[INFO] Analysis Started [INFO] Finished File Name Analyzer (0 seconds) [WARN] An unexpected error occurred during analysis of ‘/var/www/fossproxy/media/scan_files/torchmetrics-0.7.1-py3-none-any_uSCJKmJ.whl’ (Python Distribution Analyzer): Malformed \uxxxx encoding. [ERROR] java.lang.IllegalArgumentException: Malformed \uxxxx encoding. at java.util.Properties.loadConvert(Properties.java:574) at java.util.Properties.load0(Properties.java:391) at java.util.Properties.load(Properties.java:341) at org.owasp.dependencycheck.analyzer.PythonDistributionAnalyzer.getManifestProperties(PythonDistributionAnalyzer.java:376) at org.owasp.dependencycheck.analyzer.PythonDistributionAnalyzer.collectWheelMetadata(PythonDistributionAnalyzer.java:293) at org.owasp.dependencycheck.analyzer.PythonDistributionAnalyzer.collectMetadataFromArchiveFormat(PythonDistributionAnalyzer.java:232) at org.owasp.dependencycheck.analyzer.PythonDistributionAnalyzer.analyzeDependency(PythonDistributionAnalyzer.java:185) at org.owasp.dependencycheck.analyzer.AbstractAnalyzer.analyze(AbstractAnalyzer.java:131) at org.owasp.dependencycheck.AnalysisTask.call(AnalysisTask.java:88) at org.owasp.dependencycheck.AnalysisTask.call(AnalysisTask.java:37) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) [INFO] Finished Python Distribution Analyzer (0 seconds) [INFO] Finished Dependency Merging Analyzer (0 seconds) [INFO] Finished Version Filter Analyzer (0 seconds) [INFO] Finished Hint Analyzer (0 seconds) [INFO] Created CPE Index (0 seconds) [INFO] Finished NPM CPE Analyzer (0 seconds) [INFO] Created CPE Index (1 seconds) [INFO] Finished CPE Analyzer (1 seconds) [INFO] Finished False Positive Analyzer (0 seconds) [INFO] Finished NVD CVE Analyzer (0 seconds) [INFO] Finished Vulnerability Suppression Analyzer (0 seconds) [INFO] Finished Dependency Bundling Analyzer (0 seconds) [INFO] Analysis Complete (1 seconds) [INFO] Writing report to: /tmp/tmpefbnb52d/dependency-check-report.json [INFO] Writing report to: /tmp/tmpefbnb52d/dependency-check-report.html [ERROR] Malformed \uxxxx encoding.
Error output:
About this issue
- Original URL
- State: closed
- Created 2 years ago
- Comments: 20 (2 by maintainers)
Thanks for validating. I’ll promote the draft-PR to a regular PR so that it can be included in the next release. Unfortunately we just missed the cut-off for 7.0.1.
mvn verifywould’ve worked sufficiently as well, likely evenmvn package. The issue is that withmvn compilethere is no jar-file created for the dependency-check-utils submodule, so when the enforcer plugin tries to validate that all dependencies of the dependency-check-core module are compatible with Java8 it breaks because maven cannot hand it the location of the jar-file that represents the dependency-check-utils library. That jar-file is not created when running the compile phase of maven, it will be created by the package phase, which comes later. And there is no 7.0.1-SNAPSHOT vesion of the jar-file available in any of the configured repositories (which for a default maven install is only Maven Central, where only the release-versions of DependencyCheck libraries can be found in addition to the user-specific <user-home>/.m2/repository folder of locally installed libraries) nor among the ‘jar files created in the current build’ (known as the Maven Reactor). The latter is where maven would’ve been able to deliver the jar-file from for themvn verifyandmvn packagephases.The resulting CLI package you can find inside the cli/target folder as
dependency-check-7.0.1-SNAPSHOT-release.zipIt’s a bug in the metadataparsing of the PythonDistributionAnalyzer as that does not properly parse the METADATA
Python metadata according to the spec uses a format derived from email standards, with a prescribed encoding of UTF-8.
Parsing it with the java Properties class to get the key/value pairs is wrong in more than one way:
\url...which is not a unicode character reference, so the Properties class throws an illegalArgumentException as it expects any string starting with \u to represent a 4-digit unicode character reference.