django-DefectDojo: Server 500 Internal Error randomly when reimport finding
Bug description I use dd-import (https://github.com/MaibornWolff/dd-import) to import finding include:
- dependency-check XML file
- CycloneDX sbom json file
- semgrep json file
But they cannot run for the whole process, they always throw 500 Internal Server Errors when upload one of them randomly.
Steps to reproduce
docker run -e "DD_URL=https://defect:8443" -e "DD_API_KEY=xxx" -e "DD_PRODUCT_TYPE_NAME=${CI_PROJECT_NAMESPACE}" -e "DD_PRODUCT_NAME=${CI_PROJECT_NAME}" -e "DD_ENGAGEMENT_NAME=Depscan" -e "DD_TEST_NAME=DepscanCI-${CI_COMMIT_BRANCH}" -e "DD_TEST_TYPE_NAME=CycloneDX Scan" -e "DD_SSL_VERIFY=0" -e "DD_FILE_NAME=sbom-universal.vdr.json" -v $MOUNT_NAME:/builds:rw -w /builds/$CI_PROJECT_NAMESPACE/$CI_PROJECT_NAME/reports docker.xxx.com/qweqq/dd-import:1.0.12 dd-reimport-findings.sh
docker run -e "DD_URL=https://defect:8443" -e "DD_API_KEY=xxx" -e "DD_PRODUCT_TYPE_NAME=${CI_PROJECT_NAMESPACE}" -e "DD_PRODUCT_NAME=${CI_PROJECT_NAME}" -e "DD_ENGAGEMENT_NAME=DependencyCheck" -e "DD_TEST_NAME=DependencyCheckCI-${CI_COMMIT_BRANCH}" -e "DD_TEST_TYPE_NAME=Dependency Check Scan" -e "DD_SSL_VERIFY=0" -e "DD_FILE_NAME=dependency-check-report.xml" -v $MOUNT_NAME:/builds:rw -w /builds/$CI_PROJECT_NAMESPACE/$CI_PROJECT_NAME/reports docker.xxx.com/qweqq/dd-import:1.0.12 dd-reimport-findings.sh
docker run -e "DD_URL=https://defect:8443" -e "DD_API_KEY=xxx" -e "DD_PRODUCT_TYPE_NAME=${CI_PROJECT_NAMESPACE}" -e "DD_PRODUCT_NAME=${CI_PROJECT_NAME}" -e "DD_ENGAGEMENT_NAME=SemgrepScan" -e "DD_TEST_NAME=SemgrepCI-${CI_COMMIT_BRANCH}" -e "DD_TEST_TYPE_NAME=Semgrep JSON Report" -e "DD_SSL_VERIFY=0" -e "DD_FILE_NAME=semgrepScan.json" -e "DD_SSL_VERIFY=0" -v $MOUNT_NAME:/builds:rw -w /builds/$CI_PROJECT_NAMESPACE/$CI_PROJECT_NAME/reports docker.xxx.com/qweqq/dd-import:1.0.12 dd-reimport-findings.sh
Expected behavior Output: Scan results imported
Deployment method (select with an X)
- Docker Compose
- Kubernetes
- GoDojo
Environment information
- Operating System: Ubuntu 20.04
- DefectDojo version: v. 2.30.4 ( release mode )
Logs
[pid: 26|app: -|req: -/-] 192.168.67.34 (-) {42 vars in 640 bytes} [Mon Feb 5 06:52:28 2024] GET /api/v2/product_types/?name=apidemo => generated 276 bytes in 181 msecs (HTTP/1.1 200) 8 headers in 246 bytes (1 switches on core 0)
[pid: 1|app: -|req: -/-] 192.168.67.34 (-) {42 vars in 670 bytes} [Mon Feb 5 06:52:28 2024] GET /api/v2/products/?name=frontend&prod_type=3 => generated 823 bytes in 184 msecs (HTTP/1.1 200) 8 headers in 246 bytes (1 switches on core 0)
[pid: 1|app: -|req: -/-] 192.168.67.34 (-) {42 vars in 654 bytes} [Mon Feb 5 06:52:29 2024] GET /api/v2/engagements/?name=Depscan&product=6 => generated 868 bytes in 389 msecs (HTTP/1.1 200) 8 headers in 246 bytes (1 switches on core 1)
[pid: 26|app: -|req: -/-] 192.168.67.34 (-) {42 vars in 712 bytes} [Mon Feb 5 06:52:29 2024] GET /api/v2/tests/?title=DepscanCI-sonarqube%2Fdependency-check&engagement=8 => generated 622 bytes in 330 msecs (HTTP/1.1 200) 8 headers in 246 bytes (1 switches on core 1)
[05/Feb/2024 06:52:30] ERROR [dojo.api_v2.exception_handler:36] 'vector'
Traceback (most recent call last):
File "/usr/local/lib/python3.11/site-packages/rest_framework/views.py", line 506, in dispatch
response = handler(request, *args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/rest_framework/mixins.py", line 19, in create
self.perform_create(serializer)
File "/app/dojo/api_v2/views.py", line 3238, in perform_create
serializer.save(push_to_jira=push_to_jira)
File "/app/dojo/api_v2/serializers.py", line 2502, in save
) = reimporter.reimport_scan(
^^^^^^^^^^^^^^^^^^^^^^^^^
File "/app/dojo/importers/reimporter/reimporter.py", line 609, in reimport_scan
parsed_findings = parser.get_findings(scan, test)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/app/dojo/tools/cyclonedx/parser.py", line 364, in get_findings
return self._get_findings_json(file, test)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/app/dojo/tools/cyclonedx/parser.py", line 444, in _get_findings_json
raw_vector = rating["vector"]
~~~~~~^^^^^^^^^^
KeyError: 'vector'
[05/Feb/2024 06:52:30] ERROR [django.request:241] Internal Server Error: /api/v2/reimport-scan/
About this issue
- Original URL
- State: closed
- Created 5 months ago
- Comments: 16 (9 by maintainers)
I uploaded the file now via REST API with python. It worked fine.
Can you close this issue @mtesauro ?
Hi @tientmse62290 , regarding dependencycheck: You have a .xml file, but the content is clearly JSON. So you try to upload JSON content here in a .xml file. I was able to reformat the content within the file with a json formatter. regarding Cyclonedx: I am investigating.