django-DefectDojo: Server 500 Internal Error randomly when reimport finding

Bug description I use dd-import (https://github.com/MaibornWolff/dd-import) to import finding include:

  • dependency-check XML file
  • CycloneDX sbom json file
  • semgrep json file

But they cannot run for the whole process, they always throw 500 Internal Server Errors when upload one of them randomly.

Steps to reproduce

docker run -e "DD_URL=https://defect:8443" -e "DD_API_KEY=xxx" -e "DD_PRODUCT_TYPE_NAME=${CI_PROJECT_NAMESPACE}" -e "DD_PRODUCT_NAME=${CI_PROJECT_NAME}" -e "DD_ENGAGEMENT_NAME=Depscan" -e "DD_TEST_NAME=DepscanCI-${CI_COMMIT_BRANCH}" -e "DD_TEST_TYPE_NAME=CycloneDX Scan" -e "DD_SSL_VERIFY=0" -e "DD_FILE_NAME=sbom-universal.vdr.json" -v $MOUNT_NAME:/builds:rw -w /builds/$CI_PROJECT_NAMESPACE/$CI_PROJECT_NAME/reports docker.xxx.com/qweqq/dd-import:1.0.12 dd-reimport-findings.sh

docker run -e "DD_URL=https://defect:8443" -e "DD_API_KEY=xxx" -e "DD_PRODUCT_TYPE_NAME=${CI_PROJECT_NAMESPACE}" -e "DD_PRODUCT_NAME=${CI_PROJECT_NAME}" -e "DD_ENGAGEMENT_NAME=DependencyCheck" -e "DD_TEST_NAME=DependencyCheckCI-${CI_COMMIT_BRANCH}" -e "DD_TEST_TYPE_NAME=Dependency Check Scan" -e "DD_SSL_VERIFY=0" -e "DD_FILE_NAME=dependency-check-report.xml" -v $MOUNT_NAME:/builds:rw -w /builds/$CI_PROJECT_NAMESPACE/$CI_PROJECT_NAME/reports docker.xxx.com/qweqq/dd-import:1.0.12 dd-reimport-findings.sh

docker run -e "DD_URL=https://defect:8443" -e "DD_API_KEY=xxx" -e "DD_PRODUCT_TYPE_NAME=${CI_PROJECT_NAMESPACE}" -e "DD_PRODUCT_NAME=${CI_PROJECT_NAME}" -e "DD_ENGAGEMENT_NAME=SemgrepScan" -e "DD_TEST_NAME=SemgrepCI-${CI_COMMIT_BRANCH}" -e "DD_TEST_TYPE_NAME=Semgrep JSON Report" -e "DD_SSL_VERIFY=0" -e "DD_FILE_NAME=semgrepScan.json" -e "DD_SSL_VERIFY=0" -v $MOUNT_NAME:/builds:rw -w /builds/$CI_PROJECT_NAMESPACE/$CI_PROJECT_NAME/reports docker.xxx.com/qweqq/dd-import:1.0.12 dd-reimport-findings.sh

Expected behavior Output: Scan results imported

Deployment method (select with an X)

  • Docker Compose
  • Kubernetes
  • GoDojo

Environment information

  • Operating System: Ubuntu 20.04
  • DefectDojo version: v. 2.30.4 ( release mode )

Logs

[pid: 26|app: -|req: -/-] 192.168.67.34 (-) {42 vars in 640 bytes} [Mon Feb  5 06:52:28 2024] GET /api/v2/product_types/?name=apidemo => generated 276 bytes in 181 msecs (HTTP/1.1 200) 8 headers in 246 bytes (1 switches on core 0)
[pid: 1|app: -|req: -/-] 192.168.67.34 (-) {42 vars in 670 bytes} [Mon Feb  5 06:52:28 2024] GET /api/v2/products/?name=frontend&prod_type=3 => generated 823 bytes in 184 msecs (HTTP/1.1 200) 8 headers in 246 bytes (1 switches on core 0)
[pid: 1|app: -|req: -/-] 192.168.67.34 (-) {42 vars in 654 bytes} [Mon Feb  5 06:52:29 2024] GET /api/v2/engagements/?name=Depscan&product=6 => generated 868 bytes in 389 msecs (HTTP/1.1 200) 8 headers in 246 bytes (1 switches on core 1)
[pid: 26|app: -|req: -/-] 192.168.67.34 (-) {42 vars in 712 bytes} [Mon Feb  5 06:52:29 2024] GET /api/v2/tests/?title=DepscanCI-sonarqube%2Fdependency-check&engagement=8 => generated 622 bytes in 330 msecs (HTTP/1.1 200) 8 headers in 246 bytes (1 switches on core 1)
[05/Feb/2024 06:52:30] ERROR [dojo.api_v2.exception_handler:36] 'vector'
Traceback (most recent call last):
  File "/usr/local/lib/python3.11/site-packages/rest_framework/views.py", line 506, in dispatch
    response = handler(request, *args, **kwargs)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/rest_framework/mixins.py", line 19, in create
    self.perform_create(serializer)
  File "/app/dojo/api_v2/views.py", line 3238, in perform_create
    serializer.save(push_to_jira=push_to_jira)
  File "/app/dojo/api_v2/serializers.py", line 2502, in save
    ) = reimporter.reimport_scan(
        ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/app/dojo/importers/reimporter/reimporter.py", line 609, in reimport_scan
    parsed_findings = parser.get_findings(scan, test)
                      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/app/dojo/tools/cyclonedx/parser.py", line 364, in get_findings
    return self._get_findings_json(file, test)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/app/dojo/tools/cyclonedx/parser.py", line 444, in _get_findings_json
    raw_vector = rating["vector"]
                 ~~~~~~^^^^^^^^^^
KeyError: 'vector'
[05/Feb/2024 06:52:30] ERROR [django.request:241] Internal Server Error: /api/v2/reimport-scan/

About this issue

  • Original URL
  • State: closed
  • Created 5 months ago
  • Comments: 16 (9 by maintainers)

Most upvoted comments

I uploaded the file now via REST API with python. It worked fine.

nginx-1         | 172.18.0.1 - - [08/Feb/2024:07:31:42 +0000] "POST /api/v2/reimport-scan/ HTTP/1.1" 201 5116 "-" "DefectDojo_api/1.2.0." "-"
uwsgi-1         | [pid: 30|app: -|req: -/-] 172.18.0.1 (-) {42 vars in 705 bytes} [Thu Feb  8 07:31:42 2024] POST /api/v2/reimport-scan/ => generated 5116 bytes in 158 msecs (HTTP/1.1 201) 8 headers in 241 bytes (1 switches on core 1)

Can you close this issue @mtesauro ?

Hi @tientmse62290 , regarding dependencycheck: You have a .xml file, but the content is clearly JSON. So you try to upload JSON content here in a .xml file. I was able to reformat the content within the file with a json formatter. regarding Cyclonedx: I am investigating.