zaproxy: Automation plan with failOnWarning: true still ends with exit code 0 if scan produces warnings

Describe the bug I expected the automation framework parameter failOnWarning with value true to cause the execution of a plan containing it to fail, if the scan results in any warnings. That does not seem to be the case. Upon running the plan below against a target that produces warnings, the exit code remains 0. Please excuse me, if I misunderstood the meaning of failOnWarning in the first place!

To Reproduce Run the following plan against a target that produces warnings:

env:
  contexts:
  - excludePaths: []
    name: baseline
    urls:
    - http://ci-demo-master
  parameters:
    failOnError: true
    failOnWarning: true
    progressToStdout: false
jobs:
- install:
  - pscanrulesBeta
  type: addOns
  uninstall: []
- parameters:
    enableTags: false
    maxAlertsPerRule: 10
  type: passiveScan-config
- parameters:
    maxDuration: 1
    url: http://ci-demo-master
  type: spider
- parameters:
    maxDuration: 0
  type: passiveScan-wait
- parameters:
    format: Long
    summaryFile: /tmp/zap_out.json
  rules: []
  type: outputSummary
- type: report
  parameters:
    reportDir: /tmp/testing/artifacts
    template: traditional-html
- type: report
  parameters:
    reportDir: /tmp/testing/artifacts
    template: traditional-json

Expected behavior The execution of the automation plan ends with an exit code != 0

Screenshots

Software versions

  • ZAP: 2.11.0
  • Add-on: None except the defaults and the ones required by the plan above
  • OS: Official zap2docker-stable container
  • Java: 11.0.11 according to ZAP output (Found Java version 11.0.11)
  • Browser: -

Errors from the zap.log file No errors, only INFO output in zap.log

Additional context

Would you like to help fix this issue? Gladly 👍

About this issue

  • Original URL
  • State: open
  • Created 3 years ago
  • Comments: 16 (10 by maintainers)

Most upvoted comments

Yeah, I get this too 😦 Will investigate further…