cluster-api-provider-azure: conformance_test.go exit code to 0 event the non last test suite has failures
/kind bug
[Before submitting an issue, have you checked the Troubleshooting Guide?]
What steps did you take and what happened: [A clear and concise description of what the bug is.] ./conformance_test.go
What did you expect to happen: Normally there are 2 test suites (junit.e2e_suite.1.xml is the last) run. If the first one has failure but the second test suite (junit.e2e_suite.1.xml) passed. The exit code of conformance_test.go is 0. I expect the script exit with error to indicate there are failures in any of the test suites.
Anything else you would like to add: [Miscellaneous information that will assist in solving the issue.]
Environment:
- cluster-api-provider-azure version: 1.2
- Kubernetes version: (use
kubectl version): 1.23 - OS (e.g. from
/etc/os-release): ws2022
About this issue
- Original URL
- State: closed
- Created 2 years ago
- Comments: 32 (29 by maintainers)
I was able to find one where a conformance test case failed and the job actually failed: https://prow.k8s.io/view/gs/kubernetes-jenkins/logs/capz-conformance-master/1650489167919976448
So looks like this is now fixed
/close
@sbueringer that sounds like a lower-level explanation that would produce these outcomes.
Let’s keep this CAPZ issue open and move our practical energies towards fixing things to https://github.com/kubernetes-sigs/cluster-api/issues/5123