pact-jvm: When publishing verification results, only the errors are sent
When publishing verification results, only the errors are being sent. This causes the Pact broker to see the passing interactions as un-verified.
For a test with 3 interactions, if two pass and one fails, the following entries are logged:
Received test result 'au.com.dius.pact.core.pactbroker.TestResult$Ok@3f693160' for Pact pactflow-example-provider-springboot-pactflow-example-consumer and a request to get a product (Pact Broker https://testdemo.pactflow.io/pacts/provider/pactflow-example-provider-springboot/consumer/pactflow-example-consumer/pact-version/4a6c2017e4f913c5b18ee0b526eae842567bf489)
2020-12-08 12:27:12.818 DEBUG 33876 --- [ Test worker] a.c.d.p.p.DefaultTestResultAccumulator : Number of interactions #3 and results: [au.com.dius.pact.core.pactbroker.TestResult$Ok@3f693160]
2020-12-08 12:27:12.818 WARN 33876 --- [ Test worker] a.c.d.p.p.DefaultTestResultAccumulator : Not all of the 3 were verified. The following were missing:
2020-12-08 12:27:12.818 WARN 33876 --- [ Test worker] a.c.d.p.p.DefaultTestResultAccumulator : a request to get a product
2020-12-08 12:27:12.818 WARN 33876 --- [ Test worker] a.c.d.p.p.DefaultTestResultAccumulator : a request to get all products
2020-12-08 12:27:13.029 DEBUG 33876 --- [ Test worker] a.c.d.p.p.DefaultTestResultAccumulator : Received test result 'au.com.dius.pact.core.pactbroker.TestResult$Ok@3f693160' for Pact pactflow-example-provider-springboot-pactflow-example-consumer and a request to get all products (Pact Broker https://testdemo.pactflow.io/pacts/provider/pactflow-example-provider-springboot/consumer/pactflow-example-consumer/pact-version/4a6c2017e4f913c5b18ee0b526eae842567bf489)
2020-12-08 12:27:13.030 DEBUG 33876 --- [ Test worker] a.c.d.p.p.DefaultTestResultAccumulator : Number of interactions #3 and results: [au.com.dius.pact.core.pactbroker.TestResult$Ok@3f693160, Failed(results=[{interactionId=ba08b56e3fecb8a49c7f7a2d509d9fe3183a44cf, attribute=status, description=expected status of 404 but was 200}], description=Response status did not match), au.com.dius.pact.core.pactbroker.TestResult$Ok@3f693160]
2020-12-08 12:27:13.030 DEBUG 33876 --- [ Test worker] a.c.d.p.p.DefaultTestResultAccumulator : All interactions for Pact pactflow-example-provider-springboot-pactflow-example-consumer have a verification result
Then when the results are published:
2020-12-08 12:27:13.447 DEBUG 33876 --- [ Test worker] a.c.d.p.c.pactbroker.PactBrokerClient : Test result = Failed(results=[{interactionId=ba08b56e3fecb8a49c7f7a2d509d9fe3183a44cf, attribute=status, description=expected status of 404 but was 200}], description=Response status did not match)
2020-12-08 12:27:13.448 DEBUG 33876 --- [ Test worker] a.c.dius.pact.core.pactbroker.HalClient : Posting JSON to https://testdemo.pactflow.io/pacts/provider/pactflow-example-provider-springboot/consumer/pactflow-example-consumer/pact-version/4a6c2017e4f913c5b18ee0b526eae842567bf489/verification-results
{"providerApplicationVersion":"cdbbf3f+1607390821","success":false,"testResults":[{"interactionId":"ba08b56e3fecb8a49c7f7a2d509d9fe3183a44cf","mismatches":[{"attribute":"status","description":"expected status of 404 but was 200"}],"success":false}]}
Only the error is sent.
About this issue
- Original URL
- State: closed
- Created 4 years ago
- Comments: 22 (12 by maintainers)
Commits related to this issue
- refactor: change verification OK result from a singleton to a data class #1266 — committed to pact-foundation/pact-jvm by deleted user 4 years ago
- refactor: verification OK data class only needs a single interaction id #1266 — committed to pact-foundation/pact-jvm by deleted user 4 years ago
- refactor: include successful interaction results in the verification results #1266 — committed to pact-foundation/pact-jvm by deleted user 4 years ago
- fix: when merging verification results, include the interaction ID from the success results #1266 — committed to pact-foundation/pact-jvm by deleted user 4 years ago
- fix: record successful verification results by interaction ID #1266 — committed to pact-foundation/pact-jvm by deleted user 4 years ago
- fix: successful verification results were being filtered out #1266 — committed to pact-foundation/pact-jvm by deleted user 4 years ago
- chore: add test for results with both a pending and non-pending pact #1266 — committed to pact-foundation/pact-jvm by deleted user 3 years ago
- fix: use the Pact URL to disambiguate Pacts when accumilating results #1266 — committed to pact-foundation/pact-jvm by deleted user 3 years ago
- chore: add test for results with both a pending and non-pending pact #1266 — committed to pact-foundation/pact-jvm by deleted user 3 years ago
- fix: use the Pact URL to disambiguate Pacts when accumilating results #1266 — committed to pact-foundation/pact-jvm by deleted user 3 years ago
I can confirm the verification status is not unknown anymore! thanks!
@mefellows When verifications fail, can-i-deploy also fails. That works as expected.
@Rafaesp I’ve released 4.1.19 which hopefully resolves your issue
@mefellows Done! With Pact 4.1.14: https://pastebin.com/cTqBdx4V With Pact 4.1.6: https://pastebin.com/94m9g70D Thank you