pact-jvm: When publishing verification results, only the errors are sent

When publishing verification results, only the errors are being sent. This causes the Pact broker to see the passing interactions as un-verified.

For a test with 3 interactions, if two pass and one fails, the following entries are logged:

Received test result 'au.com.dius.pact.core.pactbroker.TestResult$Ok@3f693160' for Pact pactflow-example-provider-springboot-pactflow-example-consumer and a request to get a product (Pact Broker https://testdemo.pactflow.io/pacts/provider/pactflow-example-provider-springboot/consumer/pactflow-example-consumer/pact-version/4a6c2017e4f913c5b18ee0b526eae842567bf489)
2020-12-08 12:27:12.818 DEBUG 33876 --- [    Test worker] a.c.d.p.p.DefaultTestResultAccumulator   : Number of interactions #3 and results: [au.com.dius.pact.core.pactbroker.TestResult$Ok@3f693160]
2020-12-08 12:27:12.818  WARN 33876 --- [    Test worker] a.c.d.p.p.DefaultTestResultAccumulator   : Not all of the 3 were verified. The following were missing:
2020-12-08 12:27:12.818  WARN 33876 --- [    Test worker] a.c.d.p.p.DefaultTestResultAccumulator   :     a request to get a product
2020-12-08 12:27:12.818  WARN 33876 --- [    Test worker] a.c.d.p.p.DefaultTestResultAccumulator   :     a request to get all products
2020-12-08 12:27:13.029 DEBUG 33876 --- [    Test worker] a.c.d.p.p.DefaultTestResultAccumulator   : Received test result 'au.com.dius.pact.core.pactbroker.TestResult$Ok@3f693160' for Pact pactflow-example-provider-springboot-pactflow-example-consumer and a request to get all products (Pact Broker https://testdemo.pactflow.io/pacts/provider/pactflow-example-provider-springboot/consumer/pactflow-example-consumer/pact-version/4a6c2017e4f913c5b18ee0b526eae842567bf489)
2020-12-08 12:27:13.030 DEBUG 33876 --- [    Test worker] a.c.d.p.p.DefaultTestResultAccumulator   : Number of interactions #3 and results: [au.com.dius.pact.core.pactbroker.TestResult$Ok@3f693160, Failed(results=[{interactionId=ba08b56e3fecb8a49c7f7a2d509d9fe3183a44cf, attribute=status, description=expected status of 404 but was 200}], description=Response status did not match), au.com.dius.pact.core.pactbroker.TestResult$Ok@3f693160]
2020-12-08 12:27:13.030 DEBUG 33876 --- [    Test worker] a.c.d.p.p.DefaultTestResultAccumulator   : All interactions for Pact pactflow-example-provider-springboot-pactflow-example-consumer have a verification result

Then when the results are published:

2020-12-08 12:27:13.447 DEBUG 33876 --- [    Test worker] a.c.d.p.c.pactbroker.PactBrokerClient    : Test result = Failed(results=[{interactionId=ba08b56e3fecb8a49c7f7a2d509d9fe3183a44cf, attribute=status, description=expected status of 404 but was 200}], description=Response status did not match)
2020-12-08 12:27:13.448 DEBUG 33876 --- [    Test worker] a.c.dius.pact.core.pactbroker.HalClient  : Posting JSON to https://testdemo.pactflow.io/pacts/provider/pactflow-example-provider-springboot/consumer/pactflow-example-consumer/pact-version/4a6c2017e4f913c5b18ee0b526eae842567bf489/verification-results
{"providerApplicationVersion":"cdbbf3f+1607390821","success":false,"testResults":[{"interactionId":"ba08b56e3fecb8a49c7f7a2d509d9fe3183a44cf","mismatches":[{"attribute":"status","description":"expected status of 404 but was 200"}],"success":false}]}

Only the error is sent.

About this issue

  • Original URL
  • State: closed
  • Created 4 years ago
  • Comments: 22 (12 by maintainers)

Commits related to this issue

Most upvoted comments

I can confirm the verification status is not unknown anymore! thanks!

@mefellows When verifications fail, can-i-deploy also fails. That works as expected.

@Rafaesp I’ve released 4.1.19 which hopefully resolves your issue

@mefellows Done! With Pact 4.1.14: https://pastebin.com/cTqBdx4V With Pact 4.1.6: https://pastebin.com/94m9g70D Thank you