prism: [Bug] 422 Response Not Being Used For Validation Response
Describe the bug
Hello! I could really use some help and guidance!
I have an Open API spec that is valid when I used Spectral
I can successfully run it with Prism
I have unit tests that validate an endpoint, so I am using Prism like so
prism proxy --errors example.yaml http://localhost:8080
When I send an invalid request it keeps giving me this error
{
type: 'https://stoplight.io/prism/errors#UNPROCESSABLE_ENTITY',
title: 'Invalid request',
status: 422,
detail: 'Your request is not valid and no HTTP validation response was found in the spec, so Prism is generating this error for you.',
validation: [
{
location: [Array],
severity: 'Error',
code: 'required',
message: "should have required property 'firstname'"
},
{
location: [Array],
severity: 'Error',
code: 'required',
message: "should have required property 'lastname'"
},
{
location: [Array],
severity: 'Error',
code: 'required',
message: "should have required property 'email'"
},
{
location: [Array],
severity: 'Error',
code: 'required',
message: "should have required property 'password'"
}
]
}
Which is wild because this DOES exist.
It keeps stating Your request is not valid and no HTTP validation response was found in the spec, so Prism is generating this error for you which is not true at all because I have a 422 response declared for the endpoint that’s being hit!
I have tried setting a 400 response. I’ve tried setting different data for the response. And nothing is working.
What am I doing wrong?
To Reproduce
Use this file
---
openapi: '3.0.2'
servers:
- url: http://localhost:8080
description: Local deployment from your computer
paths:
/api/users/register:
post:
summary: 'Register New User'
description: 'Register a new user'
operationId: 'registerNewUser'
tags: ['Users']
requestBody:
required: true
content:
application/json:
schema:
type: 'object'
properties:
firstname:
type: 'string'
minLength: 2
maxLength: 100
example: 'John'
lastname:
type: 'string'
minLength: 2
maxLength: 100
example: 'Roberts'
email:
type: 'string'
minLength: 4
maxLength: 100
example: 'john@example.com'
password:
type: 'string'
minLength: 8
maxLength: 50
example: 'abc123456'
required:
- firstname
- lastname
- email
- password
responses:
'201':
description: Successful registration of a new user
'422':
description: Failed to register new user
content:
application/json:
schema:
type: 'object'
properties:
message:
type: 'string'
example: 'firstname is required'
Run prism proxy --errors example.yaml http://localhost:8080
Run this curl command to confirm it works
curl --location --request POST 'localhost:4010/api/users/register' \
--header 'Content-Type: application/json' \
--data-raw '{
"firstname": "First",
"lastname": "Last",
"email": "example@example.com",
"password": "This_is_4_str0ng_password!"
}'
If you want pretty printing you can add json_pp
curl --location --request POST 'localhost:4010/api/users/register' \
--header 'Content-Type: application/json' \
--data-raw '{
"firstname": "First",
"lastname": "Last",
"email": "example@example.com",
"password": "This_is_4_str0ng_password!"
}'
Run this command to confirm it fails
curl --location --request POST 'localhost:4010/api/users/register' \
--header 'Content-Type: application/json' \
--data-raw '{}'
Same, but with pretty printing
curl --location --request POST 'localhost:4010/api/users/register' \
--header 'Content-Type: application/json' \
--data-raw '{}' | json_pp
And the final output is this
~/Repositories/example feature/api_spec_and_test_reconciliation ± ⬡ v12.17.0 curl --location --request POST 'localhost:4010/api/users/register' \
--header 'Content-Type: application/json' \
--data-raw '{}' | json_pp
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 702 100 700 100 2 113k 333 --:--:-- --:--:-- --:--:-- 114k
{
"detail" : "Your request is not valid and no HTTP validation response was found in the spec, so Prism is generating this error for you.",
"title" : "Invalid request",
"type" : "https://stoplight.io/prism/errors#UNPROCESSABLE_ENTITY",
"status" : 422,
"validation" : [
{
"severity" : "Error",
"location" : [
"body"
],
"message" : "should have required property 'firstname'",
"code" : "required"
},
{
"message" : "should have required property 'lastname'",
"location" : [
"body"
],
"severity" : "Error",
"code" : "required"
},
{
"code" : "required",
"severity" : "Error",
"location" : [
"body"
],
"message" : "should have required property 'email'"
},
{
"message" : "should have required property 'password'",
"location" : [
"body"
],
"severity" : "Error",
"code" : "required"
}
]
}
Expected behavior
I expect the responses I created for 422 to be used so that I don’t have to worry about Prism returning different payloads than what my server does.
Questions
How do I get Prism to not return a generated 422 override?
Because it keeps generating that validation response I can’t integrate it with my tests, because the tests expect a message property to be there.
I’ve read through the documentation here:
- https://meta.stoplight.io/docs/prism/docs/guides/errors.md#unprocessable_entity
- https://meta.stoplight.io/docs/prism/docs/guides/02-request-validation.md#request-validation
And none of it provides an explicit example on how to make it so that Prism does not return an auto-generated response.
Additional context
- Prism: 4.1.2
- OS: MacOS
About this issue
- Original URL
- State: closed
- Created 3 years ago
- Comments: 22 (7 by maintainers)
Hello @chohmann!
I really wish there was a higher bandwith form of communication we could use 😂 I think we may be getting our wires crossed together and I would love the opportunity to understand the thought process as well as contrast it with the goals I’m trying to achieve.
RE:
--errorsFlagI want to use
prismin my CI/CD in order to call out drift between myspecandimplementation. I need this on a spiritual level, and on an organizational level 😂With the
--errorsflag the CI/CD will fail becausePrismwill return an error code when thespecdoes not match therequestorresponsepayload. I want this featureMy understanding is that if I drop the
--errorsflag that will not cause an error to be generated and instead will just be receiving the exactresponsefrom my server back, which is fine. But not what I’m looking for.What I want is
prismto allow opting out of therequestvalidation in order to allow the opportunity for theresponseto be validated against thespecBEFORE I receive the response back and run my validations on it.In this situation there would be a
responseerror generated just like therequest error.What I would like it look like is this
Request->Prism->ServerResponse->Prism->TestPrismscreams about aresponseandspecmismatchRequest->Prism->ServerResponse->Prism->TestResponsevalidation is performed, but no errors were found, so original server response is returnedIn this situation I can rely on
Prismto fail when the spec doesn’t match and succeed when it does. And then, I get an addition layer of validation that’s easy to layer on top of existing tests without having to rewrite anything, because it will checkReiterating Desired Feature
I would love love love it if
Prismgenerated errors when theresponsepayload does not match theresponsein thespecification.In order to get to this step, I NEED
Prismto allow the ability to skiprequest validationotherwise it generates a response saying therequest is invalid, which is intended.I see in your
Prism Clientdocuments here, that this type of logic is already supported https://meta.stoplight.io/docs/prism/docs/guides/http-client.md#override-the-configuration-object-on-the-request-levelHOWEVER, I do not want to code this in the
Prism Clientbecause I already have tests set up and configured. What I do want though, is the ability to pepper in something like a header to skiprequestValidationI want to have
Prismas an intermediate layer that I spin up and run requests/responses through in my CI/CDIdeally, just like the
Preferheader you all have provided, it would be radical if you had something likeRequestValidation: FalseResponseValidation: FalseFor example - See line 4
Which would then let me simply change my tests to add headers in order to skip the steps.
In this way, I can integrate
Prismon top of my tests for free AND I don’t have to write any of my tests over again. I just addheadersto indicate the validation I wantPrismto scream about.By dropping the
--errorsflag I am missing the best part aboutPrismwhich is the screaming about aspecmismatch. That is why I am so excited about the tool -Contract validation.PreferHeader Usage@chohmann Can you please clarify the issues I was having with the
Preferkeyword andexampleas well as combining the usage ofcodeandexampleOr, would it be better to open a separate ticket for that issue?
@loganknecht I’m going to try to answer all of your questions.
This is possible, just leave off the
--errorsflag and the request will be forwarded to the actual server. Prism will also still validate the request matches the spec and log out any validation errors.Here’s how I verified this use case:
so now I have my “actual” server running at http://localhost:4090
--errorsflag to make sure the requests get forwarded to our actual serverour prism proxy now accepts request at http://localhost:4010 and forwards them to http://localhost:4090
The response we get is the one returned by our actual server:
And looking at the logs of the prism proxy, you can see it logged out validation errors the request had:
Based on the use case you described, I don’t think Prism will add any value. You want to hit the actual server, receive the actual response, and then validate the string like
expect(response).toBe('message defined in spec'). You can do that without prism proxy, you’ll just have to keep the test code up to date with what response to expect.Hi @loganknecht,
I was taking a closer look at this and realized you’re using prism in
proxymode, and not inmockmode. When inproxymode a request that does not match the schema will either be forwarded to the existing api server, or prism will blow up the request based on the use of the--errorsflag. It will not return a response defined in the spec inproxymode. More information onproxymode can be found here.The error message is absolutely misleading when read in the context of
proxymode. The error object is shared across bothmockandproxymodes, but is worded primarily for themockcontext. We could look into making this error message more clear when used in theproxycontext, but probably won’t be able to do so in the short term.If you run prism in
mockmode, the example defined for the422response will be returned if an invalid request is sent to prism:I hope that answers your questions!