schemathesis: [BUG] "FailedHealthCheck : 50 filtered examples but only 0 good ones." Schemathesis not producing tests for a valid open API yaml file

Checklist

Describe the bug Error in all endpoints when testing a simple openAPI/swaggerUI specification file.

To Reproduce Steps to reproduce the behavior:

  1. Run this command
st run --base-url=http://localhost -H "Content-Type: application/json" -H "Authorization: Bearer $ACCESS_TOKEN" --validate-schema=true ./api/openapi/http.yml
  1. See error :-
================================================================================ Schemathesis test session starts ================================================================================
Schema location: file:///home/aryan/mainflux/api/openapi/http.yml
Base URL: http://localhost
Specification version: Open API 3.0.1
Workers: 1
Collected API operations: 2

POST /channels/{id}/messages E                                                                                                                                                              [ 50%]
GET /health E                                                                                                                                                                               [100%]

============================================================================================= ERRORS =============================================================================================
__________________________________________________________________________________ POST /channels/{id}/messages __________________________________________________________________________________
hypothesis.errors.FailedHealthCheck: It looks like your strategy is filtering out a lot of data. Health check found 50 filtered examples but only 0 good ones. This will make your tests much slower, and also will probably distort the data generation quite a lot. You should adapt your strategy to filter less. This can also be caused by a low max_leaves parameter in recursive() calls
See https://hypothesis.readthedocs.io/en/latest/healthchecks.html for more information about this. If you want to disable just this health check, add HealthCheck.filter_too_much to the suppress_health_check settings for this test.

__________________________________________________________________________________________ GET /health ___________________________________________________________________________________________
hypothesis.errors.Unsatisfiable: Unable to satisfy schema parameters for this API operation

Add this option to your command line parameters to see full tracebacks: --show-errors-tracebacks

If you need assistance in solving this, feel free to join our Discord server and report it:

  https://discord.gg/R9ASRAmHnA

============================================================================================ SUMMARY =============================================================================================

No checks were performed.

Hint: You can visualize test results in Schemathesis.io by using `--report` in your CLI command.

If possible, please post a minimal version of your API schema that cause this behavior:

openapi: 3.0.1
info:
  title: Mainflux http adapter
  description: |
    HTTP API for sending messages through communication channels.
    Some useful links:
    - [The Mainflux repository](https://github.com/mainflux/mainflux)
  contact:
    email: info@mainflux.com
  license:
    name: Apache 2.0
    url: https://github.com/mainflux/mainflux/blob/master/LICENSE
  version: 0.14.0

servers:
  - url: http://localhost:8008
  - url: https://localhost:8008

tags:
  - name: messages
    description: Everything about your Messages
    externalDocs:
      description: Find out more about messages
      url: http://docs.mainflux.io/
      
paths:
  /channels/{id}/messages:
    post:
      summary: Sends message to the communication channel
      description: |
        Sends message to the communication channel. Messages can be sent as
        JSON formatted SenML or as blob.
      tags:
        - messages
      parameters:
        - $ref: "#/components/parameters/ID"
      requestBody:
        $ref: "#/components/requestBodies/MessageReq"
      responses:
        "202":
          description: Message is accepted for processing.
        "400":
          description: Message discarded due to its malformed content.
        "401":
          description: Missing or invalid access token provided.
        "404":
          description: Message discarded due to invalid channel id.
        "415":
          description: Message discarded due to invalid or missing content type.
        '500':
          $ref: "#/components/responses/ServiceError"
  /health:
    get:
      summary: Retrieves service health check info.
      tags:
        - health
      responses:
        '200':
          $ref: "#/components/responses/HealthRes"
        '500':
          $ref: "#/components/responses/ServiceError"

components:
  schemas:
    SenMLRecord:
      type: object
      properties:
        bn:
          type: string
          description: Base Name
        bt:
          type: number
          format: double
          description: Base Time
        bu:
          type: number
          format: double
          description: Base Unit
        bv:
          type: number
          format: double
          description: Base Value
        bs:
          type: number
          format: double
          description: Base Sum
        bver:
          type: number
          format: double
          description: Version
        n:
          type: string
          description: Name
        u:
          type: string
          description: Unit
        v:
          type: number
          format: double
          description: Value
        vs:
          type: string
          description: String Value
        vb:
          type: boolean
          description: Boolean Value
        vd:
          type: string
          description: Data Value
        s:
          type: number
          format: double
          description: Value Sum
        t:
          type: number
          format: double
          description: Time
        ut:
          type: number
          format: double
          description: Update Time
    SenMLArray:
      type: array
      items:
        $ref: "#/components/schemas/SenMLRecord"

  parameters:
    ID:
      name: id
      description: Unique channel identifier.
      in: path
      schema:
        type: string
        format: uuid
      required: true

  requestBodies:
    MessageReq:
      description: |
        Message to be distributed. Since the platform expects messages to be
        properly formatted SenML in order to be post-processed, clients are
        obliged to specify Content-Type header for each published message.
        Note that all messages that aren't SenML will be accepted and published,
        but no post-processing will be applied.
      required: true
      content:
        application/json:
          schema:
            $ref: "#/components/schemas/SenMLArray"

  responses:
    ServiceError:
      description: Unexpected server-side error occurred.
      
    HealthRes:
      description: Service Health Check.
      content:
        application/json:
          schema:
            $ref: "./schemas/HealthInfo.yml"

  securitySchemes:
    bearerAuth:
      type: http
      scheme: bearer
      bearerFormat: uuid
      description: |
        * Thing access: "Authorization: Thing <thing_key>"

    basicAuth:
      type: http
      scheme: basic
      description: |
        * Things access: "Authorization: Basic <base64-encoded_credentials>"

security:
  - bearerAuth: []
  - basicAuth: []

Expected behavior I want to schemathesis to produce a result, where the endpoints either fail or pass, or atleasting show me example cases of why my endpoint is failing, not give up on generating usecases.

Environment (please complete the following information):

  • OS: [Linux: Ubuntu 22.04 LTS ]
  • Python version: [3.10.2]
  • Schemathesis version: [3.19.6]
  • Spec version: [Open API 3.0.1]

Additional Context

I’m using this inside a pull request for Mainflux IoT Platform Link to the Pull Request: https://github.com/mainflux/mainflux/pull/1692

About this issue

  • Original URL
  • State: closed
  • Created 10 months ago
  • Comments: 19 (8 by maintainers)

Commits related to this issue

Most upvoted comments

It could be this way:

args: '-H "Authorization: Bearer {{ env.ACCESS_TOKEN }}" --hypothesis-suppress-health-check=filter_too_much,too_slow'