fastapi: `List[...]` as type for a `Form` field doesn't work as expected

Opening a new issue since no action has been taken on https://github.com/tiangolo/fastapi/issues/842 for more than a month.

Looks like we have an issue on how fastapi handles lists of elements passed as Form parameter (so anything that is type hinted List[...] = Form(...)). The issue doesn’t arise when they are defined as Query parameters. After some digging I managed to get to the root of the problem, which is how we pass the parameters when making an api call. fastapi indeed correctly understands that the input needs to be a list of elements, but doesn’t parse the list if it comes from a single field.

Example using Query, works as expected

@app.post("/")
def api_test(l: List[int] = Query(...)) -> List[int]:
    return l

Api request generated using the swagger UI

curl -X 'GET' \
  'http://localhost:5009/?l=1&l=2' \
  -H 'accept: application/json'

note how the passed values are l=1&l=2

[
  1,
  2
]

Example using Form, doesn’t work as expected

@app.post("/")
def api_test(l: List[int] = Form(...)) -> List[int]:
    return l

Api request generated using the swagger UI

curl -X 'POST' \
  'http://localhost:5009/' \
  -H 'accept: application/json' \
  -H 'Content-Type: application/x-www-form-urlencoded' \
  -d 'l=1,2'

note how the passed values are l=1,2

{
  "detail": [
    {
      "loc": [
        "body",
        "l",
        0
      ],
      "msg": "value is not a valid integer",
      "type": "type_error.integer"
    }
  ]
}

Manual api request

curl -X 'POST' \
  'http://localhost:5009/' \
  -H 'accept: application/json' \
  -H 'Content-Type: application/x-www-form-urlencoded' \
  -d 'l=1' \
  -d 'l=2'

Now the values are passed as -d 'l=1' and -d 'l=2'

[
  1,
  2
]

I am fairly sure that l=1,2 should be an accepted way of passing a list of values as parameter, but fastapi doesn’t seem to like it. Also, if this isn’t the case, the Swagger UI doesn’t produce the correct curl requests for lists given as Form parameters.

Packages:

  • fastapi: 0.66.0
  • python: 3.7.4

Let me know if you need other details!

Thank you for coming to my TED talk

About this issue

  • Original URL
  • State: closed
  • Created 3 years ago
  • Reactions: 14
  • Comments: 17 (5 by maintainers)

Most upvoted comments

Then why does the Swagger UI produce curl requests which clearly don’t work?

You’ve got me! I don’t know what is causing it. 😢

~@HitLuca @uricholiveira probably the miss here is that FastAPI/Pydantic need to use the explode keyword in the OpenAPI specification. It looks like the default is simple in the OpenAPI spec which yields the CSV form.~

EDIT - this is a bug in swagger-ui see https://github.com/swagger-api/swagger-js/issues/2713

That linked doc talks about query parameters but there is a mention of it when describing request bodies too. Looks like the same rules apply.

This worked perfectly @phillipuniverse!

I modified it slightly to use the walrus operator (my first time using the walrus operator, in fact)

    if form_patch := (
        content.get("application/x-www-form-urlencoded")
        or content.get("multipart/form-data")
    ):

@falkben I couldn’t help myself, here’s the fully dynamic version of my monkeypatch!!

    import fastapi.openapi.utils
    orig_get_request_body = fastapi.openapi.utils.get_openapi_operation_request_body

    from typing import Optional
    from pydantic.fields import ModelField
    from pydantic.schema import field_schema
    from typing import Any, Dict, Union, Type
    from pydantic import BaseModel
    from fastapi.openapi.constants import REF_PREFIX

    from enum import Enum
    def get_request_body_with_explode(*,
        body_field: Optional[ModelField],
        model_name_map: Dict[Union[Type[BaseModel], Type[Enum]], str],
    ) -> Optional[Dict[str, Any]]:
        original = orig_get_request_body(body_field=body_field, model_name_map=model_name_map)
        if not original:
            return original
        content = original.get("content", {})
        form_patch = content.get("application/x-www-form-urlencoded", {})
        if not form_patch:
            form_patch = content.get("multipart/form-data", {})
        if form_patch:
            schema_reference, schemas, _ = field_schema(
                body_field, model_name_map=model_name_map, ref_prefix=REF_PREFIX
            )
            array_props = []
            for schema in schemas.values():  # type: Dict[str, Any]
                for prop, prop_schema in schema.get("properties", {}).items():
                    if prop_schema.get("type") == "array":
                        array_props.append(prop)

            form_patch["encoding"] = {prop: {"style": "form"} for prop in array_props} # could include "explode": True but not necessary in swagger-ui

        return original

    fastapi.openapi.utils.get_openapi_operation_request_body = get_request_body_with_explode

I think it should be possible to look into the model when creating the schema to determine if it needs an encoding section

Yup, exactly! The key part to my above code is to get the actual OpenAPI schema from Pydantic, not just a reference to it. The schema comes back in the 2nd part of the tuple in the field_schema call:

   schema_reference, schemas, _ = field_schema(
                body_field, model_name_map=model_name_map, ref_prefix=REF_PREFIX
            )

Once you’ve got that it’s trivial to determine the array properties. Shouldn’t be too bad to refactor that to a recursive version either.

Let me know if that works for you!

@falkben looking at this again, I think that there is a bug in Swagger UI. According to the ‘Form Data’ section of describing request bodies, it says:

By default, arrays are serialized as array_name=value1&array_name=value2 and objects as prop1=value1&prop=value2, but you can use other serialization strategies as defined by the OpenAPI 3.0 Specification

So the default should work the exact same as query parameters. If there are changes to code, it should be within Swagger UI, not in FastAPI or Pydantic (although perhaps FastAPI could make it easier to extend the encoding).

I messed around with a very rough way to override this with monkeypatching:

    import fastapi.openapi.utils
    orig_get_request_body = fastapi.openapi.utils.get_openapi_operation_request_body

    from typing import Optional
    from pydantic.fields import ModelField
    from typing import Dict
    from typing import Union
    from typing import Type
    from pydantic import BaseModel
    from enum import Enum
    from typing import Any

    def get_request_body_with_explode(*,
        body_field: Optional[ModelField],
        model_name_map: Dict[Union[Type[BaseModel], Type[Enum]], str],
    ) -> Optional[Dict[str, Any]]:
        original = orig_get_request_body(body_field=body_field, model_name_map=model_name_map)
        if not original:
            return original
        content = original.get("content", {})
        form_patch = content.get("application/x-www-form-urlencoded", {})
        if not form_patch:
            form_patch = content.get("multipart/form-data", {})
        if form_patch:

            form_patch["encoding"] = {
                "vals": {  # TODO: change to be more dynamic
                    "style": "form",
                    "explode": True
                }
            }

        return original

    fastapi.openapi.utils.get_openapi_operation_request_body = get_request_body_with_explode

You would monkeypatch that in the same sort of place you instantiate FastAPI, or just any time before you invoke the app.openapi() function.

This works great with an endpoint defined like:

@router.post("/withlistparams")
async def with_list_params(vals: list[str] = Form()):
    raise NotImplementedError()

The annoying part in the spec is that you have to specify the “encoding” on a per-request-property basis. But you could use the given body_field to look up the properties in the schema that are lists and ensure all of those properties have the right encoding set on the operation.

Hope that helps!

This is also a problem with Tuple, not just List. Additionally, now in FastAPI 0.68.0 (but not 0.67.0), the Swagger UI doesn’t load for endpoints with Form parameters of type Tuple. See #3665.