I’ll try to get a new release out asap.

Hi everyone You need to understand that int | str is not valid at runtime for python < 3.10 as type doesn’t implement __or__. It is valid at interpretation time by adding from __future__ import annotations but pydantic can’t do magic and change how python works at runtime. I opened a while ago https://github.com/samuelcolvin/pydantic/pull/2609, which works for python < 3.10 by using https://github.com/PrettyWood/future-typing, which changes at interpretation time int | str into Union[int, str] for example. But I don’t think it’s worth merging as moving to python 3.10 is safer Hope it clarifies things

If I may, I’d like to discourage people from posting comments like “+1”.

The issue is already confirmed and acknowledged by maintainer, fix will be available with the next release: https://github.com/samuelcolvin/pydantic/issues/3300#issuecomment-938145241.

These comments are noisy for everyone subscribed to this issue, while providing no value at all.

You can subscribe to this issue if you wish to be notified when it will be fixed.

In the meanwhile you can use this as a workaround: https://github.com/samuelcolvin/pydantic/issues/3300#issuecomment-950122191.

@Bobronium sorry for the noise then.

For me personally, “+1” comments are having the value of noticing the maintainer there are actually people interested in the fix (and I see this practice is very common in open-source projects, so I wasn’t aware this can be bad), especially considering the #3300 (comment) was actually from a few months ago and there’s still no release yet.

But you’re arguments are perfectly valid and if you see the alternative solution for that (do you see if and how many people subscribe to the issue?) then I’m happy to oblige 😅

Sorry for reopening the issue, and sorry if that’s not the best place to ask this question, but IMHO it’s pretty related:

Python 3.7 also supports the new UnionType if from __future__ import annotations is used:

$ python
Python 3.7.12 (default, Oct 13 2021, 06:51:32)
[Clang 11.0.0 (clang-1100.0.33.17)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> foo: str | None = None
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: unsupported operand type(s) for |: 'type' and 'NoneType'
>>> from __future__ import annotations
>>> foo: str | None = None
>>> foo is None
True

But it does not work in Pydantic 1.9: example.py

from __future__ import annotations

from pydantic import BaseModel


class Foo(BaseModel):
    bar: str | None

$ python example.py
Traceback (most recent call last):
  File "a.py", line 6, in <module>
    class Foo(BaseModel):
  File "pydantic/main.py", line 187, in pydantic.main.ModelMetaclass.__new__
  File "pydantic/typing.py", line 356, in pydantic.typing.resolve_annotations
    if self._name == 'Optional':
  File "/usr/local/Cellar/python@3.7/3.7.12_1/Frameworks/Python.framework/Versions/3.7/lib/python3.7/typing.py", line 263, in _eval_type
    return t._evaluate(globalns, localns)
  File "/usr/local/Cellar/python@3.7/3.7.12_1/Frameworks/Python.framework/Versions/3.7/lib/python3.7/typing.py", line 467, in _evaluate
    eval(self.__forward_code__, globalns, localns),
  File "<string>", line 1, in <module>
TypeError: unsupported operand type(s) for |: 'type' and 'NoneType'

Is it something that could be supported in the future, or such backward compatibility is out of scope?

+1

Here’s a workaround until it gets released.

from types import UnionType
from typing import Union

from pydantic.main import ModelMetaclass, BaseModel


class MyModelMetaclass(ModelMetaclass):
    def __new__(cls, name, bases, namespace, **kwargs):
        annotations = namespace.get("__annotations__", {})
        for annotation_name, annotation_type in annotations.items():
            if isinstance(annotation_type, UnionType):
                annotations[annotation_name] = Union.__getitem__(annotation_type.__args__)
        return super().__new__(cls, name, bases, namespace, **kwargs)


class MyBaseModel(BaseModel, metaclass=MyModelMetaclass):
    ...


class Python310(MyBaseModel):
    a: str | None


print(repr(Python310(a="3.10")))

Same approach code also works for SQLModel, but instead of inheriting from BaseModel and MyModelMetaclass, you’ll need to inherit from SQLModel and SQLModelMetaclass

from types import UnionType
from typing import Union

from pydantic.main import ModelMetaclass, BaseModel


class MyModelMetaclass(ModelMetaclass):
    def __new__(cls, name, bases, namespace, **kwargs):
        annotations = namespace.get("__annotations__", {})
        for annotation_name, annotation_type in annotations.items():
            if isinstance(annotation_type, UnionType):
                annotations[annotation_name] = Union.__getitem__(
                    annotation_type.__args__
                )
            elif isinstance(annotation_type, str) and "|" in annotation_type:
                # it's rather naive to think that having | can only mean that it's a Union
                # It will work in most cases, but you need to be aware that it's not always the case
                annotations[
                    annotation_name
                ] = f'Union[{", ".join(map(str.strip, annotation_type.split("|")))}]'

        return super().__new__(cls, name, bases, namespace, **kwargs)


class MyBaseModel(BaseModel, metaclass=MyModelMetaclass):
    ...


class Python310(MyBaseModel):
    a: str | None
    b: "WhatIsThat | int | None"


class WhatIsThat(BaseModel):
    foo = "1"


Python310.update_forward_refs()

print(repr(Python310(a="3.10", b=None)))

@Toreno96, no worries. Sorry if my comment read too grumpy 🙆🏻.

Let me elaborate on why I personally think that “+1” comments are usually a bad idea.

Let’s start with better alternative first: reactions. I think it’s a great feature for showing the number of people agree/interested/grateful/upset with a comment/issue. Unlike comments, it doesn’t increase the amount of noise and saves the issue from being flooded: imagine googling this issue and seeing 30+ comments with plain “+1” or “same here”. It’s not hard to imagine that people won’t even attempt to read the rest of the comments, and instead will post yet another “+1” and leave.

While I understand your concern and desire to give a notice to the maintainer, I don’t think “+1” would be the best approach for that. I believe maintainers are already very aware of this, and many other issues. I think some form of discussion here would be more applicable if, for example, pydantic had a new release that didn’t address this issue. But currently, It’s not a question “whether this issue will be fixed”, or even “when” (fix is already in master), but “when will see the new release”.

So I’m not discouraging any conversations, I just believe that “+1” is not a great start for one, especially in given situation 😃

+1 on this

Hi guys, I believe the fix suggested in https://github.com/samuelcolvin/pydantic/issues/3300#issuecomment-950122191 will not be working for the following case:

class SericesList(BaseModel):
    services: list[EcsServiceBase|EcsServiceState]

shouldn’t we replace = with : when we define variables in model? a = Optional[str] -> a: Optional[str]

+1 on this

I’m actually experiencing this on Python 3.10.4 with Pydantic 1.10.1.

class PromptData(BaseModel):
    image = PIL.Image.Image | None

results in:

    class PromptData(BaseModel):
  File "pydantic/main.py", line 222, in pydantic.main.ModelMetaclass.__new__
  File "pydantic/fields.py", line 506, in pydantic.fields.ModelField.infer
  File "pydantic/fields.py", line 436, in pydantic.fields.ModelField.__init__
  File "pydantic/fields.py", line 557, in pydantic.fields.ModelField.prepare
  File "pydantic/fields.py", line 831, in pydantic.fields.ModelField.populate_validators
  File "pydantic/validators.py", line 752, in find_validators
RuntimeError: no validator found for <class 'types.UnionType'>, see `arbitrary_types_allowed` in Config

Adding the following to PromptData

class Config:
    arbitrary_types_allowed = True  # Required so our Union in the type of instance variable image can be checked

results in

  File "pydantic/main.py", line 242, in pydantic.main.ModelMetaclass.__new__
  File "pydantic/class_validators.py", line 181, in pydantic.class_validators.ValidatorGroup.check_for_unused
pydantic.errors.ConfigError: Validators defined with incorrect fields: prevent_none (use check_fields=False if you're inheriting from the model and intended this)

I remember running into this problem when testing 3.10 with Pydantic and found this topic. It appears like it’s no longer a problem with the release of 1.9 (https://github.com/samuelcolvin/pydantic/pull/2885), so the problem is resolved.

I think there’s a related bug that, based on the existing fix in master, I think would also be fixed by this. I wanted to share the trace here in case it helps someone googling, but didn’t think it was worthy of a separate issue. If you disagree let me know and I’ll raise one.

Given:

from typing import Optional

import pydantic

class Settings(pydantic.BaseSettings):
    works: Optional[str] = None
    broken: str | None = None

Settings()

One gets:

/tmp/pydantic via 🐍 v3.10.0 (env) on ☁️  (us-east-1) took 2s 
❯ ./poc.py          

/tmp/pydantic via 🐍 v3.10.0 (env) on ☁️  (us-east-1) took 142ms 
❯ works=test ./poc.py

/tmp/pydantic via 🐍 v3.10.0 (env) on ☁️  (us-east-1) took 148ms 
❯ works=test broken=test ./poc.py
Traceback (most recent call last):
  File "/tmp/pydantic/env/lib/python3.10/site-packages/pydantic/env_settings.py", line 172, in __call__
    env_val = settings.__config__.json_loads(env_val)  # type: ignore
  File "/home/tl/.pyenv/versions/3.10.0/lib/python3.10/json/__init__.py", line 346, in loads
    return _default_decoder.decode(s)
  File "/home/tl/.pyenv/versions/3.10.0/lib/python3.10/json/decoder.py", line 337, in decode
    obj, end = self.raw_decode(s, idx=_w(s, 0).end())
  File "/home/tl/.pyenv/versions/3.10.0/lib/python3.10/json/decoder.py", line 355, in raw_decode
    raise JSONDecodeError("Expecting value", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/tmp/pydantic/./poc.py", line 10, in <module>
    Settings()
  File "/tmp/pydantic/env/lib/python3.10/site-packages/pydantic/env_settings.py", line 37, in __init__
    **__pydantic_self__._build_values(
  File "/tmp/pydantic/env/lib/python3.10/site-packages/pydantic/env_settings.py", line 63, in _build_values
    return deep_update(*reversed([source(self) for source in sources]))
  File "/tmp/pydantic/env/lib/python3.10/site-packages/pydantic/env_settings.py", line 63, in <listcomp>
    return deep_update(*reversed([source(self) for source in sources]))
  File "/tmp/pydantic/env/lib/python3.10/site-packages/pydantic/env_settings.py", line 174, in __call__
    raise SettingsError(f'error parsing JSON for "{env_name}"') from e
pydantic.env_settings.SettingsError: error parsing JSON for "broken"

/tmp/pydantic via 🐍 v3.10.0 (env) on ☁️  (us-east-1) took 137ms 
✗❯ works=test broken='"test"' ./poc.py
Traceback (most recent call last):
  File "/tmp/pydantic/./poc.py", line 10, in <module>
    Settings()
  File "/tmp/pydantic/env/lib/python3.10/site-packages/pydantic/env_settings.py", line 36, in __init__
    super().__init__(
  File "/tmp/pydantic/env/lib/python3.10/site-packages/pydantic/main.py", line 406, in __init__
    raise validation_error
pydantic.error_wrappers.ValidationError: 1 validation error for Settings
broken
  instance of UnionType expected (type=type_error.arbitrary_type; expected_arbitrary_type=UnionType)

The JSON error confused me when googling this initially after updating a FastAPI project of mine to 3.10 and making no other changes. I believe it happens because the lack of support for UnionType breaks the field.is_complex() check, causing pydantic to attempt to parse it as JSON.