pydantic: Cannot use Literal when I use typing-extension==4.6.0

List of workarounds:

• Workaround 1: pin requirement typing_extensions==4.5.0
• Workaround 2: use Python >= 3.10.1 (works because typing_extensions has some version-specific logic)
• Workaround 3: at the start of your program, before any imports, patch typing.Literal to refer to typing_extensions.Literal. (Full patch example below.) You only need to do this in the initial module.
• Workaround 4: replace every from typing import Literal with from typing_extensions import Literal.

The patch of workaround 3:

# ---- Start of patch ----
# Workaround for https://github.com/pydantic/pydantic/issues/5821
#
# Add this code once at the start of your program. It plays nice with mypy.
#
# It patches `typing.Literal` to refer to `typing_extensions.Literal`,
# and thereby avoids a bug in Pydantic<=1.10.7 code for is_literal_type.
import typing_extensions
import typing
typing.Literal = typing_extensions.Literal  # type: ignore
# ---- End of patch ----

# ---- Your program goes here ----
from pydantic import BaseModel
from typing import Literal

class Hello(BaseModel):
    x: Literal['x']   # patch prevents error
    y: typing.Literal['y]   # patch prevents error

Hey all, we’ve just released v1.10.8 to address this. It’s going through CI now and should hopefully be available through PyPI shortly. Please let us know if you run into other issues (with typing_extensions or otherwise) and we will address them as promptly as we can.

Released.

I’m getting this too. Even on pydantic v2.

As a workaround:

1. upgrade to python 3.10. This issue happens on 3.9 but not 3.10; or
2. change from typing import Literal to from typing_extensions import Literal

The docs for pydantic do say:

This is a new feature of the Python standard library as of Python 3.8; prior to Python 3.8, it requires the typing-extensions package.

But I find that documentation ambiguous. It sounds like the requirement is that typing_extensions merely be installed, and then pydantic will choose it if required. But that doesn’t seem to be the case.

So I think we need clearer documentation.

I think the source of the bug is this line:

https://github.com/pydantic/pydantic/blob/bf6b884563e7c635b9d811e97d2fdb51805398b1/pydantic/typing.py#LL412C57-L412C64

from typing_extensions import (
...
    Literal,
...
)
...
def is_literal_type(type_: Type[Any]) -> bool:
    return Literal is not None and get_origin(type_) is Literal

So I think this is a bug in pydantic. is_literal_type is comparing only to typing_extensions.Literal, not typing.Literal.

I think a patch looks something like:

def is_literal_type(type_: Type[Any]) -> bool:
    return Literal is not None and get_origin(type_) in [typing.Literal, typing_extensions.Literal]

Minimal steps to reproduce the issue (typing_extensions==4.6.0, pydantic==1.10.7, Python 3.8.10):

from typing import Literal
from pydantic import BaseModel


class MyType(BaseModel):
    literal: Literal["foobar"] = "foobar"

raises: TypeError: issubclass() arg 1 must be a class

Can confirm, the bug appears to occur with typing-extension==4.6.0.

Just as a suggestion for Pydantic, I would recommend to replace any code like is typing.X with something that checks for both typing and typing_extensions. In pyanalyze I have a helper for that: https://github.com/quora/pyanalyze/blob/master/pyanalyze/safe.py#L148 (used e.g. at https://github.com/quora/pyanalyze/blob/27fbd1831c908477baf4f9988f23a8278ce5f64e/pyanalyze/annotations.py#L783). This would be safer against future typing-extensions changes that might bring additional backports.

Thanks for reporting, @hramezani will take a look 🙏.

@jnawk @Czaki the reason that 1.10.3 was yanked was because 1.10.4 updated the minimum-allowed-version of typing_extensions. As a result, it was incompatible to use pydantic 1.10.4 with other libraries that had pinned the version of typing_extensions below 4.2.0.

However, the release of pydantic 1.10.8 did not set any new minimum dependency versions (it just fixed the bug). So unless I’m missing something, I don’t think the same reasons for yanking the 1.10.3 pydantic version apply here.

This has also been reported over at typing_extensions FYI: https://github.com/python/typing_extensions/issues/179.

I think the source of the bug is this line:

https://github.com/pydantic/pydantic/blob/bf6b884563e7c635b9d811e97d2fdb51805398b1/pydantic/typing.py#LL412C57-L412C64

from typing_extensions import (
...
    Literal,
...
)
...
def is_literal_type(type_: Type[Any]) -> bool:
    return Literal is not None and get_origin(type_) is Literal

So I think this is a bug in pydantic. is_literal_type is comparing only to typing_extensions.Literal, not typing.Literal.

I think a patch looks something like:

def is_literal_type(type_: Type[Any]) -> bool:
    return Literal is not None and get_origin(type_) in [typing.Literal, typing_extensions.Literal]

If this analysis is correct (and I haven’t checked, but it looks very plausible), you’d probably want to do something like this, in order to retain compatibility with Python 3.7 (where typing.Literal isn’t a thing yet):

_Literal_types = {typing_extensions.Literal}

if hasattr(typing, "Literal"):
    _Literal_types.add(typing.Literal)

def is_literal_type(type_: Type[Any]) -> bool:
    return Literal is not None and get_origin(type_) in _Literal_types

(Not sure under what conditions Literal can be None btw? Seems like a slight oddity in the code as it’s currently written 😃