pydantic-settings: error parsing value for field <...> from source "EnvSettingsSource"

Hi 👋

I just saw the FastAPI 0.100.0 prerelease announcement, the Pydantic v2 migration guide and wanted to give this all a try with one of my project’s test suites with the latest and greatest.

I ended up getting this error:

self = EnvSettingsSource(env_nested_delimiter=None, env_prefix_len=0)

    def __call__(self) -> dict[str, Any]:
        data: dict[str, Any] = {}
    
        for field_name, field in self.settings_cls.model_fields.items():
            try:
                field_value, field_key, value_is_complex = self.get_field_value(field, field_name)
            except Exception as e:
                raise SettingsError(
                    f'error getting value for field "{field_name}" from source "{self.__class__.__name__}"'
                ) from e
    
            try:
                field_value = self.prepare_field_value(field_name, field, field_value, value_is_complex)
            except ValueError as e:
>               raise SettingsError(
                    f'error parsing value for field "{field_name}" from source "{self.__class__.__name__}"'
                ) from e
E               pydantic_settings.sources.SettingsError: error parsing value for field "ui_languages" from source "EnvSettingsSource"

And this is my code that triggered the error when calling LanguageSettings().ui_languages:

from pathlib import Path
from pydantic import Field
from pydantic_settings import BaseSettings


def comma_separated_string_to_set(raw_value: str) -> set[str]:
    if not raw_value:
        msg = f"{raw_value} must be a comma separated string"
        raise ValueError(msg)
    return {item.strip() for item in raw_value.split(",")}


class LanguageSettings(BaseSettings):
    ui_languages: set[str] = Field(env="UI_LANGUAGES")

    class Config:
        env_file = Path("envs/test.env")

        @classmethod
        def parse_env_var(cls, field_name: str, raw_val: str) -> Any:
            if field_name == "ui_languages":
                return comma_separated_string_to_set(raw_val)

Here’s my poetry config:

[tool.poetry.dependencies]
fastapi = {extras = ["all"], version = "^0.100.0b1", allow-prereleases = true}
pydantic = {version = "^2.0b3", allow-prereleases = true}
pydantic-settings = {version = "^2.0b1", allow-prereleases = true}

Is all this expected when moving from old/current Pydantic, or is there a bug in pydantic-settings?

About this issue

Original URL
State: closed
Created a year ago
Comments: 15 (7 by maintainers)

Most upvoted comments

@hramezani many thanks for all your work on this. 😄 👍

For completeness, this is what I have now, for adding a dotenv file fallback as well as custom variable value parsing.

class Settings(BaseSettings):
    model_config = SettingsConfigDict(
        env_file=detect_dotenv_file(),  # <--- this returns e.g. str: "env/test.env" but can also return None if there is no .env file
        env_file_encoding="utf-8",
        extra="ignore",
    )

    @classmethod
    def settings_customise_sources(
        cls,
        settings_cls: type[BaseSettings],
        init_settings: PydanticBaseSettingsSource,
        env_settings: PydanticBaseSettingsSource,
        dotenv_settings: PydanticBaseSettingsSource,
        file_secret_settings: PydanticBaseSettingsSource,
    ) -> tuple[PydanticBaseSettingsSource, ...]:
        return (
            init_settings,
            env_settings,
            CustomSettingsSource(
                settings_cls,  # <--- note the omission of the env_file and env_file_encoding argument
            ),
            file_secret_settings,
        )


class CustomSettingsSource(DotEnvSettingsSource):
    def prepare_field_value(
        self,
        field_name: str,
        field: FieldInfo,
        value: Any,
        value_is_complex: bool,
    ) -> Any:
        if field_name in ["features", "ui_languages"]:
            # Return comma separated string as set
            if not value:
                msg = f"{value} must be a comma separated string"
                raise ValueError(msg)
            return {item.strip() for item in value.split(",")}

        return value


class FeatureSettings(Settings):
    features: set[str]


class LanguageSettings(Settings):
    ui_languages: set[str]
    fallback_to_default_language: bool = Field(
        default_factory=lambda: is_in_features("fallback_to_default_language")
    )


def is_in_features(raw_value: str) -> bool:
    features = FeatureSettings().features
    if raw_value in features:
        return True
    return False

fredrikaverpil on Jun 26, 2023

@fredrikaverpil I’ve created a PR to fix your mentioned problem

I’m noticing I have to pass in the env file in two places. Does this setup look reasonable to you?

hramezani on Jun 22, 2023

I’m noticing I have to pass in the env file in two places. Does this setup look reasonable to you?

I think it’s a problem in pydantic-settings and I will prepare a patch for fixing.

hramezani on Jun 22, 2023