dbt-core: [Bug] dbt-core is not compatible with latest protobuf version

Is this a new bug in dbt-core?

I believe this is a new bug in dbt-core
I have searched the existing issues, and I could not find an existing issue for this bug

Current Behavior

Protobuf 5.26.0 was released today.

When installing dbt and running dbt deps, I see:

dbt deps
18:45:37  Running with dbt=1.7.9
18:45:38  Installing dbt-labs/dbt_utils
18:45:38  Installed from version 1.1.1
18:45:38  Up to date!
18:45:38  Installing calogica/dbt_expectations
18:45:39  Installed from version 0.10.0
18:45:39  Updated version available: 0.10.3
18:45:39  Installing calogica/dbt_date
18:45:39  Installed from version 0.10.0
18:45:39  Up to date!
18:45:39  
18:45:39  Updates available for packages: ['calogica/dbt_expectations']                 
Update your versions in packages.yml, then run dbt deps
Traceback (most recent call last):
  File "/home/vscode/.pyenv/versions/3.9.18/bin/dbt", line 8, in <module>
    sys.exit(cli())
  File "/home/vscode/.pyenv/versions/3.9.18/lib/python3.9/site-packages/click/core.py", line 1157, in __call__
    return self.main(*args, **kwargs)
  File "/home/vscode/.pyenv/versions/3.9.18/lib/python3.9/site-packages/click/core.py", line 1078, in main
    rv = self.invoke(ctx)
  File "/home/vscode/.pyenv/versions/3.9.18/lib/python3.9/site-packages/click/core.py", line 1688, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/home/vscode/.pyenv/versions/3.9.18/lib/python3.9/site-packages/click/core.py", line 1434, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/home/vscode/.pyenv/versions/3.9.18/lib/python3.9/site-packages/click/core.py", line 783, in invoke
    return __callback(*args, **kwargs)
  File "/home/vscode/.pyenv/versions/3.9.18/lib/python3.9/site-packages/click/decorators.py", line 33, in new_func
    return f(get_current_context(), *args, **kwargs)
  File "/home/vscode/.pyenv/versions/3.9.18/lib/python3.9/site-packages/dbt/cli/main.py", line 155, in wrapper
    return func(*args, **kwargs)
  File "/home/vscode/.pyenv/versions/3.9.18/lib/python3.9/site-packages/dbt/cli/requires.py", line 109, in wrapper
    fire_event(
  File "/home/vscode/.pyenv/versions/3.9.18/lib/python3.9/site-packages/dbt/events/functions.py", line 270, in fire_event
    EVENT_MANAGER.fire_event(e, level=level)
  File "/home/vscode/.pyenv/versions/3.9.18/lib/python3.9/site-packages/dbt/events/eventmgr.py", line 17, in fire_event
    msg = msg_from_base_event(e, level=level)
  File "/home/vscode/.pyenv/versions/3.9.18/lib/python3.9/site-packages/dbt/events/base_types.py", line 143, in msg_from_base_event
    "msg": event.message(),
  File "/home/vscode/.pyenv/versions/3.9.18/lib/python3.9/site-packages/dbt/events/types.py", line 2414, in message
    return f"Resource report: {self.to_json()}"
  File "/home/vscode/.pyenv/versions/3.9.18/lib/python3.9/site-packages/dbt/events/base_types.py", line 104, in to_json
    return MessageToJson(
TypeError: MessageToJson() got an unexpected keyword argument 'including_default_value_fields'

Fixing protobuf dependency on protobuf 4.25.3 fixes the error.

Expected Behavior

With protobuf < 5.26.0, I don’t see an error.

Steps To Reproduce

Install dbt
Run dbt deps

Relevant log output

See above

Environment

- OS: ubuntu 22.04 (devcontainer)
- Python: 3.9.18
- dbt: 1.7.9

Which database adapter are you using with dbt?

spark

Additional Context

No response

About this issue

Original URL
State: closed
Created 4 months ago
Reactions: 10
Comments: 15 (7 by maintainers)

Most upvoted comments

@JasperHG90 Thanks for opening up this issue! Pinning to protobuf==4.25.3 solved this for us. Saved me from a lot of headache!

+23

akan72 on Mar 13, 2024

dbt-core 1.7.10 has been released.

gshank on Mar 14, 2024

A new release 1.7.10 is being built now with protobuf pinned.

gshank on Mar 14, 2024

@katepaydhealth I didn’t confirm myself, but probably by executing something like this in your dbt environment:

python -m pip install protobuf==4.25.3

dbeatty10 on Mar 13, 2024

Should be fixed by #9724

JasperHG90 on Mar 13, 2024

Hi @VShkaberda that is actually exactly what we’re doing (with only a few exceptions). We switched not too long ago to specifying upper bounds for all most all of our dependencies (see setup.py) 🙂

        ...
        # Expect compatibility with all new versions of these packages, so lower bounds only.
        "packaging>20.9",
        "protobuf>=4.0.0,<5",
        "pytz>=2015.7",
        ...

Might help future readers of the code to move the protobuf entry out of the section mentioned by the inline comment 🙂

peter-at-work on Mar 20, 2024

@katepaydhealth I didn’t confirm myself, but probably by executing something like this in your dbt environment:
python -m pip install protobuf==4.25.3

Yup this is exactly what I did!

akan72 on Mar 13, 2024