astronomer-cosmos: Jinja undefined errors when extracting openlineage events

All my cosmos-generated tasks are failing after the dbt run because of Jinja “undefined” errors during lineage processing. I have lineage disabled with OPENLINEAGE_DISABLED=True env var.

I am using Astro runtime 9.1.0 with dbt in a virtualenv as recommended. This is my Dockerfile:

FROM quay.io/astronomer/astro-runtime:9.1.0

# install dbt into a virtual environment
RUN python -m venv dbt_venv && source dbt_venv/bin/activate && \
    pip install --no-cache-dir dbt-snowflake && deactivate

ENV OPENLINEAGE_DISABLED=True

My model is very simple:

{{ config(materialized = 'table') }}

select
  try_to_timestamp('{{ run_started_at }}') as run_started_at

Everything works fine until the Task starts extracting openlineage data.

Here is the task log:

...normal dbt run above
[2023-10-18, 14:49:11 PDT] {subprocess.py:94} INFO - 21:49:11  Done. PASS=1 WARN=0 ERROR=0 SKIP=0 TOTAL=1
[2023-10-18, 14:49:12 PDT] {subprocess.py:98} INFO - Command exited with return code 0
[2023-10-18, 14:49:12 PDT] {taskinstance.py:1935} ERROR - Task failed with exception
Traceback (most recent call last):
  File "/usr/local/lib/python3.11/site-packages/cosmos/operators/local.py", line 445, in execute
    self.build_and_run_cmd(context=context, cmd_flags=cmd_flags)
  File "/usr/local/lib/python3.11/site-packages/cosmos/operators/local.py", line 357, in build_and_run_cmd
    result = self.run_command(cmd=dbt_cmd, env=env, context=context)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/cosmos/operators/local.py", line 231, in run_command
    self.calculate_openlineage_events_completes(env, Path(tmp_project_dir))
  File "/usr/local/lib/python3.11/site-packages/cosmos/operators/local.py", line 268, in calculate_openlineage_events_completes
    openlineage_processor = DbtLocalArtifactProcessor(
                            ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/openlineage/common/provider/dbt/local.py", line 53, in __init__
    dbt_project = self.load_yaml_with_jinja(
                  ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/openlineage/common/provider/dbt/local.py", line 133, in load_yaml_with_jinja
    return self.render_values_jinja(
           ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/openlineage/common/provider/dbt/local.py", line 156, in render_values_jinja
    parsed_dict[key] = cls.render_values_jinja(
                       ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/openlineage/common/provider/dbt/local.py", line 156, in render_values_jinja
    parsed_dict[key] = cls.render_values_jinja(
                       ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/openlineage/common/provider/dbt/local.py", line 156, in render_values_jinja
    parsed_dict[key] = cls.render_values_jinja(
                       ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/openlineage/common/provider/dbt/local.py", line 166, in render_values_jinja
    return environment.from_string(value).render()  # type: ignore
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/jinja2/environment.py", line 1301, in render
    self.environment.handle_exception()
  File "/usr/local/lib/python3.11/site-packages/jinja2/environment.py", line 936, in handle_exception
    raise rewrite_traceback_stack(source=source)
  File "<template>", line 1, in top-level template code
jinja2.exceptions.UndefinedError: 'run_started_at' is undefined

What am I doing wrong?

About this issue

  • Original URL
  • State: closed
  • Created 8 months ago
  • Comments: 18 (6 by maintainers)

Commits related to this issue

Most upvoted comments

@ivanstillfront we’ll be soon releasing a possible fix to most cases with failing renders: https://github.com/OpenLineage/OpenLineage/pull/2221

However, I still don’t recognize where OL integration would try to read value with run_started_at variable - it tried to render whole dbt_project.yml and needed section from profiles.yml. From issue description it seems the variable was used in model.

Anyway, I’ll be grateful if you could please check if the issue is solved in your case. Thanks for the feedback!

@sylashsu, thanks for getting back so quickly!

The openlineage library will improve to avoid this type of issue: OpenLineage/OpenLineage#2215

In the meantime, I’ll aim to release early next week to ensure Cosmos users are not blocked.

@tatiana This worked for me after upgrading to openlineage-integration-common, thanks!!

@JDarDagran thank you very much for promptly working on this! @sylashsu Could you please try to upgrade openlineage-integration-common>=1.5.0 and check if the error was solved?

This version was just released:

@sylashsu, thanks for getting back so quickly!

The openlineage library will improve to avoid this type of issue: https://github.com/OpenLineage/OpenLineage/issues/2215

In the meantime, I’ll aim to release early next week to ensure Cosmos users are not blocked.