astronomer-cosmos: Jinja undefined errors when extracting openlineage events
All my cosmos-generated tasks are failing after the dbt run because of Jinja “undefined” errors during lineage processing. I have lineage disabled with OPENLINEAGE_DISABLED=True
env var.
I am using Astro runtime 9.1.0 with dbt in a virtualenv as recommended. This is my Dockerfile:
FROM quay.io/astronomer/astro-runtime:9.1.0
# install dbt into a virtual environment
RUN python -m venv dbt_venv && source dbt_venv/bin/activate && \
pip install --no-cache-dir dbt-snowflake && deactivate
ENV OPENLINEAGE_DISABLED=True
My model is very simple:
{{ config(materialized = 'table') }}
select
try_to_timestamp('{{ run_started_at }}') as run_started_at
Everything works fine until the Task starts extracting openlineage data.
Here is the task log:
...normal dbt run above
[2023-10-18, 14:49:11 PDT] {subprocess.py:94} INFO - [0m21:49:11 Done. PASS=1 WARN=0 ERROR=0 SKIP=0 TOTAL=1
[2023-10-18, 14:49:12 PDT] {subprocess.py:98} INFO - Command exited with return code 0
[2023-10-18, 14:49:12 PDT] {taskinstance.py:1935} ERROR - Task failed with exception
Traceback (most recent call last):
File "/usr/local/lib/python3.11/site-packages/cosmos/operators/local.py", line 445, in execute
self.build_and_run_cmd(context=context, cmd_flags=cmd_flags)
File "/usr/local/lib/python3.11/site-packages/cosmos/operators/local.py", line 357, in build_and_run_cmd
result = self.run_command(cmd=dbt_cmd, env=env, context=context)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/cosmos/operators/local.py", line 231, in run_command
self.calculate_openlineage_events_completes(env, Path(tmp_project_dir))
File "/usr/local/lib/python3.11/site-packages/cosmos/operators/local.py", line 268, in calculate_openlineage_events_completes
openlineage_processor = DbtLocalArtifactProcessor(
^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/openlineage/common/provider/dbt/local.py", line 53, in __init__
dbt_project = self.load_yaml_with_jinja(
^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/openlineage/common/provider/dbt/local.py", line 133, in load_yaml_with_jinja
return self.render_values_jinja(
^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/openlineage/common/provider/dbt/local.py", line 156, in render_values_jinja
parsed_dict[key] = cls.render_values_jinja(
^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/openlineage/common/provider/dbt/local.py", line 156, in render_values_jinja
parsed_dict[key] = cls.render_values_jinja(
^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/openlineage/common/provider/dbt/local.py", line 156, in render_values_jinja
parsed_dict[key] = cls.render_values_jinja(
^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/openlineage/common/provider/dbt/local.py", line 166, in render_values_jinja
return environment.from_string(value).render() # type: ignore
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/jinja2/environment.py", line 1301, in render
self.environment.handle_exception()
File "/usr/local/lib/python3.11/site-packages/jinja2/environment.py", line 936, in handle_exception
raise rewrite_traceback_stack(source=source)
File "<template>", line 1, in top-level template code
jinja2.exceptions.UndefinedError: 'run_started_at' is undefined
What am I doing wrong?
About this issue
- Original URL
- State: closed
- Created 8 months ago
- Comments: 18 (6 by maintainers)
Commits related to this issue
- Do not block Cosmos from work if there is a jinja issue in the Openlineage package https://github.com/astronomer/astronomer-cosmos/issues/612\#issuecomment-1777563297 Closes: #612 — committed to astronomer/astronomer-cosmos by tatiana 8 months ago
- Do not block Cosmos if openlineage-common raises a jinja exception (#626) Before this change, Cosmos failed to run if there is an issue in the openlineage-common parsing of the dbt project due to a ... — committed to astronomer/astronomer-cosmos by tatiana 8 months ago
- Do not block Cosmos if openlineage-common raises a jinja exception (#626) Before this change, Cosmos failed to run if there is an issue in the openlineage-common parsing of the dbt project due to a j... — committed to astronomer/astronomer-cosmos by tatiana 8 months ago
@ivanstillfront we’ll be soon releasing a possible fix to most cases with failing renders: https://github.com/OpenLineage/OpenLineage/pull/2221
However, I still don’t recognize where OL integration would try to read value with
run_started_at
variable - it tried to render wholedbt_project.yml
and needed section fromprofiles.yml
. From issue description it seems the variable was used in model.Anyway, I’ll be grateful if you could please check if the issue is solved in your case. Thanks for the feedback!
@tatiana This worked for me after upgrading to openlineage-integration-common, thanks!!
@JDarDagran thank you very much for promptly working on this! @sylashsu Could you please try to upgrade
openlineage-integration-common>=1.5.0
and check if the error was solved?This version was just released:
@sylashsu, thanks for getting back so quickly!
The openlineage library will improve to avoid this type of issue: https://github.com/OpenLineage/OpenLineage/issues/2215
In the meantime, I’ll aim to release early next week to ensure Cosmos users are not blocked.