prefect: Persistence Missing Result error

First check

I added a descriptive title to this issue.
I used the GitHub search to find a similar issue and didn’t find it.
I searched the Prefect documentation for this issue.
I checked that this issue is related to Prefect and not one of its dependencies.

Bug summary

Hi prefect team For one of my flow i got this error (when using prefect-dbt trigger_dbt_cli_command() ) prefect.exceptions.MissingResult: State data is missing. Typically, this occurs when result persistence is disabled and the state has been retrieved from the API. The weird thing is

I didn’t restart the job it a basic scheduled jobs
It worked well before upgrading to 2.7.
I added the flag PREFECT_RESULTS_PERSIST_BY_DEFAULT: true but no effect
I tried prefect config set PREFECT_RESULTS_PERSIST_BY_DEFAULT=True

Reproduction

def run_dbt_command(self) -> None:
        trigger_dbt_cli_command(
            self.get_dbt_run_command(),
            overwrite_profiles=self.overwrite_profile,
            dbt_cli_profile=self._get_dbt_cli_profile(),
            project_dir=self._get_dbt_folder_path(),
        )

Error

prefect.exceptions.MissingResult: State data is missing. Typically, this occurs when result persistence is disabled and the state has been retrieved from the API.

Versions

2.7.8

Additional context

Thread Slack https://prefect-community.slack.com/archives/CL09KU1K7/p1673857322820719

About this issue

Original URL
State: closed
Created a year ago
Reactions: 1
Comments: 19 (6 by maintainers)

Most upvoted comments

Ok, so here is the agent log

00:51:21.175 | INFO    | prefect.infrastructure.kubernetes-job - Job 'free-pigeon-fwtzv': Pod has status 'Pending'.
00:51:21.241 | INFO    | prefect.agent - Completed submission of flow run 'a772885c-af20-48e9-b8b7-f4acc24c2397'
00:51:23.500 | INFO    | prefect.infrastructure.kubernetes-job - Job 'free-pigeon-fwtzv': Pod has status 'Running'.
00:53:07.534 | ERROR   | prefect.infrastructure.kubernetes-job - Job 'free-pigeon-fwtzv': Job did not complete within timeout of 5s.
00:53:07.700 | INFO    | prefect.agent - Reported flow run 'a772885c-af20-48e9-b8b7-f4acc24c2397' as crashed: Flow run infrastructure exited with non-zero status code -1.

Here is the job log

00:53:07.669 | INFO    | Flow run 'free-pigeon' - Created task run 'getSecrets-e47b81b4-0' for task 'getSecrets'
00:53:07.670 | INFO    | Flow run 'free-pigeon' - Executing 'getSecrets-e47b81b4-0' immediately...
00:53:08.061 | INFO    | Task run 'getSecrets-e47b81b4-0' - Task run '95f3fd31-a57f-464c-9202-77e13328933c' already finished.
00:53:08.063 | ERROR   | Flow run 'free-pigeon' - Encountered exception during execution:

Based on the timestamps alone, it makes me think this is caused by the job timeout defined in the flow deployment. The job is timing out before the flow task can even complete.

Will attempt to set that to None per the suggestion here https://github.com/PrefectHQ/prefect/issues/8050#issuecomment-1376303397

BStefansen-lightbox on Feb 28, 2023