llama_deploy: Workflow state / context is not serializable due to `events_buffer`

We are encountering an issue where the process_call of WorkflowService is failing because the context is not serializable

The line:

await self.set_workflow_state(handler.ctx, current_call)

Fails because handler.ctx has a value for the events_buffer key that is not serializable.

Traceback (most recent call last):
  File "/Users/[redacted]/projects/[redacted]/.venv/lib/python3.12/site-packages/llama_deploy/services/workflow.py", line 335, in process_call
    await self.set_workflow_state(handler.ctx, current_call)
  File "/Users/[redacted]/projects/[redacted]/.venv/lib/python3.12/site-packages/llama_deploy/services/workflow.py", line 256, in set_workflow_state
    context_str = json.dumps(context_dict)
                  ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/Cellar/python@3.12/3.12.8/Frameworks/Python.framework/Versions/3.12/lib/python3.12/json/__init__.py", line 231, in dumps
    return _default_encoder.encode(obj)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/Cellar/python@3.12/3.12.8/Frameworks/Python.framework/Versions/3.12/lib/python3.12/json/encoder.py", line 200, in encode
    chunks = self.iterencode(o, _one_shot=True)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/Cellar/python@3.12/3.12.8/Frameworks/Python.framework/Versions/3.12/lib/python3.12/json/encoder.py", line 258, in iterencode
    return _iterencode(o, 0)
           ^^^^^^^^^^^^^^^^^
TypeError: keys must be str, int, float, bool or None, not ModelMetaclass

I debugged and traced the issue down to the events_buffer value being a dictionary with Pydantic class types which can’t be serialized.

list(handler.ctx.to_dict(serializer=JsonPickleSerializer())['events_buffer'].keys())[0]
class 'llama_index.core.agent.workflow.workflow_events.ToolCallResult'

We are using llama-index 0.12.14 and llama-deploy 0.4.3.

We created the workflow using AgentWorkflow.from_tools_or_functions.

Please let us know if you need more information on this issue.

About this issue

  • Original URL
  • State: closed
  • Created 2 years ago
  • Reactions: 1
  • Comments: 15 (9 by maintainers)

Most upvoted comments

I can confirm that https://github.com/run-llama/llama_index/pull/17676 fixes this issue for us. Thanks for the help @masci and @logan-markewich !

@kylebrooks-8451 I’ll make a release later tonight (hopefully I have time) so that you don’t have to install from source 👍🏻

@logan-markewich , @masci I’m going to propose a possible fix in a PR today by changing the way we hash the context in WorkflowService.

Hmm, I can’t seem to even get it to respond now 😢 Might have to tag in @masci for this one

@logan-markewich - I was able to use your example to reproduce the error if I change the prompt to force it to call the tool:

llamactl run --deployment QuickStart --arg user_msg "What is two times two?"

This was using the latest commit on the main branch. It outputs this error for me:

ERROR:llama_deploy.services.workflow - Encountered error in task dbf13fd3-c96d-4da4-b5ce-815f894441d3! keys must be str, int, float, bool or None, not ModelMetaclass

🤔 Pydantic classes definitely can be serialized, there’s specific logic for this https://github.com/run-llama/llama_index/blob/c8a7c909c76d03d2f2e75dea1b0280045dad944d/llama-index-core/llama_index/core/workflow/context_serializers.py#L23

We even have have examples that serialize the AgentWorkflow state https://docs.llamaindex.ai/en/stable/examples/agent/agent_workflow_basic/#maintaining-state https://docs.llamaindex.ai/en/stable/examples/agent/agent_workflow_basic/#human-in-the-loop

So in any case, this is a little unexpected. I’ll take a look and see whats up