llama_deploy: Workflow state / context is not serializable due to `events_buffer`
We are encountering an issue where the process_call of WorkflowService is failing because the context is not serializable
The line:
await self.set_workflow_state(handler.ctx, current_call)
Fails because handler.ctx has a value for the events_buffer key that is not serializable.
Traceback (most recent call last):
File "/Users/[redacted]/projects/[redacted]/.venv/lib/python3.12/site-packages/llama_deploy/services/workflow.py", line 335, in process_call
await self.set_workflow_state(handler.ctx, current_call)
File "/Users/[redacted]/projects/[redacted]/.venv/lib/python3.12/site-packages/llama_deploy/services/workflow.py", line 256, in set_workflow_state
context_str = json.dumps(context_dict)
^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/homebrew/Cellar/python@3.12/3.12.8/Frameworks/Python.framework/Versions/3.12/lib/python3.12/json/__init__.py", line 231, in dumps
return _default_encoder.encode(obj)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/homebrew/Cellar/python@3.12/3.12.8/Frameworks/Python.framework/Versions/3.12/lib/python3.12/json/encoder.py", line 200, in encode
chunks = self.iterencode(o, _one_shot=True)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/homebrew/Cellar/python@3.12/3.12.8/Frameworks/Python.framework/Versions/3.12/lib/python3.12/json/encoder.py", line 258, in iterencode
return _iterencode(o, 0)
^^^^^^^^^^^^^^^^^
TypeError: keys must be str, int, float, bool or None, not ModelMetaclass
I debugged and traced the issue down to the events_buffer value being a dictionary with Pydantic class types which can’t be serialized.
list(handler.ctx.to_dict(serializer=JsonPickleSerializer())['events_buffer'].keys())[0]
class 'llama_index.core.agent.workflow.workflow_events.ToolCallResult'
We are using llama-index 0.12.14 and llama-deploy 0.4.3.
We created the workflow using AgentWorkflow.from_tools_or_functions.
Please let us know if you need more information on this issue.
About this issue
- Original URL
- State: closed
- Created 2 years ago
- Reactions: 1
- Comments: 15 (9 by maintainers)
I can confirm that https://github.com/run-llama/llama_index/pull/17676 fixes this issue for us. Thanks for the help @masci and @logan-markewich !
@kylebrooks-8451 I’ll make a release later tonight (hopefully I have time) so that you don’t have to install from source 👍🏻
@kylebrooks-8451 did you see this? https://github.com/run-llama/llama_deploy/issues/392 there’s a PR attached
@logan-markewich , @masci I’m going to propose a possible fix in a PR today by changing the way we hash the context in
WorkflowService.Hmm, I can’t seem to even get it to respond now 😢 Might have to tag in @masci for this one
@logan-markewich - I was able to use your example to reproduce the error if I change the prompt to force it to call the tool:
This was using the latest commit on the
mainbranch. It outputs this error for me:🤔 Pydantic classes definitely can be serialized, there’s specific logic for this https://github.com/run-llama/llama_index/blob/c8a7c909c76d03d2f2e75dea1b0280045dad944d/llama-index-core/llama_index/core/workflow/context_serializers.py#L23
We even have have examples that serialize the AgentWorkflow state https://docs.llamaindex.ai/en/stable/examples/agent/agent_workflow_basic/#maintaining-state https://docs.llamaindex.ai/en/stable/examples/agent/agent_workflow_basic/#human-in-the-loop
So in any case, this is a little unexpected. I’ll take a look and see whats up