opik: [Bug]: Assert error in `after_agent_callback` for trace_data when using Google ADK
What component(s) are affected?
- Python SDK
- Opik UI
- Opik Server
- Documentation
Opik version
- Opik version: 1.7.14
- google-adk: 0.4.0
Describe the problem
Firstly, thanks for being the first ADK tracing integration so early.
The error I have is when using a custom Agent before_agent_callback function that handles 2 things:
- Opik tracing via
opik_tracer.before_agent_callback, and - Setting custom state on the callback_context (initialising default user data)
the opik_tracer.after_agent_callback method has an assertion error in assert context_storage.get_trace_data() is self.trace_data
Digging further, it appears as though context_storage.get_trace_data() is None, while self.trace_data is populated.
Reproduction steps and code snippets
Set a custom before_agent_callback that applies the opik callback and also sets custom state as per:
from google.adk.agents.callback_context import CallbackContext
from opik.integrations.adk import OpikTracer
opik_tracer = OpikTracer()
def custom_before_agent_callback(callback_context: CallbackContext) -> None:
opik_tracer.before_agent_callback(callback_context)
callback_context.state["user:custom_state"] = { "custom_key1": 0, ... } # Set user state variable
example_agent = Agent(
model="gemini-2.0-flash",
name="Redacted",
description="Redacted",
instruction="Redacted",
sub_agents=[sub_agent1, sub_agent2],
before_agent_callback=custom_before_agent_callback, # custom callback to set tracing and custom state
after_agent_callback=opik_tracer.after_agent_callback,
before_model_callback=opik_tracer.before_model_callback,
after_model_callback=opik_tracer.after_model_callback,
before_tool_callback=opik_tracer.before_tool_callback,
after_tool_callback=opik_tracer.after_tool_callback,
)
Error logs or stack trace
File "/Users/<redacted_path>/lib/python3.11/site-packages/google/adk/agents/base_agent.py", line 300, in __handle_after_agent_callback
after_agent_callback_content = self.after_agent_callback(
^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/<redacted_path>/lib/python3.11/site-packages/opik/integrations/adk/opik_tracer.py", line 74, in after_agent_callback
assert context_storage.get_trace_data() is self.trace_data
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
AssertionError
Healthcheck results
*** HEALTHCHECK STARTED ***
Python version: 3.11.11
Opik version: 1.7.14
*** CONFIGURATION FILE ***
Config file path: /Users/<user>/.opik.config
Config file exists: yes
*** CURRENT CONFIGURATION ***
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓
┃ Setting ┃ Value ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━┩
│ api_key │ None │
│ background_workers │ 4 │
│ check_tls_certificate │ True │
│ console_logging_level │ INFO │
│ default_flush_timeout │ None │
│ enable_json_request_compression │ True │
│ enable_litellm_models_monitoring │ True │
│ file_logging_level │ None │
│ file_upload_background_workers │ 16 │
│ logging_file │ opik.log │
│ project_name │ Default Project │
│ pytest_experiment_enabled │ True │
│ sentry_enable │ True │
│ track_disable │ False │
│ url_override │ http://localhost:5173/api/ │
│ workspace │ default │
└──────────────────────────────────┴────────────────────────────┘
*** CONFIGURATION SCAN ***
Configuration issues: not found
*** BACKEND WORKSPACE AVAILABILITY ***
--> Checking backend workspace availability at: http://localhost:5173/api/
Backend workspace available: yes
*** HEALTHCHECK COMPLETED ***
About this issue
- Original URL
- State: closed
- Created 2 years ago
- Comments: 19 (4 by maintainers)
Hi @dylanhogg @Colstuwjx. We’ve been able to reproduce the problem when using
sub_agents. It turns out that some of our current design assumptions don’t hold up when running multiple agents with the ADK integration. We’re actively working on a fix. Unfortunately, there isn’t a workaround at the moment, but we’ll keep you updated as we make progress.Should be fixed here
@mbertani having a reproducing script that can be executed after configuring all the necessary providers API keys would be perfect
@mbertani thanks for the script! It’s very valuable because agentic applications vary a lot 😃 We’ll look into it tomorrow. (Would be nice in terms of tracking if you opened another issue since the original one was closed)
I tried
opik=1.7.20andgoogle-adk=0.5.0@alexkuzmik , but I still get errors. The difference from your example is probably that I’m using subagents with different providers, I’m mixing Gemini, Anthropic and OpenAI. The input/output tokens and cost are not calculated either.Code:
Use
call_agentwith your query.Error (a bit anonymized)
More from the logging:
I’m having similar issues with opik
1.7.18. With subagents and with async code. My workaround so far has been to use the decorator@opik.track(project_name=os.environ["OPIK_PROJECT_NAME"])on the entry function, and use also a generatedthread_id, so that the threads appear in the cloud dashboard, by passingopik_context.update_current_trace(thread_id=thread_id)internally. Cost is missing unfortunately.Thanks, Dylan for raising this issue. As part of the prototyping team at Google, I am interacting with a lot of our customers that are using ADK and potentially using Opik for evals, monitorin. I think this is something that would be good to get resolved for many of them. @Lothiraldan
There is actually one workaround, you can add the OpikTracer to only one of the agent. But you won’t be able to create other instances of the OpikTracer and you will be missing visibility into the other agents in your system.
Hi @dylanhogg ! I’m going to investigate this issue and come back to you with solution ASAP. Thank you for using OPIK and reporting an issue!