openinference: [BUG]: Disabling OpenAIAgent Tracing changes Phoenix Tracing
Where do you use Phoenix
Local (self-hosted)
What version of Phoenix are you using?
10.0.4
What operating system are you seeing the problem on?
MacOS
What version of Python are you running Phoenix with?
3.12.7
What version of Python or Node are you using instrumentation with?
3.12.7
What instrumentation are you using?
Python
openai 1.82.0
openai-agents 0.0.16
openinference-instrumentation 0.1.31
openinference-instrumentation-openai 0.1.28
openinference-instrumentation-openai-agents 0.1.12
openinference-semantic-conventions 0.1.17
opentelemetry-api 1.33.1
opentelemetry-exporter-otlp 1.33.1
opentelemetry-exporter-otlp-proto-common 1.33.1
opentelemetry-exporter-otlp-proto-grpc 1.33.1
opentelemetry-exporter-otlp-proto-http 1.33.1
opentelemetry-instrumentation 0.54b1
opentelemetry-proto 1.33.1
opentelemetry-sdk 1.33.1
opentelemetry-semantic-conventions 0.54b1
What happened?
I am trying to uninstrument OpenAIAgents so my traces do not get sent to them. Their documentation indicates that this is enabled by default. I can disable it, but if I do, my traces get sent to Phoenix as ChatCompletions rather than Agent Workflows.
I disabled it by setting the env variable OPENAI_AGENTS_DISABLE_TRACING=1 but this changes how my traces get sent to Phoenix.
What did you expect to happen?
I would have expected my Agent Workflows to get sent to Phoenix the same way regardless of OpenAI’s agent tracing being enabled.
How can we reproduce the bug?
I’m not sure how exactly you can reproduce this. Here is my code:
import os
import asyncio
from dotenv import load_dotenv
from pydantic import BaseModel
from openai import AsyncOpenAI
from agents import (
Agent,
Model,
ModelProvider,
OpenAIChatCompletionsModel,
RunConfig,
Runner,
InputGuardrail,
GuardrailFunctionOutput,
exceptions as agent_exceptions
)
load_dotenv(".env")
# Get API Settings from .env
API_KEY = os.getenv("OPENAI_API_KEY")
BASE_URL = os.getenv("BASE_URL")
MODEL_NAME = os.getenv("MODEL")
client = AsyncOpenAI(
base_url=BASE_URL,
api_key=API_KEY,
)
# configure the Phoenix tracer
from phoenix.otel import register
tracer_provider = register(
project_name="agents",
endpoint="https://my_url.com/v1/traces",
auto_instrument=True,
batch=True
)
class HomeworkOutput(BaseModel):
is_homework: bool
reasoning: str
class CustomModelProvider(ModelProvider):
def get_model(self, model_name: str | None) -> Model:
return OpenAIChatCompletionsModel(model=model_name or MODEL_NAME, openai_client=client)
async def homework_guardrail(ctx, agent, input_data):
result = await Runner.run(
starting_agent=guardrail_agent,
input=input_data,
context=ctx.context,
run_config=RunConfig(model_provider=CUSTOM_MODEL_PROVIDER),
)
final_output = result.final_output_as(HomeworkOutput)
return GuardrailFunctionOutput(
output_info=final_output,
tripwire_triggered=final_output.is_homework,
)
CUSTOM_MODEL_PROVIDER = CustomModelProvider()
# Define your agents
guardrail_agent = Agent(
name="Guardrail check",
instructions="Check if the user is asking about homework.",
output_type=HomeworkOutput,
)
math_tutor_agent = Agent(
name="Math Tutor",
handoff_description="Specialist agent for math questions",
instructions="You provide help with math problems. Explain your reasoning at each step and include examples",
)
history_tutor_agent = Agent(
name="History Tutor",
handoff_description="Specialist agent for historical questions",
instructions="You provide assistance with historical queries. Explain important events and context clearly.",
)
triage_agent = Agent(
name="Triage Agent",
instructions="You determine which agent to use based on the user's homework question",
handoffs=[history_tutor_agent, math_tutor_agent],
# input_guardrails=[
# InputGuardrail(guardrail_function=homework_guardrail),
# ],
)
async def main():
result = await Runner.run(
starting_agent=triage_agent,
input="who was the first president of the united states?",
run_config=RunConfig(model_provider=CUSTOM_MODEL_PROVIDER),
)
print(result.final_output)
if __name__ == "__main__":
try:
asyncio.run(main())
except agent_exceptions.InputGuardrailTripwireTriggered:
print("Guardrail InputGuardrail triggered tripwire")
This same code produces different results on the Phoenix dashboard if I either set or unset OPENAI_AGENTS_DISABLE_TRACING=1
Additional information
No response
About this issue
- Original URL
- State: closed
- Created a year ago
- Reactions: 1
- Comments: 16 (8 by maintainers)
Awesome! We’ll set that as the default going forward so that auto_instrument=True will work out of the box
Thanks for the report @brandonbiggs ! I’m gonna transfer this issue to OpenInference as it seems related to instrumentation. cc @Jgilhuly