dspy: [Bug] dspy doesn't handle o4 models properly
What happened?
With OpenAI’s new o4 models it seems dspy doesn’t handle them properly. Specifically this code in the dspy.clients.lm module:
# Match pattern: o[1,3] at the start, optionally followed by -mini and anything else
model_pattern = re.match(r"^o([13])(?:-mini)?", model_family)
if model_pattern:
# Handle OpenAI reasoning models (o1, o3)
assert (
max_tokens >= 5000 and temperature == 1.0
), "OpenAI's reasoning models require passing temperature=1.0 and max_tokens >= 5000 to `dspy.LM(...)`"
self.kwargs = dict(temperature=temperature, max_completion_tokens=max_tokens, **kwargs)
else:
self.kwargs = dict(temperature=temperature, max_tokens=max_tokens, **kwargs)
Seems like it should be a straightforward change to the model_pattern to support o4. Might require the latest litellm as well.
Steps to reproduce
Try to use an o4 model and you’ll see this error:
litellm.BadRequestError: OpenAIException - Unsupported parameter: 'max_tokens' is not supported with this model. Use 'max_completion_tokens' instead.
DSPy version
2.6.17
About this issue
- Original URL
- State: closed
- Created 2 years ago
- Comments: 15 (1 by maintainers)
Thank you! In
2.6.19the Qwen3 models work properly!