dspy: [Bug] dspy doesn't handle o4 models properly

What happened?

With OpenAI’s new o4 models it seems dspy doesn’t handle them properly. Specifically this code in the dspy.clients.lm module:

        # Match pattern: o[1,3] at the start, optionally followed by -mini and anything else
        model_pattern = re.match(r"^o([13])(?:-mini)?", model_family)

        if model_pattern:
            # Handle OpenAI reasoning models (o1, o3)
            assert (
                max_tokens >= 5000 and temperature == 1.0
            ), "OpenAI's reasoning models require passing temperature=1.0 and max_tokens >= 5000 to `dspy.LM(...)`"
            self.kwargs = dict(temperature=temperature, max_completion_tokens=max_tokens, **kwargs)
        else:
            self.kwargs = dict(temperature=temperature, max_tokens=max_tokens, **kwargs)

Seems like it should be a straightforward change to the model_pattern to support o4. Might require the latest litellm as well.

Steps to reproduce

Try to use an o4 model and you’ll see this error:

litellm.BadRequestError: OpenAIException - Unsupported parameter: 'max_tokens' is not supported with this model. Use 'max_completion_tokens' instead.

DSPy version

2.6.17

About this issue

  • Original URL
  • State: closed
  • Created 2 years ago
  • Comments: 15 (1 by maintainers)

Most upvoted comments

I am getting this error for local models via vLLM. It’s not just reasoning model which you can just use parser flag and only get the answer, there are other models that just don’t work and I get:

2025/06/11 10:54:22 WARNING dspy.adapters.json_adapter: Failed to use structured output format, falling back to JSON mode.
Error during person extraction: Both structured output format and JSON mode failed. Please choose a model that supports `response_format` argument. Original error: litellm.NotFoundError: NotFoundError: Hosted_vllmException - The model `google/gemma-3-27b-it` does not exist.
This might happen if the LM is not properly configured or accessible

I only managed to make llama-3.2 3B work. All the Gemma 3 models fail and none of the is a reasoning models. The Qwen2.5 VL models are also failing, they are not reasoning models. What exactly is this error and how do we avoid it?

Did you try downgrading to dspy==2.6.19

Thank you! In 2.6.19 the Qwen3 models work properly!