js-genai: Model never correctly calls any function, but it used to work

Function calling was working for me last week, but in the past few days the model never calls a function, but instead responds with text of the JSON it would use for the function call.

This used to work, so I believe my configuration and prompts are fine.

Hereโ€™s an example response:

{
  "candidates": [
    {
      "content": {
        "parts": [
          {
            "text": "```tool_code\n{\n  \"candidates\": [\n    {\n      \"content\": {\n        \"parts\": [\n          {\n            \"functionCall\": {\n              \"name\": \"get_weather\",\n              \"args\": \"{\\\"zip_code\\\":\\\"11217\\\"}\"\n            }\n          }\n        ]\n      }\n    }\n  ]\n}\n```\n"
          }
        ],
        "role": "model"
      },
      "finishReason": "STOP",
      "avgLogprobs": -0.027122719534512223
    }
  ],
  "modelVersion": "gemini-2.0-flash-001",
  "usageMetadata": {
    "promptTokenCount": 1819,
    "candidatesTokenCount": 87,
    "totalTokenCount": 1906,
    "promptTokensDetails": [
      {
        "modality": "TEXT",
        "tokenCount": 1819
      }
    ],
    "candidatesTokensDetails": [
      {
        "modality": "TEXT",
        "tokenCount": 87
      }
    ]
  }
}

Note: There is no function called get_weather, but there is a function called getWeather, so it hallucinated the function, too

Any advice?

About this issue

  • Original URL
  • State: open
  • Created a year ago
  • Reactions: 1
  • Comments: 15 (6 by maintainers)

Most upvoted comments

Currently, the gemini-2.5-flash-preview-native-audio-dialog model canโ€™t invoke function tools. Additionally, when the gemini-2.0-flash-live-001 model does trigger function tools, however it generates a tool_outputs string block at the beginning.

Iโ€™m running into this issue as well with the native audio dialog models

I have tried a couple times, can not really reproduce the issue, I think you also mentioned it is sometimes responses are text of the function call, suspecting it is a model related behavior, created a bug internally to investigate. ๐Ÿ˜ƒ

Depending on the model used, sometimes responses are text of the function call, but not the actual function call. In this case, it made up a non-existent function:

{
  "candidates": [
    {
      "content": {
        "parts": [
          {
            "text": "```tool_code\nprint(default_api.dateTime(timezone = \"UTC\"))\n```"
          }
        ],
        "role": "model"
      },
      "finishReason": "STOP",
      "avgLogprobs": -0.00004641643026843667
    }
  ],
  "modelVersion": "gemini-2.0-flash-001",
  "usageMetadata": {
    "promptTokenCount": 2880,
    "candidatesTokenCount": 20,
    "totalTokenCount": 2900,
    "promptTokensDetails": [
      {
        "modality": "TEXT",
        "tokenCount": 2880
      }
    ],
    "candidatesTokensDetails": [
      {
        "modality": "TEXT",
        "tokenCount": 20
      }
    ]
  }
}

Ohhhhh, thank you for the clarification, I think I understand your issue a lot better now. ๐Ÿ˜ƒ You meant that instead of giving a FunctionCall object, it instead gives the function call as text.

Thank you for surfacing this issue. Let me take a close look on this issue.

Can you explain a bit more on:

Function calling was working for me last week? The automatic function calling is not supported if the the tool given is not an mcp client. So when you say the function calling was working, do you mean that there was a FunctionCall generated in the response from the model?

Can you paste your test case, I can try to reproduce it?

Thank you ๐Ÿ˜ƒ