generative-ai-python: ValueError: The `response.text` quick accessor only works for simple (single-`Part`) text responses. This response is not simple text.Use the `result.parts` accessor or the full `result.candidates[index].content.parts` lookup instead.

You could try to delete max_output_tokens generation_config in the model if you use that

Hi @Ki-Zhang ,

When you set up your model, the generation_config is in it. Try ignoring it like this.

@Ki-Zhang As of January 2024, the entire list of Harm Categories can be found here. The implementation for gemini-pro or gemini-pro-vision can be carried out as follows in Python:

safety_settings = [
    {
        "category": "HARM_CATEGORY_DANGEROUS",
        "threshold": "BLOCK_NONE",
    },
    {
        "category": "HARM_CATEGORY_HARASSMENT",
        "threshold": "BLOCK_NONE",
    },
    {
        "category": "HARM_CATEGORY_HATE_SPEECH",
        "threshold": "BLOCK_NONE",
    },
    {
        "category": "HARM_CATEGORY_SEXUALLY_EXPLICIT",
        "threshold": "BLOCK_NONE",
    },
    {
        "category": "HARM_CATEGORY_DANGEROUS_CONTENT",
        "threshold": "BLOCK_NONE",
    },
]

values for each category can be found here

These settings can be applied as:

# For image model 
image_model.generate_content([your_image, prompt], safety_settings=safety_settings)

# For text model 
text_model.generate_content(prompt, safety_settings=safety_settings)

Additionally, make sure the image does not contain content related to openAI or chatgpt. Otherwise, it may result in an error. Screenshots taken through the default Snipping Tool on Windows might also lead to such errors.

I think the main reason is the model doesn’t return any text in sometimes, based on the explanation of @MarkDaoust in https://github.com/google/generative-ai-python/issues/196#issuecomment-1930503073

I still get this error with my code even when I delete generation_config.

But when I set up

for candidate in response.candidates:
    return [part.text for part in candidate.content.parts]

instead of response.text

This error does not appear anymore.

@Ki-Zhang As of January 2024, the entire list of Harm Categories can be found here. The implementation for gemini-pro or gemini-pro-vision can be carried out as follows in Python:

safety_settings = [
    {
        "category": "HARM_CATEGORY_DANGEROUS",
        "threshold": "BLOCK_NONE",
    },
    {
        "category": "HARM_CATEGORY_HARASSMENT",
        "threshold": "BLOCK_NONE",
    },
    {
        "category": "HARM_CATEGORY_HATE_SPEECH",
        "threshold": "BLOCK_NONE",
    },
    {
        "category": "HARM_CATEGORY_SEXUALLY_EXPLICIT",
        "threshold": "BLOCK_NONE",
    },
    {
        "category": "HARM_CATEGORY_DANGEROUS_CONTENT",
        "threshold": "BLOCK_NONE",
    },
]

values for each category can be found here

Threshold (Google AI Studio) Threshold (API) Description Block none BLOCK_NONE Always show regardless of probability of unsafe content Block few BLOCK_ONLY_HIGH Block when high probability of unsafe content Block some BLOCK_MEDIUM_AND_ABOVE Block when medium or high probability of unsafe content Block most BLOCK_LOW_AND_ABOVE Block when low, medium, or high probability of unsafe content HARM_BLOCK_THRESHOLD_UNSPECIFIED Threshold is unspecified, block using default threshold These settings can be applied as:

# For image model 
image_model.generate_content([your_image, prompt], safety_settings=safety_settings)

# For text model 
text_model.generate_content(prompt, safety_settings=safety_settings)

Additionally, make sure the image does not contain content related to openAI or chatgpt. Otherwise, it may result in an error. Screenshots taken through the default Snipping Tool on Windows might also lead to such errors.

Thanks for provding this! However, the safety settings does not work for me, instead, changing temperature from 0 to 0.7 works. The generated contents may be blocked since I found my input question is about black people (from MMLU dataset).

So this is caused because content was blocked on the server-side? If so, the thrown exception text is terrible.

Thank you for your answer @HienBM But I just used model = genai.GenerativeModel('gemini-pro-vision') to simply set up the genimi-pro-vision model. And I encountered the same problem.

Cell In[31], line 23, in dectect_object_why(ori_img, head_pos, gaze_pos)
      8 response = model.generate_content(
      9     [
     10         "The person outlined in the blue frame is looking at what object is marked with the red circle in the picture?",
   (...)
     20     stream=True
     21 )
     22 response.resolve()
---> 23 to_markdown(response.text)
     24 return response.text

File ~/miniconda3/envs/gemini/lib/python3.9/site-packages/google/generativeai/types/generation_types.py:328, in BaseGenerateContentResponse.text(self)
    326 parts = self.parts
    327 if len(parts) != 1 or "text" not in parts[0]:
--> 328     raise ValueError(
    329         "The `response.text` quick accessor only works for "
    330         "simple (single-`Part`) text responses. This response is not simple text."
    331         "Use the `result.parts` accessor or the full "
    332         "`result.candidates[index].content.parts` lookup "
    333         "instead."
    334     )
    335 return parts[0].text

ValueError: The `response.text` quick accessor only works for simple (single-`Part`) text responses. This response is not simple text.Use the `result.parts` accessor or the full `result.candidates[index].content.parts` lookup instead.

I don’t know how to solve this problem. But this problem does not occur when I use other image examples to input the model.