python-aiplatform: Response are stopped with finishReason: BLOCKLIST way more often than expected
As far as I know the BLOCKLIST finish reason is a fairly new feature and I’ve been experiencing a lot of stopped responses with this reason lately.
Unfortunately I do not have specific steps to reproduce this (as it seems a bit random) but it started happening when I upgraded to gemini-1.0-pro-002 and version 1.47.0 of the SDK.
The prompt I’m using is quite narrowly defined and has not produced any content that is even remotely offensive thus far (with gemini-1.0-pro-001). I found that it happens less with lower temperature settings (<0.2) which I suppose makes sense, but that’s not a solution.
I’m using the GenerativeModel.generate_content_async method to generate content, not sure if that is relevant.
About this issue
- Original URL
- State: closed
- Created 3 years ago
- Reactions: 7
- Comments: 20 (2 by maintainers)
I am also seeing this same issue with multiple different prompts. It seems to happen often, but if you retry the call over and over again, it does occasionally succeed. Here is one example:
The response from gemini comes back like this:
I tried again. The issue is still persisting
I have also experience this issue recently, specifically from tool use with gemini-1.0-pro. I’ve tested sync and async versions extensively and have about the same error rate with both (about 50%). One thing I have noticed is that when I rerun failed attempts they often fail 6-7 times before succeeding, which might indicate that certain prompts are more prone to this issue, but they will eventually succeed.
I also came across quite a few “error 500: internal server error” during my testing which I can’t recall ever seeing in other interactions with vertex. Interestingly, preview gemini-1.5 seems to not have this issue at all, at least from my testing.
Another thing I noticed is that if I set the max input tokens to an amount insufficient to make the function call, it will also return “finish_reason: BLOCKLIST” (and will do so no matter how often you rerun it) which makes it seem like its the standard message displayed when the model is attempting to make a function call but is unable to.
I am using gemini-1.0-pro-001/gemini-1.0-pro-002 for generating the output. The successful is around 50%. if fail, the error is “The candidate is likely blocked by the safety filters” Strangely, I haven’t encountered this issue for the past two weeks, and it only appeared today. The sdk version is “google-cloud-aiplatform==1.50.0”.
python code:
If the job fail, it can generate the responses, but it will block after generated the responses
My understanding is that a server-side fix is on the way, Ill update if I hear anything
I am also experiencing unreasonable amounts of BLOCKLIST triggers. I am using
gemini-1.5-pro-preview-0409My input is mostly code, but here is the response:LMK if anyone found a workaround 😃
Hi, same issue, my prompt also has a function declaration, that might be connected. I’ll post it here if it helps (or forward me to a better place to post bugs):
code used in generation:
The prompt:
The response:
EDIT: same thing happens for gemini-1.5-pro-preview-0409