dotnet-docs-samples: Gemini 1.5 Pro via Vertex AI API gives StatusCode="Unavailable" System.IO.IOException
Hello,
I’m using Google.Cloud.AIPlatform.V1 v2.27.0 to use Gemini 1.5 Pro via Vertex AI but usually I get System.IO.IOException.
This code is from the above example:
using PredictionServiceClient.StreamGenerateContentStream response = predictionServiceClient.StreamGenerateContent(generateContentRequest);
StringBuilder fullText = new();
AsyncResponseStream<GenerateContentResponse> responseStream = response.GetResponseStream();
await foreach (GenerateContentResponse responseItem in responseStream)
{
This is the exception:
Status(StatusCode="Unavailable", Detail="Error reading next message. IOException: The request was aborted. IOException: The response ended prematurely while waiting for the next frame from the server.", DebugException="System.IO.IOException: The request was aborted.")
The project is running on Ubuntu with .NET 7.
About this issue
- Original URL
- State: closed
- Created 3 years ago
- Comments: 15 (10 by maintainers)
I believe this is related to https://github.com/grpc/grpc-dotnet/issues/2358
Two workarounds - these are alternatives to each other:
AppContext.SetSwitch("System.Net.SocketsHttpHandler.Http2FlowControl.DisableDynamicWindowSizing", true);GrpcAdapter = GrpcCoreAdapter.Instance@jskeet I reproduced the issue @fatihyildizhan reported.
If you change the model name in
GeminiQuickstartto this:And run the sample, it will fail most of the time with this error:
Once in a while, the sample works fine (maybe 1 out of 5 or so). There seems to be a problem between the GAPIC library and the Gemini 1.5 endpoint when streaming is being used.
I don’t know how to further debug this, so assigning back to you for recommendations.