google-cloud-dotnet: Grpc.Core.RpcException: Status(StatusCode=Unauthenticated, Detail="Deadline Exceeded") ?
We are deploying asp.net core 2.0 project to app engine flex(while previous version is under load: about 200req/sec), and in some cases we are getting “Grpc.Core.RpcException: Status(StatusCode=Unauthenticated, Detail=“Deadline Exceeded”)” while trying to run query on datastore with standard datastore client, sometimes errors are gone and sometimes they are rising again, not sure if it depends on requests rate. So here is our app.yaml:
env: flex
runtime: aspnetcore
resources:
cpu: 16
memory_gb: 14.4
automatic_scaling:
min_num_instances: 8
max_num_instances: 20
cool_down_period_sec: 180
cpu_utilization:
target_utilization: 0.5
And here is full stack trace:
Grpc.Core.RpcException: Status(StatusCode=Unauthenticated, Detail="Deadline Exceeded")
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification
at Grpc.Core.Internal.AsyncCall`2.UnaryCall
at Grpc.Core.DefaultCallInvoker.BlockingUnaryCall[TRequest,TResponse]
at Grpc.Core.Internal.InterceptingCallInvoker.BlockingUnaryCall[TRequest,TResponse]
at Google.Cloud.Datastore.V1.Datastore.DatastoreClient.RunQuery
at Google.Api.Gax.Grpc.ApiCall.<>c__DisplayClass0_0`2.<Create>b__1
at Google.Api.Gax.Grpc.ApiCallRetryExtensions.<>c__DisplayClass1_0`2.<WithRetry>b__0
at Google.Cloud.Datastore.V1.QueryStreamer.<Sync>d__7.MoveNext
at System.Collections.Generic.List`1.AddEnumerable
at System.Linq.Enumerable.ToList[TSource]
at Google.Cloud.Datastore.V1.LazyDatastoreQuery.GetAllResults
About this issue
- Original URL
- State: closed
- Created 6 years ago
- Comments: 140 (2 by maintainers)
At this point I suspect you really need a Datastore expert to help you. I’ll reach out internally to try to find someone who can help.
I’m afraid I don’t know details about Datastore itself - I work on a lot of different APIs, so don’t know very much about each of them. I just make sure you can get requests from C# code to the server…
Here’s a small sample for you for the pooling. You’d create an instance on start-up, then keep that instance everywhere, calling
var db = pool.GetNextInstance();
each time you wanted to make a request. If you have a chance to test that out and see if it improves your latency, that would be really interesting.