google-cloud-dotnet: TailLogEntries crashes every hour
I understand that TailLogEntries
is pre-GA, but maybe it will help.
Environment details
- OS: Ubuntu 20.04
- .NET version: 5.0.9
- Package name and version: Google.Cloud.Logging.V2 3.4.0
Steps to reproduce
Code:
LoggingServiceV2Client client = await _gcpLoggingClientBuilder.BuildAsync(ct);
CallSettings callsettings = CallSettings.FromExpiration(Expiration.None);
LoggingServiceV2Client.TailLogEntriesStream response = client.TailLogEntries(callsettings);
ct.Register(async () => await response.WriteCompleteAsync());
Task responseHandlerTask = Task.Run(async () =>
{
AsyncResponseStream<TailLogEntriesResponse> responseStream = response.GetResponseStream();
await foreach (TailLogEntriesResponse responseItem in responseStream)
{
for (int i = 0; i < responseItem.Entries.Count; i++)
{
LogEntry entry = responseItem.Entries[i];
await SendMessageAsync(entry, ct);
}
}
}, CancellationToken.None);
TailLogEntriesRequest request = new()
{
ResourceNames = { _resourceName },
Filter = _filter
};
await response.WriteAsync(request);
await responseHandlerTask;
foreach
over responseStream
fails every hour with:
Grpc.Core.RpcException: Status(StatusCode="Unavailable", Detail="The service is currently unavailable."
Grpc.Core.RpcException: Status(StatusCode="Unavailable", Detail="Connection reset by peer"
Without CallSettings callsettings = CallSettings.FromExpiration(Expiration.None);
the error was
Grpc.Core.RpcException: Status(StatusCode="DeadlineExceeded", Detail="Deadline Exceeded"
UPD: Recently found that service just stopped receiving responses without any error
UPD: FYI we are doing strip --strip-debug libgrpc_csharp_ext.x64.so
for builds, because this file is very huge. Could it be the reason? I think this is connected with #4813
About this issue
- Original URL
- State: closed
- Created 3 years ago
- Comments: 16 (8 by maintainers)
As a side note (we are still looking into surfacing better errors and always bubling them up), the API team has recommended a different, more suitable setup for your use case, and that’s to export log entries to Pub/Sub for further processing. The tailing feature is intended for more interactive scenarios where a human is looking at the logs for a period of time.