google-cloud-dotnet: TailLogEntries crashes every hour

I understand that TailLogEntries is pre-GA, but maybe it will help.

Environment details

  • OS: Ubuntu 20.04
  • .NET version: 5.0.9
  • Package name and version: Google.Cloud.Logging.V2 3.4.0

Steps to reproduce

Code:

LoggingServiceV2Client client = await _gcpLoggingClientBuilder.BuildAsync(ct);

CallSettings callsettings = CallSettings.FromExpiration(Expiration.None);
LoggingServiceV2Client.TailLogEntriesStream response = client.TailLogEntries(callsettings);
ct.Register(async () => await response.WriteCompleteAsync());

Task responseHandlerTask = Task.Run(async () =>
{
    AsyncResponseStream<TailLogEntriesResponse> responseStream = response.GetResponseStream();
    await foreach (TailLogEntriesResponse responseItem in responseStream)
    {
        for (int i = 0; i < responseItem.Entries.Count; i++)
        {
            LogEntry entry = responseItem.Entries[i];
            await SendMessageAsync(entry, ct);
        }
    }
}, CancellationToken.None);

TailLogEntriesRequest request = new()
{
    ResourceNames = { _resourceName },
    Filter = _filter
};
await response.WriteAsync(request);

await responseHandlerTask;

foreach over responseStream fails every hour with: Grpc.Core.RpcException: Status(StatusCode="Unavailable", Detail="The service is currently unavailable." Grpc.Core.RpcException: Status(StatusCode="Unavailable", Detail="Connection reset by peer"

Without CallSettings callsettings = CallSettings.FromExpiration(Expiration.None); the error was Grpc.Core.RpcException: Status(StatusCode="DeadlineExceeded", Detail="Deadline Exceeded"

UPD: Recently found that service just stopped receiving responses without any error

UPD: FYI we are doing strip --strip-debug libgrpc_csharp_ext.x64.so for builds, because this file is very huge. Could it be the reason? I think this is connected with #4813

About this issue

  • Original URL
  • State: closed
  • Created 3 years ago
  • Comments: 16 (8 by maintainers)

Most upvoted comments

As a side note (we are still looking into surfacing better errors and always bubling them up), the API team has recommended a different, more suitable setup for your use case, and that’s to export log entries to Pub/Sub for further processing. The tailing feature is intended for more interactive scenarios where a human is looking at the logs for a period of time.