npgsql: Large multi statement commands can lead to deadlock when doing sync on the TP

I’ve noticed large multi statement commands can lead to a deadlock condition on connection close, where the thread is waiting on the Command.RemainingSendTask to complete but the task never gets to run as the TP becomes exhausted. This can also lead to connector pool exhaustion.

I am running 3.1.7, can also reproduce in hotfix/3.1.8 and 3.2.0.

deadlock

Here is a test case that will reproduce the issue:

using System;
using System.Collections.Generic;
using System.Diagnostics;
using System.Threading;
using Npgsql;

namespace TransactionTests {
    public class Program {
        private const string CS = "DbConnectionString";
        private static readonly string _bigOldString = new string('x', 10000);

        public static void Main(string[] args) {

            var threads = new List<ManualResetEventSlim>();
            var startEvent = new ManualResetEventSlim(false);
            for (int i = 0; i < 100000; i++) {
                var num = i;
                var doneEvent = new ManualResetEventSlim();
                ThreadPool.QueueUserWorkItem((x) => Do(num, doneEvent, startEvent));
                threads.Add(doneEvent);
            }
            Thread.Sleep(TimeSpan.FromSeconds(1));
            startEvent.Set();
            foreach (var thread in threads) {
                thread.Wait();
            }
            Console.ReadLine();
        }

        static void Do(int i, ManualResetEventSlim done, ManualResetEventSlim start) {
            start.Wait();
            using (var conn = new NpgsqlConnection(CS)) {
                conn.Open();
                var command = conn.CreateCommand();
                command.CommandText = $"SELECT repeat('{_bigOldString}',10) AS a; SELECT '{_bigOldString}' AS b;";
                var sw = Stopwatch.StartNew();
                using (var reader = command.ExecuteReader()) {
                    reader.Read();
                    var a = reader.GetString(0).Length;

                    int b = 0;
                    while (reader.Read()) {
                        b++;
                    }
                    Console.WriteLine($"{i}, {sw.Elapsed}, {a + b}");
                }
                Thread.Sleep(100);
            }
            done.Set();
        }
    }
}

About this issue

  • Original URL
  • State: closed
  • Created 8 years ago
  • Comments: 38 (33 by maintainers)

Commits related to this issue

Most upvoted comments

@roji When I tested the reverse, with Ubuntu as the client and Windows as the server, it behaved as you explained to me. WriteAsync was blocking until the server read the data (enough to drain the buffer so more could be sent). So I guess what I’m seeing is a windows networking “quirk” that shouldn’t be relied upon. This means then my idea is not useful here.