npgsql: NRE when calling Complete() in NpgsqlBinaryImporter

Not sure what we want to do here, but one option (#2398) is to just guard against the NRE and surface the underlying exception.

I think the NRE guard is good enough. An early design decision was to handle any I/O exceptions and break the connector in one, low-level place (i.e. NpgsqlReaderBuffer.Ensure()), meaning that high-level layers (such as copy) must be ready to deal with already-broken connectors.

We should backport this to 4.0.6, no?

I was able to reproduce this issue without any concurrency.

var csb = new NpgsqlConnectionStringBuilder
{
    Host = "...",
    Username = "...",
    Password = "...",
    Database = "...",
    CommandTimeout = 3 // <--
};
using (var conn = new NpgsqlConnection(csb.ConnectionString))
{
    conn.Open();
    using (var tx = conn.BeginTransaction())
    {
        const int rowCount = 1000000;
        using (var cmd = conn.CreateCommand())
        {
            cmd.CommandText = $"CREATE TABLE public.master AS SELECT * FROM generate_series(1, {rowCount}) id";
            cmd.ExecuteNonQuery();
            cmd.CommandText = "ALTER TABLE public.master ADD CONSTRAINT master_pk PRIMARY KEY (id)";
            cmd.ExecuteNonQuery();
            cmd.CommandText = "CREATE TABLE public.detail (master_id integer NOT NULL, CONSTRAINT detail_fk FOREIGN KEY(master_id) REFERENCES master(id))";
            cmd.ExecuteNonQuery();
        }
        using (var writer = conn.BeginBinaryImport("COPY public.detail FROM STDIN BINARY"))
        {
            for (var i = 1; i <= rowCount; ++i)
            {
                writer.StartRow();
                writer.Write(i);
            }
            writer.Complete(); // <-- NullReferenceException
        }
        //tx.Commit();
    }
}

The NullReferenceException is raised in the SimpleFrontendMessage.Write (line 80) if the following conditions are met:

1. The NpgsqlConnectionStringBuilder.CommandTimeout parameter is greater than 0.
2. Any command is executed before importing data into the table.
3. Time spent in the NpgsqlBinaryImporter.Complete line 304 (Expect<CommandCompleteMessage>(_connector.ReadMessage());) is greater, than the NpgsqlConnectionStringBuilder.CommandTimeout parameter value.

Npgsql 4.0.5 PostgreSQL 11.2 Ubuntu 18.04.2

@akusmin this kind of error has been reported several times in the past, and has usually been a result of an application concurrency error, where the same connection is being used at the same time by multiple threads. I’m not saying that there isn’t an Npgsql-side issue, but we need to see some sort of repro in order to work on it. Usually this kind of thing should be reproducible by running a tight loop that does the same action over and over (possibly from multiple threads) until the error is triggered.

OK.

I can recommend trying to reproduce in a small code sample (console app preferably), this is usually possible by doing an endless loop and repeating the action, and if indeed there’s a race condition in there it pops up sooner or later. This also allows us to reproduce the issue on our side and find out the root cause.

Closing this for now as suggested, but please report back with any results.