clickhouse-go: Flush batch cause panic

Issue description

Using Flush() method in clickhouse.Batch causes panic

Error log

panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x8 pc=0x895d0b]

goroutine 12 [running]:
github.com/ClickHouse/ch-go/proto.(*Buffer).PutUInt8(...)
	/go/pkg/mod/github.com/!click!house/ch-go@v0.47.3/proto/buffer.go:97
github.com/ClickHouse/ch-go/proto.(*Buffer).PutByte(...)
	/go/pkg/mod/github.com/!click!house/ch-go@v0.47.3/proto/buffer.go:82
github.com/ClickHouse/clickhouse-go/v2.(*connect).sendData(0xc000146270, 0xc00011e1c0?, {0x0, 0x0})
	/go/pkg/mod/github.com/!click!house/clickhouse-go/v2@v2.3.0/conn.go:171 +0xcb
github.com/ClickHouse/clickhouse-go/v2.(*batch).Flush(0xc00014e280)
	/go/pkg/mod/github.com/!click!house/clickhouse-go/v2@v2.3.0/conn_batch.go:165 +0x79

Configuration

OS: Alpine Linux 3.16

Interface: native

Go version: 1.19

ClickHouse Server version: 22.9.3.18

About this issue

  • Original URL
  • State: closed
  • Created 2 years ago
  • Comments: 34

Most upvoted comments

Done v.2.4.3 is out. Let me know if this doesn’t resolve the issue @moming00

Its ok ill do a PR and release shortly. Please test.

This is a different issue to the others reported I think - the fact it occurs on prepareBatch not when a batch is reused after an error. I think this happens if the connection to ClickHouse errors on any single go routine - this is causing a segmentation fault on other routines. trying to figure out how this happens.

Yes it should be safe to use in different threads…this reproduces your issue…

func writeRows(prepareSQL string, rows [][]interface{}, conn clickhouse.Conn) (err error) {
	batch, err := conn.PrepareBatch(context.Background(), prepareSQL)
	defer func() {
		_ = batch.Abort()
	}()
	for _, row := range rows {
		batch.Append(row...)
	}
	return batch.Send()
}

func Test798Concurrent(t *testing.T) {

	var (
		conn, err = clickhouse_tests.GetConnection("issues", clickhouse.Settings{
			"max_execution_time": 60,
		}, nil, &clickhouse.Compression{
			Method: clickhouse.CompressionLZ4,
		})
	)
	ctx := context.Background()
	const ddl = `
			CREATE TABLE test_issue_798 (
				  Col1 Bool
				, Col2 Bool
			) Engine MergeTree() ORDER BY tuple()
		`
	defer func() {
		conn.Exec(ctx, "DROP TABLE IF EXISTS test_issue_798")
	}()
	require.NoError(t, conn.Exec(ctx, ddl))

	require.NoError(t, err)
	todo, done := int64(10000), int64(-1)
	var wg sync.WaitGroup
	wg.Add(10)
	for i := 0; i < 10; i++ {
		go func() {
			for {
				index := atomic.AddInt64(&done, 1)
				if index >= todo {
					wg.Done()
					break
				}
				writeRows("INSERT INTO test_issue_798", [][]interface{}{{true, false}, {false, true}}, conn)
			}
		}()
	}
	wg.Wait()

}

hmm, good catch. I think there is some other trappy behavior here -

  1. empty batch causes panic - this should just do nothing. Null op.
  2. Calling Send twice or after an error on Send shouldn’t panic
  3. Reuse of batch after Send shouldn’t panic.

Protecting against these will probably help a lot of the reports here.