google-cloud-go: spanner: transaction fails with `Session not found` error
Client
Spanner: v0.37.4 (But as far as I know, it also occurs when we use latest version)
Describe Your Environment
Alpine Docker on GKE
Expected Behavior
If possible, this client library should retries a transaction when it fails with Session not found error.
Actual Behavior
We sometimes get the Session not found errors.
This client library retries transactions only when the Abort error occurred, so when taken session is not active, it just returns the NotFound error to callers without retrying.
- https://github.com/googleapis/google-cloud-go/blob/e4bd323326abdfd5f8bd53587a92c1ba2d69a6cc/spanner/client.go#L394-L398
- https://github.com/googleapis/google-cloud-go/blob/e4bd323326abdfd5f8bd53587a92c1ba2d69a6cc/spanner/client.go#L439-L454
Since this library creates the pool of the sessions, I think it should retry failed transactions caused by Session not found by taking another session. But are there any problems to take another session from the pool?
About this issue
- Original URL
- State: closed
- Created 5 years ago
- Comments: 22 (18 by maintainers)
Commits related to this issue
- spanner: Retry "Session not found" for read-only transactions Session not found errors should be retried by taking a new session from the pool and retrying the gRPC call when that is possible. This c... — committed to googleapis/google-cloud-go by olavloite 5 years ago
- spanner: make ReadWriteTransaction retry on Session not found error Updates #1527 Ref: https://github.com/googleapis/google-cloud-go/issues/1527 Change-Id: Iea12342ca098c8056abc2206b91edbeda630e718 ... — committed to googleapis/google-cloud-go by 110y 5 years ago
- spanner: retry SessionNotFound on BeginTransaction for read-only tx 'Session not found' errors on BeginTransaction calls for a read-only transaction should be retried on a new session, and the invali... — committed to googleapis/google-cloud-go by olavloite 4 years ago
- spanner: retry single use transactions on SessionNotFound Single use read-only transactions should be retried if the query returns a 'Session not found' error. These queries can safely be retried on ... — committed to googleapis/google-cloud-go by olavloite 4 years ago
@110y Yes (and additional transaction types).
@110y and @kazegusuri
Thanks for updating your CL and sorry for taking so long to merge this. I’ll have a look at this this morning and try to get it in ASAP.
@olavloite
As trial, I send a CL that makes
ReadWriteTransactionretry onSession not founderror (case2you mentioned above).@110y To my understanding, it is possible (although not very common) that Cloud Spanner deletes sessions that have been idle for less than 1 hour, which could cause this problem. The reason I asked whether you had a specific use case that would always (or often) cause this problem, was to check whether you were running into some unknown bug in the session pool. Considering your error rate of 1-2 errors per week at 1QPS, I don’t think that this is a specific bug in the Go session pool.
The Java client library for Cloud Spanner added a protection against this problem a couple of months ago along the lines that I mentioned above. I’ll have a look to see if it is feasible to add this protection for the Go client library as well.