google-cloud-ruby: Spanner: Long running queries on snapshots hit Google::Cloud::DeadlineExceededError

While looping through rows on a either a very large table or over a low-bandwidth connection I get:

Traceback (most recent call last):
	21: from migrate.rb:145:in `<main>'
	20: from /Users/untoldone/.rvm/gems/ruby-2.5.0@spanner-to-cockroach/gems/google-cloud-spanner-1.6.0/lib/google/cloud/spanner/client.rb:905:in `snapshot'
	19: from /Users/untoldone/.rvm/gems/ruby-2.5.0@spanner-to-cockroach/gems/google-cloud-spanner-1.6.0/lib/google/cloud/spanner/pool.rb:57:in `with_session'
	18: from /Users/untoldone/.rvm/gems/ruby-2.5.0@spanner-to-cockroach/gems/google-cloud-spanner-1.6.0/lib/google/cloud/spanner/client.rb:913:in `block in snapshot'
	17: from migrate.rb:151:in `block in <main>'
	16: from migrate.rb:151:in `each'
	15: from /Users/untoldone/.rvm/gems/ruby-2.5.0@spanner-to-cockroach/gems/google-cloud-spanner-1.6.0/lib/google/cloud/spanner/results.rb:177:in `rows'
	14: from /Users/untoldone/.rvm/gems/ruby-2.5.0@spanner-to-cockroach/gems/google-cloud-spanner-1.6.0/lib/google/cloud/spanner/results.rb:177:in `each'
	13: from /Users/untoldone/.rvm/gems/ruby-2.5.0@spanner-to-cockroach/gems/google-cloud-spanner-1.6.0/lib/google/cloud/spanner/results.rb:185:in `block in rows'
	12: from /Users/untoldone/.rvm/gems/ruby-2.5.0@spanner-to-cockroach/gems/google-cloud-spanner-1.6.0/lib/google/cloud/spanner/results.rb:185:in `each_slice'
	11: from /Users/untoldone/.rvm/gems/ruby-2.5.0@spanner-to-cockroach/gems/google-cloud-spanner-1.6.0/lib/google/cloud/spanner/results.rb:185:in `each'
	10: from /Users/untoldone/.rvm/gems/ruby-2.5.0@spanner-to-cockroach/gems/google-cloud-spanner-1.6.0/lib/google/cloud/spanner/results.rb:186:in `block (2 levels) in rows'
	 9: from migrate.rb:153:in `block (2 levels) in <main>'
	 8: from migrate.rb:103:in `backup_table'
	 7: from /Users/untoldone/.rvm/rubies/ruby-2.5.0/lib/ruby/2.5.0/csv.rb:1289:in `open'
	 6: from migrate.rb:113:in `block in backup_table'
	 5: from migrate.rb:113:in `to_a'
	 4: from migrate.rb:113:in `each'
	 3: from /Users/untoldone/.rvm/gems/ruby-2.5.0@spanner-to-cockroach/gems/google-cloud-spanner-1.6.0/lib/google/cloud/spanner/results.rb:119:in `rows'
	 2: from /Users/untoldone/.rvm/gems/ruby-2.5.0@spanner-to-cockroach/gems/google-cloud-spanner-1.6.0/lib/google/cloud/spanner/results.rb:119:in `loop'
	 1: from /Users/untoldone/.rvm/gems/ruby-2.5.0@spanner-to-cockroach/gems/google-cloud-spanner-1.6.0/lib/google/cloud/spanner/results.rb:120:in `block in rows'
/Users/untoldone/.rvm/gems/ruby-2.5.0@spanner-to-cockroach/gems/google-cloud-spanner-1.6.0/lib/google/cloud/spanner/results.rb:170:in `rescue in block in rows': 4:Deadline Exceeded (Google::Cloud::DeadlineExceededError)

Code is pretty strait forward:

spanner  = Google::Cloud::Spanner.new project: GOOGLE_PROJECT
client  = spanner.client SPANNER_INSTANCE_ID, SPANNER_DATABASE_ID
client.snapshot do |snapshot|
  results = snapshot.execute("SELECT * FROM table_name_here")
  results.rows.each do |row|
    # Do some stuff
  end
end

I couldn’t find anywhere documented or undocumented that I could easily extend the deadline to prevent this. I would have expected either the library handles or prevents this error or at a minimum allows me to override the default deadline.

About this issue

  • Original URL
  • State: closed
  • Created 6 years ago
  • Comments: 21 (13 by maintainers)

Most upvoted comments

This is fixed in Spanner 1.7.2. Thank you for your help! If you continue to have problems like this please open a new issue and reference this one.

Great news! I will create a PR to merge this change and release it.

Unfortunately, the exact version of google-api-client you are using had the issue and it was fixed in the next version, released a day later.