aws-record-ruby: rake aws_record:migrate broken with two table configs on the same model

When creating two global secondary indexes for the same model, this leads to the following error message:

$ rake aws_record:migrate
running db/table_config/gsi_user_login.rb
running db/table_config/user_email_gsi.rb
Aws::DynamoDB::Errors::LimitExceededException: Subscriber limit exceeded: Only 1 online index can be created or deleted simultaneously per table (Aws::DynamoDB::Errors::LimitExceededException)

It gets especially weird: When I uncomment one global_secondary_index from the user model, rake aws_record:migrate runs just fine (so it’s not the db/table_config files). When I then Re-Insert the global_secondary_index in the user model and run rake aws_record:migrate again, all indexes are created correctly.

So the gem can’t correctly handle if two global_secondary_indexes are defined in the same model and the migrations will be run after each other.

class User
  include Aws::Record

  string_attr :user_email
  string_attr :user_login

  global_secondary_index(:user_email_gsi,
                         hash_key: :user_email,
                         projection: { projection_type: 'ALL' })

  global_secondary_index(:user_login_gsi,
                         hash_key: :user_login,
                         projection: { projection_type: 'ALL' })

end

db/table_config/gsi_user_login.rb

require "aws-record"

module ModelTableConfig
  def self.config
    Aws::Record::TableConfig.define do |t|
      t.model_class User
      t.billing_mode "PAY_PER_REQUEST"
      t.global_secondary_index(:user_login_gsi)
    end
  end
end

db/table_config/user_email_gsi.rb

require "aws-record"

module ModelTableConfig
  def self.config
    Aws::Record::TableConfig.define do |t|
      t.model_class User
      t.billing_mode "PAY_PER_REQUEST"
      t.global_secondary_index(:user_email_gsi)
    end
  end
end

About this issue

  • Original URL
  • State: closed
  • Created 2 years ago
  • Comments: 24 (11 by maintainers)

Most upvoted comments

Yeah, you’re right that the TableConfig just does not map well to STI - I believe it was added in the library well before STI became a common/recommended pattern and we haven’t added better abstractions to account for it.

The way you have structured your BaseModel and User, ect models using the same table make sense to me - I think thats generally the correct way to do that.

As to the TableConfig though - it does really fall apart. Its an area of this gem that I don’t personally love - in my own projects, I’ve generally preferred using cloudwatch/cdk to manage the infrastructure… Anyway, that being said, I’d potentially recommend creating a “migration” only model that represents the desired state of the table, but doesn’t have domain behavior (ie, letting you define the name/primary/secondary keys/gsi and nothing else. I know thats not ideal though. Improving this libraries STI functionality has been on my general list for awhile, but the changes I’d want to do would definitely require a new major version.

Thanks for the reproduction steps. I’ll try to recreate this on Monday.

Deletion and creation in the same task is fine. My point is that rescuing from the base service error is fairly broad - you probably want to handle cases where deletion failed, or creation failed. Somehow I think update_table is being called. From what I can find, that error you are getting is from update_table called by migrate! in the table config. I think we just need to track down what calls are made there with a logger/wire trace.

As far as waiting goes, if the table doesn’t exist, it should rescue and return nil immediately for you:

[3] pry(Aws)> dynamodb.wait_until(:table_not_exists, table_name: 'asdf')
[Aws::DynamoDB::Client 400 0.318157 0 retries] describe_table(table_name:"asdf") Aws::DynamoDB::Errors::ResourceNotFoundException Requested resource not found: Table: asdf not found
=> nil