rails: Aws::S3::Errors::InvalidRange exception while saving ActiveStorage model with attached 0-byte length file

I have a Rails app that is using ActiveStorage direct upload to upload files to Amazon S3. If a 0-byte length file is uploaded, the saving of the ActiveStorage model (i.e. has the has_attached association) raises an “Aws::S3::Errors::InvalidRange (The requested range is not satisfiable)” exception.

Steps to reproduce

Sample Rails app demonstrating the issue is attached. Before running, make sure to add your AWS credentials to config/storage.yml.

Run the sample Rails app. Default page is a “Posts” page listing posts in the database. Click the “New Post” link. Enter a title, body, and choose a 0-byte length file. Click the"Create Post" button The file is correctly uploaded to S3, and the form is correctly posted to the Rails server, however the following exception happens while saving the ActiveStorage model instance:

Aws::S3::Errors::InvalidRange (The requested range is not satisfiable):

app/controllers/posts_controller.rb:36:in `block in create'
app/controllers/posts_controller.rb:35:in `create'

Expected behavior

The ActiveStorage model instance should get saved to the database, and no exception should be raised.

Actual behavior

The ActiveStorage model instance is not saved to the database. A “Aws::S3::Errors::InvalidRange (The requested range is not satisfiable)” is raised during the saving of the instance.

System configuration

Rails version: Rails 5.2.1

Ruby version: ruby 2.4.1p111 (2017-03-22 revision 58053) [x86_64-darwin17]

direct_upload.zip

About this issue

Original URL
State: closed
Created 6 years ago
Comments: 15 (12 by maintainers)

Commits related to this issue

Merge pull request #33795 from marceloperini/marceloperini/33450-active-storage Fix zero-byte files upload in #33630 — committed to rails/rails by rafaelfranca 6 years ago
Merge pull request #33795 from marceloperini/marceloperini/33450-active-storage Fix zero-byte files upload in #33630 — committed to rails/rails by rafaelfranca 6 years ago

Most upvoted comments

Does uploading of file that has a size smaller than 4KB but bigger than 0 bytes work?

bogdanvlviv on Sep 17, 2018

@marceloperini The issue I was experiencing no longer happens for me with the 5-2-stable branch. I am now able to upload a 0 byte file to S3, and the associated ActiveStorage records are getting created correctly in the database.

denisahearn on Sep 17, 2018

@janko-m It probably should fix that case?

diff --git a/activestorage/app/models/active_storage/blob/identifiable.rb b/activestorage/app/models/active_storage/blob/identifiable.rb
index 2c17ddc25f..fadc2dcab4 100644
--- a/activestorage/app/models/active_storage/blob/identifiable.rb
+++ b/activestorage/app/models/active_storage/blob/identifiable.rb
@@ -15,10 +15,10 @@ def identify_content_type
     end

     def download_identifiable_chunk
-      if byte_size.positive?
-        service.download_chunk key, 0...4.kilobytes
-      else
-        ""
-      end
+      bytes = 4.kilobytes
+
+      bytes = byte_size if bytes > byte_size
+
+      service.download_chunk key, 0...bytes
     end
 end

or bytes=0-0 doesn’t work either?

bogdanvlviv on Sep 12, 2018

It’s probably coming from ActiveStorage::Blob::Identifiable#download_identifiable_chunk, which downloads a fixed size: https://github.com/rails/rails/blob/53eea40d354dafbeda1703aa0e4b4a688bc26017/activestorage/app/models/active_storage/blob/identifiable.rb#L17-L19

This then makes a download request with range: "bytes=0-4096", which blows up with Aws::S3::Errors::InvalidRange when the file is smaller than 4KB due to bytes being requested which don’t exist. This includes empty files.

The following script shows that this is indeed how aws-sdk-s3 reacts:

require "aws-sdk-s3"

resource = Aws::S3::Resource.new(
  access_key_id:     "...",
  secret_access_key: "...",
  region:            "...",
)
bucket = resource.bucket("...")
object = bucket.object("foo")

object.put(body: "") # create empty object
object.get(range: "bytes=0-10") # => Aws::S3::Errors::InvalidRange: The requested range is not satisfiable

janko on Sep 2, 2018