moto: boto3 + moto + s3 + Docker: "ValueError, "I/O operation on closed file""

This simple test passes locally (when ran using py.test)

@mock_s3
def test_read_data():
    temp = tempfile.NamedTemporaryFile()
    temp.write('Something')
    temp.seek(0)

    s3 = boto3.resource('s3')
    s3.create_bucket(Bucket='mybucket')
    s3.Object('mybucket', 'hello.txt').put(Body=open(temp.name, 'rb'))

    obj = s3.Object(bucket_name='mybucket', key='hello.txt')
    response = obj.get()
    data = response['Body'].read()

    assert data == 'Something'

However, the same test fails when run inside a Docker container (docker build .):

Traceback (most recent call last):
  File "/usr/local/lib/python2.7/site-packages/botocore/endpoint.py", line 168, in _get_response
    proxies=self.proxies, timeout=self.timeout)
  File "/usr/local/lib/python2.7/site-packages/botocore/vendored/requests/sessions.py", line 573, in send
    r = adapter.send(request, **kwargs)
  File "/usr/local/lib/python2.7/site-packages/botocore/vendored/requests/adapters.py", line 370, in send
    timeout=timeout
  File "/usr/local/lib/python2.7/site-packages/botocore/vendored/requests/packages/urllib3/connectionpool.py", line 544, in urlopen
    body=body, headers=headers)
  File "/usr/local/lib/python2.7/site-packages/botocore/vendored/requests/packages/urllib3/connectionpool.py", line 372, in _make_request
    httplib_response = conn.getresponse(buffering=True)
  File "/usr/local/lib/python2.7/httplib.py", line 1129, in getresponse
    response = self.response_class(*args, **kwds)
  File "/usr/local/lib/python2.7/site-packages/botocore/awsrequest.py", line 45, in __init__
    HTTPResponse.__init__(self, *args, **kwargs)
  File "/usr/local/lib/python2.7/httplib.py", line 383, in __init__
    self.fp = sock.makefile('rb')
  File "/usr/local/lib/python2.7/site-packages/httpretty/core.py", line 332, in makefile
    self._entry.fill_filekind(self.fd)
  File "/usr/local/lib/python2.7/site-packages/httpretty/core.py", line 621, in fill_filekind
    fk.write(utf8(item) + b'\n')
  File "/usr/local/lib/python2.7/StringIO.py", line 213, in write
    _complain_ifclosed(self.closed)
  File "/usr/local/lib/python2.7/StringIO.py", line 40, in _complain_ifclosed
    raise ValueError, "I/O operation on closed file"
ValueError: I/O operation on closed file

Maybe this issue should be posted to httppretty instead of here.

Source code: boto3_pytest_docker.zip

About this issue

  • Original URL
  • State: closed
  • Created 9 years ago
  • Comments: 15

Most upvoted comments

I had a similar issue with StringIO and got rid of it by putting the contents of the file instead of the file itself.

Something like this:

output_str = 'Something'
bucket = s3.Bucket('mybucket')
bucket.put_object(Body=output_str, Key='hello.txt')