amazon-redshift-utils: Exception 'unpack_from' in ColumnEncoding only with some tables

I’ve just cloned from the master branch and am using the ColumnEncoding script through Docker by running

docker run --net host --rm -it --env-file scaling-test.env amazon-redshift-utils column-encoding

For some of the Redshift tables, I get the following error:

-- [6] [6] Running insert into public."example_table_$mig"  select * from public."example_table" order by timestamp;
Exception unpack_from requires a buffer of at least 5 bytes during analysis of example_table
Traceback (most recent call last):
  File "ColumnEncodingUtility/analyze-schema-compression.py", line 788, in analyze
    if not run_commands(get_pg_conn(), statements):
  File "ColumnEncodingUtility/analyze-schema-compression.py", line 383, in run_commands
    conn.rollback()
  File "/usr/local/lib/python2.7/site-packages/pg8000/core.py", line 1545, in rollback
    self.execute(self._cursor, "rollback", None)
  File "/usr/local/lib/python2.7/site-packages/pg8000/core.py", line 1741, in execute
    self.handle_messages(cursor)
  File "/usr/local/lib/python2.7/site-packages/pg8000/core.py", line 1875, in handle_messages
    code, data_len = ci_unpack(self._read(5))
error: unpack_from requires a buffer of at least 5 bytes

I haven’t identified any pattern among the tables that cause the issue.

About this issue

  • Original URL
  • State: closed
  • Created 6 years ago
  • Comments: 26 (18 by maintainers)

Most upvoted comments

This seems to be a timing issue. If I wait a few seconds after the docker container starts up, this error doesn’t appear.

is there any update? How did you solve this issue?

I’ve the same issue, it looks to be related to pg8000