pre-commit-hooks: UnicodeDecodeError in detect-aws-credentials is unreadable

Detect AWS Credentials...................................................Failed
hookid: detect-aws-credentials

Traceback (most recent call last):
  File "/home/ryan/.cache/pre-commit/repo8jj7q4iz/py_env-python3.7/bin/detect-aws-credentials", line 11, in <module>
    sys.exit(main())
  File "/home/ryan/.cache/pre-commit/repo8jj7q4iz/py_env-python3.7/lib/python3.7/site-packages/pre_commit_hooks/detect_aws_credentials.py", line 133, in main
    bad_filenames = check_file_for_aws_keys(args.filenames, keys)
  File "/home/ryan/.cache/pre-commit/repo8jj7q4iz/py_env-python3.7/lib/python3.7/site-packages/pre_commit_hooks/detect_aws_credentials.py", line 74, in check_file_for_aws_keys
    text_body = content.read()
  File "/home/ryan/.cache/pre-commit/repo8jj7q4iz/py_env-python3.7/lib/python3.7/codecs.py", line 322, in decode
    (result, consumed) = self._buffer_decode(data, self.errors, final)
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xeb in position 103: invalid continuation byte
Traceback (most recent call last):
  File "/home/ryan/.cache/pre-commit/repo8jj7q4iz/py_env-python3.7/bin/detect-aws-credentials", line 11, in <module>
    sys.exit(main())
  File "/home/ryan/.cache/pre-commit/repo8jj7q4iz/py_env-python3.7/lib/python3.7/site-packages/pre_commit_hooks/detect_aws_credentials.py", line 133, in main
    bad_filenames = check_file_for_aws_keys(args.filenames, keys)
  File "/home/ryan/.cache/pre-commit/repo8jj7q4iz/py_env-python3.7/lib/python3.7/site-packages/pre_commit_hooks/detect_aws_credentials.py", line 74, in check_file_for_aws_keys
    text_body = content.read()
  File "/home/ryan/.cache/pre-commit/repo8jj7q4iz/py_env-python3.7/lib/python3.7/codecs.py", line 322, in decode
    (result, consumed) = self._buffer_decode(data, self.errors, final)
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xc0 in position 2346: invalid start byte
Traceback (most recent call last):
  File "/home/ryan/.cache/pre-commit/repo8jj7q4iz/py_env-python3.7/bin/detect-aws-credentials", line 11, in <module>
    sys.exit(main())
  File "/home/ryan/.cache/pre-commit/repo8jj7q4iz/py_env-python3.7/lib/python3.7/site-packages/pre_commit_hooks/detect_aws_credentials.py", line 133, in main
    bad_filenames = check_file_for_aws_keys(args.filenames, keys)
  File "/home/ryan/.cache/pre-commit/repo8jj7q4iz/py_env-python3.7/lib/python3.7/site-packages/pre_commit_hooks/detect_aws_credentials.py", line 74, in check_file_for_aws_keys
    text_body = content.read()
  File "/home/ryan/.cache/pre-commit/repo8jj7q4iz/py_env-python3.7/lib/python3.7/codecs.py", line 322, in decode
    (result, consumed) = self._buffer_decode(data, self.errors, final)
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xe9 in position 187: invalid continuation byte
Traceback (most recent call last):
  File "/home/ryan/.cache/pre-commit/repo8jj7q4iz/py_env-python3.7/bin/detect-aws-credentials", line 11, in <module>
    sys.exit(main())
  File "/home/ryan/.cache/pre-commit/repo8jj7q4iz/py_env-python3.7/lib/python3.7/site-packages/pre_commit_hooks/detect_aws_credentials.py", line 133, in main
    bad_filenames = check_file_for_aws_keys(args.filenames, keys)
  File "/home/ryan/.cache/pre-commit/repo8jj7q4iz/py_env-python3.7/lib/python3.7/site-packages/pre_commit_hooks/detect_aws_credentials.py", line 74, in check_file_for_aws_keys
    text_body = content.read()
  File "/home/ryan/.cache/pre-commit/repo8jj7q4iz/py_env-python3.7/lib/python3.7/codecs.py", line 322, in decode
    (result, consumed) = self._buffer_decode(data, self.errors, final)
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xe9 in position 167: invalid continuation byte

It rejected the commit, so that’s good. And I’m not sure if I can recommend any improvements to the handling of invalid unicode since I haven’t looked at the code in this module.

But for sure this error message could be more useful. There are 773 files in this commit and it would sure be nice to know which one(s) have this problem.

Edit: Ideally if there’s an exception it would continue to the next file so all the files that couldn’t be processed can be listed to the user before the commit is rejected. I’m not looking forward to fixing one file, doing git commit, fixing another… 500 times. 😉