kafka-python: AttributeError: 'PartitionRecords' object has no attribute 'message_idx'

kafka-python is 1.3.5

My code is basically trying to do:

pollResults = self._consumer.poll(timeout, maxResults)

and I run into the following exception from within the library:

  File "/usr/lib/python3.6/site-packages/kafka/consumer/group.py", line 570, in poll
    records = self._poll_once(remaining, max_records)
  File "/usr/lib/python3.6/site-packages/kafka/consumer/group.py", line 604, in _poll_once
    records, partial = self._fetcher.fetched_records(max_records)
  File "/usr/lib/python3.6/site-packages/kafka/consumer/fetcher.py", line 336, in fetched_records
    if not self._next_partition_records:
  File "/usr/lib/python3.6/site-packages/kafka/consumer/fetcher.py", line 855, in __len__
    if self.messages is None or self.message_idx >= len(self.messages):
AttributeError: 'PartitionRecords' object has no attribute 'message_idx'

About this issue

Original URL
State: closed
Created 7 years ago
Comments: 15 (5 by maintainers)

Commits related to this issue

Revert ffc7caef13a120f69788bcdd43ffa01468f575f9 / PR #1239 The change caused a regression documented in issue #1290 — committed to dpkp/kafka-python by dpkp 7 years ago

Most upvoted comments

Yes, you are correct. My question is whether you see any other error log messages related to message unpacking? The root cause is a non-empty messageset yielding an empty list of actual messages. This is strange and unexpected behavior and suggests either that there is a lower level protocol issue, that message compression is failing, or that the message unpacking itself has a bug. The fix for the attribute error is obvious, but I’m much more interested in determining if there is a root cause.

dpkp on Nov 9, 2017