Home-Assistant-Mail-And-Packages: ISSUE: Failing to Parse Non-Multipart Message

Describe the bug When attempting to load the component, some email messages appear to prevent initialization with the below stacktrace.

Environment:

OS: Ubuntu 18.04
Type of device: Generic Linux
Home Assistant version: 0.109.6
Hassio/Docker/Core? Hassio via generic linux
Component version: master and 1.5

Logs

2020-05-12 17:21:46 INFO (MainThread) [homeassistant.components.sensor] Setting up sensor.mail_and_packages
2020-05-12 17:21:46 DEBUG (MainThread) [custom_components.mail_and_packages.sensor] Config scan interval: 0:05:00
2020-05-12 17:21:47 DEBUG (MainThread) [custom_components.mail_and_packages.sensor] Attempting to find Informed Delivery mail
2020-05-12 17:21:47 DEBUG (MainThread) [custom_components.mail_and_packages.sensor] Cleaning up image directory: /config/images/mail_and_packages/
2020-05-12 17:21:47 DEBUG (MainThread) [custom_components.mail_and_packages.sensor] Informed Delivery email found processing...
2020-05-12 17:21:47 DEBUG (MainThread) [custom_components.mail_and_packages.sensor] Removing duplicate images.
2020-05-12 17:21:47 DEBUG (MainThread) [custom_components.mail_and_packages.sensor] Removing USPS announcement images.
2020-05-12 17:21:47 DEBUG (MainThread) [custom_components.mail_and_packages.sensor] Image Count: 0
2020-05-12 17:21:47 INFO (MainThread) [custom_components.mail_and_packages.sensor] No mail found.
2020-05-12 17:21:47 DEBUG (MainThread) [custom_components.mail_and_packages.sensor] Copying nomail gif
2020-05-12 17:21:47 DEBUG (MainThread) [custom_components.mail_and_packages.sensor] Attempting to find mail from auto-reply@usps.com with subject 1 Item Delivered
2020-05-12 17:21:47 DEBUG (MainThread) [custom_components.mail_and_packages.sensor] Found from auto-reply@usps.com with subject 1 Item Delivered, b''
2020-05-12 17:21:47 DEBUG (MainThread) [custom_components.mail_and_packages.sensor] Attempting to find mail from auto-reply@usps.com with subject 1 Expected Delivery on
2020-05-12 17:21:47 DEBUG (MainThread) [custom_components.mail_and_packages.sensor] Found from auto-reply@usps.com with subject 1 Expected Delivery on, b''
2020-05-12 17:21:47 DEBUG (MainThread) [custom_components.mail_and_packages.sensor] Attempting to find mail from mcinfo@ups.com with subject 1 Your UPS Package was delivered
2020-05-12 17:21:47 DEBUG (MainThread) [custom_components.mail_and_packages.sensor] Found from mcinfo@ups.com with subject 1 Your UPS Package was delivered, b''
2020-05-12 17:21:47 DEBUG (MainThread) [custom_components.mail_and_packages.sensor] Attempting to find mail from mcinfo@ups.com with subject 1 UPS Update: Package Scheduled for Delivery Today
2020-05-12 17:21:47 DEBUG (MainThread) [custom_components.mail_and_packages.sensor] Found from mcinfo@ups.com with subject 1 UPS Update: Package Scheduled for Delivery Today, b''
2020-05-12 17:21:47 DEBUG (MainThread) [custom_components.mail_and_packages.sensor] Attempting to find mail from mcinfo@ups.com with subject 2 UPS Update: Follow Your Delivery on a Live Map
2020-05-12 17:21:47 DEBUG (MainThread) [custom_components.mail_and_packages.sensor] Found from mcinfo@ups.com with subject 2 UPS Update: Follow Your Delivery on a Live Map, b''
2020-05-12 17:21:47 DEBUG (MainThread) [custom_components.mail_and_packages.sensor] Attempting to find mail from TrackingUpdates@fedex.com with subject 1 Your package has been delivered
2020-05-12 17:21:47 DEBUG (MainThread) [custom_components.mail_and_packages.sensor] Found from TrackingUpdates@fedex.com with subject 1 Your package has been delivered, b''
2020-05-12 17:21:47 DEBUG (MainThread) [custom_components.mail_and_packages.sensor] Attempting to find mail from TrackingUpdates@fedex.com with subject 1 Delivery scheduled for today
2020-05-12 17:21:47 DEBUG (MainThread) [custom_components.mail_and_packages.sensor] Found from TrackingUpdates@fedex.com with subject 1 Delivery scheduled for today, b''
2020-05-12 17:21:47 DEBUG (MainThread) [custom_components.mail_and_packages.sensor] Attempting to find mail from TrackingUpdates@fedex.com with subject 2 Your package is scheduled for delivery today
2020-05-12 17:21:47 DEBUG (MainThread) [custom_components.mail_and_packages.sensor] Found from TrackingUpdates@fedex.com with subject 2 Your package is scheduled for delivery today, b''
2020-05-12 17:21:47 DEBUG (MainThread) [custom_components.mail_and_packages.sensor] Attempting to find Amazon email with item list ...
2020-05-12 17:21:47 DEBUG (MainThread) [custom_components.mail_and_packages.sensor] Amazon emails found: 3
2020-05-12 17:21:48 ERROR (MainThread) [homeassistant.components.sensor] Error while setting up mail_and_packages platform for sensor
Traceback (most recent call last):
  File "/usr/src/homeassistant/homeassistant/helpers/entity_platform.py", line 178, in _async_setup_platform
    await asyncio.wait_for(asyncio.shield(task), SLOW_SETUP_MAX_WAIT)
  File "/usr/local/lib/python3.7/asyncio/tasks.py", line 442, in wait_for
    return fut.result()
  File "/config/custom_components/mail_and_packages/sensor.py", line 146, in async_setup_entry
    sensors.append(PackagesSensor(data, variable))
  File "/config/custom_components/mail_and_packages/sensor.py", line 261, in __init__
    self.update()
  File "/config/custom_components/mail_and_packages/sensor.py", line 309, in update
    self.data.update()
  File "/usr/src/homeassistant/homeassistant/util/__init__.py", line 240, in wrapper
    result = method(*args, **kwargs)
  File "/config/custom_components/mail_and_packages/sensor.py", line 201, in update
    count[sensor] = get_items(account, "count")
  File "/config/custom_components/mail_and_packages/sensor.py", line 622, in get_items
    email_msg = str(msg.get_payload(0))
  File "/usr/local/lib/python3.7/email/message.py", line 257, in get_payload
    raise TypeError('Expected list, got %s' % type(self._payload))
TypeError: Expected list, got <class 'str'>

Additional context This is a Office 365 (Exchange Online via IMAP) account, which should be close to a Outlook Online account. Not sure if that would really matter.

The error screams this issue (https://stackoverflow.com/questions/1463074/how-can-i-get-an-email-messages-text-content-using-python) - but I’m no Python developer. The emails appear to be multi-part, removing the emails does allow the component to load. I’m having troubling dumping the raw message, since Outlook likes to hide the complexities of mime.

About this issue

Original URL
State: closed
Created 4 years ago
Comments: 17

Most upvoted comments

Looking back at my archive of Amazon emails (only shipping notices), it’s odd. Two emails within 24 hours, same order ID (not sure why I received two notifications). The message id’s were different, sent from different Amazon servers. The content is the same, just sent at different times.

1/27/2020 - Content-Type: multipart/alternative; 
1/28/2020 - Content-Type: text/html; charset="utf-8"

The headers on the different emails were different. For example, the first email had a DKIM-Signature, while the second did not. Throughout my email archive, since 12/2019, the emails switch back and forth from multipart to just text/html with all the notifications from this month being only text/html.

This makes me think, Amazon is slowly phasing in a new emailing implementation for their notifications.

I’ll try the patch and report back. Thanks for your help.

Silvenga on May 15, 2020

I’ll keep looking into it, I have a backup plan if I can’t find something more definite.

firstof9 on May 14, 2020