core: Mikrotik Integration fails with Unicode error

The problem

The Mikrotik integration is choking on a Unicode response from the router. If the integration is added with the UI, this prevents it from completing setup. If added directly to configuration.yaml, it appears to prevent updates after the initial response.

This is because, for some reason, my GE Range insists on defining its hostname as “û”

Environment

Home Assistant Core release with the issue: 0.118.5
Last working Home Assistant Core release (if known): 0.118.0
Operating environment (OS/Container/Supervised/Core): Hassio 2020.12.6
Integration causing this issue: Mikrotik
Link to integration documentation on our website: https://www.home-assistant.io/integrations/mikrotik

Problem-relevant `configuration.yaml`

mikrotik:
  - name: <HOSTNAME>
    host: <IP>
    username: homeassistant
    password: <PASSWORD>
    verify_ssl: false
    arp_ping: false
    force_dhcp: true
    detection_time: 300

Traceback/Error logs

This is the traceback placed in home_assistant.log when trying to set up the MikroTik integration from the interface.

2020-12-10 14:22:22 ERROR (MainThread) [homeassistant.config_entries] Error setting up entry Mikrotik for mikrotik
Traceback (most recent call last):
  File "/usr/src/homeassistant/homeassistant/config_entries.py", line 236, in async_setup
    result = await component.async_setup_entry(hass, self)  # type: ignore
  File "/usr/src/homeassistant/homeassistant/components/mikrotik/__init__.py", line 68, in async_setup_entry
    if not await hub.async_setup():
  File "/usr/src/homeassistant/homeassistant/components/mikrotik/hub.py", line 382, in async_setup
    await self.hass.async_add_executor_job(self._mk_data.update)
  File "/usr/local/lib/python3.8/concurrent/futures/thread.py", line 57, in run
    result = self.fn(*self.args, **self.kwargs)
  File "/usr/src/homeassistant/homeassistant/components/mikrotik/hub.py", line 278, in update
    self.update_devices()
  File "/usr/src/homeassistant/homeassistant/components/mikrotik/hub.py", line 172, in update_devices
    self.all_devices = self.get_list_from_interface(DHCP)
  File "/usr/src/homeassistant/homeassistant/components/mikrotik/hub.py", line 159, in get_list_from_interface
    result = self.command(MIKROTIK_SERVICES[interface])
  File "/usr/src/homeassistant/homeassistant/components/mikrotik/hub.py", line 253, in command
    response = list(self.api(cmd=cmd))
  File "/usr/local/lib/python3.8/site-packages/librouteros/api.py", line 28, in __call__
    yield from self.readResponse()
  File "/usr/local/lib/python3.8/site-packages/librouteros/api.py", line 60, in readResponse
    reply_word, words = self.readSentence()
  File "/usr/local/lib/python3.8/site-packages/librouteros/api.py", line 46, in readSentence
    reply_word, words = self.protocol.readSentence()
  File "/usr/local/lib/python3.8/site-packages/librouteros/protocol.py", line 189, in readSentence
    sentence = tuple(word for word in iter(self.readWord, ''))
  File "/usr/local/lib/python3.8/site-packages/librouteros/protocol.py", line 189, in <genexpr>
    sentence = tuple(word for word in iter(self.readWord, ''))
  File "/usr/local/lib/python3.8/site-packages/librouteros/protocol.py", line 206, in readWord
    return word.decode(encoding=self.encoding, errors='strict')
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xfb in position 12: invalid start byte

Additional information

About this issue

Original URL
State: closed
Created 4 years ago
Reactions: 1
Comments: 17

Most upvoted comments

Thanks for the comment, I wasn’t able to figure out how to have the router override the hostname provided by the GE appliances, so if you are successful in finding a way, please post it here!

I submitted a support request from the GE Smart Appliances group and they said this is a “known issue with all of their WiFi modules.”

Hi - that didn’t work the client GE appliances don’t honor the option 12 from the router and still go ahead and set it to the weird characters (seems Chinese but not sure). Anyways after tinkering and going thru the error in the logs I modified the encoding from UTF-8 to latin-1 and after restarting HA (I’m using the base running inside Python venv not the docker one) - it was able to come up fine. I am not sure if we can configure the encoding for mikrotik/device tracker using HA configuration.yaml or the integration - I couldn’t find it so just modified the init.py in librouteros package. Here’s the change I did: vi /srv/homeassistant/lib/python3.8/site-packages/librouteros/init.py

from librouteros.api import Api

DEFAULTS = { ‘timeout’: 10, ‘port’: 8728, ‘saddr’: ‘’, ‘subclass’: Api, ‘encoding’: ‘latin-1’, ‘ssl_wrapper’: lambda sock: sock, ‘login_method’: plain, }

I know this will impact other components using this lib but for now that worked and i can at least use my setup without issues… better way would be to set this encoding when init the conn - as a param - that’s supported (i’ll go thru the mikrotik component code in HA and try to change it there and update you once it works). let me know if this works for you. I thought there was a PR closed that supported config the encoding using config for mikrotik in HA but I couldn’t find it. That would be the best option here. Fortunately, latin-1 works and takes care of these Chinese characters that GE appliances are passing. On another note my next step is to integrate the GE appliances using the gekitchen component (if you or anyone has insight on it please help).

mhpetiwala on Jan 27, 2021

The CLI shows \11\FB\95 as the hostname for the offending leases:

GE is short for General Electric. All four of these DHCP leases are for connections to “smart” appliances made by GE. Specifically, I have the following appliances connected to the internet made by GE: https://appliances.monogram.com/us/specs/ZISS420DNSS https://appliances.monogram.com/us/specs/ZTD90DSSNSS https://appliances.monogram.com/us/specs/ZSB9131NSS https://appliances.monogram.com/us/specs/ZDT985SSNSS

As far as I can tell, there is no way to change the hostname configured on the appliance.

ChevySSinSD on Jan 14, 2021