iotedge: Restart of Windows host causes long freeze in docker pull on Iot Edge 1.2.0 running in linux vm

Expected Behavior

IoT Edge 1.2.0 immediately continues to download containers after startup like it does in version 1.0.9.5.

Current Behavior

Iot Edge 1.2.0 in Centos vm freezes the docker pull for around 15 minutes after the Windows host OS has been restarted.

Steps to Reproduce

Fresh install of iotedge 1.2.0 on Centos 7

Install config.toml and certificates in host
update-ca-trust
iotedge config apply
Docker starts downloading containers (edgeAgent, edgeHub, dispatcher)
Host OS Windows Restart (while containers are still being downloaded)
After Windows host OS has started the IoT-Edge does not continue or restart downloding containers untill a timeout of approximately 15 min has passed.

Context (Environment)

IoT-Edge running in Hyper-V VM Centos 7 At Windows shutdown the VM ‘Automatic Stop Action’ is set to ‘Save the virtual machine state’.

Output of `iotedge check`

Click here



Configuration checks (aziot-identity-service)
---------------------------------------------
√ keyd configuration is well-formed - OK
√ certd configuration is well-formed - OK
√ tpmd configuration is well-formed - OK
√ identityd configuration is well-formed - OK
√ daemon configurations up-to-date with config.toml - OK
√ identityd config toml file specifies a valid hostname - OK
× aziot-identity-service package is up-to-date - Error
    could not query https://aka.ms/latest-aziot-identity-service for latest available version
× host time is close to reference time - Error
    Could not query NTP server
√ preloaded certificates are valid - OK
√ keyd is running - OK
√ certd is running - OK
√ identityd is running - OK
√ read all preloaded certificates from the Certificates Service - OK
√ read all preloaded key pairs from the Keys Service - OK
√ ensure all preloaded certificates match preloaded private keys with the same ID - OK

Connectivity checks (aziot-identity-service)
--------------------------------------------
× host can connect to and perform TLS handshake with iothub AMQP port - Error
    Could not connect to ta01iothubt01.azure-devices.net : could not complete TLS handshake
√ host can connect to and perform TLS handshake with iothub HTTPS / WebSockets port - OK
√ host can connect to and perform TLS handshake with iothub MQTT port - OK

Configuration checks
--------------------
√ aziot-edged configuration is well-formed - OK
√ configuration up-to-date with config.toml - OK
√ container engine is installed and functional - OK
√ configuration has correct URIs for daemon mgmt endpoint - OK
‼ aziot-edge package is up-to-date - Warning
    Error while fetching latest versions of edge components: could not send HTTP request
√ container time is close to host time - OK
‼ DNS server - Warning
    Container engine is not configured with DNS server setting, which may impact connectivity to IoT Hub.
    Please see https://aka.ms/iotedge-prod-checklist-dns for best practices.
    You can ignore this warning if you are setting DNS server per module in the Edge deployment.
√ production readiness: container engine - OK
√ production readiness: logs policy - OK
‼ production readiness: Edge Agent's storage directory is persisted on the host filesystem - Warning
    The edgeAgent module is not configured to persist its /tmp/edgeAgent directory on the host filesystem.
    Data might be lost if the module is deleted or updated.
    Please see https://aka.ms/iotedge-storage-host for best practices.
‼ production readiness: Edge Hub's storage directory is persisted on the host filesystem - Warning
    The edgeHub module is not configured to persist its /tmp/edgeHub directory on the host filesystem.
    Data might be lost if the module is deleted or updated.
    Please see https://aka.ms/iotedge-storage-host for best practices.
√ Agent image is valid and can be pulled from upstream - OK

Connectivity checks
-------------------
× container on the default network can connect to upstream  AMQP port - Error
    Container on the default network could not connect to ta01iothubt01.azure-devices.net:5671
√ container on the default network can connect to upstream HTTPS / WebSockets port - OK
√ container on the default network can connect to upstream MQTT port - OK
× container on the IoT Edge module network can connect to upstream AMQP port - Error
    Container on the azure-iot-edge network could not connect to ta01iothubt01.azure-devices.net:5671
√ container on the IoT Edge module network can connect to upstream HTTPS / WebSockets port - OK
√ container on the IoT Edge module network can connect to upstream MQTT port - OK
27 check(s) succeeded.
4 check(s) raised warnings. Re-run with --verbose for more details.
5 check(s) raised errors. Re-run with --verbose for more details.

Device Information

Host OS : Windows 10
Architecture: amd64 (Intel processor)
Container OS: Centos 7

Runtime Versions

aziot-edged: 1.2.0
Edge Agent: 1.2.0
Edge Hub: 1.2.0
Docker/Moby: Tested 3.0.6 and 20.10.6+azure

Note: when using Windows containers on Windows, run docker -H npipe:////./pipe/iotedge_moby_engine version instead

Logs

aziot-edged logs


-- Logs begin at Fri 2021-05-28 01:54:52 UTC, end at Fri 2021-05-28 09:04:42 UTC. --
May 28 08:54:59 iotedge.rsu.local systemd[1]: Started Azure IoT Identity Service.
May 28 08:54:59 iotedge.rsu.local aziot-identityd[3498]: 2021-05-28T08:54:59Z [INFO] - Starting service...
May 28 08:54:59 iotedge.rsu.local aziot-identityd[3498]: 2021-05-28T08:54:59Z [INFO] - Version - 1.2.0
May 28 08:54:59 iotedge.rsu.local aziot-identityd[3498]: 2021-05-28T08:54:59Z [ERR!] - could not read config
May 28 08:54:59 iotedge.rsu.local systemd[1]: aziot-identityd.service: main process exited, code=exited, status=1/FAILURE
May 28 08:54:59 iotedge.rsu.local systemd[1]: Unit aziot-identityd.service entered failed state.
May 28 08:54:59 iotedge.rsu.local systemd[1]: aziot-identityd.service failed.
May 28 08:55:05 iotedge.rsu.local systemd[1]: aziot-identityd.service holdoff time over, scheduling restart.
May 28 08:55:05 iotedge.rsu.local systemd[1]: Stopped Azure IoT Identity Service.
May 28 08:55:05 iotedge.rsu.local systemd[1]: Started Azure IoT Identity Service.
May 28 08:55:05 iotedge.rsu.local aziot-identityd[3818]: 2021-05-28T08:55:05Z [INFO] - Starting service...
May 28 08:55:05 iotedge.rsu.local aziot-identityd[3818]: 2021-05-28T08:55:05Z [INFO] - Version - 1.2.0
May 28 08:55:05 iotedge.rsu.local aziot-identityd[3818]: 2021-05-28T08:55:05Z [ERR!] - could not read config
May 28 08:55:05 iotedge.rsu.local aziot-identityd[3818]: 2021-05-28T08:55:05Z [ERR!] - caused by: could not read config
May 28 08:55:05 iotedge.rsu.local aziot-identityd[3818]: 2021-05-28T08:55:05Z [ERR!] - caused by: missing field `hostname`
May 28 08:55:05 iotedge.rsu.local aziot-identityd[3818]: 2021-05-28T08:55:05Z [ERR!] -    0: backtrace::capture::Backtrace::create
May 28 08:55:05 iotedge.rsu.local aziot-identityd[3818]: 1: backtrace::capture::Backtrace::new
May 28 08:55:05 iotedge.rsu.local systemd[1]: aziot-identityd.service: main process exited, code=exited, status=1/FAILURE
May 28 08:55:05 iotedge.rsu.local systemd[1]: Unit aziot-identityd.service entered failed state.
May 28 08:55:05 iotedge.rsu.local systemd[1]: aziot-identityd.service failed.
May 28 08:55:06 iotedge.rsu.local systemd[1]: Started Azure IoT Edge daemon.
May 28 08:55:07 iotedge.rsu.local aziot-edged[4763]: 2021-05-28T08:55:07Z [INFO] - Starting Azure IoT Edge Module Runtime
May 28 08:55:07 iotedge.rsu.local aziot-edged[4763]: 2021-05-28T08:55:07Z [INFO] - Version - 1.2.0
May 28 08:55:07 iotedge.rsu.local systemd[1]: aziot-edged.service: main process exited, code=exited, status=153/n/a
May 28 08:55:07 iotedge.rsu.local systemd[1]: Unit aziot-edged.service entered failed state.
May 28 08:55:07 iotedge.rsu.local systemd[1]: aziot-edged.service failed.
May 28 08:55:10 iotedge.rsu.local systemd[1]: aziot-identityd.service holdoff time over, scheduling restart.
May 28 08:55:10 iotedge.rsu.local systemd[1]: Stopped Azure IoT Identity Service.
May 28 08:55:10 iotedge.rsu.local systemd[1]: Started Azure IoT Identity Service.
May 28 08:55:10 iotedge.rsu.local aziot-identityd[6367]: 2021-05-28T08:55:10Z [INFO] - Starting service...
May 28 08:55:10 iotedge.rsu.local aziot-identityd[6367]: 2021-05-28T08:55:10Z [INFO] - Version - 1.2.0
May 28 08:55:10 iotedge.rsu.local systemd[1]: aziot-identityd.service: main process exited, code=exited, status=1/FAILURE
May 28 08:55:10 iotedge.rsu.local systemd[1]: Unit aziot-identityd.service entered failed state.
May 28 08:55:10 iotedge.rsu.local systemd[1]: aziot-identityd.service failed.
May 28 08:55:15 iotedge.rsu.local systemd[1]: aziot-identityd.service holdoff time over, scheduling restart.
May 28 08:55:15 iotedge.rsu.local systemd[1]: Stopped Azure IoT Identity Service.
May 28 08:55:15 iotedge.rsu.local systemd[1]: Started Azure IoT Identity Service.
May 28 08:55:15 iotedge.rsu.local aziot-identityd[7510]: 2021-05-28T08:55:15Z [INFO] - Starting service...
May 28 08:55:15 iotedge.rsu.local aziot-identityd[7510]: 2021-05-28T08:55:15Z [INFO] - Version - 1.2.0
May 28 08:55:15 iotedge.rsu.local systemd[1]: aziot-identityd.service: main process exited, code=exited, status=1/FAILURE
May 28 08:55:15 iotedge.rsu.local systemd[1]: Unit aziot-identityd.service entered failed state.
May 28 08:55:15 iotedge.rsu.local systemd[1]: aziot-identityd.service failed.
May 28 08:55:20 iotedge.rsu.local systemd[1]: aziot-identityd.service holdoff time over, scheduling restart.
May 28 08:55:20 iotedge.rsu.local systemd[1]: Stopped Azure IoT Identity Service.
May 28 08:55:20 iotedge.rsu.local systemd[1]: Started Azure IoT Identity Service.
May 28 08:55:20 iotedge.rsu.local systemd[1]: aziot-identityd.service: main process exited, code=exited, status=1/FAILURE
May 28 08:55:20 iotedge.rsu.local systemd[1]: Unit aziot-identityd.service entered failed state.
May 28 08:55:20 iotedge.rsu.local systemd[1]: aziot-identityd.service failed.
May 28 08:55:25 iotedge.rsu.local systemd[1]: aziot-identityd.service holdoff time over, scheduling restart.
May 28 08:55:25 iotedge.rsu.local systemd[1]: Stopped Azure IoT Identity Service.
May 28 08:55:25 iotedge.rsu.local systemd[1]: Started Azure IoT Identity Service.
May 28 08:55:25 iotedge.rsu.local aziot-identityd[12467]: 2021-05-28T08:55:25Z [INFO] - Starting service...
May 28 08:55:25 iotedge.rsu.local aziot-identityd[12467]: 2021-05-28T08:55:25Z [INFO] - Version - 1.2.0
May 28 08:55:25 iotedge.rsu.local systemd[1]: aziot-identityd.service: main process exited, code=exited, status=1/FAILURE
May 28 08:55:25 iotedge.rsu.local systemd[1]: Unit aziot-identityd.service entered failed state.
May 28 08:55:25 iotedge.rsu.local systemd[1]: aziot-identityd.service failed.
May 28 08:55:31 iotedge.rsu.local systemd[1]: aziot-identityd.service holdoff time over, scheduling restart.
May 28 08:55:31 iotedge.rsu.local systemd[1]: Stopped Azure IoT Identity Service.
May 28 08:55:31 iotedge.rsu.local systemd[1]: Started Azure IoT Identity Service.
May 28 08:55:31 iotedge.rsu.local systemd[1]: aziot-identityd.service: main process exited, code=exited, status=1/FAILURE
May 28 08:55:31 iotedge.rsu.local systemd[1]: Unit aziot-identityd.service entered failed state.
May 28 08:55:31 iotedge.rsu.local systemd[1]: aziot-identityd.service failed.
May 28 08:55:36 iotedge.rsu.local systemd[1]: aziot-identityd.service holdoff time over, scheduling restart.
May 28 08:55:36 iotedge.rsu.local systemd[1]: Stopped Azure IoT Identity Service.
May 28 08:55:36 iotedge.rsu.local systemd[1]: Started Azure IoT Identity Service.
May 28 08:55:36 iotedge.rsu.local systemd[1]: aziot-identityd.service: main process exited, code=exited, status=1/FAILURE
May 28 08:55:36 iotedge.rsu.local systemd[1]: Unit aziot-identityd.service entered failed state.
May 28 08:55:36 iotedge.rsu.local systemd[1]: aziot-identityd.service failed.
May 28 08:55:41 iotedge.rsu.local systemd[1]: aziot-identityd.service holdoff time over, scheduling restart.
May 28 08:55:41 iotedge.rsu.local systemd[1]: Stopped Azure IoT Identity Service.
May 28 08:55:41 iotedge.rsu.local systemd[1]: Started Azure IoT Identity Service.
May 28 08:55:41 iotedge.rsu.local aziot-identityd[13760]: 2021-05-28T08:55:41Z [INFO] - Starting service...
May 28 08:55:41 iotedge.rsu.local aziot-identityd[13760]: 2021-05-28T08:55:41Z [INFO] - Version - 1.2.0
May 28 08:55:41 iotedge.rsu.local systemd[1]: aziot-identityd.service: main process exited, code=exited, status=1/FAILURE
May 28 08:55:41 iotedge.rsu.local systemd[1]: Unit aziot-identityd.service entered failed state.
May 28 08:55:41 iotedge.rsu.local systemd[1]: aziot-identityd.service failed.
May 28 08:55:46 iotedge.rsu.local systemd[1]: aziot-identityd.service holdoff time over, scheduling restart.
May 28 08:55:46 iotedge.rsu.local systemd[1]: Stopped Azure IoT Identity Service.
May 28 08:55:46 iotedge.rsu.local systemd[1]: Started Azure IoT Identity Service.
May 28 08:55:46 iotedge.rsu.local aziot-identityd[13775]: 2021-05-28T08:55:46Z [INFO] - Starting service...
May 28 08:55:46 iotedge.rsu.local aziot-identityd[13775]: 2021-05-28T08:55:46Z [INFO] - Version - 1.2.0
May 28 08:55:46 iotedge.rsu.local systemd[1]: aziot-identityd.service: main process exited, code=exited, status=1/FAILURE
May 28 08:55:46 iotedge.rsu.local systemd[1]: Unit aziot-identityd.service entered failed state.
May 28 08:55:46 iotedge.rsu.local systemd[1]: aziot-identityd.service failed.
May 28 08:55:52 iotedge.rsu.local systemd[1]: aziot-identityd.service holdoff time over, scheduling restart.
May 28 08:55:52 iotedge.rsu.local systemd[1]: Stopped Azure IoT Identity Service.
May 28 08:55:52 iotedge.rsu.local systemd[1]: Started Azure IoT Identity Service.
May 28 08:55:52 iotedge.rsu.local systemd[1]: aziot-identityd.service: main process exited, code=exited, status=1/FAILURE
May 28 08:55:52 iotedge.rsu.local systemd[1]: Unit aziot-identityd.service entered failed state.
May 28 08:55:52 iotedge.rsu.local systemd[1]: aziot-identityd.service failed.
May 28 08:55:57 iotedge.rsu.local systemd[1]: aziot-identityd.service holdoff time over, scheduling restart.
May 28 08:55:57 iotedge.rsu.local systemd[1]: Stopped Azure IoT Identity Service.
May 28 08:55:57 iotedge.rsu.local systemd[1]: Started Azure IoT Identity Service.
May 28 08:55:57 iotedge.rsu.local aziot-identityd[13803]: 2021-05-28T08:55:57Z [INFO] - Starting service...
May 28 08:55:57 iotedge.rsu.local aziot-identityd[13803]: 2021-05-28T08:55:57Z [INFO] - Version - 1.2.0
May 28 08:55:57 iotedge.rsu.local systemd[1]: aziot-identityd.service: main process exited, code=exited, status=1/FAILURE
May 28 08:55:57 iotedge.rsu.local systemd[1]: Unit aziot-identityd.service entered failed state.
May 28 08:55:57 iotedge.rsu.local systemd[1]: aziot-identityd.service failed.
May 28 08:56:02 iotedge.rsu.local systemd[1]: aziot-identityd.service holdoff time over, scheduling restart.
May 28 08:56:02 iotedge.rsu.local systemd[1]: Stopped Azure IoT Identity Service.
May 28 08:56:02 iotedge.rsu.local systemd[1]: Started Azure IoT Identity Service.
May 28 08:56:02 iotedge.rsu.local aziot-identityd[13817]: 2021-05-28T08:56:02Z [INFO] - Starting service...
May 28 08:56:02 iotedge.rsu.local aziot-identityd[13817]: 2021-05-28T08:56:02Z [INFO] - Version - 1.2.0
May 28 08:56:02 iotedge.rsu.local systemd[1]: aziot-identityd.service: main process exited, code=exited, status=1/FAILURE
May 28 08:56:02 iotedge.rsu.local systemd[1]: Unit aziot-identityd.service entered failed state.
May 28 08:56:02 iotedge.rsu.local systemd[1]: aziot-identityd.service failed.
May 28 08:56:07 iotedge.rsu.local systemd[1]: aziot-identityd.service holdoff time over, scheduling restart.
May 28 08:56:07 iotedge.rsu.local systemd[1]: Stopped Azure IoT Identity Service.
May 28 08:56:07 iotedge.rsu.local systemd[1]: Started Azure IoT Identity Service.
May 28 08:56:07 iotedge.rsu.local aziot-identityd[13830]: 2021-05-28T08:56:07Z [INFO] - Starting service...
May 28 08:56:07 iotedge.rsu.local aziot-identityd[13830]: 2021-05-28T08:56:07Z [INFO] - Version - 1.2.0
May 28 08:56:07 iotedge.rsu.local systemd[1]: aziot-identityd.service: main process exited, code=exited, status=1/FAILURE
May 28 08:56:07 iotedge.rsu.local systemd[1]: Unit aziot-identityd.service entered failed state.
May 28 08:56:07 iotedge.rsu.local systemd[1]: aziot-identityd.service failed.
May 28 08:56:13 iotedge.rsu.local systemd[1]: aziot-identityd.service holdoff time over, scheduling restart.
May 28 08:56:13 iotedge.rsu.local systemd[1]: Stopped Azure IoT Identity Service.
May 28 08:56:13 iotedge.rsu.local systemd[1]: Started Azure IoT Identity Service.
May 28 08:56:13 iotedge.rsu.local aziot-identityd[13845]: 2021-05-28T08:56:13Z [INFO] - Starting service...
May 28 08:56:13 iotedge.rsu.local aziot-identityd[13845]: 2021-05-28T08:56:13Z [INFO] - Version - 1.2.0
May 28 08:56:13 iotedge.rsu.local systemd[1]: aziot-identityd.service: main process exited, code=exited, status=1/FAILURE
May 28 08:56:13 iotedge.rsu.local systemd[1]: Unit aziot-identityd.service entered failed state.
May 28 08:56:13 iotedge.rsu.local systemd[1]: aziot-identityd.service failed.
May 28 08:56:18 iotedge.rsu.local systemd[1]: aziot-identityd.service holdoff time over, scheduling restart.
May 28 08:56:18 iotedge.rsu.local systemd[1]: Stopped Azure IoT Identity Service.
May 28 08:56:18 iotedge.rsu.local systemd[1]: Started Azure IoT Identity Service.
May 28 08:56:18 iotedge.rsu.local aziot-identityd[13859]: 2021-05-28T08:56:18Z [INFO] - Starting service...
May 28 08:56:18 iotedge.rsu.local aziot-identityd[13859]: 2021-05-28T08:56:18Z [INFO] - Version - 1.2.0
May 28 08:56:18 iotedge.rsu.local systemd[1]: aziot-identityd.service: main process exited, code=exited, status=1/FAILURE
May 28 08:56:18 iotedge.rsu.local systemd[1]: Unit aziot-identityd.service entered failed state.
May 28 08:56:18 iotedge.rsu.local systemd[1]: aziot-identityd.service failed.
May 28 08:56:23 iotedge.rsu.local systemd[1]: aziot-identityd.service holdoff time over, scheduling restart.
May 28 08:56:23 iotedge.rsu.local systemd[1]: Stopped Azure IoT Identity Service.
May 28 08:56:23 iotedge.rsu.local systemd[1]: Started Azure IoT Identity Service.
May 28 08:56:23 iotedge.rsu.local aziot-identityd[13874]: 2021-05-28T08:56:23Z [INFO] - Starting service...
May 28 08:56:23 iotedge.rsu.local aziot-identityd[13874]: 2021-05-28T08:56:23Z [INFO] - Version - 1.2.0
May 28 08:56:23 iotedge.rsu.local systemd[1]: aziot-identityd.service: main process exited, code=exited, status=1/FAILURE
May 28 08:56:23 iotedge.rsu.local systemd[1]: Unit aziot-identityd.service entered failed state.
May 28 08:56:23 iotedge.rsu.local systemd[1]: aziot-identityd.service failed.
May 28 08:56:28 iotedge.rsu.local systemd[1]: aziot-identityd.service holdoff time over, scheduling restart.
May 28 08:56:28 iotedge.rsu.local systemd[1]: Stopped Azure IoT Identity Service.
May 28 08:56:28 iotedge.rsu.local systemd[1]: Started Azure IoT Identity Service.
May 28 08:56:28 iotedge.rsu.local aziot-identityd[13887]: 2021-05-28T08:56:28Z [INFO] - Starting service...
May 28 08:56:28 iotedge.rsu.local aziot-identityd[13887]: 2021-05-28T08:56:28Z [INFO] - Version - 1.2.0
May 28 08:56:28 iotedge.rsu.local systemd[1]: aziot-identityd.service: main process exited, code=exited, status=1/FAILURE
May 28 08:56:28 iotedge.rsu.local systemd[1]: Unit aziot-identityd.service entered failed state.
May 28 08:56:28 iotedge.rsu.local systemd[1]: aziot-identityd.service failed.
May 28 08:56:34 iotedge.rsu.local systemd[1]: aziot-identityd.service holdoff time over, scheduling restart.
May 28 08:56:34 iotedge.rsu.local systemd[1]: Stopped Azure IoT Identity Service.
May 28 08:56:34 iotedge.rsu.local aziot-identityd[13902]: 2021-05-28T08:56:34Z [INFO] - Starting service...
May 28 08:56:34 iotedge.rsu.local aziot-identityd[13902]: 2021-05-28T08:56:34Z [INFO] - Version - 1.2.0
May 28 08:56:34 iotedge.rsu.local systemd[1]: Started Azure IoT Identity Service.
May 28 08:56:34 iotedge.rsu.local systemd[1]: aziot-identityd.service: main process exited, code=exited, status=1/FAILURE
May 28 08:56:34 iotedge.rsu.local systemd[1]: Unit aziot-identityd.service entered failed state.
May 28 08:56:34 iotedge.rsu.local systemd[1]: aziot-identityd.service failed.
May 28 08:56:39 iotedge.rsu.local systemd[1]: aziot-identityd.service holdoff time over, scheduling restart.
May 28 08:56:39 iotedge.rsu.local systemd[1]: Stopped Azure IoT Identity Service.
May 28 08:56:39 iotedge.rsu.local systemd[1]: Started Azure IoT Identity Service.
May 28 08:56:39 iotedge.rsu.local aziot-identityd[13916]: 2021-05-28T08:56:39Z [INFO] - Starting service...
May 28 08:56:39 iotedge.rsu.local aziot-identityd[13916]: 2021-05-28T08:56:39Z [INFO] - Version - 1.2.0
May 28 08:56:39 iotedge.rsu.local systemd[1]: aziot-identityd.service: main process exited, code=exited, status=1/FAILURE
May 28 08:56:39 iotedge.rsu.local systemd[1]: Unit aziot-identityd.service entered failed state.
May 28 08:56:39 iotedge.rsu.local systemd[1]: aziot-identityd.service failed.
May 28 08:56:44 iotedge.rsu.local systemd[1]: aziot-identityd.service holdoff time over, scheduling restart.
May 28 08:56:44 iotedge.rsu.local systemd[1]: Stopped Azure IoT Identity Service.
May 28 08:56:44 iotedge.rsu.local systemd[1]: Started Azure IoT Identity Service.
May 28 08:56:44 iotedge.rsu.local aziot-identityd[13930]: 2021-05-28T08:56:44Z [INFO] - Starting service...
May 28 08:56:44 iotedge.rsu.local aziot-identityd[13930]: 2021-05-28T08:56:44Z [INFO] - Version - 1.2.0
May 28 08:56:44 iotedge.rsu.local systemd[1]: aziot-identityd.service: main process exited, code=exited, status=1/FAILURE
May 28 08:56:44 iotedge.rsu.local systemd[1]: Unit aziot-identityd.service entered failed state.
May 28 08:56:44 iotedge.rsu.local systemd[1]: aziot-identityd.service failed.
May 28 08:56:49 iotedge.rsu.local systemd[1]: aziot-identityd.service holdoff time over, scheduling restart.
May 28 08:56:49 iotedge.rsu.local systemd[1]: Stopped Azure IoT Identity Service.
May 28 08:56:49 iotedge.rsu.local systemd[1]: Started Azure IoT Identity Service.
May 28 08:56:49 iotedge.rsu.local aziot-identityd[13944]: 2021-05-28T08:56:49Z [INFO] - Starting service...
May 28 08:56:49 iotedge.rsu.local aziot-identityd[13944]: 2021-05-28T08:56:49Z [INFO] - Version - 1.2.0
May 28 08:56:49 iotedge.rsu.local systemd[1]: aziot-identityd.service: main process exited, code=exited, status=1/FAILURE
May 28 08:56:49 iotedge.rsu.local systemd[1]: Unit aziot-identityd.service entered failed state.
May 28 08:56:49 iotedge.rsu.local systemd[1]: aziot-identityd.service failed.
May 28 08:56:55 iotedge.rsu.local systemd[1]: aziot-identityd.service holdoff time over, scheduling restart.
May 28 08:56:55 iotedge.rsu.local systemd[1]: Stopped Azure IoT Identity Service.
May 28 08:56:55 iotedge.rsu.local systemd[1]: Started Azure IoT Identity Service.
May 28 08:56:55 iotedge.rsu.local aziot-identityd[13959]: 2021-05-28T08:56:55Z [INFO] - Starting service...
May 28 08:56:55 iotedge.rsu.local aziot-identityd[13959]: 2021-05-28T08:56:55Z [INFO] - Version - 1.2.0
May 28 08:56:55 iotedge.rsu.local systemd[1]: aziot-identityd.service: main process exited, code=exited, status=1/FAILURE
May 28 08:56:55 iotedge.rsu.local systemd[1]: Unit aziot-identityd.service entered failed state.
May 28 08:56:55 iotedge.rsu.local systemd[1]: aziot-identityd.service failed.
May 28 08:57:00 iotedge.rsu.local systemd[1]: aziot-identityd.service holdoff time over, scheduling restart.
May 28 08:57:00 iotedge.rsu.local systemd[1]: Stopped Azure IoT Identity Service.
May 28 08:57:00 iotedge.rsu.local systemd[1]: Started Azure IoT Identity Service.
May 28 08:57:00 iotedge.rsu.local systemd[1]: aziot-identityd.service: main process exited, code=exited, status=1/FAILURE
May 28 08:57:00 iotedge.rsu.local systemd[1]: Unit aziot-identityd.service entered failed state.
May 28 08:57:00 iotedge.rsu.local systemd[1]: aziot-identityd.service failed.
May 28 08:57:05 iotedge.rsu.local systemd[1]: aziot-identityd.service holdoff time over, scheduling restart.
May 28 08:57:05 iotedge.rsu.local systemd[1]: Stopped Azure IoT Identity Service.
May 28 08:57:05 iotedge.rsu.local systemd[1]: Started Azure IoT Identity Service.
May 28 08:57:05 iotedge.rsu.local systemd[1]: aziot-identityd.service: main process exited, code=exited, status=1/FAILURE
May 28 08:57:05 iotedge.rsu.local systemd[1]: Unit aziot-identityd.service entered failed state.
May 28 08:57:05 iotedge.rsu.local systemd[1]: aziot-identityd.service failed.
May 28 08:57:10 iotedge.rsu.local systemd[1]: aziot-identityd.service holdoff time over, scheduling restart.
May 28 08:57:10 iotedge.rsu.local systemd[1]: Stopped Azure IoT Identity Service.
May 28 08:57:10 iotedge.rsu.local systemd[1]: Started Azure IoT Identity Service.
May 28 08:57:10 iotedge.rsu.local aziot-identityd[13999]: 2021-05-28T08:57:10Z [INFO] - Starting service...
May 28 08:57:10 iotedge.rsu.local aziot-identityd[13999]: 2021-05-28T08:57:10Z [INFO] - Version - 1.2.0
May 28 08:57:10 iotedge.rsu.local systemd[1]: aziot-identityd.service: main process exited, code=exited, status=1/FAILURE
May 28 08:57:10 iotedge.rsu.local systemd[1]: Unit aziot-identityd.service entered failed state.
May 28 08:57:10 iotedge.rsu.local systemd[1]: aziot-identityd.service failed.
May 28 08:57:16 iotedge.rsu.local systemd[1]: aziot-identityd.service holdoff time over, scheduling restart.
May 28 08:57:16 iotedge.rsu.local systemd[1]: Stopped Azure IoT Identity Service.
May 28 08:57:16 iotedge.rsu.local systemd[1]: Started Azure IoT Identity Service.
May 28 08:57:16 iotedge.rsu.local aziot-identityd[14013]: 2021-05-28T08:57:16Z [INFO] - Starting service...
May 28 08:57:16 iotedge.rsu.local aziot-identityd[14013]: 2021-05-28T08:57:16Z [INFO] - Version - 1.2.0
May 28 08:57:16 iotedge.rsu.local systemd[1]: aziot-identityd.service: main process exited, code=exited, status=1/FAILURE
May 28 08:57:16 iotedge.rsu.local systemd[1]: Unit aziot-identityd.service entered failed state.
May 28 08:57:16 iotedge.rsu.local systemd[1]: aziot-identityd.service failed.
May 28 08:57:21 iotedge.rsu.local systemd[1]: aziot-identityd.service holdoff time over, scheduling restart.
May 28 08:57:21 iotedge.rsu.local systemd[1]: Stopped Azure IoT Identity Service.
May 28 08:57:21 iotedge.rsu.local systemd[1]: Started Azure IoT Identity Service.
May 28 08:57:21 iotedge.rsu.local aziot-identityd[14027]: 2021-05-28T08:57:21Z [INFO] - Starting service...
May 28 08:57:21 iotedge.rsu.local aziot-identityd[14027]: 2021-05-28T08:57:21Z [INFO] - Version - 1.2.0
May 28 08:57:21 iotedge.rsu.local systemd[1]: aziot-identityd.service: main process exited, code=exited, status=1/FAILURE
May 28 08:57:21 iotedge.rsu.local systemd[1]: Unit aziot-identityd.service entered failed state.
May 28 08:57:21 iotedge.rsu.local systemd[1]: aziot-identityd.service failed.
May 28 08:57:26 iotedge.rsu.local systemd[1]: aziot-identityd.service holdoff time over, scheduling restart.
May 28 08:57:26 iotedge.rsu.local systemd[1]: Stopped Azure IoT Identity Service.
May 28 08:57:26 iotedge.rsu.local systemd[1]: Started Azure IoT Identity Service.
May 28 08:57:26 iotedge.rsu.local aziot-identityd[14041]: 2021-05-28T08:57:26Z [INFO] - Starting service...
May 28 08:57:26 iotedge.rsu.local aziot-identityd[14041]: 2021-05-28T08:57:26Z [INFO] - Version - 1.2.0
May 28 08:57:26 iotedge.rsu.local systemd[1]: aziot-identityd.service: main process exited, code=exited, status=1/FAILURE
May 28 08:57:26 iotedge.rsu.local systemd[1]: Unit aziot-identityd.service entered failed state.
May 28 08:57:26 iotedge.rsu.local systemd[1]: aziot-identityd.service failed.
May 28 08:57:31 iotedge.rsu.local systemd[1]: aziot-identityd.service holdoff time over, scheduling restart.
May 28 08:57:31 iotedge.rsu.local systemd[1]: Stopped Azure IoT Identity Service.
May 28 08:57:31 iotedge.rsu.local systemd[1]: Started Azure IoT Identity Service.
May 28 08:57:31 iotedge.rsu.local aziot-identityd[14056]: 2021-05-28T08:57:31Z [INFO] - Starting service...
May 28 08:57:31 iotedge.rsu.local aziot-identityd[14056]: 2021-05-28T08:57:31Z [INFO] - Version - 1.2.0
May 28 08:57:31 iotedge.rsu.local systemd[1]: aziot-identityd.service: main process exited, code=exited, status=1/FAILURE
May 28 08:57:31 iotedge.rsu.local systemd[1]: Unit aziot-identityd.service entered failed state.
May 28 08:57:31 iotedge.rsu.local systemd[1]: aziot-identityd.service failed.
May 28 08:57:37 iotedge.rsu.local systemd[1]: aziot-identityd.service holdoff time over, scheduling restart.
May 28 08:57:37 iotedge.rsu.local systemd[1]: Stopped Azure IoT Identity Service.
May 28 08:57:37 iotedge.rsu.local systemd[1]: Started Azure IoT Identity Service.
May 28 08:57:37 iotedge.rsu.local aziot-identityd[14072]: 2021-05-28T08:57:37Z [INFO] - Starting service...
May 28 08:57:37 iotedge.rsu.local aziot-identityd[14072]: 2021-05-28T08:57:37Z [INFO] - Version - 1.2.0
May 28 08:57:37 iotedge.rsu.local systemd[1]: aziot-identityd.service: main process exited, code=exited, status=1/FAILURE
May 28 08:57:37 iotedge.rsu.local systemd[1]: Unit aziot-identityd.service entered failed state.
May 28 08:57:37 iotedge.rsu.local systemd[1]: aziot-identityd.service failed.
May 28 08:57:42 iotedge.rsu.local systemd[1]: aziot-identityd.service holdoff time over, scheduling restart.
May 28 08:57:42 iotedge.rsu.local systemd[1]: Stopped Azure IoT Identity Service.
May 28 08:57:42 iotedge.rsu.local systemd[1]: Started Azure IoT Identity Service.
May 28 08:57:42 iotedge.rsu.local systemd[1]: aziot-identityd.service: main process exited, code=exited, status=1/FAILURE
May 28 08:57:42 iotedge.rsu.local systemd[1]: Unit aziot-identityd.service entered failed state.
May 28 08:57:42 iotedge.rsu.local systemd[1]: aziot-identityd.service failed.
May 28 08:57:47 iotedge.rsu.local systemd[1]: aziot-identityd.service holdoff time over, scheduling restart.
May 28 08:57:47 iotedge.rsu.local systemd[1]: Stopped Azure IoT Identity Service.
May 28 08:57:47 iotedge.rsu.local systemd[1]: Started Azure IoT Identity Service.
May 28 08:57:47 iotedge.rsu.local aziot-identityd[14099]: 2021-05-28T08:57:47Z [INFO] - Starting service...
May 28 08:57:47 iotedge.rsu.local aziot-identityd[14099]: 2021-05-28T08:57:47Z [INFO] - Version - 1.2.0
May 28 08:57:47 iotedge.rsu.local systemd[1]: aziot-identityd.service: main process exited, code=exited, status=1/FAILURE
May 28 08:57:47 iotedge.rsu.local systemd[1]: Unit aziot-identityd.service entered failed state.
May 28 08:57:47 iotedge.rsu.local systemd[1]: aziot-identityd.service failed.
May 28 08:57:52 iotedge.rsu.local systemd[1]: aziot-identityd.service holdoff time over, scheduling restart.
May 28 08:57:52 iotedge.rsu.local systemd[1]: Stopped Azure IoT Identity Service.
May 28 08:57:52 iotedge.rsu.local systemd[1]: Started Azure IoT Identity Service.
May 28 08:57:52 iotedge.rsu.local aziot-identityd[14113]: 2021-05-28T08:57:52Z [INFO] - Starting service...
May 28 08:57:52 iotedge.rsu.local aziot-identityd[14113]: 2021-05-28T08:57:52Z [INFO] - Version - 1.2.0
May 28 08:57:52 iotedge.rsu.local systemd[1]: aziot-identityd.service: main process exited, code=exited, status=1/FAILURE
May 28 08:57:52 iotedge.rsu.local systemd[1]: Unit aziot-identityd.service entered failed state.
May 28 08:57:52 iotedge.rsu.local systemd[1]: aziot-identityd.service failed.
May 28 08:57:58 iotedge.rsu.local systemd[1]: aziot-identityd.service holdoff time over, scheduling restart.
May 28 08:57:58 iotedge.rsu.local systemd[1]: Stopped Azure IoT Identity Service.
May 28 08:57:58 iotedge.rsu.local systemd[1]: Started Azure IoT Identity Service.
May 28 08:57:58 iotedge.rsu.local aziot-identityd[14129]: 2021-05-28T08:57:58Z [INFO] - Starting service...
May 28 08:57:58 iotedge.rsu.local aziot-identityd[14129]: 2021-05-28T08:57:58Z [INFO] - Version - 1.2.0
May 28 08:57:58 iotedge.rsu.local systemd[1]: aziot-identityd.service: main process exited, code=exited, status=1/FAILURE
May 28 08:57:58 iotedge.rsu.local systemd[1]: Unit aziot-identityd.service entered failed state.
May 28 08:57:58 iotedge.rsu.local systemd[1]: aziot-identityd.service failed.
May 28 08:58:03 iotedge.rsu.local systemd[1]: aziot-identityd.service holdoff time over, scheduling restart.
May 28 08:58:03 iotedge.rsu.local systemd[1]: Stopped Azure IoT Identity Service.
May 28 08:58:03 iotedge.rsu.local systemd[1]: Started Azure IoT Identity Service.
May 28 08:58:03 iotedge.rsu.local aziot-identityd[14144]: 2021-05-28T08:58:03Z [INFO] - Starting service...
May 28 08:58:03 iotedge.rsu.local aziot-identityd[14144]: 2021-05-28T08:58:03Z [INFO] - Version - 1.2.0
May 28 08:58:03 iotedge.rsu.local systemd[1]: aziot-identityd.service: main process exited, code=exited, status=1/FAILURE
May 28 08:58:03 iotedge.rsu.local systemd[1]: Unit aziot-identityd.service entered failed state.
May 28 08:58:03 iotedge.rsu.local systemd[1]: aziot-identityd.service failed.
May 28 08:58:08 iotedge.rsu.local systemd[1]: aziot-identityd.service holdoff time over, scheduling restart.
May 28 08:58:08 iotedge.rsu.local systemd[1]: Stopped Azure IoT Identity Service.
May 28 08:58:08 iotedge.rsu.local systemd[1]: Started Azure IoT Identity Service.
May 28 08:58:08 iotedge.rsu.local aziot-identityd[14159]: 2021-05-28T08:58:08Z [INFO] - Starting service...
May 28 08:58:08 iotedge.rsu.local aziot-identityd[14159]: 2021-05-28T08:58:08Z [INFO] - Version - 1.2.0
May 28 08:58:08 iotedge.rsu.local systemd[1]: aziot-identityd.service: main process exited, code=exited, status=1/FAILURE
May 28 08:58:08 iotedge.rsu.local systemd[1]: Unit aziot-identityd.service entered failed state.
May 28 08:58:08 iotedge.rsu.local systemd[1]: aziot-identityd.service failed.
May 28 08:58:13 iotedge.rsu.local systemd[1]: aziot-identityd.service holdoff time over, scheduling restart.
May 28 08:58:13 iotedge.rsu.local systemd[1]: Stopped Azure IoT Identity Service.
May 28 08:58:13 iotedge.rsu.local systemd[1]: Started Azure IoT Identity Service.
May 28 08:58:13 iotedge.rsu.local aziot-identityd[14172]: 2021-05-28T08:58:13Z [INFO] - Starting service...
May 28 08:58:13 iotedge.rsu.local aziot-identityd[14172]: 2021-05-28T08:58:13Z [INFO] - Version - 1.2.0
May 28 08:58:13 iotedge.rsu.local systemd[1]: aziot-identityd.service: main process exited, code=exited, status=1/FAILURE
May 28 08:58:13 iotedge.rsu.local systemd[1]: Unit aziot-identityd.service entered failed state.
May 28 08:58:13 iotedge.rsu.local systemd[1]: aziot-identityd.service failed.
May 28 08:58:19 iotedge.rsu.local systemd[1]: aziot-identityd.service holdoff time over, scheduling restart.
May 28 08:58:19 iotedge.rsu.local systemd[1]: Stopped Azure IoT Identity Service.
May 28 08:58:19 iotedge.rsu.local systemd[1]: Started Azure IoT Identity Service.
May 28 08:58:19 iotedge.rsu.local aziot-identityd[14187]: 2021-05-28T08:58:19Z [INFO] - Starting service...
May 28 08:58:19 iotedge.rsu.local aziot-identityd[14187]: 2021-05-28T08:58:19Z [INFO] - Version - 1.2.0
May 28 08:58:19 iotedge.rsu.local systemd[1]: aziot-identityd.service: main process exited, code=exited, status=1/FAILURE
May 28 08:58:19 iotedge.rsu.local systemd[1]: Unit aziot-identityd.service entered failed state.
May 28 08:58:19 iotedge.rsu.local systemd[1]: aziot-identityd.service failed.
May 28 08:58:24 iotedge.rsu.local systemd[1]: aziot-identityd.service holdoff time over, scheduling restart.
May 28 08:58:24 iotedge.rsu.local systemd[1]: Stopped Azure IoT Identity Service.
May 28 08:58:24 iotedge.rsu.local systemd[1]: Started Azure IoT Identity Service.
May 28 08:58:24 iotedge.rsu.local systemd[1]: aziot-identityd.service: main process exited, code=exited, status=1/FAILURE
May 28 08:58:24 iotedge.rsu.local systemd[1]: Unit aziot-identityd.service entered failed state.
May 28 08:58:24 iotedge.rsu.local systemd[1]: aziot-identityd.service failed.
May 28 08:58:29 iotedge.rsu.local systemd[1]: aziot-identityd.service holdoff time over, scheduling restart.
May 28 08:58:29 iotedge.rsu.local systemd[1]: Stopped Azure IoT Identity Service.
May 28 08:58:29 iotedge.rsu.local systemd[1]: Started Azure IoT Identity Service.
May 28 08:58:29 iotedge.rsu.local aziot-identityd[14216]: 2021-05-28T08:58:29Z [INFO] - Starting service...
May 28 08:58:29 iotedge.rsu.local aziot-identityd[14216]: 2021-05-28T08:58:29Z [INFO] - Version - 1.2.0
May 28 08:58:29 iotedge.rsu.local systemd[1]: aziot-identityd.service: main process exited, code=exited, status=1/FAILURE
May 28 08:58:29 iotedge.rsu.local systemd[1]: Unit aziot-identityd.service entered failed state.
May 28 08:58:29 iotedge.rsu.local systemd[1]: aziot-identityd.service failed.
May 28 08:58:34 iotedge.rsu.local systemd[1]: aziot-identityd.service holdoff time over, scheduling restart.
May 28 08:58:34 iotedge.rsu.local systemd[1]: Stopped Azure IoT Identity Service.
May 28 08:58:34 iotedge.rsu.local systemd[1]: Started Azure IoT Identity Service.
May 28 08:58:34 iotedge.rsu.local aziot-identityd[14229]: 2021-05-28T08:58:34Z [INFO] - Starting service...
May 28 08:58:34 iotedge.rsu.local aziot-identityd[14229]: 2021-05-28T08:58:34Z [INFO] - Version - 1.2.0
May 28 08:58:34 iotedge.rsu.local systemd[1]: aziot-identityd.service: main process exited, code=exited, status=1/FAILURE
May 28 08:58:34 iotedge.rsu.local systemd[1]: Unit aziot-identityd.service entered failed state.
May 28 08:58:34 iotedge.rsu.local systemd[1]: aziot-identityd.service failed.
May 28 08:58:40 iotedge.rsu.local systemd[1]: aziot-identityd.service holdoff time over, scheduling restart.
May 28 08:58:40 iotedge.rsu.local systemd[1]: Stopped Azure IoT Identity Service.
May 28 08:58:40 iotedge.rsu.local systemd[1]: Started Azure IoT Identity Service.
May 28 08:58:40 iotedge.rsu.local systemd[1]: aziot-identityd.service: main process exited, code=exited, status=1/FAILURE
May 28 08:58:40 iotedge.rsu.local systemd[1]: Unit aziot-identityd.service entered failed state.
May 28 08:58:40 iotedge.rsu.local systemd[1]: aziot-identityd.service failed.
May 28 08:58:45 iotedge.rsu.local systemd[1]: aziot-identityd.service holdoff time over, scheduling restart.
May 28 08:58:45 iotedge.rsu.local systemd[1]: Stopped Azure IoT Identity Service.
May 28 08:58:45 iotedge.rsu.local systemd[1]: Started Azure IoT Identity Service.
May 28 08:58:45 iotedge.rsu.local systemd[1]: aziot-identityd.service: main process exited, code=exited, status=1/FAILURE
May 28 08:58:45 iotedge.rsu.local systemd[1]: Unit aziot-identityd.service entered failed state.
May 28 08:58:45 iotedge.rsu.local systemd[1]: aziot-identityd.service failed.
May 28 08:58:50 iotedge.rsu.local systemd[1]: aziot-identityd.service holdoff time over, scheduling restart.
May 28 08:58:50 iotedge.rsu.local systemd[1]: Stopped Azure IoT Identity Service.
May 28 08:58:50 iotedge.rsu.local systemd[1]: Started Azure IoT Identity Service.
May 28 08:58:50 iotedge.rsu.local systemd[1]: aziot-identityd.service: main process exited, code=exited, status=1/FAILURE
May 28 08:58:50 iotedge.rsu.local systemd[1]: Unit aziot-identityd.service entered failed state.
May 28 08:58:50 iotedge.rsu.local systemd[1]: aziot-identityd.service failed.
May 28 08:58:55 iotedge.rsu.local systemd[1]: aziot-identityd.service holdoff time over, scheduling restart.
May 28 08:58:55 iotedge.rsu.local systemd[1]: Stopped Azure IoT Identity Service.
May 28 08:58:55 iotedge.rsu.local systemd[1]: Started Azure IoT Identity Service.
May 28 08:58:55 iotedge.rsu.local aziot-identityd[14284]: 2021-05-28T08:58:55Z [INFO] - Starting service...
May 28 08:58:55 iotedge.rsu.local aziot-identityd[14284]: 2021-05-28T08:58:55Z [INFO] - Version - 1.2.0
May 28 08:58:55 iotedge.rsu.local systemd[1]: aziot-identityd.service: main process exited, code=exited, status=1/FAILURE
May 28 08:58:55 iotedge.rsu.local systemd[1]: Unit aziot-identityd.service entered failed state.
May 28 08:58:55 iotedge.rsu.local systemd[1]: aziot-identityd.service failed.
May 28 08:59:01 iotedge.rsu.local systemd[1]: aziot-identityd.service holdoff time over, scheduling restart.
May 28 08:59:01 iotedge.rsu.local systemd[1]: Stopped Azure IoT Identity Service.
May 28 08:59:01 iotedge.rsu.local systemd[1]: Started Azure IoT Identity Service.
May 28 08:59:01 iotedge.rsu.local aziot-identityd[14299]: 2021-05-28T08:59:01Z [INFO] - Starting service...
May 28 08:59:01 iotedge.rsu.local aziot-identityd[14299]: 2021-05-28T08:59:01Z [INFO] - Version - 1.2.0
May 28 08:59:01 iotedge.rsu.local systemd[1]: aziot-identityd.service: main process exited, code=exited, status=1/FAILURE
May 28 08:59:01 iotedge.rsu.local systemd[1]: Unit aziot-identityd.service entered failed state.
May 28 08:59:01 iotedge.rsu.local systemd[1]: aziot-identityd.service failed.
May 28 08:59:06 iotedge.rsu.local systemd[1]: aziot-identityd.service holdoff time over, scheduling restart.
May 28 08:59:06 iotedge.rsu.local systemd[1]: Stopped Azure IoT Identity Service.
May 28 08:59:06 iotedge.rsu.local systemd[1]: Started Azure IoT Identity Service.
May 28 08:59:06 iotedge.rsu.local aziot-identityd[14313]: 2021-05-28T08:59:06Z [INFO] - Starting service...
May 28 08:59:06 iotedge.rsu.local aziot-identityd[14313]: 2021-05-28T08:59:06Z [INFO] - Version - 1.2.0
May 28 08:59:06 iotedge.rsu.local systemd[1]: aziot-identityd.service: main process exited, code=exited, status=1/FAILURE
May 28 08:59:06 iotedge.rsu.local systemd[1]: Unit aziot-identityd.service entered failed state.
May 28 08:59:06 iotedge.rsu.local systemd[1]: aziot-identityd.service failed.
May 28 08:59:11 iotedge.rsu.local systemd[1]: aziot-identityd.service holdoff time over, scheduling restart.
May 28 08:59:11 iotedge.rsu.local systemd[1]: Stopped Azure IoT Identity Service.
May 28 08:59:11 iotedge.rsu.local systemd[1]: Started Azure IoT Identity Service.
May 28 08:59:11 iotedge.rsu.local aziot-identityd[14326]: 2021-05-28T08:59:11Z [INFO] - Starting service...
May 28 08:59:11 iotedge.rsu.local aziot-identityd[14326]: 2021-05-28T08:59:11Z [INFO] - Version - 1.2.0
May 28 08:59:11 iotedge.rsu.local systemd[1]: aziot-identityd.service: main process exited, code=exited, status=1/FAILURE
May 28 08:59:11 iotedge.rsu.local systemd[1]: Unit aziot-identityd.service entered failed state.
May 28 08:59:11 iotedge.rsu.local systemd[1]: aziot-identityd.service failed.
May 28 08:59:16 iotedge.rsu.local systemd[1]: aziot-identityd.service holdoff time over, scheduling restart.
May 28 08:59:16 iotedge.rsu.local systemd[1]: Stopped Azure IoT Identity Service.
May 28 08:59:16 iotedge.rsu.local systemd[1]: Started Azure IoT Identity Service.
May 28 08:59:16 iotedge.rsu.local aziot-identityd[14339]: 2021-05-28T08:59:16Z [INFO] - Starting service...
May 28 08:59:16 iotedge.rsu.local aziot-identityd[14339]: 2021-05-28T08:59:16Z [INFO] - Version - 1.2.0
May 28 08:59:16 iotedge.rsu.local systemd[1]: aziot-identityd.service: main process exited, code=exited, status=1/FAILURE
May 28 08:59:16 iotedge.rsu.local systemd[1]: Unit aziot-identityd.service entered failed state.
May 28 08:59:16 iotedge.rsu.local systemd[1]: aziot-identityd.service failed.
May 28 08:59:22 iotedge.rsu.local systemd[1]: aziot-identityd.service holdoff time over, scheduling restart.
May 28 08:59:22 iotedge.rsu.local systemd[1]: Stopped Azure IoT Identity Service.
May 28 08:59:22 iotedge.rsu.local systemd[1]: Started Azure IoT Identity Service.
May 28 08:59:22 iotedge.rsu.local aziot-identityd[14354]: 2021-05-28T08:59:22Z [INFO] - Starting service...
May 28 08:59:22 iotedge.rsu.local systemd[1]: aziot-identityd.service: main process exited, code=exited, status=1/FAILURE
May 28 08:59:22 iotedge.rsu.local systemd[1]: Unit aziot-identityd.service entered failed state.
May 28 08:59:22 iotedge.rsu.local systemd[1]: aziot-identityd.service failed.
May 28 08:59:27 iotedge.rsu.local systemd[1]: aziot-identityd.service holdoff time over, scheduling restart.
May 28 08:59:27 iotedge.rsu.local systemd[1]: Stopped Azure IoT Identity Service.
May 28 08:59:27 iotedge.rsu.local systemd[1]: Started Azure IoT Identity Service.
May 28 08:59:27 iotedge.rsu.local aziot-identityd[14367]: 2021-05-28T08:59:27Z [INFO] - Starting service...
May 28 08:59:27 iotedge.rsu.local aziot-identityd[14367]: 2021-05-28T08:59:27Z [INFO] - Version - 1.2.0
May 28 08:59:27 iotedge.rsu.local systemd[1]: aziot-identityd.service: main process exited, code=exited, status=1/FAILURE
May 28 08:59:27 iotedge.rsu.local systemd[1]: Unit aziot-identityd.service entered failed state.
May 28 08:59:27 iotedge.rsu.local systemd[1]: aziot-identityd.service failed.
May 28 08:59:32 iotedge.rsu.local systemd[1]: aziot-identityd.service holdoff time over, scheduling restart.
May 28 08:59:32 iotedge.rsu.local systemd[1]: Stopped Azure IoT Identity Service.
May 28 08:59:32 iotedge.rsu.local systemd[1]: Started Azure IoT Identity Service.
May 28 08:59:32 iotedge.rsu.local aziot-identityd[14381]: 2021-05-28T08:59:32Z [INFO] - Starting service...
May 28 08:59:32 iotedge.rsu.local aziot-identityd[14381]: 2021-05-28T08:59:32Z [INFO] - Version - 1.2.0
May 28 08:59:32 iotedge.rsu.local systemd[1]: aziot-identityd.service: main process exited, code=exited, status=1/FAILURE
May 28 08:59:32 iotedge.rsu.local systemd[1]: Unit aziot-identityd.service entered failed state.
May 28 08:59:32 iotedge.rsu.local systemd[1]: aziot-identityd.service failed.
May 28 08:59:37 iotedge.rsu.local systemd[1]: aziot-identityd.service holdoff time over, scheduling restart.
May 28 08:59:37 iotedge.rsu.local systemd[1]: Stopped Azure IoT Identity Service.
May 28 08:59:37 iotedge.rsu.local systemd[1]: Started Azure IoT Identity Service.
May 28 08:59:37 iotedge.rsu.local aziot-identityd[14395]: 2021-05-28T08:59:37Z [INFO] - Starting service...
May 28 08:59:37 iotedge.rsu.local aziot-identityd[14395]: 2021-05-28T08:59:37Z [INFO] - Version - 1.2.0
May 28 08:59:37 iotedge.rsu.local systemd[1]: aziot-identityd.service: main process exited, code=exited, status=1/FAILURE
May 28 08:59:37 iotedge.rsu.local systemd[1]: Unit aziot-identityd.service entered failed state.
May 28 08:59:37 iotedge.rsu.local systemd[1]: aziot-identityd.service failed.
May 28 08:59:43 iotedge.rsu.local systemd[1]: aziot-identityd.service holdoff time over, scheduling restart.
May 28 08:59:43 iotedge.rsu.local systemd[1]: Stopped Azure IoT Identity Service.
May 28 08:59:43 iotedge.rsu.local systemd[1]: Started Azure IoT Identity Service.
May 28 08:59:43 iotedge.rsu.local aziot-identityd[14410]: 2021-05-28T08:59:43Z [INFO] - Starting service...
May 28 08:59:43 iotedge.rsu.local aziot-identityd[14410]: 2021-05-28T08:59:43Z [INFO] - Version - 1.2.0
May 28 08:59:43 iotedge.rsu.local systemd[1]: aziot-identityd.service: main process exited, code=exited, status=1/FAILURE
May 28 08:59:43 iotedge.rsu.local systemd[1]: Unit aziot-identityd.service entered failed state.
May 28 08:59:43 iotedge.rsu.local systemd[1]: aziot-identityd.service failed.
May 28 08:59:48 iotedge.rsu.local systemd[1]: aziot-identityd.service holdoff time over, scheduling restart.
May 28 08:59:48 iotedge.rsu.local systemd[1]: Stopped Azure IoT Identity Service.
May 28 08:59:48 iotedge.rsu.local systemd[1]: Started Azure IoT Identity Service.
May 28 08:59:48 iotedge.rsu.local aziot-identityd[14424]: 2021-05-28T08:59:48Z [INFO] - Starting service...
May 28 08:59:48 iotedge.rsu.local aziot-identityd[14424]: 2021-05-28T08:59:48Z [INFO] - Version - 1.2.0
May 28 08:59:48 iotedge.rsu.local systemd[1]: aziot-identityd.service: main process exited, code=exited, status=1/FAILURE
May 28 08:59:48 iotedge.rsu.local systemd[1]: Unit aziot-identityd.service entered failed state.
May 28 08:59:48 iotedge.rsu.local systemd[1]: aziot-identityd.service failed.
May 28 08:59:53 iotedge.rsu.local systemd[1]: aziot-identityd.service holdoff time over, scheduling restart.
May 28 08:59:53 iotedge.rsu.local systemd[1]: Stopped Azure IoT Identity Service.
May 28 08:59:53 iotedge.rsu.local systemd[1]: Started Azure IoT Identity Service.
May 28 08:59:53 iotedge.rsu.local aziot-identityd[14439]: 2021-05-28T08:59:53Z [INFO] - Starting service...
May 28 08:59:53 iotedge.rsu.local aziot-identityd[14439]: 2021-05-28T08:59:53Z [INFO] - Version - 1.2.0
May 28 08:59:53 iotedge.rsu.local systemd[1]: aziot-identityd.service: main process exited, code=exited, status=1/FAILURE
May 28 08:59:53 iotedge.rsu.local systemd[1]: Unit aziot-identityd.service entered failed state.
May 28 08:59:53 iotedge.rsu.local systemd[1]: aziot-identityd.service failed.
May 28 08:59:58 iotedge.rsu.local systemd[1]: aziot-identityd.service holdoff time over, scheduling restart.
May 28 08:59:58 iotedge.rsu.local systemd[1]: Stopped Azure IoT Identity Service.
May 28 08:59:58 iotedge.rsu.local systemd[1]: Started Azure IoT Identity Service.
May 28 08:59:58 iotedge.rsu.local aziot-identityd[14453]: 2021-05-28T08:59:58Z [INFO] - Starting service...
May 28 08:59:58 iotedge.rsu.local aziot-identityd[14453]: 2021-05-28T08:59:58Z [INFO] - Version - 1.2.0
May 28 08:59:58 iotedge.rsu.local systemd[1]: aziot-identityd.service: main process exited, code=exited, status=1/FAILURE
May 28 08:59:58 iotedge.rsu.local systemd[1]: Unit aziot-identityd.service entered failed state.
May 28 08:59:58 iotedge.rsu.local systemd[1]: aziot-identityd.service failed.
May 28 09:00:04 iotedge.rsu.local systemd[1]: aziot-identityd.service holdoff time over, scheduling restart.
May 28 09:00:04 iotedge.rsu.local systemd[1]: Stopped Azure IoT Identity Service.
May 28 09:00:04 iotedge.rsu.local systemd[1]: Started Azure IoT Identity Service.
May 28 09:00:04 iotedge.rsu.local aziot-identityd[14467]: 2021-05-28T09:00:04Z [INFO] - Starting service...
May 28 09:00:04 iotedge.rsu.local aziot-identityd[14467]: 2021-05-28T09:00:04Z [INFO] - Version - 1.2.0
May 28 09:00:04 iotedge.rsu.local systemd[1]: aziot-identityd.service: main process exited, code=exited, status=1/FAILURE
May 28 09:00:04 iotedge.rsu.local systemd[1]: Unit aziot-identityd.service entered failed state.
May 28 09:00:04 iotedge.rsu.local systemd[1]: aziot-identityd.service failed.
May 28 09:00:09 iotedge.rsu.local systemd[1]: aziot-identityd.service holdoff time over, scheduling restart.
May 28 09:00:09 iotedge.rsu.local systemd[1]: Stopped Azure IoT Identity Service.
May 28 09:00:09 iotedge.rsu.local systemd[1]: Started Azure IoT Identity Service.
May 28 09:00:09 iotedge.rsu.local systemd[1]: aziot-identityd.service: main process exited, code=exited, status=1/FAILURE
May 28 09:00:09 iotedge.rsu.local systemd[1]: Unit aziot-identityd.service entered failed state.
May 28 09:00:09 iotedge.rsu.local systemd[1]: aziot-identityd.service failed.
May 28 09:00:14 iotedge.rsu.local systemd[1]: aziot-identityd.service holdoff time over, scheduling restart.
May 28 09:00:14 iotedge.rsu.local systemd[1]: Stopped Azure IoT Identity Service.
May 28 09:00:14 iotedge.rsu.local systemd[1]: Started Azure IoT Identity Service.
May 28 09:00:14 iotedge.rsu.local aziot-identityd[14496]: 2021-05-28T09:00:14Z [INFO] - Starting service...
May 28 09:00:14 iotedge.rsu.local aziot-identityd[14496]: 2021-05-28T09:00:14Z [INFO] - Version - 1.2.0
May 28 09:00:14 iotedge.rsu.local systemd[1]: aziot-identityd.service: main process exited, code=exited, status=1/FAILURE
May 28 09:00:14 iotedge.rsu.local systemd[1]: Unit aziot-identityd.service entered failed state.
May 28 09:00:14 iotedge.rsu.local systemd[1]: aziot-identityd.service failed.
May 28 09:00:19 iotedge.rsu.local systemd[1]: aziot-identityd.service holdoff time over, scheduling restart.
May 28 09:00:19 iotedge.rsu.local systemd[1]: Stopped Azure IoT Identity Service.
May 28 09:00:19 iotedge.rsu.local systemd[1]: Started Azure IoT Identity Service.
May 28 09:00:19 iotedge.rsu.local aziot-identityd[14510]: 2021-05-28T09:00:19Z [INFO] - Starting service...
May 28 09:00:19 iotedge.rsu.local aziot-identityd[14510]: 2021-05-28T09:00:19Z [INFO] - Version - 1.2.0
May 28 09:00:19 iotedge.rsu.local systemd[1]: aziot-identityd.service: main process exited, code=exited, status=1/FAILURE
May 28 09:00:19 iotedge.rsu.local systemd[1]: Unit aziot-identityd.service entered failed state.
May 28 09:00:19 iotedge.rsu.local systemd[1]: aziot-identityd.service failed.
May 28 09:00:25 iotedge.rsu.local systemd[1]: aziot-identityd.service holdoff time over, scheduling restart.
May 28 09:00:25 iotedge.rsu.local systemd[1]: Stopped Azure IoT Identity Service.
May 28 09:00:25 iotedge.rsu.local systemd[1]: Started Azure IoT Identity Service.
May 28 09:00:25 iotedge.rsu.local aziot-identityd[14525]: 2021-05-28T09:00:25Z [INFO] - Starting service...
May 28 09:00:25 iotedge.rsu.local aziot-identityd[14525]: 2021-05-28T09:00:25Z [INFO] - Version - 1.2.0
May 28 09:00:25 iotedge.rsu.local systemd[1]: aziot-identityd.service: main process exited, code=exited, status=1/FAILURE
May 28 09:00:25 iotedge.rsu.local systemd[1]: Unit aziot-identityd.service entered failed state.
May 28 09:00:25 iotedge.rsu.local systemd[1]: aziot-identityd.service failed.
May 28 09:00:30 iotedge.rsu.local systemd[1]: aziot-identityd.service holdoff time over, scheduling restart.
May 28 09:00:30 iotedge.rsu.local systemd[1]: Stopped Azure IoT Identity Service.
May 28 09:00:30 iotedge.rsu.local systemd[1]: Started Azure IoT Identity Service.
May 28 09:00:30 iotedge.rsu.local systemd[1]: aziot-identityd.service: main process exited, code=exited, status=1/FAILURE
May 28 09:00:30 iotedge.rsu.local systemd[1]: Unit aziot-identityd.service entered failed state.
May 28 09:00:30 iotedge.rsu.local systemd[1]: aziot-identityd.service failed.
May 28 09:00:35 iotedge.rsu.local systemd[1]: aziot-identityd.service holdoff time over, scheduling restart.
May 28 09:00:35 iotedge.rsu.local systemd[1]: Stopped Azure IoT Identity Service.
May 28 09:00:35 iotedge.rsu.local systemd[1]: Started Azure IoT Identity Service.
May 28 09:00:35 iotedge.rsu.local aziot-identityd[14553]: 2021-05-28T09:00:35Z [INFO] - Starting service...
May 28 09:00:35 iotedge.rsu.local aziot-identityd[14553]: 2021-05-28T09:00:35Z [INFO] - Version - 1.2.0
May 28 09:00:35 iotedge.rsu.local systemd[1]: aziot-identityd.service: main process exited, code=exited, status=1/FAILURE
May 28 09:00:35 iotedge.rsu.local systemd[1]: Unit aziot-identityd.service entered failed state.
May 28 09:00:35 iotedge.rsu.local systemd[1]: aziot-identityd.service failed.
May 28 09:00:40 iotedge.rsu.local systemd[1]: aziot-identityd.service holdoff time over, scheduling restart.
May 28 09:00:40 iotedge.rsu.local systemd[1]: Stopped Azure IoT Identity Service.
May 28 09:00:40 iotedge.rsu.local systemd[1]: Started Azure IoT Identity Service.
May 28 09:00:40 iotedge.rsu.local aziot-identityd[14568]: 2021-05-28T09:00:40Z [INFO] - Starting service...
May 28 09:00:40 iotedge.rsu.local aziot-identityd[14568]: 2021-05-28T09:00:40Z [INFO] - Version - 1.2.0
May 28 09:00:40 iotedge.rsu.local systemd[1]: aziot-identityd.service: main process exited, code=exited, status=1/FAILURE
May 28 09:00:40 iotedge.rsu.local systemd[1]: Unit aziot-identityd.service entered failed state.
May 28 09:00:40 iotedge.rsu.local systemd[1]: aziot-identityd.service failed.
May 28 09:00:46 iotedge.rsu.local systemd[1]: aziot-identityd.service holdoff time over, scheduling restart.
May 28 09:00:46 iotedge.rsu.local systemd[1]: Stopped Azure IoT Identity Service.
May 28 09:00:46 iotedge.rsu.local systemd[1]: Started Azure IoT Identity Service.
May 28 09:00:46 iotedge.rsu.local aziot-identityd[14584]: 2021-05-28T09:00:46Z [INFO] - Starting service...
May 28 09:00:46 iotedge.rsu.local aziot-identityd[14584]: 2021-05-28T09:00:46Z [INFO] - Version - 1.2.0
May 28 09:00:46 iotedge.rsu.local systemd[1]: aziot-identityd.service: main process exited, code=exited, status=1/FAILURE
May 28 09:00:46 iotedge.rsu.local systemd[1]: Unit aziot-identityd.service entered failed state.
May 28 09:00:46 iotedge.rsu.local systemd[1]: aziot-identityd.service failed.
May 28 09:00:51 iotedge.rsu.local systemd[1]: aziot-identityd.service holdoff time over, scheduling restart.
May 28 09:00:51 iotedge.rsu.local systemd[1]: Stopped Azure IoT Identity Service.
May 28 09:00:51 iotedge.rsu.local systemd[1]: Started Azure IoT Identity Service.
May 28 09:00:51 iotedge.rsu.local aziot-identityd[14598]: 2021-05-28T09:00:51Z [INFO] - Starting service...
May 28 09:00:51 iotedge.rsu.local aziot-identityd[14598]: 2021-05-28T09:00:51Z [INFO] - Version - 1.2.0
May 28 09:00:51 iotedge.rsu.local systemd[1]: aziot-identityd.service: main process exited, code=exited, status=1/FAILURE
May 28 09:00:51 iotedge.rsu.local systemd[1]: Unit aziot-identityd.service entered failed state.
May 28 09:00:51 iotedge.rsu.local systemd[1]: aziot-identityd.service failed.
May 28 09:00:56 iotedge.rsu.local systemd[1]: aziot-identityd.service holdoff time over, scheduling restart.
May 28 09:00:56 iotedge.rsu.local systemd[1]: Stopped Azure IoT Identity Service.
May 28 09:00:56 iotedge.rsu.local systemd[1]: Started Azure IoT Identity Service.
May 28 09:00:56 iotedge.rsu.local aziot-identityd[14612]: 2021-05-28T09:00:56Z [INFO] - Starting service...
May 28 09:00:56 iotedge.rsu.local aziot-identityd[14612]: 2021-05-28T09:00:56Z [INFO] - Version - 1.2.0
May 28 09:00:56 iotedge.rsu.local systemd[1]: aziot-identityd.service: main process exited, code=exited, status=1/FAILURE
May 28 09:00:56 iotedge.rsu.local systemd[1]: Unit aziot-identityd.service entered failed state.
May 28 09:00:56 iotedge.rsu.local systemd[1]: aziot-identityd.service failed.
May 28 09:01:01 iotedge.rsu.local systemd[1]: aziot-identityd.service holdoff time over, scheduling restart.
May 28 09:01:01 iotedge.rsu.local systemd[1]: Stopped Azure IoT Identity Service.
May 28 09:01:01 iotedge.rsu.local systemd[1]: Started Azure IoT Identity Service.
May 28 09:01:01 iotedge.rsu.local aziot-identityd[14644]: 2021-05-28T09:01:01Z [INFO] - Starting service...
May 28 09:01:01 iotedge.rsu.local aziot-identityd[14644]: 2021-05-28T09:01:01Z [INFO] - Version - 1.2.0
May 28 09:01:01 iotedge.rsu.local systemd[1]: aziot-identityd.service: main process exited, code=exited, status=1/FAILURE
May 28 09:01:01 iotedge.rsu.local systemd[1]: Unit aziot-identityd.service entered failed state.
May 28 09:01:01 iotedge.rsu.local systemd[1]: aziot-identityd.service failed.
May 28 09:01:06 iotedge.rsu.local systemd[1]: aziot-identityd.service holdoff time over, scheduling restart.
May 28 09:01:06 iotedge.rsu.local systemd[1]: Stopped Azure IoT Identity Service.
May 28 09:01:06 iotedge.rsu.local systemd[1]: Started Azure IoT Identity Service.
May 28 09:01:06 iotedge.rsu.local aziot-identityd[14654]: 2021-05-28T09:01:06Z [INFO] - Starting service...
May 28 09:01:06 iotedge.rsu.local aziot-identityd[14654]: 2021-05-28T09:01:06Z [INFO] - Version - 1.2.0
May 28 09:01:06 iotedge.rsu.local systemd[1]: aziot-identityd.service: main process exited, code=exited, status=1/FAILURE
May 28 09:01:06 iotedge.rsu.local systemd[1]: Unit aziot-identityd.service entered failed state.
May 28 09:01:06 iotedge.rsu.local systemd[1]: aziot-identityd.service failed.
May 28 09:01:12 iotedge.rsu.local systemd[1]: aziot-identityd.service holdoff time over, scheduling restart.
May 28 09:01:12 iotedge.rsu.local systemd[1]: Stopped Azure IoT Identity Service.
May 28 09:01:12 iotedge.rsu.local systemd[1]: Started Azure IoT Identity Service.
May 28 09:01:12 iotedge.rsu.local aziot-identityd[14669]: 2021-05-28T09:01:12Z [INFO] - Starting service...
May 28 09:01:12 iotedge.rsu.local aziot-identityd[14669]: 2021-05-28T09:01:12Z [INFO] - Version - 1.2.0
May 28 09:01:12 iotedge.rsu.local systemd[1]: aziot-identityd.service: main process exited, code=exited, status=1/FAILURE
May 28 09:01:12 iotedge.rsu.local systemd[1]: Unit aziot-identityd.service entered failed state.
May 28 09:01:12 iotedge.rsu.local systemd[1]: aziot-identityd.service failed.
May 28 09:01:17 iotedge.rsu.local systemd[1]: aziot-identityd.service holdoff time over, scheduling restart.
May 28 09:01:17 iotedge.rsu.local systemd[1]: Stopped Azure IoT Identity Service.
May 28 09:01:17 iotedge.rsu.local systemd[1]: Started Azure IoT Identity Service.
May 28 09:01:17 iotedge.rsu.local aziot-identityd[14683]: 2021-05-28T09:01:17Z [INFO] - Starting service...
May 28 09:01:17 iotedge.rsu.local aziot-identityd[14683]: 2021-05-28T09:01:17Z [INFO] - Version - 1.2.0
May 28 09:01:17 iotedge.rsu.local systemd[1]: aziot-identityd.service: main process exited, code=exited, status=1/FAILURE
May 28 09:01:17 iotedge.rsu.local systemd[1]: Unit aziot-identityd.service entered failed state.
May 28 09:01:17 iotedge.rsu.local systemd[1]: aziot-identityd.service failed.
May 28 09:01:22 iotedge.rsu.local systemd[1]: aziot-identityd.service holdoff time over, scheduling restart.
May 28 09:01:22 iotedge.rsu.local systemd[1]: Stopped Azure IoT Identity Service.
May 28 09:01:22 iotedge.rsu.local systemd[1]: Started Azure IoT Identity Service.
May 28 09:01:22 iotedge.rsu.local systemd[1]: aziot-identityd.service: main process exited, code=exited, status=1/FAILURE
May 28 09:01:22 iotedge.rsu.local systemd[1]: Unit aziot-identityd.service entered failed state.
May 28 09:01:22 iotedge.rsu.local systemd[1]: aziot-identityd.service failed.
May 28 09:01:27 iotedge.rsu.local systemd[1]: aziot-identityd.service holdoff time over, scheduling restart.
May 28 09:01:27 iotedge.rsu.local systemd[1]: Stopped Azure IoT Identity Service.
May 28 09:01:27 iotedge.rsu.local systemd[1]: Started Azure IoT Identity Service.
May 28 09:01:27 iotedge.rsu.local aziot-identityd[14711]: 2021-05-28T09:01:27Z [INFO] - Starting service...
May 28 09:01:27 iotedge.rsu.local aziot-identityd[14711]: 2021-05-28T09:01:27Z [INFO] - Version - 1.2.0
May 28 09:01:27 iotedge.rsu.local systemd[1]: aziot-identityd.service: main process exited, code=exited, status=1/FAILURE
May 28 09:01:27 iotedge.rsu.local systemd[1]: Unit aziot-identityd.service entered failed state.
May 28 09:01:27 iotedge.rsu.local systemd[1]: aziot-identityd.service failed.
May 28 09:01:33 iotedge.rsu.local systemd[1]: aziot-identityd.service holdoff time over, scheduling restart.
May 28 09:01:33 iotedge.rsu.local systemd[1]: Stopped Azure IoT Identity Service.
May 28 09:01:33 iotedge.rsu.local systemd[1]: Started Azure IoT Identity Service.
May 28 09:01:33 iotedge.rsu.local aziot-identityd[14726]: 2021-05-28T09:01:33Z [INFO] - Starting service...
May 28 09:01:33 iotedge.rsu.local aziot-identityd[14726]: 2021-05-28T09:01:33Z [INFO] - Version - 1.2.0
May 28 09:01:33 iotedge.rsu.local systemd[1]: aziot-identityd.service: main process exited, code=exited, status=1/FAILURE
May 28 09:01:33 iotedge.rsu.local systemd[1]: Unit aziot-identityd.service entered failed state.
May 28 09:01:33 iotedge.rsu.local systemd[1]: aziot-identityd.service failed.
May 28 09:01:38 sesyiot-auto-24 systemd[1]: Stopped Azure IoT Identity Service.
May 28 09:01:38 sesyiot-auto-24 systemd[1]: Started Azure IoT Edge daemon.
May 28 09:01:38 sesyiot-auto-24 aziot-edged[14864]: 2021-05-28T09:01:38Z [INFO] - Starting Azure IoT Edge Module Runtime
May 28 09:01:38 sesyiot-auto-24 aziot-edged[14864]: 2021-05-28T09:01:38Z [INFO] - Version - 1.2.0
May 28 09:01:38 sesyiot-auto-24 aziot-edged[14864]: 2021-05-28T09:01:38Z [INFO] - Initializing the module runtime...
May 28 09:01:38 sesyiot-auto-24 aziot-edged[14864]: 2021-05-28T09:01:38Z [INFO] - Initializing module runtime...
May 28 09:01:38 sesyiot-auto-24 aziot-edged[14864]: 2021-05-28T09:01:38Z [INFO] - Using runtime network id azure-iot-edge
May 28 09:01:38 sesyiot-auto-24 aziot-edged[14864]: 2021-05-28T09:01:38Z [INFO] - Successfully initialized module runtime
May 28 09:01:38 sesyiot-auto-24 aziot-edged[14864]: 2021-05-28T09:01:38Z [INFO] - Finished initializing the module runtime.
May 28 09:01:38 sesyiot-auto-24 aziot-edged[14864]: 2021-05-28T09:01:38Z [INFO] - Obtaining edge device provisioning data...
May 28 09:01:38 sesyiot-auto-24 systemd[1]: Started Azure IoT Identity Service.
May 28 09:01:38 sesyiot-auto-24 aziot-identityd[14935]: 2021-05-28T09:01:38Z [INFO] - Starting service...
May 28 09:01:38 sesyiot-auto-24 aziot-identityd[14935]: 2021-05-28T09:01:38Z [INFO] - Version - 1.2.0
May 28 09:01:38 sesyiot-auto-24 aziot-identityd[14935]: 2021-05-28T09:01:38Z [INFO] - Detected HTTPS proxy server http://proxy.rsu.local:3128/
May 28 09:01:38 sesyiot-auto-24 aziot-identityd[14935]: 2021-05-28T09:01:38Z [INFO] - Provisioning starting. Reason: Startup
May 28 09:01:38 sesyiot-auto-24 aziot-identityd[14935]: 2021-05-28T09:01:38Z [INFO] - Provisioning complete.
May 28 09:01:38 sesyiot-auto-24 aziot-identityd[14935]: 2021-05-28T09:01:38Z [INFO] - Identity reconciliation started. Reason: Startup
May 28 09:01:38 sesyiot-auto-24 aziot-identityd[14935]: 2021-05-28T09:01:38Z [INFO] - Identity reconciliation complete.
May 28 09:01:38 sesyiot-auto-24 aziot-identityd[14935]: 2021-05-28T09:01:38Z [INFO] - Starting server...
May 28 09:01:38 sesyiot-auto-24 aziot-identityd[14935]: 2021-05-28T09:01:38Z [INFO] - <-- POST /identities/device?api-version=2020-09-01 {"content-type": "application/json", "host": "2f72756e2f617a696f742f6964656e74697479642e736f636b:0", "content-length": "16"}
May 28 09:01:38 sesyiot-auto-24 systemd[1]: Started Azure IoT Keys Service.
May 28 09:01:38 sesyiot-auto-24 aziot-keyd[14943]: 2021-05-28T09:01:38Z [INFO] - Starting service...
May 28 09:01:38 sesyiot-auto-24 aziot-keyd[14943]: 2021-05-28T09:01:38Z [INFO] - Version - 1.2.0
May 28 09:01:38 sesyiot-auto-24 aziot-keyd[14943]: 2021-05-28T09:01:38Z [INFO] - Loaded libaziot-keys with version 0x02000000
May 28 09:01:38 sesyiot-auto-24 aziot-keyd[14943]: 2021-05-28T09:01:38Z [INFO] - Starting server...
May 28 09:01:38 sesyiot-auto-24 aziot-keyd[14943]: 2021-05-28T09:01:38Z [INFO] - <-- GET /key/device-id?api-version=2020-09-01 {"host": "keyd.sock"}
May 28 09:01:38 sesyiot-auto-24 aziot-keyd[14943]: 2021-05-28T09:01:38Z [INFO] - --> 200 {"content-type": "application/json"}
May 28 09:01:38 sesyiot-auto-24 aziot-identityd[14935]: 2021-05-28T09:01:38Z [INFO] - --> 200 {"content-type": "application/json"}
May 28 09:01:38 sesyiot-auto-24 aziot-edged[14864]: 2021-05-28T09:01:38Z [INFO] - Finished provisioning edge device.
May 28 09:01:38 sesyiot-auto-24 aziot-edged[14864]: 2021-05-28T09:01:38Z [INFO] - Stopping all modules...
May 28 09:01:38 sesyiot-auto-24 aziot-edged[14864]: 2021-05-28T09:01:38Z [INFO] - Finished stopping modules.
May 28 09:01:38 sesyiot-auto-24 aziot-edged[14864]: 2021-05-28T09:01:38Z [INFO] - Detecting if device information has changed...
May 28 09:01:38 sesyiot-auto-24 aziot-edged[14864]: 2021-05-28T09:01:38Z [INFO] - Change to provisioning state detected.
May 28 09:01:38 sesyiot-auto-24 aziot-edged[14864]: 2021-05-28T09:01:38Z [INFO] - Removing all modules...
May 28 09:01:38 sesyiot-auto-24 aziot-edged[14864]: 2021-05-28T09:01:38Z [INFO] - Finished removing modules.
May 28 09:01:38 sesyiot-auto-24 aziot-edged[14864]: 2021-05-28T09:01:38Z [INFO] - Starting management API...
May 28 09:01:38 sesyiot-auto-24 aziot-edged[14864]: 2021-05-28T09:01:38Z [INFO] - Starting workload API...
May 28 09:01:38 sesyiot-auto-24 aziot-edged[14864]: 2021-05-28T09:01:38Z [INFO] - Starting watchdog with 60 second frequency...
May 28 09:01:38 sesyiot-auto-24 aziot-edged[14864]: 2021-05-28T09:01:38Z [INFO] - Listening on unix:///var/lib/iotedge/mgmt.sock with 1 thread for management API.
May 28 09:01:38 sesyiot-auto-24 aziot-edged[14864]: 2021-05-28T09:01:38Z [INFO] - Checking edge runtime status
May 28 09:01:38 sesyiot-auto-24 aziot-edged[14864]: 2021-05-28T09:01:38Z [INFO] - Creating and starting edge runtime module edgeAgent
May 28 09:01:38 sesyiot-auto-24 systemd[1]: Started Azure IoT Certificates Service.
May 28 09:01:38 sesyiot-auto-24 aziot-keyd[14943]: 2021-05-28T09:01:38Z [INFO] - <-- GET /key/device-id?api-version=2020-09-01 {"host": "keyd.sock"}
May 28 09:01:38 sesyiot-auto-24 aziot-keyd[14943]: 2021-05-28T09:01:38Z [INFO] - --> 200 {"content-type": "application/json"}
May 28 09:01:38 sesyiot-auto-24 aziot-keyd[14943]: 2021-05-28T09:01:38Z [INFO] - <-- POST /sign?api-version=2020-09-01 {"content-type": "application/json", "host": "keyd.sock", "content-length": "382"}
May 28 09:01:38 sesyiot-auto-24 aziot-keyd[14943]: 2021-05-28T09:01:38Z [INFO] - --> 200 {"content-type": "application/json"}
May 28 09:01:38 sesyiot-auto-24 aziot-identityd[14935]: 2021-05-28T09:01:38Z [INFO] - <-- PUT /identities/modules/$edgeAgent?api-version=2020-09-01&type=aziot {"content-type": "application/json", "host": "2f72756e2f617a696f742f6964656e74697479642e736f636b:0", "content-length": "40"}
May 28 09:01:38 sesyiot-auto-24 aziot-certd[14949]: 2021-05-28T09:01:38Z [INFO] - Starting service...
May 28 09:01:38 sesyiot-auto-24 aziot-certd[14949]: 2021-05-28T09:01:38Z [INFO] - Version - 1.2.0
May 28 09:01:38 sesyiot-auto-24 aziot-certd[14949]: 2021-05-28T09:01:38Z [INFO] - Starting server...
May 28 09:01:38 sesyiot-auto-24 aziot-edged[14864]: 2021-05-28T09:01:38Z [INFO] - Listening on unix:///var/lib/iotedge/workload.sock with 1 thread for workload API.
May 28 09:01:38 sesyiot-auto-24 aziot-certd[14949]: 2021-05-28T09:01:38Z [INFO] - <-- GET /certificates/aziot-edged-ca?api-version=2020-09-01 {"host": "2f72756e2f617a696f742f63657274642e736f636b:0"}
May 28 09:01:38 sesyiot-auto-24 aziot-certd[14949]: 2021-05-28T09:01:38Z [INFO] - --> 200 {"content-type": "application/json"}
May 28 09:01:39 sesyiot-auto-24 aziot-keyd[14943]: 2021-05-28T09:01:39Z [INFO] - <-- GET /key/aziot_identityd_master_id?api-version=2020-09-01 {"host": "keyd.sock"}
May 28 09:01:39 sesyiot-auto-24 aziot-keyd[14943]: 2021-05-28T09:01:39Z [ERR!] - invalid parameter "id": not found
May 28 09:01:39 sesyiot-auto-24 aziot-keyd[14943]: 2021-05-28T09:01:39Z [INFO] - !!! a parameter has an invalid value
May 28 09:01:39 sesyiot-auto-24 aziot-keyd[14943]: 2021-05-28T09:01:39Z [INFO] - --> 400 {"content-type": "application/json"}
May 28 09:01:39 sesyiot-auto-24 aziot-keyd[14943]: 2021-05-28T09:01:39Z [INFO] - <-- POST /key?api-version=2020-09-01 {"content-type": "application/json", "host": "keyd.sock", "content-length": "75"}
May 28 09:01:39 sesyiot-auto-24 aziot-keyd[14943]: 2021-05-28T09:01:39Z [INFO] - --> 200 {"content-type": "application/json"}
May 28 09:01:39 sesyiot-auto-24 aziot-keyd[14943]: 2021-05-28T09:01:39Z [INFO] - <-- POST /derivedkey?api-version=2020-09-01 {"content-type": "application/json", "host": "keyd.sock", "content-length": "340"}
May 28 09:01:39 sesyiot-auto-24 aziot-keyd[14943]: 2021-05-28T09:01:39Z [INFO] - --> 200 {"content-type": "application/json"}
May 28 09:01:39 sesyiot-auto-24 aziot-keyd[14943]: 2021-05-28T09:01:39Z [INFO] - <-- POST /derivedkey/export?api-version=2020-09-01 {"content-type": "application/json", "host": "keyd.sock", "content-length": "736"}
May 28 09:01:39 sesyiot-auto-24 aziot-keyd[14943]: 2021-05-28T09:01:39Z [INFO] - --> 200 {"content-type": "application/json"}
May 28 09:01:39 sesyiot-auto-24 aziot-keyd[14943]: 2021-05-28T09:01:39Z [INFO] - <-- POST /derivedkey?api-version=2020-09-01 {"content-type": "application/json", "host": "keyd.sock", "content-length": "340"}
May 28 09:01:39 sesyiot-auto-24 aziot-keyd[14943]: 2021-05-28T09:01:39Z [INFO] - --> 200 {"content-type": "application/json"}
May 28 09:01:39 sesyiot-auto-24 aziot-keyd[14943]: 2021-05-28T09:01:39Z [INFO] - <-- POST /derivedkey/export?api-version=2020-09-01 {"content-type": "application/json", "host": "keyd.sock", "content-length": "748"}
May 28 09:01:39 sesyiot-auto-24 aziot-keyd[14943]: 2021-05-28T09:01:39Z [INFO] - --> 200 {"content-type": "application/json"}
May 28 09:01:39 sesyiot-auto-24 aziot-keyd[14943]: 2021-05-28T09:01:39Z [INFO] - <-- GET /key/device-id?api-version=2020-09-01 {"host": "keyd.sock"}
May 28 09:01:39 sesyiot-auto-24 aziot-keyd[14943]: 2021-05-28T09:01:39Z [INFO] - --> 200 {"content-type": "application/json"}
May 28 09:01:39 sesyiot-auto-24 aziot-keyd[14943]: 2021-05-28T09:01:39Z [INFO] - <-- POST /sign?api-version=2020-09-01 {"content-type": "application/json", "host": "keyd.sock", "content-length": "382"}
May 28 09:01:39 sesyiot-auto-24 aziot-keyd[14943]: 2021-05-28T09:01:39Z [INFO] - --> 200 {"content-type": "application/json"}
May 28 09:01:39 sesyiot-auto-24 aziot-identityd[14935]: 2021-05-28T09:01:39Z [INFO] - --> 200 {"content-type": "application/json"}
May 28 09:01:39 sesyiot-auto-24 aziot-edged[14864]: 2021-05-28T09:01:39Z [INFO] - Pulling image via tag ta01iotcrt01.azurecr.io/azureiotedge-agent:1.2.0...

edge-agent logs


<Paste here between the triple backticks>

edge-hub logs


<Paste here between the triple backticks>

Additional Information

Please provide any additional information that may be helpful in understanding the issue.

About this issue

Original URL
State: closed
Created 3 years ago
Reactions: 1
Comments: 29 (19 by maintainers)

Most upvoted comments

Hi, I have not heard anything from them yet. Please don’t close this issue until the problem is resolved and the system works.

sejonssonr on Jun 15, 2021

So the steps I am following:

systemctl stop aziot-edged

remove docker containers docker ps -a copy paste all cids for docker rm -f <cid>

From hyper-v manager stop and start VM of centos7

restart windows machine hosting centos vm

checking logs for delay in pull

This is without a proxy server right now, and there is no large delay…only a 1 sec delay from logs. Want to confirm these steps are matching your exact process. If so, I will try with setting up a proxy server and see if that allows me to replicate the scenario.

I noticed now that there is at least one error in my description above and one thing missing in your “Step 2”. These are the steps necessary for reproducing the error:

systemctl stop aziot-edged
Stop containers and remove containers AND images docker rm -f $(docker ps -a -q) && docker rmi -f $(docker images -a -q) This emulates the state after a fresh install.
Run iotedge system restart to restart the service.
Restart windows machine hosting centos vm while the docker pull is running.
After restart of the Windows host and resume of the Centos VM, note how the docker pull is hanging without any tcp packets being sent (´tcpdump´).

Follow-up question, what are the steps for setting up the proxy that you are using?

The proxy is a “squid proxy” and can be setup by running a Centos in another VM on the same host and configuring the Iot Edge to use this as a proxy for the https traffic. We believe that you could replicate the problem using a vanilla squid proxy without further traffic restrictions. If needed we can provide more details.

sejonssonr on Jun 4, 2021

support_bundle.zip

We are able to reproduce the issue if we do the following on a working machine:

Run “systemctl stop aziot-edged”
Delete all containers and docker images in vm
Restart Iot Edge vm
Reboot Windows host

Side note that could be worth mentioning: A proxy server that is hosted on the same Windows host is used for the http traffic. The vm:s are running in Hyper-V. Both the Proxy vm and the IotEdge vm enter saved state at the host restart.

Support bundle is attached. Note the 17 min delay between the docker pull and the next log line in aziot-edged.txt logs:

Jun 03 13:25:13 sesyiot-auto-24 aziot-edged[14496]: 2021-06-03T13:25:13Z [INFO] - Pulling image via tag ta01iotcrt01.azurecr.io/azureiotedge-agent:1.2.0...
Jun 03 13:42:01 sesyiot-auto-24 aziot-edged[14496]: 2021-06-03T13:42:01Z [INFO] - [mgmt] - - - [2021-06-03 13:42:01.820640100 UTC] "GET /modules/?api-version=2018-06-28 HTTP/1.1" 200 OK 14 "-" "-" auth_id(-)

sejonssonr on Jun 3, 2021