iotedge: Edge runtime startup failure with long registration id

Expected Behavior

The IoT Edge runtime should startup as expected.

Current Behavior

The edgeAgent stays in the stopped state, so the rest of the bootstrap does not continue.

Steps to Reproduce

Provide a detailed set of steps to reproduce the bug.

Create an Azure VM using

publisher = "canonical"
offer     = "0001-com-ubuntu-server-focal"
sku       = "20_04-lts"
version   = "latest"

Follow the instructions for symmetric key attestation via DPS

Run iotedge list

NAME             STATUS           DESCRIPTION      CONFIG
edgeAgent        stopped          Stopped          mcr.microsoft.com/azureiotedge-agent:1.2

Context (Environment)

Output of `iotedge check`

Click here


Configuration checks (aziot-identity-service)
---------------------------------------------
√ keyd configuration is well-formed - OK
√ certd configuration is well-formed - OK
√ tpmd configuration is well-formed - OK
√ identityd configuration is well-formed - OK
√ daemon configurations up-to-date with config.toml - OK
√ identityd config toml file specifies a valid hostname - OK
× aziot-identity-service package is up-to-date - Error
    could not query https://aka.ms/latest-aziot-identity-service for latest available version
‼ host time is close to reference time - Warning
    Could not query NTP server
√ preloaded certificates are valid - OK
√ keyd is running - OK
√ certd is running - OK
√ identityd is running - OK
× read all preloaded certificates from the Certificates Service - Error
    could not load cert with ID "aziot-edged-trust-bundle"
    
    Caused by:
        parameter "id" has an invalid value
        caused by: not found
√ read all preloaded key pairs from the Keys Service - OK
√ ensure all preloaded certificates match preloaded private keys with the same ID - OK

Connectivity checks (aziot-identity-service)
--------------------------------------------
‼ host can connect to and perform TLS handshake with iothub AMQP port - Warning
    Could not retrieve iothub_hostname from provisioning file.
    Please specify the backing IoT Hub name using --iothub-hostname switch if you have that information.
    Since no hostname is provided, all hub connectivity tests will be skipped.
‼ host can connect to and perform TLS handshake with iothub HTTPS / WebSockets port - Warning
    Could not retrieve iothub_hostname from provisioning file.
    Please specify the backing IoT Hub name using --iothub-hostname switch if you have that information.
    Since no hostname is provided, all hub connectivity tests will be skipped.
‼ host can connect to and perform TLS handshake with iothub MQTT port - Warning
    Could not retrieve iothub_hostname from provisioning file.
    Please specify the backing IoT Hub name using --iothub-hostname switch if you have that information.
    Since no hostname is provided, all hub connectivity tests will be skipped.
√ host can connect to and perform TLS handshake with DPS endpoint - OK

Configuration checks
--------------------
√ aziot-edged configuration is well-formed - OK
√ configuration up-to-date with config.toml - OK
√ container engine is installed and functional - OK
√ configuration has correct URIs for daemon mgmt endpoint - OK
‼ aziot-edge package is up-to-date - Warning
    Error while fetching latest versions of edge components: could not send HTTP request
√ container time is close to host time - OK
‼ DNS server - Warning
    Container engine is not configured with DNS server setting, which may impact connectivity to IoT Hub.
    Please see https://aka.ms/iotedge-prod-checklist-dns for best practices.
    You can ignore this warning if you are setting DNS server per module in the Edge deployment.
‼ production readiness: logs policy - Warning
    Container engine is not configured to rotate module logs which may cause it run out of disk space.
    Please see https://aka.ms/iotedge-prod-checklist-logs for best practices.
    You can ignore this warning if you are setting log policy per module in the Edge deployment.
‼ production readiness: Edge Agent's storage directory is persisted on the host filesystem - Warning
    The edgeAgent module is not configured to persist its /tmp/edgeAgent directory on the host filesystem.
    Data might be lost if the module is deleted or updated.
    Please see https://aka.ms/iotedge-storage-host for best practices.
× production readiness: Edge Hub's storage directory is persisted on the host filesystem - Error
    Could not check current state of edgeHub container
√ proxy settings are consistent in aziot-edged, aziot-identityd, moby daemon and config.toml - OK

Connectivity checks
-------------------
19 check(s) succeeded.
8 check(s) raised warnings. Re-run with --verbose for more details.
3 check(s) raised errors. Re-run with --verbose for more details.
7 check(s) were skipped due to errors from other checks. Re-run with --verbose for more details.

Device Information

Host OS [e.g. Ubuntu 18.04, Windows Server IoT 2019]: Ubuntu 20.04
Architecture [e.g. amd64, arm32, arm64]: amd64
Container OS [e.g. Linux containers, Windows containers]: linux

Runtime Versions

aziot-edged [run iotedge version]: 1.2.9
Edge Agent [image tag (e.g. 1.0.0)]: 1.2
Edge Hub [image tag (e.g. 1.0.0)]: N/A
Docker/Moby [run docker version]: 20.10.14+azure-1

Note: when using Windows containers on Windows, run docker -H npipe:////./pipe/iotedge_moby_engine version instead

Logs

aziot-edged logs


-- Logs begin at Thu 2022-04-14 18:27:29 UTC, end at Thu 2022-04-14 19:00:17 UTC. --
Apr 14 18:34:12 tatooine systemd[1]: Started Azure IoT Edge daemon.
Apr 14 18:34:12 tatooine aziot-edged[6236]: 2022-04-14T18:34:12Z [INFO] - Starting Azure IoT Edge Module Runtime
Apr 14 18:34:12 tatooine aziot-edged[6236]: 2022-04-14T18:34:12Z [INFO] - Version - 1.2.9
Apr 14 18:34:12 tatooine aziot-edged[6236]: 2022-04-14T18:34:12Z [INFO] - Initializing the module runtime...
Apr 14 18:34:12 tatooine aziot-edged[6236]: 2022-04-14T18:34:12Z [INFO] - Initializing module runtime...
Apr 14 18:34:12 tatooine aziot-edged[6236]: 2022-04-14T18:34:12Z [INFO] - Using runtime network id azure-iot-edge
Apr 14 18:34:12 tatooine aziot-edged[6236]: 2022-04-14T18:34:12Z [INFO] - Successfully initialized module runtime
Apr 14 18:34:12 tatooine aziot-edged[6236]: 2022-04-14T18:34:12Z [INFO] - Finished initializing the module runtime.
Apr 14 18:34:12 tatooine aziot-edged[6236]: 2022-04-14T18:34:12Z [INFO] - Obtaining edge device provisioning data...
Apr 14 18:34:23 tatooine aziot-edged[6236]: 2022-04-14T18:34:23Z [INFO] - Finished provisioning edge device.
Apr 14 18:34:23 tatooine aziot-edged[6236]: 2022-04-14T18:34:23Z [INFO] - Stopping all modules...
Apr 14 18:34:23 tatooine aziot-edged[6236]: 2022-04-14T18:34:23Z [INFO] - Finished stopping modules.
Apr 14 18:34:23 tatooine aziot-edged[6236]: 2022-04-14T18:34:23Z [INFO] - Detecting if device information has changed...
Apr 14 18:34:23 tatooine aziot-edged[6236]: 2022-04-14T18:34:23Z [INFO] - Change to provisioning state detected.
Apr 14 18:34:23 tatooine aziot-edged[6236]: 2022-04-14T18:34:23Z [INFO] - Removing all modules...
Apr 14 18:34:23 tatooine aziot-edged[6236]: 2022-04-14T18:34:23Z [INFO] - Finished removing modules.
Apr 14 18:34:23 tatooine aziot-edged[6236]: 2022-04-14T18:34:23Z [INFO] - Starting management API...
Apr 14 18:34:23 tatooine aziot-edged[6236]: 2022-04-14T18:34:23Z [INFO] - Starting watchdog with 60 second frequency...
Apr 14 18:34:23 tatooine aziot-edged[6236]: 2022-04-14T18:34:23Z [INFO] - Listening on fd://aziot-edged.mgmt.socket with 1 thread for management API.
Apr 14 18:34:23 tatooine aziot-edged[6236]: 2022-04-14T18:34:23Z [INFO] - [mgmt] - - - [2022-04-14 18:34:23.477698125 UTC] "GET /modules?api-version=2021-12-07 HTTP/1.1" 200 OK 14 "-" "iotedge/0.1.0" auth_id(-)
Apr 14 18:34:23 tatooine aziot-edged[6236]: 2022-04-14T18:34:23Z [INFO] - Checking edge runtime status
Apr 14 18:34:23 tatooine aziot-edged[6236]: 2022-04-14T18:34:23Z [INFO] - Creating and starting edge runtime module edgeAgent
Apr 14 18:34:23 tatooine aziot-edged[6236]: 2022-04-14T18:34:23Z [INFO] - Pulling image via tag mcr.microsoft.com/azureiotedge-agent:1.2...
Apr 14 18:34:26 tatooine aziot-edged[6236]: 2022-04-14T18:34:26Z [INFO] - Successfully pulled image mcr.microsoft.com/azureiotedge-agent:1.2
Apr 14 18:34:26 tatooine aziot-edged[6236]: 2022-04-14T18:34:26Z [INFO] - Creating module edgeAgent...
Apr 14 18:34:26 tatooine aziot-edged[6236]: 2022-04-14T18:34:26Z [INFO] - Creating image via tag mcr.microsoft.com/azureiotedge-agent:1.2...
Apr 14 18:34:26 tatooine aziot-edged[6236]: 2022-04-14T18:34:26Z [INFO] - Successfully created module edgeAgent
Apr 14 18:34:26 tatooine aziot-edged[6236]: 2022-04-14T18:34:26Z [INFO] - Starting module edgeAgent...
Apr 14 18:34:26 tatooine aziot-edged[6236]: 2022-04-14T18:34:26Z [INFO] - Starting new listener for module edgeAgent
Apr 14 18:34:26 tatooine aziot-edged[6236]: 2022-04-14T18:34:26Z [WARN] - Error in watchdog when checking for edge runtime status:
Apr 14 18:34:26 tatooine aziot-edged[6236]: 2022-04-14T18:34:26Z [WARN] - A module runtime error occurred.
Apr 14 18:34:26 tatooine aziot-edged[6236]: 2022-04-14T18:34:26Z [WARN] - 	caused by: Could not get module edgeAgent
Apr 14 18:35:23 tatooine aziot-edged[6236]: 2022-04-14T18:35:23Z [INFO] - Checking edge runtime status
Apr 14 18:35:23 tatooine aziot-edged[6236]: 2022-04-14T18:35:23Z [INFO] - Edge runtime status is stopped, starting module now...
Apr 14 18:35:23 tatooine aziot-edged[6236]: 2022-04-14T18:35:23Z [INFO] - Starting module edgeAgent...
Apr 14 18:35:23 tatooine aziot-edged[6236]: 2022-04-14T18:35:23Z [INFO] - Starting new listener for module edgeAgent
Apr 14 18:35:23 tatooine aziot-edged[6236]: 2022-04-14T18:35:23Z [INFO] - Listener  edgeAgent already started, removing old listener
Apr 14 18:35:23 tatooine aziot-edged[6236]: 2022-04-14T18:35:23Z [WARN] - Error in watchdog when checking for edge runtime status:
Apr 14 18:35:23 tatooine aziot-edged[6236]: 2022-04-14T18:35:23Z [WARN] - A module runtime error occurred.
Apr 14 18:35:23 tatooine aziot-edged[6236]: 2022-04-14T18:35:23Z [WARN] - 	caused by: Could not get module edgeAgent
Apr 14 18:35:23 tatooine aziot-edged[6236]: 2022-04-14T18:35:23Z [WARN] - The workload service encountered an error
Apr 14 18:36:01 tatooine aziot-edged[6236]: 2022-04-14T18:36:01Z [INFO] - Received SIGTERM, starting shutdown
Apr 14 18:36:01 tatooine aziot-edged[6236]: 2022-04-14T18:36:01Z [INFO] - Stopping edge runtime module edgeAgent
Apr 14 18:36:01 tatooine aziot-edged[6236]: 2022-04-14T18:36:01Z [INFO] - Stopping module edgeAgent...
Apr 14 18:36:01 tatooine systemd[1]: Stopping Azure IoT Edge daemon...
Apr 14 18:36:01 tatooine aziot-edged[6236]: 2022-04-14T18:36:01Z [WARN] - Could not stop module edgeAgent
Apr 14 18:36:01 tatooine aziot-edged[6236]: 2022-04-14T18:36:01Z [WARN] - 	caused by: Target of operation already in this state
Apr 14 18:36:01 tatooine aziot-edged[6236]: 2022-04-14T18:36:01Z [ERR!] - Edge runtime will stop because the shutdown signal raised an error.
Apr 14 18:36:01 tatooine aziot-edged[6236]: 2022-04-14T18:36:01Z [ERR!] - A module runtime error occurred.
Apr 14 18:36:01 tatooine aziot-edged[6236]: 2022-04-14T18:36:01Z [ERR!] - 	caused by: Could not stop module edgeAgent
Apr 14 18:36:01 tatooine aziot-edged[6236]: 2022-04-14T18:36:01Z [ERR!] - 	caused by: Target of operation already in this state
Apr 14 18:36:01 tatooine systemd[1]: aziot-edged.service: Main process exited, code=exited, status=1/FAILURE
Apr 14 18:36:01 tatooine systemd[1]: aziot-edged.service: Failed with result 'exit-code'.
Apr 14 18:36:01 tatooine systemd[1]: Stopped Azure IoT Edge daemon.
Apr 14 18:36:01 tatooine systemd[1]: Started Azure IoT Edge daemon.
Apr 14 18:36:01 tatooine aziot-edged[6868]: 2022-04-14T18:36:01Z [INFO] - Starting Azure IoT Edge Module Runtime
Apr 14 18:36:01 tatooine aziot-edged[6868]: 2022-04-14T18:36:01Z [INFO] - Version - 1.2.9
Apr 14 18:36:01 tatooine aziot-edged[6868]: 2022-04-14T18:36:01Z [INFO] - Initializing the module runtime...
Apr 14 18:36:01 tatooine aziot-edged[6868]: 2022-04-14T18:36:01Z [INFO] - Initializing module runtime...
Apr 14 18:36:01 tatooine aziot-edged[6868]: 2022-04-14T18:36:01Z [INFO] - Using runtime network id azure-iot-edge
Apr 14 18:36:01 tatooine aziot-edged[6868]: 2022-04-14T18:36:01Z [INFO] - Successfully initialized module runtime
Apr 14 18:36:01 tatooine aziot-edged[6868]: 2022-04-14T18:36:01Z [INFO] - Finished initializing the module runtime.
Apr 14 18:36:01 tatooine aziot-edged[6868]: 2022-04-14T18:36:01Z [INFO] - Obtaining edge device provisioning data...
Apr 14 18:36:01 tatooine aziot-edged[6868]: 2022-04-14T18:36:01Z [INFO] - Finished provisioning edge device.
Apr 14 18:36:01 tatooine aziot-edged[6868]: 2022-04-14T18:36:01Z [INFO] - Stopping all modules...
Apr 14 18:36:01 tatooine aziot-edged[6868]: 2022-04-14T18:36:01Z [INFO] - Stopping module edgeAgent...
Apr 14 18:36:01 tatooine aziot-edged[6868]: 2022-04-14T18:36:01Z [WARN] - Could not stop module edgeAgent
Apr 14 18:36:01 tatooine aziot-edged[6868]: 2022-04-14T18:36:01Z [WARN] - 	caused by: Target of operation already in this state
Apr 14 18:36:01 tatooine aziot-edged[6868]: 2022-04-14T18:36:01Z [INFO] - Finished stopping modules.
Apr 14 18:36:01 tatooine aziot-edged[6868]: 2022-04-14T18:36:01Z [INFO] - Detecting if device information has changed...
Apr 14 18:36:01 tatooine aziot-edged[6868]: 2022-04-14T18:36:01Z [INFO] - Starting management API...
Apr 14 18:36:01 tatooine aziot-edged[6868]: 2022-04-14T18:36:01Z [INFO] - Starting new listener for module edgeAgent
Apr 14 18:36:01 tatooine aziot-edged[6868]: 2022-04-14T18:36:01Z [INFO] - Starting watchdog with 60 second frequency...
Apr 14 18:36:01 tatooine aziot-edged[6868]: 2022-04-14T18:36:01Z [INFO] - Listening on fd://aziot-edged.mgmt.socket with 1 thread for management API.
Apr 14 18:36:01 tatooine aziot-edged[6868]: 2022-04-14T18:36:01Z [INFO] - Checking edge runtime status
Apr 14 18:36:01 tatooine aziot-edged[6868]: 2022-04-14T18:36:01Z [INFO] - Edge runtime status is stopped, starting module now...
Apr 14 18:36:01 tatooine aziot-edged[6868]: 2022-04-14T18:36:01Z [INFO] - Starting module edgeAgent...
Apr 14 18:36:01 tatooine aziot-edged[6868]: 2022-04-14T18:36:01Z [INFO] - Starting new listener for module edgeAgent
Apr 14 18:36:01 tatooine aziot-edged[6868]: 2022-04-14T18:36:01Z [INFO] - Listener  edgeAgent already started, removing old listener
Apr 14 18:36:01 tatooine aziot-edged[6868]: 2022-04-14T18:36:01Z [WARN] - Error in watchdog when checking for edge runtime status:
Apr 14 18:36:01 tatooine aziot-edged[6868]: 2022-04-14T18:36:01Z [WARN] - A module runtime error occurred.
Apr 14 18:36:01 tatooine aziot-edged[6868]: 2022-04-14T18:36:01Z [WARN] - 	caused by: Could not get module edgeAgent
Apr 14 18:36:34 tatooine aziot-edged[6868]: 2022-04-14T18:36:34Z [INFO] - [mgmt] - - - [2022-04-14 18:36:34.190372699 UTC] "GET /modules?api-version=2021-12-07 HTTP/1.1" 200 OK 561 "-" "iotedge/0.1.0" auth_id(-)
Apr 14 18:37:01 tatooine aziot-edged[6868]: 2022-04-14T18:37:01Z [INFO] - Checking edge runtime status
Apr 14 18:37:01 tatooine aziot-edged[6868]: 2022-04-14T18:37:01Z [INFO] - Edge runtime status is stopped, starting module now...
Apr 14 18:37:01 tatooine aziot-edged[6868]: 2022-04-14T18:37:01Z [INFO] - Starting module edgeAgent...
Apr 14 18:37:01 tatooine aziot-edged[6868]: 2022-04-14T18:37:01Z [INFO] - Starting new listener for module edgeAgent
Apr 14 18:37:01 tatooine aziot-edged[6868]: 2022-04-14T18:37:01Z [INFO] - Listener  edgeAgent already started, removing old listener
Apr 14 18:37:01 tatooine aziot-edged[6868]: 2022-04-14T18:37:01Z [WARN] - The workload service encountered an error
Apr 14 18:37:01 tatooine aziot-edged[6868]: 2022-04-14T18:37:01Z [WARN] - Error in watchdog when checking for edge runtime status:
Apr 14 18:37:01 tatooine aziot-edged[6868]: 2022-04-14T18:37:01Z [WARN] - A module runtime error occurred.
Apr 14 18:37:01 tatooine aziot-edged[6868]: 2022-04-14T18:37:01Z [WARN] - 	caused by: Could not get module edgeAgent
Apr 14 18:38:01 tatooine aziot-edged[6868]: 2022-04-14T18:38:01Z [INFO] - Checking edge runtime status
Apr 14 18:38:01 tatooine aziot-edged[6868]: 2022-04-14T18:38:01Z [INFO] - Edge runtime status is stopped, starting module now...
Apr 14 18:38:01 tatooine aziot-edged[6868]: 2022-04-14T18:38:01Z [INFO] - Starting module edgeAgent...
Apr 14 18:38:01 tatooine aziot-edged[6868]: 2022-04-14T18:38:01Z [INFO] - Starting new listener for module edgeAgent
Apr 14 18:38:01 tatooine aziot-edged[6868]: 2022-04-14T18:38:01Z [WARN] - Error in watchdog when checking for edge runtime status:
Apr 14 18:38:01 tatooine aziot-edged[6868]: 2022-04-14T18:38:01Z [WARN] - A module runtime error occurred.
Apr 14 18:38:01 tatooine aziot-edged[6868]: 2022-04-14T18:38:01Z [WARN] - 	caused by: Could not get module edgeAgent
Apr 14 18:39:01 tatooine aziot-edged[6868]: 2022-04-14T18:39:01Z [INFO] - Checking edge runtime status
Apr 14 18:39:01 tatooine aziot-edged[6868]: 2022-04-14T18:39:01Z [INFO] - Edge runtime status is stopped, starting module now...
Apr 14 18:39:01 tatooine aziot-edged[6868]: 2022-04-14T18:39:01Z [INFO] - Starting module edgeAgent...
Apr 14 18:39:01 tatooine aziot-edged[6868]: 2022-04-14T18:39:01Z [INFO] - Starting new listener for module edgeAgent
Apr 14 18:39:01 tatooine aziot-edged[6868]: 2022-04-14T18:39:01Z [INFO] - Listener  edgeAgent already started, removing old listener
Apr 14 18:39:01 tatooine aziot-edged[6868]: 2022-04-14T18:39:01Z [WARN] - The workload service encountered an error
Apr 14 18:39:01 tatooine aziot-edged[6868]: 2022-04-14T18:39:01Z [WARN] - Error in watchdog when checking for edge runtime status:
Apr 14 18:39:01 tatooine aziot-edged[6868]: 2022-04-14T18:39:01Z [WARN] - A module runtime error occurred.
Apr 14 18:39:01 tatooine aziot-edged[6868]: 2022-04-14T18:39:01Z [WARN] - 	caused by: Could not get module edgeAgent
Apr 14 18:40:01 tatooine aziot-edged[6868]: 2022-04-14T18:40:01Z [INFO] - Checking edge runtime status
Apr 14 18:40:01 tatooine aziot-edged[6868]: 2022-04-14T18:40:01Z [INFO] - Edge runtime status is stopped, starting module now...
Apr 14 18:40:01 tatooine aziot-edged[6868]: 2022-04-14T18:40:01Z [INFO] - Starting module edgeAgent...
Apr 14 18:40:01 tatooine aziot-edged[6868]: 2022-04-14T18:40:01Z [INFO] - Starting new listener for module edgeAgent
Apr 14 18:40:01 tatooine aziot-edged[6868]: 2022-04-14T18:40:01Z [WARN] - Error in watchdog when checking for edge runtime status:
Apr 14 18:40:01 tatooine aziot-edged[6868]: 2022-04-14T18:40:01Z [WARN] - A module runtime error occurred.
Apr 14 18:40:01 tatooine aziot-edged[6868]: 2022-04-14T18:40:01Z [WARN] - 	caused by: Could not get module edgeAgent
Apr 14 18:40:50 tatooine aziot-edged[6868]: 2022-04-14T18:40:50Z [INFO] - Restarting module edgeAgent...
Apr 14 18:40:51 tatooine aziot-edged[6868]: 2022-04-14T18:40:51Z [INFO] - Successfully restarted module edgeAgent
Apr 14 18:40:51 tatooine aziot-edged[6868]: 2022-04-14T18:40:51Z [INFO] - [mgmt] - - - [2022-04-14 18:40:51.130956604 UTC] "POST /modules/edgeAgent/restart?api-version=2021-12-07 HTTP/1.1" 204 No Content - "-" "iotedge/0.1.0" auth_id(-)
Apr 14 18:41:01 tatooine aziot-edged[6868]: 2022-04-14T18:41:01Z [INFO] - Checking edge runtime status
Apr 14 18:41:01 tatooine aziot-edged[6868]: 2022-04-14T18:41:01Z [INFO] - Edge runtime status is failed, starting module now...
Apr 14 18:41:01 tatooine aziot-edged[6868]: 2022-04-14T18:41:01Z [INFO] - Starting module edgeAgent...
Apr 14 18:41:01 tatooine aziot-edged[6868]: 2022-04-14T18:41:01Z [INFO] - Starting new listener for module edgeAgent
Apr 14 18:41:01 tatooine aziot-edged[6868]: 2022-04-14T18:41:01Z [INFO] - Listener  edgeAgent already started, removing old listener
Apr 14 18:41:01 tatooine aziot-edged[6868]: 2022-04-14T18:41:01Z [WARN] - The workload service encountered an error
Apr 14 18:41:01 tatooine aziot-edged[6868]: 2022-04-14T18:41:01Z [WARN] - Error in watchdog when checking for edge runtime status:
Apr 14 18:41:01 tatooine aziot-edged[6868]: 2022-04-14T18:41:01Z [WARN] - A module runtime error occurred.
Apr 14 18:41:01 tatooine aziot-edged[6868]: 2022-04-14T18:41:01Z [WARN] - 	caused by: Could not get module edgeAgent
Apr 14 18:42:01 tatooine aziot-edged[6868]: 2022-04-14T18:42:01Z [INFO] - Checking edge runtime status
Apr 14 18:42:01 tatooine aziot-edged[6868]: 2022-04-14T18:42:01Z [INFO] - Edge runtime status is failed, starting module now...
Apr 14 18:42:01 tatooine aziot-edged[6868]: 2022-04-14T18:42:01Z [INFO] - Starting module edgeAgent...
Apr 14 18:42:01 tatooine aziot-edged[6868]: 2022-04-14T18:42:01Z [INFO] - Starting new listener for module edgeAgent
Apr 14 18:42:01 tatooine aziot-edged[6868]: 2022-04-14T18:42:01Z [WARN] - Error in watchdog when checking for edge runtime status:
Apr 14 18:42:01 tatooine aziot-edged[6868]: 2022-04-14T18:42:01Z [WARN] - A module runtime error occurred.
Apr 14 18:42:01 tatooine aziot-edged[6868]: 2022-04-14T18:42:01Z [WARN] - 	caused by: Could not get module edgeAgent
Apr 14 18:43:01 tatooine aziot-edged[6868]: 2022-04-14T18:43:01Z [INFO] - Checking edge runtime status
Apr 14 18:43:01 tatooine aziot-edged[6868]: 2022-04-14T18:43:01Z [INFO] - Edge runtime status is failed, starting module now...
Apr 14 18:43:01 tatooine aziot-edged[6868]: 2022-04-14T18:43:01Z [INFO] - Starting module edgeAgent...
Apr 14 18:43:01 tatooine aziot-edged[6868]: 2022-04-14T18:43:01Z [INFO] - Starting new listener for module edgeAgent
Apr 14 18:43:01 tatooine aziot-edged[6868]: 2022-04-14T18:43:01Z [INFO] - Listener  edgeAgent already started, removing old listener
Apr 14 18:43:01 tatooine aziot-edged[6868]: 2022-04-14T18:43:01Z [WARN] - The workload service encountered an error
Apr 14 18:43:01 tatooine aziot-edged[6868]: 2022-04-14T18:43:01Z [WARN] - Error in watchdog when checking for edge runtime status:
Apr 14 18:43:01 tatooine aziot-edged[6868]: 2022-04-14T18:43:01Z [WARN] - A module runtime error occurred.
Apr 14 18:43:01 tatooine aziot-edged[6868]: 2022-04-14T18:43:01Z [WARN] - 	caused by: Could not get module edgeAgent
Apr 14 18:44:01 tatooine aziot-edged[6868]: 2022-04-14T18:44:01Z [INFO] - Checking edge runtime status
Apr 14 18:44:01 tatooine aziot-edged[6868]: 2022-04-14T18:44:01Z [INFO] - Edge runtime status is failed, starting module now...
Apr 14 18:44:01 tatooine aziot-edged[6868]: 2022-04-14T18:44:01Z [INFO] - Starting module edgeAgent...
Apr 14 18:44:01 tatooine aziot-edged[6868]: 2022-04-14T18:44:01Z [INFO] - Starting new listener for module edgeAgent
Apr 14 18:44:01 tatooine aziot-edged[6868]: 2022-04-14T18:44:01Z [WARN] - Error in watchdog when checking for edge runtime status:
Apr 14 18:44:01 tatooine aziot-edged[6868]: 2022-04-14T18:44:01Z [WARN] - A module runtime error occurred.
Apr 14 18:44:01 tatooine aziot-edged[6868]: 2022-04-14T18:44:01Z [WARN] - 	caused by: Could not get module edgeAgent
Apr 14 18:45:01 tatooine aziot-edged[6868]: 2022-04-14T18:45:01Z [INFO] - Checking edge runtime status
Apr 14 18:45:01 tatooine aziot-edged[6868]: 2022-04-14T18:45:01Z [INFO] - Edge runtime status is failed, starting module now...
Apr 14 18:45:01 tatooine aziot-edged[6868]: 2022-04-14T18:45:01Z [INFO] - Starting module edgeAgent...
Apr 14 18:45:01 tatooine aziot-edged[6868]: 2022-04-14T18:45:01Z [INFO] - Starting new listener for module edgeAgent
Apr 14 18:45:01 tatooine aziot-edged[6868]: 2022-04-14T18:45:01Z [INFO] - Listener  edgeAgent already started, removing old listener
Apr 14 18:45:01 tatooine aziot-edged[6868]: 2022-04-14T18:45:01Z [WARN] - The workload service encountered an error
Apr 14 18:45:01 tatooine aziot-edged[6868]: 2022-04-14T18:45:01Z [WARN] - Error in watchdog when checking for edge runtime status:
Apr 14 18:45:01 tatooine aziot-edged[6868]: 2022-04-14T18:45:01Z [WARN] - A module runtime error occurred.
Apr 14 18:45:01 tatooine aziot-edged[6868]: 2022-04-14T18:45:01Z [WARN] - 	caused by: Could not get module edgeAgent
Apr 14 18:46:01 tatooine aziot-edged[6868]: 2022-04-14T18:46:01Z [INFO] - Checking edge runtime status
Apr 14 18:46:01 tatooine aziot-edged[6868]: 2022-04-14T18:46:01Z [INFO] - Edge runtime status is failed, starting module now...
Apr 14 18:46:01 tatooine aziot-edged[6868]: 2022-04-14T18:46:01Z [INFO] - Starting module edgeAgent...
Apr 14 18:46:01 tatooine aziot-edged[6868]: 2022-04-14T18:46:01Z [INFO] - Starting new listener for module edgeAgent
Apr 14 18:46:01 tatooine aziot-edged[6868]: 2022-04-14T18:46:01Z [WARN] - Error in watchdog when checking for edge runtime status:
Apr 14 18:46:01 tatooine aziot-edged[6868]: 2022-04-14T18:46:01Z [WARN] - A module runtime error occurred.
Apr 14 18:46:01 tatooine aziot-edged[6868]: 2022-04-14T18:46:01Z [WARN] - 	caused by: Could not get module edgeAgent
Apr 14 18:47:01 tatooine aziot-edged[6868]: 2022-04-14T18:47:01Z [INFO] - Checking edge runtime status
Apr 14 18:47:01 tatooine aziot-edged[6868]: 2022-04-14T18:47:01Z [INFO] - Edge runtime status is failed, starting module now...
Apr 14 18:47:01 tatooine aziot-edged[6868]: 2022-04-14T18:47:01Z [INFO] - Starting module edgeAgent...
Apr 14 18:47:01 tatooine aziot-edged[6868]: 2022-04-14T18:47:01Z [INFO] - Starting new listener for module edgeAgent
Apr 14 18:47:01 tatooine aziot-edged[6868]: 2022-04-14T18:47:01Z [INFO] - Listener  edgeAgent already started, removing old listener
Apr 14 18:47:01 tatooine aziot-edged[6868]: 2022-04-14T18:47:01Z [WARN] - Error in watchdog when checking for edge runtime status:
Apr 14 18:47:01 tatooine aziot-edged[6868]: 2022-04-14T18:47:01Z [WARN] - A module runtime error occurred.
Apr 14 18:47:01 tatooine aziot-edged[6868]: 2022-04-14T18:47:01Z [WARN] - 	caused by: Could not get module edgeAgent
Apr 14 18:47:01 tatooine aziot-edged[6868]: 2022-04-14T18:47:01Z [WARN] - The workload service encountered an error
Apr 14 18:47:57 tatooine aziot-edged[6868]: 2022-04-14T18:47:57Z [INFO] - Getting logs for module edgeAgent...
Apr 14 18:47:57 tatooine aziot-edged[6868]: 2022-04-14T18:47:57Z [INFO] - Successfully got logs for module edgeAgent
Apr 14 18:47:57 tatooine aziot-edged[6868]: 2022-04-14T18:47:57Z [INFO] - [mgmt] - - - [2022-04-14 18:47:57.487283663 UTC] "GET /modules/edgeAgent/logs?api-version=2021-12-07&follow=false&tail=all&timestamps=false&since=1649875677 HTTP/1.1" 200 OK - "-" "iotedge/0.1.0" auth_id(-)
Apr 14 18:48:01 tatooine aziot-edged[6868]: 2022-04-14T18:48:01Z [INFO] - Checking edge runtime status
Apr 14 18:48:01 tatooine aziot-edged[6868]: 2022-04-14T18:48:01Z [INFO] - Edge runtime status is failed, starting module now...
Apr 14 18:48:01 tatooine aziot-edged[6868]: 2022-04-14T18:48:01Z [INFO] - Starting module edgeAgent...
Apr 14 18:48:01 tatooine aziot-edged[6868]: 2022-04-14T18:48:01Z [INFO] - Starting new listener for module edgeAgent
Apr 14 18:48:01 tatooine aziot-edged[6868]: 2022-04-14T18:48:01Z [WARN] - Error in watchdog when checking for edge runtime status:
Apr 14 18:48:01 tatooine aziot-edged[6868]: 2022-04-14T18:48:01Z [WARN] - A module runtime error occurred.
Apr 14 18:48:01 tatooine aziot-edged[6868]: 2022-04-14T18:48:01Z [WARN] - 	caused by: Could not get module edgeAgent
Apr 14 18:49:01 tatooine aziot-edged[6868]: 2022-04-14T18:49:01Z [INFO] - Checking edge runtime status
Apr 14 18:49:01 tatooine aziot-edged[6868]: 2022-04-14T18:49:01Z [INFO] - Edge runtime status is failed, starting module now...
Apr 14 18:49:01 tatooine aziot-edged[6868]: 2022-04-14T18:49:01Z [INFO] - Starting module edgeAgent...
Apr 14 18:49:01 tatooine aziot-edged[6868]: 2022-04-14T18:49:01Z [INFO] - Starting new listener for module edgeAgent
Apr 14 18:49:01 tatooine aziot-edged[6868]: 2022-04-14T18:49:01Z [INFO] - Listener  edgeAgent already started, removing old listener
Apr 14 18:49:01 tatooine aziot-edged[6868]: 2022-04-14T18:49:01Z [WARN] - The workload service encountered an error
Apr 14 18:49:01 tatooine aziot-edged[6868]: 2022-04-14T18:49:01Z [WARN] - Error in watchdog when checking for edge runtime status:
Apr 14 18:49:01 tatooine aziot-edged[6868]: 2022-04-14T18:49:01Z [WARN] - A module runtime error occurred.
Apr 14 18:49:01 tatooine aziot-edged[6868]: 2022-04-14T18:49:01Z [WARN] - 	caused by: Could not get module edgeAgent
Apr 14 18:50:01 tatooine aziot-edged[6868]: 2022-04-14T18:50:01Z [INFO] - Checking edge runtime status
Apr 14 18:50:01 tatooine aziot-edged[6868]: 2022-04-14T18:50:01Z [INFO] - Edge runtime status is failed, starting module now...
Apr 14 18:50:01 tatooine aziot-edged[6868]: 2022-04-14T18:50:01Z [INFO] - Starting module edgeAgent...
Apr 14 18:50:01 tatooine aziot-edged[6868]: 2022-04-14T18:50:01Z [INFO] - Starting new listener for module edgeAgent
Apr 14 18:50:01 tatooine aziot-edged[6868]: 2022-04-14T18:50:01Z [WARN] - Error in watchdog when checking for edge runtime status:
Apr 14 18:50:01 tatooine aziot-edged[6868]: 2022-04-14T18:50:01Z [WARN] - A module runtime error occurred.
Apr 14 18:50:01 tatooine aziot-edged[6868]: 2022-04-14T18:50:01Z [WARN] - 	caused by: Could not get module edgeAgent
Apr 14 18:51:01 tatooine aziot-edged[6868]: 2022-04-14T18:51:01Z [INFO] - Checking edge runtime status
Apr 14 18:51:01 tatooine aziot-edged[6868]: 2022-04-14T18:51:01Z [INFO] - Edge runtime status is failed, starting module now...
Apr 14 18:51:01 tatooine aziot-edged[6868]: 2022-04-14T18:51:01Z [INFO] - Starting module edgeAgent...
Apr 14 18:51:01 tatooine aziot-edged[6868]: 2022-04-14T18:51:01Z [INFO] - Starting new listener for module edgeAgent
Apr 14 18:51:01 tatooine aziot-edged[6868]: 2022-04-14T18:51:01Z [INFO] - Listener  edgeAgent already started, removing old listener
Apr 14 18:51:01 tatooine aziot-edged[6868]: 2022-04-14T18:51:01Z [WARN] - The workload service encountered an error
Apr 14 18:51:01 tatooine aziot-edged[6868]: 2022-04-14T18:51:01Z [WARN] - Error in watchdog when checking for edge runtime status:
Apr 14 18:51:01 tatooine aziot-edged[6868]: 2022-04-14T18:51:01Z [WARN] - A module runtime error occurred.
Apr 14 18:51:01 tatooine aziot-edged[6868]: 2022-04-14T18:51:01Z [WARN] - 	caused by: Could not get module edgeAgent
Apr 14 18:52:01 tatooine aziot-edged[6868]: 2022-04-14T18:52:01Z [INFO] - Checking edge runtime status
Apr 14 18:52:01 tatooine aziot-edged[6868]: 2022-04-14T18:52:01Z [INFO] - Edge runtime status is failed, starting module now...
Apr 14 18:52:01 tatooine aziot-edged[6868]: 2022-04-14T18:52:01Z [INFO] - Starting module edgeAgent...
Apr 14 18:52:01 tatooine aziot-edged[6868]: 2022-04-14T18:52:01Z [INFO] - Starting new listener for module edgeAgent
Apr 14 18:52:01 tatooine aziot-edged[6868]: 2022-04-14T18:52:01Z [WARN] - Error in watchdog when checking for edge runtime status:
Apr 14 18:52:01 tatooine aziot-edged[6868]: 2022-04-14T18:52:01Z [WARN] - A module runtime error occurred.
Apr 14 18:52:01 tatooine aziot-edged[6868]: 2022-04-14T18:52:01Z [WARN] - 	caused by: Could not get module edgeAgent
Apr 14 18:53:01 tatooine aziot-edged[6868]: 2022-04-14T18:53:01Z [INFO] - Checking edge runtime status
Apr 14 18:53:01 tatooine aziot-edged[6868]: 2022-04-14T18:53:01Z [INFO] - Edge runtime status is failed, starting module now...
Apr 14 18:53:01 tatooine aziot-edged[6868]: 2022-04-14T18:53:01Z [INFO] - Starting module edgeAgent...
Apr 14 18:53:01 tatooine aziot-edged[6868]: 2022-04-14T18:53:01Z [INFO] - Starting new listener for module edgeAgent
Apr 14 18:53:01 tatooine aziot-edged[6868]: 2022-04-14T18:53:01Z [INFO] - Listener  edgeAgent already started, removing old listener
Apr 14 18:53:01 tatooine aziot-edged[6868]: 2022-04-14T18:53:01Z [WARN] - The workload service encountered an error
Apr 14 18:53:01 tatooine aziot-edged[6868]: 2022-04-14T18:53:01Z [WARN] - Error in watchdog when checking for edge runtime status:
Apr 14 18:53:01 tatooine aziot-edged[6868]: 2022-04-14T18:53:01Z [WARN] - A module runtime error occurred.
Apr 14 18:53:01 tatooine aziot-edged[6868]: 2022-04-14T18:53:01Z [WARN] - 	caused by: Could not get module edgeAgent
Apr 14 18:54:01 tatooine aziot-edged[6868]: 2022-04-14T18:54:01Z [INFO] - Checking edge runtime status
Apr 14 18:54:01 tatooine aziot-edged[6868]: 2022-04-14T18:54:01Z [INFO] - Edge runtime status is failed, starting module now...
Apr 14 18:54:01 tatooine aziot-edged[6868]: 2022-04-14T18:54:01Z [INFO] - Starting module edgeAgent...
Apr 14 18:54:01 tatooine aziot-edged[6868]: 2022-04-14T18:54:01Z [INFO] - Starting new listener for module edgeAgent
Apr 14 18:54:01 tatooine aziot-edged[6868]: 2022-04-14T18:54:01Z [WARN] - Error in watchdog when checking for edge runtime status:
Apr 14 18:54:01 tatooine aziot-edged[6868]: 2022-04-14T18:54:01Z [WARN] - A module runtime error occurred.
Apr 14 18:54:01 tatooine aziot-edged[6868]: 2022-04-14T18:54:01Z [WARN] - 	caused by: Could not get module edgeAgent
Apr 14 18:55:01 tatooine aziot-edged[6868]: 2022-04-14T18:55:01Z [INFO] - Checking edge runtime status
Apr 14 18:55:01 tatooine aziot-edged[6868]: 2022-04-14T18:55:01Z [INFO] - Edge runtime status is failed, starting module now...
Apr 14 18:55:01 tatooine aziot-edged[6868]: 2022-04-14T18:55:01Z [INFO] - Starting module edgeAgent...
Apr 14 18:55:01 tatooine aziot-edged[6868]: 2022-04-14T18:55:01Z [INFO] - Starting new listener for module edgeAgent
Apr 14 18:55:01 tatooine aziot-edged[6868]: 2022-04-14T18:55:01Z [INFO] - Listener  edgeAgent already started, removing old listener
Apr 14 18:55:01 tatooine aziot-edged[6868]: 2022-04-14T18:55:01Z [WARN] - The workload service encountered an error
Apr 14 18:55:01 tatooine aziot-edged[6868]: 2022-04-14T18:55:01Z [WARN] - Error in watchdog when checking for edge runtime status:
Apr 14 18:55:01 tatooine aziot-edged[6868]: 2022-04-14T18:55:01Z [WARN] - A module runtime error occurred.
Apr 14 18:55:01 tatooine aziot-edged[6868]: 2022-04-14T18:55:01Z [WARN] - 	caused by: Could not get module edgeAgent
Apr 14 18:56:01 tatooine aziot-edged[6868]: 2022-04-14T18:56:01Z [INFO] - Checking edge runtime status
Apr 14 18:56:01 tatooine aziot-edged[6868]: 2022-04-14T18:56:01Z [INFO] - Edge runtime status is failed, starting module now...
Apr 14 18:56:01 tatooine aziot-edged[6868]: 2022-04-14T18:56:01Z [INFO] - Starting module edgeAgent...
Apr 14 18:56:01 tatooine aziot-edged[6868]: 2022-04-14T18:56:01Z [INFO] - Starting new listener for module edgeAgent
Apr 14 18:56:01 tatooine aziot-edged[6868]: 2022-04-14T18:56:01Z [WARN] - Error in watchdog when checking for edge runtime status:
Apr 14 18:56:01 tatooine aziot-edged[6868]: 2022-04-14T18:56:01Z [WARN] - A module runtime error occurred.
Apr 14 18:56:01 tatooine aziot-edged[6868]: 2022-04-14T18:56:01Z [WARN] - 	caused by: Could not get module edgeAgent
Apr 14 18:57:01 tatooine aziot-edged[6868]: 2022-04-14T18:57:01Z [INFO] - Checking edge runtime status
Apr 14 18:57:01 tatooine aziot-edged[6868]: 2022-04-14T18:57:01Z [INFO] - Edge runtime status is failed, starting module now...
Apr 14 18:57:01 tatooine aziot-edged[6868]: 2022-04-14T18:57:01Z [INFO] - Starting module edgeAgent...
Apr 14 18:57:01 tatooine aziot-edged[6868]: 2022-04-14T18:57:01Z [INFO] - Starting new listener for module edgeAgent
Apr 14 18:57:01 tatooine aziot-edged[6868]: 2022-04-14T18:57:01Z [INFO] - Listener  edgeAgent already started, removing old listener
Apr 14 18:57:01 tatooine aziot-edged[6868]: 2022-04-14T18:57:01Z [WARN] - The workload service encountered an error
Apr 14 18:57:01 tatooine aziot-edged[6868]: 2022-04-14T18:57:01Z [WARN] - Error in watchdog when checking for edge runtime status:
Apr 14 18:57:01 tatooine aziot-edged[6868]: 2022-04-14T18:57:01Z [WARN] - A module runtime error occurred.
Apr 14 18:57:01 tatooine aziot-edged[6868]: 2022-04-14T18:57:01Z [WARN] - 	caused by: Could not get module edgeAgent
Apr 14 18:58:01 tatooine aziot-edged[6868]: 2022-04-14T18:58:01Z [INFO] - Checking edge runtime status
Apr 14 18:58:01 tatooine aziot-edged[6868]: 2022-04-14T18:58:01Z [INFO] - Edge runtime status is failed, starting module now...
Apr 14 18:58:01 tatooine aziot-edged[6868]: 2022-04-14T18:58:01Z [INFO] - Starting module edgeAgent...
Apr 14 18:58:01 tatooine aziot-edged[6868]: 2022-04-14T18:58:01Z [INFO] - Starting new listener for module edgeAgent
Apr 14 18:58:01 tatooine aziot-edged[6868]: 2022-04-14T18:58:01Z [WARN] - Error in watchdog when checking for edge runtime status:
Apr 14 18:58:01 tatooine aziot-edged[6868]: 2022-04-14T18:58:01Z [WARN] - A module runtime error occurred.
Apr 14 18:58:01 tatooine aziot-edged[6868]: 2022-04-14T18:58:01Z [WARN] - 	caused by: Could not get module edgeAgent
Apr 14 18:59:01 tatooine aziot-edged[6868]: 2022-04-14T18:59:01Z [INFO] - Checking edge runtime status
Apr 14 18:59:01 tatooine aziot-edged[6868]: 2022-04-14T18:59:01Z [INFO] - Edge runtime status is failed, starting module now...
Apr 14 18:59:01 tatooine aziot-edged[6868]: 2022-04-14T18:59:01Z [INFO] - Starting module edgeAgent...
Apr 14 18:59:01 tatooine aziot-edged[6868]: 2022-04-14T18:59:01Z [INFO] - Starting new listener for module edgeAgent
Apr 14 18:59:01 tatooine aziot-edged[6868]: 2022-04-14T18:59:01Z [INFO] - Listener  edgeAgent already started, removing old listener
Apr 14 18:59:01 tatooine aziot-edged[6868]: 2022-04-14T18:59:01Z [WARN] - The workload service encountered an error
Apr 14 18:59:01 tatooine aziot-edged[6868]: 2022-04-14T18:59:01Z [WARN] - Error in watchdog when checking for edge runtime status:
Apr 14 18:59:01 tatooine aziot-edged[6868]: 2022-04-14T18:59:01Z [WARN] - A module runtime error occurred.
Apr 14 18:59:01 tatooine aziot-edged[6868]: 2022-04-14T18:59:01Z [WARN] - 	caused by: Could not get module edgeAgent
Apr 14 19:00:01 tatooine aziot-edged[6868]: 2022-04-14T19:00:01Z [INFO] - Checking edge runtime status
Apr 14 19:00:01 tatooine aziot-edged[6868]: 2022-04-14T19:00:01Z [INFO] - Edge runtime status is failed, starting module now...
Apr 14 19:00:01 tatooine aziot-edged[6868]: 2022-04-14T19:00:01Z [INFO] - Starting module edgeAgent...
Apr 14 19:00:01 tatooine aziot-edged[6868]: 2022-04-14T19:00:01Z [INFO] - Starting new listener for module edgeAgent
Apr 14 19:00:01 tatooine aziot-edged[6868]: 2022-04-14T19:00:01Z [WARN] - Error in watchdog when checking for edge runtime status:
Apr 14 19:00:01 tatooine aziot-edged[6868]: 2022-04-14T19:00:01Z [WARN] - A module runtime error occurred.
Apr 14 19:00:01 tatooine aziot-edged[6868]: 2022-04-14T19:00:01Z [WARN] - 	caused by: Could not get module edgeAgent
Apr 14 19:00:17 tatooine aziot-edged[6868]: 2022-04-14T19:00:17Z [INFO] - [mgmt] - - - [2022-04-14 19:00:17.095566393 UTC] "GET /modules/?api-version=2018-06-28 HTTP/1.1" 200 OK 690 "-" "-" auth_id(-)
Apr 14 19:00:17 tatooine aziot-edged[6868]: 2022-04-14T19:00:17Z [INFO] - [mgmt] - - - [2022-04-14 19:00:17.720121099 UTC] "GET /modules?api-version=2021-12-07 HTTP/1.1" 200 OK 690 "-" "iotedge/0.1.0" auth_id(-)
Apr 14 19:00:17 tatooine aziot-edged[6868]: 2022-04-14T19:00:17Z [INFO] - Getting logs for module edgeAgent...
Apr 14 19:00:17 tatooine aziot-edged[6868]: 2022-04-14T19:00:17Z [INFO] - Successfully got logs for module edgeAgent
Apr 14 19:00:17 tatooine aziot-edged[6868]: 2022-04-14T19:00:17Z [INFO] - [mgmt] - - - [2022-04-14 19:00:17.720938402 UTC] "GET /modules/edgeAgent/logs?api-version=2021-12-07&follow=false&tail=all&timestamps=false&since=1649876384 HTTP/1.1" 200 OK - "-" "iotedge/0.1.0" auth_id(-)

edge-agent logs


2022-04-14 18:40:51  Starting Edge Agent
2022-04-14 18:40:51  Changing ownership of management socket: /var/run/iotedge/mgmt.sock
2022-04-14 18:40:51.161 +00:00 Edge Agent Main()
<6> 2022-04-14 18:40:51.312 +00:00 [INF] - Initializing Edge Agent.
<6> 2022-04-14 18:40:51.430 +00:00 [INF] - Version - 1.2.9.53764592 (4bbfdb99cf7afb16f23d6177865942cc22e197ce)
<6> 2022-04-14 18:40:51.430 +00:00 [INF] - 
        █████╗ ███████╗██╗   ██╗██████╗ ███████╗
       ██╔══██╗╚══███╔╝██║   ██║██╔══██╗██╔════╝
       ███████║  ███╔╝ ██║   ██║██████╔╝█████╗
       ██╔══██║ ███╔╝  ██║   ██║██╔══██╗██╔══╝
       ██║  ██║███████╗╚██████╔╝██║  ██║███████╗
       ╚═╝  ╚═╝╚══════╝ ╚═════╝ ╚═╝  ╚═╝╚══════╝

 ██╗ ██████╗ ████████╗    ███████╗██████╗  ██████╗ ███████╗
 ██║██╔═══██╗╚══██╔══╝    ██╔════╝██╔══██╗██╔════╝ ██╔════╝
 ██║██║   ██║   ██║       █████╗  ██║  ██║██║  ███╗█████╗
 ██║██║   ██║   ██║       ██╔══╝  ██║  ██║██║   ██║██╔══╝
 ██║╚██████╔╝   ██║       ███████╗██████╔╝╚██████╔╝███████╗
 ╚═╝ ╚═════╝    ╚═╝       ╚══════╝╚═════╝  ╚═════╝ ╚══════╝

<6> 2022-04-14 18:40:51.464 +00:00 [INF] - Experimental features configuration: {"Enabled":false,"DisableCloudSubscriptions":false}
<0> 2022-04-14 18:40:51.525 +00:00 [FTL] - Fatal error building application.
System.Net.Internals.SocketExceptionFactory+ExtendedSocketException (13): Permission denied /var/run/iotedge/workload.sock
   at System.Net.Sockets.Socket.DoConnect(EndPoint endPointSnapshot, SocketAddress socketAddress)
   at System.Net.Sockets.Socket.Connect(EndPoint remoteEP)
   at System.Net.Sockets.Socket.UnsafeBeginConnect(EndPoint remoteEP, AsyncCallback callback, Object state, Boolean flowContext)
   at System.Net.Sockets.Socket.BeginConnect(EndPoint remoteEP, AsyncCallback callback, Object state)
   at System.Net.Sockets.Socket.ConnectAsync(EndPoint remoteEP)
   at System.Net.Sockets.SocketTaskExtensions.ConnectAsync(Socket socket, EndPoint remoteEP)
   at Microsoft.Azure.Devices.Edge.Util.Uds.HttpUdsMessageHandler.SendAsync(HttpRequestMessage request, CancellationToken cancellationToken) in /mnt/vss/_work/1/s/edge-util/src/Microsoft.Azure.Devices.Edge.Util/uds/HttpUdsMessageHandler.cs:line 30
   at System.Net.Http.HttpClient.FinishSendAsyncUnbuffered(Task`1 sendTask, HttpRequestMessage request, CancellationTokenSource cts, Boolean disposeCts)
   at Microsoft.Azure.Devices.Edge.Util.Edged.Version_2019_01_30.GeneratedCode.HttpWorkloadClient.TrustBundleAsync(String api_version, CancellationToken cancellationToken) in /mnt/vss/_work/1/s/edge-util/src/Microsoft.Azure.Devices.Edge.Util/edged/version_2019_01_30/generatedCode/HttpWorkloadClient.cs:line 700
   at Microsoft.Azure.Devices.Edge.Util.TaskEx.TimeoutAfter[T](Task`1 task, TimeSpan timeout) in /mnt/vss/_work/1/s/edge-util/src/Microsoft.Azure.Devices.Edge.Util/TaskEx.cs:line 126
   at Microsoft.Azure.Devices.Edge.Util.Edged.WorkloadClientVersioned.Execute[T](Func`1 func, String operation) in /mnt/vss/_work/1/s/edge-util/src/Microsoft.Azure.Devices.Edge.Util/edged/WorkloadClientVersioned.cs:line 59
   at Microsoft.Azure.Devices.Edge.Util.Edged.Version_2019_01_30.WorkloadClient.HandleException(Exception ex, String operation) in /mnt/vss/_work/1/s/edge-util/src/Microsoft.Azure.Devices.Edge.Util/edged/version_2019_01_30/WorkloadClient.cs:line 122
   at Microsoft.Azure.Devices.Edge.Util.Edged.WorkloadClientVersioned.Execute[T](Func`1 func, String operation) in /mnt/vss/_work/1/s/edge-util/src/Microsoft.Azure.Devices.Edge.Util/edged/WorkloadClientVersioned.cs:line 77
   at Microsoft.Azure.Devices.Edge.Util.Edged.Version_2019_01_30.WorkloadClient.GetTrustBundleAsync() in /mnt/vss/_work/1/s/edge-util/src/Microsoft.Azure.Devices.Edge.Util/edged/version_2019_01_30/WorkloadClient.cs:line 49
   at Microsoft.Azure.Devices.Edge.Util.CertificateHelper.GetTrustBundleFromEdgelet(Uri workloadUri, String workloadApiVersion, String workloadClientApiVersion, String moduleId, String moduleGenerationId) in /mnt/vss/_work/1/s/edge-util/src/Microsoft.Azure.Devices.Edge.Util/CertificateHelper.cs:line 263
   at Microsoft.Azure.Devices.Edge.Agent.Service.Program.MainAsync(IConfiguration configuration) in /mnt/vss/_work/1/s/edge-agent/src/Microsoft.Azure.Devices.Edge.Agent.Service/Program.cs:line 197

edge-hub logs


Hub never starts

Additional Information

Please provide any additional information that may be helpful in understanding the issue.

The certs that are normally generated for the workload socket do not exist when the system fails to startup. The device registers correctly with DPS.

I’ll also be opening a support ticket. (Update: ID 2204140040008325)

About this issue

Original URL
State: closed
Created 2 years ago
Comments: 24 (9 by maintainers)

Most upvoted comments

Looks like the error message @huguesBouvier add in https://github.com/Azure/iotedge/commit/f2e5a47bb3fd109b5523cb2ae9849401f0fcc99d is working as intended. It took me so long to figure out the problem without that error message. 👌

allenhumphreys on Jun 7, 2022

This has been fixed in commit https://github.com/Azure/iotedge/commit/921840e027f1e8f2d6635f75230e4616bab2e468

varunpuranik on Jun 16, 2022