iotedge: IoT Edge module stops receiving data
Expected Behavior
The edgeAgent and the whole runtime runs reliably and stable, without interruptions.
Current Behavior
We have created an IoT Edge module for receiving data from the devices. Our module sends data as messages to IoT Hub and via our event hub triggered Azure function we store messages in the database. I created about fifteen days ago: https://github.com/Azure/iotedge/issues/4992. The issue was closed but the problem, however, has not been solved. But, now I have new information:
The IoT edge module stops receiving messages after a certain time. I improved the module for receiving messages and created two threads - one thread receives messages and stores them in the BlockingCollection. Another thread reads messages from the BlockingCollection, parses them, and forwards them to an IoT Hub. But again, the receiving messages were interrupted.
We used Windows device (Azure Windows VM - Windows 10 Pro OS), Windows container, and IoT Edge 1.0.10 for Windows but the IoT Edge module stopped receiving data. After that, we upgraded the IoT Edge to 1.1 (https://docs.microsoft.com/en-us/azure/iot-edge/how-to-install-iot-edge-windows-on-windows?view=iotedge-2018-06) but the same thing happens.
But today I saw that the interruption occurs at the same time, 2:01 AM or 02:02 AM!
I checked the IoT Hub diagram for the last twenty days and attached screenshots for the last four days. After the interruption, I usually redeployed the IoT Edge module on the device and the module received messages again. Rarely it happens that receiving messages starts slowly again and redeployment is not needed.
Why does the interruption always occur at the same time? It’s as if some Azure planned maintenance operation interrupts receiving messages by the module? The interruption is not related to the number of messages, because even on weekends there is an interruption at the same time when there are significantly fewer messages.
Output of iotedge check
We use a temporary certificate but we will create another certificate for the production.
Device Information
- Host OS [Windows 10 Pro]:
- Container OS [Windows containers]:
Runtime Versions
• iotedge: 1.1.1 • Edge Agent: 1.1.2.41361617 • Edge Hub: 1.1.2.41361617
Note: when using Windows containers on Windows, run docker -H npipe:////./pipe/iotedge_moby_engine version instead
Logs
Logs for 05-23-2021:
support_bundle_1_23.05.2021.zip
Additional Information
Other screenshots are attached:
About this issue
- Original URL
- State: closed
- Created 3 years ago
- Comments: 47 (19 by maintainers)
@yophilav You can close the issue. I implemented another C# program for receiving messages via UDP protocol and for now there weren’t any interruptions. I will apply the IoT Edge monitoring solution in the next period. That will be very useful. Thanks for your cooperation!
I followed the steps and updated the IoT module to 0.1.19. And the module is receiving messages again! But I’ll see if there are any interruptions in the next period.