iotedge: Edge Hub module can't change the permissions of the host folder

Expected Behavior

According to the documentation , giving permission 700 to user 1000 on the host storage should work.

Current Behavior

Since version 1.3.0 of edgeHub, it uses a different user (13623), which can’t access to the storage folder set for uid 1000. The script startHub.sh tries to change the owner (as edgeAgent does), but it does not work (perhaps because edgeHub does not start as root?)

Steps to Reproduce

Provide a detailed set of steps to reproduce the bug.

Configure a host folder for storage for edgeAgent and edgeHub
Set the owner to uid 1000, permissions 700
edge hub will not start

Context (Environment)

Device Information

Host OS [e.g. Ubuntu 18.04, Windows Server IoT 2019]:
Architecture [e.g. amd64, arm32, arm64]:
Container OS [e.g. Linux containers, Windows containers]:

Runtime Versions

aziot-edged [run iotedge version]: 1.3.0
Edge Agent [image tag (e.g. 1.0.0)]: 1.3.0
Edge Hub [image tag (e.g. 1.0.0)]: 1.3.0
Docker/Moby [run docker version]: 20.10.17+azure-1, build 100c70180fde3601def79a59cc3e996aa553c9b9

Note: when using Windows containers on Windows, run docker -H npipe:////./pipe/iotedge_moby_engine version instead

Logs

edge-hub logs

``` <6> 2022-07-27 13:13:24.486 +00:00 [INF] - Initializing Edge Hub <6> 2022-07-27 13:13:24.487 +00:00 [INF] - █████╗ ███████╗██╗ ██╗██████╗ ███████╗ ██╔══██╗╚══███╔╝██║ ██║██╔══██╗██╔════╝ ███████║ ███╔╝ ██║ ██║██████╔╝█████╗ ██╔══██║ ███╔╝ ██║ ██║██╔══██╗██╔══╝ ██║ ██║███████╗╚██████╔╝██║ ██║███████╗ ╚═╝ ╚═╝╚══════╝ ╚═════╝ ╚═╝ ╚═╝╚══════╝

██╗ ██████╗ ████████╗ ███████╗██████╗ ██████╗ ███████╗ ██║██╔═══██╗╚══██╔══╝ ██╔════╝██╔══██╗██╔════╝ ██╔════╝ ██║██║ ██║ ██║ █████╗ ██║ ██║██║ ███╗█████╗ ██║██║ ██║ ██║ ██╔══╝ ██║ ██║██║ ██║██╔══╝ ██║╚██████╔╝ ██║ ███████╗██████╔╝╚██████╔╝███████╗ ╚═╝ ╚═════╝ ╚═╝ ╚══════╝╚═════╝ ╚═════╝ ╚══════╝

<6> 2022-07-27 13:13:24.490 +00:00 [INF] - Version - 1.3.0.57041647 (b022069058d21deb30c7760c4e384b637694f464) <6> 2022-07-27 13:13:24.490 +00:00 [INF] - OptimizeForPerformance=True <6> 2022-07-27 13:13:24.491 +00:00 [INF] - MessageAckTimeoutSecs=30 <6> 2022-07-27 13:13:24.493 +00:00 [INF] - Loaded server certificate with expiration date of “2022-08-26T13:13:22.0000000+00:00” <6> 2022-07-27 13:13:24.516 +00:00 [INF] - Using Asp Net server for metrics <6> 2022-07-27 13:13:24.666 +00:00 [INF] - Created new message store <6> 2022-07-27 13:13:24.666 +00:00 [INF] - Started task to cleanup processed and stale messages <6> 2022-07-27 13:13:24.879 +00:00 [INF] - Created DeviceConnectivityManager with connected check frequency 00:05:00 and disconnected check frequency 00:02:00 <3> 2022-07-27 13:13:24.924 +00:00 [ERR] - Stopping with exception System.UnauthorizedAccessException: Access to the path ‘/iotedge/storage/edgeHub/EdgeHubIV’ is denied. —> System.IO.IOException: Permission denied — End of inner exception stack trace — at Interop.ThrowExceptionForIoErrno(ErrorInfo errorInfo, String path, Boolean isDirectory, Func2 errorRewriter) at Microsoft.Win32.SafeHandles.SafeFileHandle.Open(String path, OpenFlags flags, Int32 mode) at Microsoft.Win32.SafeHandles.SafeFileHandle.Open(String fullPath, FileMode mode, FileAccess access, FileShare share, FileOptions options, Int64 preallocationSize) at System.IO.Strategies.OSFileStreamStrategy..ctor(String path, FileMode mode, FileAccess access, FileShare share, FileOptions options, Int64 preallocationSize) at System.IO.Strategies.FileStreamHelpers.ChooseStrategy(FileStream fileStream, String path, FileMode mode, FileAccess access, FileShare share, Int32 bufferSize, FileOptions options, Int64 preallocationSize) at System.IO.File.Open(String path, FileMode mode) at Microsoft.Azure.Devices.Edge.Util.DiskFile.<>c__DisplayClass4_0.<<WriteAllAsync>g__WriteOperation|0>d.MoveNext() in /mnt/vss/_work/1/s/edge-util/src/Microsoft.Azure.Devices.Edge.Util/DiskFile.cs:line 34 --- End of stack trace from previous location --- at Microsoft.Azure.Devices.Edge.Util.TaskEx.TimeoutAfter(Task task, TimeSpan timeout, Action action) in /mnt/vss/_work/1/s/edge-util/src/Microsoft.Azure.Devices.Edge.Util/TaskEx.cs:line 158 at Microsoft.Azure.Devices.Edge.Util.Edged.EncryptionProvider.CreateAsync(String storagePath, Uri workloadUri, String edgeletApiVersion, String edgeletClientApiVersion, String moduleId, String genId, String initializationVectorFileName) in /mnt/vss/_work/1/s/edge-util/src/Microsoft.Azure.Devices.Edge.Util/edged/EncryptionProvider.cs:line 30 at Microsoft.Azure.Devices.Edge.Hub.Service.Modules.CommonModule.<Load>b__28_29(String uri) in /mnt/vss/_work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.Service/modules/CommonModule.cs:line 240 at Microsoft.Azure.Devices.Edge.Hub.Service.Modules.CommonModule.<Load>b__28_8(IComponentContext c) in /mnt/vss/_work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.Service/modules/CommonModule.cs:line 236 at Microsoft.Azure.Devices.Edge.Hub.Service.Modules.RoutingModule.GetTwinStore(IComponentContext context) in /mnt/vss/_work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.Service/modules/RoutingModule.cs:line 662 at Microsoft.Azure.Devices.Edge.Hub.Service.Modules.RoutingModule.<Load>b__36_25(IComponentContext c) in /mnt/vss/_work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.Service/modules/RoutingModule.cs:line 467 at Microsoft.Azure.Devices.Edge.Hub.Service.Modules.RoutingModule.<Load>b__36_28(IComponentContext c) in /mnt/vss/_work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.Service/modules/RoutingModule.cs:line 527 at Microsoft.Azure.Devices.Edge.Hub.Service.Program.MainAsync(IConfigurationRoot configuration) in /mnt/vss/_work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.Service/Program.cs:line 80 Unhandled exception. System.AggregateException: One or more errors occurred. (Access to the path '/iotedge/storage/edgeHub/EdgeHubIV' is denied.) ---> System.UnauthorizedAccessException: Access to the path '/iotedge/storage/edgeHub/EdgeHubIV' is denied. ---> System.IO.IOException: Permission denied --- End of inner exception stack trace --- at Interop.ThrowExceptionForIoErrno(ErrorInfo errorInfo, String path, Boolean isDirectory, Func2 errorRewriter) at Microsoft.Win32.SafeHandles.SafeFileHandle.Open(String path, OpenFlags flags, Int32 mode) at Microsoft.Win32.SafeHandles.SafeFileHandle.Open(String fullPath, FileMode mode, FileAccess access, FileShare share, FileOptions options, Int64 preallocationSize) at System.IO.Strategies.OSFileStreamStrategy…ctor(String path, FileMode mode, FileAccess access, FileShare share, FileOptions options, Int64 preallocationSize) at System.IO.Strategies.FileStreamHelpers.ChooseStrategy(FileStream fileStream, String path, FileMode mode, FileAccess access, FileShare share, Int32 bufferSize, FileOptions options, Int64 preallocationSize) at System.IO.File.Open(String path, FileMode mode) at Microsoft.Azure.Devices.Edge.Util.DiskFile.<>c__DisplayClass4_0.<<WriteAllAsync>g__WriteOperation|0>d.MoveNext() in /mnt/vss/_work/1/s/edge-util/src/Microsoft.Azure.Devices.Edge.Util/DiskFile.cs:line 34 — End of stack trace from previous location — at Microsoft.Azure.Devices.Edge.Util.TaskEx.TimeoutAfter(Task task, TimeSpan timeout, Action action) in /mnt/vss/_work/1/s/edge-util/src/Microsoft.Azure.Devices.Edge.Util/TaskEx.cs:line 158 at Microsoft.Azure.Devices.Edge.Util.Edged.EncryptionProvider.CreateAsync(String storagePath, Uri workloadUri, String edgeletApiVersion, String edgeletClientApiVersion, String moduleId, String genId, String initializationVectorFileName) in /mnt/vss/_work/1/s/edge-util/src/Microsoft.Azure.Devices.Edge.Util/edged/EncryptionProvider.cs:line 30 at Microsoft.Azure.Devices.Edge.Hub.Service.Modules.CommonModule.<Load>b__28_29(String uri) in /mnt/vss/_work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.Service/modules/CommonModule.cs:line 240 at Microsoft.Azure.Devices.Edge.Hub.Service.Modules.CommonModule.<Load>b__28_8(IComponentContext c) in /mnt/vss/_work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.Service/modules/CommonModule.cs:line 236 at Microsoft.Azure.Devices.Edge.Hub.Service.Modules.RoutingModule.GetTwinStore(IComponentContext context) in /mnt/vss/_work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.Service/modules/RoutingModule.cs:line 662 at Microsoft.Azure.Devices.Edge.Hub.Service.Modules.RoutingModule.<Load>b__36_25(IComponentContext c) in /mnt/vss/_work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.Service/modules/RoutingModule.cs:line 467 at Microsoft.Azure.Devices.Edge.Hub.Service.Modules.RoutingModule.<Load>b__36_28(IComponentContext c) in /mnt/vss/_work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.Service/modules/RoutingModule.cs:line 527 at Microsoft.Azure.Devices.Edge.Hub.Service.Program.MainAsync(IConfigurationRoot configuration) in /mnt/vss/_work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.Service/Program.cs:line 80 — End of inner exception stack trace — at System.Threading.Tasks.Task1.GetResultCore(Boolean waitCompletionNotification) at System.Threading.Tasks.Task1.get_Result() at Microsoft.Azure.Devices.Edge.Hub.Service.Program.Main() in /mnt/vss/_work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.Service/Program.cs:line 35

</details>

## Additional Information
Please provide any additional information that may be helpful in understanding the issue.

About this issue

Original URL
State: closed
Created 2 years ago
Reactions: 1
Comments: 16 (4 by maintainers)

Commits related to this issue

Fix user creation for edgeAgent and edgeHub (#6570) Fixes edgeAgent and edgeHub startup scripts, in response to https://github.com/Azure/iotedge/issues/6541 - Creates the edgehubuser and aziot-edged ... — committed to Azure/iotedge by gordonwang0 2 years ago

Most upvoted comments

We’ll release a fix for this. In the meantime, you can use the workarounds mentioned above, or rename the folder to edgehub.

Awesome! Any idea when this fix will be released? We’re having issues with 1.2.6 and 1.2.8 and were urged to upgrade to 1.3 because of those (as part of azure support ticket 2207070050001721)

And special thanks to the originator of this bug report, Luciano!

Cheers, Nic.

DarthLowen on Aug 6, 2022

I have merged the PR that implements the fix. It will be released officially in the 1.4 version. Unfortunately, I don’t have a estimated date of release.

gordonwang0 on Aug 11, 2022

edgeHub is supposed to be able to fix its directory permissions just like edgeAgent. However, there’s a typo in the startup script that breaks it: https://github.com/Azure/iotedge/blob/1b3f818c2eecd08d9442eee98f1b57f5502f166b/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.Service/hubStart.sh#L50

The script is expecting the default folder to be edgehub, while I’m guessing everyone here has the folder mounted as edgeHub.

We’ll release a fix for this. In the meantime, you can use the workarounds mentioned above, or rename the folder to edgehub.

gordonwang0 on Aug 6, 2022