iotedge: IotEdge modules does not conect to IotHub using TPM credential

I have Provisioned my Linux device using TPM. Everything worked fine and my modules have been successfully deployed on the iotedge. One of the modules must connect to the IotHub using DeviceClient in order to retrieve Device twin information and make some busyness logic with them. The connection using TPM fails. The working version of the sw that uses the symmetric key authentication is:

var authenticationMethod = new DeviceAuthenticationWithRegistrySymmetricKey(deviceId, deviceKey);
_deviceClient = DeviceClient.Create(iotHubHostname,
               authenticationMethod , settings);

The not working version for the Tpm is:

var authenticationMethod = new DeviceAuthenticationWithTpm(deviceId, new SecurityProviderTpmHsm(deviceId));;
_deviceClient = DeviceClient.Create(iotHubHostname,
               authenticationMethod , settings);

when a device is provisioned using Tpm the registration ID is kept equal to the iotHub Device id. The Error logged by the module is:

Debug HostingLoggerExtensions.Starting [1] - Hosting starting                                  
Exception while loading tpm2-abrmd: System.DllNotFoundException: Unable to load shared library 'libtss2-tcti-tabrmd.so' or one of its dependencies. In order to help diagnose loading problems, consider setting the LD_DEBUG environment variable: liblibtss2-tcti-tabrmd.so: cannot open shared object file: No such file or directory                                   
at Tpm2Lib.AbrmdWrapper.NativeMethods.Tss2_Tcti_Info()                                                                  
at Tpm2Lib.AbrmdWrapper.Load(IntPtr& tctiCtxPtr)                                                                        
at Tpm2Lib.LinuxTpmDevice..ctor(String tpmDevicePath)                                                                
Unhandled exception. Autofac.Core.DependencyResolutionException: An exception was thrown while activating λ:Microsoft.Extensions.Hosting.IHostedService[] -> Tenova.PDM.DeviceMonitor.DeviceMonitorApp -> Tenova.PDM.IotHubDevice.AzureDeviceClient.                                                                                                                     
---> Autofac.Core.DependencyResolutionException: An exception was thrown while invoking the constructor 'Void .ctor(Microsoft.Extensions.Logging.ILogger``1[Tenova.PDM.IotHubDevice.AzureDeviceClient], Tenova.PDM.IotHubDevice.IAzureDeviceClientConfiguration, Tenova.PDM.IotHubDevice.IClientAuthenticationFactory, Tenova.PDM.IotHubDevice.Configuration.IPropertiesStore, System.Net.IWebProxy)' on type 'AzureDeviceClient'.                                                               
---> System.AggregateException: One or more errors occurred. (Connection refused)                                       
---> System.Net.Sockets.SocketException (111): Connection refused                                                         
at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.ThrowException(SocketError error, CancellationToken cancellationToken)                                                                                                               
at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.System.Threading.Tasks.Sources.IValueTaskSource.GetResult(Int16 token)                                                                                                               
at System.Threading.Tasks.ValueTask.ValueTaskSourceAsTask.<>c.<.cctor>b__4_0(Object state)                           
--- End of stack trace from previous location ---                                                                          
at System.Net.Sockets.TcpClient.CompleteConnectAsync(Task task)                                                         
--- End of inner exception stack trace ---                                                                              
at System.Threading.Tasks.Task.Wait(Int32 millisecondsTimeout, CancellationToken cancellationToken)                     
at System.Threading.Tasks.Task.Wait()                                                                                   
at Tpm2Lib.TcpTpmDevice.ConnectWorker(String hostName, Int32 port, NetworkStream& theStream, TcpClient& theClient)      
at Tpm2Lib.TcpTpmDevice.Connect()                                                                                       
at Tpm2Lib.LinuxTpmDevice..ctor(String tpmDevicePath)                                                                   
at Microsoft.Azure.Devices.Provisioning.Security.SecurityProviderTpmHsm.CreateDefaultTpm2Device()                      
at Microsoft.Azure.Devices.Provisioning.Security.SecurityProviderTpmHsm..ctor(String registrationId)

I have already tried various steps but without success:

added the explicit installation of libtss2-tcti-tabrmd to the docker file

FROM mcr.microsoft.com/dotnet/sdk:5.0 AS build-env
WORKDIR /app
# Install the nuget credential provider
RUN wget -qO- https://raw.githubusercontent.com/Microsoft/artifacts-credprovider/master/helpers/installcredprovider.sh | bash
# TODO: remove hardcoded PAT
ARG FEED_ACCESSTOKEN
ARG FEED_USERNAME
ENV VSS_NUGET_EXTERNAL_FEED_ENDPOINTS {\"endpointCredentials\": [{\"endpoint\":\"https://pkgs.dev.azure.com/tenovadigital/_packaging/Tenova/nuget/v3/index.json\", \"username\":\"${FEED_USERNAME}\", \"password\":\"${FEED_ACCESSTOKEN}\"}]}
COPY . ./
RUN dotnet restore ./DeviceMonitor/DeviceMonitor.csproj --configfile nuget.config
COPY . ./
RUN dotnet publish ./DeviceMonitor/DeviceMonitor.csproj -c Release -o out
FROM mcr.microsoft.com/dotnet/runtime:5.0-buster-slim
WORKDIR /app
COPY --from=build-env /app/out ./
RUN apt-get update && \
    apt-get install -y libtss2-tcti-tabrmd0
RUN useradd -ms /bin/bash moduleuser
RUN mkdir -p /app/data && chown -R moduleuser:moduleuser /app/data
VOLUME /app/data
USER moduleuser
ENTRYPOINT ["dotnet", "Tenova.PDM.DeviceMonitor.dll"]

adding create options to the iotEdge module in order to allow host TPM to be accessed by module

{
  "HostConfig": {
    "Privileged": true,
    "LogConfig": {
      "Type": "json-file",
      "Config": {
        "max-size": "10m",
        "max-file": "10"
      }
    },
    "Binds": [
      "tenovadevicemonitor_data:/app/data",
      "/dev/tpm0:/dev/tpm0",
      "/dev/tpmrm0:/dev/tpmrm0"
    ]
  }
}

But none of the solutions (in any combination) worked and the error is still the same.

Some more info:

IotEdge Version 1.2.9
Runtime version 1.2.9
Host SO Ubuntu server 20.04 LTS running on VMWare VM with TPM
The project is .Net5
Azure Device Client nuget version 1.41.0
Microsoft.Azure.Devices.Provisioning.Security.Tpm nuget version 1.14.1

I looked for documentation, examples or any similar issue and I didn’t find any suggestion nor indication about how to connect IoTEdge module to IotHub using TPM Authentication

Sandro

About this issue

Original URL
State: closed
Created 2 years ago
Comments: 22 (9 by maintainers)

Most upvoted comments

I arrived to a final solution.

Module using TPM must run as privileged
The /dev/tpm{,rm}0 devices of the host must be bound the one of the docker.
User running inside that module must belong to a group of the docker whose GID is the same as the GID of the tss group of the host

In my project:

The point 1 and 2 are solved as follow in the create option on the module

{
  "HostConfig": {
    "Privileged": true,
    "Binds": [
      "tenovadevicemonitor_data:/app/data",
      "/dev/tpm0:/dev/tpm0",
      "/dev/tpmrm0:/dev/tpmrm0"
    ]
  }
}

For the point 3 the job is a bit more complex. We could change the tss GID on the host to be 1000 as suggested by @emilm here, but since this is an administrative group with special privileges we choose a different way.

On the host we changed the GID of tss to ensure it will be the same on any installation to e.g. 2000.

#find the tss group id
cat /etc/group | awk -F ":" '{ print $1,$3 }' | grep tss
#change the tss group without allowing duplicated
sudo groupmod -g 2000 tss
#change the group id for files belonging to tss 
find / -gid <old_tss_id> -exec chgrp -v 2000 '{}' \;

finally we modified the docker file in order to add the moduleuser to a group with the 2000 GID.

FROM mcr.microsoft.com/dotnet/sdk:5.0 AS build-env
WORKDIR /app

# Install the nuget credential provider
RUN wget -qO- https://raw.githubusercontent.com/Microsoft/artifacts-credprovider/master/helpers/installcredprovider.sh | bash
# TODO: remove hardcoded PAT
ARG FEED_ACCESSTOKEN
ARG FEED_USERNAME
ENV VSS_NUGET_EXTERNAL_FEED_ENDPOINTS {\"endpointCredentials\": [{\"endpoint\":\"https://pkgs.dev.azure.com/tenovadigital/_packaging/Tenova/nuget/v3/index.json\", \"username\":\"${FEED_USERNAME}\", \"password\":\"${FEED_ACCESSTOKEN}\"}]}

COPY . ./
RUN dotnet restore ./DeviceMonitor/DeviceMonitor.csproj --configfile nuget.config

COPY . ./
RUN dotnet publish ./DeviceMonitor/DeviceMonitor.csproj -c Release -o out

FROM mcr.microsoft.com/dotnet/runtime:5.0-buster-slim
WORKDIR /app
COPY --from=build-env /app/out ./

# TPM ACCESS ENABLING -- START
RUN groupadd -f -g 2000 tpmaccess 

RUN useradd -ms /bin/bash moduleuser && \
    usermod -a -G tpmaccess moduleuser
# TPM ACCESS ENABLING -- END

RUN mkdir -p /app/data && chown -R moduleuser:moduleuser /app/data
VOLUME /app/data
USER moduleuser

ENTRYPOINT ["dotnet", "Tenova.PDM.DeviceMonitor.dll"]

That’s all. If the module software only uses the DeviceClient and Microsoft.Azure.Devices.Provisioning.Security.Tpm no need to add tpm2-tools or other packages installation in the docker.

The next code will succeed:

var authenticationMethod = new DeviceAuthenticationWithTpm(deviceId, new SecurityProviderTpmHsm(deviceId));
_deviceClient = DeviceClient.Create(iotHubHostname,
               authenticationMethod , settings);

One question to @onalante-msft can these simple information be added to the DeviceClient or IotEdge documentation? This will save tons of try and fail time to a lot of people.

Thanks Sandro Varoli

SandroVaroli on Jul 25, 2022

@emilm Thank a lot, with your last suggestion I’m near to solve the issue…

The point is:

the Id of the group tss on the host must be the id of a group the moduleuser belongs to inside the docker… this because the /dev/tpm{,rm}0 files inside the docker will inherit UID and GUI from the host. No way to make different.

So basically we have two solution:

assign to tss group on the host the id 1000 that is the group of the user running the module. to do this:

#find the tss group id
cat /etc/group | awk -F ":" '{ print $1,$3 }' | grep tss
#change the tss group id allowing duplicated id
sudo groupmod -o -g 1000 tss
#change the group id for files belonging to tss 
find / -gid <old_tss_id> -exec chgrp -v 1000 '{}' \;

That solution allow to keep easy and do not modify the dockerfile leaving it potentially agnostic on convention made on the host machine. The user running on the module will have rw permission on the Tpm devices. The back-face is that on the host machine the group 1000 is the group normally used for the administrative user created during the system installation, so you will have that the tss user will have the group permission on all the staff that are assigned to the admin group. I mean; suppose that at installation time you create the user foo. Automatically a group foo is created with GID 1000 and all the files of the home directory of foo are assigned to that group… After you assign 1000 as GID to group tss there will be no distinction between the two groups (they will be same alias) and this could be configured as a security issue.

assign to tss group on the host a conventional GID (let’s say 1001) and create a group in the dokerfile with the same GUI and add the moduleuser to it. changing the GID to tss group on the host following the next steps

#find the tss group id
cat /etc/group | awk -F ":" '{ print $1,$3 }' | grep tss
#change the tss group without allowing duplicated
sudo groupmod -g 1001 tss
#change the group id for files belonging to tss 
find / -gid <old_tss_id> -exec chgrp -v 1001 '{}' \;

then in the docker file

RUN groupadd -f -g 1001 tpmaccess

RUN useradd -ms /bin/bash moduleuser && \
    usermod -a -G tpmaccess moduleuser

This method is based on a convention that Docker and host must share. That is not very clean but it allow to have always the same tss group id on all the devices and having a docked using it without breaking security rules.

I’ll clean the project and retry on a clean environment. Then I’ll post the final working solution.

Thanks a lot to @emilm for his help. it has been needful to reach a solution

Sandro

SandroVaroli on Jul 21, 2022