iotedge: Moby engine doesn't work with Raspberry Pi OS stretch and custom vision containers

Expected Behavior

When building a Custom Vision Docker Arm Raspberry Pi 3 container on a Raspberry Pi 3 running Stretch (a Tier 1 supported operating system for IoT Edge), the Docker build should work, and the container should be able to be run.

Current Behavior

Run this command:

sudo docker build .

using a container exported from Custom Vision, and you get this output:

Sending build context to Docker daemon  21.72MB
Step 1/9 : FROM python:3.7-slim
3.7-slim: Pulling from library/python
b6e5ca4da968: Pull complete 
66f8097783f7: Pull complete 
ed1b7dc0b58c: Pull complete 
de2827b1b2d4: Pull complete 
c12346225484: Pull complete 
Digest: sha256:7c669d3fc34a7b54a48028fbab955fc135fc0237b4666176709a147ab1b73ee0
Status: Downloaded newer image for python:3.7-slim
 ---> b4e1300134d3
Step 2/9 : RUN apt update && apt install -y libjpeg62-turbo libopenjp2-7 libtiff5 libatlas-base-dev libgl1-mesa-glx
 ---> Running in 1deadaa78a99

WARNING: apt does not have a stable CLI interface. Use with caution in scripts.

Get:1 http://deb.debian.org/debian bullseye InRelease [116 kB]
Get:2 http://security.debian.org/debian-security bullseye-security InRelease [44.1 kB]
Get:3 http://deb.debian.org/debian bullseye-updates InRelease [39.4 kB]
Err:2 http://security.debian.org/debian-security bullseye-security InRelease
  At least one invalid signature was encountered.
Err:1 http://deb.debian.org/debian bullseye InRelease
  At least one invalid signature was encountered.
Err:3 http://deb.debian.org/debian bullseye-updates InRelease
  At least one invalid signature was encountered.
Reading package lists...
W: GPG error: http://security.debian.org/debian-security bullseye-security InRelease: At least one invalid signature was encountered.
E: The repository 'http://security.debian.org/debian-security bullseye-security InRelease' is not signed.
W: GPG error: http://deb.debian.org/debian bullseye InRelease: At least one invalid signature was encountered.
E: The repository 'http://deb.debian.org/debian bullseye InRelease' is not signed.
W: GPG error: http://deb.debian.org/debian bullseye-updates InRelease: At least one invalid signature was encountered.
E: The repository 'http://deb.debian.org/debian bullseye-updates InRelease' is not signed.
The command '/bin/sh -c apt update && apt install -y libjpeg62-turbo libopenjp2-7 libtiff5 libatlas-base-dev libgl1-mesa-glx' returned a non-zero code: 100

Steps to Reproduce

Build a Custom Vision model on customvision.ai
Export this model as a Docker container for ARM (Raspberry Pi 3)
Set up Raspberry Pi using Stretch using the latest image from the Raspbian downloads
Work through the steps in this how to guide to install moby-engine and IoT Edge. You will also need to workaround #5882 by fixing the package list so you can install Moby-engine.
Download the Custom Vision model to the Pi, and build it using this following command from inside the unzipped folder with the docker image inside:

sudo docker build .

Device Information

Host OS [e.g. Ubuntu 18.04, Windows Server IoT 2019]: Raspberry Pi OS Stretch 2019-04-09
Architecture [e.g. amd64, arm32, arm64]: arm32
Container OS [e.g. Linux containers, Windows containers]: Linux

Runtime Versions

Docker/Moby [run docker version]:

Client:
 Version:           3.0.13+azure
 API version:       1.40
 Go version:        go1.13.11
 Git commit:        dd360c7c0de8d9132a3965db6a59d3ae74f43ba7
 Built:             Thu May 28 20:41:25 2020
 OS/Arch:           linux/arm
 Experimental:      false

Server:
 Engine:
  Version:          3.0.13+azure
  API version:      1.40 (minimum version 1.12)
  Go version:       go1.13.11
  Git commit:       77e06fda0c
  Built:            Mon Jun  1 20:22:16 2020
  OS/Arch:          linux/arm
  Experimental:     false
 containerd:
  Version:          v1.2.13
  GitCommit:        7ad184331fa3e55e52b890ea95e65ba581ae3429
 runc:
  Version:          1.0.0-rc10
  GitCommit:        dc9208a3303feef5b3839f4323d9beb36df0a9dd
 docker-init:
  Version:          0.18.0
  GitCommit:        fec3683

Additional Information

This change is described in:

The fix for the Pi is to use a different version of libseccomp2, and a version of Docker greater than 19.0.4.

About this issue

Original URL
State: closed
Created 3 years ago
Comments: 34 (17 by maintainers)

Most upvoted comments

If stretch is EOL for Moby, then are we going to move support to Buster/Bullseye?

Both still have the same problem though. The fix as I understand it is to use back ports of seccomp, and update the Moby-engine to a later version based off Docker 20. If we can get the updated Moby version, then maybe we just need a docs update to refer to the process to install the sec comp back ports (it should really be in docs rather than here so anyone who does the install will know the steps)

jimbobbennett on Dec 10, 2021

@jimbobbennett we’re working on doc updates. They will say to do the following which should already work for 1.2.9. Can you give these a try to let me know the result?

Clean install of bullseye lite, ran an apt update and apt full-upgrade.

Ran the instructions above, rebooted the Pi, and I can build the Docker container successfully! I’ll try running it through IoT Edge tomorrow as it’s later here now.

jimbobbennett on Apr 29, 2022

An animated Gif of Carlton from fresh prince of bel air dancing

jimbobbennett on Mar 31, 2022

@jimbobbennett we are planning to move to Buster / Bullseye and our guidance will be to match the container with the host. In your Buster example above I’d expect it to work if the Buster image python:3.7-slim-buster was used instead of the one based on Bullseye python:3.7-slim.

micahl on Mar 30, 2022

There are up to date builds for buster and bullseye. https://packages.microsoft.com/debian/10/prod/pool/main/m/moby-engine/

cpuguy83 on Dec 10, 2021