accelerated-container-image: Userspace conversion fails on large image (48GB)

What happened in your environment?

Trying to convert a large image (48GB) in the userspace:

/opt/overlaybd/snapshotter/convertor -r my.registry.com/project/repo -u user:password -i 0.1 --overlaybd 0.1_obd

The conversion always fails with an error ERRO[2451] failed to build overlaybd: failed to convert layer 17: failed to overlaybd-apply: 2023/09/11 15:24:27|INFO |th=000055B14C24B120|/src/build/_deps/photon-src/io/epoll.cpp:289|new_epoll_engine:Init event engine: epoll.

Output:

INFO[0016] layer 15 uploaded
INFO[0122] downloaded layer 17
INFO[1225] downloaded layer 16
INFO[2139] layer 16 committed, uuid: c0d9066d-6668-3d07-c7b7-17aa5d37f7be, parent uuid: 9fcac1fb-2ab9-586c-7039-782bfc7790ee
INFO[2141] layer 16 converted
ERRO[2451] failed to build overlaybd: failed to convert layer 17: failed to overlaybd-apply: 2023/09/11 15:24:27|INFO |th=000055B14C24B120|/src/build/_deps/photon-src/io/epoll.cpp:289|new_epoll_engine:Init event engine: epoll
2023/09/11 15:24:27|INFO |th=000055B14C24B120|/src/build/_deps/photon-src/io/signal.cpp:265|sync_signal_init:signalfd initialized
2023/09/11 15:24:27|INFO |th=000055B14C24B120|/src/build/_deps/photon-src/io/epoll.cpp:289|new_epoll_engine:Init event engine: epoll
2023/09/11 15:24:27|INFO |th=000055B14C24B120|/src/src/image_service.cpp:169|read_global_config_and_set:using config /etc/overlaybd/overlaybd.json
2023/09/11 15:24:27|INFO |th=000055B14C24B120|/src/src/image_service.cpp:183|read_global_config_and_set:set audit_path:/var/log/overlaybd-audit.log
2023/09/11 15:24:27|INFO |th=000055B14C24B120|/src/src/image_service.cpp:195|read_global_config_and_set:[global_conf.logConfig().logPath()=/var/log/overlaybd.log]
2023/09/11 15:24:27|INFO |th=000055B14C24B120|/src/src/image_service.cpp:209|read_global_config_and_set:set log_level: 1
2023/09/11 15:24:27|INFO |th=000055B14C24B120|/src/src/image_service.cpp:212|read_global_config_and_set:set log_path: /var/log/overlaybd.log, log_size: 10485760, log_num: 3
failed to extract
: exit status 255

The layer 17 mentioned in the error is the largest layer of the image which is ~45GB.

{
  "schemaVersion": 2,
  "mediaType": "application/vnd.docker.distribution.manifest.v2+json",
  "config": {
    "mediaType": "application/vnd.docker.container.image.v1+json",
    "size": 10904,
    "digest": "sha256:37aacfb983164414c15b13feff9dd015f502f1f498c26ba5f3f1fbfa9a27d6c3"
  },
  "layers": [
    .........
    {
      "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
      "size": 46814098000,
      "digest": "sha256:f4e8336fc6f74e49e99d3de5a6de2dd3e7093cb15fc705ba17f0189aaa5d204c"
    },
    {
      "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
      "size": 5084181896,
      "digest": "sha256:76775a8b477ab2c7283cac2d570756525ec86a1ba3e0cb9a91fd27b30ab1ab86"
    },
  ]
}

What did you expect to happen?

Conversion should succeed.

How can we reproduce it?

I haven’t found a way to repro it with a random image I built with a 45GB layer.

What is the version of your Accelerated Container Image?

0.6.7

What is your OS environment?

Ubuntu 20.04

Are you willing to submit PRs to fix it?

• Yes, I am willing to fix it.