coreos-assembler: Supporting aarch64/ppc64le/s390x/x86_64 in the future when `/dev/kvm` is not available

We need a solution for continuing to use coreos-assembler across all supported architectures where /dev/kvm is not available, in order to support our strongly coupled build+test model that we have adopted.

In the near future, /dev/kvm will not be available for newer ppc64le platforms and will break our ability to produce CoreOS-based builds on that platform. We may be able to continue to limp along on older versions of ppc64le platforms, but we will eventually reach a point in time where that will no longer be possible.

How will we be able to continue to support our opinionated, container-based build+test model using coreos-assembler in that future?

About this issue

Original URL
State: open
Created 2 years ago
Reactions: 1
Comments: 21 (17 by maintainers)

Most upvoted comments

The Image Builder team would be very happy to look into collaborating on building CoreOS images through our service in api.openshift.com, if that’s something you’d be interested in adopting.

The service is currently used to build RHEL for Edge and (soon) Fedora IoT, as well as all the RHEL cloud and virt images. We aspire to be able to build all Fedora/CentOS/RHEL, so CoreOS is obviously a natural candidate.

The benefits that come to mind would be:

you would no longer need to maintain any build infrastructure of your own (you’d still need to orchestrate builds, but wouldn’t need to do the actual building so wouldn’t need to worry about IBM workers etc)
we would use the same tooling/infrastructure to build all OSTree-based artefacts, ensuring feature parity of the tools
hopefully we’d be able to collaborate more closely, avoid duplicated effort and move towards uniformity where it makes sense of both OSTree and rpm-based Fedora/CentOS/RHEL.

What would need to happen:

we would need to figure out what API extensions would be necessary for the service to be useable to you
someone would need to do the work to implement any gaps

Potential challenges:

your current developer workflow might be quite different to how we currently develop image definitions, so I don’t know how much work it would be to bridge this gap.
we would need to figure out when we would all have the time to schedule all the work.

It is worth mentioning that osbuild can be made to work in (privileged) containers, which is how we develop locally, as I know that was a question in the past.

teg on Apr 5, 2022

These are the messages that it reported during “cosa build”

Just from cosa build? You’re not also doing cosa kola ...?

2022-07-22T18:00:35Z kolet: Found content in /etc/sysconfig/network-scripts

This seems like an unrelated bug…has this test been passing in current ppc64le pipelines?

it looks like the problem in the output above is about a QEMU feature that does not work anymore on new Power.

AFAICS the output from qemu here is warnings, not fatal errors. If we got this far, ISTM we must have generated a build. So the next question is around which tests run - and how many of those tests to actually run in qemu versus delegating to powervs.

cgwalters on Jul 26, 2022

So osbuild/Image Builder already supports all of this functionality. Maybe it’s a good time to consider how coreos-assembler may be able to make use of that infrastructure.

nullr0ute on Apr 1, 2022