warden: Cannot install on MacbookAir M1 processor

Update

I have forked this repository and made it run natively on both Intel and Apple Silicon chips, making it multi-platform. You can find more info at https://github.com/drpayyne/warden-multi-arch. Currently, only the one mainstream version of each service is rebuilt and provided.

I have rebuilt all necessary Dockerfiles as multi-platform Docker images (linux/arm64, linux/amd64) and published them in GitHub Packages.

Warden services

• I have re-used the MailHog Dockerfile and rebuilt it as a multi-platform image. Same for mhmailsend, which MailHog needs. Refer https://github.com/drpayyne/mhsendmail.
• Dnsmasq and sshd are rebuilt as multi-platform images. Refer https://github.com/drpayyne/docker-sshd and https://github.com/drpayyne/docker-dnsmasq-multi-arch.

Magento 2 services

• All PHP images have been switched to Fedora 34 from CentOS 8 after much efforts, as I tried to rebuild the entire PHP 7.4 binary and its extensions from Remi’s repository for arm64 but was not successful as I don’t have much experience in RPM building. Sorry! Refer https://github.com/drpayyne/docker-php.
• The base images for Magepack (Chrome with Node & Puppeteer) is rebuilt as multi-platform images. Refer https://github.com/drpayyne/chrome.
• Varnish 6.5 is rebuilt as a multi-platform image as Varnish now provides RPMs for both architectures.
• Elasticsearch 7.9 is rebuilt as a multi-platform image.
• Nginx 1.16 is rebuilt as a multi-platform image.
• RabbitMQ 3.8 is used directly from DockerHub. (already provides multi-platform images)
• All database containers now use MariaDB 10.4 directly from DockerHub. (already provides multi-platform images)
• Redis 5.0 is used directly from DockerHub. (already provides multi-platform images)

cc/ @fooman @davidalger @amarmureanuarnia

There is certainly room for some improvements on M1 currently, but complete support (i.e. no emulation required) is going to be a long time coming and there is no ETA for that, as many of these issues relate to upstream deps, many of which lack arm64 / aarch64 images and/or rpms. I’m also limited in the time I have to spend building, testing, etc.

• There are no arm64 images for MySQL, Mailhog, or the base image used for Magepack image builds
• There are arm64 images for MariaDB, Nginx, RabbitMQ and Redis, but the build process in here needs amending to pull/build/push with both architectures
• Varnish doesn’t even provide RPMs for arm64 so building a varnish image with native M1 support is not currently possible
• Elasticsearch publishes arm64 images (at least for later versions) but they use tags to denote this instead of multi-platform images as Docker Hub does, so ES specific build process changes will be needed to pull/build/push with both architectures.
• Remi does not build aarch64 RPMs for PHP, and currently has no ETA (or plans really) to do so as he lacks an arm builder and rebuilding PHP for Arm is not exactly a small endeavor (https://forum.remirepo.net/viewtopic.php?pid=11836#p11836) so the PHP-FPM images used here won’t build for arm64 / aarch64 architecture (I tried it, which is how I discovered this)

My honest take is this: Docker Desktop literally just reached “GA” (whatever that means) only this past April. It was in tech preview starting Dec 2020, and that is hardly time to become stable. Developers that know they rely on Docker Desktop racing out to buy M1 macs, many before Docker Desktop was even out of tech preview, will necessarily be accepting some level of pain in exchange for being early adopters. There is a reason the first M1 models Apple has released are not super developer focused, it takes time (a lot of time) for an entire ecosystem to catch up, particularly with complicated things like emulation and virtualization.

Even with native arm64 images (that won’t require QEMU emulation), there are “known issues” such as the following turning up in the release notes for Docker Desktop (this one is from June 6th):

On Apple Silicon in native arm64 containers, older versions of libssl in debian:buster, ubuntu:20.04 and centos:8 will segfault when connected to some TLS servers, for example curl https://dl.yarnpkg.com. The bug is fixed in newer versions of libssl in debian:bullseye, ubuntu:21.04 and fedora:35.

There is list of known issues (that I assume is kept current) here: https://docs.docker.com/docker-for-mac/apple-silicon/

An initial and first step here will be updating the image build system here to build and push multi-platform images for those where upstream dependencies exist to support it, while keeping the other image builds unchanged. This will take time to accomplish (and is something for which I also have no ETA as I cannot make it a priority at the moment)

This project is not maintained anymore. Feel free to migrate to den --> https://github.com/swiftotter/den

den is a fork from warden, providing a faster, maintained version of warden!

@DavidLambauer Just merged and rebuilt the Docker images. Thank you!

Warden package versions are available to browse both on Docker Hub and on GitHub: https://github.com/orgs/wardenenv/packages?repo_name=images

Latest PHP: 8.2 Latest OpenSearch: 2.6

I check for newer versions of all needed softwares semi-often

@amarmureanuarnia Unless I’m mistaken, the error there is related to how Homebrew is installed and not specific to Warden.

Please create a new installation in /opt/homebrew using one of the
"Alternative Installs" from:
  https://docs.brew.sh/Installation
You can migrate your previously installed formula list with:
  brew bundle dump

On that linked installation docs page, you’ll notice Homebrew states on M1 Macs it will install into /opt/homebrew instead of /urs/local and the error above indicates it’s incorrectly installed, why I can’t say, but I would look into that and try again.

If you want to give Warden a shot without needing to fix or change your Homebrew installation, feel free to use the alternate installation method documented here: https://docs.warden.dev/installing.html#alternative-installation

Quick update - I’ve now archived the https://github.com/drpayyne/warden-multi-arch repository with a deprecation notice in favor of migrating back to this repository to avoid confusion.

The fork from @drpayyne works fine for except xdebug, but there is already a pull request. Would be nice if you’d can consider to add the fork as an m1 version for warden or so

Elasticsearch publishes arm64 images (at least for later versions) but they use tags to denote this instead of multi-platform images as Docker Hub does, so ES specific build process changes will be needed to pull/build/push with both architectures.

Elasticsearch has multi-arch images in Docker Hub for versions >=7.8.0. Is this what we need? (ref: https://hub.docker.com/_/elasticsearch?tab=tags&page=1&ordering=name)

There are no arm64 images for MySQL, Mailhog, or the base image used for Magepack image builds

• MariaDB is Magento supported, so can we switch to that? Also, we don’t have any customization in any of our MariaDB Dockerfiles, so is there a reason we have custom builds for that? If it’s better to use the base MariaDB image directly, can we?
• MailHog hasn’t been updated in over a year and here are our options
  • we contact its maintainer and check if it’s on their roadmap (helps with our timelines)
  • if it’s on their roadmap, we wait for them to push an arm build
  • if not
    • we use their Dockerfile and build it on our own for multi-arch considering its dependencies and publish it under the wardenenv Docker Hub namespace
    • we switch to https://hub.docker.com/r/sj26/mailcatcher which is both multi-arch and maintained. Relevant PR in @markshust’s repo by me can be found here: https://github.com/markshust/docker-magento/pull/512. This is a proposal to switch MailHog with MailCatcher.
• There is an open issue (https://github.com/Zenika/alpine-chrome/issues/152) for Magepack’s base image and I’ve checked in to see if a multi-arch image is in their plans.

@davidalger, the above are my thoughts on this and I’m happy to collaborate and contribute to this as required. Please let me know what you think, thanks!

Varnish doesn’t even provide RPMs for arm64 so building a varnish image with native M1 support is not currently possible

Looks like they recently published aarch64 RPMs for 6.0lts, 6.5 and 6.6:

https://packagecloud.io/varnishcache/varnish60lts?filter=rpms https://packagecloud.io/varnishcache/varnish65?filter=rpms https://packagecloud.io/varnishcache/varnish66?filter=rpms

Thanks folks! Appreciate it. 😃 And thanks to you both back for the early testing and the continued support for Warden!

@pawel-snowdog I highly recommended moving to https://github.com/wardenenv/warden/, as @navarr is doing a great job maintaining it. Our team has moved from both Den and the https://github.com/drpayyne/warden-multi-arch project to Warden without any issue.

@pawel-snowdog You can use Warden 0.13 or greater and run it on an M1 mac. Den has since been deprecated in favor of Warden. Not all Den features have been ported over, but it’s being worked on.

You can also give https://github.com/rewardenv/reward a try. It’s Warden rewritten in Golang.

Find the complete documentation here: https://rewardenv.readthedocs.io

@drpayyne That’s exciting! I’m looking forward to getting an M1 Macbook Pro once they’re released (hopefully later this year) and will look into your fork at that time.

Might also be an option to switch the Mailhog image to https://hub.docker.com/r/cd2team/mailhog/tags?page=1&ordering=last_updated

Hei @drpayyne ! Unfortunately, I didn’t had time to try again. As of M1 Mac development, I can tell you that 90% of my work is still done on my old Macbook Pro late 2015 Intel based. I tried a couple of other projects with docker images, and from 4 projects only one worked without the need of doing some customization. Everyone that right now has an M1 Mac should invest time in building images for their project on the new ARM architecture. How I’m seeing it, is that even with the new Macbookpro 16 M2 there will still be missing pieces in the development process. In conclusion I still feel like an pioneer 😃))