sharp: Can't compile under Apple Silicon M1 arm64

🎉 I’m happy to report that thanks to the huge effort of the homebrew team and specially @fxcoudert, libvips can be fully installed on Apple Silicon M1 natively with all its dependencies!

Also for the most of the dependencies prebuild binaries “bottles” are available so installation goes really fast

brew info vips

vips: stable 8.10.5
Image processing library
https://github.com/libvips/libvips
/opt/homebrew/Cellar/vips/8.10.5 (167 files, 14.0MB) *
  Built from source on 2020-12-27 at 16:44:20
From: https://github.com/Homebrew/homebrew-core/blob/HEAD/Formula/vips.rb
License: LGPL-2.1-or-later
==> Dependencies
Build: pkg-config ✔
Required: cfitsio ✔, fftw ✔, fontconfig ✔, gettext ✔, giflib ✔, glib ✔, imagemagick ✔, libexif ✔, libgsf ✔, libheif ✔, libimagequant ✔, libmatio ✔, libpng ✔, librsvg ✔, libspng ✔, libtiff ✔, little-cms2 ✔, mozjpeg ✔, openexr ✔, openslide ✔, orc ✔, pango ✔, poppler ✔, webp ✔
==> Analytics
install: 4,198 (30 days), 14,926 (90 days), 42,146 (365 days)
install-on-request: 3,633 (30 days), 12,169 (90 days), 32,332 (365 days)
build-error: 0 (30 days)

brew reinstall vips

==> Downloading https://github.com/libvips/libvips/releases/download/v8.10.5/vips-8.10.5.tar.gz
Already downloaded: /Users/george/Library/Caches/Homebrew/downloads/e8752c98d6977d23909b94c1d3ee31e2c21afda72fa95eeb140bbbbde0add7c8--vips-8.10.5.tar.gz
==> Reinstalling vips 
==> ./configure --prefix=/opt/homebrew/Cellar/vips/8.10.5 --with-magick
==> make install
🍺  /opt/homebrew/Cellar/vips/8.10.5: 167 files, 14.0MB, built in 58 seconds

brew config

HOMEBREW_VERSION: 2.7.0
ORIGIN: https://github.com/Homebrew/brew
HEAD: 993bf95877143afe1cede022083499e0b4fe336c
Last commit: 6 days ago
Core tap ORIGIN: https://github.com/Homebrew/homebrew-core
Core tap HEAD: 27df7ec4599f2aa202586e4d7950afa66e820293
Core tap last commit: 2 hours ago
Core tap branch: master
HOMEBREW_PREFIX: /opt/homebrew
HOMEBREW_CASK_OPTS: []
HOMEBREW_MAKE_JOBS: 8
Homebrew Ruby: 2.6.3 => /System/Library/Frameworks/Ruby.framework/Versions/2.6/usr/bin/ruby
CPU: octa-core 64-bit arm_firestorm_icestorm
Clang: 12.0 build 1200
Git: 2.24.3 => /Applications/Xcode.app/Contents/Developer/usr/bin/git
Curl: 7.64.1 => /usr/bin/curl
macOS: 11.1-arm64
CLT: 12.3.0.0.1.1607026830
Xcode: 12.3
Rosetta 2: false

the same is valid for node and the sharp library now - they can run nicely native on M1 now!

I managed to run sharp thanks to @kenips comment Here’s what I have done:

1. Start with installing Mac ports. You will use ports to install libvips dependencies (follow next step)

2. Install these ports: pkgconfig and glib2. Use this command to install ports: $ sudo port install [port-name]

3. Now depending on the formats you’ll be handling, you’ll have to install the library for each format. Have look at the libvips dependency list and find them on Mac ports. In my case, jpg and png formats are enough, so I installed libjpeg-turbo and libpng

4. Once you’ve installed the libraries, you’re ready to install libvips.

   • Download and unpack the latest libvips release; I used this release.
   • Inside the unpacked folder, run $ ./configure.
   • Then run $ make followed by $ sudo make install

   This should install vips globally. Verify by running: $ which vips

5. When libvips is globally installed, sharp will use the global version. So clean up your node_modules folder and do a fresh install and you’re ready to go 👍

Confirmed from #2467 that this is a non-issue once you have libvips install globally. I wasn’t able to install it via brew, but downloaded the tarball and manually compiled fine, with the caveat that I had to manually specify zlib’s location: PKG_CONFIG_PATH=/opt/homebrew/Cellar/zlib/1.2.11/lib/pkgconfig ./configure. Once installed globally sharp works fine.

I’m interested in this topic as well, did anyone figure a way to get sharp running under Apple silicon m1?

@gpetrov yes my procedure is very similar to @medhoover, however using homebrew:

1. Install homebrew arm. I used the steps here.
2. Install pkg-config, zlib normally with brew install
3. Same as above, install any libraries that you need for each image format. I have jpeg, libpng, libtiff, webp installed.
4. Exactly same as @medhoover’s steps above, except for configure I had to specify the zlib path with PKG_CONFIG_PATH=/opt/homebrew/Cellar/zlib/1.2.11/lib/pkgconfig ./configure.

I have installed a few packages prior to installing libvips, so there might be additional packages that I’ve missed for a mint install, so YMMV 😃.

Commit https://github.com/lovell/sharp/commit/4264c0577e8a0e74164a768bcd17e26d9593cd59, part of the forthcoming sharp v0.27.2, should improve the experience for those using Apple M1 devices.

• For Rosetta x64, it prevents the use of a globally-installed ARM64 libvips as this conflict with the use of x64 Node.js
• For ARM64, when a globally-installed libvips is not found, it will highlight the need to install via homebrew

From sharp v0.28.0 onwards, cross-compiled libvips binaries will be provided for darwin-arm64 - thank you @abarisain for helping with this.

As soon as GitHub Actions or a similar CI service makes the M1 freely available to open source projects then natively compiled and tested prebuilt darwin-arm64 binaries will also be provided for sharp.

Latest lts node isn’t arm64: building libvips with arm homebrew will produce a library that rosetta’d node can’t use. That’s what the linker is telling you: you’re trying to link an arm64 libvips within a x86_64 library.

I guess that sharp could implement an architecture check before trying to use a system libvips to help reduce how often this issue gets reported from people not familiar with multiple architectures and the resulting errors.

Lets summarize:

• if you’re running a native arm64 node (15 and higher), you need to install libvips using arm homebrew. Note that macOS 11.2 currently breaks wasm in node 15: no fixed stable version has been tagged yet.

• if you’re running an x64 node (14 and lower), do not install libvips using homebrew. If you already did it, remove it. Note that I experienced rosetta bugs when using libvips, ending up in segfaults.

• if you need to have two node installs (one x64 and one arm64), you’re out of luck as brew will NOT build a “fat” libvips, and sharp will try to link with the system library and fail. You could manually make a fat libvips by using brew to compile both architectures and use “lipo” to make a fat libvips with both architectures, but this requires a little bit of knowledge of macOS’ way of making hybrid binaries.

To know what architecture your node uses, open up a terminal, fire up node and input process.arch

I have very similar problem. npm I / yarn goes successfully(I got node 15 (arm64) and native libvips installed with brew ) but when launching Gatsby. I get

Error: Something went wrong installing the "sharp" module dlopen(/Users/.../node_modules/sharp/build/Release/sharp.node, 1): Symbol not found: __ZTVN 4vips7VOptionE

There’s a new (hopefully temporary) section in the sharp documentation directing Apple M1 users to install libvips via homebrew - please see https://sharp.pixelplumbing.com/install#apple-m1

Work is in progress to provide prebuilt, cross-compiled (and therefore possibly untested) libvips binaries for the “darwin-arm64v8” platform - please see https://github.com/lovell/sharp-libvips/pull/74

sharp v0.28.0 is now available and provides prebuilt (cross-compiled) libvips binaries for Apple M1 (darwin-arm64v8) so you no longer need to install vips via Homebrew.

https://sharp.pixelplumbing.com/install#apple-m1

Xcode and Python are still required as sharp must be built from source. As soon as GitHub Actions or a similar CI service makes M1-based machines freely available to open source projects then natively compiled and tested prebuilt darwin-arm64 binaries will also be provided for sharp.

Like @Dominoo I have exactly same issue with Gatsby.

Using nvm you will only run node only under rosetta and not the whole terminal

I accidentally came to this problem as I am trying to install cordova-res globally using npm. I tried some mentioned solutions up there and when I try to install it again npm i -g cordova-res I get the same error.

I am using mac M1 and node 14.x

Appreciate your help.

Sorry for getting off topic again!. There are still issues when installing node < 15, on arm… Please refer to this issue for example https://github.com/nvm-sh/nvm/issues/2350

When should we expect a fix for this ?

use an ubuntu 20.10 (maybe 20.04 works too) base, there are precompiled binaries for it

Try brew install libvips first and then it should work.

I had all these issues and after all the fixes above it still didn’t work. I have now figured out the issue I was facing: I was running x86_64 brew to install the latest vips package. After removing x86_64 vips, I ran brew natively to install vips then after clearing my node_modules/lock file, sharp installed perfectly.

I was able to get it to work in the following ways with Node.js 15.3.0. Thanks @medhoover @kenips !

brew install pkg-config glib zlib
brew install libjpeg-turbo libpng webp

# in the libvips source folder
PKG_CONFIG_PATH=/opt/homebrew/Cellar/zlib/1.2.11/lib/pkgconfig ./configure
make
sudo make install

That’s what has been done for libvips.

Cross compiling the node library looks way more difficult. It might not be, but I didn’t manage to do it.

Missing those prebuilts isn’t a big deal that said, you most likely rely on dev tools being installed for other stuff (like homebrew itself, or any node gyp module)

As soon as GitHub Actions or a similar CI service makes M1-based machines freely available to open source projects then natively compiled and tested prebuilt darwin-arm64 binaries will also be provided for sharp.

you can build mac arm on intel, so infrastructure change is not really needed. Google Chromium describes it well: https://chromium.googlesource.com/chromium/src/+/master/docs/mac_arm64.md

@EMedina24 the “workaround” is to install vips via brew, or from source directly. How to do that is outlined in the thread above, for example, see https://github.com/lovell/sharp/issues/2460#issuecomment-768558230.

Once that’s installed npm i sharp will work fine. I just did it myself and it finished a few minutes ago (unlike what is implied here though, it took about 30mins to install on my machine).

@abarisain thanks for the tip about process.arch. Up until now I had been using file $(which node) which tells you the type of executable you get by running node in the terminal, your tip is more reliable though.

any workaround for this yet. i cant get gatsby or gridsome to work without Sharp

Nothing strange here: homebrew doesn’t build fat (intel + arm) libraries and therefore you will run into issues as soon as install vips using brew

All of this will be solved when sharp ships an arm prebuilt, hang on tight!

@kalu-op For linuxmusl-arm64 prebuilds, please see https://github.com/lovell/sharp-libvips/issues/72

If running under rosetta, you don’t need to install vips yourself. Sharp will pull a prebuilt library

That said, I encounter crashes when running libvips under rosetta. If you do, Best course of action is unfortunately to wait for a patched nodejs.

Hi ! This was working fine on M1 with node 15.

But since the latest Big Sur release (11.2) node 15 is not working and we have to go back to use node 14 with rosetta (see https://github.com/nodejs/node/issues/37061)

and so i have once again this error when i try to launch a node program who use sharp (gatsbyjs) under node 14 with rosetta

Error: Something went wrong installing the "sharp" module
  dlopen(/****/node_modules/sharp  /build/Release/sharp.node, 1): Symbol not found: __ZTVN4vips7VOptionE

I tried to install a x86_64 version of vips with the x86_64 version of brew (https://soffes.blog/homebrew-on-apple-silicon) but this don’t seems to work (i don’t know which version of vips is used).

If someone has an idea ! Thanks

ok, so I have it working on my dev machine but I can’t ship that cause my customers are not going to have vips installed by brew?

I installed globally with brew install libvips and it works fine now, npm install works as usual using the brew-installed version.

@Dominoo Thanks for your tip. Actually how could you remove othet x86 deps? brew uninstall *?

Unfortunately I still can’t get it to install on my M1. I guess I am doing something wrong, but I installed libvips via homebrew as mentioned by @gpetrov (thanks for the hint). But when running npm install it ignores the global libvips installation.

I dug into the code node_modules/sharp/lib/libvips.js and there in the function useGlobalLibvips simply logged the minimum and global libvips version.

const useGlobalLibvips = function () {
  if (Boolean(env.SHARP_IGNORE_GLOBAL_LIBVIPS) === true) {
    return false;
  }

  const globalVipsVersion = globalLibvipsVersion();
  
  console.log('globalVipsVersion: ' + globalVipsVersion);
  console.log('min: ' + minimumLibvipsVersion);
  
  return !!globalVipsVersion && /* istanbul ignore next */
    semver.gte(globalVipsVersion, minimumLibvipsVersion);
};

When I then run npm run install from within the node_modules/sharp directory I get:

globalVipsVersion: 8.10.2
min:8.10.5

Am I doing something wrong or has someone else run into this problem as well?

I have also fixed my mach-o, but wrong architecture issue. I made 2 changes and therefore don’t know, what exactly helped me:

1. Disable Rosetta terminal for node
2. Reinstall node with nvm (may be was using web version before)

Thanks a lot @cooknpop, @gpetrov and all others!

@iksent I succeed to solve the issue with mach-o, but wrong architecture (even if not the cleanest solution). As my mac was new, I did a fresh clean-up of all my files and installed everything from scratch (mainly homebrew, nvm/node) making sure the architecture chosen was arm and not x86. Probably the issue was that at the beginning I downloaded node directly from the web, and this version is in x86 format, so the conflicts raised. I guess there is a simpler solution (I didn’t have the chance to check sharp folder from ./npm as I already had started the clean-up process at this point, another nightmare BTW).

Said that, my suggestion: Install node directly with nvm from scratch in arm version and not from the web, with no other nodejs previously installed in the computer.

I hope it will help.

Your node_modules contain an old Mac x86 binary. You can delete them and do a fresh npm install

Thank you @medhoover and @kenips as well as @gpetrov for working on this. Just got myself a Mac Mini M1 and am struggling to get a project running.

Just wanted to add to @medhoover’s comment: the port name of pkg-config is pkgconfig. The port install command should be sudo port install pkgconfig.