gatsby: Build fails, when processing large images
Description
Hello Gatsby-Team! I am working on a designer’s portfolio website with lots of large images. These images are downloaded from a server and processed by Gatsby Image. In dev-mode everything works as expected, but when I try to trigger a build, I get messages like this:
error UNHANDLED REJECTION
Error: Failed to process image /Users/andreasfaust/websites/lisa/frontend/.cache/gatsby-source-filesystem/0cfb27ee68f5a1304c081d57405030ef/fritzlambada-06.jpg
(sharp:1306): GLib-CRITICAL **: 11:25:32.087: g_hash_table_lookup: assertion 'hash_table != NULL' failed
(sharp:1306): GLib-CRITICAL **: 11:25:32.087: g_hash_table_lookup: assertion 'hash_table != NULL' failed
(sharp:1306): GLib-CRITICAL **: 11:25:32.088: g_hash_table_lookup: assertion 'hash_table != NULL' failed
(sharp:1306): GLib-CRITICAL **: 11:25:32.088: g_hash_table_insert_internal: assertion 'hash_table != NULL' failed
(sharp:1306): GLib-CRITICAL **: 11:25:32.088: g_hash_table_lookup: assertion 'hash_table != NULL' failed
./build.sh: line 9: 1306 Segmentation fault: 11 gatsby build
When I reduce the number of images-collections, it starts working, but not necessarily. Image collections seem to work and fail randomly. The images itself are saved-for-web in Photoshop.
To source those images I am using a selfwritten source-plugin called “gatsby-source-custom-api”, which is based on “gatsby-source-filesystem”.
Expected result
Images get downloaded and processed and a build is completed. Maybe the build time is about a couple minutes, but that does not matter.
Actual result
The build fails regularly, eventually works, when I disable certain image collections. The configuration to make it work changes.
Environment
System: OS: macOS 10.14.3 CPU: (8) x64 Intel® Core™ i7-3720QM CPU @ 2.60GHz Shell: 5.5.1 - /usr/local/bin/zsh Binaries: Node: 11.6.0 - /usr/local/bin/node Yarn: 1.15.2 - ~/.yarn/bin/yarn npm: 6.9.0 - /usr/local/bin/npm Languages: Python: 2.7.15 - /usr/local/bin/python Browsers: Chrome: 73.0.3683.103 Firefox: 66.0.2 Safari: 12.0.3 npmGlobalPackages: gatsby-cli: 2.5.4
package.json: “gatsby”: “^2.3.22”, “gatsby-cli”: “^2.5.7”, “gatsby-image”: “^2.0.38”, “gatsby-plugin-sharp”: “^2.0.34”, “gatsby-source-filesystem”: “^2.0.27”,
About this issue
- Original URL
- State: closed
- Created 5 years ago
- Reactions: 3
- Comments: 70 (31 by maintainers)
I solve the problem by following these steps:
Then I install
libpng-dev:And at the end, I run an
npm installand agatsby develop.I think that the problem is related to the operating system not Gatsby, I’m running gatsby from a Ubuntu Server installed as a virtual machine:
@freemh by removing the files and npm install solved the problem for me on macos.
Updated .gitignore file with :
gatsby files
.cache/ public
it worked.
@jasonbarry @i-bsd
I figured out what was causing my issue. I am running Linux Subsystem on Windows 10.
I was unaware that I needed to install Build Tools in order for sharp to work. Details are given here: https://scottspence.com/2018/01/13/wsl-bootstrap/
I hope this helps anyone else facing the same issue.
Agree. Still an issue on FreeBSD. The only way Gatsby is usable is to use drop
gatsby-plugin-sharpback to an old version (2.0.7). This is not ideal or acceptable.This worked for me also, just make sure you delete .cache and node_modules after install and re-run npm install.
@sheelah same for us, we had exactly the same error and jpeginfo found no corrupted images. This error was the reason we moved away from gatsby for now.
Relevant discussion in the sharp repo: https://github.com/lovell/sharp/issues/409#issuecomment-210111143
It looks like it happens when the process is killed before sharp has finished resizing. That would also explain why adding a timeout would fix it (by allowing the resizing to finish).
Apologies if this is unrelated. We’re hitting a similar issue running
gatsby buildafter upgrading to Gatsby 2.9 and above. I’ve tried downgradinggatsby-plugin-sharpto 2.0.7 as mentioned above - no help there.Interestingly, I’ve had some success by injecting a small delay into
gatsby-plugin-sharp’srunJobqueue.Has anyone ever figured out how to fix this for good?
I feel like all I do when I try to build something with Gatsby is fixing these weird errors, deleting node_modules, caches and all sorts of random stuff to get it to work. Very frustrating tbh.
I did the following and it works for now:
npm updatenpm cache clear --forcenpm installI’ve tried a lot of things now i think we’re going to have to rip out all the image processing that’s done by sharp. I cannot get it to work locally
I’m seeing the same error during builds as @harrygreen, with
gatsbyv2.9.11 andgatsby-plugin-sharpv2.1.9. This is cropping up on MacOS as well as Netlify builds.I’ve run into this issue, but only in CI—so assumed it was a Linux issue since I’ve never seen it locally on macOS.
After some back and forth with the developer of sharp, he also recommended setting
sharp.cache(false)ingatsby-node.js, along withsharp.simd(true)as previously discussed.This seems to have resolved the issue, but we’re still toying with some CI settings.
this issue is still relevant, I’m still experiencing this
The same error was happening to me. For some weird reason, I just ran yarn once, and it started working fine.