gatsby: updating gatsby-plugin-sharp from 2.8.0 to 2.9.0 breaks build

I found the issue to be multiple versions of sharp. If I upgraded

• gatsby-plugin-manifest (to 2.10.0)
• gatsby-plugin-sharp (to 2.12.0)
• gatsby-transformer-sharp (to 2.10.0)

Then when running yarn why sharp I only had sharp@0.27.0 and things seem to work as expected.

@vladar In gatsby-config.js I changed 'gatsby-plugin-sharp' to:

{ resolve: 'gatsby-plugin-sharp', options: { failOnError: false } }

But I still get the same error when using "gatsby-plugin-sharp": "2.10.0".

Having the same issue

EDIT: I added this in package.json. This fixed the problem for me:

"resolutions": {
    "sharp": "0.28.1"
  },

Uups. Suddenly the following error breaks the build again:

alexa@PCA C:\Users\alexa\apf2                                                                                                                                  
$ yarn build                                                                                                                                                   
yarn run v1.22.5                                                                                                                                               
$ gatsby clean && gatsby build                                                                                                                                 
info Deleting .cache, public, C:\Users\alexa\apf2\node_modules\.cache\babel-loader, C:\Users\alexa\apf2\node_modules\.cache\terser-webpack-plugin              
info Successfully deleted directories                                                                                                                          
                                                                                                                                                               
success open and validate gatsby-configs - 0.104s                                                                                                              
success load plugins - 1.035s                                                                                                                                  
success onPreInit - 0.038s                                                                                                                                     
success delete html and css files from previous builds - 0.003s                                                                                                
success initialize cache - 0.005s                                                                                                                              
success copy gatsby files - 0.425s                                                                                                                             
success onPreBootstrap - 0.021s                                                                                                                                
success createSchemaCustomization - 0.046s                                                                                                                     
success Checking for changed pages - 0.002s                                                                                                                    
success source and transform nodes - 0.259s                                                                                                                    
success building schema - 0.363s                                                                                                                               
info Total nodes: 262, SitePage nodes: 38 (use --verbose for breakdown)                                                                                        
success createPages - 0.054s                                                                                                                                   
success Checking for changed pages - 0.001s                                                                                                                    
success createPagesStatefully - 0.076s                                                                                                                         
success update schema - 0.017s                                                                                                                                 
success onPreExtractQueries - 0.001s                                                                                                                           
success extract queries from components - 2.593s                                                                                                               
success write out redirect data - 0.002s                                                                                                                       
success Build manifest and related icons - 0.754s                                                                                                              
success onPostBootstrap - 0.762s                                                                                                                               
info bootstrap finished - 8.410s                                                                                                                               
success run static queries - 0.024s - 1/1 42.07/s                                                                                                              
warn code block or inline code language not specified in markdown. applying generic code block                                                                 
                                                                                                                                                               
 ERROR #gatsby-transformer-remark_gatsby-plugin-sharp-20000                                                                                                    
                                                                                                                                                               
Failed to process image C:/Users/alexa/apf2/src/benutzerDoku/ekPlanen/ekplanen_global.png.                                                                     
It is probably corrupt, so please try replacing it.  If it still fails, please open an issue with the image attached.                                          
                                                                                                                                                               
                                                                                                                                                               
                                                                                                                                                               
  Error: pngload: non-recoverable state                                                                                                                        
  vips2png: unable to write to target target                                                                                                                   
                                                                                                                                                               
not finished run page queries - 1.246s                                                                                                                         
not finished Generating image thumbnails - 1.137s                                                                                                              
                                                                                                                                                               
error Command failed with exit code 1.                                                                                                                         
info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command.                                                                           

Here the result from gatsby info --clipboard:

$ yarn gatsby info --clipboard                                                                              
yarn run v1.22.5                                                                                            
$ C:\Users\alexa\apf2\node_modules\.bin\gatsby info --clipboard                                             
                                                                                                            
  System:                                                                                                   
    OS: Windows 10 10.0.19042                                                                               
    CPU: (16) x64 Intel(R) Core(TM) i9-9900K CPU @ 3.60GHz                                                  
  Binaries:                                                                                                 
    Node: 15.0.1 - ~\AppData\Local\Temp\yarn--1608657460259-0.46723258413605295\node.CMD                    
    Yarn: 1.22.5 - ~\AppData\Local\Temp\yarn--1608657460259-0.46723258413605295\yarn.CMD                    
    npm: 7.0.3 - C:\Program Files\nodejs\npm.CMD                                                            
  Languages:                                                                                                
    Python: 2.7.17                                                                                          
  Browsers:                                                                                                 
    Chrome: 87.0.4280.88                                                                                    
    Edge: Spartan (44.19041.423.0), Chromium (87.0.664.66)                                                  
  npmPackages:                                                                                              
    gatsby: 2.29.1 => 2.29.1                                                                                
    gatsby-cli: 2.16.1 => 2.16.1                                                                            
    gatsby-image: 2.8.0 => 2.8.0                                                                            
    gatsby-plugin-create-client-paths: 2.7.0 => 2.7.0                                                       
    gatsby-plugin-eslint: 2.0.8 => 2.0.8                                                                    
    gatsby-plugin-manifest: 2.9.0 => 2.9.0                                                                  
    gatsby-plugin-offline: 3.7.0 => 3.7.0                                                                   
    gatsby-plugin-react-helmet: 3.7.0 => 3.7.0                                                              
    gatsby-plugin-sharp: 2.11.1 => 2.11.1                                                                   
    gatsby-plugin-styled-components: 3.7.0 => 3.7.0                                                         
    gatsby-plugin-typography: 2.9.0 => 2.9.0                                                                
    gatsby-remark-autolink-headers: 2.8.0 => 2.8.0                                                          
    gatsby-remark-copy-linked-files: 2.7.0 => 2.7.0                                                         
    gatsby-remark-embed-video: 3.0.10 => 3.0.10                                                             
    gatsby-remark-emojis: 0.4.3 => 0.4.3                                                                    
    gatsby-remark-external-links: 0.0.4 => 0.0.4                                                            
    gatsby-remark-images: 3.8.0 => 3.8.0                                                                    
    gatsby-remark-images-medium-zoom: 1.7.0 => 1.7.0                                                        
    gatsby-remark-numbered-footnotes: 1.0.1 => 1.0.1                                                        
    gatsby-remark-prismjs: 3.10.0 => 3.10.0                                                                 
    gatsby-source-filesystem: 2.8.0 => 2.8.0                                                                
    gatsby-transformer-remark: 2.13.0 => 2.13.0                                                             
    gatsby-transformer-sharp: 2.9.0 => 2.9.0                                                                

and here an excerp from my gatsby-config:

{ resolve: 'gatsby-plugin-sharp', options: { failOnError: false } }

I’ve created an internal ticket to catch these errors and log a more friendly message along the lines of “Could not process the image ‘foo.png’ because it is corrupt”

Also, gatsby-plugin-sharp supports failOnError option:

https://www.gatsbyjs.com/plugins/gatsby-plugin-sharp/#allow-build-to-continue-on-image-processing-error

Was introduced in #27345

I have just encountered the same problem (on Mac OS Cataline - pre M1), combination of few things above worked for me, it might be that only one of those three things was required, nevertheless at this point both gatsby develop and `gatsby build

1. switching to gatsby-plugin-sharp@3.1.2

2. adding to package.json

    "sharp": "0.28.1"
  },

1. Used yarn instead of npm
2. Added this configuration in gatsby-config.js

{
      resolve: "gatsby-plugin-sharp",
      options: { failOnError: false }
    }, 

same issue on m1 mac, adding resolutions did not resolve.

When updating gatsby-plugin-sharp from v2.8.0 to 2.9.0, I would expect:

• my app-build not to completely break. That is something I would only expect from a major version change
• me not needing to have to recreate potentially hundreds or thousands of images that so far have worked perfectly in order to be able to keep gatsby up to date. The thing is: This is simply the first image that causes the build to fail. Maybe all do that

The best we can do it to not make the whole build fail if there is a corrupt image

But that is exactly what is happening. I understand that at least one of the images used on my project are technically “corrupted”. But that should not stop the project from building.

The image would still be missing. Actually processing a corrupt image would require upstream changes, so is out of our control

Actually, the weird thing is that I have managed to build gatsby with one of the recent gatsby-plugin-sharp versions. And when that worked, the images kept working as well. But then suddenly the build started failing again. In the meantime I had changed nothing at all.

So the image does show when using gatsby-plugin-sharp version 2.8.0 and also sometimes when building with a higher version.

For those in this thread, make sure you have the following packages all on the same version:

• gatsby-plugin-manifest
• gatsby-plugin-sharp
• gatsby-transformer-sharp

My issue was that I had gatsby-transformer-sharp + gatsby-plugin-sharp at the latest, but I didn’t update gatsby-plugin-manifest. After updating gatsby-plugin-manifest i no longer had the build error. I hope this helps someone ❤️

That’s the reason. It doesn’t support AVIF until, I think 0.27. What happens if you remove it now?

I found that the failOnError option is not passed to sharp() for the creation of Base64 images. I opened a PR to fix this issue: https://github.com/gatsbyjs/gatsby/pull/29254

@barbalex I agree, so I’ll have a look at this. I would expect that a corrupt image would warn but continue building.

@ascorbic To really answer your question explicitly: I realize you can’t influence the fact that so far working images are now technically corrupt. But I expect the gatsby build to still work (while probably telling me about the corrupt images as it does quite nicely).

What would you do if you have a Site where users can upload images. So you now have thousands of them, without a lot of control over the pre-processing of the images the users upload. Suddenly, when updating one of dozens of dependencies of gatsby, your site build breaks and can never be built again.

What now?

@mlenser Thanks for helping.

$ yarn why sharp
yarn why v1.22.5
[1/4] Why do we have the module "sharp"...?
[2/4] Initialising dependency graph...
[3/4] Finding dependency...
[4/4] Calculating file sizes...
=> Found "sharp@0.27.0"
info Reasons this module exists
   - "gatsby-plugin-manifest" depends on it
   - Hoisted from "gatsby-plugin-manifest#sharp"
   - Hoisted from "gatsby-plugin-sharp#sharp"
   - Hoisted from "gatsby-transformer-sharp#sharp"
info Disk size without dependencies: "68.21MB"
info Disk size with unique dependencies: "69.51MB"
info Disk size with transitive dependencies: "71.95MB"
info Number of shared dependencies: 47
Done in 0.88s.

Unfortunately this does not seem to be the cause in my project.

Thanks, @mlenser! I was dealing with the same issue, and thanks to you, my gatsby site is working within only a few minutes of searching the web.

So that is just the first image that breaks the build. I would probably have to replace lots of images to get my build to work again 😢

I wonder what “corrupt” exactly means.

This image contains an ICC (colour) profile with an invalid checksum.

$ pngcheck ekplanen_ap.png 
ekplanen_ap.png  CRC error in chunk iCCP (computed 66723ff5, expected 62c14516)

The venerable libpng, as used by Chrome, ignores this error.

As of v0.26.0, sharp has started to provide the more modern libspng, hence the need to now explicitly confirm you are willing to accept/trust this image via failOnError: false.

Thanks for the report, I did run it locally and confirmed I see same result locally.

Only thing to keep in mind that repo as it as right now have mix of incompatible sharp versions:

➜  i28203 git:(master) yarn why sharp
yarn why v1.22.10
[1/4] 🤔  Why do we have the module "sharp"...?
[2/4] 🚚  Initialising dependency graph...
[3/4] 🔍  Finding dependency...
[4/4] 🚡  Calculating file sizes...
=> Found "sharp@0.26.3"
info Has been hoisted to "sharp"
info Reasons this module exists
   - Hoisted from "gatsby-plugin-manifest#sharp"
   - Hoisted from "gatsby-transformer-sharp#sharp"
info Disk size without dependencies: "25.43MB"
info Disk size with unique dependencies: "26.81MB"
info Disk size with transitive dependencies: "29.48MB"
info Number of shared dependencies: 54
=> Found "gatsby-plugin-sharp#sharp@0.25.4"
info This module exists because "gatsby-plugin-sharp" depends on it.
info Disk size without dependencies: "29.48MB"
info Disk size with unique dependencies: "31.02MB"
info Disk size with transitive dependencies: "33.94MB"
info Number of shared dependencies: 58

Which result in

success open and validate gatsby-configs - 0.036s
⠁ dyld: lazy symbol binding failed: Symbol not found: _g_once_impl
  Referenced from: /Users/misiek/test/i28203/node_modules/sharp/build/Release/sharp.node
  Expected in: /Users/misiek/test/i28203/node_modules/gatsby-plugin-sharp/node_modules/sharp/build/Release/../../vendor/lib/libvips.42.dylib

dyld: Symbol not found: _g_once_impl
  Referenced from: /Users/misiek/test/i28203/node_modules/sharp/build/Release/sharp.node
  Expected in: /Users/misiek/test/i28203/node_modules/gatsby-plugin-sharp/node_modules/sharp/build/Release/../../vendor/lib/libvips.42.dylib

/bin/sh: line 1: 45613 Killed: 9               gatsby build
error Command failed with exit code 137.

Crash almost immediately (at least on MacOS) - using yarn resolutions in package.json to set sharp version to either 0.25.4 (it works) or 0.26.3 (it crashes) was my way around this.

It feels like issue is in sharp package - given that toggling between sharp version and not even touching gatsby-plugin-sharp version cause the failure / successful builds.

Some initial details - here is example image data and arguments we pass to sharp where it fails (I’m not saying it’s faulty image - it’s just a way to be able to do minimal reproduction we could report in sharp repository):

ERROR {
  file: {
    id: '3e170652-32f4-59b6-ad85-4d7a4e836841',
    children: [ '1c940b52-b428-5f8c-92b8-b237dd0ccd5e' ],
    parent: null,
    internal: {
      contentDigest: 'db413ad7ad34df66d973d41a2829073b',
      type: 'File',
      mediaType: 'image/png',
      description: 'File "src/benutzerDoku/ekPlanen/ekplanen_global.png"',
      counter: 147,
      owner: 'gatsby-source-filesystem'
    },
    sourceInstanceName: 'benutzerDoku-pages',
    relativePath: 'ekPlanen/ekplanen_global.png',
    extension: 'png',
    prettySize: '13.7 kB',
    modifiedTime: '2020-11-20T11:54:51.702Z',
    accessTime: '2020-11-20T12:16:31.549Z',
    changeTime: '2020-11-20T11:54:51.702Z',
    birthTime: '2020-11-20T11:54:51.702Z',
    root: '/',
    dir: '/Users/misiek/test/i28203/src/benutzerDoku/ekPlanen',
    base: 'ekplanen_global.png',
    ext: '.png',
    name: 'ekplanen_global',
    absolutePath: '/Users/misiek/test/i28203/src/benutzerDoku/ekPlanen/ekplanen_global.png',
    relativeDirectory: 'ekPlanen',
    dev: 16777221,
    mode: 33188,
    nlink: 1,
    uid: 501,
    rdev: 0,
    blksize: 4096,
    ino: 205212343,
    size: 13714,
    blocks: 32,
    atimeMs: 1605874591548.6438,
    mtimeMs: 1605873291702.1418,
    ctimeMs: 1605873291702.1418,
    birthtimeMs: 1605873291702.044,
    atime: '2020-11-20T12:16:31.549Z',
    mtime: '2020-11-20T11:54:51.702Z',
    ctime: '2020-11-20T11:54:51.702Z',
    birthtime: '2020-11-20T11:54:51.702Z'
  },
  args: {
    plugins: [],
    maxWidth: 1500,
    wrapperStyle: 'margin-left: 0;',
    linkImagesToOriginal: false,
    showCaptions: false,
    markdownCaptions: false,
    sizeByPixelDensity: false,
    backgroundColor: 'white',
    quality: 50,
    withWebp: false,
    tracedSVG: false,
    loading: 'lazy',
    disableBgImageOnAlpha: false,
    disableBgImage: false,
    pathPrefix: ''
  }
}

Those are arguments we pass to fluid function in https://github.com/gatsbyjs/gatsby/blob/fe18a18812276c19c2bef654f08039f1037113f7/packages/gatsby-remark-images/src/index.js#L151-L156