dvc.org: `master` builds are failing

As noted in https://github.com/iterative/dvc.org/pull/2377#issuecomment-824484545 (and probably earlier), the build script stopped working at some point (first failed build) in master. This means we can’t deploy anything to production. The error is

       [4/4] Building fresh packages...
       error /tmp/build_7f109913/node_modules/sharp: Command failed.
       Exit code: 1
       Command: (node install/libvips && node install/dll-copy && prebuild-install) || (node-gyp rebuild && node install/dll-copy)

CI tests are all passing though. This only happens with build and on ~~prod~~ our cloud, it seems.

About this issue

Original URL
State: closed
Created 3 years ago
Comments: 17 (17 by maintainers)

Most upvoted comments

I pinned all our gatsby sharp-related plugins to an old version (quite old, now!) because our weird hand-rolled S3 caching mechanism depends on the simple public-dependent image caching that the gatsby image plugins used before the Jobs API V2 was introduced. For some reason, after that update the cache would always be invalid no matter what files we preserve.

The hand-rolled S3 cache script is in scripts/deploy-with-s3.js, which is what is used to invoke the site build in the heroku-postbuild npm script in package.json

I’ve only made a couple of early attempts to use, at the time, the latest image plugins to see if caching works. It didn’t then, but a lot has happened, especially since the Gatsby 2 -> 3 update which I still haven’t made an attempt after. It may be worth rolling an attempt to fix this into an attempt to use Gatsby 3, as the latest image plugins that use updated sharp (or no sharp at all, I don’t remember the specifics off the top of my head) are tied to Gatsby 3.

rogermparent on Apr 25, 2021

@julieg18 could you please give this one a try? it’s a p0 for us.

shcheklein on Apr 23, 2021

Good to know, I haven’t tried a local build yet but I suspect it’ll work there too.

I was correct, builds succeed on my Linux box.

rogermparent on Apr 26, 2021

The gatsby-*-sharp package did use the sharp package so I did upgrade that one when I was trying to fix the build. But since that probably breaks caching, looks like we’ll need to try something else.

We don’t necessarily have to ditch your solution just yet, if you’ve fixed builds we can at least get an emergency patch in that makes things work while we fix caching. That said, to me this immediate problem does look more like a build environment issue and we can get a more complete solution from that avenue.

rogermparent on Apr 26, 2021

From the looks of the script and what you tell here, a major upgrade may take too much time.

BTW, my local Ubuntu 20.04 installation also builds fine. ✅

@rogermparent

iesahin on Apr 25, 2021

FYI, sharp module is why I had to give up installing the site on mac M1. @rogermparent has said that some customization was applied to image generation code and we can’t directly upgrade to higher versions. @julieg18

@shcheklein @jorgeorpinel

iesahin on Apr 25, 2021

@julieg18 you have it now!

shcheklein on Apr 23, 2021

@julieg18 feel free to do any experiments, package updates/downgrades, etc- don’t worry about breaking something.

shcheklein on Apr 23, 2021

@julieg18 feel free to try it out and open a PR with a branch directly on this repo (not from a fork) so that an automatic deployment is triggered on our cloud. If the build works there it should work on prod.

jorgeorpinel on Apr 23, 2021