astro-imagetools: Abnormally high resource usage and slow builds

I’m noticing high CPU and RAM usage when building using version >0.5.3.

I’ve tested most versions from 0.5.1-0.5.8 (latest) and 0.5.2/0.5.3 seem to be the fastest, with version 0.5.1 being a close second.

Each time after I get the last “Image at xxx optimized in yyyms”, I get a “Completed in xx.” and then the CPU spikes. No transformed images are written to the build dir at that point, This last “step” takes much more time than the rest of the build when in finishes. Emphasis on when because I’ve had to forcibly terminate the build a couple of times. Using versions above 0.5.5 (I think) dramatically worsens the effect.

On the latest version, this last “step” pushes my CPU to 100% and I’ve noticed it take as much as 9.5GB of RAM. (For reference I have a desktop computer with a 6 core/12 thread i7 8700K and 16GBs of RAM). Initially I thought it was an issue with Windows so I rebooted to my Linux install and tried it there… It crashed my desktop environment and I got kicked back to the login screen.

Something of note, later versions also throw an error only on Windows when building basically anything else apart from avif files. I don’t know if these are caused by the same issue (earlier versions are ok).

I’d also like to note what I’m trying to do: I am working on a blog website that uses wordpress as a CMS. I’m currently only using the Picture component passing the src attribute as a URL. The blog currently has <50 images.

About this issue

Original URL
State: closed
Created 2 years ago
Comments: 18 (14 by maintainers)

Most upvoted comments

I created this repo to explore and demonstrate the issue astro-imagetools-example. It’s using the astro blog starter, with modifications in src/pages/index.astro. I’m basically fetching a bunch of images from the blog I’m working on and displaying them. I tried to simply display a single image from imgur multiple times but the build worked perfectly.

However with experimentation I’ve found the following:

High resource usage still happens on 0.5.3, although it’s not as bad as on 0.5.8. Builds actually finish
Once a build finishes in 0.5.3, subsequent builds are very fast and the same effect is not observed. I suppose you’re implementing some sort of cache.
The problem seems to appear (or worsen) when having images with a large total size. This example fetches 64 images but by using Array.slice() I managed to reproduce the problem on much smaller counts of images.
On windows, build fails on >=0.5.4 with Input file is missing error.
I used gitpod to make this example, on 0.5.3 - 0.5.6 the first build finishes on ~160s while subsequent builds finish in ~5s. On 0.5.7 and 0.5.8 it never finishes. Here’s a gitpod snapshot.

I changed versions on gitpod with:

rm -rf node_modules
yarn add astro-imagetools@version
yarn build

While on my windows pc I just removed node_modules/.cache since deleting and reinstalling the whole node_modules folder took a lot of time.

skourismanolis on Apr 25, 2022

@skourismanolis a fix for the regression has been merged and released in 0.6.9.

After upgrading to astro-imagetools@0.6.9 and running the same tests on GitPod:

jpg- it took 9s to generate jpg images for all the 240 images
png- it took 13s to generate png images for all the 240 images
webp- it took 20s to generate webp images for all the 240 images
avif- it took 204s to generate jpg images for all the 240 images

Now, https://github.com/skourismanolis/astro-imagetools-example builds successfully on the latest release. It’s taking 225s for a build to finish on GitPod.

Note: The increase in build time in 0.6.9 from 0.5.3 is not due to another regression. It’s because the maximum default number of breakpoints was 3 in v0.5.3 and it’s 11 since v0.5.7.

RafidMuhymin on May 21, 2022

The reason the build is slow in this case is that there are 240 .avif files being written to disk, which is pretty slow with sharp: https://github.com/lovell/sharp/issues/2597 https://github.com/lovell/sharp-libvips/issues/97

If I change the format in the example repo to just 'jpg', the build completes in 50.94s, after writing 316 image files to the /dist/assets folder.

I’m not sure there’s much more that can be done in this package to improve this, other than to find another tool that is faster than sharp, which would be a pretty major change.

IanVS on May 21, 2022