angular: Service Worker: Hash Mismatch

One reason is to conserve bandwidth (which might be significant if you cache large, rarely-changing files, like images, etc.)

But that is not the main reason (as far as I understand).

There is the concept of an app version. An app version contains all the files that comprise the app. This info is captured in a “manifest” file (ngsw.json), which is used by the SW to find out about a new app version.

Note that the SW may serve different app versions to different app instances (e.g. different tabs). The SW guarantees that each app instance will continue to receive the files that correspond to its assigned version. This is important, because if your user has the app open in tab 1, and you deploy a new version with an updated asset (which might be incompatible with the old app version - e.g. a new JS script that expects the new index.html layout and breaks if used with the old one), you don’t want the user to suddenly start getting the new incompatible asset, while still using the old index.html.

Therefore, it is important for the SW to be able to tell if an asset file belongs to a specific version. The only reliable way discussed so far is comparing the content (which btw is also what browsers do for detecting modified ServiceWorker scripts).

For example, imagine you deploy version 2 which includes foo.js as an asset. The SW detects the new version and starts downloading the assets for that version. In the meantime, you deploy version 3, with a modified foo.js. When SW requests foo.js to cache it for version 2, it will receive the new (backwards incompatible) foo.js from version 3 (because that is now deployed to the server). If it were to happily cache and serve that version with the rest of the app files for version 2, your app would be broken.

Essentially, the SW needs to be able to tell whether a received file is actually the one that belongs to the current version it is updating to. Also, lazily fetched assets (with installMode: 'lazy') complicate things even more.

I am sure there are other usecases, but in general the idea is that the SW needs to be able to tell whether the files that it receives, when fetching for a specific app version, do indeed correspond to the files that where on disk when that version’s build was created.

Verifying the content is the only reliable way that has come up so far (and doing it via a hash is a cheap way to achieve this) 😃

I got the issue fixed by:

1. Run production build.
2. Minify index.html file html-minifier -o dist/index.html dist/index.html --remove-comments --collapse-whitespace --minify-js --minify-css
3. Minify Service Worker terser dist/combined-sw.js -o dist/combined-sw.js -c --comments /^##/. (for some reason --comments false was not working so I added this --comments /^##/ to remove all comments).
4. Generate Service Worker config file again ngsw-config dist ngsw-config.json

@gkalpak @filipesilva what if Angular CLI could minify these files as well during the build process?

We had the same “Hash mismatch” issue in our PWA app for a different reason, which I want to share here for any other one who may encounter this issue. We have a PWA app used in production with thousands of users. It’s written in Angular 9 and uses App Shell for generating index.html file.

For having a better performance and load time, we were using Angular Custom Webpack and Compression Webpack Plugin plugin to make js chunks with gzip and brotli compression formats. So, they are ready in place for nginx to be served for any client requests for them. We intentionally wanted to generate the pre-compressed files in the build time, so they would be ready for nginx to be served to the clients (versus letting the nginx itself generate compressed files on-demand). As a result, the response time of nginx for such requests would be slightly better.

But here is the problem. From Angular v8.2, we weren’t able to update to the latest version of Angular. I think some minor thing was changed in the way Angular builds the app (I couldn’t find the root cause of it in the changelogs) which caused the generated compressed files to have a different hash after compression. So, the service worker couldn’t update the app, showing this error: Driver state: EXISTING_CLIENTS_ONLY (Degraded due to failed initialization: Hash mismatch (cacheBustedFetchFromNetwork). After we disabled the “Compression Webpack Plugin” in the build pipeline and let the nginx itself to compress the files, the problem has been resolved. Now, we’ve successfully upgraded to Angular v9 without any problems with the service worker.

@gkalpak That description deserves to be a part of the official tech explainer. Great job, mate!

For convenience, I have added these to my packages.json file under scripts node:

    "minify-index": "html-minifier -o dist/index.html dist/index.html --remove-comments --collapse-whitespace --minify-js --minify-css",
    "concat-ngsw-fcm-worker": "concat dist/ngsw-worker.js src/messaging-sw.js -o dist/combined-sw.js",
    "minify-combined-sw": "terser dist/combined-sw.js -o dist/combined-sw.js -c --comments /^##/",
    "prepare-ngsw-config": "ngsw-config dist ngsw-config.json",

Yes, I’ve been bitten by this myself and I suspect many others have been/will be too (with or without realizing 😁). It is hard to track down and it would be great if we could come up with an easy to use solution that remained reliable.

My first thought was to be able to automatically put the hash of a file as a comment (e.g. at the top of the file), so that the SW could read the hash from the file (instead of hashing it itself). This might not work for certain types of files though (e.g. images).

Having a configurable HashCheckStrategy would be nice iff we could ensure that it is a reliable one (which I don’t think we can). Being able to reliably determining file content equality (which is what hash comparison gives us) is essential to the SW’s working as expected (and not serving incorrect content).

Giving users the power to easily mess this fundamental check is very dangerous imo. (E.g. with the “pass all” strategy shown above, you could easily end up being served an old, outdated asset (image, stylesheet, script) along with the latest index.html and app files, with unexpected results.

@alxhub, any thoughts on that? Do you know/remember what other strategies (other than comparing hashes) have been considered/rejected?

@AaronDovTurkel, yes, it most probably will be a problem (i.e. it depends on whether the modified files are included in the ngsw.json manifest). A possible way to work around that would be to re-generate ngsw.json after modifying the files with something like: node_modules/.bin/ngsw-config dist/your-app ngsw-config.json

Hi there, we have a use case in Angular which made us hit this issue. We build our app with string tokens in environment-settings, which are being replaced on the release pipeline based on the environments it’s being deployed to. This makes one of the bundle files’ hash different from what it was during the build, and therefore NGSW stops working due to hash mismatch.

It would be great if we could manage those hashes later on or disable this check when needed.

Ok guys, after some strugglin i found and fix my hash mismatch problem. (This is not the solution to this issue anyway) I will put it here to help others who can have the same problem.

---

My infrastructure topology is: User -> Firewall -> Reverse Proxy -> Reverse Proxy (Ingress - Kubernetes NGINX) -> Web Server (NGINX)

What was happening is that the first Reverse Proxy was using some configuration that rewrite automatically links from http to https. In my case, i am using some css lib that has commented http links to its page like:

/*
  My CSS lib: http://some.random.page/
*/

Then in this case, the reverse proxy was changing the links on-the-fly (so the content was changed) and results in a hash mismatch.

Once this configuration was disabled, the hash mismatch issue goes away and the app updates successfully.

---

I think that in some cases this issue will be a very hard thing to bypass (identify/solve) or even impossible (where it is hosted in a environment out of devs control or unknow).

Anyway, thanks again for @gkalpak for the time to clarify the behaviour/issue.

In my case, when I turned off Cloudflare content optimizations, Service Worker started to work again like magic; just like @manfredsteyer stated in the ticket mentioned above.

Our release pipeline does a token replace on a configuration file in the assets folder to handle build once deploy to multiple environments. Updating the file changes the sha hash in the ngsw.json hashTables object. This node script ended up working after running this after the token replace task. This issue helped me solve my problem. Was stuck for a good 3 days.

const crypto = require('crypto');
const fs = require('fs');

var getSHA1ofJSON = function(input){
  const hash = crypto.createHash('sha1');
  hash.update(input);
  
  return hash.digest('hex');
}

const args = getArgs();

const pathToSettings = args.pathToSettings;
const pathToNgswJson = args.pathToNgswJson;

let ngsw = JSON.parse(fs.readFileSync(pathToNgswJson));

const fileBuffer = fs.readFileSync(pathToSettings);
const sha1 = getSHA1ofJSON(fileBuffer);

ngsw.hashTable['/assets/settings.tokenize.json'] = sha1;

fs.writeFileSync(pathToNgswJson, JSON.stringify(ngsw));

Just run ngsw-config dist ngsw-config.json

how can I delay or rebuild ‘ngsw.json’ with the new hashes after modifing the build’s output files (for example minifying them with terser).

so the flow will be: 1- build 2- modify the output (minify the files, for example) 3- create ngsw with the new hashes.

edit I just read @naveedahmed1, and I will try this now

Wow, good job finding that sneaky culprit 💯 And thx for posting your findings for other people to benefit from your work 👍

I think it might be worth mentioning in the docs that for the SW to work the build artifacts have to be served unmodified to the end-user. If anyone feels like submitting a pull request that adds something like that in the docs, that would be awesome!

I have run into similar issues when combining SSR (Universal) and Service Workers, as the pre-rendered index.html hash differs from the one in the manifest, making offline support impossible. I currently use the work-around to remove the hash from the index.html before deploying.

@hqrd if you can get the script tag in the build then,as @gkalpak said, you can regenerate the hash table after the script is injected.

I’m doing that already for other files with environment variables, but it doesn’t work in this case because the script tag is added after my app is started, and has some changing variables.

I also have an issue with hashes. My index.html has a <script> tag added by Dynatrace RUM and makes my client side hash always invalid. I had to remove the index.html from the ngsw-config.

Got the same issue. Server is in windows environment(IIS) and I’m using macOS. Whenever I uploaded the build output in FileZilla, index.html, style.css, ngsw-worker.js, safety-worker.js, and worker-basic.min.js, the memory size of these files seems to increase a few bytes and other files are just the same. I’m not sure why but it could be the CR and LF as mentioned by @mmatczak and @martinbianchi.

As mentioned by @naveedahmed1, I solved the hash mismatch issue by minifying the mentioned files and generated a new hash table with

./node_modules/.bin/ngsw-config ./dist/project src/ngsw-config.json

Thanks to you by the way 😀👍.

Also bypassing the hash checker could result to unwanted behavior or broken app.

UPDATE: New solution to my case is to change transfer type to Binary in FileZilla. ASCII type causes the problem because of line endings that increases the memory of the files(index.html and etc.) and when re-hash by ngsw it would result to different hash code. Now I don’t have to minify my files like what I did the last time.

@naveedahmed1, maybe you could open an issue for this to be discussed on the CLI repo.

Any update on this?

I am using Cloudflare which applies different optimizations on files, as a results the hash mismatch for index.html file.

For everybody looking for a straightforward way to update the hashes in the ngsw.json I wrote this little script:

#!/bin/bash

# This updates the hash of the index.html in the corresponding ngsw.json 
# Execute it after replacing the environment variables

# Paths to folders containing the index.html files (where the hash should be recalculated)
paths=( "de" "en")

# The first parameter has to be the path to the directory with the language folders
if [[ -z $1 ]]; then
    echo 'ERROR: No target file or directory given.'
    exit 1
fi

# Go to all folders containing index.html and ngsw.json
for i in "${paths[@]}"
do
  # Calculate hash of index.html
  replaceString=($(sha1sum $1/$i/index.html))

  if [ "$DEBUG" = true ]
  then
    # If DEBUG=true in order to log the replaced files
    sed -i "s|\"\/de\/index\.html\":\s\"\(.*\)\"|\"/""$i""/index.html\": \"""$replaceString""\"|Ig;w /dev/stdout" "$1/$i/ngsw.json"
  else
    # If DEBUG=false do it without logging
    sed -i "s|""\/de\/index\.html"":\s""\(.*\)""|""/$i/index.html"": $replaceString""|Ig" "$1/$i/ngsw.json"
  fi
done

# Execute all other parameters
exec "${@:2}"

You may have to update the search regex, as our project is served under two different paths (/de/ and /en/)

See it in action here: https://github.com/T-Systems-MMS/phonebook/pull/101/commits/d4e2b180d6f6674759465ec68ce7d8dbfe161228

As a workaround, i think we can use the sed -i 's/canonicalHash !== cacheBustedHash/false/g' ngsw-worker.js after build, which change the line https://github.com/angular/angular/blob/master/packages/service-worker/worker/src/assets.ts#L423 from

if (canonicalHash !== cacheBustedHash) {

to

if (false) {

By some testing, it will still work offline and everything. The only down side is that it will trigger the this.swUpdate.available event every time the hash change, consequently if you notify the user about the new version, it will show this message at every new hash (caused by external events). Keep in mind that the user can been notified about some new version and the version/app is the same! But i think it’s better than keeping the user with some old version (as @gkalpak said it happen https://github.com/angular/angular/issues/25307#issuecomment-410632977)!

Even with that workaround, the dev can check at update event if is really some new version of the app or any other thing that change the hash by checking some version variable that exists inside the code.

Can @gkalpak or @alxhub see if there are any side-effect i am not seeing?

Thats a point - SW should be written that way that it cache index.html after SSR and for offline mode uses latest cached one. Because SSR is not only for speed it’s main thing is SEO and if index.html is not changing then google spiders cannot gather info 😉