aws-codebuild-samples: Caching won't work with node_modules.

CodeBuild keeps its cache locally and then symlinks it to our target location as evidenced by this line:

[Container] 2019/02/27 02:24:38 Symlinking: /codebuild/output/src718408252/src/github.com/dashmug/api-tests/node_modules => /codebuild/local-cache/custom/dba27c31aad935787bb275c3e5e4e957708f15386de599eff1db476022cd7e4c

This behaviour, however, breaks Node’s module resolution causing our Node apps to fail.

internal/modules/cjs/loader.js:596
throw err;
^
Error: Cannot find module 'source-map-support'
at Function.Module._resolveFilename (internal/modules/cjs/loader.js:594:15)
at Function.Module._load (internal/modules/cjs/loader.js:520:25)
at Module.require (internal/modules/cjs/loader.js:650:17)
at require (internal/modules/cjs/helpers.js:20:18)

This is a widely-known issue. https://github.com/npm/npm/issues/9479 https://github.com/nodejs/node-v0.x-archive/issues/25090

TLDR: node’s require() doesn’t work with a symlinked node_modules directory if the source directory is not named node_modules.

Because of this bug in node, having the following in buildspec.yml will fail:

cache:
  paths:
    - 'node_modules/**/*'

About this issue

Original URL
State: closed
Created 5 years ago
Reactions: 5
Comments: 16

Most upvoted comments

Having same issue using latest CodeBuild v2.0 image and “node 10” runtime installed. Build works fine with caching disabled.

mparent61 on Sep 26, 2019

Thanks @joshhoegen I ended up removing the ‘node_modules/**/*’ and the hang time ended. I used pnpm on this project so I tweaked how that whole thing would work but still tuning. Least it finally let the lint, test jobs run built in < 6 minutes.

phases:
  install:
    commands:
      - n 14.19.0
      - echo Installing source NPM dependencies...
      # This project uses angular/cli and pnpm.  "aws/codebuild/standard:5.0" comes with Node 14
      - npm install -g @angular/cli@13.3.6 pnpm@7.1.5
      - pnpm config set store-dir .pnpm-store
      - npm config -g set prefer-offline true
      - npm config -g set cache /root/.npm
      - npm config get cache
      ```

cybercussion on May 25, 2022