redwood: Github action fails with yarn 3 since bumping to v0.49.1

We’ve bumped to v0.49.1 today in an attempt to finally get to yarn 3 - super excited.

Locally it runs very well and caused little troubles to upgrade.

The problem occurs when using a github action.

The workflow, truncated:

on:
  push:
    branches:
      - preprod

name: Deploy preprod

env:
  repo_name: ***
  node_version: 16.6.1

jobs:
  deploy:
    name: Deploy
    runs-on: ubuntu-latest
    environment: preproduction

    steps:
      - name: Configure AWS credentials
        uses: aws-actions/configure-aws-credentials@v1
        with:
          aws-access-key-id: ${{ secrets.AWS_ACCESS_KEY_ID }}
          aws-secret-access-key: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
          aws-region: ${{ secrets.AWS_REGION }}

      - name: Login to Amazon ECR
        id: login-ecr
        uses: aws-actions/amazon-ecr-login@v1

      - name: Checkout
        uses: actions/checkout@v2

      - name: Create environment files
        working-directory: aws
        run: |
          yarn

          # Generate ebextensions env file
          node ./generate_preprod_env.js ebextensions > ./.ebextensions/02-envvars.config

          # Generate dotenv file
          node ./generate_preprod_env.js dotenv > ../.env

// the rest is irrelevant

This part has been working well until now, and debugging proves to be rather difficult.

The action fails on step Create environment files and unfolding the details only show a diff of a file which should not have been edited outside of our local files. There is no explicit error message:

$ yarn rw info
  System:
    OS: Linux 5.13 Ubuntu 21.10 21.10 (Impish Indri)
    Shell: 5.8 - /usr/bin/zsh
  Binaries:
    Node: 16.6.1 - /tmp/xfs-a76dd022/node
    Yarn: 3.2.0 - /tmp/xfs-a76dd022/yarn
  Browsers:
    Chrome: 99.0.4844.51
    Firefox: 98.0
  npmPackages:
    @redwoodjs/core: ^0.49.1 => 0.49.1 
    @redwoodjs/eslint-config: ^0.49.1 => 0.49.1 
    @redwoodjs/record: ^0.47.0 => 0.47.1

About this issue

Original URL
State: closed
Created 2 years ago
Comments: 21 (4 by maintainers)

Most upvoted comments

Quick notes:

The getting yarn@3 squared away in CI/CD learnings are we might want to do a scan for yarn.locks or package.jsons that reside outside of Redwood’s standard file structure so we can at least warn framework users that there is probable danger ahead.

Another big take away is nodeLinker was needed to be configured.

nodeLinker: node-modules

this should have happened automatically via npx @redwoodjs/codemods@canary upgrade-yarn; but maybe it was not? or maybe it was not initially checked into version control?

Learnings here might be to better document how drastic of a change yarn@3 is from yarn@1; imho from an end users perspective they should be thought of as separate tools (if anyone remembers AngularJS vs Angular 2+, the distinction between architectural differences in yarn@1 and yarn@2+ is very analogous).

Also maybe have a diagnostic scan that warns if this is not configured?

Also as a learning for the redwood framework as a whole, unless I am mistaken we can not really lean into yarn’s pnp architecture until this is resolved on the prisma side: https://github.com/prisma/prisma/issues/1439

virtuoushub on Apr 9, 2022

Finally \o/

Thanks to @virtuoushub’s time and knowledge, we’ve been able to figure this out.

I’ve got a Redwood app and at root I have an aws package, configured for yarn v1. In short, there was a conflict while handling both in the github action, because our aws package gets executed in yarn 1 before deployment & then we get the application built in yarn 3. Adding to that, a Redwood JS v1 needs an updated Dockerfile.

So here are the changes we had to make:

# .yarnrc.yml
yarnPath: .yarn/releases/yarn-3.2.0.cjs
nodeLinker: node-modules    # <--- That one matters for our `aws` package.

# Dockerfile
# First we build.
FROM node:16.6.1-alpine as build

WORKDIR /app

COPY package*.json .yarnrc.yml yarn.lock redwood.toml graphql.config.js .env ./ # <--- Adding .yarnrc.yml specific to yarn 3.

COPY .yarn .yarn # <--- Copying yarn 3 cache.

COPY api api

COPY web web

[...]

At some point we’ve ran into a Docker issue, we needed to add git to the docker image, @virtuoushub I can confirm that this is not necessary.

I’m leaving this opened until I can confirm a successful prod deploy.

@thedavidprice @simoncrypta not sure how such edge case could be used. Maybe some topic in the forums? I’ve seen DT’s similar posts of issues + resolution. Something else?

noire-munich on Apr 8, 2022

unfortunately I will not be available for a call. I outlined some things to try above. If those do not work to resolve the issue, and you do end up having a repo for me to take a look at, then I will be happy to help by checking it out.

@virtuoushub No problem! Thanks a lot for the help 👍

Ok so I checked the two points:

packageManager is defined properly
engines.yarn is not, for some reason it was stuck with the old version.

I fixed according to your diff @virtuoushub , removed node_modules and yarn.lock, ran yarn dedupe then yarn install and pushed my commit. Nothing has changed, there’s still the diff between v1 and v3 yarn.lock versions.

Only thing I eventually noticed is the ending line of that diff:

➤ YN0028: │ The lockfile would have been modified by this install, which is explicitly forbidden.

I do not understand what would make that happen in the github action I’ve provided above, which got corrected according to @thedavidprice 's smoke-tests.

Last thing to know that should be brought to @simoncrypta 's attention: I upgraded to yarn 3 by using npx @redwoodjs/codemods@canary upgrade-yarn, which did not fail, but it also didn’t fix the engine section of package.json. Not sure what to make of it but I thought it was to be brought to attention, i’d have expected the code mod to have changed that as well.

noire-munich on Mar 25, 2022

I removed dependabot, still same problem with the lockfile - even after I cleaned yarn.lock. Don’t know about actions/setup-node, I didn’t have it until I took example on the smoke-tests. Also, rebuilt preprod branch several times from main, resulting in no change whatsoever.

noire-munich on Mar 21, 2022