turbo: [turborepo] turbo prune in v1.7.1 generates turbo.json file that breaks commands

What version of Turborepo are you using?

1.7.1

What package manager are you using / does the bug impact?

npm

What operating system are you using?

Linux

Describe the Bug

originally posted as a comment here: https://github.com/vercel/turbo/issues/3564, but I think it’s actually a separate issue than the one described there

My CI dockerfile build randomly started failing today when seemingly nothing changed. I noticed that my dockerfile blindly globally installs the latest turbo in order to prune the repo, so we started using 1.7.1 today. I get the following error on my turbo build step, after I’ve already pruned my repo:

Step 17/33 : RUN npm run build -- --filter=api... &&     npm prune --omit=dev
 ---> Running in 1e638563d2dc

> traveler-recruitment@0.0.0 build
> turbo run build --filter=api...

 ERROR  run failed: turbo.json: json: cannot unmarshal number into Go struct field rawTurboJSON.pipeline of type string
Turbo error: turbo.json: json: cannot unmarshal number into Go struct field rawTurboJSON.pipeline of type string
The command '/bin/sh -c npm run build -- --filter=api... &&     npm prune --omit=dev' returned a non-zero code: 1

Upon trying out the turbo prune locally on 1.7.0 and 1.7.1, the generated turbo.json files are significantly different. Nothing is wrong per se with the version produced by 1.7.1, but turbo build is clearly unhappy about this differently formatted turbo.json file.

1.7.0 turbo.json

{
  "$schema": "https://turbo.build/schema.json",
  "pipeline": {
    "dev": {
      "dependsOn": ["^build"],
      "outputs": [],
      "cache": false,
      "persistent": true
    },
    "build": {
      "dependsOn": ["^build"],
      "outputs": ["build/**", "dist/**"],
      "outputMode": "new-only"
    },
    "lint": {
      "dependsOn": ["^build"],
      "outputs": [],
      "outputMode": "new-only"
    },
    "test": {
      "outputs": ["coverage/**"],
      "dependsOn": ["^build"],
      "inputs": ["src/**/*.tsx", "src/**/*.ts", "jest.config.ts", "jest-setup.ts"],
      "outputMode": "new-only"
    },
    "test:watch": {
      "cache": false,
      "persistent": true
    },
    "clean": {
      "cache": false
    }
  }
}

1.7.1 turbo.json

{
  "pipeline": {
    "build": {
      "Outputs": { "Inclusions": ["build/**", "dist/**"], "Exclusions": null },
      "ShouldCache": true,
      "EnvVarDependencies": [],
      "TopologicalDependencies": ["build"],
      "TaskDependencies": [],
      "Inputs": null,
      "OutputMode": 3,
      "Persistent": false
    },
    "clean": {
      "Outputs": { "Inclusions": null, "Exclusions": null },
      "ShouldCache": false,
      "EnvVarDependencies": [],
      "TopologicalDependencies": [],
      "TaskDependencies": [],
      "Inputs": null,
      "OutputMode": 0,
      "Persistent": false
    },
    "dev": {
      "Outputs": { "Inclusions": null, "Exclusions": null },
      "ShouldCache": false,
      "EnvVarDependencies": [],
      "TopologicalDependencies": ["build"],
      "TaskDependencies": [],
      "Inputs": null,
      "OutputMode": 0,
      "Persistent": true
    },
    "lint": {
      "Outputs": { "Inclusions": null, "Exclusions": null },
      "ShouldCache": true,
      "EnvVarDependencies": [],
      "TopologicalDependencies": ["build"],
      "TaskDependencies": [],
      "Inputs": null,
      "OutputMode": 3,
      "Persistent": false
    },
    "test": {
      "Outputs": { "Inclusions": ["coverage/**"], "Exclusions": null },
      "ShouldCache": true,
      "EnvVarDependencies": [],
      "TopologicalDependencies": ["build"],
      "TaskDependencies": [],
      "Inputs": ["src/**/*.tsx", "src/**/*.ts", "jest.config.ts", "jest-setup.ts"],
      "OutputMode": 3,
      "Persistent": false
    },
    "test:watch": {
      "Outputs": { "Inclusions": null, "Exclusions": null },
      "ShouldCache": false,
      "EnvVarDependencies": [],
      "TopologicalDependencies": [],
      "TaskDependencies": [],
      "Inputs": null,
      "OutputMode": 0,
      "Persistent": true
    }
  },
  "remoteCache": {}
}

My workaround for now is pinning the turbo version to 1.7.0 for my docker build

Expected Behavior

turbo prune should generate a turbo.json file that can be used by turbo CLI commands

To Reproduce

run turbo prune on 1.7.1 and 1.7.0, the turbo.json file generated in the /out/ dir in 1.7.1 doesn’t work with the turbo commands you configured in your root turbo.json. Should be able to confirm this with a docker build in the with-docker example, though I haven’t tried that one yet (also it uses yarn, so not positive if this is only NPM)

I’ll try to get more detail in here tomorrow

Reproduction Repo

No response

About this issue

Original URL
State: closed
Created a year ago
Reactions: 18
Comments: 17 (1 by maintainers)

Commits related to this issue

fix(use correct serialization logic for turbo.json) (#3580) Fixes #3570 The issue here was that we implemented the custom marshalling behavior on a *pointer* of `TaskDefinition`, but the pipelin... — committed to vercel/turbo by chris-olszewski a year ago
fix: patch docker file to lock turbo version to 1.7.0 in pruning process From this issue: https://github.com/vercel/turbo/issues/3570) — committed to 2110336-2565-2/Sec33-group1-MeowMeow by PongDev a year ago
fix: Update Turbo 1.6.3 → 1.7.3.canary.1 to fix Docker builds We need this update to fix an issue with our builds that occurs on GitHub Actions only (cannot repro locally). Based on [this issue](htt... — committed to farcasterxyz/hub-monorepo by sds a year ago
fix: Update Turbo 1.6.3 → 1.7.3.canary.1 to fix Docker builds (#505) We need this update to fix an issue with our builds that occurs on GitHub Actions only (cannot repro locally). Based on [this ... — committed to farcasterxyz/hub-monorepo by sds a year ago

Most upvoted comments

Confirmed, i’m having the same issue with 1.7.1.

Everything is fine going back to 1.7.0

sebaplaza on Feb 1, 2023

I believe this will be fixed in the 1.7.3 release. Viewing the commit history, this was fixed after the 1.7.2 release 😅.

Klukies on Feb 2, 2023

@abenjaminov

in what way exactly is it related?

This code change specifically… prior to this change, the turbo.json file was being copied. Whatever is marshalling the JSON back doesn’t appear to work as expected?

eden on Feb 1, 2023

As @sebaplaza mention, I manage to fix this by patching my docker file:

# FIXME lock version to v1.7.0: https://github.com/vercel/turbo/issues/3570
RUN yarn global add turbo@1.7.0

Note that if you’re using the official example, you only need that on the AS builder image (the one that run the turbo prune command).

moshest on Feb 1, 2023

@moshest I agree, as mentioned here https://github.com/vercel/turbo/issues/3645#issuecomment-1425157546 the point I’m making is that the regression is that up until 1.7.0 turbo with no turbo.json worked fine for the prune use case.

I think this issue should be closed, and #3645 be reopened.

maximveksler on Feb 11, 2023

I just tested this again against turbo v1.7.4 and the issue seems to be resolved ✔️

jplwood on Feb 8, 2023

I’m getting the same error recently on GitHub Actions with turbo@1.7.0

turbo run deploy --filter=frontend “ERROR run failed: turbo.json: json: cannot unmarshal number into Go struct field rawTurboJSON.pipeline of type string”

HQ92 on Feb 2, 2023

@chris-olszewski Just saw you closed this. I just tried installing v1.7.2 in my dockerfile/build to try it out and I’m still getting the same error

 => [prune_repo 5/7] RUN npm install --global turbo@^1.7.2                                                                                                                                                                       3.6s 
 => [prune_repo 6/7] COPY . .                                                                                                                                                                                                    9.9s 
 => [prune_repo 7/7] RUN turbo prune --scope=api --docker  
    ...
 => ERROR [build_image 9/9] RUN npm run build -- --filter=api... &&     npm prune --omit=dev                                                                                                                                     1.4s 
------
 > [build_image 9/9] RUN npm run build -- --filter=api... &&     npm prune --omit=dev:
#22 1.247
#22 1.247 > traveler-recruitment@0.0.0 build
#22 1.247 > turbo run build --filter=api...
#22 1.247
#22 1.269 Turbo error: turbo.json: json: cannot unmarshal number into Go struct field rawTurboJSON.pipeline of type string

And the out/full/turbo.json generated by turbo prune still looks wrong:

v1.7.2 turbo.json

{
  "pipeline": {
    "build": {
      "Outputs": { "Inclusions": ["build/**", "dist/**"], "Exclusions": null },
      "ShouldCache": true,
      "EnvVarDependencies": [],
      "TopologicalDependencies": ["build"],
      "TaskDependencies": [],
      "Inputs": null,
      "OutputMode": 3,
      "Persistent": false
    },
    "clean": {
      "Outputs": { "Inclusions": null, "Exclusions": null },
      "ShouldCache": false,
      "EnvVarDependencies": [],
      "TopologicalDependencies": [],
      "TaskDependencies": [],
      "Inputs": null,
      "OutputMode": 0,
      "Persistent": false
    },
    "dev": {
      "Outputs": { "Inclusions": null, "Exclusions": null },
      "ShouldCache": false,
      "EnvVarDependencies": [],
      "TopologicalDependencies": ["build"],
      "TaskDependencies": [],
      "Inputs": null,
      "OutputMode": 0,
      "Persistent": true
    },
    "lint": {
      "Outputs": { "Inclusions": null, "Exclusions": null },
      "ShouldCache": true,
      "EnvVarDependencies": [],
      "TopologicalDependencies": ["build"],
      "TaskDependencies": [],
      "Inputs": null,
      "OutputMode": 3,
      "Persistent": false
    },
    "test": {
      "Outputs": { "Inclusions": ["coverage/**"], "Exclusions": null },
      "ShouldCache": true,
      "EnvVarDependencies": [],
      "TopologicalDependencies": ["build"],
      "TaskDependencies": [],
      "Inputs": ["src/**/*.tsx", "src/**/*.ts", "jest.config.ts", "jest-setup.ts"],
      "OutputMode": 3,
      "Persistent": false
    },
    "test:watch": {
      "Outputs": { "Inclusions": null, "Exclusions": null },
      "ShouldCache": false,
      "EnvVarDependencies": [],
      "TopologicalDependencies": [],
      "TaskDependencies": [],
      "Inputs": null,
      "OutputMode": 0,
      "Persistent": true
    }
  },
  "remoteCache": {}
}

Might have been some other problem than what you fixed, unless I’m missing something?

jplwood on Feb 1, 2023