turborepo: Persistent tasks in watch mode don't wait for first run of their dependencies to finish
Verify canary release
- I verified that the issue exists in the latest Turborepo canary release.
Link to code that reproduces this issue
https://github.com/OliverJAsh/turbo-watch-deps-of-persistent
What package manager are you using / does the bug impact?
Yarn v2/v3/v4 (node_modules linker only)
What operating system are you using?
Mac
Which canary version will you have in your reproduction?
2.0.6-canary.0
Describe the Bug
Persistent tasks in watch mode don’t wait for first run of their dependencies to finish.
- Package
bdepends on packagea. - Both packages have a build task.
In non-watch / run mode, build a correctly runs before build b:
$ TURBO_UI=false yarn run turbo run build --filter b
• Packages in scope: b
• Running build in 1 packages
• Remote caching disabled
a:build: cache bypass, force executing 7c09bd93554e1d1c
a:build: A
b:build: cache bypass, force executing 7f551e27d3b8aa66
b:build: B
Tasks: 2 successful, 2 total
Cached: 0 cached, 2 total
Time: 1.516s
However, in watch mode, Turbo runs both builds in parallel. Build b does not wait for build a to finish even though it’s a dependency:
$ TURBO_UI=false yarn run turbo watch build --filter b
• Packages in scope: b
• Running build in 1 packages
• Remote caching disabled
• Packages in scope: b
• Running build in 1 packages
• Remote caching disabled
a:build: cache bypass, force executing 7c09bd93554e1d1c
b:build: cache bypass, force executing 25c5580a26314aac
b:build: B
a:build: A
Expected Behavior
Persistent tasks in watch mode should wait for the first run of their dependencies to finish, otherwise they may fail.
To Reproduce
- Clone https://github.com/OliverJAsh/turbo-watch-deps-of-persistent.
- Run
yarn. - Run
TURBO_UI=false yarn run turbo watch build --filter b.
Additional context
I believe the idea is that persistent tasks should self-heal: https://github.com/vercel/turbo/issues/8164#issuecomment-2122878984
However, this is inefficient and can lead to confusing error messages.
In our case we’re using webpack watch and rebuilding is CPU intensive, so it would be better if webpack waited for its dependencies to finish building before starting.
Furthermore, our webpack watch task can only self-heal after the build has started—it can’t self-heal if the configuration itself is missing dependencies.
We only expect the order to be respected for the first run, not subsequent to that.
About this issue
- Original URL
- State: open
- Created 2 years ago
- Reactions: 39
- Comments: 18 (2 by maintainers)
@NicholasLYang IMHO this feels like a huge blocker of turborepo adoption. Given it isn’t a priority I feel like I am missing something. As someone trying to use
turbo watchsince it showed up in canary I don’t quite understand what it is currently solving without this. Is there an example repository somewhere that would show how someone can leverageturbo watch?If i am understanding correctly, the usual set up is:
Whenever a package changes, the dependent apps need to know about it. Without it, one would have to restart
turbo devon every change.I tried various work arounds:
The best solution for now has been using packages without a build step and just export the ts files directly as much as possible and let the host apps bundler bundle the package instead.
Appreciate any guidance / insight!
Is there a way to configure it so that only the non-persistent tasks are executed after the first run?
I have a configuration like so:
and while this does wait for the first run of dependencies to finish, subsequent changes to packages also runs
devagain.devfor me is a Nest dev server that doesn’t need to be rerun.@dylanjmcdonald In the context of the kitchen sink example, if you change the code inside of
@repo/loggertheapitsup --watch won’t detect the change as its not aware when one of its dependencies change.This led me to then make the following changes:
devscript in the rootpackage.jsontoturbo watch devdevscript inside of thepackages/loggerandpackages/uibecause we are using theturbo watchcommand nowdevtask inside ofapiinterruptible so that theapiserver is restarted when its dependencies changeThis leaves us with the following setup:
apps/directory havedevscripts that are both persistent but onlyapps/apiisinterruptiblepackages/directory have abuildscript and rely onturbo watchto be rebuiltHowever, with such setup (which i believe is an extremely common one in monorepos) there is a bug currently (
turbo@2.3.3).Repro steps:
packages/uipackagepackages/uigets rebuilt (as it should)apidevtask gets terminated and isn’t restarted.I believe this bug is documented here => https://github.com/vercel/turborepo/issues/9421
I haven’t been able to find a good workaround for this issue.
UPDATE: A neat workaround that I found was to mark all persistent tasks as non-interruptible and use the
watchcommand from tsx to monitor and restart those tasks.tsx watch(v4.19.2) also detects changes inside internal package dependencies of the watched package.@KajSzy I experienced something similar to this and it turned out it was a bug in the latest v2.2.3 release where persistent tasks were accidently stopped in watch mode: https://github.com/vercel/turborepo/pull/9330. Fixed in v2.2.4-canary.2+