turbo: --parallel Does not honor build dependencies

So if you ended here as me, let’s reiterate how Turborepo dev works (and how not) because it’s confusing.

The workflow a developer would expect is to install modules and then run the dev script, but that’s now how Turborepo works. A dev task needs two things: generate code and run watchers. From Turborepo docs:

--parallel
Default false. Run commands in parallel across workspaces and ignore the dependency graph. This is useful for developing with live reloading.

It’s confusing because Turborepo runs everything parallel, if possible. This configuration should be named --ignoreDependecyGraph

--parallel is required to run watchers, but before that, we need to build code in order so we can’t use --parallel. It’s a chicken-egg problem.

I bet many developers just run a dev task, then they see some errors, then they will rerun it, and then it magically works. That’s not ideal.

Turborepo suggests that we should use dependsOn https://turborepo.org/docs/handbook/dev#running-tasks-before-dev, but it solves nothing because --parallel is required for watchers.

As I see it, there have to be two tasks run separately from the command line. Something like buildForDev and then dev.

Because my build does not do anything special, the recommended way (for me) is always to run build before dev or start.

I have watch modes with incremental rebuilds for all packages as their dev task, one-off optimized builds as their build task.

Because I want to run all dev tasks in parallel and the incremental watch builds output to the same directories the one-off optimized builds would also output, I need to have all packages built before I start their dev modes. In the repo’s root, I have turbo run dev --continue as my dev task at the root and this in my turbo.json:

"dev": {
  "dependsOn": ["^build"],
  "cache": false
}

This starts all builds and after they finish, starts all watch modes. So far, so good.

However, if application A depends on package B which in turn depends on package C, what I’d like to see when I save a file in package C is that C does an incremental rebuild, followed by an incremental rebuild of B, followed of an incremental rebuild of A, followed by a browser refresh (which is out of turborepo scope).

What happens instead is an incremental rebuild of package C. That’s it. Is there any way to get what I want?

Since this feature is IMHO a very basic need, I don’t understand how we can use Turborepo dev mode when some packages relies on others. Did someone found any workaround on this ?

The docs do specifically say this ignores the dependency graph, but I’m struggling to see how this is useful?

In the meantime is there a workaround for local development where you build your dependencies with a tsc -w or similar? If you remove the parallel flag, as the processes never exit (watch mode) the build just hangs 😦

Since dev is a non ending tasks, dependents tasks are not executed. Any thought ?

I also realized this, So I try to let the dev task to depend on ^build as the like whats said here. https://github.com/vercel/turborepo/issues/460#issuecomment-1064161370

But if you have a dependency graph like A -> B. For example A is a typescript package, its build script is tsc. If you are developing A and B at the same time which is a common case in monorepo. Once A has a type error during development, tsc will failed and exit then the whole turbo run dev exited.

Nx seems planing to support a long runing task as dependency. https://github.com/nrwl/nx/pull/10706

It would be nice if turborepo also supported this.

The title and description here describes the intended behavior of --parallel.

What’s being described here is more like several feature requests put together. Something like --watch and some health checking across tasks. We have an issue that we’re actively working on in #986 and have ideas about depending on long-running tasks.

I’m going to close this issue since it’s describing the intended behavior - but rest assured that we hear the pains!

My understanding is that this currently works /fine/.

You must not pass --parallel flag, because that breaks deps, but by default turbo runs with 10 concurrent tasks, and if you have tasks that “hang” and never exit like dev, those will work as long as they fit within the allotted concurrency slots.

You’re welcome to use https://github.com/IPWright83/pnpm-monorepo as another example if needed. Requires pnpm, but using pnpm run dev_ideal illustrates the issue. I’ve added a dev script that also does a build to workaround it for now… turbo run build --scope=@iw* --no-deps && turbo run dev --no-cache --parallel --continue

@jaredpalmer I just tried that, and unfortunately it doesn’t work either (though I was using ^dev to keep the watch mode). The concurrency is going to max out at 2 at the start (due to the lowest level dependencies) and because they never finish it can’t start the next tasks, regardless of the concurrency limit.

Sounds like what you want to do isn’t the parallel flag, but rather increase the allowed concurrency so that you are able to run through all build processes first, and then accumulate long running dev processes without running out of the 10 default go-routines.

So have dev depend on ^build, but then run with turbo run dev --concurrency=50