prql: Playground build is slow, even without changes

What’s up?

@eitsupi - This isn’t quite a bug, but the devcontainer for commit d427c86 takes a long time (many minutes) to start up after it has already been built. Here’s what I saw:

Checkout prql d427c86 (compiler 0.6.0)
Start VS Code, it grinds away builds the new Dev Container. When it’s done…
Open a zsh terminal and enter task run-book when it’s done, you’ll be able to work on the book
Quit from VS Code

Then…

Launch VS Code, start the Dev Container
20-30 seconds later, you see the Dev Container CLI prompt vscode ➜ /workspaces/prql (main) $
Type task run-book, see the Blocking waiting for file lock on build directory message

This has persisted for 10+ minutes this morning. I will also note that during this time, the CPU on my computer was pretty heavily used, even to the point it interferes with GUI interactions.

Am I missing something? Any recommendations about what to do? Many thanks.

About this issue

Original URL
State: closed
Created a year ago
Comments: 17 (17 by maintainers)

Most upvoted comments

It’s at a different level though — at the moment npm install in playground runs npm install on prql-js, which runs wasm-pack, which runs wasm-opt. So unless we tag every source for npm install of playground, including prql-js and therefore anything in prql-compiler (!), we can’t use Taskfile here.

Thanks for explaining this. I may understand your explanation and hope the wasm-pack will improve.

eitsupi on Mar 19, 2023

Update: I think I gave it more memory successfully. I bumped Memory from 10 to 12 GBytes and Virtual Disk from 64 to 72GBytes. See image below. (I am not sure that this made a difference.) Here’s what I see:

From a fresh pull of current state of the main branch:

Start VS Code. Open the DevContainer. Let it start up, doing whatever it does, maybe it’s rebuilding the Dev Container. I show the log for the Dev Container to watch the proceedings
Open a new terminal window then task run-website- this starts pretty quickly
^C and then task run-book - This gives the “blocking waiting…” message. If I ^C and wait a while (many minutes) this command proceeds and requires a rebuild of a few cargo items (prql? mdbook? I don’t remember exactly) and takes a minute or two before it starts
^C and then task run-playground This requires a npm install for two or three minutes, then the playground opens.

Quit VS Code, then relaunch it. Here’s what I see:

Startup OK, show Dev Container log.
task run-website in a new window is pretty fast (that is, it’s available acceptably quickly)
task run-book - this time, acceptably fast, no rebuild.
task run-playground - always seems to perform npm install that takes several minutes

Docker-Advanced Settings

richb-hanover on Mar 18, 2023

As I mentioned below, unfortunately you are following the correct steps and this is a known limitation. https://github.com/PRQL/prql/pull/1893#issuecomment-1455910999

In short, a build that includes duckdb (which is automatically executed by rust-analizer) requires high machine power and takes a long time. And as you have seen, duckdb builds often fail!

However, I think duckdb is only used in the test, so I don’t think it has anything to do with the book build.

eitsupi on Mar 13, 2023