Pluto.jl: Pkg activate fails to load the correct versions of the packages

I am working on this now with @Pangoraw, the discussion about the LOAD_PATH above should be corrected: we only set a custom LOAD_PATH when you use Pluto’s package manager, when you use Pkg.activate or @quickactivate, the LOAD_PATH is unchanged (i.e. it is ["@", "@v#.#", "@stdlib"]). So the LOAD_PATH is unrelated to our problem today.

---

The reason why the notebook from https://github.com/fonsp/Pluto.jl/issues/1511#issuecomment-947210943 is not working is: you cannot use a macro in the same expression that you define it. (begin end groups the two into a single expression).

In the Julia REPL:

julia> begin
         using DrWatson
         @quickactivate "MyClass"
       end
ERROR: LoadError: UndefVarError: @quickactivate not defined
in expression starting at REPL[2]:3

julia> using DrWatson

julia> @quickactivate # in its own expression, after `using DrWatson` has been evaluated into the module
┌ Warning: DrWatson could not find find a project file by recursively checking given `dir` and its parents. Returning `nothing` instead.
│ (given dir: /Users/fons)
└ @ DrWatson ~/.julia/packages/DrWatson/MbII9/src/project_setup.jl:84

This is the case for any Julia macro, including @quickactivate. The reason that it worked before was because of a bug in our execution model, where we evaluated all using statements before evaluating a cell, which is (unfortunately) fixed in 0.16.2. We think it is the right decision to fix this bug, because our Julia execution should be the same as running the notebook as a script.

The solution will be to split the quickactivate cell in two:

using DrWatson

@quickactivate ...

begin
	...
end

This will not work properly in 0.16.2 because the cell order heuristic needs to be updated to make this work. We are working on this now, and it should be released in Pluto 0.16.3 today.

@Pangoraw fixed it and we now have a test specifically for @quickactivate:

https://github.com/fonsp/Pluto.jl/commit/b7021ae38dfd194b66d2c3a1d44f7a4bc427254f

Released as 0.16.3

It still means that notebooks with this code:

begin
	using DrWatson
	@quickactivate "..."
	...
end

will have to be rewritten, and maybe the documentation in DrWatson can be updated with the new Pluto example?

using DrWatson

@quickactivate ...

using Plots
...

Pluto sets your LOAD_PATH variable to [“@”, “@stdlib”] to prevent pulling in packages from the global environment

This argument does not make sense to me. The standard Julia load path still has @v#.#, yet it still works fine using the standard package manager and identifies correct package versions. So my understanding is you didn’t fix the bug, you just alltogether dissalowed the (normally existing standard Julia behavior) of the global environment being accessible for its pool of packages.

Having to do all these weird stuff of changing LOAD_PATH seems way too convoluted. A typical Julia user would never have heard of LOAD_PATH, and frankly, shouldn’t have to. If someone uses only exclusively Pluto.jl, things work. But as soon as one wants to use Pluto along with other editors, things become unnecessarily convoluted.

Pluto sets your LOAD_PATH variable to ["@", "@stdlib"] to prevent pulling in packages from the global environment. This issue reports a bug where Pluto would be using Packages when the load path was not yet set to this value. To use DrWatson you have to set the LOAD_PATH to ["@", "@v#.#", "@stdlib"] before using it.

Also note that @quickactivate being a macro it needs to be in a different cell than using DrWatson:

The import Pkg was added to workaround the execution order heuristics which do not play nicely with DrWatson and should probably be fixed on Pluto’s side.

@Pangoraw found the source of the bug! Fix coming soon. Thanks again for narrowing down to the issue!