cue: evalv3: high memory usage with top-level disjunctions

What version of CUE are you using (cue version)?

$ go run cuelang.org/go/cmd/cue@v0.10.0-alpha.2.0.20240726151100-e68bd632c29e version
cue version v0.10.0-alpha.2.0.20240726151100-e68bd632c29e

go version go1.22.5
      -buildmode exe
       -compiler gc
     CGO_ENABLED 1
          GOARCH amd64
            GOOS linux
         GOAMD64 v1
cue.lang.version v0.10.0

Does this issue reproduce with the latest stable release?

Yes.

What did you do?

After following this to fix another issue, I thought it would be interesting to try the new evaluator again.

https://github.com/cue-lang/cue/issues/3330#issuecomment-2252789316

❯ CUE_EXPERIMENT=evalv3 go run cuelang.org/go/cmd/cue@v0.10.0-alpha.2.0.20240726151100-e68bd632c29e export ./k8s/amour/list.cue
signal: killed

https://github.com/uhthomas/automata/commit/5a1a4fef98e2bf47227079f362e30404eed20674

What did you expect to see?

Successful build.

What did you see instead?

OOM, signal killed.

image

About this issue

  • Original URL
  • State: open
  • Created 2 years ago
  • Comments: 20 (20 by maintainers)

Commits related to this issue

Most upvoted comments

That said, we still think we can get another 10-30% reduction in allocations relatively easily, so we will definitely attempt that.

I love reading these updates, thanks for your hard work. So exciting to see such significant progress! I’m also grateful my repository has been helpful for evaluating performance.

Significant progress; I am now able to finish evaluating with evalv3, even if it’s still quite a bit slower than evalv2:

$ for n in 0 1; do time CUE_EXPERIMENT=evalv3=$n cue export ./k8s/amour/list.cue | wc -l; done
97333

real	0m5.992s
user	0m11.232s
sys	0m0.224s
97333

real	1m44.507s
user	2m54.228s
sys	0m8.351s

With @mpvl we believe the remaining work here is memory management, which evalv2 did to some extent, but evalv3 does not do yet. This is also why evalv3 above takes more than 20GiB of memory to finish.