WSL: `stack ghc` painfully slow
-
A brief description Managing haskell projects with the
stacktool is unusable due to how slow it is. -
Expected results (from a laptop running ubuntu 16.04)
time stack ghc -- --version
The Glorious Glasgow Haskell Compilation System, version 8.0.1
real 0m0.124s
user 0m0.092s
sys 0m0.036s
- Actual results (with terminal output if applicable) On desktop running WSL
time stack ghc -- --version
The Glorious Glasgow Haskell Compilation System, version 8.0.1
real 0m50.520s
user 0m0.172s
sys 1m40.547s
- Your Windows build number 15025
- Steps / All commands required to reproduce the error from a brand new installation After installation, need stack to pull in a version of GHC. This should do the trick.
stack setup
stack upgrade --install-ghc
time stack ghc -- --version
- Strace of the failing command Generating the strace output (attached) inludes a few long (multi-second) waits on FUTEX_WAIT, as well as one for mmap.
- Required packages and commands to install Install stack with the standard instructions
About this issue
- Original URL
- State: closed
- Created 10 years ago
- Reactions: 13
- Comments: 46 (7 by maintainers)
In the hopes this may be useful to somebody here: I’ve set up a GHC PPA optimised for WSL (i.e. built with
--disable-large-address-space) over athttps://launchpad.net/~hvr/+archive/ubuntu/ghc-wsl
It should merely be a matter of
and then simply prepending
/opt/ghc/bin/to your$PATHenv-var.We have improved mmap performance further in insider build 17063. I believe this makes stack ghc bearable to use now 😃.
This is on our backlog but is unlikely to make the Creators Update. I know we’re planning on looking at this soon though.
For some context, I’ve looked at what causes this slowdown. For some reason stack has mapped an mind-bogglingly huge region of memory (I’m talking dozens of terabytes). When we fork we walk the entire address range to set up the new process’s state. We have a design that should vastly speed this up, but we’re approaching “pencils down” date for Creators Update.
I have a workaround in the meantime, based on the discussion in https://ghc.haskell.org/trac/ghc/ticket/13304. It involves compiling your own GHC which does not utilize the large address space allocation, and then using that as the GHC for your stack builds. My workaround relies on then supplying that GHC as the common GHC for your projects. In my example, I will recompile GHC 8.0.2 using whatever ghc you already have on your system. I will also make sure that Cabal is installed using this GHC – otherwise, installing other packages will fall under the same problem of slowness. I suggest cleaning your
~/.stackand other stack directories to make sure you don’t have any GHC lying around with the large-allocation functionality.To fix, in the bash environment, I ran
Now you can do your
stack installandstack buildin your projects, using the specially compiled GHC.You can monitor the VIRT usage with something like
toporhtop. Trystack exec ghciand monitor VIRT before and after.@benhillis: I believe you’ve bumped into GHC 8.0’s new block-structured heap for 64-bit platforms. From the GHC 8.0.1 release notes:
For the
stack ghc -- --versioncommand, the runtime improved from 116 seconds to 13 seconds on my test vm. Similar to nvm, we understand where the remaining time is being spent and are tracking additional work to bring this down closer to native Linux speeds.At this point, it probably makes sense to do all aspects of Haskell development within a linux VM.
Performance is much better in 16215 for what it’s worth. But probably still not where you want to be if you compile Haskell all day long.
@pierrebeaucamp - Or non-desktop Ubuntu on VirtualBox with
vboxsf, if you find all the copying a pain.vboxsfis the spiritual equivalent of DrvFS (/mnt/c) on WSL. Or automate the copying withrsync, which has been standard operating procedure forever.@mstksg - I’d put up a PPA for you but I don’t ‘do’ Haskell and would be the wrong point person. That’s probably a good interim solution here though. If no one steps up in a couple of weeks ping the thread and maybe it’ll do it. But first ask over in the Haskell community. It isn’t strictly speaking a WSL Thing. The PPA is “GHC without large address space” (which would run on Real Linux too), not “GHC for WSL”. I bet a runtime option is even possible with some refactoring and deep thought, though only academically because no one would bother with the effort. WSL will catch up eventually.
@RyanGlScott - I suspect you are right. We need to modify the way our memory manager keeps track of uncommitted pages.
I’d be very curious to see some performance measurements on how much better their allocator performs versus raw mmap / munmap calls.
Another anecdote: I opted into 17074 hoping to get acceptable working conditions, but
stack setuptook exactly an hour to complete (even with windows defender temporarily disabled). For comparison,stack setupundercmd.exestarting from scratch took ~4 minutes. I’ll give working with the result a shot, but it doesn’t look promising.Sorry for the negative report; keep doing great work, you’ll get there. ❤️
@Roman2K - Thanks for the information. I’m glad that it’s much more usable for you, but we still do have a long way to go. We’re looking into ways to improve base NTFS speed to help bring Windows filesystem performance more in line with Linux.
On the latest stable non-insider as of 31/07/2017, I noticed untars and
configurescripts being extremely slow. Looking at the Task Manager, the Antimalware Service Executable seems the be the culprit, as though file writes are being inspected through that service. Both untars andconfigures do lots of small file writes.Antimalware only takes ~5% of the CPU so maybe the slow part is the transfer of contents from WSL to that process. But I don’t have any idea of what I’m talking about 😅.
So I tried the insider preview 16251 and immediately noticed a huge difference 😍. Don’t have any numbers from before to compare to, unfortunately, but it feels twice as fast. Though still a lot slower than virtualized Linux.
Here are
times from WSL compared to Alpine in VirtualBox on the same machine, within a tmpfs for reads and writes:Sorry I don’t have more to provide. I hope you will keep on improving WSL, that’s excellent work on an fantastic concept.
@mstksg I don’t know if that is an option to you, but I switched to docker to compile my haskell projects for now. It’s not ideal (it involves a lot of copying between the host and the VM) but at least the compile time is more reasonable.
@AaronFriel the fix inbound label is for the improvement mentioned above (i.e. 116 to 13 seconds). Should be much less painful 😃, and we are tracking the work to get this closer to native Linux.
@pechersky This issue affects not only GHC 8, but also anything compiled with it (
stack,pandoc, etc). The official binaries provided bystackdevelopers happen to run fine because the latest release version is built withlts-6.25and usesghc-7.10.3.FWIW, Golang also does something similar by reserving a contiguous chunk of 512 GB of memory (see this comment).
I’m certainly not qualified enough to say how they came up with that number, other than that it’s a power of two and—to use their words—“512 GB (MaxMem) should be big enough for now”.