haddock: Segfault when compiling haskell-src-exts

This is pretty much a duplicate of #837, but since that issue has been closed but the problem still exists, I am opening a new one.

I have:

$ stack --version
Version 1.7.1, Git revision 681c800873816c022739ca7ed14755e85a579565 (5807 commits) x86_64 hpack-0.28.2
$ `stack exec which haddock` --version
Haddock version 2.20.0, (c) Simon Marlow 2006
Ported to use the GHC API by David Waern 2006-2008

and when I try to build the docs for some project that uses haskell-src-exts I get:

$ stack haddock --resolver=lts-12.7
haskell-src-exts-1.20.2: configure
haskell-src-exts-1.20.2: build
haskell-src-exts-1.20.2: haddock
Progress 1/17

--  While building custom Setup.hs for package haskell-src-exts-1.20.2 using:
      [..]/.stack/setup-exe-cache/x86_64-linux/Cabal-simple_mPHDZzAJ_2.2.0.1_ghc-8.4.3 --builddir=.stack-work/dist/x86_64-linux/Cabal-2.2.0.1 haddock --html --hoogle --html-location=../$pkg-$version/ --haddock-option=--hyperlinked-source
    Process exited with code: ExitFailure (-11)
    Logs have been written to: [..]/.stack-work/logs/haskell-src-exts-1.20.2.log

    Configuring haskell-src-exts-1.20.2...
    Preprocessing library for haskell-src-exts-1.20.2..
    Building library for haskell-src-exts-1.20.2..
    [ 1 of 17] Compiling Language.Haskell.Exts.Extension ( src/Language/Haskell/Exts/Extension.hs, .stack-work/dist/x86_64-linux/Cabal-2.2.0.1/build/Language/Haskell/Exts/Extension.o )
    [ 2 of 17] Compiling Language.Haskell.Exts.ExtScheme ( src/Language/Haskell/Exts/ExtScheme.hs, .stack-work/dist/x86_64-linux/Cabal-2.2.0.1/build/Language/Haskell/Exts/ExtScheme.o )
    [ 3 of 17] Compiling Language.Haskell.Exts.SrcLoc ( src/Language/Haskell/Exts/SrcLoc.hs, .stack-work/dist/x86_64-linux/Cabal-2.2.0.1/build/Language/Haskell/Exts/SrcLoc.o )
    [ 4 of 17] Compiling Language.Haskell.Exts.Syntax ( src/Language/Haskell/Exts/Syntax.hs, .stack-work/dist/x86_64-linux/Cabal-2.2.0.1/build/Language/Haskell/Exts/Syntax.o )
    [ 5 of 17] Compiling Language.Haskell.Exts.ParseSyntax ( src/Language/Haskell/Exts/ParseSyntax.hs, .stack-work/dist/x86_64-linux/Cabal-2.2.0.1/build/Language/Haskell/Exts/ParseSyntax.o )
    [ 6 of 17] Compiling Language.Haskell.Exts.Pretty ( src/Language/Haskell/Exts/Pretty.hs, .stack-work/dist/x86_64-linux/Cabal-2.2.0.1/build/Language/Haskell/Exts/Pretty.o )
    [ 7 of 17] Compiling Language.Haskell.Exts.Fixity ( src/Language/Haskell/Exts/Fixity.hs, .stack-work/dist/x86_64-linux/Cabal-2.2.0.1/build/Language/Haskell/Exts/Fixity.o )
    [ 8 of 17] Compiling Language.Haskell.Exts.Comments ( src/Language/Haskell/Exts/Comments.hs, .stack-work/dist/x86_64-linux/Cabal-2.2.0.1/build/Language/Haskell/Exts/Comments.o )
    [ 9 of 17] Compiling Language.Haskell.Exts.ParseMonad ( src/Language/Haskell/Exts/ParseMonad.hs, .stack-work/dist/x86_64-linux/Cabal-2.2.0.1/build/Language/Haskell/Exts/ParseMonad.o )
    [10 of 17] Compiling Language.Haskell.Exts.ParseUtils ( src/Language/Haskell/Exts/ParseUtils.hs, .stack-work/dist/x86_64-linux/Cabal-2.2.0.1/build/Language/Haskell/Exts/ParseUtils.o )
    [11 of 17] Compiling Language.Haskell.Exts.InternalLexer ( src/Language/Haskell/Exts/InternalLexer.hs, .stack-work/dist/x86_64-linux/Cabal-2.2.0.1/build/Language/Haskell/Exts/InternalLexer.o )
    [12 of 17] Compiling Language.Haskell.Exts.Lexer ( src/Language/Haskell/Exts/Lexer.hs, .stack-work/dist/x86_64-linux/Cabal-2.2.0.1/build/Language/Haskell/Exts/Lexer.o )
    [13 of 17] Compiling Language.Haskell.Exts.InternalParser ( .stack-work/dist/x86_64-linux/Cabal-2.2.0.1/build/Language/Haskell/Exts/InternalParser.hs, .stack-work/dist/x86_64-linux/Cabal-2.2.0.1/build/Language/Haskell/Exts/InternalParser.o )
    [14 of 17] Compiling Language.Haskell.Exts.Parser ( src/Language/Haskell/Exts/Parser.hs, .stack-work/dist/x86_64-linux/Cabal-2.2.0.1/build/Language/Haskell/Exts/Parser.o )
    [15 of 17] Compiling Language.Haskell.Exts.ExactPrint ( src/Language/Haskell/Exts/ExactPrint.hs, .stack-work/dist/x86_64-linux/Cabal-2.2.0.1/build/Language/Haskell/Exts/ExactPrint.o )
    [16 of 17] Compiling Language.Haskell.Exts.Build ( src/Language/Haskell/Exts/Build.hs, .stack-work/dist/x86_64-linux/Cabal-2.2.0.1/build/Language/Haskell/Exts/Build.o )
    [17 of 17] Compiling Language.Haskell.Exts ( src/Language/Haskell/Exts.hs, .stack-work/dist/x86_64-linux/Cabal-2.2.0.1/build/Language/Haskell/Exts.o )
    ignoring (possibly broken) abi-depends field for packages
    Preprocessing library for haskell-src-exts-1.20.2..
    Running Haddock on library for haskell-src-exts-1.20.2..

Posts on the Haskell mailing list suggested that replacing some haddock executable somewhere would solve the issue, but I have not found a good explanation on how/what/where to do this.

About this issue

Original URL
State: closed
Created 6 years ago
Comments: 17 (7 by maintainers)

Commits related to this issue

rest of docs, switch to a more recently nightly stack resolver in order to deal with https://github.com/haskell/haddock/issues/928 — committed to cronburg/antlr-haskell by cronburg 6 years ago
Do not build haddocks for dependencies (see bug https://github.com/haskell/haddock/issues/928) — committed to mtesseract/nakadi-client by mtesseract 5 years ago

Most upvoted comments

@nh2 I don’t remember the details anymore, but the issue boiled down to overrunning a StringBuffer in the GHC API. The hyperlinker backend used to take the following approach to tokenizing:

ask GHC to lex Haskell source portions of source files (using lexTokenStream)
ask GHC to re-associate those tokens with strings in the initial source (using addSourceToTokens)

If the initial source from step 1 had {-# LINE ... #-} or {-# COLUMN ... #-} pragmas in it, these would update the location associated with the tokens produced in step 1. Then, in step 2, addSourceToTokens would get confused by token positions which were in different files. Since that function doesn’t do much more than just advancing (with nextChar) a StringBuffer until token positions matched, positions which came from different files would cause addSourceToTokens to skip straight past the end of StringBuffer.

The GHC 8.6 fix was to add a GHC option to prevent location-updating pragmas from messing with the internal position of the lexer.
Since GHC 8.8, this has been fixed more thoroughly by avoiding footgun functions like addSourceToTokens altogether.

On Linux, that usually manifested as a segfault. On Mac, I experienced hanging behaviour instead.

harpocrates on Apr 8, 2019

Closing now, given that the 8.8 release is eminent and that this is fixed in all of 8.6.

harpocrates on Feb 25, 2019