runtime: Stack overflow in F# compiler. on net5.0

With dotnet 5.0 we have seen a new stack overflow introduced in the F# compiler when running on the coreclr.

Repro: unzip StackOverflow.zip to your local zip.

The project assumes that a dotnet sdk with net5.0 is installed in c:\program files\dotnet

build this project and run it

dotnet build -t:Rebuild -c release
dotnet run -c release

observe the stack overflow.

If the project file is edited to netcoreapp3.1 and a netcoreapp3.1 runtime is installed, then executing the program will succeed without a stack overflow.

About this issue

Original URL
State: closed
Created 3 years ago
Comments: 23 (22 by maintainers)

Commits related to this issue

Disable - LargeRecordDoesNotStackOverflow due to: https://github.com/dotnet/runtime/issues/47663 (#10979) — committed to dotnet/fsharp by KevinRansom 3 years ago

Most upvoted comments

Using editbin and some binary searching, the 3.1 app’s max stack is between 0x110000 and 0x111000 bytes, and 5.0 is between 0x180000 and 0x181000.

At point of overflow, 3.1 has 8983 active frames; 5.0 has 14612 frames.

Comparing the recurring patterns (with frame sizes in hex) we see things like:

;; 3.1
80 @291.Invoke
40 targetF
50 tail call helper stub
B0 foldUnderLambda

;; 5.0
80 @291.Invoke
40 targetF
60 IL_STUB_CallTailCallTarget
50 RuntimeHelpers.DispatchTailCalls
80 exprF
80 foldUnderLambda

So there is an additional x60 bytes of stack from stubs and 0x50 from exprF’s frame remaining on stack.

Overall what’s happening is that each new “run” of tail calls on 5.0 has higher overhead than it did in 3.1 and this case has a recurring pattern where it sets up ~2500 of these runs, and that leads to higher stack usage overall, enough to cause overflow.

AndyAyersMS on Mar 20, 2021

Thanks, this repros. Partial stack trace:

   at System.Runtime.CompilerServices.RuntimeHelpers.DispatchTailCalls(IntPtr, Void (IntPtr, IntPtr, IntPtr*), IntPtr)
   at FSharp.Compiler.TypedTreeOps+ExprFolders`1[[System.__Canon, System.Private.CoreLib, Version=5.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e]].exprF(System.__Canon, Expr)
   at FSharp.Compiler.Detuple+GlobalUsageAnalysis.foldUnderLambda[[System.__Canon, System.Private.CoreLib, Version=5.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e]](Microsoft.FSharp.Core.FSharpFunc`2<Results,Microsoft.FSharp.Core.FSharpFunc`2<System.__Canon,Results>>, Results, System.__Canon)
   at System.Runtime.CompilerServices.RuntimeHelpers.DispatchTailCalls(IntPtr, Void (IntPtr, IntPtr, IntPtr*), IntPtr)
   at FSharp.Compiler.TypedTreeOps+ExprFolders`1[[System.__Canon, System.Private.CoreLib, Version=5.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e]].exprF(System.__Canon, Expr)
   at FSharp.Compiler.TypedTreeOps+ExprFolders`1[[System.__Canon, System.Private.CoreLib, Version=5.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e]].exprNoInterceptF(System.__Canon, Expr)
   at System.Runtime.CompilerServices.RuntimeHelpers.DispatchTailCalls(IntPtr, Void (IntPtr, IntPtr, IntPtr*), IntPtr)
   at FSharp.Compiler.TypedTreeOps+ExprFolders`1[[System.__Canon, System.Private.CoreLib, Version=5.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e]].exprF(System.__Canon, Expr)
   at FSharp.Compiler.Detuple+GlobalUsageAnalysis.foldUnderLambda[[System.__Canon, System.Private.CoreLib, Version=5.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e]](Microsoft.FSharp.Core.FSharpFunc`2<Results,Microsoft.FSharp.Core.FSharpFunc`2<System.__Canon,Results>>, Results, System.__Canon)

6.0 seems to hang or is very slow? It is perhaps trying to serialize out this stack trace somewhere.

AndyAyersMS on Jan 30, 2021