go: Go 1.12.1 dsymutil segmentation fault on MacOS Sierra

For posterity, here’s what’s happening with this bug.

The function causing the issue is here:

 // ReadValues will parse YAML byte data into a Values.
  func ReadValues(data []byte) (vals Values, err error) {
	  err = yaml.Unmarshal(data, &vals)
	  if len(vals) == 0 {
		  vals = Values{}
	  }
	  return
  }

This is small enough to be an inlining candidate; it gets inlined in its home package and in other packages that import that package.

During the home package compile, ‘vals’ is promoted to the heap, and in the process the dcl node for the param updated, converting its class from PPARAMOUT to PAUTOHEAP. This happens at esc.go:2175.

This change in class confuses the DWARF generation code, specifically createDwarfVars. The code there winds up treating PAUTOHEAP the same as PAUTO, hence ‘vals’ is reclassified as a local and not an output parameter. This has a cascading effect and means that the abstract subprogram DIE generated for chartutil.ReadValues is slightly different in its home package relative to what gets emitted in other packages that inline it.

The fix is fairly straightforward in createDwarfVars(); I also wrote a separate patch to add some checking to the linker to try to detect these cases early and on Linux (at the moment it seems that we only see problems on Macos and with specific more strict versions of dsymutil).

@benesch thanks for the heads-up. Backport seems reasonable. I will discuss with the other compiler/runtime folks.

Wed Mar 20 13:28:20 EDT 2019

I ran dsymutil in the debugger and it does indeed look as though it is encountering bad DWARF.

Offending compiliation unit is here:

<0><d5847a>: Abbrev Number: 1 (DW_TAG_compile_unit)
    <d5847b>   DW_AT_name        : github.com/replicatedhq/ship/vendor/k8s.io/helm/pkg/chartutil
    <d584b9>   DW_AT_language    : 22	(Go)

It appears that the routine ‘chartutil.ReadValues’ is being inlined into ‘chartutil.CoalesceValues’. The inlined routine subprogram DIE being generated looks like

<2><d5ba90>: Abbrev Number: 7 (DW_TAG_inlined_subroutine)
    <d5ba91>   DW_AT_abstract_origin: <0xd588c7>
    <d5ba95>   DW_AT_ranges      : 0x4d7c10
    <d5ba99>   DW_AT_call_file   : 0x1a
    <d5ba9d>   DW_AT_call_line   : 165
 <3><d5ba9f>: Abbrev Number: 19 (DW_TAG_formal_parameter)
    <d5baa0>   DW_AT_abstract_origin: <0xd58913>
    <d5baa4>   DW_AT_location    : 0xd4d139 (location list)
 <3><d5baa8>: Abbrev Number: 19 (DW_TAG_formal_parameter)
    <d5baa9>   DW_AT_abstract_origin: <0xd5892a>
    <d5baad>   DW_AT_location    : 0xd4d1c1 (location list)
 <3><d5bab1>: Abbrev Number: 0

The second formal parameter abstract_origin offset (0xd5892a) is bad – it looks as though it’s coming out somewhere in between the formal parameter DIEs in the subprogram it’s referring to. So almost certainly some sort of compiler problem here.

I’ll grab the bug if that is ok and see about trying to come up with a compiler test case.

I can reproduce. Additional commands I needed (in case anyone else needs to reproduce):

brew install yarn
go get -u github.com/replicatedhq/ship/pkg/cli

There’s at least a bug in dsymutil. It probably shouldn’t segfault, no matter the input we give it. Presumably we’re generating some dwarf data that dsymutil can’t handle. I don’t know what that might be. I’m having trouble convincing the go tool to leave behind enough work directories to reproduce directly with dsymutil. Still working on that, will probably have to wait until tomorrow.