delve: Cannot debug go plugins on mac

Debugging plugin on mac is really needed for Golang developers. Hope delve supports it 😃

When i read trough your project you generate some wrapper code to compile code that was written as go plugin into the main binary ?

Yes.

for me its something compiled with buildmode=plugin that gets loaded at runtime.

If --staticLinking is NOT specified when building a plugin, it is still a perfect plugin, which is not debuggable on Mac. hotswap helps me debug the code in a plugin. It is a problem solver, not a curer:)

--staticLinking is a per build choice.

FYI Sonoma (or rather the version of xcode that ships with sonoma) still has an old version of dsymutil.

I checked out the latest version of macOS (13.4) and it still has a bugged version of dsymutil.

Delve’s side of this needs to be implemented. At the moment I don’t have access to a working VM (and none of the builders have a version of macos that will receive the updated version of xcode) so I’m not going to do it any time soon, if someone else wants to do it: it should be fairly simple, just add a call to getLoadedDynamicLibraries at the end of ContinueOnce and call bi.AddImage for all the dynamic libraries.

I discovered that the fix for https://github.com/golang/go/issues/27502 is present in go1.14beta1, so I installed that version:

go get golang.org/dl/go1.14beta1
go1.14beta1 download

Then I rebuilt the binaries from the example project:

go1.14beta1 build -gcflags 'all=-N -l' -buildmode=plugin -o eng/eng.so eng/greeter.go
go1.14beta1 build -gcflags 'all=-N -l' -o greeter greeter.go

I then used otool -l to validate that the eng.so binary had the DWARF debug sections.

✗ otool -l eng/eng.so | grep debug
  sectname __zdebug_line
  sectname __zdebug_pubname
  sectname __zdebug_loc
  sectname __zdebug_ranges
  sectname __zdebug_pubtype
  sectname __zdebug_aranges
  sectname __zdebug_info
  sectname __zdebug_frame
  sectname __zdebug_abbrev
  sectname __zdebug_str

So I got my hopes up, but when I tried debugging the greeter binary I hit the same problem.

✗ dlv --check-go-version=false exec ./greeter -- english
Type 'help' for list of commands.
(dlv) b greeter.go:58
Breakpoint 1 set at 0x40f33ac for main.main() ./greeter.go:58
(dlv) c
> main.main() ./greeter.go:58 (hits goroutine(1):1 total:1) (PC: 0x40f33ac)
    53:			fmt.Println("unexpected type from module symbol")
    54:			os.Exit(1)
    55:		}
    56:	
    57:		// 4. use the module
=>  58:		greeter.Greet()
    59:	
    60:	}
(dlv) b eng/greeter.go:8
Command failed: Location "eng/greeter.go:8" not found
(dlv) s
Stopped at: 0x7ce92d0
=>no source available
(dlv) s
Hello Universe
Process has exited with status 0
(dlv) 

This is a major obstacle for me as I am working on same code base that uses plugins and I often switch between Linux and Mac while working on this.

Meantime I believe lack of debugging symbols in plugins on Darwin is still an issue in Go https://github.com/golang/go/issues/27502

@aarzilli : I’ve put in a comment on that issue. But frankly I’m too ignorant about this topic to understand what the issues are here, and how exactly golang/go#25841 is blocking this issue.

e.g., I don’t understand how you obtained the output that you cite, they look like they came from running readelf, but the ticket is about Mac. So I’m guessing maybe you targeted GOOS=linux in the build. And if so then I’m not seeing how that relates to this issue.

Furthermore, I seem to be seeing at least 2 different problems in my example above, but I’m not sure if they are symptoms of one underlying cause.

cannot set the breakpoint

On Mac, where it fails in this scenario, it looks like this:

(dlv) b eng/greeter.go:8
Command failed: Location "eng/greeter.go:8" not found

On linux where it works it looks like this:

(dlv) b eng/greeter.go:8
Breakpoint 2 set at 0x7f7d8c5c838f for plugin/unnamed-380bc3019cae9f99f0f646d134e7c21e1405b74f.greeting.Greet() ./eng/greeter.go:8

cannot step into the plugin code

On Mac. where it fails in this scenario, it looks like this:

(dlv) c
> main.main() ./greeter.go:58 (hits goroutine(1):1 total:1) (PC: 0x40f33ac)
    53:			fmt.Println("unexpected type from module symbol")
    54:			os.Exit(1)
    55:		}
    56:	
    57:		// 4. use the module
=>  58:		greeter.Greet()
    59:	
    60:	}
(dlv) s
Stopped at: 0xcde92d0
=>no source available

On linux where it works it looks like this:

(dlv) c
> main.main() ./greeter.go:58 (hits goroutine(1):1 total:1) (PC: 0x533921)
    53:			fmt.Println("unexpected type from module symbol")
    54:			os.Exit(1)
    55:		}
    56:	
    57:		// 4. use the module
=>  58:		greeter.Greet()
    59:	
    60:	}
(dlv) s
> plugin/unnamed-380bc3019cae9f99f0f646d134e7c21e1405b74f.(*greeting).Greet() <autogenerated>:1 (PC: 0x7f972c3cf421)
(dlv) s
> plugin/unnamed-380bc3019cae9f99f0f646d134e7c21e1405b74f.greeting.Greet() ./eng/greeter.go:7 (PC: 0x7f972c3cf381)
     2:	
     3:	import "fmt"
     4:	
     5:	type greeting string
     6:	
=>   7:	func (g greeting) Greet() {
     8:		fmt.Println("Hello Universe")
     9:	}
    10:	
    11:	// exported
    12:	var Greeter greeting
(dlv) s
> plugin/unnamed-380bc3019cae9f99f0f646d134e7c21e1405b74f.greeting.Greet() ./eng/greeter.go:8 (PC: 0x7f972c3cf38f)
     3:	import "fmt"
     4:	
     5:	type greeting string
     6:	
     7:	func (g greeting) Greet() {
=>   8:		fmt.Println("Hello Universe")
     9:	}
    10:	
    11:	// exported
    12:	var Greeter greeting