go: cmd/compile: slow compilation of a package with a very large API

What version of Go are you using (go version)?

go1.18.2 windows/amd64

the OS doesn’t seem to be making a big difference, and I was experiencing the issue since at least go 17

Does this issue reproduce with the latest release?

yes

What operating system and processor architecture are you using (go env)?

set GO111MODULE=
set GOARCH=amd64
set GOBIN=
set GOCACHE=C:\Users\vibiret\AppData\Local\go-build
set GOENV=C:\Users\vibiret\AppData\Roaming\go\env
set GOEXE=.exe
set GOEXPERIMENT=
set GOFLAGS=
set GOHOSTARCH=amd64
set GOHOSTOS=windows
set GOINSECURE=
set GOMODCACHE=C:\Users\vibiret\go\pkg\mod
set GONOPROXY=
set GONOSUMDB=
set GOOS=windows
set GOPATH=C:\Users\vibiret\go
set GOPRIVATE=
set GOPROXY=https://proxy.golang.org,direct
set GOROOT=C:\Program Files\Go
set GOSUMDB=sum.golang.org
set GOTMPDIR=
set GOTOOLDIR=C:\Program Files\Go\pkg\tool\windows_amd64
set GOVCS=
set GOVERSION=go1.18.2
set GCCGO=gccgo
set GOAMD64=v1
set AR=ar
set CC=gcc
set CXX=g++
set CGO_ENABLED=1
set GOMOD=C:\sources\gitlocal\gorepro\go.mod
set GOWORK=
set CGO_CFLAGS=-g -O2
set CGO_CPPFLAGS=
set CGO_CXXFLAGS=-g -O2
set CGO_FFLAGS=-g -O2
set CGO_LDFLAGS=-g -O2
set PKG_CONFIG=pkg-config
set GOGCCFLAGS=-m64 -mthreads -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -fdebug-prefix-map=C:\Users\vibiret\AppData\Local\Temp\go-build1532428258=/tmp/go-build -gno-record-gcc-switches

What did you do?

go mod init main
go get github.com/microsoftgraph/msgraph-sdk-go@v0.24.1 #takes about 20 seconds on my machine, expected, this is the large package we're maintaining
go get github.com/Azure/azure-sdk-for-go/sdk/azidentity@v1.0.0 # takes about 5 seconds, expected
go get github.com/microsoft/kiota-authentication-azure-go@v0.3.0 # takes about 2 seconds, expected

notepad main.go #or code, vim,... whatever you have on your OS

# paste the content from the appendix

go build # takes about 1m30s, long but expected

Now replace the following lines

-result, err := client.Me().Messages().Get()
+result, err := client.Me().MailFoldersById("Inbox").Messages().Get()

Then run go build again, which takes about 30 seconds, not expected, should take a couple of seconds thanks to caching.

What did you expect to see?

Subsequent builds should be in the sub seconds time as the packages should be cached.

What did you see instead?

Takes about the original build time.

Additional context

We’re maintaining a large package that’s generated from an API description. The API itself is fairly large. The initial ask from our SDK users was about the build time increasing significantly when adding our package and caching seeming like it’s not fully working from one go build to another.

With the release of 1.18 and generics, we’re already identified opportunities for improvements to generate less code, but haven’t acted on it yet. @breml @josharian @dominikh were really helpful with framing the issue and suggested we posted here before making those improvements.

Additionally, the go doc for our package is not working either.

Appendix

main.go content

package main

import (
	"context"
	"fmt"

	azidentity "github.com/Azure/azure-sdk-for-go/sdk/azidentity"
	a "github.com/microsoft/kiota-authentication-azure-go"
	r "github.com/microsoft/kiota-http-go"
	msgraphsdk "github.com/microsoftgraph/msgraph-sdk-go"
)

func main() {
	cred, err := azidentity.NewDeviceCodeCredential(&azidentity.DeviceCodeCredentialOptions{
		TenantID: "placeholder",
		ClientID: "placeholder",
		UserPrompt: func(ctx context.Context, message azidentity.DeviceCodeMessage) error {
			fmt.Println(message.Message)
			return nil
		},
	})
	if err != nil {
		fmt.Printf("Error creating credentials: %v\n", err)
	}
	auth, err := a.NewAzureIdentityAuthenticationProviderWithScopes(cred, []string{"Mail.Read", "Mail.Send"})
	if err != nil {
		fmt.Printf("Error authentication provider: %v\n", err)
		return
	}
	adapter, err := r.NewNetHttpRequestAdapter(auth)
	if err != nil {
		fmt.Printf("Error creating adapter: %v\n", err)
		return
	}
	client := msgraphsdk.NewGraphServiceClient(adapter)
	result, err := client.Me().Messages().Get()
	if err != nil {
		fmt.Printf("Error executing request: %v\n", err)
		return
	}
	if result != nil {
		fmt.Printf("Got a result")
		return
	}
}