operator-sdk: Deepcopy generation fails due to "unsupported type invalid type for invalid type"

Bug Report

When migrating an Operator that works with the SDK 0.8.2 to 0.10.0, the generate k8s command fails with a message like Hit an unsupported type invalid type for invalid type, from github.com/jaegertracing/jaeger-operator/pkg/apis/io/v1alpha1.Jaeger. It’s unclear which property has an invalid type, so, I commented out all custom types, ending up with a type like this:

type Jaeger struct {
	metav1.TypeMeta   `json:",inline"`
	metav1.ObjectMeta `json:"metadata"`
	// Spec              JaegerSpec   `json:"spec"`
	// Status            JaegerStatus `json:"status,omitempty"`
}

Even for this type, the generate k8s fails with the same message. The code for this can be seen on this branch: https://github.com/jpkrohling/jaeger-operator/tree/607-Update-Operator-SDK-to-0.10.0

What did you do? Clone the branch mentioned above and run:

$ GO111MODULE=on operator-sdk generate k8s
INFO[0000] Running deepcopy code-generation for Custom Resource group versions: [io:[v1alpha1], jaegertracing:[v1], ] 
F0822 11:01:11.492974   28811 deepcopy.go:885] Hit an unsupported type invalid type for invalid type, from github.com/jaegertracing/jaeger-operator/pkg/apis/io/v1alpha1.Jaeger
goroutine 1 [running]:
k8s.io/klog.stacks(0xc0003caf00, 0xc0006d4500, 0xb0, 0x100)
	pkg/mod/k8s.io/klog@v0.3.1/klog.go:855 +0xb1
k8s.io/klog.(*loggingT).output(0x33f8420, 0xc000000003, 0xc00030abd0, 0x323b24c, 0xb, 0x375, 0x0)
	pkg/mod/k8s.io/klog@v0.3.1/klog.go:806 +0x2d9
k8s.io/klog.(*loggingT).printf(0x33f8420, 0x3, 0x1dc88a2, 0x2a, 0xc0006d2e88, 0x3, 0x3)
	pkg/mod/k8s.io/klog@v0.3.1/klog.go:705 +0x14e
k8s.io/klog.Fatalf(...)
	pkg/mod/k8s.io/klog@v0.3.1/klog.go:1256
k8s.io/gengo/examples/deepcopy-gen/generators.(*genDeepCopy).doStruct(0xc00002f900, 0xc0007e2790, 0xc0007fa280)
	pkg/mod/k8s.io/gengo@v0.0.0-20190327210449-e17681d19d3a/examples/deepcopy-gen/generators/deepcopy.go:885 +0x7bf
k8s.io/gengo/examples/deepcopy-gen/generators.(*genDeepCopy).generateFor(0xc00002f900, 0xc0007e2790, 0xc0007fa280)
	pkg/mod/k8s.io/gengo@v0.0.0-20190327210449-e17681d19d3a/examples/deepcopy-gen/generators/deepcopy.go:695 +0xc5
k8s.io/gengo/examples/deepcopy-gen/generators.(*genDeepCopy).GenerateType(0xc00002f900, 0xc0007896e0, 0xc0007e2790, 0x209c0c0, 0xc0008143e0, 0x0, 0x0)
	pkg/mod/k8s.io/gengo@v0.0.0-20190327210449-e17681d19d3a/examples/deepcopy-gen/generators/deepcopy.go:608 +0xe86
k8s.io/gengo/generator.(*Context).executeBody(0xc0007896e0, 0x2098d80, 0xc00080e3e0, 0x2111580, 0xc00002f900, 0x60, 0x10)
	pkg/mod/k8s.io/gengo@v0.0.0-20190327210449-e17681d19d3a/generator/execute.go:304 +0x11d
k8s.io/gengo/generator.(*Context).ExecutePackage(0xc0007894a0, 0xc0004652c0, 0x25, 0x20e9e00, 0xc00002f500, 0x0, 0x0)
	pkg/mod/k8s.io/gengo@v0.0.0-20190327210449-e17681d19d3a/generator/execute.go:265 +0xbf7
k8s.io/gengo/generator.(*Context).ExecutePackages(0xc0007894a0, 0xc0004652c0, 0x25, 0xc0004af120, 0x1, 0x1, 0x0, 0xc000465320)
	pkg/mod/k8s.io/gengo@v0.0.0-20190327210449-e17681d19d3a/generator/execute.go:51 +0xc5
k8s.io/gengo/args.(*GeneratorArgs).Execute(0xc00047b860, 0xc0006d3810, 0x1d6e857, 0x6, 0x1e62798, 0xc0005b7540, 0x16)
	pkg/mod/k8s.io/gengo@v0.0.0-20190327210449-e17681d19d3a/args/args.go:194 +0x2d7
github.com/operator-framework/operator-sdk/cmd/operator-sdk/internal/genutil.deepcopyGen(0xc0005b4700, 0xe, 0xc0002e5a40, 0x2, 0x2, 0x18, 0xc0006d39b0)
	src/github.com/operator-framework/operator-sdk/cmd/operator-sdk/internal/genutil/k8s.go:94 +0x4d2
github.com/operator-framework/operator-sdk/cmd/operator-sdk/internal/genutil.K8sCodegen.func1(0xc0005b4700, 0xe, 0xc0005b4700, 0xe)
	src/github.com/operator-framework/operator-sdk/cmd/operator-sdk/internal/genutil/k8s.go:53 +0x50
github.com/operator-framework/operator-sdk/cmd/operator-sdk/internal/genutil.generateWithHeaderFile(0xc0002e5aa0, 0x0, 0x0)
	src/github.com/operator-framework/operator-sdk/cmd/operator-sdk/internal/genutil/genutil.go:104 +0x178
github.com/operator-framework/operator-sdk/cmd/operator-sdk/internal/genutil.K8sCodegen(0xc0005af180, 0x0)
	src/github.com/operator-framework/operator-sdk/cmd/operator-sdk/internal/genutil/k8s.go:54 +0x4a5
github.com/operator-framework/operator-sdk/cmd/operator-sdk/generate.k8sFunc(0xc0005af180, 0x341bfc0, 0x0, 0x0, 0x0, 0x0)
	src/github.com/operator-framework/operator-sdk/cmd/operator-sdk/generate/k8s.go:56 +0x156
github.com/spf13/cobra.(*Command).execute(0xc0005af180, 0x341bfc0, 0x0, 0x0, 0xc0005af180, 0x341bfc0)
	pkg/mod/github.com/spf13/cobra@v0.0.3/command.go:762 +0x465
github.com/spf13/cobra.(*Command).ExecuteC(0xc000546f00, 0x209d6c0, 0xc0003d0600, 0x0)
	pkg/mod/github.com/spf13/cobra@v0.0.3/command.go:852 +0x2ec
github.com/spf13/cobra.(*Command).Execute(...)
	pkg/mod/github.com/spf13/cobra@v0.0.3/command.go:800
main.main()
	src/github.com/operator-framework/operator-sdk/cmd/operator-sdk/main.go:85 +0x50f

Environment

  • operator-sdk version: operator-sdk version: v0.10.0, commit: ff80b17737a6a0aade663e4827e8af3ab5a21170
  • go version: go version go1.12.5 linux/amd64

Additional context Found while working on jaegertracing/jaeger-operator#607

About this issue

  • Original URL
  • State: closed
  • Created 5 years ago
  • Comments: 30 (16 by maintainers)

Commits related to this issue

Most upvoted comments

Alright, I think I finally have this figured out.

When the operator-sdk binary is built from source, one of the transitive dependencies that gets pulled in by controller-tools for code generation is kubernetes/gengo.

kubernetes/gengo’s AddDirRecursive function searches a set of paths, including GOROOT, when parsing Go code. It determines GOROOT using go/build.Default.GOROOT.

go/build.Default.GOROOT is initialized using runtime.GOROOT(), which first checks at runtime for the GOROOT environment variable and falls back to the compile-time GOROOT of the binary if GOROOT is not explicitly set in the environment. go env GOROOT does not appear to be used at runtime.

So when using the operator-sdk, it is necessary to either (a) set GOROOT in your environment, or (b) use the same GOROOT that was used to build the operator-sdk binary.

Obviously (b) is undesirable because you have to know what GOROOT was used and then force your environment to match it, so (a) setting GOROOT explicitly seems like the best workaround.

It appears that kubernetes/gengo is no longer used in controller-tools v0.2.0, so hopefully controller-tools’s new mechanism for Go code parsing does not have this same problem. We should be switching to controller-tools v0.2.0 very soon.

/cc @estroz @hasbro17 @jpkrohling @mandymchu @kdwodc

TL;DR: If you run into strange problems with code generation, try setting GOROOT explicitly

+27
joelanford on Oct 21, 2019

I hit this error when not exporting GOROOT, it worked for me after exporting that.

+10
mandymchu on Aug 26, 2019

Still seeing this issue on operator-sdk v0.16.0, go 1.13.8, Fedora 30.

export GOROOT=$(go env GOROOT)

Fixed it but like some others it stole a small part of my life that I’ll never get back 😄

I’ve got a few other tips I’ve been thinking I’ll add to some docs. I’ll try to figure out a spot to mention this in case others fall into it.

+8
FreedomBen on Apr 14, 2020

So to summarise

export GOROOT=$(go env GOROOT)

does the trick

+7
ytsarev on Dec 27, 2019

Soo I am running 0.11 + Modules and am still seeing this error.

+4
abergmeier on Oct 12, 2019

TL;DR: If you run into strange problems with code generation with operator-sdk<=v0.10.0, try setting GOROOT explicitly

If you always reproduce this issue, although go env GOROOT display the right path, you should export GOROOT= again. Then it can works fine.

$ go env GOROOT
/usr/local/Cellar/go/1.12.9/libexec
$ operator-sdk add api --api-version=app.example.com/v1alpha1 --kind=AppService
INFO[0000] Generating api version app.example.com/v1alpha1 for kind AppService.
INFO[0000] Created pkg/apis/app/group.go
INFO[0000] Created pkg/apis/app/v1alpha1/appservice1_types.go
INFO[0000] Created pkg/apis/addtoscheme_app1_v1alpha1.go
INFO[0000] Created pkg/apis/app/v1alpha1/register.go
INFO[0000] Created pkg/apis/app/v1alpha1/doc.go
INFO[0000] Created deploy/crds/app1_v1alpha1_appservice1_cr.yaml
panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0x18e8815]

goroutine 1 [running]:
k8s.io/gengo/parser.(*Builder).AddDirRecursive(0xc000560d70, 0xc0002af830, 0x2c, 0x2, 0x2)
	pkg/mod/k8s.io/gengo@v0.0.0-20190327210449-e17681d19d3a/parser/parse.go:229 +0xb5
$ export GOROOT='/usr/local/Cellar/go/1.12.9/libexec'
$ operator-sdk add api --api-version=app1.example.com/v1alpha1 --kind=AppService1
INFO[0000] Generating api version app1.example.com/v1alpha1 for kind AppService1.
INFO[0000] Created pkg/apis/app1/group.go
INFO[0001] Created pkg/apis/app1/v1alpha1/appservice_types.go
INFO[0001] Created pkg/apis/addtoscheme_app_v1alpha1.go
INFO[0001] Created pkg/apis/app1/v1alpha1/register.go
INFO[0001] Created pkg/apis/app1/v1alpha1/doc.go
INFO[0001] Created deploy/crds/app_v1alpha1_appservice_cr.yaml
INFO[0007] Created deploy/crds/app_v1alpha1_appservice_crd.yaml
INFO[0007] Running deepcopy code-generation for Custom Resource group versions: [app1:[v1alpha1], ]
INFO[0013] Code-generation complete.
INFO[0013] Running OpenAPI code-generation for Custom Resource group versions: [app1:[v1alpha1], ]
INFO[0026] Created deploy/crds/app_v1alpha1_appservice_crd.yaml
INFO[0026] Code-generation complete.
INFO[0026] API generation complete.

Thanks @joelanford gave the detailed answers.

+4
jiwq on Aug 27, 2019

Wonderful, thanks for digging into this! I confirm that setting GOROOT does make it work for me (Fedora 30, latest updates as of today):

$ export GOROOT=/usr/lib/golang
$ GO111MODULE=on operator-sdk generate k8s
INFO[0000] Running deepcopy code-generation for Custom Resource group versions: [io:[v1alpha1], jaegertracing:[v1], ] 
INFO[0009] Code-generation complete.                    

$ unset GOROOT
$ GO111MODULE=on operator-sdk generate k8s
INFO[0000] Running deepcopy code-generation for Custom Resource group versions: [io:[v1alpha1], jaegertracing:[v1], ] 
F0827 09:05:57.950321   12347 deepcopy.go:885] Hit an unsupported type invalid type for invalid type, from github.com/jaegertracing/jaeger-operator/pkg/apis/io/v1alpha1.ElasticsearchSpec
goroutine 1 [running]:
k8s.io/klog.stacks(0xc000814700, 0xc0002987e0, 0xbb, 0x10b)
	pkg/mod/k8s.io/klog@v0.3.1/klog.go:855 +0xb1
k8s.io/klog.(*loggingT).output(0x33f8420, 0xc000000003, 0xc0002b1260, 0x323b24c, 0xb, 0x375, 0x0)
	pkg/mod/k8s.io/klog@v0.3.1/klog.go:806 +0x2d9
k8s.io/klog.(*loggingT).printf(0x33f8420, 0x3, 0x1dc88a2, 0x2a, 0xc0006cae88, 0x3, 0x3)
	pkg/mod/k8s.io/klog@v0.3.1/klog.go:705 +0x14e
k8s.io/klog.Fatalf(...)
	pkg/mod/k8s.io/klog@v0.3.1/klog.go:1256
k8s.io/gengo/examples/deepcopy-gen/generators.(*genDeepCopy).doStruct(0xc000562200, 0xc000464f20, 0xc0007e3c20)
	pkg/mod/k8s.io/gengo@v0.0.0-20190327210449-e17681d19d3a/examples/deepcopy-gen/generators/deepcopy.go:885 +0x7bf
k8s.io/gengo/examples/deepcopy-gen/generators.(*genDeepCopy).generateFor(0xc000562200, 0xc000464f20, 0xc0007e3c20)
	pkg/mod/k8s.io/gengo@v0.0.0-20190327210449-e17681d19d3a/examples/deepcopy-gen/generators/deepcopy.go:695 +0xc5
k8s.io/gengo/examples/deepcopy-gen/generators.(*genDeepCopy).GenerateType(0xc000562200, 0xc0004f6f00, 0xc000464f20, 0x209c0c0, 0xc0007d1c40, 0x0, 0x1d9dde4)
	pkg/mod/k8s.io/gengo@v0.0.0-20190327210449-e17681d19d3a/examples/deepcopy-gen/generators/deepcopy.go:608 +0xe86
k8s.io/gengo/generator.(*Context).executeBody(0xc0004f6f00, 0x2098d80, 0xc000806720, 0x2111580, 0xc000562200, 0x60, 0x10)
	pkg/mod/k8s.io/gengo@v0.0.0-20190327210449-e17681d19d3a/generator/execute.go:304 +0x11d
k8s.io/gengo/generator.(*Context).ExecutePackage(0xc0004f6cc0, 0x0, 0x0, 0x20e9e00, 0xc0003a7e80, 0x0, 0x0)
	pkg/mod/k8s.io/gengo@v0.0.0-20190327210449-e17681d19d3a/generator/execute.go:265 +0xbf7
k8s.io/gengo/generator.(*Context).ExecutePackages(0xc0004f6cc0, 0x0, 0x0, 0xc00044a170, 0x1, 0x1, 0x0, 0xc0005868d0)
	pkg/mod/k8s.io/gengo@v0.0.0-20190327210449-e17681d19d3a/generator/execute.go:51 +0xc5
k8s.io/gengo/args.(*GeneratorArgs).Execute(0xc00049b360, 0xc0006cb810, 0x1d6e857, 0x6, 0x1e62798, 0xc00039a980, 0x16)
	pkg/mod/k8s.io/gengo@v0.0.0-20190327210449-e17681d19d3a/args/args.go:194 +0x2d7
github.com/operator-framework/operator-sdk/cmd/operator-sdk/internal/genutil.deepcopyGen(0xc00057ac90, 0xe, 0xc000568fc0, 0x2, 0x2, 0x18, 0xc0006cb9b0)
	src/github.com/operator-framework/operator-sdk/cmd/operator-sdk/internal/genutil/k8s.go:94 +0x4d2
github.com/operator-framework/operator-sdk/cmd/operator-sdk/internal/genutil.K8sCodegen.func1(0xc00057ac90, 0xe, 0xc00057ac90, 0xe)
	src/github.com/operator-framework/operator-sdk/cmd/operator-sdk/internal/genutil/k8s.go:53 +0x50
github.com/operator-framework/operator-sdk/cmd/operator-sdk/internal/genutil.generateWithHeaderFile(0xc000569000, 0x0, 0x0)
	src/github.com/operator-framework/operator-sdk/cmd/operator-sdk/internal/genutil/genutil.go:104 +0x178
github.com/operator-framework/operator-sdk/cmd/operator-sdk/internal/genutil.K8sCodegen(0xc00036c000, 0x0)
	src/github.com/operator-framework/operator-sdk/cmd/operator-sdk/internal/genutil/k8s.go:54 +0x4a5
github.com/operator-framework/operator-sdk/cmd/operator-sdk/generate.k8sFunc(0xc00036c000, 0x341bfc0, 0x0, 0x0, 0x0, 0x0)
	src/github.com/operator-framework/operator-sdk/cmd/operator-sdk/generate/k8s.go:56 +0x156
github.com/spf13/cobra.(*Command).execute(0xc00036c000, 0x341bfc0, 0x0, 0x0, 0xc00036c000, 0x341bfc0)
	pkg/mod/github.com/spf13/cobra@v0.0.3/command.go:762 +0x465
github.com/spf13/cobra.(*Command).ExecuteC(0xc000368a00, 0x209d6c0, 0xc0003ac900, 0x0)
	pkg/mod/github.com/spf13/cobra@v0.0.3/command.go:852 +0x2ec
github.com/spf13/cobra.(*Command).Execute(...)
	pkg/mod/github.com/spf13/cobra@v0.0.3/command.go:800
main.main()
	src/github.com/operator-framework/operator-sdk/cmd/operator-sdk/main.go:85 +0x50f
+4
jpkrohling on Aug 27, 2019

This is still an ongoing issue. Someone should post this on the operator-sdk install guide, as I spent atleast 30 minutes trying to figure out what was causing it. Installed go with snap package manager

+2
Nik-Novak on Feb 24, 2020
Read more comments on GitHub
← operator-sdk: CustomResourceDefinition is invalid at kubernetes 1.18
operator-sdk: Does not work with Modules →