kubernetes: go test k8s.io/kubernetes/pkg/kubectl/cmd is failing on master

make test is failing on k8s.io/kubernetes/pkg/kubectl/cmd with go1.9.4, whereas the same package passes under bazel test.

go test shows failures like

--- FAIL: TestPodAndContainerAttach (0.05s)
        attach_test.go:169: no container, no flags: unexpected error: No Auth Provider found for name "gcp"
        attach_test.go:169: container in flag: unexpected error: No Auth Provider found for name "gcp"
        attach_test.go:169: init container in flag: unexpected error: No Auth Provider found for name "gcp"
        attach_test.go:169: no container, no flags, pods and name: unexpected error: No Auth Provider found for name "gcp"
        attach_test.go:169: no container, no flags, pod/name: unexpected error: No Auth Provider found for name "gcp"
--- FAIL: TestAttach (0.00s)
        attach_test.go:267: No Auth Provider found for name "gcp"
--- FAIL: TestAttachWarnings (0.00s)
        attach_test.go:355: No Auth Provider found for name "gcp"
pod "pod0" deleted
pod "pod1" deleted
pod "pod2" deleted
pod "pod3" deleted
pod "pod4" deleted
pod "pod5" deleted
pod "pod6" deleted
pod "pod7" deleted
--- FAIL: TestPortForward (0.00s)
        portforward_test.go:124: pod portforward: Unexpected error: No Auth Provider found for name "gcp"
panic: runtime error: invalid memory address or nil pointer dereference [recovered]
        panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x38 pc=0x15ae03c]

goroutine 351 [running]:
testing.tRunner.func1(0xc4209361e0)
        [$HOME]/.gvm/gos/go1.9.4/src/testing/testing.go:711 +0x2d2
panic(0x17e1a20, 0x278b6e0)
        [$HOME]/.gvm/gos/go1.9.4/src/runtime/panic.go:491 +0x283
k8s.io/kubernetes/pkg/kubectl/cmd.testPortForward(0xc4209361e0, 0x0, 0xc420c41590, 0x3, 0x3)
        [$HOME]/code/kubernetes/src/k8s.io/kubernetes/pkg/kubectl/cmd/portforward_test.go:130 +0xcdc
k8s.io/kubernetes/pkg/kubectl/cmd.TestPortForward(0xc4209361e0)
        [$HOME]/code/kubernetes/src/k8s.io/kubernetes/pkg/kubectl/cmd/portforward_test.go:140 +0x94
testing.tRunner(0xc4209361e0, 0x1a89958)
        [$HOME]/.gvm/gos/go1.9.4/src/testing/testing.go:746 +0xd0
created by testing.(*T).Run
        [$HOME]/.gvm/gos/go1.9.4/src/testing/testing.go:789 +0x2de
FAIL    k8s.io/kubernetes/pkg/kubectl/cmd       16.572s

I dug into this a bit, and I think it started failing about two weeks ago, after https://github.com/kubernetes/kubernetes/pull/60218. Since CI only runs bazel test now, we didn’t notice the non-bazel go test failures.

It looks like the refactoring depends on init() effects from k8s.io/client-go/plugin/pkg/client/auth; if I add

import _ "k8s.io/client-go/plugin/pkg/client/auth"

to pkg/kubectl/cmd/attach_test.go, then the test passes under go test.

I’m not sure what bazel’s go rules are doing such that this effect is occurring without this explicit import. Still investigating…

(cc @jayconrod if you have any insight)

/sig testing

About this issue

Original URL
State: closed
Created 6 years ago
Reactions: 1
Comments: 16 (15 by maintainers)

Commits related to this issue

Merge pull request #60950 from juanvallejo/jvallejo/use-temp-kubeconfig-file-tests Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instr... — committed to kubernetes/kubernetes by deleted user 6 years ago

Most upvoted comments

@BenTheElder to be clear, go1.9.3 and go1.10 fail in the same way.

ixdy on Mar 8, 2018

This is getting more mysterious.

I added a panic to staging/src/k8s.io/client-go/rest/plugin.go:

--- a/staging/src/k8s.io/client-go/rest/plugin.go
+++ b/staging/src/k8s.io/client-go/rest/plugin.go
@@ -63,6 +63,7 @@ func RegisterAuthProviderPlugin(name string, plugin Factory) error {
 }
 
 func GetAuthProvider(clusterAddress string, apc *clientcmdapi.AuthProviderConfig, persister AuthProviderConfigPersister) (AuthProvider, error) {
+       panic(fmt.Sprintf("fail: auth provider name %q", apc.Name))
        pluginsLock.Lock()
        defer pluginsLock.Unlock()
        p, ok := plugins[apc.Name]

bazel test still passes. go test panics as expected.

something seems fishy.

--- FAIL: TestPodAndContainerAttach (0.02s)
panic: fail: auth provider name "gcp" [recovered]
        panic: fail: auth provider name "gcp"

goroutine 102 [running]:
testing.tRunner.func1(0xc421f27770)
        [$HOME]/.gvm/gos/go1.9.4/src/testing/testing.go:711 +0x2d2
panic(0x1738960, 0xc42305bae0)
        [$HOME]/.gvm/gos/go1.9.4/src/runtime/panic.go:491 +0x283
k8s.io/kubernetes/vendor/k8s.io/client-go/rest.GetAuthProvider(0xc4219d4980, 0x13, 0xc42266ffa0, 0x25d4120, 0xc42272a3e0, 0x0, 0x0, 0x0, 0x0)
        [$HOME]/code/kubernetes/src/k8s.io/kubernetes/vendor/k8s.io/client-go/rest/plugin.go:66 +0x10e
k8s.io/kubernetes/vendor/k8s.io/client-go/rest.(*Config).TransportConfig(0xc420137880, 0xc4218c63ee, 0x1, 0x0)
        [$HOME]/code/kubernetes/src/k8s.io/kubernetes/vendor/k8s.io/client-go/rest/transport.go:78 +0x622
k8s.io/kubernetes/vendor/k8s.io/client-go/rest.TransportFor(0xc420137880, 0xc4224b2d00, 0xc4218c63ee, 0x1, 0x0)
        [$HOME]/code/kubernetes/src/k8s.io/kubernetes/vendor/k8s.io/client-go/rest/transport.go:41 +0x2f
k8s.io/kubernetes/vendor/k8s.io/client-go/rest.UnversionedRESTClientFor(0xc420137880, 0x0, 0x0, 0x0)
        [$HOME]/code/kubernetes/src/k8s.io/kubernetes/vendor/k8s.io/client-go/rest/config.go:231 +0x9b
k8s.io/kubernetes/vendor/k8s.io/client-go/discovery.NewDiscoveryClientForConfig(0xc420136c40, 0xc420136c40, 0x0, 0x0)
        [$HOME]/code/kubernetes/src/k8s.io/kubernetes/vendor/k8s.io/client-go/discovery/discovery_client.go:391 +0x92
k8s.io/kubernetes/pkg/kubectl/cmd/util.(*discoveryFactory).DiscoveryClient(0xc42272d840, 0xc422307c00, 0x0, 0x0, 0x0)
        [$HOME]/code/kubernetes/src/k8s.io/kubernetes/pkg/kubectl/cmd/util/factory_client_access.go:122 +0x83
k8s.io/kubernetes/pkg/kubectl/cmd/util.(*ClientCache).getDefaultConfig(0xc422038480, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, ...)
        [$HOME]/code/kubernetes/src/k8s.io/kubernetes/pkg/kubectl/cmd/util/clientcache.go:102 +0xc5
k8s.io/kubernetes/pkg/kubectl/cmd/util.(*ClientCache).clientConfigForVersion(0xc422038480, 0x0, 0xc4220384ac, 0x19ce601, 0x1)
        [$HOME]/code/kubernetes/src/k8s.io/kubernetes/pkg/kubectl/cmd/util/clientcache.go:137 +0xa4
k8s.io/kubernetes/pkg/kubectl/cmd/util.(*ClientCache).ClientSetForVersion(0xc422038480, 0x0, 0x0, 0x0, 0x0, 0x0)
        [$HOME]/code/kubernetes/src/k8s.io/kubernetes/pkg/kubectl/cmd/util/clientcache.go:202 +0xea
k8s.io/kubernetes/pkg/kubectl/cmd/util.(*ring0Factory).ClientSet(0xc421fde8d0, 0x1, 0xc422038d80, 0xc4204be000, 0x38007fbdedc9a000)
        [$HOME]/code/kubernetes/src/k8s.io/kubernetes/pkg/kubectl/cmd/util/factory_client_access.go:203 +0x38
k8s.io/kubernetes/pkg/kubectl/cmd/util.(*ring1Factory).AttachablePodForObject(0xc421fde900, 0x25d9920, 0xc421f53880, 0x4a817c800, 0xc421f2d830, 0xc422f6ee10, 0xc4200e9050)
        [$HOME]/code/kubernetes/src/k8s.io/kubernetes/pkg/kubectl/cmd/util/factory_object_mapping.go:365 +0x4b
k8s.io/kubernetes/pkg/kubectl/cmd/util.(*factory).AttachablePodForObject(0xc421fde930, 0x25d9920, 0xc421f53880, 0x4a817c800, 0xc4200e9050, 0xc42191ae54, 0x2)
        <autogenerated>:1 +0x5b
k8s.io/kubernetes/pkg/kubectl/cmd/testing.(*TestFactory).AttachablePodForObject(0xc420672b00, 0x25d9920, 0xc421f53880, 0x4a817c800, 0x0, 0x1, 0xc423008b40)
        <autogenerated>:1 +0x5a
k8s.io/kubernetes/pkg/kubectl/cmd.(*AttachOptions).Complete(0xc422044460, 0x2602740, 0xc420672b00, 0xc421d85d40, 0xc422f2de20, 0x1, 0x1, 0x3, 0x4)
        [$HOME]/code/kubernetes/src/k8s.io/kubernetes/pkg/kubectl/cmd/attach.go:164 +0x1a6
k8s.io/kubernetes/pkg/kubectl/cmd.TestPodAndContainerAttach(0xc421f27770)
        [$HOME]/code/kubernetes/src/k8s.io/kubernetes/pkg/kubectl/cmd/attach_test.go:164 +0x1c53
testing.tRunner(0xc421f27770, 0x1a89940)
        [$HOME]/.gvm/gos/go1.9.4/src/testing/testing.go:746 +0xd0
created by testing.(*T).Run
        [$HOME]/.gvm/gos/go1.9.4/src/testing/testing.go:789 +0x2de                                                                                                                                                  
FAIL    k8s.io/kubernetes/pkg/kubectl/cmd       12.840s

@deads2k anything obviously wrong stand out to you here?

ixdy on Mar 8, 2018