image: panic: semaphore: bad release
I created a tool to synchronize the gcr.io image to the docker hub, but after running for a period of time, I get a panic: semaphore: bad release
INFO[2020-04-21 17:31:14] sync k8s.gcr.io/k8s-tpu-operator:0.1 => docker.io/gcrxio/k8s.gcr.io_k8s-tpu-operator:0.1
INFO[2020-04-21 17:31:14] sync k8s.gcr.io/k8s-tpu-operator:0585ea2d8131291cc041329e32bb7c81263cf7c5 => docker.io/gcrxio/k8s.gcr.io_k8s-tpu-operator:0585ea2d8131291cc041329e32bb7c81263cf7c5
INFO[2020-04-21 17:31:14] sync k8s.gcr.io/kube-addon-manager-ppc64le:v6.4-alpha.3 => docker.io/gcrxio/k8s.gcr.io_kube-addon-manager-ppc64le:v6.4-alpha.3
INFO[2020-04-21 17:31:14] sync k8s.gcr.io/k8s-jupyterhub:0.2 => docker.io/gcrxio/k8s.gcr.io_k8s-jupyterhub:0.2
INFO[2020-04-21 17:31:18] sync k8s.gcr.io/k8s-tpu-operator:1.0 => docker.io/gcrxio/k8s.gcr.io_k8s-tpu-operator:1.0
panic: semaphore: bad release
goroutine 81928 [running]:
golang.org/x/sync/semaphore.(*Weighted).Release(0xc007f61ae0, 0x1)
/Users/natural/gopath/pkg/mod/golang.org/x/sync@v0.0.0-20190227155943-e225da77a7e6/semaphore/semaphore.go:98 +0x1f5
github.com/containers/image/copy.(*imageCopier).copyLayers.func1(0xb, 0xc00066b180, 0x47, 0x15d3896, 0x0, 0x0, 0x0, 0x0, 0xc00028b900, 0x31, ...)
/Users/natural/gopath/pkg/mod/github.com/containers/image@v3.0.2+incompatible/copy/copy.go:489 +0x241
created by github.com/containers/image/copy.(*imageCopier).copyLayers.func2
/Users/natural/gopath/pkg/mod/github.com/containers/image@v3.0.2+incompatible/copy/copy.go:497 +0x1f6
For speed, I am performing image synchronization in 20 concurrency, this is my code
type Image struct {
Repo string
User string
Name string
Tag string
}
type SyncOption struct {
Timeout time.Duration
Limit int
User string
Password string
NameSpace string
Proxy string
QueryLimit int
Kubeadm bool
}
func sync2DockerHub(image *Image, opt SyncOption) error {
destImage := Image{
Repo: DefaultDockerRepo,
User: opt.User,
Name: image.MergeName(),
Tag: image.Tag,
}
logrus.Infof("sync %s => %s", image, destImage.String())
ctx, cancel := context.WithTimeout(context.Background(), opt.Timeout)
defer cancel()
policyContext, err := signature.NewPolicyContext(
&signature.Policy{
Default: []signature.PolicyRequirement{signature.NewPRInsecureAcceptAnything()},
},
)
if err != nil {
return err
}
defer func() { _ = policyContext.Destroy() }()
srcRef, err := docker.Transport.ParseReference("//" + image.String())
if err != nil {
return err
}
destRef, err := docker.Transport.ParseReference("//" + destImage.String())
if err != nil {
return err
}
sourceCtx := &types.SystemContext{DockerAuthConfig: &types.DockerAuthConfig{}}
destinationCtx := &types.SystemContext{DockerAuthConfig: &types.DockerAuthConfig{
Username: opt.User,
Password: opt.Password,
}}
m, err := copy.Image(ctx, policyContext, destRef, srcRef, ©.Options{
ReportWriter: ioutil.Discard,
SourceCtx: sourceCtx,
DestinationCtx: destinationCtx,
ProgressInterval: 1 * time.Second,
ForceManifestMIMEType: manifest.DockerV2Schema2MediaType,
})
if err != nil {
return err
}
storageDir := filepath.Join(ManifestDir, image.Repo, image.User, image.Name)
// ignore other error
if _, err := os.Stat(storageDir); err != nil {
if err := os.MkdirAll(storageDir, 0755); err != nil {
return err
}
}
return ioutil.WriteFile(filepath.Join(storageDir, image.Tag+".json"), m, 0644)
}
About this issue
- Original URL
- State: closed
- Created 4 years ago
- Comments: 15 (3 by maintainers)
@CooperLi PR: #907 and #908