restic: Restic doesn't use S3 Dual-Stack endpoints, breaking IPv6 support

Output of restic version

$ /opt/restic/0.9.3/restic version
restic 0.9.3 compiled with go1.11.1 on linux/amd64
$

How did you run restic exactly?

This being on an host with only IPv6 connectivity.

The S3 bucket arrakis-v6lab-restic have been pre-created within the eu-central-1 region.

The following environment variables have been defined for AWS access.

$ export | grep AWS
declare -x AWS_ACCESS_KEY_ID="REDACTED"
declare -x AWS_DEFAULT_REGION="eu-central-1"
declare -x AWS_SECRET_ACCESS_KEY="REDACTED"
$

Doing the initial init attempt, based on example in documentation.

$ /opt/restic/0.9.3/restic --password-file ~/.secret/restic_password --repo s3:s3.amazonaws.com/arrakis-v6lab-restic init
Fatal: create repository at s3:s3.amazonaws.com/arrakis-v6lab-restic failed: client.BucketExists: Get https://s3.amazonaws.com/arrakis-v6lab-restic/?location=: dial tcp 52.216.164.125:443: connect: network is unreachable

$

Followed up by explicitly providing a dual-stack endpoint.

$ /opt/restic/0.9.3/restic --password-file ~/.secret/restic_password --repo s3:s3.dualstack.eu-central-1.amazonaws.com/arrakis-v6lab-restic init
Fatal: create repository at s3:s3.dualstack.eu-central-1.amazonaws.com/arrakis-v6lab-restic failed: client.BucketExists: Head https://arrakis-v6lab-restic.s3-eu-central-1.amazonaws.com/: dial tcp 52.219.73.93:443: connect: network is unreachable

$

Manually providing an IPv6 address for arrakis-v6lab-restic.s3-eu-central-1.amazonaws.com did the trick.

$ host s3.dualstack.eu-central-1.amazonaws.com
s3.dualstack.eu-central-1.amazonaws.com has address 52.219.74.52
s3.dualstack.eu-central-1.amazonaws.com has IPv6 address 2a05:d050:4080:80:34db:4a31::
$

$ grep arrakis-v6lab-restic /etc/hosts
2a05:d050:4080:80:34db:4a31::    arrakis-v6lab-restic.s3-eu-central-1.amazonaws.com
$

$ /opt/restic/0.9.3/restic --password-file ~/.secret/restic_password --repo s3:s3.dualstack.eu-central-1.amazonaws.com/arrakis-v6lab-restic init
created restic repository 7601f5be4a at s3:s3.dualstack.eu-central-1.amazonaws.com/arrakis-v6lab-restic

Please note that knowledge of your password is required to access
the repository. Losing your password means that your data is
irrecoverably lost.
$

What backend/server/service did you use to store the repository?

AWS S3

Expected behavior

That restic would either use the S3 dual-stack endpoints by default, or that there would be a way to tell restic to do so.

Actual behavior

Restic uses IPv4-only S3 endpoints, unless “lied” to it.

Steps to reproduce the behavior

1. Ensure an IPv6-only environment
2. Follow the steps above.

Do you have any idea what may have caused this?

After having used the initially provided S3 endpoint to lookup the bucket location restic/minio-go appear to construct a new IPv4-only S3 endpoint based on github.com/minio/minio-go/s3-endpoints.go.

Confirmed by the fact that the following monkey patching solved my immediate problem.

$ git diff
diff --git a/vendor/github.com/minio/minio-go/s3-endpoints.go b/vendor/github.com/minio/minio-go/s3-endpoints.go
index 05892950..adc400e8 100644
--- a/vendor/github.com/minio/minio-go/s3-endpoints.go
+++ b/vendor/github.com/minio/minio-go/s3-endpoints.go
@@ -27,7 +27,7 @@ var awsS3EndpointMap = map[string]string{
        "eu-west-1":      "s3-eu-west-1.amazonaws.com",
        "eu-west-2":      "s3-eu-west-2.amazonaws.com",
        "eu-west-3":      "s3-eu-west-3.amazonaws.com",
-       "eu-central-1":   "s3-eu-central-1.amazonaws.com",
+       "eu-central-1":   "s3.dualstack.eu-central-1.amazonaws.com",
        "ap-south-1":     "s3-ap-south-1.amazonaws.com",
        "ap-southeast-1": "s3-ap-southeast-1.amazonaws.com",
        "ap-southeast-2": "s3-ap-southeast-2.amazonaws.com",
$

Do you have an idea how to solve the issue?

• Work with the minio-go library so that it either provides dual-stack endpoints by default, or provides an option to do so.

and/or.

• When the user provides a sufficient specific endpoint, continue using it.

Did restic help you or made you happy in any way?

• I love that restic allows me to perform smart backups against a dumb storage backend.
• I’m kind of happy that this bug gives me one more nudge to start learning Go.