viper: Error ``Unmarshal`` map when key contains dot

Test code:

package main

import (
	"bytes"
	"fmt"
	"github.com/spf13/viper"
)

type Config struct {
	Foo struct {
		Bar map[string]int `mapstructure:"bar"`
	} `mapstructure:"foo"`
}

var content = []byte(`
foo:
  bar:
    "x": 1
    "y": 2
    "z.z": 3
`)

func main() {
	var c1, c2 Config

	viper.SetConfigType("yaml")
	viper.ReadConfig(bytes.NewReader(content))

	viper.Unmarshal(&c1)
	fmt.Println("c1:", c1)

	viper.UnmarshalKey("foo", &c2.Foo)
	fmt.Println("c2:", c2)
}

Output:

c1: {{map[x:1 y:2]}}
c2: {{map[x:1 y:2 z.z:3]}}

As you can see, when the map key contains dot, UnmarshalKey works correctly but Unmarshal doesn’t.

About this issue

  • Original URL
  • State: open
  • Created 7 years ago
  • Reactions: 12
  • Comments: 18 (6 by maintainers)

Commits related to this issue

Most upvoted comments

hopefully it will get merged soon!

Also hit this. Seems like if there’s a key like “word.other” containing a dot as a map[string]interface{} key, the output turns it into a struct like:

{
    "word": {
        "other": "value"      
    }
}

and not

{
    "word.other": "value"
}

Is that expected? Anyone have pointers to override this to get the former behavior?

Hey guys, I’m hitting this problem too. Need to get a map with the key being IPs, but it fails.

I’ve been using this library for quite a long time now for pretty much all my personal and work projects.

Any chance we can get some brainstorming on this?

This still seems to be an issue for me. This test fails depending on v1.3.1:

package main

import (
	"strings"
	"testing"

	"github.com/spf13/viper"

	"github.com/stretchr/testify/assert"
	"github.com/stretchr/testify/require"
)

func TestConfigMarshalling(t *testing.T) {
	data := `
root:
  ele.one: value
  eletwo: value`
	reader := strings.NewReader(data)
	viper.SetConfigType("yaml")
	err := viper.ReadConfig(reader)
	require.NoError(t, err)

	// Test if the fields are even correct
	expected := map[string]interface{}{"ele.one": "value", "eletwo": "value"}
	assert.Equal(t, expected, viper.Get("root"))
	var s struct {
		Root map[string]string
	}
	err = viper.UnmarshalKey("root", &s)
	require.NoError(t, err)
}

Changing the dot to, say, an underscore fixes the issue.

Right, I fully intend to merge #794 soon, but I’m playing with replacing the setter with a constructor/factory function argument. I don’t really like setters, and in this case it could actually mess up the internal state of Viper, so that’s what blocks the PR from getting merged.

The alternative is to change the key delimiter which will be added in #794, this is on a branch of this repo and I assume will be merged at some point so could also lock the go module to that reference.

As for a more robust solution I’ve been thinking about a way forward. As discussed in #766, the problem is being unable to determine from a single string, whether a given character should be part of the key name or is treated as the delimiter, eg: sites.example.com.port can be interpreted many different ways. The reason it can work for unmarshalling, which is what I tried to fix, is that you’re telling viper the config schema in the form of a struct.

I believe the only way to also solve the problem for environment variable and pflag bindings is to provide viper with some sort of schema (like we do for unmarshalling), so that it knows explicitly what to do with a parameter address that might contain delimiters within key names, rather than assuming they’re all separate keys. I’m not yet sure if this is something that could be achieved within v1, perhaps providing a schema would be like the feature toggle @sagikazarmark mentions in https://github.com/spf13/viper/pull/794#issue-337317980.

You could stick your viper module to commit 99520c8 this commit was reverted but you don’t need to run on latest 😃 go get github.com/spf13/viper@99520c8