testify: assert.Equal randomly fails testing maps
assert.Equal (and other functions) will randomly fail testing equality of maps with the same values but different type.
For example:
package main
import (
"fmt"
"github.com/stretchr/testify/assert"
"testing"
)
func TestExactly(t *testing.T) {
map1 := map[string]interface{}{"foo": 1234, "bar": int16(4321)}
map2 := map[string]interface{}{"foo": 1234, "bar": int32(4321)}
for i := 0; i < 3; i++ {
if assert.Equal(t, map1, map2) {
fmt.Printf("Everything is ok\n")
} else {
fmt.Printf("Everything is NOT OK\n")
}
}
}
Results in:
=== RUN TestExactly
Everything is NOT OK
Everything is ok
Everything is NOT OK
--- FAIL: TestExactly (0.00s)
Location: main_test.go:14
Error: Not equal: map[string]interface {}{"foo":1234, "bar":4321} (expected)
!= map[string]interface {}{"foo":1234, "bar":4321} (actual)
Location: main_test.go:14
Error: Not equal: map[string]interface {}{"foo":1234, "bar":4321} (expected)
!= map[string]interface {}{"bar":4321, "foo":1234} (actual)
FAIL
exit status 1
FAIL _/tmp/test6 0.012s
The issue is because the type on bar is different between the maps (int16 in one, int32 in the other).
ObjectsAreEqual uses reflect.DeepEqual which properly detects this, but then it continues to check equality of a fmt.Sprintf("%#v",...) on the values, and this randomly fails depending on the order it prints the map values in.
The solution seems simple: remove the fmt.Sprintf() check. But I wasn’t sure why that was in there to begin with.
About this issue
- Original URL
- State: closed
- Created 9 years ago
- Comments: 21 (14 by maintainers)
Am I correct in my understanding that this was never resolved?
The presence of
fmt.Sprintf()check afterreflect.DeepEqual()in testing libraries likeassert.Equal()is meant to provide additional context in case of failures. It’s used to generate a human-readable representation of the values being compared, so that developers can easily see what’s different when the comparison fails.In your example, the issue arises due to the random order in which the map values are printed by
fmt.Sprintf(). Since maps in Go do not have a guaranteed order for iteration, the order of key-value pairs can vary, resulting in different string representations even when the actual values are equivalent.If you’re encountering issues with such random failures, one approach could be to explicitly convert the maps to a consistent format before comparison. For instance, you could convert the maps to JSON strings and then compare those strings. This ensures a consistent representation for comparison, regardless of the internal order of the map’s elements.
Here’s how you might modify your test function:
This modification ensures that the order of map elements doesn’t affect the comparison, and you’ll get consistent results even if the map representations are different due to type variations.
I would recommend opening a new issue for this so it can be discussed and a proper solution implemented.
On Thu, Apr 16, 2015 at 4:43 PM, Andrei notifications@github.com wrote: