testify: assert/require.Len doesn't print anything if the slice is too long

Hey there, I encountered this when attempting to use require.Len as shown below.

package tester_test

import (
	"testing"

	"github.com/stretchr/testify/require"
)

func TestVeryLongSlice(t *testing.T) {
	var bigSlice []string
	for i := 0; i < 20000; i++ {
		bigSlice = append(bigSlice, "hello")
	}

	require.Len(t, bigSlice, 10)
}

Output of test:

=== RUN   TestVeryLongSlice
    tester_test.go:15:
                Error Trace:    C:/Users/Fots/source/tester/tester/tester_test.go:15
                Error:
                Test:           TestVeryLongSlice
--- FAIL: TestVeryLongSlice (0.00s)
FAIL
FAIL    github.com/fgimian/tester/tester        0.150s
FAIL

I haven’t quite figured out why this happens, although I can imagine the output would be impractically long anyway. Worst of all, the main assertion message is not displayed which is quite confusing (especially to someone new using the library, like me 😄).

Personally, I think that it would be best not to display the slice at all for the Len function, and instead just display:

Should have X item(s), but has Y

However, I suspect my view on that may not be what some people want. The reason I think it is unwise to print the slice in any form, is that typically when many people are testing for the length of a slice, they may be comparing to a relatively large slice of items (e.g. the output of a mocked API call or similar).

An additional point that I believe has been discussed before is the inconsistency with argument ordering for the Len function which takes the actual first and expected second.

Cheers Fotis

About this issue

  • Original URL
  • State: open
  • Created 3 years ago
  • Reactions: 3
  • Comments: 19 (1 by maintainers)

Commits related to this issue

Most upvoted comments

Except we were talking about “Len”, and not “Contains”, yes it’s exactly what I’m thinking about

Should I assume there is the same issue (dumping a slice) with Contains?

Ah yes, I guess there is the same issue for any function which operates on containers (at least Len and Contains, but perhaps others?)

The argument order is covered by #146. Let’s keep this issue about the missing message.

We use a scanner to get the text which should go there line by line (it can be multiline) to indent each line until after the field name: https://github.com/stretchr/testify/blob/7caada5a3bc870e8d8e0a31bd9e03b8c3b130f05/assert/assertions.go#L304-L313

This mishandles the scanner.Scan() call. If scanner.Scan() returns false then you need to check scanner.Err() in case it isn’t io.EOF. If we did that we’d have found it is bufio.Scanner: token too long.

We need to think of a thing to do about very long lines in this code. We should also probably not produce such a long line from assert.Len, maybe we should truncate after a sane number of elements?

In other code I have written is was presented a little differently:

return fmt.Errorf("wrong result length: wanted %d, got %d: %s>", want, got, f.snippet(v))

Where snippet() was:

// snippet produces a short snippet string of an arbitrary value.                                                                                                                
func (f Formatter) snippet(v any) string {
    const snipLen = 16

    snip := fmt.Sprintf("%#v", v)
    if len(snip) > snipLen {
        // TODO: maybe add a "... %d more"
        snip = snip[:snipLen]
    }   
    return snip
}

They key parts being:

  1. The snippet goes LAST, not first.
  2. The error condition goes FIRST, not last
  3. The snippet is very small - 16 chars, or 32, but not 1024.
  4. You can call snippet() on anything and get an abbreviated snippet.

One naive solution could be to check the length of the sprinted value, and truncate it after let’s say an arbitrary value of 20-30 characters. Something with a trailing … plus tge number of items.

If the slice is short and simple, such as a slice 2-3 int/string/bool, the message displayed will be the same as now. The error message will be meaningful.

In other cases, where the message is currently useless, it would be better than nothing.

Any thoughts guys?

Personally, I think that it would be best not to display the slice at all for the Len function, and instead just display:\n\nShould have X item(s), but has Y\nHowever, I suspect my view on that may not be what some people want. The reason I think it is unwise to print the slice in any form, is that typically when many people are testing for the length of a slice, they may be comparing to a relatively large slice of items (e.g. the output of a mocked API call or similar).

I totally agree with you