nunit: CollectionTally (EquivalentTo) should throw for non-transitive comparisons

As @CharliePoole and I were discussing, we think this should probably be merged before @ggeurts’ work at https://github.com/nunit/nunit/pull/2830. I marked it as high priority with respect to that PR.

The NUnit team made a design decision in the past to ignore tolerance for EquivalentTo. It’s such a tough problem to solve and there isn’t demand for it, so I don’t think we should change that decision right now. But here is an example of code that implements tolerance manually via a non-transitive custom comparer:

Assert.That(
    new[] { 2f, 1f },
    Is.EquivalentTo(new[] { 3f, 2f })
        .Using<float>((a, b) => Math.Abs(a - b) <= 1));

It returns an incorrect answer. If you swap the order of the array elements, it’ll pass. But rather than passing or failing, I think the test should error with InvalidOperationException in the CollectionTally constructor (since CollectionTally can be used directly) if the comparer being used is non-transitive.

In addition, maybe we could hard-obsolete the Using overloads which enable non-transitivity: the ones that will cause InvalidOperationException to be thrown no matter what. This would have to be done carefully because the Using overloads are defined in the a base class, but the base class (CollectionItemsEqualConstraint) works just fine with non-transitivity, so we can’t block that.

Assert.That(
    new[] { 2f, 1f },
    Is.EquivalentTo(new[] { 3f, 2f })
        .Using<float>((a, b) => Math.Abs(a - b) <= 1));

The overloads which allow non-transitivity are:

What are your preferences? Anything we should consider?

About this issue

  • Original URL
  • State: closed
  • Created 6 years ago
  • Comments: 19 (19 by maintainers)

Most upvoted comments

@OmicronPersei As far as I can tell the problem is that you are making the change in CollectionItemsEqualConstraint and not in CollectionEquivalentConstraint.

Sounds good to me. A few comments…

  1. Title seems wrong. Don’t we have to ban all Using statements, since we know nothing of their transitivity?

  2. What if we refactor so that CollectionItemsEqualConstraint derived from a base without Using?