jasmine: Adding because descriptions to expect failures

I feel the resistance to this feature is based on a cultural difference between unit-testing developers, who are ideally dealing with low-level, simple in / simple out functions to test, and functional test engineers, who are commonly dealing with user-flow scenarios in which simple acts or operational sequences can result in complex outcomes that need to be evaluated in full.

Say for example I write a UI test that looks up an item and expects certain values of that item to be returned, as well as other conditionals. So when I do A, B, and then C, I expect to see D, E, and F. So I need three expects, for example:

expect(element(by.xpath("tr/td[1]"))).toEqual('Bob');
expect(element(by.xpath("tr/td[2]"))).toEqual('true');
expect(element(by.xpath("tr/td[3]"))).toEqual('true');

If either of the latter two asserts fail, all I will see in the output is:

Failed
    Expected false to be true.

But this doesn’t help me identify which of the assertions failed. I then need to run back to the code to examine which assertion failed. This is pretty inconvenient in a CI environment, where the person reviewing the CI report doesn’t have ready access to the code, or which might not include the lines reflecting failed line in a summary view.

But if I could write my asserts like:

expect(element(by.xpath("tr/td[1]"))).toEqual('Bob').because('Bob's name is Bob');
expect(element(by.xpath("tr/td[2]"))).toEqual('true').because('Bob is a supervisor so the supervisor    column should say "true"');
expect(element(by.xpath("tr/td[3]"))).toEqual('true').because('Bob is full time so the full-time column should say "true"');

Now if the second assert fails, I will see something like:

Failed
    Expected false to be true (because: Bob is full time so the full-time column should say "true")

and I can clearly and unambiguously see what the problem was without having to grab a shovel and go digging.

I think one of the comments earlier was pretty clear that this actually works, it’s just undocumented. You can write

describe('returnWhateverIsGiven', function() {
    it('should return whatever is given', function() {
        let someobject = new SomeObject();
        expect(someObject.returnWhateverIsGiven(true)).toBe(true, 'should return true when given true');
        expect(someObject.returnWhateverIsGiven(false)).toBe(false, 'should return false when given false');
    });
});

And it just works because if it fails it prints out all the parameters given to the matcher, even though one of those parameters was superfluous.

Since Jasmine 3.3, there’s a way to do it through withContext

Example: expect(someValue).withContext('expected someValue to be true...').toBe(true)

Source: https://github.com/jasmine/jasmine/issues/1457#issuecomment-435465924

@PaulL1 , @keithdtyler it works, it’s undocumented, it can be deprecated at any given point. There should be official API support for this, coz obviously the demand is here. Also confused why this discussion is still going. C’mon guys)

If I find myself in a situation where I have multiple asserts in one test where I really want to be clear as to what assert failed and why, I split my test up into multiple tests an wrap them in their own describe block with a beforeEach for their shared data.

For example, this:

describe('returnWhateverIsGiven', function() {
    it('should return whatever is given', function() {
        let someobject = new SomeObject();
        expect(someObject.returnWhateverIsGiven(true)).toBe(true);
        expect(someObject.returnWhateverIsGiven(false)).toBe(false);
    });
});

would become

describe('returnWhateverIsGiven', function() {
    beforeEach(function() {
        this.someobject = new SomeObject();
    });
    it('should return true when given true', function() {
        expect(this.someObject.returnWhateverIsGiven(true)).toBe(true);
    });
    it('should return false when given false', function() {
        expect(this.someObject.returnWhateverIsGiven(false)).toBe(false);
    });
});

This way it’s clear when a specific assertion fails, because each assertion has their own test name.

Many of jasmines built-in matchers will actually do something similar. It’s not documented behavior, and so could change, so I wouldn’t rely on it. How it works currently is:

1. The matcher expects to see n parameters passed to it and ignores any extras.
2. You pass n + 1 parameters.
3. When the matcher fails, and doesn’t provide a custom message
4. Jasmine creates a default failure message based on all of the parameters passed to the expectation.

This means that expect(1).toEqual(2, 'because of stuff') ends up printing out:

Expected 1 to equal 2, 'because of stuff'.

Presumably though, real code would look something more like:

expect(myObj.someProp).toEqual(3, 'this means foo');

In this case, jasmine would encourage you to instead write a custom matcher so you would have something like:

expect(myObj).toBeFoo();

or:

expect(myObj).toBeFoo(3);

And then the custom matcher can specify any failure message you want.

Is this the only assertion library that doesn’t provide custom failure messages? I’ve used many and this is the only one I’ve come across. It seems unreasonable to not implement such a useful feature that requires such a small amount of code.

@jaapz This is a workaround which requires writing more code (and more complex code) than would be otherwise required if this feature were implemented.

Frankly I find it ridiculous that this discussion is still going on.

It feels heavy to create a custom matcher when all you really want is a custom message to make debugging easier. The purpose of tests is to make our workflow more smooth and more reliable, so the ability to quickly write expressive tests that output clear messages is very valuable.

n-thing this request.

Real life example, where this would be handy: I have a JS library, that tries to generate unique CSS selector for any element. I test for various cases. But the ultimate test is a loop, that goes through every element in very complex document.

Right now, if the test does not pass, it just tells me that false should be true and I have to debug it by hand. It would be great if I could add meaningful debug data to the failing message. Like this:

expect(x.testSelector element, selector)
  .toBe(true)
  .because("Element '#{element.outerHTML}' returned selector '#{selector}'.")

Please see earlier in this thread for more discussion on this. Neither case is officially supported by Jasmine.

@MatthewHerbst I suspect not all expects work that way, especially any of the custom expects introduced by Jest.

I don’t at all disagree with the points about it being undocumented and potentially unsupported later. I would want to both support and encourage this undocumented feature to be made official and a part of all expects. I would even support enhancing the functionality to allow you to optionally include the expected and actual values as well – perhaps even as part of a sprintf-style format.

I do hear the need for some way to tell Jasmine some sort of description of the value being checked, but I don’t think it will be as an additional argument to the matcher itself. I’m leaning more toward something like what is suggested here, but I think because might be the wrong word for it. I would prefer something that will work for all matchers without them having to build in support for it in customizing their failure messages further.

Indeed. Writing a custom matcher for the sole purpose of more descriptive messages doesn’t seem like a brilliant idea, and, more to the point, every time you want a custom message, especially when a more general solution has been proposed.

Being able to write custom messages inline in tests will also improve readability of the tests.

jasmine-custom-message works well, but why can’t we implement this in jasmine?

.toBe(expected, output) seems to work OK, however .toEqual(expected, output) does not.

@jfrioux unfortunately I’m not willing to commit my entire company’s code base to an undocumented feature than could be removed at any time (and as noted above, doesn’t seem to work for me anyways).

I use: expect(something).toBe(likeThis, "because that's how it should be.")

And it fits my need and solve this problem for me.

+1

@jaapz That’s great for unit tests, not great for functional tests, especially UI flow tests. Especially when you’re hoping for functional test results in a CI situation within a certain period of time (whether or not this is a good idea is not usually up to the test developer), you need to keep your tests short, not exhaustive. Thus, multiple asserts in a single flow test is precisely what the doctor ordered - and it’s no good if the doctor can’t identify the symptoms.

You can use my library - jasmine-custom-message.

@ViniciusRio I suppose withContext made jasmine-custom-message obsolete.

Please reconsider to add .because !

We are also running integration tests with protractor and jasmine, which run on complete scenarios. Itis painful to search for the actual cause when a scenario fails and we have to check all assertions.

It would be so nice to have .because(). The output of expect(myObj.someProp).toEqual(3, 'this means foo'); is suboptimal and I’d never have expected this to work… and it is not really documented, right?

Actually felt the need for this feature as in my case I usually have 3-4 expect statements inside test case. So during failure its very difficult to tell which expect is failing.

@guy-mograbi-at-gigaspaces 👍 @slackersoft Having this feature implemented would be much more versatile than having to code a custom matcher for every single use case.

@matthewjh I think what you want for that situation is just a toBePresent custom matcher. This will give you a default message of “Expected <element description> to be present”. You would also have the ability to specify a custom message if desired. See the docs here: http://jasmine.github.io/edge/custom_matcher.html

For some reasons I have multiple asserts within single spec (mostly comparing boolean values) and I’m feeling puzzled as something like this is missing. +1