snitch: Support automatic check for unexpected exception

In my tests with Catch2, I would expect everything in test case to be checked for exceptions. If anything outside of REQUIRE_THROWS / CHECK_THROWS throws an exception, I consider it a failed test, and I want to get a message about which section / line caused the exception.

It seems Snitch doesn’t support this scenario currently. For example

#define SNITCH_IMPLEMENTATION
#include "snitch_all.hpp"

#include <exception>

void command1() {}
void command2() {}
void command3() {throw std::runtime_error("error");}

TEST_CASE("commands") {
	SECTION("command 1") {
		command1();
	}
	SECTION("command 2") {
		command2();
	}
	SECTION("command 3") {
		command3();
	}
}

outputs:

starting a with snitch v1.1.1.e557246
==========================================
failed: running test case "commands"
          at <snitch internal>:0
          unhandled std::exception caught; message: whoops
==========================================
error: some tests failed (1 out of 1 test cases, 0 assertions, 9.610000e-05 seconds)

Only if I wrap command3() in REQUIRE_NOTHROW, will I get a nice error message. But wrapping every command in every test in macro is unfeasible.

Am I missing something, or is this a necessary tradeoff for smaller footprint?

About this issue

  • Original URL
  • State: closed
  • Created 7 months ago
  • Comments: 18 (13 by maintainers)

Commits related to this issue

Most upvoted comments

I pushed an attempt at this. In summary:

  • If the exception happens inside a CHECK(...), the message will say “somewhere in check at file:line”
  • If the exception happens outside a CHECK(...) but inside a SECTION(...), the message will say “somewhere in section at file:line”
  • Else, the message will say “somewhere in test case at file:line”

How does that sound?

After a bit of refactoring, I managed to get this to work with almost zero net overhead (it did add a bit of overhead, but the refactoring was enough to gain back most of it by simplifying the assertion reporting functions).

Debug before after
Build framework 3.8s 3.8s
Build tests 68s 68s
Build all 71s 72s
Run tests 38ms 42ms
Library size 7.7MB 7.6MB
Executable size 35.3MB 35.7MB
Release before after
Build framework 4.9s 5.0s
Build tests 144s 145s
Build all 149s 150s
Run tests 29ms 26ms
Library size 1.3MB 1.3MB
Executable size 9.9MB 10.0MB

So I think this can be enabled all the time. PR to follow: https://github.com/snitch-org/snitch/pull/151