Codeception: Message output shows wrong step when using can...() methods

What are you trying to achieve?

I want to see where a test is going wrong.

What do you get instead?

I see a failure summary, showing the wrong step. In the summary you can read 133. $I->amOnPage("/settings") but the actual error is happening on this page /docs?pageId=1020&pageSlug=documentation.

I know that can…() methods don’t abort when there’s a failure, that’s intended here. If possible, all failures (or max. 3) should be shown in the summary - or at least always the first or the last.

Functional Tests (2) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
BackendLinksCept: Click backend links
Signature: BackendLinksCept
Test: tests/codeception/functional/BackendLinksCept.php
Scenario --
 I am going to start in the backend

[...]

 I can dont see element ".site-error"
 I can dont see element ".alert-warning .alert"
 I am going to check /docs?pageId=1020&pageSlug=documentation
 I am on page "/docs?pageId=1020&pageSlug=documentation"
 I can dont see element ".site-error"
 FAIL 

 I can dont see element ".alert-warning .alert"
 I am going to check /audit
 I am on page "/audit"
 I can dont see element ".site-error"
 I can dont see element ".alert-warning .alert"
 I am going to check /backend
 I am on page "/backend"
 I can dont see element ".site-error"
 I can dont see element ".alert-warning .alert"
 I am going to check /filefly

[...]

 I am going to check /help
 I am on page "/help"
 I can dont see element ".site-error"
 I can dont see element ".alert-warning .alert"
 I am going to check /guide
 I am on page "/guide"
 I can dont see element ".site-error"
 I can dont see element ".alert-warning .alert"
 I am going to check /settings
 I am on page "/settings"
 I can dont see element ".site-error"
 I can dont see element ".alert-warning .alert"
 PASSED 

VersionCept: Check application versioning
Signature: VersionCept
Test: tests/codeception/functional/VersionCept.php
Scenario --
 I don't see file found "version"
 I see file found "src/version"
 I open file "src/version"
 I don't see in this file "dev"
 I don't see in this file "dirty"
 FAIL 

-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------


Time: 11.13 seconds, Memory: 46.00MB

There were 2 failures:

---------
1) BackendLinksCept: Click backend links
 Test  tests/codeception/functional/BackendLinksCept.php
 Step  Can dont see element ".site-error"
 Fail  Element '.site-error' was found

Scenario Steps:

 135. $I->canDontSeeElement(".alert-warning .alert")
 134. $I->canDontSeeElement(".site-error")
 133. $I->amOnPage("/settings")
 132. // I am going to check /settings
 131. $I->canDontSeeElement(".alert-warning .alert")
 130. $I->canDontSeeElement(".site-error")


---------
2) VersionCept: Check application versioning
 Test  tests/codeception/functional/VersionCept.php
 Step  Don't see in this file "dirty"
 Fail  Found text 'dirty' in currently opened file
Failed asserting that '0.6.1-8-g850a0e7-dirty
' does not contain "dirty".

Scenario Steps:

 5. $I->dontSeeInThisFile("dirty")
 4. $I->dontSeeInThisFile("dev")
 3. $I->openFile("src/version")
 2. $I->seeFileFound("src/version")
 1. $I->dontSeeFileFound("version")


FAILURES!
Tests: 2, Assertions: 68, Failures: 2.
2016-08-31 16:27:49 [][-][-][warning][yii\web\Session::init] Session is already started
root@eba60ad16f38:/app#  

$I = new FunctionalTester($scenario);
$I->wantTo('Click backend links');

$I->amGoingTo('start in the backend');
$url = '/backend';
$I->amOnPage($url);

$links = $I->grabMultiple('section.content a[href^="/"]', 'href');

foreach ($links as $i => $url) {
    $I->amGoingTo('check '.$url);
    $I->amOnPage($url);

    $I->cantSeeElement('.site-error');
    $I->cantSeeElement('.alert-warning .alert');
}

Details

• Codeception version: 2.2.4
• PHP Version: 7.0.8
• Operating System: Linux (Docker)
• Installation type: Composer
• Suite configuration:

class_name: FunctionalTester
modules:
    enabled:
        # add framework module here
        - \Helper\Functional
        - Filesystem
        - Yii2
    config:
        Yii2:
            configFile: 'tests/codeception/_config/test.php'