phpunit: Inconsistency between "phpunit --list-suites" and startTestSuite events

Q A
PHPUnit version 6.5.8
PHP version 7.2.3
Installation Method Composer

Preconditions

  1. phpunit.xml:
<?xml version="1.0" encoding="UTF-8"?>
<phpunit backupGlobals="false"
         backupStaticAttributes="false"
         bootstrap="vendor/autoload.php"
         colors="true"
         convertErrorsToExceptions="true"
         convertNoticesToExceptions="true"
         convertWarningsToExceptions="true"
         processIsolation="false"
         stopOnFailure="false">
    <testsuites>
        <testsuite name="Feature">
            <directory suffix="Test.php">./tests/Feature</directory>
        </testsuite>
        <testsuite name="Unit">
            <directory suffix="Test.php">./tests/Unit</directory>
        </testsuite>
    </testsuites>
    <filter>
        <whitelist processUncoveredFilesFromWhitelist="true">
            <directory suffix=".php">./app</directory>
        </whitelist>
    </filter>
    <listeners>
        <listener class="App\Listeners\Testing\CustomTestListener">
        </listener>
    </listeners>
    <php>
        <env name="APP_ENV" value="testing"/>
        <env name="BCRYPT_ROUNDS" value="4"/>
        <env name="CACHE_DRIVER" value="array"/>
        <env name="SESSION_DRIVER" value="array"/>
        <env name="QUEUE_DRIVER" value="sync"/>
        <env name="MAIL_DRIVER" value="array"/>
    </php>
</phpunit>
  1. CustomTestListener.php:
<?php

namespace App\Listeners\Testing;

use PHPUnit\Framework\TestListener;
use PHPUnit\Framework\TestListenerDefaultImplementation;
use PHPUnit\Framework\TestSuite;

class CustomTestListener implements TestListener
{
    use TestListenerDefaultImplementation;

    public function startTestSuite(TestSuite $suite)
    {
        printf("TestSuite '%s' started.\n", $suite->getName());
    }
}
  1. I have 2 test methods in tests\Unit\RegistrationServiceTest.php

Actual behavior

  1. Running “vendor/bin/phpunit --list-suites” returns the following output: PHPUnit 6.5.8 by Sebastian Bergmann and contributors.

Available test suite(s):

  • Feature
  • Unit
  1. Running “vendor/bin/phpunit --testsuite Unit” returns the following output: PHPUnit 6.5.8 by Sebastian Bergmann and contributors.

TestSuite ‘’ started. TestSuite ‘Unit’ started. TestSuite ‘Tests\Unit\RegistrationServiceTest’ started. … 2 / 2 (100%)

Expected behavior

I do not expect going through startTestSuite() multiple when I have explicitly set the test suite I want to execute.

  1. Why are 3 suites started: ‘’, ‘Unit’, ‘Tests\Unit\RegistrationServiceTest’?
  2. Is this expected?
  3. What are the benefits of this behavior?
  4. Is it configurable?

About this issue

  • Original URL
  • State: closed
  • Created 6 years ago
  • Reactions: 1
  • Comments: 15 (6 by maintainers)

Most upvoted comments

I can only recommend to not use the TestListener interface but the new TestHook interfaces instead.