phpbench: `@BeforeMethod()` does not get called before each call?

I was working on https://github.com/laminas/laminas-servicemanager/pull/93 today, and noticed that we still have the codebase sprinkled with blocks referencing https://github.com/phpbench/phpbench/issues/304

In fact, https://github.com/phpbench/phpbench/issues/304 has been closed as “old”, but still applies today.

My assumption was that @BeforeMethods would be called before each call to said bench method: doesn’t seem to be the case, and that leads to benchmarking warmed up caches too, which is a problem (especially if we’re benchmarking said warmup).

In following example, I would expect 100 calls to mySetup(), but only 10 are occurring:

/** 
 * @BeforeMethods
 * @Revs(10)
 * @Iterations(10)
 */ 
class MyBench
{
    public function mySetup(): void
    {
        reset_all_the_things_here();
    }

    public function benchSomething(): void
    {
        // irrelevant
    }
}

What’s the best way forward here? Is a change in @BeforeMethods viable? It would change the benchmark results for bench suites I’ve worked on so far, massively, but it would lead to more honest results.