kotest: Confusing block execution with isInstancePerTest=true

I was trying to debug some test issues and was getting really confused following the code flow when isInstancePerTest is set to true. I really like isInstancePerTest as it makes ensuring state is correct a lot easier, but it’s confusing how many times each block is executed. In this example test:

internal class ClassUnderTestTest : ShouldSpec() {

    override fun isInstancePerTest(): Boolean = true

    init {
        "Scope 1" {
            println(this.context.description().name)
            "Scope 1a" {
                println(this.context.description().name)
                should("1aa") {
                    println(this.description().name)
                    1 shouldBe 1
                }
                "Scope 1ab" {
                    println(this.context.description().name)
                    should("1aba") {
                        println(this.description().name)
                        1 shouldBe 1
                    }
                }
            }
            "Scope 1b" {
                println(this.context.description().name)
                should("1ba") {
                    println(this.description().name)
                    1 shouldBe 1
                }
                "Scope 1bb" {
                    println(this.context.description().name)
                    should("1bba") {
                        println(this.description().name)
                        1 shouldBe 1
                    }
                }
            }
            "Scope 1c" {
                println(this.context.description().name)
                should("1ca") {
                    println(this.description().name)
                    1 shouldBe 1
                }
            }
        }
    }
}

With isInstancePerTest set to false, it prints:

Scope 1
Scope 1a
should 1aa
Scope 1ab
should 1aba
Scope 1b
should 1ba
Scope 1bb
should 1bba
Scope 1c
should 1ca

Which seems expected. With isInstancePerTest set to true, it prints:

Scope 1
Scope 1
Scope 1a
Scope 1
Scope 1a
should 1aa
Scope 1
Scope 1a
Scope 1ab
Scope 1
Scope 1a
Scope 1ab
should 1aba
Scope 1
Scope 1b
Scope 1
Scope 1b
should 1ba
Scope 1
Scope 1b
Scope 1bb
Scope 1
Scope 1b
Scope 1bb
should 1bba
Scope 1
Scope 1c
Scope 1
Scope 1c
should 1ca

Whereas I would have expected:

Scope 1
Scope 1a
should 1aa
Scope 1
Scope 1a
Scope 1ab
should 1aba
Scope 1
Scope 1b
should 1ba
Scope 1bb
should 1bba
Scope 1
Scope 1c
should 1ca

I’ve posted on here before about test execution order and I know it’s a tricky thing, but it’s pretty surprising to try and step through the code with the current way it executes using isInstancePerTest=true

About this issue

  • Original URL
  • State: closed
  • Created 6 years ago
  • Comments: 22 (22 by maintainers)

Commits related to this issue

Most upvoted comments

I’ve fixed this so will be in the next release candidate.

cool, thanks @sksamuel. Excited to upgrade to 3.2.