pytest: Ordering of session fixtures is wrong when combined with class or module fixtures under some conditions

This is a continuation of the discussion in #519 and #2959, and it may be related to one or both of those issues. I have a test suite where I have to run some very expensive configuration before running the entire test suite, and the set of possible configurations is the Cartesian product of modes that are fixed with flavors that I need to be able to pass in at run-time. Using the example @RonnyPfannschmidt provided in #519 with @popravich’s edit and fixing indentation, I ended up with this code:

def pytest_configure(config):
    class DynamicFixturePlugin(object):
        @pytest.fixture(scope='session', params=config.getoption('--reprovision-flavors').split(','))
        def flavor(self, request):
            return request.param
    config.pluginmanager.register(DynamicFixturePlugin(), 'flavor-fixture')

@pytest.fixture(scope='session', params=['vxlan', pytest.mark.skipif("not pytest.config.getoption('--reprovision-fabric')")('vlan')])
def encap(request):
    return request.param

@pytest.fixture(scope='session', autouse='True')
def reprovision(request, flavor, encap):
    fabric_name = request.config.getoption('--reprovision-fabric')
    if fabric_name:
        kubernetes_.deploy(fabric_name, reinstall=request.config.getoption('--reprovision-reinstall'), encap=encap, **kubernetes_.FLAVORS[flavor])

This half-works. What happens is that for the first two flavor-encap combinations, foo and bar and vxlan, reprovision runs once for each full execution of the test suite. For the second two flavor-encap combinations, reprovision runs twice for every test, as if it was function-scoped rather than session-scoped. I tried this alternate form of the above code with the same result:

def pytest_configure(config):
    class DynamicFixturePlugin(object):
        @pytest.fixture(scope='session', params=config.getoption('--reprovision-flavors').split(','))
        def flavor(self, request):
            return request.param
    config.pluginmanager.register(DynamicFixturePlugin(), 'flavor-fixture')
    class DynamicFixturePlugin2(object):
        @pytest.fixture(scope='session', params=['vxlan', 'vlan'] if config.getoption('--reprovision-fabric') else [])
        def encap(self, request):
            return request.param
    config.pluginmanager.register(DynamicFixturePlugin2(), 'encap-fixture')

@pytest.fixture(scope='session', autouse='True')
def reprovision(request, flavor, encap):
    fabric_name = request.config.getoption('--reprovision-fabric')
    if fabric_name:
        kubernetes_.deploy(fabric_name, reinstall=request.config.getoption('--reprovision-reinstall'), encap=encap, **kubernetes_.FLAVORS[flavor])

This is similar to how pytest_generate_tests breaks session-scoped fixtures.

I’m assuming these are bugs. How hard do you think it will be to fix them? Can anyone point me to where I need to look to start, assuming it’s possible?

This is with pytest 3.2.0 on Python3 and Mac OS X 10.12.6. I have pytest-notifier, pytest-html, and pytest-metadata installed.