molecule: Test running on the wrong VM with multiple-instance and Vagrant

Issue Type

  • Bug report

Molecule and Ansible details

ansible --version

ansible 2.6.1.post0 (stable-2.6 8c829778d1) last updated 2018/07/17 21:18:01 (GMT -300)
  config file = /Users/robinho/.ansible.cfg
  configured module search path = [u'/Users/robinho/.ansible/plugins/modules', u'/usr/share/ansible/plugins/modules']
  ansible python module location = /Users/robinho/app/ansible/src/lib/ansible
  executable location = /Users/robinho/app/ansible/venv/bin/ansible
  python version = 2.7.15 (default, Jun 17 2018, 12:46:58) [GCC 4.2.1 Compatible Apple LLVM 9.1.0 (clang-902.0.39.2)]
molecule --version

molecule, version 2.16.0

Molecule installation method (one of):

  • pip

Ansible installation method (one of):

  • source

Desired Behavior

Run the test on the right virtual machine

Actual Behaviour (Bug report only)

Hi guys!

I’m using molecule + vagrant to test a ansible module. I’d like to test it on Ubuntu 16.04 and 18.04.

So, my molecule.yml is:

---
dependency:
  name: galaxy
driver:
  name: vagrant
  provider:
    name: virtualbox
lint:
  name: yamllint
platforms:
  - name: ubuntu-xenial64
    box: ubuntu/xenial64
  - name: ubuntu-bionic64
    box: ubuntu/bionic64
provisioner:
  name: ansible
  lint:
    name: ansible-lint
scenario:
  name: default
verifier:
  name: testinfra
  lint:
    name: flake8

I got a fail on the test and I try to debug. As pdb doesn’t work, I add an assert forcing the failure after run the command.

def test_hostname_is_resolved(host):
    hostname_cmd = host.run('hostname -s')
    assert hostname_cmd.rc == 0
    assert hostname_cmd.stdout == ''

The output is:

___
_ test_hostname_is_resolved[ansible://ubuntu-bionic64] ____

    host = <testinfra.host.Host object at 0x109bbb550>, Ansible = <ansible>

        def test_hostname_is_resolved(host, Ansible):
            hostname_cmd = host.run('hostname -s')
            assert hostname_cmd.rc == 0
    >       assert hostname_cmd.stdout == ''
    E       AssertionError: assert 'ubuntu-xenial64' == ''
    E         - ubuntu-xenial64

    tests/test_default.py:40: AssertionError
    ____ test_hostname_is_resolved[ansible://ubuntu-xenial64] ____

    host = <testinfra.host.Host object at 0x109bbb8d0>, Ansible = <ansible>

        def test_hostname_is_resolved(host, Ansible):
            hostname_cmd = host.run('hostname -s')
            assert hostname_cmd.rc == 0
    >       assert hostname_cmd.stdout == ''
    E       AssertionError: assert 'ubuntu-bionic64' == ''
    E         - ubuntu-bionic64

So, the bionic64 tests are running on xenial64. And the xenial64 tests are running on bionic64.

But the login works well:

$ molecule login --host ubuntu-xenial64
--> Validating schema .../molecule/default/molecule.yml.
Validation completed successfully.
Warning: Permanently added '[127.0.0.1]:2222' (ECDSA) to the list of known hosts.
...
vagrant@ubuntu-xenial64:~$ hostname -s
ubuntu-xenial64

$ molecule login --host ubuntu-bionic64
--> Validating schema .../molecule/default/molecule.yml.
Validation completed successfully.
Warning: Permanently added '[127.0.0.1]:2200' (ECDSA) to the list of known hosts.
vagrant@ubuntu-bionic64:~$ hostname -s
ubuntu-bionic64

Has anyone got this error? How to solve it?

Thanks

Below has the output of molecule --debug test for the toy project https://github.com/robsonpeixoto/molecule-vagrant-bug.

https://gist.github.com/robsonpeixoto/bab49bb305b865e839f2b780b6a400f2

About this issue

  • Original URL
  • State: closed
  • Created 6 years ago
  • Comments: 16 (8 by maintainers)

Commits related to this issue

Most upvoted comments

Looks to be a testinfra 1.13.1 bug. Just opened.

I receive the following, which seems to make sense. 1 of 2 failures where ubuntu-xenial64 does not match ubuntu-bionic64.

    =================================== FAILURES ===================================
    __________________ test_hosts_file[ansible://ubuntu-xenial64] __________________

    host = <testinfra.host.Host object at 0x1119bde90>

        def test_hosts_file(host):
            hostname = host.run('hostname -s').stdout
    >       assert 'ubuntu-xenial64' == hostname
    E       AssertionError: assert 'ubuntu-xenial64' == 'ubuntu-bionic64'
    E         - ubuntu-xenial64
    E         + ubuntu-bionic64

    tests/test_default.py:20: AssertionError
    ====================== 1 failed, 1 passed in 2.82 seconds ======================```

tests/test_default.py::test_hosts_file[ansible://ubuntu-bionic64] PASSED [ 50%]
tests/test_default.py::test_hosts_file[ansible://ubuntu-xenial64] FAILED [100%]```

My bad. I’ll push it.