terraform-provider-libvirt: libvirt_network creation fails for existing bridge

Version Reports:

Distro version of host:

Ubuntu 18.04

Terraform Version Report

Terraform v0.11.8
+ provider.libvirt (unversioned)

Libvirt version

Compiled against library: libvirt 4.0.0
Using library: libvirt 4.0.0
Using API: QEMU 4.0.0
Running hypervisor: QEMU 2.11.1

terraform-provider-libvirt plugin version (git-hash)

0.4.2

Description of Issue/Question

By default virsh allows creation on a libvirt network on existing host bridge. https://libvirt.org/formatnetwork.html#examplesBridge

That works if you create libvirt network with virsh.

Using terraform it throws

* libvirt_network.default: Error crearing libvirt network: virError(Code=38, Domain=0, Message='error creating bridge interface virbr0: File exists')

But the new network is created anyway.

<network>
  <name>default</name>
  <uuid>89d03309-b412-43f5-81bc-82b46bf81ec8</uuid>
  <bridge name='virbr0' stp='on' delay='0'/>
  <mac address='52:54:00:0e:cd:ba'/>
</network>

Also rerunning terraform apply fails with

2018-08-21T15:23:36.825+0700 [DEBUG] plugin.terraform-provider-libvirt: 2018/08/21 15:23:36 [ERR] plugin: stream copy 'stderr' error: session shutdown
* libvirt_network.default: Error defining libvirt network: virError(Code=9, Domain=19, Message='operation failed: network 'default' already exists with uuid 89d03309-b412-43f5-81bc-82b46bf81ec8') -   <network>
      <name>default</name>
      <bridge name="virbr0" stp="on"></bridge>
      <domain></domain>
  </network>

since the original run fails to collect metadata, id of network.

Setup

(Please provide the full main.tf file for reproducing the issue (Be sure to remove sensitive info)

provider "libvirt" {
  uri = "qemu+tcp://root@172.16.255.254/system"
}

resource "libvirt_network" "default" {
  name = "default"
  mode = "bridge"
  bridge = "virbr0"
}

Steps to Reproduce Issue

(Include debug logs if possible and relevant.)

terraform init terraform plan

  bridge: "" => "virbr0"
  mode:   "" => "bridge"
  name:   "" => "default"
2018-08-21T15:07:42.603+0700 [DEBUG] plugin.terraform-provider-libvirt: 2018/08/21 15:07:42 [INFO] Creating libvirt network at qemu+tcp://root@172.16.255.254/system
2018-08-21T15:07:42.603+0700 [DEBUG] plugin.terraform-provider-libvirt: 2018/08/21 15:07:42 [DEBUG] Creating libvirt network at qemu+tcp://root@172.16.255.254/system:   <network>
2018-08-21T15:07:42.603+0700 [DEBUG] plugin.terraform-provider-libvirt:       <name>default</name>
2018-08-21T15:07:42.603+0700 [DEBUG] plugin.terraform-provider-libvirt:       <bridge name="virbr0" stp="on"></bridge>
2018-08-21T15:07:42.603+0700 [DEBUG] plugin.terraform-provider-libvirt:       <domain></domain>
2018-08-21T15:07:42.603+0700 [DEBUG] plugin.terraform-provider-libvirt:   </network>
2018/08/21 15:07:42 [ERROR] root: eval: *terraform.EvalApplyPost, err: 1 error(s) occurred:

* libvirt_network.default: Error crearing libvirt network: virError(Code=38, Domain=0, Message='error creating bridge interface virbr0: File exists')
2018/08/21 15:07:42 [ERROR] root: eval: *terraform.EvalSequence, err: 1 error(s) occurred:

Additional Infos:

Do you have SELinux or Apparmor/Firewall enabled? Some special configuration? Have you tried to reproduce the issue without them enabled?

About this issue

Original URL
State: closed
Created 6 years ago
Reactions: 1
Comments: 21 (3 by maintainers)

Most upvoted comments

I started working with RancherOS, as they provide a qemu-guest-agent docker image. I guess you should be able to get this working on CoreOS too. Basically, what I did:

Download the docker image on your host:

docker pull docker.io/rancher/os-qemuguestagent:v1.4.0-rc1

Export the docker image to a .tar file:

docker export rancher/os-qemuguestagent:v1.4.0-rc1  -o qemu-guest-agent.tar

Add the folder as a mount to your Terraform VMs (inside the libvirt_domain resource):

filesystem {
  source = "/media/terraform/images"
  target = "qemu_docker_image"
  readonly = true
}

And mount the folder on the host via cloudinit, ignition or just run it as a command. This here is displayed as a mount resource in cloudinit:

- - qemu_docker_image
  - /media/images
  - 9p
  - trans=virtio,version=9p2000.L,rw

Import the docker image on the host:

docker load -i /media/images/qemu-guest-agent.tar

Now that you’ve got the image on the host, you need to start a container with it. I am using the RancherOS Service template, so I did not have to figure out the options for the container. The RancherOS Service template looks like this:

qemu-guest-agent:
  image: rancher/os-qemuguestagent:v1.4.0-rc1
  command: ["/usr/bin/qemu-ga"]
  privileged: true
  restart: always
  labels:
    io.rancher.os.scope: system
  pid: host
  ipc: host
  net: host
  uts: host
  volumes_from:
  - command-volumes
  volumes:
  - /dev:/host/dev

If needed, I could provide the output of a docker inspect on the running container.

tommyknows on Nov 27, 2018

@tommyknows , i have the same issue: I want to install qemu-guest-agent on CoreOS. Do you have found a solution?

remoe on Nov 25, 2018

but there’s no networking, right? It’s not that I just can’t inspect it with KVM, the guest does not have any kind of connection (?). -> Means I’d need to copy a file onto the host by sharing a volume.

(And “installing” in CoreOS would mean running a docker container, although I’m going to try this.

tommyknows on Nov 5, 2018

yop, in bridge mode you need the qemu-guest-agent installed on the Domain. Afaik this is the only solution and yop wait_for_lease wait forever because we cannot get IP without the qemu-agent

MalloZup on Nov 5, 2018

@tommyknows as workaround you can cut the bridge generation via terraform.

So basically like this example here: https://github.com/dmacvicar/terraform-provider-libvirt/blob/master/examples/ubuntu/ubuntu-example.tf#L41

We don’t have any network/bridge creation in terraform, you just attach the domain to existing ones and the creation should not be part of the TF file.

The comment above means:

if you create a br via terraform-libvirt currently you can create it only 1 time. At moment the codebase is so that if you do an apply 2 times, you will have problem as posted in this issue. So the best solution is to don’t create network/bridge via terraform-libvirt.

You can still specify the domain external to tf network here: https://github.com/dmacvicar/terraform-provider-libvirt/blob/master/website/docs/r/domain.html.markdown#handling-network-interfaces

hope it helps 👍 💮

MalloZup on Nov 5, 2018