terraform-provider-libvirt: Permission Denied
System Information
Linux distribution
Ubuntu 18.04-lts
Terraform version
terraform -v
Terraform v0.11.11
+ provider.libvirt (unversioned)
+ provider.template v2.0.0
Provider and libvirt versions
terraform-provider-libvirt -version
terraform-provider-libvirt 0.5.1
Compiled against library: libvirt 4.0.0
Using library: libvirt 4.0.0
Running hypervisor: QEMU 2.11.1
Running against daemon: 4.0.0
If that gives you “was not built correctly”, get the Git commit hash from your local provider repository:
git describe --always --abbrev=40 --dirty
Checklist
-
Is your issue/contribution related with enabling some setting/option exposed by libvirt that the plugin does not yet support, or requires changing/extending the provider terraform schema?
- Make sure you explain why this option is important to you, why it should be important to everyone. Describe your use-case with detail and provide examples where possible.
- If it is a very special case, consider using the XSLT support in the provider to tweak the definition instead of opening an issue
- Maintainers do not have expertise in every libvirt setting, so please, describe the feature and how it is used. Link to the appropriate documentation
-
Is it a bug or something that does not work as expected? Please make sure you fill the version information below:
Description of Issue/Question
Setup
(Please provide the full main.tf file for reproducing the issue (Be sure to remove sensitive information)
provider "libvirt" {
uri = "qemu:///system"
}
#
## VM Varibles
#
variable "instance_count" {
default = "1"
}
variable "disk_img" {
default = "https://cloud-images.ubuntu.com/releases/xenial/release/ubuntu-16.04-server-cloudimg-amd64-disk1.img"
}
#
## Networking Varibles
#
variable "vm_network_addresses" {
description = "Defines the network in the CIDR format"
default = "10.0.1.0/24"
}
variable "vm_network_name" {
description = "Defines the network name"
default = "vm_network"
}
variable "domain_name" {
default = "k8s.local"
}
#---- End Of Varibles ---#
resource "libvirt_volume" "os_image" {
name = "os_image"
source = "${var.disk_img}"
}
#volume
resource "libvirt_volume" "volume" {
name = "volume-${count.index}"
base_volume_id = "${libvirt_volume.os_image.id}"
count = "${var.instance_count}"
}
# network
resource "libvirt_network" "vm_network" {
name = "${var.vm_network_name}"
addresses = ["${var.vm_network_addresses}"]
domain = "${var.domain_name}"
mode = "nat"
dhcp {
enabled = true
}
dns {
local_only = true
}
autostart = true
}
#Domain
resource "libvirt_domain" "domain" {
name = "master-${count.index}"
memory = "1024"
vcpu = "1"
network_interface {
network_id = "${libvirt_network.vm_network.id}"
hostname = "master-${count.index}"
wait_for_lease = true
}
console {
type = "pty"
target_port = "0"
target_type = "serial"
}
console {
type = "pty"
target_type = "virtio"
target_port = "1"
}
disk {
volume_id = "${libvirt_volume.volume.id}"
}
count = "${var.instance_count}"
}
#############
## Outputs ##
#############
output "disk_id" {
value = "${libvirt_volume.volume.*.id}"
}
output "network_id" {
value = "${libvirt_network.vm_network.id}"
}
output "ip_addresses" {
value = "${libvirt_domain.domain.*.network_interface.0.addresses}"
}
Steps to Reproduce Issue
(Include debug logs if possible and relevant).
Error: Error applying plan:
1 error(s) occurred:
* libvirt_domain.domain: 1 error(s) occurred:
* libvirt_domain.domain: Error creating libvirt domain: virError(Code=1, Domain=10, Message='internal error: process exited while connecting to monitor: 2019-01-28T02:29:14.861688Z qemu-system-x86_64: -drive file=/var/lib/libvirt/images/volume-0,format=qcow2,if=none,id=drive-virtio-disk0: Could not open '/var/lib/libvirt/images/volume-0': Permission denied')
Additional information:
Do you have SELinux or Apparmor/Firewall enabled? Some special configuration? Have you tried to reproduce the issue without them enabled?
About this issue
- Original URL
- State: closed
- Created 5 years ago
- Reactions: 1
- Comments: 23 (1 by maintainers)
Commits related to this issue
- Explain qemu security_driver need on Debian/Ubuntu AppArmor security driver might block in some cases qemu libvirt file access on Debian/Ubuntu. This has already been discussed on the Debian side ... — committed to EmmanuelKasper/installer by EmmanuelKasper 2 years ago
- Ubuntu 22.04 main workstation setup * fix instruction message at end of bootstrap script * Add sourcing /etc/profile to ZSH profile * Add sourcing /etc/profile to common user bash * Fix typo in KDE p... — committed to symball/the-lab by deleted user a year ago
- Ubuntu 22.04 main workstation setup * fix instruction message at end of bootstrap script * Add sourcing /etc/profile to ZSH profile * Add sourcing /etc/profile to common user bash * Fix typo in KDE p... — committed to symball/the-lab by deleted user a year ago
I fixed this problem with this:
On Ubuntu distros SELinux is enforced by qemu even if it is disabled globally, this might cause unexpected
Could not open '/var/lib/libvirt/images/<FILE_NAME>': Permission denied
errors. Double check thatsecurity_driver = "none"
is uncommented in/etc/libvirt/qemu.conf
and issuesudo systemctl restart libvirt-bin
to restart the daemon.Still running into this
libvirt
issue on 20.04.2 regardless of the pool used.Slightly modified workaround is setting
security_driver = "none"
in/etc/libvirt/qemu.conf
but followed bysudo systemctl restart libvirtd
.@gdombrov and @johnjameswhitman – That fix worked for me as well on Ubuntu 20.04.1.
@MalloZup Makes sense. Thanks for all of your work on this project!
Could very well be same as #97 see fix there, already in documentation, but very easy to miss, still an issue on ubuntu 18.04, documentation; 22f096d
If you have a problem with Could not open ‘/var/lib/libvirt/images/example.qcow2’: Permission denied’ You should add:
sudo vim /etc/apparmor.d/libvirt/TEMPLATE.qemu
/var/lib/libvirt/images/**.qcow2 rwk, /var/lib/libvirt/images/**.raw rwk, /var/lib/libvirt/images/**.img rwk,
sudo systemctl restart libvirtd
good
This seems like a known issue w/ apparmor’s handling of volumes from a libvirt pool (apparmor profiles require the full path to allow file-access, but
virt-aa-helper
doesn’t actually map the volume to its path on the filesystem). Some details:The quick-fix for me was setting
security = "none"
in/etc/libvirt/qemu.conf
until ubuntu/libvirt fixes the apparmor profiles.Terraform seems to create the pool with owner/group set to 0:
However libvirtd (which writes the uploaded files, such as the cloud init, or the base ISO) runs as non-root, writing those files as
libvirt-qemu:kvm
:I think there may be some XSLT that can be applied to the pool definitions that will change the owner/group to the right settings, but I will need to research them.
This is still true in ubuntu 20.04, thanks to @andersla that pointed to the
security_driver
thing, that did the trick. That may be added back to the readme.Ubuntu 20.04 with apparmor, I disable apparmor without any effect. Start apparmor and set a value
security_driver = none
in qemu.conf and everything right.@MalloZup - After another day of research I have found some historical context that is interesting. However I DO AGREE… This is NOT an issue with terraform-provider-libvirt.
This looks like it may be a recurrence of an issue where QEMU is failing to generate the correct permissions for qcow2 formatted volumes. Either way this issue should be closed on your side. Would be great if we could get some traction with the QEMU team and see what they can find. Thanks again for supporting this provider for terraform!