terraform-provider-lxd: zfs.block_mode or initial.zfs.block_mode broken?
Hi,
I’ve been trying to implement this for a couple of containers that will be hosting docker swarm services to try and eliminate the IO bottleneck on ZFS, however using this device block:
device {
name = "root"
type = "disk"
properties = {
"path" = "/"
"pool" = "local"
#"size" = "200GiB"
"initial.zfs.block_mode" = true
}
}
Results in the following errors:
╷
│ Error: Provider produced inconsistent result after apply
│
│ When applying changes to lxd_instance.int-dev-zvol["int-dev-zvol01"], provider "provider[\"registry.terraform.io/terraform-lxd/lxd\"]"
│ produced an unexpected new value: .device: actual set element cty.ObjectVal(map[string]cty.Value{"name":cty.StringVal("root"),
│ "properties":cty.MapVal(map[string]cty.Value{"initial.zfs.block_mode":cty.StringVal("true"), "path":cty.StringVal("/"),
│ "pool":cty.StringVal("local")}), "type":cty.StringVal("disk")}) does not correlate with any element in plan.
│
│ This is a bug in the provider, which should be reported in the provider's own issue tracker.
╵
╷
│ Error: Provider produced inconsistent result after apply
│
│ When applying changes to lxd_instance.int-dev-zvol["int-dev-zvol01"], provider "provider[\"registry.terraform.io/terraform-lxd/lxd\"]"
│ produced an unexpected new value: .device: block set length changed from 1 to 2.
│
│ This is a bug in the provider, which should be reported in the provider's own issue tracker.
╵
This does appear to create the zvol block devices as desired, however:
rpool/lxd/containers/int-dev-swarm01 946M 2.08T 1.74G legacy
rpool/lxd/containers/int-dev-swarm02 947M 2.08T 1.74G legacy
rpool/lxd/containers/int-dev-worker01 947M 2.08T 1.74G legacy
rpool/lxd/containers/int-dev-zvol01 1.03G 2.08T 1.03G -
rpool/lxd/virtual-machines/int-dev-lab04.block 3.76G 2.08T 4.52G -
(The ‘legacy’ ones are normal datasets, and lab04 has a manually created, attached and functional ext4 block device)
Am I omitting something basic? Essentially I just need the provider to create zvol block devices for specific container instances rather than zfs datasets when specified.
If I manually set the pool to use zfs.block_mode this works without issue, without the disk device being specified.
Versions: lxd 5.19 lxd provider 2.3.0 terraform 1.5.3
About this issue
- Original URL
- State: closed
- Created 2 years ago
- Comments: 28 (13 by maintainers)
Thanks for the feedback, I agree that we should include such an example in the documentation.
Morning Din,
I can confirm with the above code plus a minor tweak (specifying target), this now works as expected.
Thanks for all the help and guidance with this - I’d have thought this would be a reasonably common usecase/scenario enough for the docs.
I can confirm that after updating to 5.21.2 LTS, the profile works as expected 😃 thank you!
Regarding ‘container’ type volumes (or anything other than the ‘custom’ type), is it on the roadmap for our usecase to be achievable? aka, attaching a standard container with a zvol block volume mounted to /var/lib/docker, configured and deployed with this provider?
I think the issue was detected in our test suite, but here is the PR: https://github.com/canonical/lxd/pull/13656 and https://github.com/canonical/lxd/pull/13861
Have a nice weekend as well 😃
Excellent, thanks for letting me know. I’ll upgrade the cluster next week and we’ll go from there. Do you have a link to the issue, out of interest?
Have a good weekend!
Perfect. I’ll give this a try tomorrow and let you know the result. Thank you!
Sure.
lxd_profile:
lxd_instance:
So it seems this does function, but the zvol/disk device can only be defined under lxd_instance, not lxd_profile.