terraform-provider-proxmox: CloudInit configuration is not getting applied

@windowsrefund Yep, that’s basically what i wrote somewhere at the beginning of the thread. Not as concise though, i must admit 😃

1. You don’t need to pass all types of cloud-config (meaning user/network/vendor/meta).
2. Those elements (meaning GUI cloud-config subset and all the custom stuff) are not conflicting on the silo/type level so to say. Meaning you can pass userdata via GUI and some other data via custom config.
3. That’s basically what i came to - i pass custom cloud-config userdata but leave possibility for end users to pass network configuration via either “initialization” arguments in provider or GUI.
4. I’m still not sure if it’s possible to overwrite custom cloud-config with the stuff you pass via GUI. For now all my research points to resounding no. That’s actually was a major bummer for me - i wanted to preserve possibility for end users to change userdata via GUI, so they could for example add their own ssh-keys to the guest VM. I just gave up on that i guess.
5. Yes, you can pass custom cloud-config vendordata too - i think if you check examples in github you’ll find it. But keep in mind that in case there’s the same cloud-init modules used in user and vendor datas they are merged. And userdata takes precedence, meaning vendordata gets overwritten.

@ratiborusx Yea, it was definitely your earlier comments that perked my interest and got me heading down the rabbit hole. I’m glad I did as I have a better understanding of things. I really had to spend a few hours doing things by hand (via qm) as well as mounting the read-only cloudinit itself in order to cat the files inside. Thanks for helping to shed the light.

I’ve learned some things about Proxmox+cloud-init which may add some value to some areas of this thread.

• The Proxmox GUI only shows and interacts with a tiny subset of what is possible with cloud-init.
• In order to build a custom cloud-init, all 4 types must be passed to --cicustom as identified on this thread.
• When attempting to use qm cloudinit dump to view one or more types, only that small subset which is “used” by the Proxmox GUI is shown. In other words, the dump approach is useless when working with a custom cloud-init.
• The dump approach mentioned previously also can not report against the vendor type. That said, it is unknown to me if Proxmox supports it despite it needing to be represented when setting --cicustom.

In general, the clobbering nature of proxmox_virtual_environment_vm.initialization.ip_config and proxmox_virtual_environment_vm.initialization.user_account over their *_data_file_id counterparts seems really finicky and just ripe for pointless confusion. I’m personally going to avoid using them and just stick with the data files since that’s where all the cloud-init power lies anyway. I understand the value in the perception end users are quickly bootstrapping all the things but wonder if that’s enough reason to continue supporting these conflicting elements. Food for thought.

And another longread. I’m sorry…

@bpg I’m not exactly sure what is the reason but it probably needs to be highlighted in documentation. When you create a VM template do not forget to set “started = false”. Default is “true” which is sane but not for template considering it’s not supposed to be ran at all. Else you’ll be getting updates in-place on any plan generation even though VM itself wouldn’t be started ever:

Terraform used the selected providers to generate the following execution plan. Resource actions are indicated with the following symbols:
  ~ update in-place

Terraform will perform the following actions:

  # module.proxmox.proxmox_virtual_environment_vm.template_ubuntu_2204 will be updated in-place
  ~ resource "proxmox_virtual_environment_vm" "template_ubuntu_2204" {
        id                      = "1011"
      ~ ipv4_addresses          = [] -> (known after apply)
      ~ ipv6_addresses          = [] -> (known after apply)
        name                    = "template-ubuntu-22.04"
      ~ network_interface_names = [] -> (known after apply)
        tags                    = [
            "template-ubuntu-22.04",
            "templates",
            "terraform",
        ]
        # (21 unchanged attributes hidden)

        # (9 unchanged blocks hidden)
    }

Plan: 0 to add, 1 to change, 0 to destroy.

Also, even with “started = false” i’m still getting this “architecture” update for template VM after it was already created (once on the next plan generation):

...
module.proxmox.proxmox_virtual_environment_vm.template_ubuntu_2204: Creating...
module.proxmox.proxmox_virtual_environment_vm.template_ubuntu_2204: Creation complete after 7s [id=1011]

ratiborus@HOMEWORLD:~/WORKSPACE/terraform_yandex/proxmox$ terraform apply
...

Terraform used the selected providers to generate the following execution plan. Resource actions are indicated with the following symbols:
  ~ update in-place

Terraform will perform the following actions:

  # module.proxmox.proxmox_virtual_environment_vm.template_ubuntu_2204 will be updated in-place
  ~ resource "proxmox_virtual_environment_vm" "template_ubuntu_2204" {
        id                      = "1011"
        name                    = "template-ubuntu-22.04"
        tags                    = [
            "template-ubuntu-22.04",
            "templates",
            "terraform",
        ]
        # (24 unchanged attributes hidden)

      ~ cpu {
          + architecture = "x86_64"
            # (7 unchanged attributes hidden)
        }

        # (8 unchanged blocks hidden)
    }

Plan: 0 to add, 1 to change, 0 to destroy.

It happens only once after template VM resource creation and also does not happen if i recreate resource (i.e. with “terraform taint”). No difference whether “template = true” or “false”. I also tried it on other Linux distribution (Astra) and the result is the same. Here’s test resources:

resource "proxmox_virtual_environment_file" "cloud_image_ubuntu_2204" {
  content_type = "iso"
  datastore_id = "local"
  node_name    = "prox-srv1"

  source_file {
    path = "https://cloud-images.ubuntu.com/releases/22.04/release/ubuntu-22.04-server-cloudimg-amd64.img"
  }
}

resource "proxmox_virtual_environment_vm" "template_ubuntu_2204" {
  name        = "template-ubuntu-22.04"
  description = "Ubuntu 22.04 template"
  tags        = sort(["terraform", "templates", "template-ubuntu-22.04"])
  pool_id     = proxmox_virtual_environment_pool.templates.id

  node_name     = "prox-srv1"
  vm_id         = 1011
  migrate       = true
  on_boot       = false
  started       = false
  template      = true
  scsi_hardware = "virtio-scsi-single"

  agent {
    enabled = true
    trim    = true
  }

  cpu {
    architecture = "x86_64"
    cores        = 1
    numa         = true
    sockets      = 2
    type         = "host"
  }

  memory {
    dedicated    = 2048
  }

  disk {
    datastore_id = "local"
    file_id      = proxmox_virtual_environment_file.cloud_image_ubuntu_2204.id
    file_format  = "qcow2"
    interface    = "scsi0"
    cache        = "none"
    size         = 20
    discard      = "on"
    iothread     = true
    ssd          = true
  }

  initialization {
    datastore_id = "local"
    interface    = "ide2"
    ip_config {
      ipv4 {
        address = "dhcp"
      }
    }
    user_data_file_id = proxmox_virtual_environment_file.cloud_config_userdata.id
  }

  network_device {
    bridge = "vmbr0"
  }

  operating_system {
    type = "l26"
  }

  serial_device {
    device = "socket"
  }

  vga {
    enabled = true
    memory  = 32
    type    = "serial0"
  }

  # lifecycle {
  #   ignore_changes = [
  #     initialization[0].user_data_file_id
  #   ]
  # }
}

Not sure but maybe all that info should be covered in a dedicated issue? Just feel like i’m a bit off-topic here and deluting main question 😃

@bpg, well slightly the same.

Example terraform file:

resource "proxmox_virtual_environment_file" "vm_cloud_init" {
  content_type = "snippets"
  datastore_id = "local"
  node_name    = var.proxmox_settings.node

  source_raw {
    file_name = "cloud-init.user-data.yml"
    data      = file("cloud-init/user-data.yml")
  }
}

Example cloud-init/user-data.yml file:

#cloud-config
package_upgrade: true
packages:
  - qemu-guest-agent

timezone: Europe/Stockholm
users:
  - name: myusername
    groups: [adm, cdrom, dip, plugdev, lxd, sudo]
    lock-passwd: false
    sudo: ALL=(ALL) NOPASSWD:ALL
    shell: /bin/bash
    ssh_authorized_keys:
      - ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAINyDI3c8l5Se4w0B2ejL36WJh/PdD5MKKsL9zX0a9sT

power_state:
  delay: now
  mode: reboot
  message: Rebooting after cloud-init completion
  condition: true

Proxmox GUI:

My expectation was to see my user that was provisioned and the ssh-key, but that’s not shown.

I run version Proxmox version 8.0.3 and Terraform provider bpg/proxmox version 0.33.0.

On my part, after your other reply I seemed to made it work (see my librenms IAC repo)

But I need to retry the provider to be certain or at least report what I did wrong.

On Wed, Oct 11, 2023, 22:28 Pavel Boldyrev @.***> wrote:

Hi @webtroter https://github.com/webtroter, @tobiasehlert https://github.com/tobiasehlert! 👋🏼

I see you upvoted this issue, do you have a similar problem in your deployments? If yes, would you mind sharing your configuration?

Thanks!

— Reply to this email directly, view it on GitHub https://github.com/bpg/terraform-provider-proxmox/issues/586#issuecomment-1758815983, or unsubscribe https://github.com/notifications/unsubscribe-auth/AB77R6BIXTOD2RODIAG7G53X65ITZANCNFSM6AAAAAA5DIECAY . You are receiving this because you were mentioned.Message ID: @.***>

@bpg Thanks for the response. I tried it with the other interface and all, but it still wont work. This might be a issue with the distribution, as you areusing ubuntu and it works for you, idk. I will try your ubuntu template and clone next and see it has something to do with the linux distribution, alternatively I will also try and reinstall proxmox and see if this might fixes the issue.

Hello,

I’m also having this problem

I found this small thread in the Proxmox forums about the Regeneration process. -> https://forum.proxmox.com/threads/regenerate-cloud-init-image-using-ansible.89964/

It is basically a remove/re-add the cloudinit drive.

I hope it could help solve this problem.

Thank you

@bpg I took a brief look at the example @some-random-username-123 provided and apparently the packer template is built by specifying bios to ovmf therefore I think this is related to #575

https://github.com/some-random-username-123/Proxmox-setup/blob/2eacc4eb816018997f21833596c6867548dc0310/packer/AlmaLinux9-Template/AlmaLinux9-Template.pkr.hcl#L61

Hi @bpg, Sure I can give you more insights:

Also I'd like to clarify whther the original VM that you're cloning has a cloudinit config? I Tried both, so i have a emplate with a Cloudinit drive, and i also have a template with no cloudinit drive. I have the issue with both tamplates.

The exact issue is, that when I clone my vm with the TF provider, after the vm starts the cloudinit config (user and hostname, etc) is not applied in the template.

But when I clone it manually and press the button, all the cloudinit configuration is applied.

If you want to have a look at my code feel free to look at my repo: https://github.com/some-random-username-123/Proxmox-setup/tree/main

I hope this gives you some more insight.

Regards