cluster-api-provider-proxmox: Machine gets stuck in provisioning state

What steps did you take and what happened:

  • I’m trying to provision a kubernetes cluster in proxmox VM using talos bootstrap and controlplane provider
  • I have deployed management cluster using capi-operator-system helmchart as I want to utilize GitOps using fluxCD. Below is my management cluster configurations:
---
apiVersion: helm.toolkit.fluxcd.io/v2
kind: HelmRelease
metadata:
  name: capi-operator
  namespace: capi-operator-system
spec:
  releaseName: capi-operator
  interval: 15m
  timeout: 90s
  chart:
    spec:
      chart: "cluster-api-operator"
      version: 0.13.0
      sourceRef:
        kind: HelmRepository
        name: capi-operator
  values:
    core: "cluster-api:v1.7.5"
    infrastructure: "proxmox:v0.5.1"
    controlPlane: "talos:v0.5.6;kubeadm:v1.4.2"
    bootstrap: "talos:v0.6.5;kubeadm:v1.4.2"
    addon: "helm"
    manager:
      featureGates:
        core:
          ClusterTopology: true
          MachinePool: true
        proxmox:
          ClusterTopology: true
---
apiVersion: operator.cluster.x-k8s.io/v1alpha2
kind: IPAMProvider
metadata:
  name: proxmox-ipam
  namespace: proxmox-infrastructure-system
spec:
  version: v0.1.0-alpha.3
  fetchConfig:
    url: https://github.com/kubernetes-sigs/cluster-api-ipam-provider-in-cluster/releases/download/v0.1.0-alpha.3/ipam-components.yaml

  • I’m trying to deploy single controlPlane cluster using proxmox vm template with talos initialized in it. Below is my cluster manifest that I’m using to provision talos cluster in proxmox vm
apiVersion: cluster.x-k8s.io/v1beta1
kind: Cluster
metadata:
  name: talos-test
spec:
  clusterNetwork:
    pods:
      cidrBlocks:
      - 10.0.6.0/16
  controlPlaneRef:
    apiVersion: controlplane.cluster.x-k8s.io/v1alpha3
    kind: TalosControlPlane
    name: talos-test
  infrastructureRef:
    apiVersion: infrastructure.cluster.x-k8s.io/v1alpha1
    kind: ProxmoxCluster
    name: pride
  controlPlaneEndpoint:
    host: "10.0.1.164"
    port: 6443
  
---
apiVersion: infrastructure.cluster.x-k8s.io/v1alpha1
kind: ProxmoxCluster
metadata:
  name: pride
spec:
  controlPlaneEndpoint:
    host: "10.0.1.164"
    port: 6443
  ipv4Config:
    addresses: [10.0.1.174-10.0.1.175]
    prefix: 20
    gateway: 10.0.1.1
  dnsServers: [10.0.1.1]
  allowedNodes: [px1]
  credentialsRef:
    name: "pride-proxmox-credentials"

---
apiVersion: controlplane.cluster.x-k8s.io/v1alpha3
kind: TalosControlPlane
metadata:
  name: talos-test
spec:
  version: v1.30.1
  replicas: 1
  infrastructureTemplate:
    kind: ProxmoxMachineTemplate
    apiVersion: infrastructure.cluster.x-k8s.io/v1alpha1
    name: talos-cp
  controlPlaneConfig:
    init:
      generateType: init
    controlplane:
      generateType: controlplane
      talosVersion: v1.7.4
        
---
apiVersion: infrastructure.cluster.x-k8s.io/v1alpha1
kind: ProxmoxMachineTemplate
metadata:
  name: "talos-cp"
spec:
  template:
    spec:
      sourceNode: "px1"
      templateID: 110
      format: "qcow2"
      full: true
      numSockets: 1
      numCores: 2
      memoryMiB: 2048
      disks:
        bootVolume:
          disk: scsi0
          sizeGb: 8
      network:
        default:
          bridge: vmbr0
          model: virtio

Now the machine is created successfully in proxmox with IP assigned to it as well but machine phase in the management cluster is stuck in provisioning state as a result no further action of bootstrapping by talos takes place as it keeps waiting for infrastructure to be ready. Upon cheking the logs of capmox-controller-manager, this is what I found:

E0923 08:06:57.505071       1 controller.go:329] "Reconciler error" err="failed to reconcile VM: error waiting for agent: the operation has timed out" controller="proxmoxmachine" controllerGroup="infrastructure.cluster.x-k8s.io" controllerKind="ProxmoxMachine" ProxmoxMachine="talos-cp2-n9h6b" name="talos-cp2-n9h6b" reconcileID="f55e43c1-ba7d-43f5-bbee-8c1fd90b3202"

Note that I have already enables qemu-agent in the VM template as well.

Status of machine in management cluster:

image

What did you expect to happen: Machine is provisioned successfully and control plane is initialized.

Environment:

  • Cluster-api-provider-proxmox version:v0.5.1
  • Kubernetes version: (use kubectl version):v1.30.1
  • OS (e.g. from /etc/os-release): talos:v1.7.4

About this issue

  • Original URL
  • State: closed
  • Created 2 years ago
  • Comments: 20 (2 by maintainers)

Most upvoted comments

@khushalmer03 and @pborn-ionos I think i got it.

According to these docs It seems the new way is to use boot assets by setting .machine.install.image like so:

apiVersion: controlplane.cluster.x-k8s.io/v1alpha3
kind: TalosControlPlane
metadata:
  name: talos-cp
spec:
  version: v1.32.0
  replicas: 1
  infrastructureTemplate:
    kind: ProxmoxMachineTemplate
    apiVersion: infrastructure.cluster.x-k8s.io/v1alpha1
    name: control-plane-template
    namespace: default
  controlPlaneConfig:
    controlplane:
      generateType: controlplane
      talosVersion: v1.10
      strategicPatches:
        - |
          - op: replace
            path: /machine/install
            value:
              disk: /dev/sda
              extraKernelArgs:
                - net.ifnames=0
              image: factory.talos.dev/nocloud-installer/ce4c980550dd2ab1b17bbf2b08801c7eb59418eafe8f279833297925d67c7515:v1.10.2

I’m not too familiar with the Talos boot process, but this basically causes the VM to boot into that image after the installation image (which is the one from the Proxmox Template VM) triggers the reboot. Thanks for helping out guys!