k8s-bigip-ctlr: Shared Pool not working with v2

Setup Details

CIS Version : 2.6.1 Build: f5networks/k8s-bigip-ctlr:latest
BIGIP Version: BIG-IP v16.1.1 (Build 0.0.16) AS3 Version: 3.31.0
Agent Mode: AS3 Orchestration: K8S
Orchestration Version: 1.20.14
Pool Mode: Nodeport
Additional Setup details: CentOS 7.9

Description

When trying deploy a shared pool, controller v2 seems to run into an issue, even having the service port number defined correctly. The issue appears as followed in the logs:

021/12/15 15:11:00 [ERROR] [AS3] Template is not valid. see errors
2021/12/15 15:11:00 [ERROR] - (root): Must validate one and only one schema (oneOf)
2021/12/15 15:11:00 [ERROR] - declaration.Shared: Must validate "then" as "if" was valid
2021/12/15 15:11:00 [ERROR] - declaration.Shared: Must validate "then" as "if" was valid
2021/12/15 15:11:00 [ERROR] - declaration.Shared.members.0: Must validate "then" as "if" was valid
2021/12/15 15:11:00 [ERROR] - declaration.Shared.members.0: Must validate one and only one schema (oneOf)
2021/12/15 15:11:00 [ERROR] - declaration.Shared.members.0.serverAddresses: array items[0,3] must be unique
2021/12/15 15:11:00 [ERROR] - declaration.Shared.members.0.serverAddresses: array items[1,4] must be unique
2021/12/15 15:11:00 [ERROR] - declaration.Shared.members.0.serverAddresses: array items[2,5] must be unique
2021/12/15 15:11:00 [ERROR] - declaration.Shared.members.0: Must validate all the schemas (allOf)
2021/12/15 15:11:00 [ERROR] - declaration.Shared.members.0: Must validate all the schemas (allOf)
2021/12/15 15:11:00 [ERROR] - declaration.Shared: Must validate all the schemas (allOf)

Steps To Reproduce

  1. Create a Service with the following CIS labels
cis.f5.com/as3-app=Shared
cis.f5.com/as3-pool=alertmanager-cluster_pool
cis.f5.com/as3-tenant=sensu-system_infs_tenant
  1. Create a AS3 declaration that creates a shared pool and uses it in two VS
kind: ConfigMap
apiVersion: v1
metadata:
  name: f5-as3-sensu-system
  namespace: f5
  labels:
    f5type: virtual-server
    as3: "true"
data:
  template: |
    {
    "class": "AS3",
    "action": "deploy",
    "persist": false,
    "declaration": {
        "class": "ADC",
        "schemaVersion": "3.28.0",
        "id": "f5-as3-sensu-system",
        "label": "f5-as3-sensu-system",
        "remark": "AS3 Declaration - NS sensu-system",
        "sensu-system_infs_tenant": {
            "class": "Tenant",
            "Shared": {
                "class": "Application",
                "template": "shared",
                "alertmanager_serviceAddress": {
                    "class": "Service_Address",
                    "virtualAddress": "10.10.10.1",
                    "arpEnabled": true,
                    "icmpEcho": "enable",
                    "routeAdvertisement": "all",
                    "trafficGroup":"/Common/traffic-1"
                },
                "alertmanager-cluster_pool": {
                    "class": "Pool",
                    "loadBalancingMode": "round-robin",
                    "monitors": [
                        "tcp"
                    ],
                    "members": [
                        {
                            "servicePort": 9094,
                            "serverAddresses": [],
                            "shareNodes": true
                        }
                    ]
                }
            },
            "alertmanager-cluster_tcp": {
                "class": "Application",
                "template": "generic",
                "alertmanager-cluster_tcp": {
                    "class": "Service_TCP",
                    "remark": "alertmanager.x.x.x",
                    "virtualAddresses": [
                        {
                            "use": "/sensu-system_infs_tenant/Shared/alertmanager_serviceAddress"
                        }
                    ],
                    "virtualPort": 9094,
                    "allowVlans": [
                        "/Common/VL-lb-int"
                    ],
                    "snat": "auto",
                    "pool": "/sensu-system_infs_tenant/Shared/alertmanager-cluster_pool"
                }
            },
            "alertmanager-cluster_udp": {
                "class": "Application",
                "template": "generic",
                "alertmanager-cluster_udp": {
                    "class": "Service_UDP",
                    "remark": "alertmanager.x.x.x",
                    "virtualAddresses": [
                        {
                            "use": "/sensu-system_infs_tenant/Shared/alertmanager_serviceAddress"
                        }
                    ],
                    "virtualPort": 9094,
                    "allowVlans": [
                        "/Common/VL-lb-int"
                    ],
                    "snat": "auto",
                    "pool": "/sensu-system_infs_tenant/Shared/alertmanager-cluster_pool"
                }
            },
        }
    }
}

Expected Result

Two virtual servers should be created with a shared pool

Actual Result

CIS runs into an issue

Diagnostic Information

<Configuration files, error messages, logs>
Note: Sanitize the data. For example, be mindful of IPs, ports, application names and URLs
Note: The following F5 article outlines the information required when opening an issue.
https://support.f5.com/csp/article/K60974137

Observations (if any)

About this issue

  • Original URL
  • State: closed
  • Created 3 years ago
  • Comments: 26 (14 by maintainers)

Most upvoted comments

@vklohiya we ran it in the debugger and noticed, that the fix which was merged with #2396 does not handle duplicate pool members when the controller is deployed in NodePort mode.

we just added

} else { // Controller is in NodePort mode.
    if service.Spec.Type == v1.ServiceTypeNodePort {
        for _, port := range service.Spec.Ports {
            endpointMembers := appMgr.getEndpointsForNodePort(port.NodePort, port.Port)
            for _, newMember := range endpointMembers {
                if _, ok := uniqueMembersMap[newMember]; !ok {
                        uniqueMembersMap[newMember] = struct{}{}
                        members = append(members, newMember)
                }
            }
        }
    }
}

and made it working.

Please verify and release a bugfix as soon as possible.

Cheers Sebi

Hi @vklohiya these ports are coming from the Prometheus operator / helm chart and are unfortunately not configurable 😕