cluster-api-provider-azure: Enhancement Proposal: Break out cloud resource reconciliation

⚠️ Cluster API Azure maintainers can ask to turn an issue-proposal into a CAEP when necessary. This is to be expected for large changes that impact multiple components, breaking changes, or new large features.

Goals

1. Separate concerns for cloud resource reconciliation from transformation of Cluster API custom resource specs to cloud resource specs
2. Create parallel path for reconciliation in CAPZ which uses an Azure cloud resource controller / reconciler with Azure CRDs for the cloud resources required for Cluster-API resources, rather than using direct calls to Azure in CAPZ.

Non-Goals/Future Work

1. Future work: possibly, choose to only rely on Azure resource CRDs for reconciliation rather than parallel implementations.

User Story

As a CAPZ developer I would like to only need to interact with the API server and Custom Resource Definitions of Azure types rather than having to deal with the impedance of interacting with the Azure APIs. This would enable me to focus more on the core value of CAPZ, transforming Cluster-API specs to Azure resource specs.

Detailed Description

CAPZ currently has 2 major concerns.

1. Transforming Cluster-API types to Azure types
2. Imperatively interacting the Azure APIs to manipulate / create / delete the Azure resources

I think it would help to speed development in CAPZ if the two concerns were broken out into separate controller / reconcilers.

1. CAPZ: responsible for translating from Cluster-API CRDs to Azure CRDs
2. YTBNC (yet to be named controller): responsible for defining Azure CRDs and reconciling them with Azure

With this enhancement CAPZ could be verified to produce the correct specifications for Azure resources given an input without needing to have a dependency on the Azure API. The vast amount of verification could be done via API server and determining if a given input, Cluster API model, gets translated into the proper Azure resource specifications. At the same time, the concern on reconciling Azure resources could be focused and tested within its own component.

YTBNC (yet to be named controller)

Should have the following characteristics:

• Define versioned CRDs which describe Azure resources across Azure API versions
• Be able to reconcile Azure resources and provide status changes / conditions
• Resources should be able to be applied in any order and converge – goal seeking behavior
  • All of the ordering logic is currently built into CAPZ and is kind of brittle

Example of a ResourceGroup

Currently

      group := resources.Group{
		Location: to.StringPtr(s.Scope.Location()),
		Tags: converters.TagsToMap(infrav1.Build(infrav1.BuildParams{
			ClusterName: s.Scope.Name(),
			Lifecycle:   infrav1.ResourceLifecycleOwned,
			Name:        to.StringPtr(s.Scope.ResourceGroup()),
			Role:        to.StringPtr(infrav1.CommonRoleTagValue),
			Additional:  s.Scope.AdditionalTags(),
		})),
        }

       // Wait on Azure to be done :: **synchronous**
       _, err := s.Client.CreateOrUpdate(ctx, s.Scope.ResourceGroup(), group)

Azure Custom Resource

	rg = resources.ResourceGroup{
		ObjectMeta: v1.ObjectMeta{
			Namespace: nn.Namespace,
			Name: nn.Name,
		},
		Spec: resources.ResourceGroupSpec{
			Location: s.Scope.Location(),
			Tags: converters.TagsToMap(infrav1.Build(infrav1.BuildParams{
				ClusterName: s.Scope.Name(),
				Lifecycle:   infrav1.ResourceLifecycleOwned,
				Name:        to.StringPtr(s.Scope.ResourceGroup()),
				Role:        to.StringPtr(infrav1.CommonRoleTagValue),
				Additional:  s.Scope.AdditionalTags(),
			})),
		},
	}
       
        // PUT to API Server to create the resource
        // Just proceed and check back for condition
        err := s.Scope.Client.Create(ctx, &rg)

Contract changes [optional] None.

Data model changes [optional] The CAPZ CRDs may want to keep a reference to the underlying Azure custom resources which they own.

/kind proposal