linkerd2: RFC: Break up linkerd install into stages

This RFC proposes modification to the linkerd install flow, dividing Linkerd installation into stages grouped by permission level.

Current State

Installation of the full suite of components requires three commands, with varying combinations of flags:

linkerd install-cni

  • ClusterRole
    • linkerd-cni
  • ClusterRoleBinding
    • linkerd-cni
  • Namespace
    • linkerd
  • ServiceAccount
    • linkerd-cni
  • DaemonSet
    • linkerd-cni
  • ConfigMap
    • linkerd-cni-config

linkerd install

  • ClusterRole
    • linkerd-linkerd-controller
    • linkerd-linkerd-prometheus
  • ClusterRoleBinding
    • linkerd-linkerd-controller
    • linkerd-linkerd-prometheus
  • Namespace
    • linkerd
  • CustomResourceDefinition
    • serviceprofiles.linkerd.io
  • ServiceAccount
    • linkerd-controller
    • linkerd-grafana
    • linkerd-prometheus
    • linkerd-web
  • ConfigMap
    • linkerd-grafana-config
    • linkerd-prometheus-config
  • Service
    • linkerd-controller-api
    • linkerd-grafana
    • linkerd-prometheus
    • linkerd-proxy-api
    • linkerd-proxy-injector
    • linkerd-web
  • Deployment
    • linkerd-controller
    • linkerd-grafana
    • linkerd-prometheus
    • linkerd-web

linkerd install --tls optional

  • ClusterRole
    • linkerd-linkerd-ca
  • ClusterRoleBinding
    • linkerd-linkerd-ca
  • ServiceAccount
    • linkerd-ca
  • ConfigMap
    • linkerd-ca-bundle
  • Deployment
    • linkerd-ca

linkerd install --tls optional --proxy-auto-inject

  • ClusterRole
    • linkerd-linkerd-proxy-injector
  • ClusterRoleBinding
    • linkerd-linkerd-proxy-injector
  • ConfigMap
    • linkerd-proxy-injector-sidecar-config
  • Service
    • linkerd-proxy-injector
  • Deployment
    • linkerd-proxy-injector

linkerd install-sp

  • ServiceProfile
    • linkerd-controller-api
    • linkerd-proxy-api
    • linkerd-prometheus
    • linkerd-grafana

Proposal: Multi-stage installation

All components listed above, grouped by privilege, --admin and --user:

$ linkerd install --help
Output Kubernetes configs to install Linkerd.

Usage:
  linkerd install [flags]

Flags:
      --admin  Install components requiring cluster-wide privileges.
      --user   Install components requiring namespace-wide privileges.

The linkerd check command can be modified to mirror the linkerd install flow:

linkerd check --admin
linkerd check --user

1. Cluster Admin

Default usage:

linkerd install --admin

Possible modifier:

linkerd install --admin --cni
  • ClusterRole
    • linkerd-cni
    • linkerd-linkerd-ca
    • linkerd-linkerd-controller
    • linkerd-linkerd-prometheus
    • linkerd-linkerd-proxy-injector
  • ClusterRoleBinding
    • linkerd-cni
    • linkerd-linkerd-ca
    • linkerd-linkerd-controller
    • linkerd-linkerd-prometheus
    • linkerd-linkerd-proxy-injector
  • Namespace
    • linkerd
  • ServiceAccount
    • linkerd-cni
  • ConfigMap
    • linkerd-cni-config
  • DaemonSet
    • linkerd-cni
  • CustomResourceDefinition
    • serviceprofiles.linkerd.io

2. Cluster User

linkerd install --user
  • ServiceAccount
    • linkerd-ca
    • linkerd-controller
    • linkerd-grafana
    • linkerd-prometheus
    • linkerd-web
  • ConfigMap
    • linkerd-ca-bundle
    • linkerd-grafana-config
    • linkerd-prometheus-config
    • linkerd-proxy-injector-sidecar-config
  • Deployment
    • linkerd-ca
    • linkerd-controller
    • linkerd-grafana
    • linkerd-prometheus
    • linkerd-proxy-injector
    • linkerd-web
  • Service
    • linkerd-controller-api
    • linkerd-grafana
    • linkerd-prometheus
    • linkerd-proxy-api
    • linkerd-proxy-injector
    • linkerd-web
  • ServiceProfile
    • linkerd-controller-api
    • linkerd-proxy-api
    • linkerd-prometheus
    • linkerd-grafana

Developer installation

linkerd install shall continue to work as an atomic installation process, assuming the user has cluster access sufficient to install all Linkerd components.

To address the timing issue between installing CRD/serviceprofiles and the control-plane ServiceProfiles, the ServiceProfiles shall be installed via a job configured in the linkerd install output.

About this issue

  • Original URL
  • State: closed
  • Created 5 years ago
  • Comments: 24 (24 by maintainers)

Commits related to this issue

Most upvoted comments

What about just removing the ClusterRoleBindings entirely from linkerd install setup with a flag?

linkerd install setup --no-rbac

You can’t make a ClusterRoleBinding that is namespace specific. This feels like it’d be nicer for cluster admins as well to audit.

I’d always looked at ClusterRoleBindings and RoleBindings as an either/or kind of thing, but I like the idea of always just doing a Role/RoleBinding on the namespace level. Makes permissions a little more clear.

Just need to think through the ClusterRole and Role implications (which is the biggest differentiator right now).

If there was some YAML that a cluster admin could run to setup cluster level things (like the CRDs) and still have the control plane restricted to a single namespace, that might be a valid workaround.

Yes, I think that is how it would work. A cluster admin would run linkerd install setup | kubectl apply -f - (which creates CRDs, etc) and create a namespace for a regular user to use: kubectl create ns alex. Then a regular user would be able to linkerd install -l alex control-plane | kubectl apply -f -. This would install the control plane into the alex namespace without requiring any cluster level permissions.

I think this gets us everything we want without any special --single-namespace mode.

Just my 2c:

  • I think a linkerd install CLI wizard which performs multiple steps sequentially is a really nice user experience, but it also is in tension with the project goals of being transparent and non-magical. Having two different install methods (applying yaml and install wizard) means more surface area to support and maintain. My vote would be to not have this.
  • How important is it that we support an unprivileged mode? I don’t think it’s unreasonable to require that having a cluster admin run a setup command is a hard prerequisite for Linkerd. This lets us remove unprivileged/single-namespace entirely which greatly simplifies the code and reduces the number of configuration permutations.
  • We should standardize on if linkerd check X is intended to run before or after linkerd install x. I think running check after makes sense, which means we’d also need a linkerd check pre which can be run before any linkerd install command.
  • I also like the idea of linkerd check without any further sub-command being something that runs ALL the checks in sequence and basically tells you which command to run next. i.e. “You’re cluster is correctly set up, next run linkerd install control-plane” or whatever.

All together I think the workflow would be:

linkerd check pre # All good, cluster is ready for Linkerd setup

linkerd install setup # Must be run by admin
linkerd check setup # All good
linkerd check # Linkerd setup is complete, next run `linkerd install control-plane`

linkerd install control-plane # Can be run by user
linkerd check control-plane # All good
linkerd check # Linkerd control-plane installed, next inject some services

linkerd inject 
linkerd check data-plane # All good
linkerd check # All good

Product goals, in priority order:

  1. Each step should be clear, understandable, and fit with Kubernetes concepts and idioms.
  2. Things that require different privilege levels should be delineated between separate steps, so that they can be run by separate teams/people.
  3. Linkerd should still be usable in a locked down environment without elevated permissions, even if that mode has fewer features.
  4. The install flow be should “easy”, i.e. the common case should not require a lot of commands, a lot of flags, or navigating a big decision tree.

In other words: clarity first, easiness last.