istio: Set priorities when using distribute localityLbSetting (distribute + failover mode?)

Describe the feature request

Hi,

When using the global mesh LocalityLoadBalancerSetting in distribute mode, pilot doesn’t set any priority in the generated LocalityLbEndpoints (in https://github.com/istio/istio/blob/master/pilot/pkg/networking/core/v1alpha3/loadbalancer/loadbalancer.go#L80 )

The priorities are only applied when failover is enabled, based on locality.

It would be extremely useful if weighted and prioritised distribution could be applied: this would allow for, for example, cross-AZ traffic to be priority 0, with different weightings per-AZ, but cross-region traffic could be dropped to priority 1, as a failover case. Envoy’s docs and APIs say that this is possible, with priorities being checked before weightings.

I think this could be exposed through an additional prioritised setting, e.g.

distribute:
    - from: *
      to:
        "us-west/zone1/*": 49
        "us-west/zone2/*": 49
        "us-east/*: 2
      prioritised: true

where priorities would be determined based on locality, as in the current failover implementation. I don’t think there’s a need to specify explicit priorities.

or even by allowing both failover and distribute to be specified? I’m not sure of the reasoning behind only allowing one or the other - is there a design doc around that?

Describe alternatives you’ve considered

I tried using an EnvoyFilter to set priorities for existing LocalityLbEdndpoints but with the JSON merging semantics I was only able to append new endpoints, not modify existing list items.

Additional context

About this issue

  • Original URL
  • State: closed
  • Created 7 years ago
  • Reactions: 8
  • Comments: 21 (17 by maintainers)

Most upvoted comments

I suggest to support value 0 in distribute.to.locality[].weight.

distribute:
    - from: *
      to:
        "us-west/zone1/*": 51
        "us-west/zone2/*": 49
        "us-east/*: 0

The envoy translation will be the same as currently being done. But just for the locality whose value is 0, priority will be set to 1 which will depict failover in case all the above localities are unhealthy. According to envoy documentation, priority is picked first, so us-east/* won’t be getting any traffic and will act as failover. WDYT?

Added for you @tanujd11

With distribute, I think 2 priority levels should be enough isn’t it. I mean if we mix the regionalFailover settings with distribute, that will become more complex for users to understand. All the regions with 0 weights could be considered as least priority.