apisix-ingress-controller: bug: Upstream HealthCheck Issue - Unhealthy Upstream doesn't be excluded temporarily
Current Behavior
There is a problem with the unhealthy external service being delivered as it is without being excluded from routing targets.
Expected Behavior
Two external services (ALB configured in front of each) are configured as upstream nodes and should be temporarily excluded from routing if a 5XX error occurs through health check configuration.
Error Logs
No response
Steps to Reproduce
For reproducing the issue, one service is deployed, and the other one is not deployed (only ALB’s are set up.)
apiVersion: apisix.apache.org/v2
kind: ApisixRoute
metadata:
name: route
spec:
http:
- match:
hosts:
- service.corp.com
methods:
- POST
paths:
- /svc/*
name: route
plugins:
- enable: true
name: traffic-split
- config:
regex_uri:
- ^\/svc\/(.+)$
- /$1
enable: true
name: proxy-rewrite
upstreams:
- name: upstream
weight: 100
apiVersion: apisix.apache.org/v2
kind: ApisixUpstream
metadata:
name: upstream
namespace: apisix
spec:
externalNodes:
- name: svc01.corp.com
port: 443
type: Domain
weight: 50
- name: svc02.corp.com
port: 443
type: Domain
weight: 50
healthCheck:
active:
healthy:
httpCodes:
- 200
- 404
interval: 3s
successes: 1
httpPath: /
type: https
unhealthy:
httpCodes:
- 503
- 504
httpFailures: 1
interval: 3s
timeouts: 3
passive:
healthy:
httpCodes:
- 200
- 404
successes: 1
type: https
unhealthy:
httpCodes:
- 503
- 504
httpFailures: 1
timeouts: 3
loadbalancer:
type: roundrobin
passHost: node
scheme: https
Environment
- APISIX Ingress controller version (run
apisix-ingress-controller version --long) - Kubernetes cluster version (run
kubectl version) - OS version if running APISIX Ingress controller in a bare-metal environment (run
uname -a)
Runs on an AWS EKS Cluster (Kubernetes v1.25). Uses APISIX Helm Chart (1.11.0, App 3.8.0).
About this issue
- Original URL
- State: closed
- Created 3 years ago
- Comments: 41 (18 by maintainers)
@kworkbee Thanks for reporting. I’ll take a look
@Revolyssup It will take some time to check. I will mention it again as soon as I check. Thank you for your quick response.
The following specific logs are visible in the APISIX Log. Did SSL Handshake Failure Affect?
@Revolyssup As you just mentioned, this is the result of erasing and re-applying both
ApisixRouteandApisixUpstream.Can you delete both ApisixRoute and ApisixUpstream, reapply and show logs?