milvus: Execute hello_milvus.py, got error message "server is not healthy, please try again later"

Steps/Code to reproduce:

  1. create PVs
apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
  name: milvus-etcd
parameters:
  regionid: xxx
  type: cloud_efficiency
  zoneId: xxx
provisioner: xxx
reclaimPolicy: Retain
volumeBindingMode: WaitForFirstConsumer
------
apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
  name: milvus-pulsar
parameters:
  regionid: xxx
  type: cloud_efficiency
  zoneId: xxx
provisioner: xxx
reclaimPolicy: Retain
volumeBindingMode: WaitForFirstConsumer
  1. follow the tutorial https://milvus.io/cn/docs/v2.0.0/install_cluster-docker.md, install by Helm in Kubernetes
helm repo add milvus https://milvus-io.github.io/milvus-helm/
helm repo update
helm install my-release milvus/milvus -f value.yaml
kubectl port-forward service/my-release-milvus 19530

the value.yaml is

cluster:
  enabled: true
minio:
  enabled: false
pulsarStandalone:
  persistence:
    storageClass: milvus-pulsar
    size: 100Gi
externalS3:
  enabled: true
  host: xxx
  port: 80
  accessKey: xxx
  secretKey: xxx
  useSSL: false
  bucketName: xxx
  etcd:
    persistence:
      storageClass: milvus-ectd
      size: 20Gi
  1. recreated the PVC for pulsar
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  annotations:
    meta.helm.sh/release-name: my-release
    meta.helm.sh/release-namespace: default
  labels:
    app.kubernetes.io/instance: my-release
    app.kubernetes.io/managed-by: Helm
    app.kubernetes.io/name: milvus
    app.kubernetes.io/version: 2.0.0-rc.5-hotfix1
    helm.sh/chart: milvus-2.1.15
  name: my-release-milvus-pulsar
  namespace: default
spec:
  accessModes:
  - ReadWriteOnce
  storageClassName: milvus-pulsar
  resources:
    requests:
      storage: 100Gi                 
  1. following above tutorial https://milvus.io/cn/docs/v2.0.0/example_code.md, get hello_milvus.py and modify connections.connect(host='localhost', port='19530'), run the script in the master node where the helm is installed

Expected result:

As above tutorial shown, get logs like this

Search...

(distance: 0.0, id: 2998) -20.0

(distance: 13.2614107131958, id: 989) -11.0

(distance: 14.489648818969727, id: 1763) -19.0

(distance: 15.295698165893555, id: 968) -20.0

(distance: 15.34445571899414, id: 2049) -19.0

(distance: 0.0, id: 2999) -12.0

(distance: 14.63361930847168, id: 1259) -13.0

(distance: 15.421361923217773, id: 2530) -15.0

(distance: 15.427900314331055, id: 600) -14.0

(distance: 15.538337707519531, id: 637) -19.0

search latency = 0.0549s

Actual results:

Traceback (most recent call last):
  File "hello_milvus.py", line 89, in <module>
    hello_milvus()
  File "hello_milvus.py", line 24, in hello_milvus
    connections.connect(host='localhost', port='19530')
  File "/usr/local/lib/python3.6/dist-packages/pymilvus/orm/pymilvus_orm/connections.py", line 158, in connect
    conn = connect_milvus(**kwargs)
  File "/usr/local/lib/python3.6/dist-packages/pymilvus/orm/pymilvus_orm/connections.py", line 148, in connect_milvus
    return Milvus(tmp_host, tmp_port, handler, pool, **tmp_kwargs)
  File "/usr/local/lib/python3.6/dist-packages/pymilvus/client/stub.py", line 121, in __init__
    self._update_connection_pool(channel=channel)
  File "/usr/local/lib/python3.6/dist-packages/pymilvus/client/stub.py", line 176, in _update_connection_pool
    self._wait_for_healthy()
  File "/usr/local/lib/python3.6/dist-packages/pymilvus/client/stub.py", line 145, in _wait_for_healthy
    raise Exception("server is not healthy, please try again later")
Exception: server is not healthy, please try again later

Environment:

  • Milvus version: 2.0.0-rc.6
  • Deployment mode: cluster - kubernetes version: 1.16.6
  • SDK version: pymilvus v2.0.0rc6
  • OS: CentOS

Configuration file:

Additional context:

About this issue

  • Original URL
  • State: closed
  • Created 3 years ago
  • Comments: 18 (9 by maintainers)

Most upvoted comments

Hi all, I came cross nearly the same problem, anyone solve it ?