milvus: [Bug]: Load failed reporting "

Is there an existing issue for this?

  • I have searched the existing issues

Environment

- Milvus version: latest
- Deployment mode(standalone or cluster): standalone
- SDK version(e.g. pymilvus v2.0.0rc2): latest
- OS(Ubuntu or CentOS): 
- CPU/Memory: 
- GPU: 
- Others:

Current Behavior

Load failed:

Logs: [standalone: artifacts-milvus-standalone-master-220-pymilvus-e2e-logs.tar.gz] [please download from through the upper log name] https://ci.milvus.io:18080/jenkins/job/milvus-nightly-ci/job/master/220/

Other clues:

Failed case 1:

  1. Failed case: FAILED testcases/test_search_20.py::TestCollectionSearch::test_search_after_different_index_with_params[8-False-True-HNSW-params4]

  2. Collection name: search_collection_w4WHSbge

  3. Timeline:

[2021-11-16 15:29:18,685 - DEBUG - ci_test]: (api_request)  : [Collection.load] args: [None], kwargs: {'timeout': 20} (api_request.py:44)
[2021-11-16 15:30:36,929 - ERROR - ci_test]: Traceback (most recent call last):
  File "/home/jenkins/agent/workspace/tests/python_client/utils/api_request.py", line 18, in inner_wrapper
    res = func(*args, **kwargs)
  File "/home/jenkins/agent/workspace/tests/python_client/utils/api_request.py", line 45, in api_request
    return func(*arg, **kwargs)
  File "/usr/local/lib/python3.6/site-packages/pymilvus/orm/collection.py", line 449, in load
    conn.load_collection(self._name, timeout=timeout, **kwargs)
  File "/usr/local/lib/python3.6/site-packages/pymilvus/client/stub.py", line 58, in handler
    raise e
  File "/usr/local/lib/python3.6/site-packages/pymilvus/client/stub.py", line 42, in handler
    return func(self, *args, **kwargs)
  File "/usr/local/lib/python3.6/site-packages/pymilvus/client/stub.py", line 322, in load_collection
    return handler.load_collection("", collection_name=collection_name, timeout=timeout, **kwargs)
  File "/usr/local/lib/python3.6/site-packages/pymilvus/client/grpc_handler.py", line 75, in handler
    raise e
  File "/usr/local/lib/python3.6/site-packages/pymilvus/client/grpc_handler.py", line 67, in handler
    return func(self, *args, **kwargs)
  File "/usr/local/lib/python3.6/site-packages/pymilvus/client/grpc_handler.py", line 823, in load_collection
    self.wait_for_loading_collection(collection_name, timeout)
  File "/usr/local/lib/python3.6/site-packages/pymilvus/client/grpc_handler.py", line 75, in handler
    raise e
  File "/usr/local/lib/python3.6/site-packages/pymilvus/client/grpc_handler.py", line 67, in handler
    return func(self, *args, **kwargs)
  File "/usr/local/lib/python3.6/site-packages/pymilvus/client/grpc_handler.py", line 841, in wait_for_loading_collection
    return self._wait_for_loading_collection_v2(collection_name, timeout)
  File "/usr/local/lib/python3.6/site-packages/pymilvus/client/grpc_handler.py", line 868, in _wait_for_loading_collection_v2
    raise BaseException(response.status.error_code, response.status.reason)
pymilvus.client.exceptions.BaseException: <BaseException: (code=1, message=rpc error: code = Unknown desc = collection has not been loaded to memory or load failed)>

Failed case 2:

  1. Failed case
[2021-11-16T15:33:08.981Z] [gw0] [ 41%] FAILED testcases/test_search_20.py::TestCollectionSearch::test_search_after_index_different_metric_type[128-True-True-IVF_FLAT-params1] 
  1. collection name: search_collection_sp6bGVTh

  2. Timeline:

[2021-11-16 15:32:59,995 - ERROR - ci_test]: Traceback (most recent call last):
  File "/home/jenkins/agent/workspace/tests/python_client/utils/api_request.py", line 18, in inner_wrapper
    res = func(*args, **kwargs)
  File "/home/jenkins/agent/workspace/tests/python_client/utils/api_request.py", line 45, in api_request
    return func(*arg, **kwargs)
  File "/usr/local/lib/python3.6/site-packages/pymilvus/orm/collection.py", line 449, in load
    conn.load_collection(self._name, timeout=timeout, **kwargs)
  File "/usr/local/lib/python3.6/site-packages/pymilvus/client/stub.py", line 58, in handler
    raise e
  File "/usr/local/lib/python3.6/site-packages/pymilvus/client/stub.py", line 42, in handler
    return func(self, *args, **kwargs)
  File "/usr/local/lib/python3.6/site-packages/pymilvus/client/stub.py", line 322, in load_collection
    return handler.load_collection("", collection_name=collection_name, timeout=timeout, **kwargs)
  File "/usr/local/lib/python3.6/site-packages/pymilvus/client/grpc_handler.py", line 75, in handler
    raise e
  File "/usr/local/lib/python3.6/site-packages/pymilvus/client/grpc_handler.py", line 67, in handler
    return func(self, *args, **kwargs)
  File "/usr/local/lib/python3.6/site-packages/pymilvus/client/grpc_handler.py", line 823, in load_collection
    self.wait_for_loading_collection(collection_name, timeout)
  File "/usr/local/lib/python3.6/site-packages/pymilvus/client/grpc_handler.py", line 75, in handler
    raise e
  File "/usr/local/lib/python3.6/site-packages/pymilvus/client/grpc_handler.py", line 67, in handler
    return func(self, *args, **kwargs)
  File "/usr/local/lib/python3.6/site-packages/pymilvus/client/grpc_handler.py", line 841, in wait_for_loading_collection
    return self._wait_for_loading_collection_v2(collection_name, timeout)
  File "/usr/local/lib/python3.6/site-packages/pymilvus/client/grpc_handler.py", line 868, in _wait_for_loading_collection_v2
    raise BaseException(response.status.error_code, response.status.reason)
pymilvus.client.exceptions.BaseException: <BaseException: (code=1, message=rpc error: code = Unknown desc = collection has not been loaded to memory or load failed)>

Expected Behavior

Load success

Steps To Reproduce

No response

Anything else?

No response

About this issue

  • Original URL
  • State: closed
  • Created 3 years ago
  • Comments: 21 (18 by maintainers)

Most upvoted comments

@DragonDriver any update?