knowhere: Test failure for Ubuntu 20.04 ppc64le architecture
Hi, I want to build knowhere (as part of milvus) for ppc64le and with minimal changes I’m able to successfully build it. However, when I then run tests it’ll fail with this SIGABRT message:
root@c0f59838709d:/knowhere/build# ./Release/tests/ut/knowhere_tests
[...]
I1021 10:38:49.231748 120158 knowhere_config.cc:93] [KNOWHERE][SetBlasThreshold][knowhere_tests] Set faiss::distance_compu
te_blas_threshold to 16384
I1021 10:38:49.231760 120158 knowhere_config.cc:104] [KNOWHERE][SetEarlyStopThreshold][knowhere_tests] Set faiss::early_st
op_threshold to 0
I1021 10:38:49.231784 120158 knowhere_config.cc:115] [KNOWHERE][SetClusteringType][knowhere_tests] Set faiss::clustering_t
ype to 1
I1021 10:38:49.231792 120158 knowhere_config.cc:115] [KNOWHERE][SetClusteringType][knowhere_tests] Set faiss::clustering_t
ype to 0
I1021 10:38:49.231814 120158 knowhere_config.cc:87] [KNOWHERE][SetSimdType][knowhere_tests] FAISS hook
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
knowhere_tests is a Catch2 v3.3.1 host application.
Run with -? for options
-------------------------------------------------------------------------------
Knowhere SIMD config
-------------------------------------------------------------------------------
/knowhere/tests/ut/test_knowhere_init.cc:43
...............................................................................
/knowhere/tests/ut/test_knowhere_init.cc:48: FAILED:
REQUIRE( s.find(res) != s.end() )
with expansion:
{?} != {?}
terminate called after throwing an instance of 'Catch::TestFailureException'
/knowhere/tests/ut/test_knowhere_init.cc:48: FAILED:
{Unknown expression after the reported line}
due to a fatal error condition:
SIGABRT - Abort (abnormal termination) signal
===============================================================================
test cases: 10 | 9 passed | 1 failed
assertions: 99121199 | 99121197 passed | 2 failed
Aborted (core dumped)
The only changes to the git are the following to enable ppc64 builds:
root@c0f59838709d:/knowhere/build# git diff
diff --git a/cmake/libs/libfaiss.cmake b/cmake/libs/libfaiss.cmake
index a78f268..fd1e568 100644
--- a/cmake/libs/libfaiss.cmake
+++ b/cmake/libs/libfaiss.cmake
@@ -30,7 +30,7 @@ if(__X86_64)
target_link_libraries(knowhere_utils PUBLIC glog::glog)
endif()
-if(__AARCH64)
+if(__AARCH64 OR __PPC64)
set(UTILS_SRC src/simd/hook.cc src/simd/distances_ref.cc)
add_library(knowhere_utils STATIC ${UTILS_SRC})
target_link_libraries(knowhere_utils PUBLIC glog::glog)
@@ -85,7 +85,7 @@ if(__X86_64)
target_compile_definitions(faiss PRIVATE FINTEGER=int)
endif()
-if(__AARCH64)
+if(__AARCH64 OR __PPC64)
knowhere_file_glob(GLOB FAISS_AVX_SRCS thirdparty/faiss/faiss/impl/*avx.cpp)
list(REMOVE_ITEM FAISS_SRCS ${FAISS_AVX_SRCS})
diff --git a/cmake/utils/platform_check.cmake b/cmake/utils/platform_check.cmake
index d713a2d..953b3a3 100644
--- a/cmake/utils/platform_check.cmake
+++ b/cmake/utils/platform_check.cmake
@@ -3,8 +3,9 @@ include(CheckSymbolExists)
macro(detect_target_arch)
check_symbol_exists(__aarch64__ "" __AARCH64)
check_symbol_exists(__x86_64__ "" __X86_64)
+ check_symbol_exists(__powerpc64__ "" __PPC64)
- if(NOT __AARCH64 AND NOT __X86_64)
+ if(NOT __AARCH64 AND NOT __X86_64 AND NOT __PPC64)
message(FATAL "knowhere only support amd64 and arm64.")
endif()
endmacro()
diff --git a/conanfile.py b/conanfile.py
index 029c372..5084737 100644
--- a/conanfile.py
+++ b/conanfile.py
@@ -81,7 +81,7 @@ class KnowhereConan(ConanFile):
self.options.rm_safe("fPIC")
def requirements(self):
- self.requires("boost/1.78.0")
+ self.requires("boost/1.75.0")
self.requires("glog/0.4.0")
self.requires("nlohmann_json/3.11.2")
self.requires("openssl/1.1.1t")
I’m aware that there will be no SIMD acceleration and only scalar computation is used. I’ve also tested the exact same code on ubuntu:20.04-aarch64 and there the tests finish successfully.
Anyone know what could be the issue or how to properly debug this?
Thanks!
Information on system & build
OS: ubuntu:20.04
arch: ppc64le
gcc: 9.4.0
(ubuntu 20.04 build-essential default)
knowhere version: v2.2.1
About this issue
- Original URL
- State: closed
- Created 8 months ago
- Comments: 36 (14 by maintainers)
Well I tested this on ubuntu:20.04 (aarch64) as well as almalinux:8 (aarch64) and the tests run successfully for that architecture. I would like to contribute the PR to make this work on ppc64le architecture, but currently I have no idea why this fails 😕 That’s why I’m asking if you guys have an idea 😃
However I will create a PR to support ppc64le at all, since the build process does work.