LightGBM: segfault during predict

import pickle
model, X, kwargs = pickle.load(open("lgbsegfault.pkl", "rb"))
model.predict(X, **kwargs)

data can be anything for predict, 1 row or whatever fails. Only special thing is extra_trees=True

Backtrace:
/home/jon/minicondadai/lib/python3.6/site-packages/lightgbm_gpu/lib_lightgbm.so(_ZNK8LightGBM4GBDT10PredictRawEPKdPdPKNS_27PredictionEarlyStopInstanceE+0x730)[0x7f4d47691880]
/home/jon/minicondadai/lib/python3.6/site-packages/lightgbm_gpu/lib_lightgbm.so(_ZNK8LightGBM4GBDT7PredictEPKdPdPKNS_27PredictionEarlyStopInstanceE+0x15)[0x7f4d47692585]
/home/jon/minicondadai/lib/python3.6/site-packages/lightgbm_gpu/lib_lightgbm.so(_ZNSt17_Function_handlerIFvRKSt6vectorISt4pairIidESaIS2_EEPdEZN8LightGBM9PredictorC4EPNS9_8BoostingEiibbbbidEUlS6_S7_E3_E9_M_invokeERKSt9_Any_dataS6_OS7_+0x1fa)[0x7f4d4798ce6a]
/home/jon/minicondadai/lib/python3.6/site-packages/lightgbm_gpu/lib_lightgbm.so(+0x40c05c)[0x7f4d4798405c]
/home/jon/minicondadai/lib/python3.6/site-packages/numpy/core/../../../.././libgomp.so.1(GOMP_parallel+0x42)[0x7f4e3d716e8c]
/home/jon/minicondadai/lib/python3.6/site-packages/lightgbm_gpu/lib_lightgbm.so(_ZNK8LightGBM7Booster7PredictEiiiiiSt8functionIFSt6vectorISt4pairIidESaIS4_EEiEERKNS_6ConfigEPdPl+0x205)[0x7f4d47990ce5]
/home/jon/minicondadai/lib/python3.6/site-packages/lightgbm_gpu/lib_lightgbm.so(LGBM_BoosterPredictForMat+0xd1)[0x7f4d479800c1]
/home/jon/minicondadai/lib/python3.6/lib-dynload/../../libffi.so.6(ffi_call_unix64+0x4c)[0x7f4e40b02630]
/home/jon/minicondadai/lib/python3.6/lib-dynload/../../libffi.so.6(ffi_call+0x22d)[0x7f4e40b01fed]
/home/jon/minicondadai/lib/python3.6/lib-dynload/_ctypes.cpython-36m-x86_64-linux-gnu.so(_ctypes_callproc+0x2ce)[0x7f4e3f438f9e]
/home/jon/minicondadai/lib/python3.6/lib-dynload/_ctypes.cpython-36m-x86_64-linux-gnu.so(+0x139d5)[0x7f4e3f4399d5]

lgbsegfault.pkl.zip

About this issue

Original URL
State: closed
Created 3 years ago
Comments: 15

Most upvoted comments

@guolinke I run the example provided by @pseudotensor successfully with both gpu and cuda_exp versions. Both versions provide the same output:

[LightGBM] [Warning] Unknown parameter: silent
[LightGBM] [Warning] Unknown parameter: predict_batching
[LightGBM] [Warning] seed is set=20070863, random_state=42 will be ignored. Current value: seed=20070863
[LightGBM] [Warning] num_threads is set=8, n_jobs=8 will be ignored. Current value: num_threads=8
[402720. 402720. 402720. ... 402720. 402720. 402720.]

@pseudotensor Could you please take a look at this output to see if it is the identical with your previous trials?

shiyu1994 on May 9, 2022

I encountered a similar issue. I am deploying a lgb model within the Tornado framework. The model loaded from pickle file, in the init() function works normally, but when I call the model (used as an instance variable) in an instance method, the predict() or predict_proba() function cause a segmentation fault. I used FaultHandler to trace the exact line that caused the segfault. The result is: python3.6/site-packages/lightgbm/basic.py", line 656 in inner_predict which is: preds.ctypes.data_as(ctypes.pointer(ctypes.c_double))))

Is there a way to solve or work around this issue? Please advise.

peixin-lin on Jan 13, 2022