keras: F ./tensorflow/core/util/cuda_launch_config.h:127] Check failed: work_element_count > 0 (0 vs. 0)
my computer get two gpus，it’ s normal if i don’t add this code in the context.
model = keras.utils.training_utils.multi_gpu_model(base_model, gpus=2)
however, it will use only one to compute, i don’t understand what’s it means ‘work_element_count > 0’. is it that i have not cleared the cuda worker before ?
About this issue
- Original URL
- State: closed
- Created 6 years ago
- Reactions: 10
- Comments: 16
@mahaishou This issue was resolved when I upgraded tensorflow-gpu version to 1.9.0.
@qiuyinglin control the number of one epoch input. I use keras to train model, just like this history = multi_model.fit(X_train, Y_train, batch_size=batch_size, epochs=1, validation_data = (X_test, Y_test)) X_train is my input data, so just control the length of X_train.
beacuse there has gpu process none data in one batch. Notice the input number. I uses four gpu and input 2 sample, this will occur. it’s none business of the environment. If you have 2 gpus, make sure one batch at least has 2 sample.