tensorflow: TypeError: can't pickle _thread.RLock objects
System information
- Have I written custom code (as opposed to using a stock example script provided in TensorFlow): yes
- OS Platform and Distribution (e.g., Linux Ubuntu 16.04): Ubuntu 18.04.3 LTS
- TensorFlow installed from (source or binary): Binary
- TensorFlow version (use command below): 2.0.0-beta1
- Python version: Python 3.6.8
- Bazel version (if compiling from source):
- GCC/Compiler version (if compiling from source):
- CUDA/cuDNN version: CUDA Version 10.1
- GPU model and memory: GeForce GTX 1080 Ti
Describe the current behavior
I’m using GridSearchCV from sklearn
When I start the training process finishes with TypeError: can't pickle _thread.RLock objects
Describe the expected behavior
No error, working grid.fit(train_X, train_Y)
Code to reproduce the issue
import tensorflow
from tensorflow.compat.v2.keras.layers import *
from tensorflow.compat.v2.keras.models import Model
from tensorflow.compat.v2.keras.optimizers import *
from tensorflow.compat.v2.keras.preprocessing import image
from tensorflow.compat.v2.keras.wrappers.scikit_learn import KerasClassifier
import numpy as np
from sklearn.model_selection import GridSearchCV
def loadImages(path):
data = []
for img_name in os.listdir(path):
img = image.load_img(path + img_name, target_size=(512, 512, 1))
img = image.img_to_array(img)
img = img / 255
data.append(img)
return np.array(data)
if __name__ == "__main__":
batch_size = 4
train_frame_path = '/data/segmentation/train_frames/'
train_mask_path = '/data/segmentation/train_masks/'
[segmentation_training.txt]
train_X = loadImages(train_frame_path)
train_Y = loadImages(train_mask_path)
m = unet()
model = KerasClassifier(build_fn=m, epochs=25, batch_size=batch_size, verbose=0)
optimizer = ['SGD', 'RMSprop', 'Adagrad', 'Adadelta', 'Adam', 'Adamax', 'Nadam']
param_grid = dict(optimizer=optimizer)
grid = GridSearchCV(estimator=model, param_grid=param_grid, n_jobs=1, cv=3)
grid_result = grid.fit(train_X, train_Y)
Here there is the full code segmentation_training.txt
Other info / logs
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/home/vlongoba/singularity/testForRadio/jackzen.py", line 120, in gridSearch
grid_result = grid.fit(train_X, train_Y)
File "/home/vlongoba/singularity/.local/lib/python3.6/site-packages/sklearn/model_selection/_search.py", line 633, in fit
base_estimator = clone(self.estimator)
File "/home/vlongoba/singularity/.local/lib/python3.6/site-packages/sklearn/base.py", line 64, in clone
new_object_params[name] = clone(param, safe=False)
File "/home/vlongoba/singularity/.local/lib/python3.6/site-packages/sklearn/base.py", line 55, in clone
return copy.deepcopy(estimator)
File "/usr/lib/python3.6/copy.py", line 180, in deepcopy
y = _reconstruct(x, memo, *rv)
File "/usr/lib/python3.6/copy.py", line 280, in _reconstruct
state = deepcopy(state, memo)
File "/usr/lib/python3.6/copy.py", line 150, in deepcopy
y = copier(x, memo)
File "/usr/lib/python3.6/copy.py", line 240, in _deepcopy_dict
y[deepcopy(key, memo)] = deepcopy(value, memo)
File "/usr/lib/python3.6/copy.py", line 180, in deepcopy
y = _reconstruct(x, memo, *rv)
File "/usr/lib/python3.6/copy.py", line 280, in _reconstruct
state = deepcopy(state, memo)
File "/usr/lib/python3.6/copy.py", line 150, in deepcopy
y = copier(x, memo)
File "/usr/lib/python3.6/copy.py", line 240, in _deepcopy_dict
y[deepcopy(key, memo)] = deepcopy(value, memo)
File "/usr/lib/python3.6/copy.py", line 180, in deepcopy
y = _reconstruct(x, memo, *rv)
File "/usr/lib/python3.6/copy.py", line 280, in _reconstruct
state = deepcopy(state, memo)
File "/usr/lib/python3.6/copy.py", line 150, in deepcopy
y = copier(x, memo)
File "/usr/lib/python3.6/copy.py", line 240, in _deepcopy_dict
y[deepcopy(key, memo)] = deepcopy(value, memo)
File "/usr/lib/python3.6/copy.py", line 180, in deepcopy
y = _reconstruct(x, memo, *rv)
File "/usr/lib/python3.6/copy.py", line 280, in _reconstruct
state = deepcopy(state, memo)
File "/usr/lib/python3.6/copy.py", line 150, in deepcopy
y = copier(x, memo)
File "/usr/lib/python3.6/copy.py", line 240, in _deepcopy_dict
y[deepcopy(key, memo)] = deepcopy(value, memo)
File "/usr/lib/python3.6/copy.py", line 169, in deepcopy
rv = reductor(4)
TypeError: can't pickle _thread.RLock objects
About this issue
- Original URL
- State: closed
- Created 5 years ago
- Comments: 23 (6 by maintainers)
Thank you!! I got that work too!
The method that creates the model that I was passing to the gridsearch was inside a class, it cannot be ( apparently ) so I made a method outside the class and it worked… But I have no idea why this happened, I just figured I should try a method outside of a class and it worked 🤷
@wangxiaoshuai223 thanks so much you are right
I had the same question,maybe you can try to You can try to replace model = KerasClassifier(build_fn=m, epochs=25, batch_size=batch_size, verbose=0) with model = KerasClassifier(build_fn=unet, epochs=25, batch_size=batch_size, verbose=0), this work for me
@badjano I am having the same issue. Can you please elaborate on how you fix it? on this collab notebook : https://colab.research.google.com/drive/11s0YWZrhP9s6pL1w4uTSvvDFGAQMNfTy?usp=sharing