tensorflow: At Runtime : "Error while reading resource variable softmax/kernel from Container: localhost"

I had the same issue in tensorflow 1.13.1 which I have resolved by creating a reference to the session that is used for loading the models and then to set it to be used by keras in each request. I.e. I have done the following:

from tensorflow.python.keras.backend import set_session
from tensorflow.python.keras.models import load_model

tf_config = some_custom_config
sess = tf.Session(config=tf_config)
graph = tf.get_default_graph()

# IMPORTANT: models have to be loaded AFTER SETTING THE SESSION for keras! 
# Otherwise, their weights will be unavailable in the threads after the session there has been set
set_session(sess)
model = load_model(...)

and then in each request (i.e. in each thread):

global sess
global graph
with graph.as_default():
    set_session(sess)
    model.predict(...)

This is not Build/Installation or Bug/Performance issue. Please post this kind of support questions at Stackoverflow. There is a big community to support and learn from your questions. GitHub is mainly for addressing bugs in installation and performance. Thanks!

I had the same issue in tensorflow 1.13.1 which I have resolved by creating a reference to the session that is used for loading the models and then to set it to be used by keras in each request. I.e. I have done the following:

from tensorflow.python.keras.backend import set_session
from tensorflow.python.keras.models import load_model

tf_config = some_custom_config
sess = tf.Session(config=tf_config)
graph = tf.get_default_graph()

# IMPORTANT: models have to be loaded AFTER SETTING THE SESSION for keras! 
# Otherwise, their weights will be unavailable in the threads after the session there has been set
set_session(sess)
model = load_model(...)

and then in each request (i.e. in each thread):

global sess
global graph
with graph.as_default():
    set_session(sess)
    model.predict(...)

You are amazing!!! This is the best solution. Can you tell me why did it just work after adding session?

Thank you and you are very welcome 😃.

As far as I understand, the problem is that tensorflow graphs and sessions are not thread safe. So by default a new session (which does not contain any previously loaded weights, models a.s.o.) is created for each thread, i.e. for each request. By saving the global session that contains all your models and setting it to be used by keras in each thread the problem is solved.

I had the same issue in tensorflow 1.13.1 which I have resolved by creating a reference to the session that is used for loading the models and then to set it to be used by keras in each request. I.e. I have done the following:

from tensorflow.python.keras.backend import set_session
from tensorflow.python.keras.models import load_model

tf_config = some_custom_config
sess = tf.Session(config=tf_config)
graph = tf.get_default_graph()

# IMPORTANT: models have to be loaded AFTER SETTING THE SESSION for keras! 
# Otherwise, their weights will be unavailable in the threads after the session there has been set
set_session(sess)
model = load_model(...)

and then in each request (i.e. in each thread):

global sess
global graph
with graph.as_default():
    set_session(sess)
    model.predict(...)

You are amazing!!! This is the best solution. Can you tell me why did it just work after adding session?

Thank you and you are very welcome 😃.

As far as I understand, the problem is that tensorflow graphs and sessions are not thread safe. So by default a new session (which does not contain any previously loaded weights, models a.s.o.) is created for each thread, i.e. for each request. By saving the global session that contains all your models and setting it to be used by keras in each thread the problem is solved.

Thank you so much for this.

In my case I did it a bit differently, in case it helps anyone:

# on thread 1
session = tf.Session(graph=tf.Graph())
with session.graph.as_default():
    k.backend.set_session(session)
    model = k.models.load_model(filepath)

# on thread 2
with session.graph.as_default():
    k.backend.set_session(session)
    model.predict(x, **kwargs)

The novelty here is allowing for multiple models to be loaded (once) and used in multiple threads. By default, the “default” Session and the “default” Graph are used while loading a model. But here you create new ones. Also note the Graph is stored in the Session object, which is a bit more convenient.

Pls it’s showing some_custom_config not defined

I had the same issue in tensorflow 1.13.1 which I have resolved by creating a reference to the session that is used for loading the models and then to set it to be used by keras in each request. I.e. I have done the following:

from tensorflow.python.keras.backend import set_session
from tensorflow.python.keras.models import load_model

tf_config = some_custom_config
sess = tf.Session(config=tf_config)
graph = tf.get_default_graph()

# IMPORTANT: models have to be loaded AFTER SETTING THE SESSION for keras! 
# Otherwise, their weights will be unavailable in the threads after the session there has been set
set_session(sess)
model = load_model(...)

and then in each request (i.e. in each thread):

global sess
global graph
with graph.as_default():
    set_session(sess)
    model.predict(...)

You are amazing!!! This is the best solution. Can you tell me why did it just work after adding session?

Thank you and you are very welcome 😃.

As far as I understand, the problem is that tensorflow graphs and sessions are not thread safe. So by default a new session (which does not contain any previously loaded weights, models a.s.o.) is created for each thread, i.e. for each request. By saving the global session that contains all your models and setting it to be used by keras in each thread the problem is solved.

Pls it’s showing some_custom_config not defined

I had the same issue in tensorflow 1.13.1 which I have resolved by creating a reference to the session that is used for loading the models and then to set it to be used by keras in each request. I.e. I have done the following:

from tensorflow.python.keras.backend import set_session
from tensorflow.python.keras.models import load_model

tf_config = some_custom_config
sess = tf.Session(config=tf_config)
graph = tf.get_default_graph()

# IMPORTANT: models have to be loaded AFTER SETTING THE SESSION for keras! 
# Otherwise, their weights will be unavailable in the threads after the session there has been set
set_session(sess)
model = load_model(...)

and then in each request (i.e. in each thread):

global sess
global graph
with graph.as_default():
    set_session(sess)
    model.predict(...)

You are amazing!!! This is the best solution. Can you tell me why did it just work after adding session?

Thank you and you are very welcome 😃. As far as I understand, the problem is that tensorflow graphs and sessions are not thread safe. So by default a new session (which does not contain any previously loaded weights, models a.s.o.) is created for each thread, i.e. for each request. By saving the global session that contains all your models and setting it to be used by keras in each thread the problem is solved.

I have the same issue with tensor flow version 1.13.1, the above solution works for me.

This solution worked for me as well… Thanks @eliadl

sorry,I am also facing this issue. But your answer doesn’t fix my problem maybe it’s in different condition. In my problem, It is when I use VGG16 in keras and I want to use iterate funtion loss_value, grads_value = iterate([np.zeros((1, 150, 150, 3))]) But this similar question occured. FailedPreconditionError: Error while reading resource variable block1_conv1_12/bias from Container: localhost. This could mean that the variable was uninitialized. Not found: Resource localhost/block1_conv1_12/bias/N10tensorflow3VarE does not exist. [[{{node block1_conv1_12/BiasAdd/ReadVariableOp}}]] In https://github.com/JarvisUSTC/deep-learning-with-python-notebooks/blob/master/5.4-visualizing-what-convnets-learn.ipynb, the problem didn’t encounter.

from tensorflow.python.keras.backend import set_session
from tensorflow.python.keras.models import load_model
from tensorflow.python.keras.applications import VGG16

sess = tf.Session()
graph = tf.get_default_graph()

set_session(sess)

#model = load_model('vgg16_weights_tf_dim_ordering_tf_kernels_notop.h5')
model = VGG16(weights='imagenet',include_top=False)

layer_name = 'block3_conv1'
filter_index = 0

layer_output = model.get_layer(layer_name).output
loss = K.mean(layer_output[:, :, :, filter_index])
grads = K.gradients(loss, model.input)[0]
grads /= (K.sqrt(K.mean(K.square(grads))) + 1e-5)

import numpy as np
global sess
global graph
with graph.as_default():
    set_session(sess)
    iterate = K.function([model.input], [loss, grads])
    loss_value, grads_value = iterate([np.zeros((1, 150, 150, 3))])

I had the same issue in tensorflow 1.13.1 which I have resolved by creating a reference to the session that is used for loading the models and then to set it to be used by keras in each request. I.e. I have done the following:

from tensorflow.python.keras.backend import set_session
from tensorflow.python.keras.models import load_model

tf_config = some_custom_config
sess = tf.Session(config=tf_config)
graph = tf.get_default_graph()

# IMPORTANT: models have to be loaded AFTER SETTING THE SESSION for keras! 
# Otherwise, their weights will be unavailable in the threads after the session there has been set
set_session(sess)
model = load_model(...)

and then in each request (i.e. in each thread):

global sess
global graph
with graph.as_default():
    set_session(sess)
    model.predict(...)

You are amazing!!! This is the best solution. Can you tell me why did it just work after adding session?

Thank you and you are very welcome 😃. As far as I understand, the problem is that tensorflow graphs and sessions are not thread safe. So by default a new session (which does not contain any previously loaded weights, models a.s.o.) is created for each thread, i.e. for each request. By saving the global session that contains all your models and setting it to be used by keras in each thread the problem is solved.

Thank you so much for this.

In my case I did it a bit differently, in case it helps anyone:

# on thread 1
session = tf.Session(graph=tf.Graph())
with session.graph.as_default():
    k.backend.set_session(session)
    model = k.models.load_model(filepath)

# on thread 2
with session.graph.as_default():
    k.backend.set_session(session)
    model.predict(x, **kwargs)

The novelty here is allowing for multiple models to be loaded (once) and used in multiple threads. By default, the “default” Session and the “default” Graph are used while loading a model. But here you create new ones. Also note the Graph is stored in the Session object, which is a bit more convenient.

Thank you for this answer I am also facing this problem when using flask and multithreading, @eliadl Your solution worked for me.

Thanks 😃

I am having the same issues and was wondering what the value of some_custom_config was?

In case you want to configure your session (which I had to do), you can pass the config in this parameter. Else just leave it out.