keras: Bug with BatchNormalization(axis=1): ValueError: Shape must be rank 1 but is rank 4 for 'batch_normalization_1/cond/FusedBatchNorm'

Current master version of keras (commit b3cb261b22a73d195a527592b49ca57e8c9ac9f5), TensorFlow 1.8.0

BatchNormalization(axis=1) for 'channels_first' seems to fail.

import os
os.environ['KERAS_BACKEND'] = 'tensorflow'
import keras.backend as K
from keras.layers import Activation, Conv2D, Input
from keras.layers.normalization import BatchNormalization

# declare network model with channels first: ERROR
K.set_image_data_format('channels_first')
input = Input(shape=(3, 1001, 1001), dtype='float32')
x = Conv2D(filters=64, kernel_size=(3, 3), strides=1, padding='same')(input)
x = BatchNormalization(axis=1)(x)
x = Activation('relu')(x)

gives the error

Traceback (most recent call last):
  File "/home/rcasero/.conda/envs/cytometer_tensorflow/lib/python3.6/site-packages/tensorflow/python/framework/ops.py", line 1567, in _create_c_op
    c_op = c_api.TF_FinishOperation(op_desc)
tensorflow.python.framework.errors_impl.InvalidArgumentError: Shape must be rank 1 but is rank 4 for 'batch_normalization_1/cond/FusedBatchNorm' (op: 'FusedBatchNorm') with input shapes: [?,64,1001,1001], [1,64,1,1], [1,64,1,1], [1,64,1,1], [1,64,1,1].
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
  File "<input>", line 11, in <module>
  File "/home/rcasero/.conda/envs/cytometer_tensorflow/lib/python3.6/site-packages/keras/engine/base_layer.py", line 459, in __call__
    output = self.call(inputs, **kwargs)
  File "/home/rcasero/.conda/envs/cytometer_tensorflow/lib/python3.6/site-packages/keras/layers/normalization.py", line 204, in call
    training=training)
  File "/home/rcasero/.conda/envs/cytometer_tensorflow/lib/python3.6/site-packages/keras/backend/tensorflow_backend.py", line 3069, in in_train_phase
    x = switch(training, x, alt)
  File "/home/rcasero/.conda/envs/cytometer_tensorflow/lib/python3.6/site-packages/keras/backend/tensorflow_backend.py", line 3004, in switch
    else_expression_fn)
  File "/home/rcasero/.conda/envs/cytometer_tensorflow/lib/python3.6/site-packages/tensorflow/python/util/deprecation.py", line 432, in new_func
    return func(*args, **kwargs)
  File "/home/rcasero/.conda/envs/cytometer_tensorflow/lib/python3.6/site-packages/tensorflow/python/ops/control_flow_ops.py", line 2072, in cond
    orig_res_f, res_f = context_f.BuildCondBranch(false_fn)
  File "/home/rcasero/.conda/envs/cytometer_tensorflow/lib/python3.6/site-packages/tensorflow/python/ops/control_flow_ops.py", line 1913, in BuildCondBranch
    original_result = fn()
  File "/home/rcasero/.conda/envs/cytometer_tensorflow/lib/python3.6/site-packages/keras/layers/normalization.py", line 165, in normalize_inference
    epsilon=self.epsilon)
  File "/home/rcasero/.conda/envs/cytometer_tensorflow/lib/python3.6/site-packages/keras/backend/tensorflow_backend.py", line 1894, in batch_normalization
    is_training=False
  File "/home/rcasero/.conda/envs/cytometer_tensorflow/lib/python3.6/site-packages/tensorflow/python/ops/nn_impl.py", line 904, in fused_batch_norm
    name=name)
  File "/home/rcasero/.conda/envs/cytometer_tensorflow/lib/python3.6/site-packages/tensorflow/python/ops/gen_nn_ops.py", line 3429, in _fused_batch_norm
    is_training=is_training, name=name)
  File "/home/rcasero/.conda/envs/cytometer_tensorflow/lib/python3.6/site-packages/tensorflow/python/framework/op_def_library.py", line 787, in _apply_op_helper
    op_def=op_def)
  File "/home/rcasero/.conda/envs/cytometer_tensorflow/lib/python3.6/site-packages/tensorflow/python/framework/ops.py", line 3392, in create_op
    op_def=op_def)
  File "/home/rcasero/.conda/envs/cytometer_tensorflow/lib/python3.6/site-packages/tensorflow/python/framework/ops.py", line 1734, in __init__
    control_input_ops)
  File "/home/rcasero/.conda/envs/cytometer_tensorflow/lib/python3.6/site-packages/tensorflow/python/framework/ops.py", line 1570, in _create_c_op
    raise ValueError(str(e))
ValueError: Shape must be rank 1 but is rank 4 for 'batch_normalization_1/cond/FusedBatchNorm' (op: 'FusedBatchNorm') with input shapes: [?,64,1001,1001], [1,64,1,1], [1,64,1,1], [1,64,1,1], [1,64,1,1].

Meanwhile, BatchNormalization(axis=3) for 'channels_last' works.

import os
os.environ['KERAS_BACKEND'] = 'tensorflow'
import keras.backend as K
from keras.layers import Activation, Conv2D, Input
from keras.layers.normalization import BatchNormalization

# declare network model with channels last: NO ERROR
K.set_image_data_format('channels_last')
input = Input(shape=(1001, 1001, 3), dtype='float32')
x = Conv2D(filters=64, kernel_size=(3, 3), strides=1, padding='same')(input)
x = BatchNormalization(axis=3)(x)
x = Activation('relu')(x)

doesn’t give any error.

About this issue

Original URL
State: closed
Created 6 years ago
Reactions: 2
Comments: 43 (1 by maintainers)

Most upvoted comments

please change tensorflow_backend in keras by yourself as suggested by @see-- In keras version - 2.2.4 -> backend --> tesnorflow_backend.py : change “()” to “[ ]” in line no 1908,1910,1914, 1918.

+35

gurjar112 on Jun 18, 2019

@ymodak still not working with tf 1.12 and keras 2.2.4

+15

yaarsh on Nov 22, 2018

Fixed in e3a2f7d29f2f1c21ecc978bd0038b1d1330d33c2 @yaarsh The fix is not part of 2.2.4 (from pip). You can just install the latest keras master.

+10

see-- on Nov 22, 2018

@ymodak Did not help. Could you show the commit?

import tensorflow as tf; print(tf.__version__)
# 1.11.0-rc1
import keras; print(keras.__version__)
# 2.2.4

from keras import backend as K
from keras.layers import Input, Conv2D, BatchNormalization
K.set_image_data_format('channels_first')
input = Input(shape=(3, 1001, 1001), dtype='float32')
x = Conv2D(filters=64, kernel_size=(3, 3), strides=1, padding='same')(input)
x = BatchNormalization(axis=1)(x)

ValueError: Shape must be rank 1 but is rank 0 for ‘batch_normalization_1/cond/Reshape_4’ (op: ‘Reshape’) with input shapes: [1,64,1,1], [].

see-- on Nov 22, 2018

@kazemSafari just change your tensorflow_backend.py

jacmule on Jan 18, 2019

@rcasero For BatchNormalization layer, if you set axis = 1, the internal implementation would broadcast mean/var/beta/gama tensors to 4 dimension, but tf.nn.fused_batch_norm only accept them as 1 dimension tensor, that leads to the exception. I have sent #10684 to fix it, wish it can help you as well.

yanboliang on Jul 15, 2018

@rcasero @see-- @yaarsh @ymodak @davideboschetto @yanboliang @damnko thank you for your comments. how can i install keras master?

pipenv install -e git+https://github.com/keras-team/keras.git@master#egg=keras

rjcortese on Jun 19, 2019

@kazemSafari just change your tensorflow_backend.py

How can I change the tensorflow_backend.py in Jupyter note book(anaconda) ?

I am getting this error with 2.2.4 (keras) and tensorflow 1.13.1 ,can anyone @iperov @see-- please help?

vatsaldin on Jun 11, 2019

This one works correctly:

Name: Keras Version: 2.1.2 Summary: Deep Learning for Python Home-page: https://github.com/fchollet/keras Author: Francois Chollet Author-email: francois.chollet@gmail.com License: MIT Location: c:\users\steve\miniconda3\lib\site-packages Requires: six, numpy, pyyaml, scipy Required-by:

stiv-yakovenko on May 14, 2019

Is this still investigated? Do I really need to go back to keras 2.1.6 to solve this?

davideboschetto on Oct 31, 2018