keras: Slow BatchNormalization layers (CPU tested)

Hi,

I added BatchNormalization layers to my model and it suddenly took much more time to train.

It takes 561 seconds for one epoch with them: Epoch 1/1 4096/4096 [==============================] - 561s - loss: 0.0946 - acc: 0.9006

This is if I comment out the BatchNorm layers (186 seconds): Epoch 1/1 4096/4096 [==============================] - 186s - loss: 4.5043 - acc: 0.5933

I wrote on the user group and someone informed me that adding Dropout slows it down further, but even without Dropout this is much slower (above 500 seconds too).

Is it a CPU-related issue? Or is such a slowdown expected? Below is my model.

    model = Sequential()
    model.add(Convolution2D(32, 11, 11, subsample=(4,4),input_shape=(3,227,227)))
    model.add(PReLU())
    model.add(BatchNormalization())

    model.add(Convolution2D(64, 5, 5, subsample=(2,2)))
    model.add(PReLU())
    model.add(BatchNormalization())

    model.add(Convolution2D(64, 3, 3))
    model.add(PReLU())
    #model.add(BatchNormalization())
    model.add(MaxPooling2D((2,2), strides=(2,2)))

    model.add(Flatten())
    model.add(BatchNormalization())
    model.add(Dense(400))
    model.add(PReLU())
    model.add(BatchNormalization())
    model.add(Dropout(0.25))

    model.add(Dense(400))
    model.add(PReLU())
    model.add(BatchNormalization())
    model.add(Dropout(0.25))

    model.add(Dense(3, activation='softmax'))
    optimizer = keras.optimizers.RMSprop(lr=0.0005, rho=0.9, epsilon=1e-6)

    model.compile(loss='categorical_crossentropy',optimizer=optimizer)