tensorflow: Why the loss doesn't decrease in my CNN model?

Hi All, I am new to tensorflow, and I find the losses doesn’t decrease when train a CNN model on Cifar100 dataset.

Here is the Model code:

network = Sequential([
    layers.Conv2D(64, kernel_size=[3, 3], padding="same", activation=tf.nn.relu), # 32x32
    layers.Conv2D(64, kernel_size=[3, 3], padding="same", activation=tf.nn.relu),
    layers.MaxPooling2D((2, 2)), # 16x16

    layers.Conv2D(128, kernel_size=[3, 3], padding="same", activation=tf.nn.relu),
    layers.Conv2D(128, kernel_size=[3, 3], padding="same", activation=tf.nn.relu),
    layers.MaxPooling2D((2, 2)), # 8x8

    layers.Conv2D(256, kernel_size=[3, 3], padding="same", activation=tf.nn.relu),
    layers.Conv2D(256, kernel_size=[3, 3], padding="same", activation=tf.nn.relu),
    layers.MaxPooling2D((2, 2)), # 4x4

    layers.Conv2D(512, kernel_size=[3, 3], padding="same", activation=tf.nn.relu),
    layers.Conv2D(512, kernel_size=[3, 3], padding="same", activation=tf.nn.relu),
    layers.MaxPooling2D((2, 2)), # 2x2

    layers.Conv2D(512, kernel_size=[3, 3], padding="same", activation=tf.nn.relu),
    layers.Conv2D(512, kernel_size=[3, 3], padding="same", activation=tf.nn.relu),
    layers.MaxPooling2D((2, 2)), # 1x1

    layers.Flatten(),

    layers.Dense(256, activation=tf.nn.relu),
    layers.Dense(128, activation=tf.nn.relu),
    layers.Dense(100, activation=None),
])

and this is the link for the entire code: https://colab.research.google.com/drive/1kIT_18Z-ymEgzqgG4nh4sZdBkA1bnJdO

and, It will be OK if I use only one Conv2D layer with maxpooling, like this:

network = Sequential([
    layers.Conv2D(64, kernel_size=[3, 3], padding="same", activation=tf.nn.relu), # 32x32
    layers.MaxPooling2D((2, 2)), # 16x16

    layers.Conv2D(128, kernel_size=[3, 3], padding="same", activation=tf.nn.relu),
    layers.MaxPooling2D((2, 2)), # 8x8

    layers.Conv2D(256, kernel_size=[3, 3], padding="same", activation=tf.nn.relu),
    layers.MaxPooling2D((2, 2)), # 4x4

    layers.Conv2D(512, kernel_size=[3, 3], padding="same", activation=tf.nn.relu),
    layers.MaxPooling2D((2, 2)), # 2x2

    layers.Conv2D(512, kernel_size=[3, 3], padding="same", activation=tf.nn.relu),
    layers.MaxPooling2D((2, 2)), # 1x1

   #xxxxx
])

thanks~

About this issue

  • Original URL
  • State: closed
  • Created 4 years ago
  • Comments: 16 (3 by maintainers)

Most upvoted comments

@lihongxun945, Please refer this Stack Overflow Answer which elaborates on how to decrease the loss and improve your accuracy, and also shows how to use the functions like Dropout, Early Stopping, Regularization, Image Data Generator, etc…

Yes forgot to mention dropouts really worked much better.