tensorflow: Why the loss doesn't decrease in my CNN model?
Hi All, I am new to tensorflow, and I find the losses doesn’t decrease when train a CNN model on Cifar100 dataset.
Here is the Model code:
network = Sequential([
layers.Conv2D(64, kernel_size=[3, 3], padding="same", activation=tf.nn.relu), # 32x32
layers.Conv2D(64, kernel_size=[3, 3], padding="same", activation=tf.nn.relu),
layers.MaxPooling2D((2, 2)), # 16x16
layers.Conv2D(128, kernel_size=[3, 3], padding="same", activation=tf.nn.relu),
layers.Conv2D(128, kernel_size=[3, 3], padding="same", activation=tf.nn.relu),
layers.MaxPooling2D((2, 2)), # 8x8
layers.Conv2D(256, kernel_size=[3, 3], padding="same", activation=tf.nn.relu),
layers.Conv2D(256, kernel_size=[3, 3], padding="same", activation=tf.nn.relu),
layers.MaxPooling2D((2, 2)), # 4x4
layers.Conv2D(512, kernel_size=[3, 3], padding="same", activation=tf.nn.relu),
layers.Conv2D(512, kernel_size=[3, 3], padding="same", activation=tf.nn.relu),
layers.MaxPooling2D((2, 2)), # 2x2
layers.Conv2D(512, kernel_size=[3, 3], padding="same", activation=tf.nn.relu),
layers.Conv2D(512, kernel_size=[3, 3], padding="same", activation=tf.nn.relu),
layers.MaxPooling2D((2, 2)), # 1x1
layers.Flatten(),
layers.Dense(256, activation=tf.nn.relu),
layers.Dense(128, activation=tf.nn.relu),
layers.Dense(100, activation=None),
])
and this is the link for the entire code: https://colab.research.google.com/drive/1kIT_18Z-ymEgzqgG4nh4sZdBkA1bnJdO
and, It will be OK if I use only one Conv2D layer with maxpooling, like this:
network = Sequential([
layers.Conv2D(64, kernel_size=[3, 3], padding="same", activation=tf.nn.relu), # 32x32
layers.MaxPooling2D((2, 2)), # 16x16
layers.Conv2D(128, kernel_size=[3, 3], padding="same", activation=tf.nn.relu),
layers.MaxPooling2D((2, 2)), # 8x8
layers.Conv2D(256, kernel_size=[3, 3], padding="same", activation=tf.nn.relu),
layers.MaxPooling2D((2, 2)), # 4x4
layers.Conv2D(512, kernel_size=[3, 3], padding="same", activation=tf.nn.relu),
layers.MaxPooling2D((2, 2)), # 2x2
layers.Conv2D(512, kernel_size=[3, 3], padding="same", activation=tf.nn.relu),
layers.MaxPooling2D((2, 2)), # 1x1
#xxxxx
])
thanks~
About this issue
- Original URL
- State: closed
- Created 4 years ago
- Comments: 16 (3 by maintainers)
@lihongxun945, Please refer this Stack Overflow Answer which elaborates on how to decrease the loss and improve your accuracy, and also shows how to use the functions like Dropout, Early Stopping, Regularization, Image Data Generator, etc…
Yes forgot to mention dropouts really worked much better.