How To Build A Multi-class Convolutional Neural Network With Keras
Solution 1:
Thank you @Daniel, your suggestions helped me in the end to get the Unet to work. I managed to get results that did not just classify the whole image as background when running 500+ epochs. Also, instead of using kernel_initializer='he_normal', kernel_initializer='zeros'or kernel_initializer=TruncatedNormal(mean=0.0, stddev=0.07) worked for me. I used 'sigmoid' activation function and loss='binary_crossentropy'. I kept the 'relu' activation for all the hidden convolutional layers. I noticed that my network will sometimes be stuck in a local minimum where the loss does not improve anymore, so I need to restart.
Solution 2:
In my experience, also with a U-net for segmentation. It tends to do this:
- Go to totally black or totally white
- After a lot of time in which the loss seems to be frozen, it finds it's way.
I also use the "train just one image" method to find that convergence, then adding the other images is ok.
But I had to try a lot of times, and the only time it worked pretty fast was when I used:
- final activation = 'sigmoid'
- loss = 'binary_crossentropy'
But I wasn't using "relu" anywhere...perhaps that influences a little the convergence speed...? Thinking about "relu", which has only 0 or positive results, there is a big region in this function that does not have a gradient. Maybe having lots of "relu" activations creates a lot of "flat" areas without gradients? (Must think better about it to confirm)
Try a few times (and have patience to wait for many many epochs) with different weight initializations.
There is a chance that your learning rate is too big too.
About to_categorical(): have you tried to plot/print your masks? Do they really seem like what you expect them to?
Solution 3:
I don't see your prediction layer which as far as I know must be a dense layer and not a convolutional layer. Maybe that's your problem.
Post a Comment for "How To Build A Multi-class Convolutional Neural Network With Keras"