#Categorical cross entropy how to#

However, doing that allows us to compare the model in terms of its performance - to actually see whether sparse categorical crossentropy does as good a job as the regular one. It'll be a simple one - an extension of a CNN that we created before, with the MNIST dataset.

#Categorical cross entropy how to#

In this blog, we'll figure out how to build a convolutional neural network with sparse categorical crossentropy loss. When applied, you can start using categorical crossentropy.īut did you know that there exists another type of loss - sparse categorical crossentropy - with which you can leave the integers as they are, yet benefit from crossentropy loss? I didn't when I just started with Keras, simply because pretty much every article I read performs one-hot encoding before applying regular categorical crossentropy loss. In Keras, this can be done with to_categorical, which essentially applies one-hot encoding to your training set's targets. This means that you'll have to convert these targets first. Rather, you likely have feature vectors with integer targets - such as 0 to 9 for the numbers 0 to 9. Often, this is not what your dataset looks like when you'll start creating your models. However, traditional categorical crossentropy requires that your data is one-hot encoded and hence converted into categorical format.

For multiclass classification problems, many online tutorials - and even François Chollet's book Deep Learning with Python, which I think is one of the most intuitive books on deep learning with Keras - use categorical crossentropy for computing the loss value of your neural network.