Softmax

y_i = e^(x_i) / sum over j ^(x_j)
Easy way to get positive numbers, also called a probability distribution
John bridle, recommends calling it softargmax
Used a lot for classification
- They look like probabilities
- Used in the last layer for classification
As score gets to zero, log becomes too big
- Better off using the log of the value as the module directly to avoid numerical instability Why
This is the sigmoid function with inputs x and 1!