Tanh vs Sigmoid: Why Use Tanh Over Sigmoid in Neural Networks?

When using tanh as an activation function, sigmoid is typically not used. This is because tanh has an output range of [-1, 1] while sigmoid has an output range of [0, 1]. This means that tanh can produce better gradient flow as it is centered around zero, which reduces the risk of vanishing gradients. Sigmoid has small gradients near its center, which can lead to vanishing gradients during training. Therefore, tanh is generally considered a better alternative to sigmoid.