The derivative of softmax has been hand crafted to be different from others
Refer to https://github.com/nnstreamer/nntrainer/blob/
2a650512813db6ce3bba828b5790066fbc655f14/nntrainer/src/fc_layer.cpp#L265 for original implementation
Softmax requires softmax(x) as input for derivative while other activations require x as input for derivative_
Signed-off-by: Parichay Kapoor <pk.kapoor@samsung.com>
}
Tensor ActivationLayer::backwarding(Tensor derivative, int iteration) {
- return derivative.multiply(_act_prime_fn(hidden));
+ if (activation_type == ActiType::ACT_SOFTMAX)
+ return derivative.multiply(_act_prime_fn(hidden));
+ else
+ return derivative.multiply(_act_prime_fn(input));
}
/**