optionally zero infinite losses in CTCLoss (#16199)
authorThomas Viehmann <tv.code@beamnet.de>
Mon, 11 Feb 2019 20:26:47 +0000 (12:26 -0800)
committerFacebook Github Bot <facebook-github-bot@users.noreply.github.com>
Mon, 11 Feb 2019 21:12:55 +0000 (13:12 -0800)
commit29f096cc70cf9cc1a317ae7107228215b7dde60b
tree84f71bf39872e5a896fddc2ab5424eccda25fc9b
parent632df48207f0ff1fb7c34753f2638a90a3ff92fd
optionally zero infinite losses in CTCLoss (#16199)

Summary:
Here is a stab at implementing an option to zero out infinite losses (and NaN gradients).
It might be nicer to move the zeroing to the respective kernels.
The default is currently `False` to mimic the old behaviour, but I'd be half inclined to set the default to `True`, because the behaviour wasn't consistent between CuDNN and Native anyways and the NaN gradients aren't terribly useful.

This topic seems to come up regularly, e.g. in  #14335
Pull Request resolved: https://github.com/pytorch/pytorch/pull/16199

Differential Revision: D14020462

Pulled By: ezyang

fbshipit-source-id: 5ba8936c66ec6e61530aaf01175dc49f389ae428
aten/src/ATen/native/LossCTC.cpp
aten/src/ATen/native/cuda/LossCTC.cu
aten/src/ATen/native/cudnn/LossCTC.cpp
aten/src/ATen/native/native_functions.yaml
test/test_nn.py
tools/autograd/derivatives.yaml
torch/nn/functional.py
torch/nn/modules/loss.py