Derivatives of relu (#63027) (#63089)
authorMengeTM <34686199+MengeTM@users.noreply.github.com>
Thu, 26 Aug 2021 22:32:06 +0000 (15:32 -0700)
committerFacebook GitHub Bot <facebook-github-bot@users.noreply.github.com>
Thu, 26 Aug 2021 22:33:25 +0000 (15:33 -0700)
commit085278f8b141579c5d5481a8fb96c7dfa830b262
treef6d10757163f034565bbe82fda96446810446d93
parent7861dba7f697f91a19d7fa137a2e15799959e2ca
Derivatives of relu (#63027) (#63089)

Summary:
Optimization of relu and leaky_relu derivatives for reduction of VRAM needed for the backward-passes

Fixes https://github.com/pytorch/pytorch/issues/63027

Pull Request resolved: https://github.com/pytorch/pytorch/pull/63089

Reviewed By: iramazanli

Differential Revision: D30582049

Pulled By: albanD

fbshipit-source-id: a9481fe8c10cbfe2db485e28ce80cabfef501eb8
tools/autograd/derivatives.yaml