Fix issue re: DDP and create_graph=True (#63831)
authorRohan Varma <rvarm1@fb.com>
Thu, 26 Aug 2021 06:48:58 +0000 (23:48 -0700)
committerFacebook GitHub Bot <facebook-github-bot@users.noreply.github.com>
Thu, 26 Aug 2021 06:50:25 +0000 (23:50 -0700)
commita6f767ed3d66b4a01e5b2edead8491dfbca517e6
tree569083cc6f4e8421a215aa1a34529fa57c11a8a3
parent3b284ab0243d22bb831a1685f47061b9612e1cb3
Fix issue re: DDP and create_graph=True (#63831)

Summary:
Pull Request resolved: https://github.com/pytorch/pytorch/pull/63831

Closes https://github.com/pytorch/pytorch/issues/63812

`at::mul_out` is not supported when `grad` itself requires grad, which is useful for computing higher order derivatives.

In this case, fall back to a mul + copy instead of mul_out.
ghstack-source-id: 136614644

Test Plan: UT

Reviewed By: SciPioneer

Differential Revision: D30505573

fbshipit-source-id: 83532b6207b3d80116fcc4dff0e5520d73b3454f
torch/csrc/distributed/c10d/reducer.cpp
torch/testing/_internal/distributed/distributed_test.py