Reenable all forward-pass fusions that worked before the AD fix (#14558)
authorAdam Paszke <adam.paszke@gmail.com>
Tue, 4 Dec 2018 23:40:41 +0000 (15:40 -0800)
committerFacebook Github Bot <facebook-github-bot@users.noreply.github.com>
Tue, 4 Dec 2018 23:43:37 +0000 (15:43 -0800)
commitd76fd43294693be5a5a38251051345f2150d421c
tree16cfd85f78394fabac144d13927b33be4c1b39a2
parentc3bfa0e52bff13984a20156b1c13358bcd9ac395
Reenable all forward-pass fusions that worked before the AD fix (#14558)

Summary:
Dealing with so many `aten::size` calls (in particular calls on elements computed inside fusion groups) requires us to do some extra graph processing in the fuser (to compute the sizes by explicit broadcasts, instead of writing the intermediate tensors only to check their size). This restores the forward expects of LSTM and MiLSTM to a single big kernel. Unfortunately the backward is much harder, because as long as we can't prove that the reductions are unnecessary (or if we can't distribute them over the op), we will not be able to fuse them.
Pull Request resolved: https://github.com/pytorch/pytorch/pull/14558

Differential Revision: D13321748

Pulled By: zou3519

fbshipit-source-id: c04fc2f70d106d2bfb56206b5aec517a93b79d1f
aten/src/ATen/core/interned_strings.h
test/expect/TestScript.test_lstm_fusion_cuda-forward.expect
test/expect/TestScript.test_milstm_fusion_cuda-forward.expect
test/test_jit.py
torch/csrc/jit/fuser/executor.cpp
torch/csrc/jit/passes/alias_analysis.cpp
torch/csrc/jit/passes/graph_fuser.cpp
torch/csrc/jit/passes/python_print.cpp
torch/csrc/jit/register_prim_ops.cpp