Move weight offload inside backend construction functor (#18385)
authorYinghai Lu <yinghai@fb.com>
Wed, 27 Mar 2019 03:57:18 +0000 (20:57 -0700)
committerFacebook Github Bot <facebook-github-bot@users.noreply.github.com>
Wed, 27 Mar 2019 04:03:17 +0000 (21:03 -0700)
commitf3ddc40ca44e35d42b39cdb9a72387d9d22127aa
tree5276920d785763a78097e63e663eb442c1a4b35c
parent60538c8366587d9b8b08226c953c98a0013b833f
Move weight offload inside backend construction functor (#18385)

Summary:
Pull Request resolved: https://github.com/pytorch/pytorch/pull/18385

By moving the weight offload into the backend initialization function, we can instantiate the backend once by creating the OnnxifiOp once and then clean up the parameter workspace. And we need to keep hold of that instantiated net (OnnxifiOp) without cleaning it. Subsequent ctor of OnnxifiOp of the same model will hit the cached backend and they will not look into weight offloading, which is safe as the weight is already gone.

Reviewed By: ipiszy

Differential Revision: D14590379

fbshipit-source-id: f7f34016e09777ad3df0af487885cd14658e1044
caffe2/operators/onnxifi_op.cc
caffe2/operators/onnxifi_op.h