From: Yi Wang <wayi@fb.com>
Date: Tue, 14 Sep 2021 16:41:13 +0000 (-0700)
Subject: [Model Averaging] Revert #63895 (#64903)
X-Git-Tag: accepted/tizen/8.0/unified/20231005.095509~225
X-Git-Url: http://review.tizen.org/git/?a=commitdiff_plain;h=00e6e0c59395ba2cb6f90fae077108e69e6a7ea9;p=platform%2Fupstream%2Fpytorch.git

[Model Averaging] Revert #63895 (#64903)

Summary:
Pull Request resolved: https://github.com/pytorch/pytorch/pull/64903

Fix the accuracy regression caused by https://github.com/pytorch/pytorch/pull/63895.

Test Plan:
buck test mode/dev-nosan //caffe2/test/distributed:distributed_nccl_spawn -- test_periodic_model_averager
buck test mode/dev-nosan //caffe2/test/distributed:distributed_nccl_spawn -- test_post_localSGD_optimizer_parity

Reviewed By: rohan-varma

Differential Revision: D30894688

fbshipit-source-id: fe00b8b23b860d9f806f87c1b6caba1d0b807485
---

diff --git a/torch/distributed/algorithms/model_averaging/utils.py b/torch/distributed/algorithms/model_averaging/utils.py
index a2bbac2..1e48e54 100644
--- a/torch/distributed/algorithms/model_averaging/utils.py
+++ b/torch/distributed/algorithms/model_averaging/utils.py
@@ -32,6 +32,5 @@ def average_parameters(
 
     offset = 0
     for p in params_it2:
-        with torch.no_grad():
-            p.set_(flat_params[offset : offset + p.numel()].view_as(p).type_as(p))  # type: ignore[call-overload]
+        p.data = flat_params[offset : offset + p.numel()].view_as(p)
         offset += p.numel()