Barrier synchronizes with prior work before completing (#14386)
authorPieter Noordhuis <pietern@fb.com>
Tue, 27 Nov 2018 18:41:06 +0000 (10:41 -0800)
committerFacebook Github Bot <facebook-github-bot@users.noreply.github.com>
Tue, 27 Nov 2018 18:46:42 +0000 (10:46 -0800)
commit2cc35c161ada1fb5e27b7ed9472c21a00c981d89
tree4b5977a1c41ddb669f5da714537fc4e34f0c1be9
parent9598d380b0b9a100032e63dfeabcf55e98b4872e
Barrier synchronizes with prior work before completing (#14386)

Summary:
Pull Request resolved: https://github.com/pytorch/pytorch/pull/14386

See #13573, #14142, and #14271 for discussion.

This change updates ProcessGroupGloo to ensure that all prior
operations have completed before executing the barrier.

Reviewed By: manojkris

Differential Revision: D13205022

fbshipit-source-id: 673e7e6ca357dc843874d6dd8da590832e1de7fa
test/test_c10d.py
torch/lib/c10d/ProcessGroupGloo.cpp
torch/lib/c10d/ProcessGroupGloo.hpp