Summary:
See discussion in https://pytorch.slack.com/archives/CBHSWPNM7/p1628792389008600
Pull Request resolved: https://github.com/pytorch/pytorch/pull/63296
Reviewed By: cbalioglu
Differential Revision:
D30332042
Pulled By: mrshenli
fbshipit-source-id:
3a642fda2e106fd35b67709ed2adb60e408854c2
def get_rank(group=None):
"""
- Returns the rank of current process group
+ Returns the rank of the current process in the provided ``group`` or the
+ default group if none was provided.
Rank is a unique identifier assigned to each process within a distributed
process group. They are always consecutive integers ranging from 0 to