When a mirrored variable is fetched in cross-tower mode, fetch its primary variable.
authorIgor Saprykin <isaprykin@google.com>
Mon, 30 Apr 2018 19:41:12 +0000 (12:41 -0700)
committerTensorFlower Gardener <gardener@tensorflow.org>
Mon, 30 Apr 2018 19:43:36 +0000 (12:43 -0700)
commit8609ef4db1a2af0da0c2c20b26756031637de3ff
tree93b6e682c9eb03135ac507a43eeaeec0f7153076
parent9d79acc6aae306e0444c193e945f0c87fe5bb509
When a mirrored variable is fetched in cross-tower mode, fetch its primary variable.

This prevents errors like
ValueError: Fetch argument MirroredVariable({'/job:localhost/replica:0/task:0/device:GPU:0': <tf.Variable 'global_step:0' shape=() dtype=int64>, '/job:localhost/replica:0/task:0/device:GPU:1': <tf.Variable 'global_step/replica_1:0' shape=() dtype=int64>}) cannot be interpreted as a Tensor. (Device /job:localhost/replica:0/task:0/device:CPU:0 not found in ['/job:localhost/replica:0/task:0/device:GPU:0', '/job:localhost/replica:0/task:0/device:GPU:1'] (current device ))

I ran distribute/examples/resnet with and without the change and it fixed the problem.

PiperOrigin-RevId: 194828672
tensorflow/contrib/distribute/python/values.py
tensorflow/contrib/distribute/python/values_test.py