Fix the issue with Bahdanau attention when normalized=True and dtype = float16/32

While revisiting 18016 I noticed that Bahdanau attention has a similiar
dtype mismatch issue when normalized=True. The issue comes from:
```
     g = variable_scope.get_variable(
         "attention_g", dtype=dtype,
         initializer=math.sqrt((1. / num_units)))
```
where the initializer value does not work well with differnt dtype.

This fix converts changes the initializer to `init_ops.constant_initializer`
to address the issue, and adds additional test cases for it.

Signed-off-by: Yong Tang <yong.tang.github@outlook.com>

diff --git a/tensorflow/contrib/seq2seq/python/ops/attention_wrapper.py b/tensorflow/contrib/seq2seq/python/ops/attention_wrapper.py
index 9ba541c..867e49b 100644 (file)
--- a/tensorflow/contrib/seq2seq/python/ops/attention_wrapper.py
+++ b/tensorflow/contrib/seq2seq/python/ops/attention_wrapper.py
@@ -472,7 +472,7 @@ def _bahdanau_score(processed_query, keys, normalize):
     # Scalar used in weight normalization
     g = variable_scope.get_variable(
         "attention_g", dtype=dtype,
-        initializer=math.sqrt((1. / num_units)))
+        initializer=init_ops.constant_initializer(math.sqrt((1. / num_units))), shape=())
     # Bias added prior to the nonlinearity
     b = variable_scope.get_variable(
         "attention_b", [num_units], dtype=dtype,