- Default values of cov and inv variables are now 0. Zero-debiasing (as in Adam...
authorJames Martens <jamesmartens@google.com>
Thu, 26 Apr 2018 22:13:48 +0000 (15:13 -0700)
committerTensorFlower Gardener <gardener@tensorflow.org>
Thu, 26 Apr 2018 22:16:46 +0000 (15:16 -0700)
commitc9be1f2b19972e0b10e8c96e24b3dc3aa05ea651
tree20c766f837519ad7f80a6e81ba397f1f421e2390
parent2ce60cd2ebe835c7dea9df990b70218e418238b6
- Default values of cov and inv variables are now 0.  Zero-debiasing (as in Adam) is used for the cov matrices.  Note this this requires that cov variables, then inv variables, are all updated before the first training update is made.  All examples have been modified to do this. NOTE: you *may* have to increase the damping value you use at the start of optimization after this change (or throughout, if you are using a constant value).
- Changed the initial default approximation used for generic registrations to "diagonal"
- Convenience properties for ops and thunks have all been removed, along with "make_ops_and_vars". User should only interface with "make_vars_and_create_op_thunks" (or maybe "create_ops_and_vars_thunks").

PiperOrigin-RevId: 194461623
13 files changed:
tensorflow/contrib/kfac/examples/convnet.py
tensorflow/contrib/kfac/examples/mlp.py
tensorflow/contrib/kfac/examples/tests/convnet_test.py
tensorflow/contrib/kfac/python/kernel_tests/BUILD
tensorflow/contrib/kfac/python/kernel_tests/estimator_test.py
tensorflow/contrib/kfac/python/kernel_tests/fisher_blocks_test.py
tensorflow/contrib/kfac/python/kernel_tests/fisher_factors_test.py
tensorflow/contrib/kfac/python/kernel_tests/optimizer_test.py
tensorflow/contrib/kfac/python/ops/estimator.py
tensorflow/contrib/kfac/python/ops/fisher_factors.py
tensorflow/contrib/kfac/python/ops/layer_collection.py
tensorflow/contrib/kfac/python/ops/optimizer.py
tensorflow/contrib/kfac/python/ops/placement.py