[graph/manager] Extend gradient execution for clipping
authorParichay Kapoor <pk.kapoor@samsung.com>
Wed, 24 Nov 2021 12:15:28 +0000 (21:15 +0900)
committerJijoong Moon <jijoong.moon@samsung.com>
Wed, 1 Dec 2021 09:54:01 +0000 (18:54 +0900)
commit6c5bf3233d5934dd5b051e5895a7a3af98eebb11
treec1f597bd191b3b62ff37079338e5c68118b23cea
parentb0085cce2c1add9f71a751f0083fa023fe5cbec0
[graph/manager] Extend gradient execution for clipping

Extend the execution order for the gradients if it used for clipping by
global norm. The gradient is extended by the max execution order where
the norm of the gradients will be calculated and used to update and
apply the gradients.
This is done while requesting the gradient itself at the first time as
the layer properties are finalized by then.

Signed-off-by: Parichay Kapoor <pk.kapoor@samsung.com>
13 files changed:
nntrainer/graph/network_graph.cpp
nntrainer/graph/network_graph.h
nntrainer/layers/layer_context.h
nntrainer/layers/layer_node.cpp
nntrainer/layers/layer_node.h
nntrainer/tensor/manager.cpp
nntrainer/tensor/manager.h
nntrainer/tensor/tensor.h
nntrainer/tensor/tensor_wrap_specs.h
nntrainer/tensor/weight.cpp
nntrainer/tensor/weight.h
nntrainer/utils/node_exporter.cpp
nntrainer/utils/node_exporter.h