[Example] Concat example

This patch adds concat example with embedding

The structure is

      (input)
     /       \
  (embed1)   (embed2)
    \         /
      (concat)
         |
      (flatten)
         |
        (fc)

**Self evaluation:**
1. Build test: [X]Passed [ ]Failed [ ]Skipped
2. Run test: [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: Jihoon Lee <jhoon.it.lee@samsung.com>

diff --git a/Applications/Embedding/README.md b/Applications/Embedding/README.md
new file mode 100644 (file)
index 0000000..c4a52a3
--- /dev/null
+++ b/Applications/Embedding/README.md
@@ -0,0 +1,41 @@
+# Embedding
+
+This application contains embedding layer example with two example model.
+One with single input, another assuming there are two inputs(using split layer)
+
+## Example Model Structure
+
+There are two structure ready for the example
+
+1. Linear Embedding structure
+ (Input:           10:1:1:4 )
+ (Embedding:       10:1:4:8 )
+ (Flatten:         10:1:1:32)
+ (FullyConnected:  10:1:1:1 )
+
+2. Splitted data and seperate embedding structure
+ (Input:           10:1:2:2 )
+ (split:           10:1:1:1,                   10:1:1:1 )
+ (Embedding1:      10:1:1:8 ), (Embedding2:    10:1:1:8 )
+ (concat:          10:2:1:8 ),
+ (Flatten:         10:1:1:16),
+ (FullyConnected:  10:1:1:1 )
+
+## How to run a train epoch
+
+Once you compile, with `meson`, please file an issue if you have a problem running the example.
+
+```bash
+export ${res}
+$ cd ${build_dir}
+$ meson test app_embedding -v #this is for the first model structure
+$ meson test app_embedding_split -v #this is for the second model structure
+```
+
+Expected output is as below...
+
+```bash
+#1/100 - Training Loss: 0.692463 >> [ Accuracy: 100% - Validation Loss : 0.690833 ]
+...
+#100/100 - Training Loss: 0.535373 >> [ Accuracy: 100% - Validation Loss : 0.53367 ]
+```

diff --git a/Applications/Embedding/jni/main.cpp b/Applications/Embedding/jni/main.cpp
index 7931240..04c8ace 100644 (file)
--- a/Applications/Embedding/jni/main.cpp
+++ b/Applications/Embedding/jni/main.cpp
@@ -183,14 +183,6 @@ int main(int argc, char *argv[]) {
    */
   std::vector<std::vector<float>> inputVector, outputVector;
   nntrainer::NeuralNetwork NN;
-  std::shared_ptr<ml::train::Layer> layer;
-  std::shared_ptr<ml::train::Layer> layer_fc;
-  std::shared_ptr<nntrainer::Layer> layer_;
-  std::shared_ptr<nntrainer::Layer> layer_fc_;
-  std::string name = "embedding";
-  std::string fc_name = "outputlayer";
-  nntrainer::Tensor weight;
-  nntrainer::Tensor weight_fc;
   /**
    * @brief     Initialize NN with configuration file path
    */
@@ -216,15 +208,6 @@ int main(int argc, char *argv[]) {
 
     std::cout << "Input dimension: " << NN.getInputDimension()[0];
 
-    NN.getLayer(name.c_str(), &layer);
-    NN.getLayer(fc_name.c_str(), &layer_fc);
-
-    layer_ = nntrainer::getLayerDevel(layer);
-    layer_fc_ = nntrainer::getLayerDevel(layer_fc);
-
-    weight = layer_->getWeights()[0].getVariable();
-    weight.print(std::cout);
-
   } catch (...) {
     std::cerr << "Unexpected Error during init" << std::endl;
     return 1;
@@ -239,9 +222,6 @@ int main(int argc, char *argv[]) {
       return 1;
     }
 
-    weight = layer_->getWeights()[0].getVariable();
-    weight.print(std::cout);
-
     /****** testing with a golden data if any ********/
     nntrainer::Tensor golden(1, 1, 15, 8);
 
@@ -252,9 +232,6 @@ int main(int argc, char *argv[]) {
       nntrainer::Tensor weight_out_fc(1, 1, 32, 1);
       loadFile("fc_weight_golden.out", weight_out_fc);
       weight_out_fc.print(std::cout);
-
-      weight_fc = layer_fc_->getWeights()[0].getVariable();
-      weight_fc.print(std::cout);
     } catch (...) {
       std::cerr << "Warning: during loading golden data\n";
     }

diff --git a/Applications/Embedding/jni/meson.build b/Applications/Embedding/jni/meson.build
index 719cf68..e919b5d 100644 (file)
--- a/Applications/Embedding/jni/meson.build
+++ b/Applications/Embedding/jni/meson.build
@@ -12,3 +12,9 @@ test('app_embedding', e, args: ['train',
   nntr_app_resdir / 'Embedding' / 'Embedding.ini',
   nntr_app_resdir / 'Embedding' / 'embedding_input.txt']
 )
+
+# test split example
+test('app_embedding_split', e, args: ['train',
+  nntr_app_resdir / 'Embedding' / 'Embedding_split.ini',
+  nntr_app_resdir / 'Embedding' / 'embedding_input.txt']
+)

diff --git a/Applications/Embedding/res/Embedding.ini b/Applications/Embedding/res/Embedding.ini
index 572bf7b..1822edc 100644 (file)
--- a/Applications/Embedding/res/Embedding.ini
+++ b/Applications/Embedding/res/Embedding.ini
@@ -1,19 +1,19 @@
 # Model Section : Model
 [Model]
-Type = Regression      # Network Type : Regression, KNN, NeuralNetwork
-Learning_rate = 0.001  # Learning Rate
+Type = NeuralNetwork
 Epochs = 100           # Epochs
-Optimizer = adam       # Optimizer : sgd (stochastic gradien decent),
-                       #             adam (Adamtive Moment Estimation)
 Loss = cross           # Loss function : mse (mean squared error)
                         #                       cross ( cross entropy )
-Save_Path = "embedding_model.bin" # model path to save / read
+Save_Path = "embedding_model_split.bin" # model path to save / read
 batch_size = 10                # batch size
+
+[Optimizer]
+Type = adam
 beta1 = 0.9
 beta2 = 0.999
+Learning_rate = 0.001
 epsilon = 1e-7
 
-# Layer Section : Name
 [embedding]
 Type = embedding
 in_dim = 15

diff --git a/Applications/Embedding/res/Embedding_split.ini b/Applications/Embedding/res/Embedding_split.ini
new file mode 100644 (file)
index 0000000..d9e3c20
--- /dev/null
+++ b/Applications/Embedding/res/Embedding_split.ini
@@ -0,0 +1,51 @@
+# Model Section : Model
+[Model]
+Type = NeuralNetwork
+Epochs = 100           # Epochs
+Loss = cross           # Loss function : mse (mean squared error)
+                        #                       cross ( cross entropy )
+Save_Path = "embedding_model_split.bin" # model path to save / read
+batch_size = 10                # batch size
+
+[Optimizer]
+Type = adam
+beta1 = 0.9
+beta2 = 0.999
+Learning_rate = 0.001
+epsilon = 1e-7
+
+[input]
+Type = input
+input_shape = 1:2:2 # channel:height:width
+
+[split]
+Type = split
+split_dimension = 2 # split by height
+
+[embedding1]
+input_layers = split
+Type = embedding
+in_dim = 15
+out_dim = 8
+in_length = 1
+
+[embedding2]
+input_layers = split
+Type = embedding
+in_dim = 15
+out_dim = 8
+in_length = 1
+
+[concat]
+input_layers = embedding1, embedding2
+Type = concat
+
+[flatten]
+Type = flatten
+
+[outputlayer]
+Type = fully_connected
+input_layers = flatten
+Unit = 1
+Bias_initializer = zeros
+Activation = sigmoid