<https://github.com/ndevilla/iniparser>
## Sections
+### Model Section
-Configuration file consists of Two Sections, Network and Layer
-
-### Network Section
-
-Network section includes the hyper-parameters about Network such as batch size, name of model file to save trained weight, epochs and etc.
+Model section includes the hyper-parameters for the Network such type, epochs, loss, save path and batch size.
Start with "[Model]"
```ini
# Network Section : Network
[Model]
-Type = NeuralNetwork
-Epochs = 1500
-Loss = cross
-Save_Path = "model.bin"
+type = NeuralNetwork
+epochs = 1500
+loss = cross
+save_path = "model.bin"
batch_size = 32
```
Define the optimizer to be used for training. This is an optional section needed only for training, and can be skipped for inference.
-Start with "[ Optimizer ]"
+Start with "[Optimizer]"
1. ```type = <string>```
- Optimizer type to apply the gradients to weights.
+ Optimizer type to apply the gradients to weights. The default value is adam if the type is not used.
* adam : Adaptive Moment Estimation
* sgd : stochastic gradient decent
-2. ```learning_rate = <float>```
-
- Initial learning rate to decay
+2. ```beta1 = <float>```
-3. ```beta1 = <float>```
+ beta1 parameter for adam optimizer. Only valid for adam. The default value is 0.9.
- beta1 parameter for adam optimizer. Only valid for adam. 0.9 is default.
-4. ```beta2 = <float>```
+3. ```beta2 = <float>```
- beta2 parameter for adam optimizer. Only valid for adam. 0.999 is default.
+ beta2 parameter for adam optimizer. Only valid for adam. The default value is 0.999.
-5. ```epsilon = <float>```
+4. ```epsilon = <float>```
- Epsilon parameter for adam optimizer. Only valid for adam. 1.0e-7 is default.
+ Epsilon parameter for adam optimizer. Only valid for adam. The default value is 1.0e-7.
Below is a sample Optimizer section.
```ini
# Optimizer Section
[Optimizer]
-Type = adam
-Learning_rate = 1e-4
-batch_size = 32
+type = adam
beta1 = 0.9
beta2 = 0.999
epsilon = 1e-7
```
-### DataSet Section
-
-Define data set. training, validation, test data set.
-
-Start with "[ DataSet ]"
+### Learning Rate Scheduler Section
-1. ```buffersize = <unsigned int>```
+Define the type, learning rate, decay steps and decay rate.
- Define Buffer size. usually it is greater than batch size.
+Start with "[LearningRateScheduler]"
- Data buffer thread keeps reading the data from the file and stores the data into the data buffer.
- Meanwhile main thread gets the training data from this data buffer and feeds it to the model.
- This keyword defines the size of Data Buffer.
-
-2. ```traindata = <string>```
-
- training data file path. The data must be saved as following
+1. ```type = <string>```
- ```feature data[i], label data[i], feature data[i+1], label data[i+1], ...```
+ constant, exponential and step are supported.
+ * constant : constant learning rate
+ * exponential : exponential decay
+ * step: step decay
-3. ```validdata = <string>```
+2. ```learning_rate = <float>```
- validation data file path. The data must be saved as following
+ Initial learning rate to decay.
- ```feature data[i], label data[i], feature data[i+1], label data[i+1], ...```
+ Constant and exponential receive only one float value.
-4. ```testdata = <string>```
+ However, step must receive two or more float values separated by commas.
- test data file path. The data must be saved as following
+ ```learning_rate = <float>, <float>, ..., <float>```
- ```feature data[i], label data[i], feature data[i+1], label data[i+1], ...```
-5. ```labeldata = <string>```
+3. ```decay_steps = <float>```
- label data file path. The data must be saved as following
+ Decay steps. Only valid for exponential.
- ```class Name [i], class name [i+1],...```
+4. ```decay_rate = <float>```
-### Layer Section
+ Decay rate
-Describe hyper-parameters for layer. Order of layers in the model follows the order of definition of layers here from top to bottom.
+5. ```iteration = <unsigned int>, <unsigned int>, ..., <unsigned int>```
-Start with "[ ${layer name} ]". This layer name must be unique throughout network model.
+ Iteration, Only valid for step.
+ Step receive one or more unsigned int value separated by commas.
-1. ```type = <string>```
+Below is a sample Learning Rate scheduler Section.
- Type of Layer
- * input : input layer
- * conv2d : 2D convolution layer
- * pooling2d : 2D pooling layer
- * flatten : flatten layer
- * fully_connected : fully connected layer
- * batch_normalization : batch normalization layer
- * activation : activation layer
-
-2. ```kernel_size = <unsigned int>,<unsigned int>```
-
- Kernel size for convolution layer
-
-3. ```bias_init_zero = <bool>```
-
- token to initialize bias with zeros. Setting to False would initialize bias randomly.
-
-4. ```normalization = <bool>```
-
- normalization on the input of this layer.
+```ini
+# Learning Rate Scheduler Section
+[LearningRateScheduler]
+type=constant
+learning_rate = 1e-4 # Learning Rate
+```
-5. ```standardization = <bool>```
+### Train Set Section
- standardization on the input of this layer.
+Define the type and path of the traing data file.
-6. ```input_shape = <unsigned int>:<unsigned int>:<unsigned int>```
+Start with "[train_set]"
- shape of input (shouldn't be zero).
+1. ``` type = <string> ```
-7. ```activation = <string>```
+ Currently only file is supported.
- set activation layer
- * tanh : tanh function
- * sigmoid : sigmoid function
- * relu : ReLU function
- * softmax : softmax function
+2. ``` path = <string> ```
-8. ```weight_regularizer = <string>```
+ Data path for training, The path is mandatory.
- set weight decay
- * l2norm : L2 normalization
+Below is a sample Train Set section.
-9. ```weight_regularizer_constant = <float>```
+```ini
+# Train Set Section
+[train_set]
+type = file
+path = trainDataset.dat
+```
- coefficient for weight decay
+### Validation Set Section
-10. ```unit = <unsigned int>```
+Define the type and path of the validation data file.
- set the output layer for fully connected layer
+Start with "[valid_set]"
-11. ```weight_initializer = <string>```
+1. ``` type = <string> ```
- set weight initialization method
- * zeros : Zero initialization
- * lecun_normal : LeCun normal initialization
- * lecun_uniform : LeCun uniform initialization
- * xavier_normal : xavier normal initialization
- * xavier_uniform : xavier uniform initialization
- * he_normal : He normal initialization
- * he_uniform : He uniform initialization
+ Currently only file is supported.
-12. ```filters = <unsigned int>```
+2. ``` path = <string> ```
- set filters size for convolution layer
+ Data path for validation.
-13. ```stride = <unsigned int>,<unsigned int>```
+Below is a sample Validation Set Section.
- set stride for convolution and pooling layer
+```ini
+# Validation Set Section
+[valid_set]
+type = file
+path = validDataset.dat
+```
-14. ```padding = <unsigned int>,<unsigned int>```
+### Test Set Section
- set padding for convolution and pooling layer
+Define the type and path of the test data file.
-15. ```pool_size = <unsigned int>,<unsigned int>```
+Start with "[test_set]"
- set pooling size for pooling layer
+1. ``` type = <string> ```
-16. ```pooling = <string>```
+ Currently only file is supported.
- define type of pooling
- * max : max pooling
- * average : average pooling
- * global_max : global max pooling
- * global_average : global average pooling
+2. ``` path = <string> ```
-17. ```flatten = <bool>```
+ Data path for test.
- flattens the output of this layer.
+Below is a sample Test Set Section.
- Enabling this option is equivalent to attaching a flatten layer after the current layer.
+```ini
+# Test Set Section
+[test_set]
+type = file
+path = testDataset.dat
+```
+### Layer Section
-18. ```epsilon = <float>```
+Describe hyper-parameters for layer. Order of layers in the model follows the order of definition of layers here from top to bottom.
- Epsilon parameter for batch normalization layer. Default is 0.001.
+Start with "[ ${layer name} ]". This layer name must be unique throughout network model.
-### Properties for layer
+1. ```type = <string>```
-Each layer requires different properties.
+ Type of Layer
+ * input : input layer
+ * fully_connected : fully connected layer
+ * batch_normalization : batch normalization layer
+ * conv2d : convolution 2D layer
+ * pooling2d : pooling 2D layer
+ * flatten : flatten layer
+ * activation : activation layer
+ * addition : addition layer
+ * concat : concat layer
+ * multiout : multiout layer
+ * embedding : embedding layer
+ * rnn : RNN layer
+ * lstm : LSTM layer
+ * split : split layer
+ * gru : GRU layer
+ * permute : permute layer
+ * dropout : dropout layer
+ * backbone_nnstreamer : backbone layer using nnstreamer
+ * backbone_tflite : backbone layer using tflite
+ * centroid_knn : centroid KNN layer
+ * conv1d : convolution 1D layer
+ * lstmcell : LSTM Cell layer
+ * grucell : GRU Cell layer
+ * rnncell : RNN Cell layer
+ * zoneout_lstmcell : Zoneout LSTM Cell layer
+ * preprocess_flip : preprocess flip layer
+ * preprocess_translate : preprocess translate layer
+ * preprocess_l2norm : preprocess l2norm layer
+ * mse : MSE loss layer
+ * cross_sigmoid : cross entropy with sigmoid loss layer
+ * cross_softmax : Cross entropy with softmax loss layer
+
+2. ```key = value```
+
+ The table below shows the available keys and values for each layer type.
+ There are two types of layers. One type includes commonly trainable weights and the other type does not include. The following are the available properties for each layer type which include commonly trainable weights:
+
+Type | Key | Value | Default value | Description
+---------- | --- | ----- | ----------- | -----------
+(Universal properties) | | | | Universal properties that applies to every layer
+ | name | (string) | | An identifier for each layer
+ | trainable | (boolean) | true | Allow weights to be trained if true
+ | input_layers | (string) | | Comma-separated names of layers to be inputs of the current layer
+ | input_shape | (string) | | Comma-separated Formatted string as "channel:height:width". If there is no channel then it must be 1. First layer of the model must have input_shape. Other can be omitted as it is calculated at compile phase.
+ | flatten | (boolean) | | Flatten shape from `c:h:w` to `1:1:c*h*w`
+ | activation | (categorical) | | Activation type
+ | | tanh | | Hyperbolic tangent
+ | | sigmoid | | Sigmoid function
+ | | relu | | Relu function
+ | | softmax | | Softmax function
+ | loss | (float) | 0 | Loss
+ | weight_initializer | (categorical) | xavier_uniform | Weight initializer
+ | | zeros | | Zero initialization
+ | | lecun_normal | | LeCun normal initialization
+ | | lecun_uniform | | LeCun uniform initialization
+ | | xavier_normal | | Xavier normal initialization
+ | | xavier_uniform | | Xavier uniform initialization
+ | | he_normal | | He normal initialization
+ | | he_uniform | | He uniform initialization
+ | bias_initializer | (categorical) | zeros | Bias initializer
+ | | zeros | | Zero initialization
+ | | lecun_normal | | LeCun normal initialization
+ | | lecun_uniform | | LeCun uniform initialization
+ | | xavier_normal | | Xavier normal initialization
+ | | xavier_uniform | | Xavier uniform initialization
+ | | he_normal | | He normal initialization
+ | | he_uniform | | He uniform initialization
+ | weight_regularizer | (categorical) | | Weight regularizer. Currently, only l2norm is supported
+ | | l2norm | | L2 weight regularizer
+ | weight_regularizer_constant | (float) | 1 | Weight regularizer constant
+`fully_connected` | | | | Fully connected layer
+ | unit | (unsigned integer) | | Number of outputs
+`conv1d` | | | | 1D Convolution layer
+ | filters | (unsigned integer) | | Number of filters
+ | kernel_size | (unsigned integer) | | Kernel size
+ | stride | (unsigned integer) | 1 | Strides
+ | padding | (categorical) | valid | Padding type
+ | | valid | | No padding
+ | | same | | Preserve dimension
+ | | (unsigned integer) | | Size of padding applied uniformly to all side
+ | | (array of unsigned integer of size 2) | | Padding for left, right
+`conv2d` | | | | 2D Convolution layer
+ | filters | (unsigned integer) | | Number of filters
+ | kernel_size | (array of unsigned integer) | | Comma-separated unsigned integers for kernel size, `height, width` respectively
+ | stride | (array of unsigned integer) | 1, 1 | Comma-separated unsigned integers for strides, `height, width` respectively
+ | padding | (categorical) | valid | Padding type
+ | | valid | | No padding
+ | | same | | Preserve height/width dimension
+ | | (unsigned integer) | | Size of padding applied uniformly to all side
+ | | (array of unsigned integer of size 2) | | Padding for height, width
+ | | (array of unsigned integer of size 4) | | Padding for top, bottom, left, right
+`embedding` | | | | Embedding layer
+ | in_dim | (unsigned integer) | | Vocabulary size
+ | out_dim | (unsigned integer) | | Word embeddeing size
+`rnn` | | | | RNN layer
+ | unit | (unsigned integer) | | Number of output neurons
+ | hidden_state_activation | (categorical) | tanh | Activation type
+ | | tanh | | Hyperbolic tangent
+ | | sigmoid | | Sigmoid function
+ | | relu | | Relu function
+ | | softmax | | Softmax function
+ | return_sequences | (boolean) | false | Return only the last output if true, else return full output
+ | dropout | (float) | 0 | Dropout rate
+ | integrate_bias | (boolean) | false | Integrate bias_ih, bias_hh to bias_h
+`rnncell` | | | | RNNCELL layer
+ | unit | (unsigned integer) | | Number of output neurons
+ | hidden_state_activation | (categorical) | tanh | Activation type
+ | | tanh | | Hyperbolic tangent
+ | | sigmoid | | Sigmoid function
+ | | relu | | Relu function
+ | | softmax | | Softmax function
+ | dropout | (float) | 0 | Dropout rate
+ | integrate_bias | (boolean) | false | Integrate bias_ih, bias_hh to bias_h
+`lstm` | | | | LSTM layer
+ | unit | (unsigned integer) | | Number of output neurons
+ | hidden_state_activation | (categorical) | tanh | Activation type
+ | | tanh | | Hyperbolic tangent
+ | | sigmoid | | Sigmoid function
+ | | relu | | Relu function
+ | | softmax | | Softmax function
+ | recurrent_activation | (categorical) | sigmoid | Activation type for recurrent step
+ | | tanh | | Hyperbolic tangent
+ | | sigmoid | | Sigmoid function
+ | | relu | | Relu function
+ | | softmax | | Softmax function
+ | return_sequences | (boolean) | false | Return only the last output if true, else return full output
+ | dropout | (float) | 0 | Dropout rate
+ | integrate_bias | (boolean) | false | Integrate bias_ih, bias_hh to bias_h
+ | max_timestep | (unsigned integer) | | Maximum timestep
+`lstmcell` | | | | LSTMCELL layer
+ | unit | (unsigned integer) | | Number of output neurons
+ | hidden_state_activation | (categorical) | tanh | Activation type
+ | | tanh | | Hyperbolic tangent
+ | | sigmoid | | Sigmoid function
+ | | relu | | Relu function
+ | | softmax | | Softmax function
+ | recurrent_activation | (categorical) | sigmoid | Activation type for recurrent step
+ | | tanh | | Hyperbolic tangent
+ | | sigmoid | | Sigmoid function
+ | | relu | | Relu function
+ | | softmax | | Softmax function
+ | dropout | (float) | 0 | Dropout rate
+ | integrate_bias | (boolean) | false | Integrate bias_ih, bias_hh to bias_h
+`gru` | | | | GRU layer
+ | unit | (unsigned integer) | | Number of output neurons
+ | hidden_state_activation | (categorical) | tanh | Activation type
+ | | tanh | | Hyperbolic tangent
+ | | sigmoid | | Sigmoid function
+ | | relu | | Relu function
+ | | softmax | | Softmax function
+ | recurrent_activation | (categorical) | sigmoid | Activation type for recurrent step
+ | | tanh | | Hyperbolic tangent
+ | | sigmoid | | Sigmoid function
+ | | relu | | Relu function
+ | | softmax | | Softmax function
+ | return_sequences | (boolean) | false | Return only the last output if true, else return full output
+ | dropout | (float) | 0 | Dropout rate
+ | integrate_bias | (boolean) | false | Integrate bias_ih, bias_hh to bias_h
+ | reset_after | (boolean) | true | Apply reset gate before/after the matrix
+`grucell` | | | | GRUCELL layer
+ | unit | (unsigned integer) | | Number of output neurons
+ | reset_after | (boolean) | true | Apply reset gate before/after the matrix multiplication
+ | hidden_state_activation | (categorical) | tanh | Activation type
+ | | tanh | | Hyperbolic tangent
+ | | sigmoid | | Sigmoid function
+ | | relu | | Relu function
+ | | softmax | | Softmax function
+ | recurrent_activation | (categorical) | sigmoid | Activation type for recurrent step
+ | | tanh | | Hyperbolic tangent
+ | | sigmoid | | Sigmoid function
+ | | relu | | Relu function
+ | | softmax | | Softmax function
+ | dropout | (float) | 0 | Dropout rate
+ | integrate_bias | (boolean) | false | Integrate bias_ih, bias_hh to bias_h
+`zoneout_lstmcell` | | | | ZONEOUTLSTMCELL layer
+ | unit | (unsigned integer) | | Number of output neurons
+ | hidden_state_activation | (categorical) | tanh | Activation type
+ | | tanh | | Hyperbolic tangent
+ | | sigmoid | | Sigmoid function
+ | | relu | | Relu function
+ | | softmax | | Softmax function
+ | recurrent_activation | (categorical) | sigmoid | Activation type for recurrent step
+ | | tanh | | Hyperbolic tangent
+ | | sigmoid | | Sigmoid function
+ | | relu | | Relu function
+ | | softmax | | Softmax function
+ | cell_state_zoneout_rate | (float) | 0 | Zoneout rate for cell state
+ | hidden_state_zoneout_rate | (float) | 0 | zoneout rate for hidden state
+ | integrate_bias | (boolean) | false | Integrate bias_ih, bias_hh to bias_h
+
+
+The following are the available properties for each layer type which does not include (`weight_initializer`, `bias_initializer`, `weight_regularizer`, `weight_regularizer_constant`) properties.
+
+
+Type | Key | Value | Default value | Description
+---------- | --- | ----- | ----------- | -----------
+(Universal properties) | | | | Universal properties that applies to every layer
+ | name | (string) | | An identifier for each layer
+ | trainable | (boolean) | true | Allow weights to be trained if true
+ | input_layers | (string) | | Comma-separated names of layers to be inputs of the current layer
+ | input_shape | (string) | | Comma-separated Formatted string as "channel:height:width". If there is no channel then it must be 1. First layer of the model must have input_shape. Other can be omitted as it is calculated at compile phase.
+ | flatten | (boolean) | | Flatten shape from `c:h:w` to `1:1:c*h*w`
+ | activation | (categorical) | | Activation type
+ | | tanh | | Hyperbolic tangent
+ | | sigmoid | | Sigmoid function
+ | | relu | | Relu function
+ | | softmax | | Softmax function
+ | loss | (float) | 0 | Loss
+`input` | | | | Input layer
+ | normalization | (boolean) | false | Normalize input if true
+ | standardization | (boolean) | false | Standardize input if true
+`batch_normalization` | | | | Batch normalization layer
+ | epsilon | (float) | 0.001 | Small value to avoid divide by zero
+ | moving_mean_initializer | (categorical) | zeros | Moving mean initializer
+ | | zeros | | Zero initialization
+ | | lecun_normal | | LeCun normal initialization
+ | | lecun_uniform | | LeCun uniform initialization
+ | | xavier_normal | | Xavier normal initialization
+ | | xavier_uniform | | Xavier uniform initialization
+ | | he_normal | | He normal initialization
+ | | he_uniform | | He uniform initialization
+ | moving_variance_initializer | (categorical) | ones | Moving variance initializer
+ | | zeros | | Zero initialization
+ | | lecun_normal | | LeCun normal initialization
+ | | lecun_uniform | | LeCun uniform initialization
+ | | xavier_normal | | Xavier normal initialization
+ | | xavier_uniform | | Xavier uniform initialization
+ | | he_normal | | He normal initialization
+ | | he_uniform | | He uniform initialization
+ | gamma_initializer | (categorical) | ones | Gamma initializer
+ | | zeros | | Zero initialization
+ | | lecun_normal | | LeCun normal initialization
+ | | lecun_uniform | | LeCun uniform initialization
+ | | xavier_normal | | Xavier normal initialization
+ | | xavier_uniform | | Xavier uniform initialization
+ | | he_normal | | He normal initialization
+ | | he_uniform | | He uniform initialization
+ | beta_initializer | (categorical) | zeros | Beta initializer
+ | | zeros | | Zero initialization
+ | | lecun_normal | | LeCun normal initialization
+ | | lecun_uniform | | LeCun uniform initialization
+ | | xavier_normal | | Xavier normal initialization
+ | | xavier_uniform | | Xavier uniform initialization
+ | | he_normal | | He normal initialization
+ | | he_uniform | | He uniform initialization
+ | momentum | (float) | 0.99 | Momentum for moving average in batch normalization
+`pooling2d` | | | | Pooling layer
+ | pooling | (categorical) | | Pooling type
+ | | max | | Max pooling
+ | | average | | Average pooling
+ | | global_max | | Global max pooling
+ | | global_average | | Global average pooling
+ | pool_size | (array of unsigned integer) | | Comma-separated unsigned intergers for pooling size, `height, width` respectively
+ | stride | (array of unsigned integer) | 1, 1 | Comma-separated unsigned intergers for stride, `height, width` respectively
+ | padding | (categorical) | valid | Padding type
+ | | valid | | No padding
+ | | same | | Preserve height/width dimension
+ | | (unsigned integer) | | Size of padding applied uniformly to all side
+ | | (array of unsigned integer of size 2) | | Padding for height, width
+ | | (array of unsigned integer of size 4) | | Padding for top, bottom, left, right
+`flatten` | | | | Flatten layer
+`activation` | | | | Activation layer
+ | activation | (categorical) | | Activation type
+ | | tanh | | Hyperbolic tangent
+ | | sigmoid | | Sigmoid function
+ | | relu | | Relu function
+ | | softmax | | Softmax function
+`addition` | | | | Addition layer
+`concat` | | | | Concat layer
+`multiout` | | | | Multiout layer
+`split` | | | | Split layer
+ | split_dimension | (unsigned integer) | | Which dimension to split. Split batch dimension is not allowed
+`permute` | | | | Permute layer
+`dropout` | | | | Dropout layer
+ | dropout | (float) | 0 | Dropout rate
+`backbone_nnstreamer` | | | | NNStreamer layer
+ | model_path | (string) | | NNStreamer model path
+`backbone_tflite` | | | | TensorFlow Lite layer
+ | model_path | (string) | | TensorFlow Lite model path
+`centroid_knn` | | | | Centroid KNN layer
+ | num_class | (unsigned integer) | | Number of class
+`preprocess_flip` | | | | Preprocess flip layer
+ | flip_direction | (categorical) | | Flip direction
+ | | horizontal | | Horizontal direction
+ | | vertical | | Vertiacl direction
+ | | horizontal_and_vertical | horizontal_and_vertical | Horizontal_and vertical direction
+`preprocess_translate` | | | | Preprocess translate layer
+ | random_translate | (float) | | Translate factor value
+`preprocess_l2norm` | | | | Preprocess l2norm layer
+`mse` | | | | MSE loss layer
+`cross_sigmoid` | | | | Cross entropy with sigmoid loss layer
+`cross_softmax` | | | | Cross entropy with softmax loss layer
- | Layer | Properties |
- |:-------:|:---|
- | conv2d |<ul><li>filters</li><li>kernel_size</li><li>stride</li><li>padding</li><li>normalization</li><li>standardization</li><li>input_shape</li><li>bias_init_zero</li><li>activation</li><li>flatten</li><li>weight_regularizer</li><li>weight_regularizer_constant</li><li>weight_initializer</li></ul>|
- | pooling2d | <ul><li>pooling</li><li>pool_size</li><li>stride</li><li>padding</li></ul> |
- | flatten | - |
- | fully_connected | <lu><li>unit</li><li>normalization</li><li>standardization</li><li>input_shape</li><li>bias_initializer</li><li>activation</li><li>flatten</li><li>weight_regularizer</li><li>weight_regularizer_constant</li><li>weight_initializer</li></lu>|
- | input | <lu><li>normalization </li><li>standardization</li><li>input_shape</li><li>flatten</li></lu>|
- | batch_normalization | <lu><li>epsilon</li><li>flatten</li></lu> |
Below is sample for layers to define a model.
```ini
[conv2d_c2_layer]
-Type = conv2d
+type = conv2d
kernel_size = 5,5
-bias_initializer=zeros
-Activation=sigmoid
+bias_initializer = zeros
+activation = sigmoid
weight_initializer = xavier_uniform
filters = 12
stride = 1,1
padding = 0,0
[outputlayer]
-Type = fully_connected
+type = fully_connected
Unit = 10
weight_initializer = xavier_uniform
bias_initializer = zeros
-Activation = softmax
+activation = softmax
```
### Backbone section
If this backbone must be trained (defaults to false). Only supported for ini backbones (nntrainer models).
-3. ```Preload = <bool>```
-
- Load pretrained weights from the saved modelfile of backbone (defaults to false). Only supported for ini backbone (nntrainer models).
-
-4. ```InputShape = <string>```
-
- Set the shape of the input layer for the backbone model. Only supported for ini backbones (nntrainer models).
-
-5. ```InputLayer = <string>```
-
- Choose the start layer for the backbone. This allows taking a subgraph starting with the specified layer name as a backbone. Only supported for ini backbones (nntrainer models).
-
-6. ```OutputLayer = <string>```
-
- Choose the end layer for the backbone. This allows taking a subgraph ending with the specified layer name as a backbone. Only supported for ini backbones (nntrainer models).
-``
Below is sample backbone section.
```ini
```
### Configuration file example
+Only INI formatted files *.ini is supported to construct a model from a file.
+Special sections [Model], [Optimizers], [LearningRateScheduler], [train_set], [valid_set], [test_set] are respectively referring to model, optimizer and data provider objects. Rest of INI sections map to a layer. Keys and values from each section set properties of the layer. All keys and values are treated as case-insensitive.
+
+The following restrictions must be adhered to:
-This has one input layer, two convolution layers, two pooling layers, one flatten layer and one fully connected layer to classify MNIST example.
+ - Model file must have a `[Model]` section.
+ - Model file must have at least one layer.
+ - Valid keys must have valid properties. The invalid keys in each section result in an error.
+ - All paths inside the INI file are relative to the INI file path unless the absolute path is stated.
+Below is sample backbone section.
It takes 1 x 28 x 28 gray data (0~255) as an input. Adam optimizer is used to apply gradient and learning rate is 1.0e-4.
```ini
# Model Section
[Model]
-type = NeuralNetwork
-learning_rate = 1e-4
-epochs = 1500
-optimizer = adam
-loss = cross
-Save_Path = "model.bin"
-batch_size = 32
-beta1 = 0.9
-beta2 = 0.999
-epsilon = 1e-7
+type = NeuralNetwork # Network Type : Regression, KNN, NeuralNetwork
+epochs = 1500 # Epochs
+loss = cross # Loss function : mse (mean squared error)
+ # cross ( for cross entropy )
+save_path = "mnist_model.bin" # model path to save / read
+batch_size = 32 # batch size
-# Layer Section
+[Optimizer]
+type = adam
+beta1 = 0.9 # beta 1 for adam
+beta2 = 0.999 # beta 2 for adam
+epsilon = 1e-7 # epsilon for adam
+
+[LearningRateScheduler]
+type=constant
+learning_rate = 1e-4 # Learning Rate
+
+# Train Set Section
+[train_set]
+type = file
+path = "trainDataset.dat"
+
+# Layer Section : Name
[inputlayer]
type = input
input_shape = 1:28:28
+# Layer Section : Name
[conv2d_c1_layer]
type = conv2d
+input_layers = inputlayer
kernel_size = 5,5
-bias_initializer=zeros
-activation=sigmoid
+bias_initializer = zeros
+activation = sigmoid
weight_initializer = xavier_uniform
filters = 6
stride = 1,1
padding = 0,0
[pooling2d_p1]
-type=pooling2d
+type = pooling2d
+input_layers = conv2d_c1_layer
pool_size = 2,2
-stride =2,2
+stride = 2,2
padding = 0,0
pooling = average
[conv2d_c2_layer]
type = conv2d
+input_layers = pooling2d_p1
kernel_size = 5,5
-bias_initializer=zeros
-activation=sigmoid
+bias_initializer = zeros
+activation = sigmoid
weight_initializer = xavier_uniform
filters = 12
stride = 1,1
padding = 0,0
[pooling2d_p2]
-type=pooling2d
+type = pooling2d
+input_layers = conv2d_c2_layer
pool_size = 2,2
stride =2,2
padding = 0,0
pooling = average
[flatten]
-type=flatten
+type = flatten
+input_layers = pooling2d_p2
[outputlayer]
type = fully_connected
-unit = 10
+input_layers = flatten
+unit = 10 # Output Layer Dimension ( = Weight Width )
weight_initializer = xavier_uniform
bias_initializer = zeros
-activation = softmax
+activation = softmax # activation : sigmoid, softmax
```
projects / platform / core / ml / nntrainer.git / commitdiff