neuron layers doc

diff --git a/docs/tutorial/layers.md b/docs/tutorial/layers.md
index d22e334..72db52c 100644 (file)
--- a/docs/tutorial/layers.md
+++ b/docs/tutorial/layers.md
@@ -72,7 +72,48 @@ The `CONVOLUTION` layer convolves the input image with a set of learnable filter
 
 #### Local Response Normalization
 
-`LRN`
+* LayerType: `LRN`
+* CPU implementation: `./src/caffe/layers/lrn_layer.cpp`
+* CUDA GPU implementation: `./src/caffe/layers/lrn_layer.cu`
+* Options (`ConvolutionParameter convolution_param`)
+    - Required: `num_output` ($c_o$), the number of filters
+    - Required: `kernel_size` or (`kernel_h`, `kernel_w`), specifies height & width of each filter
+    - Strongly recommended (default `type: 'constant' value: 0`): `weight_filler`
+    - Optional (default `true`): `bias_term`, specifies whether to learn and apply a set of additive biases to the filter outputs
+    - Optional (default 0): `pad` or (`pad_h`, `pad_w`), specifies the number of pixels to (implicitly) add to each side of the input
+    - Optional (default 1): `stride` or (`stride_h`, `stride_w`), specifies the intervals at which to apply the filters to the input
+    - Optional (default 1): `group` ($g$) if $>1$, restricts the connectivity of each filter to a subset of the input.  In particular, the input to the $i^{th}$ group of $n_f / g$ filters is the $i^{th}$ group of $c_i / g$ input channels.
+* Input
+    - $n \times c_i \times h_i \times w_i$ (repeated $K \ge 1$ times)
+* Output
+    - $n \times c_o \times h_o \times w_o$ (repeated $K$ times)
+* Sample (as seen in `./examples/imagenet/imagenet_train_val.prototxt`)
+
+        layers {
+          name: "conv1"
+          type: CONVOLUTION
+          bottom: "data"
+          top: "conv1"
+          blobs_lr: 1          # learning rate multiplier for the filters
+          blobs_lr: 2          # learning rate multiplier for the biases
+          weight_decay: 1      # weight decay multiplier for the filters
+          weight_decay: 0      # weight decay multiplier for the biases
+          convolution_param {
+            num_output: 96     # learn 96 filters
+            kernel_size: 11    # each filter is 11x11
+            stride: 4          # step 4 pixels between each filter application
+            weight_filler {
+              type: "gaussian" # initialize the filters from a Gaussian
+              std: 0.01        # distribution with stdev 0.01 (default mean: 0)
+            }
+            bias_filler {
+              type: "constant" # initialize the biases to zero (0)
+              value: 0
+            }
+          }
+        }
+
+The `CONVOLUTION` layer convolves the input image with a set of learnable filters, each producing one feature map in the output image.
 
 #### im2col
 
@@ -108,6 +149,13 @@ Loss drives learning by comparing an output to a target and assigning cost to mi
 
 ### Activation / Neuron Layers
 
+In general, activation / Neuron layers are element-wise operators, taking one bottom blob and producing one top blob of the same size. In the layers below, we will ignore the input and out sizes as they are identical:
+
+* Input
+    - $n \times c \times h \times w$
+* Output
+    - $n \times c \times h \times w$
+
 #### ReLU / Rectified-Linear and Leaky-ReLU
 
 * LayerType: `RELU`
@@ -115,10 +163,6 @@ Loss drives learning by comparing an output to a target and assigning cost to mi
 * CUDA GPU implementation: `./src/caffe/layers/relu_layer.cu`
 * Options (`ReLUParameter relu_param`)
     - Optional (default 0): `negative_slope`, specifies whether to leak the negative part by multiplying it with the slope value rather than setting it to 0.
-* Input
-    - $n \times c \times h \times w$
-* Output
-    - $n \times c \times h \times w$
 * Sample (as seen in `./examples/imagenet/imagenet_train_val.prototxt`)
 
         layers {
@@ -135,10 +179,6 @@ Given an input value x, The `RELU` layer computes the output as x if x > 0 and n
 * LayerType: `SIGMOID`
 * CPU implementation: `./src/caffe/layers/sigmoid_layer.cpp`
 * CUDA GPU implementation: `./src/caffe/layers/sigmoid_layer.cu`
-* Input
-    - $n \times c \times h \times w$
-* Output
-    - $n \times c \times h \times w$
 * Sample (as seen in `./examples/imagenet/mnist_autoencoder.prototxt`)
 
         layers {
@@ -155,16 +195,12 @@ The `SIGMOID` layer computes the output as sigmoid(x) for each input element x.
 * LayerType: `TANH`
 * CPU implementation: `./src/caffe/layers/tanh_layer.cpp`
 * CUDA GPU implementation: `./src/caffe/layers/tanh_layer.cu`
-* Input
-    - $n \times c \times h \times w$
-* Output
-    - $n \times c \times h \times w$
 * Sample
 
         layers {
-          name: "encode1neuron"
-          bottom: "encode1"
-          top: "encode1neuron"
+          name: "layer"
+          bottom: "in"
+          top: "out"
           type: TANH
         }
 
@@ -172,15 +208,61 @@ The `TANH` layer computes the output as tanh(x) for each input element x.
 
 #### Absolute Value
 
-`ABSVAL`
+* LayerType: `ABSVAL`
+* CPU implementation: `./src/caffe/layers/absval_layer.cpp`
+* CUDA GPU implementation: `./src/caffe/layers/absval_layer.cu`
+* Sample
+
+        layers {
+          name: "layer"
+          bottom: "in"
+          top: "out"
+          type: ABSVAL
+        }
+
+The `ABSVAL` layer computes the output as abs(x) for each input element x.
 
 #### Power
 
-`POWER`
+* LayerType: `POWER`
+* CPU implementation: `./src/caffe/layers/power_layer.cpp`
+* CUDA GPU implementation: `./src/caffe/layers/power_layer.cu`
+* Options (`PowerParameter power_param`)
+    - Optional (default 1): `power`
+    - Optional (default 1): `scale`
+    - Optional (default 0): `shift`
+* Sample
+
+        layers {
+          name: "layer"
+          bottom: "in"
+          top: "out"
+          type: POWER
+          power_param {
+            power: 1
+            scale: 1
+            shift: 0
+          }
+        }
+
+The `POWER` layer computes the output as (shift + scale * x) ^ power for each input element x.
 
 #### BNLL
 
-`BNLL`
+* LayerType: `BNLL`
+* CPU implementation: `./src/caffe/layers/bnll_layer.cpp`
+* CUDA GPU implementation: `./src/caffe/layers/bnll_layer.cu`
+* Sample
+
+        layers {
+          name: "layer"
+          bottom: "in"
+          top: "out"
+          type: BNLL
+        }
+
+The `BNLL` (binomial normal log likelihood) layer computes the output as log(1 + exp(x)) for each input element x.
+
 
 ### Data Layers