From: Ronghang Hu Date: Thu, 28 May 2015 06:52:47 +0000 (+0800) Subject: Add MatCaffe docs to docs/tutorial/interfaces.md X-Git-Tag: submit/tizen/20180823.020014~484^2~4 X-Git-Url: http://review.tizen.org/git/?a=commitdiff_plain;h=6af8efc0bf51d475077be146e5ab7b7899ad705d;p=platform%2Fupstream%2Fcaffeonacl.git Add MatCaffe docs to docs/tutorial/interfaces.md --- diff --git a/docs/tutorial/interfaces.md b/docs/tutorial/interfaces.md index 17430b3..f824c7c 100644 --- a/docs/tutorial/interfaces.md +++ b/docs/tutorial/interfaces.md @@ -67,10 +67,211 @@ Compile pycaffe by `make pycaffe`. The module dir caffe/python/caffe should be i ## MATLAB -The MATLAB interface -- matcaffe -- is the `caffe` mex and its helper m-files in caffe/matlab. Load models, do forward and backward, extract output and read-only model weights, and load the binaryproto format mean as a matrix. +The MATLAB interface -- matcaffe -- is a `caffe` package in caffe/matlab in which you can integrate Caffe in your Matlab code. -A MATLAB demo is in caffe/matlab/caffe/matcaffe_demo.m +In MatCaffe, you can -Note that MATLAB matrices and memory are in column-major layout counter to Caffe's row-major layout! Double-check your work accordingly. +* Creating multiple Nets in Matlab +* Do forward and backward computation +* Access any layer within a network, and any parameter blob in a layer +* Get and set data or diff to any blob within a network, not restricting to input blobs or output blobs +* Save a network's parameters to file, and load parameters from file +* Reshape a blob and reshape a network +* Edit network parameter and do network surgery +* Create multiple Solvers in Matlab for training +* Resume training from solver snapshots +* Access train net and test nets in a solver +* Run for a certain number of iterations and give back control to Matlab +* Intermingle arbitrary Matlab code to with gradient steps -Compile matcaffe by `make matcaffe`. +A MATLAB demo is in caffe/matlab/matcaffe_demo.m + +### Build MatCaffe + +Build MatCaffe with `make all matcaffe`. After that, you may test it using `make mattest`. + +Common issue: if you run into error messages `libstdc++.so.6:version 'GLIBCXX_3.4.15' not found` during `make mattest`, then it means that your Matlab's runtime libraries does not match your compile-time libraries. You may need to do the following before you start matlab: + + export LD_LIBRARY_PATH=/opt/intel/mkl/lib/intel64:/usr/local/cuda/lib64 + export LD_PRELOAD=/usr/lib/x86_64-linux-gnu/libstdc++.so.6 + +Or the equivalent based on where things are installed on your system, and do `make mattest` again to see if the issue is fixed. Note: this issue is sometimes more complicated since during its startup Matlab may overwrite your `LD_LIBRARY_PATH` environment variable. You can run `!ldd ./matlab/+caffe/private/caffe_.mexa64` in Matlab to see its runtime libraries, and preload your compile-time libraries by exporting them to your `LD_PRELOAD` environment variable. + +After successful building and testing, add this package to Matlab search PATH by starting `matlab` from caffe root folder and running the following commands in Matlab command window. + + addpath ./matlab + +You can save your Matlab search PATH by running `savepath` so that you don't have to run the command above again every time you use MatCaffe. + +### Use MatCaffe + +MatCaffe is very similar to PyCaffe in usage. + +Examples below shows detailed usages and assumes you have downloaded BVLC CaffeNet from [Model Zoo](http://caffe.berkeleyvision.org/model_zoo.html) and started `matlab` from caffe root folder. + + model = './models/bvlc_reference_caffenet/deploy.prototxt'; + weights = './models/bvlc_reference_caffenet/bvlc_reference_caffenet.caffemodel'; + +#### Set mode and device + +**Mode and device should always be set BEFORE you create a net or a solver.** + +Use CPU: + + caffe.set_mode_cpu(); + +Use GPU and specify its gpu_id: + + caffe.set_mode_gpu(); + caffe.set_device(gpu_id); + +#### Create a network and access its layers and blobs + +Create a network: + + net = caffe.Net(model, weights, 'test'); % create net and load weights + +Or + + net = caffe.Net(model, 'test'); % create net but not load weights + net.copy_from(weights); % load weights + +which creates `net` object as + + Net with properties: + + layer_vec: [1x23 caffe.Layer] + blob_vec: [1x15 caffe.Blob] + inputs: {'data'} + outputs: {'prob'} + name2layer_index: [23x1 containers.Map] + name2blob_index: [15x1 containers.Map] + layer_names: {23x1 cell} + blob_names: {15x1 cell} + +The two `containers.Map` objects are useful to find the index of a layer or a blob by its name. + +You have access to every blob in this network. To fill blob 'data' with all ones: + + net.blobs('data').set_data(ones(net.blobs('data').shape)); + +To multiply all values in blob 'data' by 10: + + net.blobs('data').set_data(net.blobs('data').get_data() * 10); + +**Be aware that since Matlab is 1-indexed and column-major, the usual 4 blob dimensions in Matlab are `[width, height, channels, num]`, and `width` is the fastest dimension. Also be aware that images are in BGR channels.** Also, Caffe uses single-precision float data. If your data is not single, `set_data` will automatically convert it to single. + +You also have access to every layer, so you can do network surgery. For example, to multiply conv1 parameters by 10: + + net.params('conv1', 1).set_data(net.params('conv1', 1).get_data() * 10); % set weights + net.params('conv1', 2).set_data(net.params('conv1', 2).get_data() * 10); % set bias + +Alternatively, you can use + + net.layers('conv1').params(1).set_data(net.layers('conv1').params(1).get_data() * 10); + net.layers('conv1').params(2).set_data(net.layers('conv1').params(2).get_data() * 10); + +To save the network you just modified: + + net.save('my_net.caffemodel'); + +To get a layer's type (string): + + layer_type = net.layers('conv1').type; + +#### Forward and backward + +Forward pass can be done using `net.forward` or `net.forward_prefilled`. After creating some data for input blobs like `data = rand(net.blobs('data').shape);` you can run + + res = net.forward({data}); + prob = res{1}; + +Or + + net.blobs('data').set_data(data); + net.forward_prefilled(); + prob = net.blobs('prob').get_data(); + +Backward is similar using `net.backward` or `net.backward_prefilled` and replacing `get_data` and `set_data` with `get_diff` and `set_diff`. After creating some gradients for output blobs like `prob_diff = rand(net.blobs('prob').shape);` you can run + + res = net.backward({prob_diff}); + data_diff = res{1}; + +Or + + net.blobs('prob').set_diff(prob_diff); + net.backward_prefilled(); + data_diff = net.blobs('data').get_diff(); + +**However, the backward computation above doesn't get correct results, because Caffe decides that the network does not need backward computation. To get correct backward results, you need to set `'force_backward: true'` in your network prototxt.** + +After performing forward or backward pass, you can also get the data or diff in internal blobs. For example, to extract pool5 features after forward pass: + + pool5_feat = net.blobs('pool5').get_data(); + +#### Reshape + +Assume you want to run 1 image at a time instead of 10: + + net.blobs('data').reshape([227 227 3 1]); % reshape blob 'data' + net.reshape(); + +Then the whole network is reshaped, and now `net.blobs('prob').shape` should be `[1000 1]`; + +#### Training + +Assume you have created training and validation lmdbs following our [ImageNET Tutorial](http://caffe.berkeleyvision.org/gathered/examples/imagenet.html), to create a solver and train on ILSVRC 2012 classification dataset: + + solver = caffe.Solver('./models/bvlc_reference_caffenet/solver.prototxt'); + +which creates `solver` object as + + Solver with properties: + + net: [1x1 caffe.Net] + test_nets: [1x1 caffe.Net] + +To train: + + solver.solve(); + +Or train for only 1000 iterations (so that you can do something to its net before training more iterations) + + solver.step(1000); + +To get iteration number: + + iter = solver.iter(); + +To get its network: + + train_net = solver.net; + test_net = solver.test_nets(1); + +To resume from a snapshot "your_snapshot.solverstate": + + solver.restore('your_snapshot.solverstate'); + +#### Input and output + +`caffe.io` class provides basic input functions `load_image` and `read_mean`. For example, to read ILSVRC 2012 mean file (assume you have downloaded imagenet example auxiliary files by running `./data/ilsvrc12/get_ilsvrc_aux.sh`): + + mean_data = caffe.io.read_mean('./data/ilsvrc12/imagenet_mean.binaryproto'); + +To read Caffe's example image and resize to `[width, height]` and suppose we want `width = 256; height = 256;` + + im_data = caffe.io.load_image('./examples/images/cat.jpg'); + im_data = imresize(im_data, [width, height]); % resize using Matlab's imresize + +**Keep in mind that `width` is the fastest dimension and channels are BGR, which is different from the usual way that Matlab stores an image.** If you don't want to use `caffe.io.load_image` and prefer to load an image by yourself, you can do + + im_data = imread('./examples/images/cat.jpg'); % read image + im_data = im_data(:, :, [3, 2, 1]); % convert from RGB to BGR + im_data = permute(im_data, [2, 1, 3]); % permute width and height + im_data = single(im_data); % convert to single precision + +We do not provide extra functions for data output as Matlab itself is already quite powerful in output. + +#### Clear nets and solvers + +Call `caffe.reset()` to clear all solvers and stand-alone nets you have created.