Ronghang Hu [Wed, 12 Aug 2015 04:38:06 +0000 (21:38 -0700)]
Data Layers Parallel for Multi-GPU
Allow data layers (and also PythonLayer when used as data layer) to be shared
among worker solver's training net, and also test net for future-proof if one
wants to do Multi-GPU testing. Data layers are locked during forward to ensure
sequential forward.
Evan Shelhamer [Fri, 7 Aug 2015 20:56:49 +0000 (13:56 -0700)]
[docs] add multi-gpu usage note to interfaces
mhouston [Fri, 10 Jul 2015 23:05:48 +0000 (16:05 -0700)]
Detect topology corner cases and improve broadcast order
- Start with distant nodes in broadcast
- Fix outside loop to loop for full tree depth
Cyprien Noel [Tue, 19 May 2015 18:11:05 +0000 (11:11 -0700)]
Multi-GPU
- Parallelize batches among GPUs and tree-reduce the gradients
- The effective batch size scales with the number of devices
- Batch size is multiplied by the number of devices
- Split batches between GPUs, and tree-reduce the gradients
- Detect machine topology (twin-GPU boards, P2P connectivity)
- Track device in syncedmem (thanks @thatguymike)
- Insert a callback in the solver for minimal code change
- Accept list for gpu flag of caffe tool, e.g. '-gpu 0,1' or '-gpu all'.
Run on default GPU if no ID given.
- Add multi-GPU solver test
- Deterministic architecture for reproducible runs
Cyprien Noel [Tue, 19 May 2015 03:07:36 +0000 (20:07 -0700)]
Allocate host memory through cudaMallocHost
thanks to discussion by @thatguymike and @flx42
Cyprien Noel [Tue, 19 May 2015 01:06:09 +0000 (18:06 -0700)]
Add DataReader for parallel training with one DB session
- Make sure each solver accesses a different subset of the data
- Sequential reading of DB for performance
- Prefetch a configurable amount of data to host memory
- Distribute data to solvers in round-robin way for determinism
Cyprien Noel [Tue, 19 May 2015 00:45:20 +0000 (17:45 -0700)]
Persistent prefetch thread
Cyprien Noel [Tue, 28 Apr 2015 21:46:20 +0000 (14:46 -0700)]
Change the way threads are started and stopped
- Interrupt the thread before waiting on join
- Provide a method for looping threads to exit on demand
- CHECK if start and stop succeed instead of returning an error
Cyprien Noel [Tue, 28 Apr 2015 02:48:10 +0000 (19:48 -0700)]
Thread-local Caffe
Cyprien Noel [Tue, 28 Apr 2015 21:28:04 +0000 (14:28 -0700)]
Add BlockingQueue for inter-thread communication
Evan Shelhamer [Sun, 9 Aug 2015 19:35:04 +0000 (12:35 -0700)]
Merge pull request #2887 from shelhamer/solver-test-data
Test solvers on fixed hdf5 data
Ronghang Hu [Sun, 9 Aug 2015 18:03:23 +0000 (11:03 -0700)]
Use net_->learnable_params() instead of net_->params() in RMSprop
In RMSProp solver, use const vector<Blob<Dtype>*>& net_params = this->net_->learnable_params(); instead of const vector<shared_ptr<Blob<Dtype> > >& net_params = this->net_->params();
Ronghang Hu [Sun, 9 Aug 2015 08:45:09 +0000 (01:45 -0700)]
Merge pull request #2888 from ronghanghu/rms-prop-fix-tests
Encapsulate kRMSDecay in solver tests
Ronghang Hu [Sun, 9 Aug 2015 08:32:25 +0000 (01:32 -0700)]
Encapsulate kRMSDecay in solver tests
Instead of introducing another argument kRMSDecay and setting it for every test, this param could be set by the RMSProp test class for encapsulation.
Ronghang Hu [Sun, 9 Aug 2015 07:35:27 +0000 (00:35 -0700)]
Merge pull request #2867 from ronghanghu/rms-prop
RMSProp clean up and rebase
Eren Golge [Sun, 9 Aug 2015 06:45:08 +0000 (23:45 -0700)]
Implement RMSProp Solver
Implement RMSProp solver and cleaned up to adjust to new solver interface that uses
accumulated gradients and refactored regularization.
Evan Shelhamer [Sat, 8 Aug 2015 20:07:13 +0000 (13:07 -0700)]
TestGradientBasedSolver: replace dummy data with hdf5
Rely on fixed hdf5 data for determinism of solver tests.
- draw random Gaussian data and targets for test and save to hdf5
- use the same data for all tests without constant / gaussian condition
previously needed for accumulation
- avoid test artifacts due to order of random draws in dummy data
Evan Shelhamer [Sat, 8 Aug 2015 19:57:45 +0000 (12:57 -0700)]
TestGradientBasedSolver: drop doubled seed inititialization
Evan Shelhamer [Sat, 8 Aug 2015 04:06:29 +0000 (21:06 -0700)]
Merge pull request #2813 from longjon/net-spec-imp
Python net spec cleanups and top-less layers
Jeff Donahue [Sat, 8 Aug 2015 03:44:43 +0000 (20:44 -0700)]
Merge pull request #2886 from jeffdonahue/master
temporarily switch the snapshot_format default back to BINARYPROTO
Jeff Donahue [Sat, 8 Aug 2015 03:03:04 +0000 (20:03 -0700)]
temporarily switch the snapshot_format default back to BINARYPROTO
out of anticipation for user issues due to issue #2885, which causes
Caffe to crash when it attempts to snapshot nets with duplicate layer
names
Jon Long [Fri, 7 Aug 2015 23:28:23 +0000 (16:28 -0700)]
Merge pull request #2877 from jeffdonahue/pycaffe-shape-accessor
[pycaffe] add Blob shape accessor
Jeff Donahue [Fri, 7 Aug 2015 23:05:00 +0000 (16:05 -0700)]
Merge pull request #2866 from jeffdonahue/fix-weight-sharing
Fix weight sharing
Jeff Donahue [Tue, 21 Jul 2015 03:50:50 +0000 (20:50 -0700)]
Net: add learnable_params_ used by solvers to correctly handle shared params
-Params now share diffs as well as data (works due to layers
accumulating gradients into param diffs, rather than overwriting)
-It's now required that any shared params with specified lr_mult's,
decay_mult's match
-TestGradientBasedSolver checks that behavior remains correct with
shared weights
Jeff Donahue [Fri, 7 Aug 2015 22:34:00 +0000 (15:34 -0700)]
TestGradientBasedSolver: make tests across solver types more consistent
Jeff Donahue [Fri, 7 Aug 2015 22:27:02 +0000 (15:27 -0700)]
TestGradientBasedSolver: restore Gaussian filler to all tests except
accumulation one
Jeff Donahue [Fri, 7 Aug 2015 22:22:41 +0000 (15:22 -0700)]
Merge pull request #2836 from erictzeng/hdf5_snapshot
Snapshot model weights/solver state to HDF5 files
Eric Tzeng [Fri, 7 Aug 2015 20:17:26 +0000 (13:17 -0700)]
Update example bash scripts to expect .h5, new extensions in .gitignore
Eric Tzeng [Thu, 30 Jul 2015 01:40:38 +0000 (18:40 -0700)]
TestSnapshot expects .h5 snapshots, explicitly checks history.
Eric Tzeng [Wed, 22 Jul 2015 23:17:01 +0000 (16:17 -0700)]
Snapshot model weights/solver state to HDF5 files.
Summary of changes:
- HDF5 helper functions were moved into a separate file util/hdf5.cpp
- hdf5_save_nd_dataset now saves n-d blobs, can save diffs instead of
data
- Minor fix for memory leak in HDF5 functions (delete instead of
delete[])
- Extra methods have been added to both Net/Solver enabling
snapshotting and restoring from HDF5 files
- snapshot_format was added to SolverParameters, with possible values
HDF5 or BINARYPROTO (default HDF5)
- kMaxBlobAxes was reduced to 32 to match the limitations of HDF5
Jeff Donahue [Thu, 6 Aug 2015 20:46:11 +0000 (13:46 -0700)]
pycaffe: add shape accessor
Jon Long [Fri, 7 Aug 2015 20:51:52 +0000 (13:51 -0700)]
Merge pull request #2884 from tianzhi0549/patch-1
add [] to "delete pixels".
Jeff Donahue [Thu, 30 Jul 2015 00:27:58 +0000 (17:27 -0700)]
TestGradientBasedSolver: add TestSnapshot to verify behavior when
restoring net/solver from snapshot
Jeff Donahue [Thu, 30 Jul 2015 00:27:04 +0000 (17:27 -0700)]
add double_data, double_diff to BlobProto for weights/snapshots saved
when using Dtype == double
Jeff Donahue [Fri, 7 Aug 2015 18:58:19 +0000 (11:58 -0700)]
Merge pull request #2634 from mlopezantequera/patch-2
Update parse_log.py
Tian Zhi [Fri, 7 Aug 2015 18:31:19 +0000 (02:31 +0800)]
add [] to "delete pixels".
see https://isocpp.org/wiki/faq/freestore-mgmt#delete-array-built-ins.
Evan Shelhamer [Fri, 7 Aug 2015 16:23:20 +0000 (09:23 -0700)]
Merge pull request #2880 from koki0702/typo
[docs] Fix layer typo
koki1.saitoh [Fri, 7 Aug 2015 09:22:28 +0000 (18:22 +0900)]
Fix typo
Evan Shelhamer [Fri, 7 Aug 2015 01:41:02 +0000 (18:41 -0700)]
Merge pull request #2871 from shelhamer/python-layer-arg
[pycaffe] PythonLayer takes parameters by string
Takuya Narihira [Sat, 28 Feb 2015 03:34:51 +0000 (19:34 -0800)]
PythonLayer takes parameters by string
Evan Shelhamer [Thu, 6 Aug 2015 20:04:42 +0000 (13:04 -0700)]
Merge pull request #2876 from shelhamer/fix-py-include
[pycaffe,build] include Python first in caffe tool
Evan Shelhamer [Thu, 6 Aug 2015 20:04:15 +0000 (13:04 -0700)]
[pytest] open exception file with mode for python3
Evan Shelhamer [Thu, 6 Aug 2015 20:03:50 +0000 (13:03 -0700)]
[pycaffe,build] include Python first in caffe tool
Evan Shelhamer [Thu, 6 Aug 2015 07:27:59 +0000 (00:27 -0700)]
Merge pull request #2462 from longjon/correct-python-exceptions
Handle Python layer exceptions correctly
Evan Shelhamer [Tue, 4 Aug 2015 18:57:51 +0000 (11:57 -0700)]
Merge pull request #2859 from philkr/image_data
ImageData layer default batch size of 1, and check for zero batch size
philkr [Tue, 4 Aug 2015 17:27:34 +0000 (10:27 -0700)]
ImageData layer default batch size of 1, and check for zero batch size
Jeff Donahue [Mon, 3 Aug 2015 23:19:47 +0000 (16:19 -0700)]
Merge pull request #2583 from lukeyeager/fix-log-levels
Change log levels in upgrade_proto
Luke Yeager [Wed, 10 Jun 2015 00:50:40 +0000 (17:50 -0700)]
Change log levels in upgrade_proto
It's not an error if the upgrade succeeds
Jon Long [Sun, 2 Aug 2015 21:34:10 +0000 (14:34 -0700)]
Merge pull request #2837 from longjon/contributing-file
Add a CONTRIBUTING.md file, to appear on new Issue/PR pages
Jonathan L Long [Thu, 30 Jul 2015 03:16:31 +0000 (20:16 -0700)]
[docs] add CONTRIBUTING.md which will appear on GitHub new Issue/PR pages
Evan Shelhamer [Thu, 30 Jul 2015 00:20:31 +0000 (17:20 -0700)]
[docs] fix contrastive loss eq
make documented equation match the correct implementation of the
`max(margin - d, 0)^2` term in the loss. see #2321
Evan Shelhamer [Tue, 28 Jul 2015 17:27:39 +0000 (10:27 -0700)]
[docs] fix lmdb fetch url and path
Evan Shelhamer [Mon, 27 Jul 2015 17:58:11 +0000 (10:58 -0700)]
[docs] clear up PYTHONPATH confusion
Use the same language as the installation page to explain the Python
module path.
reported by @sdemyanov
Jonathan L Long [Fri, 24 Jul 2015 03:41:40 +0000 (20:41 -0700)]
[pytest] simple test of top-less layers
Jonathan L Long [Fri, 24 Jul 2015 03:40:02 +0000 (20:40 -0700)]
[pycaffe] net spec layers can have ntop=0
In this case, the Function is returned instead of a Top, which can be
assigned a name if desired.
Thanks @philkr for an earlier implementation of this.
Jonathan L Long [Fri, 24 Jul 2015 03:35:42 +0000 (20:35 -0700)]
[pycaffe] allow layers to have names different from their first tops
Previously, net spec only allowed names to be assigned to Tops, giving
layers the names of their first tops. Now, names can be assigned to
Functions, which become layer names in serialization. Unnamed Functions
still get named after their first top, if present, or autogenerated, if
not. (This will allow top-less layers in a natural way.)
Jonathan L Long [Fri, 24 Jul 2015 03:32:04 +0000 (20:32 -0700)]
[pycaffe] add Top._to_proto convenience function
This makes it possible to serialize Functions or Tops with a uniform
interface.
Jonathan L Long [Fri, 24 Jul 2015 03:26:46 +0000 (20:26 -0700)]
[pycaffe] use a Counter instead of a dict for counting net spec names
Jonathan L Long [Fri, 24 Jul 2015 03:23:42 +0000 (20:23 -0700)]
[pycaffe] remove dead code
Jon Long [Thu, 23 Jul 2015 03:37:17 +0000 (20:37 -0700)]
Merge pull request #2807 from Franck-Dernoncourt/patch-1
Fix path to mnist_autoencoder.prototxt
Franck Dernoncourt [Thu, 23 Jul 2015 00:32:12 +0000 (17:32 -0700)]
Fix path to mnist_autoencoder.prototxt
Eric Zeiberg [Wed, 22 Jul 2015 05:12:48 +0000 (22:12 -0700)]
[docs] set lmdb url to github mirror
Evan Shelhamer [Tue, 21 Jul 2015 17:33:31 +0000 (10:33 -0700)]
[docs] matlab 2015a compatible
Evan Shelhamer [Tue, 21 Jul 2015 05:19:46 +0000 (22:19 -0700)]
Merge pull request #2740 from philkr/travis_python3
[build] Travis scripts for python3 and pytest for cmake.
philkr [Wed, 8 Jul 2015 21:30:29 +0000 (14:30 -0700)]
Travis scripts for python3 and pytest for cmake. Also fixes CUDA CMake build issue #2722.
Evan Shelhamer [Fri, 17 Jul 2015 04:30:37 +0000 (21:30 -0700)]
[examples] fix link to point to new tutorial notebook
Jon Long [Thu, 16 Jul 2015 03:39:19 +0000 (20:39 -0700)]
Merge pull request #2762 from kashefy/fix_method_doc
tiny fix in Layer::Backward documentation
Youssef Kashef [Tue, 14 Jul 2015 09:26:03 +0000 (11:26 +0200)]
tiny fix in Layer::Backward documentation
Jon Long [Mon, 13 Jul 2015 01:35:25 +0000 (18:35 -0700)]
Merge pull request #2739 from AdamStelmaszczyk/patch-2
One command less in CIFAR10 documentation
Jon Long [Mon, 13 Jul 2015 00:44:44 +0000 (17:44 -0700)]
Merge pull request #2748 from glebm/patch-1
examples/imagenet: fix broken link
Jon Long [Mon, 13 Jul 2015 00:43:56 +0000 (17:43 -0700)]
Merge pull request #2750 from yosssi/patch-1
Update net_layer_blob.md
Keiji Yoshida [Sun, 12 Jul 2015 11:06:31 +0000 (20:06 +0900)]
Update net_layer_blob.md
Gleb Mazovetskiy [Sun, 12 Jul 2015 03:16:14 +0000 (04:16 +0100)]
examples/imagenet: fix broken link
Jeff Donahue [Sat, 11 Jul 2015 00:28:51 +0000 (17:28 -0700)]
Merge pull request #2742 from philkr/absval_warning
Removes a unused variable warning
philkr [Sat, 11 Jul 2015 00:03:03 +0000 (17:03 -0700)]
Removes a unused variable warning
AdamStelmaszczyk [Fri, 10 Jul 2015 20:41:46 +0000 (21:41 +0100)]
One command less
Also more readable and compatible with format of instructions for MNIST https://github.com/BVLC/caffe/tree/master/examples/mnist
Jeff Donahue [Fri, 10 Jul 2015 03:42:42 +0000 (20:42 -0700)]
Merge pull request #2712 from semitrivial/master
List protobuf-compiler dependency in the correct place (it is in the …
Jon Long [Wed, 8 Jul 2015 23:12:53 +0000 (16:12 -0700)]
Merge pull request #2714 from philkr/python3_netspec
Making the net_spec python3 compatible
philkr [Tue, 7 Jul 2015 23:32:46 +0000 (16:32 -0700)]
Making the net_spec python3 compatible
Evan Shelhamer [Wed, 8 Jul 2015 12:04:27 +0000 (14:04 +0200)]
Merge pull request #2716 from lukeyeager/cmake-typos
[build] fix CMake typos
Luke Yeager [Wed, 8 Jul 2015 04:30:33 +0000 (21:30 -0700)]
Fix CMake typos
No functional changes, just fixing whitespace errors and typos in
comments
semitrivial [Tue, 7 Jul 2015 07:29:16 +0000 (08:29 +0100)]
List protobuf-compiler dependency in the correct place (it is in the package managers for both 14.04 and 12.04)
Evan Shelhamer [Thu, 2 Jul 2015 23:40:25 +0000 (01:40 +0200)]
Merge pull request #2676 from lukeyeager/update-docs-boost
[docs] install boost without recommends to avoid conflicts -- close #2454
Luke Yeager [Wed, 1 Jul 2015 23:44:05 +0000 (16:44 -0700)]
Update installation docs for boost - close #2454
Evan Shelhamer [Wed, 1 Jul 2015 00:01:31 +0000 (17:01 -0700)]
Merge pull request #2213 from tnarihi/bilinear-upsampling-filler
add bilinear interpolation filler
Takuya Narihira [Fri, 27 Mar 2015 02:04:15 +0000 (19:04 -0700)]
bilinear filler -- useful for interpolation with DeconvolutionLayer
This filler is a convenience for interpolating with DeconvolutionLayer
or smoothing + downsampling with ConvolutionLayer for stride > 1.
Evan Shelhamer [Tue, 30 Jun 2015 22:37:33 +0000 (15:37 -0700)]
Merge pull request #2667 from BVLC/tutorial
bundle CVPR15 tutorial notebooks
Evan Shelhamer [Tue, 30 Jun 2015 22:35:21 +0000 (15:35 -0700)]
[examples] add Euclidean loss PythonLayer
Evan Shelhamer [Wed, 18 Mar 2015 02:01:02 +0000 (19:01 -0700)]
[examples] sequence and revise notebooks
- combine classification + filter visualization
- order by classification, learning LeNet, brewing logreg, and
fine-tuning to flickr style
- improve flow of content in classification + filter visualization
- include solver needed for learning LeNet
- edit notebook descriptions for site catalogue
Yangqing Jia [Tue, 17 Mar 2015 17:50:42 +0000 (10:50 -0700)]
[examples] flickr fine-tuning notebook
Jonathan L Long [Thu, 12 Mar 2015 08:03:51 +0000 (01:03 -0700)]
[examples] draft Python solving example
Evan Shelhamer [Tue, 30 Jun 2015 20:33:54 +0000 (13:33 -0700)]
Merge pull request #2086 from longjon/python-net-spec
Python net specification
Jonathan L Long [Thu, 18 Jun 2015 20:25:08 +0000 (13:25 -0700)]
[pytest] minimal testing of net specification
Jonathan L Long [Tue, 10 Mar 2015 00:43:53 +0000 (17:43 -0700)]
[examples] caffenet python spec
Jonathan L Long [Thu, 15 Jan 2015 09:27:42 +0000 (01:27 -0800)]
[pycaffe] basic net specification
Evan Shelhamer [Tue, 23 Jun 2015 21:48:24 +0000 (14:48 -0700)]
Merge pull request #2611 from dpaiton/test_net_bugfix
[compatibility] fix syntax for old protobuf
Evan Shelhamer [Tue, 23 Jun 2015 16:59:22 +0000 (09:59 -0700)]
copyright 2015
Evan Shelhamer [Tue, 23 Jun 2015 16:54:26 +0000 (09:54 -0700)]
Merge pull request #2579 from lukeyeager/draw-fix
small platform-specific bugfix for draw.py
Sergio Guadarrama [Mon, 22 Jun 2015 15:36:33 +0000 (08:36 -0700)]
Merge pull request #2636 from berleon/master
fixed _force_color check, fixes #2635
berleon [Mon, 22 Jun 2015 12:41:59 +0000 (14:41 +0200)]
fixed _force_color check, fixes #2635