From f2f0a7a73d25cbbd010b1405dc0bfd922d869a9d Mon Sep 17 00:00:00 2001 From: Sergey Karayev Date: Fri, 17 Oct 2014 11:34:25 -0700 Subject: [PATCH] Fixing finetune_flickr_style model reported accuracy. --- examples/finetune_flickr_style/readme.md | 6 +++--- models/finetune_flickr_style/readme.md | 11 ++++++----- 2 files changed, 9 insertions(+), 8 deletions(-) diff --git a/examples/finetune_flickr_style/readme.md b/examples/finetune_flickr_style/readme.md index c4aafbc..ecb9d3d 100644 --- a/examples/finetune_flickr_style/readme.md +++ b/examples/finetune_flickr_style/readme.md @@ -53,7 +53,6 @@ Using a script, we will download a small subset of the data and split it into tr Writing train/val for 1939 successfully downloaded images. This script downloads images and writes train/val file lists into `data/flickr_style`. -With this random seed there are 1,557 train images and 382 test images. The prototxts in this example assume this, and also assume the presence of the ImageNet mean file (run `get_ilsvrc_aux.sh` from `data/ilsvrc12` to obtain this if you haven't yet). We'll also need the ImageNet-trained model, which you can obtain by running `./scripts/download_model_binary.py models/bvlc_reference_caffenet`. @@ -106,7 +105,8 @@ Now we can train! (You can fine-tune in CPU mode by leaving out the `-gpu` flag. I0828 22:23:17.438894 11510 solver.cpp:302] Test net output #0: accuracy = 0.2356 Note how rapidly the loss went down. Although the 23.5% accuracy is only modest, it was achieved in only 1000, and evidence that the model is starting to learn quickly and well. -Once the model is fully fine-tuned on the whole training set over 100,000 iterations the final validation accuracy is 91.64%. This takes ~7 hours in Caffe on a K40 GPU. +Once the model is fully fine-tuned on the whole training set over 100,000 iterations the final validation accuracy is 39.16%. +This takes ~7 hours in Caffe on a K40 GPU. For comparison, here is how the loss goes down when we do not start with a pre-trained model: @@ -155,7 +155,7 @@ Now try fine-tuning to your own tasks and data! ## Trained model -We provide a model trained on all 80K images, with final accuracy of 98%. +We provide a model trained on all 80K images, with final accuracy of 39%. Simply do `./scripts/download_model_binary.py models/finetune_flickr_style` to obtain it. ## License diff --git a/models/finetune_flickr_style/readme.md b/models/finetune_flickr_style/readme.md index c08485f..d2a8a95 100644 --- a/models/finetune_flickr_style/readme.md +++ b/models/finetune_flickr_style/readme.md @@ -3,16 +3,17 @@ name: Finetuning CaffeNet on Flickr Style caffemodel: finetune_flickr_style.caffemodel caffemodel_url: http://dl.caffe.berkeleyvision.org/finetune_flickr_style.caffemodel license: non-commercial -sha1: 443ad95a61fb0b5cd3cee55951bcc1f299186b5e -caffe_commit: 41751046f18499b84dbaf529f64c0e664e2a09fe +sha1: b61b5cef7d771b53b0c488e78d35ccadc073e9cf +caffe_commit: 737ea5e936821b5c69f9c3952d72693ae5843370 gist_id: 034c6ac3865563b69e60 --- This model is trained exactly as described in `docs/finetune_flickr_style/readme.md`, using all 80000 images. -The final performance on the test set: +The final performance: - I0903 18:40:59.211707 11585 caffe.cpp:167] Loss: 0.407405 - I0903 18:40:59.211717 11585 caffe.cpp:179] accuracy = 0.9164 + I1017 07:36:17.370688 31333 solver.cpp:228] Iteration 100000, loss = 0.757952 + I1017 07:36:17.370730 31333 solver.cpp:247] Iteration 100000, Testing net (#0) + I1017 07:36:34.248730 31333 solver.cpp:298] Test net output #0: accuracy = 0.3916 ## License -- 2.7.4