removed mention of getting_pretrained_models page and old paths

docs/index.md

@@ -38,8 +38,8 @@ Slides about the Caffe architecture, *updated 03/14*.
 A 4-page report for the ACM Multimedia Open Source competition.
 - [Installation instructions](/installation.html)<br />
 Tested on Ubuntu, Red Hat, OS X.
-* [Pre-trained models](/getting_pretrained_models.html)<br />
-BVLC provides ready-to-use models for non-commercial use.
+* [Model Zoo](/model_zoo.html)<br />
+BVLC suggests a standard distribution format for Caffe models, and provides trained models for non-commercial use.
 * [Developing & Contributing](/development.html)<br />
 Guidelines for development and contributing to Caffe.
 * [API Documentation](/doxygen/)<br />

docs/model_zoo.md

@@ -1,31 +1,59 @@
 # Caffe Model Zoo
 
+Lots of people have used Caffe to train models of different architectures and applied to different problems, ranging from simple regression to AlexNet-alikes to Siamese networks for image similarity to speech applications.
+
+To lower the friction of sharing these models, we introduce the model zoo framework:
+
+- A standard format for packaging Caffe model info.
+- Tools to upload/download model info to/from Github Gists, and to download trained `.caffemodel` binaries.
+- A central wiki page for sharing model info Gists.
+
+## Where to get trained models
+
+First of all, we provide some trained models out of the box.
+Each one of these can be downloaded by running `scripts/download_model_binary.py <dirname>` where `<dirname>` is specified below:
+
+- **BVLC Reference CaffeNet** in `models/bvlc_reference_caffenet`: AlexNet trained on ILSVRC 2012, with a minor variation from the version as described in the NIPS 2012 paper.
+- **BVLC AlexNet** in `models/bvlc_alexnet`: AlexNet trained on ILSVRC 2012, almost exactly as described in NIPS 2012.
+- **BVLC Reference R-CNN ILSVRC-2013** in `models/bvlc_reference_rcnn_ilsvrc13`: pure Caffe implementation of [R-CNN](https://github.com/rbgirshick/rcnn).
+
+User-provided models are posted to a public-editable [wiki page](https://github.com/BVLC/caffe/wiki/Model-Zoo).
+
+## Model info format
+
 A caffe model is distributed as a directory containing:
-- solver/model prototxt(s)
-- model binary file, with .caffemodel extension
-- readme.md, containing:
-  - YAML header:
-    - model file URL or (torrent magnet link) and MD5 hash
-    - Caffe commit hash use to train this model
-    - [optional] github gist id
-    - license type or text
-  - main body: free-form description/details
-- helpful scripts
 
-It is up to the user where to host the model file.
-Dropbox or their own server are both fine.
+- Solver/model prototxt(s)
+- Readme.md containing
+    - YAML frontmatter
+        - Caffe version used to train this model (tagged release or commit hash).
+        - [optional] file URL and SHA1 of the trained `.caffemodel`.
+        - [optional] github gist id.
+    - Information about what data the model was trained on, explanation of modeling choices, etc.
+    - License information.
+- [optional] Other helpful scripts.
+
+## Hosting model info
+
+Github Gist is a good format for model info distribution because it can contain multiple files, is versionable, and has in-browser syntax highlighting and markdown rendering.
+
+- `scripts/download_model_from_gist.sh <gist_id>`: downloads the non-binary files from a Gist into `<dirname>`
+- `scripts/upload_model_to_gist.sh <dirname>`: uploads non-binary files in the model directory as a Github Gist and prints the Gist ID. If `gist_id` is already part of the `<dirname>/readme.md` frontmatter, then updates existing Gist.
 
-We provide scripts:
+### Hosting trained models
 
-- publish_model_as_gist.sh: uploads non-binary files in the model directory as a Github Gist and returns the id. If gist id is already part of the readme, then updates existing gist.
-- download_model_from_gist.sh <gist_id>: downloads the non-binary files from a Gist.
-- download_model_binary.py: downloads the .caffemodel from the URL specified in readme.
+It is up to the user where to host the `.caffemodel` file.
+We host our BVLC-provided models on our own server.
+Dropbox also works fine (tip: make sure that `?dl=1` is appended to the end of the URL).
 
-The Gist is a good format for distribution because it can contain multiple files, is versionable, and has in-browser syntax highlighting and markdown rendering.
+- `scripts/download_model_binary.py <dirname>`: downloads the `.caffemodel` from the URL specified in the `<dirname>/readme.md` frontmatter and confirms SHA1.
 
-The existing models distributed with Caffe can stay bundled with Caffe, so I am re-working them all into this format.
-All relevant examples will be updated to start with `cd models/model_of_interest && ../scripts/download_model_binary.sh`.
 
 ## Tasks
 
-- get the imagenet example to work with the new prototxt location
+x get the imagenet example to work with the new prototxt location
+x make wiki page for user-submitted models
+- add flickr model to the user-submitted models wiki page
+x make docs section listing bvlc-distributed models
+- write the publish_model_as_gist script
+- write the download_model_from_gist script

examples/classification.ipynb

@@ -3,7 +3,6 @@
   "description": "Use the pre-trained ImageNet model to classify images with the Python interface.",
   "example_name": "ImageNet classification",
   "include_in_docs": true,
-  "signature": "sha256:4f8d4c079c30d20ef4b6818e9672b1741fd1377354e5b83e291710736cecd24f"
  },
  "nbformat": 3,
  "nbformat_minor": 0,
@@ -19,7 +18,7 @@
       "\n",
       "Caffe provides a general Python interface for models with `caffe.Net` in `python/caffe/pycaffe.py`, but to make off-the-shelf classification easy we provide a `caffe.Classifier` class and `classify.py` script. Both Python and MATLAB wrappers are provided. However, the Python wrapper has more features so we will describe it here. For MATLAB, refer to `matlab/caffe/matcaffe_demo.m`.\n",
       "\n",
-      "Before we begin, you must compile Caffe and install the python wrapper by setting your `PYTHONPATH`. If you haven't yet done so, please refer to the [installation instructions](installation.html). This example uses our pre-trained ImageNet model, an ILSVRC12 image classifier. You can download it (232.57MB) by running `examples/imagenet/get_caffe_reference_imagenet_model.sh`. Note that this pre-trained model is licensed for academic research / non-commercial use only.\n",
+      "Before we begin, you must compile Caffe and install the python wrapper by setting your `PYTHONPATH`. If you haven't yet done so, please refer to the [installation instructions](installation.html). This example uses our pre-trained CaffeNet model, an ILSVRC12 image classifier. You can download it by running `./scripts/download_model_binary.py models/bvlc_reference_caffenet`. Note that this pre-trained model is licensed for academic research / non-commercial use only.\n",
       "\n",
       "Ready? Let's start."
      ]
@@ -41,8 +40,8 @@
       "\n",
       "# Set the right path to your model definition file, pretrained model weights,\n",
       "# and the image you would like to classify.\n",
-      "MODEL_FILE = 'imagenet/imagenet_deploy.prototxt'\n",
-      "PRETRAINED = 'imagenet/caffe_reference_imagenet_model'\n",
+      "MODEL_FILE = '../models/bvlc_reference_caffenet/deploy.prototxt'\n",
+      "PRETRAINED = '../models/bvlc_reference_caffenet/bvlc_reference_caffenet.caffemodel'\n",
       "IMAGE_FILE = 'images/cat.jpg'"
      ],
      "language": "python",
@@ -404,4 +403,4 @@
    "metadata": {}
   }
  ]
-}
+}

examples/detection.ipynb

@@ -3,7 +3,6 @@
   "description": "Run a pretrained model as a detector in Python.",
   "example_name": "R-CNN detection",
   "include_in_docs": true,
-  "signature": "sha256:8a744fbbb9ed80acab471247eaf50c27dcbd652105404df9feca599939f0c0ee"
  },
  "nbformat": 3,
  "nbformat_minor": 0,
@@ -26,7 +25,7 @@
       "\n",
       "- [Selective Search](http://koen.me/research/selectivesearch/) is the region proposer used by R-CNN. The [selective_search_ijcv_with_python](https://github.com/sergeyk/selective_search_ijcv_with_python) Python module takes care of extracting proposals through the selective search MATLAB implementation. To install it, download the module and name its directory `selective_search_ijcv_with_python`, run the demo in MATLAB to compile the necessary functions, then add it to your `PYTHONPATH` for importing. (If you have your own region proposals prepared, or would rather not bother with this step, [detect.py](https://github.com/BVLC/caffe/blob/master/python/detect.py) accepts a list of images and bounding boxes as CSV.)\n",
       "\n",
-      "- Follow the [model instructions](http://caffe.berkeleyvision.org/getting_pretrained_models.html) to get the Caffe R-CNN ImageNet model.\n",
+      "-Run `./scripts/download_model_binary.py models/bvlc_reference_caffenet` to get the Caffe R-CNN ImageNet model.\n",
       "\n",
       "With that done, we'll call the bundled `detect.py` to generate the region proposals and run the network. For an explanation of the arguments, do `./detect.py --help`."
      ]
@@ -37,7 +36,7 @@
      "input": [
       "!mkdir -p _temp\n",
       "!echo `pwd`/images/fish-bike.jpg > _temp/det_input.txt\n",
-      "!../python/detect.py --crop_mode=selective_search --pretrained_model=imagenet/caffe_rcnn_imagenet_model --model_def=imagenet/rcnn_imagenet_deploy.prototxt --gpu --raw_scale=255 _temp/det_input.txt _temp/det_output.h5"
+      "!../python/detect.py --crop_mode=selective_search --pretrained_model=models/bvlc_reference_rcnn_ilsvrc13/bvlc_reference_rcnn_ilsvrc13.caffemodel --model_def=models/bvlc_reference_rcnn_ilsvrc13/deploy.prototxt --gpu --raw_scale=255 _temp/det_input.txt _temp/det_output.h5"
      ],
      "language": "python",
      "metadata": {},

examples/filter_visualization.ipynb

@@ -3,7 +3,6 @@
   "description": "Extracting features and visualizing trained filters with an example image, viewed layer-by-layer.",
   "example_name": "Filter visualization",
   "include_in_docs": true,
-  "signature": "sha256:b1b0457e2b10110aca847a718a3fe631ebcfce63a61cbc33653244f52b1ff4af"
  },
  "nbformat": 3,
  "nbformat_minor": 0,
@@ -54,15 +53,15 @@
      "cell_type": "markdown",
      "metadata": {},
      "source": [
-      "Follow the [instructions](http://caffe.berkeleyvision.org/getting_pretrained_models.html) for getting the pretrained models, load the net, specify test phase and CPU mode, and configure input preprocessing."
+      "Run `./scripts/download_model_binary.py models/bvlc_reference_caffenet` to get the pretrained CaffeNet model, load the net, specify test phase and CPU mode, and configure input preprocessing."
      ]
     },
     {
      "cell_type": "code",
      "collapsed": false,
      "input": [
-      "net = caffe.Classifier(caffe_root + 'examples/imagenet/imagenet_deploy.prototxt',\n",
-      "                       caffe_root + 'examples/imagenet/caffe_reference_imagenet_model')\n",
+      "net = caffe.Classifier(caffe_root + 'models/bvlc_reference_caffenet/deploy.prototxt',\n",
+      "                       caffe_root + 'models/bvlc_reference_caffenet/bvlc_reference_caffenet.caffemodel')\n",
       "net.set_phase_test()\n",
       "net.set_mode_cpu()\n",
       "# input preprocessing: 'data' is the name of the input blob == net.inputs[0]\n",
@@ -598,4 +597,4 @@
    "metadata": {}
   }
  ]
-}
+}

examples/finetune_flickr_style/readme.md

@@ -60,13 +60,13 @@ We'll also need the ImageNet-trained model, which you can obtain by running `get
 
 Now we can train! (You can fine-tune in CPU mode by leaving out the `-gpu` flag.)
 
-    caffe % ./build/tools/caffe train -solver examples/finetune_flickr_style/flickr_style_solver.prototxt -weights examples/imagenet/caffe_reference_imagenet_model -gpu 0
+    caffe % ./build/tools/caffe train -solver examples/finetune_flickr_style/flickr_style_solver.prototxt -weights models/bvlc_reference_caffenet/bvlc_reference_caffenet.caffemodel -gpu 0
 
     [...]
 
     I0828 22:10:04.025378  9718 solver.cpp:46] Solver scaffolding done.
     I0828 22:10:04.025388  9718 caffe.cpp:95] Use GPU with device ID 0
-    I0828 22:10:04.192004  9718 caffe.cpp:107] Finetuning from examples/imagenet/caffe_reference_imagenet_model
+    I0828 22:10:04.192004  9718 caffe.cpp:107] Finetuning from models/bvlc_reference_caffenet/bvlc_reference_caffenet.caffemodel
 
     [...]
 

examples/imagenet/imagenet_full_conv.prototxt

@@ -1,3 +1,4 @@
+# This file is for the net_surgery.ipynb example notebook.
 name: "CaffeNetConv"
 input: "data"
 input_dim: 1