You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@systemml.apache.org by ni...@apache.org on 2017/04/19 22:34:48 UTC
[1/2] incubator-systemml git commit: [SYSTEMML-692] Added initial
version of DML generator for Caffe
Repository: incubator-systemml
Updated Branches:
refs/heads/gh-pages b91f9bfec -> c4918f5b6
[SYSTEMML-692] Added initial version of DML generator for Caffe
This experimental interface is called Caffe2DML and doesnot affect other functionality.
- Updated the interface to match the Caffe specification as per
@bertholdreinwald 's suggestion.
- Added support for fine-tuning.
- Added support for explain, statistics and gpu.
Closes #422.
Project: http://git-wip-us.apache.org/repos/asf/incubator-systemml/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-systemml/commit/48ba441e
Tree: http://git-wip-us.apache.org/repos/asf/incubator-systemml/tree/48ba441e
Diff: http://git-wip-us.apache.org/repos/asf/incubator-systemml/diff/48ba441e
Branch: refs/heads/gh-pages
Commit: 48ba441ee3d3ad440322c72c6a3f8a55dfcd84bc
Parents: b91f9bf
Author: Niketan Pansare <np...@us.ibm.com>
Authored: Wed Apr 19 14:07:44 2017 -0800
Committer: Niketan Pansare <np...@us.ibm.com>
Committed: Wed Apr 19 15:07:43 2017 -0700
----------------------------------------------------------------------
beginners-guide-caffe2dml.md | 124 ++++++++++++++++++++++++++++++++++++++
devdocs/deep-learning.md | 84 ++++++++++++++++++++++++++
2 files changed, 208 insertions(+)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/incubator-systemml/blob/48ba441e/beginners-guide-caffe2dml.md
----------------------------------------------------------------------
diff --git a/beginners-guide-caffe2dml.md b/beginners-guide-caffe2dml.md
new file mode 100644
index 0000000..cfcc0cb
--- /dev/null
+++ b/beginners-guide-caffe2dml.md
@@ -0,0 +1,124 @@
+---
+layout: global
+title: Beginner's Guide for Caffe2DML users
+description: Beginner's Guide for Caffe2DML users
+---
+<!--
+{% comment %}
+Licensed to the Apache Software Foundation (ASF) under one or more
+contributor license agreements. See the NOTICE file distributed with
+this work for additional information regarding copyright ownership.
+The ASF licenses this file to you under the Apache License, Version 2.0
+(the "License"); you may not use this file except in compliance with
+the License. You may obtain a copy of the License at
+
+http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+{% endcomment %}
+-->
+
+* This will become a table of contents (this text will be scraped).
+{:toc}
+
+<br/>
+
+## Introduction
+
+Caffe2DML is an experimental API that converts an Caffe specification to DML.
+
+## Frequently asked questions
+
+- How to set batch size ?
+
+Batch size is set in `data_param` of the Data layer:
+
+ layer {
+ name: "mnist"
+ type: "Data"
+ top: "data"
+ top: "label"
+ data_param {
+ source: "mnist_train"
+ batch_size: 64
+ backend: LMDB
+ }
+ }
+
+- How to set maximum number of iterations for training ?
+
+Caffe allows you to set the maximum number of iterations in solver specification
+
+ # The maximum number of iterations
+ max_iter: 2000
+
+- How to set the size of the validation dataset ?
+
+The size of the validation dataset is determined by the parameters `test_iter` and the batch size. For example: If the batch size is 64 and
+`test_iter` is 10, then the validation size is 640. This setting generates following DML code internally:
+
+ num_images = nrow(y_full)
+ BATCH_SIZE = 64
+ num_validation = 10 * BATCH_SIZE
+ X = X_full[(num_validation+1):num_images,]; y = y_full[(num_validation+1):num_images,]
+ X_val = X_full[1:num_validation,]; y_val = y_full[1:num_validation,]
+ num_images = nrow(y)
+
+- How to monitor loss via command-line ?
+
+To monitor loss, please set following parameters in the solver specification
+
+ # Display training loss and accuracy every 100 iterations
+ display: 100
+ # Carry out validation every 500 training iterations and display validation loss and accuracy.
+ test_iter: 10
+ test_interval: 500
+
+ - How to pass a single jpeg image to Caffe2DML for prediction ?
+
+ from PIL import Image
+ import systemml as sml
+ from systemml.mllearn import Caffe2DML
+ img_shape = (3, 224, 224)
+ input_image = sml.convertImageToNumPyArr(Image.open(img_file_path), img_shape=img_shape)
+ resnet = Caffe2DML(sqlCtx, solver='ResNet_50_solver.proto', weights='ResNet_50_pretrained_weights', input_shape=img_shape)
+ resnet.predict(input_image)
+
+- How to prepare a directory of jpeg images for training with Caffe2DML ?
+
+The below example assumes that the input dataset has 2 labels `cat` and `dogs` and the filename has these labels as prefix.
+We iterate through the directory and convert each jpeg image into pyspark.ml.linalg.Vector using pyspark.
+These vectors are stored as DataFrame and randomized using Spark SQL's `orderBy(rand())` function.
+The DataFrame is then saved in parquet format to reduce the cost of preprocessing for repeated training.
+
+ from systemml.mllearn import Caffe2DML
+ from pyspark.sql import SQLContext
+ import numpy as np
+ import urllib, os, scipy.ndimage
+ from pyspark.ml.linalg import Vectors
+ from pyspark import StorageLevel
+ import systemml as sml
+ from pyspark.sql.functions import rand
+ # ImageNet specific parameters
+ img_shape = (3, 224, 224)
+ train_dir = '/home/biuser/dogs_vs_cats/train'
+ def getLabelFeatures(filename):
+ from PIL import Image
+ vec = Vectors.dense(sml.convertImageToNumPyArr(Image.open(os.path.join(train_dir, filename)), img_shape=img_shape)[0,:])
+ if filename.lower().startswith('cat'):
+ return (1, vec)
+ elif filename.lower().startswith('dog'):
+ return (2, vec)
+ else:
+ raise ValueError('Expected the filename to start with either cat or dog')
+
+ list_jpeg_files = os.listdir(train_dir)
+ # 10 files per partition
+ train_df = sc.parallelize(list_jpeg_files, int(len(list_jpeg_files)/10)).map(lambda filename : getLabelFeatures(filename)).toDF(['label', 'features']).orderBy(rand())
+ # Optional: but helps seperates conversion-related from training
+ # Alternatively, this dataframe can be passed directly to `caffe2dml_model.fit(train_df)`
+ train_df.write.parquet('kaggle-cats-dogs.parquet')
\ No newline at end of file
http://git-wip-us.apache.org/repos/asf/incubator-systemml/blob/48ba441e/devdocs/deep-learning.md
----------------------------------------------------------------------
diff --git a/devdocs/deep-learning.md b/devdocs/deep-learning.md
index 1fb951a..329c6c8 100644
--- a/devdocs/deep-learning.md
+++ b/devdocs/deep-learning.md
@@ -139,3 +139,87 @@ updates for the image:
|-----------------|---------------------------------|-----------------|
| `w3*y1 + w1*y3` | `w4*y1 + w3*y2 + w2*y3 + w1*y4` | `w4*y2 + w2*y4` |
| `w3*y3` | `w4*y3 + w3*y4` | `w4*y4` |
+
+# Caffe2DML examples
+
+## Training using Caffe models on Lenet
+
+The below script also demonstrates how to save the trained model.
+
+```python
+# Download the MNIST dataset
+from mlxtend.data import mnist_data
+import numpy as np
+from sklearn.utils import shuffle
+X, y = mnist_data()
+X, y = shuffle(X, y)
+num_classes = np.unique(y).shape[0]
+img_shape = (1, 28, 28)
+
+# Split the data into training and test
+n_samples = len(X)
+X_train = X[:int(.9 * n_samples)]
+y_train = y[:int(.9 * n_samples)]
+X_test = X[int(.9 * n_samples):]
+y_test = y[int(.9 * n_samples):]
+
+# Download the Lenet network
+import urllib
+urllib.urlretrieve('https://raw.githubusercontent.com/niketanpansare/model_zoo/master/caffe/vision/lenet/mnist/lenet.proto', 'lenet.proto')
+urllib.urlretrieve('https://raw.githubusercontent.com/niketanpansare/model_zoo/master/caffe/vision/lenet/mnist/lenet_solver.proto', 'lenet_solver.proto')
+
+# Train Lenet On MNIST using scikit-learn like API
+from systemml.mllearn import Caffe2DML
+lenet = Caffe2DML(sqlCtx, solver='lenet_solver.proto').set(max_iter=500, debug=True).setStatistics(True)
+print('Lenet score: %f' % lenet.fit(X_train, y_train).score(X_test, y_test))
+
+# Save the trained model
+lenet.save('lenet_model')
+```
+
+## Load the trained model and retrain (i.e. finetuning)
+
+```python
+# Fine-tune the existing trained model
+new_lenet = Caffe2DML(sqlCtx, solver='lenet_solver.proto', weights='lenet_model').set(max_iter=500, debug=True)
+new_lenet.fit(X_train, y_train)
+new_lenet.save('lenet_model')
+```
+
+## Perform prediction using the above trained model
+
+```python
+# Use the new model for prediction
+predict_lenet = Caffe2DML(sqlCtx, solver='lenet_solver.proto', weights='lenet_model')
+print('Lenet score: %f' % predict_lenet.score(X_test, y_test))
+```
+
+Similarly, you can perform prediction using the pre-trained ResNet network
+
+```python
+from systemml.mllearn import Caffe2DML
+from pyspark.sql import SQLContext
+import numpy as np
+import urllib, os, scipy.ndimage
+from PIL import Image
+import systemml as sml
+
+# ImageNet specific parameters
+img_shape = (3, 224, 224)
+
+# Downloads a jpg image, resizes it to 224 and return as numpy array in N X CHW format
+url = 'https://upload.wikimedia.org/wikipedia/commons/thumb/5/58/MountainLion.jpg/312px-MountainLion.jpg'
+outFile = 'test.jpg'
+urllib.urlretrieve(url, outFile)
+input_image = sml.convertImageToNumPyArr(Image.open(outFile), img_shape=img_shape)
+
+# Download the ResNet network
+import urllib
+urllib.urlretrieve('https://raw.githubusercontent.com/niketanpansare/model_zoo/master/caffe/vision/resnet/ilsvrc12/ResNet_50_network.proto', 'ResNet_50_network.proto')
+urllib.urlretrieve('https://raw.githubusercontent.com/niketanpansare/model_zoo/master/caffe/vision/resnet/ilsvrc12/ResNet_50_solver.proto', 'ResNet_50_solver.proto')
+
+# Assumes that you have cloned the model_zoo repository
+# git clone https://github.com/niketanpansare/model_zoo.git
+resnet = Caffe2DML(sqlCtx, solver='ResNet_50_solver.proto', weights='~/model_zoo/caffe/vision/resnet/ilsvrc12/ResNet_50_pretrained_weights').set(input_shape=img_shape)
+resnet.predict(input_image)
+```
\ No newline at end of file
[2/2] incubator-systemml git commit: [MINOR] Updated the algorithm
documentation to clearly mark the "Argument" section.
Posted by ni...@apache.org.
[MINOR] Updated the algorithm documentation to clearly mark the "Argument" section.
Project: http://git-wip-us.apache.org/repos/asf/incubator-systemml/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-systemml/commit/c4918f5b
Tree: http://git-wip-us.apache.org/repos/asf/incubator-systemml/tree/c4918f5b
Diff: http://git-wip-us.apache.org/repos/asf/incubator-systemml/diff/c4918f5b
Branch: refs/heads/gh-pages
Commit: c4918f5b64befaebfb0cbd47c02c50cb84642a12
Parents: 48ba441
Author: Niketan Pansare <np...@us.ibm.com>
Authored: Wed Apr 19 14:20:00 2017 -0800
Committer: Niketan Pansare <np...@us.ibm.com>
Committed: Wed Apr 19 15:21:39 2017 -0700
----------------------------------------------------------------------
algorithms-classification.md | 12 ++++++------
algorithms-regression.md | 10 +++++-----
2 files changed, 11 insertions(+), 11 deletions(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/incubator-systemml/blob/c4918f5b/algorithms-classification.md
----------------------------------------------------------------------
diff --git a/algorithms-classification.md b/algorithms-classification.md
index 0ee43bf..b029e0a 100644
--- a/algorithms-classification.md
+++ b/algorithms-classification.md
@@ -180,7 +180,7 @@ val prediction = model.transform(X_test_df)
</div>
</div>
-### Arguments
+### Arguments for Spark and Hadoop invocation
**X**: Location (on HDFS) to read the input matrix of feature vectors; each row
constitutes one feature vector.
@@ -592,7 +592,7 @@ val prediction = model.transform(X_test_df)
</div>
</div>
-#### Arguments
+#### Arguments for Spark and Hadoop invocation
**X**: Location (on HDFS) to read the matrix of feature vectors; each
row constitutes one feature vector.
@@ -861,7 +861,7 @@ val prediction = model.transform(X_test_df)
</div>
-#### Arguments
+#### Arguments for Spark and Hadoop invocation
**X**: Location (on HDFS) containing the explanatory variables in
a matrix. Each row constitutes an example.
@@ -1212,7 +1212,7 @@ val prediction = model.transform(X_test_df)
</div>
-### Arguments
+### Arguments for Spark and Hadoop invocation
**X**: Location (on HDFS) to read the matrix of feature vectors; each
row constitutes one feature vector.
@@ -1472,7 +1472,7 @@ implementation is well-suited to handle large-scale data and builds a
</div>
-### Arguments
+### Arguments for Spark and Hadoop invocation
**X**: Location (on HDFS) to read the matrix of feature vectors; each row
constitutes one feature vector. Note that categorical features in $X$
@@ -1887,7 +1887,7 @@ for classification in parallel.
</div>
-### Arguments
+### Arguments for Spark and Hadoop invocation
**X**: Location (on HDFS) to read the matrix of feature vectors; each row
constitutes one feature vector. Note that categorical features in $X$
http://git-wip-us.apache.org/repos/asf/incubator-systemml/blob/c4918f5b/algorithms-regression.md
----------------------------------------------------------------------
diff --git a/algorithms-regression.md b/algorithms-regression.md
index 80b38a3..31f7ec2 100644
--- a/algorithms-regression.md
+++ b/algorithms-regression.md
@@ -168,7 +168,7 @@ y_test = lr.fit(df_train)
</div>
-### Arguments
+### Arguments for Spark and Hadoop invocation
**X**: Location (on HDFS) to read the matrix of feature vectors, each row
constitutes one feature vector
@@ -570,7 +570,7 @@ lowest AIC is computed.
</div>
</div>
-### Arguments
+### Arguments for Spark and Hadoop invocation
**X**: Location (on HDFS) to read the matrix of feature vectors, each row
contains one feature vector.
@@ -782,7 +782,7 @@ distributions and link functions, see below for details.
</div>
</div>
-### Arguments
+### Arguments for Spark and Hadoop invocation
**X**: Location (on HDFS) to read the matrix of feature vectors; each row
constitutes an example.
@@ -1255,7 +1255,7 @@ distribution family is supported (see below for details).
</div>
-### Arguments
+### Arguments for Spark and Hadoop invocation
**X**: Location (on HDFS) to read the matrix of feature vectors; each row is an
example.
@@ -1503,7 +1503,7 @@ this step outside the scope of `GLM-predict.dml` for now.
</div>
-### Arguments
+### Arguments for Spark and Hadoop invocation
**X**: Location (on HDFS) to read the $n\,{\times}\,m$-matrix $X$ of feature
vectors, each row constitutes one feature vector (one record)