You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@systemml.apache.org by du...@apache.org on 2017/06/03 01:40:03 UTC
incubator-systemml git commit: [MINOR] Updating the MNIST LeNet
example notebook.
Repository: incubator-systemml
Updated Branches:
refs/heads/master 33ebe969b -> 766cc48c0
[MINOR] Updating the MNIST LeNet example notebook.
This improves the nomenclature & documentation, adds more cleanup,
removes the unnecessary saving of the MNIST data to CSV files, and
makes everything more Pythonic.
Project: http://git-wip-us.apache.org/repos/asf/incubator-systemml/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-systemml/commit/766cc48c
Tree: http://git-wip-us.apache.org/repos/asf/incubator-systemml/tree/766cc48c
Diff: http://git-wip-us.apache.org/repos/asf/incubator-systemml/diff/766cc48c
Branch: refs/heads/master
Commit: 766cc48c00adba37fde9a34a3a06f6df8708205d
Parents: 33ebe96
Author: Mike Dusenberry <mw...@us.ibm.com>
Authored: Fri Jun 2 18:37:06 2017 -0700
Committer: Mike Dusenberry <mw...@us.ibm.com>
Committed: Fri Jun 2 18:37:06 2017 -0700
----------------------------------------------------------------------
.../Deep_Learning_Image_Classification.ipynb | 270 ++++++-------------
1 file changed, 88 insertions(+), 182 deletions(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/incubator-systemml/blob/766cc48c/samples/jupyter-notebooks/Deep_Learning_Image_Classification.ipynb
----------------------------------------------------------------------
diff --git a/samples/jupyter-notebooks/Deep_Learning_Image_Classification.ipynb b/samples/jupyter-notebooks/Deep_Learning_Image_Classification.ipynb
index 3124028..fa1e4b8 100644
--- a/samples/jupyter-notebooks/Deep_Learning_Image_Classification.ipynb
+++ b/samples/jupyter-notebooks/Deep_Learning_Image_Classification.ipynb
@@ -6,20 +6,18 @@
"source": [
"# Deep Learning Image Classification using Apache SystemML\n",
"\n",
- "This notebook shows SystemML Deep Learning functionality to map images of single digit numbers to their corresponding numeric representations. See [Getting Started with Deep Learning and Python](http://www.pyimagesearch.com/2014/09/22/getting-started-deep-learning-python/) for an explanation of the used deep learning concepts and assumptions.\n",
+ "This notebook demonstrates how to train a deep learning model on SystemML for the classic [MNIST](http://yann.lecun.com/exdb/mnist/) problem of mapping images of single digit numbers to their corresponding numeric representations, using a classic [LeNet](http://yann.lecun.com/exdb/lenet/)-like convolutional neural network model. See [Neural Networks and Deep Learning](http://neuralnetworksanddeeplearning.com/chap6.html) for more information on neural networks and deep learning.\n",
"\n",
- "The downloaded MNIST dataset contains labeled images of handwritten digits, where each example is a 28x28 pixel image of grayscale values in the range [0,255] stretched out as 784 pixels, and each label is one of 10 possible digits in [0,9]. We download 60,000 training examples, and 10,000 test examples, where the format is \"label, pixel_1, pixel_2, ..., pixel_n\". We train a SystemML LeNet model. The results of the learning algorithms have an accuracy of 98 percent.\n",
+ "The downloaded MNIST dataset contains labeled images of handwritten digits, where each example is a 28x28 pixel image of grayscale values in the range [0,255] stretched out as 784 pixels, and each label is one of 10 possible digits in [0,9]. We download 60,000 training examples, and 10,000 test examples, where the images and labels are stored in separate matrices. We then train a SystemML LeNet-like convolutional neural network (i.e. \"convnet\", \"CNN\") model. The resulting trained model has an accuracy of 98.6% on the test dataset.\n",
"\n",
- "1. [Download and Access MNIST data](#access_data)\n",
+ "1. [Download the MNIST data](#download_data)\n",
"1. [Train a CNN classifier for MNIST handwritten digits](#train)\n",
- "1. [Detect handwritten Digits](#predict)\n"
+ "1. [Detect handwritten Digits](#predict)"
]
},
{
"cell_type": "markdown",
- "metadata": {
- "collapsed": true
- },
+ "metadata": {},
"source": [
"<div style=\"text-align:center\" markdown=\"1\">\n",
"\n",
@@ -31,14 +29,14 @@
"cell_type": "markdown",
"metadata": {},
"source": [
- "### This notebook is supported with SystemML 0.14.0 and above."
+ "### Note: This notebook is supported with SystemML 0.14.0 and above."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
- "scrolled": false
+ "collapsed": true
},
"outputs": [],
"source": [
@@ -51,61 +49,36 @@
"metadata": {},
"outputs": [],
"source": [
- "from systemml import MLContext, dml\n",
- "\n",
- "ml = MLContext(sc)\n",
+ "%matplotlib inline\n",
"\n",
- "print (\"Spark Version:\" + sc.version)\n",
- "print (\"SystemML Version:\" + ml.version())\n",
- "print (\"SystemML Built-Time:\" + ml.buildTime())"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "metadata": {
- "scrolled": true
- },
- "outputs": [],
- "source": [
- "import warnings\n",
- "warnings.filterwarnings(\"ignore\")\n",
- "from sklearn import datasets\n",
- "from sklearn.cross_validation import train_test_split\n",
- "from sklearn.metrics import classification_report\n",
- "import pandas as pd\n",
- "import numpy as np\n",
"import matplotlib.pyplot as plt\n",
- "#import matplotlib.image as mpimg\n",
- "%matplotlib inline"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "Create data directory."
+ "import numpy as np\n",
+ "import pandas as pd\n",
+ "from sklearn import datasets\n",
+ "from sklearn.cross_validation import train_test_split # module deprecated in 0.18\n",
+ "#from sklearn.model_selection import train_test_split # use this module for >=0.18\n",
+ "from sklearn import metrics\n",
+ "from systemml import MLContext, dml"
]
},
{
"cell_type": "code",
"execution_count": null,
- "metadata": {
- "collapsed": true
- },
+ "metadata": {},
"outputs": [],
"source": [
- "%%sh\n",
- "mkdir -p data/mnist/\n",
- "cd data/mnist/"
+ "ml = MLContext(sc)\n",
+ "print(\"Spark Version: {}\".format(sc.version))\n",
+ "print(\"SystemML Version: {}\".format(ml.version()))\n",
+ "print(\"SystemML Built-Time: {}\".format(ml.buildTime()))"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
- "<a id=\"access_data\"></a>\n",
- "## Download and Access MNIST data\n",
+ "<a id=\"download_data\"></a>\n",
+ "## Download the MNIST data\n",
"\n",
"Download the [MNIST data from the MLData repository](http://mldata.org/repository/data/viewslug/mnist-original/), and then split and save."
]
@@ -113,87 +86,28 @@
{
"cell_type": "code",
"execution_count": null,
- "metadata": {
- "scrolled": true
- },
+ "metadata": {},
"outputs": [],
"source": [
"mnist = datasets.fetch_mldata(\"MNIST Original\")\n",
"\n",
- "print (\"Mnist data features:\" + str(mnist.data.shape))\n",
- "print (\"Mnist data label:\" + str(mnist.target.shape))\n",
- "\n",
- "trainX, testX, trainY, testY = train_test_split(mnist.data, mnist.target.astype(\"int0\"), test_size = 0.142857)\n",
- "\n",
- "trainD = np.concatenate((trainY.reshape(trainY.size, 1), trainX),axis=1)\n",
- "testD = np.concatenate((testY.reshape (testY.size, 1), testX),axis=1)\n",
+ "print(\"MNIST data features: {}\".format(mnist.data.shape))\n",
+ "print(\"MNIST data labels: {}\".format(mnist.target.shape))\n",
"\n",
- "print (\"Images for training:\" + str(trainD.shape))\n",
- "print (\"Images used for testing:\" + str(testD.shape))\n",
- "pix = int(np.sqrt(trainD.shape[1]))\n",
- "print (\"Each image is: \" + str(pix) + \" by \" + str(pix) + \" pixels\")\n",
+ "X_train, X_test, y_train, y_test = train_test_split(\n",
+ " mnist.data, mnist.target.astype(np.uint8).reshape(-1, 1),\n",
+ " test_size = 10000)\n",
"\n",
- "np.savetxt('data/mnist/mnist_train.csv', trainD, fmt='%u', delimiter=\",\")\n",
- "np.savetxt('data/mnist/mnist_test.csv', testD, fmt='%u', delimiter=\",\")"
+ "print(\"Training images, labels: {}, {}\".format(X_train.shape, y_train.shape))\n",
+ "print(\"Testing images, labels: {}, {}\".format(X_test.shape, y_test.shape))\n",
+ "print(\"Each image is: {0:d}x{0:d} pixels\".format(int(np.sqrt(X_train.shape[1]))))"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
- "Alternatively get the data from here. (Uncomment curl commands from following cell if you want to download using following approach)"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "metadata": {
- "scrolled": true
- },
- "outputs": [],
- "source": [
- "%%sh\n",
- "cd data/mnist\n",
- "# curl -O https://pjreddie.com/media/files/mnist_train.csv\n",
- "# curl -O https://pjreddie.com/media/files/mnist_test.csv\n",
- "wc -l mnist*"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "### Read the data."
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "metadata": {},
- "outputs": [],
- "source": [
- "trainData = np.genfromtxt('data/mnist/mnist_train.csv', delimiter=\",\")\n",
- "testData = np.genfromtxt('data/mnist/mnist_test.csv', delimiter=\",\")\n",
- "\n",
- "print (\"Training data: \" + str(trainData.shape))\n",
- "print (\"Test data: \" + str(testData.shape))"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "metadata": {},
- "outputs": [],
- "source": [
- "pd.set_option('display.max_columns', 200)\n",
- "pd.DataFrame(testData[1:10,],dtype='uint')"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "### Following command is not required for code above SystemML 0.14 (master branch dated 05/15/2017 or later)"
+ "### Note: The following command is not required for code above SystemML 0.14 (master branch dated 05/15/2017 or later)."
]
},
{
@@ -210,7 +124,7 @@
"metadata": {},
"source": [
"<a id=\"train\"></a>\n",
- "## Develop LeNet CNN classifier on Training Data"
+ "## Train a LeNet-like CNN classifier on the training data"
]
},
{
@@ -227,37 +141,23 @@
"cell_type": "markdown",
"metadata": {},
"source": [
- "### Train Model using SystemML LeNet CNN."
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "(on a Mac Book, this takes approx. 5-6 mins for 1 epoch)"
+ "### Train a LeNet-like CNN model using SystemML"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
- "collapsed": true,
- "scrolled": true
+ "collapsed": true
},
"outputs": [],
"source": [
"script = \"\"\"\n",
" source(\"nn/examples/mnist_lenet.dml\") as mnist_lenet\n",
"\n",
- " # Bind training data\n",
- " n = nrow(data)\n",
- "\n",
- " # Extract images and labels\n",
- " images = data[,2:ncol(data)]\n",
- " labels = data[,1]\n",
- "\n",
" # Scale images to [-1,1], and one-hot encode the labels\n",
- " images = (images / 255.0) * 2 - 1\n",
+ " images = (images / 255) * 2 - 1\n",
+ " n = nrow(images)\n",
" labels = table(seq(1, n), labels+1, n, 10)\n",
"\n",
" # Split into training (55,000 examples) and validation (5,000 examples)\n",
@@ -266,49 +166,56 @@
" y = labels[5001:nrow(images),]\n",
" y_val = labels[1:5000,]\n",
"\n",
- " # Train the model using channel, height, and width to produce weights/biases.\n",
+ " # Train the model to produce weights & biases.\n",
" [W1, b1, W2, b2, W3, b3, W4, b4] = mnist_lenet::train(X, y, X_val, y_val, C, Hin, Win, epochs)\n",
"\"\"\"\n",
- "rets = ('W1', 'b1','W2','b2','W3','b3','W4','b4')\n",
+ "out = ('W1', 'b1', 'W2', 'b2', 'W3', 'b3', 'W4', 'b4')\n",
+ "prog = (dml(script).input(images=X_train, labels=y_train, epochs=1, C=1, Hin=28, Win=28)\n",
+ " .output(*out))\n",
"\n",
- "script = (dml(script).input(data=trainData, epochs=1, C=1, Hin=28, Win=28)\n",
- " .output(*rets)) \n",
- "\n",
- "W1, b1, W2, b2, W3, b3, W4, b4 = (ml.execute(script).get(*rets))"
+ "W1, b1, W2, b2, W3, b3, W4, b4 = ml.execute(prog).get(*out)"
]
},
{
"cell_type": "markdown",
- "metadata": {
- "collapsed": true
- },
+ "metadata": {},
"source": [
- "Use trained model and predict on test data, and evaluate the quality of the predictions for each digit."
+ "Use the trained model to make predictions for the test data, and evaluate the quality of the predictions."
]
},
{
"cell_type": "code",
"execution_count": null,
- "metadata": {},
+ "metadata": {
+ "collapsed": true
+ },
"outputs": [],
"source": [
- "scriptPredict = \"\"\"\n",
+ "script_predict = \"\"\"\n",
" source(\"nn/examples/mnist_lenet.dml\") as mnist_lenet\n",
"\n",
- " # Separate images from lables and scale images to [-1,1]\n",
- " X_test = data[,2:ncol(data)]\n",
- " X_test = (X_test / 255.0) * 2 - 1\n",
+ " # Scale images to [-1,1]\n",
+ " X_test = (X_test / 255) * 2 - 1\n",
"\n",
" # Predict\n",
- " probs = mnist_lenet::predict(X_test, C, Hin, Win, W1, b1, W2, b2, W3, b3, W4, b4)\n",
- " predictions = rowIndexMax(probs) - 1\n",
+ " y_prob = mnist_lenet::predict(X_test, C, Hin, Win, W1, b1, W2, b2, W3, b3, W4, b4)\n",
+ " y_pred = rowIndexMax(y_prob) - 1\n",
"\"\"\"\n",
- "script = (dml(scriptPredict).input(data=testData, C=1, Hin=28, Win=28, W1=W1, b1=b1, W2=W2, b2=b2, W3=W3, b3=b3, W4=W4, b4=b4)\n",
- " .output(\"predictions\"))\n",
+ "prog = (dml(script_predict).input(X_test=X_test, C=1, Hin=28, Win=28, W1=W1, b1=b1,\n",
+ " W2=W2, b2=b2, W3=W3, b3=b3, W4=W4, b4=b4)\n",
+ " .output(\"y_pred\"))\n",
"\n",
- "predictions = ml.execute(script).get(\"predictions\").toNumPy()\n",
- "\n",
- "print (classification_report(testData[:,0], predictions))"
+ "y_pred = ml.execute(prog).get(\"y_pred\").toNumPy()"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "print(metrics.accuracy_score(y_test, y_pred))\n",
+ "print(metrics.classification_report(y_test, y_pred))"
]
},
{
@@ -316,14 +223,14 @@
"metadata": {},
"source": [
"<a id=\"predict\"></a>\n",
- "## Detect handwritten Digits"
+ "## Detect handwritten digits"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
- "Define a function that randomly selects a test image, display the image, and scores it."
+ "Define a function that randomly selects a test image, displays the image, and scores it."
]
},
{
@@ -334,11 +241,11 @@
},
"outputs": [],
"source": [
- "img_size = int(np.sqrt(testData.shape[1] - 1))\n",
+ "img_size = int(np.sqrt(X_test.shape[1]))\n",
"\n",
"def displayImage(i):\n",
- " image = (testData[i,1:]).reshape((img_size, img_size)).astype(\"uint8\")\n",
- " imgplot = plt.imshow(image, cmap='gray') "
+ " image = (X_test[i]).reshape(img_size, img_size).astype(np.uint8)\n",
+ " imgplot = plt.imshow(image, cmap='gray') "
]
},
{
@@ -350,28 +257,27 @@
"outputs": [],
"source": [
"def predictImage(i):\n",
- " image = testData[i,:].reshape(1,testData.shape[1])\n",
- " prog = dml(scriptPredict).input(data=image, C=1, Hin=28, Win=28, W1=W1, b1=b1, W2=W2, b2=b2, W3=W3, b3=b3, W4=W4, b4=b4) \\\n",
- " .output(\"predictions\")\n",
- " result = ml.execute(prog)\n",
- " return (result.get(\"predictions\").toNumPy())[0]"
+ " image = X_test[i].reshape(1, -1)\n",
+ " out = \"y_pred\"\n",
+ " prog = (dml(script_predict).input(X_test=image, C=1, Hin=28, Win=28, W1=W1, b1=b1,\n",
+ " W2=W2, b2=b2, W3=W3, b3=b3, W4=W4, b4=b4)\n",
+ " .output(out))\n",
+ " pred = int(ml.execute(prog).get(out).toNumPy())\n",
+ " return pred"
]
},
{
"cell_type": "code",
"execution_count": null,
- "metadata": {
- "scrolled": true
- },
+ "metadata": {},
"outputs": [],
"source": [
- "i = np.random.choice(np.arange(0, len(testData)), size = (1,))\n",
- "\n",
+ "i = np.random.randint(len(X_test))\n",
"p = predictImage(i)\n",
"\n",
- "print (\"Image \" + str(i) + \"\\nPredicted digit: \" + str(p) + \"\\nActual digit: \" + str(testData[i,0]) + \"\\nResult: \" + str(p == testData[i,0]))\n",
+ "print(\"Image {}\\nPredicted digit: {}\\nActual digit: {}\\nResult: {}\".format(\n",
+ " i, p, int(y_test[i]), p == int(y_test[i])))\n",
"\n",
- "p\n",
"displayImage(i)"
]
},
@@ -382,27 +288,27 @@
"outputs": [],
"source": [
"pd.set_option('display.max_columns', 28)\n",
- "pd.DataFrame((testData[i,1:]).reshape(img_size, img_size),dtype='uint')"
+ "pd.DataFrame((X_test[i]).reshape(img_size, img_size), dtype='uint')"
]
}
],
"metadata": {
"kernelspec": {
- "display_name": "Python 2",
+ "display_name": "Python 3 + Spark 2.x + SystemML",
"language": "python",
- "name": "python2"
+ "name": "pyspark3_2.x"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
- "version": 2
+ "version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
- "pygments_lexer": "ipython2",
- "version": "2.7.13"
+ "pygments_lexer": "ipython3",
+ "version": "3.6.1"
}
},
"nbformat": 4,