You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@systemml.apache.org by du...@apache.org on 2016/09/10 01:57:14 UTC
incubator-systemml git commit: [SYSTEMML-618][SYSTEMML-904] Update
the API calls in the deep learning notebook examples.
Repository: incubator-systemml
Updated Branches:
refs/heads/master 63e2060fe -> b21d73d60
[SYSTEMML-618][SYSTEMML-904] Update the API calls in the deep learning notebook examples.
This simply updates the MLContext API function calls to their new forms.
Project: http://git-wip-us.apache.org/repos/asf/incubator-systemml/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-systemml/commit/b21d73d6
Tree: http://git-wip-us.apache.org/repos/asf/incubator-systemml/tree/b21d73d6
Diff: http://git-wip-us.apache.org/repos/asf/incubator-systemml/diff/b21d73d6
Branch: refs/heads/master
Commit: b21d73d60be8655d015169ed98bbbe1895a1d92a
Parents: 63e2060
Author: Mike Dusenberry <mw...@us.ibm.com>
Authored: Fri Sep 9 18:55:10 2016 -0700
Committer: Mike Dusenberry <mw...@us.ibm.com>
Committed: Fri Sep 9 18:55:10 2016 -0700
----------------------------------------------------------------------
scripts/staging/SystemML-NN/README.md | 2 +-
.../staging/SystemML-NN/examples/Example - MNIST LeNet.ipynb | 7 ++-----
.../examples/Example - MNIST Softmax Classifier.ipynb | 7 ++-----
scripts/staging/SystemML-NN/examples/README.md | 4 ++--
4 files changed, 7 insertions(+), 13 deletions(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/incubator-systemml/blob/b21d73d6/scripts/staging/SystemML-NN/README.md
----------------------------------------------------------------------
diff --git a/scripts/staging/SystemML-NN/README.md b/scripts/staging/SystemML-NN/README.md
index ca5067e..3943765 100644
--- a/scripts/staging/SystemML-NN/README.md
+++ b/scripts/staging/SystemML-NN/README.md
@@ -22,7 +22,7 @@ limitations under the License.
### A deep learning library for [Apache SystemML](https://github.com/apache/incubator-systemml).
## Examples:
-#### Please see the `examples` folder for more detailed examples, or view the following two quick examples.
+#### Please see the [`examples`](examples) folder for more detailed examples, or view the following two quick examples.
### Neural net for regression with vanilla SGD:
```python
# Imports
http://git-wip-us.apache.org/repos/asf/incubator-systemml/blob/b21d73d6/scripts/staging/SystemML-NN/examples/Example - MNIST LeNet.ipynb
----------------------------------------------------------------------
diff --git a/scripts/staging/SystemML-NN/examples/Example - MNIST LeNet.ipynb b/scripts/staging/SystemML-NN/examples/Example - MNIST LeNet.ipynb
index 4ed79d5..3ad210e 100644
--- a/scripts/staging/SystemML-NN/examples/Example - MNIST LeNet.ipynb
+++ b/scripts/staging/SystemML-NN/examples/Example - MNIST LeNet.ipynb
@@ -15,11 +15,8 @@
},
"outputs": [],
"source": [
- "# Add SystemML PySpark API file.\n",
- "sc.addPyFile(\"https://raw.githubusercontent.com/apache/incubator-systemml/2f3ab98804dace8b1ac1fef70d841c8df88290ab/src/main/python/SystemML.py\")\n",
- "\n",
"# Create a SystemML MLContext object\n",
- "from SystemML import MLContext, dml\n",
+ "from systemml import MLContext, dml\n",
"ml = MLContext(sc)"
]
},
@@ -100,7 +97,7 @@
"\"\"\"\n",
"script = (dml(script_string).input(\"$data\", \"data/mnist/mnist_train.csv\")\n",
" .input(C=1, Hin=28, Win=28)\n",
- " .out(\"W1\", \"b1\", \"W2\", \"b2\", \"W3\", \"b3\", \"W4\", \"b4\"))\n",
+ " .output(\"W1\", \"b1\", \"W2\", \"b2\", \"W3\", \"b3\", \"W4\", \"b4\"))\n",
"W1, b1, W2, b2, W3, b3, W4, b4 = (ml.execute(script)\n",
" .get(\"W1\", \"b1\", \"W2\", \"b2\", \"W3\", \"b3\", \"W4\", \"b4\"))"
]
http://git-wip-us.apache.org/repos/asf/incubator-systemml/blob/b21d73d6/scripts/staging/SystemML-NN/examples/Example - MNIST Softmax Classifier.ipynb
----------------------------------------------------------------------
diff --git a/scripts/staging/SystemML-NN/examples/Example - MNIST Softmax Classifier.ipynb b/scripts/staging/SystemML-NN/examples/Example - MNIST Softmax Classifier.ipynb
index f383ef8..7f2c2f0 100644
--- a/scripts/staging/SystemML-NN/examples/Example - MNIST Softmax Classifier.ipynb
+++ b/scripts/staging/SystemML-NN/examples/Example - MNIST Softmax Classifier.ipynb
@@ -16,11 +16,8 @@
},
"outputs": [],
"source": [
- "# Add SystemML PySpark API file.\n",
- "sc.addPyFile(\"https://raw.githubusercontent.com/apache/incubator-systemml/2f3ab98804dace8b1ac1fef70d841c8df88290ab/src/main/python/SystemML.py\")\n",
- "\n",
"# Create a SystemML MLContext object\n",
- "from SystemML import MLContext, dml\n",
+ "from systemml import MLContext, dml\n",
"ml = MLContext(sc)"
]
},
@@ -99,7 +96,7 @@
"# Train\n",
"[W, b] = mnist_softmax::train(X, y, X_val, y_val)\n",
"\"\"\"\n",
- "script = dml(training).input(\"$data\", \"data/mnist/mnist_train.csv\").out(\"W\", \"b\")\n",
+ "script = dml(training).input(\"$data\", \"data/mnist/mnist_train.csv\").output(\"W\", \"b\")\n",
"W, b = ml.execute(script).get(\"W\", \"b\")"
]
},
http://git-wip-us.apache.org/repos/asf/incubator-systemml/blob/b21d73d6/scripts/staging/SystemML-NN/examples/README.md
----------------------------------------------------------------------
diff --git a/scripts/staging/SystemML-NN/examples/README.md b/scripts/staging/SystemML-NN/examples/README.md
index cb0454f..b1ccf22 100644
--- a/scripts/staging/SystemML-NN/examples/README.md
+++ b/scripts/staging/SystemML-NN/examples/README.md
@@ -64,10 +64,10 @@ limitations under the License.
## Execution
* These examples contain scripts written in SystemML's R-like language (`*.dml`), as well as PySpark Jupyter notebooks (`*.ipynb`). The scripts contain the math for the algorithms, enclosed in functions, and the notebooks serve as full, end-to-end examples of reading in data, training models using the functions within the scripts, and evaluating final performance.
-* **Notebooks**: To run the notebook examples, please startup Jupyter in the following manner from this directory (or for more information, please see [this great blog post](http://spark.tc/0-to-life-changing-application-with-apache-systemml/)):
+* **Notebooks**: To run the notebook examples, please install the SystemML Python package with `pip install systemml`, and then startup Jupyter in the following manner from this directory (or for more information, please see [this great blog post](http://spark.tc/0-to-life-changing-application-with-apache-systemml/)):
```
- PYSPARK_DRIVER_PYTHON=jupyter PYSPARK_DRIVER_PYTHON_OPTS="notebook" $SPARK_HOME/bin/pyspark --master local[*] --driver-memory 3G --driver-class-path $SYSTEMML_HOME/SystemML.jar
+ PYSPARK_DRIVER_PYTHON=jupyter PYSPARK_DRIVER_PYTHON_OPTS="notebook" pyspark --master local[*] --driver-memory 3G --driver-class-path $SYSTEMML_HOME/SystemML.jar --jars $SYSTEMML_HOME/SystemML.jar
```
Note that all printed output, such as training statistics, from the SystemML scripts will be sent to the terminal in which Jupyter was started (for now...).