You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@mnemonic.apache.org by ga...@apache.org on 2017/11/28 07:56:39 UTC
incubator-mnemonic git commit: MNEMONIC-416: Add README to
mnemonic-spark-kmeans-bench module
Repository: incubator-mnemonic
Updated Branches:
refs/heads/master a1495c47b -> 35f59c403
MNEMONIC-416: Add README to mnemonic-spark-kmeans-bench module
Project: http://git-wip-us.apache.org/repos/asf/incubator-mnemonic/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-mnemonic/commit/35f59c40
Tree: http://git-wip-us.apache.org/repos/asf/incubator-mnemonic/tree/35f59c40
Diff: http://git-wip-us.apache.org/repos/asf/incubator-mnemonic/diff/35f59c40
Branch: refs/heads/master
Commit: 35f59c403b1d1229fe6b70d9497afeeee48faec1
Parents: a1495c4
Author: Wang, Gang(Gary) <ga...@intel.com>
Authored: Mon Nov 27 23:53:56 2017 -0800
Committer: Wang, Gang(Gary) <ga...@intel.com>
Committed: Mon Nov 27 23:55:36 2017 -0800
----------------------------------------------------------------------
.../mnemonic-spark-kmeans-bench/README.md | 37 ++++++++++++++++++++
1 file changed, 37 insertions(+)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/incubator-mnemonic/blob/35f59c40/mnemonic-benches/mnemonic-spark-kmeans-bench/README.md
----------------------------------------------------------------------
diff --git a/mnemonic-benches/mnemonic-spark-kmeans-bench/README.md b/mnemonic-benches/mnemonic-spark-kmeans-bench/README.md
new file mode 100644
index 0000000..fd345ec
--- /dev/null
+++ b/mnemonic-benches/mnemonic-spark-kmeans-bench/README.md
@@ -0,0 +1,37 @@
+### Generate dataset
+
+
+```bash
+ $ bin/gen_data.py <the number of dimensions> <the number of vectors> [-s seed]
+```
+
+
+### Configure Apache Spark
+
+```text
+# conf/spark-defaults.conf for local mode
+
+spark.master local[4,0]
+
+spark.executor.memory <size>
+
+spark.driver.memory <size>
+
+spark.local.dir <dir>
+
+spark.jars <dir>/target/service-dist/mnemonic-pmalloc-service-<version>-linux-x86_64.jar,<dir>/target/service-dist/mnemonic-nvml-pmem-service-<version>-linux-x86_64.jar
+
+
+```
+
+
+### Run Apache Spark Benchmark Workloads
+
+
+```bash
+ # regular version
+ $ $SPARK_HOME/bin/spark-submit --class org.apache.mnemonic.bench.RegularKMeans --conf spark.durable-basedir=./mne incubator-mnemonic/mnemonic-benches/mnemonic-spark-kmeans-bench/target/mnemonic-spark-kmeans-bench-<version>.jar <path-to-dataset-file>
+
+ # durable version
+ $ $SPARK_HOME/bin/spark-submit --class org.apache.mnemonic.bench.DurableKMeans --conf spark.durable-basedir=./mne incubator-mnemonic/mnemonic-benches/mnemonic-spark-kmeans-bench/target/mnemonic-spark-kmeans-bench-<version>.jar <path-to-dataset-file>
+```