You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@systemml.apache.org by du...@apache.org on 2015/12/02 02:05:24 UTC
[46/47] incubator-systemml git commit: [SYSML-341] Parfor check
option documented
[SYSML-341] Parfor check option documented
Project: http://git-wip-us.apache.org/repos/asf/incubator-systemml/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-systemml/commit/c8cc6eb7
Tree: http://git-wip-us.apache.org/repos/asf/incubator-systemml/tree/c8cc6eb7
Diff: http://git-wip-us.apache.org/repos/asf/incubator-systemml/diff/c8cc6eb7
Branch: refs/heads/gh-pages
Commit: c8cc6eb7a0cfa31453b6498b8cb145ebf7fd523d
Parents: bf8d44e
Author: Deron Eriksson <de...@us.ibm.com>
Authored: Tue Nov 3 10:35:13 2015 -0300
Committer: Luciano Resende <lr...@apache.org>
Committed: Tue Nov 3 10:35:13 2015 -0300
----------------------------------------------------------------------
dml-language-reference.md | 33 +++++++++++++++++++++++++++++++--
1 file changed, 31 insertions(+), 2 deletions(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/incubator-systemml/blob/c8cc6eb7/dml-language-reference.md
----------------------------------------------------------------------
diff --git a/dml-language-reference.md b/dml-language-reference.md
index f6915a3..dff0886 100644
--- a/dml-language-reference.md
+++ b/dml-language-reference.md
@@ -304,7 +304,7 @@ The for loop body may contain any sequence of statements. The statements in the
#### ParFor Statement
-The syntax and semantics of a parfor statement are equivalent to a for statement except for the different keyword and a list of optional parameters.
+The syntax and semantics of a `parfor` (parallel `for`) statement are equivalent to a `for` statement except for the different keyword and a list of optional parameters.
parfor (var in <for_predicate> <parfor_paramslist> ) {
<statement>*
@@ -329,7 +329,36 @@ The syntax and semantics of a parfor statement are equivalent to a for statement
<result_merge_mode> is one of the following tokens: LOCAL_MEM LOCAL_FILE LOCAL_AUTOMATIC REMOTE_MR
<optimization_mode> is one of the following tokens: NONE RULEBASED HEURISTIC GREEDY FULL_DP
-If any of these parameters is not specified, the following respective defaults are used: check = 1, par = [number of virtual processors on master node], mode = LOCAL, taskpartitioner = FIXED, tasksize =1, datapartitioner = NONE, resultmerge = LOCAL_AUTOMATIC, opt = RULEBASED.
+If any of these parameters is not specified, the following respective defaults are used: `check = 1`, `par = [number of virtual processors on master node]`, `mode = LOCAL`, `taskpartitioner = FIXED`, `tasksize = 1`, `datapartitioner = NONE`, `resultmerge = LOCAL_AUTOMATIC`, `opt = RULEBASED`.
+
+Of particular note is the `check` parameter. SystemML's `parfor` statement by default (`check = 1`) performs dependency analysis in an
+attempt to guarantee result correctness for parallel execution. For example, the following `parfor` statement is **incorrect** because
+the iterations do not act independently, so they are not parallizable. The iterations incorrectly try to increment the same `sum` variable.
+
+ sum = 0
+ parfor(i in 1:3) {
+ sum = sum + i; # not parallizable - generates error
+ }
+ print(sum)
+
+SystemML's `parfor` dependency analysis can occasionally result in false positives, as in the following example. This example creates a 2x30
+matrix. It then utilizes a `parfor` loop to write 10 2x3 matrices into the 2x30 matrix. This `parfor` statement is parallizable and correct,
+but the dependency analysis generates a false positive dependency error for the variable `ms`.
+
+ ms = matrix(0, rows=2, cols=3*10)
+ parfor (v in 1:10) { # parallizable - false positive
+ mv = matrix(v, rows=2, cols=3)
+ ms[,(v-1)*3+1:v*3] = mv
+ }
+
+If a false positive arises but you are certain that the `parfor` is parallizable, the `parfor` dependency check can be disabled via
+the `check = 0` option.
+
+ ms = matrix(0, rows=2, cols=3*10)
+ parfor (v in 1:10, check=0) { # parallizable
+ mv = matrix(v, rows=2, cols=3)
+ ms[,(v-1)*3+1:v*3] = mv
+ }
### User-Defined Function (UDF)