[27/47] incubator-systemml git commit: [SYSML-282] Use "select" vector to remove empty rows/cols

[du...@apache.org]

[SYSML-282] Use "select" vector to remove empty rows/cols

This change was based on observation that time consumed by eviction was
high in experimental data. In few algorithms such as -- DecisionTree,
RandomForest -- in which Matrix was operated with RemoveEmpty operator.
Before doing RemoveEmpty operation, select matrix was operated on target
matrix with append and elementwise multiplication. This is repeated in
loop operation. Instead of operating select vector on target matrix,
select vector can be passed to RemoveEmpty to avoid expensive operations
causing delay in eviction.

This fix will provide an option for caller to send select vector in
RemoveEmpty operator.

Enhancement to RemoveEmpty include following:

- It will use "Select vector" if passed in to figure out rows/columns
  to be removed.
- Memory requirement estimate update.
- Test cases to evaluate this change updated.


Project: http://git-wip-us.apache.org/repos/asf/incubator-systemml/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-systemml/commit/442e83a4
Tree: http://git-wip-us.apache.org/repos/asf/incubator-systemml/tree/442e83a4
Diff: http://git-wip-us.apache.org/repos/asf/incubator-systemml/diff/442e83a4

Branch: refs/heads/gh-pages
Commit: 442e83a469b99bcf1f33d2489a25b9ba3857de3a
Parents: b966a81
Author: Arvind Surve <ac...@yahoo.com>
Authored: Tue Sep 15 14:08:04 2015 -0700
Committer: Luciano Resende <lr...@apache.org>
Committed: Tue Sep 15 14:08:04 2015 -0700

----------------------------------------------------------------------
 Language Reference/SystemML_Language_Reference.html | 11 +++++++----
 1 file changed, 7 insertions(+), 4 deletions(-)
----------------------------------------------------------------------


http://git-wip-us.apache.org/repos/asf/incubator-systemml/blob/442e83a4/Language Reference/SystemML_Language_Reference.html
----------------------------------------------------------------------
diff --git a/Language Reference/SystemML_Language_Reference.html b/Language Reference/SystemML_Language_Reference.html
index 76e2c04..ce3635d 100644
--- a/Language Reference/SystemML_Language_Reference.html	
+++ b/Language Reference/SystemML_Language_Reference.html	
@@ -4421,8 +4421,9 @@ Manipulation, and Aggregation Built-In Functions<o:p></o:p></span></p>
   mso-border-top-alt:solid windowtext .5pt;mso-border-left-alt:solid windowtext .5pt;
   mso-border-alt:solid windowtext .5pt;padding:0in 5.4pt 0in 5.4pt'>
   <p class=MsoNormal>Removes all empty rows or columns from the input matrix <i>target</i>
-  X according to the specified <i>margin.</i><span style='mso-fareast-font-family:
-  "Times New Roman";mso-fareast-theme-font:minor-fareast;color:black'><o:p></o:p></span></p>
+  X according to the specified <i>margin.</i>  <br>Optionally vector select can be passed to remove empty rows/columns. 
+  <br>select vector should contain values either 1.0 or 0.0 for rows/columns to be retained or removed respectively. 
+  <span style='mso-fareast-font-family: "Times New Roman";mso-fareast-theme-font:minor-fareast;color:black'><o:p></o:p></span></p>
   </td>
   <td width=162 valign=top style='width:121.75pt;border-top:none;border-left:
   none;border-bottom:solid windowtext 1.0pt;border-right:solid windowtext 1.0pt;
@@ -4430,7 +4431,7 @@ Manipulation, and Aggregation Built-In Functions<o:p></o:p></span></p>
   mso-border-alt:solid windowtext .5pt;padding:0in 5.4pt 0in 5.4pt'>
   <p class=MsoNormal><span class=GramE><b style='mso-bidi-font-weight:normal'>Input</b>
   :</span><b style='mso-bidi-font-weight:normal'> </b>(target= X
-  &lt;matrix&gt;, margin=&quot;...&quot;)</p>
+  &lt;matrix&gt;, margin=&quot;...&quot;[,select=I])</p>
   <p class=MsoNormal><b style='mso-bidi-font-weight:normal'>Output</b> :
   &lt;matrix&gt;</p>
   <p class=MsoNormal>Valid values for margin are &quot;rows&quot; or
@@ -4442,7 +4443,9 @@ Manipulation, and Aggregation Built-In Functions<o:p></o:p></span></p>
   mso-border-top-alt:solid windowtext .5pt;mso-border-left-alt:solid windowtext .5pt;
   mso-border-alt:solid windowtext .5pt;padding:0in 5.4pt 0in 5.4pt'>
   <p class=MsoNormal>A = <span class=SpellE>removeEmpty</span>(target=X,
-  margin=&quot;rows&quot;)<span style='mso-fareast-font-family:"Times New Roman";
+  margin=&quot;rows&quot;) <br> A = <span class=SpellE>removeEmpty</span>(target=X,
+  margin=&quot;rows&quot;,select=I)
+  <span style='mso-fareast-font-family:"Times New Roman";
   mso-fareast-theme-font:minor-fareast;color:black'><o:p></o:p></span></p>
   </td>
  </tr>