[systemds] branch master updated: [SYSTEMDS-3157] Fix performance MSVM for multinomial w/ intercept

[mb...@apache.org]

This is an automated email from the ASF dual-hosted git repository.

mboehm7 pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/systemds.git


The following commit(s) were added to refs/heads/master by this push:
     new c509e89  [SYSTEMDS-3157] Fix performance MSVM for multinomial w/ intercept
c509e89 is described below

commit c509e8906cdd2e0fc7dd21f93030c5a1be113c88
Author: Matthias Boehm <mb...@gmail.com>
AuthorDate: Sun Oct 10 19:49:11 2021 +0200

    [SYSTEMDS-3157] Fix performance MSVM for multinomial w/ intercept
    
    This patch fixes a performance issue of MSVM perftest on multinomial
    data of 80Mx1K sparse with 5 classes, which took 652s with 585s garbage
    collection time. One reason of the large GC was that the intercept flag
    was passed to the l2svm binary classifiers which copied the large
    feature matrix for every class and appended the same column of ones.
    Since lineage-based reuse (which would automatically eliminate this
    redundancy) is still not enabled by default, we now handle the intercept
    once in the MSVM function, and always invoke L2SVM with intercept=FALSE.
    
    On the perftest scenario, this patch improved end-to-end performance
    from 652s to 109s (which close to the invocation without intercept of
    98s).
---
 scripts/builtin/msvm.dml | 19 +++++++++++--------
 1 file changed, 11 insertions(+), 8 deletions(-)

diff --git a/scripts/builtin/msvm.dml b/scripts/builtin/msvm.dml
index 63dfa5e..d8dbc37 100644
--- a/scripts/builtin/msvm.dml
+++ b/scripts/builtin/msvm.dml
@@ -44,21 +44,24 @@
 # model           Double   ---        model matrix
 
 m_msvm = function(Matrix[Double] X, Matrix[Double] Y, Boolean intercept = FALSE,
-    Double epsilon = 0.001, Double lambda = 1.0, Integer maxIterations = 100, Boolean verbose = FALSE)
+    Double epsilon = 0.001, Double lambda = 1.0, Integer maxIterations = 100,
+Boolean verbose = FALSE)
   return(Matrix[Double] model)
 {
   if(min(Y) < 0)
     stop("MSVM: Invalid Y input, containing negative values")
-
   if(verbose)
     print("Running Multiclass-SVM")
 
   num_rows_in_w = ncol(X)
   if(intercept) {
-    num_rows_in_w = num_rows_in_w + 1
+    # append once, and call l2svm always with intercept=FALSE 
+    ones = matrix(1, rows=nrow(X), cols=1)
+    X = cbind(X, ones);
+    num_rows_in_w += 1
   }
 
-  if(ncol(Y) > 1) 
+  if(ncol(Y) > 1)
     Y = rowMaxs(Y * t(seq(1,ncol(Y))))
 
   # Assuming number of classes to be max contained in Y
@@ -66,10 +69,10 @@ m_msvm = function(Matrix[Double] X, Matrix[Double] Y, Boolean intercept = FALSE,
 
   parfor(class in 1:max(Y)) {
     Y_local = 2 * (Y == class) - 1
-    w[,class] = l2svm(X=X, Y=Y_local, intercept=intercept,
-        epsilon=epsilon, lambda=lambda, maxIterations=maxIterations, 
-        verbose= verbose, columnId=class)
+    w[,class] = l2svm(X=X, Y=Y_local, intercept=FALSE,
+        epsilon=epsilon, lambda=lambda, maxIterations=maxIterations,
+        verbose=verbose, columnId=class)
   }
-  
+
   model = w
 }