[systemds] branch master updated: [SYSTEMDS-3141] Fix memory estimates of ultra-sparse matrix blocks

[mb...@apache.org]

This is an automated email from the ASF dual-hosted git repository.

mboehm7 pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/systemds.git


The following commit(s) were added to refs/heads/master by this push:
     new c1496ae  [SYSTEMDS-3141] Fix memory estimates of ultra-sparse matrix blocks
c1496ae is described below

commit c1496ae0a78d5a5173ccaff5bd732563c89c7c37
Author: Matthias Boehm <mb...@gmail.com>
AuthorDate: Sun Sep 19 22:15:46 2021 +0200

    [SYSTEMDS-3141] Fix memory estimates of ultra-sparse matrix blocks
    
    The recently changed memory estimates of matrix blocks, introduced a bug
    that misses the object reference for empty rows leading to severe
    underestimation of ultra-sparse matrices with many empty rows.
    
    When computing connected components on the DBLP author graph with 2.4M
    authors and only 35K non-empty rows, the wrong memory estimate led to
    column vectors being mistakenly allocated in sparse. With this fix,
    end-to-end performance for 100 iterations improved from 20.1s to 9.1s.
---
 src/main/java/org/apache/sysds/runtime/data/SparseBlockMCSR.java | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/src/main/java/org/apache/sysds/runtime/data/SparseBlockMCSR.java b/src/main/java/org/apache/sysds/runtime/data/SparseBlockMCSR.java
index a733ea9..66f63a4 100644
--- a/src/main/java/org/apache/sysds/runtime/data/SparseBlockMCSR.java
+++ b/src/main/java/org/apache/sysds/runtime/data/SparseBlockMCSR.java
@@ -111,7 +111,7 @@ public class SparseBlockMCSR extends SparseBlock
 		//Overheads for arrays, objects, and references refer to 64bit JVMs
 		//If nnz < rows we have guaranteed also empty rows.
 		double size = 16; //object
-		size += MemoryEstimates.objectArrayCost((long)rlen); //references
+		size += MemoryEstimates.objectArrayCost((long)nrows); //references
 		long sparseRowSize = 16; // object
 		sparseRowSize += 4*4; // 3 integers + padding
 		sparseRowSize += MemoryEstimates.intArrayCost(0);