You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@systemds.apache.org by GitBox <gi...@apache.org> on 2021/01/18 17:07:48 UTC

[GitHub] [systemds] Baunsgaard opened a new pull request #1164: [SYSTEMDS-2800] Agressive Compression Rewrite

Baunsgaard opened a new pull request #1164:
URL: https://github.com/apache/systemds/pull/1164


   This commit change the COST based approach of the rewrite rules for compression to an "aggressive" compression approach.
   The basic idea is to introduce many compression instructions into the computation graph, such that we have an compression command at all location that have a potential to:
   
   1. Compress nicely from uncompressed representation
   2. Restructure an already compressed matrix, into a better compression 
   
   For the 2. case operations such as less than resulting in only 0 and 1 in the matrix have high potential for better restructuring.
   But in general if the matrix inputted before the less than operation is not compressed it will try to compress anyway according to case 1, so no matter which case the instruction introduced should go the same place.
   
   For this aggressive compression instruction to execute the object to compress have a few criteria.
   
   1. There has to be at least 3 instructions executed on the object, that is "supported compressed operations" or the matrix is used in a loop.
   2. The compressed operations that are used should not have many "supported but slow operations".
   3. The matrix to compress have to be at least 1000 rows and below 10 columns, or have a ratio of 1000 to nrow/ncol.
   
   
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [systemds] Baunsgaard closed pull request #1164: [SYSTEMDS-2800] Agressive Compression Rewrite

Posted by GitBox <gi...@apache.org>.
Baunsgaard closed pull request #1164:
URL: https://github.com/apache/systemds/pull/1164


   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [systemds] Baunsgaard commented on pull request #1164: [SYSTEMDS-2800] Agressive Compression Rewrite

Posted by GitBox <gi...@apache.org>.
Baunsgaard commented on pull request #1164:
URL: https://github.com/apache/systemds/pull/1164#issuecomment-762374399


   Further work is to introduce decompression in cases where the compressed matrix is used in more than 1 operation that would decompress.


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org