You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@systemds.apache.org by GitBox <gi...@apache.org> on 2022/10/21 13:47:41 UTC

[GitHub] [systemds] Baunsgaard opened a new pull request, #1709: [SYSTEMDS-3447] SDC AppendN

Baunsgaard opened a new pull request, #1709:
URL: https://github.com/apache/systemds/pull/1709

   This PR addresses the effeciency and implementation of AppendN in case of Offsets contained in all SDC Groups.
   Initial performance numbers looks as follows with variable sparsity:
   ![image](https://user-images.githubusercontent.com/9947148/197210999-663d0135-9b03-4f56-b073-b3d70184fe4a.png)
   
   Here all increases in the read line for read performance is because of unsupported append N for SDC groups.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscribe@systemds.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [systemds] Baunsgaard commented on pull request #1709: [SYSTEMDS-3447] SDC AppendN

Posted by GitBox <gi...@apache.org>.
Baunsgaard commented on PR #1709:
URL: https://github.com/apache/systemds/pull/1709#issuecomment-1287029330

   Generic AppendN offsets:
   
   ![image](https://user-images.githubusercontent.com/9947148/197217372-e7f0aacb-ad85-4eb8-968c-d4a583e25266.png)
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscribe@systemds.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [systemds] Baunsgaard commented on pull request #1709: [SYSTEMDS-3447] SDC AppendN

Posted by GitBox <gi...@apache.org>.
Baunsgaard commented on PR #1709:
URL: https://github.com/apache/systemds/pull/1709#issuecomment-1288152387

   Specialized Append, and multi threading the append:
   
   ![image](https://user-images.githubusercontent.com/9947148/197404710-7de13f31-64f5-4109-a8d6-0b1a6136e304.png)
   
   Unfortunately the parallelization have increased the base time, of the previous already good execution time, so further tuning is needed to make decisions for parallelization.
   
   To break down the execution time better, i have chosen to change to only plot IO time, instead of end-to-end time:
   
   ![image](https://user-images.githubusercontent.com/9947148/197404668-7374515d-343f-4d14-baf6-180c94bbc20d.png)
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscribe@systemds.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [systemds] Baunsgaard merged pull request #1709: [SYSTEMDS-3447] SDC AppendN

Posted by GitBox <gi...@apache.org>.
Baunsgaard merged PR #1709:
URL: https://github.com/apache/systemds/pull/1709


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscribe@systemds.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [systemds] Baunsgaard commented on pull request #1709: [SYSTEMDS-3447] SDC AppendN

Posted by GitBox <gi...@apache.org>.
Baunsgaard commented on PR #1709:
URL: https://github.com/apache/systemds/pull/1709#issuecomment-1288152681

   The compiler seems to be doing something questionable, since each execution have an overhead of 0.3 to 0.6 sec in compiler.
   Is this something we can optimize @mboehm7 ?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscribe@systemds.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org