You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@systemds.apache.org by GitBox <gi...@apache.org> on 2023/01/05 00:01:12 UTC
[GitHub] [systemds] Baunsgaard opened a new pull request, #1759: [??] Frame Append cBind optimizations
Baunsgaard opened a new pull request, #1759:
URL: https://github.com/apache/systemds/pull/1759
Frame Cbind is implemented naively via appending rows one at a time and executing single threaded.
Current execution of a 64kx2k matrix of:
```txt
m = read($1)
for(i in 1:$2){
m2 = rbind(m,m)
}
print(toString(m2[1:10]))
```
```txt
SystemDS Statistics:
Total elapsed time: 50.224 sec.
Total compilation time: 0.385 sec.
Total execution time: 49.840 sec.
Cache hits (Mem/Li/WB/FS/HDFS): 11/0/0/0/1.
Cache writes (Li/WB/FS/HDFS): 0/6/0/0.
Cache times (ACQr/m, RLS, EXP): 0.122/0.002/0.002/0.000 sec.
HOP DAGs recompiled (PRED, SB): 0/0.
HOP DAGs recompile time: 0.000 sec.
Total JIT compile time: 0.459 sec.
Total JVM GC count: 0.
Total JVM GC time: 0.0 sec.
Heavy hitter instructions:
# Instruction Time(s) Count
1 append 49.813 5
2 toString 0.012 1
```
There are optimizations available in parallelization and appending a block at a time, both going to be implemented in this PR.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@systemds.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [systemds] Baunsgaard closed pull request #1759: [SYSTEMDS-3484] Frame Append Optimizations
Posted by GitBox <gi...@apache.org>.
Baunsgaard closed pull request #1759: [SYSTEMDS-3484] Frame Append Optimizations
URL: https://github.com/apache/systemds/pull/1759
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@systemds.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [systemds] Baunsgaard commented on pull request #1759: [??] Frame Append cBind optimizations
Posted by GitBox <gi...@apache.org>.
Baunsgaard commented on PR #1759:
URL: https://github.com/apache/systemds/pull/1759#issuecomment-1371967674
After:
```txt
SystemDS Statistics:
Total elapsed time: 0.489 sec.
Total compilation time: 0.255 sec.
Total execution time: 0.234 sec.
Cache hits (Mem/Li/WB/FS/HDFS): 11/0/0/0/1.
Cache writes (Li/WB/FS/HDFS): 0/6/0/0.
Cache times (ACQr/m, RLS, EXP): 0.095/0.002/0.002/0.000 sec.
HOP DAGs recompiled (PRED, SB): 0/0.
HOP DAGs recompile time: 0.000 sec.
Total JIT compile time: 0.502 sec.
Total JVM GC count: 0.
Total JVM GC time: 0.0 sec.
Heavy hitter instructions:
# Instruction Time(s) Count
1 append 0.211 5
2 createvar 0.010 7
3 rightIndex 0.005 1
4 toString 0.005 1
5 assignvar 0.001 3
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@systemds.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org