You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@systemds.apache.org by GitBox <gi...@apache.org> on 2022/12/21 19:00:32 UTC
[GitHub] [systemds] Baunsgaard opened a new pull request, #1755: [SYSTEMDS-3481] FrameFromMatrix Improvements
Baunsgaard opened a new pull request, #1755:
URL: https://github.com/apache/systemds/pull/1755
This commit update the change of matrix to frame to more efficiently change the MatrixBlock to frames.
The previous implementation has nice cache blocks and allocation for direct double to double change, this PR simply adds support for this change in the case of changing into other types, like boolean.
Changing a Matrix 64kx2k to boolean frame:
After:
22/12/21 19:56:11 ERROR frame.FrameFromMatrixBlockTest: 1055.994364
22/12/21 19:56:12 ERROR frame.FrameFromMatrixBlockTest: 1039.756463
22/12/21 19:56:13 ERROR frame.FrameFromMatrixBlockTest: 946.029085
22/12/21 19:56:14 ERROR frame.FrameFromMatrixBlockTest: 928.161053
22/12/21 19:56:15 ERROR frame.FrameFromMatrixBlockTest: 943.132151
22/12/21 19:56:16 ERROR frame.FrameFromMatrixBlockTest: 950.212744
22/12/21 19:56:17 ERROR frame.FrameFromMatrixBlockTest: 964.515222
22/12/21 19:56:17 ERROR frame.FrameFromMatrixBlockTest: 966.944032
22/12/21 19:56:18 ERROR frame.FrameFromMatrixBlockTest: 965.85695
22/12/21 19:56:19 ERROR frame.FrameFromMatrixBlockTest: 956.783357
Before:
22/12/21 19:59:56 ERROR frame.FrameFromMatrixBlockTest: 2199.846241
22/12/21 19:59:58 ERROR frame.FrameFromMatrixBlockTest: 2373.381971
22/12/21 20:00:01 ERROR frame.FrameFromMatrixBlockTest: 2270.362306
22/12/21 20:00:03 ERROR frame.FrameFromMatrixBlockTest: 2324.07255
22/12/21 20:00:05 ERROR frame.FrameFromMatrixBlockTest: 2294.39046
22/12/21 20:00:08 ERROR frame.FrameFromMatrixBlockTest: 2284.978142
22/12/21 20:00:10 ERROR frame.FrameFromMatrixBlockTest: 2295.71655
22/12/21 20:00:12 ERROR frame.FrameFromMatrixBlockTest: 2297.712022
22/12/21 20:00:14 ERROR frame.FrameFromMatrixBlockTest: 2311.518135
22/12/21 20:00:17 ERROR frame.FrameFromMatrixBlockTest: 2467.055097
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@systemds.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [systemds] Baunsgaard commented on pull request #1755: [SYSTEMDS-3481] FrameFromMatrix Improvements
Posted by GitBox <gi...@apache.org>.
Baunsgaard commented on PR #1755:
URL: https://github.com/apache/systemds/pull/1755#issuecomment-1362088597
And now with parallelization:
22/12/21 22:03:53 ERROR frame.FrameFromMatrixBlockTest: 197.328218
22/12/21 22:03:53 ERROR frame.FrameFromMatrixBlockTest: 136.362345
22/12/21 22:03:53 ERROR frame.FrameFromMatrixBlockTest: 130.854457
22/12/21 22:03:53 ERROR frame.FrameFromMatrixBlockTest: 148.331303
22/12/21 22:03:53 ERROR frame.FrameFromMatrixBlockTest: 129.145552
22/12/21 22:03:54 ERROR frame.FrameFromMatrixBlockTest: 129.617421
22/12/21 22:03:54 ERROR frame.FrameFromMatrixBlockTest: 130.632412
22/12/21 22:03:54 ERROR frame.FrameFromMatrixBlockTest: 133.174287
22/12/21 22:03:54 ERROR frame.FrameFromMatrixBlockTest: 195.967821
22/12/21 22:03:54 ERROR frame.FrameFromMatrixBlockTest: 132.401876
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@systemds.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [systemds] Baunsgaard commented on pull request #1755: [SYSTEMDS-3481] FrameFromMatrix Improvements
Posted by GitBox <gi...@apache.org>.
Baunsgaard commented on PR #1755:
URL: https://github.com/apache/systemds/pull/1755#issuecomment-1363796788
Before:
blockSize (k)
0.5 0.557+-0.025
1.0 0.296+-0.003
2.0 0.210+-0.010
4.0 0.159+-0.009
8.0 0.137+-0.003
16.0 0.139+-0.009
32.0 0.152+-0.023
64.0 0.159+-0.012
128.0 0.112+-0.004
After:
Read Time
0.134+-0.005
0.111+-0.001
0.107+-0.006
0.106+-0.007
0.110+-0.006
0.108+-0.005
0.129+-0.006
0.130+-0.002
0.117+-0.004
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@systemds.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [systemds] Baunsgaard commented on pull request #1755: [SYSTEMDS-3481] FrameFromMatrix Improvements
Posted by GitBox <gi...@apache.org>.
Baunsgaard commented on PR #1755:
URL: https://github.com/apache/systemds/pull/1755#issuecomment-1362118570
@phaniarnab @mboehm7
Any ideas on how to add (or where to add) threads to all methods/internals/instructions.
It is now the---i don't know---millionth time i run into the issue of not having the instruction include number of threads allowed for the instruction.
It should be set somewhere for all instructions such that no matter what all of them include a thread number, even if it is not used.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@systemds.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [systemds] Baunsgaard closed pull request #1755: [SYSTEMDS-3481] FrameFromMatrix Improvements
Posted by GitBox <gi...@apache.org>.
Baunsgaard closed pull request #1755: [SYSTEMDS-3481] FrameFromMatrix Improvements
URL: https://github.com/apache/systemds/pull/1755
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@systemds.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org