You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@systemds.apache.org by GitBox <gi...@apache.org> on 2021/01/24 16:27:08 UTC
[GitHub] [systemds] ywcb00 opened a new pull request #1170: [SYSTEMDS-2747] Federated Als-CG Test
ywcb00 opened a new pull request #1170:
URL: https://github.com/apache/systemds/pull/1170
Hi,
This is a PR for adding federated Als-CG as a first algorithm-level test for federated quaternary operations.
I also added a branch inside FEDInstructionUtils for rewriting BinarySP instructions into BinaryFED instructions in order to support the "fed_!=" instruction needed inside the als-CG algorithm. Since I haven't found a test for these federated operations, I added a test for MatrixScalar and MatrixMatrix federated logical operations.
I made a small fix (I hope it was a fix xD) inside BinaryMatrixMatrixFEDInstruction by changing a broadcast() to a broadcastSliced().
Hope that was not too much, otherwise just revert the last commit (bce7fdc) and the AlsCG test is using "sp_!=" for spark execution mode again.
Thanks for review :)
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [systemds] mboehm7 commented on pull request #1170: [SYSTEMDS-2747] Federated Als-CG Test
Posted by GitBox <gi...@apache.org>.
mboehm7 commented on pull request #1170:
URL: https://github.com/apache/systemds/pull/1170#issuecomment-770292186
LGTM - thanks @ywcb00. I just disabled the spark tests for ALS because forced spark exec mode hurts here in terms of execution (repeated lazy evaluation as the no intermediates are materialized).
As we discussed offline, please go ahead and do an additional cleanup over the federated binary element-wise operations to clearly differentiate between row and column partitioned federated data for matrix-row vector, matrix-column vector, and matrix-matrix. Thanks.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [systemds] ywcb00 commented on pull request #1170: [SYSTEMDS-2747] Federated Als-CG Test
Posted by GitBox <gi...@apache.org>.
ywcb00 commented on pull request #1170:
URL: https://github.com/apache/systemds/pull/1170#issuecomment-767560778
I've made a new commit (8c69017) to revert the changes of BinaryMatrixMatrixFEDInstruction.java because some federated algorithm tests were failing. In addition, I ignored the MatrixMatrix tests of the new FederatedLogicalTest (not working without the previous change) while still keeping the MatrixScalar tests (still working).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [systemds] ywcb00 commented on pull request #1170: [SYSTEMDS-2747] Federated Als-CG Test
Posted by GitBox <gi...@apache.org>.
ywcb00 commented on pull request #1170:
URL: https://github.com/apache/systemds/pull/1170#issuecomment-768453570
Found my mistake - we need to distinguish between column vector (normal broadcast) and matrix (broadcast sliced if row partitioned) in BinaryMatrixMatrixFEDInstruction.java.
Commit 8059c5e
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [systemds] asfgit closed pull request #1170: [SYSTEMDS-2747] Federated Als-CG Test
Posted by GitBox <gi...@apache.org>.
asfgit closed pull request #1170:
URL: https://github.com/apache/systemds/pull/1170
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org