You are viewing a plain text version of this content. The canonical link for it is here.

Posted to dev@systemds.apache.org by GitBox <gi...@apache.org> on 2021/05/19 15:33:20 UTC

[GitHub] [systemds] ywcb00 opened a new pull request #1283: [SYSTEMDS-2922] Federated Fused Codegen Operator - Outer Product

ywcb00 opened a new pull request #1283:
URL: https://github.com/apache/systemds/pull/1283


   Hi,
   This is a PR for adding federated code generation for outer product templates, and the respective tests (hybrid execution mode to reduce the test time).
   I created some nested classes in _SpoofFEDInstruction.java_ for the different spoof template types to better distinguish between them (e.g., cellwise, rowwise, multiagg, outerproduct). Each class has its own _setOutput()_ method, whereas only the class of the outerproduct template has its own _needsBroadcastSliced()_ method, because the other classes can share it.
   
   The federated spoof instruction for outerproduct templates is determining if an input matrix gets broadcast sliced based on its dimensions. This could create some problems, for example when computing
   `X * log(U %*% t(V))`
   with federated matrix _X_ in dimensions _2000x2000_,  _U_ in _2000x10_, and _V_ in _2000x10_, it broadcasts both, _U_ and _V_, in a sliced manner because the row number matches with the dimension of the federated matrix. Obviously, depending on the federation type of _X_, either _U_ or _V_ should've been broadcast sliced. However, when _U_ and _V_ differ in the dimensions, everything is working how it should. How can we handle this case of similar dimensions?
   
   Thanks for review :)


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [systemds] asfgit closed pull request #1283: [SYSTEMDS-2922] Federated Fused Codegen Operator - Outer Product

Posted by GitBox <gi...@apache.org>.

asfgit closed pull request #1283:
URL: https://github.com/apache/systemds/pull/1283


   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [systemds] ywcb00 commented on pull request #1283: [SYSTEMDS-2922] Federated Fused Codegen Operator - Outer Product

Posted by GitBox <gi...@apache.org>.

ywcb00 commented on pull request #1283:
URL: https://github.com/apache/systemds/pull/1283#issuecomment-846431957


   I changed the _needsBroadcastSliced()_ method to check whether it is the side input _U_ or _V_ based on the index of the inputs.
   If _X_ is row partitioned, _V_ is not getting broadcast sliced - and vice versa.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [systemds] ywcb00 commented on pull request #1283: [SYSTEMDS-2922] Federated Fused Codegen Operator - Outer Product

Posted by GitBox <gi...@apache.org>.

ywcb00 commented on pull request #1283:
URL: https://github.com/apache/systemds/pull/1283#issuecomment-846071273


   Hi,
   I figured out how to differentiate between the two side inputs U and V to correctly decide wether to broadcast sliced or not. I will update the PR soon :)


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [systemds] mboehm7 commented on pull request #1283: [SYSTEMDS-2922] Federated Fused Codegen Operator - Outer Product

Posted by GitBox <gi...@apache.org>.

mboehm7 commented on pull request #1283:
URL: https://github.com/apache/systemds/pull/1283#issuecomment-846455666


   LGTM - thanks for the patch @ywcb00. Great to see the federated codegen support completed.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org