[GitHub] [systemml] kev-inn commented on issue #871: [SYSTEMDS-315] Python Federated Matrices

[GitBox <gi...@apache.org>]

kev-inn commented on issue #871:
URL: https://github.com/apache/systemml/pull/871#issuecomment-616539062


   Ok, so currently we have the bugs:
   1. ~~bug is if you create a federated matrix that is not 100% filled in by federated workers, then operations does not take into account the remaining cells.~~ This should be fixed, I committed the fix to your sb_fed_python branch, seems like the change didn't make it in to this PR. Should we do a separate one?
   2. bug is if you create a federated matrix that is larger than the input size, it crashes, while expected behavior is to pad the matrix with 0's for consistent behavior, with other functionality.
   3. ~~feature~~/bug, currently if a single federated matrix is having overlapping sub matrices, see tests "federated basic test" for examples, the system does not complain, and in some cases even works by overwriting the matrix values of the first selected federated matrix with the later matrices.
   
   I am not sure if the second bug is actually a bug or a problem of our function definition. Our definition (the way I implemented it) expects the ranges to match the actual sizes of the matrices on the workers. And the actual size of the federated matrix goes from (0,0) to (max row, max col) given in the ranges, making it therefore not possible to get 0 rows and 0 cols after our ranges are finished. It is also bad that I forgot some very important checks to actually throw errors if the function definition is used wrong (3. could be regarded as fixed if we just check for overlaps).


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org