You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@systemds.apache.org by GitBox <gi...@apache.org> on 2022/05/22 09:50:29 UTC
[GitHub] [systemds] ywcb00 opened a new pull request, #1622: [SYSTEMDS-3382] Federated Left Indexing with Scalar
ywcb00 opened a new pull request, #1622:
URL: https://github.com/apache/systemds/pull/1622
Hi,
this PR adds support for the federated left indexing instruction with a scalar rhs, including the respective tests.
When running the FederatedLeftIndexTest, it throws an exception because of invalid matrix dimensions (-1, -1) at the first test run. The second test run passes. Somehow this exception is only thrown if the two frame tests (_testLeftIndexFullDenseFrameCP_, _testLeftIndexFullDenseFrameSP_) are executed before the failing scalar test (_testLeftIndexScalarCP_). I tried to put the empty matrix for the output to the federated workers with the correct sliced dimensions instead of [-1, -1], but then we get a "file does not exist on hdfs" exception. Any idea or suspicion why this could be the case?
Thanks for review :)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@systemds.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [systemds] asfgit closed pull request #1622: [SYSTEMDS-3382] Federated Left Indexing with Scalar
Posted by GitBox <gi...@apache.org>.
asfgit closed pull request #1622: [SYSTEMDS-3382] Federated Left Indexing with Scalar
URL: https://github.com/apache/systemds/pull/1622
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@systemds.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [systemds] mboehm7 commented on pull request #1622: [SYSTEMDS-3382] Federated Left Indexing with Scalar
Posted by GitBox <gi...@apache.org>.
mboehm7 commented on PR #1622:
URL: https://github.com/apache/systemds/pull/1622#issuecomment-1146883114
LGTM - thanks for the patch @ywcb00 as well as catching and reporting this issue.
After spending way too much time on debugging this issue, I found the reason and fixed it. A simple `FederationUtils.resetFedDataID()` at the beginning of the test mitigated the described problem, which however pointed to a more severe issue. In detail, after running the two other tests, we got an output ID 13 for the federated left indexing. The scalar literal was also 13 which lead to incorrect replacements when preparing the individual instructions sent to the workers. By rearranging the order to input/output replacements, we now generally prevent such problems. Thanks.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@systemds.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org