[GitHub] [systemds] sebwrede commented on pull request #1409: [SYSTEMDS-3148] Federated ALS Performance Tests

[GitBox <gi...@apache.org>]

sebwrede commented on pull request #1409:
URL: https://github.com/apache/systemds/pull/1409#issuecomment-937863036


   > Hi, This PR adds the first federated performance test, ALS-CG. The script generates the data, creates a specific amount (in our case 5) of federated worker processes on localhost, splits the data into partitions, creates a federated object from the different partitions and workers, runs the ALS-CG algorithm with the federated object as data input, and kills the woker processes again. I tried to make the shell scripts for starting and killing the federated workers, and also the dml script for splitting the data and creating a federated object, as general as possible so that the same scripts can be used the further federated performance tests.
   > 
   > Thanks for review :)
   
   Thanks for this PR!
   
   I ran the performance test and have two comments: 
   
   1. I get the following message continuously during the execution: `TRACE nio.NioEventLoop: instrumented a special java.util.Set into: sun.nio.ch.EPollSelectorImpl@`. Do you have any idea why this happens and how to fix it? 
   2. The script prints `Max iteration achieved but not converged!`. Could we increase the maximum number of iterations or change something else to make sure that it reaches its target? Otherwise it is difficult to determine from the prints whether the algorithm actually succeeded or if it was even close to succeeding. 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscribe@systemds.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org