[GitHub] [systemds] Baunsgaard commented on pull request #1481: [SYSTEMDS-2832] Refactoring of old performance benchmarks.

[GitBox <gi...@apache.org>]

Baunsgaard commented on pull request #1481:
URL: https://github.com/apache/systemds/pull/1481#issuecomment-992732966


   > Couple of notes, let me know what you think:
   
   >     * I've added an extra dot to data generation scripts, to go back a directory, if the command is "systemds". This is done to compensate the different folder structure of the perftests when ran locally, instead of on a cluster.
   
   Don't like this since it implies a different folder to run the scripts from, as seen in the comments above.
   
   >     * I'm assuming the federated tests are to be ran locally only, not on a cluster.
   
   True, they are running with localhost workers, We want to add the federated things, but it require a bit of setup and remote machines so not obvious for a perf benchmark.
   
   >     * The stratified statistics benchmark needs a heap size of at least 10 GB. This setting is exported only when it's ran locally via "export SYSTEMDS_STANDALONE_OPTS="-Xmx10g -Xms10g -Xmn2000m"
   
   Change this, so that it is set via a settings file, or other means.
   
   >     * The K-means built-in function does a predict implicitly, meaning the benchmark predicts twice, as of now.
   
   For now i think we just ignore this.
   
   >     * Could not get the decision tree training built-in function to run.
   
   Okay, then leave it out commented and focus on the other parts.
   
   >       * It does not stop when ran with logistic regression data.
   
   hmm, this might be a bug, please if you want to, report it as such in JIRA with a way to reproduce it.
   
    https://issues.apache.org/jira/secure/Dashboard.jspa?selectPageId=12335852
   
   >       * There are scripts to generate decision tree data, but these can not be ran either as the built-in transform function does not exist anymore, apparently.
   >     * Could also not get the old decision tree and random forest algorithms running without looking deeper into it. (null pointer exception)
   
   Here leave them out commented, the decision tree and random forest have to be worked on anyway..
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscribe@systemds.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org