[GitHub] [madlib] orhankislal opened a new pull request #425: DL: Add training for multiple models

[GitBox <gi...@apache.org>]

orhankislal opened a new pull request #425: DL: Add training for multiple models
URL: https://github.com/apache/madlib/pull/425
 
 
   This commit adds a new function to train multiple models in parallel
   with model hopping method.
   
   Model hopping method involves the following steps:
   
   - Train models in parallel for a single epoch using the data local
   to each segment.
   - Move the models to the next segment in round-robin fashion.
   
   This method ensures that all of the models visit the entitre dataset,
   which eliminates the need to average the model at the end.
   
   This commit also fixes an issue with the regular fit function where
   excessive amounts of threads were being created and left over by keras
   sessions. It is fixed by reusing the same session and the same
   computational graph throughout the process.
   
   Co-authored-by: Orhan Kislal <ok...@apache.org>
   Co-authored-by: Nandish Jayaram <nj...@apache.org>
   Co-authored-by: Ekta Khanna <ek...@pivotal.io>
   Co-authored-by: Nikhil Kak <nk...@pivotal.io>

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services