[jira] [Commented] (SPARK-20767) The LDA model update

["Cezary Dendek (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/SPARK-20767?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16012205#comment-16012205 ] 

Cezary Dendek commented on SPARK-20767:
---------------------------------------

Sure.
I'm proposing loading the trained model & continuation of the training.

> The LDA model update
> --------------------
>
>                 Key: SPARK-20767
>                 URL: https://issues.apache.org/jira/browse/SPARK-20767
>             Project: Spark
>          Issue Type: Improvement
>          Components: ML
>    Affects Versions: 2.1.1
>            Reporter: Cezary Dendek
>            Priority: Minor
>
> Current online implementation of the LDA model fit (OnlineLDAOptimizer) does not support the model update (ie. to account for the population/covariates drift) nor the continuation of model fitting in case of the insufficient number of iterations.
> Technical aspects:
> 1. The implementation of LDA fitting does not currently allow the coefficients pre-setting (private setter), as noted by a comment in the source code of OnlineLDAOptimizer.setLambda: "This is only used for testing now. In the future, it can help support training stop/resume".
> 2. The lambda matrix is always randomly initialized by the optimizer, which needs fixing for preset lambda matrix.
> The adaptation of the classes by the user is not possible due to protected setters & sealed / final classes.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org