[jira] [Resolved] (SPARK-5563) LDA with online variational inference

["Joseph K. Bradley (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/SPARK-5563?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Joseph K. Bradley resolved SPARK-5563.
--------------------------------------
       Resolution: Fixed
    Fix Version/s: 1.4.0

Issue resolved by pull request 4419
[https://github.com/apache/spark/pull/4419]

> LDA with online variational inference
> -------------------------------------
>
>                 Key: SPARK-5563
>                 URL: https://issues.apache.org/jira/browse/SPARK-5563
>             Project: Spark
>          Issue Type: Improvement
>          Components: MLlib
>    Affects Versions: 1.3.0
>            Reporter: Joseph K. Bradley
>            Assignee: yuhao yang
>             Fix For: 1.4.0
>
>
> Latent Dirichlet Allocation (LDA) parameters can be inferred using online variational inference, as in Hoffman, Blei and Bach. “Online Learning for Latent Dirichlet Allocation.”  NIPS, 2010.  This algorithm should be very efficient and should be able to handle much larger datasets than batch algorithms for LDA.
> This algorithm will also be important for supporting Streaming versions of LDA.
> The implementation will ideally use the same API as the existing LDA but use a different underlying optimizer.
> This will require hooking in to the existing mllib.optimization frameworks.
> This will require some discussion about whether batch versions of online variational inference should be supported, as well as what variational approximation should be used now or in the future.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org