Re: Dirichlet: "Sampling from the posterior distribution for the model parameters"

[Ted Dunning <te...@gmail.com>]

Where k-means would be re-computing centroids of clusters based on data
points in the cluster, DP samples model parameters from the posterior
distribution defined by the data points assigned to that model.  I need to
look at the code, though, to be sure of what is happening there.

I will look at that right now.

So, yes, your change makes sense.

On Thu, Mar 5, 2009 at 10:49 AM, Jeff Eastman <jd...@windwardsolutions.com>wrote:

> Ted's description of DP includes the following text: "After points are
> assigned [to models in an iteration], new parameters for each model are
> sampled from the posterior distribution for the model parameters considering
> all of the observed data points that were assigned to the model.  Models
> without any data points are also sampled, but since they have no points
> assigned, the new samples are effectively taken from the prior distribution
> for model parameters."
>
> Looking at the code in MAHOUT-30e <DirichletClusterer.iterate()>:
>
>   // create new prior models
>   Model<Observation>[] newModels = modelFactory
>       .sampleFromPrior(state.numClusters, dist);
>   // iterate over the samples, assigning each to a model
>
> This seems to be inconsistent with the text. By always sampling from the
> prior we lose all coherence between the model parameters between iterations.
> Look at the first trace output, the parameters are all over the map between
> iterations as one would expect from the code. I have tried changing this
> code to:
>
>   // create new posterior models
>   Model<Observation>[] newModels = modelFactory
>       .sampleFromPosterior(state.models, dist);
>   // iterate over the samples, assigning each to a model
>
> Where sampleFromPosterior() returns the exact parameters from the posterior
> models. As you can see from the second trace output, the parameters of the
> k-th model are a lot more consistent between iterations. Here the
> variability is only due to the statistical assignment of points to models
> and the effect that has on the posterior model parameters. Using the exact
> posterior parameters is not "sampling from the posterior distribution for
> the model parameters"; however, and I wonder how to improve upon this?
>



-- 
Ted Dunning, CTO
DeepDyve

111 West Evelyn Ave. Ste. 202
Sunnyvale, CA 94086
www.deepdyve.com
408-773-0110 ext. 738
858-414-0013 (m)
408-773-0220 (fax)