You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@commons.apache.org by Patrick Meyer <me...@gmail.com> on 2012/10/18 14:13:52 UTC
RE: [Math] MATH-816 (mixture model distribution) . .
I vote for simplicity. Current practice in the social sciences is to fit
multiple models, each with a different number of components, and use fit
statistics to choose the best model.
There are some additional features I would like to see added and I have the
code to contribute if it is not currently there. To be consistent with
Mplus, we need have the algorithm use multiple random starts and run a few
of the best starts to completion. Mplus uses this strategy to effectively
overcome local minima.
-----Original Message-----
From: Becksfort, Jared [mailto:Jared.Becksfort@STJUDE.ORG]
Sent: Wednesday, October 17, 2012 11:37 PM
To: Commons Developers List
Subject: RE: [Math] MATH-816 (mixture model distribution) . .
I see. I am planning to submit the EM fit for multivariate normal mixture
models in the next couple of weeks (Math-817). A Gibbs sampling DP fit may
be a bit further out. I am not opposed to allowing the number of
components to change, but I also like the simplicity of this class.
Whatever you guys decide is probably fine.
Jared
________________________________________
From: Ted Dunning [ted.dunning@gmail.com]
Sent: Wednesday, October 17, 2012 9:41 PM
To: Commons Developers List
Subject: Re: [Math] MATH-816 (mixture model distribution)
=?utf-8?B?LiAgICAu? ==?utf-8?B?LiAgICAu?=
The issue is that with a fixed number of components, you need to do multiple
runs to find a best fit number of components. Gibbs sampling against a
Dirichlet process can get you to the same answer in about the same cost as a
single run of EM with a fixed number of models.
On Wed, Oct 17, 2012 at 7:31 PM, Becksfort, Jared <
Jared.Becksfort@stjude.org> wrote:
> Ted,
>
> I am not sure I understand the problem with the fixed number of
> components. My understanding is that CM prefers immutable objects.
> Adding a component to an object would require reweighting in addition
> to modifying the component list. A new mixture model could be
> instantiated using the getComponents function and then adding or
> removing more components if necessary.
>
> Jared
> ________________________________________
> From: Ted Dunning [ted.dunning@gmail.com]
> Sent: Wednesday, October 17, 2012 5:21 PM
> To: Commons Developers List
> Subject: Re: [Math] MATH-816 (mixture model
> distribution)=?utf-8?B?LiAgICAu? =
>
> Seems fine.
>
> I think that the limitation to a fixed number of mixture components is
> a bit limiting. So is the limitation to a uniform set of components.
> Both limitations can be eased without a huge difficultly.
>
> Avoiding the fixed number of components can be done by using some
> variant of Dirichlet processes. Simply picking k_max relatively large
> and then using an approximate DP over that finite set works well.
>
> That said, mixture models are pretty nice to have.
>
> On Wed, Oct 17, 2012 at 2:13 PM, Gilles Sadowski <
> gilles@harfang.homelinux.org> wrote:
>
> > Hello.
> >
> > Any objection to commit the code as proposed on the report page?
> > https://issues.apache.org/jira/browse/MATH-816
> >
> >
> > Regards,
> > Gilles
> >
> > --------------------------------------------------------------------
> > - To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
> > For additional commands, e-mail: dev-help@commons.apache.org
> >
> >
>
> Email Disclaimer: www.stjude.org/emaildisclaimer Consultation
> Disclaimer: www.stjude.org/consultationdisclaimer
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
> For additional commands, e-mail: dev-help@commons.apache.org
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
For additional commands, e-mail: dev-help@commons.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
For additional commands, e-mail: dev-help@commons.apache.org
Re: [Math] MATH-816 (mixture model distribution) . .
Posted by Ted Dunning <te...@gmail.com>.
Existing code does have a certain cachet to it.
On Thu, Oct 18, 2012 at 5:13 AM, Patrick Meyer <me...@gmail.com> wrote:
> I vote for simplicity. Current practice in the social sciences is to fit
> multiple models, each with a different number of components, and use fit
> statistics to choose the best model.
>
> There are some additional features I would like to see added and I have the
> code to contribute if it is not currently there. To be consistent with
> Mplus, we need have the algorithm use multiple random starts and run a few
> of the best starts to completion. Mplus uses this strategy to effectively
> overcome local minima.
>
>
> -----Original Message-----
> From: Becksfort, Jared [mailto:Jared.Becksfort@STJUDE.ORG]
> Sent: Wednesday, October 17, 2012 11:37 PM
> To: Commons Developers List
> Subject: RE: [Math] MATH-816 (mixture model distribution) . .
>
> I see. I am planning to submit the EM fit for multivariate normal mixture
> models in the next couple of weeks (Math-817). A Gibbs sampling DP fit may
> be a bit further out. I am not opposed to allowing the number of
> components to change, but I also like the simplicity of this class.
> Whatever you guys decide is probably fine.
>
> Jared
> ________________________________________
> From: Ted Dunning [ted.dunning@gmail.com]
> Sent: Wednesday, October 17, 2012 9:41 PM
> To: Commons Developers List
> Subject: Re: [Math] MATH-816 (mixture model distribution)
> =?utf-8?B?LiAgICAu? ==?utf-8?B?LiAgICAu?=
>
> The issue is that with a fixed number of components, you need to do
> multiple
> runs to find a best fit number of components. Gibbs sampling against a
> Dirichlet process can get you to the same answer in about the same cost as
> a
> single run of EM with a fixed number of models.
>
> On Wed, Oct 17, 2012 at 7:31 PM, Becksfort, Jared <
> Jared.Becksfort@stjude.org> wrote:
>
> > Ted,
> >
> > I am not sure I understand the problem with the fixed number of
> > components. My understanding is that CM prefers immutable objects.
> > Adding a component to an object would require reweighting in addition
> > to modifying the component list. A new mixture model could be
> > instantiated using the getComponents function and then adding or
> > removing more components if necessary.
> >
> > Jared
> > ________________________________________
> > From: Ted Dunning [ted.dunning@gmail.com]
> > Sent: Wednesday, October 17, 2012 5:21 PM
> > To: Commons Developers List
> > Subject: Re: [Math] MATH-816 (mixture model
> > distribution)=?utf-8?B?LiAgICAu? =
> >
> > Seems fine.
> >
> > I think that the limitation to a fixed number of mixture components is
> > a bit limiting. So is the limitation to a uniform set of components.
> > Both limitations can be eased without a huge difficultly.
> >
> > Avoiding the fixed number of components can be done by using some
> > variant of Dirichlet processes. Simply picking k_max relatively large
> > and then using an approximate DP over that finite set works well.
> >
> > That said, mixture models are pretty nice to have.
> >
> > On Wed, Oct 17, 2012 at 2:13 PM, Gilles Sadowski <
> > gilles@harfang.homelinux.org> wrote:
> >
> > > Hello.
> > >
> > > Any objection to commit the code as proposed on the report page?
> > > https://issues.apache.org/jira/browse/MATH-816
> > >
> > >
> > > Regards,
> > > Gilles
> > >
> > > --------------------------------------------------------------------
> > > - To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
> > > For additional commands, e-mail: dev-help@commons.apache.org
> > >
> > >
> >
> > Email Disclaimer: www.stjude.org/emaildisclaimer Consultation
> > Disclaimer: www.stjude.org/consultationdisclaimer
> >
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
> > For additional commands, e-mail: dev-help@commons.apache.org
> >
> >
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
> For additional commands, e-mail: dev-help@commons.apache.org
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
> For additional commands, e-mail: dev-help@commons.apache.org
>
>
Re: [Math] MATH-816 (mixture model distribution)
Posted by Gilles Sadowski <gi...@harfang.homelinux.org>.
On Thu, Oct 18, 2012 at 11:00:17AM -0400, Patrick Meyer wrote:
> Yes, I like the latest changes. It looks cleaner to me.
>
> It seems that the attachments only describe the mixture distribution and do
> not provide the EM estimation algorithm. Am I missing something? Didn't the
> original contributor mention the estimation part too? The parts I would like
> to add will need the EM part first.
Jared has broken up his proposal into three parts; the enclosing issue is
https://issues.apache.org/jira/browse/MATH-817
which explicitly refers to the EM part, but no implementation of this has
been submitted yet.
Regards,
Gilles
> [...]
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
For additional commands, e-mail: dev-help@commons.apache.org
RE: [Math] MATH-816 (mixture model distribution)
Posted by Patrick Meyer <me...@gmail.com>.
Yes, I like the latest changes. It looks cleaner to me.
It seems that the attachments only describe the mixture distribution and do
not provide the EM estimation algorithm. Am I missing something? Didn't the
original contributor mention the estimation part too? The parts I would like
to add will need the EM part first.
Thanks!
-----Original Message-----
From: Gilles Sadowski [mailto:gilles@harfang.homelinux.org]
Sent: Thursday, October 18, 2012 9:33 AM
To: dev@commons.apache.org
Subject: Re: [Math] MATH-816 (mixture model distribution)
On Thu, Oct 18, 2012 at 08:13:52AM -0400, Patrick Meyer wrote:
> I vote for simplicity. Current practice in the social sciences is to
> fit multiple models, each with a different number of components, and
> use fit statistics to choose the best model.
So... Do you vote for the current proposal (as in the latest attachment on
the JIRA page)? [Sorry for being dense. :-)]
[The (simple) "mixture model" code could be in 3.1, due to be out a couple
of weeks _ago_. :-}]
>
> There are some additional features I would like to see added and I
> have the code to contribute if it is not currently there. To be
> consistent with Mplus, we need have the algorithm use multiple random
> starts and run a few of the best starts to completion. Mplus uses this
> strategy to effectively overcome local minima.
Proposals welcome; please open a feature request with an outline of the
implementation.
Thanks,
Gilles
> [...]
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
For additional commands, e-mail: dev-help@commons.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
For additional commands, e-mail: dev-help@commons.apache.org
Re: [Math] MATH-816 (mixture model distribution)
Posted by Gilles Sadowski <gi...@harfang.homelinux.org>.
On Thu, Oct 18, 2012 at 08:13:52AM -0400, Patrick Meyer wrote:
> I vote for simplicity. Current practice in the social sciences is to fit
> multiple models, each with a different number of components, and use fit
> statistics to choose the best model.
So... Do you vote for the current proposal (as in the latest attachment
on the JIRA page)? [Sorry for being dense. :-)]
[The (simple) "mixture model" code could be in 3.1, due to be out a couple
of weeks _ago_. :-}]
>
> There are some additional features I would like to see added and I have the
> code to contribute if it is not currently there. To be consistent with
> Mplus, we need have the algorithm use multiple random starts and run a few
> of the best starts to completion. Mplus uses this strategy to effectively
> overcome local minima.
Proposals welcome; please open a feature request with an outline of the
implementation.
Thanks,
Gilles
> [...]
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
For additional commands, e-mail: dev-help@commons.apache.org