Code Freeze

[Grant Ingersoll <gs...@apache.org>]

OK, this time I really mean I think we are finally there!    Please  
review http://people.apache.org/~gsingers/staging-repo/mahout/org/apache/mahout/

I believe we have:
project tarball that contains the project source and is easily  
compilable (we don't have a pure "binary" distribution yet, but I can  
live with it)
Individual artifacts all published to Maven repo
Prebuilt Hadoop Job jars for both core and example
Taste war and easy demo setup

I've downloaded the tarballs and checked signatures and run the  
examples Job for both k-means and canopy

All in all, I think it is a pretty good 0.1.

I would like to cut a release candidate today at 4 PM EDT and call a  
vote.  I believe all the pieces are in place w/ the possible exception  
of the Dirichlet Job code.  Thus, can we agree no I will use what is  
on trunk at 4 PM EDT as the source for the RC?

How does that sound to everyone?

Cheers,
Grant

Re: Code Freeze

[Jeff Eastman <jd...@windwardsolutions.com>]

I added a routine to dump all the clusters. It's not as bad as I thought 
originally. It looks like my posterior std calculation is inflating the 
values by - I think - averaging the 60 component stds in the 
calculation. This causes a rapid collapse in the number of clusters to a 
single cluster which contains all the points. It seems to have a 
reasonable mean, but its std is too high.

Perhaps just a caveat about the initial nature of the model would suffice?

Jeff

Ted Dunning wrote:
> I would love to help, but I can't help before the deadline.
>
> On Thu, Mar 19, 2009 at 10:48 AM, Jeff Eastman
> <jd...@windwardsolutions.com>wrote:
>
>   
>> ...
>> I can commit what I have if anybody (Ted?) wants to go spelunking with me.
>> I can work on this up to the 4pm EDT deadline.
>>
>>
>>     
>
>   

Re: Code Freeze

[Ted Dunning <te...@gmail.com>]

I would love to help, but I can't help before the deadline.

On Thu, Mar 19, 2009 at 10:48 AM, Jeff Eastman
<jd...@windwardsolutions.com>wrote:

> ...
> I can commit what I have if anybody (Ted?) wants to go spelunking with me.
> I can work on this up to the 4pm EDT deadline.
>
>

Re: Code Freeze

[Jeff Eastman <jd...@windwardsolutions.com>]

I have a Dirichlet Job that runs, but it produces very odd results that 
I have not been able to debug, not yet at least. Basically the challenge 
is needing to produce a ModelDistribution and Model implementation that 
is suitable for a 60-d sample vector (what's the prior? pdf?). I've 
adapted the code from the 2-d Normal, but it suffers from underflow in 
the pdf() calculation which may be the source of the problem.

Also, using the SequenceFile format for the clusters which have 60-d 
centers is obfuscating. Finally, Dirichlet does not assign points to a 
single model so the final OutputDriver step would have to output the pdf 
of each point to each cluster.  I will add an output tool to help with 
the interpretation.

I can commit what I have if anybody (Ted?) wants to go spelunking with 
me. I can work on this up to the 4pm EDT deadline.

Jeff

Grant Ingersoll wrote:
> OK, this time I really mean I think we are finally there!    Please 
> review 
> http://people.apache.org/~gsingers/staging-repo/mahout/org/apache/mahout/
>
> I believe we have:
> project tarball that contains the project source and is easily 
> compilable (we don't have a pure "binary" distribution yet, but I can 
> live with it)
> Individual artifacts all published to Maven repo
> Prebuilt Hadoop Job jars for both core and example
> Taste war and easy demo setup
>
> I've downloaded the tarballs and checked signatures and run the 
> examples Job for both k-means and canopy
>
> All in all, I think it is a pretty good 0.1.
>
> I would like to cut a release candidate today at 4 PM EDT and call a 
> vote.  I believe all the pieces are in place w/ the possible exception 
> of the Dirichlet Job code.  Thus, can we agree no I will use what is 
> on trunk at 4 PM EDT as the source for the RC?
>
> How does that sound to everyone?
>
> Cheers,
> Grant
>
>