You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@commons.apache.org by Ajo Fod <aj...@gmail.com> on 2013/11/16 12:55:29 UTC
[math] Links to documents in javadocs for moment computation
Hello,
The document:
http://commons.apache.org/proper/commons-math/javadocs/api-3.2/index.htmlis
relevant to First/Second/Moments, could someone add it to those
classes
as well?
This link is already documented in :
*org.apache.commons.math3.stat.correlation.StorelessCovariance*
Re: [math] Links to documents in javadocs for moment computation
Posted by Ajo Fod <aj...@gmail.com>.
I'm sorry I was talking about the document : Formulas for Robust, One-Pass
Parallel Computation of Covariances and Arbitrary-Order Statistical
Moments<http://prod.sandia.gov/techlib/access-control.cgi/2008/086212.pdf>,
2008, Technical Report SAND2008-6212, Sandia National Laboratories.
... being relevant to the First/Second/... Moments.
-Ajo
On Sat, Nov 16, 2013 at 3:55 AM, Ajo Fod <aj...@gmail.com> wrote:
> Hello,
>
> The document:
> http://commons.apache.org/proper/commons-math/javadocs/api-3.2/index.htmlis relevant to First/Second/Moments, could someone add it to those classes
> as well?
>
>
> This link is already documented in :
>
>
>
>
> *org.apache.commons.math3.stat.correlation.StorelessCovariance*
>
>
Re: [math] Links to documents in javadocs for moment computation
Posted by Ajo Fod <aj...@gmail.com>.
Hello,
Unless someone wants to buy the West's reference, the closest thing to an
explanation for what is in those classes is in the Sandia report.
Perhaps you can add a link to the Sandia reference with the qualification
that it is not an exact description of the algo?
Cheers,
-Ajo.
On Sun, Nov 17, 2013 at 8:10 AM, Phil Steitz <ph...@gmail.com> wrote:
> On 11/17/13 6:02 AM, Ajo Fod wrote:
> > The Sandia report is free. The javadoc link is paywalled. I'd recommend
> > linking a free version if you want people to have unfettered access.
> >
> > Does the paywalled document describe how to do the
> aggregate/combine/reduce
> > operation we talked about in
> https://issues.apache.org/jira/browse/MATH-1016?
> > The Sandia report does.
> >
> > Also, I've been working off the Sandia report and I think that adequately
> > describes what is going on in the classes. Let me know if you are aware
> of
> > any differences.
>
> What the moment classes do is described exactly in the class javadoc
> inline. Per comment above, SecondMoment is *not* what the Sandia
> reference describes. West's algorithm is *not* what that reference
> describes either. If you can find a free reference to West's
> algorithm for the variance computation I will gladly add it.
>
> Phil
> >
> > Cheers,
> > -Ajo.
> >
> >
> > On Sat, Nov 16, 2013 at 11:34 AM, Phil Steitz <ph...@gmail.com>
> wrote:
> >
> >> On 11/16/13 3:55 AM, Ajo Fod wrote:
> >>> Hello,
> >>>
> >>> The document:
> >>>
> >>
> http://commons.apache.org/proper/commons-math/javadocs/api-3.2/index.htmlis
> >>> relevant to First/Second/Moments, could someone add it to those
> >>> classes
> >>> as well?
> >> The javadoc for first and second moments and variance now describes
> >> exactly what the updating algorithms are. The moments use
> >> recurrences similar to what the reference you provided suggests, but
> >> they are explicitly stated in the javadoc and pretty elementary. In
> >> general, when we can provide full algorithm descriptions in [math]
> >> javadoc, we prefer to do that, as it makes it clear exactly what we
> >> are computing. If what you are suggesting is that we change the
> >> updating formulas, please clarify and explain the numerical
> >> rationale for the change. The (storeless) variance computation uses
> >> West's updating algorithm [1], as documented in the class javadoc
> >> for Variance. One other thing to bear in mind here is that what we
> >> call "SecondMoment" is a not really the standard second moment
> >> statistic. The difference is made clear in the class javadoc. This
> >> statistic is a building block used by other stats and its definition
> >> reflects that.
> >>
> >> Phil
> >>
> >> [1] http://doi.acm.org/10.1145/359146.359152
> >>>
> >>> This link is already documented in :
> >>>
> >>>
> >>>
> >>>
> >>> *org.apache.commons.math3.stat.correlation.StorelessCovariance*
> >>>
> >>
> >> ---------------------------------------------------------------------
> >> To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
> >> For additional commands, e-mail: dev-help@commons.apache.org
> >>
> >>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
> For additional commands, e-mail: dev-help@commons.apache.org
>
>
Re: [math] Links to documents in javadocs for moment computation
Posted by Phil Steitz <ph...@gmail.com>.
On 11/17/13 6:02 AM, Ajo Fod wrote:
> The Sandia report is free. The javadoc link is paywalled. I'd recommend
> linking a free version if you want people to have unfettered access.
>
> Does the paywalled document describe how to do the aggregate/combine/reduce
> operation we talked about in https://issues.apache.org/jira/browse/MATH-1016?
> The Sandia report does.
>
> Also, I've been working off the Sandia report and I think that adequately
> describes what is going on in the classes. Let me know if you are aware of
> any differences.
What the moment classes do is described exactly in the class javadoc
inline. Per comment above, SecondMoment is *not* what the Sandia
reference describes. West's algorithm is *not* what that reference
describes either. If you can find a free reference to West's
algorithm for the variance computation I will gladly add it.
Phil
>
> Cheers,
> -Ajo.
>
>
> On Sat, Nov 16, 2013 at 11:34 AM, Phil Steitz <ph...@gmail.com> wrote:
>
>> On 11/16/13 3:55 AM, Ajo Fod wrote:
>>> Hello,
>>>
>>> The document:
>>>
>> http://commons.apache.org/proper/commons-math/javadocs/api-3.2/index.htmlis
>>> relevant to First/Second/Moments, could someone add it to those
>>> classes
>>> as well?
>> The javadoc for first and second moments and variance now describes
>> exactly what the updating algorithms are. The moments use
>> recurrences similar to what the reference you provided suggests, but
>> they are explicitly stated in the javadoc and pretty elementary. In
>> general, when we can provide full algorithm descriptions in [math]
>> javadoc, we prefer to do that, as it makes it clear exactly what we
>> are computing. If what you are suggesting is that we change the
>> updating formulas, please clarify and explain the numerical
>> rationale for the change. The (storeless) variance computation uses
>> West's updating algorithm [1], as documented in the class javadoc
>> for Variance. One other thing to bear in mind here is that what we
>> call "SecondMoment" is a not really the standard second moment
>> statistic. The difference is made clear in the class javadoc. This
>> statistic is a building block used by other stats and its definition
>> reflects that.
>>
>> Phil
>>
>> [1] http://doi.acm.org/10.1145/359146.359152
>>>
>>> This link is already documented in :
>>>
>>>
>>>
>>>
>>> *org.apache.commons.math3.stat.correlation.StorelessCovariance*
>>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
>> For additional commands, e-mail: dev-help@commons.apache.org
>>
>>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
For additional commands, e-mail: dev-help@commons.apache.org
Re: [math] Links to documents in javadocs for moment computation
Posted by Ajo Fod <aj...@gmail.com>.
The Sandia report is free. The javadoc link is paywalled. I'd recommend
linking a free version if you want people to have unfettered access.
Does the paywalled document describe how to do the aggregate/combine/reduce
operation we talked about in https://issues.apache.org/jira/browse/MATH-1016?
The Sandia report does.
Also, I've been working off the Sandia report and I think that adequately
describes what is going on in the classes. Let me know if you are aware of
any differences.
Cheers,
-Ajo.
On Sat, Nov 16, 2013 at 11:34 AM, Phil Steitz <ph...@gmail.com> wrote:
> On 11/16/13 3:55 AM, Ajo Fod wrote:
> > Hello,
> >
> > The document:
> >
> http://commons.apache.org/proper/commons-math/javadocs/api-3.2/index.htmlis
> > relevant to First/Second/Moments, could someone add it to those
> > classes
> > as well?
>
> The javadoc for first and second moments and variance now describes
> exactly what the updating algorithms are. The moments use
> recurrences similar to what the reference you provided suggests, but
> they are explicitly stated in the javadoc and pretty elementary. In
> general, when we can provide full algorithm descriptions in [math]
> javadoc, we prefer to do that, as it makes it clear exactly what we
> are computing. If what you are suggesting is that we change the
> updating formulas, please clarify and explain the numerical
> rationale for the change. The (storeless) variance computation uses
> West's updating algorithm [1], as documented in the class javadoc
> for Variance. One other thing to bear in mind here is that what we
> call "SecondMoment" is a not really the standard second moment
> statistic. The difference is made clear in the class javadoc. This
> statistic is a building block used by other stats and its definition
> reflects that.
>
> Phil
>
> [1] http://doi.acm.org/10.1145/359146.359152
> >
> >
> > This link is already documented in :
> >
> >
> >
> >
> > *org.apache.commons.math3.stat.correlation.StorelessCovariance*
> >
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
> For additional commands, e-mail: dev-help@commons.apache.org
>
>
Re: [math] Links to documents in javadocs for moment computation
Posted by Phil Steitz <ph...@gmail.com>.
On 11/16/13 3:55 AM, Ajo Fod wrote:
> Hello,
>
> The document:
> http://commons.apache.org/proper/commons-math/javadocs/api-3.2/index.htmlis
> relevant to First/Second/Moments, could someone add it to those
> classes
> as well?
The javadoc for first and second moments and variance now describes
exactly what the updating algorithms are. The moments use
recurrences similar to what the reference you provided suggests, but
they are explicitly stated in the javadoc and pretty elementary. In
general, when we can provide full algorithm descriptions in [math]
javadoc, we prefer to do that, as it makes it clear exactly what we
are computing. If what you are suggesting is that we change the
updating formulas, please clarify and explain the numerical
rationale for the change. The (storeless) variance computation uses
West's updating algorithm [1], as documented in the class javadoc
for Variance. One other thing to bear in mind here is that what we
call "SecondMoment" is a not really the standard second moment
statistic. The difference is made clear in the class javadoc. This
statistic is a building block used by other stats and its definition
reflects that.
Phil
[1] http://doi.acm.org/10.1145/359146.359152
>
>
> This link is already documented in :
>
>
>
>
> *org.apache.commons.math3.stat.correlation.StorelessCovariance*
>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
For additional commands, e-mail: dev-help@commons.apache.org