You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@commons.apache.org by Jörg Henning <jo...@spectos.com> on 2008/09/24 10:48:52 UTC
[MATH] Strange issue with FDistribution
Hello,
I'm relatively new to this kind of stuff, so pardon me, if I get some
things wrong.
We are using the FDistributionImpl from the commons.math project to do
some statistical calculations, namely receiving the upper and lower
boundaries of a confidence interval. Everything is working fine and the
results are matching our reference calculations.
However, the FDistribution behaves strange if a
denominatorDegreeOfFreedom of 2 is used, with an alpha-value of 0.95.
This results in an IllegalArgumentsException, stating:
Invalid endpoint parameters: lowerBound=0.0 initial=Infinity
upperBound=1.7976931348623157E308
coming from
org.apache.commons.math.analysis.UnivariateRealSolverUtils.bracket
The problem is the 'initial' parameter to that function, wich is
POSITIVE_INFINITY and therefore not within the boundaries. I already
pinned down the problem to the FDistributions getInitialDomain()-method,
wich goes like:
return getDenominatorDegreesOfFreedom() /
(getDenominatorDegreesOfFreedom() - 2.0);
Obviously, in case of denominatorDegreesOfFreedom == 2, this must lead
to a division-by-zero, resulting in POSTIVE_INFINITY. The result of this
operation is then directly passed into the
UnivariateRealSolverUtils.bracket() - method as second argument.
Can someone explain what this behaviour is about? Why should a
degreeOfFreedom of 2 not be allowed? Shouldn't this fact be mentioned in
the API-documentation? Is there a possible workaround?
Thanks in advance,
joerx
---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@commons.apache.org
For additional commands, e-mail: user-help@commons.apache.org
Re: [MATH] Strange issue with FDistribution
Posted by Jörg Henning <jo...@spectos.com>.
I already took Brents solution from JIRA and subclassed the
FDistribution from 1.2, to avoid using the svn version of commons.math.
It works as far as I see, the results match our reference calculations.
I do not understand too much of that mathematical theory, I just ported
some algorithm from PHP (*douh*) ;-)
Thanks for all help, you do great work here, regards
Joerg
On Fri, 2008-09-26 at 00:19 -0700, Ted Dunning wrote:
> Take my comment with a grain of salt. I didn't look at the code (this
> time... looked previously).
>
> On Fri, Sep 26, 2008 at 12:09 AM, Jörg Henning <jo...@spectos.com>wrote:
>
> > This was exactly what I needed to know. :-) Thank you for your help
> > guys!
> >
> > On Thu, 2008-09-25 at 17:27 -0500, Brent Worden wrote:
> > > You can always subclass FDistributionImpl and override the method.
> > > The return value from this method serves as a starting value for a
> > > root finding rountine; the default mechnanism used to compute inverse
> > > CDFs. So, the exact number returned is not crucial to obtain accurate
> > > results. The inverse CDF rountines will just perform better based on
> > > this starting value.
> > >
> > > HTH,
> > >
> > > Brent.
> > >
> > > On Thu, Sep 25, 2008 at 3:27 AM, Jörg Henning <jo...@spectos.com>
> > wrote:
> > > > thanks for your reply Phil. As requested, I opened a Bug in JIRA.
> > > > (Please give me a hint if I messed anything up)
> > > >
> > > > Anyone knows a possible quick-fix or workaround on the issue?
> > > >
> > >
> > > ---------------------------------------------------------------------
> > > To unsubscribe, e-mail: user-unsubscribe@commons.apache.org
> > > For additional commands, e-mail: user-help@commons.apache.org
> > >
> > Jörg Henning
> > Teamleiter SpectosSolutions
> >
> > --
> > Spectos GmbH
> > Theaterstrasse 6
> > 01067 Dresden
> >
> > Fon: +49 351 89 66 34 10
> > Fax: +49 351 46 56 10 42
> > Mobil: +49 163 444 17 37
> >
> > Web: www.spectos.com
> >
> > Spectos GmbH
> > Firmensitz: Theaterstrasse 6, 01067 Dresden
> > Registergericht: Amtsgericht Dresden HRB 24787
> > Geschäftsführer: Niels Delater
> >
> > Diese E-Mail und ihre eventuell beigefügten Anhänge enthalten
> > vertrauliche Informationen, die rechtlich besonders geschützt sein
> > können, z.B. durch die Berufspflichten des Steuerberaters. Diese
> > Informationen sind ausschließlich für die als Adressaten genannten
> > Personen bestimmt. Wenn Sie nicht der angeschriebene Empfänger sind und
> > diese E-Mail durch einen Übertragungsfehler erhalten haben, informieren
> > Sie uns bitte sofort per E-Mail, Telefon oder Fax und löschen Sie danach
> > vorliegende E-Mail. Das unbefugte Kopieren dieser E-Mail, ihrer
> > eventuell beigefügten Anhänge sowie die unbefugte Weitergabe der
> > enthaltenen Informationen an Dritte ist nicht gestattet. Wir danken für
> > Ihre Hilfe.
> >
> > This e-mail message together with its attachments, if any, is
> > confidential and may contain information subject to legal privilege,
> > e.g. to professional obligations of tax advisors. The information
> > contained in this e-mail or its attachments is intended solely for the
> > persons named as addressees. If you are not the intended recipient or
> > have received this e-mail in error, please advise us immediately by
> > e-mail, telephone or fax and delete this message. Any unauthorized
> > copying of this message or unauthorized distribution of the information
> > contained herein is prohibited. Thank you for your co-operation.
> >
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: user-unsubscribe@commons.apache.org
> > For additional commands, e-mail: user-help@commons.apache.org
> >
> >
>
>
Jörg Henning
Teamleiter SpectosSolutions
--
Spectos GmbH
Theaterstrasse 6
01067 Dresden
Fon: +49 351 89 66 34 10
Fax: +49 351 46 56 10 42
Mobil: +49 163 444 17 37
Web: www.spectos.com
Spectos GmbH
Firmensitz: Theaterstrasse 6, 01067 Dresden
Registergericht: Amtsgericht Dresden HRB 24787
Geschäftsführer: Niels Delater
Diese E-Mail und ihre eventuell beigefügten Anhänge enthalten
vertrauliche Informationen, die rechtlich besonders geschützt sein
können, z.B. durch die Berufspflichten des Steuerberaters. Diese
Informationen sind ausschließlich für die als Adressaten genannten
Personen bestimmt. Wenn Sie nicht der angeschriebene Empfänger sind und
diese E-Mail durch einen Übertragungsfehler erhalten haben, informieren
Sie uns bitte sofort per E-Mail, Telefon oder Fax und löschen Sie danach
vorliegende E-Mail. Das unbefugte Kopieren dieser E-Mail, ihrer
eventuell beigefügten Anhänge sowie die unbefugte Weitergabe der
enthaltenen Informationen an Dritte ist nicht gestattet. Wir danken für
Ihre Hilfe.
This e-mail message together with its attachments, if any, is
confidential and may contain information subject to legal privilege,
e.g. to professional obligations of tax advisors. The information
contained in this e-mail or its attachments is intended solely for the
persons named as addressees. If you are not the intended recipient or
have received this e-mail in error, please advise us immediately by
e-mail, telephone or fax and delete this message. Any unauthorized
copying of this message or unauthorized distribution of the information
contained herein is prohibited. Thank you for your co-operation.
---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@commons.apache.org
For additional commands, e-mail: user-help@commons.apache.org
Re: [MATH] Strange issue with FDistribution
Posted by Ted Dunning <te...@gmail.com>.
Take my comment with a grain of salt. I didn't look at the code (this
time... looked previously).
On Fri, Sep 26, 2008 at 12:09 AM, Jörg Henning <jo...@spectos.com>wrote:
> This was exactly what I needed to know. :-) Thank you for your help
> guys!
>
> On Thu, 2008-09-25 at 17:27 -0500, Brent Worden wrote:
> > You can always subclass FDistributionImpl and override the method.
> > The return value from this method serves as a starting value for a
> > root finding rountine; the default mechnanism used to compute inverse
> > CDFs. So, the exact number returned is not crucial to obtain accurate
> > results. The inverse CDF rountines will just perform better based on
> > this starting value.
> >
> > HTH,
> >
> > Brent.
> >
> > On Thu, Sep 25, 2008 at 3:27 AM, Jörg Henning <jo...@spectos.com>
> wrote:
> > > thanks for your reply Phil. As requested, I opened a Bug in JIRA.
> > > (Please give me a hint if I messed anything up)
> > >
> > > Anyone knows a possible quick-fix or workaround on the issue?
> > >
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: user-unsubscribe@commons.apache.org
> > For additional commands, e-mail: user-help@commons.apache.org
> >
> Jörg Henning
> Teamleiter SpectosSolutions
>
> --
> Spectos GmbH
> Theaterstrasse 6
> 01067 Dresden
>
> Fon: +49 351 89 66 34 10
> Fax: +49 351 46 56 10 42
> Mobil: +49 163 444 17 37
>
> Web: www.spectos.com
>
> Spectos GmbH
> Firmensitz: Theaterstrasse 6, 01067 Dresden
> Registergericht: Amtsgericht Dresden HRB 24787
> Geschäftsführer: Niels Delater
>
> Diese E-Mail und ihre eventuell beigefügten Anhänge enthalten
> vertrauliche Informationen, die rechtlich besonders geschützt sein
> können, z.B. durch die Berufspflichten des Steuerberaters. Diese
> Informationen sind ausschließlich für die als Adressaten genannten
> Personen bestimmt. Wenn Sie nicht der angeschriebene Empfänger sind und
> diese E-Mail durch einen Übertragungsfehler erhalten haben, informieren
> Sie uns bitte sofort per E-Mail, Telefon oder Fax und löschen Sie danach
> vorliegende E-Mail. Das unbefugte Kopieren dieser E-Mail, ihrer
> eventuell beigefügten Anhänge sowie die unbefugte Weitergabe der
> enthaltenen Informationen an Dritte ist nicht gestattet. Wir danken für
> Ihre Hilfe.
>
> This e-mail message together with its attachments, if any, is
> confidential and may contain information subject to legal privilege,
> e.g. to professional obligations of tax advisors. The information
> contained in this e-mail or its attachments is intended solely for the
> persons named as addressees. If you are not the intended recipient or
> have received this e-mail in error, please advise us immediately by
> e-mail, telephone or fax and delete this message. Any unauthorized
> copying of this message or unauthorized distribution of the information
> contained herein is prohibited. Thank you for your co-operation.
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: user-unsubscribe@commons.apache.org
> For additional commands, e-mail: user-help@commons.apache.org
>
>
--
ted
Re: [MATH] Strange issue with FDistribution
Posted by Jörg Henning <jo...@spectos.com>.
This was exactly what I needed to know. :-) Thank you for your help
guys!
On Thu, 2008-09-25 at 17:27 -0500, Brent Worden wrote:
> You can always subclass FDistributionImpl and override the method.
> The return value from this method serves as a starting value for a
> root finding rountine; the default mechnanism used to compute inverse
> CDFs. So, the exact number returned is not crucial to obtain accurate
> results. The inverse CDF rountines will just perform better based on
> this starting value.
>
> HTH,
>
> Brent.
>
> On Thu, Sep 25, 2008 at 3:27 AM, Jörg Henning <jo...@spectos.com> wrote:
> > thanks for your reply Phil. As requested, I opened a Bug in JIRA.
> > (Please give me a hint if I messed anything up)
> >
> > Anyone knows a possible quick-fix or workaround on the issue?
> >
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: user-unsubscribe@commons.apache.org
> For additional commands, e-mail: user-help@commons.apache.org
>
Jörg Henning
Teamleiter SpectosSolutions
--
Spectos GmbH
Theaterstrasse 6
01067 Dresden
Fon: +49 351 89 66 34 10
Fax: +49 351 46 56 10 42
Mobil: +49 163 444 17 37
Web: www.spectos.com
Spectos GmbH
Firmensitz: Theaterstrasse 6, 01067 Dresden
Registergericht: Amtsgericht Dresden HRB 24787
Geschäftsführer: Niels Delater
Diese E-Mail und ihre eventuell beigefügten Anhänge enthalten
vertrauliche Informationen, die rechtlich besonders geschützt sein
können, z.B. durch die Berufspflichten des Steuerberaters. Diese
Informationen sind ausschließlich für die als Adressaten genannten
Personen bestimmt. Wenn Sie nicht der angeschriebene Empfänger sind und
diese E-Mail durch einen Übertragungsfehler erhalten haben, informieren
Sie uns bitte sofort per E-Mail, Telefon oder Fax und löschen Sie danach
vorliegende E-Mail. Das unbefugte Kopieren dieser E-Mail, ihrer
eventuell beigefügten Anhänge sowie die unbefugte Weitergabe der
enthaltenen Informationen an Dritte ist nicht gestattet. Wir danken für
Ihre Hilfe.
This e-mail message together with its attachments, if any, is
confidential and may contain information subject to legal privilege,
e.g. to professional obligations of tax advisors. The information
contained in this e-mail or its attachments is intended solely for the
persons named as addressees. If you are not the intended recipient or
have received this e-mail in error, please advise us immediately by
e-mail, telephone or fax and delete this message. Any unauthorized
copying of this message or unauthorized distribution of the information
contained herein is prohibited. Thank you for your co-operation.
---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@commons.apache.org
For additional commands, e-mail: user-help@commons.apache.org
Re: [MATH] Strange issue with FDistribution
Posted by Ted Dunning <te...@gmail.com>.
More importantly, I think that all of the root finding routines will expand
their original region as necessary. Thus, you can just check for the
exception case and put in some nominal first value like 1e6. This may cost
you a little speed in some cases, but not all that much.
On Thu, Sep 25, 2008 at 3:27 PM, Brent Worden <br...@gmail.com>wrote:
> You can always subclass FDistributionImpl and override the method.
> The return value from this method serves as a starting value for a
> root finding rountine; the default mechnanism used to compute inverse
> CDFs. So, the exact number returned is not crucial to obtain accurate
> results. The inverse CDF rountines will just perform better based on
> this starting value.
>
> HTH,
>
> Brent.
>
> On Thu, Sep 25, 2008 at 3:27 AM, Jörg Henning <jo...@spectos.com>
> wrote:
> > thanks for your reply Phil. As requested, I opened a Bug in JIRA.
> > (Please give me a hint if I messed anything up)
> >
> > Anyone knows a possible quick-fix or workaround on the issue?
> >
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: user-unsubscribe@commons.apache.org
> For additional commands, e-mail: user-help@commons.apache.org
>
>
--
ted
Re: [MATH] Strange issue with FDistribution
Posted by Brent Worden <br...@gmail.com>.
You can always subclass FDistributionImpl and override the method.
The return value from this method serves as a starting value for a
root finding rountine; the default mechnanism used to compute inverse
CDFs. So, the exact number returned is not crucial to obtain accurate
results. The inverse CDF rountines will just perform better based on
this starting value.
HTH,
Brent.
On Thu, Sep 25, 2008 at 3:27 AM, Jörg Henning <jo...@spectos.com> wrote:
> thanks for your reply Phil. As requested, I opened a Bug in JIRA.
> (Please give me a hint if I messed anything up)
>
> Anyone knows a possible quick-fix or workaround on the issue?
>
---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@commons.apache.org
For additional commands, e-mail: user-help@commons.apache.org
Re: [MATH] Strange issue with FDistribution
Posted by Jörg Henning <jo...@spectos.com>.
thanks for your reply Phil. As requested, I opened a Bug in JIRA.
(Please give me a hint if I messed anything up)
Anyone knows a possible quick-fix or workaround on the issue?
On Wed, 2008-09-24 at 21:37 -0400, Phil Steitz wrote:
> This looks like a bug. At least it is a documentation bug. Can you pls
> open a JIRA ticket so we do not lose track of it. Follow the
> instructions here
> http://commons.apache.org/math/issue-tracking.html
> Thanks for reporting this.
>
> Phil
>
>
> Jörg Henning wrote:
> > Hello,
> >
> > I'm relatively new to this kind of stuff, so pardon me, if I get some
> > things wrong.
> >
> > We are using the FDistributionImpl from the commons.math project to do
> > some statistical calculations, namely receiving the upper and lower
> > boundaries of a confidence interval. Everything is working fine and the
> > results are matching our reference calculations.
> >
> > However, the FDistribution behaves strange if a
> > denominatorDegreeOfFreedom of 2 is used, with an alpha-value of 0.95.
> > This results in an IllegalArgumentsException, stating:
> >
> > Invalid endpoint parameters: lowerBound=0.0 initial=Infinity
> > upperBound=1.7976931348623157E308
> >
> > coming from
> > org.apache.commons.math.analysis.UnivariateRealSolverUtils.bracket
> >
> > The problem is the 'initial' parameter to that function, wich is
> > POSITIVE_INFINITY and therefore not within the boundaries. I already
> > pinned down the problem to the FDistributions getInitialDomain()-method,
> > wich goes like:
> >
> > return getDenominatorDegreesOfFreedom() /
> > (getDenominatorDegreesOfFreedom() - 2.0);
> >
> > Obviously, in case of denominatorDegreesOfFreedom == 2, this must lead
> > to a division-by-zero, resulting in POSTIVE_INFINITY. The result of this
> > operation is then directly passed into the
> > UnivariateRealSolverUtils.bracket() - method as second argument.
> >
> >
> > Can someone explain what this behaviour is about? Why should a
> > degreeOfFreedom of 2 not be allowed? Shouldn't this fact be mentioned in
> > the API-documentation? Is there a possible workaround?
> >
> > Thanks in advance,
> >
> >
> > joerx
> >
> >
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: user-unsubscribe@commons.apache.org
> > For additional commands, e-mail: user-help@commons.apache.org
> >
> >
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: user-unsubscribe@commons.apache.org
> For additional commands, e-mail: user-help@commons.apache.org
>
Jörg Henning
Teamleiter SpectosSolutions
--
Spectos GmbH
Theaterstrasse 6
01067 Dresden
Fon: +49 351 89 66 34 10
Fax: +49 351 46 56 10 42
Mobil: +49 163 444 17 37
Web: www.spectos.com
Spectos GmbH
Firmensitz: Theaterstrasse 6, 01067 Dresden
Registergericht: Amtsgericht Dresden HRB 24787
Geschäftsführer: Niels Delater
Diese E-Mail und ihre eventuell beigefügten Anhänge enthalten
vertrauliche Informationen, die rechtlich besonders geschützt sein
können, z.B. durch die Berufspflichten des Steuerberaters. Diese
Informationen sind ausschließlich für die als Adressaten genannten
Personen bestimmt. Wenn Sie nicht der angeschriebene Empfänger sind und
diese E-Mail durch einen Übertragungsfehler erhalten haben, informieren
Sie uns bitte sofort per E-Mail, Telefon oder Fax und löschen Sie danach
vorliegende E-Mail. Das unbefugte Kopieren dieser E-Mail, ihrer
eventuell beigefügten Anhänge sowie die unbefugte Weitergabe der
enthaltenen Informationen an Dritte ist nicht gestattet. Wir danken für
Ihre Hilfe.
This e-mail message together with its attachments, if any, is
confidential and may contain information subject to legal privilege,
e.g. to professional obligations of tax advisors. The information
contained in this e-mail or its attachments is intended solely for the
persons named as addressees. If you are not the intended recipient or
have received this e-mail in error, please advise us immediately by
e-mail, telephone or fax and delete this message. Any unauthorized
copying of this message or unauthorized distribution of the information
contained herein is prohibited. Thank you for your co-operation.
---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@commons.apache.org
For additional commands, e-mail: user-help@commons.apache.org
Re: [MATH] Strange issue with FDistribution
Posted by Phil Steitz <ph...@steitz.com>.
This looks like a bug. At least it is a documentation bug. Can you pls
open a JIRA ticket so we do not lose track of it. Follow the
instructions here
http://commons.apache.org/math/issue-tracking.html
Thanks for reporting this.
Phil
Jörg Henning wrote:
> Hello,
>
> I'm relatively new to this kind of stuff, so pardon me, if I get some
> things wrong.
>
> We are using the FDistributionImpl from the commons.math project to do
> some statistical calculations, namely receiving the upper and lower
> boundaries of a confidence interval. Everything is working fine and the
> results are matching our reference calculations.
>
> However, the FDistribution behaves strange if a
> denominatorDegreeOfFreedom of 2 is used, with an alpha-value of 0.95.
> This results in an IllegalArgumentsException, stating:
>
> Invalid endpoint parameters: lowerBound=0.0 initial=Infinity
> upperBound=1.7976931348623157E308
>
> coming from
> org.apache.commons.math.analysis.UnivariateRealSolverUtils.bracket
>
> The problem is the 'initial' parameter to that function, wich is
> POSITIVE_INFINITY and therefore not within the boundaries. I already
> pinned down the problem to the FDistributions getInitialDomain()-method,
> wich goes like:
>
> return getDenominatorDegreesOfFreedom() /
> (getDenominatorDegreesOfFreedom() - 2.0);
>
> Obviously, in case of denominatorDegreesOfFreedom == 2, this must lead
> to a division-by-zero, resulting in POSTIVE_INFINITY. The result of this
> operation is then directly passed into the
> UnivariateRealSolverUtils.bracket() - method as second argument.
>
>
> Can someone explain what this behaviour is about? Why should a
> degreeOfFreedom of 2 not be allowed? Shouldn't this fact be mentioned in
> the API-documentation? Is there a possible workaround?
>
> Thanks in advance,
>
>
> joerx
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: user-unsubscribe@commons.apache.org
> For additional commands, e-mail: user-help@commons.apache.org
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@commons.apache.org
For additional commands, e-mail: user-help@commons.apache.org