You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@commons.apache.org by "Patrick Meyer (JIRA)" <ji...@apache.org> on 2012/11/26 00:38:58 UTC
[jira] [Created] (MATH-909) FDistribution NoBracketingException in
BrentSolver
Patrick Meyer created MATH-909:
----------------------------------
Summary: FDistribution NoBracketingException in BrentSolver
Key: MATH-909
URL: https://issues.apache.org/jira/browse/MATH-909
Project: Commons Math
Issue Type: Bug
Affects Versions: 3.0
Reporter: Patrick Meyer
I get an exception when running the code below. the exception is
{code}
function values at endpoints do not have different signs, endpoints: [0, 1.002], values: [-0.025, -∞]
{code}
The problematic code:
{code}
double df1 = 10675;
double df2 = 501725;
FDistribution fDist = new FDistribution(df1, df2); System.out.println(fDist.inverseCumulativeProbability(0.025));//NoBracketingException
{code}
However, R returns the value 0.9733505. The R code is:
{code}
qf(p=.025, df1=10675, df2=501725)
{code}
I don't know enough about the FDistribution class to know the solution to the exception, but I thought I would report it.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (MATH-909) FDistribution NoBracketingException in
BrentSolver
Posted by "Sébastien Brisard (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MATH-909?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Sébastien Brisard resolved MATH-909.
------------------------------------
Resolution: Cannot Reproduce
New implementation of {{Gamma.logGamma}} seems to solve this issue.
A test equivalent to the extreme test proposed by Patrick will be included in the unit tests of the incomplete beta function.
> FDistribution NoBracketingException in BrentSolver
> --------------------------------------------------
>
> Key: MATH-909
> URL: https://issues.apache.org/jira/browse/MATH-909
> Project: Commons Math
> Issue Type: Bug
> Affects Versions: 3.0
> Reporter: Patrick Meyer
> Original Estimate: 24h
> Remaining Estimate: 24h
>
> I get an exception when running the code below. the exception is
> {code}
> function values at endpoints do not have different signs, endpoints: [0, 1.002], values: [-0.025, -∞]
> {code}
> The problematic code:
> {code}
> double df1 = 10675;
> double df2 = 501725;
> FDistribution fDist = new FDistribution(df1, df2);
> System.out.println(fDist.inverseCumulativeProbability(0.025));//NoBracketingException
> {code}
> However, R returns the value 0.9733505. The R code is:
> {code}
> qf(p=.025, df1=10675, df2=501725)
> {code}
> I don't know enough about the FDistribution class to know the solution to the exception, but I thought I would report it.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MATH-909) FDistribution NoBracketingException
in BrentSolver
Posted by "Sébastien Brisard (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MATH-909?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13503793#comment-13503793 ]
Sébastien Brisard commented on MATH-909:
----------------------------------------
Patrick, the arguments of the incomplete beta function are large, so a loss of accuracy is to be expected. We are working on it (see MATH-738), but there is still a lot to do.
Meanwhile, I've used MAXIMA to compute in multiple precision the result you are asking for. Here is the script I wrote
{code}
kill(all);
load(newton1);
fpprec : 128;
d1 : 10675;
d2 : 501725;
p : 0.025b0;
F(x) := block(
y : d1 * x / (d1 * x + d2),
beta_incomplete(0.5b0 * d1, 0.5b0 * d2, y) / beta(0.5b0 * d1, 0.5b0 * d2)
);
x0 : 0.9733505b0;
x1 : newton(F(x) - p, x, x0, 10**(-fpprec+1));
print(x1);
{code}
Basically, {{F(X)}} is the cumulative distribution function, and {{p}} is the target. I then look for an approximate solution of {{F(X) == p}}, starting from {{x0 = 0.9733505b0}} which is the value returned by R.
Here is the result I get
{code}
9.730779455235159798387713198169726833852358770410804399004797817813867760559669310857118225960097152689836562586854107391295331b-1
{code}
It seems to me that the value returned by CM is more accurate than the one returned by R. Could you carry out an independent check on that?
> FDistribution NoBracketingException in BrentSolver
> --------------------------------------------------
>
> Key: MATH-909
> URL: https://issues.apache.org/jira/browse/MATH-909
> Project: Commons Math
> Issue Type: Bug
> Affects Versions: 3.0
> Reporter: Patrick Meyer
> Original Estimate: 24h
> Remaining Estimate: 24h
>
> I get an exception when running the code below. the exception is
> {code}
> function values at endpoints do not have different signs, endpoints: [0, 1.002], values: [-0.025, -∞]
> {code}
> The problematic code:
> {code}
> double df1 = 10675;
> double df2 = 501725;
> FDistribution fDist = new FDistribution(df1, df2);
> System.out.println(fDist.inverseCumulativeProbability(0.025));//NoBracketingException
> {code}
> However, R returns the value 0.9733505. The R code is:
> {code}
> qf(p=.025, df1=10675, df2=501725)
> {code}
> I don't know enough about the FDistribution class to know the solution to the exception, but I thought I would report it.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MATH-909) FDistribution NoBracketingException
in BrentSolver
Posted by "Gilles (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MATH-909?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13503849#comment-13503849 ]
Gilles commented on MATH-909:
-----------------------------
bq. I suggest we keep it open until the unit test has been implemented.
As this test would not demonstrate a current buggy behaviour of CM, it should not be linked to this issue (i.e. not need to create a "testMath909()" method).
I think that the issue can be resolved now as "Cannot reproduce".
Of course, unit tests are always welcome. But in this case, your approach of fairly exhaustively checking the result of the underlying functions by comparison with "maxima" makes it superfluous to separately test the "client" (i.e. "FDistribution").
> FDistribution NoBracketingException in BrentSolver
> --------------------------------------------------
>
> Key: MATH-909
> URL: https://issues.apache.org/jira/browse/MATH-909
> Project: Commons Math
> Issue Type: Bug
> Affects Versions: 3.0
> Reporter: Patrick Meyer
> Original Estimate: 24h
> Remaining Estimate: 24h
>
> I get an exception when running the code below. the exception is
> {code}
> function values at endpoints do not have different signs, endpoints: [0, 1.002], values: [-0.025, -∞]
> {code}
> The problematic code:
> {code}
> double df1 = 10675;
> double df2 = 501725;
> FDistribution fDist = new FDistribution(df1, df2);
> System.out.println(fDist.inverseCumulativeProbability(0.025));//NoBracketingException
> {code}
> However, R returns the value 0.9733505. The R code is:
> {code}
> qf(p=.025, df1=10675, df2=501725)
> {code}
> I don't know enough about the FDistribution class to know the solution to the exception, but I thought I would report it.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MATH-909) FDistribution NoBracketingException
in BrentSolver
Posted by "Patrick Meyer (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MATH-909?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13503773#comment-13503773 ]
Patrick Meyer commented on MATH-909:
------------------------------------
I tested it with the most recent version and I also got 0.9730779455126357. The exception no longer occurs, but the result still seems to be to different from the value reported by R.
> FDistribution NoBracketingException in BrentSolver
> --------------------------------------------------
>
> Key: MATH-909
> URL: https://issues.apache.org/jira/browse/MATH-909
> Project: Commons Math
> Issue Type: Bug
> Affects Versions: 3.0
> Reporter: Patrick Meyer
> Original Estimate: 24h
> Remaining Estimate: 24h
>
> I get an exception when running the code below. the exception is
> {code}
> function values at endpoints do not have different signs, endpoints: [0, 1.002], values: [-0.025, -∞]
> {code}
> The problematic code:
> {code}
> double df1 = 10675;
> double df2 = 501725;
> FDistribution fDist = new FDistribution(df1, df2);
> System.out.println(fDist.inverseCumulativeProbability(0.025));//NoBracketingException
> {code}
> However, R returns the value 0.9733505. The R code is:
> {code}
> qf(p=.025, df1=10675, df2=501725)
> {code}
> I don't know enough about the FDistribution class to know the solution to the exception, but I thought I would report it.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MATH-909) FDistribution NoBracketingException
in BrentSolver
Posted by "Patrick Meyer (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MATH-909?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13503818#comment-13503818 ]
Patrick Meyer commented on MATH-909:
------------------------------------
Ha, you're right! R is less accurate. I checked the value with Stata (code listed below) and the result was 0.97307795. I'm satisfied. CM returns a more accurate value.
{code}
display invF(10675, 501725, 0.025)
{code}
> FDistribution NoBracketingException in BrentSolver
> --------------------------------------------------
>
> Key: MATH-909
> URL: https://issues.apache.org/jira/browse/MATH-909
> Project: Commons Math
> Issue Type: Bug
> Affects Versions: 3.0
> Reporter: Patrick Meyer
> Original Estimate: 24h
> Remaining Estimate: 24h
>
> I get an exception when running the code below. the exception is
> {code}
> function values at endpoints do not have different signs, endpoints: [0, 1.002], values: [-0.025, -∞]
> {code}
> The problematic code:
> {code}
> double df1 = 10675;
> double df2 = 501725;
> FDistribution fDist = new FDistribution(df1, df2);
> System.out.println(fDist.inverseCumulativeProbability(0.025));//NoBracketingException
> {code}
> However, R returns the value 0.9733505. The R code is:
> {code}
> qf(p=.025, df1=10675, df2=501725)
> {code}
> I don't know enough about the FDistribution class to know the solution to the exception, but I thought I would report it.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MATH-909) FDistribution NoBracketingException
in BrentSolver
Posted by "Thomas Neidhart (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MATH-909?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13503703#comment-13503703 ]
Thomas Neidhart commented on MATH-909:
--------------------------------------
Ah ok my fault, but adding more regression tests is definitely useful.
Will add the test case later on.
> FDistribution NoBracketingException in BrentSolver
> --------------------------------------------------
>
> Key: MATH-909
> URL: https://issues.apache.org/jira/browse/MATH-909
> Project: Commons Math
> Issue Type: Bug
> Affects Versions: 3.0
> Reporter: Patrick Meyer
> Original Estimate: 24h
> Remaining Estimate: 24h
>
> I get an exception when running the code below. the exception is
> {code}
> function values at endpoints do not have different signs, endpoints: [0, 1.002], values: [-0.025, -∞]
> {code}
> The problematic code:
> {code}
> double df1 = 10675;
> double df2 = 501725;
> FDistribution fDist = new FDistribution(df1, df2);
> System.out.println(fDist.inverseCumulativeProbability(0.025));//NoBracketingException
> {code}
> However, R returns the value 0.9733505. The R code is:
> {code}
> qf(p=.025, df1=10675, df2=501725)
> {code}
> I don't know enough about the FDistribution class to know the solution to the exception, but I thought I would report it.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MATH-909) FDistribution NoBracketingException in
BrentSolver
Posted by "Patrick Meyer (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MATH-909?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Patrick Meyer updated MATH-909:
-------------------------------
Description:
I get an exception when running the code below. the exception is
{code}
function values at endpoints do not have different signs, endpoints: [0, 1.002], values: [-0.025, -∞]
{code}
The problematic code:
{code}
double df1 = 10675;
double df2 = 501725;
FDistribution fDist = new FDistribution(df1, df2);
System.out.println(fDist.inverseCumulativeProbability(0.025));//NoBracketingException
{code}
However, R returns the value 0.9733505. The R code is:
{code}
qf(p=.025, df1=10675, df2=501725)
{code}
I don't know enough about the FDistribution class to know the solution to the exception, but I thought I would report it.
was:
I get an exception when running the code below. the exception is
{code}
function values at endpoints do not have different signs, endpoints: [0, 1.002], values: [-0.025, -∞]
{code}
The problematic code:
{code}
double df1 = 10675;
double df2 = 501725;
FDistribution fDist = new FDistribution(df1, df2); System.out.println(fDist.inverseCumulativeProbability(0.025));//NoBracketingException
{code}
However, R returns the value 0.9733505. The R code is:
{code}
qf(p=.025, df1=10675, df2=501725)
{code}
I don't know enough about the FDistribution class to know the solution to the exception, but I thought I would report it.
> FDistribution NoBracketingException in BrentSolver
> --------------------------------------------------
>
> Key: MATH-909
> URL: https://issues.apache.org/jira/browse/MATH-909
> Project: Commons Math
> Issue Type: Bug
> Affects Versions: 3.0
> Reporter: Patrick Meyer
> Original Estimate: 24h
> Remaining Estimate: 24h
>
> I get an exception when running the code below. the exception is
> {code}
> function values at endpoints do not have different signs, endpoints: [0, 1.002], values: [-0.025, -∞]
> {code}
> The problematic code:
> {code}
> double df1 = 10675;
> double df2 = 501725;
> FDistribution fDist = new FDistribution(df1, df2);
> System.out.println(fDist.inverseCumulativeProbability(0.025));//NoBracketingException
> {code}
> However, R returns the value 0.9733505. The R code is:
> {code}
> qf(p=.025, df1=10675, df2=501725)
> {code}
> I don't know enough about the FDistribution class to know the solution to the exception, but I thought I would report it.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Comment Edited] (MATH-909) FDistribution
NoBracketingException in BrentSolver
Posted by "Sébastien Brisard (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MATH-909?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13503793#comment-13503793 ]
Sébastien Brisard edited comment on MATH-909 at 11/26/12 2:19 PM:
------------------------------------------------------------------
Patrick, the arguments of the incomplete beta function are large, so a loss of accuracy is to be expected. We are working on it (see MATH-738), but there is still a lot to do.
Meanwhile, I've used MAXIMA to compute in multiple precision the result you are asking for. Here is the script I wrote
{code}
kill(all);
load(newton1);
fpprec : 128;
d1 : 10675;
d2 : 501725;
p : 0.025b0;
F(x) := block(
y : d1 * x / (d1 * x + d2),
beta_incomplete(0.5b0 * d1, 0.5b0 * d2, y) / beta(0.5b0 * d1, 0.5b0 * d2)
);
x0 : 0.9733505b0;
x1 : newton(F(x) - p, x, x0, 10**(-fpprec+1));
print(x1);
{code}
Basically, {{F(X)}} is the cumulative distribution function, and {{p}} is the target. I then look for an approximate solution of {{F(X) == p}}, starting from {{x0 = 0.9733505b0}} which is the value returned by R.
Here is the result I get
{code}
9.730779455235159798387713198169726833852358770410804399004797817813867760559669310857118225960097152689836562586854107391295331b-1
{code}
It seems to me that the value returned by CM is more accurate than the one returned by R. Could you carry out an independent check on that?
NOTA: for some reason, I had to compute the regularized incomplete beta function as the ratio of the incomplete beta function and the beta function, as the function {{beta_incomplete_regularized (a, b, z)}} led to errors.
was (Author: celestin):
Patrick, the arguments of the incomplete beta function are large, so a loss of accuracy is to be expected. We are working on it (see MATH-738), but there is still a lot to do.
Meanwhile, I've used MAXIMA to compute in multiple precision the result you are asking for. Here is the script I wrote
{code}
kill(all);
load(newton1);
fpprec : 128;
d1 : 10675;
d2 : 501725;
p : 0.025b0;
F(x) := block(
y : d1 * x / (d1 * x + d2),
beta_incomplete(0.5b0 * d1, 0.5b0 * d2, y) / beta(0.5b0 * d1, 0.5b0 * d2)
);
x0 : 0.9733505b0;
x1 : newton(F(x) - p, x, x0, 10**(-fpprec+1));
print(x1);
{code}
Basically, {{F(X)}} is the cumulative distribution function, and {{p}} is the target. I then look for an approximate solution of {{F(X) == p}}, starting from {{x0 = 0.9733505b0}} which is the value returned by R.
Here is the result I get
{code}
9.730779455235159798387713198169726833852358770410804399004797817813867760559669310857118225960097152689836562586854107391295331b-1
{code}
It seems to me that the value returned by CM is more accurate than the one returned by R. Could you carry out an independent check on that?
> FDistribution NoBracketingException in BrentSolver
> --------------------------------------------------
>
> Key: MATH-909
> URL: https://issues.apache.org/jira/browse/MATH-909
> Project: Commons Math
> Issue Type: Bug
> Affects Versions: 3.0
> Reporter: Patrick Meyer
> Original Estimate: 24h
> Remaining Estimate: 24h
>
> I get an exception when running the code below. the exception is
> {code}
> function values at endpoints do not have different signs, endpoints: [0, 1.002], values: [-0.025, -∞]
> {code}
> The problematic code:
> {code}
> double df1 = 10675;
> double df2 = 501725;
> FDistribution fDist = new FDistribution(df1, df2);
> System.out.println(fDist.inverseCumulativeProbability(0.025));//NoBracketingException
> {code}
> However, R returns the value 0.9733505. The R code is:
> {code}
> qf(p=.025, df1=10675, df2=501725)
> {code}
> I don't know enough about the FDistribution class to know the solution to the exception, but I thought I would report it.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MATH-909) FDistribution NoBracketingException
in BrentSolver
Posted by "Sébastien Brisard (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MATH-909?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13503825#comment-13503825 ]
Sébastien Brisard commented on MATH-909:
----------------------------------------
Excellent! I'm glad CM turns out to be more accurate. Be cautious, though, since we are still working on the incomplete beta function (MATH-738).
Just out of curiosity: I Haven't R installed on my computer. Could you check the doc for the incomplete beta function, and let us know on which implementation it is based?
If I remember correctly, it's slatec. I've noticed that slatec is in fact *not very accurate* for the gamma and beta functions, so I ruled it out for MATH-738.
Patrick: are you satisfied? Can we consider this issue as solved? If that's OK with you, I suggest we keep it open until the unit test has been implemented. Then, we will close it. What do you think?
> FDistribution NoBracketingException in BrentSolver
> --------------------------------------------------
>
> Key: MATH-909
> URL: https://issues.apache.org/jira/browse/MATH-909
> Project: Commons Math
> Issue Type: Bug
> Affects Versions: 3.0
> Reporter: Patrick Meyer
> Original Estimate: 24h
> Remaining Estimate: 24h
>
> I get an exception when running the code below. the exception is
> {code}
> function values at endpoints do not have different signs, endpoints: [0, 1.002], values: [-0.025, -∞]
> {code}
> The problematic code:
> {code}
> double df1 = 10675;
> double df2 = 501725;
> FDistribution fDist = new FDistribution(df1, df2);
> System.out.println(fDist.inverseCumulativeProbability(0.025));//NoBracketingException
> {code}
> However, R returns the value 0.9733505. The R code is:
> {code}
> qf(p=.025, df1=10675, df2=501725)
> {code}
> I don't know enough about the FDistribution class to know the solution to the exception, but I thought I would report it.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MATH-909) FDistribution NoBracketingException
in BrentSolver
Posted by "Thomas Neidhart (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MATH-909?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13503683#comment-13503683 ]
Thomas Neidhart commented on MATH-909:
--------------------------------------
I tested it with the latest trunk version, and the result is: 0.9730779455126357
The FDistribution relies on the Beta function, which got several improvements in terms of accuracy since the 3.0 release.
Could you please test yourself with a more recent version of CM?
Thanks,
Thomas
> FDistribution NoBracketingException in BrentSolver
> --------------------------------------------------
>
> Key: MATH-909
> URL: https://issues.apache.org/jira/browse/MATH-909
> Project: Commons Math
> Issue Type: Bug
> Affects Versions: 3.0
> Reporter: Patrick Meyer
> Original Estimate: 24h
> Remaining Estimate: 24h
>
> I get an exception when running the code below. the exception is
> {code}
> function values at endpoints do not have different signs, endpoints: [0, 1.002], values: [-0.025, -∞]
> {code}
> The problematic code:
> {code}
> double df1 = 10675;
> double df2 = 501725;
> FDistribution fDist = new FDistribution(df1, df2);
> System.out.println(fDist.inverseCumulativeProbability(0.025));//NoBracketingException
> {code}
> However, R returns the value 0.9733505. The R code is:
> {code}
> qf(p=.025, df1=10675, df2=501725)
> {code}
> I don't know enough about the FDistribution class to know the solution to the exception, but I thought I would report it.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MATH-909) FDistribution NoBracketingException
in BrentSolver
Posted by "Patrick Meyer (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MATH-909?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13503910#comment-13503910 ]
Patrick Meyer commented on MATH-909:
------------------------------------
According to the R documentation, the gamma and beta functions are C translations of the SLATEC Fortran subroutines, as you suspected. The incomplete gamma appears to have a different origin. According to the R documentation, the pbeta function is related to the incomplete beta function of Abramowitz and Stegun. They cite two different sources for the function depending on whether it is a central or non-central pbeta.
Central pbeta:
Didonato, A. and Morris, A., Jr, (1992) Algorithm 708: Significant digit computation of the incomplete beta function ratios, ACM Transactions on Mathematical Software, 18, 360–373. (See also
Brown, B. and Lawrence Levy, L. (1994) Certification of algorithm 708: Significant digit computation of the incomplete beta, ACM Transactions on Mathematical Software, 20, 393–397.)
Non-central pbeta:
Lenth, R. V. (1987) Algorithm AS226: Computing noncentral beta probabilities. Appl. Statist, 36, 241–244, incorporating
Frick, H. (1990)'s AS R84, Appl. Statist, 39, 311–2, and
Lam, M.L. (1995)'s AS R95, Appl. Statist, 44, 551–2.
As far as test cases go, I think we should include a test case, given the proposed work on the underlying incomplete beta function. The test case does not have to be specific to this issue, but it would be safe to include a test.
> FDistribution NoBracketingException in BrentSolver
> --------------------------------------------------
>
> Key: MATH-909
> URL: https://issues.apache.org/jira/browse/MATH-909
> Project: Commons Math
> Issue Type: Bug
> Affects Versions: 3.0
> Reporter: Patrick Meyer
> Original Estimate: 24h
> Remaining Estimate: 24h
>
> I get an exception when running the code below. the exception is
> {code}
> function values at endpoints do not have different signs, endpoints: [0, 1.002], values: [-0.025, -∞]
> {code}
> The problematic code:
> {code}
> double df1 = 10675;
> double df2 = 501725;
> FDistribution fDist = new FDistribution(df1, df2);
> System.out.println(fDist.inverseCumulativeProbability(0.025));//NoBracketingException
> {code}
> However, R returns the value 0.9733505. The R code is:
> {code}
> qf(p=.025, df1=10675, df2=501725)
> {code}
> I don't know enough about the FDistribution class to know the solution to the exception, but I thought I would report it.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MATH-909) FDistribution NoBracketingException
in BrentSolver
Posted by "Sébastien Brisard (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MATH-909?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13504009#comment-13504009 ]
Sébastien Brisard commented on MATH-909:
----------------------------------------
bq. According to the R documentation, the gamma and beta functions are C translations of the SLATEC Fortran subroutines, as you suspected. The incomplete gamma appears to have a different origin. According to the R documentation, the pbeta function is related to the incomplete beta function of Abramowitz and Stegun. They cite two different sources for the function depending on whether it is a central or non-central pbeta.
Thank you Patrick for checking the references. Didonato and Morris (1992) is actually the reference I used for our new implementation of {{Gamma.logGamma}} and {{Gamma.gamma}}, as well as {{Beta.logBeta}} (not yet committed).
I'm worried about the incomplete beta function, as R uses this paper, and the accuracy seems to be not so good. I will look into it.
> FDistribution NoBracketingException in BrentSolver
> --------------------------------------------------
>
> Key: MATH-909
> URL: https://issues.apache.org/jira/browse/MATH-909
> Project: Commons Math
> Issue Type: Bug
> Affects Versions: 3.0
> Reporter: Patrick Meyer
> Original Estimate: 24h
> Remaining Estimate: 24h
>
> I get an exception when running the code below. the exception is
> {code}
> function values at endpoints do not have different signs, endpoints: [0, 1.002], values: [-0.025, -∞]
> {code}
> The problematic code:
> {code}
> double df1 = 10675;
> double df2 = 501725;
> FDistribution fDist = new FDistribution(df1, df2);
> System.out.println(fDist.inverseCumulativeProbability(0.025));//NoBracketingException
> {code}
> However, R returns the value 0.9733505. The R code is:
> {code}
> qf(p=.025, df1=10675, df2=501725)
> {code}
> I don't know enough about the FDistribution class to know the solution to the exception, but I thought I would report it.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MATH-909) FDistribution NoBracketingException
in BrentSolver
Posted by "Sébastien Brisard (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MATH-909?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13503701#comment-13503701 ]
Sébastien Brisard commented on MATH-909:
----------------------------------------
bq. The FDistribution relies on the Beta function, which got several improvements in terms of accuracy since the 3.0 release.
Actually, in the lastest trunk {{r1413531}}, {{Beta}} hasn't been changed yet (ongoing work in MATH-738). Accuracy of {{Gamma.logGamma}} has improved a lot (see MATH-849) which might explain why this bug no longer shows up.
Even if this should not be considered as a bug, I propose we include this simple test in our unit tests, as {{Beta}} will undergo some changes in the coming days, hopefully for the best, but it never hurts to check non-regression, does it?
> FDistribution NoBracketingException in BrentSolver
> --------------------------------------------------
>
> Key: MATH-909
> URL: https://issues.apache.org/jira/browse/MATH-909
> Project: Commons Math
> Issue Type: Bug
> Affects Versions: 3.0
> Reporter: Patrick Meyer
> Original Estimate: 24h
> Remaining Estimate: 24h
>
> I get an exception when running the code below. the exception is
> {code}
> function values at endpoints do not have different signs, endpoints: [0, 1.002], values: [-0.025, -∞]
> {code}
> The problematic code:
> {code}
> double df1 = 10675;
> double df2 = 501725;
> FDistribution fDist = new FDistribution(df1, df2);
> System.out.println(fDist.inverseCumulativeProbability(0.025));//NoBracketingException
> {code}
> However, R returns the value 0.9733505. The R code is:
> {code}
> qf(p=.025, df1=10675, df2=501725)
> {code}
> I don't know enough about the FDistribution class to know the solution to the exception, but I thought I would report it.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Comment Edited] (MATH-909) FDistribution
NoBracketingException in BrentSolver
Posted by "Patrick Meyer (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MATH-909?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13503773#comment-13503773 ]
Patrick Meyer edited comment on MATH-909 at 11/26/12 1:17 PM:
--------------------------------------------------------------
I tested it with the most recent version and I also got 0.9730779455126357. The exception no longer occurs, but the result still seems to be too different from the value reported by R.
was (Author: meyerjp):
I tested it with the most recent version and I also got 0.9730779455126357. The exception no longer occurs, but the result still seems to be to different from the value reported by R.
> FDistribution NoBracketingException in BrentSolver
> --------------------------------------------------
>
> Key: MATH-909
> URL: https://issues.apache.org/jira/browse/MATH-909
> Project: Commons Math
> Issue Type: Bug
> Affects Versions: 3.0
> Reporter: Patrick Meyer
> Original Estimate: 24h
> Remaining Estimate: 24h
>
> I get an exception when running the code below. the exception is
> {code}
> function values at endpoints do not have different signs, endpoints: [0, 1.002], values: [-0.025, -∞]
> {code}
> The problematic code:
> {code}
> double df1 = 10675;
> double df2 = 501725;
> FDistribution fDist = new FDistribution(df1, df2);
> System.out.println(fDist.inverseCumulativeProbability(0.025));//NoBracketingException
> {code}
> However, R returns the value 0.9733505. The R code is:
> {code}
> qf(p=.025, df1=10675, df2=501725)
> {code}
> I don't know enough about the FDistribution class to know the solution to the exception, but I thought I would report it.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MATH-909) FDistribution NoBracketingException
in BrentSolver
Posted by "Sébastien Brisard (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MATH-909?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13503951#comment-13503951 ]
Sébastien Brisard commented on MATH-909:
----------------------------------------
bq. Of course, unit tests are always welcome. But in this case, your approach of fairly exhaustively checking the result of the underlying functions by comparison with "maxima" makes it superfluous to separately test the "client" (i.e. "FDistribution").
I wouldn't say superfluous. Maybe (maybe) redundant. But remember that the incomplete beta function has three arguments. Exhaustive checking of this function is going to be difficult...
On the whole, I agree with you. This test would seem a bit weird. I'll make sure that comparison with Maxima are "exhaustive enough" to include a test equivalent to the values proposed by Patrick in this issue.
I agree this issue should be tagged as "cannot reproduce".
> FDistribution NoBracketingException in BrentSolver
> --------------------------------------------------
>
> Key: MATH-909
> URL: https://issues.apache.org/jira/browse/MATH-909
> Project: Commons Math
> Issue Type: Bug
> Affects Versions: 3.0
> Reporter: Patrick Meyer
> Original Estimate: 24h
> Remaining Estimate: 24h
>
> I get an exception when running the code below. the exception is
> {code}
> function values at endpoints do not have different signs, endpoints: [0, 1.002], values: [-0.025, -∞]
> {code}
> The problematic code:
> {code}
> double df1 = 10675;
> double df2 = 501725;
> FDistribution fDist = new FDistribution(df1, df2);
> System.out.println(fDist.inverseCumulativeProbability(0.025));//NoBracketingException
> {code}
> However, R returns the value 0.9733505. The R code is:
> {code}
> qf(p=.025, df1=10675, df2=501725)
> {code}
> I don't know enough about the FDistribution class to know the solution to the exception, but I thought I would report it.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira