[jira] [Issue Comment Edited] (MATH-718) inverseCumulativeProbability of BinomialDistribution returns wrong value for large trials.

["Thomas Neidhart (Issue Comment Edited) (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/MATH-718?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13253360#comment-13253360 ] 

Thomas Neidhart edited comment on MATH-718 at 4/13/12 1:22 PM:
---------------------------------------------------------------

The problem Christian described wrt the PascalDistribution is a simple integer overflow in the class itself:

{noformat}
    public double cumulativeProbability(int x) {
        double ret;
        if (x < 0) {
            ret = 0.0;
        } else {
            ret = Beta.regularizedBeta(probabilityOfSuccess,
                    numberOfSuccesses, x + 1);
        }
        return ret;
    }
{noformat}

when x = Integer.MAX_VALUE, adding 1 to it will result in an overflow. As the parameter of regularizedBeta is anyway a double, it should be cast to long/double before the addition.

Edit: Similar things happen btw also in other Distribution implementations, so it should be fixed also there, e.g. BinomialDistribution
                
      was (Author: tn):
    The problem Christian described wrt the PascalDistribution is a simple integer overflow in the class itself:

{noformat}
    public double cumulativeProbability(int x) {
        double ret;
        if (x < 0) {
            ret = 0.0;
        } else {
            ret = Beta.regularizedBeta(probabilityOfSuccess,
                    numberOfSuccesses, x + 1);
        }
        return ret;
    }
{noformat}

when x = Integer.MAX_VALUE, adding 1 to it will result in an overflow. As the parameter of regularizedBeta is anyway a double, it should be cast to long/double before the addition.
                  
> inverseCumulativeProbability of BinomialDistribution returns wrong value for large trials.
> ------------------------------------------------------------------------------------------
>
>                 Key: MATH-718
>                 URL: https://issues.apache.org/jira/browse/MATH-718
>             Project: Commons Math
>          Issue Type: Bug
>    Affects Versions: 2.2, 3.0
>            Reporter: Yuji Uchiyama
>            Assignee: Sébastien Brisard
>             Fix For: 3.1, 4.0
>
>
> The inverseCumulativeProbability method of the BinomialDistributionImpl class returns wrong value for large trials.  Following code will be reproduce the problem.
> {{System.out.println(new BinomialDistributionImpl(1000000, 0.5).inverseCumulativeProbability(0.5));}}
> This returns 499525, though it should be 499999.
> I'm not sure how it should be fixed, but the cause is that the cumulativeProbability method returns Infinity, not NaN.  As the result the checkedCumulativeProbability method doesn't work as expected.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira