[jira] Updated: (MATH-282) ChiSquaredDistributionImpl.cumulativeProbability > 1

["Phil Steitz (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/MATH-282?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Phil Steitz updated MATH-282:
-----------------------------

    Attachment: distributions.patch

The attached patch resolves this issue as well as MATH-301 and the problems described above with the Poisson distribution.  The patch bundles quite a few changes, some of which are not strictly necessary to resolve these issues.  The following is a summary.

* BrentSolver has been changed to expose its configured absolute accuracy.   This solver is used by the default inverse cum implementation in AbstractContinuousDistribution and the hard-coded setting (1E-6) was limiting accuracy in inverse cumulative probability estimates.  AbstractContinuousDistribution was changed to allow distributions to set this value and NormalDistributionImpl was changed to set it to 1E-9 by default and allow users to configure it via a constructor argument.   If all are happy with this change, I will similarly change other distributions to override the new getSolverAbsoluteAccuracy method and add constructors to configure the value.

* AbstractContinuousDistribution and AbstractIntegerDistribution inverseCumulativeProbability methods have been modified to check for NaN values returned by cumulativeProbability and throw MathExceptions when this happens.

* The criteria for choosing between the Lanczos series and continued fraction expansion when computing regularized gamma functions has been changed to (x >= a + 1).  When using the series approximation (regularizedGammaP), divergence to infinity is checked and when this happens, 1 is returned. 

* When scaling continued fractions to (try to) avoid divergence to infinity, the larger of a and b is used as a scale factor and the attempt to scale is repeated up to 5 times, using successive powers of the scale factor.  

* The maximum number of iterations used in estimating cumulative probabilities for PoissonDistributionImpl has been decreased from Integer.MAX_VALUE to 10000000 and made configurable.

Review and comment much appreciated.  One thing that I would like improve is to get decent top-coding in place in terms of the arguments to the regularized gamma functions.  The Poisson inverse cum tests take a very long time now because for very large values of x, the continued fractions are taking a long time to converge.  This is needless computation, as the value returned is 1.  We should be able to analytically determine bounds here.

> ChiSquaredDistributionImpl.cumulativeProbability > 1
> ----------------------------------------------------
>
>                 Key: MATH-282
>                 URL: https://issues.apache.org/jira/browse/MATH-282
>             Project: Commons Math
>          Issue Type: Bug
>    Affects Versions: 1.0, 1.1, 1.2, 2.0
>         Environment: called from Scala code
>            Reporter: Adam Kiezun
>            Assignee: Phil Steitz
>             Fix For: 2.1
>
>         Attachments: distributions.patch, math-282.patch
>
>
> Calling 
> new ChiSquaredDistributionImpl(1.0).cumulativeProbability(66.41528551683048)
> returns 1.000000000000004, which is bogus (should never be > 1)

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.