[jira] [Commented] (STATISTICS-82) Raise exception for integer overflow during statistic computation.

["Alex Herbert (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/STATISTICS-82?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17800889#comment-17800889 ] 

Alex Herbert commented on STATISTICS-82:
----------------------------------------

Statistics that use the count at the end to compute the statistic:
 * IntMean/LongMean
 * IntVariance/LongVariance
 * IntStandardDeviation/LongStandardDeviation
 * GeometricMean

These statistics could:
 # Raise an error if the count is negative.
 # Use the count as an unsigned 64-bit integer thus allowing support via incrementing to be up to 2^64 observations. If a count is created by combine to just under 2^63, this allows incrementing beyond 2^63 and the statistics is still correct.
 # Ignore this unlikely use case.

Statistics that use the count within the accept method for each additional value:
 * Mean
 * Higher moments: Variance/Skewness/Kurtosis

These statistics are limited to < 2^63 observations. Options are:
 # Guard against incrementing above this level for each new value. This would affect performance.
 # Ignore this unlikely use case.

 

Not handling overflow in counts is how this has been implemented in the JDK in e.g. {{{}java.util.DoubleSummaryStatistics{}}}. A note in the class header javadoc states:
{noformat}
"This implementation does not check for overflow of the count."{noformat}
This includes in the method to combine two instances of the summary statistics.

 

Given the unlikely case of creating over 2^63 actual observations a minimal solution is to document the behaviour:
{noformat}
"Supports up to 2^63 (exclusive) observations. This implementation does not check for overflow of the count."{noformat}
 

> Raise exception for integer overflow during statistic computation.
> ------------------------------------------------------------------
>
>                 Key: STATISTICS-82
>                 URL: https://issues.apache.org/jira/browse/STATISTICS-82
>             Project: Commons Statistics
>          Issue Type: Improvement
>          Components: descriptive
>            Reporter: Alex Herbert
>            Priority: Minor
>
> Many statistics count the number of observations using a long. This limited computation to ~2^63 values.
> This is effectively impossible to achieve using single values. It is very easy to achieve using a combine:
> {code:java}
> final IntMean m = IntMean.of(2, 3);
> System.out.println(m.getAsDouble());
> for (int i = 0; i < 62; i++) {
>     m.combine(m);
> }
> // n = 2^63 = -9223372036854775808
> System.out.println(m.getAsDouble());
> {code}
> Prints:
> {noformat}
> 2.5
> -0.5
> {noformat}
> Overflow can be detected in combine, e.g.
> {code:java}
> // throws ArithmeticException when n+m overflows
> n = Math.addExact(n, other.n)
> {code}
> However it is possible to create a statistic with a count very close to 2^63 via combine and then increment it past 2^63 with single values. To avoid an expensive check at each addition, this case will have to detect overflow of n to negative during the final computation of the statistic.
> Raising an exception will guard against misuse. It is unlikely to affect any real use cases. Without an exception the user will have an invalid result which may cause downstream problems in the dependent code.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)