[jira] Commented: (DERBY-131) AVG precision is INTEGER instead of DOUBLE

["Rainer Gemulla (JIRA)" <de...@db.apache.org>]

     [ http://issues.apache.org/jira/browse/DERBY-131?page=comments#action_58007 ]
     
Rainer Gemulla commented on DERBY-131:
--------------------------------------

Quick Fix:

--------------------------------
in AvgAggregator.java change:

avg = value.getBigDecimal().divide(BigDecimal.valueOf(count), scale, BigDecimal.ROUND_DOWN);

to:

avg = value.getBigDecimal().divide(BigDecimal.valueOf(count), TypeId.DECIMAL_SCALE, BigDecimal.ROUND_DOWN);


--------------------------------
in SumAvgAggregator.java change:

DataTypeDescriptor outDts = tc.resolveArithmeticOperation(dts, dts, getOperator());

to:

DataTypeDescriptor outDts;
                if (isSum) {
                    outDts = tc.resolveArithmeticOperation(dts, dts, getOperator());
                } else { // avg returns always double!
                    outDts = DataTypeDescriptor.getSQLDataTypeDescriptor(TypeId.DOUBLE_NAME);
                }

> AVG precision is INTEGER instead of DOUBLE
> ------------------------------------------
>
>          Key: DERBY-131
>          URL: http://issues.apache.org/jira/browse/DERBY-131
>      Project: Derby
>         Type: Bug
>   Components: SQL
>     Versions: 10.0.2.1
>     Reporter: Rainer Gemulla

>
> The AVG aggregator returns an INTEGER value, if the data tyoe of the respective attribute is INTEGER. This is wrong, because the average of a set of integers is not always an integer, too.

-- 
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
   http://issues.apache.org/jira/secure/Administrators.jspa
-
If you want more information on JIRA, or have a bug to report see:
   http://www.atlassian.com/software/jira