[jira] [Closed] (GROOVY-12355) STC infers BigDecimal for division of Number or wrapper Double/Float operands

["Paul King (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/GROOVY-12355?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Paul King closed GROOVY-12355.
------------------------------

> STC infers BigDecimal for division of Number or wrapper Double/Float operands
> -----------------------------------------------------------------------------
>
>                 Key: GROOVY-12355
>                 URL: https://issues.apache.org/jira/browse/GROOVY-12355
>             Project: Groovy
>          Issue Type: Bug
>    Affects Versions: 6.0.0-RC-1, 5.1.2
>            Reporter: Paul King
>            Assignee: Paul King
>            Priority: Major
>             Fix For: 6.0.0-RC-2
>
>
> Under {{@CompileStatic}} (and {{@TypeChecked}}), the static type checker infers {{BigDecimal}} as the result of a division whenever neither operand is a *primitive* {{float}} or {{double}}. This is wrong in two cases:
> * wrapper {{Double}}/{{Float}} operands, e.g. {{Double / Integer}}, which dynamic Groovy evaluates to a {{Double}}
> * operands whose static type is {{Number}} (or another non-leaf numeric type), where the runtime category is unknown
> In {{StaticTypeCheckingVisitor.getMathResultType}}, the {{DIVIDE}} branch uses {{isFloatingCategory}}, which only recognises primitives, and otherwise falls back to {{BigDecimal_TYPE}}. Group operations ({{+}}, {{-}}, {{*}}) already fall back to {{Number_TYPE}} via {{getGroupOperationResultType}}; division should do the same, i.e. infer the LUB {{Number}} when the operand category can't be determined, and {{Double}} when either operand is a wrapper floating type. This matches the runtime signature {{NumberMath.divide(Number, Number): Number}}.
> The wrong inference is masked in a simple assignment, because the assignment path runs {{castToType}} which converts a {{Double}} to a {{BigDecimal}}. So {{def q = a / b}} silently returns a different type to dynamic Groovy. In a chained expression the compiler emits a {{CHECKCAST}} to {{BigDecimal}} and the code throws {{ClassCastException}}.
> h3. Reproducer
> {code:groovy}
> import groovy.transform.CompileStatic
> @CompileStatic
> class P {
>     static Object nn(Number a, Number b) { a / b }
>     static Object nnChain(Number a, Number b) { ((a / b) * 8).intValue() }
>     static Object di(Double a, Integer b) { a / b }
>     static Object diChain(Double a, Integer b) { ((a / b) * 8).intValue() }
> }
> class D { // dynamic, for comparison
>     static Object nn(Number a, Number b) { a / b }
>     static Object nnChain(Number a, Number b) { ((a / b) * 8).intValue() }
> }
> assert D.nn(1.5d, 2) instanceof Double        // 0.75
> assert D.nnChain(1.5d, 2) == 6
> assert P.nn(1.5d, 2) instanceof BigDecimal    // differs from dynamic
> assert P.di(1.5d, 2) instanceof BigDecimal    // differs from dynamic
> P.nnChain(1.5d, 2)  // ClassCastException: Double cannot be cast to BigDecimal
> P.diChain(1.5d, 2)  // ClassCastException: Double cannot be cast to BigDecimal
> {code}
> Results, called with {{(1.5d, 2)}}:
> ||expression||dynamic||@CompileStatic||
> |{{Number / Number}}|{{Double}} 0.75|{{BigDecimal}} 0.75|
> |{{Double / Integer}}|{{Double}} 0.75|{{BigDecimal}} 0.75|
> |{{((Number / Number) * 8).intValue()}}|6|{{ClassCastException}}|
> |{{((Double / Integer) * 8).intValue()}}|6|{{ClassCastException}}|
> |{{double / int}}|{{Double}}|{{double}}|
> Behaviour is identical on 4.0.33, 5.1.1 and 6.0.0-beta-3, so this is long-standing rather than a regression.
> h3. Real-world impact
> {{org.codehaus.groovy.util.StringUtil.bar(Number x, Number min, Number max, int width)}} (added in 5.0.0) is compiled statically and contains exactly the chained shape above:
> {code:groovy}
> int barWidth = ((x - min) / interval * fracWidth).intValue()
> {code}
> so {{bar(0.45d, 0, 2)}} (or any {{Double}}/{{Float}} argument) throws {{ClassCastException}}, while {{bar(0.45, 0, 2)}} with a {{BigDecimal}} literal works. Fixing the inference makes {{bar}} work without changes, since every intermediate becomes {{Number}}.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)