[jira] [Updated] (CALCITE-2684) AssertionError on RexBuilder when creating BigDecimal RexLiteral

["Ruben Quesada Lopez (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/CALCITE-2684?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Ruben Quesada Lopez updated CALCITE-2684:
-----------------------------------------
    Description: 
The method {{RexBuilder#makeExactLiteral(java.math.BigDecimal)}} throws an AssertionError if the BigDecimal parameter has an unscaled value that overflows long:
{code:java}
public RexLiteral makeExactLiteral(BigDecimal bd) {
    ...
    long l = bd.unscaledValue().longValue(); // narrowing conversion BigInteter to long
    ...
    assert BigDecimal.valueOf(l, scale).equals(bd); // assert fails if l overflew long
    ...
{code}

Moreover, with the current implementation, it can be possible to have this AssertionError, even in the cases when the variable {{l}} would not be used at all (decimal number)
{code:java}
    ...
    assert BigDecimal.valueOf(l, scale).equals(bd); // assert can fail, even if scale == 0 (l would not be needed)
    if (scale == 0) {
      if ((l >= Integer.MIN_VALUE) && (l <= Integer.MAX_VALUE))
        relType = typeFactory.createSqlType(SqlTypeName.INTEGER);
      else
        relType = typeFactory.createSqlType(SqlTypeName.BIGINT);
    } else {
      int precision = bd.unscaledValue().abs().toString().length();
      if (precision > scale)
        relType = typeFactory.createSqlType(SqlTypeName.DECIMAL, precision, scale);
      else
        relType = typeFactory.createSqlType(SqlTypeName.DECIMAL, scale + 1, scale);
    }
{code}


  was:
The method {{RexBuilder#makeExactLiteral(java.math.BigDecimal)}} throws an AssertionError if the BigDecimal parameter has an unscaled value that overflows long:
{code:java}
public RexLiteral makeExactLiteral(BigDecimal bd) {
    ...
    long l = bd.unscaledValue().longValue(); // narrowing conversion BigInteter to long
    ...
    assert BigDecimal.valueOf(l, scale).equals(bd); // assert fails if l overflew long
    ...
{code}


> AssertionError on RexBuilder when creating BigDecimal RexLiteral
> ----------------------------------------------------------------
>
>                 Key: CALCITE-2684
>                 URL: https://issues.apache.org/jira/browse/CALCITE-2684
>             Project: Calcite
>          Issue Type: Bug
>    Affects Versions: 1.17.0
>            Reporter: Ruben Quesada Lopez
>            Assignee: Julian Hyde
>            Priority: Minor
>             Fix For: 1.18.0
>
>
> The method {{RexBuilder#makeExactLiteral(java.math.BigDecimal)}} throws an AssertionError if the BigDecimal parameter has an unscaled value that overflows long:
> {code:java}
> public RexLiteral makeExactLiteral(BigDecimal bd) {
>     ...
>     long l = bd.unscaledValue().longValue(); // narrowing conversion BigInteter to long
>     ...
>     assert BigDecimal.valueOf(l, scale).equals(bd); // assert fails if l overflew long
>     ...
> {code}
> Moreover, with the current implementation, it can be possible to have this AssertionError, even in the cases when the variable {{l}} would not be used at all (decimal number)
> {code:java}
>     ...
>     assert BigDecimal.valueOf(l, scale).equals(bd); // assert can fail, even if scale == 0 (l would not be needed)
>     if (scale == 0) {
>       if ((l >= Integer.MIN_VALUE) && (l <= Integer.MAX_VALUE))
>         relType = typeFactory.createSqlType(SqlTypeName.INTEGER);
>       else
>         relType = typeFactory.createSqlType(SqlTypeName.BIGINT);
>     } else {
>       int precision = bd.unscaledValue().abs().toString().length();
>       if (precision > scale)
>         relType = typeFactory.createSqlType(SqlTypeName.DECIMAL, precision, scale);
>       else
>         relType = typeFactory.createSqlType(SqlTypeName.DECIMAL, scale + 1, scale);
>     }
> {code}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)