[jira] [Updated] (SPARK-39316) Merge PromotePrecision and CheckOverflow into decimal binary arithmetic

["XiDuo You (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/SPARK-39316?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

XiDuo You updated SPARK-39316:
------------------------------
    Description: 
Fix the bug of `TypeCoercion`, for example:

{code:java}
  SELECT CAST(1 AS DECIMAL(28, 2))
  UNION ALL
  SELECT CAST(1 AS DECIMAL(18, 2)) / CAST(1 AS DECIMAL(18, 2));
{code}

The union result data type is not correct according to the formula:

|| Operation || Result Precision || Result Scale ||
|e1 union e2 | max(s1, s2) + max(p1-s1, p2-s2) | max(s1, s2) |


{code:java}
-- before
-- query schema
decimal(28,2)
-- query output
1.00
1.00

-- after
-- query schema
decimal(38,20)
-- query output
1.00000000000000000000
1.00000000000000000000
{code}


  was:
Merge {{PromotePrecision}} into {{{}dataType{}}}, for example, {{{}Add{}}}:
{code:java}
override def dataType: DataType = (left, right) match {
  case (DecimalType.Expression(p1, s1), DecimalType.Expression(p2, s2)) =>
    val resultScale = max(s1, s2)
    if (allowPrecisionLoss) {
      DecimalType.adjustPrecisionScale(max(p1 - s1, p2 - s2) + resultScale + 1,
        resultScale)
    } else {
      DecimalType.bounded(max(p1 - s1, p2 - s2) + resultScale + 1, resultScale)
    }
  case _ => super.dataType
} {code}
Merge {{{}CheckOverflow{}}}, for example, {{Add}} eval:
{code:java}
dataType match {
  case decimalType: DecimalType =>
    val value = numeric.plus(input1, input2)
    checkOverflow(value.asInstanceOf[Decimal], decimalType)
  ...
} {code}


> Merge PromotePrecision and CheckOverflow into decimal binary arithmetic
> -----------------------------------------------------------------------
>
>                 Key: SPARK-39316
>                 URL: https://issues.apache.org/jira/browse/SPARK-39316
>             Project: Spark
>          Issue Type: Sub-task
>          Components: SQL
>    Affects Versions: 3.4.0
>            Reporter: XiDuo You
>            Assignee: XiDuo You
>            Priority: Major
>             Fix For: 3.4.0
>
>
> Fix the bug of `TypeCoercion`, for example:
> {code:java}
>   SELECT CAST(1 AS DECIMAL(28, 2))
>   UNION ALL
>   SELECT CAST(1 AS DECIMAL(18, 2)) / CAST(1 AS DECIMAL(18, 2));
> {code}
> The union result data type is not correct according to the formula:
> || Operation || Result Precision || Result Scale ||
> |e1 union e2 | max(s1, s2) + max(p1-s1, p2-s2) | max(s1, s2) |
> {code:java}
> -- before
> -- query schema
> decimal(28,2)
> -- query output
> 1.00
> 1.00
> -- after
> -- query schema
> decimal(38,20)
> -- query output
> 1.00000000000000000000
> 1.00000000000000000000
> {code}



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org