You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "L. C. Hsieh (Jira)" <ji...@apache.org> on 2026/10/10 17:51:00 UTC

[jira] [Created] (SPARK-60132) Bound the integral-digit count of BigDecimal values in Decimal.set to avoid raw ArithmeticException for huge exponents

L. C. Hsieh created SPARK-60132:
-----------------------------------

             Summary: Bound the integral-digit count of BigDecimal values in Decimal.set to avoid raw ArithmeticException for huge exponents
                 Key: SPARK-60132
                 URL: https://issues.apache.org/jira/browse/SPARK-60132
             Project: Spark
          Issue Type: Bug
          Components: SQL
    Affects Versions: 4.4.0
            Reporter: L. C. Hsieh


SPARK-60118 bounds the number of integral digits (precision - scale, now computed in Long) only in Decimal.fromString / fromStringANSI, i.e. the string-to-decimal cast path. Other entry points that build a Decimal from a BigDecimal have no such bound and can still fail with a raw java.lang.ArithmeticException (or spend a long time in BigDecimal.setScale) for values with a huge exponent:

* Decimal.set(BigDecimal) computes this._precision = decimal.precision - decimal.scale as an Int, which overflows for e.g. 1e2147483647 or 0e2147483647, and then calls decimal.setScale(0). Decimal(String) and Decimal(java.math.BigDecimal) go through this.
* Decimal.set(BigDecimal, precision, scale), used by Decimal(bd, precision, scale), calls decimal.setScale(scale) directly. The CSV reader (UnivocityParser) and the JSON reader (JacksonParser) use it for decimal columns. There the exception is currently turned into a bad record (or a failure in FAILFAST mode) rather than a NULL field.

Suggested direction: move the Long-based integral-digit bound into Decimal.set or a shared helper, so these paths don't depend on the string-cast fast path.

Reported in the review of https://github.com/apache/spark/pull/59334.




--
This message was sent by Atlassian Jira
(v8.20.10#820010)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org