[jira] [Created] (SPARK-59934) pmod silently wraps around for INT/BIGINT with large-magnitude negative divisors

["David Mollitor (Jira)" <ji...@apache.org>]

David Mollitor created SPARK-59934:
--------------------------------------

             Summary:  pmod silently wraps around for INT/BIGINT with large-magnitude negative divisors
                 Key: SPARK-59934
                 URL: https://issues.apache.org/jira/browse/SPARK-59934
             Project: Spark
          Issue Type: Improvement
          Components: SQL
    Affects Versions: 4.0.0, 4.1.0
            Reporter: David Mollitor


{code:sql}
-- INT: expected -1, actual 2147483647
SELECT pmod(-1, CAST(-2147483648 AS INT));

-- BIGINT: expected -1, actual 9223372036854775807
SELECT pmod(CAST(-1 AS BIGINT), CAST(-9223372036854775808 AS BIGINT));

-- Correct result for the same magnitude via DECIMAL: -1
SELECT pmod(CAST(-1 AS DECIMAL(38,0)), CAST(-2147483648 AS DECIMAL(38,0)));
{code}
h3. Root cause

For {{{}n < 0{}}}, {{MathUtils.pmod}} computes {{r = a % n}} and, when {{{}r < 0{}}}, returns
{{(r + n) % n}} to give a result that shares the sign of the dividend. For sufficiently
large-magnitude negative divisors (roughly {{n < -2^30}} for {{{}INT{}}}, {{n < -2^62}} for
{{{}BIGINT{}}}), the intermediate {{r + n}} overflows the integral type and wraps, so the
final value is positive and no longer congruent to {{a}} modulo {{{}n{}}}. The {{DECIMAL}}
path uses {{BigDecimal}} arithmetic and does not overflow, hence the cross-type
disagreement.

 
h3. Expected behavior

{{pmod(a, n)}} should return a result consistent across all integral types (matching the
{{DECIMAL}}/{{BYTE}}/{{SHORT}} result), or raise {{ARITHMETIC_OVERFLOW}} under ANSI mode
rather than silently returning a wrapped value.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org