(spark) branch branch-4.x updated: [SPARK-59093][PS][FOLLOWUP] Reuse the floating floor division expression

[ru...@apache.org]

This is an automated email from the ASF dual-hosted git repository.

zhengruifeng pushed a commit to branch branch-4.x
in repository https://gitbox.apache.org/repos/asf/spark.git


The following commit(s) were added to refs/heads/branch-4.x by this push:
     new ac24e314c02e [SPARK-59093][PS][FOLLOWUP] Reuse the floating floor division expression
ac24e314c02e is described below

commit ac24e314c02eabfe4410906316368e40f24098df
Author: Spenser Sun <hs...@gmail.com>
AuthorDate: Fri Sep 11 08:15:33 2026 +0800

    [SPARK-59093][PS][FOLLOWUP] Reuse the floating floor division expression
    
    ### What changes were proposed in this pull request?
    
    This is a small follow-up to #58397: `_floor_divide_func` now constructs the floating-point result once and reuses it for the floating and decimal branches. The adjacent `F.floor` comment now distinguishes ANSI overflow from non-ANSI saturation.
    
    ### Why are the changes needed?
    
    The same lazy `Column` expression was constructed twice while building floor division, and the comment described only the ANSI behavior.
    
    ### Does this PR introduce _any_ user-facing change?
    
    No.
    
    ### How was this patch tested?
    
    Ran `test_floordiv` and `test_rfloordiv`; a targeted construction probe confirmed one helper call instead of two with an identical Spark expression.
    
    ### Was this patch authored or co-authored using generative AI tooling?
    
    Generated-by: Codex (GPT-5)
    
    Closes #58715 from Spenserrrr/floordiv-digest-followup.
    
    Authored-by: Spenser Sun <hs...@gmail.com>
    Signed-off-by: Ruifeng Zheng <ru...@foxmail.com>
    (cherry picked from commit 0edaeb88a5df98c67905143065e7cf8d1965d43a)
    Signed-off-by: Ruifeng Zheng <ru...@foxmail.com>
---
 python/pyspark/pandas/utils.py | 8 +++++---
 1 file changed, 5 insertions(+), 3 deletions(-)

diff --git a/python/pyspark/pandas/utils.py b/python/pyspark/pandas/utils.py
index 6a3eb14daac9..a96220217f0b 100644
--- a/python/pyspark/pandas/utils.py
+++ b/python/pyspark/pandas/utils.py
@@ -1061,7 +1061,8 @@ def _floor_divide_floating(c1: Column, c2: Column) -> Column:
         (remainder != 0) & ((remainder < 0) != (c2 < 0)), truncated - F.lit(1.0)
     ).otherwise(truncated)
     # The quotient is whole in exact arithmetic, but the division can leave it a few bits off, so
-    # round it back. F.floor cannot do this: it returns a bigint, which raises on an infinity.
+    # round it back. F.floor cannot do this: its bigint result raises on an infinity in ANSI mode
+    # and saturates otherwise.
     floor = quotient - F.pmod(quotient, F.lit(1.0))
     return (
         # An infinite quotient is its own floor, and has to be returned before the line above
@@ -1102,6 +1103,7 @@ def _floor_divide_func(c1: Column, c2: Column) -> Column:
     c1_double = c1.cast("double")
     c2_double = c2.cast("double")
     integral_types = ["tinyint", "smallint", "int", "bigint"]
+    floating_result = _floor_divide_floating(c1_double, c2_double)
 
     return (
         # Null, nan and a zero divisor are handled the same way for every operand type.
@@ -1135,10 +1137,10 @@ def _floor_divide_func(c1: Column, c2: Column) -> Column:
             )
             # Dividing a zero dividend keeps its sign, so -0.0 // 3.0 is -0.0.
             .when(c1_double == 0, c1_double / c2_double)
-            .otherwise(_floor_divide_floating(c1_double, c2_double)),
+            .otherwise(floating_result),
         )
         # A decimal cannot be matched by name: typeof reports its precision, as in decimal(10,2).
-        .otherwise(_floor_divide_floating(c1_double, c2_double))
+        .otherwise(floating_result)
     )
 
 


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@spark.apache.org
For additional commands, e-mail: commits-help@spark.apache.org