(spark) branch master updated: [SPARK-59271][PYTHON][TEST][FOLLOWUP] Fix pandas.api.types golden test with NumPy 1.23

[ru...@apache.org]

This is an automated email from the ASF dual-hosted git repository.

zhengruifeng pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/spark.git


The following commit(s) were added to refs/heads/master by this push:
     new 9459e6c4ef77 [SPARK-59271][PYTHON][TEST][FOLLOWUP] Fix pandas.api.types golden test with NumPy 1.23
9459e6c4ef77 is described below

commit 9459e6c4ef7725999db3e048144926763306f211
Author: Spenser Sun <hs...@gmail.com>
AuthorDate: Fri Sep 18 14:56:45 2026 +0800

    [SPARK-59271][PYTHON][TEST][FOLLOWUP] Fix pandas.api.types golden test with NumPy 1.23
    
    ### What changes were proposed in this pull request?
    
    This PR adds a NumPy-version override for the input-type cell in the pandas.api.types object-predicate golden test.
    
    ### Why are the changes needed?
    
    Spark's minimum dependency profile uses NumPy 1.23.2, where `type(np.dtype("int64")).__name__` is `dtype[int64]` rather than `Int64DType`. Without the override, the scheduled minimum-dependency build fails although the predicate results are unchanged.
    
    ### Does this PR introduce _any_ user-facing change?
    
    No.
    
    ### How was this patch tested?
    
    The focused test passed with NumPy 1.23.2, 1.24.4, 1.25.0, and 2.4.1; the [minimum-dependency workflow](https://github.com/Spenserrrr/spark/actions/runs/35135479035) passed on this branch, while the scheduled run demonstrates the failure without the override.
    
    ### Was this patch authored or co-authored using generative AI tooling?
    
    Generated-by: Codex (GPT-5)
    
    Closes #58869 from Spenserrrr/pandas-api-types-mindep-fix.
    
    Authored-by: Spenser Sun <hs...@gmail.com>
    Signed-off-by: Ruifeng Zheng <ru...@apache.org>
---
 python/pyspark/tests/upstream/pandas/test_pandas_api_types.py | 3 +++
 1 file changed, 3 insertions(+)

diff --git a/python/pyspark/tests/upstream/pandas/test_pandas_api_types.py b/python/pyspark/tests/upstream/pandas/test_pandas_api_types.py
index 1d07fca6b21d..7a662954ab3a 100644
--- a/python/pyspark/tests/upstream/pandas/test_pandas_api_types.py
+++ b/python/pyspark/tests/upstream/pandas/test_pandas_api_types.py
@@ -340,6 +340,9 @@ class PandasApiTypesObjectTests(_PandasApiTypesTestBase):
         predicates = self._predicates()
         # pandas/numpy-version-specific expected cells; empty at the pandas 2.3 baseline.
         overrides: dict[tuple[str, str], str] = {}
+        if LooseVersion(np.__version__) < LooseVersion("1.25.0"):
+            # Spark's minimum profile pins NumPy 1.23.2; the compatibility sweep used 1.26.4.
+            overrides[("np.dtype('int64')", COL_INPUT_TYPE)] = "dtype[int64]"
         if sys.version_info >= (3, 12):
             # slice became hashable in Python 3.12, and is_hashable is exactly "does hash()
             # raise", so this cell follows the interpreter. The golden holds the 3.11 answer.


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@spark.apache.org
For additional commands, e-mail: commits-help@spark.apache.org