[jira] [Created] (SPARK-59271) Add tests for pandas.api.types is_* predicates

["Haotian Sun (Jira)" <ji...@apache.org>]

Haotian Sun created SPARK-59271:
-----------------------------------

             Summary: Add tests for pandas.api.types is_* predicates
                 Key: SPARK-59271
                 URL: https://issues.apache.org/jira/browse/SPARK-59271
             Project: Spark
          Issue Type: Sub-task
          Components: PySpark
    Affects Versions: 5.0.0
            Reporter: Haotian Sun


Add golden-file tests pinning the behavior of pandas' type-introspection predicates
(pandas.api.types.is_*) that PySpark depends on.

PySpark branches on the boolean these predicates return, so an upstream change misroutes
control flow rather than corrupting a single value: the conversion layer picks a Spark type
from is_datetime64_dtype / is_timedelta64_dtype, is_integer_dtype decides whether integer
data is rewritten as Decimal, and pandas-on-Spark dispatches implementations and raises
user-visible TypeErrors from is_hashable / is_list_like.

The tests run without a Spark session and act as a drift canary, following the pattern of
the existing python/pyspark/tests/upstream/pyarrow suite.




--
This message was sent by Atlassian Jira
(v8.20.10#820010)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org