[spark] branch branch-3.3 updated: [SPARK-40376][PYTHON] Avoid Numpy deprecation warning

[sr...@apache.org]

This is an automated email from the ASF dual-hosted git repository.

srowen pushed a commit to branch branch-3.3
in repository https://gitbox.apache.org/repos/asf/spark.git


The following commit(s) were added to refs/heads/branch-3.3 by this push:
     new 52d52a158c9 [SPARK-40376][PYTHON] Avoid Numpy deprecation warning
52d52a158c9 is described below

commit 52d52a158c9a169d97122523142b7c3026ee92bb
Author: ELHoussineT <el...@gmail.com>
AuthorDate: Mon Sep 12 20:46:15 2022 -0500

    [SPARK-40376][PYTHON] Avoid Numpy deprecation warning
    
    ### What changes were proposed in this pull request?
    
    Use `bool` instead of `np.bool` as `np.bool` will be deprecated (see: https://numpy.org/devdocs/release/1.20.0-notes.html#deprecations)
    
    Using `np.bool` generates this warning:
    
    ```
    UserWarning: toPandas attempted Arrow optimization because 'spark.sql.execution.arrow.pyspark.enabled' is set to true, but has reached the error below and can not continue. Note that 'spark.sql.execution.arrow.pyspark.fallback.enabled' does not have an effect on failures in the middle of computation.
    3070E                     `np.bool` is a deprecated alias for the builtin `bool`. To silence this warning, use `bool` by itself. Doing this will not modify any behavior and is safe. If you specifically wanted the numpy scalar type, use `np.bool_` here.
    3071E                   Deprecated in NumPy 1.20; for more details and guidance: https://numpy.org/devdocs/release/1.20.0-notes.html#deprecations
    ```
    
    ### Why are the changes needed?
    Deprecation soon: https://numpy.org/devdocs/release/1.20.0-notes.html#deprecations.
    
    ### Does this PR introduce _any_ user-facing change?
    The warning will be suppressed
    
    ### How was this patch tested?
    Existing tests should suffice.
    
    Closes #37817 from ELHoussineT/patch-1.
    
    Authored-by: ELHoussineT <el...@gmail.com>
    Signed-off-by: Sean Owen <sr...@gmail.com>
---
 python/pyspark/sql/pandas/conversion.py | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/python/pyspark/sql/pandas/conversion.py b/python/pyspark/sql/pandas/conversion.py
index fff0bac5480..22717241fde 100644
--- a/python/pyspark/sql/pandas/conversion.py
+++ b/python/pyspark/sql/pandas/conversion.py
@@ -295,7 +295,7 @@ class PandasConversionMixin:
         elif type(dt) == DoubleType:
             return np.float64
         elif type(dt) == BooleanType:
-            return np.bool  # type: ignore[attr-defined]
+            return bool
         elif type(dt) == TimestampType:
             return np.datetime64
         elif type(dt) == TimestampNTZType:


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@spark.apache.org
For additional commands, e-mail: commits-help@spark.apache.org