[spark] branch master updated: [SPARK-43917][PS][INFRA] Upgrade `pandas` to 2.0.2

[do...@apache.org]

This is an automated email from the ASF dual-hosted git repository.

dongjoon pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/spark.git


The following commit(s) were added to refs/heads/master by this push:
     new af61526bea7 [SPARK-43917][PS][INFRA] Upgrade `pandas` to 2.0.2
af61526bea7 is described below

commit af61526bea7d2a9e02c3d4acd691fc03695c4573
Author: Bjørn Jørgensen <bj...@gmail.com>
AuthorDate: Sat Jun 3 23:21:58 2023 -0700

    [SPARK-43917][PS][INFRA] Upgrade `pandas` to 2.0.2
    
    ### What changes were proposed in this pull request?
    Upgrade pandas from 2.0.0 to 2.0.2
    
    ### Why are the changes needed?
    This fixes some regressions and bugs.
    
    [Whats new in 2.0.2](https://pandas.pydata.org/docs/whatsnew/v2.0.2.html)
    
    [Whats new in 2.0.1](https://pandas.pydata.org/docs/whatsnew/v2.0.1.html)
    
    ### Does this PR introduce _any_ user-facing change?
    No.
    
    ### How was this patch tested?
    Pass GA
    
    Closes #41437 from bjornjorgensen/pandas2.0.2.
    
    Lead-authored-by: Bjørn Jørgensen <bj...@gmail.com>
    Co-authored-by: bjornjorgensen <bj...@gmail.com>
    Signed-off-by: Dongjoon Hyun <do...@apache.org>
---
 dev/infra/Dockerfile                       | 4 ++--
 python/pyspark/pandas/supported_api_gen.py | 2 +-
 2 files changed, 3 insertions(+), 3 deletions(-)

diff --git a/dev/infra/Dockerfile b/dev/infra/Dockerfile
index 888b4e00b39..3b95467389a 100644
--- a/dev/infra/Dockerfile
+++ b/dev/infra/Dockerfile
@@ -64,8 +64,8 @@ RUN Rscript -e "devtools::install_version('roxygen2', version='7.2.0', repos='ht
 # See more in SPARK-39735
 ENV R_LIBS_SITE "/usr/local/lib/R/site-library:${R_LIBS_SITE}:/usr/lib/R/library"
 
-RUN pypy3 -m pip install numpy 'pandas<=2.0.0' scipy coverage matplotlib
-RUN python3.9 -m pip install numpy pyarrow 'pandas<=2.0.0' scipy unittest-xml-reporting plotly>=4.8 'mlflow>=2.3.1' coverage matplotlib openpyxl 'memory-profiler==0.60.0' 'scikit-learn==1.1.*'
+RUN pypy3 -m pip install numpy 'pandas<=2.0.2' scipy coverage matplotlib
+RUN python3.9 -m pip install numpy pyarrow 'pandas<=2.0.2' scipy unittest-xml-reporting plotly>=4.8 'mlflow>=2.3.1' coverage matplotlib openpyxl 'memory-profiler==0.60.0' 'scikit-learn==1.1.*'
 
 # Add Python deps for Spark Connect.
 RUN python3.9 -m pip install grpcio protobuf googleapis-common-protos grpcio-status
diff --git a/python/pyspark/pandas/supported_api_gen.py b/python/pyspark/pandas/supported_api_gen.py
index b5d6cadd3ca..d259171ecb9 100644
--- a/python/pyspark/pandas/supported_api_gen.py
+++ b/python/pyspark/pandas/supported_api_gen.py
@@ -98,7 +98,7 @@ def generate_supported_api(output_rst_file_path: str) -> None:
 
     Write supported APIs documentation.
     """
-    pandas_latest_version = "2.0.0"
+    pandas_latest_version = "2.0.2"
     if LooseVersion(pd.__version__) != LooseVersion(pandas_latest_version):
         msg = (
             "Warning: Latest version of pandas (%s) is required to generate the documentation; "


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@spark.apache.org
For additional commands, e-mail: commits-help@spark.apache.org