[spark] branch master updated: [SPARK-39762][INFRA][PS] Support latest numpy in infra

[gu...@apache.org]

This is an automated email from the ASF dual-hosted git repository.

gurwls223 pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/spark.git


The following commit(s) were added to refs/heads/master by this push:
     new efb5bb7c52a [SPARK-39762][INFRA][PS] Support latest numpy in infra
efb5bb7c52a is described below

commit efb5bb7c52a294ef691f722dbbab86a0aa876407
Author: Yikun Jiang <yi...@gmail.com>
AuthorDate: Wed Jul 13 20:49:40 2022 +0900

    [SPARK-39762][INFRA][PS] Support latest numpy in infra
    
    ### What changes were proposed in this pull request?
    Remove infra numpy<1.23.0 version limit to support numpy 1.23+ (latest) in infra.
    
    ### Why are the changes needed?
    After below two PRs merged:
    
    https://github.com/apache/spark/pull/37117: Fix annotation: `python/pyspark/pandas/frame.py:9970: error: Need type annotation for "raveled_column_labels"  [var-annotated]`
    https://github.com/apache/spark/pull/37078: Fix wrong aliases in __array_ufunc__: `NotImplementedError: pandas-on-Spark objects currently do not support <ufunc 'divide'>`
    
    We can now remove limit on infra file to support numpy > 1.23.0.
    
    ### Does this PR introduce _any_ user-facing change?
    No
    
    ### How was this patch tested?
    CI passed and [numpy 1.23.1](https://github.com/Yikun/spark/runs/7314545823?check_suite_focus=true#step:9:49) installed in CI
    
    Closes #37175 from Yikun/patch-24.
    
    Authored-by: Yikun Jiang <yi...@gmail.com>
    Signed-off-by: Hyukjin Kwon <gu...@apache.org>
---
 dev/infra/Dockerfile | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/dev/infra/Dockerfile b/dev/infra/Dockerfile
index 7c46058a289..005727a518a 100644
--- a/dev/infra/Dockerfile
+++ b/dev/infra/Dockerfile
@@ -31,7 +31,7 @@ RUN $APT_INSTALL software-properties-common git libxml2-dev pkg-config curl wget
 RUN update-alternatives --set java /usr/lib/jvm/java-8-openjdk-amd64/jre/bin/java
 
 RUN curl -sS https://bootstrap.pypa.io/get-pip.py | python3.9
-RUN python3.9 -m pip install 'numpy<1.23.0' pyarrow 'pandas<1.4.0' scipy xmlrunner plotly>=4.8 sklearn 'mlflow>=1.0' coverage matplotlib
+RUN python3.9 -m pip install numpy pyarrow 'pandas<1.4.0' scipy xmlrunner plotly>=4.8 sklearn 'mlflow>=1.0' coverage matplotlib
 
 RUN add-apt-repository ppa:pypy/ppa
 RUN apt update
@@ -44,7 +44,7 @@ RUN mkdir -p /usr/local/pypy/pypy3.7 && \
     ln -sf /usr/local/pypy/pypy3.7/bin/pypy /usr/local/bin/pypy3
 
 RUN curl -sS https://bootstrap.pypa.io/get-pip.py | pypy3
-RUN pypy3 -m pip install 'numpy<1.23.0' 'pandas<1.4.0' scipy coverage matplotlib
+RUN pypy3 -m pip install numpy 'pandas<1.4.0' scipy coverage matplotlib
 
 RUN $APT_INSTALL gnupg ca-certificates pandoc
 RUN echo 'deb https://cloud.r-project.org/bin/linux/ubuntu focal-cran40/' >> /etc/apt/sources.list


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@spark.apache.org
For additional commands, e-mail: commits-help@spark.apache.org