You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@spark.apache.org by gu...@apache.org on 2020/10/22 05:36:42 UTC
[spark] branch branch-3.0 updated: [SPARK-32247][INFRA] Install and
test scipy with PyPy in GitHub Actions
This is an automated email from the ASF dual-hosted git repository.
gurwls223 pushed a commit to branch branch-3.0
in repository https://gitbox.apache.org/repos/asf/spark.git
The following commit(s) were added to refs/heads/branch-3.0 by this push:
new 933dc6c [SPARK-32247][INFRA] Install and test scipy with PyPy in GitHub Actions
933dc6c is described below
commit 933dc6cb7b3de1d8ccaf73d124d6eb95b947ed19
Author: HyukjinKwon <gu...@apache.org>
AuthorDate: Thu Oct 15 09:08:14 2020 -0700
[SPARK-32247][INFRA] Install and test scipy with PyPy in GitHub Actions
### What changes were proposed in this pull request?
This PR proposes to install `scipy` as well in PyPy. It will test several ML specific test cases in PyPy as well. For example, https://github.com/apache/spark/blob/31a16fbb405a19dc3eb732347e0e1f873b16971d/python/pyspark/mllib/tests/test_linalg.py#L487
It was not installed when GitHub Actions build was added because it failed to install for an unknown reason. Seems like it's fixed in the latest scipy.
### Why are the changes needed?
To improve test coverage.
### Does this PR introduce _any_ user-facing change?
No, dev-only.
### How was this patch tested?
GitHub Actions build in this PR will test it out.
Closes #30054 from HyukjinKwon/SPARK-32247.
Authored-by: HyukjinKwon <gu...@apache.org>
Signed-off-by: Dongjoon Hyun <dh...@apple.com>
---
.github/workflows/build_and_test.yml | 3 +--
1 file changed, 1 insertion(+), 2 deletions(-)
diff --git a/.github/workflows/build_and_test.yml b/.github/workflows/build_and_test.yml
index d1203c6..6afa41b 100644
--- a/.github/workflows/build_and_test.yml
+++ b/.github/workflows/build_and_test.yml
@@ -147,12 +147,11 @@ jobs:
- name: Install Python packages (Python 2.7 and PyPy3)
if: contains(matrix.modules, 'pyspark')
# PyArrow is not supported in PyPy yet, see ARROW-2651.
- # TODO(SPARK-32247): scipy installation with PyPy fails for an unknown reason.
run: |
python2.7 -m pip install numpy 'pyarrow<3.0.0' pandas scipy xmlrunner
python2.7 -m pip list
# PyPy does not have xmlrunner
- pypy3 -m pip install numpy pandas
+ pypy3 -m pip install numpy pandas scipy
pypy3 -m pip list
- name: Install Python packages (Python 3.8)
if: contains(matrix.modules, 'pyspark') || (contains(matrix.modules, 'sql') && !contains(matrix.modules, 'sql-'))
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@spark.apache.org
For additional commands, e-mail: commits-help@spark.apache.org