You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@spark.apache.org by gu...@apache.org on 2022/03/29 02:10:49 UTC
[spark] branch branch-3.3 updated: [SPARK-38680][INFRA] Set upperbound for pandas-stubs in CI
This is an automated email from the ASF dual-hosted git repository.
gurwls223 pushed a commit to branch branch-3.3
in repository https://gitbox.apache.org/repos/asf/spark.git
The following commit(s) were added to refs/heads/branch-3.3 by this push:
new 2b3a089 [SPARK-38680][INFRA] Set upperbound for pandas-stubs in CI
2b3a089 is described below
commit 2b3a089d78c00103a4aaa183ab7f96fed39547e5
Author: Hyukjin Kwon <gu...@apache.org>
AuthorDate: Tue Mar 29 11:07:22 2022 +0900
[SPARK-38680][INFRA] Set upperbound for pandas-stubs in CI
### What changes were proposed in this pull request?
`pandas-stubs` released `1.2.0.54` but that seems like havng a breaking change:
e.g., ) https://github.com/apache/spark/runs/5729037000?check_suite_focus=true
```
starting mypy annotations test...
annotations failed mypy checks:
python/pyspark/pandas/ml.py:62: error: Incompatible types in assignment (expression has type "Index", variable has type "MultiIndex") [assignment]
python/pyspark/pandas/frame.py:6133: error: unused "type: ignore" comment
python/pyspark/pandas/frame.py:6292: error: Incompatible types in assignment (expression has type "Index", variable has type "MultiIndex") [assignment]
Found 3 errors in 2 files (checked 325 source files)
```
This PR proposes to pin pandas-stubs version for now to avoid breaking CI.
### Why are the changes needed?
To have the stable CI status.
### Does this PR introduce _any_ user-facing change?
No, dev-only.
### How was this patch tested?
It should be tested in this CI.
Closes #35996 from HyukjinKwon/SPARK-38680.
Authored-by: Hyukjin Kwon <gu...@apache.org>
Signed-off-by: Hyukjin Kwon <gu...@apache.org>
(cherry picked from commit 2d90659d09b153ebd4100db686d5c63e8dbe63db)
Signed-off-by: Hyukjin Kwon <gu...@apache.org>
---
.github/workflows/build_and_test.yml | 2 +-
dev/requirements.txt | 3 ++-
2 files changed, 3 insertions(+), 2 deletions(-)
diff --git a/.github/workflows/build_and_test.yml b/.github/workflows/build_and_test.yml
index 7baabc7..1f5df70 100644
--- a/.github/workflows/build_and_test.yml
+++ b/.github/workflows/build_and_test.yml
@@ -503,7 +503,7 @@ jobs:
# Jinja2 3.0.0+ causes error when building with Sphinx.
# See also https://issues.apache.org/jira/browse/SPARK-35375.
python3.9 -m pip install 'flake8==3.9.0' pydata_sphinx_theme 'mypy==0.920' 'pytest-mypy-plugins==1.9.3' numpydoc 'jinja2<3.0.0' 'black==21.12b0'
- python3.9 -m pip install pandas-stubs
+ python3.9 -m pip install 'pandas-stubs==1.2.0.53'
- name: Install R linter dependencies and SparkR
run: |
apt-get install -y libcurl4-openssl-dev libgit2-dev libssl-dev libxml2-dev
diff --git a/dev/requirements.txt b/dev/requirements.txt
index b17694a..22e72d5 100644
--- a/dev/requirements.txt
+++ b/dev/requirements.txt
@@ -21,7 +21,8 @@ coverage
mypy
pytest-mypy-plugins==1.9.3
flake8==3.9.0
-pandas-stubs
+# See SPARK-38680.
+pandas-stubs<1.2.0.54
# Documentation (SQL)
mkdocs
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@spark.apache.org
For additional commands, e-mail: commits-help@spark.apache.org