You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by GitBox <gi...@apache.org> on 2020/10/22 03:36:45 UTC
[GitHub] [spark] dongjoon-hyun opened a new pull request #30127: [SPARK-33216][INFRA][PYTHON][2.4] Set upper bound of Pandas version in GitHub Actions
dongjoon-hyun opened a new pull request #30127:
URL: https://github.com/apache/spark/pull/30127
### What changes were proposed in this pull request?
This PR aims to pin `Pandas` version up to `1.1.0` released on `July 28`.
### Why are the changes needed?
`branch-2.4` GitHub Action was broken since `Aug 26` due to Pandas API incompatibility.
- https://github.com/apache/spark/runs/1034685663
The following is just one example.
```
File "/home/runner/work/spark/spark/python/pyspark/sql/tests.py", line 5948, in change_col_order
return pd.DataFrame.from_items([
AttributeError: type object 'DataFrame' has no attribute 'from_items'
```
### Does this PR introduce _any_ user-facing change?
No.
### How was this patch tested?
Pass the GitHub Action.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #30127: [SPARK-33216][INFRA][PYTHON][2.4] Set upper bound of Pandas version in GitHub Actions
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #30127:
URL: https://github.com/apache/spark/pull/30127#issuecomment-714243577
**[Test build #130120 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/130120/testReport)** for PR 30127 at commit [`9729565`](https://github.com/apache/spark/commit/9729565d4db4e68206289284539ff21e5b4a47e1).
* This patch passes all tests.
* This patch merges cleanly.
* This patch adds no public classes.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #30127: [SPARK-33216][INFRA][PYTHON][2.4] Set upper bound of Pandas version in GitHub Actions
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #30127:
URL: https://github.com/apache/spark/pull/30127#issuecomment-714199097
Kubernetes integration test status success
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/34728/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #30127: [SPARK-33216][INFRA][PYTHON][2.4] Set upper bound of Pandas version in GitHub Actions
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #30127:
URL: https://github.com/apache/spark/pull/30127#issuecomment-714241455
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #30127: [WIP][INFRA][PYTHON][2.4] Set upper bound of Pandas version in GitHub Actions
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #30127:
URL: https://github.com/apache/spark/pull/30127#issuecomment-714187612
**[Test build #130120 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/130120/testReport)** for PR 30127 at commit [`9729565`](https://github.com/apache/spark/commit/9729565d4db4e68206289284539ff21e5b4a47e1).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] dongjoon-hyun commented on pull request #30127: [SPARK-33216][INFRA][PYTHON][2.4] Set upper bound of Pandas version in GitHub Actions
Posted by GitBox <gi...@apache.org>.
dongjoon-hyun commented on pull request #30127:
URL: https://github.com/apache/spark/pull/30127#issuecomment-714200852
Thank you for investigating all the stuff and guiding this, @HyukjinKwon !
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] dongjoon-hyun edited a comment on pull request #30127: [SPARK-33216][INFRA][PYTHON][2.4] Set upper bound of Pandas version in GitHub Actions
Posted by GitBox <gi...@apache.org>.
dongjoon-hyun edited a comment on pull request #30127:
URL: https://github.com/apache/spark/pull/30127#issuecomment-714188408
@HyukjinKwon How do you think about this? `branch-2.4` GitHub Action seems to be broken for a while.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] HyukjinKwon closed pull request #30127: [SPARK-33216][INFRA][PYTHON][2.4] Set upper bound of Pandas version in GitHub Actions
Posted by GitBox <gi...@apache.org>.
HyukjinKwon closed pull request #30127:
URL: https://github.com/apache/spark/pull/30127
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #30127: [SPARK-33216][INFRA][PYTHON][2.4] Set upper bound of Pandas version in GitHub Actions
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #30127:
URL: https://github.com/apache/spark/pull/30127#issuecomment-714190019
**[Test build #130121 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/130121/testReport)** for PR 30127 at commit [`260a5d5`](https://github.com/apache/spark/commit/260a5d5926c71a881389be09ce75265ed778253e).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #30127: [SPARK-33216][INFRA][PYTHON][2.4] Set upper bound of Pandas version in GitHub Actions
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #30127:
URL: https://github.com/apache/spark/pull/30127#issuecomment-714199110
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #30127: [SPARK-33216][INFRA][PYTHON][2.4] Set upper bound of Pandas version in GitHub Actions
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #30127:
URL: https://github.com/apache/spark/pull/30127#issuecomment-714211996
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] HyukjinKwon commented on a change in pull request #30127: [SPARK-33216][INFRA][PYTHON][2.4] Set upper bound of Pandas version in GitHub Actions
Posted by GitBox <gi...@apache.org>.
HyukjinKwon commented on a change in pull request #30127:
URL: https://github.com/apache/spark/pull/30127#discussion_r509853641
##########
File path: .github/workflows/build_and_test.yml
##########
@@ -131,16 +131,16 @@ jobs:
# PyArrow is not supported in PyPy yet, see ARROW-2651.
# TODO(SPARK-32247): scipy installation with PyPy fails for an unknown reason.
run: |
- python3.6 -m pip install numpy 'pyarrow<3.0.0' pandas scipy xmlrunner
+ python3.6 -m pip install numpy 'pyarrow<3.0.0' 'pandas<1.1.1' scipy xmlrunner
Review comment:
Sure, either setting an upper bound or porting https://github.com/apache/spark/commit/07a9885f2792be1353f4a923d649e90bc431cb38 back should be fine. We're picking one version to test anyway.
BTW, I think `DataFrame.from_item` was removed in pandas 1.0.0 (https://github.com/pandas-dev/pandas/blob/master/doc/source/whatsnew/v1.0.0.rst#removal-of-prior-version-deprecationschanges).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #30127: [SPARK-33216][INFRA][PYTHON][2.4] Set upper bound of Pandas version in GitHub Actions
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #30127:
URL: https://github.com/apache/spark/pull/30127#issuecomment-714211970
Kubernetes integration test status success
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/34729/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #30127: [SPARK-33216][INFRA][PYTHON][2.4] Set upper bound of Pandas version in GitHub Actions
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #30127:
URL: https://github.com/apache/spark/pull/30127#issuecomment-714244076
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #30127: [SPARK-33216][INFRA][PYTHON][2.4] Set upper bound of Pandas version in GitHub Actions
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #30127:
URL: https://github.com/apache/spark/pull/30127#issuecomment-714187612
**[Test build #130120 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/130120/testReport)** for PR 30127 at commit [`9729565`](https://github.com/apache/spark/commit/9729565d4db4e68206289284539ff21e5b4a47e1).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] HyukjinKwon edited a comment on pull request #30127: [SPARK-33216][INFRA][PYTHON][2.4] Set upper bound of Pandas version in GitHub Actions
Posted by GitBox <gi...@apache.org>.
HyukjinKwon edited a comment on pull request #30127:
URL: https://github.com/apache/spark/pull/30127#issuecomment-714200325
I discussed offline with @dongjoon-hyun. It requires port two PRs https://github.com/apache/spark/commit/07a9885f2792be1353f4a923d649e90bc431cb38 and https://github.com/apache/spark/commit/16990f929921b3f784a85f3afbe1a22fbe77d895 + Arrow dependency upgrade in POM side which is not possible. Setting upperbound looks correct apporach. It was my bad that I thought it just needs one backport.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #30127: [SPARK-33216][INFRA][PYTHON][2.4] Set upper bound of Pandas version in GitHub Actions
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #30127:
URL: https://github.com/apache/spark/pull/30127#issuecomment-714193969
Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/34728/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] dongjoon-hyun commented on a change in pull request #30127: [SPARK-33216][INFRA][PYTHON][2.4] Set upper bound of Pandas version in GitHub Actions
Posted by GitBox <gi...@apache.org>.
dongjoon-hyun commented on a change in pull request #30127:
URL: https://github.com/apache/spark/pull/30127#discussion_r509854432
##########
File path: .github/workflows/build_and_test.yml
##########
@@ -131,16 +131,16 @@ jobs:
# PyArrow is not supported in PyPy yet, see ARROW-2651.
# TODO(SPARK-32247): scipy installation with PyPy fails for an unknown reason.
run: |
- python3.6 -m pip install numpy 'pyarrow<3.0.0' pandas scipy xmlrunner
+ python3.6 -m pip install numpy 'pyarrow<3.0.0' 'pandas<1.1.1' scipy xmlrunner
Review comment:
Got it. In that case, could you backport 07a9885 ?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] dongjoon-hyun commented on pull request #30127: [SPARK-33216][INFRA][PYTHON][2.4] Set upper bound of Pandas version in GitHub Actions
Posted by GitBox <gi...@apache.org>.
dongjoon-hyun commented on pull request #30127:
URL: https://github.com/apache/spark/pull/30127#issuecomment-714194661
Please recover branch-2.4~ Thank you in advance.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] HyukjinKwon edited a comment on pull request #30127: [SPARK-33216][INFRA][PYTHON][2.4] Set upper bound of Pandas version in GitHub Actions
Posted by GitBox <gi...@apache.org>.
HyukjinKwon edited a comment on pull request #30127:
URL: https://github.com/apache/spark/pull/30127#issuecomment-714200325
I discussed offline with @dongjoon-hyun. It requires to port at least two PRs https://github.com/apache/spark/commit/07a9885f2792be1353f4a923d649e90bc431cb38 and https://github.com/apache/spark/commit/16990f929921b3f784a85f3afbe1a22fbe77d895 + Arrow dependency upgrade in POM side which is not possible. Setting the upper bound looks correct approach. It was my bad that I thought it just needs one backport.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] HyukjinKwon edited a comment on pull request #30127: [SPARK-33216][INFRA][PYTHON][2.4] Set upper bound of Pandas version in GitHub Actions
Posted by GitBox <gi...@apache.org>.
HyukjinKwon edited a comment on pull request #30127:
URL: https://github.com/apache/spark/pull/30127#issuecomment-714200325
I discussed offline with @dongjoon-hyun. It requires port at least two PRs https://github.com/apache/spark/commit/07a9885f2792be1353f4a923d649e90bc431cb38 and https://github.com/apache/spark/commit/16990f929921b3f784a85f3afbe1a22fbe77d895 + Arrow dependency upgrade in POM side which is not possible. Setting the upper bound looks correct approach. It was my bad that I thought it just needs one backport.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #30127: [SPARK-33216][INFRA][PYTHON][2.4] Set upper bound of Pandas version in GitHub Actions
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #30127:
URL: https://github.com/apache/spark/pull/30127#issuecomment-714199110
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] dongjoon-hyun closed pull request #30127: [SPARK-33216][INFRA][PYTHON][2.4] Set upper bound of Pandas version in GitHub Actions
Posted by GitBox <gi...@apache.org>.
dongjoon-hyun closed pull request #30127:
URL: https://github.com/apache/spark/pull/30127
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] HyukjinKwon commented on pull request #30127: [SPARK-33216][INFRA][PYTHON][2.4] Set upper bound of Pandas version in GitHub Actions
Posted by GitBox <gi...@apache.org>.
HyukjinKwon commented on pull request #30127:
URL: https://github.com/apache/spark/pull/30127#issuecomment-714200325
I discussed offline. It requires port two PRs https://github.com/apache/spark/commit/07a9885f2792be1353f4a923d649e90bc431cb38 and https://github.com/apache/spark/commit/16990f929921b3f784a85f3afbe1a22fbe77d895 + Arrow dependency upgrade in POM side which is not possible. Setting upperbound looks correct apporach. It was my bad that I thought it just needs one backport.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #30127: [SPARK-33216][INFRA][PYTHON][2.4] Set upper bound of Pandas version in GitHub Actions
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #30127:
URL: https://github.com/apache/spark/pull/30127#issuecomment-714201874
**[Test build #130122 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/130122/testReport)** for PR 30127 at commit [`260a5d5`](https://github.com/apache/spark/commit/260a5d5926c71a881389be09ce75265ed778253e).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #30127: [SPARK-33216][INFRA][PYTHON][2.4] Set upper bound of Pandas version in GitHub Actions
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #30127:
URL: https://github.com/apache/spark/pull/30127#issuecomment-714207750
Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/34729/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] dongjoon-hyun commented on pull request #30127: [SPARK-33216][INFRA][PYTHON][2.4] Set upper bound of Pandas version in GitHub Actions
Posted by GitBox <gi...@apache.org>.
dongjoon-hyun commented on pull request #30127:
URL: https://github.com/apache/spark/pull/30127#issuecomment-714194082
@HyukjinKwon . I'll close this PR in favor of SPARK-30777.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] HyukjinKwon commented on a change in pull request #30127: [SPARK-33216][INFRA][PYTHON][2.4] Set upper bound of Pandas version in GitHub Actions
Posted by GitBox <gi...@apache.org>.
HyukjinKwon commented on a change in pull request #30127:
URL: https://github.com/apache/spark/pull/30127#discussion_r509854538
##########
File path: .github/workflows/build_and_test.yml
##########
@@ -131,16 +131,16 @@ jobs:
# PyArrow is not supported in PyPy yet, see ARROW-2651.
# TODO(SPARK-32247): scipy installation with PyPy fails for an unknown reason.
run: |
- python3.6 -m pip install numpy 'pyarrow<3.0.0' pandas scipy xmlrunner
+ python3.6 -m pip install numpy 'pyarrow<3.0.0' 'pandas<1.1.1' scipy xmlrunner
Review comment:
Sure, no problem.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #30127: [SPARK-33216][INFRA][PYTHON][2.4] Set upper bound of Pandas version in GitHub Actions
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #30127:
URL: https://github.com/apache/spark/pull/30127#issuecomment-714241455
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #30127: [SPARK-33216][INFRA][PYTHON][2.4] Set upper bound of Pandas version in GitHub Actions
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #30127:
URL: https://github.com/apache/spark/pull/30127#issuecomment-714190019
**[Test build #130121 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/130121/testReport)** for PR 30127 at commit [`260a5d5`](https://github.com/apache/spark/commit/260a5d5926c71a881389be09ce75265ed778253e).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] HyukjinKwon commented on pull request #30127: [SPARK-33216][INFRA][PYTHON][2.4] Set upper bound of Pandas version in GitHub Actions
Posted by GitBox <gi...@apache.org>.
HyukjinKwon commented on pull request #30127:
URL: https://github.com/apache/spark/pull/30127#issuecomment-714198928
Some of them require Arrow dep upgrade in pom which isn't possible. I thought I fixed it by setting upper bound of PyArrow. Hm, let me fix it by the same approach proposed here.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #30127: [SPARK-33216][INFRA][PYTHON][2.4] Set upper bound of Pandas version in GitHub Actions
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #30127:
URL: https://github.com/apache/spark/pull/30127#issuecomment-714211996
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] HyukjinKwon commented on pull request #30127: [SPARK-33216][INFRA][PYTHON][2.4] Set upper bound of Pandas version in GitHub Actions
Posted by GitBox <gi...@apache.org>.
HyukjinKwon commented on pull request #30127:
URL: https://github.com/apache/spark/pull/30127#issuecomment-714209051
@dongjoon-hyun, I will open another PR with picking your commits - seems it fails.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #30127: [SPARK-33216][INFRA][PYTHON][2.4] Set upper bound of Pandas version in GitHub Actions
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #30127:
URL: https://github.com/apache/spark/pull/30127#issuecomment-714244076
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #30127: [SPARK-33216][INFRA][PYTHON][2.4] Set upper bound of Pandas version in GitHub Actions
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #30127:
URL: https://github.com/apache/spark/pull/30127#issuecomment-714270666
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] HyukjinKwon commented on pull request #30127: [SPARK-33216][INFRA][PYTHON][2.4] Set upper bound of Pandas version in GitHub Actions
Posted by GitBox <gi...@apache.org>.
HyukjinKwon commented on pull request #30127:
URL: https://github.com/apache/spark/pull/30127#issuecomment-714198721
Ohh okay got it. There are more tests being failed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #30127: [SPARK-33216][INFRA][PYTHON][2.4] Set upper bound of Pandas version in GitHub Actions
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #30127:
URL: https://github.com/apache/spark/pull/30127#issuecomment-714270666
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] dongjoon-hyun commented on pull request #30127: [SPARK-33216][INFRA][PYTHON][2.4] Set upper bound of Pandas version in GitHub Actions
Posted by GitBox <gi...@apache.org>.
dongjoon-hyun commented on pull request #30127:
URL: https://github.com/apache/spark/pull/30127#issuecomment-714188408
How do you think about this? `branch-2.4` GitHub Action seems to be broken for a while.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #30127: [SPARK-33216][INFRA][PYTHON][2.4] Set upper bound of Pandas version in GitHub Actions
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #30127:
URL: https://github.com/apache/spark/pull/30127#issuecomment-714240938
**[Test build #130121 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/130121/testReport)** for PR 30127 at commit [`260a5d5`](https://github.com/apache/spark/commit/260a5d5926c71a881389be09ce75265ed778253e).
* This patch passes all tests.
* This patch merges cleanly.
* This patch adds no public classes.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #30127: [SPARK-33216][INFRA][PYTHON][2.4] Set upper bound of Pandas version in GitHub Actions
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #30127:
URL: https://github.com/apache/spark/pull/30127#issuecomment-714269742
**[Test build #130122 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/130122/testReport)** for PR 30127 at commit [`260a5d5`](https://github.com/apache/spark/commit/260a5d5926c71a881389be09ce75265ed778253e).
* This patch passes all tests.
* This patch merges cleanly.
* This patch adds no public classes.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #30127: [SPARK-33216][INFRA][PYTHON][2.4] Set upper bound of Pandas version in GitHub Actions
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #30127:
URL: https://github.com/apache/spark/pull/30127#issuecomment-714201874
**[Test build #130122 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/130122/testReport)** for PR 30127 at commit [`260a5d5`](https://github.com/apache/spark/commit/260a5d5926c71a881389be09ce75265ed778253e).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org