You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by GitBox <gi...@apache.org> on 2021/01/19 21:42:08 UTC
[GitHub] [spark] dongjoon-hyun opened a new pull request #31251: [SPARK-34162][DOCS][PYSPARK] Add PyArrow compatibility note for Python 3.9
dongjoon-hyun opened a new pull request #31251:
URL: https://github.com/apache/spark/pull/31251
### What changes were proposed in this pull request?
This PR aims to add a note for Apache Arrow project's `PyArrow` compatibility for Python 3.9.
### Why are the changes needed?
Officially, `PyArrow` is not compatible with Python 3.9.x yet.
### Does this PR introduce _any_ user-facing change?
https://arrow.apache.org/docs/python/install.html#python-compatibility
### How was this patch tested?
Manual.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #31251: [SPARK-34162][DOCS][PYSPARK] Add PyArrow compatibility note for Python 3.9
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31251:
URL: https://github.com/apache/spark/pull/31251#issuecomment-763175375
**[Test build #134241 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134241/testReport)** for PR 31251 at commit [`975efff`](https://github.com/apache/spark/commit/975efff97d1ec0b47c88a3d157754d8308238932).
* This patch passes all tests.
* This patch merges cleanly.
* This patch adds no public classes.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #31251: [SPARK-34162][DOCS][PYSPARK] Add PyArrow compatibility note for Python 3.9
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #31251:
URL: https://github.com/apache/spark/pull/31251#issuecomment-763297948
**[Test build #134248 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134248/testReport)** for PR 31251 at commit [`c306a38`](https://github.com/apache/spark/commit/c306a3881c1e69c8e8bc9cc39f757e39e4c39f30).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #31251: [SPARK-34162][DOCS][PYSPARK] Add PyArrow compatibility note for Python 3.9
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31251:
URL: https://github.com/apache/spark/pull/31251#issuecomment-763200735
Kubernetes integration test status success
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/38826/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] HyukjinKwon commented on a change in pull request #31251: [SPARK-34162][DOCS][PYSPARK] Add PyArrow compatibility note for Python 3.9
Posted by GitBox <gi...@apache.org>.
HyukjinKwon commented on a change in pull request #31251:
URL: https://github.com/apache/spark/pull/31251#discussion_r560588397
##########
File path: docs/index.md
##########
@@ -50,6 +50,7 @@ For the Scala API, Spark {{site.SPARK_VERSION}}
uses Scala {{site.SCALA_BINARY_VERSION}}. You will need to use a compatible Scala version
({{site.SCALA_BINARY_VERSION}}.x).
+For Python 3.9, Apache Arrow's PyArrow library is not compatible. Please check [Python Compatibility](https://arrow.apache.org/docs/python/install.html#python-compatibility) page.
Review comment:
and seems like they are running RCs for 3.0.0 (https://mail-archives.apache.org/mod_mbox/arrow-dev/202101.mbox/browser)
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] HyukjinKwon commented on a change in pull request #31251: [SPARK-34162][DOCS][PYSPARK] Add PyArrow compatibility note for Python 3.9
Posted by GitBox <gi...@apache.org>.
HyukjinKwon commented on a change in pull request #31251:
URL: https://github.com/apache/spark/pull/31251#discussion_r560586910
##########
File path: docs/index.md
##########
@@ -50,6 +50,7 @@ For the Scala API, Spark {{site.SPARK_VERSION}}
uses Scala {{site.SCALA_BINARY_VERSION}}. You will need to use a compatible Scala version
({{site.SCALA_BINARY_VERSION}}.x).
+For Python 3.9, Apache Arrow's PyArrow library is not compatible. Please check [Python Compatibility](https://arrow.apache.org/docs/python/install.html#python-compatibility) page.
Review comment:
@dongjoon-hyun, how about:
```suggestion
For Python 3.9, it will be supported in Apache Arrow 3.0+. Please check the latest [Python Compatibility](https://arrow.apache.org/docs/python/install.html#python-compatibility) page.
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #31251: [SPARK-34162][DOCS][PYSPARK] Add PyArrow compatibility note for Python 3.9
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31251:
URL: https://github.com/apache/spark/pull/31251#issuecomment-763314103
Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/38833/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #31251: [SPARK-34162][DOCS][PYSPARK] Add PyArrow compatibility note for Python 3.9
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31251:
URL: https://github.com/apache/spark/pull/31251#issuecomment-763170419
**[Test build #134241 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134241/testReport)** for PR 31251 at commit [`975efff`](https://github.com/apache/spark/commit/975efff97d1ec0b47c88a3d157754d8308238932).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #31251: [SPARK-34162][DOCS][PYSPARK] Add PyArrow compatibility note for Python 3.9
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #31251:
URL: https://github.com/apache/spark/pull/31251#issuecomment-763200756
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/38826/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #31251: [SPARK-34162][DOCS][PYSPARK] Add PyArrow compatibility note for Python 3.9
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #31251:
URL: https://github.com/apache/spark/pull/31251#issuecomment-763313575
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/134248/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] HyukjinKwon commented on a change in pull request #31251: [SPARK-34162][DOCS][PYSPARK] Add PyArrow compatibility note for Python 3.9
Posted by GitBox <gi...@apache.org>.
HyukjinKwon commented on a change in pull request #31251:
URL: https://github.com/apache/spark/pull/31251#discussion_r560590007
##########
File path: docs/index.md
##########
@@ -50,6 +50,7 @@ For the Scala API, Spark {{site.SPARK_VERSION}}
uses Scala {{site.SCALA_BINARY_VERSION}}. You will need to use a compatible Scala version
({{site.SCALA_BINARY_VERSION}}.x).
+For Python 3.9, Apache Arrow's PyArrow library is not compatible. Please check [Python Compatibility](https://arrow.apache.org/docs/python/install.html#python-compatibility) page.
Review comment:
On the other hand, it will be incorrect when Spark 3.1.1 is released as it looks Arrow 3.0.0 is likely released almost same timing.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] HyukjinKwon commented on a change in pull request #31251: [SPARK-34162][DOCS][PYSPARK] Add PyArrow compatibility note for Python 3.9
Posted by GitBox <gi...@apache.org>.
HyukjinKwon commented on a change in pull request #31251:
URL: https://github.com/apache/spark/pull/31251#discussion_r560591001
##########
File path: docs/index.md
##########
@@ -50,6 +50,7 @@ For the Scala API, Spark {{site.SPARK_VERSION}}
uses Scala {{site.SCALA_BINARY_VERSION}}. You will need to use a compatible Scala version
({{site.SCALA_BINARY_VERSION}}.x).
+For Python 3.9, Apache Arrow's PyArrow library is not compatible. Please check [Python Compatibility](https://arrow.apache.org/docs/python/install.html#python-compatibility) page.
Review comment:
How about just simply say like "For Python 3.9, Arrow optimization and pandas UDFs might not work due to the supported Python versions in Apache Arrow. Please refer to the latest Python Compatibility"?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #31251: [SPARK-34162][DOCS][PYSPARK] Add PyArrow compatibility note for Python 3.9
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #31251:
URL: https://github.com/apache/spark/pull/31251#issuecomment-763170419
**[Test build #134241 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134241/testReport)** for PR 31251 at commit [`975efff`](https://github.com/apache/spark/commit/975efff97d1ec0b47c88a3d157754d8308238932).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] dongjoon-hyun commented on a change in pull request #31251: [SPARK-34162][DOCS][PYSPARK] Add PyArrow compatibility note for Python 3.9
Posted by GitBox <gi...@apache.org>.
dongjoon-hyun commented on a change in pull request #31251:
URL: https://github.com/apache/spark/pull/31251#discussion_r560588440
##########
File path: docs/index.md
##########
@@ -50,6 +50,7 @@ For the Scala API, Spark {{site.SPARK_VERSION}}
uses Scala {{site.SCALA_BINARY_VERSION}}. You will need to use a compatible Scala version
({{site.SCALA_BINARY_VERSION}}.x).
+For Python 3.9, Apache Arrow's PyArrow library is not compatible. Please check [Python Compatibility](https://arrow.apache.org/docs/python/install.html#python-compatibility) page.
Review comment:
IMO, `will be supported in Apache Arrow` is not what we can say in the Apache Spark official website.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] dongjoon-hyun commented on pull request #31251: [SPARK-34162][DOCS][PYSPARK] Add PyArrow compatibility note for Python 3.9
Posted by GitBox <gi...@apache.org>.
dongjoon-hyun commented on pull request #31251:
URL: https://github.com/apache/spark/pull/31251#issuecomment-763163484
cc @HyukjinKwon
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #31251: [SPARK-34162][DOCS][PYSPARK] Add PyArrow compatibility note for Python 3.9
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #31251:
URL: https://github.com/apache/spark/pull/31251#issuecomment-763333115
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/38833/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #31251: [SPARK-34162][DOCS][PYSPARK] Add PyArrow compatibility note for Python 3.9
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #31251:
URL: https://github.com/apache/spark/pull/31251#issuecomment-763197416
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/134241/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] HyukjinKwon commented on a change in pull request #31251: [SPARK-34162][DOCS][PYSPARK] Add PyArrow compatibility note for Python 3.9
Posted by GitBox <gi...@apache.org>.
HyukjinKwon commented on a change in pull request #31251:
URL: https://github.com/apache/spark/pull/31251#discussion_r560588397
##########
File path: docs/index.md
##########
@@ -50,6 +50,7 @@ For the Scala API, Spark {{site.SPARK_VERSION}}
uses Scala {{site.SCALA_BINARY_VERSION}}. You will need to use a compatible Scala version
({{site.SCALA_BINARY_VERSION}}.x).
+For Python 3.9, Apache Arrow's PyArrow library is not compatible. Please check [Python Compatibility](https://arrow.apache.org/docs/python/install.html#python-compatibility) page.
Review comment:
and seems like they are running RCs for 3.0.0 (https://mail-archives.apache.org/mod_mbox/arrow-dev/202101.mbox/%3CCAHM19a68yOwWiFB%2B6U67zGOiTxNT-O1MOosZ3oyOpzuGHEg72A%40mail.gmail.com%3E)
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #31251: [SPARK-34162][DOCS][PYSPARK] Add PyArrow compatibility note for Python 3.9
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31251:
URL: https://github.com/apache/spark/pull/31251#issuecomment-763320096
Kubernetes integration test status failure
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/38833/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] dongjoon-hyun closed pull request #31251: [SPARK-34162][DOCS][PYSPARK] Add PyArrow compatibility note for Python 3.9
Posted by GitBox <gi...@apache.org>.
dongjoon-hyun closed pull request #31251:
URL: https://github.com/apache/spark/pull/31251
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] dongjoon-hyun commented on a change in pull request #31251: [SPARK-34162][DOCS][PYSPARK] Add PyArrow compatibility note for Python 3.9
Posted by GitBox <gi...@apache.org>.
dongjoon-hyun commented on a change in pull request #31251:
URL: https://github.com/apache/spark/pull/31251#discussion_r560588170
##########
File path: docs/index.md
##########
@@ -50,6 +50,7 @@ For the Scala API, Spark {{site.SPARK_VERSION}}
uses Scala {{site.SCALA_BINARY_VERSION}}. You will need to use a compatible Scala version
({{site.SCALA_BINARY_VERSION}}.x).
+For Python 3.9, Apache Arrow's PyArrow library is not compatible. Please check [Python Compatibility](https://arrow.apache.org/docs/python/install.html#python-compatibility) page.
Review comment:
Thanks. However, it may sounds inappropriate to claim some other open source community's suggestion. Apache Spark also drop Scala 2.13 from Spark 3.1.0.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] dongjoon-hyun commented on pull request #31251: [SPARK-34162][DOCS][PYSPARK] Add PyArrow compatibility note for Python 3.9
Posted by GitBox <gi...@apache.org>.
dongjoon-hyun commented on pull request #31251:
URL: https://github.com/apache/spark/pull/31251#issuecomment-763298073
Thank you for your review and advice, @HyukjinKwon . The newly generated doc is verified and the snapshot of PR description is also updated.
Merged to master/3.1.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #31251: [SPARK-34162][DOCS][PYSPARK] Add PyArrow compatibility note for Python 3.9
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31251:
URL: https://github.com/apache/spark/pull/31251#issuecomment-763303114
**[Test build #134248 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134248/testReport)** for PR 31251 at commit [`c306a38`](https://github.com/apache/spark/commit/c306a3881c1e69c8e8bc9cc39f757e39e4c39f30).
* This patch passes all tests.
* This patch merges cleanly.
* This patch adds no public classes.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #31251: [SPARK-34162][DOCS][PYSPARK] Add PyArrow compatibility note for Python 3.9
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #31251:
URL: https://github.com/apache/spark/pull/31251#issuecomment-763200756
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/38826/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] HyukjinKwon commented on a change in pull request #31251: [SPARK-34162][DOCS][PYSPARK] Add PyArrow compatibility note for Python 3.9
Posted by GitBox <gi...@apache.org>.
HyukjinKwon commented on a change in pull request #31251:
URL: https://github.com/apache/spark/pull/31251#discussion_r560587070
##########
File path: docs/index.md
##########
@@ -50,6 +50,7 @@ For the Scala API, Spark {{site.SPARK_VERSION}}
uses Scala {{site.SCALA_BINARY_VERSION}}. You will need to use a compatible Scala version
({{site.SCALA_BINARY_VERSION}}.x).
+For Python 3.9, Apache Arrow's PyArrow library is not compatible. Please check [Python Compatibility](https://arrow.apache.org/docs/python/install.html#python-compatibility) page.
Review comment:
related JIRAs:
- https://issues.apache.org/jira/browse/ARROW-10905
- https://issues.apache.org/jira/browse/ARROW-10904
- https://issues.apache.org/jira/browse/ARROW-10224
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #31251: [SPARK-34162][DOCS][PYSPARK] Add PyArrow compatibility note for Python 3.9
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31251:
URL: https://github.com/apache/spark/pull/31251#issuecomment-763188866
Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/38826/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] dongjoon-hyun edited a comment on pull request #31251: [SPARK-34162][DOCS][PYSPARK] Add PyArrow compatibility note for Python 3.9
Posted by GitBox <gi...@apache.org>.
dongjoon-hyun edited a comment on pull request #31251:
URL: https://github.com/apache/spark/pull/31251#issuecomment-763163484
cc @HyukjinKwon and @holdenk
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #31251: [SPARK-34162][DOCS][PYSPARK] Add PyArrow compatibility note for Python 3.9
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31251:
URL: https://github.com/apache/spark/pull/31251#issuecomment-763297948
**[Test build #134248 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134248/testReport)** for PR 31251 at commit [`c306a38`](https://github.com/apache/spark/commit/c306a3881c1e69c8e8bc9cc39f757e39e4c39f30).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #31251: [SPARK-34162][DOCS][PYSPARK] Add PyArrow compatibility note for Python 3.9
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #31251:
URL: https://github.com/apache/spark/pull/31251#issuecomment-763197416
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/134241/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #31251: [SPARK-34162][DOCS][PYSPARK] Add PyArrow compatibility note for Python 3.9
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #31251:
URL: https://github.com/apache/spark/pull/31251#issuecomment-763333115
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/38833/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] dongjoon-hyun commented on a change in pull request #31251: [SPARK-34162][DOCS][PYSPARK] Add PyArrow compatibility note for Python 3.9
Posted by GitBox <gi...@apache.org>.
dongjoon-hyun commented on a change in pull request #31251:
URL: https://github.com/apache/spark/pull/31251#discussion_r560643426
##########
File path: docs/index.md
##########
@@ -50,6 +50,7 @@ For the Scala API, Spark {{site.SPARK_VERSION}}
uses Scala {{site.SCALA_BINARY_VERSION}}. You will need to use a compatible Scala version
({{site.SCALA_BINARY_VERSION}}.x).
+For Python 3.9, Apache Arrow's PyArrow library is not compatible. Please check [Python Compatibility](https://arrow.apache.org/docs/python/install.html#python-compatibility) page.
Review comment:
That sounds reasonable for now.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #31251: [SPARK-34162][DOCS][PYSPARK] Add PyArrow compatibility note for Python 3.9
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #31251:
URL: https://github.com/apache/spark/pull/31251#issuecomment-763313575
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/134248/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org