You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@spark.apache.org by gu...@apache.org on 2021/04/28 00:56:57 UTC
[spark] branch master updated: [SPARK-34979][PYTHON][DOC] Add
PyArrow installation note for PySpark aarch64 user
This is an automated email from the ASF dual-hosted git repository.
gurwls223 pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/spark.git
The following commit(s) were added to refs/heads/master by this push:
new 0769049 [SPARK-34979][PYTHON][DOC] Add PyArrow installation note for PySpark aarch64 user
0769049 is described below
commit 0769049ee1701535646a3e5e709e333ca46fac1f
Author: Yikun Jiang <yi...@gmail.com>
AuthorDate: Wed Apr 28 09:56:17 2021 +0900
[SPARK-34979][PYTHON][DOC] Add PyArrow installation note for PySpark aarch64 user
### What changes were proposed in this pull request?
This patch adds a note for aarch64 user to install the specific pyarrow>=4.0.0.
### Why are the changes needed?
The pyarrow aarch64 support is [introduced](https://github.com/apache/arrow/pull/9285) in [PyArrow 4.0.0](https://github.com/apache/arrow/releases/tag/apache-arrow-4.0.0), and it has been published 27.Apr.2021.
See more in [SPARK-34979](https://issues.apache.org/jira/browse/SPARK-34979).
### Does this PR introduce _any_ user-facing change?
Yes, this doc can help user install arrow on aarch64.
### How was this patch tested?
doc test passed.
Closes #32363 from Yikun/SPARK-34979.
Authored-by: Yikun Jiang <yi...@gmail.com>
Signed-off-by: hyukjinkwon <gu...@apache.org>
---
python/docs/source/getting_started/install.rst | 8 ++++++++
1 file changed, 8 insertions(+)
diff --git a/python/docs/source/getting_started/install.rst b/python/docs/source/getting_started/install.rst
index a14f2b8..4bc68a9 100644
--- a/python/docs/source/getting_started/install.rst
+++ b/python/docs/source/getting_started/install.rst
@@ -164,3 +164,11 @@ Package Minimum supported version Note
Note that PySpark requires Java 8 or later with ``JAVA_HOME`` properly set.
If using JDK 11, set ``-Dio.netty.tryReflectionSetAccessible=true`` for Arrow related features and refer
to |downloading|_.
+
+Note for AArch64 (ARM64) users: PyArrow is required by PySpark SQL, but PyArrow support for AArch64
+is introduced in PyArrow 4.0.0. If PySpark installation fails on AArch64 due to PyArrow
+installation errors, you can install PyArrow >= 4.0.0 as below:
+
+.. code-block:: bash
+
+ pip install "pyarrow>=4.0.0" --prefer-binary
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@spark.apache.org
For additional commands, e-mail: commits-help@spark.apache.org