[jira] [Comment Edited] (ARROW-10599) Prebuilt distributions (aka. pyarrow and libarrow-dev) should use the same ABI (with or without the DUAL abi)

["Kouhei Sutou (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/ARROW-10599?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17232439#comment-17232439 ] 

Kouhei Sutou edited comment on ARROW-10599 at 11/16/20, 1:59 AM:
-----------------------------------------------------------------

You should use {{python3 -m pip3 install --no-binary pyarrow pyarrow}} when you use libarrow-dev and system Python on Ubuntu. Note that you can't use libarrow-dev with non-system Python. It's built for system Python.

The manylinux1 wheel for pyarrow isn't only for Ubuntu. It can be used for other Linux distributions such as CentOS.



was (Author: kou):
You should use {{python3 -m pip3 install --no-binary pyarrow pyarrow}} when you use libarrow-dev}} and system Python on Ubuntu. Note that you can't use libarrow-dev with non-system Python. It's built for system Python.

The manylinux1 wheel for pyarrow isn't only for Ubuntu. It can be used for other Linux distributions such as CentOS.


> Prebuilt distributions (aka. pyarrow and libarrow-dev) should use the same ABI (with or without the DUAL abi)
> -------------------------------------------------------------------------------------------------------------
>
>                 Key: ARROW-10599
>                 URL: https://issues.apache.org/jira/browse/ARROW-10599
>             Project: Apache Arrow
>          Issue Type: New Feature
>          Components: C++, Python
>    Affects Versions: 0.17.0, 1.0.1, 2.0.0
>            Reporter: Tao He
>            Priority: Critical
>
> I have observed that the python release (pyarrow) and c++ release (libarrow-dev for ubuntu) are built using the different GCC ABI.
> The former, pyarrow, builtin within the manylinux1 environment, using gcc-4.8, however the later's ABI has a `[cxx11]` tag. That blocks users to develop python C extensions that depends on libarrow-dev. For example, we have developed `lib` A in C++, which use arrow's `Arrow::Buffer` from libarrow-dev, and wrap it using things like `pybind11` to a python module `liba`. After building the `liba` on commodity Ubuntu (which could install libarrow-dev with apt-get), the user import both `liba` and `pyarrow` to the python's script, it won't work correctly due to the ABI confliction (especially when it comes to the string cases).
> I can see two options to make it works:
> 1. build arrow's python package using static link, that the pyarrow won't contains so many shared libraries (libarrow.so, libarrow_python.so, etc.)
> 2. distribute `libarrow-dev` with `-D_GLIBCXX_USE_CXX11_ABI=0`
> I'm also wondering if there's any technical issues that not distributing packages in different languages with the same ABI.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)