[jira] [Created] (ARROW-10599) Prebuilt distributions (aka. pyarrow and libarrow-dev) should use the same ABI (with or without the DUAL abi)

["Tao He (Jira)" <ji...@apache.org>]

Tao He created ARROW-10599:
------------------------------

             Summary: Prebuilt distributions (aka. pyarrow and libarrow-dev) should use the same ABI (with or without the DUAL abi)
                 Key: ARROW-10599
                 URL: https://issues.apache.org/jira/browse/ARROW-10599
             Project: Apache Arrow
          Issue Type: New Feature
          Components: C++, Python
    Affects Versions: 2.0.0, 1.0.1, 0.17.0
            Reporter: Tao He


I have observed that the python release (pyarrow) and c++ release (libarrow-dev for ubuntu) are built using the different GCC ABI.

The former, pyarrow, builtin within the manylinux1 environment, using gcc-4.8, however the later's ABI has a `[cxx11]` tag. That blocks users to develop python C extensions that depends on libarrow-dev. For example, we have developed `lib` A in C++, which use arrow's `Arrow::Buffer` from libarrow-dev, and wrap it using things like `pybind11` to a python module `liba`. After building the `liba` on commodity Ubuntu (which could install libarrow-dev with apt-get), the user import both `liba` and `pyarrow` to the python's script, it won't work correctly due to the ABI confliction (especially when it comes to the string cases).

I can see two options to make it works:

1. build arrow's python package using static link, that the pyarrow won't contains so many shared libraries (libarrow.so, libarrow_python.so, etc.)
2. distribute `libarrow-dev` with `-D_GLIBCXX_USE_CXX11_ABI=0`

I'm also wondering if there's any technical issues that not distributing packages in different languages with the same ABI.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)