cython and pyarrow -- undefined symbol

[Aldrin <ak...@ucsc.edu>]

Hi all!

For my problem, I have put a lot of context into this gist:

https://gist.github.com/drin/f0059adee42a508923ead67b3c30a7dc

The summary is that I am able to compile cython code that uses pyarrow and
my own C++ code (which calls C code and uses the arrow C++ libs), but when
I try to import the compiled cpython shared library, it produces the
following error:

/.../skytether.cpython-39-x86_64-linux-gnu.so: undefined symbol:
_ZNK5arrow6Status8ToStringB5cxx11Ev

In my gist, I include outputs of `ldd` and `readelf -Ws` on various
libraries, to show that I think it should be resolving the symbols, but I
suspect that maybe my cython build is statically linking some symbols that
should be dynamically linked?

I'm not well experienced in fixing linker errors, and this is my first time
using cython, so I'm not sure if the problem is how I'm compiling the
cython code or what I've written, or if the problem is something related to
how I've compiled the C++ and pyarrow libs.

I define my cython build in this `build.cython` file (because I assume
"setup.py" isn't an important name when I'm building locally):

https://gitlab.com/skyhookdm/skytether-singlecell/-/blob/feature-cythonize/cpp/build.cython

Also, here is a folder of my cython code:

https://gitlab.com/skyhookdm/skytether-singlecell/-/tree/feature-cythonize/cpp/cybindings

Thank you for any help you can provide!

Aldrin Montana
Computer Science PhD Student
UC Santa Cruz

Re: cython and pyarrow -- undefined symbol

[Aldrin <ak...@ucsc.edu>]

I finally got this to work. I confused the 'libraries' argument to
setuptools.Extension with 'library_dirs' and I wasn't specifying my C++
libraries correctly.

Apparently, if the libraries aren't specified correctly, linking doesn't
fail, there are just symbols that are undefined which cause an error at
runtime.

Aldrin Montana
Computer Science PhD Student
UC Santa Cruz


On Fri, Jul 9, 2021 at 4:43 PM Aldrin <ak...@ucsc.edu> wrote:

> Hi all!
>
> For my problem, I have put a lot of context into this gist:
>
> https://gist.github.com/drin/f0059adee42a508923ead67b3c30a7dc
>
> The summary is that I am able to compile cython code that uses pyarrow and
> my own C++ code (which calls C code and uses the arrow C++ libs), but when
> I try to import the compiled cpython shared library, it produces the
> following error:
>
> /.../skytether.cpython-39-x86_64-linux-gnu.so: undefined symbol:
> _ZNK5arrow6Status8ToStringB5cxx11Ev
>
> In my gist, I include outputs of `ldd` and `readelf -Ws` on various
> libraries, to show that I think it should be resolving the symbols, but I
> suspect that maybe my cython build is statically linking some symbols that
> should be dynamically linked?
>
> I'm not well experienced in fixing linker errors, and this is my first
> time using cython, so I'm not sure if the problem is how I'm compiling the
> cython code or what I've written, or if the problem is something related to
> how I've compiled the C++ and pyarrow libs.
>
> I define my cython build in this `build.cython` file (because I assume
> "setup.py" isn't an important name when I'm building locally):
>
>
> https://gitlab.com/skyhookdm/skytether-singlecell/-/blob/feature-cythonize/cpp/build.cython
>
> Also, here is a folder of my cython code:
>
>
> https://gitlab.com/skyhookdm/skytether-singlecell/-/tree/feature-cythonize/cpp/cybindings
>
> Thank you for any help you can provide!
>
> Aldrin Montana
> Computer Science PhD Student
> UC Santa Cruz
>