You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@arrow.apache.org by Kirill Lykov <ly...@gmail.com> on 2020/12/21 16:20:04 UTC
building and debugging on Mac without rpath
Hi,
I experience, probably, a well known problem with mac that
DYLD_LIBRARY_PATH is flashed.
Maybe there is a flag in the arrow which helps overcoming this problem
already?
It appears when I try to debug my code:
```bash
lldb python
run
c
>> import pyarrow as pa
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File
"/Users/klykov1/Documents/workspace/arrow_copy/python/pyarrow/__init__.py",
line 63, in <module>
import pyarrow.lib as _lib
ImportError:
dlopen(/Users/klykov1/Documents/workspace/arrow_copy/python/pyarrow/
lib.cpython-38-darwin.so, 2): Library not loaded: @rpath/libarrow.200.dylib
Referenced from:
/Users/klykov1/Documents/workspace/arrow_copy/python/pyarrow/
lib.cpython-38-darwin.so
Reason: image not found
```
I tried the following but seems it is not enough:
```
set(CMAKE_CXX_FLAGS "-Wl,-rpath,/arrowdir/dist/lib")
```
--
Best regards,
Kirill Lykov
Re: building and debugging on Mac without rpath
Posted by Kirill Lykov <ly...@gmail.com>.
I'm not sure what is the right place to document my experience with
building on macos.
I will write it here yet not sure it might be useful for a broader audience.
On MacOS Mojave:
1. To build, I had to use clang provided with macos together with python
built with the same clang.
2. A possible problem with brew update: after clang update by brew, I get
an error with `ld` because ld is shipped with macos and was not updated. So
be careful with version of clang.
3. On mac there is "system integrity protection" which is flushing
DYLD_LIBRARY_PATH which prevents debugging of C++ code from python code. It
cannot revolve rpath and there is a workaround to replace rpath to
physical path. It can be turned off, I guess, but not on my laptop which is
not personal.
On MacOS Catalina (10.15.7):
clang version is 12.0. So when I was trying to build Gandiva, I got error
that llvm version is too fresh:
Could not find a configuration file for package "LLVM" that is compatible
with requested version "10".
...
CMake Error at
/usr/local/Cellar/cmake/3.19.2/share/cmake/Modules/FindPackageHandleStandardArgs.cmake:218
(message):
Could NOT find LLVMAlt (missing: LLVM_PACKAGE_VERSION CLANG_EXECUTABLE
LLVM_FOUND LLVM_LINK_EXECUTABLE)
Call Stack (most recent call first):
/usr/local/Cellar/cmake/3.19.2/share/cmake/Modules/FindPackageHandleStandardArgs.cmake:582
(_FPHSA_FAILURE_MESSAGE)
cmake_modules/FindLLVMAlt.cmake:71 (find_package_handle_standard_args)
src/gandiva/CMakeLists.txt:28 (find_package)
I guess it might be due to gandiva, but haven't checked it without gandiva.
I haven't read many posts about building on mac so not sure if it is
interesting for a broad audience since one can use docker to build arrow.
On Tue, Dec 22, 2020 at 10:03 PM Wes McKinney <we...@gmail.com> wrote:
> What does it mean that DYLD_LIBRARY_PATH is "flashed"? It seems like
> there are some issues here which may affect other developers, in which
> case we should try to document them in our docs for future reference.
>
> On Mon, Dec 21, 2020 at 1:19 PM Neal Richardson
> <ne...@gmail.com> wrote:
> >
> > Building with -DARROW_INSTALL_NAME_RPATH=OFF may fix this (it seems to
> > resolve the similar issue I've experienced in R).
> >
> > Neal
> >
> > On Mon, Dec 21, 2020 at 8:20 AM Kirill Lykov <ly...@gmail.com>
> wrote:
> >
> > > Hi,
> > >
> > > I experience, probably, a well known problem with mac that
> > > DYLD_LIBRARY_PATH is flashed.
> > > Maybe there is a flag in the arrow which helps overcoming this problem
> > > already?
> > >
> > > It appears when I try to debug my code:
> > > ```bash
> > > lldb python
> > > run
> > > c
> > > >> import pyarrow as pa
> > > Traceback (most recent call last):
> > > File "<stdin>", line 1, in <module>
> > > File
> > >
> "/Users/klykov1/Documents/workspace/arrow_copy/python/pyarrow/__init__.py",
> > > line 63, in <module>
> > > import pyarrow.lib as _lib
> > > ImportError:
> > > dlopen(/Users/klykov1/Documents/workspace/arrow_copy/python/pyarrow/
> > > lib.cpython-38-darwin.so, 2): Library not loaded:
> > > @rpath/libarrow.200.dylib
> > > Referenced from:
> > > /Users/klykov1/Documents/workspace/arrow_copy/python/pyarrow/
> > > lib.cpython-38-darwin.so
> > > Reason: image not found
> > > ```
> > >
> > > I tried the following but seems it is not enough:
> > > ```
> > > set(CMAKE_CXX_FLAGS "-Wl,-rpath,/arrowdir/dist/lib")
> > > ```
> > >
> > > --
> > > Best regards,
> > > Kirill Lykov
> > >
>
--
Best regards,
Kirill Lykov
Re: building and debugging on Mac without rpath
Posted by Wes McKinney <we...@gmail.com>.
What does it mean that DYLD_LIBRARY_PATH is "flashed"? It seems like
there are some issues here which may affect other developers, in which
case we should try to document them in our docs for future reference.
On Mon, Dec 21, 2020 at 1:19 PM Neal Richardson
<ne...@gmail.com> wrote:
>
> Building with -DARROW_INSTALL_NAME_RPATH=OFF may fix this (it seems to
> resolve the similar issue I've experienced in R).
>
> Neal
>
> On Mon, Dec 21, 2020 at 8:20 AM Kirill Lykov <ly...@gmail.com> wrote:
>
> > Hi,
> >
> > I experience, probably, a well known problem with mac that
> > DYLD_LIBRARY_PATH is flashed.
> > Maybe there is a flag in the arrow which helps overcoming this problem
> > already?
> >
> > It appears when I try to debug my code:
> > ```bash
> > lldb python
> > run
> > c
> > >> import pyarrow as pa
> > Traceback (most recent call last):
> > File "<stdin>", line 1, in <module>
> > File
> > "/Users/klykov1/Documents/workspace/arrow_copy/python/pyarrow/__init__.py",
> > line 63, in <module>
> > import pyarrow.lib as _lib
> > ImportError:
> > dlopen(/Users/klykov1/Documents/workspace/arrow_copy/python/pyarrow/
> > lib.cpython-38-darwin.so, 2): Library not loaded:
> > @rpath/libarrow.200.dylib
> > Referenced from:
> > /Users/klykov1/Documents/workspace/arrow_copy/python/pyarrow/
> > lib.cpython-38-darwin.so
> > Reason: image not found
> > ```
> >
> > I tried the following but seems it is not enough:
> > ```
> > set(CMAKE_CXX_FLAGS "-Wl,-rpath,/arrowdir/dist/lib")
> > ```
> >
> > --
> > Best regards,
> > Kirill Lykov
> >
Re: building and debugging on Mac without rpath
Posted by Neal Richardson <ne...@gmail.com>.
Building with -DARROW_INSTALL_NAME_RPATH=OFF may fix this (it seems to
resolve the similar issue I've experienced in R).
Neal
On Mon, Dec 21, 2020 at 8:20 AM Kirill Lykov <ly...@gmail.com> wrote:
> Hi,
>
> I experience, probably, a well known problem with mac that
> DYLD_LIBRARY_PATH is flashed.
> Maybe there is a flag in the arrow which helps overcoming this problem
> already?
>
> It appears when I try to debug my code:
> ```bash
> lldb python
> run
> c
> >> import pyarrow as pa
> Traceback (most recent call last):
> File "<stdin>", line 1, in <module>
> File
> "/Users/klykov1/Documents/workspace/arrow_copy/python/pyarrow/__init__.py",
> line 63, in <module>
> import pyarrow.lib as _lib
> ImportError:
> dlopen(/Users/klykov1/Documents/workspace/arrow_copy/python/pyarrow/
> lib.cpython-38-darwin.so, 2): Library not loaded:
> @rpath/libarrow.200.dylib
> Referenced from:
> /Users/klykov1/Documents/workspace/arrow_copy/python/pyarrow/
> lib.cpython-38-darwin.so
> Reason: image not found
> ```
>
> I tried the following but seems it is not enough:
> ```
> set(CMAKE_CXX_FLAGS "-Wl,-rpath,/arrowdir/dist/lib")
> ```
>
> --
> Best regards,
> Kirill Lykov
>