You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@arrow.apache.org by Ivan Gozali <iv...@lecida.com> on 2019/07/29 22:57:03 UTC

pyarrow & numba compatibility issue in macOS

Hi Arrow maintainers,

I wanted to report an issue that was previously discussed here:
https://github.com/numba/numba/issues/4256.

The main issue is that importing pyarrow, and then numba causes some weird
LLVM error, as follows:
$ pip3 install pyarrow numba
$ pip3 freeze
llvmlite==0.29.0
numba==0.45.0
numpy==1.17.0
pyarrow==0.14.1
six==1.12.0
$ python3
Python 3.7.4 (default, Jul 29 2019, 14:51:28)
[Clang 10.0.1 (clang-1001.0.46.4)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import pyarrow
>>> import numba
Assertion failed: (PassInf && "Expected all immutable passes to be
initialized"), function addImmutablePass, file
/Users/buildbot/miniconda3/conda-bld/llvmdev_1556270736866/work/lib/IR/LegacyPassManager.cpp,
line 849.
Abort trap: 6

After some additional digging, I found the first problematic version to be
pyarrow==0.13.0, because if I install pyarrow==0.12.1 the issue seems to
disappear.

Thank you!
--
Regards,


Ivan Gozali
Lecida
Email: ivan@lecida.com

Re: pyarrow & numba compatibility issue in macOS

Posted by Ivan Gozali <iv...@lecida.com>.
Hi Wes, I understand, and thanks for the email - that was extremely useful
background.

--
Regards,


Ivan Gozali
Lecida
Email: ivan@lecida.com


On Mon, Jul 29, 2019 at 4:01 PM Wes McKinney <we...@gmail.com> wrote:

> hi Ivan,
>
> We have statically linked LLVM symbols in our wheels on all platforms.
> We're looking for new maintainers for the wheels to take care of
> issues like this, see [1]. In the meantime, I suggest you use conda
> packages if you need to use numba and pyarrow together.
>
> Thanks
>
> [1]:
> https://lists.apache.org/thread.html/128a2bec285ad45aa4189ebb39a15b39dcf6d91c4ab0278ff4f7cdea@%3Cdev.arrow.apache.org%3E
>
> On Mon, Jul 29, 2019 at 5:57 PM Ivan Gozali <iv...@lecida.com> wrote:
> >
> > Hi Arrow maintainers,
> >
> > I wanted to report an issue that was previously discussed here:
> > https://github.com/numba/numba/issues/4256.
> >
> > The main issue is that importing pyarrow, and then numba causes some
> weird
> > LLVM error, as follows:
> > $ pip3 install pyarrow numba
> > $ pip3 freeze
> > llvmlite==0.29.0
> > numba==0.45.0
> > numpy==1.17.0
> > pyarrow==0.14.1
> > six==1.12.0
> > $ python3
> > Python 3.7.4 (default, Jul 29 2019, 14:51:28)
> > [Clang 10.0.1 (clang-1001.0.46.4)] on darwin
> > Type "help", "copyright", "credits" or "license" for more information.
> > >>> import pyarrow
> > >>> import numba
> > Assertion failed: (PassInf && "Expected all immutable passes to be
> > initialized"), function addImmutablePass, file
> >
> /Users/buildbot/miniconda3/conda-bld/llvmdev_1556270736866/work/lib/IR/LegacyPassManager.cpp,
> > line 849.
> > Abort trap: 6
> >
> > After some additional digging, I found the first problematic version to
> be
> > pyarrow==0.13.0, because if I install pyarrow==0.12.1 the issue seems to
> > disappear.
> >
> > Thank you!
> > --
> > Regards,
> >
> >
> > Ivan Gozali
> > Lecida
> > Email: ivan@lecida.com
>

Re: pyarrow & numba compatibility issue in macOS

Posted by Wes McKinney <we...@gmail.com>.
hi Ivan,

We have statically linked LLVM symbols in our wheels on all platforms.
We're looking for new maintainers for the wheels to take care of
issues like this, see [1]. In the meantime, I suggest you use conda
packages if you need to use numba and pyarrow together.

Thanks

[1]: https://lists.apache.org/thread.html/128a2bec285ad45aa4189ebb39a15b39dcf6d91c4ab0278ff4f7cdea@%3Cdev.arrow.apache.org%3E

On Mon, Jul 29, 2019 at 5:57 PM Ivan Gozali <iv...@lecida.com> wrote:
>
> Hi Arrow maintainers,
>
> I wanted to report an issue that was previously discussed here:
> https://github.com/numba/numba/issues/4256.
>
> The main issue is that importing pyarrow, and then numba causes some weird
> LLVM error, as follows:
> $ pip3 install pyarrow numba
> $ pip3 freeze
> llvmlite==0.29.0
> numba==0.45.0
> numpy==1.17.0
> pyarrow==0.14.1
> six==1.12.0
> $ python3
> Python 3.7.4 (default, Jul 29 2019, 14:51:28)
> [Clang 10.0.1 (clang-1001.0.46.4)] on darwin
> Type "help", "copyright", "credits" or "license" for more information.
> >>> import pyarrow
> >>> import numba
> Assertion failed: (PassInf && "Expected all immutable passes to be
> initialized"), function addImmutablePass, file
> /Users/buildbot/miniconda3/conda-bld/llvmdev_1556270736866/work/lib/IR/LegacyPassManager.cpp,
> line 849.
> Abort trap: 6
>
> After some additional digging, I found the first problematic version to be
> pyarrow==0.13.0, because if I install pyarrow==0.12.1 the issue seems to
> disappear.
>
> Thank you!
> --
> Regards,
>
>
> Ivan Gozali
> Lecida
> Email: ivan@lecida.com