You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@arrow.apache.org by Quang Vu <vh...@gmail.com> on 2018/05/14 16:34:43 UTC
Symbol not found: _PyCObject_Type (MacOS El Capitan, Python 3.6)
Hi Arrow dev,
I am having trouble with installing and setting my development environment
for Arrow. I wonder if anyone is familiar with the issue. My system info:
- MacOS 10.11.6 (El Capitan)
- conda 4.5.1
- python 3.6.5
- arrow's current commit: 4b8511
Installing Arrow C++ libraries and Pacquet are both successful. But
importing `pyarrow` fail:
$ python -c 'import pyarrow'
Traceback (most recent call last):
File "<string>", line 1, in <module>
File "/Users/myuser/code/arrow/python/pyarrow/__init__.py", line 47, in
<module>
from pyarrow.lib import cpu_count, set_cpu_count
ImportError: dlopen(/Users/myuser/code/arrow/python/pyarrow/
lib.cpython-36m-darwin.so, 2): Symbol not found: _PyCObject_Type
Referenced from:
/Users/myuser/miniconda3/envs/pyarrow-test/lib/libarrow_python.10.dylib
Expected in: flat namespace
in /Users/myuser/miniconda3/envs/pyarrow-test/lib/libarrow_python.10.dylib
If anyone have suggestion on what the problem is about, please let me know.
Thanks!
Re: Symbol not found: _PyCObject_Type (MacOS El Capitan, Python 3.6)
Posted by Quang Vu <vh...@gmail.com>.
Hi Wes,
Thank you for your suggestion. Clearing out CMake temporary files, and
rebuilding them again, helps fixing the issue. I am so glad I have a
working dev environment now. Thanks again!
On Tue, May 15, 2018 at 9:45 PM Wes McKinney <we...@gmail.com> wrote:
> hi Quang -- I recommend clearing out your CMake temporary files after
> making any conda environment changes. If you activate a different
> conda environment, CMake will not know to recompute variables related
> to Python's header files and libraries. So it might have been that you
> invoked CMake with Python 2 activated and later activated Python 3
>
> - Wes
>
> On Tue, May 15, 2018 at 5:15 AM, Quang Vu <vh...@gmail.com> wrote:
> > Yes Antoine, that happens when compiling Arrow under an activated conda
> > environment.
> > Thank you for all the info you are helping me with!
> >
> > Quang.
> >
> > On Mon, May 14, 2018 at 3:34 PM Antoine Pitrou <an...@python.org>
> wrote:
> >
> >>
> >> To give a bit more insight: you should compile Arrow with your conda
> >> environment activated, so that it picks the right Python version (3.6.5,
> >> in your case). If it's still picking the wrong Python version, that
> >> might be a bug.
> >>
> >> Regards
> >>
> >> Antoine.
> >>
> >>
> >> Le 14/05/2018 à 20:50, Quang Vu a écrit :
> >> > Thanks Antoine,
> >> >
> >> > I will need to learn more about the compiling process that happens on
> my
> >> > Mac, to see how that link to Python 2.
> >> > I am not familiar with that process. But this is a good pointer for
> my
> >> > issue. Thank you for your response to my issue!
> >> >
> >> > Quang.
> >> >
> >> > On Mon, May 14, 2018 at 12:50 PM Antoine Pitrou <an...@python.org>
> >> wrote:
> >> >
> >> >>
> >> >> Hi Quang,
> >> >>
> >> >> It sounds like you have compiled Arrow against a Python 2 install but
> >> >> are now trying to use it with Python 3. This won't work, the same
> >> >> Python version must be used when compiling and when using PyArrow.
> >> >>
> >> >> ("PyCObject" is a Python 2-specific API that doesn't exist anymore in
> >> >> Python 3)
> >> >>
> >> >> Regards
> >> >>
> >> >> Antoine.
> >> >>
> >> >>
> >> >> Le 14/05/2018 à 18:34, Quang Vu a écrit :
> >> >>> Hi Arrow dev,
> >> >>>
> >> >>> I am having trouble with installing and setting my development
> >> >> environment
> >> >>> for Arrow. I wonder if anyone is familiar with the issue. My system
> >> info:
> >> >>> - MacOS 10.11.6 (El Capitan)
> >> >>> - conda 4.5.1
> >> >>> - python 3.6.5
> >> >>> - arrow's current commit: 4b8511
> >> >>>
> >> >>> Installing Arrow C++ libraries and Pacquet are both successful. But
> >> >>> importing `pyarrow` fail:
> >> >>>
> >> >>> $ python -c 'import pyarrow'
> >> >>>
> >> >>> Traceback (most recent call last):
> >> >>> File "<string>", line 1, in <module>
> >> >>> File "/Users/myuser/code/arrow/python/pyarrow/__init__.py", line
> 47,
> >> in
> >> >>> <module>
> >> >>> from pyarrow.lib import cpu_count, set_cpu_count
> >> >>> ImportError: dlopen(/Users/myuser/code/arrow/python/pyarrow/
> >> >>> lib.cpython-36m-darwin.so, 2): Symbol not found: _PyCObject_Type
> >> >>> Referenced from:
> >> >>>
> /Users/myuser/miniconda3/envs/pyarrow-test/lib/libarrow_python.10.dylib
> >> >>> Expected in: flat namespace
> >> >>> in
> >> >>
> /Users/myuser/miniconda3/envs/pyarrow-test/lib/libarrow_python.10.dylib
> >> >>>
> >> >>> If anyone have suggestion on what the problem is about, please let
> me
> >> >> know.
> >> >>> Thanks!
> >> >>>
> >> >>
> >> >
> >>
>
Re: Symbol not found: _PyCObject_Type (MacOS El Capitan, Python 3.6)
Posted by Wes McKinney <we...@gmail.com>.
hi Quang -- I recommend clearing out your CMake temporary files after
making any conda environment changes. If you activate a different
conda environment, CMake will not know to recompute variables related
to Python's header files and libraries. So it might have been that you
invoked CMake with Python 2 activated and later activated Python 3
- Wes
On Tue, May 15, 2018 at 5:15 AM, Quang Vu <vh...@gmail.com> wrote:
> Yes Antoine, that happens when compiling Arrow under an activated conda
> environment.
> Thank you for all the info you are helping me with!
>
> Quang.
>
> On Mon, May 14, 2018 at 3:34 PM Antoine Pitrou <an...@python.org> wrote:
>
>>
>> To give a bit more insight: you should compile Arrow with your conda
>> environment activated, so that it picks the right Python version (3.6.5,
>> in your case). If it's still picking the wrong Python version, that
>> might be a bug.
>>
>> Regards
>>
>> Antoine.
>>
>>
>> Le 14/05/2018 à 20:50, Quang Vu a écrit :
>> > Thanks Antoine,
>> >
>> > I will need to learn more about the compiling process that happens on my
>> > Mac, to see how that link to Python 2.
>> > I am not familiar with that process. But this is a good pointer for my
>> > issue. Thank you for your response to my issue!
>> >
>> > Quang.
>> >
>> > On Mon, May 14, 2018 at 12:50 PM Antoine Pitrou <an...@python.org>
>> wrote:
>> >
>> >>
>> >> Hi Quang,
>> >>
>> >> It sounds like you have compiled Arrow against a Python 2 install but
>> >> are now trying to use it with Python 3. This won't work, the same
>> >> Python version must be used when compiling and when using PyArrow.
>> >>
>> >> ("PyCObject" is a Python 2-specific API that doesn't exist anymore in
>> >> Python 3)
>> >>
>> >> Regards
>> >>
>> >> Antoine.
>> >>
>> >>
>> >> Le 14/05/2018 à 18:34, Quang Vu a écrit :
>> >>> Hi Arrow dev,
>> >>>
>> >>> I am having trouble with installing and setting my development
>> >> environment
>> >>> for Arrow. I wonder if anyone is familiar with the issue. My system
>> info:
>> >>> - MacOS 10.11.6 (El Capitan)
>> >>> - conda 4.5.1
>> >>> - python 3.6.5
>> >>> - arrow's current commit: 4b8511
>> >>>
>> >>> Installing Arrow C++ libraries and Pacquet are both successful. But
>> >>> importing `pyarrow` fail:
>> >>>
>> >>> $ python -c 'import pyarrow'
>> >>>
>> >>> Traceback (most recent call last):
>> >>> File "<string>", line 1, in <module>
>> >>> File "/Users/myuser/code/arrow/python/pyarrow/__init__.py", line 47,
>> in
>> >>> <module>
>> >>> from pyarrow.lib import cpu_count, set_cpu_count
>> >>> ImportError: dlopen(/Users/myuser/code/arrow/python/pyarrow/
>> >>> lib.cpython-36m-darwin.so, 2): Symbol not found: _PyCObject_Type
>> >>> Referenced from:
>> >>> /Users/myuser/miniconda3/envs/pyarrow-test/lib/libarrow_python.10.dylib
>> >>> Expected in: flat namespace
>> >>> in
>> >> /Users/myuser/miniconda3/envs/pyarrow-test/lib/libarrow_python.10.dylib
>> >>>
>> >>> If anyone have suggestion on what the problem is about, please let me
>> >> know.
>> >>> Thanks!
>> >>>
>> >>
>> >
>>
Re: Symbol not found: _PyCObject_Type (MacOS El Capitan, Python 3.6)
Posted by Quang Vu <vh...@gmail.com>.
Yes Antoine, that happens when compiling Arrow under an activated conda
environment.
Thank you for all the info you are helping me with!
Quang.
On Mon, May 14, 2018 at 3:34 PM Antoine Pitrou <an...@python.org> wrote:
>
> To give a bit more insight: you should compile Arrow with your conda
> environment activated, so that it picks the right Python version (3.6.5,
> in your case). If it's still picking the wrong Python version, that
> might be a bug.
>
> Regards
>
> Antoine.
>
>
> Le 14/05/2018 à 20:50, Quang Vu a écrit :
> > Thanks Antoine,
> >
> > I will need to learn more about the compiling process that happens on my
> > Mac, to see how that link to Python 2.
> > I am not familiar with that process. But this is a good pointer for my
> > issue. Thank you for your response to my issue!
> >
> > Quang.
> >
> > On Mon, May 14, 2018 at 12:50 PM Antoine Pitrou <an...@python.org>
> wrote:
> >
> >>
> >> Hi Quang,
> >>
> >> It sounds like you have compiled Arrow against a Python 2 install but
> >> are now trying to use it with Python 3. This won't work, the same
> >> Python version must be used when compiling and when using PyArrow.
> >>
> >> ("PyCObject" is a Python 2-specific API that doesn't exist anymore in
> >> Python 3)
> >>
> >> Regards
> >>
> >> Antoine.
> >>
> >>
> >> Le 14/05/2018 à 18:34, Quang Vu a écrit :
> >>> Hi Arrow dev,
> >>>
> >>> I am having trouble with installing and setting my development
> >> environment
> >>> for Arrow. I wonder if anyone is familiar with the issue. My system
> info:
> >>> - MacOS 10.11.6 (El Capitan)
> >>> - conda 4.5.1
> >>> - python 3.6.5
> >>> - arrow's current commit: 4b8511
> >>>
> >>> Installing Arrow C++ libraries and Pacquet are both successful. But
> >>> importing `pyarrow` fail:
> >>>
> >>> $ python -c 'import pyarrow'
> >>>
> >>> Traceback (most recent call last):
> >>> File "<string>", line 1, in <module>
> >>> File "/Users/myuser/code/arrow/python/pyarrow/__init__.py", line 47,
> in
> >>> <module>
> >>> from pyarrow.lib import cpu_count, set_cpu_count
> >>> ImportError: dlopen(/Users/myuser/code/arrow/python/pyarrow/
> >>> lib.cpython-36m-darwin.so, 2): Symbol not found: _PyCObject_Type
> >>> Referenced from:
> >>> /Users/myuser/miniconda3/envs/pyarrow-test/lib/libarrow_python.10.dylib
> >>> Expected in: flat namespace
> >>> in
> >> /Users/myuser/miniconda3/envs/pyarrow-test/lib/libarrow_python.10.dylib
> >>>
> >>> If anyone have suggestion on what the problem is about, please let me
> >> know.
> >>> Thanks!
> >>>
> >>
> >
>
Re: Symbol not found: _PyCObject_Type (MacOS El Capitan, Python 3.6)
Posted by Antoine Pitrou <an...@python.org>.
To give a bit more insight: you should compile Arrow with your conda
environment activated, so that it picks the right Python version (3.6.5,
in your case). If it's still picking the wrong Python version, that
might be a bug.
Regards
Antoine.
Le 14/05/2018 à 20:50, Quang Vu a écrit :
> Thanks Antoine,
>
> I will need to learn more about the compiling process that happens on my
> Mac, to see how that link to Python 2.
> I am not familiar with that process. But this is a good pointer for my
> issue. Thank you for your response to my issue!
>
> Quang.
>
> On Mon, May 14, 2018 at 12:50 PM Antoine Pitrou <an...@python.org> wrote:
>
>>
>> Hi Quang,
>>
>> It sounds like you have compiled Arrow against a Python 2 install but
>> are now trying to use it with Python 3. This won't work, the same
>> Python version must be used when compiling and when using PyArrow.
>>
>> ("PyCObject" is a Python 2-specific API that doesn't exist anymore in
>> Python 3)
>>
>> Regards
>>
>> Antoine.
>>
>>
>> Le 14/05/2018 à 18:34, Quang Vu a écrit :
>>> Hi Arrow dev,
>>>
>>> I am having trouble with installing and setting my development
>> environment
>>> for Arrow. I wonder if anyone is familiar with the issue. My system info:
>>> - MacOS 10.11.6 (El Capitan)
>>> - conda 4.5.1
>>> - python 3.6.5
>>> - arrow's current commit: 4b8511
>>>
>>> Installing Arrow C++ libraries and Pacquet are both successful. But
>>> importing `pyarrow` fail:
>>>
>>> $ python -c 'import pyarrow'
>>>
>>> Traceback (most recent call last):
>>> File "<string>", line 1, in <module>
>>> File "/Users/myuser/code/arrow/python/pyarrow/__init__.py", line 47, in
>>> <module>
>>> from pyarrow.lib import cpu_count, set_cpu_count
>>> ImportError: dlopen(/Users/myuser/code/arrow/python/pyarrow/
>>> lib.cpython-36m-darwin.so, 2): Symbol not found: _PyCObject_Type
>>> Referenced from:
>>> /Users/myuser/miniconda3/envs/pyarrow-test/lib/libarrow_python.10.dylib
>>> Expected in: flat namespace
>>> in
>> /Users/myuser/miniconda3/envs/pyarrow-test/lib/libarrow_python.10.dylib
>>>
>>> If anyone have suggestion on what the problem is about, please let me
>> know.
>>> Thanks!
>>>
>>
>
Re: Symbol not found: _PyCObject_Type (MacOS El Capitan, Python 3.6)
Posted by Quang Vu <vh...@gmail.com>.
Thanks Antoine,
I will need to learn more about the compiling process that happens on my
Mac, to see how that link to Python 2.
I am not familiar with that process. But this is a good pointer for my
issue. Thank you for your response to my issue!
Quang.
On Mon, May 14, 2018 at 12:50 PM Antoine Pitrou <an...@python.org> wrote:
>
> Hi Quang,
>
> It sounds like you have compiled Arrow against a Python 2 install but
> are now trying to use it with Python 3. This won't work, the same
> Python version must be used when compiling and when using PyArrow.
>
> ("PyCObject" is a Python 2-specific API that doesn't exist anymore in
> Python 3)
>
> Regards
>
> Antoine.
>
>
> Le 14/05/2018 à 18:34, Quang Vu a écrit :
> > Hi Arrow dev,
> >
> > I am having trouble with installing and setting my development
> environment
> > for Arrow. I wonder if anyone is familiar with the issue. My system info:
> > - MacOS 10.11.6 (El Capitan)
> > - conda 4.5.1
> > - python 3.6.5
> > - arrow's current commit: 4b8511
> >
> > Installing Arrow C++ libraries and Pacquet are both successful. But
> > importing `pyarrow` fail:
> >
> > $ python -c 'import pyarrow'
> >
> > Traceback (most recent call last):
> > File "<string>", line 1, in <module>
> > File "/Users/myuser/code/arrow/python/pyarrow/__init__.py", line 47, in
> > <module>
> > from pyarrow.lib import cpu_count, set_cpu_count
> > ImportError: dlopen(/Users/myuser/code/arrow/python/pyarrow/
> > lib.cpython-36m-darwin.so, 2): Symbol not found: _PyCObject_Type
> > Referenced from:
> > /Users/myuser/miniconda3/envs/pyarrow-test/lib/libarrow_python.10.dylib
> > Expected in: flat namespace
> > in
> /Users/myuser/miniconda3/envs/pyarrow-test/lib/libarrow_python.10.dylib
> >
> > If anyone have suggestion on what the problem is about, please let me
> know.
> > Thanks!
> >
>
Re: Symbol not found: _PyCObject_Type (MacOS El Capitan, Python 3.6)
Posted by Antoine Pitrou <an...@python.org>.
Hi Quang,
It sounds like you have compiled Arrow against a Python 2 install but
are now trying to use it with Python 3. This won't work, the same
Python version must be used when compiling and when using PyArrow.
("PyCObject" is a Python 2-specific API that doesn't exist anymore in
Python 3)
Regards
Antoine.
Le 14/05/2018 à 18:34, Quang Vu a écrit :
> Hi Arrow dev,
>
> I am having trouble with installing and setting my development environment
> for Arrow. I wonder if anyone is familiar with the issue. My system info:
> - MacOS 10.11.6 (El Capitan)
> - conda 4.5.1
> - python 3.6.5
> - arrow's current commit: 4b8511
>
> Installing Arrow C++ libraries and Pacquet are both successful. But
> importing `pyarrow` fail:
>
> $ python -c 'import pyarrow'
>
> Traceback (most recent call last):
> File "<string>", line 1, in <module>
> File "/Users/myuser/code/arrow/python/pyarrow/__init__.py", line 47, in
> <module>
> from pyarrow.lib import cpu_count, set_cpu_count
> ImportError: dlopen(/Users/myuser/code/arrow/python/pyarrow/
> lib.cpython-36m-darwin.so, 2): Symbol not found: _PyCObject_Type
> Referenced from:
> /Users/myuser/miniconda3/envs/pyarrow-test/lib/libarrow_python.10.dylib
> Expected in: flat namespace
> in /Users/myuser/miniconda3/envs/pyarrow-test/lib/libarrow_python.10.dylib
>
> If anyone have suggestion on what the problem is about, please let me know.
> Thanks!
>