You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@arrow.apache.org by Tom Augspurger <to...@gmail.com> on 2020/02/10 18:17:33 UTC
AttributeError importing pyarrow 0.16.0
Hi all,
I'm seeing a strange issue when importing pyarrow on the intake CI. I get an
exception saying
AttributeError: type object 'pyarrow.lib.Message' has no attribute
'__reduce_cython__'
The full traceback is:
______________________________ test_arrow_import
_______________________________
def test_arrow_import():
> import pyarrow
intake/cli/server/tests/test_server.py:32:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
_ _
../../../virtualenv/python3.7.1/lib/python3.7/site-packages/pyarrow/__init__.py:49:
in <module>
from pyarrow.lib import cpu_count, set_cpu_count
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
_ _
> ???
E AttributeError: type object 'pyarrow.lib.Message' has no attribute
'__reduce_cython__'
pyarrow/ipc.pxi:21: AttributeError
_________________ TestServerV1Source.test_read_part_compressed
_________________
I'm unable to reproduce this locally, and was wondering if anyone else has
seen something similar.
Pyarrow was installed using pip / a wheel (
https://travis-ci.org/intake/intake/jobs/648523104#L311).
A common cause of this error message is building with too old of a Cython.
While checking this, I noticed
that some of the files are generated with Cython 0.29.8, while others were
generated with 0.29.14.
I have no idea if this is a problem in general of if it's causing this
specific issue.
```
_hdfs.cpp:1:/* Generated by Cython 0.29.14 */
include/arrow/python/pyarrow_lib.h:20:/* Generated by Cython 0.29.8 */
include/arrow/python/pyarrow_api.h:21:/* Generated by Cython 0.29.8 */
_plasma.cpp:1:/* Generated by Cython 0.29.14 */
_fs.cpp:1:/* Generated by Cython 0.29.14 */
lib_api.h:1:/* Generated by Cython 0.29.14 */
gandiva.cpp:1:/* Generated by Cython 0.29.14 */
_json.cpp:1:/* Generated by Cython 0.29.14 */
_parquet.cpp:1:/* Generated by Cython 0.29.14 */
_csv.cpp:1:/* Generated by Cython 0.29.14 */
_compute.cpp:1:/* Generated by Cython 0.29.14 */
_dataset.cpp:1:/* Generated by Cython 0.29.14 */
_flight.cpp:1:/* Generated by Cython 0.29.14 */
lib.cpp:1:/* Generated by Cython 0.29.14 */
```
See the https://travis-ci.org/intake/intake/jobs/648523104 for the full log.
Thanks for any pointers!
Re: AttributeError importing pyarrow 0.16.0
Posted by Tom Augspurger <to...@gmail.com>.
Thanks for linking to that. The Python there does seems problematic.
Upgrading to
TravisCI's "bionic" image (with Python 3.7.5 instead of 3.7.1) seems to
have fixed it.
Tom
On Mon, Feb 10, 2020 at 1:34 PM Wes McKinney <we...@gmail.com> wrote:
> hi Tom,
>
> Looks like it could be https://bugs.python.org/issue32973, but I'm not
> sure. I wasn't able to reproduce locally. The Python version 3.7.1
> running in CI is also potentially suspicious.
>
> This class of error seems to have a lot of bug reports based on Google
> searches
>
> Message isn't picklable so we should probably fix that regardless
>
> https://issues.apache.org/jira/browse/ARROW-7826
>
> - Wes
>
> On Mon, Feb 10, 2020 at 12:17 PM Tom Augspurger
> <to...@gmail.com> wrote:
> >
> > Hi all,
> >
> > I'm seeing a strange issue when importing pyarrow on the intake CI. I
> get an
> > exception saying
> >
> > AttributeError: type object 'pyarrow.lib.Message' has no attribute
> > '__reduce_cython__'
> >
> > The full traceback is:
> >
> > ______________________________ test_arrow_import
> > _______________________________
> >
> > def test_arrow_import():
> >
> > > import pyarrow
> >
> > intake/cli/server/tests/test_server.py:32:
> >
> > _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
> _ _
> > _ _
> >
> >
> ../../../virtualenv/python3.7.1/lib/python3.7/site-packages/pyarrow/__init__.py:49:
> > in <module>
> >
> > from pyarrow.lib import cpu_count, set_cpu_count
> >
> > _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
> _ _
> > _ _
> >
> > > ???
> >
> > E AttributeError: type object 'pyarrow.lib.Message' has no attribute
> > '__reduce_cython__'
> >
> > pyarrow/ipc.pxi:21: AttributeError
> >
> > _________________ TestServerV1Source.test_read_part_compressed
> > _________________
> >
> >
> > I'm unable to reproduce this locally, and was wondering if anyone else
> has
> > seen something similar.
> > Pyarrow was installed using pip / a wheel (
> > https://travis-ci.org/intake/intake/jobs/648523104#L311).
> >
> > A common cause of this error message is building with too old of a
> Cython.
> > While checking this, I noticed
> > that some of the files are generated with Cython 0.29.8, while others
> were
> > generated with 0.29.14.
> > I have no idea if this is a problem in general of if it's causing this
> > specific issue.
> >
> > ```
> > _hdfs.cpp:1:/* Generated by Cython 0.29.14 */
> > include/arrow/python/pyarrow_lib.h:20:/* Generated by Cython 0.29.8 */
> > include/arrow/python/pyarrow_api.h:21:/* Generated by Cython 0.29.8 */
> > _plasma.cpp:1:/* Generated by Cython 0.29.14 */
> > _fs.cpp:1:/* Generated by Cython 0.29.14 */
> > lib_api.h:1:/* Generated by Cython 0.29.14 */
> > gandiva.cpp:1:/* Generated by Cython 0.29.14 */
> > _json.cpp:1:/* Generated by Cython 0.29.14 */
> > _parquet.cpp:1:/* Generated by Cython 0.29.14 */
> > _csv.cpp:1:/* Generated by Cython 0.29.14 */
> > _compute.cpp:1:/* Generated by Cython 0.29.14 */
> > _dataset.cpp:1:/* Generated by Cython 0.29.14 */
> > _flight.cpp:1:/* Generated by Cython 0.29.14 */
> > lib.cpp:1:/* Generated by Cython 0.29.14 */
> > ```
> >
> > See the https://travis-ci.org/intake/intake/jobs/648523104 for the full
> log.
> >
> >
> > Thanks for any pointers!
>
Re: AttributeError importing pyarrow 0.16.0
Posted by Wes McKinney <we...@gmail.com>.
hi Tom,
Looks like it could be https://bugs.python.org/issue32973, but I'm not
sure. I wasn't able to reproduce locally. The Python version 3.7.1
running in CI is also potentially suspicious.
This class of error seems to have a lot of bug reports based on Google searches
Message isn't picklable so we should probably fix that regardless
https://issues.apache.org/jira/browse/ARROW-7826
- Wes
On Mon, Feb 10, 2020 at 12:17 PM Tom Augspurger
<to...@gmail.com> wrote:
>
> Hi all,
>
> I'm seeing a strange issue when importing pyarrow on the intake CI. I get an
> exception saying
>
> AttributeError: type object 'pyarrow.lib.Message' has no attribute
> '__reduce_cython__'
>
> The full traceback is:
>
> ______________________________ test_arrow_import
> _______________________________
>
> def test_arrow_import():
>
> > import pyarrow
>
> intake/cli/server/tests/test_server.py:32:
>
> _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
> _ _
>
> ../../../virtualenv/python3.7.1/lib/python3.7/site-packages/pyarrow/__init__.py:49:
> in <module>
>
> from pyarrow.lib import cpu_count, set_cpu_count
>
> _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
> _ _
>
> > ???
>
> E AttributeError: type object 'pyarrow.lib.Message' has no attribute
> '__reduce_cython__'
>
> pyarrow/ipc.pxi:21: AttributeError
>
> _________________ TestServerV1Source.test_read_part_compressed
> _________________
>
>
> I'm unable to reproduce this locally, and was wondering if anyone else has
> seen something similar.
> Pyarrow was installed using pip / a wheel (
> https://travis-ci.org/intake/intake/jobs/648523104#L311).
>
> A common cause of this error message is building with too old of a Cython.
> While checking this, I noticed
> that some of the files are generated with Cython 0.29.8, while others were
> generated with 0.29.14.
> I have no idea if this is a problem in general of if it's causing this
> specific issue.
>
> ```
> _hdfs.cpp:1:/* Generated by Cython 0.29.14 */
> include/arrow/python/pyarrow_lib.h:20:/* Generated by Cython 0.29.8 */
> include/arrow/python/pyarrow_api.h:21:/* Generated by Cython 0.29.8 */
> _plasma.cpp:1:/* Generated by Cython 0.29.14 */
> _fs.cpp:1:/* Generated by Cython 0.29.14 */
> lib_api.h:1:/* Generated by Cython 0.29.14 */
> gandiva.cpp:1:/* Generated by Cython 0.29.14 */
> _json.cpp:1:/* Generated by Cython 0.29.14 */
> _parquet.cpp:1:/* Generated by Cython 0.29.14 */
> _csv.cpp:1:/* Generated by Cython 0.29.14 */
> _compute.cpp:1:/* Generated by Cython 0.29.14 */
> _dataset.cpp:1:/* Generated by Cython 0.29.14 */
> _flight.cpp:1:/* Generated by Cython 0.29.14 */
> lib.cpp:1:/* Generated by Cython 0.29.14 */
> ```
>
> See the https://travis-ci.org/intake/intake/jobs/648523104 for the full log.
>
>
> Thanks for any pointers!