You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@arrow.apache.org by filippo medri <fi...@gmail.com> on 2020/02/26 22:50:14 UTC
[pyarrow] Table.from_pandas fails with TypeError: infer_dtype() takes
no keyword arguments
Hi,
trying the example here https://arrow.apache.org/docs/python/pandas.html on
DataFrames paragraph with python 2.7.17, numpy==1.16.2 , pandas ==0.20.3,
pyarrow==0.16.0 or pyarrow==0.15.1 got an error (see below).
import pandas as pd
import pyarrow as pa
df = pd.DataFrame({"a": [1,2,3]})
table = pa.Table.from_pandas(df)
fails with the following stacktrace:
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "pyarrow/table.pxi", line 1177, in pyarrow.lib.Table.from_pandas
File
"/home/ubuntu/.pyenv/versions/2.7.17/envs/py27/lib/python2.7/site-packages/pyarrow/pandas_compat.py",
line 593, in dataframe_to_arrays
types)
File
"/home/ubuntu/.pyenv/versions/2.7.17/envs/py27/lib/python2.7/site-packages/pyarrow/pandas_compat.py",
line 234, in construct_metadata
metadata = _get_simple_index_descriptor(level, name)
File
"/home/ubuntu/.pyenv/versions/2.7.17/envs/py27/lib/python2.7/site-packages/pyarrow/pandas_compat.py",
line 255, in _get_simple_index_descriptor
pandas_type = get_logical_type_from_numpy(level)
File
"/home/ubuntu/.pyenv/versions/2.7.17/envs/py27/lib/python2.7/site-packages/pyarrow/pandas_compat.py",
line 113, in get_logical_type_from_numpy
result = _pandas_api.infer_dtype(pandas_collection)
File "pyarrow/pandas-shim.pxi", line 131, in
pyarrow.lib._PandasAPIShim.infer_dtype
File "pyarrow/pandas-shim.pxi", line 134, in
pyarrow.lib._PandasAPIShim.infer_dtype
TypeError: infer_dtype() takes no keyword arguments
Is some known bug, some incompatibility or some mis-configuration?
Thank you very much,
Filippo Medri
Re: [pyarrow] Table.from_pandas fails with TypeError: infer_dtype()
takes no keyword arguments
Posted by filippo medri <fi...@gmail.com>.
That makes sense to me.
Thank you very much.
Filippo Medri
Il giorno mer 26 feb 2020 alle ore 14:57 Wes McKinney <we...@gmail.com>
ha scritto:
> hi Filippo,
>
> The pandas version is too old to use the pyarrow-pandas functionality
> (0.20.3 was released in July 2017) -- note however that pandas isn't
> required to use pyarrow. We wouldn't necessarily be opposed to
> supporting older versions of pandas so long as there were volunteers
> willing to do the support / maintenance and testing.
>
> I just opened https://issues.apache.org/jira/browse/ARROW-7950 about
> providing more user feedback when the installed pandas version is too
> old
>
> - Wes
>
> On Wed, Feb 26, 2020 at 4:50 PM filippo medri <fi...@gmail.com>
> wrote:
> >
> > Hi,
> > trying the example here https://arrow.apache.org/docs/python/pandas.html
> on DataFrames paragraph with python 2.7.17, numpy==1.16.2 , pandas
> ==0.20.3, pyarrow==0.16.0 or pyarrow==0.15.1 got an error (see below).
> >
> > import pandas as pd
> > import pyarrow as pa
> >
> > df = pd.DataFrame({"a": [1,2,3]})
> > table = pa.Table.from_pandas(df)
> >
> > fails with the following stacktrace:
> >
> > Traceback (most recent call last):
> > File "<stdin>", line 1, in <module>
> > File "pyarrow/table.pxi", line 1177, in pyarrow.lib.Table.from_pandas
> > File
> "/home/ubuntu/.pyenv/versions/2.7.17/envs/py27/lib/python2.7/site-packages/pyarrow/pandas_compat.py",
> line 593, in dataframe_to_arrays
> > types)
> > File
> "/home/ubuntu/.pyenv/versions/2.7.17/envs/py27/lib/python2.7/site-packages/pyarrow/pandas_compat.py",
> line 234, in construct_metadata
> > metadata = _get_simple_index_descriptor(level, name)
> > File
> "/home/ubuntu/.pyenv/versions/2.7.17/envs/py27/lib/python2.7/site-packages/pyarrow/pandas_compat.py",
> line 255, in _get_simple_index_descriptor
> > pandas_type = get_logical_type_from_numpy(level)
> > File
> "/home/ubuntu/.pyenv/versions/2.7.17/envs/py27/lib/python2.7/site-packages/pyarrow/pandas_compat.py",
> line 113, in get_logical_type_from_numpy
> > result = _pandas_api.infer_dtype(pandas_collection)
> > File "pyarrow/pandas-shim.pxi", line 131, in
> pyarrow.lib._PandasAPIShim.infer_dtype
> > File "pyarrow/pandas-shim.pxi", line 134, in
> pyarrow.lib._PandasAPIShim.infer_dtype
> > TypeError: infer_dtype() takes no keyword arguments
> >
> > Is some known bug, some incompatibility or some mis-configuration?
> > Thank you very much,
> > Filippo Medri
>
Re: [pyarrow] Table.from_pandas fails with TypeError: infer_dtype()
takes no keyword arguments
Posted by Wes McKinney <we...@gmail.com>.
hi Filippo,
The pandas version is too old to use the pyarrow-pandas functionality
(0.20.3 was released in July 2017) -- note however that pandas isn't
required to use pyarrow. We wouldn't necessarily be opposed to
supporting older versions of pandas so long as there were volunteers
willing to do the support / maintenance and testing.
I just opened https://issues.apache.org/jira/browse/ARROW-7950 about
providing more user feedback when the installed pandas version is too
old
- Wes
On Wed, Feb 26, 2020 at 4:50 PM filippo medri <fi...@gmail.com> wrote:
>
> Hi,
> trying the example here https://arrow.apache.org/docs/python/pandas.html on DataFrames paragraph with python 2.7.17, numpy==1.16.2 , pandas ==0.20.3, pyarrow==0.16.0 or pyarrow==0.15.1 got an error (see below).
>
> import pandas as pd
> import pyarrow as pa
>
> df = pd.DataFrame({"a": [1,2,3]})
> table = pa.Table.from_pandas(df)
>
> fails with the following stacktrace:
>
> Traceback (most recent call last):
> File "<stdin>", line 1, in <module>
> File "pyarrow/table.pxi", line 1177, in pyarrow.lib.Table.from_pandas
> File "/home/ubuntu/.pyenv/versions/2.7.17/envs/py27/lib/python2.7/site-packages/pyarrow/pandas_compat.py", line 593, in dataframe_to_arrays
> types)
> File "/home/ubuntu/.pyenv/versions/2.7.17/envs/py27/lib/python2.7/site-packages/pyarrow/pandas_compat.py", line 234, in construct_metadata
> metadata = _get_simple_index_descriptor(level, name)
> File "/home/ubuntu/.pyenv/versions/2.7.17/envs/py27/lib/python2.7/site-packages/pyarrow/pandas_compat.py", line 255, in _get_simple_index_descriptor
> pandas_type = get_logical_type_from_numpy(level)
> File "/home/ubuntu/.pyenv/versions/2.7.17/envs/py27/lib/python2.7/site-packages/pyarrow/pandas_compat.py", line 113, in get_logical_type_from_numpy
> result = _pandas_api.infer_dtype(pandas_collection)
> File "pyarrow/pandas-shim.pxi", line 131, in pyarrow.lib._PandasAPIShim.infer_dtype
> File "pyarrow/pandas-shim.pxi", line 134, in pyarrow.lib._PandasAPIShim.infer_dtype
> TypeError: infer_dtype() takes no keyword arguments
>
> Is some known bug, some incompatibility or some mis-configuration?
> Thank you very much,
> Filippo Medri