You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@arrow.apache.org by GitBox <gi...@apache.org> on 2021/07/23 17:51:09 UTC
[GitHub] [arrow] sjperkins edited a comment on pull request #10565: ARROW-638: [C++] Complex Number Support via ExtensionTypes
sjperkins edited a comment on pull request #10565:
URL: https://github.com/apache/arrow/pull/10565#issuecomment-885786527
The Complex Extension Type now seems to be somewhat understood by the Python layer, but seems to be failing because there's no `__arrow_ext_class__` attribute, which seems to be [required ](https://arrow.apache.org/docs/python/extending_types.html )by the Python layer
https://github.com/sjperkins/arrow/runs/3144270259?check_suite_focus=true#step:8:9416
```traceback
Traceback (most recent call last):
File "/opt/conda/envs/arrow/lib/python3.7/multiprocessing/pool.py", line 121, in worker
result = (True, func(*args, **kwds))
File "/opt/conda/envs/arrow/lib/python3.7/site-packages/pyarrow/tests/test_pandas.py", line 2533, in _threaded_conversion
_check_pandas_roundtrip(df, use_threads=True)
File "/opt/conda/envs/arrow/lib/python3.7/site-packages/pyarrow/tests/test_pandas.py", line 91, in _check_pandas_roundtrip
nthreads=2 if use_threads else 1)
File "pyarrow/table.pxi", line 1561, in pyarrow.lib.Table.from_pandas
File "/opt/conda/envs/arrow/lib/python3.7/site-packages/pyarrow/pandas_compat.py", line 607, in dataframe_to_arrays
arrays[i] = maybe_fut.result()
File "/opt/conda/envs/arrow/lib/python3.7/concurrent/futures/_base.py", line 428, in result
return self.__get_result()
File "/opt/conda/envs/arrow/lib/python3.7/concurrent/futures/_base.py", line 384, in __get_result
raise self._exception
File "/opt/conda/envs/arrow/lib/python3.7/concurrent/futures/thread.py", line 57, in run
result = self.fn(*self.args, **self.kwargs)
File "/opt/conda/envs/arrow/lib/python3.7/site-packages/pyarrow/pandas_compat.py", line 575, in convert_column
result = pa.array(col, type=type_, from_pandas=True, safe=safe)
File "pyarrow/array.pxi", line 302, in pyarrow.lib.array
File "pyarrow/array.pxi", line 89, in pyarrow.lib._ndarray_to_array
File "pyarrow/types.pxi", line 200, in pyarrow.lib.DataType.to_pandas_dtype
File "pyarrow/array.pxi", line 2402, in pyarrow.lib.get_array_class_from_type
AttributeError: 'pyarrow.lib.BaseExtensionType' object has no attribute '__arrow_ext_class__'
```
I **think** this may be happening because of the following lines
https://github.com/apache/arrow/blob/42e5726b543135f18db1560278e062b9a7941f5c/python/pyarrow/public-api.pxi#L110-L116
which are invoked in this location:
https://github.com/apache/arrow/blob/42e5726b543135f18db1560278e062b9a7941f5c/python/pyarrow/array.pxi#L2400-L2402
So at this stage, it doesn't look like the underlying C++ Extension Type is fully exposed in the Python layer. For example, `__arrow_ext_class__` should probably return some Python wrapper of the underlying C++ ExtensionArray.
I'd appreciate some guidance at this point on how to achieve this. /cc @pitrou who looks like he may have worked on this last in https://github.com/apache/arrow/pull/4532
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org