[GitHub] [arrow] sjperkins commented on pull request #10565: ARROW-638: [C++] Complex Number Support via ExtensionTypes

[GitBox <gi...@apache.org>]

sjperkins commented on pull request #10565:
URL: https://github.com/apache/arrow/pull/10565#issuecomment-885786527


   The Complex Extension Type now seems to be somewhat understood by the Python layer, but seems to be failing because there's no `__arrow_ext_class__` attribute, which seems to be [required ](https://arrow.apache.org/docs/python/extending_types.html )by the Python layer
   
   https://github.com/sjperkins/arrow/runs/3144270259?check_suite_focus=true#step:8:9416
   
   ```traceback
    Traceback (most recent call last):
     File "/opt/conda/envs/arrow/lib/python3.7/multiprocessing/pool.py", line 121, in worker
       result = (True, func(*args, **kwds))
     File "/opt/conda/envs/arrow/lib/python3.7/site-packages/pyarrow/tests/test_pandas.py", line 2533, in _threaded_conversion
       _check_pandas_roundtrip(df, use_threads=True)
     File "/opt/conda/envs/arrow/lib/python3.7/site-packages/pyarrow/tests/test_pandas.py", line 91, in _check_pandas_roundtrip
       nthreads=2 if use_threads else 1)
     File "pyarrow/table.pxi", line 1561, in pyarrow.lib.Table.from_pandas
     File "/opt/conda/envs/arrow/lib/python3.7/site-packages/pyarrow/pandas_compat.py", line 607, in dataframe_to_arrays
       arrays[i] = maybe_fut.result()
     File "/opt/conda/envs/arrow/lib/python3.7/concurrent/futures/_base.py", line 428, in result
       return self.__get_result()
     File "/opt/conda/envs/arrow/lib/python3.7/concurrent/futures/_base.py", line 384, in __get_result
       raise self._exception
     File "/opt/conda/envs/arrow/lib/python3.7/concurrent/futures/thread.py", line 57, in run
       result = self.fn(*self.args, **self.kwargs)
     File "/opt/conda/envs/arrow/lib/python3.7/site-packages/pyarrow/pandas_compat.py", line 575, in convert_column
       result = pa.array(col, type=type_, from_pandas=True, safe=safe)
     File "pyarrow/array.pxi", line 302, in pyarrow.lib.array
     File "pyarrow/array.pxi", line 89, in pyarrow.lib._ndarray_to_array
     File "pyarrow/types.pxi", line 200, in pyarrow.lib.DataType.to_pandas_dtype
     File "pyarrow/array.pxi", line 2402, in pyarrow.lib.get_array_class_from_type
   AttributeError: 'pyarrow.lib.BaseExtensionType' object has no attribute '__arrow_ext_class__'
   ```
   
   I **think** this may be happening because of the following lines
   
   https://github.com/apache/arrow/blob/42e5726b543135f18db1560278e062b9a7941f5c/python/pyarrow/public-api.pxi#L110-L116
   
   which are invoked in this location:
   
   https://github.com/apache/arrow/blob/42e5726b543135f18db1560278e062b9a7941f5c/python/pyarrow/array.pxi#L2400-L2402
   
   So at this stage, it doesn't look like the underlying C++ Extension Type is fully exposed in the Python layer. For example, the `__arrow_ext_class__` should return some Python wrapper of the underlying C++ ExtensionArray.
   
   I'd appreciate some guidance at this point on how to achieve this. /cc @pitrou who looks like he may have worked on this last in https://github.com/apache/arrow/pull/4532
   
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org