You are viewing a plain text version of this content. The canonical link for it is here.
Posted to jira@arrow.apache.org by "Joris Van den Bossche (Jira)" <ji...@apache.org> on 2021/11/25 16:36:00 UTC
[jira] [Comment Edited] (ARROW-14378) [R] Make custom extension classes for (some) cols with row-level metadata
[ https://issues.apache.org/jira/browse/ARROW-14378?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17449256#comment-17449256 ]
Joris Van den Bossche edited comment on ARROW-14378 at 11/25/21, 4:35 PM:
--------------------------------------------------------------------------
For the Interval example, there are some different parts living in different places:
* The Arrow Extension Type definition for pandas' interval type (what Dewey linked above): https://github.com/pandas-dev/pandas/blob/fc6b441ba527ca32b460ae4f4f5a6802335497f9/pandas/core/arrays/_arrow_utils.py#L88-L136
* The {{\_\_from_arrow\_\_}} implementation (doing the conversion to arrow): https://github.com/pandas-dev/pandas/blob/fc6b441ba527ca32b460ae4f4f5a6802335497f9/pandas/core/arrays/interval.py#L1405-L1455
* The {{\_\_from_arrow\_\_}} implementation (conversion arrow -> pandas): https://github.com/pandas-dev/pandas/blob/fc6b441ba527ca32b460ae4f4f5a6802335497f9/pandas/core/dtypes/dtypes.py#L1227-L1255
(in theory, if designed differently in the past, those could also all live on the extension type, + a way to get the equivalent extension type for a certain R object)
was (Author: jorisvandenbossche):
For the Interval example, there are some different parts living in different places:
* The Arrow Extension Type definition (what Dewey linked above): https://github.com/pandas-dev/pandas/blob/fc6b441ba527ca32b460ae4f4f5a6802335497f9/pandas/core/arrays/_arrow_utils.py#L88-L136
* The {{\_\_from_arrow\_\_}} implementation (doing the conversion to arrow): https://github.com/pandas-dev/pandas/blob/fc6b441ba527ca32b460ae4f4f5a6802335497f9/pandas/core/arrays/interval.py#L1405-L1455
* The {{\_\_from_arrow\_\_}} implementation (conversion arrow -> pandas): https://github.com/pandas-dev/pandas/blob/fc6b441ba527ca32b460ae4f4f5a6802335497f9/pandas/core/dtypes/dtypes.py#L1227-L1255
(in theory, if designed differently in the past, those could also all live on the extension type, + a way to get the equivalent extension type for a certain R object)
> [R] Make custom extension classes for (some) cols with row-level metadata
> -------------------------------------------------------------------------
>
> Key: ARROW-14378
> URL: https://issues.apache.org/jira/browse/ARROW-14378
> Project: Apache Arrow
> Issue Type: Improvement
> Components: R
> Reporter: Jonathan Keane
> Assignee: Dewey Dunnington
> Priority: Major
>
> The major usecase for this is SF columns which have attributes/metadata for each element of a column. We originally stored these in our standard column-level metadata, but that was very fragile and took forever, so we disabled it ARROW-13189
> This will likely take some steps to accomplish. I've sketched out some in the subtasks here (though if we have a different approach, we could do that directly)
--
This message was sent by Atlassian Jira
(v8.20.1#820001)