[jira] [Commented] (ARROW-14378) [R] Make custom extension classes for (some) cols with row-level metadata

["Dewey Dunnington (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/ARROW-14378?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17438041#comment-17438041 ] 

Dewey Dunnington commented on ARROW-14378:
------------------------------------------

It's worth checking out how jsonlite does this because it's a similar idea (arbitrary R objects to type agnostic storage format with restricted storage modes). It's done through a (non-exported) generic `jsonlite::asJSON()` and sf/sfc is supported (via GeoJSON): [https://github.com/jeroen/jsonlite/blob/master/R/asJSON.sf.R]

It might be worth exploring exported S3 generics (maybe `as_ArrowArray()`/`restore_ArrowArray()`?) that theoretically other packages could implement to serialize and unserialize objects to- and from Arrow storage. If that were the case just like jsonlite, we'd probably implement methods for the most common types.

Just spitballing here!

> [R] Make custom extension classes for (some) cols with row-level metadata
> -------------------------------------------------------------------------
>
>                 Key: ARROW-14378
>                 URL: https://issues.apache.org/jira/browse/ARROW-14378
>             Project: Apache Arrow
>          Issue Type: Improvement
>          Components: R
>            Reporter: Jonathan Keane
>            Assignee: Dewey Dunnington
>            Priority: Major
>
> The major usecase for this is SF columns which have attributes/metadata for each element of a column. We originally stored these in our standard column-level metadata, but that was very fragile and took forever, so we disabled it ARROW-13189
> This will likely take some steps to accomplish. I've sketched out some in the subtasks here (though if we have a different approach, we could do that directly)



--
This message was sent by Atlassian Jira
(v8.3.4#803005)