You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@arrow.apache.org by "stemangiola (via GitHub)" <gi...@apache.org> on 2025/03/14 06:27:17 UTC
[I] Creation of parquet from column with complex nested objects [arrow]
stemangiola opened a new issue, #45786:
URL: https://github.com/apache/arrow/issues/45786
### Describe the usage question you have. Please include as many useful details as possible.
The command, which tries to save a tibble in which a column include `SummarizedExperiment` biological data container fails
```r
> tf <- tempfile()
> on.exit(unlink(tf))
> write_parquet(tibble(a = list(tidybulk::breast_tcga_mini_SE)), tf)
Error:
! Can't infer Arrow data type from object inheriting from SummarizedExperiment
Run `rlang::last_trace()` to see where the error occurred.
```
Can Parquet include arbitrary complex nested columns somehow? If there any plan to allow that?
Thanks a lot.
### Component(s)
R
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@arrow.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [R] Creation of parquet from column with complex nested objects [arrow]
Posted by "thisisnic (via GitHub)" <gi...@apache.org>.
thisisnic closed issue #45786: [R] Creation of parquet from column with complex nested objects
URL: https://github.com/apache/arrow/issues/45786
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [R] Creation of parquet from column with complex nested objects [arrow]
Posted by "thisisnic (via GitHub)" <gi...@apache.org>.
thisisnic commented on issue #45786:
URL: https://github.com/apache/arrow/issues/45786#issuecomment-2868975549
Hi @stemangiola, thanks for reporting this!
It looks like the issue here is that the `SummarizedExperiment` object is an S4 class and there is no direct equivalent in Arrow or Parquet. We do supported things like list-columns, but only when the underlying object can be mapped to Arrow/Parquet supported data types. Simpler classes like S3 classes can be preserved (see https://arrowrbook.com/files_and_formats.html#preservation-of-other-metadata for an example).
One workaround could be to extract the relevant data from the `SummarizedExperiment` object (e.g., as a data frame or tibble) and write that to Parquet, rather than trying to serialize the entire object.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org