[jira] [Commented] (ARROW-15460) [R] Add as.data.frame.Dataset method

["Dewey Dunnington (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/ARROW-15460?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17484678#comment-17484678 ] 

Dewey Dunnington commented on ARROW-15460:
------------------------------------------

Reprex:

{code:R}
library(arrow, warn.conflicts = FALSE)

# possible implementation
as.data.frame.Dataset <- function(x, ...) {
  as.data.frame(Scanner$create(ds)$ToTable())
}

ds <- InMemoryDataset$create(data.frame(int = 1:10))
as.data.frame(ds)
#>    int
#> 1    1
#> 2    2
#> 3    3
#> 4    4
#> 5    5
#> 6    6
#> 7    7
#> 8    8
#> 9    9
#> 10  10

# for a future print method we can already do head()
as.data.frame(head(ds))
#>   int
#> 1   1
#> 2   2
#> 3   3
#> 4   4
#> 5   5
#> 6   6
{code}


> [R] Add as.data.frame.Dataset method
> ------------------------------------
>
>                 Key: ARROW-15460
>                 URL: https://issues.apache.org/jira/browse/ARROW-15460
>             Project: Apache Arrow
>          Issue Type: New Feature
>          Components: R
>            Reporter: Dragoș Moldovan-Grünfeld
>            Priority: Major
>
> Started with a question from Jim Hester on Twitter:
> bq. Is there a way to take an arrow::Dataset and collect all the data into a data.frame without using `dplyr::collect()`?
> bq. I have a code path I just want to return a regular data.frame, but I don't really want to add a soft dplyr dependency just for this.
> Twitter thread: https://twitter.com/jimhester_/status/1484624519612579841
> This might also be useful for pillar/tibble. Maybe add a {{max_memory_argument}} to avoid allocating to much memory. (see suggestion from Kirill Müller)



--
This message was sent by Atlassian Jira
(v8.20.1#820001)