Re: [I] slow nanoarrow_array_stream to data.frame for strings; slow ALTREP materialization [arrow-nanoarrow]

["klin333 (via GitHub)" <gi...@apache.org>]

klin333 commented on issue #822:
URL: https://github.com/apache/arrow-nanoarrow/issues/822#issuecomment-3489807102

   this following path, which also tries to go via arrow::Table, is just as slow as the direct nanoarrow->data.frame path. not sure if this give any ideas to root cause of problem?
   ```r
   library(tidyverse)
   uri <- "secret"
   con <- DBI::dbConnect(adbi::adbi("adbcsnowflake::adbcsnowflake"), uri = uri)
   
   t1 <- Sys.time()
   x_tbl <- tbl(con, 'LARGE1') 
   stream <- DBI::dbGetQueryArrow(con, as.character(dbplyr::sql_render(x_tbl, con)))
   df <- arrow::as_arrow_table(stream) %>% as.data.frame()
   .Internal(inspect(df[[1]])) # 16 STRSXP g0c0 [REF(65535)] arrow::array_string_vector<0x000002468862c460, string
   df <- as_tibble(df)
   print(df) # just as slow as direct path
   .Internal(inspect(df[[1]])) # 16 STRSXP g0c0 [REF(65535)] materialized arrow::array_string_vector len=100000
   print(Sys.time() - t1)
   ```


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org