Re: [I] [R] `Invalid metadata$r` warning when feeding parquet file into dplyr [arrow]

["r2evans (via GitHub)" <gi...@apache.org>]

r2evans commented on issue #40423:
URL: https://github.com/apache/arrow/issues/40423#issuecomment-3705937655

   I'm not sure if it is related, but I can reliably reproduce that warning with named vectors in a column:
   
   ```r
   mutate(mtcars, cyl = setNames(nm = cyl)) |>
     arrow_table() |>
     rename_with(.fn = toupper)
   # Warning: Invalid metadata$r
   # Warning: Invalid metadata$r
   # Table (query)
   # MPG: double
   # CYL: double
   # DISP: double
   # HP: double
   # DRAT: double
   # WT: double
   # QSEC: double
   # VS: double
   # AM: double
   # GEAR: double
   # CARB: double
   # See $.data for the source Arrow object
   ```
   
   The `rename_with(.)` is the causative step, without it there are no warnings (with or without `collect()`). I had not thought that having the name as a vector attribute would in effect defeat some of the "lazy load" in that if an arrow table has 1M+ rows, in its lazy state it is memory-efficient (not all loaded), but the attributes are always loaded and can be largish. Bummer (not this issue).
   
   When the data is read in using `open_dataset()` (not in the above example) then I can override with `..$schema$metadata$r$columns$cyl$attributes$names <- NULL` and then the `rename_with(.)` works. This overwrite does not appear to work the same using `arrow_table()` (not sure if that's by-design or just a feature).
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org