[jira] [Created] (ARROW-16783) [R] write_dataset fails with an uninformative message when duplicated column names

["Andy Teucher (Jira)" <ji...@apache.org>]

Andy Teucher created ARROW-16783:
------------------------------------

             Summary: [R] write_dataset fails with an uninformative message when duplicated column names
                 Key: ARROW-16783
                 URL: https://issues.apache.org/jira/browse/ARROW-16783
             Project: Apache Arrow
          Issue Type: Improvement
          Components: R
    Affects Versions: 8.0.0
            Reporter: Andy Teucher


{{write_dataset()}} fails when the object being written has duplicated column names. This is probably reasonable behaviour, but the error message is misleading:
{code:r}
library(arrow, warn.conflicts = FALSE)

df <- data.frame(
  id = c("a", "b", "c"),
  x = 1:3, 
  x = 4:6,
  check.names = FALSE
)

write_dataset(df, "df.parquet")
#> Error: 'dataset' must be a Dataset, RecordBatch, Table, arrow_dplyr_query, or data.frame, not "data.frame"
{code}
[{{write_dataset()}} calls {{as_adq()}} inside a {{tryCatch()}} statement|https://github.com/apache/arrow/blob/0d5cf1882228624271062e6c19583c8b0c361a20/r/R/dataset-write.R#L146-L160], so any error from {{as_adq()}} is swallowed and the error emitted is about the class of the object.

The real error comes from here:
{code:r}
arrow:::as_adq(df)
#> Error in `arrow_dplyr_query()`:
#> ! Duplicated field names
#> ✖ The following field names were found more than once in the data: "x"
{code}
I'm not sure what your preferred fix is here... two options that come to mind are:

1. Explicitly check for compatible classes before calling {{as_adq()}} instead of using {{tryCatch()}}

OR

2. Check for duplicate column names before the {{tryCatc}} block

My thought is that option 1 is better, as option 2 means that checking for duplicates would happen twice (once inside {{write_dataset()}} and once again inside {{{}as_adq(){}}}).

I'm happy to work a fix if you like!



--
This message was sent by Atlassian Jira
(v8.20.7#820007)