[jira] [Commented] (ARROW-12792) [R] DatasetFactory could sniff file formats

["Neal Richardson (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/ARROW-12792?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17344636#comment-17344636 ] 

Neal Richardson commented on ARROW-12792:
-----------------------------------------

I'm not sure we want to go down this path (though not sure we don't). If I understand what you're proposing, if you provide file paths (and possibly only local file system paths, not S3/cloud), we sniff their file type, but if you provide a reference to a directory, we don't. I wonder if that step alone just adds confusion.

> [R] DatasetFactory could sniff file formats
> -------------------------------------------
>
>                 Key: ARROW-12792
>                 URL: https://issues.apache.org/jira/browse/ARROW-12792
>             Project: Apache Arrow
>          Issue Type: New Feature
>          Components: R
>            Reporter: Nic Crane
>            Assignee: Nic Crane
>            Priority: Minor
>
> I was running the following code:
> {code:java}
> tf <- tempfile()
> dir.create(tf)
> on.exit(unlink(tf))
> write_csv_arrow(mtcars[1:5,], file.path(tf, "file1.csv"))
> write_csv_arrow(mtcars[6:11,], file.path(tf, "file2.csv"))
> # ds <- open_dataset(c(file.path(tf, "file1.csv"), file.path(tf, "file2.csv")))
> ds <- open_dataset(c(file.path(tf, "file1.csv"), file.path(tf, "file2.csv")), 
>                    schema = Table$create(mtcars)$schema
>                    )
> {code}
> But when I print the ds object, it reports that the files are Parquet files not CSVs
> {code:java}
> > ds
>  FileSystemDataset with 2 Parquet files
>  mpg: double
>  cyl: double
>  disp: double
>  hp: double
>  drat: double
>  wt: double
>  qsec: double
>  vs: double
>  am: double
>  gear: double
>  carb: double{code}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)