[GitHub] [arrow] nealrichardson commented on a change in pull request #9685: ARROW-10372: [Dataset][C++][Python][R] Support reading compressed CSV

[GitBox <gi...@apache.org>]

nealrichardson commented on a change in pull request #9685:
URL: https://github.com/apache/arrow/pull/9685#discussion_r593318541



##########
File path: r/tests/testthat/test-dataset.R
##########
@@ -295,6 +295,29 @@ test_that("CSV dataset", {
   )
 })
 
+test_that("compressed CSV dataset", {
+  skip_if_not_available("gzip")
+  dst_dir <- make_temp_dir()
+  dst_file <- file.path(dst_dir, "data.csv.gz")
+  write.csv(df1, gzfile(dst_file), row.names = FALSE, quote = FALSE)
+  format <- FileFormat$create("csv", compression = CompressionType$GZIP)

Review comment:
       With the above suggestion, this becomes: 
   
   ```suggestion
     format <- FileFormat$create("csv", compression = "gzip")
   ```

##########
File path: r/R/dataset-format.R
##########
@@ -101,8 +101,10 @@ IpcFileFormat <- R6Class("IpcFileFormat", inherit = FileFormat)
 #' @rdname FileFormat
 #' @export
 CsvFileFormat <- R6Class("CsvFileFormat", inherit = FileFormat)
-CsvFileFormat$create <- function(..., opts = csv_file_format_parse_options(...)) {
-  dataset___CsvFileFormat__Make(opts)
+CsvFileFormat$create <- function(...,
+                                 opts = csv_file_format_parse_options(...),
+                                 compression = CompressionType$UNCOMPRESSED) {
+  dataset___CsvFileFormat__Make(opts, compression)

Review comment:
       We have a helper used in a few places that makes this a little more human-friendly:
   
   ```suggestion
                                    compression = "uncompressed") {
     dataset___CsvFileFormat__Make(opts, compression_from_name(compression))
   ```
   
   Should also add a note in the FileFormat$create docs above (around L48) that compression is an option now.




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org