[GitHub] [arrow] paleolimbot commented on a diff in pull request #37658: GH-34640: [R] Can't read in partitioning column in CSV datasets when both (non-hive) partition and schema supplied

["paleolimbot (via GitHub)" <gi...@apache.org>]

paleolimbot commented on code in PR #37658:
URL: https://github.com/apache/arrow/pull/37658#discussion_r1338638956


##########
r/tests/testthat/test-dataset-csv.R:
##########
@@ -593,3 +593,36 @@ test_that("CSVReadOptions field access", {
   expect_equal(options$block_size, 1048576L)
   expect_equal(options$encoding, "UTF-8")
 })
+
+test_that("GH-34640 - CSV datasets are read in correctly when both schema and partitioning supplied", {
+
+  target_schema <- schema(
+      int = int64(), dbl = float16(), lgl = bool(), chr = utf8(),
+      fct = utf8(), ts = timestamp(unit = "s"), part = int32()
+    )
+
+  ds <- open_dataset(
+    csv_dir,
+    partitioning = schema(part = int32()),
+    format = "csv",
+    schema = target_schema,
+    skip = 1
+  )
+  expect_r6_class(ds$format, "CsvFileFormat")
+  expect_r6_class(ds$filesystem, "LocalFileSystem")
+  expect_identical(names(ds), c(names(df1), "part"))
+  expect_identical(dim(ds), c(20L, 7L))
+  expect_equal(schema(ds), target_schema)
+

Review Comment:
   Maybe also:
   
   ```r
   expect_identical(names(collect(ds)), c(names(df1), "part"))
   ```
   
   (I seem to remember there are some things that don't happen until a dataset is collected)



##########
r/R/dataset-format.R:
##########
@@ -223,7 +227,7 @@ check_csv_file_format_args <- function(args) {
   }
 
   if (is.null(args$read_options)) {
-    options$read_options <- do.call(csv_file_format_read_opts, args)
+    options$read_options <- do.call(csv_file_format_read_opts, c(args, partitioning = partitioning))

Review Comment:
   ```suggestion
       options$read_options <- do.call(csv_file_format_read_opts, c(args, list(partitioning = partitioning)))
   ```
   
   (I think `c(something_that_is_a_list, some_name = something_that_could_be_anything)` often, but not always, results in the output being a list...wrapping the second bit in a `list()` always works)



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org