You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@arrow.apache.org by GitBox <gi...@apache.org> on 2022/07/04 12:40:14 UTC

[GitHub] [arrow] rok opened a new pull request, #13506: ARROW-16653: [R] All formats are supported with the lubridate `parse_date_time` binding

rok opened a new pull request, #13506:
URL: https://github.com/apache/arrow/pull/13506

   This is to resolve [ARROW-16653](https://issues.apache.org/jira/browse/ARROW-16653).
   Please note the intent here is to map out currently available formats via a new test. That is to support [ARROW-16395](https://issues.apache.org/jira/browse/ARROW-16395) that will add `ymd_hms() ymd_hm() ymd_h() dmy_hms() dmy_hm() dmy_h() mdy_hms() mdy_hm() mdy_h() ydm_hms() ydm_hm() ydm_h()`.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [arrow] ursabot commented on pull request #13506: ARROW-16653: [R] All formats are supported with the lubridate `parse_date_time` binding

Posted by GitBox <gi...@apache.org>.
ursabot commented on PR #13506:
URL: https://github.com/apache/arrow/pull/13506#issuecomment-1199053075

   Benchmark runs are scheduled for baseline = f645ffa2a089805ed6a4d2d64fd50ef40d7ddbc7 and contender = 3e87c66ae00cd25cb9e5e42424b90ace988856ac. 3e87c66ae00cd25cb9e5e42424b90ace988856ac is a master commit associated with this PR. Results will be available as each benchmark for each run completes.
   Conbench compare runs links:
   [Failed :arrow_down:3.77% :arrow_up:0.0%] [ec2-t3-xlarge-us-east-2](https://conbench.ursa.dev/compare/runs/4cce8f5e3105466d939e6a0a984c6d34...63d29f5bad9c4884b8c612f0c59e61f7/)
   [Finished :arrow_down:0.65% :arrow_up:0.0%] [test-mac-arm](https://conbench.ursa.dev/compare/runs/fded4659588242f8a0d19baf4f5eb19f...bffb84b0d9a04a88bd41a5d7c097dab6/)
   [Finished :arrow_down:0.0% :arrow_up:0.0%] [ursa-i9-9960x](https://conbench.ursa.dev/compare/runs/55d29b5309f8427a9058a43e2197b364...111c914674c2414a9ceed586739e96ad/)
   [Finished :arrow_down:0.39% :arrow_up:0.07%] [ursa-thinkcentre-m75q](https://conbench.ursa.dev/compare/runs/52dc1a5eff8c43348ae28ceabda1d063...4b457df5b0124bdf85d697bbd1a1a0ee/)
   Buildkite builds:
   [Failed] [`3e87c66a` ec2-t3-xlarge-us-east-2](https://buildkite.com/apache-arrow/arrow-bci-benchmark-on-ec2-t3-xlarge-us-east-2/builds/1222)
   [Finished] [`3e87c66a` test-mac-arm](https://buildkite.com/apache-arrow/arrow-bci-benchmark-on-test-mac-arm/builds/1234)
   [Finished] [`3e87c66a` ursa-i9-9960x](https://buildkite.com/apache-arrow/arrow-bci-benchmark-on-ursa-i9-9960x/builds/1216)
   [Finished] [`3e87c66a` ursa-thinkcentre-m75q](https://buildkite.com/apache-arrow/arrow-bci-benchmark-on-ursa-thinkcentre-m75q/builds/1236)
   [Failed] [`f645ffa2` ec2-t3-xlarge-us-east-2](https://buildkite.com/apache-arrow/arrow-bci-benchmark-on-ec2-t3-xlarge-us-east-2/builds/1221)
   [Finished] [`f645ffa2` test-mac-arm](https://buildkite.com/apache-arrow/arrow-bci-benchmark-on-test-mac-arm/builds/1233)
   [Finished] [`f645ffa2` ursa-i9-9960x](https://buildkite.com/apache-arrow/arrow-bci-benchmark-on-ursa-i9-9960x/builds/1215)
   [Finished] [`f645ffa2` ursa-thinkcentre-m75q](https://buildkite.com/apache-arrow/arrow-bci-benchmark-on-ursa-thinkcentre-m75q/builds/1235)
   Supported benchmarks:
   ec2-t3-xlarge-us-east-2: Supported benchmark langs: Python, R. Runs only benchmarks with cloud = True
   test-mac-arm: Supported benchmark langs: C++, Python, R
   ursa-i9-9960x: Supported benchmark langs: Python, R, JavaScript
   ursa-thinkcentre-m75q: Supported benchmark langs: C++, Java
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [arrow] dragosmg commented on pull request #13506: ARROW-16653: [R] All formats are supported with the lubridate `parse_date_time` binding

Posted by GitBox <gi...@apache.org>.
dragosmg commented on PR #13506:
URL: https://github.com/apache/arrow/pull/13506#issuecomment-1192791651

   LGTM once we have a test with the additional formats we now support & the CI passes. 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [arrow] rok commented on a diff in pull request #13506: ARROW-16653: [R] All formats are supported with the lubridate `parse_date_time` binding

Posted by GitBox <gi...@apache.org>.
rok commented on code in PR #13506:
URL: https://github.com/apache/arrow/pull/13506#discussion_r930133620


##########
r/R/dplyr-datetime-helpers.R:
##########
@@ -166,6 +166,18 @@ build_formats <- function(orders) {
   orders <- gsub("[^A-Za-z]", "", orders)
   orders <- gsub("Y", "y", orders)
 
+  invalid_orders <- nchar(gsub("[ymdqIHMS]", "", orders)) > 0

Review Comment:
   I've made this into a different check. Basically the format needs to be longer than 6 chars to be considered valid (e.g.: "ymd" -> "%Y%m%d" is 6 chars long and is probably the shortest format we have).



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [arrow] paleolimbot commented on pull request #13506: ARROW-16653: [R] All formats are supported with the lubridate `parse_date_time` binding

Posted by GitBox <gi...@apache.org>.
paleolimbot commented on PR #13506:
URL: https://github.com/apache/arrow/pull/13506#issuecomment-1198784246

   (The sanitizer error was fixed already and valgrind will likely fail when it completes but that is a separate issue)


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [arrow] dragosmg commented on a diff in pull request #13506: ARROW-16653: [R] All formats are supported with the lubridate `parse_date_time` binding

Posted by GitBox <gi...@apache.org>.
dragosmg commented on code in PR #13506:
URL: https://github.com/apache/arrow/pull/13506#discussion_r928544138


##########
r/R/dplyr-datetime-helpers.R:
##########
@@ -240,25 +206,29 @@ build_formats <- function(orders) {
 #' @noRd
 build_format_from_order <- function(order) {
   char_list <- list(
+    "T" = "%H-%M-%S",

Review Comment:
   Probably a swap of `"T"` to "HMS" should happen in `build_formats` and not here.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [arrow] rok commented on a diff in pull request #13506: ARROW-16653: [R] All formats are supported with the lubridate `parse_date_time` binding

Posted by GitBox <gi...@apache.org>.
rok commented on code in PR #13506:
URL: https://github.com/apache/arrow/pull/13506#discussion_r927889222


##########
r/tests/testthat/test-dplyr-funcs-datetime.R:
##########
@@ -152,13 +152,14 @@ test_that("strptime", {
   )
 
   tz <- "Pacific/Marquesas"
-  times <- seq(as.POSIXct("1999-02-07", tz = tz), as.POSIXct("2000-01-01", tz = tz), by = "hour")
-  times <- c(as.POSIXct("1999-12-31T12:34:56.01", tz = "UTC"))
+  set.seed(42)
+  times <- seq(as.POSIXct("1999-02-07", tz = tz), as.POSIXct("2000-01-01", tz = tz), by = "sec")

Review Comment:
   It's about a year's worth of seconds (`~3e+7`). I stopped the full vector test after three minutes and sampled to arbitrary number.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [arrow] dragosmg commented on a diff in pull request #13506: ARROW-16653: [R] All formats are supported with the lubridate `parse_date_time` binding

Posted by GitBox <gi...@apache.org>.
dragosmg commented on code in PR #13506:
URL: https://github.com/apache/arrow/pull/13506#discussion_r928534338


##########
r/R/dplyr-datetime-helpers.R:
##########
@@ -158,74 +158,40 @@ binding_as_date_numeric <- function(x, origin = "1970-01-01") {
 #'
 #' @noRd
 build_formats <- function(orders) {
-  # only keep the letters and the underscore as separator -> allow the users to
-  # pass strptime-like formats (with "%"). We process the data -> we need to
-  # process the `orders` (even if supplied in the desired format)
-  # Processing is needed (instead of passing
-  # formats as-is) due to the processing of the character vector in parse_date_time()
-  orders <- gsub("[^A-Za-z]", "", orders)
-  orders <- gsub("Y", "y", orders)
-
   # we separate "ym', "my", and "yq" from the rest of the `orders` vector and
   # transform them. `ym` and `yq` -> `ymd` & `my` -> `myd`
   # this is needed for 2 reasons:
   # 1. strptime does not parse "2022-05" -> we add "-01", thus changing the format,
   # 2. for equivalence to lubridate, which parses `ym` to the first day of the month
-  short_orders <- c("ym", "my")
+  short_orders <- c("ym", "my", "Ym", "mY")
+  quarter_orders <- c("yq", "Yq", "qy", "qY")
 
   if (any(orders %in% short_orders)) {
     orders1 <- setdiff(orders, short_orders)
     orders2 <- intersect(orders, short_orders)
     orders2 <- paste0(orders2, "d")
     orders <- unique(c(orders2, orders1))
   }
-
-  if (any(orders == "yq")) {
-    orders1 <- setdiff(orders, "yq")
-    orders2 <- "ymd"
-    orders <- unique(c(orders1, orders2))
+  if (any(orders %in% quarter_orders)) {
+    orders <- c(setdiff(orders, quarter_orders), "ymd")
   }
+  orders <- unique(orders)
 
-  if (any(orders == "qy")) {
-    orders1 <- setdiff(orders, "qy")
-    orders2 <- "ymd"
-    orders <- unique(c(orders1, orders2))
-  }
-
-  ymd_orders <- c("ymd", "ydm", "mdy", "myd", "dmy", "dym")
-  ymd_hms_orders <- c(
-    "ymd_HMS", "ymd_HM", "ymd_H", "dmy_HMS", "dmy_HM", "dmy_H", "mdy_HMS",
-    "mdy_HM", "mdy_H", "ydm_HMS", "ydm_HM", "ydm_H"
-  )
-  # support "%I" hour formats
-  ymd_ims_orders <- gsub("H", "I", ymd_hms_orders)
-
-  supported_orders <- c(
-    ymd_orders,
-    ymd_hms_orders,
-    gsub("_", " ", ymd_hms_orders), # allow "_", " " and "" as order separators
-    gsub("_", "", ymd_hms_orders),
-    ymd_ims_orders,
-    gsub("_", " ", ymd_ims_orders), # allow "_", " " and "" as order separators
-    gsub("_", "", ymd_ims_orders)
-  )
-
-  unsupported_passed_orders <- setdiff(orders, supported_orders)
-  supported_passed_orders <- intersect(orders, supported_orders)
+  formats_list <- map(orders, build_format_from_order)
+  formats_length <- map(map(formats_list, nchar), max)
+  invalid_orders <- formats_length < 6
 
-  # error only if there isn't at least one valid order we can try
-  if (length(supported_passed_orders) == 0) {
+  if (any(invalid_orders)) {
     arrow_not_supported(
       paste0(
         oxford_paste(
-          unsupported_passed_orders
+          orders[invalid_orders]
         ),
         " `orders`"
       )
     )
   }

Review Comment:
   If you end up removing it completely - which I think is the way to go - could you please also update the description? https://github.com/apache/arrow/blob/ef6049a2ee5673d0944a0b4f70ff9c70e0419a22/r/R/dplyr-datetime-helpers.R#L152-L153, we should probably remove:
   #' it also checks if the supplied
   #' orders are currently supported. 



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [arrow] dragosmg commented on pull request #13506: ARROW-16653: [R] All formats are supported with the lubridate `parse_date_time` binding

Posted by GitBox <gi...@apache.org>.
dragosmg commented on PR #13506:
URL: https://github.com/apache/arrow/pull/13506#issuecomment-1192853211

   Once we remove the guardrails, it would be good to test `orders` that aren't covered by #13627, such as: `parse_date_time("12/17/1996 04:00:00  PM", "mdYHMSp")` 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [arrow] dragosmg commented on a diff in pull request #13506: ARROW-16653: [R] All formats are supported with the lubridate `parse_date_time` binding

Posted by GitBox <gi...@apache.org>.
dragosmg commented on code in PR #13506:
URL: https://github.com/apache/arrow/pull/13506#discussion_r930806145


##########
r/R/dplyr-datetime-helpers.R:
##########
@@ -240,25 +214,24 @@ build_formats <- function(orders) {
 #' @noRd
 build_format_from_order <- function(order) {
   char_list <- list(
-    "y" = c("%y", "%Y"),
-    "m" = c("%m", "%B", "%b"),
-    "d" = "%d",
-    "H" = "%H",
-    "M" = "%M",
-    "S" = "%S",
-    "I" = "%I"
+    "%T" = "%H-%M-%S",
+    "%y" = c("%y", "%Y"),
+    "%m" = c("%m", "%B", "%b"),
+    "%b" = c("%m", "%B", "%b")
   )
 
-  split_order <- strsplit(order, split = "")[[1]]
+  formats <- regmatches(order, gregexpr("(O{0,1}[a-zA-Z])", order))[[1]]

Review Comment:
   Same as above, I think a comment / comments would make this easier to follow. 



##########
r/R/dplyr-datetime-helpers.R:
##########
@@ -171,61 +171,35 @@ build_formats <- function(orders) {
   # this is needed for 2 reasons:
   # 1. strptime does not parse "2022-05" -> we add "-01", thus changing the format,
   # 2. for equivalence to lubridate, which parses `ym` to the first day of the month
-  short_orders <- c("ym", "my")
+  short_orders <- c("ym", "my", "yOm", "Omy")
+  quarter_orders <- c("yq", "qy")
 
   if (any(orders %in% short_orders)) {
     orders1 <- setdiff(orders, short_orders)
     orders2 <- intersect(orders, short_orders)
     orders2 <- paste0(orders2, "d")
     orders <- unique(c(orders2, orders1))
   }
-
-  if (any(orders == "yq")) {
-    orders1 <- setdiff(orders, "yq")
-    orders2 <- "ymd"
-    orders <- unique(c(orders1, orders2))
-  }
-
-  if (any(orders == "qy")) {
-    orders1 <- setdiff(orders, "qy")
-    orders2 <- "ymd"
-    orders <- unique(c(orders1, orders2))
+  if (any(orders %in% quarter_orders)) {
+    orders <- c(setdiff(orders, quarter_orders), "ymd")
   }
+  orders <- unique(orders)
 
-  ymd_orders <- c("ymd", "ydm", "mdy", "myd", "dmy", "dym")
-  ymd_hms_orders <- c(
-    "ymd_HMS", "ymd_HM", "ymd_H", "dmy_HMS", "dmy_HM", "dmy_H", "mdy_HMS",
-    "mdy_HM", "mdy_H", "ydm_HMS", "ydm_HM", "ydm_H"
-  )
-  # support "%I" hour formats
-  ymd_ims_orders <- gsub("H", "I", ymd_hms_orders)
-
-  supported_orders <- c(
-    ymd_orders,
-    ymd_hms_orders,
-    gsub("_", " ", ymd_hms_orders), # allow "_", " " and "" as order separators
-    gsub("_", "", ymd_hms_orders),
-    ymd_ims_orders,
-    gsub("_", " ", ymd_ims_orders), # allow "_", " " and "" as order separators
-    gsub("_", "", ymd_ims_orders)
-  )
-
-  unsupported_passed_orders <- setdiff(orders, supported_orders)
-  supported_passed_orders <- intersect(orders, supported_orders)
+  formats_list <- map(orders, build_format_from_order)
+  formats_length <- map(map(formats_list, nchar), max)
+  invalid_orders <- formats_length < 6

Review Comment:
   I think adding some comments here would help for our future selves. For example, on why you think 6 is a good threshold. 



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [arrow] dragosmg commented on a diff in pull request #13506: ARROW-16653: [R] All formats are supported with the lubridate `parse_date_time` binding

Posted by GitBox <gi...@apache.org>.
dragosmg commented on code in PR #13506:
URL: https://github.com/apache/arrow/pull/13506#discussion_r928530697


##########
r/R/dplyr-datetime-helpers.R:
##########
@@ -158,74 +158,40 @@ binding_as_date_numeric <- function(x, origin = "1970-01-01") {
 #'
 #' @noRd
 build_formats <- function(orders) {
-  # only keep the letters and the underscore as separator -> allow the users to
-  # pass strptime-like formats (with "%"). We process the data -> we need to
-  # process the `orders` (even if supplied in the desired format)
-  # Processing is needed (instead of passing
-  # formats as-is) due to the processing of the character vector in parse_date_time()
-  orders <- gsub("[^A-Za-z]", "", orders)
-  orders <- gsub("Y", "y", orders)
-
   # we separate "ym', "my", and "yq" from the rest of the `orders` vector and
   # transform them. `ym` and `yq` -> `ymd` & `my` -> `myd`
   # this is needed for 2 reasons:
   # 1. strptime does not parse "2022-05" -> we add "-01", thus changing the format,
   # 2. for equivalence to lubridate, which parses `ym` to the first day of the month
-  short_orders <- c("ym", "my")
+  short_orders <- c("ym", "my", "Ym", "mY")
+  quarter_orders <- c("yq", "Yq", "qy", "qY")
 
   if (any(orders %in% short_orders)) {
     orders1 <- setdiff(orders, short_orders)
     orders2 <- intersect(orders, short_orders)
     orders2 <- paste0(orders2, "d")
     orders <- unique(c(orders2, orders1))
   }
-
-  if (any(orders == "yq")) {
-    orders1 <- setdiff(orders, "yq")
-    orders2 <- "ymd"
-    orders <- unique(c(orders1, orders2))
+  if (any(orders %in% quarter_orders)) {
+    orders <- c(setdiff(orders, quarter_orders), "ymd")
   }
+  orders <- unique(orders)
 
-  if (any(orders == "qy")) {
-    orders1 <- setdiff(orders, "qy")
-    orders2 <- "ymd"
-    orders <- unique(c(orders1, orders2))
-  }
-
-  ymd_orders <- c("ymd", "ydm", "mdy", "myd", "dmy", "dym")
-  ymd_hms_orders <- c(
-    "ymd_HMS", "ymd_HM", "ymd_H", "dmy_HMS", "dmy_HM", "dmy_H", "mdy_HMS",
-    "mdy_HM", "mdy_H", "ydm_HMS", "ydm_HM", "ydm_H"
-  )
-  # support "%I" hour formats
-  ymd_ims_orders <- gsub("H", "I", ymd_hms_orders)
-
-  supported_orders <- c(
-    ymd_orders,
-    ymd_hms_orders,
-    gsub("_", " ", ymd_hms_orders), # allow "_", " " and "" as order separators
-    gsub("_", "", ymd_hms_orders),
-    ymd_ims_orders,
-    gsub("_", " ", ymd_ims_orders), # allow "_", " " and "" as order separators
-    gsub("_", "", ymd_ims_orders)
-  )
-
-  unsupported_passed_orders <- setdiff(orders, supported_orders)
-  supported_passed_orders <- intersect(orders, supported_orders)
+  formats_list <- map(orders, build_format_from_order)
+  formats_length <- map(map(formats_list, nchar), max)
+  invalid_orders <- formats_length < 6
 
-  # error only if there isn't at least one valid order we can try
-  if (length(supported_passed_orders) == 0) {
+  if (any(invalid_orders)) {
     arrow_not_supported(
       paste0(
         oxford_paste(
-          unsupported_passed_orders
+          orders[invalid_orders]
         ),
         " `orders`"
       )
     )
   }

Review Comment:
   I agree. It was there while we were implementing additional formats. Once this PR is done, it doesn't serve that purpose anymore.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [arrow] rok commented on a diff in pull request #13506: ARROW-16653: [R] All formats are supported with the lubridate `parse_date_time` binding

Posted by GitBox <gi...@apache.org>.
rok commented on code in PR #13506:
URL: https://github.com/apache/arrow/pull/13506#discussion_r928527638


##########
r/R/dplyr-datetime-helpers.R:
##########
@@ -158,74 +158,40 @@ binding_as_date_numeric <- function(x, origin = "1970-01-01") {
 #'
 #' @noRd
 build_formats <- function(orders) {
-  # only keep the letters and the underscore as separator -> allow the users to
-  # pass strptime-like formats (with "%"). We process the data -> we need to
-  # process the `orders` (even if supplied in the desired format)
-  # Processing is needed (instead of passing
-  # formats as-is) due to the processing of the character vector in parse_date_time()
-  orders <- gsub("[^A-Za-z]", "", orders)
-  orders <- gsub("Y", "y", orders)
-
   # we separate "ym', "my", and "yq" from the rest of the `orders` vector and
   # transform them. `ym` and `yq` -> `ymd` & `my` -> `myd`
   # this is needed for 2 reasons:
   # 1. strptime does not parse "2022-05" -> we add "-01", thus changing the format,
   # 2. for equivalence to lubridate, which parses `ym` to the first day of the month
-  short_orders <- c("ym", "my")
+  short_orders <- c("ym", "my", "Ym", "mY")
+  quarter_orders <- c("yq", "Yq", "qy", "qY")
 
   if (any(orders %in% short_orders)) {
     orders1 <- setdiff(orders, short_orders)
     orders2 <- intersect(orders, short_orders)
     orders2 <- paste0(orders2, "d")
     orders <- unique(c(orders2, orders1))
   }
-
-  if (any(orders == "yq")) {
-    orders1 <- setdiff(orders, "yq")
-    orders2 <- "ymd"
-    orders <- unique(c(orders1, orders2))
+  if (any(orders %in% quarter_orders)) {
+    orders <- c(setdiff(orders, quarter_orders), "ymd")
   }
+  orders <- unique(orders)
 
-  if (any(orders == "qy")) {
-    orders1 <- setdiff(orders, "qy")
-    orders2 <- "ymd"
-    orders <- unique(c(orders1, orders2))
-  }
-
-  ymd_orders <- c("ymd", "ydm", "mdy", "myd", "dmy", "dym")
-  ymd_hms_orders <- c(
-    "ymd_HMS", "ymd_HM", "ymd_H", "dmy_HMS", "dmy_HM", "dmy_H", "mdy_HMS",
-    "mdy_HM", "mdy_H", "ydm_HMS", "ydm_HM", "ydm_H"
-  )
-  # support "%I" hour formats
-  ymd_ims_orders <- gsub("H", "I", ymd_hms_orders)
-
-  supported_orders <- c(
-    ymd_orders,
-    ymd_hms_orders,
-    gsub("_", " ", ymd_hms_orders), # allow "_", " " and "" as order separators
-    gsub("_", "", ymd_hms_orders),
-    ymd_ims_orders,
-    gsub("_", " ", ymd_ims_orders), # allow "_", " " and "" as order separators
-    gsub("_", "", ymd_ims_orders)
-  )
-
-  unsupported_passed_orders <- setdiff(orders, supported_orders)
-  supported_passed_orders <- intersect(orders, supported_orders)
+  formats_list <- map(orders, build_format_from_order)
+  formats_length <- map(map(formats_list, nchar), max)
+  invalid_orders <- formats_length < 6
 
-  # error only if there isn't at least one valid order we can try
-  if (length(supported_passed_orders) == 0) {
+  if (any(invalid_orders)) {
     arrow_not_supported(
       paste0(
         oxford_paste(
-          unsupported_passed_orders
+          orders[invalid_orders]
         ),
         " `orders`"
       )
     )
   }

Review Comment:
   Should this logic really live in R? Wouldn't it be better to have strptime detect and raise this?



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [arrow] dragosmg commented on a diff in pull request #13506: ARROW-16653: [R] All formats are supported with the lubridate `parse_date_time` binding

Posted by GitBox <gi...@apache.org>.
dragosmg commented on code in PR #13506:
URL: https://github.com/apache/arrow/pull/13506#discussion_r930151838


##########
r/R/dplyr-datetime-helpers.R:
##########
@@ -240,25 +214,24 @@ build_formats <- function(orders) {
 #' @noRd
 build_format_from_order <- function(order) {
   char_list <- list(
-    "y" = c("%y", "%Y"),
-    "m" = c("%m", "%B", "%b"),
-    "d" = "%d",
-    "H" = "%H",
-    "M" = "%M",
-    "S" = "%S",
-    "I" = "%I"
+    "%T" = "%H-%M-%S",
+    "%y" = c("%y", "%Y"),
+    "%m" = c("%m", "%B", "%b"),
+    "%b" = c("%m", "%B", "%b")
   )
 
-  split_order <- strsplit(order, split = "")[[1]]
+  formats <- stringr::str_extract_all(order, "%{0,1}(O*[a-zA-Z])")[[1]]

Review Comment:
   We currently do not depend on {stringr} - i.e. `stringr` is not listed under `Imports` in DESCRIPTION. I'm not sure it's worth the extra dependency for a single function call, which we could reproduce with base R functions.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [arrow] rok commented on a diff in pull request #13506: ARROW-16653: [R] All formats are supported with the lubridate `parse_date_time` binding

Posted by GitBox <gi...@apache.org>.
rok commented on code in PR #13506:
URL: https://github.com/apache/arrow/pull/13506#discussion_r931730491


##########
r/R/dplyr-datetime-helpers.R:
##########
@@ -171,61 +171,35 @@ build_formats <- function(orders) {
   # this is needed for 2 reasons:
   # 1. strptime does not parse "2022-05" -> we add "-01", thus changing the format,
   # 2. for equivalence to lubridate, which parses `ym` to the first day of the month
-  short_orders <- c("ym", "my")
+  short_orders <- c("ym", "my", "yOm", "Omy")
+  quarter_orders <- c("yq", "qy")
 
   if (any(orders %in% short_orders)) {
     orders1 <- setdiff(orders, short_orders)
     orders2 <- intersect(orders, short_orders)
     orders2 <- paste0(orders2, "d")
     orders <- unique(c(orders2, orders1))
   }
-
-  if (any(orders == "yq")) {
-    orders1 <- setdiff(orders, "yq")
-    orders2 <- "ymd"
-    orders <- unique(c(orders1, orders2))
-  }
-
-  if (any(orders == "qy")) {
-    orders1 <- setdiff(orders, "qy")
-    orders2 <- "ymd"
-    orders <- unique(c(orders1, orders2))
+  if (any(orders %in% quarter_orders)) {
+    orders <- c(setdiff(orders, quarter_orders), "ymd")
   }
+  orders <- unique(orders)
 
-  ymd_orders <- c("ymd", "ydm", "mdy", "myd", "dmy", "dym")
-  ymd_hms_orders <- c(
-    "ymd_HMS", "ymd_HM", "ymd_H", "dmy_HMS", "dmy_HM", "dmy_H", "mdy_HMS",
-    "mdy_HM", "mdy_H", "ydm_HMS", "ydm_HM", "ydm_H"
-  )
-  # support "%I" hour formats
-  ymd_ims_orders <- gsub("H", "I", ymd_hms_orders)
-
-  supported_orders <- c(
-    ymd_orders,
-    ymd_hms_orders,
-    gsub("_", " ", ymd_hms_orders), # allow "_", " " and "" as order separators
-    gsub("_", "", ymd_hms_orders),
-    ymd_ims_orders,
-    gsub("_", " ", ymd_ims_orders), # allow "_", " " and "" as order separators
-    gsub("_", "", ymd_ims_orders)
-  )
-
-  unsupported_passed_orders <- setdiff(orders, supported_orders)
-  supported_passed_orders <- intersect(orders, supported_orders)
+  formats_list <- map(orders, build_format_from_order)
+  formats_length <- map(map(formats_list, nchar), max)
+  invalid_orders <- formats_length < 6

Review Comment:
   Lubridate checks against an example of it's input, which is something we can't/won't do.
   I'd say erroring on invalid formats and returning NA on unsuccessful parsing would be the thing to do.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [arrow] rok commented on a diff in pull request #13506: ARROW-16653: [R] All formats are supported with the lubridate `parse_date_time` binding

Posted by GitBox <gi...@apache.org>.
rok commented on code in PR #13506:
URL: https://github.com/apache/arrow/pull/13506#discussion_r931729522


##########
r/R/dplyr-datetime-helpers.R:
##########
@@ -240,25 +214,24 @@ build_formats <- function(orders) {
 #' @noRd
 build_format_from_order <- function(order) {
   char_list <- list(
-    "y" = c("%y", "%Y"),
-    "m" = c("%m", "%B", "%b"),
-    "d" = "%d",
-    "H" = "%H",
-    "M" = "%M",
-    "S" = "%S",
-    "I" = "%I"
+    "%T" = "%H-%M-%S",
+    "%y" = c("%y", "%Y"),
+    "%m" = c("%m", "%B", "%b"),
+    "%b" = c("%m", "%B", "%b")
   )
 
-  split_order <- strsplit(order, split = "")[[1]]
+  formats <- regmatches(order, gregexpr("(O{0,1}[a-zA-Z])", order))[[1]]
+  formats <- paste0("%", formats)
+  formats <- ifelse(formats %in% names(char_list), char_list[formats], formats)

Review Comment:
   Switching back to the older implementation.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [arrow] rok commented on a diff in pull request #13506: ARROW-16653: [R] All formats are supported with the lubridate `parse_date_time` binding

Posted by GitBox <gi...@apache.org>.
rok commented on code in PR #13506:
URL: https://github.com/apache/arrow/pull/13506#discussion_r930221417


##########
r/R/dplyr-datetime-helpers.R:
##########
@@ -166,6 +166,18 @@ build_formats <- function(orders) {
   orders <- gsub("[^A-Za-z]", "", orders)
   orders <- gsub("Y", "y", orders)
 
+  invalid_orders <- nchar(gsub("[ymdqIHMS]", "", orders)) > 0

Review Comment:
   I've made this into a different check. Basically the format needs to be longer than 6 chars to be considered valid (e.g.: "ymd" -> "%Y%m%d" is 6 chars long and is probably the shortest format we have).



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [arrow] rok commented on a diff in pull request #13506: ARROW-16653: [R] All formats are supported with the lubridate `parse_date_time` binding

Posted by GitBox <gi...@apache.org>.
rok commented on code in PR #13506:
URL: https://github.com/apache/arrow/pull/13506#discussion_r930133620


##########
r/R/dplyr-datetime-helpers.R:
##########
@@ -166,6 +166,18 @@ build_formats <- function(orders) {
   orders <- gsub("[^A-Za-z]", "", orders)
   orders <- gsub("Y", "y", orders)
 
+  invalid_orders <- nchar(gsub("[ymdqIHMS]", "", orders)) > 0

Review Comment:
   I've made this into a different check. Basically the format needs to be longer than 6 chars to be considered valid (e.g.: "ymd" -> "%Y%m%d" is 6 chars long and is probably the shortest format we have).



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [arrow] dragosmg commented on a diff in pull request #13506: ARROW-16653: [R] All formats are supported with the lubridate `parse_date_time` binding

Posted by GitBox <gi...@apache.org>.
dragosmg commented on code in PR #13506:
URL: https://github.com/apache/arrow/pull/13506#discussion_r930151838


##########
r/R/dplyr-datetime-helpers.R:
##########
@@ -240,25 +214,24 @@ build_formats <- function(orders) {
 #' @noRd
 build_format_from_order <- function(order) {
   char_list <- list(
-    "y" = c("%y", "%Y"),
-    "m" = c("%m", "%B", "%b"),
-    "d" = "%d",
-    "H" = "%H",
-    "M" = "%M",
-    "S" = "%S",
-    "I" = "%I"
+    "%T" = "%H-%M-%S",
+    "%y" = c("%y", "%Y"),
+    "%m" = c("%m", "%B", "%b"),
+    "%b" = c("%m", "%B", "%b")
   )
 
-  split_order <- strsplit(order, split = "")[[1]]
+  formats <- stringr::str_extract_all(order, "%{0,1}(O*[a-zA-Z])")[[1]]

Review Comment:
   We currently do not depend on {stringr} - i.e. `stringr` is not listed under `Imports` in DESCRIPTION. I'm not sure it's worth the extra dependency for a single function call, which we could reproduce with base R functions.
   
   Moreover, the regex starts with match `%` 0 or 1 times in orders, given that we strip anything that is not a letter with:
   https://github.com/apache/arrow/blob/bbf249e056315af0a18d5c0834de9adef117a25f/r/R/dplyr-datetime-helpers.R#L166
   , I don't think that regex is entirely needed. 



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [arrow] rok commented on pull request #13506: ARROW-16653: [R] All formats are supported with the lubridate `parse_date_time` binding

Posted by GitBox <gi...@apache.org>.
rok commented on PR #13506:
URL: https://github.com/apache/arrow/pull/13506#issuecomment-1197587262

   Thanks for the reviews @dragosmg @paleolimbot !
   I've pushed a new set of changes, could you please review them? I'm still hoping to get this into the release 🤞 .


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [arrow] dragosmg commented on a diff in pull request #13506: ARROW-16653: [R] All formats are supported with the lubridate `parse_date_time` binding

Posted by GitBox <gi...@apache.org>.
dragosmg commented on code in PR #13506:
URL: https://github.com/apache/arrow/pull/13506#discussion_r928540866


##########
r/tests/testthat/test-dplyr-funcs-datetime.R:
##########
@@ -2638,14 +2694,18 @@ test_that("build_formats() and build_format_from_order()", {
     )
   )
 
-  # ab not supported yet
-  expect_error(
-    build_formats("abd"),
-    '"abd" `orders` not supported in Arrow'
+  expect_equal(
+    build_format_from_order("abp"),
+    c("%a", "%b","%p", "%a", "%b", "%p")
+  )
+
+  expect_equal(
+    build_format_from_order("uYM"),
+    c("%y-%M", "%Y-%M", "%y%M", "%Y%M")

Review Comment:
   For example, due to line 166, no order should have a `%` inside.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [arrow] rok commented on a diff in pull request #13506: ARROW-16653: [R] All formats are supported with the lubridate `parse_date_time` binding

Posted by GitBox <gi...@apache.org>.
rok commented on code in PR #13506:
URL: https://github.com/apache/arrow/pull/13506#discussion_r927607591


##########
r/tests/testthat/test-dplyr-funcs-datetime.R:
##########
@@ -150,6 +150,30 @@ test_that("strptime", {
     as.POSIXct(tstamp),
     ignore_attr = "tzone"
   )
+
+  tz <- "Pacific/Marquesas"
+  times <- seq(as.POSIXct("1999-02-07", tz = tz), as.POSIXct("2000-01-01", tz = tz), by = "hour")
+  times <- c(as.POSIXct("1999-12-31T12:34:56.01", tz = "UTC"))
+
+  # The following formats are currently not supported by strptime: %q %Op
+  formats <- c("%a", "%A", "%b", "%B", "%d", "%H", "%j", "%m", "%Om", "%T", "%OS", "%I%p",
+               "%S", "%q", "%M", "%p", "%U", "%w", "%W", "%y", "%Y", "%r", "%R", "%T%z")
+  base_format <- c("%Y-%m-%d")
+
+  for (fmt in formats) {
+    fmt <- paste(base_format, fmt)
+    test_df <- tibble::tibble(x = strftime(times, format = fmt))
+    expect_equal(
+      test_df %>%
+        arrow_table() %>%
+          mutate(x = strptime(x, format = fmt)) %>%
+          collect(),
+      test_df %>%
+        mutate(x = strptime(x, format = fmt)) %>%
+        mutate(x = as.POSIXct(x)) %>%

Review Comment:
   No reason, just how I originally wrote it.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [arrow] dragosmg commented on a diff in pull request #13506: ARROW-16653: [R] All formats are supported with the lubridate `parse_date_time` binding

Posted by GitBox <gi...@apache.org>.
dragosmg commented on code in PR #13506:
URL: https://github.com/apache/arrow/pull/13506#discussion_r927934525


##########
r/R/dplyr-datetime-helpers.R:
##########
@@ -166,6 +166,18 @@ build_formats <- function(orders) {
   orders <- gsub("[^A-Za-z]", "", orders)
   orders <- gsub("Y", "y", orders)
 
+  invalid_orders <- nchar(gsub("[ymdqIHMS]", "", orders)) > 0

Review Comment:
   You might need to make this more permissive if you want to include the additional formats.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [arrow] dragosmg commented on a diff in pull request #13506: ARROW-16653: [R] All formats are supported with the lubridate `parse_date_time` binding

Posted by GitBox <gi...@apache.org>.
dragosmg commented on code in PR #13506:
URL: https://github.com/apache/arrow/pull/13506#discussion_r932416919


##########
r/R/dplyr-datetime-helpers.R:
##########
@@ -171,61 +171,35 @@ build_formats <- function(orders) {
   # this is needed for 2 reasons:
   # 1. strptime does not parse "2022-05" -> we add "-01", thus changing the format,
   # 2. for equivalence to lubridate, which parses `ym` to the first day of the month
-  short_orders <- c("ym", "my")
+  short_orders <- c("ym", "my", "yOm", "Omy")
+  quarter_orders <- c("yq", "qy")
 
   if (any(orders %in% short_orders)) {
     orders1 <- setdiff(orders, short_orders)
     orders2 <- intersect(orders, short_orders)
     orders2 <- paste0(orders2, "d")
     orders <- unique(c(orders2, orders1))
   }
-
-  if (any(orders == "yq")) {
-    orders1 <- setdiff(orders, "yq")
-    orders2 <- "ymd"
-    orders <- unique(c(orders1, orders2))
-  }
-
-  if (any(orders == "qy")) {
-    orders1 <- setdiff(orders, "qy")
-    orders2 <- "ymd"
-    orders <- unique(c(orders1, orders2))
+  if (any(orders %in% quarter_orders)) {
+    orders <- c(setdiff(orders, quarter_orders), "ymd")
   }
+  orders <- unique(orders)
 
-  ymd_orders <- c("ymd", "ydm", "mdy", "myd", "dmy", "dym")
-  ymd_hms_orders <- c(
-    "ymd_HMS", "ymd_HM", "ymd_H", "dmy_HMS", "dmy_HM", "dmy_H", "mdy_HMS",
-    "mdy_HM", "mdy_H", "ydm_HMS", "ydm_HM", "ydm_H"
-  )
-  # support "%I" hour formats
-  ymd_ims_orders <- gsub("H", "I", ymd_hms_orders)
-
-  supported_orders <- c(
-    ymd_orders,
-    ymd_hms_orders,
-    gsub("_", " ", ymd_hms_orders), # allow "_", " " and "" as order separators
-    gsub("_", "", ymd_hms_orders),
-    ymd_ims_orders,
-    gsub("_", " ", ymd_ims_orders), # allow "_", " " and "" as order separators
-    gsub("_", "", ymd_ims_orders)
-  )
-
-  unsupported_passed_orders <- setdiff(orders, supported_orders)
-  supported_passed_orders <- intersect(orders, supported_orders)
+  formats_list <- map(orders, build_format_from_order)
+  formats_length <- map(map(formats_list, nchar), max)
+  invalid_orders <- formats_length < 6

Review Comment:
   Agree. That sounds like a good plan



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [arrow] dragosmg commented on a diff in pull request #13506: ARROW-16653: [R] All formats are supported with the lubridate `parse_date_time` binding

Posted by GitBox <gi...@apache.org>.
dragosmg commented on code in PR #13506:
URL: https://github.com/apache/arrow/pull/13506#discussion_r932415495


##########
r/R/dplyr-datetime-helpers.R:
##########
@@ -240,25 +214,24 @@ build_formats <- function(orders) {
 #' @noRd
 build_format_from_order <- function(order) {
   char_list <- list(
-    "y" = c("%y", "%Y"),
-    "m" = c("%m", "%B", "%b"),
-    "d" = "%d",
-    "H" = "%H",
-    "M" = "%M",
-    "S" = "%S",
-    "I" = "%I"
+    "%T" = "%H-%M-%S",
+    "%y" = c("%y", "%Y"),
+    "%m" = c("%m", "%B", "%b"),
+    "%b" = c("%m", "%B", "%b")
   )
 
-  split_order <- strsplit(order, split = "")[[1]]
+  formats <- regmatches(order, gregexpr("(O{0,1}[a-zA-Z])", order))[[1]]
+  formats <- paste0("%", formats)
+  formats <- ifelse(formats %in% names(char_list), char_list[formats], formats)
 
-  outcome <- expand.grid(char_list[split_order])
+  outcome <- expand.grid(formats)
   # we combine formats with and without the "-" separator, we will later
   # coalesce through all of them (benchmarking indicated this is a more
   # computationally efficient approach rather than figuring out if a string has
   # separators or not and applying only )

Review Comment:
   Yep, new comment is good. Thanks!



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [arrow] rok commented on pull request #13506: ARROW-16653: [R] All formats are supported with the lubridate `parse_date_time` binding

Posted by GitBox <gi...@apache.org>.
rok commented on PR #13506:
URL: https://github.com/apache/arrow/pull/13506#issuecomment-1198664639

   @github-actions crossbow submit -g r


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [arrow] rok commented on pull request #13506: ARROW-16653: [R] All formats are supported with the lubridate `parse_date_time` binding

Posted by GitBox <gi...@apache.org>.
rok commented on PR #13506:
URL: https://github.com/apache/arrow/pull/13506#issuecomment-1197585586

   > ```r
   > # all the formatting characters should be supported
   > build_formats("wIpz")
   > #> Error: "wIpz" `orders` not supported in Arrow
   > ```
   
   Added this to tests.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [arrow] github-actions[bot] commented on pull request #13506: ARROW-16653: [R] All formats are supported with the lubridate `parse_date_time` binding

Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on PR #13506:
URL: https://github.com/apache/arrow/pull/13506#issuecomment-1173774739

   https://issues.apache.org/jira/browse/ARROW-16653


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [arrow] rok commented on pull request #13506: ARROW-16653: [R] All formats are supported with the lubridate `parse_date_time` binding

Posted by GitBox <gi...@apache.org>.
rok commented on PR #13506:
URL: https://github.com/apache/arrow/pull/13506#issuecomment-1193656226

   @paleolimbot @dragosmg I've pushed this a bit further and I think most formats are now covered. Could you please check if this makes sense to you?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [arrow] rok commented on a diff in pull request #13506: ARROW-16653: [R] All formats are supported with the lubridate `parse_date_time` binding

Posted by GitBox <gi...@apache.org>.
rok commented on code in PR #13506:
URL: https://github.com/apache/arrow/pull/13506#discussion_r930118337


##########
r/R/dplyr-datetime-helpers.R:
##########
@@ -158,74 +158,40 @@ binding_as_date_numeric <- function(x, origin = "1970-01-01") {
 #'
 #' @noRd
 build_formats <- function(orders) {
-  # only keep the letters and the underscore as separator -> allow the users to
-  # pass strptime-like formats (with "%"). We process the data -> we need to
-  # process the `orders` (even if supplied in the desired format)
-  # Processing is needed (instead of passing
-  # formats as-is) due to the processing of the character vector in parse_date_time()
-  orders <- gsub("[^A-Za-z]", "", orders)
-  orders <- gsub("Y", "y", orders)
-
   # we separate "ym', "my", and "yq" from the rest of the `orders` vector and
   # transform them. `ym` and `yq` -> `ymd` & `my` -> `myd`
   # this is needed for 2 reasons:
   # 1. strptime does not parse "2022-05" -> we add "-01", thus changing the format,
   # 2. for equivalence to lubridate, which parses `ym` to the first day of the month
-  short_orders <- c("ym", "my")
+  short_orders <- c("ym", "my", "Ym", "mY")
+  quarter_orders <- c("yq", "Yq", "qy", "qY")
 
   if (any(orders %in% short_orders)) {
     orders1 <- setdiff(orders, short_orders)
     orders2 <- intersect(orders, short_orders)
     orders2 <- paste0(orders2, "d")
     orders <- unique(c(orders2, orders1))
   }
-
-  if (any(orders == "yq")) {
-    orders1 <- setdiff(orders, "yq")
-    orders2 <- "ymd"
-    orders <- unique(c(orders1, orders2))
+  if (any(orders %in% quarter_orders)) {
+    orders <- c(setdiff(orders, quarter_orders), "ymd")
   }
+  orders <- unique(orders)
 
-  if (any(orders == "qy")) {
-    orders1 <- setdiff(orders, "qy")
-    orders2 <- "ymd"
-    orders <- unique(c(orders1, orders2))
-  }
-
-  ymd_orders <- c("ymd", "ydm", "mdy", "myd", "dmy", "dym")
-  ymd_hms_orders <- c(
-    "ymd_HMS", "ymd_HM", "ymd_H", "dmy_HMS", "dmy_HM", "dmy_H", "mdy_HMS",
-    "mdy_HM", "mdy_H", "ydm_HMS", "ydm_HM", "ydm_H"
-  )
-  # support "%I" hour formats
-  ymd_ims_orders <- gsub("H", "I", ymd_hms_orders)
-
-  supported_orders <- c(
-    ymd_orders,
-    ymd_hms_orders,
-    gsub("_", " ", ymd_hms_orders), # allow "_", " " and "" as order separators
-    gsub("_", "", ymd_hms_orders),
-    ymd_ims_orders,
-    gsub("_", " ", ymd_ims_orders), # allow "_", " " and "" as order separators
-    gsub("_", "", ymd_ims_orders)
-  )
-
-  unsupported_passed_orders <- setdiff(orders, supported_orders)
-  supported_passed_orders <- intersect(orders, supported_orders)
+  formats_list <- map(orders, build_format_from_order)
+  formats_length <- map(map(formats_list, nchar), max)
+  invalid_orders <- formats_length < 6
 
-  # error only if there isn't at least one valid order we can try
-  if (length(supported_passed_orders) == 0) {
+  if (any(invalid_orders)) {
     arrow_not_supported(
       paste0(
         oxford_paste(
-          unsupported_passed_orders
+          orders[invalid_orders]
         ),
         " `orders`"
       )
     )
   }

Review Comment:
   I'll leave this for now as it probably needs to be addressed in c++ as well. I'll open a Jira and refer to this.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [arrow] rok commented on pull request #13506: ARROW-16653: [R] All formats are supported with the lubridate `parse_date_time` binding

Posted by GitBox <gi...@apache.org>.
rok commented on PR #13506:
URL: https://github.com/apache/arrow/pull/13506#issuecomment-1195834863

   @nealrichardson @jonkeane could either of you review this? There's still a small chance to get this into the release.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [arrow] rok commented on pull request #13506: ARROW-16653: [R] All formats are supported with the lubridate `parse_date_time` binding

Posted by GitBox <gi...@apache.org>.
rok commented on PR #13506:
URL: https://github.com/apache/arrow/pull/13506#issuecomment-1192207742

   @dragosmg 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [arrow] paleolimbot commented on pull request #13506: ARROW-16653: [R] All formats are supported with the lubridate `parse_date_time` binding

Posted by GitBox <gi...@apache.org>.
paleolimbot commented on PR #13506:
URL: https://github.com/apache/arrow/pull/13506#issuecomment-1192856368

   @dragosmg Can you create a follow-up ticket for that?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [arrow] github-actions[bot] commented on pull request #13506: ARROW-16653: [R] All formats are supported with the lubridate `parse_date_time` binding

Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on PR #13506:
URL: https://github.com/apache/arrow/pull/13506#issuecomment-1198665487

   Revision: 17b46ab264beb29e1cc73db322c09990e8965df7
   
   Submitted crossbow builds: [ursacomputing/crossbow @ actions-156d8fedd0](https://github.com/ursacomputing/crossbow/branches/all?query=actions-156d8fedd0)
   
   |Task|Status|
   |----|------|
   |conda-linux-gcc-py37-cpu-r40|[![Azure](https://dev.azure.com/ursacomputing/crossbow/_apis/build/status/ursacomputing.crossbow?branchName=actions-156d8fedd0-azure-conda-linux-gcc-py37-cpu-r40)](https://dev.azure.com/ursacomputing/crossbow/_build/latest?definitionId=1&branchName=actions-156d8fedd0-azure-conda-linux-gcc-py37-cpu-r40)|
   |conda-linux-gcc-py37-cpu-r41|[![Azure](https://dev.azure.com/ursacomputing/crossbow/_apis/build/status/ursacomputing.crossbow?branchName=actions-156d8fedd0-azure-conda-linux-gcc-py37-cpu-r41)](https://dev.azure.com/ursacomputing/crossbow/_build/latest?definitionId=1&branchName=actions-156d8fedd0-azure-conda-linux-gcc-py37-cpu-r41)|
   |conda-osx-clang-py37-r40|[![Azure](https://dev.azure.com/ursacomputing/crossbow/_apis/build/status/ursacomputing.crossbow?branchName=actions-156d8fedd0-azure-conda-osx-clang-py37-r40)](https://dev.azure.com/ursacomputing/crossbow/_build/latest?definitionId=1&branchName=actions-156d8fedd0-azure-conda-osx-clang-py37-r40)|
   |conda-osx-clang-py37-r41|[![Azure](https://dev.azure.com/ursacomputing/crossbow/_apis/build/status/ursacomputing.crossbow?branchName=actions-156d8fedd0-azure-conda-osx-clang-py37-r41)](https://dev.azure.com/ursacomputing/crossbow/_build/latest?definitionId=1&branchName=actions-156d8fedd0-azure-conda-osx-clang-py37-r41)|
   |conda-win-vs2017-py37-r40|[![Azure](https://dev.azure.com/ursacomputing/crossbow/_apis/build/status/ursacomputing.crossbow?branchName=actions-156d8fedd0-azure-conda-win-vs2017-py37-r40)](https://dev.azure.com/ursacomputing/crossbow/_build/latest?definitionId=1&branchName=actions-156d8fedd0-azure-conda-win-vs2017-py37-r40)|
   |conda-win-vs2017-py37-r41|[![Azure](https://dev.azure.com/ursacomputing/crossbow/_apis/build/status/ursacomputing.crossbow?branchName=actions-156d8fedd0-azure-conda-win-vs2017-py37-r41)](https://dev.azure.com/ursacomputing/crossbow/_build/latest?definitionId=1&branchName=actions-156d8fedd0-azure-conda-win-vs2017-py37-r41)|
   |homebrew-r-autobrew|[![Github Actions](https://github.com/ursacomputing/crossbow/workflows/Crossbow/badge.svg?branch=actions-156d8fedd0-github-homebrew-r-autobrew)](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-156d8fedd0-github-homebrew-r-autobrew)|
   |homebrew-r-brew|[![Github Actions](https://github.com/ursacomputing/crossbow/workflows/Crossbow/badge.svg?branch=actions-156d8fedd0-github-homebrew-r-brew)](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-156d8fedd0-github-homebrew-r-brew)|
   |r-binary-packages|[![Github Actions](https://github.com/ursacomputing/crossbow/workflows/Crossbow/badge.svg?branch=actions-156d8fedd0-github-r-binary-packages)](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-156d8fedd0-github-r-binary-packages)|
   |test-fedora-r-clang-sanitizer|[![Azure](https://dev.azure.com/ursacomputing/crossbow/_apis/build/status/ursacomputing.crossbow?branchName=actions-156d8fedd0-azure-test-fedora-r-clang-sanitizer)](https://dev.azure.com/ursacomputing/crossbow/_build/latest?definitionId=1&branchName=actions-156d8fedd0-azure-test-fedora-r-clang-sanitizer)|
   |test-r-arrow-backwards-compatibility|[![Github Actions](https://github.com/ursacomputing/crossbow/workflows/Crossbow/badge.svg?branch=actions-156d8fedd0-github-test-r-arrow-backwards-compatibility)](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-156d8fedd0-github-test-r-arrow-backwards-compatibility)|
   |test-r-depsource-bundled|[![Azure](https://dev.azure.com/ursacomputing/crossbow/_apis/build/status/ursacomputing.crossbow?branchName=actions-156d8fedd0-azure-test-r-depsource-bundled)](https://dev.azure.com/ursacomputing/crossbow/_build/latest?definitionId=1&branchName=actions-156d8fedd0-azure-test-r-depsource-bundled)|
   |test-r-depsource-system|[![Github Actions](https://github.com/ursacomputing/crossbow/workflows/Crossbow/badge.svg?branch=actions-156d8fedd0-github-test-r-depsource-system)](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-156d8fedd0-github-test-r-depsource-system)|
   |test-r-dev-duckdb|[![Github Actions](https://github.com/ursacomputing/crossbow/workflows/Crossbow/badge.svg?branch=actions-156d8fedd0-github-test-r-dev-duckdb)](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-156d8fedd0-github-test-r-dev-duckdb)|
   |test-r-devdocs|[![Github Actions](https://github.com/ursacomputing/crossbow/workflows/Crossbow/badge.svg?branch=actions-156d8fedd0-github-test-r-devdocs)](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-156d8fedd0-github-test-r-devdocs)|
   |test-r-gcc-11|[![Github Actions](https://github.com/ursacomputing/crossbow/workflows/Crossbow/badge.svg?branch=actions-156d8fedd0-github-test-r-gcc-11)](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-156d8fedd0-github-test-r-gcc-11)|
   |test-r-gcc-12|[![Github Actions](https://github.com/ursacomputing/crossbow/workflows/Crossbow/badge.svg?branch=actions-156d8fedd0-github-test-r-gcc-12)](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-156d8fedd0-github-test-r-gcc-12)|
   |test-r-install-local|[![Github Actions](https://github.com/ursacomputing/crossbow/workflows/Crossbow/badge.svg?branch=actions-156d8fedd0-github-test-r-install-local)](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-156d8fedd0-github-test-r-install-local)|
   |test-r-linux-as-cran|[![Github Actions](https://github.com/ursacomputing/crossbow/workflows/Crossbow/badge.svg?branch=actions-156d8fedd0-github-test-r-linux-as-cran)](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-156d8fedd0-github-test-r-linux-as-cran)|
   |test-r-linux-rchk|[![Github Actions](https://github.com/ursacomputing/crossbow/workflows/Crossbow/badge.svg?branch=actions-156d8fedd0-github-test-r-linux-rchk)](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-156d8fedd0-github-test-r-linux-rchk)|
   |test-r-linux-valgrind|[![Azure](https://dev.azure.com/ursacomputing/crossbow/_apis/build/status/ursacomputing.crossbow?branchName=actions-156d8fedd0-azure-test-r-linux-valgrind)](https://dev.azure.com/ursacomputing/crossbow/_build/latest?definitionId=1&branchName=actions-156d8fedd0-azure-test-r-linux-valgrind)|
   |test-r-minimal-build|[![Azure](https://dev.azure.com/ursacomputing/crossbow/_apis/build/status/ursacomputing.crossbow?branchName=actions-156d8fedd0-azure-test-r-minimal-build)](https://dev.azure.com/ursacomputing/crossbow/_build/latest?definitionId=1&branchName=actions-156d8fedd0-azure-test-r-minimal-build)|
   |test-r-offline-maximal|[![Github Actions](https://github.com/ursacomputing/crossbow/workflows/Crossbow/badge.svg?branch=actions-156d8fedd0-github-test-r-offline-maximal)](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-156d8fedd0-github-test-r-offline-maximal)|
   |test-r-offline-minimal|[![Azure](https://dev.azure.com/ursacomputing/crossbow/_apis/build/status/ursacomputing.crossbow?branchName=actions-156d8fedd0-azure-test-r-offline-minimal)](https://dev.azure.com/ursacomputing/crossbow/_build/latest?definitionId=1&branchName=actions-156d8fedd0-azure-test-r-offline-minimal)|
   |test-r-rhub-debian-gcc-devel-lto-latest|[![Azure](https://dev.azure.com/ursacomputing/crossbow/_apis/build/status/ursacomputing.crossbow?branchName=actions-156d8fedd0-azure-test-r-rhub-debian-gcc-devel-lto-latest)](https://dev.azure.com/ursacomputing/crossbow/_build/latest?definitionId=1&branchName=actions-156d8fedd0-azure-test-r-rhub-debian-gcc-devel-lto-latest)|
   |test-r-rhub-debian-gcc-release-custom-ccache|[![Azure](https://dev.azure.com/ursacomputing/crossbow/_apis/build/status/ursacomputing.crossbow?branchName=actions-156d8fedd0-azure-test-r-rhub-debian-gcc-release-custom-ccache)](https://dev.azure.com/ursacomputing/crossbow/_build/latest?definitionId=1&branchName=actions-156d8fedd0-azure-test-r-rhub-debian-gcc-release-custom-ccache)|
   |test-r-rhub-ubuntu-gcc-release-latest|[![Azure](https://dev.azure.com/ursacomputing/crossbow/_apis/build/status/ursacomputing.crossbow?branchName=actions-156d8fedd0-azure-test-r-rhub-ubuntu-gcc-release-latest)](https://dev.azure.com/ursacomputing/crossbow/_build/latest?definitionId=1&branchName=actions-156d8fedd0-azure-test-r-rhub-ubuntu-gcc-release-latest)|
   |test-r-rocker-r-base-latest|[![Azure](https://dev.azure.com/ursacomputing/crossbow/_apis/build/status/ursacomputing.crossbow?branchName=actions-156d8fedd0-azure-test-r-rocker-r-base-latest)](https://dev.azure.com/ursacomputing/crossbow/_build/latest?definitionId=1&branchName=actions-156d8fedd0-azure-test-r-rocker-r-base-latest)|
   |test-r-rstudio-r-base-4.1-opensuse153|[![Azure](https://dev.azure.com/ursacomputing/crossbow/_apis/build/status/ursacomputing.crossbow?branchName=actions-156d8fedd0-azure-test-r-rstudio-r-base-4.1-opensuse153)](https://dev.azure.com/ursacomputing/crossbow/_build/latest?definitionId=1&branchName=actions-156d8fedd0-azure-test-r-rstudio-r-base-4.1-opensuse153)|
   |test-r-rstudio-r-base-4.2-centos7-devtoolset-8|[![Azure](https://dev.azure.com/ursacomputing/crossbow/_apis/build/status/ursacomputing.crossbow?branchName=actions-156d8fedd0-azure-test-r-rstudio-r-base-4.2-centos7-devtoolset-8)](https://dev.azure.com/ursacomputing/crossbow/_build/latest?definitionId=1&branchName=actions-156d8fedd0-azure-test-r-rstudio-r-base-4.2-centos7-devtoolset-8)|
   |test-r-rstudio-r-base-4.2-focal|[![Azure](https://dev.azure.com/ursacomputing/crossbow/_apis/build/status/ursacomputing.crossbow?branchName=actions-156d8fedd0-azure-test-r-rstudio-r-base-4.2-focal)](https://dev.azure.com/ursacomputing/crossbow/_build/latest?definitionId=1&branchName=actions-156d8fedd0-azure-test-r-rstudio-r-base-4.2-focal)|
   |test-r-ubuntu-22.04|[![Github Actions](https://github.com/ursacomputing/crossbow/workflows/Crossbow/badge.svg?branch=actions-156d8fedd0-github-test-r-ubuntu-22.04)](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-156d8fedd0-github-test-r-ubuntu-22.04)|
   |test-r-versions|[![Github Actions](https://github.com/ursacomputing/crossbow/workflows/Crossbow/badge.svg?branch=actions-156d8fedd0-github-test-r-versions)](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-156d8fedd0-github-test-r-versions)|
   |test-ubuntu-18.04-r-sanitizer|[![Azure](https://dev.azure.com/ursacomputing/crossbow/_apis/build/status/ursacomputing.crossbow?branchName=actions-156d8fedd0-azure-test-ubuntu-18.04-r-sanitizer)](https://dev.azure.com/ursacomputing/crossbow/_build/latest?definitionId=1&branchName=actions-156d8fedd0-azure-test-ubuntu-18.04-r-sanitizer)|


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [arrow] rok commented on pull request #13506: ARROW-16653: [R] All formats are supported with the lubridate `parse_date_time` binding

Posted by GitBox <gi...@apache.org>.
rok commented on PR #13506:
URL: https://github.com/apache/arrow/pull/13506#issuecomment-1198986302

   Thank you for fast reviews and feedback @dragosmg and @paleolimbot !


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [arrow] rok commented on pull request #13506: ARROW-16653: [R] All formats are supported with the lubridate `parse_date_time` binding

Posted by GitBox <gi...@apache.org>.
rok commented on PR #13506:
URL: https://github.com/apache/arrow/pull/13506#issuecomment-1192867698

   > to clarify, that test would be nice to have, but it doesn't mean that without it the features aren't tested. ultimately, `parse_date_time()` calls `strptime` so, as long as those formats are tested with `strptime` they should work just fine with `parse_date_time()`.
   
   👍 
   The thing is I started looking into frankenformats (e.g. `Y-%b-d`) which are supported by lubridate, but they brake our orders parsing here. I'll try to make it work and make it reject frankenformats if I can't. Does that seem ok?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [arrow] dragosmg commented on pull request #13506: ARROW-16653: [R] All formats are supported with the lubridate `parse_date_time` binding

Posted by GitBox <gi...@apache.org>.
dragosmg commented on PR #13506:
URL: https://github.com/apache/arrow/pull/13506#issuecomment-1192864687

   to clarify, that test would be nice to have, but it doesn't mean that without it the features aren't tested. ultimately, `parse_date_time()` calls `strptime` so, as long as those formats are tested with `strptime` they should work just fine with `parse_date_time()`. 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [arrow] rok commented on a diff in pull request #13506: ARROW-16653: [R] All formats are supported with the lubridate `parse_date_time` binding

Posted by GitBox <gi...@apache.org>.
rok commented on code in PR #13506:
URL: https://github.com/apache/arrow/pull/13506#discussion_r930223895


##########
r/R/dplyr-datetime-helpers.R:
##########
@@ -158,74 +158,40 @@ binding_as_date_numeric <- function(x, origin = "1970-01-01") {
 #'
 #' @noRd
 build_formats <- function(orders) {
-  # only keep the letters and the underscore as separator -> allow the users to
-  # pass strptime-like formats (with "%"). We process the data -> we need to
-  # process the `orders` (even if supplied in the desired format)
-  # Processing is needed (instead of passing
-  # formats as-is) due to the processing of the character vector in parse_date_time()
-  orders <- gsub("[^A-Za-z]", "", orders)
-  orders <- gsub("Y", "y", orders)
-
   # we separate "ym', "my", and "yq" from the rest of the `orders` vector and
   # transform them. `ym` and `yq` -> `ymd` & `my` -> `myd`
   # this is needed for 2 reasons:
   # 1. strptime does not parse "2022-05" -> we add "-01", thus changing the format,
   # 2. for equivalence to lubridate, which parses `ym` to the first day of the month
-  short_orders <- c("ym", "my")
+  short_orders <- c("ym", "my", "Ym", "mY")
+  quarter_orders <- c("yq", "Yq", "qy", "qY")
 
   if (any(orders %in% short_orders)) {
     orders1 <- setdiff(orders, short_orders)
     orders2 <- intersect(orders, short_orders)
     orders2 <- paste0(orders2, "d")
     orders <- unique(c(orders2, orders1))
   }
-
-  if (any(orders == "yq")) {
-    orders1 <- setdiff(orders, "yq")
-    orders2 <- "ymd"
-    orders <- unique(c(orders1, orders2))
+  if (any(orders %in% quarter_orders)) {
+    orders <- c(setdiff(orders, quarter_orders), "ymd")
   }
+  orders <- unique(orders)
 
-  if (any(orders == "qy")) {
-    orders1 <- setdiff(orders, "qy")
-    orders2 <- "ymd"
-    orders <- unique(c(orders1, orders2))
-  }
-
-  ymd_orders <- c("ymd", "ydm", "mdy", "myd", "dmy", "dym")
-  ymd_hms_orders <- c(
-    "ymd_HMS", "ymd_HM", "ymd_H", "dmy_HMS", "dmy_HM", "dmy_H", "mdy_HMS",
-    "mdy_HM", "mdy_H", "ydm_HMS", "ydm_HM", "ydm_H"
-  )
-  # support "%I" hour formats
-  ymd_ims_orders <- gsub("H", "I", ymd_hms_orders)
-
-  supported_orders <- c(
-    ymd_orders,
-    ymd_hms_orders,
-    gsub("_", " ", ymd_hms_orders), # allow "_", " " and "" as order separators
-    gsub("_", "", ymd_hms_orders),
-    ymd_ims_orders,
-    gsub("_", " ", ymd_ims_orders), # allow "_", " " and "" as order separators
-    gsub("_", "", ymd_ims_orders)
-  )
-
-  unsupported_passed_orders <- setdiff(orders, supported_orders)
-  supported_passed_orders <- intersect(orders, supported_orders)
+  formats_list <- map(orders, build_format_from_order)
+  formats_length <- map(map(formats_list, nchar), max)
+  invalid_orders <- formats_length < 6
 
-  # error only if there isn't at least one valid order we can try
-  if (length(supported_passed_orders) == 0) {
+  if (any(invalid_orders)) {
     arrow_not_supported(
       paste0(
         oxford_paste(
-          unsupported_passed_orders
+          orders[invalid_orders]
         ),
         " `orders`"
       )
     )
   }

Review Comment:
   [ARROW-17218](https://issues.apache.org/jira/browse/ARROW-17218).



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [arrow] rok commented on a diff in pull request #13506: ARROW-16653: [R] All formats are supported with the lubridate `parse_date_time` binding

Posted by GitBox <gi...@apache.org>.
rok commented on code in PR #13506:
URL: https://github.com/apache/arrow/pull/13506#discussion_r928528243


##########
r/tests/testthat/test-dplyr-funcs-datetime.R:
##########
@@ -2638,14 +2694,18 @@ test_that("build_formats() and build_format_from_order()", {
     )
   )
 
-  # ab not supported yet
-  expect_error(
-    build_formats("abd"),
-    '"abd" `orders` not supported in Arrow'
+  expect_equal(
+    build_format_from_order("abp"),
+    c("%a", "%b","%p", "%a", "%b", "%p")
+  )
+
+  expect_equal(
+    build_format_from_order("uYM"),
+    c("%y-%M", "%Y-%M", "%y%M", "%Y%M")

Review Comment:
   These are odd. My suspicion is something is not quite right here: https://github.com/apache/arrow/pull/13506/files#diff-8fdef8da9950901a3c1535c97b0370aad5a4fced647e29c76c22bc6114cf4880R222



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [arrow] dragosmg commented on a diff in pull request #13506: ARROW-16653: [R] All formats are supported with the lubridate `parse_date_time` binding

Posted by GitBox <gi...@apache.org>.
dragosmg commented on code in PR #13506:
URL: https://github.com/apache/arrow/pull/13506#discussion_r930901753


##########
r/R/dplyr-datetime-helpers.R:
##########
@@ -240,25 +214,24 @@ build_formats <- function(orders) {
 #' @noRd
 build_format_from_order <- function(order) {
   char_list <- list(
-    "y" = c("%y", "%Y"),
-    "m" = c("%m", "%B", "%b"),
-    "d" = "%d",
-    "H" = "%H",
-    "M" = "%M",
-    "S" = "%S",
-    "I" = "%I"
+    "%T" = "%H-%M-%S",
+    "%y" = c("%y", "%Y"),
+    "%m" = c("%m", "%B", "%b"),
+    "%b" = c("%m", "%B", "%b")
   )
 
-  split_order <- strsplit(order, split = "")[[1]]
+  formats <- regmatches(order, gregexpr("(O{0,1}[a-zA-Z])", order))[[1]]
+  formats <- paste0("%", formats)
+  formats <- ifelse(formats %in% names(char_list), char_list[formats], formats)

Review Comment:
   I think `formats` is type-unstable here; it returns a list or a vector depending.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [arrow] paleolimbot commented on a diff in pull request #13506: ARROW-16653: [R] All formats are supported with the lubridate `parse_date_time` binding

Posted by GitBox <gi...@apache.org>.
paleolimbot commented on code in PR #13506:
URL: https://github.com/apache/arrow/pull/13506#discussion_r930902571


##########
r/tests/testthat/test-dplyr-funcs-datetime.R:
##########
@@ -150,6 +150,71 @@ test_that("strptime", {
     as.POSIXct(tstamp),
     ignore_attr = "tzone"
   )
+
+  tz <- "Pacific/Marquesas"
+  set.seed(42)
+  times <- seq(as.POSIXct("1999-02-07", tz = tz), as.POSIXct("2000-01-01", tz = tz), by = "sec")
+  times <- sample(times, 100)
+
+  # %Op format is currently not supported by strptime
+  # We support a subset of flags on Windows
+  if (tolower(Sys.info()[["sysname"]]) == "windows") {
+    formats <- c(
+      "%d", "%H", "%j", "%m", "%T", "%S", "%q", "%M", "%U", "%w", "%W", "%y", "%Y", "%R"
+    )
+    formats2 <- c("d", "H", "j", "m", "T", "S", "q", "M", "U", "w", "W", "y", "Y", "R")
+  } else {
+    formats <- c(
+      "%a", "%A", "%b", "%B", "%d", "%H", "%j", "%m", "%Om", "%T", "%OS", "%I%p",
+      "%S", "%q", "%M", "%U", "%w", "%W", "%y", "%Y", "%r", "%R", "%T%z"
+    )
+    formats2 <- c(
+      "a", "A", "b", "B", "d", "H", "j", "m", "Om", "T", "OS", "Ip",
+      "S", "q", "M", "U", "w", "W", "y", "Y", "r", "R", "Tz"
+    )
+  }
+
+  base_format <- c("%Y-%m-%d")
+  base_format2 <- c("ymd")

Review Comment:
   ```suggestion
     base_format <- "%Y-%m-%d"
     base_format2 <- "ymd"
   ```



##########
r/R/dplyr-datetime-helpers.R:
##########
@@ -171,61 +171,35 @@ build_formats <- function(orders) {
   # this is needed for 2 reasons:
   # 1. strptime does not parse "2022-05" -> we add "-01", thus changing the format,
   # 2. for equivalence to lubridate, which parses `ym` to the first day of the month
-  short_orders <- c("ym", "my")
+  short_orders <- c("ym", "my", "yOm", "Omy")
+  quarter_orders <- c("yq", "qy")
 
   if (any(orders %in% short_orders)) {
     orders1 <- setdiff(orders, short_orders)
     orders2 <- intersect(orders, short_orders)
     orders2 <- paste0(orders2, "d")
     orders <- unique(c(orders2, orders1))
   }
-
-  if (any(orders == "yq")) {
-    orders1 <- setdiff(orders, "yq")
-    orders2 <- "ymd"
-    orders <- unique(c(orders1, orders2))
-  }
-
-  if (any(orders == "qy")) {
-    orders1 <- setdiff(orders, "qy")
-    orders2 <- "ymd"
-    orders <- unique(c(orders1, orders2))
+  if (any(orders %in% quarter_orders)) {
+    orders <- c(setdiff(orders, quarter_orders), "ymd")
   }
+  orders <- unique(orders)
 
-  ymd_orders <- c("ymd", "ydm", "mdy", "myd", "dmy", "dym")
-  ymd_hms_orders <- c(
-    "ymd_HMS", "ymd_HM", "ymd_H", "dmy_HMS", "dmy_HM", "dmy_H", "mdy_HMS",
-    "mdy_HM", "mdy_H", "ydm_HMS", "ydm_HM", "ydm_H"
-  )
-  # support "%I" hour formats
-  ymd_ims_orders <- gsub("H", "I", ymd_hms_orders)
-
-  supported_orders <- c(
-    ymd_orders,
-    ymd_hms_orders,
-    gsub("_", " ", ymd_hms_orders), # allow "_", " " and "" as order separators
-    gsub("_", "", ymd_hms_orders),
-    ymd_ims_orders,
-    gsub("_", " ", ymd_ims_orders), # allow "_", " " and "" as order separators
-    gsub("_", "", ymd_ims_orders)
-  )
-
-  unsupported_passed_orders <- setdiff(orders, supported_orders)
-  supported_passed_orders <- intersect(orders, supported_orders)
+  formats_list <- map(orders, build_format_from_order)
+  formats_length <- map(map(formats_list, nchar), max)

Review Comment:
   ```suggestion
     formats_length <- map_int(map_int(formats_list, nchar), max)
   ```
   
   ...is almost certainly what you mean here?



##########
r/R/dplyr-datetime-helpers.R:
##########
@@ -240,25 +214,24 @@ build_formats <- function(orders) {
 #' @noRd
 build_format_from_order <- function(order) {
   char_list <- list(
-    "y" = c("%y", "%Y"),
-    "m" = c("%m", "%B", "%b"),
-    "d" = "%d",
-    "H" = "%H",
-    "M" = "%M",
-    "S" = "%S",
-    "I" = "%I"
+    "%T" = "%H-%M-%S",
+    "%y" = c("%y", "%Y"),
+    "%m" = c("%m", "%B", "%b"),
+    "%b" = c("%m", "%B", "%b")
   )
 
-  split_order <- strsplit(order, split = "")[[1]]
+  formats <- regmatches(order, gregexpr("(O{0,1}[a-zA-Z])", order))[[1]]
+  formats <- paste0("%", formats)
+  formats <- ifelse(formats %in% names(char_list), char_list[formats], formats)
 
-  outcome <- expand.grid(char_list[split_order])
+  outcome <- expand.grid(formats)
   # we combine formats with and without the "-" separator, we will later
   # coalesce through all of them (benchmarking indicated this is a more
   # computationally efficient approach rather than figuring out if a string has
   # separators or not and applying only )

Review Comment:
   It might be nice to complete this sentence while you're here



##########
r/R/dplyr-datetime-helpers.R:
##########
@@ -171,61 +171,35 @@ build_formats <- function(orders) {
   # this is needed for 2 reasons:
   # 1. strptime does not parse "2022-05" -> we add "-01", thus changing the format,
   # 2. for equivalence to lubridate, which parses `ym` to the first day of the month
-  short_orders <- c("ym", "my")
+  short_orders <- c("ym", "my", "yOm", "Omy")
+  quarter_orders <- c("yq", "qy")
 
   if (any(orders %in% short_orders)) {
     orders1 <- setdiff(orders, short_orders)
     orders2 <- intersect(orders, short_orders)
     orders2 <- paste0(orders2, "d")
     orders <- unique(c(orders2, orders1))
   }
-
-  if (any(orders == "yq")) {
-    orders1 <- setdiff(orders, "yq")
-    orders2 <- "ymd"
-    orders <- unique(c(orders1, orders2))
-  }
-
-  if (any(orders == "qy")) {
-    orders1 <- setdiff(orders, "qy")
-    orders2 <- "ymd"
-    orders <- unique(c(orders1, orders2))
+  if (any(orders %in% quarter_orders)) {
+    orders <- c(setdiff(orders, quarter_orders), "ymd")
   }
+  orders <- unique(orders)
 
-  ymd_orders <- c("ymd", "ydm", "mdy", "myd", "dmy", "dym")
-  ymd_hms_orders <- c(
-    "ymd_HMS", "ymd_HM", "ymd_H", "dmy_HMS", "dmy_HM", "dmy_H", "mdy_HMS",
-    "mdy_HM", "mdy_H", "ydm_HMS", "ydm_HM", "ydm_H"
-  )
-  # support "%I" hour formats
-  ymd_ims_orders <- gsub("H", "I", ymd_hms_orders)
-
-  supported_orders <- c(
-    ymd_orders,
-    ymd_hms_orders,
-    gsub("_", " ", ymd_hms_orders), # allow "_", " " and "" as order separators
-    gsub("_", "", ymd_hms_orders),
-    ymd_ims_orders,
-    gsub("_", " ", ymd_ims_orders), # allow "_", " " and "" as order separators
-    gsub("_", "", ymd_ims_orders)
-  )
-
-  unsupported_passed_orders <- setdiff(orders, supported_orders)
-  supported_passed_orders <- intersect(orders, supported_orders)
+  formats_list <- map(orders, build_format_from_order)
+  formats_length <- map(map(formats_list, nchar), max)
+  invalid_orders <- formats_length < 6

Review Comment:
   It seems unlikely that < 6 characters is the *only* threshold for an invalid order?



##########
r/R/dplyr-datetime-helpers.R:
##########
@@ -240,25 +214,24 @@ build_formats <- function(orders) {
 #' @noRd
 build_format_from_order <- function(order) {
   char_list <- list(
-    "y" = c("%y", "%Y"),
-    "m" = c("%m", "%B", "%b"),
-    "d" = "%d",
-    "H" = "%H",
-    "M" = "%M",
-    "S" = "%S",
-    "I" = "%I"
+    "%T" = "%H-%M-%S",
+    "%y" = c("%y", "%Y"),
+    "%m" = c("%m", "%B", "%b"),
+    "%b" = c("%m", "%B", "%b")

Review Comment:
   ```suggestion
       "%y" = "%y", "%Y",
       "%m" = "%m", "%B", "%b",
       "%b" = "%m", "%B", "%b"
   ```



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [arrow] dragosmg commented on a diff in pull request #13506: ARROW-16653: [R] All formats are supported with the lubridate `parse_date_time` binding

Posted by GitBox <gi...@apache.org>.
dragosmg commented on code in PR #13506:
URL: https://github.com/apache/arrow/pull/13506#discussion_r932416919


##########
r/R/dplyr-datetime-helpers.R:
##########
@@ -171,61 +171,35 @@ build_formats <- function(orders) {
   # this is needed for 2 reasons:
   # 1. strptime does not parse "2022-05" -> we add "-01", thus changing the format,
   # 2. for equivalence to lubridate, which parses `ym` to the first day of the month
-  short_orders <- c("ym", "my")
+  short_orders <- c("ym", "my", "yOm", "Omy")
+  quarter_orders <- c("yq", "qy")
 
   if (any(orders %in% short_orders)) {
     orders1 <- setdiff(orders, short_orders)
     orders2 <- intersect(orders, short_orders)
     orders2 <- paste0(orders2, "d")
     orders <- unique(c(orders2, orders1))
   }
-
-  if (any(orders == "yq")) {
-    orders1 <- setdiff(orders, "yq")
-    orders2 <- "ymd"
-    orders <- unique(c(orders1, orders2))
-  }
-
-  if (any(orders == "qy")) {
-    orders1 <- setdiff(orders, "qy")
-    orders2 <- "ymd"
-    orders <- unique(c(orders1, orders2))
+  if (any(orders %in% quarter_orders)) {
+    orders <- c(setdiff(orders, quarter_orders), "ymd")
   }
+  orders <- unique(orders)
 
-  ymd_orders <- c("ymd", "ydm", "mdy", "myd", "dmy", "dym")
-  ymd_hms_orders <- c(
-    "ymd_HMS", "ymd_HM", "ymd_H", "dmy_HMS", "dmy_HM", "dmy_H", "mdy_HMS",
-    "mdy_HM", "mdy_H", "ydm_HMS", "ydm_HM", "ydm_H"
-  )
-  # support "%I" hour formats
-  ymd_ims_orders <- gsub("H", "I", ymd_hms_orders)
-
-  supported_orders <- c(
-    ymd_orders,
-    ymd_hms_orders,
-    gsub("_", " ", ymd_hms_orders), # allow "_", " " and "" as order separators
-    gsub("_", "", ymd_hms_orders),
-    ymd_ims_orders,
-    gsub("_", " ", ymd_ims_orders), # allow "_", " " and "" as order separators
-    gsub("_", "", ymd_ims_orders)
-  )
-
-  unsupported_passed_orders <- setdiff(orders, supported_orders)
-  supported_passed_orders <- intersect(orders, supported_orders)
+  formats_list <- map(orders, build_format_from_order)
+  formats_length <- map(map(formats_list, nchar), max)
+  invalid_orders <- formats_length < 6

Review Comment:
   Agree



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [arrow] rok commented on a diff in pull request #13506: ARROW-16653: [R] All formats are supported with the lubridate `parse_date_time` binding

Posted by GitBox <gi...@apache.org>.
rok commented on code in PR #13506:
URL: https://github.com/apache/arrow/pull/13506#discussion_r931723596


##########
r/R/dplyr-datetime-helpers.R:
##########
@@ -171,61 +171,35 @@ build_formats <- function(orders) {
   # this is needed for 2 reasons:
   # 1. strptime does not parse "2022-05" -> we add "-01", thus changing the format,
   # 2. for equivalence to lubridate, which parses `ym` to the first day of the month
-  short_orders <- c("ym", "my")
+  short_orders <- c("ym", "my", "yOm", "Omy")
+  quarter_orders <- c("yq", "qy")
 
   if (any(orders %in% short_orders)) {
     orders1 <- setdiff(orders, short_orders)
     orders2 <- intersect(orders, short_orders)
     orders2 <- paste0(orders2, "d")
     orders <- unique(c(orders2, orders1))
   }
-
-  if (any(orders == "yq")) {
-    orders1 <- setdiff(orders, "yq")
-    orders2 <- "ymd"
-    orders <- unique(c(orders1, orders2))
-  }
-
-  if (any(orders == "qy")) {
-    orders1 <- setdiff(orders, "qy")
-    orders2 <- "ymd"
-    orders <- unique(c(orders1, orders2))
+  if (any(orders %in% quarter_orders)) {
+    orders <- c(setdiff(orders, quarter_orders), "ymd")
   }
+  orders <- unique(orders)
 
-  ymd_orders <- c("ymd", "ydm", "mdy", "myd", "dmy", "dym")
-  ymd_hms_orders <- c(
-    "ymd_HMS", "ymd_HM", "ymd_H", "dmy_HMS", "dmy_HM", "dmy_H", "mdy_HMS",
-    "mdy_HM", "mdy_H", "ydm_HMS", "ydm_HM", "ydm_H"
-  )
-  # support "%I" hour formats
-  ymd_ims_orders <- gsub("H", "I", ymd_hms_orders)
-
-  supported_orders <- c(
-    ymd_orders,
-    ymd_hms_orders,
-    gsub("_", " ", ymd_hms_orders), # allow "_", " " and "" as order separators
-    gsub("_", "", ymd_hms_orders),
-    ymd_ims_orders,
-    gsub("_", " ", ymd_ims_orders), # allow "_", " " and "" as order separators
-    gsub("_", "", ymd_ims_orders)
-  )
-
-  unsupported_passed_orders <- setdiff(orders, supported_orders)
-  supported_passed_orders <- intersect(orders, supported_orders)
+  formats_list <- map(orders, build_format_from_order)
+  formats_length <- map(map(formats_list, nchar), max)
+  invalid_orders <- formats_length < 6

Review Comment:
   I'm not exactly sure how to check if an order is valid as it really comes down to strptime implementation used. I'm switching to detecting if no valid formats are present or not and will check what lubridate does.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [arrow] paleolimbot merged pull request #13506: ARROW-16653: [R] All formats are supported with the lubridate `parse_date_time` binding

Posted by GitBox <gi...@apache.org>.
paleolimbot merged PR #13506:
URL: https://github.com/apache/arrow/pull/13506


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [arrow] dragosmg commented on a diff in pull request #13506: ARROW-16653: [R] All formats are supported with the lubridate `parse_date_time` binding

Posted by GitBox <gi...@apache.org>.
dragosmg commented on code in PR #13506:
URL: https://github.com/apache/arrow/pull/13506#discussion_r930901753


##########
r/R/dplyr-datetime-helpers.R:
##########
@@ -240,25 +214,24 @@ build_formats <- function(orders) {
 #' @noRd
 build_format_from_order <- function(order) {
   char_list <- list(
-    "y" = c("%y", "%Y"),
-    "m" = c("%m", "%B", "%b"),
-    "d" = "%d",
-    "H" = "%H",
-    "M" = "%M",
-    "S" = "%S",
-    "I" = "%I"
+    "%T" = "%H-%M-%S",
+    "%y" = c("%y", "%Y"),
+    "%m" = c("%m", "%B", "%b"),
+    "%b" = c("%m", "%B", "%b")
   )
 
-  split_order <- strsplit(order, split = "")[[1]]
+  formats <- regmatches(order, gregexpr("(O{0,1}[a-zA-Z])", order))[[1]]
+  formats <- paste0("%", formats)
+  formats <- ifelse(formats %in% names(char_list), char_list[formats], formats)

Review Comment:
   I think `formats` is type-unstable here; it returns a list or a vector.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [arrow] rok commented on pull request #13506: ARROW-16653: [R] All formats are supported with the lubridate `parse_date_time` binding

Posted by GitBox <gi...@apache.org>.
rok commented on PR #13506:
URL: https://github.com/apache/arrow/pull/13506#issuecomment-1192775535

   > I think the scope of the Jira is to also test/ check/ make sure all formats are supported with `parse_date_time()`. This means we might need to get rid of the `supported_orders` [list](https://github.com/apache/arrow/blob/010b5921bd33e9849820f981647561f19936b899/r/R/dplyr-datetime-helpers.R#L203-L211) and test they actually work.
   
   I forgot about that! 😄
   Please check [772ee78](https://github.com/apache/arrow/pull/13506/commits/772ee7873a5c27eebce6608afa3f9f232613321e).


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [arrow] dragosmg commented on pull request #13506: ARROW-16653: [R] All formats are supported with the lubridate `parse_date_time` binding

Posted by GitBox <gi...@apache.org>.
dragosmg commented on PR #13506:
URL: https://github.com/apache/arrow/pull/13506#issuecomment-1192790807

   > I forgot about that! 😄 Please check [772ee78](https://github.com/apache/arrow/pull/13506/commits/772ee7873a5c27eebce6608afa3f9f232613321e).
   
   Could you add a test/ tests for `parse_date_time()` with some of the formats / orders this enables? 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [arrow] paleolimbot commented on a diff in pull request #13506: ARROW-16653: [R] All formats are supported with the lubridate `parse_date_time` binding

Posted by GitBox <gi...@apache.org>.
paleolimbot commented on code in PR #13506:
URL: https://github.com/apache/arrow/pull/13506#discussion_r927868176


##########
r/tests/testthat/test-dplyr-funcs-datetime.R:
##########
@@ -152,13 +152,14 @@ test_that("strptime", {
   )
 
   tz <- "Pacific/Marquesas"
-  times <- seq(as.POSIXct("1999-02-07", tz = tz), as.POSIXct("2000-01-01", tz = tz), by = "hour")
-  times <- c(as.POSIXct("1999-12-31T12:34:56.01", tz = "UTC"))
+  set.seed(42)
+  times <- seq(as.POSIXct("1999-02-07", tz = tz), as.POSIXct("2000-01-01", tz = tz), by = "sec")

Review Comment:
   Does this take sufficiently long that you need to sample it (I suspect it doesn't)



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [arrow] dragosmg commented on pull request #13506: ARROW-16653: [R] All formats are supported with the lubridate `parse_date_time` binding

Posted by GitBox <gi...@apache.org>.
dragosmg commented on PR #13506:
URL: https://github.com/apache/arrow/pull/13506#issuecomment-1192858584

   I can, but @rok might be close to a unit test for those additional formats.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [arrow] dragosmg commented on a diff in pull request #13506: ARROW-16653: [R] All formats are supported with the lubridate `parse_date_time` binding

Posted by GitBox <gi...@apache.org>.
dragosmg commented on code in PR #13506:
URL: https://github.com/apache/arrow/pull/13506#discussion_r928538333


##########
r/tests/testthat/test-dplyr-funcs-datetime.R:
##########
@@ -2638,14 +2694,18 @@ test_that("build_formats() and build_format_from_order()", {
     )
   )
 
-  # ab not supported yet
-  expect_error(
-    build_formats("abd"),
-    '"abd" `orders` not supported in Arrow'
+  expect_equal(
+    build_format_from_order("abp"),
+    c("%a", "%b","%p", "%a", "%b", "%p")
+  )
+
+  expect_equal(
+    build_format_from_order("uYM"),
+    c("%y-%M", "%Y-%M", "%y%M", "%Y%M")

Review Comment:
   The processing of `order` happens in `build_formats`, https://github.com/apache/arrow/blob/ef6049a2ee5673d0944a0b4f70ff9c70e0419a22/r/R/dplyr-datetime-helpers.R#L166-L167, so that orders getting to `build_format_from_order()` are "clean". 



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [arrow] rok commented on a diff in pull request #13506: ARROW-16653: [R] All formats are supported with the lubridate `parse_date_time` binding

Posted by GitBox <gi...@apache.org>.
rok commented on code in PR #13506:
URL: https://github.com/apache/arrow/pull/13506#discussion_r930130764


##########
r/R/dplyr-datetime-helpers.R:
##########
@@ -240,25 +206,29 @@ build_formats <- function(orders) {
 #' @noRd
 build_format_from_order <- function(order) {
   char_list <- list(
+    "T" = "%H-%M-%S",

Review Comment:
   I'm ok either way. I've moved things around a bit, do you still think so? If yes I'll move it.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [arrow] dragosmg commented on a diff in pull request #13506: ARROW-16653: [R] All formats are supported with the lubridate `parse_date_time` binding

Posted by GitBox <gi...@apache.org>.
dragosmg commented on code in PR #13506:
URL: https://github.com/apache/arrow/pull/13506#discussion_r928541762


##########
r/R/dplyr-datetime-helpers.R:
##########
@@ -158,74 +158,40 @@ binding_as_date_numeric <- function(x, origin = "1970-01-01") {
 #'
 #' @noRd
 build_formats <- function(orders) {
-  # only keep the letters and the underscore as separator -> allow the users to
-  # pass strptime-like formats (with "%"). We process the data -> we need to
-  # process the `orders` (even if supplied in the desired format)
-  # Processing is needed (instead of passing
-  # formats as-is) due to the processing of the character vector in parse_date_time()
-  orders <- gsub("[^A-Za-z]", "", orders)

Review Comment:
   I'm not quite sure I follow why these 2 lines were removed.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [arrow] paleolimbot commented on a diff in pull request #13506: ARROW-16653: [R] All formats are supported with the lubridate `parse_date_time` binding

Posted by GitBox <gi...@apache.org>.
paleolimbot commented on code in PR #13506:
URL: https://github.com/apache/arrow/pull/13506#discussion_r927583175


##########
r/tests/testthat/test-dplyr-funcs-datetime.R:
##########
@@ -150,6 +150,30 @@ test_that("strptime", {
     as.POSIXct(tstamp),
     ignore_attr = "tzone"
   )
+
+  tz <- "Pacific/Marquesas"
+  times <- seq(as.POSIXct("1999-02-07", tz = tz), as.POSIXct("2000-01-01", tz = tz), by = "hour")
+  times <- c(as.POSIXct("1999-12-31T12:34:56.01", tz = "UTC"))

Review Comment:
   Did you mean to keep this second line in? I'm unclear if you meant to actually test the first line (because it's overwritten here).



##########
r/tests/testthat/test-dplyr-funcs-datetime.R:
##########
@@ -150,6 +150,30 @@ test_that("strptime", {
     as.POSIXct(tstamp),
     ignore_attr = "tzone"
   )
+
+  tz <- "Pacific/Marquesas"
+  times <- seq(as.POSIXct("1999-02-07", tz = tz), as.POSIXct("2000-01-01", tz = tz), by = "hour")
+  times <- c(as.POSIXct("1999-12-31T12:34:56.01", tz = "UTC"))
+
+  # The following formats are currently not supported by strptime: %q %Op
+  formats <- c("%a", "%A", "%b", "%B", "%d", "%H", "%j", "%m", "%Om", "%T", "%OS", "%I%p",
+               "%S", "%q", "%M", "%p", "%U", "%w", "%W", "%y", "%Y", "%r", "%R", "%T%z")

Review Comment:
   ```suggestion
     formats <- c(
       "%a", "%A", "%b", "%B", "%d", "%H", "%j", "%m", "%Om", "%T", "%OS", "%I%p",
       "%S", "%q", "%M", "%p", "%U", "%w", "%W", "%y", "%Y", "%r", "%R", "%T%z"
     )
   ```



##########
r/tests/testthat/test-dplyr-funcs-datetime.R:
##########
@@ -150,6 +150,30 @@ test_that("strptime", {
     as.POSIXct(tstamp),
     ignore_attr = "tzone"
   )
+
+  tz <- "Pacific/Marquesas"
+  times <- seq(as.POSIXct("1999-02-07", tz = tz), as.POSIXct("2000-01-01", tz = tz), by = "hour")
+  times <- c(as.POSIXct("1999-12-31T12:34:56.01", tz = "UTC"))
+
+  # The following formats are currently not supported by strptime: %q %Op
+  formats <- c("%a", "%A", "%b", "%B", "%d", "%H", "%j", "%m", "%Om", "%T", "%OS", "%I%p",
+               "%S", "%q", "%M", "%p", "%U", "%w", "%W", "%y", "%Y", "%r", "%R", "%T%z")
+  base_format <- c("%Y-%m-%d")
+
+  for (fmt in formats) {
+    fmt <- paste(base_format, fmt)
+    test_df <- tibble::tibble(x = strftime(times, format = fmt))
+    expect_equal(
+      test_df %>%
+        arrow_table() %>%
+          mutate(x = strptime(x, format = fmt)) %>%
+          collect(),
+      test_df %>%
+        mutate(x = strptime(x, format = fmt)) %>%
+        mutate(x = as.POSIXct(x)) %>%

Review Comment:
   ```suggestion
           mutate(x = as.POSIXct(strptime(x, format = fmt))) %>%
   ```



##########
r/tests/testthat/test-dplyr-funcs-datetime.R:
##########
@@ -150,6 +150,30 @@ test_that("strptime", {
     as.POSIXct(tstamp),
     ignore_attr = "tzone"
   )
+
+  tz <- "Pacific/Marquesas"
+  times <- seq(as.POSIXct("1999-02-07", tz = tz), as.POSIXct("2000-01-01", tz = tz), by = "hour")
+  times <- c(as.POSIXct("1999-12-31T12:34:56.01", tz = "UTC"))
+
+  # The following formats are currently not supported by strptime: %q %Op
+  formats <- c("%a", "%A", "%b", "%B", "%d", "%H", "%j", "%m", "%Om", "%T", "%OS", "%I%p",
+               "%S", "%q", "%M", "%p", "%U", "%w", "%W", "%y", "%Y", "%r", "%R", "%T%z")
+  base_format <- c("%Y-%m-%d")
+
+  for (fmt in formats) {
+    fmt <- paste(base_format, fmt)
+    test_df <- tibble::tibble(x = strftime(times, format = fmt))
+    expect_equal(
+      test_df %>%
+        arrow_table() %>%
+          mutate(x = strptime(x, format = fmt)) %>%
+          collect(),
+      test_df %>%
+        mutate(x = strptime(x, format = fmt)) %>%
+        mutate(x = as.POSIXct(x)) %>%

Review Comment:
   Why does this need `as.POSIXct()` and the top doesn't? (I'm sure it's fine, I just want to make sure it's not covering up something).



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [arrow] dragosmg commented on pull request #13506: ARROW-16653: [R] All formats are supported with the lubridate `parse_date_time` binding

Posted by GitBox <gi...@apache.org>.
dragosmg commented on PR #13506:
URL: https://github.com/apache/arrow/pull/13506#issuecomment-1192540546

   I think the scope of the Jira is to test/ check/ make sure all formats are supported with `parse_date_time()` too. This means we might need to get rid of the `supported_orders` [list](https://github.com/apache/arrow/blob/010b5921bd33e9849820f981647561f19936b899/r/R/dplyr-datetime-helpers.R#L203-L211) and test they actually work.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [arrow] dragosmg commented on a diff in pull request #13506: ARROW-16653: [R] All formats are supported with the lubridate `parse_date_time` binding

Posted by GitBox <gi...@apache.org>.
dragosmg commented on code in PR #13506:
URL: https://github.com/apache/arrow/pull/13506#discussion_r928534338


##########
r/R/dplyr-datetime-helpers.R:
##########
@@ -158,74 +158,40 @@ binding_as_date_numeric <- function(x, origin = "1970-01-01") {
 #'
 #' @noRd
 build_formats <- function(orders) {
-  # only keep the letters and the underscore as separator -> allow the users to
-  # pass strptime-like formats (with "%"). We process the data -> we need to
-  # process the `orders` (even if supplied in the desired format)
-  # Processing is needed (instead of passing
-  # formats as-is) due to the processing of the character vector in parse_date_time()
-  orders <- gsub("[^A-Za-z]", "", orders)
-  orders <- gsub("Y", "y", orders)
-
   # we separate "ym', "my", and "yq" from the rest of the `orders` vector and
   # transform them. `ym` and `yq` -> `ymd` & `my` -> `myd`
   # this is needed for 2 reasons:
   # 1. strptime does not parse "2022-05" -> we add "-01", thus changing the format,
   # 2. for equivalence to lubridate, which parses `ym` to the first day of the month
-  short_orders <- c("ym", "my")
+  short_orders <- c("ym", "my", "Ym", "mY")
+  quarter_orders <- c("yq", "Yq", "qy", "qY")
 
   if (any(orders %in% short_orders)) {
     orders1 <- setdiff(orders, short_orders)
     orders2 <- intersect(orders, short_orders)
     orders2 <- paste0(orders2, "d")
     orders <- unique(c(orders2, orders1))
   }
-
-  if (any(orders == "yq")) {
-    orders1 <- setdiff(orders, "yq")
-    orders2 <- "ymd"
-    orders <- unique(c(orders1, orders2))
+  if (any(orders %in% quarter_orders)) {
+    orders <- c(setdiff(orders, quarter_orders), "ymd")
   }
+  orders <- unique(orders)
 
-  if (any(orders == "qy")) {
-    orders1 <- setdiff(orders, "qy")
-    orders2 <- "ymd"
-    orders <- unique(c(orders1, orders2))
-  }
-
-  ymd_orders <- c("ymd", "ydm", "mdy", "myd", "dmy", "dym")
-  ymd_hms_orders <- c(
-    "ymd_HMS", "ymd_HM", "ymd_H", "dmy_HMS", "dmy_HM", "dmy_H", "mdy_HMS",
-    "mdy_HM", "mdy_H", "ydm_HMS", "ydm_HM", "ydm_H"
-  )
-  # support "%I" hour formats
-  ymd_ims_orders <- gsub("H", "I", ymd_hms_orders)
-
-  supported_orders <- c(
-    ymd_orders,
-    ymd_hms_orders,
-    gsub("_", " ", ymd_hms_orders), # allow "_", " " and "" as order separators
-    gsub("_", "", ymd_hms_orders),
-    ymd_ims_orders,
-    gsub("_", " ", ymd_ims_orders), # allow "_", " " and "" as order separators
-    gsub("_", "", ymd_ims_orders)
-  )
-
-  unsupported_passed_orders <- setdiff(orders, supported_orders)
-  supported_passed_orders <- intersect(orders, supported_orders)
+  formats_list <- map(orders, build_format_from_order)
+  formats_length <- map(map(formats_list, nchar), max)
+  invalid_orders <- formats_length < 6
 
-  # error only if there isn't at least one valid order we can try
-  if (length(supported_passed_orders) == 0) {
+  if (any(invalid_orders)) {
     arrow_not_supported(
       paste0(
         oxford_paste(
-          unsupported_passed_orders
+          orders[invalid_orders]
         ),
         " `orders`"
       )
     )
   }

Review Comment:
   If you end up removing it completely - which I think is the way to go - could you please also update the description? https://github.com/apache/arrow/blob/ef6049a2ee5673d0944a0b4f70ff9c70e0419a22/r/R/dplyr-datetime-helpers.R#L152-L153, we should probably remove:
   ```r
   #' ... it also checks if the supplied
   #' orders are currently supported. 
   ```



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [arrow] rok commented on a diff in pull request #13506: ARROW-16653: [R] All formats are supported with the lubridate `parse_date_time` binding

Posted by GitBox <gi...@apache.org>.
rok commented on code in PR #13506:
URL: https://github.com/apache/arrow/pull/13506#discussion_r927656598


##########
r/tests/testthat/test-dplyr-funcs-datetime.R:
##########
@@ -150,6 +150,30 @@ test_that("strptime", {
     as.POSIXct(tstamp),
     ignore_attr = "tzone"
   )
+
+  tz <- "Pacific/Marquesas"
+  times <- seq(as.POSIXct("1999-02-07", tz = tz), as.POSIXct("2000-01-01", tz = tz), by = "hour")
+  times <- c(as.POSIXct("1999-12-31T12:34:56.01", tz = "UTC"))

Review Comment:
   Done.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [arrow] rok commented on a diff in pull request #13506: ARROW-16653: [R] All formats are supported with the lubridate `parse_date_time` binding

Posted by GitBox <gi...@apache.org>.
rok commented on code in PR #13506:
URL: https://github.com/apache/arrow/pull/13506#discussion_r927607032


##########
r/tests/testthat/test-dplyr-funcs-datetime.R:
##########
@@ -150,6 +150,30 @@ test_that("strptime", {
     as.POSIXct(tstamp),
     ignore_attr = "tzone"
   )
+
+  tz <- "Pacific/Marquesas"
+  times <- seq(as.POSIXct("1999-02-07", tz = tz), as.POSIXct("2000-01-01", tz = tz), by = "hour")
+  times <- c(as.POSIXct("1999-12-31T12:34:56.01", tz = "UTC"))

Review Comment:
   Ooop. Will fix.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [arrow] dragosmg commented on a diff in pull request #13506: ARROW-16653: [R] All formats are supported with the lubridate `parse_date_time` binding

Posted by GitBox <gi...@apache.org>.
dragosmg commented on code in PR #13506:
URL: https://github.com/apache/arrow/pull/13506#discussion_r928568182


##########
r/R/dplyr-datetime-helpers.R:
##########
@@ -158,74 +158,40 @@ binding_as_date_numeric <- function(x, origin = "1970-01-01") {
 #'
 #' @noRd
 build_formats <- function(orders) {
-  # only keep the letters and the underscore as separator -> allow the users to
-  # pass strptime-like formats (with "%"). We process the data -> we need to
-  # process the `orders` (even if supplied in the desired format)
-  # Processing is needed (instead of passing
-  # formats as-is) due to the processing of the character vector in parse_date_time()
-  orders <- gsub("[^A-Za-z]", "", orders)
-  orders <- gsub("Y", "y", orders)
-
   # we separate "ym', "my", and "yq" from the rest of the `orders` vector and
   # transform them. `ym` and `yq` -> `ymd` & `my` -> `myd`
   # this is needed for 2 reasons:
   # 1. strptime does not parse "2022-05" -> we add "-01", thus changing the format,
   # 2. for equivalence to lubridate, which parses `ym` to the first day of the month
-  short_orders <- c("ym", "my")
+  short_orders <- c("ym", "my", "Ym", "mY")
+  quarter_orders <- c("yq", "Yq", "qy", "qY")
 
   if (any(orders %in% short_orders)) {
     orders1 <- setdiff(orders, short_orders)
     orders2 <- intersect(orders, short_orders)
     orders2 <- paste0(orders2, "d")
     orders <- unique(c(orders2, orders1))
   }
-
-  if (any(orders == "yq")) {
-    orders1 <- setdiff(orders, "yq")
-    orders2 <- "ymd"
-    orders <- unique(c(orders1, orders2))
+  if (any(orders %in% quarter_orders)) {
+    orders <- c(setdiff(orders, quarter_orders), "ymd")
   }
+  orders <- unique(orders)
 
-  if (any(orders == "qy")) {
-    orders1 <- setdiff(orders, "qy")
-    orders2 <- "ymd"
-    orders <- unique(c(orders1, orders2))
-  }
-
-  ymd_orders <- c("ymd", "ydm", "mdy", "myd", "dmy", "dym")
-  ymd_hms_orders <- c(
-    "ymd_HMS", "ymd_HM", "ymd_H", "dmy_HMS", "dmy_HM", "dmy_H", "mdy_HMS",
-    "mdy_HM", "mdy_H", "ydm_HMS", "ydm_HM", "ydm_H"
-  )
-  # support "%I" hour formats
-  ymd_ims_orders <- gsub("H", "I", ymd_hms_orders)
-
-  supported_orders <- c(
-    ymd_orders,
-    ymd_hms_orders,
-    gsub("_", " ", ymd_hms_orders), # allow "_", " " and "" as order separators
-    gsub("_", "", ymd_hms_orders),
-    ymd_ims_orders,
-    gsub("_", " ", ymd_ims_orders), # allow "_", " " and "" as order separators
-    gsub("_", "", ymd_ims_orders)
-  )
-
-  unsupported_passed_orders <- setdiff(orders, supported_orders)
-  supported_passed_orders <- intersect(orders, supported_orders)
+  formats_list <- map(orders, build_format_from_order)
+  formats_length <- map(map(formats_list, nchar), max)
+  invalid_orders <- formats_length < 6
 
-  # error only if there isn't at least one valid order we can try
-  if (length(supported_passed_orders) == 0) {
+  if (any(invalid_orders)) {
     arrow_not_supported(
       paste0(
         oxford_paste(
-          unsupported_passed_orders
+          orders[invalid_orders]
         ),
         " `orders`"
       )
     )
   }
 
-  formats_list <- map(orders, build_format_from_order)

Review Comment:
   Iterating through the various orders and build the corresponding formats is needed. 



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [arrow] rok commented on a diff in pull request #13506: ARROW-16653: [R] All formats are supported with the lubridate `parse_date_time` binding

Posted by GitBox <gi...@apache.org>.
rok commented on code in PR #13506:
URL: https://github.com/apache/arrow/pull/13506#discussion_r931729718


##########
r/R/dplyr-datetime-helpers.R:
##########
@@ -240,25 +214,24 @@ build_formats <- function(orders) {
 #' @noRd
 build_format_from_order <- function(order) {
   char_list <- list(
-    "y" = c("%y", "%Y"),
-    "m" = c("%m", "%B", "%b"),
-    "d" = "%d",
-    "H" = "%H",
-    "M" = "%M",
-    "S" = "%S",
-    "I" = "%I"
+    "%T" = "%H-%M-%S",
+    "%y" = c("%y", "%Y"),
+    "%m" = c("%m", "%B", "%b"),
+    "%b" = c("%m", "%B", "%b")
   )
 
-  split_order <- strsplit(order, split = "")[[1]]
+  formats <- regmatches(order, gregexpr("(O{0,1}[a-zA-Z])", order))[[1]]

Review Comment:
   Switching back to older implementation.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [arrow] rok commented on a diff in pull request #13506: ARROW-16653: [R] All formats are supported with the lubridate `parse_date_time` binding

Posted by GitBox <gi...@apache.org>.
rok commented on code in PR #13506:
URL: https://github.com/apache/arrow/pull/13506#discussion_r931723894


##########
r/R/dplyr-datetime-helpers.R:
##########
@@ -240,25 +214,24 @@ build_formats <- function(orders) {
 #' @noRd
 build_format_from_order <- function(order) {
   char_list <- list(
-    "y" = c("%y", "%Y"),
-    "m" = c("%m", "%B", "%b"),
-    "d" = "%d",
-    "H" = "%H",
-    "M" = "%M",
-    "S" = "%S",
-    "I" = "%I"
+    "%T" = "%H-%M-%S",
+    "%y" = c("%y", "%Y"),
+    "%m" = c("%m", "%B", "%b"),
+    "%b" = c("%m", "%B", "%b")
   )
 
-  split_order <- strsplit(order, split = "")[[1]]
+  formats <- regmatches(order, gregexpr("(O{0,1}[a-zA-Z])", order))[[1]]
+  formats <- paste0("%", formats)
+  formats <- ifelse(formats %in% names(char_list), char_list[formats], formats)
 
-  outcome <- expand.grid(char_list[split_order])
+  outcome <- expand.grid(formats)
   # we combine formats with and without the "-" separator, we will later
   # coalesce through all of them (benchmarking indicated this is a more
   # computationally efficient approach rather than figuring out if a string has
   # separators or not and applying only )

Review Comment:
   Done, @dragosmg does the new comment make sense?



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [arrow] rok commented on a diff in pull request #13506: ARROW-16653: [R] All formats are supported with the lubridate `parse_date_time` binding

Posted by GitBox <gi...@apache.org>.
rok commented on code in PR #13506:
URL: https://github.com/apache/arrow/pull/13506#discussion_r930210160


##########
r/R/dplyr-datetime-helpers.R:
##########
@@ -240,25 +214,24 @@ build_formats <- function(orders) {
 #' @noRd
 build_format_from_order <- function(order) {
   char_list <- list(
-    "y" = c("%y", "%Y"),
-    "m" = c("%m", "%B", "%b"),
-    "d" = "%d",
-    "H" = "%H",
-    "M" = "%M",
-    "S" = "%S",
-    "I" = "%I"
+    "%T" = "%H-%M-%S",
+    "%y" = c("%y", "%Y"),
+    "%m" = c("%m", "%B", "%b"),
+    "%b" = c("%m", "%B", "%b")
   )
 
-  split_order <- strsplit(order, split = "")[[1]]
+  formats <- stringr::str_extract_all(order, "%{0,1}(O*[a-zA-Z])")[[1]]

Review Comment:
   Indeed `%` is already removed in `build_formats`, removing.
   Also switching `str_extract_all` to:
   ```R
   formats <- regmatches(order, gregexpr("(O{0,1}[a-zA-Z])", order))[[1]]
   ```
   
   I do think we need regex to keep `O{format_letter}` formats intact (otherwise `"Om"` -> `c("O", "m")`).
   Lubridate also [does test for the O prefix](https://github.com/tidyverse/lubridate/blob/main/tests/testthat/test-parsers.R#L807).



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org