You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@arrow.apache.org by "hamgamb (via GitHub)" <gi...@apache.org> on 2024/01/23 04:05:04 UTC
[I] Inconsistent behaviour of lubridate::dmy on Windows. [arrow]
hamgamb opened a new issue, #39754:
URL: https://github.com/apache/arrow/issues/39754
### Describe the bug, including details regarding any error messages, version, and platform.
``` r
library(arrow)
#> Warning: package 'arrow' was built under R version 4.3.2
#>
#> Attaching package: 'arrow'
#> The following object is masked from 'package:utils':
#>
#> timestamp
library(lubridate)
#>
#> Attaching package: 'lubridate'
#> The following object is masked from 'package:arrow':
#>
#> duration
#> The following objects are masked from 'package:base':
#>
#> date, intersect, setdiff, union
library(dplyr)
#>
#> Attaching package: 'dplyr'
#> The following objects are masked from 'package:stats':
#>
#> filter, lag
#> The following objects are masked from 'package:base':
#>
#> intersect, setdiff, setequal, union
arrow_table(date = "12JAN2004") %>%
mutate(date = dmy(date)) %>%
collect()
#> # A tibble: 1 × 1
#> date
#> <date>
#> 1 NA
arrow_table(date = "12JAN2004") %>%
collect() %>%
mutate(date = dmy(date))
#> # A tibble: 1 × 1
#> date
#> <date>
#> 1 2004-01-12
```
<sup>Created on 2024-01-23 with [reprex v2.0.2](https://reprex.tidyverse.org)</sup>
<details style="margin-bottom:10px;">
<summary>
Session info
</summary>
``` r
sessioninfo::session_info()
#> ─ Session info ───────────────────────────────────────────────────────────────
#> setting value
#> version R version 4.3.1 (2023-06-16 ucrt)
#> os Windows 10 x64 (build 19045)
#> system x86_64, mingw32
#> ui RTerm
#> language (EN)
#> collate English_Australia.utf8
#> ctype English_Australia.utf8
#> tz Australia/Adelaide
#> date 2024-01-23
#> pandoc 3.1.1 @ C:/Users/gamb0043/RStudio/resources/app/bin/quarto/bin/tools/ (via rmarkdown)
#>
#> ─ Packages ───────────────────────────────────────────────────────────────────
#> package * version date (UTC) lib source
#> arrow * 14.0.0.2 2023-12-02 [1] CRAN (R 4.3.2)
#> assertthat 0.2.1 2019-03-21 [1] CRAN (R 4.3.1)
#> bit 4.0.5 2022-11-15 [1] CRAN (R 4.3.1)
#> bit64 4.0.5 2020-08-30 [1] CRAN (R 4.3.1)
#> cli 3.6.2 2023-12-11 [1] CRAN (R 4.3.2)
#> digest 0.6.34 2024-01-11 [1] CRAN (R 4.3.2)
#> dplyr * 1.1.2 2023-04-20 [1] CRAN (R 4.3.1)
#> evaluate 0.23 2023-11-01 [1] CRAN (R 4.3.2)
#> fansi 1.0.6 2023-12-08 [1] CRAN (R 4.3.2)
#> fastmap 1.1.1 2023-02-24 [1] CRAN (R 4.3.1)
#> fs 1.6.3 2023-07-20 [1] CRAN (R 4.3.1)
#> generics 0.1.3 2022-07-05 [1] CRAN (R 4.0.2)
#> glue 1.7.0 2024-01-09 [1] CRAN (R 4.3.2)
#> htmltools 0.5.7 2023-11-03 [1] CRAN (R 4.3.2)
#> knitr 1.45 2023-10-30 [1] CRAN (R 4.3.2)
#> lifecycle 1.0.4 2023-11-07 [1] CRAN (R 4.3.2)
#> lubridate * 1.9.2 2023-02-10 [1] CRAN (R 4.3.1)
#> magrittr 2.0.3 2022-03-30 [1] CRAN (R 4.0.5)
#> pillar 1.9.0 2023-03-22 [1] CRAN (R 4.0.2)
#> pkgconfig 2.0.3 2019-09-22 [1] CRAN (R 4.3.1)
#> purrr 1.0.2 2023-08-10 [1] CRAN (R 4.3.1)
#> R.cache 0.16.0 2022-07-21 [1] CRAN (R 4.3.2)
#> R.methodsS3 1.8.2 2022-06-13 [1] CRAN (R 4.3.1)
#> R.oo 1.25.0 2022-06-12 [1] CRAN (R 4.3.1)
#> R.utils 2.12.2 2022-11-11 [1] CRAN (R 4.3.1)
#> R6 2.5.1 2021-08-19 [1] CRAN (R 4.3.1)
#> reprex 2.0.2 2022-08-17 [1] CRAN (R 4.3.1)
#> rlang 1.1.3 2024-01-10 [1] CRAN (R 4.3.2)
#> rmarkdown 2.25 2023-09-18 [1] CRAN (R 4.3.1)
#> rstudioapi 0.15.0 2023-07-07 [1] CRAN (R 4.3.1)
#> sessioninfo 1.2.2 2021-12-06 [1] CRAN (R 4.3.1)
#> styler 1.10.2 2023-08-29 [1] CRAN (R 4.3.2)
#> tibble 3.2.1 2023-03-20 [1] CRAN (R 4.3.1)
#> tidyselect 1.2.0 2022-10-10 [1] CRAN (R 4.0.2)
#> timechange 0.2.0 2023-01-11 [1] CRAN (R 4.3.1)
#> tzdb 0.4.0 2023-05-12 [1] CRAN (R 4.3.1)
#> utf8 1.2.4 2023-10-22 [1] CRAN (R 4.3.2)
#> vctrs 0.6.5 2023-12-01 [1] CRAN (R 4.3.2)
#> withr 2.5.2 2023-10-30 [1] CRAN (R 4.3.2)
#> xfun 0.41 2023-11-01 [1] CRAN (R 4.3.2)
#> yaml 2.3.8 2023-12-11 [1] CRAN (R 4.3.2)
#>
#> [1] C:/Users/gamb0043/R
#> [2] C:/Program Files/R/R-4.3.1/library
#>
#> ──────────────────────────────────────────────────────────────────────────────
```
</details>
### Component(s)
R
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@arrow.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [R] Docs are not clear on expected behaviour of date parsing functions (e.g. dmy()) on Windows vs. Linux/MacOS [arrow]
Posted by "thisisnic (via GitHub)" <gi...@apache.org>.
thisisnic commented on issue #39754:
URL: https://github.com/apache/arrow/issues/39754#issuecomment-4669694438
Issue resolved by pull request 49708
https://github.com/apache/arrow/pull/49708
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [R] Inconsistent behaviour of lubridate::dmy on Windows. [arrow]
Posted by "hamgamb (via GitHub)" <gi...@apache.org>.
hamgamb commented on issue #39754:
URL: https://github.com/apache/arrow/issues/39754#issuecomment-1905294446
From others I've spoken to, this isn't reproducible on Mac either. When you say other conversions, do you mean same string format, different methods (as below?)
``` r
library(arrow)
#> Warning: package 'arrow' was built under R version 4.3.2
#>
#> Attaching package: 'arrow'
#> The following object is masked from 'package:utils':
#>
#> timestamp
library(lubridate)
#>
#> Attaching package: 'lubridate'
#> The following object is masked from 'package:arrow':
#>
#> duration
#> The following objects are masked from 'package:base':
#>
#> date, intersect, setdiff, union
library(dplyr)
#>
#> Attaching package: 'dplyr'
#> The following objects are masked from 'package:stats':
#>
#> filter, lag
#> The following objects are masked from 'package:base':
#>
#> intersect, setdiff, setequal, union
arrow_table(date = "12JAN2004") |>
mutate(date = dmy(date)) |>
collect()
#> # A tibble: 1 × 1
#> date
#> <date>
#> 1 NA
arrow_table(date = "12JAN2004") |>
mutate(date = as.Date(date, format = "%d%B%Y")) |>
collect()
#> Error in `compute.arrow_dplyr_query()`:
#> ! Invalid: Failed to parse string: '12JAN2004' as a scalar of type timestamp[s]
#> Backtrace:
#> ▆
#> 1. ├─dplyr::collect(...)
#> 2. └─arrow:::collect.arrow_dplyr_query(...)
#> 3. └─arrow:::compute.arrow_dplyr_query(x)
#> 4. └─base::tryCatch(...)
#> 5. └─base (local) tryCatchList(expr, classes, parentenv, handlers)
#> 6. └─base (local) tryCatchOne(expr, names, parentenv, handlers[[1L]])
#> 7. └─value[[3L]](cond)
#> 8. └─arrow:::augment_io_error_msg(e, call, schema = schema())
#> 9. └─rlang::abort(msg, call = call)
arrow_table(date = "12JAN2004") |>
mutate(date = as_date(date, format = "%d%B%Y")) |>
collect()
#> Error in `compute.arrow_dplyr_query()`:
#> ! Invalid: Failed to parse string: '12JAN2004' as a scalar of type timestamp[s]
#> Backtrace:
#> ▆
#> 1. ├─dplyr::collect(...)
#> 2. └─arrow:::collect.arrow_dplyr_query(...)
#> 3. └─arrow:::compute.arrow_dplyr_query(x)
#> 4. └─base::tryCatch(...)
#> 5. └─base (local) tryCatchList(expr, classes, parentenv, handlers)
#> 6. └─base (local) tryCatchOne(expr, names, parentenv, handlers[[1L]])
#> 7. └─value[[3L]](cond)
#> 8. └─arrow:::augment_io_error_msg(e, call, schema = schema())
#> 9. └─rlang::abort(msg, call = call)
```
<sup>Created on 2024-01-23 with [reprex v2.0.2](https://reprex.tidyverse.org)</sup>
<details style="margin-bottom:10px;">
<summary>
Session info
</summary>
``` r
sessioninfo::session_info()
#> ─ Session info ───────────────────────────────────────────────────────────────
#> setting value
#> version R version 4.3.1 (2023-06-16 ucrt)
#> os Windows 10 x64 (build 19045)
#> system x86_64, mingw32
#> ui RTerm
#> language (EN)
#> collate English_Australia.utf8
#> ctype English_Australia.utf8
#> tz Australia/Adelaide
#> date 2024-01-23
#> pandoc 3.1.1 @ C:/Users/gamb0043/RStudio/resources/app/bin/quarto/bin/tools/ (via rmarkdown)
#>
#> ─ Packages ───────────────────────────────────────────────────────────────────
#> package * version date (UTC) lib source
#> arrow * 14.0.0.2 2023-12-02 [1] CRAN (R 4.3.2)
#> assertthat 0.2.1 2019-03-21 [1] CRAN (R 4.3.1)
#> bit 4.0.5 2022-11-15 [1] CRAN (R 4.3.1)
#> bit64 4.0.5 2020-08-30 [1] CRAN (R 4.3.1)
#> cli 3.6.2 2023-12-11 [1] CRAN (R 4.3.2)
#> digest 0.6.34 2024-01-11 [1] CRAN (R 4.3.2)
#> dplyr * 1.1.2 2023-04-20 [1] CRAN (R 4.3.1)
#> evaluate 0.23 2023-11-01 [1] CRAN (R 4.3.2)
#> fansi 1.0.6 2023-12-08 [1] CRAN (R 4.3.2)
#> fastmap 1.1.1 2023-02-24 [1] CRAN (R 4.3.1)
#> fs 1.6.3 2023-07-20 [1] CRAN (R 4.3.1)
#> generics 0.1.3 2022-07-05 [1] CRAN (R 4.0.2)
#> glue 1.7.0 2024-01-09 [1] CRAN (R 4.3.2)
#> htmltools 0.5.7 2023-11-03 [1] CRAN (R 4.3.2)
#> knitr 1.45 2023-10-30 [1] CRAN (R 4.3.2)
#> lifecycle 1.0.4 2023-11-07 [1] CRAN (R 4.3.2)
#> lubridate * 1.9.2 2023-02-10 [1] CRAN (R 4.3.1)
#> magrittr 2.0.3 2022-03-30 [1] CRAN (R 4.0.5)
#> pillar 1.9.0 2023-03-22 [1] CRAN (R 4.0.2)
#> pkgconfig 2.0.3 2019-09-22 [1] CRAN (R 4.3.1)
#> purrr 1.0.2 2023-08-10 [1] CRAN (R 4.3.1)
#> R.cache 0.16.0 2022-07-21 [1] CRAN (R 4.3.2)
#> R.methodsS3 1.8.2 2022-06-13 [1] CRAN (R 4.3.1)
#> R.oo 1.25.0 2022-06-12 [1] CRAN (R 4.3.1)
#> R.utils 2.12.2 2022-11-11 [1] CRAN (R 4.3.1)
#> R6 2.5.1 2021-08-19 [1] CRAN (R 4.3.1)
#> reprex 2.0.2 2022-08-17 [1] CRAN (R 4.3.1)
#> rlang 1.1.3 2024-01-10 [1] CRAN (R 4.3.2)
#> rmarkdown 2.25 2023-09-18 [1] CRAN (R 4.3.1)
#> rstudioapi 0.15.0 2023-07-07 [1] CRAN (R 4.3.1)
#> sessioninfo 1.2.2 2021-12-06 [1] CRAN (R 4.3.1)
#> styler 1.10.2 2023-08-29 [1] CRAN (R 4.3.2)
#> tibble 3.2.1 2023-03-20 [1] CRAN (R 4.3.1)
#> tidyselect 1.2.0 2022-10-10 [1] CRAN (R 4.0.2)
#> timechange 0.2.0 2023-01-11 [1] CRAN (R 4.3.1)
#> tzdb 0.4.0 2023-05-12 [1] CRAN (R 4.3.1)
#> utf8 1.2.4 2023-10-22 [1] CRAN (R 4.3.2)
#> vctrs 0.6.5 2023-12-01 [1] CRAN (R 4.3.2)
#> withr 2.5.2 2023-10-30 [1] CRAN (R 4.3.2)
#> xfun 0.41 2023-11-01 [1] CRAN (R 4.3.2)
#> yaml 2.3.8 2023-12-11 [1] CRAN (R 4.3.2)
#>
#> [1] C:/Users/gamb0043/R
#> [2] C:/Program Files/R/R-4.3.1/library
#>
#> ──────────────────────────────────────────────────────────────────────────────
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [R] Inconsistent behaviour of lubridate::dmy on Windows. [arrow]
Posted by "thisisnic (via GitHub)" <gi...@apache.org>.
thisisnic commented on issue #39754:
URL: https://github.com/apache/arrow/issues/39754#issuecomment-1917056416
What's happening here is that "12JAN2004" is in the format which lubridate refers to as dBY or dbY (see `?lubridate::parse_date_time` for the full spec). The `dmy()` binding in the arrow package is a wrapper around the `parse_date_time()` binding.
In our docs for `parse_date_time()`, it notes that "parse_date_time(): quiet = FALSE is not supported Available formats are H, I, j, M, S, U, w, W, y, Y, R, T. On Linux and OS X additionally a, A, b, B, Om, p, r are available." Therefore on Windows, we wouldn't expect that code to work as b and B are not supported. I assume this is due to tzdata on Windows (which will be doing the parsing in the background) as noted by @assignUser.
Therefore, this isn't a bug, but is expected behaviour. We should update our docs to make it easier to find this information though, as it's not immediately obvious to find without knowing that `dmy()` calls `parse_date_time()`.
Thanks for reporting this @hamgamb!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Inconsistent behaviour of lubridate::dmy on Windows. [arrow]
Posted by "assignUser (via GitHub)" <gi...@apache.org>.
assignUser commented on issue #39754:
URL: https://github.com/apache/arrow/issues/39754#issuecomment-1905284866
Hey, thanks for the report.
I can't reproduce with the same R and arrow version though I am on linux so it might be an issue with tzdata on windows... do other conversions work correctly?
``` r
library(arrow)
#>
#> Attaching package: 'arrow'
#> The following object is masked from 'package:utils':
#>
#> timestamp
library(lubridate)
#>
#> Attaching package: 'lubridate'
#> The following object is masked from 'package:arrow':
#>
#> duration
#> The following objects are masked from 'package:base':
#>
#> date, intersect, setdiff, union
library(dplyr)
#>
#> Attaching package: 'dplyr'
#> The following objects are masked from 'package:stats':
#>
#> filter, lag
#> The following objects are masked from 'package:base':
#>
#> intersect, setdiff, setequal, union
arrow_table(date = "12JAN2004") %>%
mutate(date = dmy(date)) %>%
collect()
#> # A tibble: 1 × 1
#> date
#> <date>
#> 1 2004-01-12
arrow_table(date = "12JAN2004") %>%
collect() %>%
mutate(date = dmy(date))
#> # A tibble: 1 × 1
#> date
#> <date>
#> 1 2004-01-12
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [R] Docs are not clear on expected behaviour of date parsing functions (e.g. dmy()) on Windows vs. Linux/MacOS [arrow]
Posted by "thisisnic (via GitHub)" <gi...@apache.org>.
thisisnic closed issue #39754: [R] Docs are not clear on expected behaviour of date parsing functions (e.g. dmy()) on Windows vs. Linux/MacOS
URL: https://github.com/apache/arrow/issues/39754
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org