You are viewing a plain text version of this content. The canonical link for it is here.
Posted to jira@arrow.apache.org by "Dragoș Moldovan-Grünfeld (Jira)" <ji...@apache.org> on 2022/03/22 12:30:00 UTC
[jira] [Updated] (ARROW-15948) [C++] strptime rolls-over dates not in range for current month
[ https://issues.apache.org/jira/browse/ARROW-15948?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Dragoș Moldovan-Grünfeld updated ARROW-15948:
---------------------------------------------
Priority: Critical (was: Major)
> [C++] strptime rolls-over dates not in range for current month
> ---------------------------------------------------------------
>
> Key: ARROW-15948
> URL: https://issues.apache.org/jira/browse/ARROW-15948
> Project: Apache Arrow
> Issue Type: Bug
> Components: C++, Python, R
> Reporter: Dragoș Moldovan-Grünfeld
> Priority: Critical
>
> I noticed some potentially unexpected behaviour when converting from string to date. Days that are out of bounds for the given month are rolled over into the following month.
> I think the expected behaviour would be to either error (Python) or return NULL/NA (R), but not to roll over dates in the following month.
> {code:r}
> library(arrow, warn.conflicts = FALSE)
> library(lubridate, warn.conflicts = FALSE)
> library(dplyr, warn.conflicts = FALSE)
> df <- tibble::tibble(string_date = "1999-02-30")
> # base R returns NA
> df %>%
> mutate(date = strptime(string_date, format = "%Y-%m-%d"))
> #> # A tibble: 1 × 2
> #> string_date date
> #> <chr> <dttm>
> #> 1 1999-02-30 NA
> # arrow rolls over the 30th of February into the 2nd of March
> df %>%
> arrow_table() %>%
> mutate(date = strptime(string_date, format = "%Y-%m-%d")) %>%
> collect()
> #> # A tibble: 1 × 2
> #> string_date date
> #> <chr> <dttm>
> #> 1 1999-02-30 1999-03-02 00:00:00
> {code}
> Thanks Alenka, Joris and Rok for helping me with the Python examples:
> pandas:
> {code:python}
> >>> import pandas as pd
> >>> pd.to_datetime("1999-02-30", format="%Y-%m-%d")
> ...
> ValueError: time data 1999-02-30 doesn't match format specified
> {code}
> datetime:
> {code:python}
> >>> import datetime
> >>> from datetime import datetime
> >>> datetime.strptime("1999-02-30", "%Y-%m-%d")
> ...
> ValueError: day is out of range for month
> {code}
> arrow:
> {code:python}
> >>> import pyarrow.compute as pc
> >>> print(pc.strptime("1999-02-30", format="%Y-%m-%d", unit="s"))
> 1999-03-02 00:00:00
> {code}
--
This message was sent by Atlassian Jira
(v8.20.1#820001)