[jira] [Commented] (ARROW-15948) [C++] strptime rolls-over dates not in range for current month

["Joris Van den Bossche (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/ARROW-15948?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17507601#comment-17507601 ] 

Joris Van den Bossche commented on ARROW-15948:
-----------------------------------------------

Copying my comment from the other issue: personally, I don't like that behaviour, but I suppose we get this from the system strptime? (so that might even depend on your OS?)
It might be interesting to check what date.h's version of strptime does.

> I think the expected behaviour would be to either error (Python) or return NULL/NA (R), 

I would say it is both: by default error, and optionally return null (after ARROW-15665)


> [C++] strptime rolls-over dates not in range for current month 
> ---------------------------------------------------------------
>
>                 Key: ARROW-15948
>                 URL: https://issues.apache.org/jira/browse/ARROW-15948
>             Project: Apache Arrow
>          Issue Type: Bug
>          Components: C++, Python, R
>            Reporter: Dragoș Moldovan-Grünfeld
>            Priority: Major
>
> I noticed some potentially unexpected behaviour when converting from string to date. Days that are out of bounds for the given month are rolled over into the following month.
> I think the expected behaviour would be to either error (Python) or return NULL/NA (R), but not to roll over dates in the following month.
> {code:r}
> library(arrow, warn.conflicts = FALSE)
> library(lubridate, warn.conflicts = FALSE)
> library(dplyr, warn.conflicts = FALSE)
> df <- tibble::tibble(string_date = "1999-02-30")
> # base R returns NA
> df %>% 
>   mutate(date = strptime(string_date, format = "%Y-%m-%d"))
> #> # A tibble: 1 × 2
> #>   string_date date  
> #>   <chr>       <dttm>
> #> 1 1999-02-30  NA
> # arrow rolls over the 30th of February into the 2nd of March
> df %>% 
>   arrow_table() %>% 
>   mutate(date = strptime(string_date, format = "%Y-%m-%d")) %>% 
>   collect()
> #> # A tibble: 1 × 2
> #>   string_date date               
> #>   <chr>       <dttm>             
> #> 1 1999-02-30  1999-03-02 00:00:00
> {code}
> Thanks Alenka, Joris and Rok for helping me with the Python examples: 
> pandas:
> {code:python}
> >>> import pandas as pd
> >>> pd.to_datetime("1999-02-30", format="%Y-%m-%d")
> ...
> ValueError: time data 1999-02-30 doesn't match format specified
> {code}
> datetime:
> {code:python}
> >>> import datetime
> >>> from datetime import datetime
> >>> datetime.strptime("1999-02-30", "%Y-%m-%d")
> ...
> ValueError: day is out of range for month
> {code}
> arrow:
> {code:python}
> >>> import pyarrow.compute as pc
> >>> print(pc.strptime("1999-02-30", format="%Y-%m-%d", unit="s"))
> 1999-03-02 00:00:00
> {code}



--
This message was sent by Atlassian Jira
(v8.20.1#820001)