You are viewing a plain text version of this content. The canonical link for it is here.
Posted to jira@arrow.apache.org by "Dragoș Moldovan-Grünfeld (Jira)" <ji...@apache.org> on 2022/03/15 15:03:00 UTC

[jira] [Comment Edited] (ARROW-15665) [C++] Add error handling option to StrptimeOptions

    [ https://issues.apache.org/jira/browse/ARROW-15665?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17506955#comment-17506955 ] 

Dragoș Moldovan-Grünfeld edited comment on ARROW-15665 at 3/15/22, 3:02 PM:
----------------------------------------------------------------------------

[~rokm], [~jorisvandenbossche] & [~jonkeane]
I will ask the question here and move it if it is the wrong place / follow-up Jira required. In R {{strptime}} returns NA / NULL in the following circumstances:
# 1 {{format}} doesn't match {{string}} - e.g. {{"1999-12-31" and "%Y-%d-%M"}} 
# 2 {{string}} doesn't make sense, given the format - e.g. {{string}} is {{"this is a string that doesn't make sense"}}
# 3 the {{string}} can be parsed with the given {{format}}, but implicitly relies on rollover - {{string}} is {{"1999-02-30"}} and is parsed as the {{"1999-03-02"}} Date.


not sure 1 and 2 are actually different, but the 3rd part is different from the current Arrow behaviour

{code:r}
library(arrow, warn.conflicts = FALSE)
library(lubridate, warn.conflicts = FALSE)
library(dplyr, warn.conflicts = FALSE)

df <- tibble::tibble(string_date = "1999-02-30")

df %>% 
  mutate(date = strptime(string_date, format = "%Y-%m-%d"))
#> # A tibble: 1 × 2
#>   string_date date  
#>   <chr>       <dttm>
#> 1 1999-02-30  NA

df %>% 
  arrow_table() %>% 
  mutate(date = strptime(string_date, format = "%Y-%m-%d")) %>% 
  collect()
#> # A tibble: 1 × 2
#>   string_date date               
#>   <chr>       <dttm>             
#> 1 1999-02-30  1999-03-02 00:00:00
{code} 

How are things done in Python? Does the R behaviour align with your expectations / Is it breaking any ISO Standard?


was (Author: dragosmg):
[~rokm][~jorisvandenbossche][~jonkeane]
I will ask the question here and move it if it is the wrong place / follow-up Jira required. In R {{strptime}} returns NA / NULL in the following circumstances:
# 1 {{format}} doesn't match {{string}} - e.g. {{"1999-12-31" and "%Y-%d-%M"}} 
# 2 {{string}} doesn't make sense, given the format - e.g. {{string}} is {{"this is a string that doesn't make sense"}}
# 3 the {{string}} can be parsed with the given {{format}}, but implicitly relies on rollover - {{string}} is {{"1999-02-30"}} and is parsed as the {{"1999-03-02"}} Date.


not sure 1 and 2 are actually different, but the 3rd part is different from the current Arrow behaviour

{code:r}
library(arrow, warn.conflicts = FALSE)
library(lubridate, warn.conflicts = FALSE)
library(dplyr, warn.conflicts = FALSE)

df <- tibble::tibble(string_date = "1999-02-30")

df %>% 
  mutate(date = strptime(string_date, format = "%Y-%m-%d"))
#> # A tibble: 1 × 2
#>   string_date date  
#>   <chr>       <dttm>
#> 1 1999-02-30  NA

df %>% 
  arrow_table() %>% 
  mutate(date = strptime(string_date, format = "%Y-%m-%d")) %>% 
  collect()
#> # A tibble: 1 × 2
#>   string_date date               
#>   <chr>       <dttm>             
#> 1 1999-02-30  1999-03-02 00:00:00
{code} 

How are things done in Python? Does the R behaviour align with your expectations / Is it breaking any ISO Standard?

> [C++] Add error handling option to StrptimeOptions
> --------------------------------------------------
>
>                 Key: ARROW-15665
>                 URL: https://issues.apache.org/jira/browse/ARROW-15665
>             Project: Apache Arrow
>          Issue Type: Improvement
>          Components: C++
>            Reporter: Rok Mihevc
>            Assignee: Rok Mihevc
>            Priority: Major
>              Labels: kernel, pull-request-available
>          Time Spent: 40m
>  Remaining Estimate: 0h
>
> We want to have an option to either raise, ignore or return NA in case of format mismatch.
> See [pandas.to_datetime|https://pandas.pydata.org/docs/reference/api/pandas.to_datetime.html] and lubridate [parse_date_time|https://lubridate.tidyverse.org/reference/parse_date_time.html] for examples.



--
This message was sent by Atlassian Jira
(v8.20.1#820001)