You are viewing a plain text version of this content. The canonical link for it is here.
Posted to jira@arrow.apache.org by "Joris Van den Bossche (Jira)" <ji...@apache.org> on 2022/03/15 11:38:00 UTC
[jira] [Commented] (ARROW-15862) [C++] Provide a way to go from numeric to duration
[ https://issues.apache.org/jira/browse/ARROW-15862?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17506864#comment-17506864 ]
Joris Van den Bossche commented on ARROW-15862:
-----------------------------------------------
bq. Would it make sense to support float -> duration or int32 -> duration?
Do you mean in the (C++) Cast machinery?
We also don't support this for example timestamp:
{code:python}
# casting int64 to timestamp is fine
>>> pa.array([1, 2, 3]).cast(pa.timestamp("s"))
<pyarrow.lib.TimestampArray object at 0x7fb5c314c340>
[
1970-01-01 00:00:01,
1970-01-01 00:00:02,
1970-01-01 00:00:03
]
# casting int32 not
>>> pa.array([1, 2, 3], type="int32").cast(pa.timestamp("s"))
... ArrowNotImplementedError: Unsupported cast from int32 to timestamp using function cast_timestamp
../src/arrow/compute/function.cc:231 DispatchBest(&inputs)
{code}
We might want to do this, but I think it is a general issue / question for the casting machinery of Arrow how to handle casts that might need an additional step (like "int32 -> timestamp" can be done as "int32 -> int64 -> timestamp" which are all supported)
> [C++] Provide a way to go from numeric to duration
> --------------------------------------------------
>
> Key: ARROW-15862
> URL: https://issues.apache.org/jira/browse/ARROW-15862
> Project: Apache Arrow
> Issue Type: Improvement
> Components: C++
> Reporter: Dragoș Moldovan-Grünfeld
> Priority: Critical
>
> *Update*: I keep forgetting that R integers are translated into int32 (since R does not have int64). int64 -> duration is supported.
> Would it make sense to support float -> duration or int32 -> duration?
> ======================
> Currently it is not possible to directly create a duration object from a numeric one (for example through casting).
> {code:r}
> library(arrow)
> a <- Array$create(32L)
> a$cast(duration("s"))
> #> Error: NotImplemented: Unsupported cast from int32 to duration using function cast_duration
> #> /Users/dragos/Documents/arrow/cpp/src/arrow/compute/function.cc:231 DispatchBest(&inputs)
> {code}
> This underpins a lot of the date-time arithmetic in R, which support the conversion/ coercion of an integer to difftime (R's equivalent for duration), such as in the pipeline below.
> {code:r}
> library(arrow, warn.conflicts = FALSE)
> #> See arrow_info() for available features
> library(dplyr, warn.conflicts = FALSE)
> library(lubridate, warn.conflicts = FALSE)
> df <- tibble(time = as_datetime(c("2022-03-07 15:00:28", "2022-03-06 14:00:28")))
> df
> #> # A tibble: 2 × 1
> #> time
> #> <dttm>
> #> 1 2022-03-07 15:00:28
> #> 2 2022-03-06 14:00:28
> df %>%
> mutate(time2 = time + seconds(2))
> #> # A tibble: 2 × 2
> #> time time2
> #> <dttm> <dttm>
> #> 1 2022-03-07 15:00:28 2022-03-07 15:00:30
> #> 2 2022-03-06 14:00:28 2022-03-06 14:00:30
> {code}
--
This message was sent by Atlassian Jira
(v8.20.1#820001)