You are viewing a plain text version of this content. The canonical link for it is here.
Posted to jira@arrow.apache.org by "Dragoș Moldovan-Grünfeld (Jira)" <ji...@apache.org> on 2022/05/16 08:41:00 UTC

[jira] [Commented] (ARROW-14575) [R] Allow functions with {{pkg::}} prefixes

    [ https://issues.apache.org/jira/browse/ARROW-14575?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17537402#comment-17537402 ] 

Dragoș Moldovan-Grünfeld commented on ARROW-14575:
--------------------------------------------------

I have an draft [PR|https://github.com/apache/arrow/pull/13160] for allowing {{pkg::}} prefixes. What do you think of this approach?
* change the naming convention for bindings, by adding {{"package_"}} before the function name. For example {{"as_datetime"}} becomes {{{}"lubridate_as_datetime"{}}}. Doing this mainly for 2 reasons:
 ** it gives us an indication of the package / namespace we're linking to
 ** it avoids the use of the double-colon ({{{}::{}}}) operator (easier dealing with strings only)
* we register each binding both with its _short_ name ({{{}"as_datetime"{}}}) and its _full_ name ({{{}"lubridate_as_datetime"{}}})
* we change the expressions supplied by the user by replacing the double colon ({{{}::{}}}) with an underscore ({{{}_{}}})
* we can still include the original expression (containing {{::}} in the error message)
* we don't have to define an allow list as the user will get the same message they are getting: "now {{pkg::fun() not supported in Arrow}}".

> [R] Allow functions with {{pkg::}} prefixes
> -------------------------------------------
>
>                 Key: ARROW-14575
>                 URL: https://issues.apache.org/jira/browse/ARROW-14575
>             Project: Apache Arrow
>          Issue Type: Bug
>          Components: R
>            Reporter: Jonathan Keane
>            Assignee: Dragoș Moldovan-Grünfeld
>            Priority: Major
>              Labels: pull-request-available
>          Time Spent: 1.5h
>  Remaining Estimate: 0h
>
> Currently we implement a number of functions from packages like {lubridate} which work well when called without namespacing (e.g. {{year()}}), however if someone calls {{lubridate::year()}} we get a not-implemented method (e.g. {{Warning: Expression lubridate::year(time_hour) not supported in Arrow}}). Is it possible for us to look and see if we have an arrow function that matches the function itself.
> {code:r}
> library(arrow, warn.conflicts = FALSE)
> library(dplyr, warn.conflicts = FALSE)
> ds <- InMemoryDataset$create(nycflights13::flights)
> ds %>% 
>   mutate(year = lubridate::year(time_hour)) %>%
>   collect()
> #> Warning: Expression lubridate::year(time_hour) not supported in Arrow; pulling
> #> data into R
> #> # A tibble: 336,776 × 19
> #>     year month   day dep_time sched_dep_time dep_delay arr_time sched_arr_time
> #>    <dbl> <int> <int>    <int>          <int>     <dbl>    <int>          <int>
> #>  1  2013     1     1      517            515         2      830            819
> #>  2  2013     1     1      533            529         4      850            830
> #>  3  2013     1     1      542            540         2      923            850
> #>  4  2013     1     1      544            545        -1     1004           1022
> #>  5  2013     1     1      554            600        -6      812            837
> #>  6  2013     1     1      554            558        -4      740            728
> #>  7  2013     1     1      555            600        -5      913            854
> #>  8  2013     1     1      557            600        -3      709            723
> #>  9  2013     1     1      557            600        -3      838            846
> #> 10  2013     1     1      558            600        -2      753            745
> #> # … with 336,766 more rows, and 11 more variables: arr_delay <dbl>,
> #> #   carrier <chr>, flight <int>, tailnum <chr>, origin <chr>, dest <chr>,
> #> #   air_time <dbl>, distance <dbl>, hour <dbl>, minute <dbl>, time_hour <dttm>
> ds %>% 
>   mutate(year = year(time_hour)) %>%
>   collect()
> #> # A tibble: 336,776 × 19
> #>     year month   day dep_time sched_dep_time dep_delay arr_time sched_arr_time
> #>    <int> <int> <int>    <int>          <int>     <dbl>    <int>          <int>
> #>  1  2013     1     1      517            515         2      830            819
> #>  2  2013     1     1      533            529         4      850            830
> #>  3  2013     1     1      542            540         2      923            850
> #>  4  2013     1     1      544            545        -1     1004           1022
> #>  5  2013     1     1      554            600        -6      812            837
> #>  6  2013     1     1      554            558        -4      740            728
> #>  7  2013     1     1      555            600        -5      913            854
> #>  8  2013     1     1      557            600        -3      709            723
> #>  9  2013     1     1      557            600        -3      838            846
> #> 10  2013     1     1      558            600        -2      753            745
> #> # … with 336,766 more rows, and 11 more variables: arr_delay <dbl>,
> #> #   carrier <chr>, flight <int>, tailnum <chr>, origin <chr>, dest <chr>,
> #> #   air_time <dbl>, distance <dbl>, hour <dbl>, minute <dbl>, time_hour <dttm>
> {code}



--
This message was sent by Atlassian Jira
(v8.20.7#820007)