Arrow FunctionRegsitry usage in Python

[Murali S <mu...@gmail.com>]

Hi ,

I was wondering if it is possible to add a C++ Function to the Compute
FunctionRegistry
<https://arrow.apache.org/docs/cpp/api/compute.html#_CPPv4N5arrow7compute16FunctionRegistryE>
and then use the functions in python. Would be great if you could provide
examples of such usage.
Also are all functions added to the FunctionRegistry only callable using
the GetFunction API with the function name as string ? Would like to know
if there is a way to just do arrow::compute::FuncA where FuncA is the newly
added function

Thanks in advance
Mura

Re: Arrow FunctionRegsitry usage in Python

[Weston Pace <we...@gmail.com>]

> I was wondering if it is possible to add a C++ Function to the Compute FunctionRegistry and then use the functions in python.

Yes.  This is partly handled automatically I believe when functions
are added to the C++ function registry.  There are generally two ways
to call the function.  For example, consider the "take" function:

```
>>> import pyarrow as pa
>>> import pyarrow.compute as pc
>>> arr = pa.array([1, 2, 3])
>>> idxs = pa.array([1, 2])
>>> arr.take(idxs)
<pyarrow.lib.Int64Array object at 0x00000211586BF0A0>
[
  2,
  3
]
>>> pc.call_function('take', [arr, idxs])
<pyarrow.lib.Int64Array object at 0x00000211586BF100>
[
  2,
  3
]
```

The first method (`arr.take(idxs)`) is shorthand for the second.  I
believe these convenience shorthands have to be manually added to
pyarrow for each new function.  The second method (`pc.call_function`)
is more generic and I don't think you need to do anything special to
call a function in this way once it has been added to the function
registry.

> Would be great if you could provide examples of such usage.

Some of these shorthand methods are shown in the cookbook[1].
However, I don't see anything geared towards creating new functions
and calling them.  Can you create a JIRA issue (or, even better, a PR)
describing what you'd like to see?

> Also are all functions added to the FunctionRegistry only callable using the GetFunction API with the function name as string ? Would like to know if there is a way to just do arrow::compute::FuncA where FuncA is the newly added function

The C++ API is similar to python.  Some functions have been manually
given a shortcut.  For example, the Take function has a shortcut
here[2].  However, this does not happen automatically and not all
functions have shortcuts.

[1] https://arrow.apache.org/cookbook/py/data.html
[2] https://github.com/apache/arrow/blob/86915807af6fe10f44bc881e57b2f425f97c56c7/cpp/src/arrow/compute/api_vector.h#L352

On Tue, Jun 21, 2022 at 6:31 PM Murali S <mu...@gmail.com> wrote:
>
> Hi ,
>
> I was wondering if it is possible to add a C++ Function to the Compute FunctionRegistry and then use the functions in python. Would be great if you could provide examples of such usage.
> Also are all functions added to the FunctionRegistry only callable using the GetFunction API with the function name as string ? Would like to know if there is a way to just do arrow::compute::FuncA where FuncA is the newly added function
>
> Thanks in advance
> Mura
>
>