You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@arrow.apache.org by GitBox <gi...@apache.org> on 2020/07/09 19:05:12 UTC
[GitHub] [arrow] pitrou opened a new pull request #7695: ARROW-8989: [C++][Doc] Document available compute functions
pitrou opened a new pull request #7695:
URL: https://github.com/apache/arrow/pull/7695
Also fix glaring bugs in arithmetic kernels
(signed overflow detection was broken).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] jorisvandenbossche commented on a change in pull request #7695: ARROW-8989: [C++][Doc] Document available compute functions
Posted by GitBox <gi...@apache.org>.
jorisvandenbossche commented on a change in pull request #7695:
URL: https://github.com/apache/arrow/pull/7695#discussion_r453541169
##########
File path: docs/source/cpp/compute.rst
##########
@@ -0,0 +1,419 @@
+.. Licensed to the Apache Software Foundation (ASF) under one
+.. or more contributor license agreements. See the NOTICE file
+.. distributed with this work for additional information
+.. regarding copyright ownership. The ASF licenses this file
+.. to you under the Apache License, Version 2.0 (the
+.. "License"); you may not use this file except in compliance
+.. with the License. You may obtain a copy of the License at
+
+.. http://www.apache.org/licenses/LICENSE-2.0
+
+.. Unless required by applicable law or agreed to in writing,
+.. software distributed under the License is distributed on an
+.. "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+.. KIND, either express or implied. See the License for the
+.. specific language governing permissions and limitations
+.. under the License.
+
+.. default-domain:: cpp
+.. highlight:: cpp
+.. cpp:namespace:: arrow::compute
+
+=================
+Compute Functions
+=================
+
+.. TODO: describe API and how to invoke compute functions
+
+Available functions
+===================
+
+Aggregations
+------------
+
++--------------------------+------------+--------------------+-----------------------+--------------------------------------------+
+| Function name | Arity | Input types | Output type | Options class |
++==========================+============+====================+=======================+============================================+
+| count | Unary | Any | Scalar Int64 | :struct:`CountOptions` |
++--------------------------+------------+--------------------+-----------------------+--------------------------------------------+
+| mean | Unary | Numeric | Scalar Float64 | |
++--------------------------+------------+--------------------+-----------------------+--------------------------------------------+
+| minmax | Unary | Numeric | Scalar Struct (1) | :struct:`MinMaxOptions` |
++--------------------------+------------+--------------------+-----------------------+--------------------------------------------+
+| sum | Unary | Numeric | Scalar Numeric (2) | |
++--------------------------+------------+--------------------+-----------------------+--------------------------------------------+
+
+Notes:
+
+* \(1) Output is a ``{"min": input type, "max": input type}`` Struct
+
+* \(2) Output is Int64, UInt64 or Float64, depending on the input type
+
+
+Element-wise ("scalar") functions
+---------------------------------
+
+Arithmetic functions
+~~~~~~~~~~~~~~~~~~~~
+
+Those functions expect two inputs of the same type and apply a given binary
+operation to each pair of elements gathered from the inputs. Each function
+is also available in an overflow-checking variant, suffixed ``_checked``.
+
+If any of the input elements in a pair is null, the corresponding output
+element is null.
+
++--------------------------+------------+--------------------+---------------------+
+| Function name | Arity | Input types | Output type |
++==========================+============+====================+=====================+
+| add | Binary | Numeric | Numeric |
++--------------------------+------------+--------------------+---------------------+
+| add_checked | Binary | Numeric | Numeric |
++--------------------------+------------+--------------------+---------------------+
+| multiply | Binary | Numeric | Numeric |
++--------------------------+------------+--------------------+---------------------+
+| multiply_checked | Binary | Numeric | Numeric |
++--------------------------+------------+--------------------+---------------------+
+| subtract | Binary | Numeric | Numeric |
++--------------------------+------------+--------------------+---------------------+
+| subtract_checked | Binary | Numeric | Numeric |
++--------------------------+------------+--------------------+---------------------+
+
+Comparisons
+~~~~~~~~~~~
+
+Those functions expect two inputs of the same type and apply a given
+comparison operator. If any of the input elements in a pair is null,
+the corresponding output element is null.
+
++--------------------------+------------+---------------------------------+---------------------+
+| Function names | Arity | Input types | Output type |
++==========================+============+=================================+=====================+
+| equal, not_equal | Binary | Numeric | Boolean |
++--------------------------+------------+---------------------------------+---------------------+
+| equal, not_equal | Binary | Binary- and String-like | Boolean |
++--------------------------+------------+---------------------------------+---------------------+
+| equal, not_equal | Binary | Temporal | Boolean |
++--------------------------+------------+---------------------------------+---------------------+
+| greater, greater_equal, | Binary | Numeric | Boolean |
+| less, less_equal | | | |
++--------------------------+------------+---------------------------------+---------------------+
+| greater, greater_equal, | Binary | Binary- and String-like | Boolean |
+| less, less_equal | | | |
++--------------------------+------------+---------------------------------+---------------------+
+| greater, greater_equal, | Binary | Temporal | Boolean |
+| less, less_equal | | | |
++--------------------------+------------+---------------------------------+---------------------+
+
+Logical functions
+~~~~~~~~~~~~~~~~~~
+
+The normal behaviour for these functions is to emit a null if any of the
+inputs is null.
+
+Some of them are also available in a "`Kleene logic`_" variant (suffixed
Review comment:
Yes, numpy doesn't do this. But, numpy doesn't know the concept of "nulls", so it's actually not even a question for numpy (and pandas only started to do this in the new experimental opt-in nullable dtypes, but for its default numpy-based dtypes, ti follows numpy)
Kleene logic is also what Julia does by default when working with their "Missing" union type (https://docs.julialang.org/en/v1/manual/missing/index.html#Logical-operators-1, they use the term "three-valued logic").
I am personally not aware of a system that has "null" support, and doesn't use Kleene logic.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] pitrou commented on a change in pull request #7695: ARROW-8989: [C++][Doc] Document available compute functions
Posted by GitBox <gi...@apache.org>.
pitrou commented on a change in pull request #7695:
URL: https://github.com/apache/arrow/pull/7695#discussion_r453001323
##########
File path: docs/source/cpp/compute.rst
##########
@@ -0,0 +1,419 @@
+.. Licensed to the Apache Software Foundation (ASF) under one
+.. or more contributor license agreements. See the NOTICE file
+.. distributed with this work for additional information
+.. regarding copyright ownership. The ASF licenses this file
+.. to you under the Apache License, Version 2.0 (the
+.. "License"); you may not use this file except in compliance
+.. with the License. You may obtain a copy of the License at
+
+.. http://www.apache.org/licenses/LICENSE-2.0
+
+.. Unless required by applicable law or agreed to in writing,
+.. software distributed under the License is distributed on an
+.. "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+.. KIND, either express or implied. See the License for the
+.. specific language governing permissions and limitations
+.. under the License.
+
+.. default-domain:: cpp
+.. highlight:: cpp
+.. cpp:namespace:: arrow::compute
+
+=================
+Compute Functions
+=================
+
+.. TODO: describe API and how to invoke compute functions
+
+Available functions
+===================
+
+Aggregations
+------------
+
++--------------------------+------------+--------------------+-----------------------+--------------------------------------------+
+| Function name | Arity | Input types | Output type | Options class |
++==========================+============+====================+=======================+============================================+
+| count | Unary | Any | Scalar Int64 | :struct:`CountOptions` |
++--------------------------+------------+--------------------+-----------------------+--------------------------------------------+
+| mean | Unary | Numeric | Scalar Float64 | |
++--------------------------+------------+--------------------+-----------------------+--------------------------------------------+
+| minmax | Unary | Numeric | Scalar Struct (1) | :struct:`MinMaxOptions` |
++--------------------------+------------+--------------------+-----------------------+--------------------------------------------+
+| sum | Unary | Numeric | Scalar Numeric (2) | |
++--------------------------+------------+--------------------+-----------------------+--------------------------------------------+
+
+Notes:
+
+* \(1) Output is a ``{"min": input type, "max": input type}`` Struct
+
+* \(2) Output is Int64, UInt64 or Float64, depending on the input type
+
+
+Element-wise ("scalar") functions
+---------------------------------
+
+Arithmetic functions
+~~~~~~~~~~~~~~~~~~~~
+
+Those functions expect two inputs of the same type and apply a given binary
+operation to each pair of elements gathered from the inputs. Each function
+is also available in an overflow-checking variant, suffixed ``_checked``.
+
+If any of the input elements in a pair is null, the corresponding output
+element is null.
+
++--------------------------+------------+--------------------+---------------------+
+| Function name | Arity | Input types | Output type |
++==========================+============+====================+=====================+
+| add | Binary | Numeric | Numeric |
++--------------------------+------------+--------------------+---------------------+
+| add_checked | Binary | Numeric | Numeric |
++--------------------------+------------+--------------------+---------------------+
+| multiply | Binary | Numeric | Numeric |
++--------------------------+------------+--------------------+---------------------+
+| multiply_checked | Binary | Numeric | Numeric |
++--------------------------+------------+--------------------+---------------------+
+| subtract | Binary | Numeric | Numeric |
++--------------------------+------------+--------------------+---------------------+
+| subtract_checked | Binary | Numeric | Numeric |
++--------------------------+------------+--------------------+---------------------+
+
+Comparisons
+~~~~~~~~~~~
+
+Those functions expect two inputs of the same type and apply a given
+comparison operator. If any of the input elements in a pair is null,
+the corresponding output element is null.
+
++--------------------------+------------+---------------------------------+---------------------+
+| Function names | Arity | Input types | Output type |
++==========================+============+=================================+=====================+
+| equal, not_equal | Binary | Numeric | Boolean |
++--------------------------+------------+---------------------------------+---------------------+
+| equal, not_equal | Binary | Binary- and String-like | Boolean |
++--------------------------+------------+---------------------------------+---------------------+
+| equal, not_equal | Binary | Temporal | Boolean |
++--------------------------+------------+---------------------------------+---------------------+
+| greater, greater_equal, | Binary | Numeric | Boolean |
+| less, less_equal | | | |
++--------------------------+------------+---------------------------------+---------------------+
+| greater, greater_equal, | Binary | Binary- and String-like | Boolean |
+| less, less_equal | | | |
++--------------------------+------------+---------------------------------+---------------------+
+| greater, greater_equal, | Binary | Temporal | Boolean |
+| less, less_equal | | | |
++--------------------------+------------+---------------------------------+---------------------+
+
+Logical functions
+~~~~~~~~~~~~~~~~~~
+
+The normal behaviour for these functions is to emit a null if any of the
+inputs is null.
+
+Some of them are also available in a "`Kleene logic`_" variant (suffixed
+``_kleene``) where null is taken to mean "undefined". For those variants
+therefore:
+
+* "true AND null", "null AND true" give "null" (the result is undefined)
+* "true OR null", "null OR true" give "true"
+* "false AND null", "null AND false" give "false"
+* "false OR null", "null OR false" give "null" (the result is undefined)
+
++--------------------------+------------+--------------------+---------------------+
+| Function name | Arity | Input types | Output type |
++==========================+============+====================+=====================+
+| and | Binary | Boolean | Boolean |
++--------------------------+------------+--------------------+---------------------+
+| and_kleene | Binary | Boolean | Boolean |
++--------------------------+------------+--------------------+---------------------+
+| invert | Unary | Boolean | Boolean |
++--------------------------+------------+--------------------+---------------------+
+| or | Binary | Boolean | Boolean |
++--------------------------+------------+--------------------+---------------------+
+| or_kleene | Binary | Boolean | Boolean |
++--------------------------+------------+--------------------+---------------------+
+| xor | Binary | Boolean | Boolean |
++--------------------------+------------+--------------------+---------------------+
+
+.. _Kleene logic: https://en.wikipedia.org/wiki/Three-valued_logic#Kleene_and_Priest_logics
+
+String functions
+~~~~~~~~~~~~~~~~
+
++--------------------------+------------+--------------------+---------------------+---------+
+| Function name | Arity | Input types | Output type | Notes |
++==========================+============+====================+=====================+=========+
+| ascii_length | Unary | String-like | Int32 or Int64 | \(1) |
++--------------------------+------------+--------------------+---------------------+---------+
+| ascii_lower | Unary | String-like | String-like | \(2) |
++--------------------------+------------+--------------------+---------------------+---------+
+| ascii_upper | Unary | String-like | String-like | \(2) |
++--------------------------+------------+--------------------+---------------------+---------+
+| utf8_lower | Unary | String-like | String-like | \(3) |
++--------------------------+------------+--------------------+---------------------+---------+
+| utf8_upper | Unary | String-like | String-like | \(3) |
++--------------------------+------------+--------------------+---------------------+---------+
+
+* \(1) Output is the physical length in bytes of each input element.
+
+* \(2) Each ASCII character in the input is converted to lowercase or
+ uppercase. Non-ASCII characters are left untouched.
+
+* \(3) Each UTF8-encoded character in the input is converted to lowercase or
+ uppercase.
+
+Containment tests
+~~~~~~~~~~~~~~~~~
+
++--------------------------+------------+----------------------------------+-----------------------+--------------------------------------------+
+| Function name | Arity | Input types | Output type | Options class |
++==========================+============+==================================+=======================+============================================+
+| binary_contains_exact | Unary | String-like | Boolean (1) | :struct:`BinaryContainsExactOptions` |
++--------------------------+------------+----------------------------------+-----------------------+--------------------------------------------+
+| isin | Unary | Binary- and String-like | Boolean (2) | :struct:`SetLookupOptions` |
++--------------------------+------------+----------------------------------+-----------------------+--------------------------------------------+
+| isin | Unary | Null | Boolean (2) | :struct:`SetLookupOptions` |
++--------------------------+------------+----------------------------------+-----------------------+--------------------------------------------+
+| isin | Unary | Boolean,Numeric, Temporal | Boolean (2) | :struct:`SetLookupOptions` |
++--------------------------+------------+----------------------------------+-----------------------+--------------------------------------------+
+| match | Unary | Binary- and String-like | Int32 (3) | :struct:`SetLookupOptions` |
++--------------------------+------------+----------------------------------+-----------------------+--------------------------------------------+
+| match | Unary | Null | Int32 (3) | :struct:`SetLookupOptions` |
++--------------------------+------------+----------------------------------+-----------------------+--------------------------------------------+
+| match | Unary | Boolean,Numeric, Temporal | Int32 (3) | :struct:`SetLookupOptions` |
++--------------------------+------------+----------------------------------+-----------------------+--------------------------------------------+
+
+* \(1) Output is true iff :member:`BinaryContainsExactOptions::pattern`
+ is a substring of the corresponding input element.
+
+* \(2) Output is true iff the corresponding input element is equal to one
+ of the elements in :member:`SetLookupOptions::value_set`.
+
+* \(3) Output is the index of the corresponding input element in
+ :member:`SetLookupOptions::value_set`, if found there. Otherwise,
+ output is null.
+
+Structural transforms
+~~~~~~~~~~~~~~~~~~~~~
+
++--------------------------+------------+--------------------+---------------------+---------+
+| Function name | Arity | Input types | Output type | Notes |
++==========================+============+====================+=====================+=========+
+| is_null | Unary | Any | Boolean | \(1) |
++--------------------------+------------+--------------------+---------------------+---------+
+| is_valid | Unary | Any | Boolean | \(2) |
++--------------------------+------------+--------------------+---------------------+---------+
+| list_value_lengths | Unary | List-like | Int32 or Int64 | \(3) |
Review comment:
Whether the type is List or LargeList.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] pitrou commented on a change in pull request #7695: ARROW-8989: [C++][Doc] Document available compute functions
Posted by GitBox <gi...@apache.org>.
pitrou commented on a change in pull request #7695:
URL: https://github.com/apache/arrow/pull/7695#discussion_r453525416
##########
File path: docs/source/cpp/compute.rst
##########
@@ -0,0 +1,419 @@
+.. Licensed to the Apache Software Foundation (ASF) under one
+.. or more contributor license agreements. See the NOTICE file
+.. distributed with this work for additional information
+.. regarding copyright ownership. The ASF licenses this file
+.. to you under the Apache License, Version 2.0 (the
+.. "License"); you may not use this file except in compliance
+.. with the License. You may obtain a copy of the License at
+
+.. http://www.apache.org/licenses/LICENSE-2.0
+
+.. Unless required by applicable law or agreed to in writing,
+.. software distributed under the License is distributed on an
+.. "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+.. KIND, either express or implied. See the License for the
+.. specific language governing permissions and limitations
+.. under the License.
+
+.. default-domain:: cpp
+.. highlight:: cpp
+.. cpp:namespace:: arrow::compute
+
+=================
+Compute Functions
+=================
+
+.. TODO: describe API and how to invoke compute functions
+
+Available functions
+===================
+
+Aggregations
+------------
+
++--------------------------+------------+--------------------+-----------------------+--------------------------------------------+
+| Function name | Arity | Input types | Output type | Options class |
++==========================+============+====================+=======================+============================================+
+| count | Unary | Any | Scalar Int64 | :struct:`CountOptions` |
++--------------------------+------------+--------------------+-----------------------+--------------------------------------------+
+| mean | Unary | Numeric | Scalar Float64 | |
++--------------------------+------------+--------------------+-----------------------+--------------------------------------------+
+| minmax | Unary | Numeric | Scalar Struct (1) | :struct:`MinMaxOptions` |
++--------------------------+------------+--------------------+-----------------------+--------------------------------------------+
+| sum | Unary | Numeric | Scalar Numeric (2) | |
++--------------------------+------------+--------------------+-----------------------+--------------------------------------------+
+
+Notes:
+
+* \(1) Output is a ``{"min": input type, "max": input type}`` Struct
+
+* \(2) Output is Int64, UInt64 or Float64, depending on the input type
+
+
+Element-wise ("scalar") functions
+---------------------------------
+
+Arithmetic functions
+~~~~~~~~~~~~~~~~~~~~
+
+Those functions expect two inputs of the same type and apply a given binary
+operation to each pair of elements gathered from the inputs. Each function
+is also available in an overflow-checking variant, suffixed ``_checked``.
+
+If any of the input elements in a pair is null, the corresponding output
+element is null.
+
++--------------------------+------------+--------------------+---------------------+
+| Function name | Arity | Input types | Output type |
++==========================+============+====================+=====================+
+| add | Binary | Numeric | Numeric |
++--------------------------+------------+--------------------+---------------------+
+| add_checked | Binary | Numeric | Numeric |
++--------------------------+------------+--------------------+---------------------+
+| multiply | Binary | Numeric | Numeric |
++--------------------------+------------+--------------------+---------------------+
+| multiply_checked | Binary | Numeric | Numeric |
++--------------------------+------------+--------------------+---------------------+
+| subtract | Binary | Numeric | Numeric |
++--------------------------+------------+--------------------+---------------------+
+| subtract_checked | Binary | Numeric | Numeric |
++--------------------------+------------+--------------------+---------------------+
+
+Comparisons
+~~~~~~~~~~~
+
+Those functions expect two inputs of the same type and apply a given
+comparison operator. If any of the input elements in a pair is null,
+the corresponding output element is null.
+
++--------------------------+------------+---------------------------------+---------------------+
+| Function names | Arity | Input types | Output type |
++==========================+============+=================================+=====================+
+| equal, not_equal | Binary | Numeric | Boolean |
++--------------------------+------------+---------------------------------+---------------------+
+| equal, not_equal | Binary | Binary- and String-like | Boolean |
++--------------------------+------------+---------------------------------+---------------------+
+| equal, not_equal | Binary | Temporal | Boolean |
++--------------------------+------------+---------------------------------+---------------------+
+| greater, greater_equal, | Binary | Numeric | Boolean |
+| less, less_equal | | | |
++--------------------------+------------+---------------------------------+---------------------+
+| greater, greater_equal, | Binary | Binary- and String-like | Boolean |
+| less, less_equal | | | |
++--------------------------+------------+---------------------------------+---------------------+
+| greater, greater_equal, | Binary | Temporal | Boolean |
+| less, less_equal | | | |
++--------------------------+------------+---------------------------------+---------------------+
+
+Logical functions
+~~~~~~~~~~~~~~~~~~
+
+The normal behaviour for these functions is to emit a null if any of the
+inputs is null.
+
+Some of them are also available in a "`Kleene logic`_" variant (suffixed
+``_kleene``) where null is taken to mean "undefined". For those variants
+therefore:
+
+* "true AND null", "null AND true" give "null" (the result is undefined)
+* "true OR null", "null OR true" give "true"
+* "false AND null", "null AND false" give "false"
+* "false OR null", "null OR false" give "null" (the result is undefined)
+
++--------------------------+------------+--------------------+---------------------+
+| Function name | Arity | Input types | Output type |
++==========================+============+====================+=====================+
+| and | Binary | Boolean | Boolean |
++--------------------------+------------+--------------------+---------------------+
+| and_kleene | Binary | Boolean | Boolean |
++--------------------------+------------+--------------------+---------------------+
+| invert | Unary | Boolean | Boolean |
++--------------------------+------------+--------------------+---------------------+
+| or | Binary | Boolean | Boolean |
++--------------------------+------------+--------------------+---------------------+
+| or_kleene | Binary | Boolean | Boolean |
++--------------------------+------------+--------------------+---------------------+
+| xor | Binary | Boolean | Boolean |
++--------------------------+------------+--------------------+---------------------+
+
+.. _Kleene logic: https://en.wikipedia.org/wiki/Three-valued_logic#Kleene_and_Priest_logics
+
+String functions
+~~~~~~~~~~~~~~~~
+
++--------------------------+------------+--------------------+---------------------+---------+
+| Function name | Arity | Input types | Output type | Notes |
++==========================+============+====================+=====================+=========+
+| ascii_length | Unary | String-like | Int32 or Int64 | \(1) |
++--------------------------+------------+--------------------+---------------------+---------+
+| ascii_lower | Unary | String-like | String-like | \(2) |
++--------------------------+------------+--------------------+---------------------+---------+
+| ascii_upper | Unary | String-like | String-like | \(2) |
++--------------------------+------------+--------------------+---------------------+---------+
+| utf8_lower | Unary | String-like | String-like | \(3) |
++--------------------------+------------+--------------------+---------------------+---------+
+| utf8_upper | Unary | String-like | String-like | \(3) |
++--------------------------+------------+--------------------+---------------------+---------+
+
+* \(1) Output is the physical length in bytes of each input element.
+
+* \(2) Each ASCII character in the input is converted to lowercase or
+ uppercase. Non-ASCII characters are left untouched.
+
+* \(3) Each UTF8-encoded character in the input is converted to lowercase or
+ uppercase.
+
+Containment tests
+~~~~~~~~~~~~~~~~~
+
++--------------------------+------------+----------------------------------+-----------------------+--------------------------------------------+
+| Function name | Arity | Input types | Output type | Options class |
++==========================+============+==================================+=======================+============================================+
+| binary_contains_exact | Unary | String-like | Boolean (1) | :struct:`BinaryContainsExactOptions` |
++--------------------------+------------+----------------------------------+-----------------------+--------------------------------------------+
+| isin | Unary | Binary- and String-like | Boolean (2) | :struct:`SetLookupOptions` |
++--------------------------+------------+----------------------------------+-----------------------+--------------------------------------------+
+| isin | Unary | Null | Boolean (2) | :struct:`SetLookupOptions` |
++--------------------------+------------+----------------------------------+-----------------------+--------------------------------------------+
+| isin | Unary | Boolean,Numeric, Temporal | Boolean (2) | :struct:`SetLookupOptions` |
++--------------------------+------------+----------------------------------+-----------------------+--------------------------------------------+
+| match | Unary | Binary- and String-like | Int32 (3) | :struct:`SetLookupOptions` |
++--------------------------+------------+----------------------------------+-----------------------+--------------------------------------------+
+| match | Unary | Null | Int32 (3) | :struct:`SetLookupOptions` |
++--------------------------+------------+----------------------------------+-----------------------+--------------------------------------------+
+| match | Unary | Boolean,Numeric, Temporal | Int32 (3) | :struct:`SetLookupOptions` |
++--------------------------+------------+----------------------------------+-----------------------+--------------------------------------------+
+
+* \(1) Output is true iff :member:`BinaryContainsExactOptions::pattern`
+ is a substring of the corresponding input element.
+
+* \(2) Output is true iff the corresponding input element is equal to one
+ of the elements in :member:`SetLookupOptions::value_set`.
+
+* \(3) Output is the index of the corresponding input element in
+ :member:`SetLookupOptions::value_set`, if found there. Otherwise,
+ output is null.
+
+Structural transforms
+~~~~~~~~~~~~~~~~~~~~~
+
++--------------------------+------------+--------------------+---------------------+---------+
+| Function name | Arity | Input types | Output type | Notes |
++==========================+============+====================+=====================+=========+
+| is_null | Unary | Any | Boolean | \(1) |
++--------------------------+------------+--------------------+---------------------+---------+
+| is_valid | Unary | Any | Boolean | \(2) |
++--------------------------+------------+--------------------+---------------------+---------+
+| list_value_lengths | Unary | List-like | Int32 or Int64 | \(3) |
++--------------------------+------------+--------------------+---------------------+---------+
+
+* \(1) Output is true iff the corresponding input element is non-null.
+
+* \(2) Output is true iff the corresponding input element is null.
+
+* \(3) Each output element is the length of the corresponding input element
+ (null if input is null).
+
+Conversions
+~~~~~~~~~~~
+
+A general conversion function named ``cast`` is provided which accepts a large
+number of input and output types. The type to cast to can be passed in a
+:struct:`CastOptions` instance.
+
++--------------------------+------------+--------------------+-----------------------+--------------------------------------------+
+| Function name | Arity | Input types | Output type | Options class |
++==========================+============+====================+=======================+============================================+
+| cast | Unary | Many | Variable | :struct:`CastOptions` |
++--------------------------+------------+--------------------+-----------------------+--------------------------------------------+
+| strptime | Unary | String-like | Timestamp | :struct:`StrptimeOptions` |
++--------------------------+------------+--------------------+-----------------------+--------------------------------------------+
+
+The conversions available with ``cast`` are listed below. In all cases, a
+null input value is converted into a null output value.
+
+**Truth value extraction**
+
++-----------------------------+------------------------------------+--------------+
+| Input type | Output type | Notes |
++=============================+====================================+==============+
+| Binary- and String-like | Boolean | \(1) |
++-----------------------------+------------------------------------+--------------+
+| Numeric | Boolean | \(2) |
++-----------------------------+------------------------------------+--------------+
+
+* \(1) Output is true iff the corresponding input value has non-zero length.
+
+* \(2) Output is true iff the corresponding input value is non-zero.
+
+**Same-kind conversion**
+
++-----------------------------+------------------------------------+--------------+
+| Input type | Output type | Notes |
++=============================+====================================+==============+
+| Int32 | 32-bit Temporal | \(1) |
++-----------------------------+------------------------------------+--------------+
+| Int64 | 64-bit Temporal | \(1) |
++-----------------------------+------------------------------------+--------------+
+| (Large)Binary | (Large)String | \(2) |
++-----------------------------+------------------------------------+--------------+
+| (Large)String | (Large)Binary | \(3) |
++-----------------------------+------------------------------------+--------------+
+| Numeric | Numeric | \(4) \(5) |
++-----------------------------+------------------------------------+--------------+
+| 32-bit Temporal | Int32 | \(1) |
++-----------------------------+------------------------------------+--------------+
+| 64-bit Temporal | Int64 | \(1) |
++-----------------------------+------------------------------------+--------------+
+| Temporal | Temporal | \(4) \(5) |
++-----------------------------+------------------------------------+--------------+
+
+* \(1) No-operation cast: the raw values are kept identical, only
+ the type is changed.
+
+* \(2) Validates the contents if :member:`CastOptions::allow_invalid_utf8`
+ is false.
+
+* \(3) No-operation cast: only the type is changed.
+
+* \(4) Overflow and truncation checks are enabled depending on
+ the given :struct:`CastOptions`.
+
+* \(5) Not all such casts have been implemented.
+
+**String representations**
+
++-----------------------------+------------------------------------+---------+
+| Input type | Output type | Notes |
++=============================+====================================+=========+
+| Boolean | String-like | |
++-----------------------------+------------------------------------+---------+
+| Numeric | String-like | |
++-----------------------------+------------------------------------+---------+
+
+**Generic conversions**
+
++-----------------------------+------------------------------------+---------+
+| Input type | Output type | Notes |
++=============================+====================================+=========+
+| Dictionary | Dictionary value type | |
++-----------------------------+------------------------------------+---------+
+| Extension | Extension storage type | |
++-----------------------------+------------------------------------+---------+
+| List-like | List-like | \(1) |
++-----------------------------+------------------------------------+---------+
+| Null | Any | |
++-----------------------------+------------------------------------+---------+
+
+* \(1) The list offsets are unchanged, the list values are cast from the
+ input value type to the output value type (if a conversion is
+ available).
+
+
+.. TODO: add C++ cast example
+
+Array-wise ("vector") functions
+-------------------------------
+
+Associative transforms
+~~~~~~~~~~~~~~~~~~~~~~
+
++--------------------------+------------+----------------------------+----------------------------+
+| Function name | Arity | Input types | Output type |
++==========================+============+============================+============================+
+| dictionary_encode | Unary | Binary- and String-like | Dictionary (1) |
++--------------------------+------------+----------------------------+----------------------------+
+| dictionary_encode | Unary | Boolean, Numeric, Temporal | Dictionary (1) |
++--------------------------+------------+----------------------------+----------------------------+
+| dictionary_encode | Unary | Null | Dictionary (1) |
++--------------------------+------------+----------------------------+----------------------------+
+| unique | Unary | Binary- and String-like | Input type (2) |
++--------------------------+------------+----------------------------+----------------------------+
+| unique | Unary | Boolean, Numeric, Temporal | Input type (2) |
++--------------------------+------------+----------------------------+----------------------------+
+| unique | Unary | Null | Input type (2) |
++--------------------------+------------+----------------------------+----------------------------+
+| value_counts | Unary | Binary- and String-like | Struct (3) |
++--------------------------+------------+----------------------------+----------------------------+
+| value_counts | Unary | Boolean, Numeric, Temporal | Struct (3) |
++--------------------------+------------+----------------------------+----------------------------+
+| value_counts | Unary | Null | Struct (3) |
++--------------------------+------------+----------------------------+----------------------------+
+
+* \(1) Output is ``Dictionary(Int32, input type)``.
+
+* \(2) Duplicates are removed from the output while the original order is
+ maintained.
+
+* \(3) Output is a ``{"values": input type, "counts": Int64}`` Struct.
+ Each output element corresponds to a unique value in the input, along
+ with the number of times this value has appeared.
+
+Selections
+~~~~~~~~~~
+
+These functions select a subset of the first input defined by the second input.
+
++-----------------+------------+---------------+--------------+------------------+-------------------------+-------------+
+| Function name | Arity | Input type 1 | Input type 2 | Output type | Options class | Notes |
++=================+============+===============+==============+==================+=========================+=============+
+| filter | Binary | Any (1) | Boolean | Input type 1 | :struct:`FilterOptions` | \(2) |
++-----------------+------------+---------------+--------------+------------------+-------------------------+-------------+
+| take | Binary | Any (1) | Integer | Input type 1 | :struct:`TakeOptions` | \(3) |
Review comment:
I don't know. Does it help the user to know about the non-metafunctions? Personally, I wouldn't know what to do with them.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] pitrou commented on pull request #7695: ARROW-8989: [C++][Doc] Document available compute functions
Posted by GitBox <gi...@apache.org>.
pitrou commented on pull request #7695:
URL: https://github.com/apache/arrow/pull/7695#issuecomment-657467563
So there are three formats to choose from:
* "full" reST table layout:
```restructuredtext
+--------------------------+------------+---------------------------------------------+---------------------+
| Function names | Arity | Input types | Output type |
+==========================+============+=============================================+=====================+
| equal, not_equal | Binary | Numeric, Temporal, Binary- and String-like | Boolean |
+--------------------------+------------+---------------------------------------------+---------------------+
| greater, greater_equal, | Binary | Numeric, Temporal, Binary- and String-like | Boolean |
| less, less_equal | | | |
+--------------------------+------------+---------------------------------------------+---------------------+
```
* "simplified" reST table layout:
```restructuredtext
========================================= =========== ============================================== =================
Function names Arity Input types Output type
========================================= =========== ============================================== =================
equal, not_equal Binary Numeric, Temporal, Binary- and String-like Boolean
greater, greater_equal, less, less_equal Binary Numeric, Temporal, Binary- and String-like Boolean
========================================= =========== ============================================== =================
```
* list table layout:
```restructuredtext
.. list-table::
:header-rows: 1
* - Function names
- Arity
- Input types
- Output type
* - equal, not_equal
- Binary
- Numeric, Temporal, Binary- and String-like
- Boolean
* - greater, greater_equal, less, less_equal
- Binary
- Numeric, Temporal, Binary- and String-like
- Boolean
```
The simplified table layout doesn't allow multi-line cells or cell fusion. It's also not much easier to edit than full table layout.
The list table layout isn't easily reviewable in source format, you have to build the docs to see clearly what happens.
Personally, I would favour the full table layout. Mostly you need to get used to it.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] pitrou commented on a change in pull request #7695: ARROW-8989: [C++][Doc] Document available compute functions
Posted by GitBox <gi...@apache.org>.
pitrou commented on a change in pull request #7695:
URL: https://github.com/apache/arrow/pull/7695#discussion_r453000458
##########
File path: docs/source/cpp/compute.rst
##########
@@ -0,0 +1,419 @@
+.. Licensed to the Apache Software Foundation (ASF) under one
+.. or more contributor license agreements. See the NOTICE file
+.. distributed with this work for additional information
+.. regarding copyright ownership. The ASF licenses this file
+.. to you under the Apache License, Version 2.0 (the
+.. "License"); you may not use this file except in compliance
+.. with the License. You may obtain a copy of the License at
+
+.. http://www.apache.org/licenses/LICENSE-2.0
+
+.. Unless required by applicable law or agreed to in writing,
+.. software distributed under the License is distributed on an
+.. "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+.. KIND, either express or implied. See the License for the
+.. specific language governing permissions and limitations
+.. under the License.
+
+.. default-domain:: cpp
+.. highlight:: cpp
+.. cpp:namespace:: arrow::compute
+
+=================
+Compute Functions
+=================
+
+.. TODO: describe API and how to invoke compute functions
+
+Available functions
+===================
+
+Aggregations
+------------
+
++--------------------------+------------+--------------------+-----------------------+--------------------------------------------+
+| Function name | Arity | Input types | Output type | Options class |
++==========================+============+====================+=======================+============================================+
+| count | Unary | Any | Scalar Int64 | :struct:`CountOptions` |
Review comment:
It will. That's the whole point of this markup, and the API docs I added :-)
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] wesm edited a comment on pull request #7695: ARROW-8989: [C++][Doc] Document available compute functions
Posted by GitBox <gi...@apache.org>.
wesm edited a comment on pull request #7695:
URL: https://github.com/apache/arrow/pull/7695#issuecomment-657697264
I guess it's a matter of perspective. I don't feel at all comfortable editing the RST tables, whereas I would be fine editing a CSV file. Many text editors (e.g. emacs, vim) have a CSV editing mode so in many cases a separate tool is not needed
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] jorisvandenbossche commented on a change in pull request #7695: ARROW-8989: [C++][Doc] Document available compute functions
Posted by GitBox <gi...@apache.org>.
jorisvandenbossche commented on a change in pull request #7695:
URL: https://github.com/apache/arrow/pull/7695#discussion_r453541169
##########
File path: docs/source/cpp/compute.rst
##########
@@ -0,0 +1,419 @@
+.. Licensed to the Apache Software Foundation (ASF) under one
+.. or more contributor license agreements. See the NOTICE file
+.. distributed with this work for additional information
+.. regarding copyright ownership. The ASF licenses this file
+.. to you under the Apache License, Version 2.0 (the
+.. "License"); you may not use this file except in compliance
+.. with the License. You may obtain a copy of the License at
+
+.. http://www.apache.org/licenses/LICENSE-2.0
+
+.. Unless required by applicable law or agreed to in writing,
+.. software distributed under the License is distributed on an
+.. "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+.. KIND, either express or implied. See the License for the
+.. specific language governing permissions and limitations
+.. under the License.
+
+.. default-domain:: cpp
+.. highlight:: cpp
+.. cpp:namespace:: arrow::compute
+
+=================
+Compute Functions
+=================
+
+.. TODO: describe API and how to invoke compute functions
+
+Available functions
+===================
+
+Aggregations
+------------
+
++--------------------------+------------+--------------------+-----------------------+--------------------------------------------+
+| Function name | Arity | Input types | Output type | Options class |
++==========================+============+====================+=======================+============================================+
+| count | Unary | Any | Scalar Int64 | :struct:`CountOptions` |
++--------------------------+------------+--------------------+-----------------------+--------------------------------------------+
+| mean | Unary | Numeric | Scalar Float64 | |
++--------------------------+------------+--------------------+-----------------------+--------------------------------------------+
+| minmax | Unary | Numeric | Scalar Struct (1) | :struct:`MinMaxOptions` |
++--------------------------+------------+--------------------+-----------------------+--------------------------------------------+
+| sum | Unary | Numeric | Scalar Numeric (2) | |
++--------------------------+------------+--------------------+-----------------------+--------------------------------------------+
+
+Notes:
+
+* \(1) Output is a ``{"min": input type, "max": input type}`` Struct
+
+* \(2) Output is Int64, UInt64 or Float64, depending on the input type
+
+
+Element-wise ("scalar") functions
+---------------------------------
+
+Arithmetic functions
+~~~~~~~~~~~~~~~~~~~~
+
+Those functions expect two inputs of the same type and apply a given binary
+operation to each pair of elements gathered from the inputs. Each function
+is also available in an overflow-checking variant, suffixed ``_checked``.
+
+If any of the input elements in a pair is null, the corresponding output
+element is null.
+
++--------------------------+------------+--------------------+---------------------+
+| Function name | Arity | Input types | Output type |
++==========================+============+====================+=====================+
+| add | Binary | Numeric | Numeric |
++--------------------------+------------+--------------------+---------------------+
+| add_checked | Binary | Numeric | Numeric |
++--------------------------+------------+--------------------+---------------------+
+| multiply | Binary | Numeric | Numeric |
++--------------------------+------------+--------------------+---------------------+
+| multiply_checked | Binary | Numeric | Numeric |
++--------------------------+------------+--------------------+---------------------+
+| subtract | Binary | Numeric | Numeric |
++--------------------------+------------+--------------------+---------------------+
+| subtract_checked | Binary | Numeric | Numeric |
++--------------------------+------------+--------------------+---------------------+
+
+Comparisons
+~~~~~~~~~~~
+
+Those functions expect two inputs of the same type and apply a given
+comparison operator. If any of the input elements in a pair is null,
+the corresponding output element is null.
+
++--------------------------+------------+---------------------------------+---------------------+
+| Function names | Arity | Input types | Output type |
++==========================+============+=================================+=====================+
+| equal, not_equal | Binary | Numeric | Boolean |
++--------------------------+------------+---------------------------------+---------------------+
+| equal, not_equal | Binary | Binary- and String-like | Boolean |
++--------------------------+------------+---------------------------------+---------------------+
+| equal, not_equal | Binary | Temporal | Boolean |
++--------------------------+------------+---------------------------------+---------------------+
+| greater, greater_equal, | Binary | Numeric | Boolean |
+| less, less_equal | | | |
++--------------------------+------------+---------------------------------+---------------------+
+| greater, greater_equal, | Binary | Binary- and String-like | Boolean |
+| less, less_equal | | | |
++--------------------------+------------+---------------------------------+---------------------+
+| greater, greater_equal, | Binary | Temporal | Boolean |
+| less, less_equal | | | |
++--------------------------+------------+---------------------------------+---------------------+
+
+Logical functions
+~~~~~~~~~~~~~~~~~~
+
+The normal behaviour for these functions is to emit a null if any of the
+inputs is null.
+
+Some of them are also available in a "`Kleene logic`_" variant (suffixed
Review comment:
Yes, numpy doesn't do this. But, numpy doesn't know the concept of "nulls", so it's actually not even a question for numpy (and pandas only started to do this in the new experimental opt-in nullable dtypes, but for its default numpy-baed dtypes, ti follows numpy)
Kleene logic is also what Julia does by default when working with their "Missing" union type (https://docs.julialang.org/en/v1/manual/missing/index.html#Logical-operators-1, they use the term "three-valued logic").
I am personally not aware of a system that has "null" support, and doesn't use Kleene logic.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] pitrou commented on pull request #7695: ARROW-8989: [C++][Doc] Document available compute functions
Posted by GitBox <gi...@apache.org>.
pitrou commented on pull request #7695:
URL: https://github.com/apache/arrow/pull/7695#issuecomment-657557007
> Does that include tables which use the file: directive to refer to an out-of-line table source?
Do you mean you would like to edit a CSV table in a spreadsheet?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] wesm commented on pull request #7695: ARROW-8989: [C++][Doc] Document available compute functions
Posted by GitBox <gi...@apache.org>.
wesm commented on pull request #7695:
URL: https://github.com/apache/arrow/pull/7695#issuecomment-657697264
I guess it's a matter of perspective. I don't feel at all comfortable editing the RST tables, whereas I would be fine editing a CSV file
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] pitrou edited a comment on pull request #7695: ARROW-8989: [C++][Doc] Document available compute functions
Posted by GitBox <gi...@apache.org>.
pitrou edited a comment on pull request #7695:
URL: https://github.com/apache/arrow/pull/7695#issuecomment-657467563
So there are three formats to choose from:
* "full" reST table layout:
```restructuredtext
+--------------------------+------------+---------------------------------------------+---------------------+
| Function names | Arity | Input types | Output type |
+==========================+============+=============================================+=====================+
| equal, not_equal | Binary | Numeric, Temporal, Binary- and String-like | Boolean |
+--------------------------+------------+---------------------------------------------+---------------------+
| greater, greater_equal, | Binary | Numeric, Temporal, Binary- and String-like | Boolean |
| less, less_equal | | | |
+--------------------------+------------+---------------------------------------------+---------------------+
```
* "simplified" reST table layout:
```restructuredtext
========================================= =========== ============================================== =================
Function names Arity Input types Output type
========================================= =========== ============================================== =================
equal, not_equal Binary Numeric, Temporal, Binary- and String-like Boolean
greater, greater_equal, less, less_equal Binary Numeric, Temporal, Binary- and String-like Boolean
========================================= =========== ============================================== =================
```
* list table layout:
```restructuredtext
.. list-table::
:header-rows: 1
* - Function names
- Arity
- Input types
- Output type
* - equal, not_equal
- Binary
- Numeric, Temporal, Binary- and String-like
- Boolean
* - greater, greater_equal, less, less_equal
- Binary
- Numeric, Temporal, Binary- and String-like
- Boolean
```
The simplified table layout doesn't allow multi-line cells or cell fusion. It's also not much easier to edit than full table layout.
The list table layout isn't easily reviewable in source format, you have to build the docs to see clearly what happens. And it will get quite unwieldy if you have 10 rows instead of 3 here.
Personally, I would favour the full table layout. Mostly you need to get used to it.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] pitrou commented on a change in pull request #7695: ARROW-8989: [C++][Doc] Document available compute functions
Posted by GitBox <gi...@apache.org>.
pitrou commented on a change in pull request #7695:
URL: https://github.com/apache/arrow/pull/7695#discussion_r453000744
##########
File path: docs/source/cpp/compute.rst
##########
@@ -0,0 +1,419 @@
+.. Licensed to the Apache Software Foundation (ASF) under one
+.. or more contributor license agreements. See the NOTICE file
+.. distributed with this work for additional information
+.. regarding copyright ownership. The ASF licenses this file
+.. to you under the Apache License, Version 2.0 (the
+.. "License"); you may not use this file except in compliance
+.. with the License. You may obtain a copy of the License at
+
+.. http://www.apache.org/licenses/LICENSE-2.0
+
+.. Unless required by applicable law or agreed to in writing,
+.. software distributed under the License is distributed on an
+.. "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+.. KIND, either express or implied. See the License for the
+.. specific language governing permissions and limitations
+.. under the License.
+
+.. default-domain:: cpp
+.. highlight:: cpp
+.. cpp:namespace:: arrow::compute
+
+=================
+Compute Functions
+=================
+
+.. TODO: describe API and how to invoke compute functions
+
+Available functions
+===================
+
+Aggregations
+------------
+
++--------------------------+------------+--------------------+-----------------------+--------------------------------------------+
+| Function name | Arity | Input types | Output type | Options class |
++==========================+============+====================+=======================+============================================+
+| count | Unary | Any | Scalar Int64 | :struct:`CountOptions` |
++--------------------------+------------+--------------------+-----------------------+--------------------------------------------+
+| mean | Unary | Numeric | Scalar Float64 | |
++--------------------------+------------+--------------------+-----------------------+--------------------------------------------+
+| minmax | Unary | Numeric | Scalar Struct (1) | :struct:`MinMaxOptions` |
++--------------------------+------------+--------------------+-----------------------+--------------------------------------------+
+| sum | Unary | Numeric | Scalar Numeric (2) | |
++--------------------------+------------+--------------------+-----------------------+--------------------------------------------+
+
+Notes:
+
+* \(1) Output is a ``{"min": input type, "max": input type}`` Struct
+
+* \(2) Output is Int64, UInt64 or Float64, depending on the input type
+
+
+Element-wise ("scalar") functions
+---------------------------------
+
+Arithmetic functions
+~~~~~~~~~~~~~~~~~~~~
+
+Those functions expect two inputs of the same type and apply a given binary
+operation to each pair of elements gathered from the inputs. Each function
+is also available in an overflow-checking variant, suffixed ``_checked``.
+
+If any of the input elements in a pair is null, the corresponding output
+element is null.
+
++--------------------------+------------+--------------------+---------------------+
+| Function name | Arity | Input types | Output type |
++==========================+============+====================+=====================+
+| add | Binary | Numeric | Numeric |
++--------------------------+------------+--------------------+---------------------+
+| add_checked | Binary | Numeric | Numeric |
++--------------------------+------------+--------------------+---------------------+
+| multiply | Binary | Numeric | Numeric |
++--------------------------+------------+--------------------+---------------------+
+| multiply_checked | Binary | Numeric | Numeric |
++--------------------------+------------+--------------------+---------------------+
+| subtract | Binary | Numeric | Numeric |
++--------------------------+------------+--------------------+---------------------+
+| subtract_checked | Binary | Numeric | Numeric |
++--------------------------+------------+--------------------+---------------------+
+
+Comparisons
+~~~~~~~~~~~
+
+Those functions expect two inputs of the same type and apply a given
+comparison operator. If any of the input elements in a pair is null,
+the corresponding output element is null.
+
++--------------------------+------------+---------------------------------+---------------------+
+| Function names | Arity | Input types | Output type |
++==========================+============+=================================+=====================+
+| equal, not_equal | Binary | Numeric | Boolean |
++--------------------------+------------+---------------------------------+---------------------+
+| equal, not_equal | Binary | Binary- and String-like | Boolean |
++--------------------------+------------+---------------------------------+---------------------+
+| equal, not_equal | Binary | Temporal | Boolean |
++--------------------------+------------+---------------------------------+---------------------+
+| greater, greater_equal, | Binary | Numeric | Boolean |
+| less, less_equal | | | |
++--------------------------+------------+---------------------------------+---------------------+
+| greater, greater_equal, | Binary | Binary- and String-like | Boolean |
+| less, less_equal | | | |
++--------------------------+------------+---------------------------------+---------------------+
+| greater, greater_equal, | Binary | Temporal | Boolean |
+| less, less_equal | | | |
++--------------------------+------------+---------------------------------+---------------------+
+
+Logical functions
+~~~~~~~~~~~~~~~~~~
+
+The normal behaviour for these functions is to emit a null if any of the
+inputs is null.
+
+Some of them are also available in a "`Kleene logic`_" variant (suffixed
Review comment:
I don't think Numpy does anything, since it doesn't have nulls. Perhaps you mean Pandas?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] pitrou commented on a change in pull request #7695: ARROW-8989: [C++][Doc] Document available compute functions
Posted by GitBox <gi...@apache.org>.
pitrou commented on a change in pull request #7695:
URL: https://github.com/apache/arrow/pull/7695#discussion_r453001677
##########
File path: docs/source/cpp/compute.rst
##########
@@ -0,0 +1,419 @@
+.. Licensed to the Apache Software Foundation (ASF) under one
+.. or more contributor license agreements. See the NOTICE file
+.. distributed with this work for additional information
+.. regarding copyright ownership. The ASF licenses this file
+.. to you under the Apache License, Version 2.0 (the
+.. "License"); you may not use this file except in compliance
+.. with the License. You may obtain a copy of the License at
+
+.. http://www.apache.org/licenses/LICENSE-2.0
+
+.. Unless required by applicable law or agreed to in writing,
+.. software distributed under the License is distributed on an
+.. "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+.. KIND, either express or implied. See the License for the
+.. specific language governing permissions and limitations
+.. under the License.
+
+.. default-domain:: cpp
+.. highlight:: cpp
+.. cpp:namespace:: arrow::compute
+
+=================
+Compute Functions
+=================
+
+.. TODO: describe API and how to invoke compute functions
+
+Available functions
+===================
+
+Aggregations
+------------
+
++--------------------------+------------+--------------------+-----------------------+--------------------------------------------+
+| Function name | Arity | Input types | Output type | Options class |
++==========================+============+====================+=======================+============================================+
+| count | Unary | Any | Scalar Int64 | :struct:`CountOptions` |
++--------------------------+------------+--------------------+-----------------------+--------------------------------------------+
+| mean | Unary | Numeric | Scalar Float64 | |
++--------------------------+------------+--------------------+-----------------------+--------------------------------------------+
+| minmax | Unary | Numeric | Scalar Struct (1) | :struct:`MinMaxOptions` |
++--------------------------+------------+--------------------+-----------------------+--------------------------------------------+
+| sum | Unary | Numeric | Scalar Numeric (2) | |
++--------------------------+------------+--------------------+-----------------------+--------------------------------------------+
+
+Notes:
+
+* \(1) Output is a ``{"min": input type, "max": input type}`` Struct
+
+* \(2) Output is Int64, UInt64 or Float64, depending on the input type
+
+
+Element-wise ("scalar") functions
+---------------------------------
+
+Arithmetic functions
+~~~~~~~~~~~~~~~~~~~~
+
+Those functions expect two inputs of the same type and apply a given binary
+operation to each pair of elements gathered from the inputs. Each function
+is also available in an overflow-checking variant, suffixed ``_checked``.
+
+If any of the input elements in a pair is null, the corresponding output
+element is null.
+
++--------------------------+------------+--------------------+---------------------+
+| Function name | Arity | Input types | Output type |
++==========================+============+====================+=====================+
+| add | Binary | Numeric | Numeric |
++--------------------------+------------+--------------------+---------------------+
+| add_checked | Binary | Numeric | Numeric |
++--------------------------+------------+--------------------+---------------------+
+| multiply | Binary | Numeric | Numeric |
++--------------------------+------------+--------------------+---------------------+
+| multiply_checked | Binary | Numeric | Numeric |
++--------------------------+------------+--------------------+---------------------+
+| subtract | Binary | Numeric | Numeric |
++--------------------------+------------+--------------------+---------------------+
+| subtract_checked | Binary | Numeric | Numeric |
++--------------------------+------------+--------------------+---------------------+
+
+Comparisons
+~~~~~~~~~~~
+
+Those functions expect two inputs of the same type and apply a given
+comparison operator. If any of the input elements in a pair is null,
+the corresponding output element is null.
+
++--------------------------+------------+---------------------------------+---------------------+
+| Function names | Arity | Input types | Output type |
++==========================+============+=================================+=====================+
+| equal, not_equal | Binary | Numeric | Boolean |
++--------------------------+------------+---------------------------------+---------------------+
+| equal, not_equal | Binary | Binary- and String-like | Boolean |
++--------------------------+------------+---------------------------------+---------------------+
+| equal, not_equal | Binary | Temporal | Boolean |
++--------------------------+------------+---------------------------------+---------------------+
+| greater, greater_equal, | Binary | Numeric | Boolean |
+| less, less_equal | | | |
++--------------------------+------------+---------------------------------+---------------------+
+| greater, greater_equal, | Binary | Binary- and String-like | Boolean |
+| less, less_equal | | | |
++--------------------------+------------+---------------------------------+---------------------+
+| greater, greater_equal, | Binary | Temporal | Boolean |
+| less, less_equal | | | |
++--------------------------+------------+---------------------------------+---------------------+
+
+Logical functions
+~~~~~~~~~~~~~~~~~~
+
+The normal behaviour for these functions is to emit a null if any of the
+inputs is null.
+
+Some of them are also available in a "`Kleene logic`_" variant (suffixed
+``_kleene``) where null is taken to mean "undefined". For those variants
+therefore:
+
+* "true AND null", "null AND true" give "null" (the result is undefined)
+* "true OR null", "null OR true" give "true"
+* "false AND null", "null AND false" give "false"
+* "false OR null", "null OR false" give "null" (the result is undefined)
+
++--------------------------+------------+--------------------+---------------------+
+| Function name | Arity | Input types | Output type |
++==========================+============+====================+=====================+
+| and | Binary | Boolean | Boolean |
++--------------------------+------------+--------------------+---------------------+
+| and_kleene | Binary | Boolean | Boolean |
++--------------------------+------------+--------------------+---------------------+
+| invert | Unary | Boolean | Boolean |
++--------------------------+------------+--------------------+---------------------+
+| or | Binary | Boolean | Boolean |
++--------------------------+------------+--------------------+---------------------+
+| or_kleene | Binary | Boolean | Boolean |
++--------------------------+------------+--------------------+---------------------+
+| xor | Binary | Boolean | Boolean |
++--------------------------+------------+--------------------+---------------------+
+
+.. _Kleene logic: https://en.wikipedia.org/wiki/Three-valued_logic#Kleene_and_Priest_logics
+
+String functions
+~~~~~~~~~~~~~~~~
+
++--------------------------+------------+--------------------+---------------------+---------+
+| Function name | Arity | Input types | Output type | Notes |
++==========================+============+====================+=====================+=========+
+| ascii_length | Unary | String-like | Int32 or Int64 | \(1) |
++--------------------------+------------+--------------------+---------------------+---------+
+| ascii_lower | Unary | String-like | String-like | \(2) |
++--------------------------+------------+--------------------+---------------------+---------+
+| ascii_upper | Unary | String-like | String-like | \(2) |
++--------------------------+------------+--------------------+---------------------+---------+
+| utf8_lower | Unary | String-like | String-like | \(3) |
++--------------------------+------------+--------------------+---------------------+---------+
+| utf8_upper | Unary | String-like | String-like | \(3) |
++--------------------------+------------+--------------------+---------------------+---------+
+
+* \(1) Output is the physical length in bytes of each input element.
+
+* \(2) Each ASCII character in the input is converted to lowercase or
+ uppercase. Non-ASCII characters are left untouched.
+
+* \(3) Each UTF8-encoded character in the input is converted to lowercase or
+ uppercase.
+
+Containment tests
+~~~~~~~~~~~~~~~~~
+
++--------------------------+------------+----------------------------------+-----------------------+--------------------------------------------+
+| Function name | Arity | Input types | Output type | Options class |
++==========================+============+==================================+=======================+============================================+
+| binary_contains_exact | Unary | String-like | Boolean (1) | :struct:`BinaryContainsExactOptions` |
++--------------------------+------------+----------------------------------+-----------------------+--------------------------------------------+
+| isin | Unary | Binary- and String-like | Boolean (2) | :struct:`SetLookupOptions` |
++--------------------------+------------+----------------------------------+-----------------------+--------------------------------------------+
+| isin | Unary | Null | Boolean (2) | :struct:`SetLookupOptions` |
++--------------------------+------------+----------------------------------+-----------------------+--------------------------------------------+
+| isin | Unary | Boolean,Numeric, Temporal | Boolean (2) | :struct:`SetLookupOptions` |
++--------------------------+------------+----------------------------------+-----------------------+--------------------------------------------+
+| match | Unary | Binary- and String-like | Int32 (3) | :struct:`SetLookupOptions` |
++--------------------------+------------+----------------------------------+-----------------------+--------------------------------------------+
+| match | Unary | Null | Int32 (3) | :struct:`SetLookupOptions` |
++--------------------------+------------+----------------------------------+-----------------------+--------------------------------------------+
+| match | Unary | Boolean,Numeric, Temporal | Int32 (3) | :struct:`SetLookupOptions` |
++--------------------------+------------+----------------------------------+-----------------------+--------------------------------------------+
+
+* \(1) Output is true iff :member:`BinaryContainsExactOptions::pattern`
+ is a substring of the corresponding input element.
+
+* \(2) Output is true iff the corresponding input element is equal to one
+ of the elements in :member:`SetLookupOptions::value_set`.
+
+* \(3) Output is the index of the corresponding input element in
+ :member:`SetLookupOptions::value_set`, if found there. Otherwise,
+ output is null.
+
+Structural transforms
+~~~~~~~~~~~~~~~~~~~~~
+
++--------------------------+------------+--------------------+---------------------+---------+
+| Function name | Arity | Input types | Output type | Notes |
++==========================+============+====================+=====================+=========+
+| is_null | Unary | Any | Boolean | \(1) |
++--------------------------+------------+--------------------+---------------------+---------+
+| is_valid | Unary | Any | Boolean | \(2) |
++--------------------------+------------+--------------------+---------------------+---------+
+| list_value_lengths | Unary | List-like | Int32 or Int64 | \(3) |
++--------------------------+------------+--------------------+---------------------+---------+
+
+* \(1) Output is true iff the corresponding input element is non-null.
+
+* \(2) Output is true iff the corresponding input element is null.
+
+* \(3) Each output element is the length of the corresponding input element
+ (null if input is null).
+
+Conversions
+~~~~~~~~~~~
+
+A general conversion function named ``cast`` is provided which accepts a large
+number of input and output types. The type to cast to can be passed in a
+:struct:`CastOptions` instance.
Review comment:
You get the answer by clicking on the `CastOptions` link not rendered :-) I'm not sure it's worth repeating here.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] pitrou commented on a change in pull request #7695: ARROW-8989: [C++][Doc] Document available compute functions
Posted by GitBox <gi...@apache.org>.
pitrou commented on a change in pull request #7695:
URL: https://github.com/apache/arrow/pull/7695#discussion_r453571743
##########
File path: docs/source/cpp/compute.rst
##########
@@ -0,0 +1,419 @@
+.. Licensed to the Apache Software Foundation (ASF) under one
+.. or more contributor license agreements. See the NOTICE file
+.. distributed with this work for additional information
+.. regarding copyright ownership. The ASF licenses this file
+.. to you under the Apache License, Version 2.0 (the
+.. "License"); you may not use this file except in compliance
+.. with the License. You may obtain a copy of the License at
+
+.. http://www.apache.org/licenses/LICENSE-2.0
+
+.. Unless required by applicable law or agreed to in writing,
+.. software distributed under the License is distributed on an
+.. "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+.. KIND, either express or implied. See the License for the
+.. specific language governing permissions and limitations
+.. under the License.
+
+.. default-domain:: cpp
+.. highlight:: cpp
+.. cpp:namespace:: arrow::compute
+
+=================
+Compute Functions
+=================
+
+.. TODO: describe API and how to invoke compute functions
+
+Available functions
+===================
+
+Aggregations
+------------
+
++--------------------------+------------+--------------------+-----------------------+--------------------------------------------+
+| Function name | Arity | Input types | Output type | Options class |
++==========================+============+====================+=======================+============================================+
+| count | Unary | Any | Scalar Int64 | :struct:`CountOptions` |
++--------------------------+------------+--------------------+-----------------------+--------------------------------------------+
+| mean | Unary | Numeric | Scalar Float64 | |
++--------------------------+------------+--------------------+-----------------------+--------------------------------------------+
+| minmax | Unary | Numeric | Scalar Struct (1) | :struct:`MinMaxOptions` |
++--------------------------+------------+--------------------+-----------------------+--------------------------------------------+
+| sum | Unary | Numeric | Scalar Numeric (2) | |
++--------------------------+------------+--------------------+-----------------------+--------------------------------------------+
+
+Notes:
+
+* \(1) Output is a ``{"min": input type, "max": input type}`` Struct
+
+* \(2) Output is Int64, UInt64 or Float64, depending on the input type
+
+
+Element-wise ("scalar") functions
+---------------------------------
+
+Arithmetic functions
+~~~~~~~~~~~~~~~~~~~~
+
+Those functions expect two inputs of the same type and apply a given binary
+operation to each pair of elements gathered from the inputs. Each function
+is also available in an overflow-checking variant, suffixed ``_checked``.
Review comment:
I'd also recommend the checked versions. But I'm not sure we need to recommend anything. AFAIU, the compute layer isn't supposed to be a user-facing API like Pandas.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] wesm commented on pull request #7695: ARROW-8989: [C++][Doc] Document available compute functions
Posted by GitBox <gi...@apache.org>.
wesm commented on pull request #7695:
URL: https://github.com/apache/arrow/pull/7695#issuecomment-657692574
> Do you mean you would like to edit a CSV table in a spreadsheet?
Yeah I think that would be the idea
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] wesm closed pull request #7695: ARROW-8989: [C++][Doc] Document available compute functions
Posted by GitBox <gi...@apache.org>.
wesm closed pull request #7695:
URL: https://github.com/apache/arrow/pull/7695
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] pitrou commented on a change in pull request #7695: ARROW-8989: [C++][Doc] Document available compute functions
Posted by GitBox <gi...@apache.org>.
pitrou commented on a change in pull request #7695:
URL: https://github.com/apache/arrow/pull/7695#discussion_r453525042
##########
File path: docs/source/cpp/compute.rst
##########
@@ -0,0 +1,419 @@
+.. Licensed to the Apache Software Foundation (ASF) under one
+.. or more contributor license agreements. See the NOTICE file
+.. distributed with this work for additional information
+.. regarding copyright ownership. The ASF licenses this file
+.. to you under the Apache License, Version 2.0 (the
+.. "License"); you may not use this file except in compliance
+.. with the License. You may obtain a copy of the License at
+
+.. http://www.apache.org/licenses/LICENSE-2.0
+
+.. Unless required by applicable law or agreed to in writing,
+.. software distributed under the License is distributed on an
+.. "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+.. KIND, either express or implied. See the License for the
+.. specific language governing permissions and limitations
+.. under the License.
+
+.. default-domain:: cpp
+.. highlight:: cpp
+.. cpp:namespace:: arrow::compute
+
+=================
+Compute Functions
+=================
+
+.. TODO: describe API and how to invoke compute functions
+
+Available functions
+===================
+
+Aggregations
+------------
+
++--------------------------+------------+--------------------+-----------------------+--------------------------------------------+
Review comment:
I don't find JSON files easy to edit at all. I'd much rather keep the reST table format. It's not the most natural format to edit in a text editor, but it's still reasonable if your editor has a [block selection mode](https://stackoverflow.com/questions/1802616/how-to-select-columns-in-editors-atom-notepad-kate-vim-sublime-textpad-et).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] pitrou commented on a change in pull request #7695: ARROW-8989: [C++][Doc] Document available compute functions
Posted by GitBox <gi...@apache.org>.
pitrou commented on a change in pull request #7695:
URL: https://github.com/apache/arrow/pull/7695#discussion_r453520967
##########
File path: docs/source/cpp/compute.rst
##########
@@ -0,0 +1,419 @@
+.. Licensed to the Apache Software Foundation (ASF) under one
+.. or more contributor license agreements. See the NOTICE file
+.. distributed with this work for additional information
+.. regarding copyright ownership. The ASF licenses this file
+.. to you under the Apache License, Version 2.0 (the
+.. "License"); you may not use this file except in compliance
+.. with the License. You may obtain a copy of the License at
+
+.. http://www.apache.org/licenses/LICENSE-2.0
+
+.. Unless required by applicable law or agreed to in writing,
+.. software distributed under the License is distributed on an
+.. "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+.. KIND, either express or implied. See the License for the
+.. specific language governing permissions and limitations
+.. under the License.
+
+.. default-domain:: cpp
+.. highlight:: cpp
+.. cpp:namespace:: arrow::compute
+
+=================
+Compute Functions
+=================
+
+.. TODO: describe API and how to invoke compute functions
+
+Available functions
+===================
+
+Aggregations
+------------
+
++--------------------------+------------+--------------------+-----------------------+--------------------------------------------+
+| Function name | Arity | Input types | Output type | Options class |
++==========================+============+====================+=======================+============================================+
+| count | Unary | Any | Scalar Int64 | :struct:`CountOptions` |
++--------------------------+------------+--------------------+-----------------------+--------------------------------------------+
+| mean | Unary | Numeric | Scalar Float64 | |
++--------------------------+------------+--------------------+-----------------------+--------------------------------------------+
+| minmax | Unary | Numeric | Scalar Struct (1) | :struct:`MinMaxOptions` |
++--------------------------+------------+--------------------+-----------------------+--------------------------------------------+
+| sum | Unary | Numeric | Scalar Numeric (2) | |
++--------------------------+------------+--------------------+-----------------------+--------------------------------------------+
+
+Notes:
+
+* \(1) Output is a ``{"min": input type, "max": input type}`` Struct
+
+* \(2) Output is Int64, UInt64 or Float64, depending on the input type
+
+
+Element-wise ("scalar") functions
+---------------------------------
+
+Arithmetic functions
+~~~~~~~~~~~~~~~~~~~~
+
+Those functions expect two inputs of the same type and apply a given binary
+operation to each pair of elements gathered from the inputs. Each function
+is also available in an overflow-checking variant, suffixed ``_checked``.
+
+If any of the input elements in a pair is null, the corresponding output
+element is null.
+
++--------------------------+------------+--------------------+---------------------+
+| Function name | Arity | Input types | Output type |
++==========================+============+====================+=====================+
+| add | Binary | Numeric | Numeric |
++--------------------------+------------+--------------------+---------------------+
+| add_checked | Binary | Numeric | Numeric |
++--------------------------+------------+--------------------+---------------------+
+| multiply | Binary | Numeric | Numeric |
++--------------------------+------------+--------------------+---------------------+
+| multiply_checked | Binary | Numeric | Numeric |
++--------------------------+------------+--------------------+---------------------+
+| subtract | Binary | Numeric | Numeric |
++--------------------------+------------+--------------------+---------------------+
+| subtract_checked | Binary | Numeric | Numeric |
++--------------------------+------------+--------------------+---------------------+
+
+Comparisons
+~~~~~~~~~~~
+
+Those functions expect two inputs of the same type and apply a given
+comparison operator. If any of the input elements in a pair is null,
+the corresponding output element is null.
+
++--------------------------+------------+---------------------------------+---------------------+
+| Function names | Arity | Input types | Output type |
++==========================+============+=================================+=====================+
+| equal, not_equal | Binary | Numeric | Boolean |
++--------------------------+------------+---------------------------------+---------------------+
+| equal, not_equal | Binary | Binary- and String-like | Boolean |
Review comment:
To fit all types. I could also put them all on a single line, but the table may end up quite larger.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] pitrou commented on pull request #7695: ARROW-8989: [C++][Doc] Document available compute functions
Posted by GitBox <gi...@apache.org>.
pitrou commented on pull request #7695:
URL: https://github.com/apache/arrow/pull/7695#issuecomment-657449500
I could give a try to list tables, but otherwise I think CSV or JSON would be a major PITA to edit later.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] github-actions[bot] commented on pull request #7695: ARROW-8989: [C++][Doc] Document available compute functions
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on pull request #7695:
URL: https://github.com/apache/arrow/pull/7695#issuecomment-656304218
https://issues.apache.org/jira/browse/ARROW-8989
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] pitrou commented on pull request #7695: ARROW-8989: [C++][Doc] Document available compute functions
Posted by GitBox <gi...@apache.org>.
pitrou commented on pull request #7695:
URL: https://github.com/apache/arrow/pull/7695#issuecomment-657454065
Ok, list tables may be workable, but they don't make it easy to review docs simply by reading the source reST code. I'd rather keep the usual reST layout, unless you're using an editor that doesn't have a block selection mode at all.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] pitrou commented on pull request #7695: ARROW-8989: [C++][Doc] Document available compute functions
Posted by GitBox <gi...@apache.org>.
pitrou commented on pull request #7695:
URL: https://github.com/apache/arrow/pull/7695#issuecomment-657696511
I think that would be a rather terrible doc writing experience: each time you want to update those docs, you have to fire a different tool for some part of the page. Also, you would have a number of small CSV files to open, not a single one...
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] pitrou commented on pull request #7695: ARROW-8989: [C++][Doc] Document available compute functions
Posted by GitBox <gi...@apache.org>.
pitrou commented on pull request #7695:
URL: https://github.com/apache/arrow/pull/7695#issuecomment-656300001
@nealrichardson You should like this.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] pitrou commented on a change in pull request #7695: ARROW-8989: [C++][Doc] Document available compute functions
Posted by GitBox <gi...@apache.org>.
pitrou commented on a change in pull request #7695:
URL: https://github.com/apache/arrow/pull/7695#discussion_r453000941
##########
File path: docs/source/cpp/compute.rst
##########
@@ -0,0 +1,419 @@
+.. Licensed to the Apache Software Foundation (ASF) under one
+.. or more contributor license agreements. See the NOTICE file
+.. distributed with this work for additional information
+.. regarding copyright ownership. The ASF licenses this file
+.. to you under the Apache License, Version 2.0 (the
+.. "License"); you may not use this file except in compliance
+.. with the License. You may obtain a copy of the License at
+
+.. http://www.apache.org/licenses/LICENSE-2.0
+
+.. Unless required by applicable law or agreed to in writing,
+.. software distributed under the License is distributed on an
+.. "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+.. KIND, either express or implied. See the License for the
+.. specific language governing permissions and limitations
+.. under the License.
+
+.. default-domain:: cpp
+.. highlight:: cpp
+.. cpp:namespace:: arrow::compute
+
+=================
+Compute Functions
+=================
+
+.. TODO: describe API and how to invoke compute functions
+
+Available functions
+===================
+
+Aggregations
+------------
+
++--------------------------+------------+--------------------+-----------------------+--------------------------------------------+
+| Function name | Arity | Input types | Output type | Options class |
++==========================+============+====================+=======================+============================================+
+| count | Unary | Any | Scalar Int64 | :struct:`CountOptions` |
++--------------------------+------------+--------------------+-----------------------+--------------------------------------------+
+| mean | Unary | Numeric | Scalar Float64 | |
++--------------------------+------------+--------------------+-----------------------+--------------------------------------------+
+| minmax | Unary | Numeric | Scalar Struct (1) | :struct:`MinMaxOptions` |
++--------------------------+------------+--------------------+-----------------------+--------------------------------------------+
+| sum | Unary | Numeric | Scalar Numeric (2) | |
++--------------------------+------------+--------------------+-----------------------+--------------------------------------------+
+
+Notes:
+
+* \(1) Output is a ``{"min": input type, "max": input type}`` Struct
+
+* \(2) Output is Int64, UInt64 or Float64, depending on the input type
+
+
+Element-wise ("scalar") functions
+---------------------------------
+
+Arithmetic functions
+~~~~~~~~~~~~~~~~~~~~
+
+Those functions expect two inputs of the same type and apply a given binary
+operation to each pair of elements gathered from the inputs. Each function
+is also available in an overflow-checking variant, suffixed ``_checked``.
+
+If any of the input elements in a pair is null, the corresponding output
+element is null.
+
++--------------------------+------------+--------------------+---------------------+
+| Function name | Arity | Input types | Output type |
++==========================+============+====================+=====================+
+| add | Binary | Numeric | Numeric |
++--------------------------+------------+--------------------+---------------------+
+| add_checked | Binary | Numeric | Numeric |
++--------------------------+------------+--------------------+---------------------+
+| multiply | Binary | Numeric | Numeric |
++--------------------------+------------+--------------------+---------------------+
+| multiply_checked | Binary | Numeric | Numeric |
++--------------------------+------------+--------------------+---------------------+
+| subtract | Binary | Numeric | Numeric |
++--------------------------+------------+--------------------+---------------------+
+| subtract_checked | Binary | Numeric | Numeric |
++--------------------------+------------+--------------------+---------------------+
+
+Comparisons
+~~~~~~~~~~~
+
+Those functions expect two inputs of the same type and apply a given
+comparison operator. If any of the input elements in a pair is null,
+the corresponding output element is null.
+
++--------------------------+------------+---------------------------------+---------------------+
+| Function names | Arity | Input types | Output type |
++==========================+============+=================================+=====================+
+| equal, not_equal | Binary | Numeric | Boolean |
++--------------------------+------------+---------------------------------+---------------------+
+| equal, not_equal | Binary | Binary- and String-like | Boolean |
++--------------------------+------------+---------------------------------+---------------------+
+| equal, not_equal | Binary | Temporal | Boolean |
++--------------------------+------------+---------------------------------+---------------------+
+| greater, greater_equal, | Binary | Numeric | Boolean |
+| less, less_equal | | | |
++--------------------------+------------+---------------------------------+---------------------+
+| greater, greater_equal, | Binary | Binary- and String-like | Boolean |
+| less, less_equal | | | |
++--------------------------+------------+---------------------------------+---------------------+
+| greater, greater_equal, | Binary | Temporal | Boolean |
+| less, less_equal | | | |
++--------------------------+------------+---------------------------------+---------------------+
+
+Logical functions
+~~~~~~~~~~~~~~~~~~
+
+The normal behaviour for these functions is to emit a null if any of the
+inputs is null.
+
+Some of them are also available in a "`Kleene logic`_" variant (suffixed
+``_kleene``) where null is taken to mean "undefined". For those variants
+therefore:
+
+* "true AND null", "null AND true" give "null" (the result is undefined)
Review comment:
The link to "Kleene logic" show the truth tables. I'm not sure it's worth repeating them here.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] pitrou commented on a change in pull request #7695: ARROW-8989: [C++][Doc] Document available compute functions
Posted by GitBox <gi...@apache.org>.
pitrou commented on a change in pull request #7695:
URL: https://github.com/apache/arrow/pull/7695#discussion_r453001192
##########
File path: docs/source/cpp/compute.rst
##########
@@ -0,0 +1,419 @@
+.. Licensed to the Apache Software Foundation (ASF) under one
+.. or more contributor license agreements. See the NOTICE file
+.. distributed with this work for additional information
+.. regarding copyright ownership. The ASF licenses this file
+.. to you under the Apache License, Version 2.0 (the
+.. "License"); you may not use this file except in compliance
+.. with the License. You may obtain a copy of the License at
+
+.. http://www.apache.org/licenses/LICENSE-2.0
+
+.. Unless required by applicable law or agreed to in writing,
+.. software distributed under the License is distributed on an
+.. "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+.. KIND, either express or implied. See the License for the
+.. specific language governing permissions and limitations
+.. under the License.
+
+.. default-domain:: cpp
+.. highlight:: cpp
+.. cpp:namespace:: arrow::compute
+
+=================
+Compute Functions
+=================
+
+.. TODO: describe API and how to invoke compute functions
+
+Available functions
+===================
+
+Aggregations
+------------
+
++--------------------------+------------+--------------------+-----------------------+--------------------------------------------+
+| Function name | Arity | Input types | Output type | Options class |
++==========================+============+====================+=======================+============================================+
+| count | Unary | Any | Scalar Int64 | :struct:`CountOptions` |
++--------------------------+------------+--------------------+-----------------------+--------------------------------------------+
+| mean | Unary | Numeric | Scalar Float64 | |
++--------------------------+------------+--------------------+-----------------------+--------------------------------------------+
+| minmax | Unary | Numeric | Scalar Struct (1) | :struct:`MinMaxOptions` |
++--------------------------+------------+--------------------+-----------------------+--------------------------------------------+
+| sum | Unary | Numeric | Scalar Numeric (2) | |
++--------------------------+------------+--------------------+-----------------------+--------------------------------------------+
+
+Notes:
+
+* \(1) Output is a ``{"min": input type, "max": input type}`` Struct
+
+* \(2) Output is Int64, UInt64 or Float64, depending on the input type
+
+
+Element-wise ("scalar") functions
+---------------------------------
+
+Arithmetic functions
+~~~~~~~~~~~~~~~~~~~~
+
+Those functions expect two inputs of the same type and apply a given binary
+operation to each pair of elements gathered from the inputs. Each function
+is also available in an overflow-checking variant, suffixed ``_checked``.
+
+If any of the input elements in a pair is null, the corresponding output
+element is null.
+
++--------------------------+------------+--------------------+---------------------+
+| Function name | Arity | Input types | Output type |
++==========================+============+====================+=====================+
+| add | Binary | Numeric | Numeric |
++--------------------------+------------+--------------------+---------------------+
+| add_checked | Binary | Numeric | Numeric |
++--------------------------+------------+--------------------+---------------------+
+| multiply | Binary | Numeric | Numeric |
++--------------------------+------------+--------------------+---------------------+
+| multiply_checked | Binary | Numeric | Numeric |
++--------------------------+------------+--------------------+---------------------+
+| subtract | Binary | Numeric | Numeric |
++--------------------------+------------+--------------------+---------------------+
+| subtract_checked | Binary | Numeric | Numeric |
++--------------------------+------------+--------------------+---------------------+
+
+Comparisons
+~~~~~~~~~~~
+
+Those functions expect two inputs of the same type and apply a given
+comparison operator. If any of the input elements in a pair is null,
+the corresponding output element is null.
+
++--------------------------+------------+---------------------------------+---------------------+
+| Function names | Arity | Input types | Output type |
++==========================+============+=================================+=====================+
+| equal, not_equal | Binary | Numeric | Boolean |
++--------------------------+------------+---------------------------------+---------------------+
+| equal, not_equal | Binary | Binary- and String-like | Boolean |
++--------------------------+------------+---------------------------------+---------------------+
+| equal, not_equal | Binary | Temporal | Boolean |
++--------------------------+------------+---------------------------------+---------------------+
+| greater, greater_equal, | Binary | Numeric | Boolean |
+| less, less_equal | | | |
++--------------------------+------------+---------------------------------+---------------------+
+| greater, greater_equal, | Binary | Binary- and String-like | Boolean |
+| less, less_equal | | | |
++--------------------------+------------+---------------------------------+---------------------+
+| greater, greater_equal, | Binary | Temporal | Boolean |
+| less, less_equal | | | |
++--------------------------+------------+---------------------------------+---------------------+
+
+Logical functions
+~~~~~~~~~~~~~~~~~~
+
+The normal behaviour for these functions is to emit a null if any of the
+inputs is null.
+
+Some of them are also available in a "`Kleene logic`_" variant (suffixed
+``_kleene``) where null is taken to mean "undefined". For those variants
+therefore:
+
+* "true AND null", "null AND true" give "null" (the result is undefined)
+* "true OR null", "null OR true" give "true"
+* "false AND null", "null AND false" give "false"
+* "false OR null", "null OR false" give "null" (the result is undefined)
+
++--------------------------+------------+--------------------+---------------------+
+| Function name | Arity | Input types | Output type |
++==========================+============+====================+=====================+
+| and | Binary | Boolean | Boolean |
++--------------------------+------------+--------------------+---------------------+
+| and_kleene | Binary | Boolean | Boolean |
++--------------------------+------------+--------------------+---------------------+
+| invert | Unary | Boolean | Boolean |
++--------------------------+------------+--------------------+---------------------+
+| or | Binary | Boolean | Boolean |
++--------------------------+------------+--------------------+---------------------+
+| or_kleene | Binary | Boolean | Boolean |
++--------------------------+------------+--------------------+---------------------+
+| xor | Binary | Boolean | Boolean |
++--------------------------+------------+--------------------+---------------------+
+
+.. _Kleene logic: https://en.wikipedia.org/wiki/Three-valued_logic#Kleene_and_Priest_logics
+
+String functions
+~~~~~~~~~~~~~~~~
+
++--------------------------+------------+--------------------+---------------------+---------+
+| Function name | Arity | Input types | Output type | Notes |
++==========================+============+====================+=====================+=========+
+| ascii_length | Unary | String-like | Int32 or Int64 | \(1) |
++--------------------------+------------+--------------------+---------------------+---------+
+| ascii_lower | Unary | String-like | String-like | \(2) |
++--------------------------+------------+--------------------+---------------------+---------+
+| ascii_upper | Unary | String-like | String-like | \(2) |
++--------------------------+------------+--------------------+---------------------+---------+
+| utf8_lower | Unary | String-like | String-like | \(3) |
++--------------------------+------------+--------------------+---------------------+---------+
+| utf8_upper | Unary | String-like | String-like | \(3) |
++--------------------------+------------+--------------------+---------------------+---------+
+
+* \(1) Output is the physical length in bytes of each input element.
+
+* \(2) Each ASCII character in the input is converted to lowercase or
+ uppercase. Non-ASCII characters are left untouched.
+
+* \(3) Each UTF8-encoded character in the input is converted to lowercase or
+ uppercase.
+
+Containment tests
+~~~~~~~~~~~~~~~~~
+
++--------------------------+------------+----------------------------------+-----------------------+--------------------------------------------+
+| Function name | Arity | Input types | Output type | Options class |
++==========================+============+==================================+=======================+============================================+
+| binary_contains_exact | Unary | String-like | Boolean (1) | :struct:`BinaryContainsExactOptions` |
++--------------------------+------------+----------------------------------+-----------------------+--------------------------------------------+
+| isin | Unary | Binary- and String-like | Boolean (2) | :struct:`SetLookupOptions` |
++--------------------------+------------+----------------------------------+-----------------------+--------------------------------------------+
+| isin | Unary | Null | Boolean (2) | :struct:`SetLookupOptions` |
++--------------------------+------------+----------------------------------+-----------------------+--------------------------------------------+
+| isin | Unary | Boolean,Numeric, Temporal | Boolean (2) | :struct:`SetLookupOptions` |
++--------------------------+------------+----------------------------------+-----------------------+--------------------------------------------+
+| match | Unary | Binary- and String-like | Int32 (3) | :struct:`SetLookupOptions` |
++--------------------------+------------+----------------------------------+-----------------------+--------------------------------------------+
+| match | Unary | Null | Int32 (3) | :struct:`SetLookupOptions` |
++--------------------------+------------+----------------------------------+-----------------------+--------------------------------------------+
+| match | Unary | Boolean,Numeric, Temporal | Int32 (3) | :struct:`SetLookupOptions` |
++--------------------------+------------+----------------------------------+-----------------------+--------------------------------------------+
+
+* \(1) Output is true iff :member:`BinaryContainsExactOptions::pattern`
Review comment:
You get the detailed API docs by clicking on the `BinaryContainsExactOptions` hyperlink (in the rendered docs only, though :-)).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] nealrichardson commented on a change in pull request #7695: ARROW-8989: [C++][Doc] Document available compute functions
Posted by GitBox <gi...@apache.org>.
nealrichardson commented on a change in pull request #7695:
URL: https://github.com/apache/arrow/pull/7695#discussion_r452911334
##########
File path: docs/source/cpp/compute.rst
##########
@@ -0,0 +1,419 @@
+.. Licensed to the Apache Software Foundation (ASF) under one
+.. or more contributor license agreements. See the NOTICE file
+.. distributed with this work for additional information
+.. regarding copyright ownership. The ASF licenses this file
+.. to you under the Apache License, Version 2.0 (the
+.. "License"); you may not use this file except in compliance
+.. with the License. You may obtain a copy of the License at
+
+.. http://www.apache.org/licenses/LICENSE-2.0
+
+.. Unless required by applicable law or agreed to in writing,
+.. software distributed under the License is distributed on an
+.. "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+.. KIND, either express or implied. See the License for the
+.. specific language governing permissions and limitations
+.. under the License.
+
+.. default-domain:: cpp
+.. highlight:: cpp
+.. cpp:namespace:: arrow::compute
+
+=================
+Compute Functions
+=================
+
+.. TODO: describe API and how to invoke compute functions
+
+Available functions
+===================
+
+Aggregations
+------------
+
++--------------------------+------------+--------------------+-----------------------+--------------------------------------------+
+| Function name | Arity | Input types | Output type | Options class |
++==========================+============+====================+=======================+============================================+
+| count | Unary | Any | Scalar Int64 | :struct:`CountOptions` |
Review comment:
Will this automatically link to some generated docs that say what the possible options are? If not, can you add them here?
##########
File path: docs/source/cpp/compute.rst
##########
@@ -0,0 +1,419 @@
+.. Licensed to the Apache Software Foundation (ASF) under one
+.. or more contributor license agreements. See the NOTICE file
+.. distributed with this work for additional information
+.. regarding copyright ownership. The ASF licenses this file
+.. to you under the Apache License, Version 2.0 (the
+.. "License"); you may not use this file except in compliance
+.. with the License. You may obtain a copy of the License at
+
+.. http://www.apache.org/licenses/LICENSE-2.0
+
+.. Unless required by applicable law or agreed to in writing,
+.. software distributed under the License is distributed on an
+.. "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+.. KIND, either express or implied. See the License for the
+.. specific language governing permissions and limitations
+.. under the License.
+
+.. default-domain:: cpp
+.. highlight:: cpp
+.. cpp:namespace:: arrow::compute
+
+=================
+Compute Functions
+=================
+
+.. TODO: describe API and how to invoke compute functions
Review comment:
Yes please, at least an example
##########
File path: docs/source/cpp/compute.rst
##########
@@ -0,0 +1,419 @@
+.. Licensed to the Apache Software Foundation (ASF) under one
+.. or more contributor license agreements. See the NOTICE file
+.. distributed with this work for additional information
+.. regarding copyright ownership. The ASF licenses this file
+.. to you under the Apache License, Version 2.0 (the
+.. "License"); you may not use this file except in compliance
+.. with the License. You may obtain a copy of the License at
+
+.. http://www.apache.org/licenses/LICENSE-2.0
+
+.. Unless required by applicable law or agreed to in writing,
+.. software distributed under the License is distributed on an
+.. "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+.. KIND, either express or implied. See the License for the
+.. specific language governing permissions and limitations
+.. under the License.
+
+.. default-domain:: cpp
+.. highlight:: cpp
+.. cpp:namespace:: arrow::compute
+
+=================
+Compute Functions
+=================
+
+.. TODO: describe API and how to invoke compute functions
+
+Available functions
+===================
+
+Aggregations
+------------
+
++--------------------------+------------+--------------------+-----------------------+--------------------------------------------+
+| Function name | Arity | Input types | Output type | Options class |
++==========================+============+====================+=======================+============================================+
+| count | Unary | Any | Scalar Int64 | :struct:`CountOptions` |
++--------------------------+------------+--------------------+-----------------------+--------------------------------------------+
+| mean | Unary | Numeric | Scalar Float64 | |
++--------------------------+------------+--------------------+-----------------------+--------------------------------------------+
+| minmax | Unary | Numeric | Scalar Struct (1) | :struct:`MinMaxOptions` |
++--------------------------+------------+--------------------+-----------------------+--------------------------------------------+
+| sum | Unary | Numeric | Scalar Numeric (2) | |
++--------------------------+------------+--------------------+-----------------------+--------------------------------------------+
+
+Notes:
+
+* \(1) Output is a ``{"min": input type, "max": input type}`` Struct
+
+* \(2) Output is Int64, UInt64 or Float64, depending on the input type
+
+
+Element-wise ("scalar") functions
+---------------------------------
+
+Arithmetic functions
+~~~~~~~~~~~~~~~~~~~~
+
+Those functions expect two inputs of the same type and apply a given binary
+operation to each pair of elements gathered from the inputs. Each function
+is also available in an overflow-checking variant, suffixed ``_checked``.
+
+If any of the input elements in a pair is null, the corresponding output
+element is null.
+
++--------------------------+------------+--------------------+---------------------+
+| Function name | Arity | Input types | Output type |
++==========================+============+====================+=====================+
+| add | Binary | Numeric | Numeric |
++--------------------------+------------+--------------------+---------------------+
+| add_checked | Binary | Numeric | Numeric |
++--------------------------+------------+--------------------+---------------------+
+| multiply | Binary | Numeric | Numeric |
++--------------------------+------------+--------------------+---------------------+
+| multiply_checked | Binary | Numeric | Numeric |
++--------------------------+------------+--------------------+---------------------+
+| subtract | Binary | Numeric | Numeric |
++--------------------------+------------+--------------------+---------------------+
+| subtract_checked | Binary | Numeric | Numeric |
++--------------------------+------------+--------------------+---------------------+
+
+Comparisons
+~~~~~~~~~~~
+
+Those functions expect two inputs of the same type and apply a given
+comparison operator. If any of the input elements in a pair is null,
+the corresponding output element is null.
+
++--------------------------+------------+---------------------------------+---------------------+
+| Function names | Arity | Input types | Output type |
++==========================+============+=================================+=====================+
+| equal, not_equal | Binary | Numeric | Boolean |
++--------------------------+------------+---------------------------------+---------------------+
+| equal, not_equal | Binary | Binary- and String-like | Boolean |
++--------------------------+------------+---------------------------------+---------------------+
+| equal, not_equal | Binary | Temporal | Boolean |
++--------------------------+------------+---------------------------------+---------------------+
+| greater, greater_equal, | Binary | Numeric | Boolean |
+| less, less_equal | | | |
++--------------------------+------------+---------------------------------+---------------------+
+| greater, greater_equal, | Binary | Binary- and String-like | Boolean |
+| less, less_equal | | | |
++--------------------------+------------+---------------------------------+---------------------+
+| greater, greater_equal, | Binary | Temporal | Boolean |
Review comment:
Since this doc uses them throughout, it would be good to have an explicit list somewhere of what is included in "numeric", "binary- and string-like", "temporal", and any other type categories used.
##########
File path: docs/source/cpp/compute.rst
##########
@@ -0,0 +1,419 @@
+.. Licensed to the Apache Software Foundation (ASF) under one
+.. or more contributor license agreements. See the NOTICE file
+.. distributed with this work for additional information
+.. regarding copyright ownership. The ASF licenses this file
+.. to you under the Apache License, Version 2.0 (the
+.. "License"); you may not use this file except in compliance
+.. with the License. You may obtain a copy of the License at
+
+.. http://www.apache.org/licenses/LICENSE-2.0
+
+.. Unless required by applicable law or agreed to in writing,
+.. software distributed under the License is distributed on an
+.. "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+.. KIND, either express or implied. See the License for the
+.. specific language governing permissions and limitations
+.. under the License.
+
+.. default-domain:: cpp
+.. highlight:: cpp
+.. cpp:namespace:: arrow::compute
+
+=================
+Compute Functions
+=================
+
+.. TODO: describe API and how to invoke compute functions
+
+Available functions
+===================
+
+Aggregations
+------------
+
++--------------------------+------------+--------------------+-----------------------+--------------------------------------------+
+| Function name | Arity | Input types | Output type | Options class |
++==========================+============+====================+=======================+============================================+
+| count | Unary | Any | Scalar Int64 | :struct:`CountOptions` |
++--------------------------+------------+--------------------+-----------------------+--------------------------------------------+
+| mean | Unary | Numeric | Scalar Float64 | |
++--------------------------+------------+--------------------+-----------------------+--------------------------------------------+
+| minmax | Unary | Numeric | Scalar Struct (1) | :struct:`MinMaxOptions` |
++--------------------------+------------+--------------------+-----------------------+--------------------------------------------+
+| sum | Unary | Numeric | Scalar Numeric (2) | |
++--------------------------+------------+--------------------+-----------------------+--------------------------------------------+
+
+Notes:
+
+* \(1) Output is a ``{"min": input type, "max": input type}`` Struct
+
+* \(2) Output is Int64, UInt64 or Float64, depending on the input type
+
+
+Element-wise ("scalar") functions
+---------------------------------
+
+Arithmetic functions
+~~~~~~~~~~~~~~~~~~~~
+
+Those functions expect two inputs of the same type and apply a given binary
+operation to each pair of elements gathered from the inputs. Each function
+is also available in an overflow-checking variant, suffixed ``_checked``.
Review comment:
What happens if you don't use a checked version and it does overflow? Do we recommend one or the other for general use?
##########
File path: docs/source/cpp/compute.rst
##########
@@ -0,0 +1,419 @@
+.. Licensed to the Apache Software Foundation (ASF) under one
+.. or more contributor license agreements. See the NOTICE file
+.. distributed with this work for additional information
+.. regarding copyright ownership. The ASF licenses this file
+.. to you under the Apache License, Version 2.0 (the
+.. "License"); you may not use this file except in compliance
+.. with the License. You may obtain a copy of the License at
+
+.. http://www.apache.org/licenses/LICENSE-2.0
+
+.. Unless required by applicable law or agreed to in writing,
+.. software distributed under the License is distributed on an
+.. "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+.. KIND, either express or implied. See the License for the
+.. specific language governing permissions and limitations
+.. under the License.
+
+.. default-domain:: cpp
+.. highlight:: cpp
+.. cpp:namespace:: arrow::compute
+
+=================
+Compute Functions
+=================
+
+.. TODO: describe API and how to invoke compute functions
+
+Available functions
+===================
+
+Aggregations
+------------
+
++--------------------------+------------+--------------------+-----------------------+--------------------------------------------+
+| Function name | Arity | Input types | Output type | Options class |
++==========================+============+====================+=======================+============================================+
+| count | Unary | Any | Scalar Int64 | :struct:`CountOptions` |
++--------------------------+------------+--------------------+-----------------------+--------------------------------------------+
+| mean | Unary | Numeric | Scalar Float64 | |
++--------------------------+------------+--------------------+-----------------------+--------------------------------------------+
+| minmax | Unary | Numeric | Scalar Struct (1) | :struct:`MinMaxOptions` |
++--------------------------+------------+--------------------+-----------------------+--------------------------------------------+
+| sum | Unary | Numeric | Scalar Numeric (2) | |
++--------------------------+------------+--------------------+-----------------------+--------------------------------------------+
+
+Notes:
+
+* \(1) Output is a ``{"min": input type, "max": input type}`` Struct
+
+* \(2) Output is Int64, UInt64 or Float64, depending on the input type
+
+
+Element-wise ("scalar") functions
+---------------------------------
+
+Arithmetic functions
+~~~~~~~~~~~~~~~~~~~~
+
+Those functions expect two inputs of the same type and apply a given binary
+operation to each pair of elements gathered from the inputs. Each function
+is also available in an overflow-checking variant, suffixed ``_checked``.
+
+If any of the input elements in a pair is null, the corresponding output
+element is null.
+
++--------------------------+------------+--------------------+---------------------+
+| Function name | Arity | Input types | Output type |
++==========================+============+====================+=====================+
+| add | Binary | Numeric | Numeric |
++--------------------------+------------+--------------------+---------------------+
+| add_checked | Binary | Numeric | Numeric |
++--------------------------+------------+--------------------+---------------------+
+| multiply | Binary | Numeric | Numeric |
++--------------------------+------------+--------------------+---------------------+
+| multiply_checked | Binary | Numeric | Numeric |
++--------------------------+------------+--------------------+---------------------+
+| subtract | Binary | Numeric | Numeric |
++--------------------------+------------+--------------------+---------------------+
+| subtract_checked | Binary | Numeric | Numeric |
++--------------------------+------------+--------------------+---------------------+
+
+Comparisons
+~~~~~~~~~~~
+
+Those functions expect two inputs of the same type and apply a given
Review comment:
```suggestion
These functions expect two inputs of the same type and apply a given
```
##########
File path: docs/source/cpp/compute.rst
##########
@@ -0,0 +1,419 @@
+.. Licensed to the Apache Software Foundation (ASF) under one
+.. or more contributor license agreements. See the NOTICE file
+.. distributed with this work for additional information
+.. regarding copyright ownership. The ASF licenses this file
+.. to you under the Apache License, Version 2.0 (the
+.. "License"); you may not use this file except in compliance
+.. with the License. You may obtain a copy of the License at
+
+.. http://www.apache.org/licenses/LICENSE-2.0
+
+.. Unless required by applicable law or agreed to in writing,
+.. software distributed under the License is distributed on an
+.. "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+.. KIND, either express or implied. See the License for the
+.. specific language governing permissions and limitations
+.. under the License.
+
+.. default-domain:: cpp
+.. highlight:: cpp
+.. cpp:namespace:: arrow::compute
+
+=================
+Compute Functions
+=================
+
+.. TODO: describe API and how to invoke compute functions
+
+Available functions
+===================
+
+Aggregations
+------------
+
++--------------------------+------------+--------------------+-----------------------+--------------------------------------------+
+| Function name | Arity | Input types | Output type | Options class |
++==========================+============+====================+=======================+============================================+
+| count | Unary | Any | Scalar Int64 | :struct:`CountOptions` |
++--------------------------+------------+--------------------+-----------------------+--------------------------------------------+
+| mean | Unary | Numeric | Scalar Float64 | |
++--------------------------+------------+--------------------+-----------------------+--------------------------------------------+
+| minmax | Unary | Numeric | Scalar Struct (1) | :struct:`MinMaxOptions` |
++--------------------------+------------+--------------------+-----------------------+--------------------------------------------+
+| sum | Unary | Numeric | Scalar Numeric (2) | |
++--------------------------+------------+--------------------+-----------------------+--------------------------------------------+
+
+Notes:
+
+* \(1) Output is a ``{"min": input type, "max": input type}`` Struct
+
+* \(2) Output is Int64, UInt64 or Float64, depending on the input type
+
+
+Element-wise ("scalar") functions
+---------------------------------
+
+Arithmetic functions
+~~~~~~~~~~~~~~~~~~~~
+
+Those functions expect two inputs of the same type and apply a given binary
+operation to each pair of elements gathered from the inputs. Each function
+is also available in an overflow-checking variant, suffixed ``_checked``.
+
+If any of the input elements in a pair is null, the corresponding output
+element is null.
+
++--------------------------+------------+--------------------+---------------------+
+| Function name | Arity | Input types | Output type |
++==========================+============+====================+=====================+
+| add | Binary | Numeric | Numeric |
++--------------------------+------------+--------------------+---------------------+
+| add_checked | Binary | Numeric | Numeric |
++--------------------------+------------+--------------------+---------------------+
+| multiply | Binary | Numeric | Numeric |
++--------------------------+------------+--------------------+---------------------+
+| multiply_checked | Binary | Numeric | Numeric |
++--------------------------+------------+--------------------+---------------------+
+| subtract | Binary | Numeric | Numeric |
++--------------------------+------------+--------------------+---------------------+
+| subtract_checked | Binary | Numeric | Numeric |
++--------------------------+------------+--------------------+---------------------+
+
+Comparisons
+~~~~~~~~~~~
+
+Those functions expect two inputs of the same type and apply a given
+comparison operator. If any of the input elements in a pair is null,
+the corresponding output element is null.
+
++--------------------------+------------+---------------------------------+---------------------+
+| Function names | Arity | Input types | Output type |
++==========================+============+=================================+=====================+
+| equal, not_equal | Binary | Numeric | Boolean |
++--------------------------+------------+---------------------------------+---------------------+
+| equal, not_equal | Binary | Binary- and String-like | Boolean |
++--------------------------+------------+---------------------------------+---------------------+
+| equal, not_equal | Binary | Temporal | Boolean |
++--------------------------+------------+---------------------------------+---------------------+
+| greater, greater_equal, | Binary | Numeric | Boolean |
+| less, less_equal | | | |
++--------------------------+------------+---------------------------------+---------------------+
+| greater, greater_equal, | Binary | Binary- and String-like | Boolean |
+| less, less_equal | | | |
++--------------------------+------------+---------------------------------+---------------------+
+| greater, greater_equal, | Binary | Temporal | Boolean |
+| less, less_equal | | | |
++--------------------------+------------+---------------------------------+---------------------+
+
+Logical functions
+~~~~~~~~~~~~~~~~~~
+
+The normal behaviour for these functions is to emit a null if any of the
+inputs is null.
+
+Some of them are also available in a "`Kleene logic`_" variant (suffixed
+``_kleene``) where null is taken to mean "undefined". For those variants
+therefore:
+
+* "true AND null", "null AND true" give "null" (the result is undefined)
+* "true OR null", "null OR true" give "true"
+* "false AND null", "null AND false" give "false"
+* "false OR null", "null OR false" give "null" (the result is undefined)
+
++--------------------------+------------+--------------------+---------------------+
+| Function name | Arity | Input types | Output type |
++==========================+============+====================+=====================+
+| and | Binary | Boolean | Boolean |
++--------------------------+------------+--------------------+---------------------+
+| and_kleene | Binary | Boolean | Boolean |
++--------------------------+------------+--------------------+---------------------+
+| invert | Unary | Boolean | Boolean |
++--------------------------+------------+--------------------+---------------------+
+| or | Binary | Boolean | Boolean |
++--------------------------+------------+--------------------+---------------------+
+| or_kleene | Binary | Boolean | Boolean |
++--------------------------+------------+--------------------+---------------------+
+| xor | Binary | Boolean | Boolean |
++--------------------------+------------+--------------------+---------------------+
+
+.. _Kleene logic: https://en.wikipedia.org/wiki/Three-valued_logic#Kleene_and_Priest_logics
+
+String functions
+~~~~~~~~~~~~~~~~
+
++--------------------------+------------+--------------------+---------------------+---------+
+| Function name | Arity | Input types | Output type | Notes |
++==========================+============+====================+=====================+=========+
+| ascii_length | Unary | String-like | Int32 or Int64 | \(1) |
++--------------------------+------------+--------------------+---------------------+---------+
+| ascii_lower | Unary | String-like | String-like | \(2) |
++--------------------------+------------+--------------------+---------------------+---------+
+| ascii_upper | Unary | String-like | String-like | \(2) |
++--------------------------+------------+--------------------+---------------------+---------+
+| utf8_lower | Unary | String-like | String-like | \(3) |
++--------------------------+------------+--------------------+---------------------+---------+
+| utf8_upper | Unary | String-like | String-like | \(3) |
++--------------------------+------------+--------------------+---------------------+---------+
+
+* \(1) Output is the physical length in bytes of each input element.
+
+* \(2) Each ASCII character in the input is converted to lowercase or
+ uppercase. Non-ASCII characters are left untouched.
+
+* \(3) Each UTF8-encoded character in the input is converted to lowercase or
+ uppercase.
+
+Containment tests
+~~~~~~~~~~~~~~~~~
+
++--------------------------+------------+----------------------------------+-----------------------+--------------------------------------------+
+| Function name | Arity | Input types | Output type | Options class |
++==========================+============+==================================+=======================+============================================+
+| binary_contains_exact | Unary | String-like | Boolean (1) | :struct:`BinaryContainsExactOptions` |
++--------------------------+------------+----------------------------------+-----------------------+--------------------------------------------+
+| isin | Unary | Binary- and String-like | Boolean (2) | :struct:`SetLookupOptions` |
++--------------------------+------------+----------------------------------+-----------------------+--------------------------------------------+
+| isin | Unary | Null | Boolean (2) | :struct:`SetLookupOptions` |
++--------------------------+------------+----------------------------------+-----------------------+--------------------------------------------+
+| isin | Unary | Boolean,Numeric, Temporal | Boolean (2) | :struct:`SetLookupOptions` |
++--------------------------+------------+----------------------------------+-----------------------+--------------------------------------------+
+| match | Unary | Binary- and String-like | Int32 (3) | :struct:`SetLookupOptions` |
++--------------------------+------------+----------------------------------+-----------------------+--------------------------------------------+
+| match | Unary | Null | Int32 (3) | :struct:`SetLookupOptions` |
++--------------------------+------------+----------------------------------+-----------------------+--------------------------------------------+
+| match | Unary | Boolean,Numeric, Temporal | Int32 (3) | :struct:`SetLookupOptions` |
++--------------------------+------------+----------------------------------+-----------------------+--------------------------------------------+
+
+* \(1) Output is true iff :member:`BinaryContainsExactOptions::pattern`
Review comment:
Could you clarify what kinds of objects are used as `pattern` and `value_set` here? Array, Scalar, std::something, etc.?
Also might be worth calling out that these are unary and not binary, as they often appear to be elsewhere. It certainly confused me when I tried to call `isin` like `isin(x, y)` and got the error message that "isin" only takes 1 argument--which doesn't help you understand how to properly construct the call, it just makes you question your sanity.
##########
File path: docs/source/cpp/compute.rst
##########
@@ -0,0 +1,419 @@
+.. Licensed to the Apache Software Foundation (ASF) under one
+.. or more contributor license agreements. See the NOTICE file
+.. distributed with this work for additional information
+.. regarding copyright ownership. The ASF licenses this file
+.. to you under the Apache License, Version 2.0 (the
+.. "License"); you may not use this file except in compliance
+.. with the License. You may obtain a copy of the License at
+
+.. http://www.apache.org/licenses/LICENSE-2.0
+
+.. Unless required by applicable law or agreed to in writing,
+.. software distributed under the License is distributed on an
+.. "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+.. KIND, either express or implied. See the License for the
+.. specific language governing permissions and limitations
+.. under the License.
+
+.. default-domain:: cpp
+.. highlight:: cpp
+.. cpp:namespace:: arrow::compute
+
+=================
+Compute Functions
+=================
+
+.. TODO: describe API and how to invoke compute functions
+
+Available functions
+===================
+
+Aggregations
+------------
+
++--------------------------+------------+--------------------+-----------------------+--------------------------------------------+
+| Function name | Arity | Input types | Output type | Options class |
++==========================+============+====================+=======================+============================================+
+| count | Unary | Any | Scalar Int64 | :struct:`CountOptions` |
++--------------------------+------------+--------------------+-----------------------+--------------------------------------------+
+| mean | Unary | Numeric | Scalar Float64 | |
++--------------------------+------------+--------------------+-----------------------+--------------------------------------------+
+| minmax | Unary | Numeric | Scalar Struct (1) | :struct:`MinMaxOptions` |
++--------------------------+------------+--------------------+-----------------------+--------------------------------------------+
+| sum | Unary | Numeric | Scalar Numeric (2) | |
++--------------------------+------------+--------------------+-----------------------+--------------------------------------------+
+
+Notes:
+
+* \(1) Output is a ``{"min": input type, "max": input type}`` Struct
+
+* \(2) Output is Int64, UInt64 or Float64, depending on the input type
+
+
+Element-wise ("scalar") functions
+---------------------------------
+
+Arithmetic functions
+~~~~~~~~~~~~~~~~~~~~
+
+Those functions expect two inputs of the same type and apply a given binary
+operation to each pair of elements gathered from the inputs. Each function
+is also available in an overflow-checking variant, suffixed ``_checked``.
+
+If any of the input elements in a pair is null, the corresponding output
+element is null.
+
++--------------------------+------------+--------------------+---------------------+
+| Function name | Arity | Input types | Output type |
++==========================+============+====================+=====================+
+| add | Binary | Numeric | Numeric |
++--------------------------+------------+--------------------+---------------------+
+| add_checked | Binary | Numeric | Numeric |
++--------------------------+------------+--------------------+---------------------+
+| multiply | Binary | Numeric | Numeric |
++--------------------------+------------+--------------------+---------------------+
+| multiply_checked | Binary | Numeric | Numeric |
++--------------------------+------------+--------------------+---------------------+
+| subtract | Binary | Numeric | Numeric |
++--------------------------+------------+--------------------+---------------------+
+| subtract_checked | Binary | Numeric | Numeric |
++--------------------------+------------+--------------------+---------------------+
+
+Comparisons
+~~~~~~~~~~~
+
+Those functions expect two inputs of the same type and apply a given
+comparison operator. If any of the input elements in a pair is null,
+the corresponding output element is null.
+
++--------------------------+------------+---------------------------------+---------------------+
+| Function names | Arity | Input types | Output type |
++==========================+============+=================================+=====================+
+| equal, not_equal | Binary | Numeric | Boolean |
++--------------------------+------------+---------------------------------+---------------------+
+| equal, not_equal | Binary | Binary- and String-like | Boolean |
++--------------------------+------------+---------------------------------+---------------------+
+| equal, not_equal | Binary | Temporal | Boolean |
++--------------------------+------------+---------------------------------+---------------------+
+| greater, greater_equal, | Binary | Numeric | Boolean |
+| less, less_equal | | | |
++--------------------------+------------+---------------------------------+---------------------+
+| greater, greater_equal, | Binary | Binary- and String-like | Boolean |
+| less, less_equal | | | |
++--------------------------+------------+---------------------------------+---------------------+
+| greater, greater_equal, | Binary | Temporal | Boolean |
+| less, less_equal | | | |
++--------------------------+------------+---------------------------------+---------------------+
+
+Logical functions
+~~~~~~~~~~~~~~~~~~
+
+The normal behaviour for these functions is to emit a null if any of the
+inputs is null.
+
+Some of them are also available in a "`Kleene logic`_" variant (suffixed
+``_kleene``) where null is taken to mean "undefined". For those variants
+therefore:
+
+* "true AND null", "null AND true" give "null" (the result is undefined)
+* "true OR null", "null OR true" give "true"
+* "false AND null", "null AND false" give "false"
+* "false OR null", "null OR false" give "null" (the result is undefined)
+
++--------------------------+------------+--------------------+---------------------+
+| Function name | Arity | Input types | Output type |
++==========================+============+====================+=====================+
+| and | Binary | Boolean | Boolean |
++--------------------------+------------+--------------------+---------------------+
+| and_kleene | Binary | Boolean | Boolean |
++--------------------------+------------+--------------------+---------------------+
+| invert | Unary | Boolean | Boolean |
++--------------------------+------------+--------------------+---------------------+
+| or | Binary | Boolean | Boolean |
++--------------------------+------------+--------------------+---------------------+
+| or_kleene | Binary | Boolean | Boolean |
++--------------------------+------------+--------------------+---------------------+
+| xor | Binary | Boolean | Boolean |
++--------------------------+------------+--------------------+---------------------+
+
+.. _Kleene logic: https://en.wikipedia.org/wiki/Three-valued_logic#Kleene_and_Priest_logics
+
+String functions
+~~~~~~~~~~~~~~~~
+
++--------------------------+------------+--------------------+---------------------+---------+
+| Function name | Arity | Input types | Output type | Notes |
++==========================+============+====================+=====================+=========+
+| ascii_length | Unary | String-like | Int32 or Int64 | \(1) |
++--------------------------+------------+--------------------+---------------------+---------+
+| ascii_lower | Unary | String-like | String-like | \(2) |
++--------------------------+------------+--------------------+---------------------+---------+
+| ascii_upper | Unary | String-like | String-like | \(2) |
++--------------------------+------------+--------------------+---------------------+---------+
+| utf8_lower | Unary | String-like | String-like | \(3) |
++--------------------------+------------+--------------------+---------------------+---------+
+| utf8_upper | Unary | String-like | String-like | \(3) |
++--------------------------+------------+--------------------+---------------------+---------+
+
+* \(1) Output is the physical length in bytes of each input element.
+
+* \(2) Each ASCII character in the input is converted to lowercase or
+ uppercase. Non-ASCII characters are left untouched.
+
+* \(3) Each UTF8-encoded character in the input is converted to lowercase or
+ uppercase.
+
+Containment tests
+~~~~~~~~~~~~~~~~~
+
++--------------------------+------------+----------------------------------+-----------------------+--------------------------------------------+
+| Function name | Arity | Input types | Output type | Options class |
++==========================+============+==================================+=======================+============================================+
+| binary_contains_exact | Unary | String-like | Boolean (1) | :struct:`BinaryContainsExactOptions` |
++--------------------------+------------+----------------------------------+-----------------------+--------------------------------------------+
+| isin | Unary | Binary- and String-like | Boolean (2) | :struct:`SetLookupOptions` |
++--------------------------+------------+----------------------------------+-----------------------+--------------------------------------------+
+| isin | Unary | Null | Boolean (2) | :struct:`SetLookupOptions` |
++--------------------------+------------+----------------------------------+-----------------------+--------------------------------------------+
+| isin | Unary | Boolean,Numeric, Temporal | Boolean (2) | :struct:`SetLookupOptions` |
++--------------------------+------------+----------------------------------+-----------------------+--------------------------------------------+
+| match | Unary | Binary- and String-like | Int32 (3) | :struct:`SetLookupOptions` |
++--------------------------+------------+----------------------------------+-----------------------+--------------------------------------------+
+| match | Unary | Null | Int32 (3) | :struct:`SetLookupOptions` |
++--------------------------+------------+----------------------------------+-----------------------+--------------------------------------------+
+| match | Unary | Boolean,Numeric, Temporal | Int32 (3) | :struct:`SetLookupOptions` |
++--------------------------+------------+----------------------------------+-----------------------+--------------------------------------------+
+
+* \(1) Output is true iff :member:`BinaryContainsExactOptions::pattern`
+ is a substring of the corresponding input element.
+
+* \(2) Output is true iff the corresponding input element is equal to one
+ of the elements in :member:`SetLookupOptions::value_set`.
+
+* \(3) Output is the index of the corresponding input element in
+ :member:`SetLookupOptions::value_set`, if found there. Otherwise,
+ output is null.
+
+Structural transforms
+~~~~~~~~~~~~~~~~~~~~~
+
++--------------------------+------------+--------------------+---------------------+---------+
+| Function name | Arity | Input types | Output type | Notes |
++==========================+============+====================+=====================+=========+
+| is_null | Unary | Any | Boolean | \(1) |
++--------------------------+------------+--------------------+---------------------+---------+
+| is_valid | Unary | Any | Boolean | \(2) |
++--------------------------+------------+--------------------+---------------------+---------+
+| list_value_lengths | Unary | List-like | Int32 or Int64 | \(3) |
++--------------------------+------------+--------------------+---------------------+---------+
+
+* \(1) Output is true iff the corresponding input element is non-null.
+
+* \(2) Output is true iff the corresponding input element is null.
+
+* \(3) Each output element is the length of the corresponding input element
+ (null if input is null).
+
+Conversions
+~~~~~~~~~~~
+
+A general conversion function named ``cast`` is provided which accepts a large
+number of input and output types. The type to cast to can be passed in a
+:struct:`CastOptions` instance.
Review comment:
How?
##########
File path: docs/source/cpp/compute.rst
##########
@@ -0,0 +1,419 @@
+.. Licensed to the Apache Software Foundation (ASF) under one
+.. or more contributor license agreements. See the NOTICE file
+.. distributed with this work for additional information
+.. regarding copyright ownership. The ASF licenses this file
+.. to you under the Apache License, Version 2.0 (the
+.. "License"); you may not use this file except in compliance
+.. with the License. You may obtain a copy of the License at
+
+.. http://www.apache.org/licenses/LICENSE-2.0
+
+.. Unless required by applicable law or agreed to in writing,
+.. software distributed under the License is distributed on an
+.. "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+.. KIND, either express or implied. See the License for the
+.. specific language governing permissions and limitations
+.. under the License.
+
+.. default-domain:: cpp
+.. highlight:: cpp
+.. cpp:namespace:: arrow::compute
+
+=================
+Compute Functions
+=================
+
+.. TODO: describe API and how to invoke compute functions
+
+Available functions
+===================
+
+Aggregations
+------------
+
++--------------------------+------------+--------------------+-----------------------+--------------------------------------------+
+| Function name | Arity | Input types | Output type | Options class |
++==========================+============+====================+=======================+============================================+
+| count | Unary | Any | Scalar Int64 | :struct:`CountOptions` |
++--------------------------+------------+--------------------+-----------------------+--------------------------------------------+
+| mean | Unary | Numeric | Scalar Float64 | |
++--------------------------+------------+--------------------+-----------------------+--------------------------------------------+
+| minmax | Unary | Numeric | Scalar Struct (1) | :struct:`MinMaxOptions` |
++--------------------------+------------+--------------------+-----------------------+--------------------------------------------+
+| sum | Unary | Numeric | Scalar Numeric (2) | |
++--------------------------+------------+--------------------+-----------------------+--------------------------------------------+
+
+Notes:
+
+* \(1) Output is a ``{"min": input type, "max": input type}`` Struct
+
+* \(2) Output is Int64, UInt64 or Float64, depending on the input type
+
+
+Element-wise ("scalar") functions
+---------------------------------
+
+Arithmetic functions
+~~~~~~~~~~~~~~~~~~~~
+
+Those functions expect two inputs of the same type and apply a given binary
+operation to each pair of elements gathered from the inputs. Each function
+is also available in an overflow-checking variant, suffixed ``_checked``.
+
+If any of the input elements in a pair is null, the corresponding output
+element is null.
+
++--------------------------+------------+--------------------+---------------------+
+| Function name | Arity | Input types | Output type |
++==========================+============+====================+=====================+
+| add | Binary | Numeric | Numeric |
++--------------------------+------------+--------------------+---------------------+
+| add_checked | Binary | Numeric | Numeric |
++--------------------------+------------+--------------------+---------------------+
+| multiply | Binary | Numeric | Numeric |
++--------------------------+------------+--------------------+---------------------+
+| multiply_checked | Binary | Numeric | Numeric |
++--------------------------+------------+--------------------+---------------------+
+| subtract | Binary | Numeric | Numeric |
++--------------------------+------------+--------------------+---------------------+
+| subtract_checked | Binary | Numeric | Numeric |
++--------------------------+------------+--------------------+---------------------+
+
+Comparisons
+~~~~~~~~~~~
+
+Those functions expect two inputs of the same type and apply a given
+comparison operator. If any of the input elements in a pair is null,
+the corresponding output element is null.
+
++--------------------------+------------+---------------------------------+---------------------+
+| Function names | Arity | Input types | Output type |
++==========================+============+=================================+=====================+
+| equal, not_equal | Binary | Numeric | Boolean |
++--------------------------+------------+---------------------------------+---------------------+
+| equal, not_equal | Binary | Binary- and String-like | Boolean |
++--------------------------+------------+---------------------------------+---------------------+
+| equal, not_equal | Binary | Temporal | Boolean |
++--------------------------+------------+---------------------------------+---------------------+
+| greater, greater_equal, | Binary | Numeric | Boolean |
+| less, less_equal | | | |
++--------------------------+------------+---------------------------------+---------------------+
+| greater, greater_equal, | Binary | Binary- and String-like | Boolean |
+| less, less_equal | | | |
++--------------------------+------------+---------------------------------+---------------------+
+| greater, greater_equal, | Binary | Temporal | Boolean |
+| less, less_equal | | | |
++--------------------------+------------+---------------------------------+---------------------+
+
+Logical functions
+~~~~~~~~~~~~~~~~~~
+
+The normal behaviour for these functions is to emit a null if any of the
+inputs is null.
+
+Some of them are also available in a "`Kleene logic`_" variant (suffixed
+``_kleene``) where null is taken to mean "undefined". For those variants
+therefore:
+
+* "true AND null", "null AND true" give "null" (the result is undefined)
+* "true OR null", "null OR true" give "true"
+* "false AND null", "null AND false" give "false"
+* "false OR null", "null OR false" give "null" (the result is undefined)
+
++--------------------------+------------+--------------------+---------------------+
+| Function name | Arity | Input types | Output type |
++==========================+============+====================+=====================+
+| and | Binary | Boolean | Boolean |
++--------------------------+------------+--------------------+---------------------+
+| and_kleene | Binary | Boolean | Boolean |
++--------------------------+------------+--------------------+---------------------+
+| invert | Unary | Boolean | Boolean |
++--------------------------+------------+--------------------+---------------------+
+| or | Binary | Boolean | Boolean |
++--------------------------+------------+--------------------+---------------------+
+| or_kleene | Binary | Boolean | Boolean |
++--------------------------+------------+--------------------+---------------------+
+| xor | Binary | Boolean | Boolean |
++--------------------------+------------+--------------------+---------------------+
+
+.. _Kleene logic: https://en.wikipedia.org/wiki/Three-valued_logic#Kleene_and_Priest_logics
+
+String functions
+~~~~~~~~~~~~~~~~
+
++--------------------------+------------+--------------------+---------------------+---------+
+| Function name | Arity | Input types | Output type | Notes |
++==========================+============+====================+=====================+=========+
+| ascii_length | Unary | String-like | Int32 or Int64 | \(1) |
++--------------------------+------------+--------------------+---------------------+---------+
+| ascii_lower | Unary | String-like | String-like | \(2) |
++--------------------------+------------+--------------------+---------------------+---------+
+| ascii_upper | Unary | String-like | String-like | \(2) |
++--------------------------+------------+--------------------+---------------------+---------+
+| utf8_lower | Unary | String-like | String-like | \(3) |
++--------------------------+------------+--------------------+---------------------+---------+
+| utf8_upper | Unary | String-like | String-like | \(3) |
++--------------------------+------------+--------------------+---------------------+---------+
+
+* \(1) Output is the physical length in bytes of each input element.
+
+* \(2) Each ASCII character in the input is converted to lowercase or
+ uppercase. Non-ASCII characters are left untouched.
+
+* \(3) Each UTF8-encoded character in the input is converted to lowercase or
+ uppercase.
+
+Containment tests
+~~~~~~~~~~~~~~~~~
+
++--------------------------+------------+----------------------------------+-----------------------+--------------------------------------------+
+| Function name | Arity | Input types | Output type | Options class |
++==========================+============+==================================+=======================+============================================+
+| binary_contains_exact | Unary | String-like | Boolean (1) | :struct:`BinaryContainsExactOptions` |
++--------------------------+------------+----------------------------------+-----------------------+--------------------------------------------+
+| isin | Unary | Binary- and String-like | Boolean (2) | :struct:`SetLookupOptions` |
++--------------------------+------------+----------------------------------+-----------------------+--------------------------------------------+
+| isin | Unary | Null | Boolean (2) | :struct:`SetLookupOptions` |
++--------------------------+------------+----------------------------------+-----------------------+--------------------------------------------+
+| isin | Unary | Boolean,Numeric, Temporal | Boolean (2) | :struct:`SetLookupOptions` |
++--------------------------+------------+----------------------------------+-----------------------+--------------------------------------------+
+| match | Unary | Binary- and String-like | Int32 (3) | :struct:`SetLookupOptions` |
++--------------------------+------------+----------------------------------+-----------------------+--------------------------------------------+
+| match | Unary | Null | Int32 (3) | :struct:`SetLookupOptions` |
++--------------------------+------------+----------------------------------+-----------------------+--------------------------------------------+
+| match | Unary | Boolean,Numeric, Temporal | Int32 (3) | :struct:`SetLookupOptions` |
++--------------------------+------------+----------------------------------+-----------------------+--------------------------------------------+
+
+* \(1) Output is true iff :member:`BinaryContainsExactOptions::pattern`
+ is a substring of the corresponding input element.
+
+* \(2) Output is true iff the corresponding input element is equal to one
+ of the elements in :member:`SetLookupOptions::value_set`.
+
+* \(3) Output is the index of the corresponding input element in
+ :member:`SetLookupOptions::value_set`, if found there. Otherwise,
+ output is null.
+
+Structural transforms
+~~~~~~~~~~~~~~~~~~~~~
+
++--------------------------+------------+--------------------+---------------------+---------+
+| Function name | Arity | Input types | Output type | Notes |
++==========================+============+====================+=====================+=========+
+| is_null | Unary | Any | Boolean | \(1) |
++--------------------------+------------+--------------------+---------------------+---------+
+| is_valid | Unary | Any | Boolean | \(2) |
++--------------------------+------------+--------------------+---------------------+---------+
+| list_value_lengths | Unary | List-like | Int32 or Int64 | \(3) |
++--------------------------+------------+--------------------+---------------------+---------+
+
+* \(1) Output is true iff the corresponding input element is non-null.
+
+* \(2) Output is true iff the corresponding input element is null.
+
+* \(3) Each output element is the length of the corresponding input element
+ (null if input is null).
+
+Conversions
+~~~~~~~~~~~
+
+A general conversion function named ``cast`` is provided which accepts a large
+number of input and output types. The type to cast to can be passed in a
+:struct:`CastOptions` instance.
+
++--------------------------+------------+--------------------+-----------------------+--------------------------------------------+
+| Function name | Arity | Input types | Output type | Options class |
++==========================+============+====================+=======================+============================================+
+| cast | Unary | Many | Variable | :struct:`CastOptions` |
++--------------------------+------------+--------------------+-----------------------+--------------------------------------------+
+| strptime | Unary | String-like | Timestamp | :struct:`StrptimeOptions` |
++--------------------------+------------+--------------------+-----------------------+--------------------------------------------+
+
+The conversions available with ``cast`` are listed below. In all cases, a
+null input value is converted into a null output value.
+
+**Truth value extraction**
+
++-----------------------------+------------------------------------+--------------+
+| Input type | Output type | Notes |
++=============================+====================================+==============+
+| Binary- and String-like | Boolean | \(1) |
++-----------------------------+------------------------------------+--------------+
+| Numeric | Boolean | \(2) |
++-----------------------------+------------------------------------+--------------+
+
+* \(1) Output is true iff the corresponding input value has non-zero length.
+
+* \(2) Output is true iff the corresponding input value is non-zero.
+
+**Same-kind conversion**
+
++-----------------------------+------------------------------------+--------------+
+| Input type | Output type | Notes |
++=============================+====================================+==============+
+| Int32 | 32-bit Temporal | \(1) |
++-----------------------------+------------------------------------+--------------+
+| Int64 | 64-bit Temporal | \(1) |
++-----------------------------+------------------------------------+--------------+
+| (Large)Binary | (Large)String | \(2) |
++-----------------------------+------------------------------------+--------------+
+| (Large)String | (Large)Binary | \(3) |
++-----------------------------+------------------------------------+--------------+
+| Numeric | Numeric | \(4) \(5) |
++-----------------------------+------------------------------------+--------------+
+| 32-bit Temporal | Int32 | \(1) |
++-----------------------------+------------------------------------+--------------+
+| 64-bit Temporal | Int64 | \(1) |
++-----------------------------+------------------------------------+--------------+
+| Temporal | Temporal | \(4) \(5) |
++-----------------------------+------------------------------------+--------------+
+
+* \(1) No-operation cast: the raw values are kept identical, only
+ the type is changed.
+
+* \(2) Validates the contents if :member:`CastOptions::allow_invalid_utf8`
+ is false.
+
+* \(3) No-operation cast: only the type is changed.
+
+* \(4) Overflow and truncation checks are enabled depending on
+ the given :struct:`CastOptions`.
+
+* \(5) Not all such casts have been implemented.
+
+**String representations**
+
++-----------------------------+------------------------------------+---------+
+| Input type | Output type | Notes |
++=============================+====================================+=========+
+| Boolean | String-like | |
++-----------------------------+------------------------------------+---------+
+| Numeric | String-like | |
++-----------------------------+------------------------------------+---------+
+
+**Generic conversions**
+
++-----------------------------+------------------------------------+---------+
+| Input type | Output type | Notes |
++=============================+====================================+=========+
+| Dictionary | Dictionary value type | |
++-----------------------------+------------------------------------+---------+
+| Extension | Extension storage type | |
++-----------------------------+------------------------------------+---------+
+| List-like | List-like | \(1) |
++-----------------------------+------------------------------------+---------+
+| Null | Any | |
++-----------------------------+------------------------------------+---------+
+
+* \(1) The list offsets are unchanged, the list values are cast from the
+ input value type to the output value type (if a conversion is
+ available).
+
+
+.. TODO: add C++ cast example
Review comment:
OMG yes please, probably more than one
##########
File path: docs/source/cpp/compute.rst
##########
@@ -0,0 +1,419 @@
+.. Licensed to the Apache Software Foundation (ASF) under one
+.. or more contributor license agreements. See the NOTICE file
+.. distributed with this work for additional information
+.. regarding copyright ownership. The ASF licenses this file
+.. to you under the Apache License, Version 2.0 (the
+.. "License"); you may not use this file except in compliance
+.. with the License. You may obtain a copy of the License at
+
+.. http://www.apache.org/licenses/LICENSE-2.0
+
+.. Unless required by applicable law or agreed to in writing,
+.. software distributed under the License is distributed on an
+.. "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+.. KIND, either express or implied. See the License for the
+.. specific language governing permissions and limitations
+.. under the License.
+
+.. default-domain:: cpp
+.. highlight:: cpp
+.. cpp:namespace:: arrow::compute
+
+=================
+Compute Functions
+=================
+
+.. TODO: describe API and how to invoke compute functions
+
+Available functions
+===================
+
+Aggregations
+------------
+
++--------------------------+------------+--------------------+-----------------------+--------------------------------------------+
+| Function name | Arity | Input types | Output type | Options class |
++==========================+============+====================+=======================+============================================+
+| count | Unary | Any | Scalar Int64 | :struct:`CountOptions` |
++--------------------------+------------+--------------------+-----------------------+--------------------------------------------+
+| mean | Unary | Numeric | Scalar Float64 | |
++--------------------------+------------+--------------------+-----------------------+--------------------------------------------+
+| minmax | Unary | Numeric | Scalar Struct (1) | :struct:`MinMaxOptions` |
++--------------------------+------------+--------------------+-----------------------+--------------------------------------------+
+| sum | Unary | Numeric | Scalar Numeric (2) | |
++--------------------------+------------+--------------------+-----------------------+--------------------------------------------+
+
+Notes:
+
+* \(1) Output is a ``{"min": input type, "max": input type}`` Struct
+
+* \(2) Output is Int64, UInt64 or Float64, depending on the input type
+
+
+Element-wise ("scalar") functions
+---------------------------------
+
+Arithmetic functions
+~~~~~~~~~~~~~~~~~~~~
+
+Those functions expect two inputs of the same type and apply a given binary
+operation to each pair of elements gathered from the inputs. Each function
+is also available in an overflow-checking variant, suffixed ``_checked``.
+
+If any of the input elements in a pair is null, the corresponding output
+element is null.
+
++--------------------------+------------+--------------------+---------------------+
+| Function name | Arity | Input types | Output type |
++==========================+============+====================+=====================+
+| add | Binary | Numeric | Numeric |
++--------------------------+------------+--------------------+---------------------+
+| add_checked | Binary | Numeric | Numeric |
++--------------------------+------------+--------------------+---------------------+
+| multiply | Binary | Numeric | Numeric |
++--------------------------+------------+--------------------+---------------------+
+| multiply_checked | Binary | Numeric | Numeric |
++--------------------------+------------+--------------------+---------------------+
+| subtract | Binary | Numeric | Numeric |
++--------------------------+------------+--------------------+---------------------+
+| subtract_checked | Binary | Numeric | Numeric |
++--------------------------+------------+--------------------+---------------------+
+
+Comparisons
+~~~~~~~~~~~
+
+Those functions expect two inputs of the same type and apply a given
+comparison operator. If any of the input elements in a pair is null,
+the corresponding output element is null.
+
++--------------------------+------------+---------------------------------+---------------------+
+| Function names | Arity | Input types | Output type |
++==========================+============+=================================+=====================+
+| equal, not_equal | Binary | Numeric | Boolean |
++--------------------------+------------+---------------------------------+---------------------+
+| equal, not_equal | Binary | Binary- and String-like | Boolean |
++--------------------------+------------+---------------------------------+---------------------+
+| equal, not_equal | Binary | Temporal | Boolean |
++--------------------------+------------+---------------------------------+---------------------+
+| greater, greater_equal, | Binary | Numeric | Boolean |
+| less, less_equal | | | |
++--------------------------+------------+---------------------------------+---------------------+
+| greater, greater_equal, | Binary | Binary- and String-like | Boolean |
+| less, less_equal | | | |
++--------------------------+------------+---------------------------------+---------------------+
+| greater, greater_equal, | Binary | Temporal | Boolean |
+| less, less_equal | | | |
++--------------------------+------------+---------------------------------+---------------------+
+
+Logical functions
+~~~~~~~~~~~~~~~~~~
+
+The normal behaviour for these functions is to emit a null if any of the
+inputs is null.
+
+Some of them are also available in a "`Kleene logic`_" variant (suffixed
+``_kleene``) where null is taken to mean "undefined". For those variants
+therefore:
+
+* "true AND null", "null AND true" give "null" (the result is undefined)
+* "true OR null", "null OR true" give "true"
+* "false AND null", "null AND false" give "false"
+* "false OR null", "null OR false" give "null" (the result is undefined)
+
++--------------------------+------------+--------------------+---------------------+
+| Function name | Arity | Input types | Output type |
++==========================+============+====================+=====================+
+| and | Binary | Boolean | Boolean |
++--------------------------+------------+--------------------+---------------------+
+| and_kleene | Binary | Boolean | Boolean |
++--------------------------+------------+--------------------+---------------------+
+| invert | Unary | Boolean | Boolean |
++--------------------------+------------+--------------------+---------------------+
+| or | Binary | Boolean | Boolean |
++--------------------------+------------+--------------------+---------------------+
+| or_kleene | Binary | Boolean | Boolean |
++--------------------------+------------+--------------------+---------------------+
+| xor | Binary | Boolean | Boolean |
++--------------------------+------------+--------------------+---------------------+
+
+.. _Kleene logic: https://en.wikipedia.org/wiki/Three-valued_logic#Kleene_and_Priest_logics
+
+String functions
+~~~~~~~~~~~~~~~~
+
++--------------------------+------------+--------------------+---------------------+---------+
+| Function name | Arity | Input types | Output type | Notes |
++==========================+============+====================+=====================+=========+
+| ascii_length | Unary | String-like | Int32 or Int64 | \(1) |
++--------------------------+------------+--------------------+---------------------+---------+
+| ascii_lower | Unary | String-like | String-like | \(2) |
++--------------------------+------------+--------------------+---------------------+---------+
+| ascii_upper | Unary | String-like | String-like | \(2) |
++--------------------------+------------+--------------------+---------------------+---------+
+| utf8_lower | Unary | String-like | String-like | \(3) |
++--------------------------+------------+--------------------+---------------------+---------+
+| utf8_upper | Unary | String-like | String-like | \(3) |
++--------------------------+------------+--------------------+---------------------+---------+
+
+* \(1) Output is the physical length in bytes of each input element.
+
+* \(2) Each ASCII character in the input is converted to lowercase or
+ uppercase. Non-ASCII characters are left untouched.
+
+* \(3) Each UTF8-encoded character in the input is converted to lowercase or
+ uppercase.
+
+Containment tests
+~~~~~~~~~~~~~~~~~
+
++--------------------------+------------+----------------------------------+-----------------------+--------------------------------------------+
+| Function name | Arity | Input types | Output type | Options class |
++==========================+============+==================================+=======================+============================================+
+| binary_contains_exact | Unary | String-like | Boolean (1) | :struct:`BinaryContainsExactOptions` |
++--------------------------+------------+----------------------------------+-----------------------+--------------------------------------------+
+| isin | Unary | Binary- and String-like | Boolean (2) | :struct:`SetLookupOptions` |
++--------------------------+------------+----------------------------------+-----------------------+--------------------------------------------+
+| isin | Unary | Null | Boolean (2) | :struct:`SetLookupOptions` |
++--------------------------+------------+----------------------------------+-----------------------+--------------------------------------------+
+| isin | Unary | Boolean,Numeric, Temporal | Boolean (2) | :struct:`SetLookupOptions` |
++--------------------------+------------+----------------------------------+-----------------------+--------------------------------------------+
+| match | Unary | Binary- and String-like | Int32 (3) | :struct:`SetLookupOptions` |
++--------------------------+------------+----------------------------------+-----------------------+--------------------------------------------+
+| match | Unary | Null | Int32 (3) | :struct:`SetLookupOptions` |
++--------------------------+------------+----------------------------------+-----------------------+--------------------------------------------+
+| match | Unary | Boolean,Numeric, Temporal | Int32 (3) | :struct:`SetLookupOptions` |
++--------------------------+------------+----------------------------------+-----------------------+--------------------------------------------+
+
+* \(1) Output is true iff :member:`BinaryContainsExactOptions::pattern`
+ is a substring of the corresponding input element.
+
+* \(2) Output is true iff the corresponding input element is equal to one
+ of the elements in :member:`SetLookupOptions::value_set`.
+
+* \(3) Output is the index of the corresponding input element in
+ :member:`SetLookupOptions::value_set`, if found there. Otherwise,
+ output is null.
+
+Structural transforms
+~~~~~~~~~~~~~~~~~~~~~
+
++--------------------------+------------+--------------------+---------------------+---------+
+| Function name | Arity | Input types | Output type | Notes |
++==========================+============+====================+=====================+=========+
+| is_null | Unary | Any | Boolean | \(1) |
++--------------------------+------------+--------------------+---------------------+---------+
+| is_valid | Unary | Any | Boolean | \(2) |
++--------------------------+------------+--------------------+---------------------+---------+
+| list_value_lengths | Unary | List-like | Int32 or Int64 | \(3) |
++--------------------------+------------+--------------------+---------------------+---------+
+
+* \(1) Output is true iff the corresponding input element is non-null.
+
+* \(2) Output is true iff the corresponding input element is null.
+
+* \(3) Each output element is the length of the corresponding input element
+ (null if input is null).
+
+Conversions
+~~~~~~~~~~~
+
+A general conversion function named ``cast`` is provided which accepts a large
+number of input and output types. The type to cast to can be passed in a
+:struct:`CastOptions` instance.
+
++--------------------------+------------+--------------------+-----------------------+--------------------------------------------+
+| Function name | Arity | Input types | Output type | Options class |
++==========================+============+====================+=======================+============================================+
+| cast | Unary | Many | Variable | :struct:`CastOptions` |
++--------------------------+------------+--------------------+-----------------------+--------------------------------------------+
+| strptime | Unary | String-like | Timestamp | :struct:`StrptimeOptions` |
++--------------------------+------------+--------------------+-----------------------+--------------------------------------------+
+
+The conversions available with ``cast`` are listed below. In all cases, a
+null input value is converted into a null output value.
+
+**Truth value extraction**
+
++-----------------------------+------------------------------------+--------------+
+| Input type | Output type | Notes |
++=============================+====================================+==============+
+| Binary- and String-like | Boolean | \(1) |
++-----------------------------+------------------------------------+--------------+
+| Numeric | Boolean | \(2) |
++-----------------------------+------------------------------------+--------------+
+
+* \(1) Output is true iff the corresponding input value has non-zero length.
+
+* \(2) Output is true iff the corresponding input value is non-zero.
+
+**Same-kind conversion**
+
++-----------------------------+------------------------------------+--------------+
+| Input type | Output type | Notes |
++=============================+====================================+==============+
+| Int32 | 32-bit Temporal | \(1) |
++-----------------------------+------------------------------------+--------------+
+| Int64 | 64-bit Temporal | \(1) |
++-----------------------------+------------------------------------+--------------+
+| (Large)Binary | (Large)String | \(2) |
++-----------------------------+------------------------------------+--------------+
+| (Large)String | (Large)Binary | \(3) |
++-----------------------------+------------------------------------+--------------+
+| Numeric | Numeric | \(4) \(5) |
++-----------------------------+------------------------------------+--------------+
+| 32-bit Temporal | Int32 | \(1) |
++-----------------------------+------------------------------------+--------------+
+| 64-bit Temporal | Int64 | \(1) |
++-----------------------------+------------------------------------+--------------+
+| Temporal | Temporal | \(4) \(5) |
++-----------------------------+------------------------------------+--------------+
+
+* \(1) No-operation cast: the raw values are kept identical, only
+ the type is changed.
+
+* \(2) Validates the contents if :member:`CastOptions::allow_invalid_utf8`
+ is false.
+
+* \(3) No-operation cast: only the type is changed.
+
+* \(4) Overflow and truncation checks are enabled depending on
+ the given :struct:`CastOptions`.
+
+* \(5) Not all such casts have been implemented.
+
+**String representations**
+
++-----------------------------+------------------------------------+---------+
+| Input type | Output type | Notes |
++=============================+====================================+=========+
+| Boolean | String-like | |
++-----------------------------+------------------------------------+---------+
+| Numeric | String-like | |
++-----------------------------+------------------------------------+---------+
+
+**Generic conversions**
+
++-----------------------------+------------------------------------+---------+
+| Input type | Output type | Notes |
++=============================+====================================+=========+
+| Dictionary | Dictionary value type | |
++-----------------------------+------------------------------------+---------+
+| Extension | Extension storage type | |
++-----------------------------+------------------------------------+---------+
+| List-like | List-like | \(1) |
++-----------------------------+------------------------------------+---------+
+| Null | Any | |
++-----------------------------+------------------------------------+---------+
+
+* \(1) The list offsets are unchanged, the list values are cast from the
+ input value type to the output value type (if a conversion is
+ available).
+
+
+.. TODO: add C++ cast example
+
+Array-wise ("vector") functions
+-------------------------------
+
+Associative transforms
+~~~~~~~~~~~~~~~~~~~~~~
+
++--------------------------+------------+----------------------------+----------------------------+
+| Function name | Arity | Input types | Output type |
++==========================+============+============================+============================+
+| dictionary_encode | Unary | Binary- and String-like | Dictionary (1) |
Review comment:
Why isn't `dictionary_encode` just cast to Dictionary?
##########
File path: docs/source/cpp/compute.rst
##########
@@ -0,0 +1,419 @@
+.. Licensed to the Apache Software Foundation (ASF) under one
+.. or more contributor license agreements. See the NOTICE file
+.. distributed with this work for additional information
+.. regarding copyright ownership. The ASF licenses this file
+.. to you under the Apache License, Version 2.0 (the
+.. "License"); you may not use this file except in compliance
+.. with the License. You may obtain a copy of the License at
+
+.. http://www.apache.org/licenses/LICENSE-2.0
+
+.. Unless required by applicable law or agreed to in writing,
+.. software distributed under the License is distributed on an
+.. "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+.. KIND, either express or implied. See the License for the
+.. specific language governing permissions and limitations
+.. under the License.
+
+.. default-domain:: cpp
+.. highlight:: cpp
+.. cpp:namespace:: arrow::compute
+
+=================
+Compute Functions
+=================
+
+.. TODO: describe API and how to invoke compute functions
+
+Available functions
+===================
+
+Aggregations
+------------
+
++--------------------------+------------+--------------------+-----------------------+--------------------------------------------+
+| Function name | Arity | Input types | Output type | Options class |
++==========================+============+====================+=======================+============================================+
+| count | Unary | Any | Scalar Int64 | :struct:`CountOptions` |
++--------------------------+------------+--------------------+-----------------------+--------------------------------------------+
+| mean | Unary | Numeric | Scalar Float64 | |
++--------------------------+------------+--------------------+-----------------------+--------------------------------------------+
+| minmax | Unary | Numeric | Scalar Struct (1) | :struct:`MinMaxOptions` |
++--------------------------+------------+--------------------+-----------------------+--------------------------------------------+
+| sum | Unary | Numeric | Scalar Numeric (2) | |
++--------------------------+------------+--------------------+-----------------------+--------------------------------------------+
+
+Notes:
+
+* \(1) Output is a ``{"min": input type, "max": input type}`` Struct
+
+* \(2) Output is Int64, UInt64 or Float64, depending on the input type
+
+
+Element-wise ("scalar") functions
+---------------------------------
+
+Arithmetic functions
+~~~~~~~~~~~~~~~~~~~~
+
+Those functions expect two inputs of the same type and apply a given binary
+operation to each pair of elements gathered from the inputs. Each function
+is also available in an overflow-checking variant, suffixed ``_checked``.
+
+If any of the input elements in a pair is null, the corresponding output
+element is null.
+
++--------------------------+------------+--------------------+---------------------+
+| Function name | Arity | Input types | Output type |
++==========================+============+====================+=====================+
+| add | Binary | Numeric | Numeric |
++--------------------------+------------+--------------------+---------------------+
+| add_checked | Binary | Numeric | Numeric |
++--------------------------+------------+--------------------+---------------------+
+| multiply | Binary | Numeric | Numeric |
++--------------------------+------------+--------------------+---------------------+
+| multiply_checked | Binary | Numeric | Numeric |
++--------------------------+------------+--------------------+---------------------+
+| subtract | Binary | Numeric | Numeric |
++--------------------------+------------+--------------------+---------------------+
+| subtract_checked | Binary | Numeric | Numeric |
++--------------------------+------------+--------------------+---------------------+
+
+Comparisons
+~~~~~~~~~~~
+
+Those functions expect two inputs of the same type and apply a given
+comparison operator. If any of the input elements in a pair is null,
+the corresponding output element is null.
+
++--------------------------+------------+---------------------------------+---------------------+
+| Function names | Arity | Input types | Output type |
++==========================+============+=================================+=====================+
+| equal, not_equal | Binary | Numeric | Boolean |
++--------------------------+------------+---------------------------------+---------------------+
+| equal, not_equal | Binary | Binary- and String-like | Boolean |
++--------------------------+------------+---------------------------------+---------------------+
+| equal, not_equal | Binary | Temporal | Boolean |
++--------------------------+------------+---------------------------------+---------------------+
+| greater, greater_equal, | Binary | Numeric | Boolean |
+| less, less_equal | | | |
++--------------------------+------------+---------------------------------+---------------------+
+| greater, greater_equal, | Binary | Binary- and String-like | Boolean |
+| less, less_equal | | | |
++--------------------------+------------+---------------------------------+---------------------+
+| greater, greater_equal, | Binary | Temporal | Boolean |
+| less, less_equal | | | |
++--------------------------+------------+---------------------------------+---------------------+
+
+Logical functions
+~~~~~~~~~~~~~~~~~~
+
+The normal behaviour for these functions is to emit a null if any of the
+inputs is null.
+
+Some of them are also available in a "`Kleene logic`_" variant (suffixed
+``_kleene``) where null is taken to mean "undefined". For those variants
+therefore:
+
+* "true AND null", "null AND true" give "null" (the result is undefined)
Review comment:
Maybe some 3 x 3 truth tables (T, F, NA) would more clearly show the expectations.
##########
File path: docs/source/cpp/compute.rst
##########
@@ -0,0 +1,419 @@
+.. Licensed to the Apache Software Foundation (ASF) under one
+.. or more contributor license agreements. See the NOTICE file
+.. distributed with this work for additional information
+.. regarding copyright ownership. The ASF licenses this file
+.. to you under the Apache License, Version 2.0 (the
+.. "License"); you may not use this file except in compliance
+.. with the License. You may obtain a copy of the License at
+
+.. http://www.apache.org/licenses/LICENSE-2.0
+
+.. Unless required by applicable law or agreed to in writing,
+.. software distributed under the License is distributed on an
+.. "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+.. KIND, either express or implied. See the License for the
+.. specific language governing permissions and limitations
+.. under the License.
+
+.. default-domain:: cpp
+.. highlight:: cpp
+.. cpp:namespace:: arrow::compute
+
+=================
+Compute Functions
+=================
+
+.. TODO: describe API and how to invoke compute functions
+
+Available functions
+===================
+
+Aggregations
+------------
+
++--------------------------+------------+--------------------+-----------------------+--------------------------------------------+
+| Function name | Arity | Input types | Output type | Options class |
++==========================+============+====================+=======================+============================================+
+| count | Unary | Any | Scalar Int64 | :struct:`CountOptions` |
++--------------------------+------------+--------------------+-----------------------+--------------------------------------------+
+| mean | Unary | Numeric | Scalar Float64 | |
++--------------------------+------------+--------------------+-----------------------+--------------------------------------------+
+| minmax | Unary | Numeric | Scalar Struct (1) | :struct:`MinMaxOptions` |
++--------------------------+------------+--------------------+-----------------------+--------------------------------------------+
+| sum | Unary | Numeric | Scalar Numeric (2) | |
++--------------------------+------------+--------------------+-----------------------+--------------------------------------------+
+
+Notes:
+
+* \(1) Output is a ``{"min": input type, "max": input type}`` Struct
+
+* \(2) Output is Int64, UInt64 or Float64, depending on the input type
+
+
+Element-wise ("scalar") functions
+---------------------------------
+
+Arithmetic functions
+~~~~~~~~~~~~~~~~~~~~
+
+Those functions expect two inputs of the same type and apply a given binary
+operation to each pair of elements gathered from the inputs. Each function
+is also available in an overflow-checking variant, suffixed ``_checked``.
+
+If any of the input elements in a pair is null, the corresponding output
+element is null.
+
++--------------------------+------------+--------------------+---------------------+
+| Function name | Arity | Input types | Output type |
++==========================+============+====================+=====================+
+| add | Binary | Numeric | Numeric |
++--------------------------+------------+--------------------+---------------------+
+| add_checked | Binary | Numeric | Numeric |
++--------------------------+------------+--------------------+---------------------+
+| multiply | Binary | Numeric | Numeric |
++--------------------------+------------+--------------------+---------------------+
+| multiply_checked | Binary | Numeric | Numeric |
++--------------------------+------------+--------------------+---------------------+
+| subtract | Binary | Numeric | Numeric |
++--------------------------+------------+--------------------+---------------------+
+| subtract_checked | Binary | Numeric | Numeric |
++--------------------------+------------+--------------------+---------------------+
+
+Comparisons
+~~~~~~~~~~~
+
+Those functions expect two inputs of the same type and apply a given
+comparison operator. If any of the input elements in a pair is null,
+the corresponding output element is null.
+
++--------------------------+------------+---------------------------------+---------------------+
+| Function names | Arity | Input types | Output type |
++==========================+============+=================================+=====================+
+| equal, not_equal | Binary | Numeric | Boolean |
++--------------------------+------------+---------------------------------+---------------------+
+| equal, not_equal | Binary | Binary- and String-like | Boolean |
Review comment:
I'm not sure I understand why there are three rows for `equal, not_equal`
##########
File path: docs/source/cpp/compute.rst
##########
@@ -0,0 +1,419 @@
+.. Licensed to the Apache Software Foundation (ASF) under one
+.. or more contributor license agreements. See the NOTICE file
+.. distributed with this work for additional information
+.. regarding copyright ownership. The ASF licenses this file
+.. to you under the Apache License, Version 2.0 (the
+.. "License"); you may not use this file except in compliance
+.. with the License. You may obtain a copy of the License at
+
+.. http://www.apache.org/licenses/LICENSE-2.0
+
+.. Unless required by applicable law or agreed to in writing,
+.. software distributed under the License is distributed on an
+.. "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+.. KIND, either express or implied. See the License for the
+.. specific language governing permissions and limitations
+.. under the License.
+
+.. default-domain:: cpp
+.. highlight:: cpp
+.. cpp:namespace:: arrow::compute
+
+=================
+Compute Functions
+=================
+
+.. TODO: describe API and how to invoke compute functions
+
+Available functions
+===================
+
+Aggregations
+------------
+
++--------------------------+------------+--------------------+-----------------------+--------------------------------------------+
+| Function name | Arity | Input types | Output type | Options class |
++==========================+============+====================+=======================+============================================+
+| count | Unary | Any | Scalar Int64 | :struct:`CountOptions` |
++--------------------------+------------+--------------------+-----------------------+--------------------------------------------+
+| mean | Unary | Numeric | Scalar Float64 | |
++--------------------------+------------+--------------------+-----------------------+--------------------------------------------+
+| minmax | Unary | Numeric | Scalar Struct (1) | :struct:`MinMaxOptions` |
++--------------------------+------------+--------------------+-----------------------+--------------------------------------------+
+| sum | Unary | Numeric | Scalar Numeric (2) | |
++--------------------------+------------+--------------------+-----------------------+--------------------------------------------+
+
+Notes:
+
+* \(1) Output is a ``{"min": input type, "max": input type}`` Struct
+
+* \(2) Output is Int64, UInt64 or Float64, depending on the input type
+
+
+Element-wise ("scalar") functions
+---------------------------------
+
+Arithmetic functions
+~~~~~~~~~~~~~~~~~~~~
+
+Those functions expect two inputs of the same type and apply a given binary
+operation to each pair of elements gathered from the inputs. Each function
+is also available in an overflow-checking variant, suffixed ``_checked``.
+
+If any of the input elements in a pair is null, the corresponding output
+element is null.
+
++--------------------------+------------+--------------------+---------------------+
+| Function name | Arity | Input types | Output type |
++==========================+============+====================+=====================+
+| add | Binary | Numeric | Numeric |
++--------------------------+------------+--------------------+---------------------+
+| add_checked | Binary | Numeric | Numeric |
++--------------------------+------------+--------------------+---------------------+
+| multiply | Binary | Numeric | Numeric |
++--------------------------+------------+--------------------+---------------------+
+| multiply_checked | Binary | Numeric | Numeric |
++--------------------------+------------+--------------------+---------------------+
+| subtract | Binary | Numeric | Numeric |
++--------------------------+------------+--------------------+---------------------+
+| subtract_checked | Binary | Numeric | Numeric |
++--------------------------+------------+--------------------+---------------------+
+
+Comparisons
+~~~~~~~~~~~
+
+Those functions expect two inputs of the same type and apply a given
+comparison operator. If any of the input elements in a pair is null,
+the corresponding output element is null.
+
++--------------------------+------------+---------------------------------+---------------------+
+| Function names | Arity | Input types | Output type |
++==========================+============+=================================+=====================+
+| equal, not_equal | Binary | Numeric | Boolean |
++--------------------------+------------+---------------------------------+---------------------+
+| equal, not_equal | Binary | Binary- and String-like | Boolean |
++--------------------------+------------+---------------------------------+---------------------+
+| equal, not_equal | Binary | Temporal | Boolean |
++--------------------------+------------+---------------------------------+---------------------+
+| greater, greater_equal, | Binary | Numeric | Boolean |
+| less, less_equal | | | |
++--------------------------+------------+---------------------------------+---------------------+
+| greater, greater_equal, | Binary | Binary- and String-like | Boolean |
+| less, less_equal | | | |
++--------------------------+------------+---------------------------------+---------------------+
+| greater, greater_equal, | Binary | Temporal | Boolean |
+| less, less_equal | | | |
++--------------------------+------------+---------------------------------+---------------------+
+
+Logical functions
+~~~~~~~~~~~~~~~~~~
+
+The normal behaviour for these functions is to emit a null if any of the
+inputs is null.
+
+Some of them are also available in a "`Kleene logic`_" variant (suffixed
Review comment:
It might be worth noting that this is what R, Numpy, etc. do. And is the "normal behavior" default for some other systems? Just to help users understand which one they should use to get their expected results.
##########
File path: docs/source/cpp/compute.rst
##########
@@ -0,0 +1,419 @@
+.. Licensed to the Apache Software Foundation (ASF) under one
+.. or more contributor license agreements. See the NOTICE file
+.. distributed with this work for additional information
+.. regarding copyright ownership. The ASF licenses this file
+.. to you under the Apache License, Version 2.0 (the
+.. "License"); you may not use this file except in compliance
+.. with the License. You may obtain a copy of the License at
+
+.. http://www.apache.org/licenses/LICENSE-2.0
+
+.. Unless required by applicable law or agreed to in writing,
+.. software distributed under the License is distributed on an
+.. "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+.. KIND, either express or implied. See the License for the
+.. specific language governing permissions and limitations
+.. under the License.
+
+.. default-domain:: cpp
+.. highlight:: cpp
+.. cpp:namespace:: arrow::compute
+
+=================
+Compute Functions
+=================
+
+.. TODO: describe API and how to invoke compute functions
+
+Available functions
+===================
+
+Aggregations
+------------
+
++--------------------------+------------+--------------------+-----------------------+--------------------------------------------+
+| Function name | Arity | Input types | Output type | Options class |
++==========================+============+====================+=======================+============================================+
+| count | Unary | Any | Scalar Int64 | :struct:`CountOptions` |
++--------------------------+------------+--------------------+-----------------------+--------------------------------------------+
+| mean | Unary | Numeric | Scalar Float64 | |
++--------------------------+------------+--------------------+-----------------------+--------------------------------------------+
+| minmax | Unary | Numeric | Scalar Struct (1) | :struct:`MinMaxOptions` |
++--------------------------+------------+--------------------+-----------------------+--------------------------------------------+
+| sum | Unary | Numeric | Scalar Numeric (2) | |
++--------------------------+------------+--------------------+-----------------------+--------------------------------------------+
+
+Notes:
+
+* \(1) Output is a ``{"min": input type, "max": input type}`` Struct
+
+* \(2) Output is Int64, UInt64 or Float64, depending on the input type
+
+
+Element-wise ("scalar") functions
+---------------------------------
+
+Arithmetic functions
+~~~~~~~~~~~~~~~~~~~~
+
+Those functions expect two inputs of the same type and apply a given binary
Review comment:
What are the constraints on the pairs of inputs? I haven't tested all combinations, but it seems that generally you can do `Array OPERATOR Scalar`, `Array OPERATOR Array` iff the arrays are the same length, etc., and also for ChunkedArray. Some are also defined for RecordBatch and Table too right?
##########
File path: docs/source/cpp/compute.rst
##########
@@ -0,0 +1,419 @@
+.. Licensed to the Apache Software Foundation (ASF) under one
+.. or more contributor license agreements. See the NOTICE file
+.. distributed with this work for additional information
+.. regarding copyright ownership. The ASF licenses this file
+.. to you under the Apache License, Version 2.0 (the
+.. "License"); you may not use this file except in compliance
+.. with the License. You may obtain a copy of the License at
+
+.. http://www.apache.org/licenses/LICENSE-2.0
+
+.. Unless required by applicable law or agreed to in writing,
+.. software distributed under the License is distributed on an
+.. "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+.. KIND, either express or implied. See the License for the
+.. specific language governing permissions and limitations
+.. under the License.
+
+.. default-domain:: cpp
+.. highlight:: cpp
+.. cpp:namespace:: arrow::compute
+
+=================
+Compute Functions
+=================
+
+.. TODO: describe API and how to invoke compute functions
+
+Available functions
+===================
+
+Aggregations
+------------
+
++--------------------------+------------+--------------------+-----------------------+--------------------------------------------+
+| Function name | Arity | Input types | Output type | Options class |
++==========================+============+====================+=======================+============================================+
+| count | Unary | Any | Scalar Int64 | :struct:`CountOptions` |
++--------------------------+------------+--------------------+-----------------------+--------------------------------------------+
+| mean | Unary | Numeric | Scalar Float64 | |
++--------------------------+------------+--------------------+-----------------------+--------------------------------------------+
+| minmax | Unary | Numeric | Scalar Struct (1) | :struct:`MinMaxOptions` |
++--------------------------+------------+--------------------+-----------------------+--------------------------------------------+
+| sum | Unary | Numeric | Scalar Numeric (2) | |
++--------------------------+------------+--------------------+-----------------------+--------------------------------------------+
+
+Notes:
+
+* \(1) Output is a ``{"min": input type, "max": input type}`` Struct
+
+* \(2) Output is Int64, UInt64 or Float64, depending on the input type
+
+
+Element-wise ("scalar") functions
+---------------------------------
+
+Arithmetic functions
+~~~~~~~~~~~~~~~~~~~~
+
+Those functions expect two inputs of the same type and apply a given binary
+operation to each pair of elements gathered from the inputs. Each function
+is also available in an overflow-checking variant, suffixed ``_checked``.
+
+If any of the input elements in a pair is null, the corresponding output
+element is null.
+
++--------------------------+------------+--------------------+---------------------+
+| Function name | Arity | Input types | Output type |
++==========================+============+====================+=====================+
+| add | Binary | Numeric | Numeric |
++--------------------------+------------+--------------------+---------------------+
+| add_checked | Binary | Numeric | Numeric |
++--------------------------+------------+--------------------+---------------------+
+| multiply | Binary | Numeric | Numeric |
++--------------------------+------------+--------------------+---------------------+
+| multiply_checked | Binary | Numeric | Numeric |
++--------------------------+------------+--------------------+---------------------+
+| subtract | Binary | Numeric | Numeric |
++--------------------------+------------+--------------------+---------------------+
+| subtract_checked | Binary | Numeric | Numeric |
++--------------------------+------------+--------------------+---------------------+
+
+Comparisons
+~~~~~~~~~~~
+
+Those functions expect two inputs of the same type and apply a given
+comparison operator. If any of the input elements in a pair is null,
+the corresponding output element is null.
+
++--------------------------+------------+---------------------------------+---------------------+
+| Function names | Arity | Input types | Output type |
++==========================+============+=================================+=====================+
+| equal, not_equal | Binary | Numeric | Boolean |
++--------------------------+------------+---------------------------------+---------------------+
+| equal, not_equal | Binary | Binary- and String-like | Boolean |
++--------------------------+------------+---------------------------------+---------------------+
+| equal, not_equal | Binary | Temporal | Boolean |
++--------------------------+------------+---------------------------------+---------------------+
+| greater, greater_equal, | Binary | Numeric | Boolean |
+| less, less_equal | | | |
++--------------------------+------------+---------------------------------+---------------------+
+| greater, greater_equal, | Binary | Binary- and String-like | Boolean |
+| less, less_equal | | | |
++--------------------------+------------+---------------------------------+---------------------+
+| greater, greater_equal, | Binary | Temporal | Boolean |
+| less, less_equal | | | |
++--------------------------+------------+---------------------------------+---------------------+
+
+Logical functions
+~~~~~~~~~~~~~~~~~~
+
+The normal behaviour for these functions is to emit a null if any of the
+inputs is null.
+
+Some of them are also available in a "`Kleene logic`_" variant (suffixed
+``_kleene``) where null is taken to mean "undefined". For those variants
+therefore:
+
+* "true AND null", "null AND true" give "null" (the result is undefined)
+* "true OR null", "null OR true" give "true"
+* "false AND null", "null AND false" give "false"
+* "false OR null", "null OR false" give "null" (the result is undefined)
+
++--------------------------+------------+--------------------+---------------------+
+| Function name | Arity | Input types | Output type |
++==========================+============+====================+=====================+
+| and | Binary | Boolean | Boolean |
++--------------------------+------------+--------------------+---------------------+
+| and_kleene | Binary | Boolean | Boolean |
++--------------------------+------------+--------------------+---------------------+
+| invert | Unary | Boolean | Boolean |
++--------------------------+------------+--------------------+---------------------+
+| or | Binary | Boolean | Boolean |
++--------------------------+------------+--------------------+---------------------+
+| or_kleene | Binary | Boolean | Boolean |
++--------------------------+------------+--------------------+---------------------+
+| xor | Binary | Boolean | Boolean |
++--------------------------+------------+--------------------+---------------------+
+
+.. _Kleene logic: https://en.wikipedia.org/wiki/Three-valued_logic#Kleene_and_Priest_logics
+
+String functions
+~~~~~~~~~~~~~~~~
+
++--------------------------+------------+--------------------+---------------------+---------+
+| Function name | Arity | Input types | Output type | Notes |
++==========================+============+====================+=====================+=========+
+| ascii_length | Unary | String-like | Int32 or Int64 | \(1) |
++--------------------------+------------+--------------------+---------------------+---------+
+| ascii_lower | Unary | String-like | String-like | \(2) |
++--------------------------+------------+--------------------+---------------------+---------+
+| ascii_upper | Unary | String-like | String-like | \(2) |
++--------------------------+------------+--------------------+---------------------+---------+
+| utf8_lower | Unary | String-like | String-like | \(3) |
++--------------------------+------------+--------------------+---------------------+---------+
+| utf8_upper | Unary | String-like | String-like | \(3) |
++--------------------------+------------+--------------------+---------------------+---------+
+
+* \(1) Output is the physical length in bytes of each input element.
+
+* \(2) Each ASCII character in the input is converted to lowercase or
+ uppercase. Non-ASCII characters are left untouched.
+
+* \(3) Each UTF8-encoded character in the input is converted to lowercase or
+ uppercase.
+
+Containment tests
+~~~~~~~~~~~~~~~~~
+
++--------------------------+------------+----------------------------------+-----------------------+--------------------------------------------+
+| Function name | Arity | Input types | Output type | Options class |
++==========================+============+==================================+=======================+============================================+
+| binary_contains_exact | Unary | String-like | Boolean (1) | :struct:`BinaryContainsExactOptions` |
++--------------------------+------------+----------------------------------+-----------------------+--------------------------------------------+
+| isin | Unary | Binary- and String-like | Boolean (2) | :struct:`SetLookupOptions` |
++--------------------------+------------+----------------------------------+-----------------------+--------------------------------------------+
+| isin | Unary | Null | Boolean (2) | :struct:`SetLookupOptions` |
++--------------------------+------------+----------------------------------+-----------------------+--------------------------------------------+
+| isin | Unary | Boolean,Numeric, Temporal | Boolean (2) | :struct:`SetLookupOptions` |
++--------------------------+------------+----------------------------------+-----------------------+--------------------------------------------+
+| match | Unary | Binary- and String-like | Int32 (3) | :struct:`SetLookupOptions` |
++--------------------------+------------+----------------------------------+-----------------------+--------------------------------------------+
+| match | Unary | Null | Int32 (3) | :struct:`SetLookupOptions` |
++--------------------------+------------+----------------------------------+-----------------------+--------------------------------------------+
+| match | Unary | Boolean,Numeric, Temporal | Int32 (3) | :struct:`SetLookupOptions` |
++--------------------------+------------+----------------------------------+-----------------------+--------------------------------------------+
+
+* \(1) Output is true iff :member:`BinaryContainsExactOptions::pattern`
+ is a substring of the corresponding input element.
+
+* \(2) Output is true iff the corresponding input element is equal to one
+ of the elements in :member:`SetLookupOptions::value_set`.
+
+* \(3) Output is the index of the corresponding input element in
+ :member:`SetLookupOptions::value_set`, if found there. Otherwise,
+ output is null.
+
+Structural transforms
+~~~~~~~~~~~~~~~~~~~~~
+
++--------------------------+------------+--------------------+---------------------+---------+
+| Function name | Arity | Input types | Output type | Notes |
++==========================+============+====================+=====================+=========+
+| is_null | Unary | Any | Boolean | \(1) |
++--------------------------+------------+--------------------+---------------------+---------+
+| is_valid | Unary | Any | Boolean | \(2) |
++--------------------------+------------+--------------------+---------------------+---------+
+| list_value_lengths | Unary | List-like | Int32 or Int64 | \(3) |
Review comment:
What determines whether it's int32 or int64?
##########
File path: docs/source/cpp/compute.rst
##########
@@ -0,0 +1,419 @@
+.. Licensed to the Apache Software Foundation (ASF) under one
+.. or more contributor license agreements. See the NOTICE file
+.. distributed with this work for additional information
+.. regarding copyright ownership. The ASF licenses this file
+.. to you under the Apache License, Version 2.0 (the
+.. "License"); you may not use this file except in compliance
+.. with the License. You may obtain a copy of the License at
+
+.. http://www.apache.org/licenses/LICENSE-2.0
+
+.. Unless required by applicable law or agreed to in writing,
+.. software distributed under the License is distributed on an
+.. "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+.. KIND, either express or implied. See the License for the
+.. specific language governing permissions and limitations
+.. under the License.
+
+.. default-domain:: cpp
+.. highlight:: cpp
+.. cpp:namespace:: arrow::compute
+
+=================
+Compute Functions
+=================
+
+.. TODO: describe API and how to invoke compute functions
+
+Available functions
+===================
+
+Aggregations
+------------
+
++--------------------------+------------+--------------------+-----------------------+--------------------------------------------+
+| Function name | Arity | Input types | Output type | Options class |
++==========================+============+====================+=======================+============================================+
+| count | Unary | Any | Scalar Int64 | :struct:`CountOptions` |
++--------------------------+------------+--------------------+-----------------------+--------------------------------------------+
+| mean | Unary | Numeric | Scalar Float64 | |
++--------------------------+------------+--------------------+-----------------------+--------------------------------------------+
+| minmax | Unary | Numeric | Scalar Struct (1) | :struct:`MinMaxOptions` |
++--------------------------+------------+--------------------+-----------------------+--------------------------------------------+
+| sum | Unary | Numeric | Scalar Numeric (2) | |
++--------------------------+------------+--------------------+-----------------------+--------------------------------------------+
+
+Notes:
+
+* \(1) Output is a ``{"min": input type, "max": input type}`` Struct
+
+* \(2) Output is Int64, UInt64 or Float64, depending on the input type
+
+
+Element-wise ("scalar") functions
+---------------------------------
+
+Arithmetic functions
+~~~~~~~~~~~~~~~~~~~~
+
+Those functions expect two inputs of the same type and apply a given binary
+operation to each pair of elements gathered from the inputs. Each function
+is also available in an overflow-checking variant, suffixed ``_checked``.
+
+If any of the input elements in a pair is null, the corresponding output
+element is null.
+
++--------------------------+------------+--------------------+---------------------+
+| Function name | Arity | Input types | Output type |
++==========================+============+====================+=====================+
+| add | Binary | Numeric | Numeric |
++--------------------------+------------+--------------------+---------------------+
+| add_checked | Binary | Numeric | Numeric |
++--------------------------+------------+--------------------+---------------------+
+| multiply | Binary | Numeric | Numeric |
++--------------------------+------------+--------------------+---------------------+
+| multiply_checked | Binary | Numeric | Numeric |
++--------------------------+------------+--------------------+---------------------+
+| subtract | Binary | Numeric | Numeric |
++--------------------------+------------+--------------------+---------------------+
+| subtract_checked | Binary | Numeric | Numeric |
++--------------------------+------------+--------------------+---------------------+
+
+Comparisons
+~~~~~~~~~~~
+
+Those functions expect two inputs of the same type and apply a given
+comparison operator. If any of the input elements in a pair is null,
+the corresponding output element is null.
+
++--------------------------+------------+---------------------------------+---------------------+
+| Function names | Arity | Input types | Output type |
++==========================+============+=================================+=====================+
+| equal, not_equal | Binary | Numeric | Boolean |
++--------------------------+------------+---------------------------------+---------------------+
+| equal, not_equal | Binary | Binary- and String-like | Boolean |
++--------------------------+------------+---------------------------------+---------------------+
+| equal, not_equal | Binary | Temporal | Boolean |
++--------------------------+------------+---------------------------------+---------------------+
+| greater, greater_equal, | Binary | Numeric | Boolean |
+| less, less_equal | | | |
++--------------------------+------------+---------------------------------+---------------------+
+| greater, greater_equal, | Binary | Binary- and String-like | Boolean |
+| less, less_equal | | | |
++--------------------------+------------+---------------------------------+---------------------+
+| greater, greater_equal, | Binary | Temporal | Boolean |
+| less, less_equal | | | |
++--------------------------+------------+---------------------------------+---------------------+
+
+Logical functions
+~~~~~~~~~~~~~~~~~~
+
+The normal behaviour for these functions is to emit a null if any of the
+inputs is null.
+
+Some of them are also available in a "`Kleene logic`_" variant (suffixed
+``_kleene``) where null is taken to mean "undefined". For those variants
+therefore:
+
+* "true AND null", "null AND true" give "null" (the result is undefined)
+* "true OR null", "null OR true" give "true"
+* "false AND null", "null AND false" give "false"
+* "false OR null", "null OR false" give "null" (the result is undefined)
+
++--------------------------+------------+--------------------+---------------------+
+| Function name | Arity | Input types | Output type |
++==========================+============+====================+=====================+
+| and | Binary | Boolean | Boolean |
++--------------------------+------------+--------------------+---------------------+
+| and_kleene | Binary | Boolean | Boolean |
++--------------------------+------------+--------------------+---------------------+
+| invert | Unary | Boolean | Boolean |
++--------------------------+------------+--------------------+---------------------+
+| or | Binary | Boolean | Boolean |
++--------------------------+------------+--------------------+---------------------+
+| or_kleene | Binary | Boolean | Boolean |
++--------------------------+------------+--------------------+---------------------+
+| xor | Binary | Boolean | Boolean |
++--------------------------+------------+--------------------+---------------------+
+
+.. _Kleene logic: https://en.wikipedia.org/wiki/Three-valued_logic#Kleene_and_Priest_logics
+
+String functions
+~~~~~~~~~~~~~~~~
+
++--------------------------+------------+--------------------+---------------------+---------+
+| Function name | Arity | Input types | Output type | Notes |
++==========================+============+====================+=====================+=========+
+| ascii_length | Unary | String-like | Int32 or Int64 | \(1) |
++--------------------------+------------+--------------------+---------------------+---------+
+| ascii_lower | Unary | String-like | String-like | \(2) |
++--------------------------+------------+--------------------+---------------------+---------+
+| ascii_upper | Unary | String-like | String-like | \(2) |
++--------------------------+------------+--------------------+---------------------+---------+
+| utf8_lower | Unary | String-like | String-like | \(3) |
++--------------------------+------------+--------------------+---------------------+---------+
+| utf8_upper | Unary | String-like | String-like | \(3) |
++--------------------------+------------+--------------------+---------------------+---------+
+
+* \(1) Output is the physical length in bytes of each input element.
+
+* \(2) Each ASCII character in the input is converted to lowercase or
+ uppercase. Non-ASCII characters are left untouched.
+
+* \(3) Each UTF8-encoded character in the input is converted to lowercase or
+ uppercase.
+
+Containment tests
+~~~~~~~~~~~~~~~~~
+
++--------------------------+------------+----------------------------------+-----------------------+--------------------------------------------+
+| Function name | Arity | Input types | Output type | Options class |
++==========================+============+==================================+=======================+============================================+
+| binary_contains_exact | Unary | String-like | Boolean (1) | :struct:`BinaryContainsExactOptions` |
++--------------------------+------------+----------------------------------+-----------------------+--------------------------------------------+
+| isin | Unary | Binary- and String-like | Boolean (2) | :struct:`SetLookupOptions` |
++--------------------------+------------+----------------------------------+-----------------------+--------------------------------------------+
+| isin | Unary | Null | Boolean (2) | :struct:`SetLookupOptions` |
++--------------------------+------------+----------------------------------+-----------------------+--------------------------------------------+
+| isin | Unary | Boolean,Numeric, Temporal | Boolean (2) | :struct:`SetLookupOptions` |
++--------------------------+------------+----------------------------------+-----------------------+--------------------------------------------+
+| match | Unary | Binary- and String-like | Int32 (3) | :struct:`SetLookupOptions` |
++--------------------------+------------+----------------------------------+-----------------------+--------------------------------------------+
+| match | Unary | Null | Int32 (3) | :struct:`SetLookupOptions` |
++--------------------------+------------+----------------------------------+-----------------------+--------------------------------------------+
+| match | Unary | Boolean,Numeric, Temporal | Int32 (3) | :struct:`SetLookupOptions` |
++--------------------------+------------+----------------------------------+-----------------------+--------------------------------------------+
+
+* \(1) Output is true iff :member:`BinaryContainsExactOptions::pattern`
+ is a substring of the corresponding input element.
+
+* \(2) Output is true iff the corresponding input element is equal to one
+ of the elements in :member:`SetLookupOptions::value_set`.
+
+* \(3) Output is the index of the corresponding input element in
+ :member:`SetLookupOptions::value_set`, if found there. Otherwise,
+ output is null.
+
+Structural transforms
+~~~~~~~~~~~~~~~~~~~~~
+
++--------------------------+------------+--------------------+---------------------+---------+
+| Function name | Arity | Input types | Output type | Notes |
++==========================+============+====================+=====================+=========+
+| is_null | Unary | Any | Boolean | \(1) |
++--------------------------+------------+--------------------+---------------------+---------+
+| is_valid | Unary | Any | Boolean | \(2) |
++--------------------------+------------+--------------------+---------------------+---------+
+| list_value_lengths | Unary | List-like | Int32 or Int64 | \(3) |
++--------------------------+------------+--------------------+---------------------+---------+
+
+* \(1) Output is true iff the corresponding input element is non-null.
+
+* \(2) Output is true iff the corresponding input element is null.
+
+* \(3) Each output element is the length of the corresponding input element
+ (null if input is null).
+
+Conversions
+~~~~~~~~~~~
+
+A general conversion function named ``cast`` is provided which accepts a large
+number of input and output types. The type to cast to can be passed in a
+:struct:`CastOptions` instance.
+
++--------------------------+------------+--------------------+-----------------------+--------------------------------------------+
+| Function name | Arity | Input types | Output type | Options class |
++==========================+============+====================+=======================+============================================+
+| cast | Unary | Many | Variable | :struct:`CastOptions` |
++--------------------------+------------+--------------------+-----------------------+--------------------------------------------+
+| strptime | Unary | String-like | Timestamp | :struct:`StrptimeOptions` |
Review comment:
What do StrptimeOptions look like?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] pitrou commented on a change in pull request #7695: ARROW-8989: [C++][Doc] Document available compute functions
Posted by GitBox <gi...@apache.org>.
pitrou commented on a change in pull request #7695:
URL: https://github.com/apache/arrow/pull/7695#discussion_r453521794
##########
File path: docs/source/cpp/compute.rst
##########
@@ -0,0 +1,419 @@
+.. Licensed to the Apache Software Foundation (ASF) under one
+.. or more contributor license agreements. See the NOTICE file
+.. distributed with this work for additional information
+.. regarding copyright ownership. The ASF licenses this file
+.. to you under the Apache License, Version 2.0 (the
+.. "License"); you may not use this file except in compliance
+.. with the License. You may obtain a copy of the License at
+
+.. http://www.apache.org/licenses/LICENSE-2.0
+
+.. Unless required by applicable law or agreed to in writing,
+.. software distributed under the License is distributed on an
+.. "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+.. KIND, either express or implied. See the License for the
+.. specific language governing permissions and limitations
+.. under the License.
+
+.. default-domain:: cpp
+.. highlight:: cpp
+.. cpp:namespace:: arrow::compute
+
+=================
+Compute Functions
+=================
+
+.. TODO: describe API and how to invoke compute functions
+
+Available functions
+===================
+
+Aggregations
+------------
+
++--------------------------+------------+--------------------+-----------------------+--------------------------------------------+
+| Function name | Arity | Input types | Output type | Options class |
++==========================+============+====================+=======================+============================================+
+| count | Unary | Any | Scalar Int64 | :struct:`CountOptions` |
++--------------------------+------------+--------------------+-----------------------+--------------------------------------------+
+| mean | Unary | Numeric | Scalar Float64 | |
++--------------------------+------------+--------------------+-----------------------+--------------------------------------------+
+| minmax | Unary | Numeric | Scalar Struct (1) | :struct:`MinMaxOptions` |
++--------------------------+------------+--------------------+-----------------------+--------------------------------------------+
+| sum | Unary | Numeric | Scalar Numeric (2) | |
++--------------------------+------------+--------------------+-----------------------+--------------------------------------------+
+
+Notes:
+
+* \(1) Output is a ``{"min": input type, "max": input type}`` Struct
+
+* \(2) Output is Int64, UInt64 or Float64, depending on the input type
+
+
+Element-wise ("scalar") functions
+---------------------------------
+
+Arithmetic functions
+~~~~~~~~~~~~~~~~~~~~
+
+Those functions expect two inputs of the same type and apply a given binary
+operation to each pair of elements gathered from the inputs. Each function
+is also available in an overflow-checking variant, suffixed ``_checked``.
+
+If any of the input elements in a pair is null, the corresponding output
+element is null.
+
++--------------------------+------------+--------------------+---------------------+
+| Function name | Arity | Input types | Output type |
++==========================+============+====================+=====================+
+| add | Binary | Numeric | Numeric |
++--------------------------+------------+--------------------+---------------------+
+| add_checked | Binary | Numeric | Numeric |
++--------------------------+------------+--------------------+---------------------+
+| multiply | Binary | Numeric | Numeric |
++--------------------------+------------+--------------------+---------------------+
+| multiply_checked | Binary | Numeric | Numeric |
++--------------------------+------------+--------------------+---------------------+
+| subtract | Binary | Numeric | Numeric |
++--------------------------+------------+--------------------+---------------------+
+| subtract_checked | Binary | Numeric | Numeric |
++--------------------------+------------+--------------------+---------------------+
+
+Comparisons
+~~~~~~~~~~~
+
+Those functions expect two inputs of the same type and apply a given
+comparison operator. If any of the input elements in a pair is null,
+the corresponding output element is null.
+
++--------------------------+------------+---------------------------------+---------------------+
+| Function names | Arity | Input types | Output type |
++==========================+============+=================================+=====================+
+| equal, not_equal | Binary | Numeric | Boolean |
++--------------------------+------------+---------------------------------+---------------------+
+| equal, not_equal | Binary | Binary- and String-like | Boolean |
++--------------------------+------------+---------------------------------+---------------------+
+| equal, not_equal | Binary | Temporal | Boolean |
++--------------------------+------------+---------------------------------+---------------------+
+| greater, greater_equal, | Binary | Numeric | Boolean |
+| less, less_equal | | | |
++--------------------------+------------+---------------------------------+---------------------+
+| greater, greater_equal, | Binary | Binary- and String-like | Boolean |
+| less, less_equal | | | |
++--------------------------+------------+---------------------------------+---------------------+
+| greater, greater_equal, | Binary | Temporal | Boolean |
Review comment:
Will do, though unfortunately the real answer may be slightly function-specific.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] wesm commented on a change in pull request #7695: ARROW-8989: [C++][Doc] Document available compute functions
Posted by GitBox <gi...@apache.org>.
wesm commented on a change in pull request #7695:
URL: https://github.com/apache/arrow/pull/7695#discussion_r453364998
##########
File path: docs/source/cpp/compute.rst
##########
@@ -0,0 +1,419 @@
+.. Licensed to the Apache Software Foundation (ASF) under one
+.. or more contributor license agreements. See the NOTICE file
+.. distributed with this work for additional information
+.. regarding copyright ownership. The ASF licenses this file
+.. to you under the Apache License, Version 2.0 (the
+.. "License"); you may not use this file except in compliance
+.. with the License. You may obtain a copy of the License at
+
+.. http://www.apache.org/licenses/LICENSE-2.0
+
+.. Unless required by applicable law or agreed to in writing,
+.. software distributed under the License is distributed on an
+.. "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+.. KIND, either express or implied. See the License for the
+.. specific language governing permissions and limitations
+.. under the License.
+
+.. default-domain:: cpp
+.. highlight:: cpp
+.. cpp:namespace:: arrow::compute
+
+=================
+Compute Functions
+=================
+
+.. TODO: describe API and how to invoke compute functions
+
+Available functions
+===================
+
+Aggregations
+------------
+
++--------------------------+------------+--------------------+-----------------------+--------------------------------------------+
+| Function name | Arity | Input types | Output type | Options class |
++==========================+============+====================+=======================+============================================+
+| count | Unary | Any | Scalar Int64 | :struct:`CountOptions` |
++--------------------------+------------+--------------------+-----------------------+--------------------------------------------+
+| mean | Unary | Numeric | Scalar Float64 | |
++--------------------------+------------+--------------------+-----------------------+--------------------------------------------+
+| minmax | Unary | Numeric | Scalar Struct (1) | :struct:`MinMaxOptions` |
++--------------------------+------------+--------------------+-----------------------+--------------------------------------------+
+| sum | Unary | Numeric | Scalar Numeric (2) | |
++--------------------------+------------+--------------------+-----------------------+--------------------------------------------+
+
+Notes:
+
+* \(1) Output is a ``{"min": input type, "max": input type}`` Struct
+
+* \(2) Output is Int64, UInt64 or Float64, depending on the input type
+
+
+Element-wise ("scalar") functions
+---------------------------------
+
+Arithmetic functions
+~~~~~~~~~~~~~~~~~~~~
+
+Those functions expect two inputs of the same type and apply a given binary
+operation to each pair of elements gathered from the inputs. Each function
+is also available in an overflow-checking variant, suffixed ``_checked``.
+
+If any of the input elements in a pair is null, the corresponding output
+element is null.
+
++--------------------------+------------+--------------------+---------------------+
+| Function name | Arity | Input types | Output type |
++==========================+============+====================+=====================+
+| add | Binary | Numeric | Numeric |
++--------------------------+------------+--------------------+---------------------+
+| add_checked | Binary | Numeric | Numeric |
++--------------------------+------------+--------------------+---------------------+
+| multiply | Binary | Numeric | Numeric |
++--------------------------+------------+--------------------+---------------------+
+| multiply_checked | Binary | Numeric | Numeric |
++--------------------------+------------+--------------------+---------------------+
+| subtract | Binary | Numeric | Numeric |
++--------------------------+------------+--------------------+---------------------+
+| subtract_checked | Binary | Numeric | Numeric |
++--------------------------+------------+--------------------+---------------------+
+
+Comparisons
+~~~~~~~~~~~
+
+Those functions expect two inputs of the same type and apply a given
+comparison operator. If any of the input elements in a pair is null,
+the corresponding output element is null.
+
++--------------------------+------------+---------------------------------+---------------------+
+| Function names | Arity | Input types | Output type |
++==========================+============+=================================+=====================+
+| equal, not_equal | Binary | Numeric | Boolean |
++--------------------------+------------+---------------------------------+---------------------+
+| equal, not_equal | Binary | Binary- and String-like | Boolean |
++--------------------------+------------+---------------------------------+---------------------+
+| equal, not_equal | Binary | Temporal | Boolean |
++--------------------------+------------+---------------------------------+---------------------+
+| greater, greater_equal, | Binary | Numeric | Boolean |
+| less, less_equal | | | |
++--------------------------+------------+---------------------------------+---------------------+
+| greater, greater_equal, | Binary | Binary- and String-like | Boolean |
+| less, less_equal | | | |
++--------------------------+------------+---------------------------------+---------------------+
+| greater, greater_equal, | Binary | Temporal | Boolean |
+| less, less_equal | | | |
++--------------------------+------------+---------------------------------+---------------------+
+
+Logical functions
+~~~~~~~~~~~~~~~~~~
+
+The normal behaviour for these functions is to emit a null if any of the
+inputs is null.
+
+Some of them are also available in a "`Kleene logic`_" variant (suffixed
+``_kleene``) where null is taken to mean "undefined". For those variants
+therefore:
+
+* "true AND null", "null AND true" give "null" (the result is undefined)
+* "true OR null", "null OR true" give "true"
+* "false AND null", "null AND false" give "false"
+* "false OR null", "null OR false" give "null" (the result is undefined)
+
++--------------------------+------------+--------------------+---------------------+
+| Function name | Arity | Input types | Output type |
++==========================+============+====================+=====================+
+| and | Binary | Boolean | Boolean |
++--------------------------+------------+--------------------+---------------------+
+| and_kleene | Binary | Boolean | Boolean |
++--------------------------+------------+--------------------+---------------------+
+| invert | Unary | Boolean | Boolean |
++--------------------------+------------+--------------------+---------------------+
+| or | Binary | Boolean | Boolean |
++--------------------------+------------+--------------------+---------------------+
+| or_kleene | Binary | Boolean | Boolean |
++--------------------------+------------+--------------------+---------------------+
+| xor | Binary | Boolean | Boolean |
++--------------------------+------------+--------------------+---------------------+
+
+.. _Kleene logic: https://en.wikipedia.org/wiki/Three-valued_logic#Kleene_and_Priest_logics
+
+String functions
+~~~~~~~~~~~~~~~~
+
++--------------------------+------------+--------------------+---------------------+---------+
+| Function name | Arity | Input types | Output type | Notes |
++==========================+============+====================+=====================+=========+
+| ascii_length | Unary | String-like | Int32 or Int64 | \(1) |
++--------------------------+------------+--------------------+---------------------+---------+
+| ascii_lower | Unary | String-like | String-like | \(2) |
++--------------------------+------------+--------------------+---------------------+---------+
+| ascii_upper | Unary | String-like | String-like | \(2) |
++--------------------------+------------+--------------------+---------------------+---------+
+| utf8_lower | Unary | String-like | String-like | \(3) |
++--------------------------+------------+--------------------+---------------------+---------+
+| utf8_upper | Unary | String-like | String-like | \(3) |
++--------------------------+------------+--------------------+---------------------+---------+
+
+* \(1) Output is the physical length in bytes of each input element.
+
+* \(2) Each ASCII character in the input is converted to lowercase or
+ uppercase. Non-ASCII characters are left untouched.
+
+* \(3) Each UTF8-encoded character in the input is converted to lowercase or
+ uppercase.
+
+Containment tests
+~~~~~~~~~~~~~~~~~
+
++--------------------------+------------+----------------------------------+-----------------------+--------------------------------------------+
+| Function name | Arity | Input types | Output type | Options class |
++==========================+============+==================================+=======================+============================================+
+| binary_contains_exact | Unary | String-like | Boolean (1) | :struct:`BinaryContainsExactOptions` |
++--------------------------+------------+----------------------------------+-----------------------+--------------------------------------------+
+| isin | Unary | Binary- and String-like | Boolean (2) | :struct:`SetLookupOptions` |
++--------------------------+------------+----------------------------------+-----------------------+--------------------------------------------+
+| isin | Unary | Null | Boolean (2) | :struct:`SetLookupOptions` |
++--------------------------+------------+----------------------------------+-----------------------+--------------------------------------------+
+| isin | Unary | Boolean,Numeric, Temporal | Boolean (2) | :struct:`SetLookupOptions` |
++--------------------------+------------+----------------------------------+-----------------------+--------------------------------------------+
+| match | Unary | Binary- and String-like | Int32 (3) | :struct:`SetLookupOptions` |
++--------------------------+------------+----------------------------------+-----------------------+--------------------------------------------+
+| match | Unary | Null | Int32 (3) | :struct:`SetLookupOptions` |
++--------------------------+------------+----------------------------------+-----------------------+--------------------------------------------+
+| match | Unary | Boolean,Numeric, Temporal | Int32 (3) | :struct:`SetLookupOptions` |
Review comment:
Same comment here, having multiple lines per function seems really tedious
##########
File path: docs/source/cpp/compute.rst
##########
@@ -0,0 +1,419 @@
+.. Licensed to the Apache Software Foundation (ASF) under one
+.. or more contributor license agreements. See the NOTICE file
+.. distributed with this work for additional information
+.. regarding copyright ownership. The ASF licenses this file
+.. to you under the Apache License, Version 2.0 (the
+.. "License"); you may not use this file except in compliance
+.. with the License. You may obtain a copy of the License at
+
+.. http://www.apache.org/licenses/LICENSE-2.0
+
+.. Unless required by applicable law or agreed to in writing,
+.. software distributed under the License is distributed on an
+.. "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+.. KIND, either express or implied. See the License for the
+.. specific language governing permissions and limitations
+.. under the License.
+
+.. default-domain:: cpp
+.. highlight:: cpp
+.. cpp:namespace:: arrow::compute
+
+=================
+Compute Functions
+=================
+
+.. TODO: describe API and how to invoke compute functions
Review comment:
Note: it isn't necessary to use `Datum(...)` explicitly on most cases because of implicit conversions
##########
File path: docs/source/cpp/compute.rst
##########
@@ -0,0 +1,419 @@
+.. Licensed to the Apache Software Foundation (ASF) under one
+.. or more contributor license agreements. See the NOTICE file
+.. distributed with this work for additional information
+.. regarding copyright ownership. The ASF licenses this file
+.. to you under the Apache License, Version 2.0 (the
+.. "License"); you may not use this file except in compliance
+.. with the License. You may obtain a copy of the License at
+
+.. http://www.apache.org/licenses/LICENSE-2.0
+
+.. Unless required by applicable law or agreed to in writing,
+.. software distributed under the License is distributed on an
+.. "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+.. KIND, either express or implied. See the License for the
+.. specific language governing permissions and limitations
+.. under the License.
+
+.. default-domain:: cpp
+.. highlight:: cpp
+.. cpp:namespace:: arrow::compute
+
+=================
+Compute Functions
+=================
+
+.. TODO: describe API and how to invoke compute functions
+
+Available functions
+===================
+
+Aggregations
+------------
+
++--------------------------+------------+--------------------+-----------------------+--------------------------------------------+
+| Function name | Arity | Input types | Output type | Options class |
++==========================+============+====================+=======================+============================================+
+| count | Unary | Any | Scalar Int64 | :struct:`CountOptions` |
++--------------------------+------------+--------------------+-----------------------+--------------------------------------------+
+| mean | Unary | Numeric | Scalar Float64 | |
++--------------------------+------------+--------------------+-----------------------+--------------------------------------------+
+| minmax | Unary | Numeric | Scalar Struct (1) | :struct:`MinMaxOptions` |
++--------------------------+------------+--------------------+-----------------------+--------------------------------------------+
+| sum | Unary | Numeric | Scalar Numeric (2) | |
++--------------------------+------------+--------------------+-----------------------+--------------------------------------------+
+
+Notes:
+
+* \(1) Output is a ``{"min": input type, "max": input type}`` Struct
+
+* \(2) Output is Int64, UInt64 or Float64, depending on the input type
+
+
+Element-wise ("scalar") functions
+---------------------------------
+
+Arithmetic functions
+~~~~~~~~~~~~~~~~~~~~
+
+Those functions expect two inputs of the same type and apply a given binary
+operation to each pair of elements gathered from the inputs. Each function
+is also available in an overflow-checking variant, suffixed ``_checked``.
+
+If any of the input elements in a pair is null, the corresponding output
+element is null.
+
++--------------------------+------------+--------------------+---------------------+
+| Function name | Arity | Input types | Output type |
++==========================+============+====================+=====================+
+| add | Binary | Numeric | Numeric |
++--------------------------+------------+--------------------+---------------------+
+| add_checked | Binary | Numeric | Numeric |
++--------------------------+------------+--------------------+---------------------+
+| multiply | Binary | Numeric | Numeric |
++--------------------------+------------+--------------------+---------------------+
+| multiply_checked | Binary | Numeric | Numeric |
++--------------------------+------------+--------------------+---------------------+
+| subtract | Binary | Numeric | Numeric |
++--------------------------+------------+--------------------+---------------------+
+| subtract_checked | Binary | Numeric | Numeric |
++--------------------------+------------+--------------------+---------------------+
+
+Comparisons
+~~~~~~~~~~~
+
+Those functions expect two inputs of the same type and apply a given
+comparison operator. If any of the input elements in a pair is null,
+the corresponding output element is null.
+
++--------------------------+------------+---------------------------------+---------------------+
+| Function names | Arity | Input types | Output type |
++==========================+============+=================================+=====================+
+| equal, not_equal | Binary | Numeric | Boolean |
++--------------------------+------------+---------------------------------+---------------------+
+| equal, not_equal | Binary | Binary- and String-like | Boolean |
++--------------------------+------------+---------------------------------+---------------------+
+| equal, not_equal | Binary | Temporal | Boolean |
++--------------------------+------------+---------------------------------+---------------------+
+| greater, greater_equal, | Binary | Numeric | Boolean |
+| less, less_equal | | | |
++--------------------------+------------+---------------------------------+---------------------+
+| greater, greater_equal, | Binary | Binary- and String-like | Boolean |
+| less, less_equal | | | |
++--------------------------+------------+---------------------------------+---------------------+
+| greater, greater_equal, | Binary | Temporal | Boolean |
+| less, less_equal | | | |
++--------------------------+------------+---------------------------------+---------------------+
+
+Logical functions
+~~~~~~~~~~~~~~~~~~
+
+The normal behaviour for these functions is to emit a null if any of the
+inputs is null.
+
+Some of them are also available in a "`Kleene logic`_" variant (suffixed
+``_kleene``) where null is taken to mean "undefined". For those variants
+therefore:
+
+* "true AND null", "null AND true" give "null" (the result is undefined)
+* "true OR null", "null OR true" give "true"
+* "false AND null", "null AND false" give "false"
+* "false OR null", "null OR false" give "null" (the result is undefined)
+
++--------------------------+------------+--------------------+---------------------+
+| Function name | Arity | Input types | Output type |
++==========================+============+====================+=====================+
+| and | Binary | Boolean | Boolean |
++--------------------------+------------+--------------------+---------------------+
+| and_kleene | Binary | Boolean | Boolean |
++--------------------------+------------+--------------------+---------------------+
+| invert | Unary | Boolean | Boolean |
++--------------------------+------------+--------------------+---------------------+
+| or | Binary | Boolean | Boolean |
++--------------------------+------------+--------------------+---------------------+
+| or_kleene | Binary | Boolean | Boolean |
++--------------------------+------------+--------------------+---------------------+
+| xor | Binary | Boolean | Boolean |
++--------------------------+------------+--------------------+---------------------+
+
+.. _Kleene logic: https://en.wikipedia.org/wiki/Three-valued_logic#Kleene_and_Priest_logics
+
+String functions
+~~~~~~~~~~~~~~~~
+
++--------------------------+------------+--------------------+---------------------+---------+
+| Function name | Arity | Input types | Output type | Notes |
++==========================+============+====================+=====================+=========+
+| ascii_length | Unary | String-like | Int32 or Int64 | \(1) |
++--------------------------+------------+--------------------+---------------------+---------+
+| ascii_lower | Unary | String-like | String-like | \(2) |
++--------------------------+------------+--------------------+---------------------+---------+
+| ascii_upper | Unary | String-like | String-like | \(2) |
++--------------------------+------------+--------------------+---------------------+---------+
+| utf8_lower | Unary | String-like | String-like | \(3) |
++--------------------------+------------+--------------------+---------------------+---------+
+| utf8_upper | Unary | String-like | String-like | \(3) |
++--------------------------+------------+--------------------+---------------------+---------+
+
+* \(1) Output is the physical length in bytes of each input element.
+
+* \(2) Each ASCII character in the input is converted to lowercase or
+ uppercase. Non-ASCII characters are left untouched.
+
+* \(3) Each UTF8-encoded character in the input is converted to lowercase or
+ uppercase.
+
+Containment tests
+~~~~~~~~~~~~~~~~~
+
++--------------------------+------------+----------------------------------+-----------------------+--------------------------------------------+
+| Function name | Arity | Input types | Output type | Options class |
++==========================+============+==================================+=======================+============================================+
+| binary_contains_exact | Unary | String-like | Boolean (1) | :struct:`BinaryContainsExactOptions` |
++--------------------------+------------+----------------------------------+-----------------------+--------------------------------------------+
+| isin | Unary | Binary- and String-like | Boolean (2) | :struct:`SetLookupOptions` |
++--------------------------+------------+----------------------------------+-----------------------+--------------------------------------------+
+| isin | Unary | Null | Boolean (2) | :struct:`SetLookupOptions` |
++--------------------------+------------+----------------------------------+-----------------------+--------------------------------------------+
+| isin | Unary | Boolean,Numeric, Temporal | Boolean (2) | :struct:`SetLookupOptions` |
++--------------------------+------------+----------------------------------+-----------------------+--------------------------------------------+
+| match | Unary | Binary- and String-like | Int32 (3) | :struct:`SetLookupOptions` |
++--------------------------+------------+----------------------------------+-----------------------+--------------------------------------------+
+| match | Unary | Null | Int32 (3) | :struct:`SetLookupOptions` |
++--------------------------+------------+----------------------------------+-----------------------+--------------------------------------------+
+| match | Unary | Boolean,Numeric, Temporal | Int32 (3) | :struct:`SetLookupOptions` |
++--------------------------+------------+----------------------------------+-----------------------+--------------------------------------------+
+
+* \(1) Output is true iff :member:`BinaryContainsExactOptions::pattern`
+ is a substring of the corresponding input element.
+
+* \(2) Output is true iff the corresponding input element is equal to one
+ of the elements in :member:`SetLookupOptions::value_set`.
+
+* \(3) Output is the index of the corresponding input element in
+ :member:`SetLookupOptions::value_set`, if found there. Otherwise,
+ output is null.
+
+Structural transforms
+~~~~~~~~~~~~~~~~~~~~~
+
++--------------------------+------------+--------------------+---------------------+---------+
+| Function name | Arity | Input types | Output type | Notes |
++==========================+============+====================+=====================+=========+
+| is_null | Unary | Any | Boolean | \(1) |
++--------------------------+------------+--------------------+---------------------+---------+
+| is_valid | Unary | Any | Boolean | \(2) |
++--------------------------+------------+--------------------+---------------------+---------+
+| list_value_lengths | Unary | List-like | Int32 or Int64 | \(3) |
++--------------------------+------------+--------------------+---------------------+---------+
+
+* \(1) Output is true iff the corresponding input element is non-null.
+
+* \(2) Output is true iff the corresponding input element is null.
+
+* \(3) Each output element is the length of the corresponding input element
+ (null if input is null).
+
+Conversions
+~~~~~~~~~~~
+
+A general conversion function named ``cast`` is provided which accepts a large
+number of input and output types. The type to cast to can be passed in a
+:struct:`CastOptions` instance.
+
++--------------------------+------------+--------------------+-----------------------+--------------------------------------------+
+| Function name | Arity | Input types | Output type | Options class |
++==========================+============+====================+=======================+============================================+
+| cast | Unary | Many | Variable | :struct:`CastOptions` |
++--------------------------+------------+--------------------+-----------------------+--------------------------------------------+
+| strptime | Unary | String-like | Timestamp | :struct:`StrptimeOptions` |
++--------------------------+------------+--------------------+-----------------------+--------------------------------------------+
+
+The conversions available with ``cast`` are listed below. In all cases, a
+null input value is converted into a null output value.
+
+**Truth value extraction**
+
++-----------------------------+------------------------------------+--------------+
+| Input type | Output type | Notes |
++=============================+====================================+==============+
+| Binary- and String-like | Boolean | \(1) |
++-----------------------------+------------------------------------+--------------+
+| Numeric | Boolean | \(2) |
++-----------------------------+------------------------------------+--------------+
+
+* \(1) Output is true iff the corresponding input value has non-zero length.
+
+* \(2) Output is true iff the corresponding input value is non-zero.
+
+**Same-kind conversion**
+
++-----------------------------+------------------------------------+--------------+
+| Input type | Output type | Notes |
++=============================+====================================+==============+
+| Int32 | 32-bit Temporal | \(1) |
++-----------------------------+------------------------------------+--------------+
+| Int64 | 64-bit Temporal | \(1) |
++-----------------------------+------------------------------------+--------------+
+| (Large)Binary | (Large)String | \(2) |
++-----------------------------+------------------------------------+--------------+
+| (Large)String | (Large)Binary | \(3) |
++-----------------------------+------------------------------------+--------------+
+| Numeric | Numeric | \(4) \(5) |
++-----------------------------+------------------------------------+--------------+
+| 32-bit Temporal | Int32 | \(1) |
++-----------------------------+------------------------------------+--------------+
+| 64-bit Temporal | Int64 | \(1) |
++-----------------------------+------------------------------------+--------------+
+| Temporal | Temporal | \(4) \(5) |
++-----------------------------+------------------------------------+--------------+
+
+* \(1) No-operation cast: the raw values are kept identical, only
+ the type is changed.
+
+* \(2) Validates the contents if :member:`CastOptions::allow_invalid_utf8`
+ is false.
+
+* \(3) No-operation cast: only the type is changed.
+
+* \(4) Overflow and truncation checks are enabled depending on
+ the given :struct:`CastOptions`.
+
+* \(5) Not all such casts have been implemented.
+
+**String representations**
+
++-----------------------------+------------------------------------+---------+
+| Input type | Output type | Notes |
++=============================+====================================+=========+
+| Boolean | String-like | |
++-----------------------------+------------------------------------+---------+
+| Numeric | String-like | |
++-----------------------------+------------------------------------+---------+
+
+**Generic conversions**
+
++-----------------------------+------------------------------------+---------+
+| Input type | Output type | Notes |
++=============================+====================================+=========+
+| Dictionary | Dictionary value type | |
++-----------------------------+------------------------------------+---------+
+| Extension | Extension storage type | |
++-----------------------------+------------------------------------+---------+
+| List-like | List-like | \(1) |
++-----------------------------+------------------------------------+---------+
+| Null | Any | |
++-----------------------------+------------------------------------+---------+
+
+* \(1) The list offsets are unchanged, the list values are cast from the
+ input value type to the output value type (if a conversion is
+ available).
+
+
+.. TODO: add C++ cast example
+
+Array-wise ("vector") functions
+-------------------------------
+
+Associative transforms
+~~~~~~~~~~~~~~~~~~~~~~
+
++--------------------------+------------+----------------------------+----------------------------+
+| Function name | Arity | Input types | Output type |
++==========================+============+============================+============================+
+| dictionary_encode | Unary | Binary- and String-like | Dictionary (1) |
++--------------------------+------------+----------------------------+----------------------------+
+| dictionary_encode | Unary | Boolean, Numeric, Temporal | Dictionary (1) |
++--------------------------+------------+----------------------------+----------------------------+
+| dictionary_encode | Unary | Null | Dictionary (1) |
++--------------------------+------------+----------------------------+----------------------------+
+| unique | Unary | Binary- and String-like | Input type (2) |
++--------------------------+------------+----------------------------+----------------------------+
+| unique | Unary | Boolean, Numeric, Temporal | Input type (2) |
++--------------------------+------------+----------------------------+----------------------------+
+| unique | Unary | Null | Input type (2) |
++--------------------------+------------+----------------------------+----------------------------+
+| value_counts | Unary | Binary- and String-like | Struct (3) |
++--------------------------+------------+----------------------------+----------------------------+
+| value_counts | Unary | Boolean, Numeric, Temporal | Struct (3) |
++--------------------------+------------+----------------------------+----------------------------+
+| value_counts | Unary | Null | Struct (3) |
++--------------------------+------------+----------------------------+----------------------------+
+
+* \(1) Output is ``Dictionary(Int32, input type)``.
+
+* \(2) Duplicates are removed from the output while the original order is
+ maintained.
+
+* \(3) Output is a ``{"values": input type, "counts": Int64}`` Struct.
+ Each output element corresponds to a unique value in the input, along
+ with the number of times this value has appeared.
+
+Selections
+~~~~~~~~~~
+
+These functions select a subset of the first input defined by the second input.
+
++-----------------+------------+---------------+--------------+------------------+-------------------------+-------------+
+| Function name | Arity | Input type 1 | Input type 2 | Output type | Options class | Notes |
++=================+============+===============+==============+==================+=========================+=============+
+| filter | Binary | Any (1) | Boolean | Input type 1 | :struct:`FilterOptions` | \(2) |
++-----------------+------------+---------------+--------------+------------------+-------------------------+-------------+
+| take | Binary | Any (1) | Integer | Input type 1 | :struct:`TakeOptions` | \(3) |
Review comment:
These are metafunctions. Probably also want to list array_filter and array_take
##########
File path: docs/source/cpp/compute.rst
##########
@@ -0,0 +1,419 @@
+.. Licensed to the Apache Software Foundation (ASF) under one
+.. or more contributor license agreements. See the NOTICE file
+.. distributed with this work for additional information
+.. regarding copyright ownership. The ASF licenses this file
+.. to you under the Apache License, Version 2.0 (the
+.. "License"); you may not use this file except in compliance
+.. with the License. You may obtain a copy of the License at
+
+.. http://www.apache.org/licenses/LICENSE-2.0
+
+.. Unless required by applicable law or agreed to in writing,
+.. software distributed under the License is distributed on an
+.. "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+.. KIND, either express or implied. See the License for the
+.. specific language governing permissions and limitations
+.. under the License.
+
+.. default-domain:: cpp
+.. highlight:: cpp
+.. cpp:namespace:: arrow::compute
+
+=================
+Compute Functions
+=================
+
+.. TODO: describe API and how to invoke compute functions
+
+Available functions
+===================
+
+Aggregations
+------------
+
++--------------------------+------------+--------------------+-----------------------+--------------------------------------------+
Review comment:
I've always found these reStructuredText tables to be immensely tedious. What would you say about putting the source of the documentation in e.g. a JSON file (which would be much easier to edit) and then generating the RST markup from the JSON? Then if we need to restructure the output in some way we won't have to tear our hair out manually editing these tables
##########
File path: docs/source/cpp/compute.rst
##########
@@ -0,0 +1,419 @@
+.. Licensed to the Apache Software Foundation (ASF) under one
+.. or more contributor license agreements. See the NOTICE file
+.. distributed with this work for additional information
+.. regarding copyright ownership. The ASF licenses this file
+.. to you under the Apache License, Version 2.0 (the
+.. "License"); you may not use this file except in compliance
+.. with the License. You may obtain a copy of the License at
+
+.. http://www.apache.org/licenses/LICENSE-2.0
+
+.. Unless required by applicable law or agreed to in writing,
+.. software distributed under the License is distributed on an
+.. "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+.. KIND, either express or implied. See the License for the
+.. specific language governing permissions and limitations
+.. under the License.
+
+.. default-domain:: cpp
+.. highlight:: cpp
+.. cpp:namespace:: arrow::compute
+
+=================
+Compute Functions
+=================
+
+.. TODO: describe API and how to invoke compute functions
+
+Available functions
+===================
+
+Aggregations
+------------
+
++--------------------------+------------+--------------------+-----------------------+--------------------------------------------+
+| Function name | Arity | Input types | Output type | Options class |
++==========================+============+====================+=======================+============================================+
+| count | Unary | Any | Scalar Int64 | :struct:`CountOptions` |
++--------------------------+------------+--------------------+-----------------------+--------------------------------------------+
+| mean | Unary | Numeric | Scalar Float64 | |
++--------------------------+------------+--------------------+-----------------------+--------------------------------------------+
+| minmax | Unary | Numeric | Scalar Struct (1) | :struct:`MinMaxOptions` |
++--------------------------+------------+--------------------+-----------------------+--------------------------------------------+
+| sum | Unary | Numeric | Scalar Numeric (2) | |
++--------------------------+------------+--------------------+-----------------------+--------------------------------------------+
+
+Notes:
+
+* \(1) Output is a ``{"min": input type, "max": input type}`` Struct
+
+* \(2) Output is Int64, UInt64 or Float64, depending on the input type
+
+
+Element-wise ("scalar") functions
+---------------------------------
+
+Arithmetic functions
+~~~~~~~~~~~~~~~~~~~~
+
+Those functions expect two inputs of the same type and apply a given binary
+operation to each pair of elements gathered from the inputs. Each function
+is also available in an overflow-checking variant, suffixed ``_checked``.
+
+If any of the input elements in a pair is null, the corresponding output
+element is null.
+
++--------------------------+------------+--------------------+---------------------+
+| Function name | Arity | Input types | Output type |
++==========================+============+====================+=====================+
+| add | Binary | Numeric | Numeric |
++--------------------------+------------+--------------------+---------------------+
+| add_checked | Binary | Numeric | Numeric |
++--------------------------+------------+--------------------+---------------------+
+| multiply | Binary | Numeric | Numeric |
++--------------------------+------------+--------------------+---------------------+
+| multiply_checked | Binary | Numeric | Numeric |
++--------------------------+------------+--------------------+---------------------+
+| subtract | Binary | Numeric | Numeric |
++--------------------------+------------+--------------------+---------------------+
+| subtract_checked | Binary | Numeric | Numeric |
++--------------------------+------------+--------------------+---------------------+
+
+Comparisons
+~~~~~~~~~~~
+
+Those functions expect two inputs of the same type and apply a given
+comparison operator. If any of the input elements in a pair is null,
+the corresponding output element is null.
+
++--------------------------+------------+---------------------------------+---------------------+
+| Function names | Arity | Input types | Output type |
++==========================+============+=================================+=====================+
+| equal, not_equal | Binary | Numeric | Boolean |
++--------------------------+------------+---------------------------------+---------------------+
+| equal, not_equal | Binary | Binary- and String-like | Boolean |
++--------------------------+------------+---------------------------------+---------------------+
+| equal, not_equal | Binary | Temporal | Boolean |
Review comment:
What is the rationale for having multiple lines for each function?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] pitrou commented on a change in pull request #7695: ARROW-8989: [C++][Doc] Document available compute functions
Posted by GitBox <gi...@apache.org>.
pitrou commented on a change in pull request #7695:
URL: https://github.com/apache/arrow/pull/7695#discussion_r453001863
##########
File path: docs/source/cpp/compute.rst
##########
@@ -0,0 +1,419 @@
+.. Licensed to the Apache Software Foundation (ASF) under one
+.. or more contributor license agreements. See the NOTICE file
+.. distributed with this work for additional information
+.. regarding copyright ownership. The ASF licenses this file
+.. to you under the Apache License, Version 2.0 (the
+.. "License"); you may not use this file except in compliance
+.. with the License. You may obtain a copy of the License at
+
+.. http://www.apache.org/licenses/LICENSE-2.0
+
+.. Unless required by applicable law or agreed to in writing,
+.. software distributed under the License is distributed on an
+.. "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+.. KIND, either express or implied. See the License for the
+.. specific language governing permissions and limitations
+.. under the License.
+
+.. default-domain:: cpp
+.. highlight:: cpp
+.. cpp:namespace:: arrow::compute
+
+=================
+Compute Functions
+=================
+
+.. TODO: describe API and how to invoke compute functions
+
+Available functions
+===================
+
+Aggregations
+------------
+
++--------------------------+------------+--------------------+-----------------------+--------------------------------------------+
+| Function name | Arity | Input types | Output type | Options class |
++==========================+============+====================+=======================+============================================+
+| count | Unary | Any | Scalar Int64 | :struct:`CountOptions` |
++--------------------------+------------+--------------------+-----------------------+--------------------------------------------+
+| mean | Unary | Numeric | Scalar Float64 | |
++--------------------------+------------+--------------------+-----------------------+--------------------------------------------+
+| minmax | Unary | Numeric | Scalar Struct (1) | :struct:`MinMaxOptions` |
++--------------------------+------------+--------------------+-----------------------+--------------------------------------------+
+| sum | Unary | Numeric | Scalar Numeric (2) | |
++--------------------------+------------+--------------------+-----------------------+--------------------------------------------+
+
+Notes:
+
+* \(1) Output is a ``{"min": input type, "max": input type}`` Struct
+
+* \(2) Output is Int64, UInt64 or Float64, depending on the input type
+
+
+Element-wise ("scalar") functions
+---------------------------------
+
+Arithmetic functions
+~~~~~~~~~~~~~~~~~~~~
+
+Those functions expect two inputs of the same type and apply a given binary
+operation to each pair of elements gathered from the inputs. Each function
+is also available in an overflow-checking variant, suffixed ``_checked``.
+
+If any of the input elements in a pair is null, the corresponding output
+element is null.
+
++--------------------------+------------+--------------------+---------------------+
+| Function name | Arity | Input types | Output type |
++==========================+============+====================+=====================+
+| add | Binary | Numeric | Numeric |
++--------------------------+------------+--------------------+---------------------+
+| add_checked | Binary | Numeric | Numeric |
++--------------------------+------------+--------------------+---------------------+
+| multiply | Binary | Numeric | Numeric |
++--------------------------+------------+--------------------+---------------------+
+| multiply_checked | Binary | Numeric | Numeric |
++--------------------------+------------+--------------------+---------------------+
+| subtract | Binary | Numeric | Numeric |
++--------------------------+------------+--------------------+---------------------+
+| subtract_checked | Binary | Numeric | Numeric |
++--------------------------+------------+--------------------+---------------------+
+
+Comparisons
+~~~~~~~~~~~
+
+Those functions expect two inputs of the same type and apply a given
+comparison operator. If any of the input elements in a pair is null,
+the corresponding output element is null.
+
++--------------------------+------------+---------------------------------+---------------------+
+| Function names | Arity | Input types | Output type |
++==========================+============+=================================+=====================+
+| equal, not_equal | Binary | Numeric | Boolean |
++--------------------------+------------+---------------------------------+---------------------+
+| equal, not_equal | Binary | Binary- and String-like | Boolean |
++--------------------------+------------+---------------------------------+---------------------+
+| equal, not_equal | Binary | Temporal | Boolean |
++--------------------------+------------+---------------------------------+---------------------+
+| greater, greater_equal, | Binary | Numeric | Boolean |
+| less, less_equal | | | |
++--------------------------+------------+---------------------------------+---------------------+
+| greater, greater_equal, | Binary | Binary- and String-like | Boolean |
+| less, less_equal | | | |
++--------------------------+------------+---------------------------------+---------------------+
+| greater, greater_equal, | Binary | Temporal | Boolean |
+| less, less_equal | | | |
++--------------------------+------------+---------------------------------+---------------------+
+
+Logical functions
+~~~~~~~~~~~~~~~~~~
+
+The normal behaviour for these functions is to emit a null if any of the
+inputs is null.
+
+Some of them are also available in a "`Kleene logic`_" variant (suffixed
+``_kleene``) where null is taken to mean "undefined". For those variants
+therefore:
+
+* "true AND null", "null AND true" give "null" (the result is undefined)
+* "true OR null", "null OR true" give "true"
+* "false AND null", "null AND false" give "false"
+* "false OR null", "null OR false" give "null" (the result is undefined)
+
++--------------------------+------------+--------------------+---------------------+
+| Function name | Arity | Input types | Output type |
++==========================+============+====================+=====================+
+| and | Binary | Boolean | Boolean |
++--------------------------+------------+--------------------+---------------------+
+| and_kleene | Binary | Boolean | Boolean |
++--------------------------+------------+--------------------+---------------------+
+| invert | Unary | Boolean | Boolean |
++--------------------------+------------+--------------------+---------------------+
+| or | Binary | Boolean | Boolean |
++--------------------------+------------+--------------------+---------------------+
+| or_kleene | Binary | Boolean | Boolean |
++--------------------------+------------+--------------------+---------------------+
+| xor | Binary | Boolean | Boolean |
++--------------------------+------------+--------------------+---------------------+
+
+.. _Kleene logic: https://en.wikipedia.org/wiki/Three-valued_logic#Kleene_and_Priest_logics
+
+String functions
+~~~~~~~~~~~~~~~~
+
++--------------------------+------------+--------------------+---------------------+---------+
+| Function name | Arity | Input types | Output type | Notes |
++==========================+============+====================+=====================+=========+
+| ascii_length | Unary | String-like | Int32 or Int64 | \(1) |
++--------------------------+------------+--------------------+---------------------+---------+
+| ascii_lower | Unary | String-like | String-like | \(2) |
++--------------------------+------------+--------------------+---------------------+---------+
+| ascii_upper | Unary | String-like | String-like | \(2) |
++--------------------------+------------+--------------------+---------------------+---------+
+| utf8_lower | Unary | String-like | String-like | \(3) |
++--------------------------+------------+--------------------+---------------------+---------+
+| utf8_upper | Unary | String-like | String-like | \(3) |
++--------------------------+------------+--------------------+---------------------+---------+
+
+* \(1) Output is the physical length in bytes of each input element.
+
+* \(2) Each ASCII character in the input is converted to lowercase or
+ uppercase. Non-ASCII characters are left untouched.
+
+* \(3) Each UTF8-encoded character in the input is converted to lowercase or
+ uppercase.
+
+Containment tests
+~~~~~~~~~~~~~~~~~
+
++--------------------------+------------+----------------------------------+-----------------------+--------------------------------------------+
+| Function name | Arity | Input types | Output type | Options class |
++==========================+============+==================================+=======================+============================================+
+| binary_contains_exact | Unary | String-like | Boolean (1) | :struct:`BinaryContainsExactOptions` |
++--------------------------+------------+----------------------------------+-----------------------+--------------------------------------------+
+| isin | Unary | Binary- and String-like | Boolean (2) | :struct:`SetLookupOptions` |
++--------------------------+------------+----------------------------------+-----------------------+--------------------------------------------+
+| isin | Unary | Null | Boolean (2) | :struct:`SetLookupOptions` |
++--------------------------+------------+----------------------------------+-----------------------+--------------------------------------------+
+| isin | Unary | Boolean,Numeric, Temporal | Boolean (2) | :struct:`SetLookupOptions` |
++--------------------------+------------+----------------------------------+-----------------------+--------------------------------------------+
+| match | Unary | Binary- and String-like | Int32 (3) | :struct:`SetLookupOptions` |
++--------------------------+------------+----------------------------------+-----------------------+--------------------------------------------+
+| match | Unary | Null | Int32 (3) | :struct:`SetLookupOptions` |
++--------------------------+------------+----------------------------------+-----------------------+--------------------------------------------+
+| match | Unary | Boolean,Numeric, Temporal | Int32 (3) | :struct:`SetLookupOptions` |
++--------------------------+------------+----------------------------------+-----------------------+--------------------------------------------+
+
+* \(1) Output is true iff :member:`BinaryContainsExactOptions::pattern`
+ is a substring of the corresponding input element.
+
+* \(2) Output is true iff the corresponding input element is equal to one
+ of the elements in :member:`SetLookupOptions::value_set`.
+
+* \(3) Output is the index of the corresponding input element in
+ :member:`SetLookupOptions::value_set`, if found there. Otherwise,
+ output is null.
+
+Structural transforms
+~~~~~~~~~~~~~~~~~~~~~
+
++--------------------------+------------+--------------------+---------------------+---------+
+| Function name | Arity | Input types | Output type | Notes |
++==========================+============+====================+=====================+=========+
+| is_null | Unary | Any | Boolean | \(1) |
++--------------------------+------------+--------------------+---------------------+---------+
+| is_valid | Unary | Any | Boolean | \(2) |
++--------------------------+------------+--------------------+---------------------+---------+
+| list_value_lengths | Unary | List-like | Int32 or Int64 | \(3) |
++--------------------------+------------+--------------------+---------------------+---------+
+
+* \(1) Output is true iff the corresponding input element is non-null.
+
+* \(2) Output is true iff the corresponding input element is null.
+
+* \(3) Each output element is the length of the corresponding input element
+ (null if input is null).
+
+Conversions
+~~~~~~~~~~~
+
+A general conversion function named ``cast`` is provided which accepts a large
+number of input and output types. The type to cast to can be passed in a
+:struct:`CastOptions` instance.
+
++--------------------------+------------+--------------------+-----------------------+--------------------------------------------+
+| Function name | Arity | Input types | Output type | Options class |
++==========================+============+====================+=======================+============================================+
+| cast | Unary | Many | Variable | :struct:`CastOptions` |
++--------------------------+------------+--------------------+-----------------------+--------------------------------------------+
+| strptime | Unary | String-like | Timestamp | :struct:`StrptimeOptions` |
Review comment:
I think you can guess the answer now :-)
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] wesm commented on pull request #7695: ARROW-8989: [C++][Doc] Document available compute functions
Posted by GitBox <gi...@apache.org>.
wesm commented on pull request #7695:
URL: https://github.com/apache/arrow/pull/7695#issuecomment-657277758
I think either CSV or List tables would be an improvement over the current
https://docutils.sourceforge.io/docs/ref/rst/directives.html#tables
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] bkietz commented on a change in pull request #7695: ARROW-8989: [C++][Doc] Document available compute functions
Posted by GitBox <gi...@apache.org>.
bkietz commented on a change in pull request #7695:
URL: https://github.com/apache/arrow/pull/7695#discussion_r453365336
##########
File path: docs/source/cpp/compute.rst
##########
@@ -0,0 +1,419 @@
+.. Licensed to the Apache Software Foundation (ASF) under one
+.. or more contributor license agreements. See the NOTICE file
+.. distributed with this work for additional information
+.. regarding copyright ownership. The ASF licenses this file
+.. to you under the Apache License, Version 2.0 (the
+.. "License"); you may not use this file except in compliance
+.. with the License. You may obtain a copy of the License at
+
+.. http://www.apache.org/licenses/LICENSE-2.0
+
+.. Unless required by applicable law or agreed to in writing,
+.. software distributed under the License is distributed on an
+.. "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+.. KIND, either express or implied. See the License for the
+.. specific language governing permissions and limitations
+.. under the License.
+
+.. default-domain:: cpp
+.. highlight:: cpp
+.. cpp:namespace:: arrow::compute
+
+=================
+Compute Functions
+=================
+
+.. TODO: describe API and how to invoke compute functions
Review comment:
This is true, the explicit typing was for clarity of exposition
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] bkietz commented on pull request #7695: ARROW-8989: [C++][Doc] Document available compute functions
Posted by GitBox <gi...@apache.org>.
bkietz commented on pull request #7695:
URL: https://github.com/apache/arrow/pull/7695#issuecomment-657556478
> I could give a try to list tables, but otherwise I think CSV or JSON would be a major PITA to edit later.
Does that include tables which use the `file:` directive to refer to an out-of-line table source?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] pitrou commented on pull request #7695: ARROW-8989: [C++][Doc] Document available compute functions
Posted by GitBox <gi...@apache.org>.
pitrou commented on pull request #7695:
URL: https://github.com/apache/arrow/pull/7695#issuecomment-657699568
The three options I would be comfortable with are those I proposed in my comment above. I think it would be clumsy to have to edit separate _files_ using a spreadsheet editor to update a single page of the docs, though (especially as there are footnotes from the tables to the main text).
I would also suggest we discuss this later, since this PR probably deserves to be in 1.0 :-) Someone may want to submit a later draft PR converting this doc to a different table format.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] pitrou commented on a change in pull request #7695: ARROW-8989: [C++][Doc] Document available compute functions
Posted by GitBox <gi...@apache.org>.
pitrou commented on a change in pull request #7695:
URL: https://github.com/apache/arrow/pull/7695#discussion_r453002174
##########
File path: docs/source/cpp/compute.rst
##########
@@ -0,0 +1,419 @@
+.. Licensed to the Apache Software Foundation (ASF) under one
+.. or more contributor license agreements. See the NOTICE file
+.. distributed with this work for additional information
+.. regarding copyright ownership. The ASF licenses this file
+.. to you under the Apache License, Version 2.0 (the
+.. "License"); you may not use this file except in compliance
+.. with the License. You may obtain a copy of the License at
+
+.. http://www.apache.org/licenses/LICENSE-2.0
+
+.. Unless required by applicable law or agreed to in writing,
+.. software distributed under the License is distributed on an
+.. "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+.. KIND, either express or implied. See the License for the
+.. specific language governing permissions and limitations
+.. under the License.
+
+.. default-domain:: cpp
+.. highlight:: cpp
+.. cpp:namespace:: arrow::compute
+
+=================
+Compute Functions
+=================
+
+.. TODO: describe API and how to invoke compute functions
+
+Available functions
+===================
+
+Aggregations
+------------
+
++--------------------------+------------+--------------------+-----------------------+--------------------------------------------+
+| Function name | Arity | Input types | Output type | Options class |
++==========================+============+====================+=======================+============================================+
+| count | Unary | Any | Scalar Int64 | :struct:`CountOptions` |
++--------------------------+------------+--------------------+-----------------------+--------------------------------------------+
+| mean | Unary | Numeric | Scalar Float64 | |
++--------------------------+------------+--------------------+-----------------------+--------------------------------------------+
+| minmax | Unary | Numeric | Scalar Struct (1) | :struct:`MinMaxOptions` |
++--------------------------+------------+--------------------+-----------------------+--------------------------------------------+
+| sum | Unary | Numeric | Scalar Numeric (2) | |
++--------------------------+------------+--------------------+-----------------------+--------------------------------------------+
+
+Notes:
+
+* \(1) Output is a ``{"min": input type, "max": input type}`` Struct
+
+* \(2) Output is Int64, UInt64 or Float64, depending on the input type
+
+
+Element-wise ("scalar") functions
+---------------------------------
+
+Arithmetic functions
+~~~~~~~~~~~~~~~~~~~~
+
+Those functions expect two inputs of the same type and apply a given binary
+operation to each pair of elements gathered from the inputs. Each function
+is also available in an overflow-checking variant, suffixed ``_checked``.
+
+If any of the input elements in a pair is null, the corresponding output
+element is null.
+
++--------------------------+------------+--------------------+---------------------+
+| Function name | Arity | Input types | Output type |
++==========================+============+====================+=====================+
+| add | Binary | Numeric | Numeric |
++--------------------------+------------+--------------------+---------------------+
+| add_checked | Binary | Numeric | Numeric |
++--------------------------+------------+--------------------+---------------------+
+| multiply | Binary | Numeric | Numeric |
++--------------------------+------------+--------------------+---------------------+
+| multiply_checked | Binary | Numeric | Numeric |
++--------------------------+------------+--------------------+---------------------+
+| subtract | Binary | Numeric | Numeric |
++--------------------------+------------+--------------------+---------------------+
+| subtract_checked | Binary | Numeric | Numeric |
++--------------------------+------------+--------------------+---------------------+
+
+Comparisons
+~~~~~~~~~~~
+
+Those functions expect two inputs of the same type and apply a given
+comparison operator. If any of the input elements in a pair is null,
+the corresponding output element is null.
+
++--------------------------+------------+---------------------------------+---------------------+
+| Function names | Arity | Input types | Output type |
++==========================+============+=================================+=====================+
+| equal, not_equal | Binary | Numeric | Boolean |
++--------------------------+------------+---------------------------------+---------------------+
+| equal, not_equal | Binary | Binary- and String-like | Boolean |
++--------------------------+------------+---------------------------------+---------------------+
+| equal, not_equal | Binary | Temporal | Boolean |
++--------------------------+------------+---------------------------------+---------------------+
+| greater, greater_equal, | Binary | Numeric | Boolean |
+| less, less_equal | | | |
++--------------------------+------------+---------------------------------+---------------------+
+| greater, greater_equal, | Binary | Binary- and String-like | Boolean |
+| less, less_equal | | | |
++--------------------------+------------+---------------------------------+---------------------+
+| greater, greater_equal, | Binary | Temporal | Boolean |
+| less, less_equal | | | |
++--------------------------+------------+---------------------------------+---------------------+
+
+Logical functions
+~~~~~~~~~~~~~~~~~~
+
+The normal behaviour for these functions is to emit a null if any of the
+inputs is null.
+
+Some of them are also available in a "`Kleene logic`_" variant (suffixed
+``_kleene``) where null is taken to mean "undefined". For those variants
+therefore:
+
+* "true AND null", "null AND true" give "null" (the result is undefined)
+* "true OR null", "null OR true" give "true"
+* "false AND null", "null AND false" give "false"
+* "false OR null", "null OR false" give "null" (the result is undefined)
+
++--------------------------+------------+--------------------+---------------------+
+| Function name | Arity | Input types | Output type |
++==========================+============+====================+=====================+
+| and | Binary | Boolean | Boolean |
++--------------------------+------------+--------------------+---------------------+
+| and_kleene | Binary | Boolean | Boolean |
++--------------------------+------------+--------------------+---------------------+
+| invert | Unary | Boolean | Boolean |
++--------------------------+------------+--------------------+---------------------+
+| or | Binary | Boolean | Boolean |
++--------------------------+------------+--------------------+---------------------+
+| or_kleene | Binary | Boolean | Boolean |
++--------------------------+------------+--------------------+---------------------+
+| xor | Binary | Boolean | Boolean |
++--------------------------+------------+--------------------+---------------------+
+
+.. _Kleene logic: https://en.wikipedia.org/wiki/Three-valued_logic#Kleene_and_Priest_logics
+
+String functions
+~~~~~~~~~~~~~~~~
+
++--------------------------+------------+--------------------+---------------------+---------+
+| Function name | Arity | Input types | Output type | Notes |
++==========================+============+====================+=====================+=========+
+| ascii_length | Unary | String-like | Int32 or Int64 | \(1) |
++--------------------------+------------+--------------------+---------------------+---------+
+| ascii_lower | Unary | String-like | String-like | \(2) |
++--------------------------+------------+--------------------+---------------------+---------+
+| ascii_upper | Unary | String-like | String-like | \(2) |
++--------------------------+------------+--------------------+---------------------+---------+
+| utf8_lower | Unary | String-like | String-like | \(3) |
++--------------------------+------------+--------------------+---------------------+---------+
+| utf8_upper | Unary | String-like | String-like | \(3) |
++--------------------------+------------+--------------------+---------------------+---------+
+
+* \(1) Output is the physical length in bytes of each input element.
+
+* \(2) Each ASCII character in the input is converted to lowercase or
+ uppercase. Non-ASCII characters are left untouched.
+
+* \(3) Each UTF8-encoded character in the input is converted to lowercase or
+ uppercase.
+
+Containment tests
+~~~~~~~~~~~~~~~~~
+
++--------------------------+------------+----------------------------------+-----------------------+--------------------------------------------+
+| Function name | Arity | Input types | Output type | Options class |
++==========================+============+==================================+=======================+============================================+
+| binary_contains_exact | Unary | String-like | Boolean (1) | :struct:`BinaryContainsExactOptions` |
++--------------------------+------------+----------------------------------+-----------------------+--------------------------------------------+
+| isin | Unary | Binary- and String-like | Boolean (2) | :struct:`SetLookupOptions` |
++--------------------------+------------+----------------------------------+-----------------------+--------------------------------------------+
+| isin | Unary | Null | Boolean (2) | :struct:`SetLookupOptions` |
++--------------------------+------------+----------------------------------+-----------------------+--------------------------------------------+
+| isin | Unary | Boolean,Numeric, Temporal | Boolean (2) | :struct:`SetLookupOptions` |
++--------------------------+------------+----------------------------------+-----------------------+--------------------------------------------+
+| match | Unary | Binary- and String-like | Int32 (3) | :struct:`SetLookupOptions` |
++--------------------------+------------+----------------------------------+-----------------------+--------------------------------------------+
+| match | Unary | Null | Int32 (3) | :struct:`SetLookupOptions` |
++--------------------------+------------+----------------------------------+-----------------------+--------------------------------------------+
+| match | Unary | Boolean,Numeric, Temporal | Int32 (3) | :struct:`SetLookupOptions` |
++--------------------------+------------+----------------------------------+-----------------------+--------------------------------------------+
+
+* \(1) Output is true iff :member:`BinaryContainsExactOptions::pattern`
+ is a substring of the corresponding input element.
+
+* \(2) Output is true iff the corresponding input element is equal to one
+ of the elements in :member:`SetLookupOptions::value_set`.
+
+* \(3) Output is the index of the corresponding input element in
+ :member:`SetLookupOptions::value_set`, if found there. Otherwise,
+ output is null.
+
+Structural transforms
+~~~~~~~~~~~~~~~~~~~~~
+
++--------------------------+------------+--------------------+---------------------+---------+
+| Function name | Arity | Input types | Output type | Notes |
++==========================+============+====================+=====================+=========+
+| is_null | Unary | Any | Boolean | \(1) |
++--------------------------+------------+--------------------+---------------------+---------+
+| is_valid | Unary | Any | Boolean | \(2) |
++--------------------------+------------+--------------------+---------------------+---------+
+| list_value_lengths | Unary | List-like | Int32 or Int64 | \(3) |
++--------------------------+------------+--------------------+---------------------+---------+
+
+* \(1) Output is true iff the corresponding input element is non-null.
+
+* \(2) Output is true iff the corresponding input element is null.
+
+* \(3) Each output element is the length of the corresponding input element
+ (null if input is null).
+
+Conversions
+~~~~~~~~~~~
+
+A general conversion function named ``cast`` is provided which accepts a large
+number of input and output types. The type to cast to can be passed in a
+:struct:`CastOptions` instance.
+
++--------------------------+------------+--------------------+-----------------------+--------------------------------------------+
+| Function name | Arity | Input types | Output type | Options class |
++==========================+============+====================+=======================+============================================+
+| cast | Unary | Many | Variable | :struct:`CastOptions` |
++--------------------------+------------+--------------------+-----------------------+--------------------------------------------+
+| strptime | Unary | String-like | Timestamp | :struct:`StrptimeOptions` |
++--------------------------+------------+--------------------+-----------------------+--------------------------------------------+
+
+The conversions available with ``cast`` are listed below. In all cases, a
+null input value is converted into a null output value.
+
+**Truth value extraction**
+
++-----------------------------+------------------------------------+--------------+
+| Input type | Output type | Notes |
++=============================+====================================+==============+
+| Binary- and String-like | Boolean | \(1) |
++-----------------------------+------------------------------------+--------------+
+| Numeric | Boolean | \(2) |
++-----------------------------+------------------------------------+--------------+
+
+* \(1) Output is true iff the corresponding input value has non-zero length.
+
+* \(2) Output is true iff the corresponding input value is non-zero.
+
+**Same-kind conversion**
+
++-----------------------------+------------------------------------+--------------+
+| Input type | Output type | Notes |
++=============================+====================================+==============+
+| Int32 | 32-bit Temporal | \(1) |
++-----------------------------+------------------------------------+--------------+
+| Int64 | 64-bit Temporal | \(1) |
++-----------------------------+------------------------------------+--------------+
+| (Large)Binary | (Large)String | \(2) |
++-----------------------------+------------------------------------+--------------+
+| (Large)String | (Large)Binary | \(3) |
++-----------------------------+------------------------------------+--------------+
+| Numeric | Numeric | \(4) \(5) |
++-----------------------------+------------------------------------+--------------+
+| 32-bit Temporal | Int32 | \(1) |
++-----------------------------+------------------------------------+--------------+
+| 64-bit Temporal | Int64 | \(1) |
++-----------------------------+------------------------------------+--------------+
+| Temporal | Temporal | \(4) \(5) |
++-----------------------------+------------------------------------+--------------+
+
+* \(1) No-operation cast: the raw values are kept identical, only
+ the type is changed.
+
+* \(2) Validates the contents if :member:`CastOptions::allow_invalid_utf8`
+ is false.
+
+* \(3) No-operation cast: only the type is changed.
+
+* \(4) Overflow and truncation checks are enabled depending on
+ the given :struct:`CastOptions`.
+
+* \(5) Not all such casts have been implemented.
+
+**String representations**
+
++-----------------------------+------------------------------------+---------+
+| Input type | Output type | Notes |
++=============================+====================================+=========+
+| Boolean | String-like | |
++-----------------------------+------------------------------------+---------+
+| Numeric | String-like | |
++-----------------------------+------------------------------------+---------+
+
+**Generic conversions**
+
++-----------------------------+------------------------------------+---------+
+| Input type | Output type | Notes |
++=============================+====================================+=========+
+| Dictionary | Dictionary value type | |
++-----------------------------+------------------------------------+---------+
+| Extension | Extension storage type | |
++-----------------------------+------------------------------------+---------+
+| List-like | List-like | \(1) |
++-----------------------------+------------------------------------+---------+
+| Null | Any | |
++-----------------------------+------------------------------------+---------+
+
+* \(1) The list offsets are unchanged, the list values are cast from the
+ input value type to the output value type (if a conversion is
+ available).
+
+
+.. TODO: add C++ cast example
+
+Array-wise ("vector") functions
+-------------------------------
+
+Associative transforms
+~~~~~~~~~~~~~~~~~~~~~~
+
++--------------------------+------------+----------------------------+----------------------------+
+| Function name | Arity | Input types | Output type |
++==========================+============+============================+============================+
+| dictionary_encode | Unary | Binary- and String-like | Dictionary (1) |
Review comment:
Perhaps because it's special enough, though @wesm would know the answer better than me.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] bkietz commented on a change in pull request #7695: ARROW-8989: [C++][Doc] Document available compute functions
Posted by GitBox <gi...@apache.org>.
bkietz commented on a change in pull request #7695:
URL: https://github.com/apache/arrow/pull/7695#discussion_r453046448
##########
File path: docs/source/cpp/compute.rst
##########
@@ -0,0 +1,419 @@
+.. Licensed to the Apache Software Foundation (ASF) under one
+.. or more contributor license agreements. See the NOTICE file
+.. distributed with this work for additional information
+.. regarding copyright ownership. The ASF licenses this file
+.. to you under the Apache License, Version 2.0 (the
+.. "License"); you may not use this file except in compliance
+.. with the License. You may obtain a copy of the License at
+
+.. http://www.apache.org/licenses/LICENSE-2.0
+
+.. Unless required by applicable law or agreed to in writing,
+.. software distributed under the License is distributed on an
+.. "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+.. KIND, either express or implied. See the License for the
+.. specific language governing permissions and limitations
+.. under the License.
+
+.. default-domain:: cpp
+.. highlight:: cpp
+.. cpp:namespace:: arrow::compute
+
+=================
+Compute Functions
+=================
+
+.. TODO: describe API and how to invoke compute functions
+
+Available functions
+===================
+
+Aggregations
+------------
+
++--------------------------+------------+--------------------+-----------------------+--------------------------------------------+
+| Function name | Arity | Input types | Output type | Options class |
++==========================+============+====================+=======================+============================================+
+| count | Unary | Any | Scalar Int64 | :struct:`CountOptions` |
++--------------------------+------------+--------------------+-----------------------+--------------------------------------------+
+| mean | Unary | Numeric | Scalar Float64 | |
++--------------------------+------------+--------------------+-----------------------+--------------------------------------------+
+| minmax | Unary | Numeric | Scalar Struct (1) | :struct:`MinMaxOptions` |
++--------------------------+------------+--------------------+-----------------------+--------------------------------------------+
+| sum | Unary | Numeric | Scalar Numeric (2) | |
++--------------------------+------------+--------------------+-----------------------+--------------------------------------------+
+
+Notes:
+
+* \(1) Output is a ``{"min": input type, "max": input type}`` Struct
+
+* \(2) Output is Int64, UInt64 or Float64, depending on the input type
+
+
+Element-wise ("scalar") functions
+---------------------------------
+
+Arithmetic functions
+~~~~~~~~~~~~~~~~~~~~
+
+Those functions expect two inputs of the same type and apply a given binary
+operation to each pair of elements gathered from the inputs. Each function
+is also available in an overflow-checking variant, suffixed ``_checked``.
+
+If any of the input elements in a pair is null, the corresponding output
+element is null.
+
++--------------------------+------------+--------------------+---------------------+
+| Function name | Arity | Input types | Output type |
++==========================+============+====================+=====================+
+| add | Binary | Numeric | Numeric |
++--------------------------+------------+--------------------+---------------------+
+| add_checked | Binary | Numeric | Numeric |
++--------------------------+------------+--------------------+---------------------+
+| multiply | Binary | Numeric | Numeric |
++--------------------------+------------+--------------------+---------------------+
+| multiply_checked | Binary | Numeric | Numeric |
++--------------------------+------------+--------------------+---------------------+
+| subtract | Binary | Numeric | Numeric |
++--------------------------+------------+--------------------+---------------------+
+| subtract_checked | Binary | Numeric | Numeric |
++--------------------------+------------+--------------------+---------------------+
+
+Comparisons
+~~~~~~~~~~~
+
+Those functions expect two inputs of the same type and apply a given
+comparison operator. If any of the input elements in a pair is null,
+the corresponding output element is null.
+
++--------------------------+------------+---------------------------------+---------------------+
+| Function names | Arity | Input types | Output type |
++==========================+============+=================================+=====================+
+| equal, not_equal | Binary | Numeric | Boolean |
++--------------------------+------------+---------------------------------+---------------------+
+| equal, not_equal | Binary | Binary- and String-like | Boolean |
++--------------------------+------------+---------------------------------+---------------------+
+| equal, not_equal | Binary | Temporal | Boolean |
++--------------------------+------------+---------------------------------+---------------------+
+| greater, greater_equal, | Binary | Numeric | Boolean |
+| less, less_equal | | | |
++--------------------------+------------+---------------------------------+---------------------+
+| greater, greater_equal, | Binary | Binary- and String-like | Boolean |
+| less, less_equal | | | |
++--------------------------+------------+---------------------------------+---------------------+
+| greater, greater_equal, | Binary | Temporal | Boolean |
+| less, less_equal | | | |
++--------------------------+------------+---------------------------------+---------------------+
+
+Logical functions
+~~~~~~~~~~~~~~~~~~
+
+The normal behaviour for these functions is to emit a null if any of the
+inputs is null.
Review comment:
```suggestion
inputs is null, similar to the way any operation involving ``NaN`` devolves to ``NaN``.
```
##########
File path: docs/source/cpp/compute.rst
##########
@@ -0,0 +1,419 @@
+.. Licensed to the Apache Software Foundation (ASF) under one
+.. or more contributor license agreements. See the NOTICE file
+.. distributed with this work for additional information
+.. regarding copyright ownership. The ASF licenses this file
+.. to you under the Apache License, Version 2.0 (the
+.. "License"); you may not use this file except in compliance
+.. with the License. You may obtain a copy of the License at
+
+.. http://www.apache.org/licenses/LICENSE-2.0
+
+.. Unless required by applicable law or agreed to in writing,
+.. software distributed under the License is distributed on an
+.. "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+.. KIND, either express or implied. See the License for the
+.. specific language governing permissions and limitations
+.. under the License.
+
+.. default-domain:: cpp
+.. highlight:: cpp
+.. cpp:namespace:: arrow::compute
+
+=================
+Compute Functions
+=================
+
+.. TODO: describe API and how to invoke compute functions
+
+Available functions
+===================
+
+Aggregations
+------------
+
++--------------------------+------------+--------------------+-----------------------+--------------------------------------------+
+| Function name | Arity | Input types | Output type | Options class |
++==========================+============+====================+=======================+============================================+
+| count | Unary | Any | Scalar Int64 | :struct:`CountOptions` |
++--------------------------+------------+--------------------+-----------------------+--------------------------------------------+
+| mean | Unary | Numeric | Scalar Float64 | |
++--------------------------+------------+--------------------+-----------------------+--------------------------------------------+
+| minmax | Unary | Numeric | Scalar Struct (1) | :struct:`MinMaxOptions` |
++--------------------------+------------+--------------------+-----------------------+--------------------------------------------+
+| sum | Unary | Numeric | Scalar Numeric (2) | |
++--------------------------+------------+--------------------+-----------------------+--------------------------------------------+
+
+Notes:
+
+* \(1) Output is a ``{"min": input type, "max": input type}`` Struct
+
+* \(2) Output is Int64, UInt64 or Float64, depending on the input type
+
+
+Element-wise ("scalar") functions
+---------------------------------
+
+Arithmetic functions
+~~~~~~~~~~~~~~~~~~~~
+
+Those functions expect two inputs of the same type and apply a given binary
+operation to each pair of elements gathered from the inputs. Each function
+is also available in an overflow-checking variant, suffixed ``_checked``.
+
+If any of the input elements in a pair is null, the corresponding output
+element is null.
+
++--------------------------+------------+--------------------+---------------------+
+| Function name | Arity | Input types | Output type |
++==========================+============+====================+=====================+
+| add | Binary | Numeric | Numeric |
++--------------------------+------------+--------------------+---------------------+
+| add_checked | Binary | Numeric | Numeric |
++--------------------------+------------+--------------------+---------------------+
+| multiply | Binary | Numeric | Numeric |
++--------------------------+------------+--------------------+---------------------+
+| multiply_checked | Binary | Numeric | Numeric |
++--------------------------+------------+--------------------+---------------------+
+| subtract | Binary | Numeric | Numeric |
++--------------------------+------------+--------------------+---------------------+
+| subtract_checked | Binary | Numeric | Numeric |
++--------------------------+------------+--------------------+---------------------+
+
+Comparisons
+~~~~~~~~~~~
+
+Those functions expect two inputs of the same type and apply a given
+comparison operator. If any of the input elements in a pair is null,
+the corresponding output element is null.
+
++--------------------------+------------+---------------------------------+---------------------+
+| Function names | Arity | Input types | Output type |
++==========================+============+=================================+=====================+
+| equal, not_equal | Binary | Numeric | Boolean |
++--------------------------+------------+---------------------------------+---------------------+
+| equal, not_equal | Binary | Binary- and String-like | Boolean |
++--------------------------+------------+---------------------------------+---------------------+
+| equal, not_equal | Binary | Temporal | Boolean |
++--------------------------+------------+---------------------------------+---------------------+
+| greater, greater_equal, | Binary | Numeric | Boolean |
+| less, less_equal | | | |
++--------------------------+------------+---------------------------------+---------------------+
+| greater, greater_equal, | Binary | Binary- and String-like | Boolean |
+| less, less_equal | | | |
++--------------------------+------------+---------------------------------+---------------------+
+| greater, greater_equal, | Binary | Temporal | Boolean |
+| less, less_equal | | | |
++--------------------------+------------+---------------------------------+---------------------+
+
+Logical functions
+~~~~~~~~~~~~~~~~~~
+
+The normal behaviour for these functions is to emit a null if any of the
+inputs is null.
+
+Some of them are also available in a "`Kleene logic`_" variant (suffixed
+``_kleene``) where null is taken to mean "undefined". For those variants
+therefore:
+
+* "true AND null", "null AND true" give "null" (the result is undefined)
+* "true OR null", "null OR true" give "true"
+* "false AND null", "null AND false" give "false"
+* "false OR null", "null OR false" give "null" (the result is undefined)
+
++--------------------------+------------+--------------------+---------------------+
+| Function name | Arity | Input types | Output type |
++==========================+============+====================+=====================+
+| and | Binary | Boolean | Boolean |
++--------------------------+------------+--------------------+---------------------+
+| and_kleene | Binary | Boolean | Boolean |
++--------------------------+------------+--------------------+---------------------+
+| invert | Unary | Boolean | Boolean |
++--------------------------+------------+--------------------+---------------------+
+| or | Binary | Boolean | Boolean |
++--------------------------+------------+--------------------+---------------------+
+| or_kleene | Binary | Boolean | Boolean |
++--------------------------+------------+--------------------+---------------------+
+| xor | Binary | Boolean | Boolean |
++--------------------------+------------+--------------------+---------------------+
+
+.. _Kleene logic: https://en.wikipedia.org/wiki/Three-valued_logic#Kleene_and_Priest_logics
+
+String functions
+~~~~~~~~~~~~~~~~
+
++--------------------------+------------+--------------------+---------------------+---------+
+| Function name | Arity | Input types | Output type | Notes |
++==========================+============+====================+=====================+=========+
+| ascii_length | Unary | String-like | Int32 or Int64 | \(1) |
++--------------------------+------------+--------------------+---------------------+---------+
+| ascii_lower | Unary | String-like | String-like | \(2) |
++--------------------------+------------+--------------------+---------------------+---------+
+| ascii_upper | Unary | String-like | String-like | \(2) |
++--------------------------+------------+--------------------+---------------------+---------+
+| utf8_lower | Unary | String-like | String-like | \(3) |
++--------------------------+------------+--------------------+---------------------+---------+
+| utf8_upper | Unary | String-like | String-like | \(3) |
++--------------------------+------------+--------------------+---------------------+---------+
+
+* \(1) Output is the physical length in bytes of each input element.
+
+* \(2) Each ASCII character in the input is converted to lowercase or
+ uppercase. Non-ASCII characters are left untouched.
+
+* \(3) Each UTF8-encoded character in the input is converted to lowercase or
+ uppercase.
+
+Containment tests
+~~~~~~~~~~~~~~~~~
+
++--------------------------+------------+----------------------------------+-----------------------+--------------------------------------------+
+| Function name | Arity | Input types | Output type | Options class |
++==========================+============+==================================+=======================+============================================+
+| binary_contains_exact | Unary | String-like | Boolean (1) | :struct:`BinaryContainsExactOptions` |
++--------------------------+------------+----------------------------------+-----------------------+--------------------------------------------+
+| isin | Unary | Binary- and String-like | Boolean (2) | :struct:`SetLookupOptions` |
++--------------------------+------------+----------------------------------+-----------------------+--------------------------------------------+
+| isin | Unary | Null | Boolean (2) | :struct:`SetLookupOptions` |
++--------------------------+------------+----------------------------------+-----------------------+--------------------------------------------+
+| isin | Unary | Boolean,Numeric, Temporal | Boolean (2) | :struct:`SetLookupOptions` |
++--------------------------+------------+----------------------------------+-----------------------+--------------------------------------------+
+| match | Unary | Binary- and String-like | Int32 (3) | :struct:`SetLookupOptions` |
++--------------------------+------------+----------------------------------+-----------------------+--------------------------------------------+
+| match | Unary | Null | Int32 (3) | :struct:`SetLookupOptions` |
++--------------------------+------------+----------------------------------+-----------------------+--------------------------------------------+
+| match | Unary | Boolean,Numeric, Temporal | Int32 (3) | :struct:`SetLookupOptions` |
++--------------------------+------------+----------------------------------+-----------------------+--------------------------------------------+
+
+* \(1) Output is true iff :member:`BinaryContainsExactOptions::pattern`
+ is a substring of the corresponding input element.
+
+* \(2) Output is true iff the corresponding input element is equal to one
+ of the elements in :member:`SetLookupOptions::value_set`.
+
+* \(3) Output is the index of the corresponding input element in
+ :member:`SetLookupOptions::value_set`, if found there. Otherwise,
+ output is null.
+
+Structural transforms
+~~~~~~~~~~~~~~~~~~~~~
+
++--------------------------+------------+--------------------+---------------------+---------+
+| Function name | Arity | Input types | Output type | Notes |
++==========================+============+====================+=====================+=========+
+| is_null | Unary | Any | Boolean | \(1) |
++--------------------------+------------+--------------------+---------------------+---------+
+| is_valid | Unary | Any | Boolean | \(2) |
++--------------------------+------------+--------------------+---------------------+---------+
+| list_value_lengths | Unary | List-like | Int32 or Int64 | \(3) |
++--------------------------+------------+--------------------+---------------------+---------+
+
+* \(1) Output is true iff the corresponding input element is non-null.
+
+* \(2) Output is true iff the corresponding input element is null.
+
+* \(3) Each output element is the length of the corresponding input element
+ (null if input is null).
Review comment:
```suggestion
(null if input is null). The output type is Int32 for List, Int64 for LargeList
```
##########
File path: docs/source/cpp/compute.rst
##########
@@ -0,0 +1,419 @@
+.. Licensed to the Apache Software Foundation (ASF) under one
+.. or more contributor license agreements. See the NOTICE file
+.. distributed with this work for additional information
+.. regarding copyright ownership. The ASF licenses this file
+.. to you under the Apache License, Version 2.0 (the
+.. "License"); you may not use this file except in compliance
+.. with the License. You may obtain a copy of the License at
+
+.. http://www.apache.org/licenses/LICENSE-2.0
+
+.. Unless required by applicable law or agreed to in writing,
+.. software distributed under the License is distributed on an
+.. "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+.. KIND, either express or implied. See the License for the
+.. specific language governing permissions and limitations
+.. under the License.
+
+.. default-domain:: cpp
+.. highlight:: cpp
+.. cpp:namespace:: arrow::compute
+
+=================
+Compute Functions
+=================
+
+.. TODO: describe API and how to invoke compute functions
+
+Available functions
+===================
+
+Aggregations
+------------
+
++--------------------------+------------+--------------------+-----------------------+--------------------------------------------+
+| Function name | Arity | Input types | Output type | Options class |
++==========================+============+====================+=======================+============================================+
+| count | Unary | Any | Scalar Int64 | :struct:`CountOptions` |
++--------------------------+------------+--------------------+-----------------------+--------------------------------------------+
+| mean | Unary | Numeric | Scalar Float64 | |
++--------------------------+------------+--------------------+-----------------------+--------------------------------------------+
+| minmax | Unary | Numeric | Scalar Struct (1) | :struct:`MinMaxOptions` |
++--------------------------+------------+--------------------+-----------------------+--------------------------------------------+
+| sum | Unary | Numeric | Scalar Numeric (2) | |
++--------------------------+------------+--------------------+-----------------------+--------------------------------------------+
+
+Notes:
+
+* \(1) Output is a ``{"min": input type, "max": input type}`` Struct
+
+* \(2) Output is Int64, UInt64 or Float64, depending on the input type
+
+
+Element-wise ("scalar") functions
+---------------------------------
+
+Arithmetic functions
+~~~~~~~~~~~~~~~~~~~~
+
+Those functions expect two inputs of the same type and apply a given binary
+operation to each pair of elements gathered from the inputs. Each function
+is also available in an overflow-checking variant, suffixed ``_checked``.
Review comment:
If we made a recommendation for general use it'd probably be `_checked` so that users become aware of overflow sooner and can be intentional about handling it
##########
File path: docs/source/cpp/compute.rst
##########
@@ -0,0 +1,419 @@
+.. Licensed to the Apache Software Foundation (ASF) under one
+.. or more contributor license agreements. See the NOTICE file
+.. distributed with this work for additional information
+.. regarding copyright ownership. The ASF licenses this file
+.. to you under the Apache License, Version 2.0 (the
+.. "License"); you may not use this file except in compliance
+.. with the License. You may obtain a copy of the License at
+
+.. http://www.apache.org/licenses/LICENSE-2.0
+
+.. Unless required by applicable law or agreed to in writing,
+.. software distributed under the License is distributed on an
+.. "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+.. KIND, either express or implied. See the License for the
+.. specific language governing permissions and limitations
+.. under the License.
+
+.. default-domain:: cpp
+.. highlight:: cpp
+.. cpp:namespace:: arrow::compute
+
+=================
+Compute Functions
+=================
+
+.. TODO: describe API and how to invoke compute functions
Review comment:
```suggestion
The inputs and outputs of compute functions are of type :class:`arrow::Datum`, which are
a discriminated union of several shapes of data, including :class:`arrow::Scalar`,
:class:`arrow::Array`, and :class:`arrow::ChunkedArray`.
Compute functions can be invoked by name using the :func:`arrow::compute::CallFunction`
or via dedicated c++ convenience functions.
.. code-block:: cpp
std::shared_ptr<arrow::Int32Array> ints = ...;
std::shared_ptr<arrow::Int32Scalar> increment = ...;
ARROW_ASSIGN_OR_RAISE(arrow::Datum incremented, arrow::compute::CallFunction("add_checked",
{arrow::Datum(ints), arrow::Datum(increment)}));
arrow::ArithmeticOptions options;
options.check_overflow = true;
ARROW_ASSIGN_OR_RAISE(arrow::Datum equivalent, arrow::compute::Add(
{arrow::Datum(ints), arrow::Datum(increment)}, arithmetic_options));
```
##########
File path: docs/source/cpp/compute.rst
##########
@@ -0,0 +1,419 @@
+.. Licensed to the Apache Software Foundation (ASF) under one
+.. or more contributor license agreements. See the NOTICE file
+.. distributed with this work for additional information
+.. regarding copyright ownership. The ASF licenses this file
+.. to you under the Apache License, Version 2.0 (the
+.. "License"); you may not use this file except in compliance
+.. with the License. You may obtain a copy of the License at
+
+.. http://www.apache.org/licenses/LICENSE-2.0
+
+.. Unless required by applicable law or agreed to in writing,
+.. software distributed under the License is distributed on an
+.. "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+.. KIND, either express or implied. See the License for the
+.. specific language governing permissions and limitations
+.. under the License.
+
+.. default-domain:: cpp
+.. highlight:: cpp
+.. cpp:namespace:: arrow::compute
+
+=================
+Compute Functions
+=================
+
+.. TODO: describe API and how to invoke compute functions
+
+Available functions
+===================
+
+Aggregations
+------------
+
++--------------------------+------------+--------------------+-----------------------+--------------------------------------------+
+| Function name | Arity | Input types | Output type | Options class |
++==========================+============+====================+=======================+============================================+
+| count | Unary | Any | Scalar Int64 | :struct:`CountOptions` |
++--------------------------+------------+--------------------+-----------------------+--------------------------------------------+
+| mean | Unary | Numeric | Scalar Float64 | |
++--------------------------+------------+--------------------+-----------------------+--------------------------------------------+
+| minmax | Unary | Numeric | Scalar Struct (1) | :struct:`MinMaxOptions` |
++--------------------------+------------+--------------------+-----------------------+--------------------------------------------+
+| sum | Unary | Numeric | Scalar Numeric (2) | |
++--------------------------+------------+--------------------+-----------------------+--------------------------------------------+
+
+Notes:
+
+* \(1) Output is a ``{"min": input type, "max": input type}`` Struct
+
+* \(2) Output is Int64, UInt64 or Float64, depending on the input type
+
+
+Element-wise ("scalar") functions
+---------------------------------
+
+Arithmetic functions
+~~~~~~~~~~~~~~~~~~~~
+
+Those functions expect two inputs of the same type and apply a given binary
+operation to each pair of elements gathered from the inputs. Each function
+is also available in an overflow-checking variant, suffixed ``_checked``.
+
+If any of the input elements in a pair is null, the corresponding output
+element is null.
+
++--------------------------+------------+--------------------+---------------------+
+| Function name | Arity | Input types | Output type |
++==========================+============+====================+=====================+
+| add | Binary | Numeric | Numeric |
++--------------------------+------------+--------------------+---------------------+
+| add_checked | Binary | Numeric | Numeric |
++--------------------------+------------+--------------------+---------------------+
+| multiply | Binary | Numeric | Numeric |
++--------------------------+------------+--------------------+---------------------+
+| multiply_checked | Binary | Numeric | Numeric |
++--------------------------+------------+--------------------+---------------------+
+| subtract | Binary | Numeric | Numeric |
++--------------------------+------------+--------------------+---------------------+
+| subtract_checked | Binary | Numeric | Numeric |
++--------------------------+------------+--------------------+---------------------+
+
+Comparisons
+~~~~~~~~~~~
+
+Those functions expect two inputs of the same type and apply a given
+comparison operator. If any of the input elements in a pair is null,
+the corresponding output element is null.
+
++--------------------------+------------+---------------------------------+---------------------+
+| Function names | Arity | Input types | Output type |
++==========================+============+=================================+=====================+
+| equal, not_equal | Binary | Numeric | Boolean |
++--------------------------+------------+---------------------------------+---------------------+
+| equal, not_equal | Binary | Binary- and String-like | Boolean |
++--------------------------+------------+---------------------------------+---------------------+
+| equal, not_equal | Binary | Temporal | Boolean |
++--------------------------+------------+---------------------------------+---------------------+
+| greater, greater_equal, | Binary | Numeric | Boolean |
+| less, less_equal | | | |
++--------------------------+------------+---------------------------------+---------------------+
+| greater, greater_equal, | Binary | Binary- and String-like | Boolean |
+| less, less_equal | | | |
++--------------------------+------------+---------------------------------+---------------------+
+| greater, greater_equal, | Binary | Temporal | Boolean |
+| less, less_equal | | | |
++--------------------------+------------+---------------------------------+---------------------+
+
+Logical functions
+~~~~~~~~~~~~~~~~~~
+
+The normal behaviour for these functions is to emit a null if any of the
+inputs is null.
+
+Some of them are also available in a "`Kleene logic`_" variant (suffixed
+``_kleene``) where null is taken to mean "undefined". For those variants
+therefore:
+
+* "true AND null", "null AND true" give "null" (the result is undefined)
+* "true OR null", "null OR true" give "true"
+* "false AND null", "null AND false" give "false"
+* "false OR null", "null OR false" give "null" (the result is undefined)
+
++--------------------------+------------+--------------------+---------------------+
+| Function name | Arity | Input types | Output type |
++==========================+============+====================+=====================+
+| and | Binary | Boolean | Boolean |
++--------------------------+------------+--------------------+---------------------+
+| and_kleene | Binary | Boolean | Boolean |
++--------------------------+------------+--------------------+---------------------+
+| invert | Unary | Boolean | Boolean |
++--------------------------+------------+--------------------+---------------------+
+| or | Binary | Boolean | Boolean |
++--------------------------+------------+--------------------+---------------------+
+| or_kleene | Binary | Boolean | Boolean |
++--------------------------+------------+--------------------+---------------------+
+| xor | Binary | Boolean | Boolean |
++--------------------------+------------+--------------------+---------------------+
+
+.. _Kleene logic: https://en.wikipedia.org/wiki/Three-valued_logic#Kleene_and_Priest_logics
+
+String functions
+~~~~~~~~~~~~~~~~
+
++--------------------------+------------+--------------------+---------------------+---------+
+| Function name | Arity | Input types | Output type | Notes |
++==========================+============+====================+=====================+=========+
+| ascii_length | Unary | String-like | Int32 or Int64 | \(1) |
++--------------------------+------------+--------------------+---------------------+---------+
+| ascii_lower | Unary | String-like | String-like | \(2) |
++--------------------------+------------+--------------------+---------------------+---------+
+| ascii_upper | Unary | String-like | String-like | \(2) |
++--------------------------+------------+--------------------+---------------------+---------+
+| utf8_lower | Unary | String-like | String-like | \(3) |
++--------------------------+------------+--------------------+---------------------+---------+
+| utf8_upper | Unary | String-like | String-like | \(3) |
++--------------------------+------------+--------------------+---------------------+---------+
+
+* \(1) Output is the physical length in bytes of each input element.
+
+* \(2) Each ASCII character in the input is converted to lowercase or
+ uppercase. Non-ASCII characters are left untouched.
+
+* \(3) Each UTF8-encoded character in the input is converted to lowercase or
+ uppercase.
+
+Containment tests
+~~~~~~~~~~~~~~~~~
+
++--------------------------+------------+----------------------------------+-----------------------+--------------------------------------------+
+| Function name | Arity | Input types | Output type | Options class |
++==========================+============+==================================+=======================+============================================+
+| binary_contains_exact | Unary | String-like | Boolean (1) | :struct:`BinaryContainsExactOptions` |
++--------------------------+------------+----------------------------------+-----------------------+--------------------------------------------+
+| isin | Unary | Binary- and String-like | Boolean (2) | :struct:`SetLookupOptions` |
++--------------------------+------------+----------------------------------+-----------------------+--------------------------------------------+
+| isin | Unary | Null | Boolean (2) | :struct:`SetLookupOptions` |
++--------------------------+------------+----------------------------------+-----------------------+--------------------------------------------+
+| isin | Unary | Boolean,Numeric, Temporal | Boolean (2) | :struct:`SetLookupOptions` |
++--------------------------+------------+----------------------------------+-----------------------+--------------------------------------------+
+| match | Unary | Binary- and String-like | Int32 (3) | :struct:`SetLookupOptions` |
++--------------------------+------------+----------------------------------+-----------------------+--------------------------------------------+
+| match | Unary | Null | Int32 (3) | :struct:`SetLookupOptions` |
++--------------------------+------------+----------------------------------+-----------------------+--------------------------------------------+
+| match | Unary | Boolean,Numeric, Temporal | Int32 (3) | :struct:`SetLookupOptions` |
++--------------------------+------------+----------------------------------+-----------------------+--------------------------------------------+
+
+* \(1) Output is true iff :member:`BinaryContainsExactOptions::pattern`
+ is a substring of the corresponding input element.
+
+* \(2) Output is true iff the corresponding input element is equal to one
+ of the elements in :member:`SetLookupOptions::value_set`.
+
+* \(3) Output is the index of the corresponding input element in
+ :member:`SetLookupOptions::value_set`, if found there. Otherwise,
+ output is null.
+
+Structural transforms
+~~~~~~~~~~~~~~~~~~~~~
+
++--------------------------+------------+--------------------+---------------------+---------+
+| Function name | Arity | Input types | Output type | Notes |
++==========================+============+====================+=====================+=========+
+| is_null | Unary | Any | Boolean | \(1) |
++--------------------------+------------+--------------------+---------------------+---------+
+| is_valid | Unary | Any | Boolean | \(2) |
++--------------------------+------------+--------------------+---------------------+---------+
+| list_value_lengths | Unary | List-like | Int32 or Int64 | \(3) |
++--------------------------+------------+--------------------+---------------------+---------+
+
+* \(1) Output is true iff the corresponding input element is non-null.
+
+* \(2) Output is true iff the corresponding input element is null.
+
+* \(3) Each output element is the length of the corresponding input element
+ (null if input is null).
+
+Conversions
+~~~~~~~~~~~
+
+A general conversion function named ``cast`` is provided which accepts a large
+number of input and output types. The type to cast to can be passed in a
+:struct:`CastOptions` instance.
+
++--------------------------+------------+--------------------+-----------------------+--------------------------------------------+
+| Function name | Arity | Input types | Output type | Options class |
++==========================+============+====================+=======================+============================================+
+| cast | Unary | Many | Variable | :struct:`CastOptions` |
++--------------------------+------------+--------------------+-----------------------+--------------------------------------------+
+| strptime | Unary | String-like | Timestamp | :struct:`StrptimeOptions` |
++--------------------------+------------+--------------------+-----------------------+--------------------------------------------+
+
+The conversions available with ``cast`` are listed below. In all cases, a
+null input value is converted into a null output value.
+
+**Truth value extraction**
+
++-----------------------------+------------------------------------+--------------+
+| Input type | Output type | Notes |
++=============================+====================================+==============+
+| Binary- and String-like | Boolean | \(1) |
++-----------------------------+------------------------------------+--------------+
+| Numeric | Boolean | \(2) |
++-----------------------------+------------------------------------+--------------+
+
+* \(1) Output is true iff the corresponding input value has non-zero length.
+
+* \(2) Output is true iff the corresponding input value is non-zero.
+
+**Same-kind conversion**
+
++-----------------------------+------------------------------------+--------------+
+| Input type | Output type | Notes |
++=============================+====================================+==============+
+| Int32 | 32-bit Temporal | \(1) |
++-----------------------------+------------------------------------+--------------+
+| Int64 | 64-bit Temporal | \(1) |
++-----------------------------+------------------------------------+--------------+
+| (Large)Binary | (Large)String | \(2) |
++-----------------------------+------------------------------------+--------------+
+| (Large)String | (Large)Binary | \(3) |
++-----------------------------+------------------------------------+--------------+
+| Numeric | Numeric | \(4) \(5) |
++-----------------------------+------------------------------------+--------------+
+| 32-bit Temporal | Int32 | \(1) |
++-----------------------------+------------------------------------+--------------+
+| 64-bit Temporal | Int64 | \(1) |
++-----------------------------+------------------------------------+--------------+
+| Temporal | Temporal | \(4) \(5) |
++-----------------------------+------------------------------------+--------------+
+
+* \(1) No-operation cast: the raw values are kept identical, only
+ the type is changed.
+
+* \(2) Validates the contents if :member:`CastOptions::allow_invalid_utf8`
+ is false.
+
+* \(3) No-operation cast: only the type is changed.
+
+* \(4) Overflow and truncation checks are enabled depending on
+ the given :struct:`CastOptions`.
+
+* \(5) Not all such casts have been implemented.
+
+**String representations**
+
++-----------------------------+------------------------------------+---------+
+| Input type | Output type | Notes |
++=============================+====================================+=========+
+| Boolean | String-like | |
++-----------------------------+------------------------------------+---------+
+| Numeric | String-like | |
++-----------------------------+------------------------------------+---------+
+
+**Generic conversions**
+
++-----------------------------+------------------------------------+---------+
+| Input type | Output type | Notes |
++=============================+====================================+=========+
+| Dictionary | Dictionary value type | |
++-----------------------------+------------------------------------+---------+
+| Extension | Extension storage type | |
++-----------------------------+------------------------------------+---------+
+| List-like | List-like | \(1) |
++-----------------------------+------------------------------------+---------+
+| Null | Any | |
++-----------------------------+------------------------------------+---------+
+
+* \(1) The list offsets are unchanged, the list values are cast from the
+ input value type to the output value type (if a conversion is
+ available).
+
+
+.. TODO: add C++ cast example
Review comment:
```suggestion
C++ cast example::
std::shared_ptr<arrow::StringArray> inputs = ...;
ARROW_ASSIGN_OR_RAISE(arrow::Datum converted, arrow::compute::Cast(inputs, arrow::int32()));
auto inputs32 = std::static_pointer_cast<arrow::Int32Array>(converted.make_array());
for (int64_t i = 0; i < inputs32.length(); ++i) {
ReportInput(inputs32.Value(i));
}
```
##########
File path: docs/source/cpp/compute.rst
##########
@@ -0,0 +1,419 @@
+.. Licensed to the Apache Software Foundation (ASF) under one
+.. or more contributor license agreements. See the NOTICE file
+.. distributed with this work for additional information
+.. regarding copyright ownership. The ASF licenses this file
+.. to you under the Apache License, Version 2.0 (the
+.. "License"); you may not use this file except in compliance
+.. with the License. You may obtain a copy of the License at
+
+.. http://www.apache.org/licenses/LICENSE-2.0
+
+.. Unless required by applicable law or agreed to in writing,
+.. software distributed under the License is distributed on an
+.. "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+.. KIND, either express or implied. See the License for the
+.. specific language governing permissions and limitations
+.. under the License.
+
+.. default-domain:: cpp
+.. highlight:: cpp
+.. cpp:namespace:: arrow::compute
+
+=================
+Compute Functions
+=================
+
+.. TODO: describe API and how to invoke compute functions
+
+Available functions
+===================
+
+Aggregations
+------------
+
++--------------------------+------------+--------------------+-----------------------+--------------------------------------------+
+| Function name | Arity | Input types | Output type | Options class |
++==========================+============+====================+=======================+============================================+
+| count | Unary | Any | Scalar Int64 | :struct:`CountOptions` |
++--------------------------+------------+--------------------+-----------------------+--------------------------------------------+
+| mean | Unary | Numeric | Scalar Float64 | |
++--------------------------+------------+--------------------+-----------------------+--------------------------------------------+
+| minmax | Unary | Numeric | Scalar Struct (1) | :struct:`MinMaxOptions` |
++--------------------------+------------+--------------------+-----------------------+--------------------------------------------+
+| sum | Unary | Numeric | Scalar Numeric (2) | |
++--------------------------+------------+--------------------+-----------------------+--------------------------------------------+
+
+Notes:
+
+* \(1) Output is a ``{"min": input type, "max": input type}`` Struct
+
+* \(2) Output is Int64, UInt64 or Float64, depending on the input type
+
+
+Element-wise ("scalar") functions
+---------------------------------
+
+Arithmetic functions
+~~~~~~~~~~~~~~~~~~~~
+
+Those functions expect two inputs of the same type and apply a given binary
+operation to each pair of elements gathered from the inputs. Each function
+is also available in an overflow-checking variant, suffixed ``_checked``.
+
+If any of the input elements in a pair is null, the corresponding output
+element is null.
+
++--------------------------+------------+--------------------+---------------------+
+| Function name | Arity | Input types | Output type |
++==========================+============+====================+=====================+
+| add | Binary | Numeric | Numeric |
++--------------------------+------------+--------------------+---------------------+
+| add_checked | Binary | Numeric | Numeric |
++--------------------------+------------+--------------------+---------------------+
+| multiply | Binary | Numeric | Numeric |
++--------------------------+------------+--------------------+---------------------+
+| multiply_checked | Binary | Numeric | Numeric |
++--------------------------+------------+--------------------+---------------------+
+| subtract | Binary | Numeric | Numeric |
++--------------------------+------------+--------------------+---------------------+
+| subtract_checked | Binary | Numeric | Numeric |
++--------------------------+------------+--------------------+---------------------+
+
+Comparisons
+~~~~~~~~~~~
+
+Those functions expect two inputs of the same type and apply a given
+comparison operator. If any of the input elements in a pair is null,
+the corresponding output element is null.
+
++--------------------------+------------+---------------------------------+---------------------+
+| Function names | Arity | Input types | Output type |
++==========================+============+=================================+=====================+
+| equal, not_equal | Binary | Numeric | Boolean |
++--------------------------+------------+---------------------------------+---------------------+
+| equal, not_equal | Binary | Binary- and String-like | Boolean |
++--------------------------+------------+---------------------------------+---------------------+
+| equal, not_equal | Binary | Temporal | Boolean |
++--------------------------+------------+---------------------------------+---------------------+
+| greater, greater_equal, | Binary | Numeric | Boolean |
+| less, less_equal | | | |
++--------------------------+------------+---------------------------------+---------------------+
+| greater, greater_equal, | Binary | Binary- and String-like | Boolean |
+| less, less_equal | | | |
++--------------------------+------------+---------------------------------+---------------------+
+| greater, greater_equal, | Binary | Temporal | Boolean |
+| less, less_equal | | | |
++--------------------------+------------+---------------------------------+---------------------+
+
+Logical functions
+~~~~~~~~~~~~~~~~~~
+
+The normal behaviour for these functions is to emit a null if any of the
+inputs is null.
+
+Some of them are also available in a "`Kleene logic`_" variant (suffixed
+``_kleene``) where null is taken to mean "undefined". For those variants
Review comment:
```suggestion
``_kleene``) where null is taken to mean "undefined". (This is the interpretation of null
used in ``R`` and ``SQL``, for example.) For those variants
```
##########
File path: docs/source/cpp/compute.rst
##########
@@ -0,0 +1,419 @@
+.. Licensed to the Apache Software Foundation (ASF) under one
+.. or more contributor license agreements. See the NOTICE file
+.. distributed with this work for additional information
+.. regarding copyright ownership. The ASF licenses this file
+.. to you under the Apache License, Version 2.0 (the
+.. "License"); you may not use this file except in compliance
+.. with the License. You may obtain a copy of the License at
+
+.. http://www.apache.org/licenses/LICENSE-2.0
+
+.. Unless required by applicable law or agreed to in writing,
+.. software distributed under the License is distributed on an
+.. "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+.. KIND, either express or implied. See the License for the
+.. specific language governing permissions and limitations
+.. under the License.
+
+.. default-domain:: cpp
+.. highlight:: cpp
+.. cpp:namespace:: arrow::compute
+
+=================
+Compute Functions
+=================
+
+.. TODO: describe API and how to invoke compute functions
+
+Available functions
+===================
+
+Aggregations
+------------
+
++--------------------------+------------+--------------------+-----------------------+--------------------------------------------+
+| Function name | Arity | Input types | Output type | Options class |
++==========================+============+====================+=======================+============================================+
+| count | Unary | Any | Scalar Int64 | :struct:`CountOptions` |
++--------------------------+------------+--------------------+-----------------------+--------------------------------------------+
+| mean | Unary | Numeric | Scalar Float64 | |
++--------------------------+------------+--------------------+-----------------------+--------------------------------------------+
+| minmax | Unary | Numeric | Scalar Struct (1) | :struct:`MinMaxOptions` |
++--------------------------+------------+--------------------+-----------------------+--------------------------------------------+
+| sum | Unary | Numeric | Scalar Numeric (2) | |
++--------------------------+------------+--------------------+-----------------------+--------------------------------------------+
+
+Notes:
+
+* \(1) Output is a ``{"min": input type, "max": input type}`` Struct
+
+* \(2) Output is Int64, UInt64 or Float64, depending on the input type
+
+
+Element-wise ("scalar") functions
+---------------------------------
+
+Arithmetic functions
+~~~~~~~~~~~~~~~~~~~~
+
+Those functions expect two inputs of the same type and apply a given binary
+operation to each pair of elements gathered from the inputs. Each function
+is also available in an overflow-checking variant, suffixed ``_checked``.
Review comment:
```suggestion
operation to each pair of elements gathered from the inputs. Integer overflow is
handled by wrapping; the sum of ``1`` and the maximum value will evaluate to the
minimum value for any integer type. Each function is also available in an overflow-
checking variant which raises an error if overflow would occur, suffixed ``_checked``.
```
##########
File path: docs/source/cpp/compute.rst
##########
@@ -0,0 +1,419 @@
+.. Licensed to the Apache Software Foundation (ASF) under one
+.. or more contributor license agreements. See the NOTICE file
+.. distributed with this work for additional information
+.. regarding copyright ownership. The ASF licenses this file
+.. to you under the Apache License, Version 2.0 (the
+.. "License"); you may not use this file except in compliance
+.. with the License. You may obtain a copy of the License at
+
+.. http://www.apache.org/licenses/LICENSE-2.0
+
+.. Unless required by applicable law or agreed to in writing,
+.. software distributed under the License is distributed on an
+.. "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+.. KIND, either express or implied. See the License for the
+.. specific language governing permissions and limitations
+.. under the License.
+
+.. default-domain:: cpp
+.. highlight:: cpp
+.. cpp:namespace:: arrow::compute
+
+=================
+Compute Functions
+=================
+
+.. TODO: describe API and how to invoke compute functions
+
+Available functions
+===================
+
+Aggregations
+------------
+
++--------------------------+------------+--------------------+-----------------------+--------------------------------------------+
+| Function name | Arity | Input types | Output type | Options class |
++==========================+============+====================+=======================+============================================+
+| count | Unary | Any | Scalar Int64 | :struct:`CountOptions` |
++--------------------------+------------+--------------------+-----------------------+--------------------------------------------+
+| mean | Unary | Numeric | Scalar Float64 | |
++--------------------------+------------+--------------------+-----------------------+--------------------------------------------+
+| minmax | Unary | Numeric | Scalar Struct (1) | :struct:`MinMaxOptions` |
++--------------------------+------------+--------------------+-----------------------+--------------------------------------------+
+| sum | Unary | Numeric | Scalar Numeric (2) | |
++--------------------------+------------+--------------------+-----------------------+--------------------------------------------+
+
+Notes:
+
+* \(1) Output is a ``{"min": input type, "max": input type}`` Struct
+
+* \(2) Output is Int64, UInt64 or Float64, depending on the input type
+
+
+Element-wise ("scalar") functions
+---------------------------------
+
+Arithmetic functions
+~~~~~~~~~~~~~~~~~~~~
+
+Those functions expect two inputs of the same type and apply a given binary
+operation to each pair of elements gathered from the inputs. Each function
+is also available in an overflow-checking variant, suffixed ``_checked``.
+
+If any of the input elements in a pair is null, the corresponding output
+element is null.
+
++--------------------------+------------+--------------------+---------------------+
+| Function name | Arity | Input types | Output type |
++==========================+============+====================+=====================+
+| add | Binary | Numeric | Numeric |
++--------------------------+------------+--------------------+---------------------+
+| add_checked | Binary | Numeric | Numeric |
++--------------------------+------------+--------------------+---------------------+
+| multiply | Binary | Numeric | Numeric |
++--------------------------+------------+--------------------+---------------------+
+| multiply_checked | Binary | Numeric | Numeric |
++--------------------------+------------+--------------------+---------------------+
+| subtract | Binary | Numeric | Numeric |
++--------------------------+------------+--------------------+---------------------+
+| subtract_checked | Binary | Numeric | Numeric |
++--------------------------+------------+--------------------+---------------------+
+
+Comparisons
+~~~~~~~~~~~
+
+Those functions expect two inputs of the same type and apply a given
+comparison operator. If any of the input elements in a pair is null,
+the corresponding output element is null.
+
++--------------------------+------------+---------------------------------+---------------------+
+| Function names | Arity | Input types | Output type |
++==========================+============+=================================+=====================+
+| equal, not_equal | Binary | Numeric | Boolean |
++--------------------------+------------+---------------------------------+---------------------+
+| equal, not_equal | Binary | Binary- and String-like | Boolean |
++--------------------------+------------+---------------------------------+---------------------+
+| equal, not_equal | Binary | Temporal | Boolean |
++--------------------------+------------+---------------------------------+---------------------+
+| greater, greater_equal, | Binary | Numeric | Boolean |
+| less, less_equal | | | |
++--------------------------+------------+---------------------------------+---------------------+
+| greater, greater_equal, | Binary | Binary- and String-like | Boolean |
+| less, less_equal | | | |
++--------------------------+------------+---------------------------------+---------------------+
+| greater, greater_equal, | Binary | Temporal | Boolean |
+| less, less_equal | | | |
++--------------------------+------------+---------------------------------+---------------------+
+
+Logical functions
+~~~~~~~~~~~~~~~~~~
+
+The normal behaviour for these functions is to emit a null if any of the
+inputs is null.
+
+Some of them are also available in a "`Kleene logic`_" variant (suffixed
+``_kleene``) where null is taken to mean "undefined". For those variants
+therefore:
+
+* "true AND null", "null AND true" give "null" (the result is undefined)
+* "true OR null", "null OR true" give "true"
+* "false AND null", "null AND false" give "false"
+* "false OR null", "null OR false" give "null" (the result is undefined)
+
++--------------------------+------------+--------------------+---------------------+
+| Function name | Arity | Input types | Output type |
++==========================+============+====================+=====================+
+| and | Binary | Boolean | Boolean |
++--------------------------+------------+--------------------+---------------------+
+| and_kleene | Binary | Boolean | Boolean |
++--------------------------+------------+--------------------+---------------------+
+| invert | Unary | Boolean | Boolean |
++--------------------------+------------+--------------------+---------------------+
+| or | Binary | Boolean | Boolean |
++--------------------------+------------+--------------------+---------------------+
+| or_kleene | Binary | Boolean | Boolean |
++--------------------------+------------+--------------------+---------------------+
+| xor | Binary | Boolean | Boolean |
++--------------------------+------------+--------------------+---------------------+
+
+.. _Kleene logic: https://en.wikipedia.org/wiki/Three-valued_logic#Kleene_and_Priest_logics
+
+String functions
+~~~~~~~~~~~~~~~~
+
++--------------------------+------------+--------------------+---------------------+---------+
+| Function name | Arity | Input types | Output type | Notes |
++==========================+============+====================+=====================+=========+
+| ascii_length | Unary | String-like | Int32 or Int64 | \(1) |
++--------------------------+------------+--------------------+---------------------+---------+
+| ascii_lower | Unary | String-like | String-like | \(2) |
++--------------------------+------------+--------------------+---------------------+---------+
+| ascii_upper | Unary | String-like | String-like | \(2) |
++--------------------------+------------+--------------------+---------------------+---------+
+| utf8_lower | Unary | String-like | String-like | \(3) |
++--------------------------+------------+--------------------+---------------------+---------+
+| utf8_upper | Unary | String-like | String-like | \(3) |
++--------------------------+------------+--------------------+---------------------+---------+
+
+* \(1) Output is the physical length in bytes of each input element.
+
+* \(2) Each ASCII character in the input is converted to lowercase or
+ uppercase. Non-ASCII characters are left untouched.
+
+* \(3) Each UTF8-encoded character in the input is converted to lowercase or
+ uppercase.
+
+Containment tests
+~~~~~~~~~~~~~~~~~
+
++--------------------------+------------+----------------------------------+-----------------------+--------------------------------------------+
+| Function name | Arity | Input types | Output type | Options class |
++==========================+============+==================================+=======================+============================================+
+| binary_contains_exact | Unary | String-like | Boolean (1) | :struct:`BinaryContainsExactOptions` |
++--------------------------+------------+----------------------------------+-----------------------+--------------------------------------------+
+| isin | Unary | Binary- and String-like | Boolean (2) | :struct:`SetLookupOptions` |
++--------------------------+------------+----------------------------------+-----------------------+--------------------------------------------+
+| isin | Unary | Null | Boolean (2) | :struct:`SetLookupOptions` |
++--------------------------+------------+----------------------------------+-----------------------+--------------------------------------------+
+| isin | Unary | Boolean,Numeric, Temporal | Boolean (2) | :struct:`SetLookupOptions` |
++--------------------------+------------+----------------------------------+-----------------------+--------------------------------------------+
+| match | Unary | Binary- and String-like | Int32 (3) | :struct:`SetLookupOptions` |
++--------------------------+------------+----------------------------------+-----------------------+--------------------------------------------+
+| match | Unary | Null | Int32 (3) | :struct:`SetLookupOptions` |
++--------------------------+------------+----------------------------------+-----------------------+--------------------------------------------+
+| match | Unary | Boolean,Numeric, Temporal | Int32 (3) | :struct:`SetLookupOptions` |
++--------------------------+------------+----------------------------------+-----------------------+--------------------------------------------+
+
+* \(1) Output is true iff :member:`BinaryContainsExactOptions::pattern`
+ is a substring of the corresponding input element.
+
+* \(2) Output is true iff the corresponding input element is equal to one
+ of the elements in :member:`SetLookupOptions::value_set`.
+
+* \(3) Output is the index of the corresponding input element in
+ :member:`SetLookupOptions::value_set`, if found there. Otherwise,
+ output is null.
+
+Structural transforms
+~~~~~~~~~~~~~~~~~~~~~
+
++--------------------------+------------+--------------------+---------------------+---------+
+| Function name | Arity | Input types | Output type | Notes |
++==========================+============+====================+=====================+=========+
+| is_null | Unary | Any | Boolean | \(1) |
++--------------------------+------------+--------------------+---------------------+---------+
+| is_valid | Unary | Any | Boolean | \(2) |
++--------------------------+------------+--------------------+---------------------+---------+
+| list_value_lengths | Unary | List-like | Int32 or Int64 | \(3) |
++--------------------------+------------+--------------------+---------------------+---------+
+
+* \(1) Output is true iff the corresponding input element is non-null.
+
+* \(2) Output is true iff the corresponding input element is null.
+
+* \(3) Each output element is the length of the corresponding input element
+ (null if input is null).
+
+Conversions
+~~~~~~~~~~~
+
+A general conversion function named ``cast`` is provided which accepts a large
+number of input and output types. The type to cast to can be passed in a
+:struct:`CastOptions` instance.
+
++--------------------------+------------+--------------------+-----------------------+--------------------------------------------+
+| Function name | Arity | Input types | Output type | Options class |
++==========================+============+====================+=======================+============================================+
+| cast | Unary | Many | Variable | :struct:`CastOptions` |
++--------------------------+------------+--------------------+-----------------------+--------------------------------------------+
+| strptime | Unary | String-like | Timestamp | :struct:`StrptimeOptions` |
++--------------------------+------------+--------------------+-----------------------+--------------------------------------------+
+
+The conversions available with ``cast`` are listed below. In all cases, a
+null input value is converted into a null output value.
+
+**Truth value extraction**
+
++-----------------------------+------------------------------------+--------------+
+| Input type | Output type | Notes |
++=============================+====================================+==============+
+| Binary- and String-like | Boolean | \(1) |
++-----------------------------+------------------------------------+--------------+
+| Numeric | Boolean | \(2) |
++-----------------------------+------------------------------------+--------------+
+
+* \(1) Output is true iff the corresponding input value has non-zero length.
+
+* \(2) Output is true iff the corresponding input value is non-zero.
+
+**Same-kind conversion**
+
++-----------------------------+------------------------------------+--------------+
+| Input type | Output type | Notes |
++=============================+====================================+==============+
+| Int32 | 32-bit Temporal | \(1) |
++-----------------------------+------------------------------------+--------------+
+| Int64 | 64-bit Temporal | \(1) |
++-----------------------------+------------------------------------+--------------+
+| (Large)Binary | (Large)String | \(2) |
++-----------------------------+------------------------------------+--------------+
+| (Large)String | (Large)Binary | \(3) |
++-----------------------------+------------------------------------+--------------+
+| Numeric | Numeric | \(4) \(5) |
++-----------------------------+------------------------------------+--------------+
+| 32-bit Temporal | Int32 | \(1) |
++-----------------------------+------------------------------------+--------------+
+| 64-bit Temporal | Int64 | \(1) |
++-----------------------------+------------------------------------+--------------+
+| Temporal | Temporal | \(4) \(5) |
++-----------------------------+------------------------------------+--------------+
+
+* \(1) No-operation cast: the raw values are kept identical, only
+ the type is changed.
+
+* \(2) Validates the contents if :member:`CastOptions::allow_invalid_utf8`
+ is false.
+
+* \(3) No-operation cast: only the type is changed.
+
+* \(4) Overflow and truncation checks are enabled depending on
+ the given :struct:`CastOptions`.
+
+* \(5) Not all such casts have been implemented.
+
+**String representations**
+
++-----------------------------+------------------------------------+---------+
+| Input type | Output type | Notes |
++=============================+====================================+=========+
+| Boolean | String-like | |
++-----------------------------+------------------------------------+---------+
+| Numeric | String-like | |
++-----------------------------+------------------------------------+---------+
+
+**Generic conversions**
+
++-----------------------------+------------------------------------+---------+
+| Input type | Output type | Notes |
++=============================+====================================+=========+
+| Dictionary | Dictionary value type | |
++-----------------------------+------------------------------------+---------+
+| Extension | Extension storage type | |
++-----------------------------+------------------------------------+---------+
+| List-like | List-like | \(1) |
++-----------------------------+------------------------------------+---------+
+| Null | Any | |
++-----------------------------+------------------------------------+---------+
+
+* \(1) The list offsets are unchanged, the list values are cast from the
+ input value type to the output value type (if a conversion is
+ available).
+
+
+.. TODO: add C++ cast example
+
+Array-wise ("vector") functions
+-------------------------------
+
+Associative transforms
+~~~~~~~~~~~~~~~~~~~~~~
+
++--------------------------+------------+----------------------------+----------------------------+
+| Function name | Arity | Input types | Output type |
++==========================+============+============================+============================+
+| dictionary_encode | Unary | Binary- and String-like | Dictionary (1) |
Review comment:
dictionary_encode infers the dictionary's type from the argument and always uses int32 indices. Cast requires that you specify the destination type
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org