You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@arrow.apache.org by GitBox <gi...@apache.org> on 2020/06/09 17:24:18 UTC
[GitHub] [arrow] nealrichardson commented on a change in pull request #7350: ARROW-6602: [Doc] Add a feature/implementation matrix
nealrichardson commented on a change in pull request #7350:
URL: https://github.com/apache/arrow/pull/7350#discussion_r437592494
##########
File path: README.md
##########
@@ -32,16 +32,24 @@ set of technologies that enable big data systems to process and move data fast.
Major components of the project include:
- - [The Arrow Columnar In-Memory Format](https://github.com/apache/arrow/tree/master/format)
+ - [The Arrow Columnar In-Memory Format](https://github.com/apache/arrow/tree/master/format):
+ a standard and efficient in-memory representation of various datatypes, plain or nested
+ - [The Arrow IPC Format](https://github.com/apache/arrow/tree/master/format):
Review comment:
Can/should this link to something more specific? It's the same link as the previous line.
##########
File path: docs/source/status.rst
##########
@@ -0,0 +1,199 @@
+.. Licensed to the Apache Software Foundation (ASF) under one
+.. or more contributor license agreements. See the NOTICE file
+.. distributed with this work for additional information
+.. regarding copyright ownership. The ASF licenses this file
+.. to you under the Apache License, Version 2.0 (the
+.. "License"); you may not use this file except in compliance
+.. with the License. You may obtain a copy of the License at
+
+.. http://www.apache.org/licenses/LICENSE-2.0
+
+.. Unless required by applicable law or agreed to in writing,
+.. software distributed under the License is distributed on an
+.. "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+.. KIND, either express or implied. See the License for the
+.. specific language governing permissions and limitations
+.. under the License.
+
+=====================
+Implementation Status
+=====================
+
+The following tables summarize the features available in the various official
+Arrow libraries. Unless otherwise stated, the Python, R, Ruby and C/GLib
+libraries follow the C++ Arrow library.
+
+Data Types
+==========
+
++-------------------+-------+-------+-------+------------+-------+-------+
+| Data type | C++ | Java | Go | JavaScript | C# | Rust |
+| (primitive) | | | | | | |
++===================+=======+=======+=======+============+=======+=======+
+| Null | ✓ | ✓ | | | | ✓ |
++-------------------+-------+-------+-------+------------+-------+-------+
+| Boolean | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ |
++-------------------+-------+-------+-------+------------+-------+-------+
+| Int8/16/32/64 | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ |
++-------------------+-------+-------+-------+------------+-------+-------+
+| UInt8/16/32/64 | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ |
++-------------------+-------+-------+-------+------------+-------+-------+
+| Float16 | | | ✓ | | | |
++-------------------+-------+-------+-------+------------+-------+-------+
+| Float32/64 | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ |
++-------------------+-------+-------+-------+------------+-------+-------+
+| Decimal128 | ✓ | ✓ | | | | |
++-------------------+-------+-------+-------+------------+-------+-------+
+| Date32/64 | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ |
++-------------------+-------+-------+-------+------------+-------+-------+
+| Time32/64 | ✓ | ✓ | ✓ | ✓ | | ✓ |
++-------------------+-------+-------+-------+------------+-------+-------+
+| Timestamp | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ |
++-------------------+-------+-------+-------+------------+-------+-------+
+| Duration | ✓ | ✓ | ✓ | | | ✓ |
++-------------------+-------+-------+-------+------------+-------+-------+
+| Interval | ✓ | ✓ | ✓ | | | ✓ |
++-------------------+-------+-------+-------+------------+-------+-------+
+| Fixed Size Binary | ✓ | ✓ | ✓ | ✓ | | ✓ |
++-------------------+-------+-------+-------+------------+-------+-------+
+| Binary | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ |
++-------------------+-------+-------+-------+------------+-------+-------+
+| Large Binary | ✓ | ✓ | ✓ | ✓ | | |
Review comment:
R doesn't have bindings for the Large* types yet (ARROW-6543)
##########
File path: docs/source/status.rst
##########
@@ -0,0 +1,199 @@
+.. Licensed to the Apache Software Foundation (ASF) under one
+.. or more contributor license agreements. See the NOTICE file
+.. distributed with this work for additional information
+.. regarding copyright ownership. The ASF licenses this file
+.. to you under the Apache License, Version 2.0 (the
+.. "License"); you may not use this file except in compliance
+.. with the License. You may obtain a copy of the License at
+
+.. http://www.apache.org/licenses/LICENSE-2.0
+
+.. Unless required by applicable law or agreed to in writing,
+.. software distributed under the License is distributed on an
+.. "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+.. KIND, either express or implied. See the License for the
+.. specific language governing permissions and limitations
+.. under the License.
+
+=====================
+Implementation Status
+=====================
+
+The following tables summarize the features available in the various official
+Arrow libraries. Unless otherwise stated, the Python, R, Ruby and C/GLib
+libraries follow the C++ Arrow library.
+
+Data Types
+==========
+
++-------------------+-------+-------+-------+------------+-------+-------+
+| Data type | C++ | Java | Go | JavaScript | C# | Rust |
+| (primitive) | | | | | | |
++===================+=======+=======+=======+============+=======+=======+
+| Null | ✓ | ✓ | | | | ✓ |
++-------------------+-------+-------+-------+------------+-------+-------+
+| Boolean | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ |
++-------------------+-------+-------+-------+------------+-------+-------+
+| Int8/16/32/64 | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ |
++-------------------+-------+-------+-------+------------+-------+-------+
+| UInt8/16/32/64 | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ |
++-------------------+-------+-------+-------+------------+-------+-------+
+| Float16 | | | ✓ | | | |
++-------------------+-------+-------+-------+------------+-------+-------+
+| Float32/64 | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ |
++-------------------+-------+-------+-------+------------+-------+-------+
+| Decimal128 | ✓ | ✓ | | | | |
++-------------------+-------+-------+-------+------------+-------+-------+
+| Date32/64 | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ |
++-------------------+-------+-------+-------+------------+-------+-------+
+| Time32/64 | ✓ | ✓ | ✓ | ✓ | | ✓ |
++-------------------+-------+-------+-------+------------+-------+-------+
+| Timestamp | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ |
++-------------------+-------+-------+-------+------------+-------+-------+
+| Duration | ✓ | ✓ | ✓ | | | ✓ |
++-------------------+-------+-------+-------+------------+-------+-------+
+| Interval | ✓ | ✓ | ✓ | | | ✓ |
++-------------------+-------+-------+-------+------------+-------+-------+
+| Fixed Size Binary | ✓ | ✓ | ✓ | ✓ | | ✓ |
++-------------------+-------+-------+-------+------------+-------+-------+
+| Binary | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ |
++-------------------+-------+-------+-------+------------+-------+-------+
+| Large Binary | ✓ | ✓ | ✓ | ✓ | | |
++-------------------+-------+-------+-------+------------+-------+-------+
+| Utf8 | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ |
++-------------------+-------+-------+-------+------------+-------+-------+
+| Large Utf8 | ✓ | ✓ | ✓ | ✓ | | |
++-------------------+-------+-------+-------+------------+-------+-------+
+
++-------------------+-------+-------+-------+------------+-------+-------+
+| Data type | C++ | Java | Go | JavaScript | C# | Rust |
+| (nested) | | | | | | |
++===================+=======+=======+=======+============+=======+=======+
+| Fixed Size List | ✓ | ✓ | ✓ | ✓ | | ✓ |
++-------------------+-------+-------+-------+------------+-------+-------+
+| List | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ |
++-------------------+-------+-------+-------+------------+-------+-------+
+| Large List | ✓ | | | | | |
++-------------------+-------+-------+-------+------------+-------+-------+
+| Struct | ✓ | ✓ | ✓ | ✓ | | ✓ |
++-------------------+-------+-------+-------+------------+-------+-------+
+| Map | ✓ | ✓ | | ✓ | | |
++-------------------+-------+-------+-------+------------+-------+-------+
+| Dense Union | ✓ | ✓ (1) | | | | ✓ |
++-------------------+-------+-------+-------+------------+-------+-------+
+| Sparse Union | ✓ | ✓ (1) | | | | ✓ (2) |
++-------------------+-------+-------+-------+------------+-------+-------+
+
++-------------------+-------+-------+-------+------------+-------+-------+
+| Data type | C++ | Java | Go | JavaScript | C# | Rust |
+| (special) | | | | | | |
++===================+=======+=======+=======+============+=======+=======+
+| Dictionary | ✓ | ✓ (3) | | ✓ (3) | | ✓ (3) |
++-------------------+-------+-------+-------+------------+-------+-------+
+| Extension | ✓ | ✓ | | | | |
++-------------------+-------+-------+-------+------------+-------+-------+
+
+Notes:
+
+* \(1) Union support in Java is currently not compliant with the specification
+
+* \(2) Union support in Rust is currently incomplete
+
+* \(3) Nested dictionaries not supported
+
+.. seealso::
+ The :ref:`format_columnar` specification.
+
+
+IPC Format
+==========
+
++-----------------------------+-------+-------+-------+------------+-------+-------+
+| IPC Feature | C++ | Java | Go | JavaScript | C# | Rust |
+| | | | | | | |
++=============================+=======+=======+=======+============+=======+=======+
+| Arrow stream format | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ |
++-----------------------------+-------+-------+-------+------------+-------+-------+
+| Arrow file format | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ |
++-----------------------------+-------+-------+-------+------------+-------+-------+
+| Record batches | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ |
++-----------------------------+-------+-------+-------+------------+-------+-------+
+| Dictionaries | ✓ | ✓ | ✓ | ✓ | | |
++-----------------------------+-------+-------+-------+------------+-------+-------+
+| Replacement dictionaries | ✓ | ✓ | | | | |
++-----------------------------+-------+-------+-------+------------+-------+-------+
+| Delta dictionaries | | | | | | |
++-----------------------------+-------+-------+-------+------------+-------+-------+
+| Tensors | ✓ | | | | | |
++-----------------------------+-------+-------+-------+------------+-------+-------+
+| Sparse tensors | ✓ | | | | | |
++-----------------------------+-------+-------+-------+------------+-------+-------+
+| Custom schema metadata | ✓ | ✓ | | | | |
++-----------------------------+-------+-------+-------+------------+-------+-------+
+
+
+Flight RPC
+==========
+
++-----------------------------+-------+-------+-------+------------+-------+-------+
+| Flight RPC Feature | C++ | Java | Go | JavaScript | C# | Rust |
+| | | | | | | |
++=============================+=======+=======+=======+============+=======+=======+
+| gRPC transport | ✓ | ✓ | | | | |
++-----------------------------+-------+-------+-------+------------+-------+-------+
+| gRPC + TLS transport | ✓ | ✓ | | | | |
++-----------------------------+-------+-------+-------+------------+-------+-------+
+| RPC error codes | ✓ | ✓ | | | | |
++-----------------------------+-------+-------+-------+------------+-------+-------+
+| Authentication handlers | ✓ | ✓ | | | | |
++-----------------------------+-------+-------+-------+------------+-------+-------+
+| Custom client middleware | ✓ | ✓ | | | | |
++-----------------------------+-------+-------+-------+------------+-------+-------+
+| Custom server middleware | ✓ | ✓ | | | | |
++-----------------------------+-------+-------+-------+------------+-------+-------+
+
+.. seealso::
+ The :ref:`flight-rpc` specification.
+
+
+C Data Interface
+================
+
++-----------------------------+-------+--------+-------+
+| Feature | C++ | Python | R |
+| | | | |
++=============================+=======+========+=======+
+| Schema export | ✓ | ✓ | ✓ |
++-----------------------------+-------+--------+-------+
+| Array export | ✓ | ✓ | ✓ |
++-----------------------------+-------+--------+-------+
+| Schema import | ✓ | ✓ | ✓ |
++-----------------------------+-------+--------+-------+
+| Array import | ✓ | ✓ | ✓ |
++-----------------------------+-------+--------+-------+
+
+.. seealso::
+ The :ref:`C Data Interface <c-data-interface>` specification.
+
+
+Third-Party Data Formats
+========================
+
++-----------------------------+---------+---------+-------+------------+-------+-------+
+| Format | C++ | Java | Go | JavaScript | C# | Rust |
+| | | | | | | |
++=============================+=========+=========+=======+============+=======+=======+
+| Avro | | R | | | | |
++-----------------------------+---------+---------+-------+------------+-------+-------+
+| CSV | R | | R/W | | | R/W |
++-----------------------------+---------+---------+-------+------------+-------+-------+
+| ORC | R | | | | | |
++-----------------------------+---------+---------+-------+------------+-------+-------+
+| Parquet | R/W | | | | | |
Review comment:
Should we note the nested type parquet limitation?
##########
File path: docs/source/status.rst
##########
@@ -0,0 +1,199 @@
+.. Licensed to the Apache Software Foundation (ASF) under one
+.. or more contributor license agreements. See the NOTICE file
+.. distributed with this work for additional information
+.. regarding copyright ownership. The ASF licenses this file
+.. to you under the Apache License, Version 2.0 (the
+.. "License"); you may not use this file except in compliance
+.. with the License. You may obtain a copy of the License at
+
+.. http://www.apache.org/licenses/LICENSE-2.0
+
+.. Unless required by applicable law or agreed to in writing,
+.. software distributed under the License is distributed on an
+.. "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+.. KIND, either express or implied. See the License for the
+.. specific language governing permissions and limitations
+.. under the License.
+
+=====================
+Implementation Status
+=====================
+
+The following tables summarize the features available in the various official
Review comment:
What is the criteria for getting a check? An implementation, or an integration test?
##########
File path: docs/source/status.rst
##########
@@ -0,0 +1,199 @@
+.. Licensed to the Apache Software Foundation (ASF) under one
+.. or more contributor license agreements. See the NOTICE file
+.. distributed with this work for additional information
+.. regarding copyright ownership. The ASF licenses this file
+.. to you under the Apache License, Version 2.0 (the
+.. "License"); you may not use this file except in compliance
+.. with the License. You may obtain a copy of the License at
+
+.. http://www.apache.org/licenses/LICENSE-2.0
+
+.. Unless required by applicable law or agreed to in writing,
+.. software distributed under the License is distributed on an
+.. "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+.. KIND, either express or implied. See the License for the
+.. specific language governing permissions and limitations
+.. under the License.
+
+=====================
+Implementation Status
+=====================
+
+The following tables summarize the features available in the various official
+Arrow libraries. Unless otherwise stated, the Python, R, Ruby and C/GLib
+libraries follow the C++ Arrow library.
+
+Data Types
+==========
+
++-------------------+-------+-------+-------+------------+-------+-------+
+| Data type | C++ | Java | Go | JavaScript | C# | Rust |
+| (primitive) | | | | | | |
++===================+=======+=======+=======+============+=======+=======+
+| Null | ✓ | ✓ | | | | ✓ |
++-------------------+-------+-------+-------+------------+-------+-------+
+| Boolean | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ |
++-------------------+-------+-------+-------+------------+-------+-------+
+| Int8/16/32/64 | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ |
++-------------------+-------+-------+-------+------------+-------+-------+
+| UInt8/16/32/64 | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ |
++-------------------+-------+-------+-------+------------+-------+-------+
+| Float16 | | | ✓ | | | |
++-------------------+-------+-------+-------+------------+-------+-------+
+| Float32/64 | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ |
++-------------------+-------+-------+-------+------------+-------+-------+
+| Decimal128 | ✓ | ✓ | | | | |
++-------------------+-------+-------+-------+------------+-------+-------+
+| Date32/64 | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ |
++-------------------+-------+-------+-------+------------+-------+-------+
+| Time32/64 | ✓ | ✓ | ✓ | ✓ | | ✓ |
++-------------------+-------+-------+-------+------------+-------+-------+
+| Timestamp | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ |
++-------------------+-------+-------+-------+------------+-------+-------+
+| Duration | ✓ | ✓ | ✓ | | | ✓ |
++-------------------+-------+-------+-------+------------+-------+-------+
+| Interval | ✓ | ✓ | ✓ | | | ✓ |
++-------------------+-------+-------+-------+------------+-------+-------+
+| Fixed Size Binary | ✓ | ✓ | ✓ | ✓ | | ✓ |
++-------------------+-------+-------+-------+------------+-------+-------+
+| Binary | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ |
Review comment:
R doesn't know Binary yet (ARROW-6235)
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org