You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@arrow.apache.org by GitBox <gi...@apache.org> on 2020/12/14 18:31:04 UTC
[GitHub] [arrow] pitrou opened a new pull request #8914: ARROW-10887: [Doc][C++] Document C++ IPC API
pitrou opened a new pull request #8914:
URL: https://github.com/apache/arrow/pull/8914
Also add some tables of contents here and there for long nested documents.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] pitrou commented on a change in pull request #8914: ARROW-10887: [Doc][C++] Document C++ IPC API
Posted by GitBox <gi...@apache.org>.
pitrou commented on a change in pull request #8914:
URL: https://github.com/apache/arrow/pull/8914#discussion_r544175713
##########
File path: docs/source/cpp/api/ipc.rst
##########
@@ -0,0 +1,91 @@
+.. Licensed to the Apache Software Foundation (ASF) under one
+.. or more contributor license agreements. See the NOTICE file
+.. distributed with this work for additional information
+.. regarding copyright ownership. The ASF licenses this file
+.. to you under the Apache License, Version 2.0 (the
+.. "License"); you may not use this file except in compliance
+.. with the License. You may obtain a copy of the License at
+
+.. http://www.apache.org/licenses/LICENSE-2.0
+
+.. Unless required by applicable law or agreed to in writing,
+.. software distributed under the License is distributed on an
+.. "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+.. KIND, either express or implied. See the License for the
+.. specific language governing permissions and limitations
+.. under the License.
+
+.. default-domain:: cpp
+.. highlight:: cpp
+
+=========
+Arrow IPC
+=========
+
+.. sidebar:: Contents
+
+ .. contents:: :local:
+
+IPC options
+===========
+
+.. doxygenstruct:: arrow::ipc::IpcReadOptions
+ :members:
+
+.. doxygenstruct:: arrow::ipc::IpcWriteOptions
+ :members:
+
+Reading IPC streams and files
+=============================
+
+Blocking API
+------------
+
+Use either of these two classes, depending on which IPC format you want
+to read. The file format requires a random-access file, while the stream
+format only requires a sequential input stream.
+
+.. doxygenclass:: arrow::ipc::RecordBatchStreamReader
+ :members:
+
+.. doxygenclass:: arrow::ipc::RecordBatchFileReader
+ :members:
+
+Event-driven API
+----------------
+
+To read an IPC stream in event-driven fashion, you must implement a
+:class:`~arrow::ipc::Listener` subclass that you will pass to
+:class:`~arrow::ipc::StreamDecoder`.
+
+.. doxygenclass:: arrow::ipc::Listener
+ :members:
+
+.. doxygenclass:: arrow::ipc::StreamDecoder
+ :members:
+
+Statistics
+----------
+
+.. doxygenstruct:: arrow::ipc::ReadStats
+ :members:
+
+Writing IPC streams and files
+=============================
+
+Blocking API
+------------
Review comment:
Neat suggestion, thank you.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] github-actions[bot] commented on pull request #8914: ARROW-10887: [Doc][C++] Document C++ IPC API
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on pull request #8914:
URL: https://github.com/apache/arrow/pull/8914#issuecomment-744643673
https://issues.apache.org/jira/browse/ARROW-10887
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] pitrou closed pull request #8914: ARROW-10887: [Doc][C++] Document C++ IPC API
Posted by GitBox <gi...@apache.org>.
pitrou closed pull request #8914:
URL: https://github.com/apache/arrow/pull/8914
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] bkietz commented on a change in pull request #8914: ARROW-10887: [Doc][C++] Document C++ IPC API
Posted by GitBox <gi...@apache.org>.
bkietz commented on a change in pull request #8914:
URL: https://github.com/apache/arrow/pull/8914#discussion_r543674843
##########
File path: docs/source/cpp/api/ipc.rst
##########
@@ -0,0 +1,91 @@
+.. Licensed to the Apache Software Foundation (ASF) under one
+.. or more contributor license agreements. See the NOTICE file
+.. distributed with this work for additional information
+.. regarding copyright ownership. The ASF licenses this file
+.. to you under the Apache License, Version 2.0 (the
+.. "License"); you may not use this file except in compliance
+.. with the License. You may obtain a copy of the License at
+
+.. http://www.apache.org/licenses/LICENSE-2.0
+
+.. Unless required by applicable law or agreed to in writing,
+.. software distributed under the License is distributed on an
+.. "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+.. KIND, either express or implied. See the License for the
+.. specific language governing permissions and limitations
+.. under the License.
+
+.. default-domain:: cpp
+.. highlight:: cpp
+
+=========
+Arrow IPC
+=========
+
+.. sidebar:: Contents
+
+ .. contents:: :local:
+
+IPC options
+===========
+
+.. doxygenstruct:: arrow::ipc::IpcReadOptions
+ :members:
+
+.. doxygenstruct:: arrow::ipc::IpcWriteOptions
+ :members:
+
+Reading IPC streams and files
+=============================
+
+Blocking API
+------------
+
+Use either of these two classes, depending on which IPC format you want
+to read. The file format requires a random-access file, while the stream
+format only requires a sequential input stream.
+
+.. doxygenclass:: arrow::ipc::RecordBatchStreamReader
+ :members:
+
+.. doxygenclass:: arrow::ipc::RecordBatchFileReader
+ :members:
+
+Event-driven API
+----------------
+
+To read an IPC stream in event-driven fashion, you must implement a
+:class:`~arrow::ipc::Listener` subclass that you will pass to
+:class:`~arrow::ipc::StreamDecoder`.
+
+.. doxygenclass:: arrow::ipc::Listener
+ :members:
+
+.. doxygenclass:: arrow::ipc::StreamDecoder
+ :members:
+
+Statistics
+----------
+
+.. doxygenstruct:: arrow::ipc::ReadStats
+ :members:
+
+Writing IPC streams and files
+=============================
+
+Blocking API
+------------
Review comment:
```suggestion
------------
The IPC stream format is only optionally terminated, whereas the IPC file format
must include a terminating footer. Thus a writer of the IPC file format must be
explicitly finalized with :func:`arrow::RecordBatchWriter::Close()` or the resulting
file will be corrupt.
```
##########
File path: docs/source/cpp/ipc.rst
##########
@@ -0,0 +1,75 @@
+.. Licensed to the Apache Software Foundation (ASF) under one
+.. or more contributor license agreements. See the NOTICE file
+.. distributed with this work for additional information
+.. regarding copyright ownership. The ASF licenses this file
+.. to you under the Apache License, Version 2.0 (the
+.. "License"); you may not use this file except in compliance
+.. with the License. You may obtain a copy of the License at
+
+.. http://www.apache.org/licenses/LICENSE-2.0
+
+.. Unless required by applicable law or agreed to in writing,
+.. software distributed under the License is distributed on an
+.. "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+.. KIND, either express or implied. See the License for the
+.. specific language governing permissions and limitations
+.. under the License.
+
+.. default-domain:: cpp
+.. highlight:: cpp
+.. cpp:namespace:: arrow::ipc
+
+========================================
+Reading and writing the Arrow IPC format
+========================================
+
+.. seealso::
+ :ref:`Arrow IPC format specification <format-ipc>`.
+
+ :doc:`API reference for IPC readers and writers <api/ipc>`.
+
+Arrow C++ provides abstracted readers and writers for the Arrow IPC format.
+Actual input/output is handled through the :doc:`IO interfaces <io>`.
+For reading, there is also an event-driven API that allows to feed arbitrary
+data into the IPC decoding layer.
Review comment:
```suggestion
Arrow C++ provides readers and writers for the Arrow IPC format which wrap
lower level input/output, handled through the :doc:`IO interfaces <io>`.
For reading, there is also an event-driven API that enables feeding
arbitrary data into the IPC decoding layer asynchronously.
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org