You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@arrow.apache.org by "zeroshade (via GitHub)" <gi...@apache.org> on 2024/08/09 20:12:00 UTC
[PR] GH-43631: [C][Format] Add ArrowAsyncDeviceStreamHandler interface [arrow]
zeroshade opened a new pull request, #43632:
URL: https://github.com/apache/arrow/pull/43632
<!--
Thanks for opening a pull request!
If this is your first pull request you can find detailed information on how
to contribute here:
* [New Contributor's Guide](https://arrow.apache.org/docs/dev/developers/guide/step_by_step/pr_lifecycle.html#reviews-and-merge-of-the-pull-request)
* [Contributing Overview](https://arrow.apache.org/docs/dev/developers/overview.html)
If this is not a [minor PR](https://github.com/apache/arrow/blob/main/CONTRIBUTING.md#Minor-Fixes). Could you open an issue for this pull request on GitHub? https://github.com/apache/arrow/issues/new/choose
Opening GitHub issues ahead of time contributes to the [Openness](http://theapacheway.com/open/#:~:text=Openness%20allows%20new%20users%20the,must%20happen%20in%20the%20open.) of the Apache Arrow project.
Then could you also rename the pull request title in the following format?
GH-${GITHUB_ISSUE_ID}: [${COMPONENT}] ${SUMMARY}
or
MINOR: [${COMPONENT}] ${SUMMARY}
In the case of PARQUET issues on JIRA the title also supports:
PARQUET-${JIRA_ISSUE_ID}: [${COMPONENT}] ${SUMMARY}
-->
### Rationale for this change
See https://github.com/apache/arrow-adbc/issues/811 and https://github.com/apache/arrow/issues/43631
<!--
Why are you proposing this change? If this is already explained clearly in the issue then this section is not needed.
Explaining clearly why changes are proposed helps reviewers understand your changes and offer better suggestions for fixes.
-->
### What changes are included in this PR?
Definition of `ArrowAsyncDeviceStreamHandler` and addition of it to the docs.
<!--
There is no need to duplicate the description in the issue here but it is sometimes worth providing a summary of the individual changes in this PR.
-->
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-43631: [C][Format] Add ArrowAsyncDeviceStreamHandler interface [arrow]
Posted by "bkietz (via GitHub)" <gi...@apache.org>.
bkietz commented on code in PR #43632:
URL: https://github.com/apache/arrow/pull/43632#discussion_r1765870066
##########
docs/source/format/CDeviceDataInterface.rst:
##########
@@ -650,6 +652,331 @@ The stream source is not assumed to be thread-safe. Consumers wanting to
call ``get_next`` from several threads should ensure those calls are
serialized.
+Async Device Stream Interface
+=============================
+
+The :ref:`C stream interface <_c-device-stream-interface>` provides a synchronous
+API centered around the consumer calling the callback functions to retrieve
+the next record batch. For some bindings, use cases, and interoperability, a more
+asynchronous, producer-focused interface may be required. These scenarios can utilize
+the ``ArrowAsyncDeviceStreamHandler``.
+
+Semantics
+---------
+
+Rather than the producer providing a structure of callbacks for a consumer to
+call and retrieve records, the Async interface is a consumer allocated structure.
+The consumer allocated struct provides handler callbacks for the producer to call
+when the schema and chunks of data are available, rather than the consumer using
+a blocking pull-style iteration.
+
+In addition to the ``ArrowAsyncDeviceStreamHandler``, there are also two additional
+structs used for the full data flow: ``ArrowAsyncTask`` and ``ArrowAsyncProducer``.
+
+Structure Definition
+--------------------
+
+The C device async stream interface is defined with a single ``struct`` definition:
+
+.. code-block:: c
+
+ #ifndef ARROW_C_ASYNC_STREAM_INTERFACE
+ #define ARROW_C_ASYNC_STREAM_INTERFACE
+
+ struct ArrowAsyncTask {
+ int (*get_data)(struct ArrowArrayTask* self, struct ArrowDeviceArray* out);
+
+ void* private_data;
+ };
+
+ struct ArrowAsyncProducer {
+ void (*request)(struct ArrowAsyncProducer* self, uint64_t n);
+ void (*cancel)(struct ArrowAsyncProducer* self);
+
+ void (*release)(struct ArrowAsyncProducer* self);
+ void* private_data;
+ };
+
+ struct ArrowAsyncDeviceStreamHandler {
+ // handlers
+ int (*on_schema)(struct ArrowAsyncDeviceStreamHandler* self,
+ struct ArrowAsyncProducer* producer,
+ struct ArrowSchema* stream_schema, const char* addl_metadata);
+ int (*on_next_task)(struct ArrowAsyncDeviceStreamHandler* self,
+ struct ArrowAsyncTask* task, const char* metadata);
+ void (*on_error)(struct ArrowAsyncDeviceStreamHandler* self,
+ int code, const char* message, const char* metadata);
+
+ // release callback
+ void (*release)(struct ArrowAsyncDeviceStreamHandler* self);
+
+ // opaque handler-specific data
+ void* private_data;
+ };
+
+ #endif // ARROW_C_ASYNC_STREAM_INTERFACE
+
+.. note::
+ The canonical guard ``ARROW_C_ASYNC_STREAM_INTERFACE`` is meant to avoid
+ duplicate definitions if two projects copy the C async stream interface
+ definitions into their own headers, and a third-party project includes
+ from these two projects. It is therefore important that this guard is kept
+ exactly as-is when these definitions are copied.
+
+The ArrowAsyncDeviceStreamHandler structure
+'''''''''''''''''''''''''''''''''''''''''''
+
+The structure has the following fields:
+
+.. c:member:: int (*ArrowAsyncDeviceStreamHandler.on_schema)(struct ArrowAsyncDeviceStreamHandler*, struct ArrowAsyncProducer*, struct ArrowSchema*, const char*)
+
+ *Mandatory.* Handler for receiving the schema of the stream. All records should
+ match the provided schema. If successful, the function should return 0, otherwise
+ it should return an ``errno``-compatible error code.
+
+ The ``const char*`` parameter exists for producers to provide any extra contextual information
+ they want, such as the total number of rows in the stream, statistics, or otherwise. This is
+ encoded in the same format as :c:member:`ArrowSchema.metadata`. If not ``NULL``,
+ the lifetime is only the scope of the call to this function. A consumer who wants to maintain
+ the additional metadata beyond the lifetime of this call *MUST* copy the value themselves.
+
+ Unless the ``on_error`` handler is called, this will always get called exactly once and will be
+ the first method called on this object. As such the producer *MUST* provide an ``ArrowAsyncProducer``
+ object when calling this function to allow the consumer to manage back-pressure and flow control.
+ The producer maintains ownership of the ``ArrowAsyncProducer`` and must clean it up before or after
+ calling the release callback on this object.
+
+ A producer that receives a non-zero result here must not subsequently call anything other than
+ the release callback on this object.
+
+.. c:member:: int (*ArrowAsyncDeviceStreamHandler.on_next_task)(struct ArrowAsyncDeviceStreamHandler*, struct ArrowAsyncTask*, const char*)
+
+ *Mandatory.* Handler to be called when a new record is available for processing. The
+ schema for each record should be the same as the schema that ``on_schema`` was called with.
+ If successfully handled, the function should return 0, otherwise it should return an
+ ``errno``-compatible error code.
+
+ Rather than passing the record itself it receives an ``ArrowAsyncTask`` instead to facilitate
+ better consumer-focused thread control as far as receiving the data. A call to this function
+ simply indicates that data is available via the provided task.
+
+ The ``const char*`` parameter exists for producers to provide any extra contextual information
+ they want. This is encoded in the same format as :c:member:`ArrowSchema.metadata`. If not ``NULL``,
+ the lifetime is only the scope of the call to this function. A consumer who wants to maintain
+ the additional metadata beyond the lifetime of this call *MUST* copy the value themselves.
+
+ A producer *MUST NOT* call this concurrently from multiple different threads.
+
+ The :c:member:`ArrowAsyncProducer.request` callback must be called to start receiving calls to this
+ handler.
+
+ This function *MUST* be able to be called re-entrantly on the same thread to allow for the
+ :c:member:`ArrowAsyncProducer.request` callback to potentially call this method recursively.
+
+.. c:member:: void (*ArrowAsyncDeviceStreamHandler.on_error)(struct ArrowAsyncDeviceStreamHandler, int, const char*, const char*)
+
+ *Mandatory.* Handler to be called when an error is encountered by the producer. After calling
+ this, the ``release`` callback will be called as the last call on this struct. The parameters
+ are an ``errno``-compatible error code and an optional error message and metadata.
+
+ If the message and metadata are not ``NULL``, their lifetime is only valid during the scope
+ of this call. A consumer who wants to maintain these values past the return of this function
+ *MUST* copy the values themselves.
+
+ If the metadata parameter is not ``NULL``, to provide key-value error metadata, then it should
+ be encoded identically to the way that metadata is encoded in :c:member:`ArrowSchema.metadata`.
+
+ It is valid for this to be called by a producer with or without a preceding call to
+ :c:member:`ArrowAsyncProducer.request`. This callback *MUST NOT* call any methods of an
+ ``ArrowAsyncProducer`` object.
+
+.. c:member:: void (*ArrowAsyncDeviceStreamHandler.release)(struct ArrowAsyncDeviceStreamHandler*)
+
+ *Mandatory.* A pointer to a consumer-provided release callback for the handler.
+
+ It is valid for this to be called by a producer with or without a preceding call to
+ :c:member:`ArrowAsyncProducer.request`. This must not call any methods of an ``ArrowAsyncProducer``
+ object.
+
+.. c:member:: void* ArrowAsyncDeviceStreamHandler.private_data
+
+ *Optional.* An opaque pointer to consumer-provided private data.
+
+ Producers *MUST NOT* process this member. Lifetime of this member is handled by
+ the consumer, and especially by the release callback.
+
+The ArrowAsyncTask structure
+''''''''''''''''''''''''''''
+
+The purpose of using a Task object rather than passing the array directly to the ``on_next``
+callback is to allow for more complex and efficient thread handling. Utilizing a Task
+object allows for a producer to separate the "decoding" logic from the I/O, enabling a
+consumer to avoid transferring data between CPU cores (e.g. from one L1/L2 cache to another).
Review Comment:
I think task structure is useful but we're not highlighting the real sweet spot for this flexibility. The reason not to do I/O and decoding on the same thread is that frequently we have separate thread pools for each, and we don't want to block for IO on a work thread nor do we want to do anything in an IO thread other than queue IO as immediately as we can. Any consumer who has distinct thread pools like this might be annoyed by a producer which insists on doing CPU bound work in a thread pool the consumer can't control. Tasks enable the producer to defer CPU bound work to the consumer
```suggestion
Using a Task object rather than simply passing the array directly to an ``on_next``
callback enables consumers to keep separate I/O and CPU work thread pools.
For example, a producer might construct Task objects after I/O is complete but
leave decoding logic until ``get_data`` is called. A consumer who knows this can
avoid blocking I/O threads by transferring Tasks to the CPU work thread pool.
After ``get_data`` completes the data will be processed immediately on the same
CPU work thread, avoiding transfer between CPU cores.
```
IIUC, Antoine's case of fanning out decoding is behavior that would most benefit a simpler consumer which has only one thread. That's also a useful case to support; what's the story for how a consumer would pass `use_threads=True` to a producer? Is that out of scope for this protocol, part of the factory interface which accepts a `ArrowAsyncDeviceStreamHandler*`?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-43631: [C][Format] Add ArrowAsyncDeviceStreamHandler interface [arrow]
Posted by "pitrou (via GitHub)" <gi...@apache.org>.
pitrou commented on code in PR #43632:
URL: https://github.com/apache/arrow/pull/43632#discussion_r1764970636
##########
docs/source/format/CDeviceDataInterface.rst:
##########
@@ -650,6 +652,331 @@ The stream source is not assumed to be thread-safe. Consumers wanting to
call ``get_next`` from several threads should ensure those calls are
serialized.
+Async Device Stream Interface
+=============================
+
+The :ref:`C stream interface <_c-device-stream-interface>` provides a synchronous
+API centered around the consumer calling the callback functions to retrieve
+the next record batch. For some bindings, use cases, and interoperability, a more
+asynchronous, producer-focused interface may be required. These scenarios can utilize
+the ``ArrowAsyncDeviceStreamHandler``.
+
+Semantics
+---------
+
+Rather than the producer providing a structure of callbacks for a consumer to
+call and retrieve records, the Async interface is a consumer allocated structure.
+The consumer allocated struct provides handler callbacks for the producer to call
+when the schema and chunks of data are available, rather than the consumer using
+a blocking pull-style iteration.
+
+In addition to the ``ArrowAsyncDeviceStreamHandler``, there are also two additional
+structs used for the full data flow: ``ArrowAsyncTask`` and ``ArrowAsyncProducer``.
+
+Structure Definition
+--------------------
+
+The C device async stream interface is defined with a single ``struct`` definition:
+
+.. code-block:: c
+
+ #ifndef ARROW_C_ASYNC_STREAM_INTERFACE
+ #define ARROW_C_ASYNC_STREAM_INTERFACE
+
+ struct ArrowAsyncTask {
+ int (*get_data)(struct ArrowArrayTask* self, struct ArrowDeviceArray* out);
+
+ void* private_data;
+ };
+
+ struct ArrowAsyncProducer {
+ void (*request)(struct ArrowAsyncProducer* self, uint64_t n);
+ void (*cancel)(struct ArrowAsyncProducer* self);
+
+ void (*release)(struct ArrowAsyncProducer* self);
+ void* private_data;
+ };
+
+ struct ArrowAsyncDeviceStreamHandler {
+ // handlers
+ int (*on_schema)(struct ArrowAsyncDeviceStreamHandler* self,
+ struct ArrowAsyncProducer* producer,
+ struct ArrowSchema* stream_schema, const char* addl_metadata);
+ int (*on_next_task)(struct ArrowAsyncDeviceStreamHandler* self,
+ struct ArrowAsyncTask* task, const char* metadata);
+ void (*on_error)(struct ArrowAsyncDeviceStreamHandler* self,
+ int code, const char* message, const char* metadata);
+
+ // release callback
+ void (*release)(struct ArrowAsyncDeviceStreamHandler* self);
+
+ // opaque handler-specific data
+ void* private_data;
+ };
+
+ #endif // ARROW_C_ASYNC_STREAM_INTERFACE
+
+.. note::
+ The canonical guard ``ARROW_C_ASYNC_STREAM_INTERFACE`` is meant to avoid
+ duplicate definitions if two projects copy the C async stream interface
+ definitions into their own headers, and a third-party project includes
+ from these two projects. It is therefore important that this guard is kept
+ exactly as-is when these definitions are copied.
+
+The ArrowAsyncDeviceStreamHandler structure
+'''''''''''''''''''''''''''''''''''''''''''
+
+The structure has the following fields:
+
+.. c:member:: int (*ArrowAsyncDeviceStreamHandler.on_schema)(struct ArrowAsyncDeviceStreamHandler*, struct ArrowAsyncProducer*, struct ArrowSchema*, const char*)
+
+ *Mandatory.* Handler for receiving the schema of the stream. All records should
+ match the provided schema. If successful, the function should return 0, otherwise
+ it should return an ``errno``-compatible error code.
+
+ The ``const char*`` parameter exists for producers to provide any extra contextual information
+ they want, such as the total number of rows in the stream, statistics, or otherwise. This is
+ encoded in the same format as :c:member:`ArrowSchema.metadata`. If not ``NULL``,
+ the lifetime is only the scope of the call to this function. A consumer who wants to maintain
+ the additional metadata beyond the lifetime of this call *MUST* copy the value themselves.
+
+ Unless the ``on_error`` handler is called, this will always get called exactly once and will be
+ the first method called on this object. As such the producer *MUST* provide an ``ArrowAsyncProducer``
+ object when calling this function to allow the consumer to manage back-pressure and flow control.
+ The producer maintains ownership of the ``ArrowAsyncProducer`` and must clean it up before or after
+ calling the release callback on this object.
+
+ A producer that receives a non-zero result here must not subsequently call anything other than
+ the release callback on this object.
+
+.. c:member:: int (*ArrowAsyncDeviceStreamHandler.on_next_task)(struct ArrowAsyncDeviceStreamHandler*, struct ArrowAsyncTask*, const char*)
+
+ *Mandatory.* Handler to be called when a new record is available for processing. The
+ schema for each record should be the same as the schema that ``on_schema`` was called with.
+ If successfully handled, the function should return 0, otherwise it should return an
+ ``errno``-compatible error code.
+
+ Rather than passing the record itself it receives an ``ArrowAsyncTask`` instead to facilitate
+ better consumer-focused thread control as far as receiving the data. A call to this function
+ simply indicates that data is available via the provided task.
+
+ The ``const char*`` parameter exists for producers to provide any extra contextual information
+ they want. This is encoded in the same format as :c:member:`ArrowSchema.metadata`. If not ``NULL``,
+ the lifetime is only the scope of the call to this function. A consumer who wants to maintain
+ the additional metadata beyond the lifetime of this call *MUST* copy the value themselves.
+
+ A producer *MUST NOT* call this concurrently from multiple different threads.
+
+ The :c:member:`ArrowAsyncProducer.request` callback must be called to start receiving calls to this
+ handler.
+
+ This function *MUST* be able to be called re-entrantly on the same thread to allow for the
+ :c:member:`ArrowAsyncProducer.request` callback to potentially call this method recursively.
+
+.. c:member:: void (*ArrowAsyncDeviceStreamHandler.on_error)(struct ArrowAsyncDeviceStreamHandler, int, const char*, const char*)
+
+ *Mandatory.* Handler to be called when an error is encountered by the producer. After calling
+ this, the ``release`` callback will be called as the last call on this struct. The parameters
+ are an ``errno``-compatible error code and an optional error message and metadata.
+
+ If the message and metadata are not ``NULL``, their lifetime is only valid during the scope
+ of this call. A consumer who wants to maintain these values past the return of this function
+ *MUST* copy the values themselves.
+
+ If the metadata parameter is not ``NULL``, to provide key-value error metadata, then it should
+ be encoded identically to the way that metadata is encoded in :c:member:`ArrowSchema.metadata`.
+
+ It is valid for this to be called by a producer with or without a preceding call to
+ :c:member:`ArrowAsyncProducer.request`. This callback *MUST NOT* call any methods of an
+ ``ArrowAsyncProducer`` object.
+
+.. c:member:: void (*ArrowAsyncDeviceStreamHandler.release)(struct ArrowAsyncDeviceStreamHandler*)
+
+ *Mandatory.* A pointer to a consumer-provided release callback for the handler.
+
+ It is valid for this to be called by a producer with or without a preceding call to
+ :c:member:`ArrowAsyncProducer.request`. This must not call any methods of an ``ArrowAsyncProducer``
+ object.
+
+.. c:member:: void* ArrowAsyncDeviceStreamHandler.private_data
+
+ *Optional.* An opaque pointer to consumer-provided private data.
+
+ Producers *MUST NOT* process this member. Lifetime of this member is handled by
+ the consumer, and especially by the release callback.
+
+The ArrowAsyncTask structure
+''''''''''''''''''''''''''''
+
+The purpose of using a Task object rather than passing the array directly to the ``on_next``
+callback is to allow for more complex and efficient thread handling. Utilizing a Task
+object allows for a producer to separate the "decoding" logic from the I/O, enabling a
+consumer to avoid transferring data between CPU cores (e.g. from one L1/L2 cache to another).
+
+This producer-provided structure has the following fields:
+
+.. c:member:: int (*ArrowArrayTask.get_data)(struct ArrowArrayTask*, struct ArrowDeviceArray*)
+
+ *Mandatory.* A callback to populate the provided ``ArrowDeviceArray`` with the available data.
+ The order of ArrowAsyncTasks provided by the producer enables a consumer to know the order of
+ the data to process.
+
+ If a non-zero value is returned from this, it should be followed only by the producer calling
+ the ``on_error`` callback of the ``ArrowAsyncDeviceStreamHandler``. Because calling this method
+ is likely to be separate from the current control flow, returning a non-zero value to signal
+ an error occuring allows the current thread to decide handle the case accordingly, while still
+ allowing all error logging and handling to be centralized in the
+ :c:member:`ArrowAsyncDeviceStreamHandler.on_error` callback.
+
+ Rather than having a separate release callback, any required cleanup should be performed as part
+ of the invocation of this callback, with ownership of the Array being passed to the consumer-provided
+ parameter calling this, and must be released separately.
+
+ It is only valid to call this method exactly once.
+
+.. c:member:: void* ArrowArrayTask.private_data
+
+ *Optional.* An opaque pointer to consumer-provided private data.
+
+ Consumers *MUST NOT* process this member. Lifetime of this member is handled by
+ the producer who created this object, and should be cleaned up if necessary during
+ the call to :c:member:`ArrowArrayTask.get_data`.
+
+The ArrowAsyncProducer structure
+''''''''''''''''''''''''''''''''
+
+This producer-provided and managed object has the following fields:
+
+.. c:member:: void (*ArrowAsyncProducer.request)(struct ArrowAsyncProducer*, uint64_t)
+
+ *Mandatory.* This function must be called by a consumer to start receiving calls to
+ :c:member:`ArrowAsyncDeviceStreamHandler.on_next_task`. It *MUST* be valid to call
+ this synchronously from within :c:member:`ArrowAsyncDeviceStreamHandler.on_next_task`
+ or :c:member:`ArrowAsyncDeviceStreamHandler.on_schema`. As a result, this function
+ *MUST NOT* synchronously call ``on_next_task`` or ``on_error`` to avoid recursive
+ and reentrant callbacks.
+
+ After ``cancel`` is called, additional calls to this function must be a NOP, but allowed.
+
+ While not cancelled, calling this function registers the given number of additional
+ arrays/batches to be produced by the producer. A producer should only call
+ the appropriate ``on_next_task`` callback up to a maximum of the total sum of calls to
+ this method before propagating back-pressure / waiting.
+
+ Any error encountered by calling request must be propagated by calling the ``on_error``
+ callback of the ``ArrowAsyncDeviceStreamHandler``.
+
+ A producer *MUST* support an unbounded number of calls to this callback and *MUST* support
+ a total registered demand (sum requested - sum delivered) of up to ``UINT64_MAX``.
Review Comment:
I don't know what "Reactive Streams", but why would it proscribe any implementation-specific limit? This sounds like a silly thing to put in a spec, IMHO.
##########
docs/source/format/CDeviceDataInterface.rst:
##########
@@ -650,6 +652,331 @@ The stream source is not assumed to be thread-safe. Consumers wanting to
call ``get_next`` from several threads should ensure those calls are
serialized.
+Async Device Stream Interface
+=============================
+
+The :ref:`C stream interface <_c-device-stream-interface>` provides a synchronous
+API centered around the consumer calling the callback functions to retrieve
+the next record batch. For some bindings, use cases, and interoperability, a more
+asynchronous, producer-focused interface may be required. These scenarios can utilize
+the ``ArrowAsyncDeviceStreamHandler``.
+
+Semantics
+---------
+
+Rather than the producer providing a structure of callbacks for a consumer to
+call and retrieve records, the Async interface is a consumer allocated structure.
+The consumer allocated struct provides handler callbacks for the producer to call
+when the schema and chunks of data are available, rather than the consumer using
+a blocking pull-style iteration.
+
+In addition to the ``ArrowAsyncDeviceStreamHandler``, there are also two additional
+structs used for the full data flow: ``ArrowAsyncTask`` and ``ArrowAsyncProducer``.
+
+Structure Definition
+--------------------
+
+The C device async stream interface is defined with a single ``struct`` definition:
+
+.. code-block:: c
+
+ #ifndef ARROW_C_ASYNC_STREAM_INTERFACE
+ #define ARROW_C_ASYNC_STREAM_INTERFACE
+
+ struct ArrowAsyncTask {
+ int (*get_data)(struct ArrowArrayTask* self, struct ArrowDeviceArray* out);
+
+ void* private_data;
+ };
+
+ struct ArrowAsyncProducer {
+ void (*request)(struct ArrowAsyncProducer* self, uint64_t n);
+ void (*cancel)(struct ArrowAsyncProducer* self);
+
+ void (*release)(struct ArrowAsyncProducer* self);
+ void* private_data;
+ };
+
+ struct ArrowAsyncDeviceStreamHandler {
+ // handlers
+ int (*on_schema)(struct ArrowAsyncDeviceStreamHandler* self,
+ struct ArrowAsyncProducer* producer,
+ struct ArrowSchema* stream_schema, const char* addl_metadata);
+ int (*on_next_task)(struct ArrowAsyncDeviceStreamHandler* self,
+ struct ArrowAsyncTask* task, const char* metadata);
+ void (*on_error)(struct ArrowAsyncDeviceStreamHandler* self,
+ int code, const char* message, const char* metadata);
+
+ // release callback
+ void (*release)(struct ArrowAsyncDeviceStreamHandler* self);
+
+ // opaque handler-specific data
+ void* private_data;
+ };
+
+ #endif // ARROW_C_ASYNC_STREAM_INTERFACE
+
+.. note::
+ The canonical guard ``ARROW_C_ASYNC_STREAM_INTERFACE`` is meant to avoid
+ duplicate definitions if two projects copy the C async stream interface
+ definitions into their own headers, and a third-party project includes
+ from these two projects. It is therefore important that this guard is kept
+ exactly as-is when these definitions are copied.
+
+The ArrowAsyncDeviceStreamHandler structure
+'''''''''''''''''''''''''''''''''''''''''''
+
+The structure has the following fields:
+
+.. c:member:: int (*ArrowAsyncDeviceStreamHandler.on_schema)(struct ArrowAsyncDeviceStreamHandler*, struct ArrowAsyncProducer*, struct ArrowSchema*, const char*)
+
+ *Mandatory.* Handler for receiving the schema of the stream. All records should
+ match the provided schema. If successful, the function should return 0, otherwise
+ it should return an ``errno``-compatible error code.
+
+ The ``const char*`` parameter exists for producers to provide any extra contextual information
+ they want, such as the total number of rows in the stream, statistics, or otherwise. This is
+ encoded in the same format as :c:member:`ArrowSchema.metadata`. If not ``NULL``,
+ the lifetime is only the scope of the call to this function. A consumer who wants to maintain
+ the additional metadata beyond the lifetime of this call *MUST* copy the value themselves.
+
+ Unless the ``on_error`` handler is called, this will always get called exactly once and will be
+ the first method called on this object. As such the producer *MUST* provide an ``ArrowAsyncProducer``
+ object when calling this function to allow the consumer to manage back-pressure and flow control.
+ The producer maintains ownership of the ``ArrowAsyncProducer`` and must clean it up before or after
+ calling the release callback on this object.
+
+ A producer that receives a non-zero result here must not subsequently call anything other than
+ the release callback on this object.
+
+.. c:member:: int (*ArrowAsyncDeviceStreamHandler.on_next_task)(struct ArrowAsyncDeviceStreamHandler*, struct ArrowAsyncTask*, const char*)
+
+ *Mandatory.* Handler to be called when a new record is available for processing. The
+ schema for each record should be the same as the schema that ``on_schema`` was called with.
+ If successfully handled, the function should return 0, otherwise it should return an
+ ``errno``-compatible error code.
+
+ Rather than passing the record itself it receives an ``ArrowAsyncTask`` instead to facilitate
+ better consumer-focused thread control as far as receiving the data. A call to this function
+ simply indicates that data is available via the provided task.
+
+ The ``const char*`` parameter exists for producers to provide any extra contextual information
+ they want. This is encoded in the same format as :c:member:`ArrowSchema.metadata`. If not ``NULL``,
+ the lifetime is only the scope of the call to this function. A consumer who wants to maintain
+ the additional metadata beyond the lifetime of this call *MUST* copy the value themselves.
+
+ A producer *MUST NOT* call this concurrently from multiple different threads.
+
+ The :c:member:`ArrowAsyncProducer.request` callback must be called to start receiving calls to this
+ handler.
+
+ This function *MUST* be able to be called re-entrantly on the same thread to allow for the
+ :c:member:`ArrowAsyncProducer.request` callback to potentially call this method recursively.
+
+.. c:member:: void (*ArrowAsyncDeviceStreamHandler.on_error)(struct ArrowAsyncDeviceStreamHandler, int, const char*, const char*)
+
+ *Mandatory.* Handler to be called when an error is encountered by the producer. After calling
+ this, the ``release`` callback will be called as the last call on this struct. The parameters
+ are an ``errno``-compatible error code and an optional error message and metadata.
+
+ If the message and metadata are not ``NULL``, their lifetime is only valid during the scope
+ of this call. A consumer who wants to maintain these values past the return of this function
+ *MUST* copy the values themselves.
+
+ If the metadata parameter is not ``NULL``, to provide key-value error metadata, then it should
+ be encoded identically to the way that metadata is encoded in :c:member:`ArrowSchema.metadata`.
+
+ It is valid for this to be called by a producer with or without a preceding call to
+ :c:member:`ArrowAsyncProducer.request`. This callback *MUST NOT* call any methods of an
+ ``ArrowAsyncProducer`` object.
+
+.. c:member:: void (*ArrowAsyncDeviceStreamHandler.release)(struct ArrowAsyncDeviceStreamHandler*)
+
+ *Mandatory.* A pointer to a consumer-provided release callback for the handler.
+
+ It is valid for this to be called by a producer with or without a preceding call to
+ :c:member:`ArrowAsyncProducer.request`. This must not call any methods of an ``ArrowAsyncProducer``
+ object.
+
+.. c:member:: void* ArrowAsyncDeviceStreamHandler.private_data
+
+ *Optional.* An opaque pointer to consumer-provided private data.
+
+ Producers *MUST NOT* process this member. Lifetime of this member is handled by
+ the consumer, and especially by the release callback.
+
+The ArrowAsyncTask structure
+''''''''''''''''''''''''''''
+
+The purpose of using a Task object rather than passing the array directly to the ``on_next``
+callback is to allow for more complex and efficient thread handling. Utilizing a Task
+object allows for a producer to separate the "decoding" logic from the I/O, enabling a
+consumer to avoid transferring data between CPU cores (e.g. from one L1/L2 cache to another).
+
+This producer-provided structure has the following fields:
+
+.. c:member:: int (*ArrowArrayTask.get_data)(struct ArrowArrayTask*, struct ArrowDeviceArray*)
+
+ *Mandatory.* A callback to populate the provided ``ArrowDeviceArray`` with the available data.
+ The order of ArrowAsyncTasks provided by the producer enables a consumer to know the order of
+ the data to process.
+
+ If a non-zero value is returned from this, it should be followed only by the producer calling
+ the ``on_error`` callback of the ``ArrowAsyncDeviceStreamHandler``. Because calling this method
+ is likely to be separate from the current control flow, returning a non-zero value to signal
+ an error occuring allows the current thread to decide handle the case accordingly, while still
+ allowing all error logging and handling to be centralized in the
+ :c:member:`ArrowAsyncDeviceStreamHandler.on_error` callback.
+
+ Rather than having a separate release callback, any required cleanup should be performed as part
+ of the invocation of this callback, with ownership of the Array being passed to the consumer-provided
+ parameter calling this, and must be released separately.
+
+ It is only valid to call this method exactly once.
+
+.. c:member:: void* ArrowArrayTask.private_data
+
+ *Optional.* An opaque pointer to consumer-provided private data.
+
+ Consumers *MUST NOT* process this member. Lifetime of this member is handled by
+ the producer who created this object, and should be cleaned up if necessary during
+ the call to :c:member:`ArrowArrayTask.get_data`.
+
+The ArrowAsyncProducer structure
+''''''''''''''''''''''''''''''''
+
+This producer-provided and managed object has the following fields:
+
+.. c:member:: void (*ArrowAsyncProducer.request)(struct ArrowAsyncProducer*, uint64_t)
+
+ *Mandatory.* This function must be called by a consumer to start receiving calls to
+ :c:member:`ArrowAsyncDeviceStreamHandler.on_next_task`. It *MUST* be valid to call
+ this synchronously from within :c:member:`ArrowAsyncDeviceStreamHandler.on_next_task`
+ or :c:member:`ArrowAsyncDeviceStreamHandler.on_schema`. As a result, this function
+ *MUST NOT* synchronously call ``on_next_task`` or ``on_error`` to avoid recursive
+ and reentrant callbacks.
+
+ After ``cancel`` is called, additional calls to this function must be a NOP, but allowed.
+
+ While not cancelled, calling this function registers the given number of additional
+ arrays/batches to be produced by the producer. A producer should only call
+ the appropriate ``on_next_task`` callback up to a maximum of the total sum of calls to
+ this method before propagating back-pressure / waiting.
+
+ Any error encountered by calling request must be propagated by calling the ``on_error``
+ callback of the ``ArrowAsyncDeviceStreamHandler``.
+
+ A producer *MUST* support an unbounded number of calls to this callback and *MUST* support
+ a total registered demand (sum requested - sum delivered) of up to ``UINT64_MAX``.
Review Comment:
I don't know what "Reactive Streams" it, but why would it proscribe any implementation-specific limit? This sounds like a silly thing to put in a spec, IMHO.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-43631: [C][Format] Add ArrowAsyncDeviceStreamHandler interface [arrow]
Posted by "pitrou (via GitHub)" <gi...@apache.org>.
pitrou commented on code in PR #43632:
URL: https://github.com/apache/arrow/pull/43632#discussion_r1765708075
##########
docs/source/format/CDeviceDataInterface.rst:
##########
@@ -650,6 +652,331 @@ The stream source is not assumed to be thread-safe. Consumers wanting to
call ``get_next`` from several threads should ensure those calls are
serialized.
+Async Device Stream Interface
+=============================
+
+The :ref:`C stream interface <_c-device-stream-interface>` provides a synchronous
+API centered around the consumer calling the callback functions to retrieve
+the next record batch. For some bindings, use cases, and interoperability, a more
+asynchronous, producer-focused interface may be required. These scenarios can utilize
+the ``ArrowAsyncDeviceStreamHandler``.
+
+Semantics
+---------
+
+Rather than the producer providing a structure of callbacks for a consumer to
+call and retrieve records, the Async interface is a consumer allocated structure.
+The consumer allocated struct provides handler callbacks for the producer to call
+when the schema and chunks of data are available, rather than the consumer using
+a blocking pull-style iteration.
+
+In addition to the ``ArrowAsyncDeviceStreamHandler``, there are also two additional
+structs used for the full data flow: ``ArrowAsyncTask`` and ``ArrowAsyncProducer``.
+
+Structure Definition
+--------------------
+
+The C device async stream interface is defined with a single ``struct`` definition:
+
+.. code-block:: c
+
+ #ifndef ARROW_C_ASYNC_STREAM_INTERFACE
+ #define ARROW_C_ASYNC_STREAM_INTERFACE
+
+ struct ArrowAsyncTask {
+ int (*get_data)(struct ArrowArrayTask* self, struct ArrowDeviceArray* out);
+
+ void* private_data;
+ };
+
+ struct ArrowAsyncProducer {
+ void (*request)(struct ArrowAsyncProducer* self, uint64_t n);
+ void (*cancel)(struct ArrowAsyncProducer* self);
+
+ void (*release)(struct ArrowAsyncProducer* self);
+ void* private_data;
+ };
+
+ struct ArrowAsyncDeviceStreamHandler {
+ // handlers
+ int (*on_schema)(struct ArrowAsyncDeviceStreamHandler* self,
+ struct ArrowAsyncProducer* producer,
+ struct ArrowSchema* stream_schema, const char* addl_metadata);
+ int (*on_next_task)(struct ArrowAsyncDeviceStreamHandler* self,
+ struct ArrowAsyncTask* task, const char* metadata);
+ void (*on_error)(struct ArrowAsyncDeviceStreamHandler* self,
+ int code, const char* message, const char* metadata);
+
+ // release callback
+ void (*release)(struct ArrowAsyncDeviceStreamHandler* self);
+
+ // opaque handler-specific data
+ void* private_data;
+ };
+
+ #endif // ARROW_C_ASYNC_STREAM_INTERFACE
+
+.. note::
+ The canonical guard ``ARROW_C_ASYNC_STREAM_INTERFACE`` is meant to avoid
+ duplicate definitions if two projects copy the C async stream interface
+ definitions into their own headers, and a third-party project includes
+ from these two projects. It is therefore important that this guard is kept
+ exactly as-is when these definitions are copied.
+
+The ArrowAsyncDeviceStreamHandler structure
+'''''''''''''''''''''''''''''''''''''''''''
+
+The structure has the following fields:
+
+.. c:member:: int (*ArrowAsyncDeviceStreamHandler.on_schema)(struct ArrowAsyncDeviceStreamHandler*, struct ArrowAsyncProducer*, struct ArrowSchema*, const char*)
+
+ *Mandatory.* Handler for receiving the schema of the stream. All records should
+ match the provided schema. If successful, the function should return 0, otherwise
+ it should return an ``errno``-compatible error code.
+
+ The ``const char*`` parameter exists for producers to provide any extra contextual information
+ they want, such as the total number of rows in the stream, statistics, or otherwise. This is
+ encoded in the same format as :c:member:`ArrowSchema.metadata`. If not ``NULL``,
+ the lifetime is only the scope of the call to this function. A consumer who wants to maintain
+ the additional metadata beyond the lifetime of this call *MUST* copy the value themselves.
+
+ Unless the ``on_error`` handler is called, this will always get called exactly once and will be
+ the first method called on this object. As such the producer *MUST* provide an ``ArrowAsyncProducer``
+ object when calling this function to allow the consumer to manage back-pressure and flow control.
+ The producer maintains ownership of the ``ArrowAsyncProducer`` and must clean it up before or after
+ calling the release callback on this object.
+
+ A producer that receives a non-zero result here must not subsequently call anything other than
+ the release callback on this object.
+
+.. c:member:: int (*ArrowAsyncDeviceStreamHandler.on_next_task)(struct ArrowAsyncDeviceStreamHandler*, struct ArrowAsyncTask*, const char*)
+
+ *Mandatory.* Handler to be called when a new record is available for processing. The
+ schema for each record should be the same as the schema that ``on_schema`` was called with.
+ If successfully handled, the function should return 0, otherwise it should return an
+ ``errno``-compatible error code.
+
+ Rather than passing the record itself it receives an ``ArrowAsyncTask`` instead to facilitate
+ better consumer-focused thread control as far as receiving the data. A call to this function
+ simply indicates that data is available via the provided task.
+
+ The ``const char*`` parameter exists for producers to provide any extra contextual information
+ they want. This is encoded in the same format as :c:member:`ArrowSchema.metadata`. If not ``NULL``,
+ the lifetime is only the scope of the call to this function. A consumer who wants to maintain
+ the additional metadata beyond the lifetime of this call *MUST* copy the value themselves.
+
+ A producer *MUST NOT* call this concurrently from multiple different threads.
+
+ The :c:member:`ArrowAsyncProducer.request` callback must be called to start receiving calls to this
+ handler.
+
+ This function *MUST* be able to be called re-entrantly on the same thread to allow for the
+ :c:member:`ArrowAsyncProducer.request` callback to potentially call this method recursively.
+
+.. c:member:: void (*ArrowAsyncDeviceStreamHandler.on_error)(struct ArrowAsyncDeviceStreamHandler, int, const char*, const char*)
+
+ *Mandatory.* Handler to be called when an error is encountered by the producer. After calling
+ this, the ``release`` callback will be called as the last call on this struct. The parameters
+ are an ``errno``-compatible error code and an optional error message and metadata.
+
+ If the message and metadata are not ``NULL``, their lifetime is only valid during the scope
+ of this call. A consumer who wants to maintain these values past the return of this function
+ *MUST* copy the values themselves.
+
+ If the metadata parameter is not ``NULL``, to provide key-value error metadata, then it should
+ be encoded identically to the way that metadata is encoded in :c:member:`ArrowSchema.metadata`.
+
+ It is valid for this to be called by a producer with or without a preceding call to
+ :c:member:`ArrowAsyncProducer.request`. This callback *MUST NOT* call any methods of an
+ ``ArrowAsyncProducer`` object.
+
+.. c:member:: void (*ArrowAsyncDeviceStreamHandler.release)(struct ArrowAsyncDeviceStreamHandler*)
+
+ *Mandatory.* A pointer to a consumer-provided release callback for the handler.
+
+ It is valid for this to be called by a producer with or without a preceding call to
+ :c:member:`ArrowAsyncProducer.request`. This must not call any methods of an ``ArrowAsyncProducer``
+ object.
+
+.. c:member:: void* ArrowAsyncDeviceStreamHandler.private_data
+
+ *Optional.* An opaque pointer to consumer-provided private data.
+
+ Producers *MUST NOT* process this member. Lifetime of this member is handled by
+ the consumer, and especially by the release callback.
+
+The ArrowAsyncTask structure
+''''''''''''''''''''''''''''
+
+The purpose of using a Task object rather than passing the array directly to the ``on_next``
+callback is to allow for more complex and efficient thread handling. Utilizing a Task
+object allows for a producer to separate the "decoding" logic from the I/O, enabling a
+consumer to avoid transferring data between CPU cores (e.g. from one L1/L2 cache to another).
Review Comment:
You didn't answer my objection. In the IPC reader with buffer compression example, the producer could be able to leverage parallelism _for a single task_. But that's only if it manages threads by itself.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-43631: [C][Format] Add ArrowAsyncDeviceStreamHandler interface [arrow]
Posted by "leventov (via GitHub)" <gi...@apache.org>.
leventov commented on code in PR #43632:
URL: https://github.com/apache/arrow/pull/43632#discussion_r1721537318
##########
cpp/src/arrow/c/abi.h:
##########
@@ -228,6 +228,65 @@ struct ArrowDeviceArrayStream {
#endif // ARROW_C_DEVICE_STREAM_INTERFACE
+#ifndef ARROW_C_ASYNC_STREAM_INTERFACE
+#define ARROW_C_ASYNC_STREAM_INTERFACE
+
+// Similar to ArrowDeviceArrayStream, except designed for an asynchronous
+// style of interaction. While ArrowDeviceArrayStream provides producer
+// defined callbacks, this is intended to be created by the consumer instead.
+// The consumer passes this handler to the producer, which in turn uses the
+// callbacks to inform the consumer of events in the stream.
+struct ArrowAsyncDeviceStreamHandler {
+ // Handler for receiving a schema. The passed in stream_schema should be
+ // released or moved by the handler (producer is giving ownership of it to
+ // the handler).
+ //
+ // The `extension_param` argument can be null or can be used by a producer
+ // to pass arbitrary extra information to the consumer (such as total number
+ // of rows, context info, or otherwise).
+ //
+ // Return value: 0 if successful, `errno`-compatible error otherwise
Review Comment:
In general, I would suggest that ADBC's async APIs should be compatible with [Reactive Streams](https://en.wikipedia.org/wiki/Reactive_Streams) (RS) semantics in all places except for the semantics of `request(n)` signal, which for Arrow shouldn't be tied to the number of calls to on_next(), but rather indicate the number of rows. This is to avoid the burden of reinventing the wheel in async model aspects which reactive streams working group already thought and debated long and hard about.
Also, users of ADBC's async APIs in JVM and C# will be able to leverage some very high quality code for these platforms that implement RS semantics, such as [j.u.c.SubmissionPublisher](https://docs.oracle.com/en/java/javase/17/docs/api/java.base/java/util/concurrent/SubmissionPublisher.html), [kotlinx.coroutines.reactive](https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-reactive/), [dotnet/reactive](https://github.com/dotnet/reactive), hopefully with minimal modifications, or perhaps no modifications at all (the different semantics of `request(n)` could perhaps piggy-back existing libs because per RS spec, producers are allowed to send a complete signal after calling on_next fewer times than was requested).
Per [RS 1.7](https://github.com/reactive-streams/reactive-streams-jvm/blob/v1.0.4/README.md#1.7), it's required that producer stops on error from consumer callback:
> Once a [terminal state](https://github.com/reactive-streams/reactive-streams-jvm/blob/v1.0.4/README.md#term_terminal_state) has been signaled (onError, onComplete) it is REQUIRED that no further signals occur.
Re:
> the docs explicitly state that if an error is returned from the callback, the producer should only call release after that and not call further callbacks. though there isn't anything to functionally prevent it.
Indeed, per [RS 2.5](https://github.com/reactive-streams/reactive-streams-jvm/blob/v1.0.4/README.md#2.5), consumer MUST go out of its way even to prevent buggy concurrent use by the producer, let alone not to call any of its methods itself concurrently with the producer, or ofter the producer has returned error or completion signal.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-43631: [C][Format] Add ArrowAsyncDeviceStreamHandler interface [arrow]
Posted by "pitrou (via GitHub)" <gi...@apache.org>.
pitrou commented on code in PR #43632:
URL: https://github.com/apache/arrow/pull/43632#discussion_r1718558356
##########
cpp/src/arrow/c/abi.h:
##########
@@ -228,6 +228,65 @@ struct ArrowDeviceArrayStream {
#endif // ARROW_C_DEVICE_STREAM_INTERFACE
+#ifndef ARROW_C_ASYNC_STREAM_INTERFACE
+#define ARROW_C_ASYNC_STREAM_INTERFACE
+
+// Similar to ArrowDeviceArrayStream, except designed for an asynchronous
+// style of interaction. While ArrowDeviceArrayStream provides producer
+// defined callbacks, this is intended to be created by the consumer instead.
+// The consumer passes this handler to the producer, which in turn uses the
+// callbacks to inform the consumer of events in the stream.
+struct ArrowAsyncDeviceStreamHandler {
+ // Handler for receiving a schema. The passed in stream_schema should be
+ // released or moved by the handler (producer is giving ownership of it to
+ // the handler).
+ //
+ // The `extension_param` argument can be null or can be used by a producer
+ // to pass arbitrary extra information to the consumer (such as total number
+ // of rows, context info, or otherwise).
+ //
+ // Return value: 0 if successful, `errno`-compatible error otherwise
+ int (*on_schema)(struct ArrowAsyncDeviceStreamHandler* self,
+ struct ArrowSchema* stream_schema, void* extension_param);
+
+ // Handler for receiving an array/record batch. Always called at least once
+ // unless an error is encountered (which would result in calling on_error).
+ // An empty/released array is passed to indicate the end of the stream if no
+ // errors have been encountered.
+ //
+ // The `extension_param` argument can be null or can be used by a producer
+ // to pass arbitrary extra information to the consumer.
+ //
+ // Return value: 0 if successful, `errno`-compatible error otherwise.
+ int (*on_next)(struct ArrowAsyncDeviceStreamHandler* self,
+ struct ArrowDeviceArray* next, void* extension_param);
+
+ // Handler for encountering an error. The producer should call release after
+ // this returns to clean up any resources.
+ //
+ // If the message or metadata are non-null, they will only last as long as this
+ // function call. The consumer would need to perform a copy of the data if it is
+ // it is necessary for them live past the lifetime of this call.
+ //
+ // Error metadata should be encoded as with metadata in ArrowSchema, defined in
+ // the spec at
+ // https://arrow.apache.org/docs/format/CDataInterface.html#c.ArrowSchema.metadata
+ //
+ // After this call, producers should follow-up by calling the release callback.
+ void (*on_error)(struct ArrowAsyncDeviceStreamHandler* self, int code,
+ const char* message, const char* metadata);
Review Comment:
Why a key-value metadata instead of the other `void* extension_param` convention above? Do you have something particular in mind?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-43631: [C][Format] Add ArrowAsyncDeviceStreamHandler interface [arrow]
Posted by "lidavidm (via GitHub)" <gi...@apache.org>.
lidavidm commented on PR #43632:
URL: https://github.com/apache/arrow/pull/43632#issuecomment-2303920845
> Change `on_next` to instead be `get_next_task` and create an `ArrowArrayTask` struct that populates the `ArrowDeviceArray`
There's the partial compromise option:
> Would it be sufficient for that use case if we had a callback approach that produced a task instead of directly producing an array?
>
>> Off the top of my head, yes, that should be fine. I think you can separate thread transfer from push/pull in that way.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-43631: [C][Format] Add ArrowAsyncDeviceStreamHandler interface [arrow]
Posted by "lidavidm (via GitHub)" <gi...@apache.org>.
lidavidm commented on code in PR #43632:
URL: https://github.com/apache/arrow/pull/43632#discussion_r1712483891
##########
cpp/src/arrow/c/abi.h:
##########
@@ -228,6 +228,65 @@ struct ArrowDeviceArrayStream {
#endif // ARROW_C_DEVICE_STREAM_INTERFACE
+#ifndef ARROW_C_ASYNC_STREAM_INTERFACE
+#define ARROW_C_ASYNC_STREAM_INTERFACE
+
+// Similar to ArrowDeviceArrayStream, except designed for an asynchronous
+// style of interaction. While ArrowDeviceArrayStream provides producer
+// defined callbacks, this is intended to be created by the consumer instead.
+// The consumer passes this handler to the producer, which in turn uses the
+// callbacks to inform the consumer of events in the stream.
+struct ArrowAsyncDeviceStreamHandler {
+ // Handler for receiving a schema. The passed in stream_schema should be
+ // released or moved by the handler (producer is giving ownership of it to
+ // the handler).
+ //
+ // The `extension_param` argument can be null or can be used by a producer
+ // to pass arbitrary extra information to the consumer (such as total number
+ // of rows, context info, or otherwise).
+ //
+ // Return value: 0 if successful, `errno`-compatible error otherwise
Review Comment:
Right, but it might make more sense to just have a unified cancellation mechanism. And again, what is the producer really supposed to do with the error code? It's just going to drop it on the floor.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-43631: [C][Format] Add ArrowAsyncDeviceStreamHandler interface [arrow]
Posted by "felipecrv (via GitHub)" <gi...@apache.org>.
felipecrv commented on code in PR #43632:
URL: https://github.com/apache/arrow/pull/43632#discussion_r1767114741
##########
docs/source/format/CDeviceDataInterface.rst:
##########
@@ -650,6 +652,340 @@ The stream source is not assumed to be thread-safe. Consumers wanting to
call ``get_next`` from several threads should ensure those calls are
serialized.
+Async Device Stream Interface
+=============================
+
+The :ref:`C stream interface <_c-device-stream-interface>` provides a synchronous
+API centered around the consumer calling the callback functions to retrieve
Review Comment:
Are these really "call-back" functions? I know the spec and code calls them callbacks, but "producer functions" will be less confusing here.
```suggestion
API centered around the consumer calling the producer functions to retrieve
```
##########
docs/source/format/CDeviceDataInterface.rst:
##########
@@ -650,6 +652,340 @@ The stream source is not assumed to be thread-safe. Consumers wanting to
call ``get_next`` from several threads should ensure those calls are
serialized.
+Async Device Stream Interface
+=============================
+
+The :ref:`C stream interface <_c-device-stream-interface>` provides a synchronous
+API centered around the consumer calling the callback functions to retrieve
+the next record batch. For some bindings, use cases, and interoperability, a more
+asynchronous, producer-focused interface may be required. These scenarios can utilize
Review Comment:
What "producer-focused" means here?
##########
docs/source/format/CDeviceDataInterface.rst:
##########
@@ -650,6 +652,340 @@ The stream source is not assumed to be thread-safe. Consumers wanting to
call ``get_next`` from several threads should ensure those calls are
serialized.
+Async Device Stream Interface
+=============================
+
+The :ref:`C stream interface <_c-device-stream-interface>` provides a synchronous
+API centered around the consumer calling the callback functions to retrieve
+the next record batch. For some bindings, use cases, and interoperability, a more
+asynchronous, producer-focused interface may be required. These scenarios can utilize
+the ``ArrowAsyncDeviceStreamHandler``.
+
+Semantics
+---------
+
+Rather than the producer providing a structure of callbacks for a consumer to
+call and retrieve records, the Async interface is a consumer allocated structure.
+The consumer allocated struct provides handler callbacks for the producer to call
+when the schema and chunks of data are available, rather than the consumer using
+a blocking pull-style iteration.
+
+In addition to the ``ArrowAsyncDeviceStreamHandler``, there are also two additional
+structs used for the full data flow: ``ArrowAsyncTask`` and ``ArrowAsyncProducer``.
+
+Structure Definition
+--------------------
+
+The C device async stream interface is defined with a single ``struct`` definition:
+
+.. code-block:: c
+
+ #ifndef ARROW_C_ASYNC_STREAM_INTERFACE
+ #define ARROW_C_ASYNC_STREAM_INTERFACE
+
+ struct ArrowAsyncTask {
+ int (*get_data)(struct ArrowArrayTask* self, struct ArrowDeviceArray* out);
+
+ void* private_data;
+ };
+
+ struct ArrowAsyncProducer {
+ void (*request)(struct ArrowAsyncProducer* self, int64_t n);
+ void (*cancel)(struct ArrowAsyncProducer* self);
+
+ void (*release)(struct ArrowAsyncProducer* self);
+ void* private_data;
+ };
+
+ struct ArrowAsyncDeviceStreamHandler {
+ // handlers
Review Comment:
```suggestion
// consumer-specific handlers
```
##########
docs/source/format/CDeviceDataInterface.rst:
##########
@@ -650,6 +652,340 @@ The stream source is not assumed to be thread-safe. Consumers wanting to
call ``get_next`` from several threads should ensure those calls are
serialized.
+Async Device Stream Interface
+=============================
+
+The :ref:`C stream interface <_c-device-stream-interface>` provides a synchronous
+API centered around the consumer calling the callback functions to retrieve
+the next record batch. For some bindings, use cases, and interoperability, a more
+asynchronous, producer-focused interface may be required. These scenarios can utilize
+the ``ArrowAsyncDeviceStreamHandler``.
+
+Semantics
+---------
+
+Rather than the producer providing a structure of callbacks for a consumer to
+call and retrieve records, the Async interface is a consumer allocated structure.
Review Comment:
```suggestion
call and retrieve records, the Async interface is a structure allocated and populated by the consumer.
```
##########
docs/source/format/CDeviceDataInterface.rst:
##########
@@ -650,6 +652,340 @@ The stream source is not assumed to be thread-safe. Consumers wanting to
call ``get_next`` from several threads should ensure those calls are
serialized.
+Async Device Stream Interface
+=============================
+
+The :ref:`C stream interface <_c-device-stream-interface>` provides a synchronous
+API centered around the consumer calling the callback functions to retrieve
+the next record batch. For some bindings, use cases, and interoperability, a more
+asynchronous, producer-focused interface may be required. These scenarios can utilize
+the ``ArrowAsyncDeviceStreamHandler``.
+
+Semantics
+---------
+
+Rather than the producer providing a structure of callbacks for a consumer to
+call and retrieve records, the Async interface is a consumer allocated structure.
+The consumer allocated struct provides handler callbacks for the producer to call
+when the schema and chunks of data are available, rather than the consumer using
+a blocking pull-style iteration.
+
+In addition to the ``ArrowAsyncDeviceStreamHandler``, there are also two additional
+structs used for the full data flow: ``ArrowAsyncTask`` and ``ArrowAsyncProducer``.
+
+Structure Definition
+--------------------
+
+The C device async stream interface is defined with a single ``struct`` definition:
+
+.. code-block:: c
+
+ #ifndef ARROW_C_ASYNC_STREAM_INTERFACE
+ #define ARROW_C_ASYNC_STREAM_INTERFACE
+
+ struct ArrowAsyncTask {
+ int (*get_data)(struct ArrowArrayTask* self, struct ArrowDeviceArray* out);
+
+ void* private_data;
+ };
+
+ struct ArrowAsyncProducer {
+ void (*request)(struct ArrowAsyncProducer* self, int64_t n);
+ void (*cancel)(struct ArrowAsyncProducer* self);
+
+ void (*release)(struct ArrowAsyncProducer* self);
+ void* private_data;
+ };
+
+ struct ArrowAsyncDeviceStreamHandler {
+ // handlers
+ int (*on_schema)(struct ArrowAsyncDeviceStreamHandler* self,
+ struct ArrowAsyncProducer* producer,
+ struct ArrowSchema* stream_schema, const char* addl_metadata);
+ int (*on_next_task)(struct ArrowAsyncDeviceStreamHandler* self,
+ struct ArrowAsyncTask* task, const char* metadata);
+ void (*on_error)(struct ArrowAsyncDeviceStreamHandler* self,
+ int code, const char* message, const char* metadata);
+
+ // release callback
+ void (*release)(struct ArrowAsyncDeviceStreamHandler* self);
+
+ // opaque handler-specific data
+ void* private_data;
+ };
+
+ #endif // ARROW_C_ASYNC_STREAM_INTERFACE
+
+.. note::
+ The canonical guard ``ARROW_C_ASYNC_STREAM_INTERFACE`` is meant to avoid
+ duplicate definitions if two projects copy the C async stream interface
+ definitions into their own headers, and a third-party project includes
+ from these two projects. It is therefore important that this guard is kept
+ exactly as-is when these definitions are copied.
+
+The ArrowAsyncDeviceStreamHandler structure
+'''''''''''''''''''''''''''''''''''''''''''
+
+The structure has the following fields:
+
+.. c:member:: int (*ArrowAsyncDeviceStreamHandler.on_schema)(struct ArrowAsyncDeviceStreamHandler*, struct ArrowAsyncProducer*, struct ArrowSchema*, const char*)
+
+ *Mandatory.* Handler for receiving the schema of the stream. All records should
+ match the provided schema. If successful, the function should return 0, otherwise
+ it should return an ``errno``-compatible error code.
+
+ The ``const char*`` parameter exists for producers to provide any extra contextual information
+ they want, such as the total number of rows in the stream, statistics, or otherwise. This is
+ encoded in the same format as :c:member:`ArrowSchema.metadata`. If not ``NULL``,
+ the lifetime is only the scope of the call to this function. A consumer who wants to maintain
+ the additional metadata beyond the lifetime of this call *MUST* copy the value themselves.
+
+ Unless the ``on_error`` handler is called, this will always get called exactly once and will be
+ the first method called on this object. As such the producer *MUST* provide an ``ArrowAsyncProducer``
+ object when calling this function to allow the consumer to manage back-pressure and flow control.
+ The producer maintains ownership of the ``ArrowAsyncProducer`` and must clean it up *after*
+ calling the release callback on this object.
+
+ A producer that receives a non-zero result here must not subsequently call anything other than
+ the release callback on this object.
+
+.. c:member:: int (*ArrowAsyncDeviceStreamHandler.on_next_task)(struct ArrowAsyncDeviceStreamHandler*, struct ArrowAsyncTask*, const char*)
+
+ *Mandatory.* Handler to be called when a new record is available for processing. The
+ schema for each record should be the same as the schema that ``on_schema`` was called with.
+ If successfully handled, the function should return 0, otherwise it should return an
+ ``errno``-compatible error code.
+
+ Rather than passing the record itself it receives an ``ArrowAsyncTask`` instead to facilitate
+ better consumer-focused thread control as far as receiving the data. A call to this function
+ simply indicates that data is available via the provided task.
+
+ The producer signals the end of the stream by passing ``NULL`` for the ``ArrowAsyncTask``
+ pointer instead of a valid address. This task object is only valid during the lifetime of
+ this function call. If the consumer wants to use the task beyond the scope of this method, it
+ must copy or move its contents to a new ArrowAsyncTask object.
+
+ The ``const char*`` parameter exists for producers to provide any extra contextual information
+ they want. This is encoded in the same format as :c:member:`ArrowSchema.metadata`. If not ``NULL``,
+ the lifetime is only the scope of the call to this function. A consumer who wants to maintain
+ the additional metadata beyond the lifetime of this call *MUST* copy the value themselves.
+
+ A producer *MUST NOT* call this concurrently from multiple threads.
+
+ The :c:member:`ArrowAsyncProducer.request` callback must be called to start receiving calls to this
+ handler.
+
+.. c:member:: void (*ArrowAsyncDeviceStreamHandler.on_error)(struct ArrowAsyncDeviceStreamHandler, int, const char*, const char*)
+
+ *Mandatory.* Handler to be called when an error is encountered by the producer. After calling
+ this, the ``release`` callback will be called as the last call on this struct. The parameters
+ are an ``errno``-compatible error code and an optional error message and metadata.
+
+ If the message and metadata are not ``NULL``, their lifetime is only valid during the scope
+ of this call. A consumer who wants to maintain these values past the return of this function
+ *MUST* copy the values themselves.
+
+ If the metadata parameter is not ``NULL``, to provide key-value error metadata, then it should
+ be encoded identically to the way that metadata is encoded in :c:member:`ArrowSchema.metadata`.
+
+ It is valid for this to be called by a producer with or without a preceding call to
+ :c:member:`ArrowAsyncProducer.request`. This callback *MUST NOT* call any methods of an
+ ``ArrowAsyncProducer`` object.
+
+.. c:member:: void (*ArrowAsyncDeviceStreamHandler.release)(struct ArrowAsyncDeviceStreamHandler*)
+
+ *Mandatory.* A pointer to a consumer-provided release callback for the handler.
+
+ It is valid for this to be called by a producer with or without a preceding call to
+ :c:member:`ArrowAsyncProducer.request`. This must not call any methods of an ``ArrowAsyncProducer``
+ object.
+
+.. c:member:: void* ArrowAsyncDeviceStreamHandler.private_data
+
+ *Optional.* An opaque pointer to consumer-provided private data.
+
+ Producers *MUST NOT* process this member. Lifetime of this member is handled by
+ the consumer, and especially by the release callback.
+
+The ArrowAsyncTask structure
+''''''''''''''''''''''''''''
+
+The purpose of using a Task object rather than passing the array directly to the ``on_next``
+callback is to allow for more complex and efficient thread handling. Utilizing a Task
+object allows for a producer to separate the "decoding" logic from the I/O, enabling a
+consumer to avoid transferring data between CPU cores (e.g. from one L1/L2 cache to another).
+
+This producer-provided structure has the following fields:
+
+.. c:member:: int (*ArrowArrayTask.get_data)(struct ArrowArrayTask*, struct ArrowDeviceArray*)
+
+ *Mandatory.* A callback to populate the provided ``ArrowDeviceArray`` with the available data.
+ The order of ArrowAsyncTasks provided by the producer enables a consumer to know the order of
+ the data to process. If the consumer does not care about the data that is owned by this task,
+ it must still call ``get_data`` so that the producer can perform any required cleanup. ``NULL``
+ should be passed as the device array pointer to indicate that the consumer doesn't want the
+ actual data, letting the task perform necessary cleanup.
Review Comment:
Oh, it's not a pure function. Yet another reason to call this `consume_data` instead of `get_data`.
##########
docs/source/format/CDeviceDataInterface.rst:
##########
@@ -650,6 +652,340 @@ The stream source is not assumed to be thread-safe. Consumers wanting to
call ``get_next`` from several threads should ensure those calls are
serialized.
+Async Device Stream Interface
+=============================
+
+The :ref:`C stream interface <_c-device-stream-interface>` provides a synchronous
+API centered around the consumer calling the callback functions to retrieve
+the next record batch. For some bindings, use cases, and interoperability, a more
+asynchronous, producer-focused interface may be required. These scenarios can utilize
+the ``ArrowAsyncDeviceStreamHandler``.
+
+Semantics
+---------
+
+Rather than the producer providing a structure of callbacks for a consumer to
+call and retrieve records, the Async interface is a consumer allocated structure.
+The consumer allocated struct provides handler callbacks for the producer to call
+when the schema and chunks of data are available, rather than the consumer using
+a blocking pull-style iteration.
+
+In addition to the ``ArrowAsyncDeviceStreamHandler``, there are also two additional
+structs used for the full data flow: ``ArrowAsyncTask`` and ``ArrowAsyncProducer``.
+
+Structure Definition
+--------------------
+
+The C device async stream interface is defined with a single ``struct`` definition:
+
+.. code-block:: c
+
+ #ifndef ARROW_C_ASYNC_STREAM_INTERFACE
+ #define ARROW_C_ASYNC_STREAM_INTERFACE
+
+ struct ArrowAsyncTask {
+ int (*get_data)(struct ArrowArrayTask* self, struct ArrowDeviceArray* out);
+
+ void* private_data;
+ };
+
+ struct ArrowAsyncProducer {
+ void (*request)(struct ArrowAsyncProducer* self, int64_t n);
+ void (*cancel)(struct ArrowAsyncProducer* self);
+
+ void (*release)(struct ArrowAsyncProducer* self);
+ void* private_data;
+ };
+
+ struct ArrowAsyncDeviceStreamHandler {
+ // handlers
+ int (*on_schema)(struct ArrowAsyncDeviceStreamHandler* self,
+ struct ArrowAsyncProducer* producer,
+ struct ArrowSchema* stream_schema, const char* addl_metadata);
+ int (*on_next_task)(struct ArrowAsyncDeviceStreamHandler* self,
+ struct ArrowAsyncTask* task, const char* metadata);
+ void (*on_error)(struct ArrowAsyncDeviceStreamHandler* self,
+ int code, const char* message, const char* metadata);
+
+ // release callback
+ void (*release)(struct ArrowAsyncDeviceStreamHandler* self);
+
+ // opaque handler-specific data
+ void* private_data;
+ };
+
+ #endif // ARROW_C_ASYNC_STREAM_INTERFACE
+
+.. note::
+ The canonical guard ``ARROW_C_ASYNC_STREAM_INTERFACE`` is meant to avoid
+ duplicate definitions if two projects copy the C async stream interface
+ definitions into their own headers, and a third-party project includes
+ from these two projects. It is therefore important that this guard is kept
+ exactly as-is when these definitions are copied.
+
+The ArrowAsyncDeviceStreamHandler structure
+'''''''''''''''''''''''''''''''''''''''''''
+
+The structure has the following fields:
+
+.. c:member:: int (*ArrowAsyncDeviceStreamHandler.on_schema)(struct ArrowAsyncDeviceStreamHandler*, struct ArrowAsyncProducer*, struct ArrowSchema*, const char*)
+
+ *Mandatory.* Handler for receiving the schema of the stream. All records should
+ match the provided schema. If successful, the function should return 0, otherwise
+ it should return an ``errno``-compatible error code.
+
+ The ``const char*`` parameter exists for producers to provide any extra contextual information
+ they want, such as the total number of rows in the stream, statistics, or otherwise. This is
+ encoded in the same format as :c:member:`ArrowSchema.metadata`. If not ``NULL``,
+ the lifetime is only the scope of the call to this function. A consumer who wants to maintain
+ the additional metadata beyond the lifetime of this call *MUST* copy the value themselves.
+
+ Unless the ``on_error`` handler is called, this will always get called exactly once and will be
+ the first method called on this object. As such the producer *MUST* provide an ``ArrowAsyncProducer``
+ object when calling this function to allow the consumer to manage back-pressure and flow control.
+ The producer maintains ownership of the ``ArrowAsyncProducer`` and must clean it up *after*
+ calling the release callback on this object.
+
+ A producer that receives a non-zero result here must not subsequently call anything other than
+ the release callback on this object.
+
+.. c:member:: int (*ArrowAsyncDeviceStreamHandler.on_next_task)(struct ArrowAsyncDeviceStreamHandler*, struct ArrowAsyncTask*, const char*)
+
+ *Mandatory.* Handler to be called when a new record is available for processing. The
+ schema for each record should be the same as the schema that ``on_schema`` was called with.
+ If successfully handled, the function should return 0, otherwise it should return an
+ ``errno``-compatible error code.
+
+ Rather than passing the record itself it receives an ``ArrowAsyncTask`` instead to facilitate
+ better consumer-focused thread control as far as receiving the data. A call to this function
+ simply indicates that data is available via the provided task.
+
+ The producer signals the end of the stream by passing ``NULL`` for the ``ArrowAsyncTask``
+ pointer instead of a valid address. This task object is only valid during the lifetime of
+ this function call. If the consumer wants to use the task beyond the scope of this method, it
+ must copy or move its contents to a new ArrowAsyncTask object.
+
+ The ``const char*`` parameter exists for producers to provide any extra contextual information
+ they want. This is encoded in the same format as :c:member:`ArrowSchema.metadata`. If not ``NULL``,
+ the lifetime is only the scope of the call to this function. A consumer who wants to maintain
+ the additional metadata beyond the lifetime of this call *MUST* copy the value themselves.
+
+ A producer *MUST NOT* call this concurrently from multiple threads.
+
+ The :c:member:`ArrowAsyncProducer.request` callback must be called to start receiving calls to this
+ handler.
+
+.. c:member:: void (*ArrowAsyncDeviceStreamHandler.on_error)(struct ArrowAsyncDeviceStreamHandler, int, const char*, const char*)
+
+ *Mandatory.* Handler to be called when an error is encountered by the producer. After calling
+ this, the ``release`` callback will be called as the last call on this struct. The parameters
+ are an ``errno``-compatible error code and an optional error message and metadata.
+
+ If the message and metadata are not ``NULL``, their lifetime is only valid during the scope
+ of this call. A consumer who wants to maintain these values past the return of this function
+ *MUST* copy the values themselves.
+
+ If the metadata parameter is not ``NULL``, to provide key-value error metadata, then it should
+ be encoded identically to the way that metadata is encoded in :c:member:`ArrowSchema.metadata`.
+
+ It is valid for this to be called by a producer with or without a preceding call to
+ :c:member:`ArrowAsyncProducer.request`. This callback *MUST NOT* call any methods of an
+ ``ArrowAsyncProducer`` object.
+
+.. c:member:: void (*ArrowAsyncDeviceStreamHandler.release)(struct ArrowAsyncDeviceStreamHandler*)
+
+ *Mandatory.* A pointer to a consumer-provided release callback for the handler.
+
+ It is valid for this to be called by a producer with or without a preceding call to
+ :c:member:`ArrowAsyncProducer.request`. This must not call any methods of an ``ArrowAsyncProducer``
+ object.
+
+.. c:member:: void* ArrowAsyncDeviceStreamHandler.private_data
+
+ *Optional.* An opaque pointer to consumer-provided private data.
+
+ Producers *MUST NOT* process this member. Lifetime of this member is handled by
+ the consumer, and especially by the release callback.
+
+The ArrowAsyncTask structure
+''''''''''''''''''''''''''''
+
+The purpose of using a Task object rather than passing the array directly to the ``on_next``
+callback is to allow for more complex and efficient thread handling. Utilizing a Task
+object allows for a producer to separate the "decoding" logic from the I/O, enabling a
+consumer to avoid transferring data between CPU cores (e.g. from one L1/L2 cache to another).
+
+This producer-provided structure has the following fields:
+
+.. c:member:: int (*ArrowArrayTask.get_data)(struct ArrowArrayTask*, struct ArrowDeviceArray*)
+
+ *Mandatory.* A callback to populate the provided ``ArrowDeviceArray`` with the available data.
+ The order of ArrowAsyncTasks provided by the producer enables a consumer to know the order of
+ the data to process. If the consumer does not care about the data that is owned by this task,
+ it must still call ``get_data`` so that the producer can perform any required cleanup. ``NULL``
+ should be passed as the device array pointer to indicate that the consumer doesn't want the
+ actual data, letting the task perform necessary cleanup.
+
+ If a non-zero value is returned from this, it should be followed only by the producer calling
+ the ``on_error`` callback of the ``ArrowAsyncDeviceStreamHandler``. Because calling this method
Review Comment:
This assumes the error returned here can only originate from the process that produced the task (i.e. the producer). Consuming the data in the task is forced to be infallible.
##########
docs/source/format/CDeviceDataInterface.rst:
##########
@@ -650,6 +652,340 @@ The stream source is not assumed to be thread-safe. Consumers wanting to
call ``get_next`` from several threads should ensure those calls are
serialized.
+Async Device Stream Interface
+=============================
+
+The :ref:`C stream interface <_c-device-stream-interface>` provides a synchronous
+API centered around the consumer calling the callback functions to retrieve
+the next record batch. For some bindings, use cases, and interoperability, a more
+asynchronous, producer-focused interface may be required. These scenarios can utilize
+the ``ArrowAsyncDeviceStreamHandler``.
+
+Semantics
+---------
+
+Rather than the producer providing a structure of callbacks for a consumer to
+call and retrieve records, the Async interface is a consumer allocated structure.
+The consumer allocated struct provides handler callbacks for the producer to call
+when the schema and chunks of data are available, rather than the consumer using
+a blocking pull-style iteration.
+
+In addition to the ``ArrowAsyncDeviceStreamHandler``, there are also two additional
+structs used for the full data flow: ``ArrowAsyncTask`` and ``ArrowAsyncProducer``.
+
+Structure Definition
+--------------------
+
+The C device async stream interface is defined with a single ``struct`` definition:
+
+.. code-block:: c
+
+ #ifndef ARROW_C_ASYNC_STREAM_INTERFACE
+ #define ARROW_C_ASYNC_STREAM_INTERFACE
+
+ struct ArrowAsyncTask {
+ int (*get_data)(struct ArrowArrayTask* self, struct ArrowDeviceArray* out);
+
+ void* private_data;
+ };
+
+ struct ArrowAsyncProducer {
+ void (*request)(struct ArrowAsyncProducer* self, int64_t n);
+ void (*cancel)(struct ArrowAsyncProducer* self);
+
+ void (*release)(struct ArrowAsyncProducer* self);
+ void* private_data;
+ };
+
+ struct ArrowAsyncDeviceStreamHandler {
+ // handlers
+ int (*on_schema)(struct ArrowAsyncDeviceStreamHandler* self,
+ struct ArrowAsyncProducer* producer,
+ struct ArrowSchema* stream_schema, const char* addl_metadata);
+ int (*on_next_task)(struct ArrowAsyncDeviceStreamHandler* self,
+ struct ArrowAsyncTask* task, const char* metadata);
+ void (*on_error)(struct ArrowAsyncDeviceStreamHandler* self,
+ int code, const char* message, const char* metadata);
+
+ // release callback
+ void (*release)(struct ArrowAsyncDeviceStreamHandler* self);
+
+ // opaque handler-specific data
+ void* private_data;
+ };
+
+ #endif // ARROW_C_ASYNC_STREAM_INTERFACE
+
+.. note::
+ The canonical guard ``ARROW_C_ASYNC_STREAM_INTERFACE`` is meant to avoid
+ duplicate definitions if two projects copy the C async stream interface
+ definitions into their own headers, and a third-party project includes
+ from these two projects. It is therefore important that this guard is kept
+ exactly as-is when these definitions are copied.
+
+The ArrowAsyncDeviceStreamHandler structure
+'''''''''''''''''''''''''''''''''''''''''''
+
+The structure has the following fields:
+
+.. c:member:: int (*ArrowAsyncDeviceStreamHandler.on_schema)(struct ArrowAsyncDeviceStreamHandler*, struct ArrowAsyncProducer*, struct ArrowSchema*, const char*)
+
+ *Mandatory.* Handler for receiving the schema of the stream. All records should
+ match the provided schema. If successful, the function should return 0, otherwise
+ it should return an ``errno``-compatible error code.
+
+ The ``const char*`` parameter exists for producers to provide any extra contextual information
+ they want, such as the total number of rows in the stream, statistics, or otherwise. This is
+ encoded in the same format as :c:member:`ArrowSchema.metadata`. If not ``NULL``,
+ the lifetime is only the scope of the call to this function. A consumer who wants to maintain
+ the additional metadata beyond the lifetime of this call *MUST* copy the value themselves.
+
+ Unless the ``on_error`` handler is called, this will always get called exactly once and will be
+ the first method called on this object. As such the producer *MUST* provide an ``ArrowAsyncProducer``
+ object when calling this function to allow the consumer to manage back-pressure and flow control.
+ The producer maintains ownership of the ``ArrowAsyncProducer`` and must clean it up *after*
+ calling the release callback on this object.
+
+ A producer that receives a non-zero result here must not subsequently call anything other than
+ the release callback on this object.
+
+.. c:member:: int (*ArrowAsyncDeviceStreamHandler.on_next_task)(struct ArrowAsyncDeviceStreamHandler*, struct ArrowAsyncTask*, const char*)
+
+ *Mandatory.* Handler to be called when a new record is available for processing. The
+ schema for each record should be the same as the schema that ``on_schema`` was called with.
+ If successfully handled, the function should return 0, otherwise it should return an
+ ``errno``-compatible error code.
+
+ Rather than passing the record itself it receives an ``ArrowAsyncTask`` instead to facilitate
+ better consumer-focused thread control as far as receiving the data. A call to this function
+ simply indicates that data is available via the provided task.
+
+ The producer signals the end of the stream by passing ``NULL`` for the ``ArrowAsyncTask``
+ pointer instead of a valid address. This task object is only valid during the lifetime of
+ this function call. If the consumer wants to use the task beyond the scope of this method, it
+ must copy or move its contents to a new ArrowAsyncTask object.
+
+ The ``const char*`` parameter exists for producers to provide any extra contextual information
+ they want. This is encoded in the same format as :c:member:`ArrowSchema.metadata`. If not ``NULL``,
+ the lifetime is only the scope of the call to this function. A consumer who wants to maintain
+ the additional metadata beyond the lifetime of this call *MUST* copy the value themselves.
+
+ A producer *MUST NOT* call this concurrently from multiple threads.
+
+ The :c:member:`ArrowAsyncProducer.request` callback must be called to start receiving calls to this
+ handler.
+
+.. c:member:: void (*ArrowAsyncDeviceStreamHandler.on_error)(struct ArrowAsyncDeviceStreamHandler, int, const char*, const char*)
+
+ *Mandatory.* Handler to be called when an error is encountered by the producer. After calling
+ this, the ``release`` callback will be called as the last call on this struct. The parameters
+ are an ``errno``-compatible error code and an optional error message and metadata.
+
+ If the message and metadata are not ``NULL``, their lifetime is only valid during the scope
+ of this call. A consumer who wants to maintain these values past the return of this function
+ *MUST* copy the values themselves.
+
+ If the metadata parameter is not ``NULL``, to provide key-value error metadata, then it should
+ be encoded identically to the way that metadata is encoded in :c:member:`ArrowSchema.metadata`.
+
+ It is valid for this to be called by a producer with or without a preceding call to
+ :c:member:`ArrowAsyncProducer.request`. This callback *MUST NOT* call any methods of an
+ ``ArrowAsyncProducer`` object.
+
+.. c:member:: void (*ArrowAsyncDeviceStreamHandler.release)(struct ArrowAsyncDeviceStreamHandler*)
+
+ *Mandatory.* A pointer to a consumer-provided release callback for the handler.
+
+ It is valid for this to be called by a producer with or without a preceding call to
+ :c:member:`ArrowAsyncProducer.request`. This must not call any methods of an ``ArrowAsyncProducer``
+ object.
+
+.. c:member:: void* ArrowAsyncDeviceStreamHandler.private_data
+
+ *Optional.* An opaque pointer to consumer-provided private data.
+
+ Producers *MUST NOT* process this member. Lifetime of this member is handled by
+ the consumer, and especially by the release callback.
+
+The ArrowAsyncTask structure
+''''''''''''''''''''''''''''
+
+The purpose of using a Task object rather than passing the array directly to the ``on_next``
+callback is to allow for more complex and efficient thread handling. Utilizing a Task
+object allows for a producer to separate the "decoding" logic from the I/O, enabling a
+consumer to avoid transferring data between CPU cores (e.g. from one L1/L2 cache to another).
+
+This producer-provided structure has the following fields:
+
+.. c:member:: int (*ArrowArrayTask.get_data)(struct ArrowArrayTask*, struct ArrowDeviceArray*)
+
+ *Mandatory.* A callback to populate the provided ``ArrowDeviceArray`` with the available data.
+ The order of ArrowAsyncTasks provided by the producer enables a consumer to know the order of
+ the data to process. If the consumer does not care about the data that is owned by this task,
+ it must still call ``get_data`` so that the producer can perform any required cleanup. ``NULL``
+ should be passed as the device array pointer to indicate that the consumer doesn't want the
+ actual data, letting the task perform necessary cleanup.
+
+ If a non-zero value is returned from this, it should be followed only by the producer calling
+ the ``on_error`` callback of the ``ArrowAsyncDeviceStreamHandler``. Because calling this method
+ is likely to be separate from the current control flow, returning a non-zero value to signal
+ an error occuring allows the current thread to decide handle the case accordingly, while still
+ allowing all error logging and handling to be centralized in the
+ :c:member:`ArrowAsyncDeviceStreamHandler.on_error` callback.
+
+ Rather than having a separate release callback, any required cleanup should be performed as part
+ of the invocation of this callback. Ownership of the Array is given to the pointer passed in as
+ a parameter, and this array must be released separately.
+
+ It is only valid to call this method exactly once.
+
+.. c:member:: void* ArrowArrayTask.private_data
+
+ *Optional.* An opaque pointer to producer-provided private data.
+
+ Consumers *MUST NOT* process this member. Lifetime of this member is handled by
+ the producer who created this object, and should be cleaned up if necessary during
+ the call to :c:member:`ArrowArrayTask.get_data`.
Review Comment:
```suggestion
Consumers *MUST NOT* process this member. Lifetime of this member is initiated by
the producer when creating this object, and cleanup occurs during
the call to :c:member:`ArrowArrayTask.consume_data`.
```
##########
docs/source/format/CDeviceDataInterface.rst:
##########
@@ -650,6 +652,340 @@ The stream source is not assumed to be thread-safe. Consumers wanting to
call ``get_next`` from several threads should ensure those calls are
serialized.
+Async Device Stream Interface
+=============================
+
+The :ref:`C stream interface <_c-device-stream-interface>` provides a synchronous
+API centered around the consumer calling the callback functions to retrieve
+the next record batch. For some bindings, use cases, and interoperability, a more
+asynchronous, producer-focused interface may be required. These scenarios can utilize
+the ``ArrowAsyncDeviceStreamHandler``.
+
+Semantics
+---------
+
+Rather than the producer providing a structure of callbacks for a consumer to
+call and retrieve records, the Async interface is a consumer allocated structure.
+The consumer allocated struct provides handler callbacks for the producer to call
+when the schema and chunks of data are available, rather than the consumer using
+a blocking pull-style iteration.
+
+In addition to the ``ArrowAsyncDeviceStreamHandler``, there are also two additional
+structs used for the full data flow: ``ArrowAsyncTask`` and ``ArrowAsyncProducer``.
+
+Structure Definition
+--------------------
+
+The C device async stream interface is defined with a single ``struct`` definition:
+
+.. code-block:: c
+
+ #ifndef ARROW_C_ASYNC_STREAM_INTERFACE
+ #define ARROW_C_ASYNC_STREAM_INTERFACE
+
+ struct ArrowAsyncTask {
+ int (*get_data)(struct ArrowArrayTask* self, struct ArrowDeviceArray* out);
+
+ void* private_data;
+ };
+
+ struct ArrowAsyncProducer {
+ void (*request)(struct ArrowAsyncProducer* self, int64_t n);
+ void (*cancel)(struct ArrowAsyncProducer* self);
+
+ void (*release)(struct ArrowAsyncProducer* self);
+ void* private_data;
+ };
+
+ struct ArrowAsyncDeviceStreamHandler {
+ // handlers
+ int (*on_schema)(struct ArrowAsyncDeviceStreamHandler* self,
+ struct ArrowAsyncProducer* producer,
+ struct ArrowSchema* stream_schema, const char* addl_metadata);
+ int (*on_next_task)(struct ArrowAsyncDeviceStreamHandler* self,
+ struct ArrowAsyncTask* task, const char* metadata);
+ void (*on_error)(struct ArrowAsyncDeviceStreamHandler* self,
+ int code, const char* message, const char* metadata);
+
+ // release callback
+ void (*release)(struct ArrowAsyncDeviceStreamHandler* self);
+
+ // opaque handler-specific data
+ void* private_data;
+ };
+
+ #endif // ARROW_C_ASYNC_STREAM_INTERFACE
+
+.. note::
+ The canonical guard ``ARROW_C_ASYNC_STREAM_INTERFACE`` is meant to avoid
+ duplicate definitions if two projects copy the C async stream interface
+ definitions into their own headers, and a third-party project includes
+ from these two projects. It is therefore important that this guard is kept
+ exactly as-is when these definitions are copied.
+
+The ArrowAsyncDeviceStreamHandler structure
+'''''''''''''''''''''''''''''''''''''''''''
+
+The structure has the following fields:
+
+.. c:member:: int (*ArrowAsyncDeviceStreamHandler.on_schema)(struct ArrowAsyncDeviceStreamHandler*, struct ArrowAsyncProducer*, struct ArrowSchema*, const char*)
+
+ *Mandatory.* Handler for receiving the schema of the stream. All records should
+ match the provided schema. If successful, the function should return 0, otherwise
+ it should return an ``errno``-compatible error code.
Review Comment:
Link to `errno` specification of choice here maybe?
##########
docs/source/format/CDeviceDataInterface.rst:
##########
@@ -650,6 +652,340 @@ The stream source is not assumed to be thread-safe. Consumers wanting to
call ``get_next`` from several threads should ensure those calls are
serialized.
+Async Device Stream Interface
+=============================
+
+The :ref:`C stream interface <_c-device-stream-interface>` provides a synchronous
+API centered around the consumer calling the callback functions to retrieve
+the next record batch. For some bindings, use cases, and interoperability, a more
+asynchronous, producer-focused interface may be required. These scenarios can utilize
+the ``ArrowAsyncDeviceStreamHandler``.
+
+Semantics
+---------
+
+Rather than the producer providing a structure of callbacks for a consumer to
+call and retrieve records, the Async interface is a consumer allocated structure.
+The consumer allocated struct provides handler callbacks for the producer to call
+when the schema and chunks of data are available, rather than the consumer using
+a blocking pull-style iteration.
+
+In addition to the ``ArrowAsyncDeviceStreamHandler``, there are also two additional
+structs used for the full data flow: ``ArrowAsyncTask`` and ``ArrowAsyncProducer``.
+
+Structure Definition
+--------------------
+
+The C device async stream interface is defined with a single ``struct`` definition:
+
+.. code-block:: c
+
+ #ifndef ARROW_C_ASYNC_STREAM_INTERFACE
+ #define ARROW_C_ASYNC_STREAM_INTERFACE
+
+ struct ArrowAsyncTask {
+ int (*get_data)(struct ArrowArrayTask* self, struct ArrowDeviceArray* out);
+
+ void* private_data;
+ };
+
+ struct ArrowAsyncProducer {
+ void (*request)(struct ArrowAsyncProducer* self, int64_t n);
+ void (*cancel)(struct ArrowAsyncProducer* self);
+
+ void (*release)(struct ArrowAsyncProducer* self);
+ void* private_data;
+ };
+
+ struct ArrowAsyncDeviceStreamHandler {
+ // handlers
+ int (*on_schema)(struct ArrowAsyncDeviceStreamHandler* self,
+ struct ArrowAsyncProducer* producer,
+ struct ArrowSchema* stream_schema, const char* addl_metadata);
+ int (*on_next_task)(struct ArrowAsyncDeviceStreamHandler* self,
+ struct ArrowAsyncTask* task, const char* metadata);
+ void (*on_error)(struct ArrowAsyncDeviceStreamHandler* self,
+ int code, const char* message, const char* metadata);
+
+ // release callback
+ void (*release)(struct ArrowAsyncDeviceStreamHandler* self);
+
+ // opaque handler-specific data
+ void* private_data;
+ };
+
+ #endif // ARROW_C_ASYNC_STREAM_INTERFACE
+
+.. note::
+ The canonical guard ``ARROW_C_ASYNC_STREAM_INTERFACE`` is meant to avoid
+ duplicate definitions if two projects copy the C async stream interface
+ definitions into their own headers, and a third-party project includes
+ from these two projects. It is therefore important that this guard is kept
+ exactly as-is when these definitions are copied.
+
+The ArrowAsyncDeviceStreamHandler structure
+'''''''''''''''''''''''''''''''''''''''''''
+
+The structure has the following fields:
+
+.. c:member:: int (*ArrowAsyncDeviceStreamHandler.on_schema)(struct ArrowAsyncDeviceStreamHandler*, struct ArrowAsyncProducer*, struct ArrowSchema*, const char*)
+
+ *Mandatory.* Handler for receiving the schema of the stream. All records should
+ match the provided schema. If successful, the function should return 0, otherwise
+ it should return an ``errno``-compatible error code.
+
+ The ``const char*`` parameter exists for producers to provide any extra contextual information
+ they want, such as the total number of rows in the stream, statistics, or otherwise. This is
+ encoded in the same format as :c:member:`ArrowSchema.metadata`. If not ``NULL``,
+ the lifetime is only the scope of the call to this function. A consumer who wants to maintain
+ the additional metadata beyond the lifetime of this call *MUST* copy the value themselves.
+
+ Unless the ``on_error`` handler is called, this will always get called exactly once and will be
+ the first method called on this object. As such the producer *MUST* provide an ``ArrowAsyncProducer``
+ object when calling this function to allow the consumer to manage back-pressure and flow control.
Review Comment:
"manage back-pressure" is synonym with "flow control in this context.
```suggestion
object when calling this function to allow the consumer to apply back-pressure and control the flow of data.
```
##########
docs/source/format/CDeviceDataInterface.rst:
##########
@@ -650,6 +652,340 @@ The stream source is not assumed to be thread-safe. Consumers wanting to
call ``get_next`` from several threads should ensure those calls are
serialized.
+Async Device Stream Interface
+=============================
+
+The :ref:`C stream interface <_c-device-stream-interface>` provides a synchronous
+API centered around the consumer calling the callback functions to retrieve
+the next record batch. For some bindings, use cases, and interoperability, a more
+asynchronous, producer-focused interface may be required. These scenarios can utilize
+the ``ArrowAsyncDeviceStreamHandler``.
+
+Semantics
+---------
+
+Rather than the producer providing a structure of callbacks for a consumer to
+call and retrieve records, the Async interface is a consumer allocated structure.
+The consumer allocated struct provides handler callbacks for the producer to call
+when the schema and chunks of data are available, rather than the consumer using
+a blocking pull-style iteration.
Review Comment:
This repeats what was said on the first line of this paragraph.
```suggestion
when the schema and chunks of data are available.
```
##########
docs/source/format/CDeviceDataInterface.rst:
##########
@@ -650,6 +652,340 @@ The stream source is not assumed to be thread-safe. Consumers wanting to
call ``get_next`` from several threads should ensure those calls are
serialized.
+Async Device Stream Interface
+=============================
+
+The :ref:`C stream interface <_c-device-stream-interface>` provides a synchronous
+API centered around the consumer calling the callback functions to retrieve
+the next record batch. For some bindings, use cases, and interoperability, a more
+asynchronous, producer-focused interface may be required. These scenarios can utilize
+the ``ArrowAsyncDeviceStreamHandler``.
Review Comment:
This paragraph is good as it states the problem before the solution, but it leaves the goals somewhat implicit.
```suggestion
the next record batch. For concurrent communication between producer and consumer,
the ``ArrowAsyncDeviceStreamHandler`` can be used. This interface is non-opinionated
and may fit into different concurrent communication models.
```
##########
docs/source/format/CDeviceDataInterface.rst:
##########
@@ -650,6 +652,340 @@ The stream source is not assumed to be thread-safe. Consumers wanting to
call ``get_next`` from several threads should ensure those calls are
serialized.
+Async Device Stream Interface
+=============================
+
+The :ref:`C stream interface <_c-device-stream-interface>` provides a synchronous
+API centered around the consumer calling the callback functions to retrieve
+the next record batch. For some bindings, use cases, and interoperability, a more
+asynchronous, producer-focused interface may be required. These scenarios can utilize
+the ``ArrowAsyncDeviceStreamHandler``.
+
+Semantics
+---------
+
+Rather than the producer providing a structure of callbacks for a consumer to
+call and retrieve records, the Async interface is a consumer allocated structure.
+The consumer allocated struct provides handler callbacks for the producer to call
+when the schema and chunks of data are available, rather than the consumer using
+a blocking pull-style iteration.
+
+In addition to the ``ArrowAsyncDeviceStreamHandler``, there are also two additional
+structs used for the full data flow: ``ArrowAsyncTask`` and ``ArrowAsyncProducer``.
+
+Structure Definition
+--------------------
+
+The C device async stream interface is defined with a single ``struct`` definition:
Review Comment:
I see 3 structs :)
```suggestion
The C device async stream interface consists of three ``struct`` definitions:
```
##########
docs/source/format/CDeviceDataInterface.rst:
##########
@@ -650,6 +652,340 @@ The stream source is not assumed to be thread-safe. Consumers wanting to
call ``get_next`` from several threads should ensure those calls are
serialized.
+Async Device Stream Interface
+=============================
+
+The :ref:`C stream interface <_c-device-stream-interface>` provides a synchronous
+API centered around the consumer calling the callback functions to retrieve
+the next record batch. For some bindings, use cases, and interoperability, a more
+asynchronous, producer-focused interface may be required. These scenarios can utilize
+the ``ArrowAsyncDeviceStreamHandler``.
+
+Semantics
+---------
+
+Rather than the producer providing a structure of callbacks for a consumer to
+call and retrieve records, the Async interface is a consumer allocated structure.
+The consumer allocated struct provides handler callbacks for the producer to call
+when the schema and chunks of data are available, rather than the consumer using
+a blocking pull-style iteration.
+
+In addition to the ``ArrowAsyncDeviceStreamHandler``, there are also two additional
+structs used for the full data flow: ``ArrowAsyncTask`` and ``ArrowAsyncProducer``.
+
+Structure Definition
+--------------------
+
+The C device async stream interface is defined with a single ``struct`` definition:
+
+.. code-block:: c
+
+ #ifndef ARROW_C_ASYNC_STREAM_INTERFACE
+ #define ARROW_C_ASYNC_STREAM_INTERFACE
+
+ struct ArrowAsyncTask {
+ int (*get_data)(struct ArrowArrayTask* self, struct ArrowDeviceArray* out);
+
+ void* private_data;
+ };
+
+ struct ArrowAsyncProducer {
+ void (*request)(struct ArrowAsyncProducer* self, int64_t n);
+ void (*cancel)(struct ArrowAsyncProducer* self);
+
+ void (*release)(struct ArrowAsyncProducer* self);
+ void* private_data;
+ };
+
+ struct ArrowAsyncDeviceStreamHandler {
+ // handlers
+ int (*on_schema)(struct ArrowAsyncDeviceStreamHandler* self,
+ struct ArrowAsyncProducer* producer,
+ struct ArrowSchema* stream_schema, const char* addl_metadata);
+ int (*on_next_task)(struct ArrowAsyncDeviceStreamHandler* self,
+ struct ArrowAsyncTask* task, const char* metadata);
+ void (*on_error)(struct ArrowAsyncDeviceStreamHandler* self,
+ int code, const char* message, const char* metadata);
+
+ // release callback
+ void (*release)(struct ArrowAsyncDeviceStreamHandler* self);
+
+ // opaque handler-specific data
+ void* private_data;
+ };
+
+ #endif // ARROW_C_ASYNC_STREAM_INTERFACE
+
+.. note::
+ The canonical guard ``ARROW_C_ASYNC_STREAM_INTERFACE`` is meant to avoid
+ duplicate definitions if two projects copy the C async stream interface
+ definitions into their own headers, and a third-party project includes
+ from these two projects. It is therefore important that this guard is kept
+ exactly as-is when these definitions are copied.
+
+The ArrowAsyncDeviceStreamHandler structure
+'''''''''''''''''''''''''''''''''''''''''''
+
+The structure has the following fields:
+
+.. c:member:: int (*ArrowAsyncDeviceStreamHandler.on_schema)(struct ArrowAsyncDeviceStreamHandler*, struct ArrowAsyncProducer*, struct ArrowSchema*, const char*)
+
+ *Mandatory.* Handler for receiving the schema of the stream. All records should
Review Comment:
```suggestion
*Mandatory.* Handler for receiving the schema of the stream. All incoming records should
```
##########
docs/source/format/CDeviceDataInterface.rst:
##########
@@ -650,6 +652,340 @@ The stream source is not assumed to be thread-safe. Consumers wanting to
call ``get_next`` from several threads should ensure those calls are
serialized.
+Async Device Stream Interface
+=============================
+
+The :ref:`C stream interface <_c-device-stream-interface>` provides a synchronous
+API centered around the consumer calling the callback functions to retrieve
+the next record batch. For some bindings, use cases, and interoperability, a more
+asynchronous, producer-focused interface may be required. These scenarios can utilize
+the ``ArrowAsyncDeviceStreamHandler``.
+
+Semantics
+---------
+
+Rather than the producer providing a structure of callbacks for a consumer to
+call and retrieve records, the Async interface is a consumer allocated structure.
+The consumer allocated struct provides handler callbacks for the producer to call
+when the schema and chunks of data are available, rather than the consumer using
+a blocking pull-style iteration.
+
+In addition to the ``ArrowAsyncDeviceStreamHandler``, there are also two additional
+structs used for the full data flow: ``ArrowAsyncTask`` and ``ArrowAsyncProducer``.
+
+Structure Definition
+--------------------
+
+The C device async stream interface is defined with a single ``struct`` definition:
+
+.. code-block:: c
+
+ #ifndef ARROW_C_ASYNC_STREAM_INTERFACE
+ #define ARROW_C_ASYNC_STREAM_INTERFACE
+
+ struct ArrowAsyncTask {
+ int (*get_data)(struct ArrowArrayTask* self, struct ArrowDeviceArray* out);
+
+ void* private_data;
+ };
+
+ struct ArrowAsyncProducer {
+ void (*request)(struct ArrowAsyncProducer* self, int64_t n);
+ void (*cancel)(struct ArrowAsyncProducer* self);
+
+ void (*release)(struct ArrowAsyncProducer* self);
+ void* private_data;
+ };
+
+ struct ArrowAsyncDeviceStreamHandler {
+ // handlers
+ int (*on_schema)(struct ArrowAsyncDeviceStreamHandler* self,
+ struct ArrowAsyncProducer* producer,
+ struct ArrowSchema* stream_schema, const char* addl_metadata);
+ int (*on_next_task)(struct ArrowAsyncDeviceStreamHandler* self,
+ struct ArrowAsyncTask* task, const char* metadata);
+ void (*on_error)(struct ArrowAsyncDeviceStreamHandler* self,
+ int code, const char* message, const char* metadata);
+
+ // release callback
+ void (*release)(struct ArrowAsyncDeviceStreamHandler* self);
+
+ // opaque handler-specific data
+ void* private_data;
+ };
+
+ #endif // ARROW_C_ASYNC_STREAM_INTERFACE
+
+.. note::
+ The canonical guard ``ARROW_C_ASYNC_STREAM_INTERFACE`` is meant to avoid
+ duplicate definitions if two projects copy the C async stream interface
+ definitions into their own headers, and a third-party project includes
+ from these two projects. It is therefore important that this guard is kept
+ exactly as-is when these definitions are copied.
+
+The ArrowAsyncDeviceStreamHandler structure
+'''''''''''''''''''''''''''''''''''''''''''
+
+The structure has the following fields:
+
+.. c:member:: int (*ArrowAsyncDeviceStreamHandler.on_schema)(struct ArrowAsyncDeviceStreamHandler*, struct ArrowAsyncProducer*, struct ArrowSchema*, const char*)
+
+ *Mandatory.* Handler for receiving the schema of the stream. All records should
+ match the provided schema. If successful, the function should return 0, otherwise
+ it should return an ``errno``-compatible error code.
+
+ The ``const char*`` parameter exists for producers to provide any extra contextual information
+ they want, such as the total number of rows in the stream, statistics, or otherwise. This is
+ encoded in the same format as :c:member:`ArrowSchema.metadata`. If not ``NULL``,
+ the lifetime is only the scope of the call to this function. A consumer who wants to maintain
+ the additional metadata beyond the lifetime of this call *MUST* copy the value themselves.
+
+ Unless the ``on_error`` handler is called, this will always get called exactly once and will be
+ the first method called on this object. As such the producer *MUST* provide an ``ArrowAsyncProducer``
+ object when calling this function to allow the consumer to manage back-pressure and flow control.
+ The producer maintains ownership of the ``ArrowAsyncProducer`` and must clean it up *after*
+ calling the release callback on this object.
Review Comment:
```suggestion
calling the release callback on the ``ArrowAsyncDeviceStreamHandler``.
```
##########
docs/source/format/CDeviceDataInterface.rst:
##########
@@ -650,6 +652,340 @@ The stream source is not assumed to be thread-safe. Consumers wanting to
call ``get_next`` from several threads should ensure those calls are
serialized.
+Async Device Stream Interface
+=============================
+
+The :ref:`C stream interface <_c-device-stream-interface>` provides a synchronous
+API centered around the consumer calling the callback functions to retrieve
+the next record batch. For some bindings, use cases, and interoperability, a more
+asynchronous, producer-focused interface may be required. These scenarios can utilize
+the ``ArrowAsyncDeviceStreamHandler``.
+
+Semantics
+---------
+
+Rather than the producer providing a structure of callbacks for a consumer to
+call and retrieve records, the Async interface is a consumer allocated structure.
+The consumer allocated struct provides handler callbacks for the producer to call
+when the schema and chunks of data are available, rather than the consumer using
+a blocking pull-style iteration.
+
+In addition to the ``ArrowAsyncDeviceStreamHandler``, there are also two additional
+structs used for the full data flow: ``ArrowAsyncTask`` and ``ArrowAsyncProducer``.
+
+Structure Definition
+--------------------
+
+The C device async stream interface is defined with a single ``struct`` definition:
+
+.. code-block:: c
+
+ #ifndef ARROW_C_ASYNC_STREAM_INTERFACE
+ #define ARROW_C_ASYNC_STREAM_INTERFACE
+
+ struct ArrowAsyncTask {
+ int (*get_data)(struct ArrowArrayTask* self, struct ArrowDeviceArray* out);
+
+ void* private_data;
+ };
+
+ struct ArrowAsyncProducer {
+ void (*request)(struct ArrowAsyncProducer* self, int64_t n);
+ void (*cancel)(struct ArrowAsyncProducer* self);
+
+ void (*release)(struct ArrowAsyncProducer* self);
+ void* private_data;
+ };
+
+ struct ArrowAsyncDeviceStreamHandler {
+ // handlers
+ int (*on_schema)(struct ArrowAsyncDeviceStreamHandler* self,
+ struct ArrowAsyncProducer* producer,
+ struct ArrowSchema* stream_schema, const char* addl_metadata);
+ int (*on_next_task)(struct ArrowAsyncDeviceStreamHandler* self,
+ struct ArrowAsyncTask* task, const char* metadata);
+ void (*on_error)(struct ArrowAsyncDeviceStreamHandler* self,
+ int code, const char* message, const char* metadata);
+
+ // release callback
+ void (*release)(struct ArrowAsyncDeviceStreamHandler* self);
+
+ // opaque handler-specific data
+ void* private_data;
+ };
+
+ #endif // ARROW_C_ASYNC_STREAM_INTERFACE
+
+.. note::
+ The canonical guard ``ARROW_C_ASYNC_STREAM_INTERFACE`` is meant to avoid
+ duplicate definitions if two projects copy the C async stream interface
+ definitions into their own headers, and a third-party project includes
+ from these two projects. It is therefore important that this guard is kept
+ exactly as-is when these definitions are copied.
+
+The ArrowAsyncDeviceStreamHandler structure
+'''''''''''''''''''''''''''''''''''''''''''
+
+The structure has the following fields:
+
+.. c:member:: int (*ArrowAsyncDeviceStreamHandler.on_schema)(struct ArrowAsyncDeviceStreamHandler*, struct ArrowAsyncProducer*, struct ArrowSchema*, const char*)
+
+ *Mandatory.* Handler for receiving the schema of the stream. All records should
+ match the provided schema. If successful, the function should return 0, otherwise
+ it should return an ``errno``-compatible error code.
+
+ The ``const char*`` parameter exists for producers to provide any extra contextual information
+ they want, such as the total number of rows in the stream, statistics, or otherwise. This is
+ encoded in the same format as :c:member:`ArrowSchema.metadata`. If not ``NULL``,
+ the lifetime is only the scope of the call to this function. A consumer who wants to maintain
+ the additional metadata beyond the lifetime of this call *MUST* copy the value themselves.
+
+ Unless the ``on_error`` handler is called, this will always get called exactly once and will be
+ the first method called on this object. As such the producer *MUST* provide an ``ArrowAsyncProducer``
+ object when calling this function to allow the consumer to manage back-pressure and flow control.
+ The producer maintains ownership of the ``ArrowAsyncProducer`` and must clean it up *after*
+ calling the release callback on this object.
+
+ A producer that receives a non-zero result here must not subsequently call anything other than
+ the release callback on this object.
+
+.. c:member:: int (*ArrowAsyncDeviceStreamHandler.on_next_task)(struct ArrowAsyncDeviceStreamHandler*, struct ArrowAsyncTask*, const char*)
+
+ *Mandatory.* Handler to be called when a new record is available for processing. The
+ schema for each record should be the same as the schema that ``on_schema`` was called with.
+ If successfully handled, the function should return 0, otherwise it should return an
+ ``errno``-compatible error code.
+
+ Rather than passing the record itself it receives an ``ArrowAsyncTask`` instead to facilitate
+ better consumer-focused thread control as far as receiving the data. A call to this function
+ simply indicates that data is available via the provided task.
+
+ The producer signals the end of the stream by passing ``NULL`` for the ``ArrowAsyncTask``
+ pointer instead of a valid address. This task object is only valid during the lifetime of
+ this function call. If the consumer wants to use the task beyond the scope of this method, it
+ must copy or move its contents to a new ArrowAsyncTask object.
Review Comment:
Do you think it makes sense to change `get_data` to `consume_data` and replace "must copy or move" with "must consume the data in the task"?
##########
docs/source/format/CDeviceDataInterface.rst:
##########
@@ -650,6 +652,340 @@ The stream source is not assumed to be thread-safe. Consumers wanting to
call ``get_next`` from several threads should ensure those calls are
serialized.
+Async Device Stream Interface
+=============================
+
+The :ref:`C stream interface <_c-device-stream-interface>` provides a synchronous
+API centered around the consumer calling the callback functions to retrieve
+the next record batch. For some bindings, use cases, and interoperability, a more
+asynchronous, producer-focused interface may be required. These scenarios can utilize
+the ``ArrowAsyncDeviceStreamHandler``.
+
+Semantics
+---------
+
+Rather than the producer providing a structure of callbacks for a consumer to
+call and retrieve records, the Async interface is a consumer allocated structure.
+The consumer allocated struct provides handler callbacks for the producer to call
+when the schema and chunks of data are available, rather than the consumer using
+a blocking pull-style iteration.
+
+In addition to the ``ArrowAsyncDeviceStreamHandler``, there are also two additional
+structs used for the full data flow: ``ArrowAsyncTask`` and ``ArrowAsyncProducer``.
+
+Structure Definition
+--------------------
+
+The C device async stream interface is defined with a single ``struct`` definition:
+
+.. code-block:: c
+
+ #ifndef ARROW_C_ASYNC_STREAM_INTERFACE
+ #define ARROW_C_ASYNC_STREAM_INTERFACE
+
+ struct ArrowAsyncTask {
+ int (*get_data)(struct ArrowArrayTask* self, struct ArrowDeviceArray* out);
+
+ void* private_data;
+ };
+
+ struct ArrowAsyncProducer {
+ void (*request)(struct ArrowAsyncProducer* self, int64_t n);
+ void (*cancel)(struct ArrowAsyncProducer* self);
+
+ void (*release)(struct ArrowAsyncProducer* self);
+ void* private_data;
+ };
+
+ struct ArrowAsyncDeviceStreamHandler {
+ // handlers
+ int (*on_schema)(struct ArrowAsyncDeviceStreamHandler* self,
+ struct ArrowAsyncProducer* producer,
+ struct ArrowSchema* stream_schema, const char* addl_metadata);
+ int (*on_next_task)(struct ArrowAsyncDeviceStreamHandler* self,
+ struct ArrowAsyncTask* task, const char* metadata);
+ void (*on_error)(struct ArrowAsyncDeviceStreamHandler* self,
+ int code, const char* message, const char* metadata);
+
+ // release callback
+ void (*release)(struct ArrowAsyncDeviceStreamHandler* self);
+
+ // opaque handler-specific data
+ void* private_data;
+ };
+
+ #endif // ARROW_C_ASYNC_STREAM_INTERFACE
+
+.. note::
+ The canonical guard ``ARROW_C_ASYNC_STREAM_INTERFACE`` is meant to avoid
+ duplicate definitions if two projects copy the C async stream interface
+ definitions into their own headers, and a third-party project includes
+ from these two projects. It is therefore important that this guard is kept
+ exactly as-is when these definitions are copied.
+
+The ArrowAsyncDeviceStreamHandler structure
+'''''''''''''''''''''''''''''''''''''''''''
+
+The structure has the following fields:
+
+.. c:member:: int (*ArrowAsyncDeviceStreamHandler.on_schema)(struct ArrowAsyncDeviceStreamHandler*, struct ArrowAsyncProducer*, struct ArrowSchema*, const char*)
+
+ *Mandatory.* Handler for receiving the schema of the stream. All records should
+ match the provided schema. If successful, the function should return 0, otherwise
+ it should return an ``errno``-compatible error code.
+
+ The ``const char*`` parameter exists for producers to provide any extra contextual information
+ they want, such as the total number of rows in the stream, statistics, or otherwise. This is
+ encoded in the same format as :c:member:`ArrowSchema.metadata`. If not ``NULL``,
+ the lifetime is only the scope of the call to this function. A consumer who wants to maintain
+ the additional metadata beyond the lifetime of this call *MUST* copy the value themselves.
+
+ Unless the ``on_error`` handler is called, this will always get called exactly once and will be
+ the first method called on this object. As such the producer *MUST* provide an ``ArrowAsyncProducer``
+ object when calling this function to allow the consumer to manage back-pressure and flow control.
+ The producer maintains ownership of the ``ArrowAsyncProducer`` and must clean it up *after*
+ calling the release callback on this object.
+
+ A producer that receives a non-zero result here must not subsequently call anything other than
+ the release callback on this object.
+
+.. c:member:: int (*ArrowAsyncDeviceStreamHandler.on_next_task)(struct ArrowAsyncDeviceStreamHandler*, struct ArrowAsyncTask*, const char*)
+
+ *Mandatory.* Handler to be called when a new record is available for processing. The
+ schema for each record should be the same as the schema that ``on_schema`` was called with.
+ If successfully handled, the function should return 0, otherwise it should return an
+ ``errno``-compatible error code.
+
+ Rather than passing the record itself it receives an ``ArrowAsyncTask`` instead to facilitate
+ better consumer-focused thread control as far as receiving the data. A call to this function
+ simply indicates that data is available via the provided task.
+
+ The producer signals the end of the stream by passing ``NULL`` for the ``ArrowAsyncTask``
+ pointer instead of a valid address. This task object is only valid during the lifetime of
+ this function call. If the consumer wants to use the task beyond the scope of this method, it
+ must copy or move its contents to a new ArrowAsyncTask object.
+
+ The ``const char*`` parameter exists for producers to provide any extra contextual information
+ they want. This is encoded in the same format as :c:member:`ArrowSchema.metadata`. If not ``NULL``,
+ the lifetime is only the scope of the call to this function. A consumer who wants to maintain
+ the additional metadata beyond the lifetime of this call *MUST* copy the value themselves.
+
+ A producer *MUST NOT* call this concurrently from multiple threads.
+
+ The :c:member:`ArrowAsyncProducer.request` callback must be called to start receiving calls to this
+ handler.
+
+.. c:member:: void (*ArrowAsyncDeviceStreamHandler.on_error)(struct ArrowAsyncDeviceStreamHandler, int, const char*, const char*)
+
+ *Mandatory.* Handler to be called when an error is encountered by the producer. After calling
+ this, the ``release`` callback will be called as the last call on this struct. The parameters
+ are an ``errno``-compatible error code and an optional error message and metadata.
+
+ If the message and metadata are not ``NULL``, their lifetime is only valid during the scope
+ of this call. A consumer who wants to maintain these values past the return of this function
+ *MUST* copy the values themselves.
+
+ If the metadata parameter is not ``NULL``, to provide key-value error metadata, then it should
+ be encoded identically to the way that metadata is encoded in :c:member:`ArrowSchema.metadata`.
+
+ It is valid for this to be called by a producer with or without a preceding call to
+ :c:member:`ArrowAsyncProducer.request`. This callback *MUST NOT* call any methods of an
+ ``ArrowAsyncProducer`` object.
+
+.. c:member:: void (*ArrowAsyncDeviceStreamHandler.release)(struct ArrowAsyncDeviceStreamHandler*)
+
+ *Mandatory.* A pointer to a consumer-provided release callback for the handler.
+
+ It is valid for this to be called by a producer with or without a preceding call to
+ :c:member:`ArrowAsyncProducer.request`. This must not call any methods of an ``ArrowAsyncProducer``
+ object.
+
+.. c:member:: void* ArrowAsyncDeviceStreamHandler.private_data
+
+ *Optional.* An opaque pointer to consumer-provided private data.
+
+ Producers *MUST NOT* process this member. Lifetime of this member is handled by
+ the consumer, and especially by the release callback.
+
+The ArrowAsyncTask structure
+''''''''''''''''''''''''''''
+
+The purpose of using a Task object rather than passing the array directly to the ``on_next``
+callback is to allow for more complex and efficient thread handling. Utilizing a Task
+object allows for a producer to separate the "decoding" logic from the I/O, enabling a
+consumer to avoid transferring data between CPU cores (e.g. from one L1/L2 cache to another).
+
+This producer-provided structure has the following fields:
+
+.. c:member:: int (*ArrowArrayTask.get_data)(struct ArrowArrayTask*, struct ArrowDeviceArray*)
+
+ *Mandatory.* A callback to populate the provided ``ArrowDeviceArray`` with the available data.
+ The order of ArrowAsyncTasks provided by the producer enables a consumer to know the order of
+ the data to process. If the consumer does not care about the data that is owned by this task,
+ it must still call ``get_data`` so that the producer can perform any required cleanup. ``NULL``
+ should be passed as the device array pointer to indicate that the consumer doesn't want the
+ actual data, letting the task perform necessary cleanup.
+
+ If a non-zero value is returned from this, it should be followed only by the producer calling
+ the ``on_error`` callback of the ``ArrowAsyncDeviceStreamHandler``. Because calling this method
+ is likely to be separate from the current control flow, returning a non-zero value to signal
+ an error occuring allows the current thread to decide handle the case accordingly, while still
+ allowing all error logging and handling to be centralized in the
+ :c:member:`ArrowAsyncDeviceStreamHandler.on_error` callback.
+
+ Rather than having a separate release callback, any required cleanup should be performed as part
+ of the invocation of this callback. Ownership of the Array is given to the pointer passed in as
+ a parameter, and this array must be released separately.
+
+ It is only valid to call this method exactly once.
Review Comment:
`consume_data` it is.
##########
docs/source/format/CDeviceDataInterface.rst:
##########
@@ -650,6 +652,340 @@ The stream source is not assumed to be thread-safe. Consumers wanting to
call ``get_next`` from several threads should ensure those calls are
serialized.
+Async Device Stream Interface
+=============================
+
+The :ref:`C stream interface <_c-device-stream-interface>` provides a synchronous
+API centered around the consumer calling the callback functions to retrieve
+the next record batch. For some bindings, use cases, and interoperability, a more
+asynchronous, producer-focused interface may be required. These scenarios can utilize
+the ``ArrowAsyncDeviceStreamHandler``.
+
+Semantics
+---------
+
+Rather than the producer providing a structure of callbacks for a consumer to
+call and retrieve records, the Async interface is a consumer allocated structure.
+The consumer allocated struct provides handler callbacks for the producer to call
+when the schema and chunks of data are available, rather than the consumer using
+a blocking pull-style iteration.
+
+In addition to the ``ArrowAsyncDeviceStreamHandler``, there are also two additional
+structs used for the full data flow: ``ArrowAsyncTask`` and ``ArrowAsyncProducer``.
+
+Structure Definition
+--------------------
+
+The C device async stream interface is defined with a single ``struct`` definition:
+
+.. code-block:: c
+
+ #ifndef ARROW_C_ASYNC_STREAM_INTERFACE
+ #define ARROW_C_ASYNC_STREAM_INTERFACE
+
+ struct ArrowAsyncTask {
+ int (*get_data)(struct ArrowArrayTask* self, struct ArrowDeviceArray* out);
+
+ void* private_data;
+ };
+
+ struct ArrowAsyncProducer {
+ void (*request)(struct ArrowAsyncProducer* self, int64_t n);
+ void (*cancel)(struct ArrowAsyncProducer* self);
+
+ void (*release)(struct ArrowAsyncProducer* self);
+ void* private_data;
+ };
+
+ struct ArrowAsyncDeviceStreamHandler {
+ // handlers
+ int (*on_schema)(struct ArrowAsyncDeviceStreamHandler* self,
+ struct ArrowAsyncProducer* producer,
+ struct ArrowSchema* stream_schema, const char* addl_metadata);
+ int (*on_next_task)(struct ArrowAsyncDeviceStreamHandler* self,
+ struct ArrowAsyncTask* task, const char* metadata);
+ void (*on_error)(struct ArrowAsyncDeviceStreamHandler* self,
+ int code, const char* message, const char* metadata);
+
+ // release callback
+ void (*release)(struct ArrowAsyncDeviceStreamHandler* self);
+
+ // opaque handler-specific data
+ void* private_data;
+ };
+
+ #endif // ARROW_C_ASYNC_STREAM_INTERFACE
+
+.. note::
+ The canonical guard ``ARROW_C_ASYNC_STREAM_INTERFACE`` is meant to avoid
+ duplicate definitions if two projects copy the C async stream interface
+ definitions into their own headers, and a third-party project includes
+ from these two projects. It is therefore important that this guard is kept
+ exactly as-is when these definitions are copied.
+
+The ArrowAsyncDeviceStreamHandler structure
+'''''''''''''''''''''''''''''''''''''''''''
+
+The structure has the following fields:
+
+.. c:member:: int (*ArrowAsyncDeviceStreamHandler.on_schema)(struct ArrowAsyncDeviceStreamHandler*, struct ArrowAsyncProducer*, struct ArrowSchema*, const char*)
+
+ *Mandatory.* Handler for receiving the schema of the stream. All records should
+ match the provided schema. If successful, the function should return 0, otherwise
+ it should return an ``errno``-compatible error code.
+
+ The ``const char*`` parameter exists for producers to provide any extra contextual information
+ they want, such as the total number of rows in the stream, statistics, or otherwise. This is
+ encoded in the same format as :c:member:`ArrowSchema.metadata`. If not ``NULL``,
+ the lifetime is only the scope of the call to this function. A consumer who wants to maintain
+ the additional metadata beyond the lifetime of this call *MUST* copy the value themselves.
+
+ Unless the ``on_error`` handler is called, this will always get called exactly once and will be
+ the first method called on this object. As such the producer *MUST* provide an ``ArrowAsyncProducer``
+ object when calling this function to allow the consumer to manage back-pressure and flow control.
+ The producer maintains ownership of the ``ArrowAsyncProducer`` and must clean it up *after*
+ calling the release callback on this object.
+
+ A producer that receives a non-zero result here must not subsequently call anything other than
+ the release callback on this object.
+
+.. c:member:: int (*ArrowAsyncDeviceStreamHandler.on_next_task)(struct ArrowAsyncDeviceStreamHandler*, struct ArrowAsyncTask*, const char*)
+
+ *Mandatory.* Handler to be called when a new record is available for processing. The
+ schema for each record should be the same as the schema that ``on_schema`` was called with.
+ If successfully handled, the function should return 0, otherwise it should return an
+ ``errno``-compatible error code.
+
+ Rather than passing the record itself it receives an ``ArrowAsyncTask`` instead to facilitate
+ better consumer-focused thread control as far as receiving the data. A call to this function
+ simply indicates that data is available via the provided task.
+
+ The producer signals the end of the stream by passing ``NULL`` for the ``ArrowAsyncTask``
+ pointer instead of a valid address. This task object is only valid during the lifetime of
+ this function call. If the consumer wants to use the task beyond the scope of this method, it
+ must copy or move its contents to a new ArrowAsyncTask object.
+
+ The ``const char*`` parameter exists for producers to provide any extra contextual information
+ they want. This is encoded in the same format as :c:member:`ArrowSchema.metadata`. If not ``NULL``,
+ the lifetime is only the scope of the call to this function. A consumer who wants to maintain
+ the additional metadata beyond the lifetime of this call *MUST* copy the value themselves.
+
+ A producer *MUST NOT* call this concurrently from multiple threads.
+
+ The :c:member:`ArrowAsyncProducer.request` callback must be called to start receiving calls to this
+ handler.
+
+.. c:member:: void (*ArrowAsyncDeviceStreamHandler.on_error)(struct ArrowAsyncDeviceStreamHandler, int, const char*, const char*)
+
+ *Mandatory.* Handler to be called when an error is encountered by the producer. After calling
+ this, the ``release`` callback will be called as the last call on this struct. The parameters
+ are an ``errno``-compatible error code and an optional error message and metadata.
+
+ If the message and metadata are not ``NULL``, their lifetime is only valid during the scope
+ of this call. A consumer who wants to maintain these values past the return of this function
+ *MUST* copy the values themselves.
+
+ If the metadata parameter is not ``NULL``, to provide key-value error metadata, then it should
+ be encoded identically to the way that metadata is encoded in :c:member:`ArrowSchema.metadata`.
+
+ It is valid for this to be called by a producer with or without a preceding call to
+ :c:member:`ArrowAsyncProducer.request`. This callback *MUST NOT* call any methods of an
+ ``ArrowAsyncProducer`` object.
+
+.. c:member:: void (*ArrowAsyncDeviceStreamHandler.release)(struct ArrowAsyncDeviceStreamHandler*)
+
+ *Mandatory.* A pointer to a consumer-provided release callback for the handler.
+
+ It is valid for this to be called by a producer with or without a preceding call to
+ :c:member:`ArrowAsyncProducer.request`. This must not call any methods of an ``ArrowAsyncProducer``
+ object.
+
+.. c:member:: void* ArrowAsyncDeviceStreamHandler.private_data
+
+ *Optional.* An opaque pointer to consumer-provided private data.
+
+ Producers *MUST NOT* process this member. Lifetime of this member is handled by
+ the consumer, and especially by the release callback.
+
+The ArrowAsyncTask structure
+''''''''''''''''''''''''''''
+
+The purpose of using a Task object rather than passing the array directly to the ``on_next``
+callback is to allow for more complex and efficient thread handling. Utilizing a Task
+object allows for a producer to separate the "decoding" logic from the I/O, enabling a
+consumer to avoid transferring data between CPU cores (e.g. from one L1/L2 cache to another).
+
+This producer-provided structure has the following fields:
+
+.. c:member:: int (*ArrowArrayTask.get_data)(struct ArrowArrayTask*, struct ArrowDeviceArray*)
+
+ *Mandatory.* A callback to populate the provided ``ArrowDeviceArray`` with the available data.
+ The order of ArrowAsyncTasks provided by the producer enables a consumer to know the order of
+ the data to process. If the consumer does not care about the data that is owned by this task,
+ it must still call ``get_data`` so that the producer can perform any required cleanup. ``NULL``
+ should be passed as the device array pointer to indicate that the consumer doesn't want the
+ actual data, letting the task perform necessary cleanup.
+
+ If a non-zero value is returned from this, it should be followed only by the producer calling
+ the ``on_error`` callback of the ``ArrowAsyncDeviceStreamHandler``. Because calling this method
+ is likely to be separate from the current control flow, returning a non-zero value to signal
+ an error occuring allows the current thread to decide handle the case accordingly, while still
+ allowing all error logging and handling to be centralized in the
+ :c:member:`ArrowAsyncDeviceStreamHandler.on_error` callback.
+
+ Rather than having a separate release callback, any required cleanup should be performed as part
+ of the invocation of this callback. Ownership of the Array is given to the pointer passed in as
+ a parameter, and this array must be released separately.
Review Comment:
Ok. Now this must definitely be renamed to `consume_data` as I suspected earlier.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-43631: [C][Format] Add ArrowAsyncDeviceStreamHandler interface [arrow]
Posted by "bkietz (via GitHub)" <gi...@apache.org>.
bkietz commented on PR #43632:
URL: https://github.com/apache/arrow/pull/43632#issuecomment-2461021652
🎉
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-43631: [C][Format] Add ArrowAsyncDeviceStreamHandler interface [arrow]
Posted by "pitrou (via GitHub)" <gi...@apache.org>.
pitrou commented on code in PR #43632:
URL: https://github.com/apache/arrow/pull/43632#discussion_r1757486825
##########
docs/source/format/CDeviceDataInterface.rst:
##########
@@ -650,6 +652,331 @@ The stream source is not assumed to be thread-safe. Consumers wanting to
call ``get_next`` from several threads should ensure those calls are
serialized.
+Async Device Stream Interface
+=============================
+
+The :ref:`C stream interface <_c-device-stream-interface>` provides a synchronous
+API centered around the consumer calling the callback functions to retrieve
+the next record batch. For some bindings, use cases, and interoperability, a more
+asynchronous, producer-focused interface may be required. These scenarios can utilize
+the ``ArrowAsyncDeviceStreamHandler``.
+
+Semantics
+---------
+
+Rather than the producer providing a structure of callbacks for a consumer to
+call and retrieve records, the Async interface is a consumer allocated structure.
+The consumer allocated struct provides handler callbacks for the producer to call
+when the schema and chunks of data are available, rather than the consumer using
+a blocking pull-style iteration.
+
+In addition to the ``ArrowAsyncDeviceStreamHandler``, there are also two additional
+structs used for the full data flow: ``ArrowAsyncTask`` and ``ArrowAsyncProducer``.
+
+Structure Definition
+--------------------
+
+The C device async stream interface is defined with a single ``struct`` definition:
+
+.. code-block:: c
+
+ #ifndef ARROW_C_ASYNC_STREAM_INTERFACE
+ #define ARROW_C_ASYNC_STREAM_INTERFACE
+
+ struct ArrowAsyncTask {
+ int (*get_data)(struct ArrowArrayTask* self, struct ArrowDeviceArray* out);
+
+ void* private_data;
+ };
+
+ struct ArrowAsyncProducer {
+ void (*request)(struct ArrowAsyncProducer* self, uint64_t n);
+ void (*cancel)(struct ArrowAsyncProducer* self);
+
+ void (*release)(struct ArrowAsyncProducer* self);
+ void* private_data;
+ };
+
+ struct ArrowAsyncDeviceStreamHandler {
+ // handlers
+ int (*on_schema)(struct ArrowAsyncDeviceStreamHandler* self,
+ struct ArrowAsyncProducer* producer,
+ struct ArrowSchema* stream_schema, const char* addl_metadata);
+ int (*on_next_task)(struct ArrowAsyncDeviceStreamHandler* self,
+ struct ArrowAsyncTask* task, const char* metadata);
+ void (*on_error)(struct ArrowAsyncDeviceStreamHandler* self,
+ int code, const char* message, const char* metadata);
+
+ // release callback
+ void (*release)(struct ArrowAsyncDeviceStreamHandler* self);
+
+ // opaque handler-specific data
+ void* private_data;
+ };
+
+ #endif // ARROW_C_ASYNC_STREAM_INTERFACE
+
+.. note::
+ The canonical guard ``ARROW_C_ASYNC_STREAM_INTERFACE`` is meant to avoid
+ duplicate definitions if two projects copy the C async stream interface
+ definitions into their own headers, and a third-party project includes
+ from these two projects. It is therefore important that this guard is kept
+ exactly as-is when these definitions are copied.
+
+The ArrowAsyncDeviceStreamHandler structure
+'''''''''''''''''''''''''''''''''''''''''''
+
+The structure has the following fields:
+
+.. c:member:: int (*ArrowAsyncDeviceStreamHandler.on_schema)(struct ArrowAsyncDeviceStreamHandler*, struct ArrowAsyncProducer*, struct ArrowSchema*, const char*)
+
+ *Mandatory.* Handler for receiving the schema of the stream. All records should
+ match the provided schema. If successful, the function should return 0, otherwise
+ it should return an ``errno``-compatible error code.
+
+ The ``const char*`` parameter exists for producers to provide any extra contextual information
+ they want, such as the total number of rows in the stream, statistics, or otherwise. This is
+ encoded in the same format as :c:member:`ArrowSchema.metadata`. If not ``NULL``,
+ the lifetime is only the scope of the call to this function. A consumer who wants to maintain
+ the additional metadata beyond the lifetime of this call *MUST* copy the value themselves.
+
+ Unless the ``on_error`` handler is called, this will always get called exactly once and will be
+ the first method called on this object. As such the producer *MUST* provide an ``ArrowAsyncProducer``
+ object when calling this function to allow the consumer to manage back-pressure and flow control.
+ The producer maintains ownership of the ``ArrowAsyncProducer`` and must clean it up before or after
+ calling the release callback on this object.
+
+ A producer that receives a non-zero result here must not subsequently call anything other than
+ the release callback on this object.
+
+.. c:member:: int (*ArrowAsyncDeviceStreamHandler.on_next_task)(struct ArrowAsyncDeviceStreamHandler*, struct ArrowAsyncTask*, const char*)
+
+ *Mandatory.* Handler to be called when a new record is available for processing. The
+ schema for each record should be the same as the schema that ``on_schema`` was called with.
+ If successfully handled, the function should return 0, otherwise it should return an
+ ``errno``-compatible error code.
+
+ Rather than passing the record itself it receives an ``ArrowAsyncTask`` instead to facilitate
+ better consumer-focused thread control as far as receiving the data. A call to this function
+ simply indicates that data is available via the provided task.
+
+ The ``const char*`` parameter exists for producers to provide any extra contextual information
+ they want. This is encoded in the same format as :c:member:`ArrowSchema.metadata`. If not ``NULL``,
+ the lifetime is only the scope of the call to this function. A consumer who wants to maintain
+ the additional metadata beyond the lifetime of this call *MUST* copy the value themselves.
+
+ A producer *MUST NOT* call this concurrently from multiple different threads.
+
+ The :c:member:`ArrowAsyncProducer.request` callback must be called to start receiving calls to this
+ handler.
+
+ This function *MUST* be able to be called re-entrantly on the same thread to allow for the
+ :c:member:`ArrowAsyncProducer.request` callback to potentially call this method recursively.
+
+.. c:member:: void (*ArrowAsyncDeviceStreamHandler.on_error)(struct ArrowAsyncDeviceStreamHandler, int, const char*, const char*)
+
+ *Mandatory.* Handler to be called when an error is encountered by the producer. After calling
+ this, the ``release`` callback will be called as the last call on this struct. The parameters
+ are an ``errno``-compatible error code and an optional error message and metadata.
+
+ If the message and metadata are not ``NULL``, their lifetime is only valid during the scope
+ of this call. A consumer who wants to maintain these values past the return of this function
+ *MUST* copy the values themselves.
+
+ If the metadata parameter is not ``NULL``, to provide key-value error metadata, then it should
+ be encoded identically to the way that metadata is encoded in :c:member:`ArrowSchema.metadata`.
+
+ It is valid for this to be called by a producer with or without a preceding call to
+ :c:member:`ArrowAsyncProducer.request`. This callback *MUST NOT* call any methods of an
+ ``ArrowAsyncProducer`` object.
+
+.. c:member:: void (*ArrowAsyncDeviceStreamHandler.release)(struct ArrowAsyncDeviceStreamHandler*)
+
+ *Mandatory.* A pointer to a consumer-provided release callback for the handler.
+
+ It is valid for this to be called by a producer with or without a preceding call to
+ :c:member:`ArrowAsyncProducer.request`. This must not call any methods of an ``ArrowAsyncProducer``
+ object.
+
+.. c:member:: void* ArrowAsyncDeviceStreamHandler.private_data
+
+ *Optional.* An opaque pointer to consumer-provided private data.
+
+ Producers *MUST NOT* process this member. Lifetime of this member is handled by
+ the consumer, and especially by the release callback.
+
+The ArrowAsyncTask structure
+''''''''''''''''''''''''''''
+
+The purpose of using a Task object rather than passing the array directly to the ``on_next``
+callback is to allow for more complex and efficient thread handling. Utilizing a Task
+object allows for a producer to separate the "decoding" logic from the I/O, enabling a
+consumer to avoid transferring data between CPU cores (e.g. from one L1/L2 cache to another).
+
+This producer-provided structure has the following fields:
+
+.. c:member:: int (*ArrowArrayTask.get_data)(struct ArrowArrayTask*, struct ArrowDeviceArray*)
+
+ *Mandatory.* A callback to populate the provided ``ArrowDeviceArray`` with the available data.
+ The order of ArrowAsyncTasks provided by the producer enables a consumer to know the order of
+ the data to process.
+
+ If a non-zero value is returned from this, it should be followed only by the producer calling
+ the ``on_error`` callback of the ``ArrowAsyncDeviceStreamHandler``. Because calling this method
+ is likely to be separate from the current control flow, returning a non-zero value to signal
+ an error occuring allows the current thread to decide handle the case accordingly, while still
+ allowing all error logging and handling to be centralized in the
+ :c:member:`ArrowAsyncDeviceStreamHandler.on_error` callback.
+
+ Rather than having a separate release callback, any required cleanup should be performed as part
+ of the invocation of this callback, with ownership of the Array being passed to the consumer-provided
+ parameter calling this, and must be released separately.
+
+ It is only valid to call this method exactly once.
+
+.. c:member:: void* ArrowArrayTask.private_data
+
+ *Optional.* An opaque pointer to consumer-provided private data.
+
+ Consumers *MUST NOT* process this member. Lifetime of this member is handled by
+ the producer who created this object, and should be cleaned up if necessary during
+ the call to :c:member:`ArrowArrayTask.get_data`.
+
+The ArrowAsyncProducer structure
+''''''''''''''''''''''''''''''''
+
+This producer-provided and managed object has the following fields:
+
+.. c:member:: void (*ArrowAsyncProducer.request)(struct ArrowAsyncProducer*, uint64_t)
+
+ *Mandatory.* This function must be called by a consumer to start receiving calls to
+ :c:member:`ArrowAsyncDeviceStreamHandler.on_next_task`. It *MUST* be valid to call
+ this synchronously from within :c:member:`ArrowAsyncDeviceStreamHandler.on_next_task`
+ or :c:member:`ArrowAsyncDeviceStreamHandler.on_schema`. As a result, this function
+ *MUST NOT* synchronously call ``on_next_task`` or ``on_error`` to avoid recursive
+ and reentrant callbacks.
+
+ After ``cancel`` is called, additional calls to this function must be a NOP, but allowed.
+
+ While not cancelled, calling this function registers the given number of additional
+ arrays/batches to be produced by the producer. A producer should only call
+ the appropriate ``on_next_task`` callback up to a maximum of the total sum of calls to
+ this method before propagating back-pressure / waiting.
+
+ Any error encountered by calling request must be propagated by calling the ``on_error``
+ callback of the ``ArrowAsyncDeviceStreamHandler``.
+
+ A producer *MUST* support an unbounded number of calls to this callback and *MUST* support
Review Comment:
This is a very strange requirement. Why unbounded?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-43631: [C][Format] Add ArrowAsyncDeviceStreamHandler interface [arrow]
Posted by "pitrou (via GitHub)" <gi...@apache.org>.
pitrou commented on code in PR #43632:
URL: https://github.com/apache/arrow/pull/43632#discussion_r1757491988
##########
docs/source/format/CDeviceDataInterface.rst:
##########
@@ -650,6 +652,331 @@ The stream source is not assumed to be thread-safe. Consumers wanting to
call ``get_next`` from several threads should ensure those calls are
serialized.
+Async Device Stream Interface
+=============================
+
+The :ref:`C stream interface <_c-device-stream-interface>` provides a synchronous
+API centered around the consumer calling the callback functions to retrieve
+the next record batch. For some bindings, use cases, and interoperability, a more
+asynchronous, producer-focused interface may be required. These scenarios can utilize
+the ``ArrowAsyncDeviceStreamHandler``.
+
+Semantics
+---------
+
+Rather than the producer providing a structure of callbacks for a consumer to
+call and retrieve records, the Async interface is a consumer allocated structure.
+The consumer allocated struct provides handler callbacks for the producer to call
+when the schema and chunks of data are available, rather than the consumer using
+a blocking pull-style iteration.
+
+In addition to the ``ArrowAsyncDeviceStreamHandler``, there are also two additional
+structs used for the full data flow: ``ArrowAsyncTask`` and ``ArrowAsyncProducer``.
+
+Structure Definition
+--------------------
+
+The C device async stream interface is defined with a single ``struct`` definition:
+
+.. code-block:: c
+
+ #ifndef ARROW_C_ASYNC_STREAM_INTERFACE
+ #define ARROW_C_ASYNC_STREAM_INTERFACE
+
+ struct ArrowAsyncTask {
+ int (*get_data)(struct ArrowArrayTask* self, struct ArrowDeviceArray* out);
+
+ void* private_data;
+ };
+
+ struct ArrowAsyncProducer {
+ void (*request)(struct ArrowAsyncProducer* self, uint64_t n);
+ void (*cancel)(struct ArrowAsyncProducer* self);
+
+ void (*release)(struct ArrowAsyncProducer* self);
+ void* private_data;
+ };
+
+ struct ArrowAsyncDeviceStreamHandler {
+ // handlers
+ int (*on_schema)(struct ArrowAsyncDeviceStreamHandler* self,
+ struct ArrowAsyncProducer* producer,
+ struct ArrowSchema* stream_schema, const char* addl_metadata);
+ int (*on_next_task)(struct ArrowAsyncDeviceStreamHandler* self,
+ struct ArrowAsyncTask* task, const char* metadata);
+ void (*on_error)(struct ArrowAsyncDeviceStreamHandler* self,
+ int code, const char* message, const char* metadata);
+
+ // release callback
+ void (*release)(struct ArrowAsyncDeviceStreamHandler* self);
+
+ // opaque handler-specific data
+ void* private_data;
+ };
+
+ #endif // ARROW_C_ASYNC_STREAM_INTERFACE
+
+.. note::
+ The canonical guard ``ARROW_C_ASYNC_STREAM_INTERFACE`` is meant to avoid
+ duplicate definitions if two projects copy the C async stream interface
+ definitions into their own headers, and a third-party project includes
+ from these two projects. It is therefore important that this guard is kept
+ exactly as-is when these definitions are copied.
+
+The ArrowAsyncDeviceStreamHandler structure
+'''''''''''''''''''''''''''''''''''''''''''
+
+The structure has the following fields:
+
+.. c:member:: int (*ArrowAsyncDeviceStreamHandler.on_schema)(struct ArrowAsyncDeviceStreamHandler*, struct ArrowAsyncProducer*, struct ArrowSchema*, const char*)
+
+ *Mandatory.* Handler for receiving the schema of the stream. All records should
+ match the provided schema. If successful, the function should return 0, otherwise
+ it should return an ``errno``-compatible error code.
+
+ The ``const char*`` parameter exists for producers to provide any extra contextual information
+ they want, such as the total number of rows in the stream, statistics, or otherwise. This is
+ encoded in the same format as :c:member:`ArrowSchema.metadata`. If not ``NULL``,
+ the lifetime is only the scope of the call to this function. A consumer who wants to maintain
+ the additional metadata beyond the lifetime of this call *MUST* copy the value themselves.
+
+ Unless the ``on_error`` handler is called, this will always get called exactly once and will be
+ the first method called on this object. As such the producer *MUST* provide an ``ArrowAsyncProducer``
+ object when calling this function to allow the consumer to manage back-pressure and flow control.
+ The producer maintains ownership of the ``ArrowAsyncProducer`` and must clean it up before or after
+ calling the release callback on this object.
+
+ A producer that receives a non-zero result here must not subsequently call anything other than
+ the release callback on this object.
+
+.. c:member:: int (*ArrowAsyncDeviceStreamHandler.on_next_task)(struct ArrowAsyncDeviceStreamHandler*, struct ArrowAsyncTask*, const char*)
+
+ *Mandatory.* Handler to be called when a new record is available for processing. The
+ schema for each record should be the same as the schema that ``on_schema`` was called with.
+ If successfully handled, the function should return 0, otherwise it should return an
+ ``errno``-compatible error code.
+
+ Rather than passing the record itself it receives an ``ArrowAsyncTask`` instead to facilitate
+ better consumer-focused thread control as far as receiving the data. A call to this function
+ simply indicates that data is available via the provided task.
+
+ The ``const char*`` parameter exists for producers to provide any extra contextual information
+ they want. This is encoded in the same format as :c:member:`ArrowSchema.metadata`. If not ``NULL``,
+ the lifetime is only the scope of the call to this function. A consumer who wants to maintain
+ the additional metadata beyond the lifetime of this call *MUST* copy the value themselves.
+
+ A producer *MUST NOT* call this concurrently from multiple different threads.
+
+ The :c:member:`ArrowAsyncProducer.request` callback must be called to start receiving calls to this
+ handler.
+
+ This function *MUST* be able to be called re-entrantly on the same thread to allow for the
+ :c:member:`ArrowAsyncProducer.request` callback to potentially call this method recursively.
+
+.. c:member:: void (*ArrowAsyncDeviceStreamHandler.on_error)(struct ArrowAsyncDeviceStreamHandler, int, const char*, const char*)
+
+ *Mandatory.* Handler to be called when an error is encountered by the producer. After calling
+ this, the ``release`` callback will be called as the last call on this struct. The parameters
+ are an ``errno``-compatible error code and an optional error message and metadata.
+
+ If the message and metadata are not ``NULL``, their lifetime is only valid during the scope
+ of this call. A consumer who wants to maintain these values past the return of this function
+ *MUST* copy the values themselves.
+
+ If the metadata parameter is not ``NULL``, to provide key-value error metadata, then it should
+ be encoded identically to the way that metadata is encoded in :c:member:`ArrowSchema.metadata`.
+
+ It is valid for this to be called by a producer with or without a preceding call to
+ :c:member:`ArrowAsyncProducer.request`. This callback *MUST NOT* call any methods of an
+ ``ArrowAsyncProducer`` object.
+
+.. c:member:: void (*ArrowAsyncDeviceStreamHandler.release)(struct ArrowAsyncDeviceStreamHandler*)
+
+ *Mandatory.* A pointer to a consumer-provided release callback for the handler.
+
+ It is valid for this to be called by a producer with or without a preceding call to
+ :c:member:`ArrowAsyncProducer.request`. This must not call any methods of an ``ArrowAsyncProducer``
+ object.
+
+.. c:member:: void* ArrowAsyncDeviceStreamHandler.private_data
+
+ *Optional.* An opaque pointer to consumer-provided private data.
+
+ Producers *MUST NOT* process this member. Lifetime of this member is handled by
+ the consumer, and especially by the release callback.
+
+The ArrowAsyncTask structure
+''''''''''''''''''''''''''''
+
+The purpose of using a Task object rather than passing the array directly to the ``on_next``
+callback is to allow for more complex and efficient thread handling. Utilizing a Task
+object allows for a producer to separate the "decoding" logic from the I/O, enabling a
+consumer to avoid transferring data between CPU cores (e.g. from one L1/L2 cache to another).
+
+This producer-provided structure has the following fields:
+
+.. c:member:: int (*ArrowArrayTask.get_data)(struct ArrowArrayTask*, struct ArrowDeviceArray*)
+
+ *Mandatory.* A callback to populate the provided ``ArrowDeviceArray`` with the available data.
+ The order of ArrowAsyncTasks provided by the producer enables a consumer to know the order of
+ the data to process.
+
+ If a non-zero value is returned from this, it should be followed only by the producer calling
+ the ``on_error`` callback of the ``ArrowAsyncDeviceStreamHandler``. Because calling this method
+ is likely to be separate from the current control flow, returning a non-zero value to signal
+ an error occuring allows the current thread to decide handle the case accordingly, while still
+ allowing all error logging and handling to be centralized in the
+ :c:member:`ArrowAsyncDeviceStreamHandler.on_error` callback.
+
+ Rather than having a separate release callback, any required cleanup should be performed as part
+ of the invocation of this callback, with ownership of the Array being passed to the consumer-provided
+ parameter calling this, and must be released separately.
+
+ It is only valid to call this method exactly once.
+
+.. c:member:: void* ArrowArrayTask.private_data
+
+ *Optional.* An opaque pointer to consumer-provided private data.
+
+ Consumers *MUST NOT* process this member. Lifetime of this member is handled by
+ the producer who created this object, and should be cleaned up if necessary during
+ the call to :c:member:`ArrowArrayTask.get_data`.
+
+The ArrowAsyncProducer structure
+''''''''''''''''''''''''''''''''
+
+This producer-provided and managed object has the following fields:
+
+.. c:member:: void (*ArrowAsyncProducer.request)(struct ArrowAsyncProducer*, uint64_t)
+
+ *Mandatory.* This function must be called by a consumer to start receiving calls to
+ :c:member:`ArrowAsyncDeviceStreamHandler.on_next_task`. It *MUST* be valid to call
+ this synchronously from within :c:member:`ArrowAsyncDeviceStreamHandler.on_next_task`
+ or :c:member:`ArrowAsyncDeviceStreamHandler.on_schema`. As a result, this function
+ *MUST NOT* synchronously call ``on_next_task`` or ``on_error`` to avoid recursive
+ and reentrant callbacks.
+
+ After ``cancel`` is called, additional calls to this function must be a NOP, but allowed.
+
+ While not cancelled, calling this function registers the given number of additional
+ arrays/batches to be produced by the producer. A producer should only call
+ the appropriate ``on_next_task`` callback up to a maximum of the total sum of calls to
+ this method before propagating back-pressure / waiting.
+
+ Any error encountered by calling request must be propagated by calling the ``on_error``
+ callback of the ``ArrowAsyncDeviceStreamHandler``.
+
+ A producer *MUST* support an unbounded number of calls to this callback and *MUST* support
+ a total registered demand (sum requested - sum delivered) of up to ``UINT64_MAX``.
+
+.. c:member:: void (*ArrowAsyncProducer.cancel)(struct ArrowAsyncProducer*)
+
+ *Mandatory.* This function signals to the producer that it must *eventually* stop calling
+ ``on_next_task``. Calls to ``cancel`` must be idempotent and thread-safe. After calling
+ it once, subsequent calls *MUST* be a NOP.
+
+ It is not required that calling ``cancel`` affect the producer *immediately*, only that it
+ must eventually stop calling ``on_next_task`` and then subsequently call ``release``
+ on the async handler object. As such, a consumer *MUST* be prepared to receive one or more
+ calls to ``on_next_task`` or ``on_error`` even after calling ``cancel`` if there are still
+ requested arrays pending.
+
+ Successful cancelling *MUST NOT* result in a producer calling
+ :c:member:`ArrowAsyncDeviceStreamHandler.on_error`, instead it should finish out any remaining
+ tasks (calling ``on_next_task`` accordingly) and eventually just call ``release``.
+
+ Any error encountered during handling a call to cancel must be reported via the ``on_error``
+ callback on the async stream handler.
+
+.. c:member:: void* ArrowAsyncProducer.private_data
+
+ *Optional.* An opaque pointer to producer-provided specific data.
+
+ Consumers *MUST NOT* process this member, the lifetime is owned by the producer
+ that constructed this object.
+
+Error Handling
+''''''''''''''
+
+Unlike the regular C Stream interface, the Async interface allows for errors to flow in
+both directions. As a result, error handling can be slightly more complex. Thus this spec
+designates the following rules:
+
+* If the producer encounters an error during processing, it should call the ``on_error``
+callback, and then call ``release`` after it returns.
+
+* If ``on_schema`` or ``on_next_task`` returns a non-zero integer value, the producer *should not*
+call the ``on_error`` callback, but instead should eventually call ``release`` at some point
+before or after any logging or processing of the error code.
+
+Result lifetimes
+''''''''''''''''
+
+The ``ArrowSchema`` passed to the ``on_schema`` callback must be released independantly,
+with the object itself needing to be moved to a consumer owned ``ArrowSchema`` object. The
+``ArrowSchema*`` passed as a parameter to the callback *MUST NOT* be stored and kept.
+
+The ``ArrowAsyncTask`` object provided to ``on_next_task`` is owned by the producer and
+will be cleaned up during the invocation of calling ``get_data`` on it. If the consumer
+doesn't care about the data, it should pass ``NULL`` instead of a valid ``ArrowDeviceArray*``.
Review Comment:
This NULL special case should be documenting in `get_data` above instead?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-43631: [C][Format] Add ArrowAsyncDeviceStreamHandler interface [arrow]
Posted by "zeroshade (via GitHub)" <gi...@apache.org>.
zeroshade commented on code in PR #43632:
URL: https://github.com/apache/arrow/pull/43632#discussion_r1759129213
##########
docs/source/format/CDeviceDataInterface.rst:
##########
@@ -650,6 +652,331 @@ The stream source is not assumed to be thread-safe. Consumers wanting to
call ``get_next`` from several threads should ensure those calls are
serialized.
+Async Device Stream Interface
+=============================
+
+The :ref:`C stream interface <_c-device-stream-interface>` provides a synchronous
+API centered around the consumer calling the callback functions to retrieve
+the next record batch. For some bindings, use cases, and interoperability, a more
+asynchronous, producer-focused interface may be required. These scenarios can utilize
+the ``ArrowAsyncDeviceStreamHandler``.
+
+Semantics
+---------
+
+Rather than the producer providing a structure of callbacks for a consumer to
+call and retrieve records, the Async interface is a consumer allocated structure.
+The consumer allocated struct provides handler callbacks for the producer to call
+when the schema and chunks of data are available, rather than the consumer using
+a blocking pull-style iteration.
+
+In addition to the ``ArrowAsyncDeviceStreamHandler``, there are also two additional
+structs used for the full data flow: ``ArrowAsyncTask`` and ``ArrowAsyncProducer``.
+
+Structure Definition
+--------------------
+
+The C device async stream interface is defined with a single ``struct`` definition:
+
+.. code-block:: c
+
+ #ifndef ARROW_C_ASYNC_STREAM_INTERFACE
+ #define ARROW_C_ASYNC_STREAM_INTERFACE
+
+ struct ArrowAsyncTask {
+ int (*get_data)(struct ArrowArrayTask* self, struct ArrowDeviceArray* out);
+
+ void* private_data;
+ };
+
+ struct ArrowAsyncProducer {
+ void (*request)(struct ArrowAsyncProducer* self, uint64_t n);
+ void (*cancel)(struct ArrowAsyncProducer* self);
+
+ void (*release)(struct ArrowAsyncProducer* self);
+ void* private_data;
+ };
+
+ struct ArrowAsyncDeviceStreamHandler {
+ // handlers
+ int (*on_schema)(struct ArrowAsyncDeviceStreamHandler* self,
+ struct ArrowAsyncProducer* producer,
+ struct ArrowSchema* stream_schema, const char* addl_metadata);
+ int (*on_next_task)(struct ArrowAsyncDeviceStreamHandler* self,
+ struct ArrowAsyncTask* task, const char* metadata);
+ void (*on_error)(struct ArrowAsyncDeviceStreamHandler* self,
+ int code, const char* message, const char* metadata);
+
+ // release callback
+ void (*release)(struct ArrowAsyncDeviceStreamHandler* self);
+
+ // opaque handler-specific data
+ void* private_data;
+ };
+
+ #endif // ARROW_C_ASYNC_STREAM_INTERFACE
+
+.. note::
+ The canonical guard ``ARROW_C_ASYNC_STREAM_INTERFACE`` is meant to avoid
+ duplicate definitions if two projects copy the C async stream interface
+ definitions into their own headers, and a third-party project includes
+ from these two projects. It is therefore important that this guard is kept
+ exactly as-is when these definitions are copied.
+
+The ArrowAsyncDeviceStreamHandler structure
+'''''''''''''''''''''''''''''''''''''''''''
+
+The structure has the following fields:
+
+.. c:member:: int (*ArrowAsyncDeviceStreamHandler.on_schema)(struct ArrowAsyncDeviceStreamHandler*, struct ArrowAsyncProducer*, struct ArrowSchema*, const char*)
+
+ *Mandatory.* Handler for receiving the schema of the stream. All records should
+ match the provided schema. If successful, the function should return 0, otherwise
+ it should return an ``errno``-compatible error code.
+
+ The ``const char*`` parameter exists for producers to provide any extra contextual information
+ they want, such as the total number of rows in the stream, statistics, or otherwise. This is
+ encoded in the same format as :c:member:`ArrowSchema.metadata`. If not ``NULL``,
+ the lifetime is only the scope of the call to this function. A consumer who wants to maintain
+ the additional metadata beyond the lifetime of this call *MUST* copy the value themselves.
+
+ Unless the ``on_error`` handler is called, this will always get called exactly once and will be
+ the first method called on this object. As such the producer *MUST* provide an ``ArrowAsyncProducer``
+ object when calling this function to allow the consumer to manage back-pressure and flow control.
+ The producer maintains ownership of the ``ArrowAsyncProducer`` and must clean it up before or after
+ calling the release callback on this object.
+
+ A producer that receives a non-zero result here must not subsequently call anything other than
+ the release callback on this object.
+
+.. c:member:: int (*ArrowAsyncDeviceStreamHandler.on_next_task)(struct ArrowAsyncDeviceStreamHandler*, struct ArrowAsyncTask*, const char*)
+
+ *Mandatory.* Handler to be called when a new record is available for processing. The
+ schema for each record should be the same as the schema that ``on_schema`` was called with.
+ If successfully handled, the function should return 0, otherwise it should return an
+ ``errno``-compatible error code.
+
+ Rather than passing the record itself it receives an ``ArrowAsyncTask`` instead to facilitate
+ better consumer-focused thread control as far as receiving the data. A call to this function
+ simply indicates that data is available via the provided task.
+
+ The ``const char*`` parameter exists for producers to provide any extra contextual information
+ they want. This is encoded in the same format as :c:member:`ArrowSchema.metadata`. If not ``NULL``,
+ the lifetime is only the scope of the call to this function. A consumer who wants to maintain
+ the additional metadata beyond the lifetime of this call *MUST* copy the value themselves.
+
+ A producer *MUST NOT* call this concurrently from multiple different threads.
+
+ The :c:member:`ArrowAsyncProducer.request` callback must be called to start receiving calls to this
+ handler.
+
+ This function *MUST* be able to be called re-entrantly on the same thread to allow for the
+ :c:member:`ArrowAsyncProducer.request` callback to potentially call this method recursively.
+
+.. c:member:: void (*ArrowAsyncDeviceStreamHandler.on_error)(struct ArrowAsyncDeviceStreamHandler, int, const char*, const char*)
+
+ *Mandatory.* Handler to be called when an error is encountered by the producer. After calling
+ this, the ``release`` callback will be called as the last call on this struct. The parameters
+ are an ``errno``-compatible error code and an optional error message and metadata.
+
+ If the message and metadata are not ``NULL``, their lifetime is only valid during the scope
+ of this call. A consumer who wants to maintain these values past the return of this function
+ *MUST* copy the values themselves.
+
+ If the metadata parameter is not ``NULL``, to provide key-value error metadata, then it should
+ be encoded identically to the way that metadata is encoded in :c:member:`ArrowSchema.metadata`.
+
+ It is valid for this to be called by a producer with or without a preceding call to
+ :c:member:`ArrowAsyncProducer.request`. This callback *MUST NOT* call any methods of an
+ ``ArrowAsyncProducer`` object.
+
+.. c:member:: void (*ArrowAsyncDeviceStreamHandler.release)(struct ArrowAsyncDeviceStreamHandler*)
+
+ *Mandatory.* A pointer to a consumer-provided release callback for the handler.
+
+ It is valid for this to be called by a producer with or without a preceding call to
+ :c:member:`ArrowAsyncProducer.request`. This must not call any methods of an ``ArrowAsyncProducer``
+ object.
+
+.. c:member:: void* ArrowAsyncDeviceStreamHandler.private_data
+
+ *Optional.* An opaque pointer to consumer-provided private data.
+
+ Producers *MUST NOT* process this member. Lifetime of this member is handled by
+ the consumer, and especially by the release callback.
+
+The ArrowAsyncTask structure
+''''''''''''''''''''''''''''
+
+The purpose of using a Task object rather than passing the array directly to the ``on_next``
+callback is to allow for more complex and efficient thread handling. Utilizing a Task
+object allows for a producer to separate the "decoding" logic from the I/O, enabling a
+consumer to avoid transferring data between CPU cores (e.g. from one L1/L2 cache to another).
+
+This producer-provided structure has the following fields:
+
+.. c:member:: int (*ArrowArrayTask.get_data)(struct ArrowArrayTask*, struct ArrowDeviceArray*)
+
+ *Mandatory.* A callback to populate the provided ``ArrowDeviceArray`` with the available data.
+ The order of ArrowAsyncTasks provided by the producer enables a consumer to know the order of
+ the data to process.
+
+ If a non-zero value is returned from this, it should be followed only by the producer calling
+ the ``on_error`` callback of the ``ArrowAsyncDeviceStreamHandler``. Because calling this method
+ is likely to be separate from the current control flow, returning a non-zero value to signal
+ an error occuring allows the current thread to decide handle the case accordingly, while still
+ allowing all error logging and handling to be centralized in the
+ :c:member:`ArrowAsyncDeviceStreamHandler.on_error` callback.
+
+ Rather than having a separate release callback, any required cleanup should be performed as part
+ of the invocation of this callback, with ownership of the Array being passed to the consumer-provided
+ parameter calling this, and must be released separately.
+
+ It is only valid to call this method exactly once.
+
+.. c:member:: void* ArrowArrayTask.private_data
+
+ *Optional.* An opaque pointer to consumer-provided private data.
+
+ Consumers *MUST NOT* process this member. Lifetime of this member is handled by
+ the producer who created this object, and should be cleaned up if necessary during
+ the call to :c:member:`ArrowArrayTask.get_data`.
+
+The ArrowAsyncProducer structure
+''''''''''''''''''''''''''''''''
+
+This producer-provided and managed object has the following fields:
+
+.. c:member:: void (*ArrowAsyncProducer.request)(struct ArrowAsyncProducer*, uint64_t)
+
+ *Mandatory.* This function must be called by a consumer to start receiving calls to
+ :c:member:`ArrowAsyncDeviceStreamHandler.on_next_task`. It *MUST* be valid to call
+ this synchronously from within :c:member:`ArrowAsyncDeviceStreamHandler.on_next_task`
+ or :c:member:`ArrowAsyncDeviceStreamHandler.on_schema`. As a result, this function
+ *MUST NOT* synchronously call ``on_next_task`` or ``on_error`` to avoid recursive
+ and reentrant callbacks.
+
+ After ``cancel`` is called, additional calls to this function must be a NOP, but allowed.
+
+ While not cancelled, calling this function registers the given number of additional
+ arrays/batches to be produced by the producer. A producer should only call
+ the appropriate ``on_next_task`` callback up to a maximum of the total sum of calls to
+ this method before propagating back-pressure / waiting.
+
+ Any error encountered by calling request must be propagated by calling the ``on_error``
+ callback of the ``ArrowAsyncDeviceStreamHandler``.
+
+ A producer *MUST* support an unbounded number of calls to this callback and *MUST* support
+ a total registered demand (sum requested - sum delivered) of up to ``UINT64_MAX``.
+
+.. c:member:: void (*ArrowAsyncProducer.cancel)(struct ArrowAsyncProducer*)
+
+ *Mandatory.* This function signals to the producer that it must *eventually* stop calling
+ ``on_next_task``. Calls to ``cancel`` must be idempotent and thread-safe. After calling
+ it once, subsequent calls *MUST* be a NOP.
+
+ It is not required that calling ``cancel`` affect the producer *immediately*, only that it
+ must eventually stop calling ``on_next_task`` and then subsequently call ``release``
+ on the async handler object. As such, a consumer *MUST* be prepared to receive one or more
+ calls to ``on_next_task`` or ``on_error`` even after calling ``cancel`` if there are still
+ requested arrays pending.
+
+ Successful cancelling *MUST NOT* result in a producer calling
+ :c:member:`ArrowAsyncDeviceStreamHandler.on_error`, instead it should finish out any remaining
+ tasks (calling ``on_next_task`` accordingly) and eventually just call ``release``.
+
+ Any error encountered during handling a call to cancel must be reported via the ``on_error``
+ callback on the async stream handler.
+
+.. c:member:: void* ArrowAsyncProducer.private_data
+
+ *Optional.* An opaque pointer to producer-provided specific data.
+
+ Consumers *MUST NOT* process this member, the lifetime is owned by the producer
+ that constructed this object.
+
+Error Handling
+''''''''''''''
+
+Unlike the regular C Stream interface, the Async interface allows for errors to flow in
+both directions. As a result, error handling can be slightly more complex. Thus this spec
+designates the following rules:
+
+* If the producer encounters an error during processing, it should call the ``on_error``
+callback, and then call ``release`` after it returns.
+
+* If ``on_schema`` or ``on_next_task`` returns a non-zero integer value, the producer *should not*
+call the ``on_error`` callback, but instead should eventually call ``release`` at some point
+before or after any logging or processing of the error code.
+
+Result lifetimes
+''''''''''''''''
+
+The ``ArrowSchema`` passed to the ``on_schema`` callback must be released independantly,
+with the object itself needing to be moved to a consumer owned ``ArrowSchema`` object. The
+``ArrowSchema*`` passed as a parameter to the callback *MUST NOT* be stored and kept.
+
+The ``ArrowAsyncTask`` object provided to ``on_next_task`` is owned by the producer and
+will be cleaned up during the invocation of calling ``get_data`` on it. If the consumer
+doesn't care about the data, it should pass ``NULL`` instead of a valid ``ArrowDeviceArray*``.
+
+The ``const char*`` error ``message`` and ``metadata`` which are passed to ``on_error``
+are only valid within the scope of the ``on_error`` function itself. They must be copied
+if it is necessary for them to exist after it returns.
+
+Stream Handler Lifetime
+'''''''''''''''''''''''
+
+Lifetime of the async stream handler is managed using a release callback with similar
+usage as in :ref:`C data interface <c-data-interface-released>`.
+
+ArrowAsyncProducer Lifetime
+'''''''''''''''''''''''''''
+
+The lifetime of the ``ArrowAsyncProducer`` passed to ``on_schema`` is owned by the producer
+itself and should be managed by it. It *MUST* remain valid at least until just before
+calling ``release`` on the stream handler object.
+
+Thread safety
+'''''''''''''
+
+All handler functions should only be called in a serialized manner, but are not guaranteed
+to be called from the same thread every time. A producer should wait for handler callbacks to
+return before calling the next handler callback, and before calling the ``release`` callback.
+As a result, back-pressure is managed by how long it takes for the ``on_next`` handler to return.
Review Comment:
Removed this sentence, added a sentence that backpressure is managed by calls to `request`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-43631: [C][Format] Add ArrowAsyncDeviceStreamHandler interface [arrow]
Posted by "bkietz (via GitHub)" <gi...@apache.org>.
bkietz commented on PR #43632:
URL: https://github.com/apache/arrow/pull/43632#issuecomment-2293742608
> Do we want to handle cancellation here?
If the producer is free to respond slowly or even ignore signals sent through a `struct ScanHandle` then maybe this could be handled as simply as `int *ArrowAsyncDeviceStreamHandler::status`. This may be null or the consumer may update it for polling by the producer (all atomically). The producer might drop messages, like a transition from `continue -> pause` then `continue -> cancel` but that might not be a problem
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-43631: [C][Format] Add ArrowAsyncDeviceStreamHandler interface [arrow]
Posted by "felipecrv (via GitHub)" <gi...@apache.org>.
felipecrv commented on code in PR #43632:
URL: https://github.com/apache/arrow/pull/43632#discussion_r1722012720
##########
cpp/src/arrow/c/abi.h:
##########
@@ -228,6 +228,65 @@ struct ArrowDeviceArrayStream {
#endif // ARROW_C_DEVICE_STREAM_INTERFACE
+#ifndef ARROW_C_ASYNC_STREAM_INTERFACE
+#define ARROW_C_ASYNC_STREAM_INTERFACE
+
+// Similar to ArrowDeviceArrayStream, except designed for an asynchronous
+// style of interaction. While ArrowDeviceArrayStream provides producer
+// defined callbacks, this is intended to be created by the consumer instead.
+// The consumer passes this handler to the producer, which in turn uses the
+// callbacks to inform the consumer of events in the stream.
+struct ArrowAsyncDeviceStreamHandler {
+ // Handler for receiving a schema. The passed in stream_schema should be
+ // released or moved by the handler (producer is giving ownership of it to
+ // the handler).
+ //
+ // The `extension_param` argument can be null or can be used by a producer
+ // to pass arbitrary extra information to the consumer (such as total number
+ // of rows, context info, or otherwise).
+ //
+ // Return value: 0 if successful, `errno`-compatible error otherwise
+ int (*on_schema)(struct ArrowAsyncDeviceStreamHandler* self,
+ struct ArrowSchema* stream_schema, void* extension_param);
+
+ // Handler for receiving an array/record batch. Always called at least once
+ // unless an error is encountered (which would result in calling on_error).
+ // An empty/released array is passed to indicate the end of the stream if no
+ // errors have been encountered.
+ //
+ // The `extension_param` argument can be null or can be used by a producer
+ // to pass arbitrary extra information to the consumer.
+ //
+ // Return value: 0 if successful, `errno`-compatible error otherwise.
+ int (*on_next)(struct ArrowAsyncDeviceStreamHandler* self,
+ struct ArrowDeviceArray* next, void* extension_param);
Review Comment:
@leventov cool. Nice that we can keep C Data simple and not prevent advanced usages like the one you described.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-43631: [C][Format] Add ArrowAsyncDeviceStreamHandler interface [arrow]
Posted by "leventov (via GitHub)" <gi...@apache.org>.
leventov commented on PR #43632:
URL: https://github.com/apache/arrow/pull/43632#issuecomment-2337501707
### DataFusion
[`ExecutionPlan.execute`](https://docs.rs/datafusion/latest/datafusion/physical_plan/trait.ExecutionPlan.html#tymethod.execute):
> Begin execution of partition, returning a [Stream](https://docs.rs/futures-core/0.3.30/x86_64-unknown-linux-gnu/futures_core/stream/trait.Stream.html) of [RecordBatch](https://docs.rs/datafusion/latest/datafusion/common/arrow/record_batch/struct.RecordBatch.html)es.
Notes
The execute method itself is not async but it returns an async [futures::stream::Stream](https://docs.rs/futures-core/0.3.30/x86_64-unknown-linux-gnu/futures_core/stream/trait.Stream.html). This Stream should incrementally compute the output, RecordBatch by RecordBatch (in a streaming fashion). Most ExecutionPlans should not do any work before the first RecordBatch is requested from the stream.
[RecordBatchStreamAdapter](https://docs.rs/datafusion/latest/datafusion/physical_plan/stream/struct.RecordBatchStreamAdapter.html) can be used to convert an async [Stream](https://docs.rs/futures-core/0.3.30/x86_64-unknown-linux-gnu/futures_core/stream/trait.Stream.html) into a [SendableRecordBatchStream](https://docs.rs/datafusion/latest/datafusion/physical_plan/type.SendableRecordBatchStream.html).
Using async Streams allows for network I/O during execution and takes advantage of Rust’s built in support for async continuations and crate ecosystem.
Cancellation / Aborting Execution
The [Stream](https://docs.rs/futures-core/0.3.30/x86_64-unknown-linux-gnu/futures_core/stream/trait.Stream.html) that is returned must ensure that any allocated resources are freed when the stream itself is dropped. This is particularly important for [spawn](https://docs.rs/tokio/1.39.2/x86_64-unknown-linux-gnu/tokio/task/spawn/fn.spawn.html)ed tasks or threads. Unless care is taken to “abort” such tasks, they may continue to consume resources even after the plan is dropped, generating intermediate results that are never used. Thus, [spawn](https://docs.rs/tokio/1.39.2/x86_64-unknown-linux-gnu/tokio/task/spawn/fn.spawn.html) is disallowed, and instead use [SpawnedTask](https://docs.rs/datafusion/latest/datafusion/common/runtime/struct.SpawnedTask.html).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-43631: [C][Format] Add ArrowAsyncDeviceStreamHandler interface [arrow]
Posted by "pitrou (via GitHub)" <gi...@apache.org>.
pitrou commented on code in PR #43632:
URL: https://github.com/apache/arrow/pull/43632#discussion_r1764968946
##########
docs/source/format/CDeviceDataInterface.rst:
##########
@@ -650,6 +652,331 @@ The stream source is not assumed to be thread-safe. Consumers wanting to
call ``get_next`` from several threads should ensure those calls are
serialized.
+Async Device Stream Interface
+=============================
+
+The :ref:`C stream interface <_c-device-stream-interface>` provides a synchronous
+API centered around the consumer calling the callback functions to retrieve
+the next record batch. For some bindings, use cases, and interoperability, a more
+asynchronous, producer-focused interface may be required. These scenarios can utilize
+the ``ArrowAsyncDeviceStreamHandler``.
+
+Semantics
+---------
+
+Rather than the producer providing a structure of callbacks for a consumer to
+call and retrieve records, the Async interface is a consumer allocated structure.
+The consumer allocated struct provides handler callbacks for the producer to call
+when the schema and chunks of data are available, rather than the consumer using
+a blocking pull-style iteration.
+
+In addition to the ``ArrowAsyncDeviceStreamHandler``, there are also two additional
+structs used for the full data flow: ``ArrowAsyncTask`` and ``ArrowAsyncProducer``.
+
+Structure Definition
+--------------------
+
+The C device async stream interface is defined with a single ``struct`` definition:
+
+.. code-block:: c
+
+ #ifndef ARROW_C_ASYNC_STREAM_INTERFACE
+ #define ARROW_C_ASYNC_STREAM_INTERFACE
+
+ struct ArrowAsyncTask {
+ int (*get_data)(struct ArrowArrayTask* self, struct ArrowDeviceArray* out);
+
+ void* private_data;
+ };
+
+ struct ArrowAsyncProducer {
+ void (*request)(struct ArrowAsyncProducer* self, uint64_t n);
+ void (*cancel)(struct ArrowAsyncProducer* self);
+
+ void (*release)(struct ArrowAsyncProducer* self);
+ void* private_data;
+ };
+
+ struct ArrowAsyncDeviceStreamHandler {
+ // handlers
+ int (*on_schema)(struct ArrowAsyncDeviceStreamHandler* self,
+ struct ArrowAsyncProducer* producer,
+ struct ArrowSchema* stream_schema, const char* addl_metadata);
+ int (*on_next_task)(struct ArrowAsyncDeviceStreamHandler* self,
+ struct ArrowAsyncTask* task, const char* metadata);
+ void (*on_error)(struct ArrowAsyncDeviceStreamHandler* self,
+ int code, const char* message, const char* metadata);
+
+ // release callback
+ void (*release)(struct ArrowAsyncDeviceStreamHandler* self);
+
+ // opaque handler-specific data
+ void* private_data;
+ };
+
+ #endif // ARROW_C_ASYNC_STREAM_INTERFACE
+
+.. note::
+ The canonical guard ``ARROW_C_ASYNC_STREAM_INTERFACE`` is meant to avoid
+ duplicate definitions if two projects copy the C async stream interface
+ definitions into their own headers, and a third-party project includes
+ from these two projects. It is therefore important that this guard is kept
+ exactly as-is when these definitions are copied.
+
+The ArrowAsyncDeviceStreamHandler structure
+'''''''''''''''''''''''''''''''''''''''''''
+
+The structure has the following fields:
+
+.. c:member:: int (*ArrowAsyncDeviceStreamHandler.on_schema)(struct ArrowAsyncDeviceStreamHandler*, struct ArrowAsyncProducer*, struct ArrowSchema*, const char*)
+
+ *Mandatory.* Handler for receiving the schema of the stream. All records should
+ match the provided schema. If successful, the function should return 0, otherwise
+ it should return an ``errno``-compatible error code.
+
+ The ``const char*`` parameter exists for producers to provide any extra contextual information
+ they want, such as the total number of rows in the stream, statistics, or otherwise. This is
+ encoded in the same format as :c:member:`ArrowSchema.metadata`. If not ``NULL``,
+ the lifetime is only the scope of the call to this function. A consumer who wants to maintain
+ the additional metadata beyond the lifetime of this call *MUST* copy the value themselves.
+
+ Unless the ``on_error`` handler is called, this will always get called exactly once and will be
+ the first method called on this object. As such the producer *MUST* provide an ``ArrowAsyncProducer``
+ object when calling this function to allow the consumer to manage back-pressure and flow control.
+ The producer maintains ownership of the ``ArrowAsyncProducer`` and must clean it up before or after
+ calling the release callback on this object.
+
+ A producer that receives a non-zero result here must not subsequently call anything other than
+ the release callback on this object.
+
+.. c:member:: int (*ArrowAsyncDeviceStreamHandler.on_next_task)(struct ArrowAsyncDeviceStreamHandler*, struct ArrowAsyncTask*, const char*)
+
+ *Mandatory.* Handler to be called when a new record is available for processing. The
+ schema for each record should be the same as the schema that ``on_schema`` was called with.
+ If successfully handled, the function should return 0, otherwise it should return an
+ ``errno``-compatible error code.
+
+ Rather than passing the record itself it receives an ``ArrowAsyncTask`` instead to facilitate
+ better consumer-focused thread control as far as receiving the data. A call to this function
+ simply indicates that data is available via the provided task.
+
+ The ``const char*`` parameter exists for producers to provide any extra contextual information
+ they want. This is encoded in the same format as :c:member:`ArrowSchema.metadata`. If not ``NULL``,
+ the lifetime is only the scope of the call to this function. A consumer who wants to maintain
+ the additional metadata beyond the lifetime of this call *MUST* copy the value themselves.
+
+ A producer *MUST NOT* call this concurrently from multiple different threads.
+
+ The :c:member:`ArrowAsyncProducer.request` callback must be called to start receiving calls to this
+ handler.
+
+ This function *MUST* be able to be called re-entrantly on the same thread to allow for the
+ :c:member:`ArrowAsyncProducer.request` callback to potentially call this method recursively.
+
+.. c:member:: void (*ArrowAsyncDeviceStreamHandler.on_error)(struct ArrowAsyncDeviceStreamHandler, int, const char*, const char*)
+
+ *Mandatory.* Handler to be called when an error is encountered by the producer. After calling
+ this, the ``release`` callback will be called as the last call on this struct. The parameters
+ are an ``errno``-compatible error code and an optional error message and metadata.
+
+ If the message and metadata are not ``NULL``, their lifetime is only valid during the scope
+ of this call. A consumer who wants to maintain these values past the return of this function
+ *MUST* copy the values themselves.
+
+ If the metadata parameter is not ``NULL``, to provide key-value error metadata, then it should
+ be encoded identically to the way that metadata is encoded in :c:member:`ArrowSchema.metadata`.
+
+ It is valid for this to be called by a producer with or without a preceding call to
+ :c:member:`ArrowAsyncProducer.request`. This callback *MUST NOT* call any methods of an
+ ``ArrowAsyncProducer`` object.
+
+.. c:member:: void (*ArrowAsyncDeviceStreamHandler.release)(struct ArrowAsyncDeviceStreamHandler*)
+
+ *Mandatory.* A pointer to a consumer-provided release callback for the handler.
+
+ It is valid for this to be called by a producer with or without a preceding call to
+ :c:member:`ArrowAsyncProducer.request`. This must not call any methods of an ``ArrowAsyncProducer``
+ object.
+
+.. c:member:: void* ArrowAsyncDeviceStreamHandler.private_data
+
+ *Optional.* An opaque pointer to consumer-provided private data.
+
+ Producers *MUST NOT* process this member. Lifetime of this member is handled by
+ the consumer, and especially by the release callback.
+
+The ArrowAsyncTask structure
+''''''''''''''''''''''''''''
+
+The purpose of using a Task object rather than passing the array directly to the ``on_next``
+callback is to allow for more complex and efficient thread handling. Utilizing a Task
+object allows for a producer to separate the "decoding" logic from the I/O, enabling a
+consumer to avoid transferring data between CPU cores (e.g. from one L1/L2 cache to another).
+
+This producer-provided structure has the following fields:
+
+.. c:member:: int (*ArrowArrayTask.get_data)(struct ArrowArrayTask*, struct ArrowDeviceArray*)
+
+ *Mandatory.* A callback to populate the provided ``ArrowDeviceArray`` with the available data.
+ The order of ArrowAsyncTasks provided by the producer enables a consumer to know the order of
+ the data to process.
+
+ If a non-zero value is returned from this, it should be followed only by the producer calling
+ the ``on_error`` callback of the ``ArrowAsyncDeviceStreamHandler``. Because calling this method
+ is likely to be separate from the current control flow, returning a non-zero value to signal
+ an error occuring allows the current thread to decide handle the case accordingly, while still
+ allowing all error logging and handling to be centralized in the
+ :c:member:`ArrowAsyncDeviceStreamHandler.on_error` callback.
+
+ Rather than having a separate release callback, any required cleanup should be performed as part
+ of the invocation of this callback, with ownership of the Array being passed to the consumer-provided
+ parameter calling this, and must be released separately.
+
+ It is only valid to call this method exactly once.
+
+.. c:member:: void* ArrowArrayTask.private_data
+
+ *Optional.* An opaque pointer to consumer-provided private data.
+
+ Consumers *MUST NOT* process this member. Lifetime of this member is handled by
+ the producer who created this object, and should be cleaned up if necessary during
+ the call to :c:member:`ArrowArrayTask.get_data`.
+
+The ArrowAsyncProducer structure
+''''''''''''''''''''''''''''''''
+
+This producer-provided and managed object has the following fields:
+
+.. c:member:: void (*ArrowAsyncProducer.request)(struct ArrowAsyncProducer*, uint64_t)
+
+ *Mandatory.* This function must be called by a consumer to start receiving calls to
+ :c:member:`ArrowAsyncDeviceStreamHandler.on_next_task`. It *MUST* be valid to call
+ this synchronously from within :c:member:`ArrowAsyncDeviceStreamHandler.on_next_task`
+ or :c:member:`ArrowAsyncDeviceStreamHandler.on_schema`. As a result, this function
+ *MUST NOT* synchronously call ``on_next_task`` or ``on_error`` to avoid recursive
+ and reentrant callbacks.
+
+ After ``cancel`` is called, additional calls to this function must be a NOP, but allowed.
+
+ While not cancelled, calling this function registers the given number of additional
+ arrays/batches to be produced by the producer. A producer should only call
+ the appropriate ``on_next_task`` callback up to a maximum of the total sum of calls to
+ this method before propagating back-pressure / waiting.
Review Comment:
Ah, I see.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-43631: [C][Format] Add ArrowAsyncDeviceStreamHandler interface [arrow]
Posted by "zeroshade (via GitHub)" <gi...@apache.org>.
zeroshade commented on code in PR #43632:
URL: https://github.com/apache/arrow/pull/43632#discussion_r1765372798
##########
docs/source/format/CDeviceDataInterface.rst:
##########
@@ -650,6 +652,331 @@ The stream source is not assumed to be thread-safe. Consumers wanting to
call ``get_next`` from several threads should ensure those calls are
serialized.
+Async Device Stream Interface
+=============================
+
+The :ref:`C stream interface <_c-device-stream-interface>` provides a synchronous
+API centered around the consumer calling the callback functions to retrieve
+the next record batch. For some bindings, use cases, and interoperability, a more
+asynchronous, producer-focused interface may be required. These scenarios can utilize
+the ``ArrowAsyncDeviceStreamHandler``.
+
+Semantics
+---------
+
+Rather than the producer providing a structure of callbacks for a consumer to
+call and retrieve records, the Async interface is a consumer allocated structure.
+The consumer allocated struct provides handler callbacks for the producer to call
+when the schema and chunks of data are available, rather than the consumer using
+a blocking pull-style iteration.
+
+In addition to the ``ArrowAsyncDeviceStreamHandler``, there are also two additional
+structs used for the full data flow: ``ArrowAsyncTask`` and ``ArrowAsyncProducer``.
+
+Structure Definition
+--------------------
+
+The C device async stream interface is defined with a single ``struct`` definition:
+
+.. code-block:: c
+
+ #ifndef ARROW_C_ASYNC_STREAM_INTERFACE
+ #define ARROW_C_ASYNC_STREAM_INTERFACE
+
+ struct ArrowAsyncTask {
+ int (*get_data)(struct ArrowArrayTask* self, struct ArrowDeviceArray* out);
+
+ void* private_data;
+ };
+
+ struct ArrowAsyncProducer {
+ void (*request)(struct ArrowAsyncProducer* self, uint64_t n);
+ void (*cancel)(struct ArrowAsyncProducer* self);
+
+ void (*release)(struct ArrowAsyncProducer* self);
+ void* private_data;
+ };
+
+ struct ArrowAsyncDeviceStreamHandler {
+ // handlers
+ int (*on_schema)(struct ArrowAsyncDeviceStreamHandler* self,
+ struct ArrowAsyncProducer* producer,
+ struct ArrowSchema* stream_schema, const char* addl_metadata);
+ int (*on_next_task)(struct ArrowAsyncDeviceStreamHandler* self,
+ struct ArrowAsyncTask* task, const char* metadata);
+ void (*on_error)(struct ArrowAsyncDeviceStreamHandler* self,
+ int code, const char* message, const char* metadata);
+
+ // release callback
+ void (*release)(struct ArrowAsyncDeviceStreamHandler* self);
+
+ // opaque handler-specific data
+ void* private_data;
+ };
+
+ #endif // ARROW_C_ASYNC_STREAM_INTERFACE
+
+.. note::
+ The canonical guard ``ARROW_C_ASYNC_STREAM_INTERFACE`` is meant to avoid
+ duplicate definitions if two projects copy the C async stream interface
+ definitions into their own headers, and a third-party project includes
+ from these two projects. It is therefore important that this guard is kept
+ exactly as-is when these definitions are copied.
+
+The ArrowAsyncDeviceStreamHandler structure
+'''''''''''''''''''''''''''''''''''''''''''
+
+The structure has the following fields:
+
+.. c:member:: int (*ArrowAsyncDeviceStreamHandler.on_schema)(struct ArrowAsyncDeviceStreamHandler*, struct ArrowAsyncProducer*, struct ArrowSchema*, const char*)
+
+ *Mandatory.* Handler for receiving the schema of the stream. All records should
+ match the provided schema. If successful, the function should return 0, otherwise
+ it should return an ``errno``-compatible error code.
+
+ The ``const char*`` parameter exists for producers to provide any extra contextual information
+ they want, such as the total number of rows in the stream, statistics, or otherwise. This is
+ encoded in the same format as :c:member:`ArrowSchema.metadata`. If not ``NULL``,
+ the lifetime is only the scope of the call to this function. A consumer who wants to maintain
+ the additional metadata beyond the lifetime of this call *MUST* copy the value themselves.
+
+ Unless the ``on_error`` handler is called, this will always get called exactly once and will be
+ the first method called on this object. As such the producer *MUST* provide an ``ArrowAsyncProducer``
+ object when calling this function to allow the consumer to manage back-pressure and flow control.
+ The producer maintains ownership of the ``ArrowAsyncProducer`` and must clean it up before or after
+ calling the release callback on this object.
+
+ A producer that receives a non-zero result here must not subsequently call anything other than
+ the release callback on this object.
+
+.. c:member:: int (*ArrowAsyncDeviceStreamHandler.on_next_task)(struct ArrowAsyncDeviceStreamHandler*, struct ArrowAsyncTask*, const char*)
+
+ *Mandatory.* Handler to be called when a new record is available for processing. The
+ schema for each record should be the same as the schema that ``on_schema`` was called with.
+ If successfully handled, the function should return 0, otherwise it should return an
+ ``errno``-compatible error code.
+
+ Rather than passing the record itself it receives an ``ArrowAsyncTask`` instead to facilitate
+ better consumer-focused thread control as far as receiving the data. A call to this function
+ simply indicates that data is available via the provided task.
+
+ The ``const char*`` parameter exists for producers to provide any extra contextual information
+ they want. This is encoded in the same format as :c:member:`ArrowSchema.metadata`. If not ``NULL``,
+ the lifetime is only the scope of the call to this function. A consumer who wants to maintain
+ the additional metadata beyond the lifetime of this call *MUST* copy the value themselves.
+
+ A producer *MUST NOT* call this concurrently from multiple different threads.
+
+ The :c:member:`ArrowAsyncProducer.request` callback must be called to start receiving calls to this
+ handler.
+
+ This function *MUST* be able to be called re-entrantly on the same thread to allow for the
+ :c:member:`ArrowAsyncProducer.request` callback to potentially call this method recursively.
+
+.. c:member:: void (*ArrowAsyncDeviceStreamHandler.on_error)(struct ArrowAsyncDeviceStreamHandler, int, const char*, const char*)
+
+ *Mandatory.* Handler to be called when an error is encountered by the producer. After calling
+ this, the ``release`` callback will be called as the last call on this struct. The parameters
+ are an ``errno``-compatible error code and an optional error message and metadata.
+
+ If the message and metadata are not ``NULL``, their lifetime is only valid during the scope
+ of this call. A consumer who wants to maintain these values past the return of this function
+ *MUST* copy the values themselves.
+
+ If the metadata parameter is not ``NULL``, to provide key-value error metadata, then it should
+ be encoded identically to the way that metadata is encoded in :c:member:`ArrowSchema.metadata`.
+
+ It is valid for this to be called by a producer with or without a preceding call to
+ :c:member:`ArrowAsyncProducer.request`. This callback *MUST NOT* call any methods of an
+ ``ArrowAsyncProducer`` object.
+
+.. c:member:: void (*ArrowAsyncDeviceStreamHandler.release)(struct ArrowAsyncDeviceStreamHandler*)
+
+ *Mandatory.* A pointer to a consumer-provided release callback for the handler.
+
+ It is valid for this to be called by a producer with or without a preceding call to
+ :c:member:`ArrowAsyncProducer.request`. This must not call any methods of an ``ArrowAsyncProducer``
+ object.
+
+.. c:member:: void* ArrowAsyncDeviceStreamHandler.private_data
+
+ *Optional.* An opaque pointer to consumer-provided private data.
+
+ Producers *MUST NOT* process this member. Lifetime of this member is handled by
+ the consumer, and especially by the release callback.
+
+The ArrowAsyncTask structure
+''''''''''''''''''''''''''''
+
+The purpose of using a Task object rather than passing the array directly to the ``on_next``
+callback is to allow for more complex and efficient thread handling. Utilizing a Task
+object allows for a producer to separate the "decoding" logic from the I/O, enabling a
+consumer to avoid transferring data between CPU cores (e.g. from one L1/L2 cache to another).
Review Comment:
As long as this function is still not blocking on IO for the consumer, and the logic is managed by the producer, I don't see that as being an issue here. We're not dealing with multiple processes here as this is a C ABI, not a cross process interface, so ultimately it's just a matter of who is managing the thread pool.
In the example you've provided, one of two things could happen:
1. The producer could decide that there's no benefit to the consumer to offload the decompression and simply perform that decompression itself before it calls `on_next_task`, so `get_data` is just handing off the `ArrowDeviceArray` object.
2. The producer decides there is benefit to allowing the consumer to determine *when* the decompression happens and bakes this logic into the `get_data` callback. The consumer chooses when to call `get_data` (a synchronous function call) which then utilizes the producer logic to perform the decompression (possibly across multiple producer threads if really desired) and then receives the final result.
In both cases, it allows for reduced pressure on the L2/L1 caches for handing the data off and the producer is making the decision as to whether or not there is benefit to offloading the logic.
The documentation explicitly states that `get_data` is only allowed to be called **exactly once** on a given `ArrowAsyncTask` so there isn't any concern of calling it on a single task from multiple threads.
@lidavidm @westonpace @bkietz do any of you have any thoughts on this discussion? As this is a core aspect of the current design I'd like to get this hammered out before I officially send this to the mailing list for a vote (in case I need to explicitly change the design as a result of this discussion).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-43631: [C][Format] Add ArrowAsyncDeviceStreamHandler interface [arrow]
Posted by "pitrou (via GitHub)" <gi...@apache.org>.
pitrou commented on code in PR #43632:
URL: https://github.com/apache/arrow/pull/43632#discussion_r1757495841
##########
docs/source/format/CDeviceDataInterface.rst:
##########
@@ -650,6 +652,331 @@ The stream source is not assumed to be thread-safe. Consumers wanting to
call ``get_next`` from several threads should ensure those calls are
serialized.
+Async Device Stream Interface
+=============================
+
+The :ref:`C stream interface <_c-device-stream-interface>` provides a synchronous
+API centered around the consumer calling the callback functions to retrieve
+the next record batch. For some bindings, use cases, and interoperability, a more
+asynchronous, producer-focused interface may be required. These scenarios can utilize
+the ``ArrowAsyncDeviceStreamHandler``.
+
+Semantics
+---------
+
+Rather than the producer providing a structure of callbacks for a consumer to
+call and retrieve records, the Async interface is a consumer allocated structure.
+The consumer allocated struct provides handler callbacks for the producer to call
+when the schema and chunks of data are available, rather than the consumer using
+a blocking pull-style iteration.
+
+In addition to the ``ArrowAsyncDeviceStreamHandler``, there are also two additional
+structs used for the full data flow: ``ArrowAsyncTask`` and ``ArrowAsyncProducer``.
+
+Structure Definition
+--------------------
+
+The C device async stream interface is defined with a single ``struct`` definition:
+
+.. code-block:: c
+
+ #ifndef ARROW_C_ASYNC_STREAM_INTERFACE
+ #define ARROW_C_ASYNC_STREAM_INTERFACE
+
+ struct ArrowAsyncTask {
+ int (*get_data)(struct ArrowArrayTask* self, struct ArrowDeviceArray* out);
+
+ void* private_data;
+ };
+
+ struct ArrowAsyncProducer {
+ void (*request)(struct ArrowAsyncProducer* self, uint64_t n);
+ void (*cancel)(struct ArrowAsyncProducer* self);
+
+ void (*release)(struct ArrowAsyncProducer* self);
+ void* private_data;
+ };
+
+ struct ArrowAsyncDeviceStreamHandler {
+ // handlers
+ int (*on_schema)(struct ArrowAsyncDeviceStreamHandler* self,
+ struct ArrowAsyncProducer* producer,
+ struct ArrowSchema* stream_schema, const char* addl_metadata);
+ int (*on_next_task)(struct ArrowAsyncDeviceStreamHandler* self,
+ struct ArrowAsyncTask* task, const char* metadata);
+ void (*on_error)(struct ArrowAsyncDeviceStreamHandler* self,
+ int code, const char* message, const char* metadata);
+
+ // release callback
+ void (*release)(struct ArrowAsyncDeviceStreamHandler* self);
+
+ // opaque handler-specific data
+ void* private_data;
+ };
+
+ #endif // ARROW_C_ASYNC_STREAM_INTERFACE
+
+.. note::
+ The canonical guard ``ARROW_C_ASYNC_STREAM_INTERFACE`` is meant to avoid
+ duplicate definitions if two projects copy the C async stream interface
+ definitions into their own headers, and a third-party project includes
+ from these two projects. It is therefore important that this guard is kept
+ exactly as-is when these definitions are copied.
+
+The ArrowAsyncDeviceStreamHandler structure
+'''''''''''''''''''''''''''''''''''''''''''
+
+The structure has the following fields:
+
+.. c:member:: int (*ArrowAsyncDeviceStreamHandler.on_schema)(struct ArrowAsyncDeviceStreamHandler*, struct ArrowAsyncProducer*, struct ArrowSchema*, const char*)
+
+ *Mandatory.* Handler for receiving the schema of the stream. All records should
+ match the provided schema. If successful, the function should return 0, otherwise
+ it should return an ``errno``-compatible error code.
+
+ The ``const char*`` parameter exists for producers to provide any extra contextual information
+ they want, such as the total number of rows in the stream, statistics, or otherwise. This is
+ encoded in the same format as :c:member:`ArrowSchema.metadata`. If not ``NULL``,
+ the lifetime is only the scope of the call to this function. A consumer who wants to maintain
+ the additional metadata beyond the lifetime of this call *MUST* copy the value themselves.
+
+ Unless the ``on_error`` handler is called, this will always get called exactly once and will be
+ the first method called on this object. As such the producer *MUST* provide an ``ArrowAsyncProducer``
+ object when calling this function to allow the consumer to manage back-pressure and flow control.
+ The producer maintains ownership of the ``ArrowAsyncProducer`` and must clean it up before or after
+ calling the release callback on this object.
+
+ A producer that receives a non-zero result here must not subsequently call anything other than
+ the release callback on this object.
+
+.. c:member:: int (*ArrowAsyncDeviceStreamHandler.on_next_task)(struct ArrowAsyncDeviceStreamHandler*, struct ArrowAsyncTask*, const char*)
+
+ *Mandatory.* Handler to be called when a new record is available for processing. The
+ schema for each record should be the same as the schema that ``on_schema`` was called with.
+ If successfully handled, the function should return 0, otherwise it should return an
+ ``errno``-compatible error code.
+
+ Rather than passing the record itself it receives an ``ArrowAsyncTask`` instead to facilitate
+ better consumer-focused thread control as far as receiving the data. A call to this function
+ simply indicates that data is available via the provided task.
+
+ The ``const char*`` parameter exists for producers to provide any extra contextual information
+ they want. This is encoded in the same format as :c:member:`ArrowSchema.metadata`. If not ``NULL``,
+ the lifetime is only the scope of the call to this function. A consumer who wants to maintain
+ the additional metadata beyond the lifetime of this call *MUST* copy the value themselves.
+
+ A producer *MUST NOT* call this concurrently from multiple different threads.
+
+ The :c:member:`ArrowAsyncProducer.request` callback must be called to start receiving calls to this
+ handler.
+
+ This function *MUST* be able to be called re-entrantly on the same thread to allow for the
+ :c:member:`ArrowAsyncProducer.request` callback to potentially call this method recursively.
+
+.. c:member:: void (*ArrowAsyncDeviceStreamHandler.on_error)(struct ArrowAsyncDeviceStreamHandler, int, const char*, const char*)
+
+ *Mandatory.* Handler to be called when an error is encountered by the producer. After calling
+ this, the ``release`` callback will be called as the last call on this struct. The parameters
+ are an ``errno``-compatible error code and an optional error message and metadata.
+
+ If the message and metadata are not ``NULL``, their lifetime is only valid during the scope
+ of this call. A consumer who wants to maintain these values past the return of this function
+ *MUST* copy the values themselves.
+
+ If the metadata parameter is not ``NULL``, to provide key-value error metadata, then it should
+ be encoded identically to the way that metadata is encoded in :c:member:`ArrowSchema.metadata`.
+
+ It is valid for this to be called by a producer with or without a preceding call to
+ :c:member:`ArrowAsyncProducer.request`. This callback *MUST NOT* call any methods of an
+ ``ArrowAsyncProducer`` object.
+
+.. c:member:: void (*ArrowAsyncDeviceStreamHandler.release)(struct ArrowAsyncDeviceStreamHandler*)
+
+ *Mandatory.* A pointer to a consumer-provided release callback for the handler.
+
+ It is valid for this to be called by a producer with or without a preceding call to
+ :c:member:`ArrowAsyncProducer.request`. This must not call any methods of an ``ArrowAsyncProducer``
+ object.
+
+.. c:member:: void* ArrowAsyncDeviceStreamHandler.private_data
+
+ *Optional.* An opaque pointer to consumer-provided private data.
+
+ Producers *MUST NOT* process this member. Lifetime of this member is handled by
+ the consumer, and especially by the release callback.
+
+The ArrowAsyncTask structure
+''''''''''''''''''''''''''''
+
+The purpose of using a Task object rather than passing the array directly to the ``on_next``
+callback is to allow for more complex and efficient thread handling. Utilizing a Task
+object allows for a producer to separate the "decoding" logic from the I/O, enabling a
+consumer to avoid transferring data between CPU cores (e.g. from one L1/L2 cache to another).
+
+This producer-provided structure has the following fields:
+
+.. c:member:: int (*ArrowArrayTask.get_data)(struct ArrowArrayTask*, struct ArrowDeviceArray*)
+
+ *Mandatory.* A callback to populate the provided ``ArrowDeviceArray`` with the available data.
+ The order of ArrowAsyncTasks provided by the producer enables a consumer to know the order of
+ the data to process.
+
+ If a non-zero value is returned from this, it should be followed only by the producer calling
+ the ``on_error`` callback of the ``ArrowAsyncDeviceStreamHandler``. Because calling this method
+ is likely to be separate from the current control flow, returning a non-zero value to signal
+ an error occuring allows the current thread to decide handle the case accordingly, while still
+ allowing all error logging and handling to be centralized in the
+ :c:member:`ArrowAsyncDeviceStreamHandler.on_error` callback.
+
+ Rather than having a separate release callback, any required cleanup should be performed as part
+ of the invocation of this callback, with ownership of the Array being passed to the consumer-provided
+ parameter calling this, and must be released separately.
+
+ It is only valid to call this method exactly once.
+
+.. c:member:: void* ArrowArrayTask.private_data
+
+ *Optional.* An opaque pointer to consumer-provided private data.
+
+ Consumers *MUST NOT* process this member. Lifetime of this member is handled by
+ the producer who created this object, and should be cleaned up if necessary during
+ the call to :c:member:`ArrowArrayTask.get_data`.
+
+The ArrowAsyncProducer structure
+''''''''''''''''''''''''''''''''
+
+This producer-provided and managed object has the following fields:
+
+.. c:member:: void (*ArrowAsyncProducer.request)(struct ArrowAsyncProducer*, uint64_t)
+
+ *Mandatory.* This function must be called by a consumer to start receiving calls to
+ :c:member:`ArrowAsyncDeviceStreamHandler.on_next_task`. It *MUST* be valid to call
+ this synchronously from within :c:member:`ArrowAsyncDeviceStreamHandler.on_next_task`
+ or :c:member:`ArrowAsyncDeviceStreamHandler.on_schema`. As a result, this function
+ *MUST NOT* synchronously call ``on_next_task`` or ``on_error`` to avoid recursive
+ and reentrant callbacks.
+
+ After ``cancel`` is called, additional calls to this function must be a NOP, but allowed.
+
+ While not cancelled, calling this function registers the given number of additional
+ arrays/batches to be produced by the producer. A producer should only call
+ the appropriate ``on_next_task`` callback up to a maximum of the total sum of calls to
+ this method before propagating back-pressure / waiting.
+
+ Any error encountered by calling request must be propagated by calling the ``on_error``
+ callback of the ``ArrowAsyncDeviceStreamHandler``.
+
+ A producer *MUST* support an unbounded number of calls to this callback and *MUST* support
+ a total registered demand (sum requested - sum delivered) of up to ``UINT64_MAX``.
+
+.. c:member:: void (*ArrowAsyncProducer.cancel)(struct ArrowAsyncProducer*)
+
+ *Mandatory.* This function signals to the producer that it must *eventually* stop calling
+ ``on_next_task``. Calls to ``cancel`` must be idempotent and thread-safe. After calling
+ it once, subsequent calls *MUST* be a NOP.
+
+ It is not required that calling ``cancel`` affect the producer *immediately*, only that it
+ must eventually stop calling ``on_next_task`` and then subsequently call ``release``
+ on the async handler object. As such, a consumer *MUST* be prepared to receive one or more
+ calls to ``on_next_task`` or ``on_error`` even after calling ``cancel`` if there are still
+ requested arrays pending.
+
+ Successful cancelling *MUST NOT* result in a producer calling
+ :c:member:`ArrowAsyncDeviceStreamHandler.on_error`, instead it should finish out any remaining
+ tasks (calling ``on_next_task`` accordingly) and eventually just call ``release``.
+
+ Any error encountered during handling a call to cancel must be reported via the ``on_error``
+ callback on the async stream handler.
+
+.. c:member:: void* ArrowAsyncProducer.private_data
+
+ *Optional.* An opaque pointer to producer-provided specific data.
+
+ Consumers *MUST NOT* process this member, the lifetime is owned by the producer
+ that constructed this object.
+
+Error Handling
+''''''''''''''
+
+Unlike the regular C Stream interface, the Async interface allows for errors to flow in
+both directions. As a result, error handling can be slightly more complex. Thus this spec
+designates the following rules:
+
+* If the producer encounters an error during processing, it should call the ``on_error``
+callback, and then call ``release`` after it returns.
+
+* If ``on_schema`` or ``on_next_task`` returns a non-zero integer value, the producer *should not*
+call the ``on_error`` callback, but instead should eventually call ``release`` at some point
+before or after any logging or processing of the error code.
+
+Result lifetimes
+''''''''''''''''
+
+The ``ArrowSchema`` passed to the ``on_schema`` callback must be released independantly,
+with the object itself needing to be moved to a consumer owned ``ArrowSchema`` object. The
+``ArrowSchema*`` passed as a parameter to the callback *MUST NOT* be stored and kept.
+
+The ``ArrowAsyncTask`` object provided to ``on_next_task`` is owned by the producer and
+will be cleaned up during the invocation of calling ``get_data`` on it. If the consumer
+doesn't care about the data, it should pass ``NULL`` instead of a valid ``ArrowDeviceArray*``.
+
+The ``const char*`` error ``message`` and ``metadata`` which are passed to ``on_error``
+are only valid within the scope of the ``on_error`` function itself. They must be copied
+if it is necessary for them to exist after it returns.
+
+Stream Handler Lifetime
+'''''''''''''''''''''''
+
+Lifetime of the async stream handler is managed using a release callback with similar
+usage as in :ref:`C data interface <c-data-interface-released>`.
+
+ArrowAsyncProducer Lifetime
+'''''''''''''''''''''''''''
+
+The lifetime of the ``ArrowAsyncProducer`` passed to ``on_schema`` is owned by the producer
+itself and should be managed by it. It *MUST* remain valid at least until just before
+calling ``release`` on the stream handler object.
+
+Thread safety
+'''''''''''''
+
+All handler functions should only be called in a serialized manner, but are not guaranteed
+to be called from the same thread every time. A producer should wait for handler callbacks to
+return before calling the next handler callback, and before calling the ``release`` callback.
+As a result, back-pressure is managed by how long it takes for the ``on_next`` handler to return.
+
+The ``ArrowAsyncDeviceStreamHandler`` object should be able to handle callbacks as soon as
+it is passed to the producer, any initialization should be performed before it is provided.
+
+Possible Sequence Diagram
+-------------------------
+
+.. mermaid::
Review Comment:
Is there an easy way to see a rendered version of this? :-)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-43631: [C][Format] Add ArrowAsyncDeviceStreamHandler interface [arrow]
Posted by "pitrou (via GitHub)" <gi...@apache.org>.
pitrou commented on code in PR #43632:
URL: https://github.com/apache/arrow/pull/43632#discussion_r1757483286
##########
docs/source/format/CDeviceDataInterface.rst:
##########
@@ -650,6 +652,331 @@ The stream source is not assumed to be thread-safe. Consumers wanting to
call ``get_next`` from several threads should ensure those calls are
serialized.
+Async Device Stream Interface
+=============================
+
+The :ref:`C stream interface <_c-device-stream-interface>` provides a synchronous
+API centered around the consumer calling the callback functions to retrieve
+the next record batch. For some bindings, use cases, and interoperability, a more
+asynchronous, producer-focused interface may be required. These scenarios can utilize
+the ``ArrowAsyncDeviceStreamHandler``.
+
+Semantics
+---------
+
+Rather than the producer providing a structure of callbacks for a consumer to
+call and retrieve records, the Async interface is a consumer allocated structure.
+The consumer allocated struct provides handler callbacks for the producer to call
+when the schema and chunks of data are available, rather than the consumer using
+a blocking pull-style iteration.
+
+In addition to the ``ArrowAsyncDeviceStreamHandler``, there are also two additional
+structs used for the full data flow: ``ArrowAsyncTask`` and ``ArrowAsyncProducer``.
+
+Structure Definition
+--------------------
+
+The C device async stream interface is defined with a single ``struct`` definition:
+
+.. code-block:: c
+
+ #ifndef ARROW_C_ASYNC_STREAM_INTERFACE
+ #define ARROW_C_ASYNC_STREAM_INTERFACE
+
+ struct ArrowAsyncTask {
+ int (*get_data)(struct ArrowArrayTask* self, struct ArrowDeviceArray* out);
+
+ void* private_data;
+ };
+
+ struct ArrowAsyncProducer {
+ void (*request)(struct ArrowAsyncProducer* self, uint64_t n);
+ void (*cancel)(struct ArrowAsyncProducer* self);
+
+ void (*release)(struct ArrowAsyncProducer* self);
+ void* private_data;
+ };
+
+ struct ArrowAsyncDeviceStreamHandler {
+ // handlers
+ int (*on_schema)(struct ArrowAsyncDeviceStreamHandler* self,
+ struct ArrowAsyncProducer* producer,
+ struct ArrowSchema* stream_schema, const char* addl_metadata);
+ int (*on_next_task)(struct ArrowAsyncDeviceStreamHandler* self,
+ struct ArrowAsyncTask* task, const char* metadata);
+ void (*on_error)(struct ArrowAsyncDeviceStreamHandler* self,
+ int code, const char* message, const char* metadata);
+
+ // release callback
+ void (*release)(struct ArrowAsyncDeviceStreamHandler* self);
+
+ // opaque handler-specific data
+ void* private_data;
+ };
+
+ #endif // ARROW_C_ASYNC_STREAM_INTERFACE
+
+.. note::
+ The canonical guard ``ARROW_C_ASYNC_STREAM_INTERFACE`` is meant to avoid
+ duplicate definitions if two projects copy the C async stream interface
+ definitions into their own headers, and a third-party project includes
+ from these two projects. It is therefore important that this guard is kept
+ exactly as-is when these definitions are copied.
+
+The ArrowAsyncDeviceStreamHandler structure
+'''''''''''''''''''''''''''''''''''''''''''
+
+The structure has the following fields:
+
+.. c:member:: int (*ArrowAsyncDeviceStreamHandler.on_schema)(struct ArrowAsyncDeviceStreamHandler*, struct ArrowAsyncProducer*, struct ArrowSchema*, const char*)
+
+ *Mandatory.* Handler for receiving the schema of the stream. All records should
+ match the provided schema. If successful, the function should return 0, otherwise
+ it should return an ``errno``-compatible error code.
+
+ The ``const char*`` parameter exists for producers to provide any extra contextual information
+ they want, such as the total number of rows in the stream, statistics, or otherwise. This is
+ encoded in the same format as :c:member:`ArrowSchema.metadata`. If not ``NULL``,
+ the lifetime is only the scope of the call to this function. A consumer who wants to maintain
+ the additional metadata beyond the lifetime of this call *MUST* copy the value themselves.
+
+ Unless the ``on_error`` handler is called, this will always get called exactly once and will be
+ the first method called on this object. As such the producer *MUST* provide an ``ArrowAsyncProducer``
+ object when calling this function to allow the consumer to manage back-pressure and flow control.
+ The producer maintains ownership of the ``ArrowAsyncProducer`` and must clean it up before or after
+ calling the release callback on this object.
+
+ A producer that receives a non-zero result here must not subsequently call anything other than
+ the release callback on this object.
+
+.. c:member:: int (*ArrowAsyncDeviceStreamHandler.on_next_task)(struct ArrowAsyncDeviceStreamHandler*, struct ArrowAsyncTask*, const char*)
+
+ *Mandatory.* Handler to be called when a new record is available for processing. The
+ schema for each record should be the same as the schema that ``on_schema`` was called with.
+ If successfully handled, the function should return 0, otherwise it should return an
+ ``errno``-compatible error code.
+
+ Rather than passing the record itself it receives an ``ArrowAsyncTask`` instead to facilitate
+ better consumer-focused thread control as far as receiving the data. A call to this function
+ simply indicates that data is available via the provided task.
+
+ The ``const char*`` parameter exists for producers to provide any extra contextual information
+ they want. This is encoded in the same format as :c:member:`ArrowSchema.metadata`. If not ``NULL``,
+ the lifetime is only the scope of the call to this function. A consumer who wants to maintain
+ the additional metadata beyond the lifetime of this call *MUST* copy the value themselves.
+
+ A producer *MUST NOT* call this concurrently from multiple different threads.
+
+ The :c:member:`ArrowAsyncProducer.request` callback must be called to start receiving calls to this
+ handler.
+
+ This function *MUST* be able to be called re-entrantly on the same thread to allow for the
+ :c:member:`ArrowAsyncProducer.request` callback to potentially call this method recursively.
+
+.. c:member:: void (*ArrowAsyncDeviceStreamHandler.on_error)(struct ArrowAsyncDeviceStreamHandler, int, const char*, const char*)
+
+ *Mandatory.* Handler to be called when an error is encountered by the producer. After calling
+ this, the ``release`` callback will be called as the last call on this struct. The parameters
+ are an ``errno``-compatible error code and an optional error message and metadata.
+
+ If the message and metadata are not ``NULL``, their lifetime is only valid during the scope
+ of this call. A consumer who wants to maintain these values past the return of this function
+ *MUST* copy the values themselves.
+
+ If the metadata parameter is not ``NULL``, to provide key-value error metadata, then it should
+ be encoded identically to the way that metadata is encoded in :c:member:`ArrowSchema.metadata`.
+
+ It is valid for this to be called by a producer with or without a preceding call to
+ :c:member:`ArrowAsyncProducer.request`. This callback *MUST NOT* call any methods of an
+ ``ArrowAsyncProducer`` object.
+
+.. c:member:: void (*ArrowAsyncDeviceStreamHandler.release)(struct ArrowAsyncDeviceStreamHandler*)
+
+ *Mandatory.* A pointer to a consumer-provided release callback for the handler.
+
+ It is valid for this to be called by a producer with or without a preceding call to
+ :c:member:`ArrowAsyncProducer.request`. This must not call any methods of an ``ArrowAsyncProducer``
+ object.
+
+.. c:member:: void* ArrowAsyncDeviceStreamHandler.private_data
+
+ *Optional.* An opaque pointer to consumer-provided private data.
+
+ Producers *MUST NOT* process this member. Lifetime of this member is handled by
+ the consumer, and especially by the release callback.
+
+The ArrowAsyncTask structure
+''''''''''''''''''''''''''''
+
+The purpose of using a Task object rather than passing the array directly to the ``on_next``
+callback is to allow for more complex and efficient thread handling. Utilizing a Task
+object allows for a producer to separate the "decoding" logic from the I/O, enabling a
+consumer to avoid transferring data between CPU cores (e.g. from one L1/L2 cache to another).
+
+This producer-provided structure has the following fields:
+
+.. c:member:: int (*ArrowArrayTask.get_data)(struct ArrowArrayTask*, struct ArrowDeviceArray*)
+
+ *Mandatory.* A callback to populate the provided ``ArrowDeviceArray`` with the available data.
+ The order of ArrowAsyncTasks provided by the producer enables a consumer to know the order of
+ the data to process.
+
+ If a non-zero value is returned from this, it should be followed only by the producer calling
+ the ``on_error`` callback of the ``ArrowAsyncDeviceStreamHandler``. Because calling this method
+ is likely to be separate from the current control flow, returning a non-zero value to signal
+ an error occuring allows the current thread to decide handle the case accordingly, while still
+ allowing all error logging and handling to be centralized in the
+ :c:member:`ArrowAsyncDeviceStreamHandler.on_error` callback.
+
+ Rather than having a separate release callback, any required cleanup should be performed as part
+ of the invocation of this callback, with ownership of the Array being passed to the consumer-provided
+ parameter calling this, and must be released separately.
+
+ It is only valid to call this method exactly once.
+
+.. c:member:: void* ArrowArrayTask.private_data
+
+ *Optional.* An opaque pointer to consumer-provided private data.
+
+ Consumers *MUST NOT* process this member. Lifetime of this member is handled by
+ the producer who created this object, and should be cleaned up if necessary during
+ the call to :c:member:`ArrowArrayTask.get_data`.
+
+The ArrowAsyncProducer structure
+''''''''''''''''''''''''''''''''
+
+This producer-provided and managed object has the following fields:
+
+.. c:member:: void (*ArrowAsyncProducer.request)(struct ArrowAsyncProducer*, uint64_t)
+
+ *Mandatory.* This function must be called by a consumer to start receiving calls to
+ :c:member:`ArrowAsyncDeviceStreamHandler.on_next_task`. It *MUST* be valid to call
+ this synchronously from within :c:member:`ArrowAsyncDeviceStreamHandler.on_next_task`
+ or :c:member:`ArrowAsyncDeviceStreamHandler.on_schema`. As a result, this function
+ *MUST NOT* synchronously call ``on_next_task`` or ``on_error`` to avoid recursive
+ and reentrant callbacks.
Review Comment:
Wait, the opposite is said above above about `on_next_task`:
```restructuredtext
This function *MUST* be able to be called re-entrantly on the same thread to allow for the
:c:member:`ArrowAsyncProducer.request` callback to potentially call this method recursively.
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-43631: [C][Format] Add ArrowAsyncDeviceStreamHandler interface [arrow]
Posted by "pitrou (via GitHub)" <gi...@apache.org>.
pitrou commented on code in PR #43632:
URL: https://github.com/apache/arrow/pull/43632#discussion_r1757481415
##########
docs/source/format/CDeviceDataInterface.rst:
##########
@@ -650,6 +652,331 @@ The stream source is not assumed to be thread-safe. Consumers wanting to
call ``get_next`` from several threads should ensure those calls are
serialized.
+Async Device Stream Interface
+=============================
+
+The :ref:`C stream interface <_c-device-stream-interface>` provides a synchronous
+API centered around the consumer calling the callback functions to retrieve
+the next record batch. For some bindings, use cases, and interoperability, a more
+asynchronous, producer-focused interface may be required. These scenarios can utilize
+the ``ArrowAsyncDeviceStreamHandler``.
+
+Semantics
+---------
+
+Rather than the producer providing a structure of callbacks for a consumer to
+call and retrieve records, the Async interface is a consumer allocated structure.
+The consumer allocated struct provides handler callbacks for the producer to call
+when the schema and chunks of data are available, rather than the consumer using
+a blocking pull-style iteration.
+
+In addition to the ``ArrowAsyncDeviceStreamHandler``, there are also two additional
+structs used for the full data flow: ``ArrowAsyncTask`` and ``ArrowAsyncProducer``.
+
+Structure Definition
+--------------------
+
+The C device async stream interface is defined with a single ``struct`` definition:
+
+.. code-block:: c
+
+ #ifndef ARROW_C_ASYNC_STREAM_INTERFACE
+ #define ARROW_C_ASYNC_STREAM_INTERFACE
+
+ struct ArrowAsyncTask {
+ int (*get_data)(struct ArrowArrayTask* self, struct ArrowDeviceArray* out);
+
+ void* private_data;
+ };
+
+ struct ArrowAsyncProducer {
+ void (*request)(struct ArrowAsyncProducer* self, uint64_t n);
+ void (*cancel)(struct ArrowAsyncProducer* self);
+
+ void (*release)(struct ArrowAsyncProducer* self);
+ void* private_data;
+ };
+
+ struct ArrowAsyncDeviceStreamHandler {
+ // handlers
+ int (*on_schema)(struct ArrowAsyncDeviceStreamHandler* self,
+ struct ArrowAsyncProducer* producer,
+ struct ArrowSchema* stream_schema, const char* addl_metadata);
+ int (*on_next_task)(struct ArrowAsyncDeviceStreamHandler* self,
+ struct ArrowAsyncTask* task, const char* metadata);
+ void (*on_error)(struct ArrowAsyncDeviceStreamHandler* self,
+ int code, const char* message, const char* metadata);
+
+ // release callback
+ void (*release)(struct ArrowAsyncDeviceStreamHandler* self);
+
+ // opaque handler-specific data
+ void* private_data;
+ };
+
+ #endif // ARROW_C_ASYNC_STREAM_INTERFACE
+
+.. note::
+ The canonical guard ``ARROW_C_ASYNC_STREAM_INTERFACE`` is meant to avoid
+ duplicate definitions if two projects copy the C async stream interface
+ definitions into their own headers, and a third-party project includes
+ from these two projects. It is therefore important that this guard is kept
+ exactly as-is when these definitions are copied.
+
+The ArrowAsyncDeviceStreamHandler structure
+'''''''''''''''''''''''''''''''''''''''''''
+
+The structure has the following fields:
+
+.. c:member:: int (*ArrowAsyncDeviceStreamHandler.on_schema)(struct ArrowAsyncDeviceStreamHandler*, struct ArrowAsyncProducer*, struct ArrowSchema*, const char*)
+
+ *Mandatory.* Handler for receiving the schema of the stream. All records should
+ match the provided schema. If successful, the function should return 0, otherwise
+ it should return an ``errno``-compatible error code.
+
+ The ``const char*`` parameter exists for producers to provide any extra contextual information
+ they want, such as the total number of rows in the stream, statistics, or otherwise. This is
+ encoded in the same format as :c:member:`ArrowSchema.metadata`. If not ``NULL``,
+ the lifetime is only the scope of the call to this function. A consumer who wants to maintain
+ the additional metadata beyond the lifetime of this call *MUST* copy the value themselves.
+
+ Unless the ``on_error`` handler is called, this will always get called exactly once and will be
+ the first method called on this object. As such the producer *MUST* provide an ``ArrowAsyncProducer``
+ object when calling this function to allow the consumer to manage back-pressure and flow control.
+ The producer maintains ownership of the ``ArrowAsyncProducer`` and must clean it up before or after
+ calling the release callback on this object.
+
+ A producer that receives a non-zero result here must not subsequently call anything other than
+ the release callback on this object.
+
+.. c:member:: int (*ArrowAsyncDeviceStreamHandler.on_next_task)(struct ArrowAsyncDeviceStreamHandler*, struct ArrowAsyncTask*, const char*)
+
+ *Mandatory.* Handler to be called when a new record is available for processing. The
+ schema for each record should be the same as the schema that ``on_schema`` was called with.
+ If successfully handled, the function should return 0, otherwise it should return an
+ ``errno``-compatible error code.
+
+ Rather than passing the record itself it receives an ``ArrowAsyncTask`` instead to facilitate
+ better consumer-focused thread control as far as receiving the data. A call to this function
+ simply indicates that data is available via the provided task.
+
+ The ``const char*`` parameter exists for producers to provide any extra contextual information
+ they want. This is encoded in the same format as :c:member:`ArrowSchema.metadata`. If not ``NULL``,
+ the lifetime is only the scope of the call to this function. A consumer who wants to maintain
+ the additional metadata beyond the lifetime of this call *MUST* copy the value themselves.
+
+ A producer *MUST NOT* call this concurrently from multiple different threads.
+
+ The :c:member:`ArrowAsyncProducer.request` callback must be called to start receiving calls to this
+ handler.
+
+ This function *MUST* be able to be called re-entrantly on the same thread to allow for the
+ :c:member:`ArrowAsyncProducer.request` callback to potentially call this method recursively.
+
+.. c:member:: void (*ArrowAsyncDeviceStreamHandler.on_error)(struct ArrowAsyncDeviceStreamHandler, int, const char*, const char*)
+
+ *Mandatory.* Handler to be called when an error is encountered by the producer. After calling
+ this, the ``release`` callback will be called as the last call on this struct. The parameters
+ are an ``errno``-compatible error code and an optional error message and metadata.
+
+ If the message and metadata are not ``NULL``, their lifetime is only valid during the scope
+ of this call. A consumer who wants to maintain these values past the return of this function
+ *MUST* copy the values themselves.
+
+ If the metadata parameter is not ``NULL``, to provide key-value error metadata, then it should
+ be encoded identically to the way that metadata is encoded in :c:member:`ArrowSchema.metadata`.
+
+ It is valid for this to be called by a producer with or without a preceding call to
+ :c:member:`ArrowAsyncProducer.request`. This callback *MUST NOT* call any methods of an
+ ``ArrowAsyncProducer`` object.
+
+.. c:member:: void (*ArrowAsyncDeviceStreamHandler.release)(struct ArrowAsyncDeviceStreamHandler*)
+
+ *Mandatory.* A pointer to a consumer-provided release callback for the handler.
+
+ It is valid for this to be called by a producer with or without a preceding call to
+ :c:member:`ArrowAsyncProducer.request`. This must not call any methods of an ``ArrowAsyncProducer``
+ object.
+
+.. c:member:: void* ArrowAsyncDeviceStreamHandler.private_data
+
+ *Optional.* An opaque pointer to consumer-provided private data.
+
+ Producers *MUST NOT* process this member. Lifetime of this member is handled by
+ the consumer, and especially by the release callback.
+
+The ArrowAsyncTask structure
+''''''''''''''''''''''''''''
+
+The purpose of using a Task object rather than passing the array directly to the ``on_next``
+callback is to allow for more complex and efficient thread handling. Utilizing a Task
+object allows for a producer to separate the "decoding" logic from the I/O, enabling a
+consumer to avoid transferring data between CPU cores (e.g. from one L1/L2 cache to another).
+
+This producer-provided structure has the following fields:
+
+.. c:member:: int (*ArrowArrayTask.get_data)(struct ArrowArrayTask*, struct ArrowDeviceArray*)
+
+ *Mandatory.* A callback to populate the provided ``ArrowDeviceArray`` with the available data.
+ The order of ArrowAsyncTasks provided by the producer enables a consumer to know the order of
+ the data to process.
+
+ If a non-zero value is returned from this, it should be followed only by the producer calling
+ the ``on_error`` callback of the ``ArrowAsyncDeviceStreamHandler``. Because calling this method
+ is likely to be separate from the current control flow, returning a non-zero value to signal
+ an error occuring allows the current thread to decide handle the case accordingly, while still
+ allowing all error logging and handling to be centralized in the
+ :c:member:`ArrowAsyncDeviceStreamHandler.on_error` callback.
+
+ Rather than having a separate release callback, any required cleanup should be performed as part
+ of the invocation of this callback, with ownership of the Array being passed to the consumer-provided
+ parameter calling this, and must be released separately.
+
+ It is only valid to call this method exactly once.
+
+.. c:member:: void* ArrowArrayTask.private_data
+
+ *Optional.* An opaque pointer to consumer-provided private data.
+
+ Consumers *MUST NOT* process this member. Lifetime of this member is handled by
+ the producer who created this object, and should be cleaned up if necessary during
+ the call to :c:member:`ArrowArrayTask.get_data`.
+
+The ArrowAsyncProducer structure
+''''''''''''''''''''''''''''''''
+
+This producer-provided and managed object has the following fields:
+
+.. c:member:: void (*ArrowAsyncProducer.request)(struct ArrowAsyncProducer*, uint64_t)
Review Comment:
Please let's make it `int64_t`, not `uint64_t`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-43631: [C][Format] Add ArrowAsyncDeviceStreamHandler interface [arrow]
Posted by "pitrou (via GitHub)" <gi...@apache.org>.
pitrou commented on code in PR #43632:
URL: https://github.com/apache/arrow/pull/43632#discussion_r1765348390
##########
docs/source/format/CDeviceDataInterface.rst:
##########
@@ -650,6 +652,331 @@ The stream source is not assumed to be thread-safe. Consumers wanting to
call ``get_next`` from several threads should ensure those calls are
serialized.
+Async Device Stream Interface
+=============================
+
+The :ref:`C stream interface <_c-device-stream-interface>` provides a synchronous
+API centered around the consumer calling the callback functions to retrieve
+the next record batch. For some bindings, use cases, and interoperability, a more
+asynchronous, producer-focused interface may be required. These scenarios can utilize
+the ``ArrowAsyncDeviceStreamHandler``.
+
+Semantics
+---------
+
+Rather than the producer providing a structure of callbacks for a consumer to
+call and retrieve records, the Async interface is a consumer allocated structure.
+The consumer allocated struct provides handler callbacks for the producer to call
+when the schema and chunks of data are available, rather than the consumer using
+a blocking pull-style iteration.
+
+In addition to the ``ArrowAsyncDeviceStreamHandler``, there are also two additional
+structs used for the full data flow: ``ArrowAsyncTask`` and ``ArrowAsyncProducer``.
+
+Structure Definition
+--------------------
+
+The C device async stream interface is defined with a single ``struct`` definition:
+
+.. code-block:: c
+
+ #ifndef ARROW_C_ASYNC_STREAM_INTERFACE
+ #define ARROW_C_ASYNC_STREAM_INTERFACE
+
+ struct ArrowAsyncTask {
+ int (*get_data)(struct ArrowArrayTask* self, struct ArrowDeviceArray* out);
+
+ void* private_data;
+ };
+
+ struct ArrowAsyncProducer {
+ void (*request)(struct ArrowAsyncProducer* self, uint64_t n);
+ void (*cancel)(struct ArrowAsyncProducer* self);
+
+ void (*release)(struct ArrowAsyncProducer* self);
+ void* private_data;
+ };
+
+ struct ArrowAsyncDeviceStreamHandler {
+ // handlers
+ int (*on_schema)(struct ArrowAsyncDeviceStreamHandler* self,
+ struct ArrowAsyncProducer* producer,
+ struct ArrowSchema* stream_schema, const char* addl_metadata);
+ int (*on_next_task)(struct ArrowAsyncDeviceStreamHandler* self,
+ struct ArrowAsyncTask* task, const char* metadata);
+ void (*on_error)(struct ArrowAsyncDeviceStreamHandler* self,
+ int code, const char* message, const char* metadata);
+
+ // release callback
+ void (*release)(struct ArrowAsyncDeviceStreamHandler* self);
+
+ // opaque handler-specific data
+ void* private_data;
+ };
+
+ #endif // ARROW_C_ASYNC_STREAM_INTERFACE
+
+.. note::
+ The canonical guard ``ARROW_C_ASYNC_STREAM_INTERFACE`` is meant to avoid
+ duplicate definitions if two projects copy the C async stream interface
+ definitions into their own headers, and a third-party project includes
+ from these two projects. It is therefore important that this guard is kept
+ exactly as-is when these definitions are copied.
+
+The ArrowAsyncDeviceStreamHandler structure
+'''''''''''''''''''''''''''''''''''''''''''
+
+The structure has the following fields:
+
+.. c:member:: int (*ArrowAsyncDeviceStreamHandler.on_schema)(struct ArrowAsyncDeviceStreamHandler*, struct ArrowAsyncProducer*, struct ArrowSchema*, const char*)
+
+ *Mandatory.* Handler for receiving the schema of the stream. All records should
+ match the provided schema. If successful, the function should return 0, otherwise
+ it should return an ``errno``-compatible error code.
+
+ The ``const char*`` parameter exists for producers to provide any extra contextual information
+ they want, such as the total number of rows in the stream, statistics, or otherwise. This is
+ encoded in the same format as :c:member:`ArrowSchema.metadata`. If not ``NULL``,
+ the lifetime is only the scope of the call to this function. A consumer who wants to maintain
+ the additional metadata beyond the lifetime of this call *MUST* copy the value themselves.
+
+ Unless the ``on_error`` handler is called, this will always get called exactly once and will be
+ the first method called on this object. As such the producer *MUST* provide an ``ArrowAsyncProducer``
+ object when calling this function to allow the consumer to manage back-pressure and flow control.
+ The producer maintains ownership of the ``ArrowAsyncProducer`` and must clean it up before or after
+ calling the release callback on this object.
+
+ A producer that receives a non-zero result here must not subsequently call anything other than
+ the release callback on this object.
+
+.. c:member:: int (*ArrowAsyncDeviceStreamHandler.on_next_task)(struct ArrowAsyncDeviceStreamHandler*, struct ArrowAsyncTask*, const char*)
+
+ *Mandatory.* Handler to be called when a new record is available for processing. The
+ schema for each record should be the same as the schema that ``on_schema`` was called with.
+ If successfully handled, the function should return 0, otherwise it should return an
+ ``errno``-compatible error code.
+
+ Rather than passing the record itself it receives an ``ArrowAsyncTask`` instead to facilitate
+ better consumer-focused thread control as far as receiving the data. A call to this function
+ simply indicates that data is available via the provided task.
+
+ The ``const char*`` parameter exists for producers to provide any extra contextual information
+ they want. This is encoded in the same format as :c:member:`ArrowSchema.metadata`. If not ``NULL``,
+ the lifetime is only the scope of the call to this function. A consumer who wants to maintain
+ the additional metadata beyond the lifetime of this call *MUST* copy the value themselves.
+
+ A producer *MUST NOT* call this concurrently from multiple different threads.
+
+ The :c:member:`ArrowAsyncProducer.request` callback must be called to start receiving calls to this
+ handler.
+
+ This function *MUST* be able to be called re-entrantly on the same thread to allow for the
+ :c:member:`ArrowAsyncProducer.request` callback to potentially call this method recursively.
+
+.. c:member:: void (*ArrowAsyncDeviceStreamHandler.on_error)(struct ArrowAsyncDeviceStreamHandler, int, const char*, const char*)
+
+ *Mandatory.* Handler to be called when an error is encountered by the producer. After calling
+ this, the ``release`` callback will be called as the last call on this struct. The parameters
+ are an ``errno``-compatible error code and an optional error message and metadata.
+
+ If the message and metadata are not ``NULL``, their lifetime is only valid during the scope
+ of this call. A consumer who wants to maintain these values past the return of this function
+ *MUST* copy the values themselves.
+
+ If the metadata parameter is not ``NULL``, to provide key-value error metadata, then it should
+ be encoded identically to the way that metadata is encoded in :c:member:`ArrowSchema.metadata`.
+
+ It is valid for this to be called by a producer with or without a preceding call to
+ :c:member:`ArrowAsyncProducer.request`. This callback *MUST NOT* call any methods of an
+ ``ArrowAsyncProducer`` object.
+
+.. c:member:: void (*ArrowAsyncDeviceStreamHandler.release)(struct ArrowAsyncDeviceStreamHandler*)
+
+ *Mandatory.* A pointer to a consumer-provided release callback for the handler.
+
+ It is valid for this to be called by a producer with or without a preceding call to
+ :c:member:`ArrowAsyncProducer.request`. This must not call any methods of an ``ArrowAsyncProducer``
+ object.
+
+.. c:member:: void* ArrowAsyncDeviceStreamHandler.private_data
+
+ *Optional.* An opaque pointer to consumer-provided private data.
+
+ Producers *MUST NOT* process this member. Lifetime of this member is handled by
+ the consumer, and especially by the release callback.
+
+The ArrowAsyncTask structure
+''''''''''''''''''''''''''''
+
+The purpose of using a Task object rather than passing the array directly to the ``on_next``
+callback is to allow for more complex and efficient thread handling. Utilizing a Task
+object allows for a producer to separate the "decoding" logic from the I/O, enabling a
+consumer to avoid transferring data between CPU cores (e.g. from one L1/L2 cache to another).
Review Comment:
The producer implementation for `get_data` may itself be multithreaded, in which case calling `get_data` from multiple threads may actually be detrimental.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-43631: [C][Format] Add ArrowAsyncDeviceStreamHandler interface [arrow]
Posted by "lidavidm (via GitHub)" <gi...@apache.org>.
lidavidm commented on code in PR #43632:
URL: https://github.com/apache/arrow/pull/43632#discussion_r1804183282
##########
docs/source/format/CDeviceDataInterface.rst:
##########
@@ -650,6 +652,340 @@ The stream source is not assumed to be thread-safe. Consumers wanting to
call ``get_next`` from several threads should ensure those calls are
serialized.
+Async Device Stream Interface
+=============================
+
+The :ref:`C stream interface <_c-device-stream-interface>` provides a synchronous
+API centered around the consumer calling the producer functions to retrieve
+the next record batch. For concurrent communication between producer and consumer,
+the ``ArrowAsyncDeviceStreamHandler`` can be used. This interface is non-opinionated
+and may fit into different concurrent communication models.
+
+Semantics
+---------
+
+Rather than the producer providing a structure of callbacks for a consumer to
+call and retrieve records, the Async interface is a structure allocated and populated by the consumer.
+The consumer allocated struct provides handler callbacks for the producer to call
+when the schema and chunks of data are available.
+
+In addition to the ``ArrowAsyncDeviceStreamHandler``, there are also two additional
+structs used for the full data flow: ``ArrowAsyncTask`` and ``ArrowAsyncProducer``.
+
+Structure Definition
+--------------------
+
+The C device async stream interface consists of three ``struct`` definitions:
+
+.. code-block:: c
+
+ #ifndef ARROW_C_ASYNC_STREAM_INTERFACE
+ #define ARROW_C_ASYNC_STREAM_INTERFACE
+
+ struct ArrowAsyncTask {
+ int (*extract_data)(struct ArrowArrayTask* self, struct ArrowDeviceArray* out);
+
+ void* private_data;
+ };
+
+ struct ArrowAsyncProducer {
+ void (*request)(struct ArrowAsyncProducer* self, int64_t n);
+ void (*cancel)(struct ArrowAsyncProducer* self);
+
+ void (*release)(struct ArrowAsyncProducer* self);
+ void* private_data;
+ };
+
+ struct ArrowAsyncDeviceStreamHandler {
+ // consumer-specific handlers
+ int (*on_schema)(struct ArrowAsyncDeviceStreamHandler* self,
+ struct ArrowAsyncProducer* producer,
+ struct ArrowSchema* stream_schema, const char* addl_metadata);
+ int (*on_next_task)(struct ArrowAsyncDeviceStreamHandler* self,
+ struct ArrowAsyncTask* task, const char* metadata);
+ void (*on_error)(struct ArrowAsyncDeviceStreamHandler* self,
+ int code, const char* message, const char* metadata);
+
+ // release callback
+ void (*release)(struct ArrowAsyncDeviceStreamHandler* self);
+
+ // opaque handler-specific data
+ void* private_data;
+ };
+
+ #endif // ARROW_C_ASYNC_STREAM_INTERFACE
+
+.. note::
+ The canonical guard ``ARROW_C_ASYNC_STREAM_INTERFACE`` is meant to avoid
+ duplicate definitions if two projects copy the C async stream interface
+ definitions into their own headers, and a third-party project includes
+ from these two projects. It is therefore important that this guard is kept
+ exactly as-is when these definitions are copied.
+
+The ArrowAsyncDeviceStreamHandler structure
+'''''''''''''''''''''''''''''''''''''''''''
+
+The structure has the following fields:
+
+.. c:member:: int (*ArrowAsyncDeviceStreamHandler.on_schema)(struct ArrowAsyncDeviceStreamHandler*, struct ArrowAsyncProducer*, struct ArrowSchema*, const char*)
+
+ *Mandatory.* Handler for receiving the schema of the stream. All incoming records should
+ match the provided schema. If successful, the function should return 0, otherwise
+ it should return an ``errno``-compatible error code.
+
+ The ``const char*`` parameter exists for producers to provide any extra contextual information
+ they want, such as the total number of rows in the stream, statistics, or otherwise. This is
+ encoded in the same format as :c:member:`ArrowSchema.metadata`. If not ``NULL``,
+ the lifetime is only the scope of the call to this function. A consumer who wants to maintain
+ the additional metadata beyond the lifetime of this call *MUST* copy the value themselves.
+
+ Unless the ``on_error`` handler is called, this will always get called exactly once and will be
+ the first method called on this object. As such the producer *MUST* provide an ``ArrowAsyncProducer``
+ object when calling this function to allow the consumer to apply back-pressure and control the flow of data.
+ The producer maintains ownership of the ``ArrowAsyncProducer`` and must clean it up *after*
+ calling the release callback on the ``ArrowAsyncDeviceStreamHandler``.
+
+ A producer that receives a non-zero result here must not subsequently call anything other than
+ the release callback on this object.
+
+.. c:member:: int (*ArrowAsyncDeviceStreamHandler.on_next_task)(struct ArrowAsyncDeviceStreamHandler*, struct ArrowAsyncTask*, const char*)
+
+ *Mandatory.* Handler to be called when a new record is available for processing. The
+ schema for each record should be the same as the schema that ``on_schema`` was called with.
+ If successfully handled, the function should return 0, otherwise it should return an
+ ``errno``-compatible error code.
+
+ Rather than passing the record itself it receives an ``ArrowAsyncTask`` instead to facilitate
+ better consumer-focused thread control as far as receiving the data. A call to this function
+ simply indicates that data is available via the provided task.
+
+ The producer signals the end of the stream by passing ``NULL`` for the ``ArrowAsyncTask``
+ pointer instead of a valid address. This task object is only valid during the lifetime of
+ this function call. If the consumer wants to use the task beyond the scope of this method, it
+ must copy or move its contents to a new ArrowAsyncTask object.
+
+ The ``const char*`` parameter exists for producers to provide any extra contextual information
+ they want. This is encoded in the same format as :c:member:`ArrowSchema.metadata`. If not ``NULL``,
+ the lifetime is only the scope of the call to this function. A consumer who wants to maintain
+ the additional metadata beyond the lifetime of this call *MUST* copy the value themselves.
+
+ A producer *MUST NOT* call this concurrently from multiple threads.
+
+ The :c:member:`ArrowAsyncProducer.request` callback must be called to start receiving calls to this
+ handler.
+
+.. c:member:: void (*ArrowAsyncDeviceStreamHandler.on_error)(struct ArrowAsyncDeviceStreamHandler, int, const char*, const char*)
+
+ *Mandatory.* Handler to be called when an error is encountered by the producer. After calling
+ this, the ``release`` callback will be called as the last call on this struct. The parameters
+ are an ``errno``-compatible error code and an optional error message and metadata.
+
+ If the message and metadata are not ``NULL``, their lifetime is only valid during the scope
+ of this call. A consumer who wants to maintain these values past the return of this function
+ *MUST* copy the values themselves.
+
+ If the metadata parameter is not ``NULL``, to provide key-value error metadata, then it should
+ be encoded identically to the way that metadata is encoded in :c:member:`ArrowSchema.metadata`.
+
+ It is valid for this to be called by a producer with or without a preceding call to
+ :c:member:`ArrowAsyncProducer.request`. This callback *MUST NOT* call any methods of an
+ ``ArrowAsyncProducer`` object.
+
+.. c:member:: void (*ArrowAsyncDeviceStreamHandler.release)(struct ArrowAsyncDeviceStreamHandler*)
+
+ *Mandatory.* A pointer to a consumer-provided release callback for the handler.
+
+ It is valid for this to be called by a producer with or without a preceding call to
+ :c:member:`ArrowAsyncProducer.request`. This must not call any methods of an ``ArrowAsyncProducer``
+ object.
+
+.. c:member:: void* ArrowAsyncDeviceStreamHandler.private_data
+
+ *Optional.* An opaque pointer to consumer-provided private data.
+
+ Producers *MUST NOT* process this member. Lifetime of this member is handled by
+ the consumer, and especially by the release callback.
+
+The ArrowAsyncTask structure
+''''''''''''''''''''''''''''
+
+The purpose of using a Task object rather than passing the array directly to the ``on_next``
+callback is to allow for more complex and efficient thread handling. Utilizing a Task
+object allows for a producer to separate the "decoding" logic from the I/O, enabling a
+consumer to avoid transferring data between CPU cores (e.g. from one L1/L2 cache to another).
+
+This producer-provided structure has the following fields:
+
+.. c:member:: int (*ArrowArrayTask.extract_data)(struct ArrowArrayTask*, struct ArrowDeviceArray*)
+
+ *Mandatory.* A callback to populate the provided ``ArrowDeviceArray`` with the available data.
+ The order of ArrowAsyncTasks provided by the producer enables a consumer to know the order of
+ the data to process. If the consumer does not care about the data that is owned by this task,
+ it must still call ``extract_data`` so that the producer can perform any required cleanup. ``NULL``
+ should be passed as the device array pointer to indicate that the consumer doesn't want the
+ actual data, letting the task perform necessary cleanup.
+
+ If a non-zero value is returned from this, it should be followed only by the producer calling
+ the ``on_error`` callback of the ``ArrowAsyncDeviceStreamHandler``. Because calling this method
+ is likely to be separate from the current control flow, returning a non-zero value to signal
+ an error occuring allows the current thread to decide handle the case accordingly, while still
+ allowing all error logging and handling to be centralized in the
+ :c:member:`ArrowAsyncDeviceStreamHandler.on_error` callback.
+
+ Rather than having a separate release callback, any required cleanup should be performed as part
+ of the invocation of this callback. Ownership of the Array is given to the pointer passed in as
+ a parameter, and this array must be released separately.
+
+ It is only valid to call this method exactly once.
+
+.. c:member:: void* ArrowArrayTask.private_data
+
+ *Optional.* An opaque pointer to producer-provided private data.
+
+ Consumers *MUST NOT* process this member. Lifetime of this member is handled by
+ the producer who created this object, and should be cleaned up if necessary during
+ the call to :c:member:`ArrowArrayTask.extract_data`.
+
+The ArrowAsyncProducer structure
+''''''''''''''''''''''''''''''''
+
+This producer-provided and managed object has the following fields:
+
+.. c:member:: void (*ArrowAsyncProducer.request)(struct ArrowAsyncProducer*, uint64_t)
+
+ *Mandatory.* This function must be called by a consumer to start receiving calls to
+ :c:member:`ArrowAsyncDeviceStreamHandler.on_next_task`. It *MUST* be valid to call
+ this synchronously from within :c:member:`ArrowAsyncDeviceStreamHandler.on_next_task`
+ or :c:member:`ArrowAsyncDeviceStreamHandler.on_schema`. As a result, this function
+ *MUST NOT* synchronously call ``on_next_task`` or ``on_error`` to avoid recursive
+ and reentrant callbacks.
+
+ After ``cancel`` is called, additional calls to this function must be a NOP, but allowed.
+
+ While not cancelled, calling this function registers the given number of additional
+ arrays/batches to be produced by the producer. A producer should only call
+ the appropriate ``on_next_task`` callback up to a maximum of the total sum of calls to
+ this method before propagating back-pressure / waiting.
+
+ Any error encountered by calling request must be propagated by calling the ``on_error``
+ callback of the ``ArrowAsyncDeviceStreamHandler``.
+
+ It is invalid to call this function with a value of ``n`` that is ``<= 0``. Producers should
+ error (e.g. call ``on_error``) if receiving such a value for ``n``.
+
+.. c:member:: void (*ArrowAsyncProducer.cancel)(struct ArrowAsyncProducer*)
+
+ *Mandatory.* This function signals to the producer that it must *eventually* stop calling
+ ``on_next_task``. Calls to ``cancel`` must be idempotent and thread-safe. After calling
+ it once, subsequent calls *MUST* be a NOP. This *MUST NOT* call any consumer-side handlers
+ other than ``on_error``.
+
+ It is not required that calling ``cancel`` affect the producer *immediately*, only that it
+ must eventually stop calling ``on_next_task`` and then subsequently call ``release``
+ on the async handler object. As such, a consumer *MUST* be prepared to receive one or more
+ calls to ``on_next_task`` or ``on_error`` even after calling ``cancel`` if there are still
+ requested arrays pending.
+
+ Successful cancelling *MUST NOT* result in a producer calling
+ :c:member:`ArrowAsyncDeviceStreamHandler.on_error`, instead it should finish out any remaining
+ tasks (calling ``on_next_task`` accordingly) and eventually just call ``release``.
+
+ Any error encountered during handling a call to cancel must be reported via the ``on_error``
+ callback on the async stream handler.
+
+.. c:member:: void* ArrowAsyncProducer.private_data
+
+ *Optional.* An opaque pointer to producer-provided specific data.
+
+ Consumers *MUST NOT* process this member, the lifetime is owned by the producer
+ that constructed this object.
+
+Error Handling
+''''''''''''''
+
+Unlike the regular C Stream interface, the Async interface allows for errors to flow in
+both directions. As a result, error handling can be slightly more complex. Thus this spec
+designates the following rules:
+
+* If the producer encounters an error during processing, it should call the ``on_error``
+callback, and then call ``release`` after it returns.
+
+* If ``on_schema`` or ``on_next_task`` returns a non-zero integer value, the producer *should not*
+call the ``on_error`` callback, but instead should eventually call ``release`` at some point
+before or after any logging or processing of the error code.
+
+Result lifetimes
+''''''''''''''''
+
+The ``ArrowSchema`` passed to the ``on_schema`` callback must be released independantly,
Review Comment:
```suggestion
The ``ArrowSchema`` passed to the ``on_schema`` callback must be released independently,
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-43631: [C][Format] Add ArrowAsyncDeviceStreamHandler interface [arrow]
Posted by "felipecrv (via GitHub)" <gi...@apache.org>.
felipecrv commented on code in PR #43632:
URL: https://github.com/apache/arrow/pull/43632#discussion_r1767114387
##########
docs/source/format/CDeviceDataInterface.rst:
##########
@@ -650,6 +652,340 @@ The stream source is not assumed to be thread-safe. Consumers wanting to
call ``get_next`` from several threads should ensure those calls are
serialized.
+Async Device Stream Interface
+=============================
+
+The :ref:`C stream interface <_c-device-stream-interface>` provides a synchronous
+API centered around the consumer calling the callback functions to retrieve
Review Comment:
Are these really "call-back" functions? I know the spec and code calls them callbacks, but "producer functions" will be less confusing here.
```suggestion
API centered around the consumer calling the producer functions to retrieve
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-43631: [C][Format] Add ArrowAsyncDeviceStreamHandler interface [arrow]
Posted by "pitrou (via GitHub)" <gi...@apache.org>.
pitrou commented on code in PR #43632:
URL: https://github.com/apache/arrow/pull/43632#discussion_r1757471380
##########
docs/source/format/CDeviceDataInterface.rst:
##########
@@ -650,6 +652,331 @@ The stream source is not assumed to be thread-safe. Consumers wanting to
call ``get_next`` from several threads should ensure those calls are
serialized.
+Async Device Stream Interface
+=============================
+
+The :ref:`C stream interface <_c-device-stream-interface>` provides a synchronous
+API centered around the consumer calling the callback functions to retrieve
+the next record batch. For some bindings, use cases, and interoperability, a more
+asynchronous, producer-focused interface may be required. These scenarios can utilize
+the ``ArrowAsyncDeviceStreamHandler``.
+
+Semantics
+---------
+
+Rather than the producer providing a structure of callbacks for a consumer to
+call and retrieve records, the Async interface is a consumer allocated structure.
+The consumer allocated struct provides handler callbacks for the producer to call
+when the schema and chunks of data are available, rather than the consumer using
+a blocking pull-style iteration.
+
+In addition to the ``ArrowAsyncDeviceStreamHandler``, there are also two additional
+structs used for the full data flow: ``ArrowAsyncTask`` and ``ArrowAsyncProducer``.
+
+Structure Definition
+--------------------
+
+The C device async stream interface is defined with a single ``struct`` definition:
+
+.. code-block:: c
+
+ #ifndef ARROW_C_ASYNC_STREAM_INTERFACE
+ #define ARROW_C_ASYNC_STREAM_INTERFACE
+
+ struct ArrowAsyncTask {
+ int (*get_data)(struct ArrowArrayTask* self, struct ArrowDeviceArray* out);
+
+ void* private_data;
+ };
+
+ struct ArrowAsyncProducer {
+ void (*request)(struct ArrowAsyncProducer* self, uint64_t n);
+ void (*cancel)(struct ArrowAsyncProducer* self);
+
+ void (*release)(struct ArrowAsyncProducer* self);
+ void* private_data;
+ };
+
+ struct ArrowAsyncDeviceStreamHandler {
+ // handlers
+ int (*on_schema)(struct ArrowAsyncDeviceStreamHandler* self,
+ struct ArrowAsyncProducer* producer,
+ struct ArrowSchema* stream_schema, const char* addl_metadata);
+ int (*on_next_task)(struct ArrowAsyncDeviceStreamHandler* self,
+ struct ArrowAsyncTask* task, const char* metadata);
+ void (*on_error)(struct ArrowAsyncDeviceStreamHandler* self,
+ int code, const char* message, const char* metadata);
+
+ // release callback
+ void (*release)(struct ArrowAsyncDeviceStreamHandler* self);
+
+ // opaque handler-specific data
+ void* private_data;
+ };
+
+ #endif // ARROW_C_ASYNC_STREAM_INTERFACE
+
+.. note::
+ The canonical guard ``ARROW_C_ASYNC_STREAM_INTERFACE`` is meant to avoid
+ duplicate definitions if two projects copy the C async stream interface
+ definitions into their own headers, and a third-party project includes
+ from these two projects. It is therefore important that this guard is kept
+ exactly as-is when these definitions are copied.
+
+The ArrowAsyncDeviceStreamHandler structure
+'''''''''''''''''''''''''''''''''''''''''''
+
+The structure has the following fields:
+
+.. c:member:: int (*ArrowAsyncDeviceStreamHandler.on_schema)(struct ArrowAsyncDeviceStreamHandler*, struct ArrowAsyncProducer*, struct ArrowSchema*, const char*)
+
+ *Mandatory.* Handler for receiving the schema of the stream. All records should
+ match the provided schema. If successful, the function should return 0, otherwise
+ it should return an ``errno``-compatible error code.
+
+ The ``const char*`` parameter exists for producers to provide any extra contextual information
+ they want, such as the total number of rows in the stream, statistics, or otherwise. This is
+ encoded in the same format as :c:member:`ArrowSchema.metadata`. If not ``NULL``,
+ the lifetime is only the scope of the call to this function. A consumer who wants to maintain
+ the additional metadata beyond the lifetime of this call *MUST* copy the value themselves.
+
+ Unless the ``on_error`` handler is called, this will always get called exactly once and will be
+ the first method called on this object. As such the producer *MUST* provide an ``ArrowAsyncProducer``
+ object when calling this function to allow the consumer to manage back-pressure and flow control.
+ The producer maintains ownership of the ``ArrowAsyncProducer`` and must clean it up before or after
+ calling the release callback on this object.
+
+ A producer that receives a non-zero result here must not subsequently call anything other than
+ the release callback on this object.
+
+.. c:member:: int (*ArrowAsyncDeviceStreamHandler.on_next_task)(struct ArrowAsyncDeviceStreamHandler*, struct ArrowAsyncTask*, const char*)
+
+ *Mandatory.* Handler to be called when a new record is available for processing. The
+ schema for each record should be the same as the schema that ``on_schema`` was called with.
+ If successfully handled, the function should return 0, otherwise it should return an
+ ``errno``-compatible error code.
+
+ Rather than passing the record itself it receives an ``ArrowAsyncTask`` instead to facilitate
+ better consumer-focused thread control as far as receiving the data. A call to this function
+ simply indicates that data is available via the provided task.
+
+ The ``const char*`` parameter exists for producers to provide any extra contextual information
+ they want. This is encoded in the same format as :c:member:`ArrowSchema.metadata`. If not ``NULL``,
+ the lifetime is only the scope of the call to this function. A consumer who wants to maintain
+ the additional metadata beyond the lifetime of this call *MUST* copy the value themselves.
+
+ A producer *MUST NOT* call this concurrently from multiple different threads.
Review Comment:
Presumably it cannot be called concurrently from multiple identical threads
```suggestion
A producer *MUST NOT* call this concurrently from multiple threads.
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-43631: [C][Format] Add ArrowAsyncDeviceStreamHandler interface [arrow]
Posted by "zeroshade (via GitHub)" <gi...@apache.org>.
zeroshade commented on code in PR #43632:
URL: https://github.com/apache/arrow/pull/43632#discussion_r1759033830
##########
docs/source/format/CDeviceDataInterface.rst:
##########
@@ -650,6 +652,331 @@ The stream source is not assumed to be thread-safe. Consumers wanting to
call ``get_next`` from several threads should ensure those calls are
serialized.
+Async Device Stream Interface
+=============================
+
+The :ref:`C stream interface <_c-device-stream-interface>` provides a synchronous
+API centered around the consumer calling the callback functions to retrieve
+the next record batch. For some bindings, use cases, and interoperability, a more
+asynchronous, producer-focused interface may be required. These scenarios can utilize
+the ``ArrowAsyncDeviceStreamHandler``.
+
+Semantics
+---------
+
+Rather than the producer providing a structure of callbacks for a consumer to
+call and retrieve records, the Async interface is a consumer allocated structure.
+The consumer allocated struct provides handler callbacks for the producer to call
+when the schema and chunks of data are available, rather than the consumer using
+a blocking pull-style iteration.
+
+In addition to the ``ArrowAsyncDeviceStreamHandler``, there are also two additional
+structs used for the full data flow: ``ArrowAsyncTask`` and ``ArrowAsyncProducer``.
+
+Structure Definition
+--------------------
+
+The C device async stream interface is defined with a single ``struct`` definition:
+
+.. code-block:: c
+
+ #ifndef ARROW_C_ASYNC_STREAM_INTERFACE
+ #define ARROW_C_ASYNC_STREAM_INTERFACE
+
+ struct ArrowAsyncTask {
+ int (*get_data)(struct ArrowArrayTask* self, struct ArrowDeviceArray* out);
+
+ void* private_data;
+ };
+
+ struct ArrowAsyncProducer {
+ void (*request)(struct ArrowAsyncProducer* self, uint64_t n);
+ void (*cancel)(struct ArrowAsyncProducer* self);
+
+ void (*release)(struct ArrowAsyncProducer* self);
+ void* private_data;
+ };
+
+ struct ArrowAsyncDeviceStreamHandler {
+ // handlers
+ int (*on_schema)(struct ArrowAsyncDeviceStreamHandler* self,
+ struct ArrowAsyncProducer* producer,
+ struct ArrowSchema* stream_schema, const char* addl_metadata);
+ int (*on_next_task)(struct ArrowAsyncDeviceStreamHandler* self,
+ struct ArrowAsyncTask* task, const char* metadata);
+ void (*on_error)(struct ArrowAsyncDeviceStreamHandler* self,
+ int code, const char* message, const char* metadata);
+
+ // release callback
+ void (*release)(struct ArrowAsyncDeviceStreamHandler* self);
+
+ // opaque handler-specific data
+ void* private_data;
+ };
+
+ #endif // ARROW_C_ASYNC_STREAM_INTERFACE
+
+.. note::
+ The canonical guard ``ARROW_C_ASYNC_STREAM_INTERFACE`` is meant to avoid
+ duplicate definitions if two projects copy the C async stream interface
+ definitions into their own headers, and a third-party project includes
+ from these two projects. It is therefore important that this guard is kept
+ exactly as-is when these definitions are copied.
+
+The ArrowAsyncDeviceStreamHandler structure
+'''''''''''''''''''''''''''''''''''''''''''
+
+The structure has the following fields:
+
+.. c:member:: int (*ArrowAsyncDeviceStreamHandler.on_schema)(struct ArrowAsyncDeviceStreamHandler*, struct ArrowAsyncProducer*, struct ArrowSchema*, const char*)
+
+ *Mandatory.* Handler for receiving the schema of the stream. All records should
+ match the provided schema. If successful, the function should return 0, otherwise
+ it should return an ``errno``-compatible error code.
+
+ The ``const char*`` parameter exists for producers to provide any extra contextual information
+ they want, such as the total number of rows in the stream, statistics, or otherwise. This is
+ encoded in the same format as :c:member:`ArrowSchema.metadata`. If not ``NULL``,
+ the lifetime is only the scope of the call to this function. A consumer who wants to maintain
+ the additional metadata beyond the lifetime of this call *MUST* copy the value themselves.
+
+ Unless the ``on_error`` handler is called, this will always get called exactly once and will be
+ the first method called on this object. As such the producer *MUST* provide an ``ArrowAsyncProducer``
+ object when calling this function to allow the consumer to manage back-pressure and flow control.
+ The producer maintains ownership of the ``ArrowAsyncProducer`` and must clean it up before or after
+ calling the release callback on this object.
+
+ A producer that receives a non-zero result here must not subsequently call anything other than
+ the release callback on this object.
+
+.. c:member:: int (*ArrowAsyncDeviceStreamHandler.on_next_task)(struct ArrowAsyncDeviceStreamHandler*, struct ArrowAsyncTask*, const char*)
+
+ *Mandatory.* Handler to be called when a new record is available for processing. The
+ schema for each record should be the same as the schema that ``on_schema`` was called with.
+ If successfully handled, the function should return 0, otherwise it should return an
+ ``errno``-compatible error code.
+
+ Rather than passing the record itself it receives an ``ArrowAsyncTask`` instead to facilitate
+ better consumer-focused thread control as far as receiving the data. A call to this function
+ simply indicates that data is available via the provided task.
+
+ The ``const char*`` parameter exists for producers to provide any extra contextual information
+ they want. This is encoded in the same format as :c:member:`ArrowSchema.metadata`. If not ``NULL``,
+ the lifetime is only the scope of the call to this function. A consumer who wants to maintain
+ the additional metadata beyond the lifetime of this call *MUST* copy the value themselves.
+
+ A producer *MUST NOT* call this concurrently from multiple different threads.
+
+ The :c:member:`ArrowAsyncProducer.request` callback must be called to start receiving calls to this
+ handler.
+
+ This function *MUST* be able to be called re-entrantly on the same thread to allow for the
+ :c:member:`ArrowAsyncProducer.request` callback to potentially call this method recursively.
+
+.. c:member:: void (*ArrowAsyncDeviceStreamHandler.on_error)(struct ArrowAsyncDeviceStreamHandler, int, const char*, const char*)
+
+ *Mandatory.* Handler to be called when an error is encountered by the producer. After calling
+ this, the ``release`` callback will be called as the last call on this struct. The parameters
+ are an ``errno``-compatible error code and an optional error message and metadata.
+
+ If the message and metadata are not ``NULL``, their lifetime is only valid during the scope
+ of this call. A consumer who wants to maintain these values past the return of this function
+ *MUST* copy the values themselves.
+
+ If the metadata parameter is not ``NULL``, to provide key-value error metadata, then it should
+ be encoded identically to the way that metadata is encoded in :c:member:`ArrowSchema.metadata`.
+
+ It is valid for this to be called by a producer with or without a preceding call to
+ :c:member:`ArrowAsyncProducer.request`. This callback *MUST NOT* call any methods of an
+ ``ArrowAsyncProducer`` object.
+
+.. c:member:: void (*ArrowAsyncDeviceStreamHandler.release)(struct ArrowAsyncDeviceStreamHandler*)
+
+ *Mandatory.* A pointer to a consumer-provided release callback for the handler.
+
+ It is valid for this to be called by a producer with or without a preceding call to
+ :c:member:`ArrowAsyncProducer.request`. This must not call any methods of an ``ArrowAsyncProducer``
+ object.
+
+.. c:member:: void* ArrowAsyncDeviceStreamHandler.private_data
+
+ *Optional.* An opaque pointer to consumer-provided private data.
+
+ Producers *MUST NOT* process this member. Lifetime of this member is handled by
+ the consumer, and especially by the release callback.
+
+The ArrowAsyncTask structure
+''''''''''''''''''''''''''''
+
+The purpose of using a Task object rather than passing the array directly to the ``on_next``
+callback is to allow for more complex and efficient thread handling. Utilizing a Task
+object allows for a producer to separate the "decoding" logic from the I/O, enabling a
+consumer to avoid transferring data between CPU cores (e.g. from one L1/L2 cache to another).
+
+This producer-provided structure has the following fields:
+
+.. c:member:: int (*ArrowArrayTask.get_data)(struct ArrowArrayTask*, struct ArrowDeviceArray*)
+
+ *Mandatory.* A callback to populate the provided ``ArrowDeviceArray`` with the available data.
+ The order of ArrowAsyncTasks provided by the producer enables a consumer to know the order of
+ the data to process.
+
+ If a non-zero value is returned from this, it should be followed only by the producer calling
+ the ``on_error`` callback of the ``ArrowAsyncDeviceStreamHandler``. Because calling this method
+ is likely to be separate from the current control flow, returning a non-zero value to signal
+ an error occuring allows the current thread to decide handle the case accordingly, while still
+ allowing all error logging and handling to be centralized in the
+ :c:member:`ArrowAsyncDeviceStreamHandler.on_error` callback.
+
+ Rather than having a separate release callback, any required cleanup should be performed as part
+ of the invocation of this callback, with ownership of the Array being passed to the consumer-provided
+ parameter calling this, and must be released separately.
+
+ It is only valid to call this method exactly once.
+
+.. c:member:: void* ArrowArrayTask.private_data
+
+ *Optional.* An opaque pointer to consumer-provided private data.
+
+ Consumers *MUST NOT* process this member. Lifetime of this member is handled by
+ the producer who created this object, and should be cleaned up if necessary during
+ the call to :c:member:`ArrowArrayTask.get_data`.
+
+The ArrowAsyncProducer structure
+''''''''''''''''''''''''''''''''
+
+This producer-provided and managed object has the following fields:
+
+.. c:member:: void (*ArrowAsyncProducer.request)(struct ArrowAsyncProducer*, uint64_t)
+
+ *Mandatory.* This function must be called by a consumer to start receiving calls to
+ :c:member:`ArrowAsyncDeviceStreamHandler.on_next_task`. It *MUST* be valid to call
+ this synchronously from within :c:member:`ArrowAsyncDeviceStreamHandler.on_next_task`
+ or :c:member:`ArrowAsyncDeviceStreamHandler.on_schema`. As a result, this function
+ *MUST NOT* synchronously call ``on_next_task`` or ``on_error`` to avoid recursive
+ and reentrant callbacks.
+
+ After ``cancel`` is called, additional calls to this function must be a NOP, but allowed.
+
+ While not cancelled, calling this function registers the given number of additional
+ arrays/batches to be produced by the producer. A producer should only call
+ the appropriate ``on_next_task`` callback up to a maximum of the total sum of calls to
+ this method before propagating back-pressure / waiting.
Review Comment:
because of the disconnect between the consumer, the producer and whatever the *source* of the data is.
For instance, if the data is coming from a stream on the producer's side then just because the consumer requests N arrays, doesn't mean the producer won't receive >N arrays from the data source. Thus, back-pressure potentially needs to be handled by the producer to push back on the data source. This may be as simple as the producer *only* retrieving N arrays from its source, or it might be more complex. I didn't want to proscribe a sleep or any other particular way of handling the back-pressure.
Does that make more sense?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-43631: [C][Format] Add ArrowAsyncDeviceStreamHandler interface [arrow]
Posted by "zeroshade (via GitHub)" <gi...@apache.org>.
zeroshade commented on code in PR #43632:
URL: https://github.com/apache/arrow/pull/43632#discussion_r1722291200
##########
cpp/src/arrow/c/abi.h:
##########
@@ -228,6 +228,65 @@ struct ArrowDeviceArrayStream {
#endif // ARROW_C_DEVICE_STREAM_INTERFACE
+#ifndef ARROW_C_ASYNC_STREAM_INTERFACE
+#define ARROW_C_ASYNC_STREAM_INTERFACE
+
+// Similar to ArrowDeviceArrayStream, except designed for an asynchronous
+// style of interaction. While ArrowDeviceArrayStream provides producer
+// defined callbacks, this is intended to be created by the consumer instead.
+// The consumer passes this handler to the producer, which in turn uses the
+// callbacks to inform the consumer of events in the stream.
+struct ArrowAsyncDeviceStreamHandler {
+ // Handler for receiving a schema. The passed in stream_schema should be
+ // released or moved by the handler (producer is giving ownership of it to
+ // the handler).
+ //
+ // The `extension_param` argument can be null or can be used by a producer
+ // to pass arbitrary extra information to the consumer (such as total number
+ // of rows, context info, or otherwise).
+ //
+ // Return value: 0 if successful, `errno`-compatible error otherwise
+ int (*on_schema)(struct ArrowAsyncDeviceStreamHandler* self,
+ struct ArrowSchema* stream_schema, void* extension_param);
+
+ // Handler for receiving an array/record batch. Always called at least once
+ // unless an error is encountered (which would result in calling on_error).
+ // An empty/released array is passed to indicate the end of the stream if no
+ // errors have been encountered.
+ //
+ // The `extension_param` argument can be null or can be used by a producer
+ // to pass arbitrary extra information to the consumer.
+ //
+ // Return value: 0 if successful, `errno`-compatible error otherwise.
+ int (*on_next)(struct ArrowAsyncDeviceStreamHandler* self,
+ struct ArrowDeviceArray* next, void* extension_param);
Review Comment:
I agree with @leventov that out-of-order delivery is outside the scope of this.
@bkietz The documentation explicitly states that the callbacks must be called serially, but can be called from different threads.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-43631: [C][Format] Add ArrowAsyncDeviceStreamHandler interface [arrow]
Posted by "lidavidm (via GitHub)" <gi...@apache.org>.
lidavidm commented on PR #43632:
URL: https://github.com/apache/arrow/pull/43632#issuecomment-2305909247
> > There's the partial compromise option:
>
> Something like `int on_next_task(ArrowAsyncDeviceStreamHandler*, ArrowArrayTask*)` ?
Yes, though I wonder if it's really worth it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-43631: [C][Format] Add ArrowAsyncDeviceStreamHandler interface [arrow]
Posted by "leventov (via GitHub)" <gi...@apache.org>.
leventov commented on PR #43632:
URL: https://github.com/apache/arrow/pull/43632#issuecomment-2311887589
I'd suggest that to help resolve the callbacks vs. tasks dilemma, someone could take several representative examples of processing engines that are going to be users of this API, such as DataFusion, Velox, ClickHouse, Doris, or the like, and see whether callbacks or tasks fit better into their threading/task execution model.
### ClickHouse
For example, here's the ClickHouse's model: https://github.com/ClickHouse/ClickHouse/blob/6289c65e0286127689303b5b7a543212ca38e0c7/src/Processors/IProcessor.h
-- Clearly, it would work better with a task-based Arrow Async handler (it would be a "source" IProcessor, as they call IProcessors with a single "input port").
> My big concern is the case of an IPC stream where we don't know beforehand how many record batches are in the stream. If get_next_task is called multiple times in a row, historically we've used a return of 0 + null for the Array to indicate the end of the stream. So the only reasonable way to handle get_next_task being called multiple times in a row while there are tasks pending is to return a valid task for each call until the producer knows that the stream has ended and then we define semantics for the task struct to indicate that the stream has ended successfully.
@zeroshade FWIW please compare with how IProcessor handles this: `IProcessor.prepare()` should return `Status.Finished`.
IProcessor also has explicit `.cancel()` callback.
### Other systems
Could someone please help with research for other systems?
I don't suggest copying any specific system's approach verbatim, but probably taking the approach that is most compatible with most systems' approaches is best.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-43631: [C][Format] Add ArrowAsyncDeviceStreamHandler interface [arrow]
Posted by "zeroshade (via GitHub)" <gi...@apache.org>.
zeroshade commented on PR #43632:
URL: https://github.com/apache/arrow/pull/43632#issuecomment-2284467244
@westonpace your last example confuses me a bit, particularly because the entire purpose of this is to create a *push-based* approach for async handling, rather than a pull-based approach.
Below are my questions:
```c
struct Waker {
// Signal to the producer that more data is available, the consumer shall release any resources
// associated with the waker after this method is called. The producer shall not call any other
// methods after calling this method.
//
// The producer must always call this method even if an error is encountered (in which case the
// error will be reported on the following call to `get_next`).
void wake(Waker* waker);
void* private_data;
};
```
I don't understand signalling to the producer that data is available, the producer knows when data is available right? it's producing the data in the first place. Isn't the point that the producer needs to signal the *consumer* that more data is available? `Waker` would need to be consumer created, with the producer calling `wake` when data is available. Was your comment just a typo, or am I missing something?
```c
struct ArrowAsyncDeviceArrayStream {
// Callback to get the next array task
// (if no error and the task is released, the stream has ended)
//
// Return value: 0 if successful, an `errno`-compatible error code otherwise.
//
// The consumer is allowed to call this method again even if the tasks returned by a previous
// call have not completed. However, the consumer shall not call this re-entrantly.
//
// If successful, the ArrowDeviceArray must be released independently from the stream.
int (*get_next_task)(struct ArrowAsyncDeviceArrayStream* self, struct ArrowArrayTask* task);
// The rest is identical to `ArrowDeviceArrayStream`
ArrowDeviceType device_type;
const char* (*get_last_error)(struct ArrowAsyncDeviceArrayStream* self);
void (*release)(struct ArrowAsyncDeviceArrayStream* self);
void* private_data;
};
```
Why the abstraction of a task in this case? Couldn't we remove the task abstraction and just put the `Waker` here? It feels like the `get_next_task` abstraction here is just a way to re-invert the direction of the calls. Producer calls `get_next_task` which turns around and calls `task->get_next`? At that point, it feels like we could bypass the idea of needing `ArrowAsyncDeviceArrayStream` to be constructed by the consumer and instead build the entire API around the waker?
Also in the case of this being able to handle concurrency, the `get_last_error` function becomes mostly meaningless, right? Depending on when it is called, you end up with a race condition as far as what the error might be.
Finally, If a consumer can call `get_next_task` before the previous one has completed, what should happen in the following scenario:
* There is only one record in the stream, it is delayed
* Consumer calls `get_next_task` 5 times in a loop to get the next 5 tasks
Do all 5 calls from the consumer wait until the record comes in? Does each one provide its own task object which then turns around and waits until the record comes in to call wake? The producer then needs to manage the order the calls came in, along with the order that the records come in for the calls to `get_next`. This seems like a lot of added complexity without much benefit.
All in all, i'm not completely sold on the idea of this re-inversion of the paradigm. It honestly feels sorta hacky, but that might just be me.
@paleolimbot @lidavidm thoughts on @westonpace's suggestion here?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-43631: [C][Format] Add ArrowAsyncDeviceStreamHandler interface [arrow]
Posted by "westonpace (via GitHub)" <gi...@apache.org>.
westonpace commented on PR #43632:
URL: https://github.com/apache/arrow/pull/43632#issuecomment-2284669419
> @westonpace your last example confuses me a bit, particularly because the entire purpose of this is to create a push-based approach for async handling, rather than a pull-based approach.
I read the purpose as "create an async interface". There are both push-based and pull-based asynchronous interfaces. Is there a reason you specifically want a pull-based interface?
> I don't understand signalling to the producer that data is available, the producer knows when data is available right? it's producing the data in the first place. Isn't the point that the producer needs to signal the consumer that more data is available? Waker would need to be consumer created, with the producer calling wake when data is available. Was your comment just a typo, or am I missing something?
Typo. The waker is for the producer to signal the consumer that data is available.
> Also in the case of this being able to handle concurrency, the get_last_error function becomes mostly meaningless, right? Depending on when it is called, you end up with a race condition as far as what the error might be.
Good point. The error handling would have to move to the task.
> All in all, i'm not completely sold on the idea of this re-inversion of the paradigm. It honestly feels sorta hacky, but that might just be me.
I'm basically modeling this on the current lance file reader. The file reader knows (from the metadata) how many tasks there will be. `get_next_task` is a synchronous method that creates a task which is an asynchronous task. Therefore:
>
> There is only one record in the stream, it is delayed
> Consumer calls get_next_task 5 times in a loop to get the next 5 tasks
This would never happen. The first call to `get_next_task` returns the first task. The second call returns null because it knows all tasks are done. However, if you didn't know ahead of time how many tasks there are then tasks 2-5 would just all return NULL.
> The original idea and semantics here were that the ArrowAsyncDeviceArrayStream is constructed by the consumer and given to the producer, given the comments you've put here it seems like this is the other way around now? The producer still provides the struct, and the consumer calls get_next providing the waker. Do I have that right? What happens if get_next is called again before wake is called on the waker? Are we defining that get_next must not be called until wake is called? The semantics are a bit confusing to me here as far as what you're proposing.
Yeah, sorry for the typo, but you got it. The producer creates the `ArrowAsyncDeviceArrayStream`. The consumer creates the `Waker` and passes it into the call to `get_next`.
> What happens if get_next is called again before wake is called on the waker?
There are ways this could work but I don't see any advantage in allowing it so it'd be easier to say that must not happen.
> Are we defining that get_next must not be called until wake is called?
Sure, if this simplifies things there is no harm in defining that.
> The semantics are a bit confusing to me here as far as what you're proposing.
This is based on Rust's asynchronous model which I've found to work well. The consumer in this case is running some kind of event-loop based coroutine asynchronous logic.
```
while True:
for task in ready_tasks:
task.run()
```
When calling an asynchronous method the logic looks something like this (taking many liberties here)
```
cur_task = get_current_task()
waker = lambda: tasks.make_task_active(cur_task.id)
res = call_async_method(..., waker=waker)
if res == EWOULDBLOCK:
return yield_current_task() # makes the task inactive
```
## Resetting the conversation
My goals here are currently based on the approach I'm taking with the lance file reader where I go pretty far out of my way to avoid data being transferred from one core to another (e.g. from one L1/L2 cache to another). It may be that the complexity of achieving these goals is not worth the benefit. Currently the approach I'm using is the task based approach described above:
* Consumer calls `get_next_task` (synchronous, single-threaded)
* Producer figures out what I/O is needed to fulfill the task
* Producer schedules the I/O
* Producer creates a task to decode the data and returns it
* Consumer checks if task is ready, if not, it adds it to the inactive task list.
* I/O finishes on producer. Producer marks task ready.
* Once consumer has a free thread it assigns it the ready task. Consumer runs task, decodes data (the decode logic is producer logic), and then processes the decoded data (consumer logic) all without a CPU core transfer (data stays in L2 cache).
So my main goal here is to have some way so that the handoff from producer to consumer can happen without forcing a CPU core transfer.
> As described, the intent was that a consumer doesn't need to handle concurrency by default because a producer should only call the on_next when the previous one returns. But in the scenario you describe, where the processing inside of on_next might be lengthy, I think it's fine for the consumer to have to make the decision as to whether or not it should create a new thread task (if it wants to handle things in parallel) or do it inside of on_next (and potentially tie up the producer, but that's basically just backpressure essentially).
This doesn't meet my goal because, by this point, the producer has already done the decoding. If the consumer decides to make a new thread task it is incurring the cost of a CPU core transfer.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-43631: [C][Format] Add ArrowAsyncDeviceStreamHandler interface [arrow]
Posted by "pitrou (via GitHub)" <gi...@apache.org>.
pitrou commented on code in PR #43632:
URL: https://github.com/apache/arrow/pull/43632#discussion_r1757490158
##########
docs/source/format/CDeviceDataInterface.rst:
##########
@@ -650,6 +652,331 @@ The stream source is not assumed to be thread-safe. Consumers wanting to
call ``get_next`` from several threads should ensure those calls are
serialized.
+Async Device Stream Interface
+=============================
+
+The :ref:`C stream interface <_c-device-stream-interface>` provides a synchronous
+API centered around the consumer calling the callback functions to retrieve
+the next record batch. For some bindings, use cases, and interoperability, a more
+asynchronous, producer-focused interface may be required. These scenarios can utilize
+the ``ArrowAsyncDeviceStreamHandler``.
+
+Semantics
+---------
+
+Rather than the producer providing a structure of callbacks for a consumer to
+call and retrieve records, the Async interface is a consumer allocated structure.
+The consumer allocated struct provides handler callbacks for the producer to call
+when the schema and chunks of data are available, rather than the consumer using
+a blocking pull-style iteration.
+
+In addition to the ``ArrowAsyncDeviceStreamHandler``, there are also two additional
+structs used for the full data flow: ``ArrowAsyncTask`` and ``ArrowAsyncProducer``.
+
+Structure Definition
+--------------------
+
+The C device async stream interface is defined with a single ``struct`` definition:
+
+.. code-block:: c
+
+ #ifndef ARROW_C_ASYNC_STREAM_INTERFACE
+ #define ARROW_C_ASYNC_STREAM_INTERFACE
+
+ struct ArrowAsyncTask {
+ int (*get_data)(struct ArrowArrayTask* self, struct ArrowDeviceArray* out);
+
+ void* private_data;
+ };
+
+ struct ArrowAsyncProducer {
+ void (*request)(struct ArrowAsyncProducer* self, uint64_t n);
+ void (*cancel)(struct ArrowAsyncProducer* self);
+
+ void (*release)(struct ArrowAsyncProducer* self);
+ void* private_data;
+ };
+
+ struct ArrowAsyncDeviceStreamHandler {
+ // handlers
+ int (*on_schema)(struct ArrowAsyncDeviceStreamHandler* self,
+ struct ArrowAsyncProducer* producer,
+ struct ArrowSchema* stream_schema, const char* addl_metadata);
+ int (*on_next_task)(struct ArrowAsyncDeviceStreamHandler* self,
+ struct ArrowAsyncTask* task, const char* metadata);
+ void (*on_error)(struct ArrowAsyncDeviceStreamHandler* self,
+ int code, const char* message, const char* metadata);
+
+ // release callback
+ void (*release)(struct ArrowAsyncDeviceStreamHandler* self);
+
+ // opaque handler-specific data
+ void* private_data;
+ };
+
+ #endif // ARROW_C_ASYNC_STREAM_INTERFACE
+
+.. note::
+ The canonical guard ``ARROW_C_ASYNC_STREAM_INTERFACE`` is meant to avoid
+ duplicate definitions if two projects copy the C async stream interface
+ definitions into their own headers, and a third-party project includes
+ from these two projects. It is therefore important that this guard is kept
+ exactly as-is when these definitions are copied.
+
+The ArrowAsyncDeviceStreamHandler structure
+'''''''''''''''''''''''''''''''''''''''''''
+
+The structure has the following fields:
+
+.. c:member:: int (*ArrowAsyncDeviceStreamHandler.on_schema)(struct ArrowAsyncDeviceStreamHandler*, struct ArrowAsyncProducer*, struct ArrowSchema*, const char*)
+
+ *Mandatory.* Handler for receiving the schema of the stream. All records should
+ match the provided schema. If successful, the function should return 0, otherwise
+ it should return an ``errno``-compatible error code.
+
+ The ``const char*`` parameter exists for producers to provide any extra contextual information
+ they want, such as the total number of rows in the stream, statistics, or otherwise. This is
+ encoded in the same format as :c:member:`ArrowSchema.metadata`. If not ``NULL``,
+ the lifetime is only the scope of the call to this function. A consumer who wants to maintain
+ the additional metadata beyond the lifetime of this call *MUST* copy the value themselves.
+
+ Unless the ``on_error`` handler is called, this will always get called exactly once and will be
+ the first method called on this object. As such the producer *MUST* provide an ``ArrowAsyncProducer``
+ object when calling this function to allow the consumer to manage back-pressure and flow control.
+ The producer maintains ownership of the ``ArrowAsyncProducer`` and must clean it up before or after
+ calling the release callback on this object.
+
+ A producer that receives a non-zero result here must not subsequently call anything other than
+ the release callback on this object.
+
+.. c:member:: int (*ArrowAsyncDeviceStreamHandler.on_next_task)(struct ArrowAsyncDeviceStreamHandler*, struct ArrowAsyncTask*, const char*)
+
+ *Mandatory.* Handler to be called when a new record is available for processing. The
+ schema for each record should be the same as the schema that ``on_schema`` was called with.
+ If successfully handled, the function should return 0, otherwise it should return an
+ ``errno``-compatible error code.
+
+ Rather than passing the record itself it receives an ``ArrowAsyncTask`` instead to facilitate
+ better consumer-focused thread control as far as receiving the data. A call to this function
+ simply indicates that data is available via the provided task.
+
+ The ``const char*`` parameter exists for producers to provide any extra contextual information
+ they want. This is encoded in the same format as :c:member:`ArrowSchema.metadata`. If not ``NULL``,
+ the lifetime is only the scope of the call to this function. A consumer who wants to maintain
+ the additional metadata beyond the lifetime of this call *MUST* copy the value themselves.
+
+ A producer *MUST NOT* call this concurrently from multiple different threads.
+
+ The :c:member:`ArrowAsyncProducer.request` callback must be called to start receiving calls to this
+ handler.
+
+ This function *MUST* be able to be called re-entrantly on the same thread to allow for the
+ :c:member:`ArrowAsyncProducer.request` callback to potentially call this method recursively.
+
+.. c:member:: void (*ArrowAsyncDeviceStreamHandler.on_error)(struct ArrowAsyncDeviceStreamHandler, int, const char*, const char*)
+
+ *Mandatory.* Handler to be called when an error is encountered by the producer. After calling
+ this, the ``release`` callback will be called as the last call on this struct. The parameters
+ are an ``errno``-compatible error code and an optional error message and metadata.
+
+ If the message and metadata are not ``NULL``, their lifetime is only valid during the scope
+ of this call. A consumer who wants to maintain these values past the return of this function
+ *MUST* copy the values themselves.
+
+ If the metadata parameter is not ``NULL``, to provide key-value error metadata, then it should
+ be encoded identically to the way that metadata is encoded in :c:member:`ArrowSchema.metadata`.
+
+ It is valid for this to be called by a producer with or without a preceding call to
+ :c:member:`ArrowAsyncProducer.request`. This callback *MUST NOT* call any methods of an
+ ``ArrowAsyncProducer`` object.
+
+.. c:member:: void (*ArrowAsyncDeviceStreamHandler.release)(struct ArrowAsyncDeviceStreamHandler*)
+
+ *Mandatory.* A pointer to a consumer-provided release callback for the handler.
+
+ It is valid for this to be called by a producer with or without a preceding call to
+ :c:member:`ArrowAsyncProducer.request`. This must not call any methods of an ``ArrowAsyncProducer``
+ object.
+
+.. c:member:: void* ArrowAsyncDeviceStreamHandler.private_data
+
+ *Optional.* An opaque pointer to consumer-provided private data.
+
+ Producers *MUST NOT* process this member. Lifetime of this member is handled by
+ the consumer, and especially by the release callback.
+
+The ArrowAsyncTask structure
+''''''''''''''''''''''''''''
+
+The purpose of using a Task object rather than passing the array directly to the ``on_next``
+callback is to allow for more complex and efficient thread handling. Utilizing a Task
+object allows for a producer to separate the "decoding" logic from the I/O, enabling a
+consumer to avoid transferring data between CPU cores (e.g. from one L1/L2 cache to another).
+
+This producer-provided structure has the following fields:
+
+.. c:member:: int (*ArrowArrayTask.get_data)(struct ArrowArrayTask*, struct ArrowDeviceArray*)
+
+ *Mandatory.* A callback to populate the provided ``ArrowDeviceArray`` with the available data.
+ The order of ArrowAsyncTasks provided by the producer enables a consumer to know the order of
+ the data to process.
+
+ If a non-zero value is returned from this, it should be followed only by the producer calling
+ the ``on_error`` callback of the ``ArrowAsyncDeviceStreamHandler``. Because calling this method
+ is likely to be separate from the current control flow, returning a non-zero value to signal
+ an error occuring allows the current thread to decide handle the case accordingly, while still
+ allowing all error logging and handling to be centralized in the
+ :c:member:`ArrowAsyncDeviceStreamHandler.on_error` callback.
+
+ Rather than having a separate release callback, any required cleanup should be performed as part
+ of the invocation of this callback, with ownership of the Array being passed to the consumer-provided
+ parameter calling this, and must be released separately.
+
+ It is only valid to call this method exactly once.
+
+.. c:member:: void* ArrowArrayTask.private_data
+
+ *Optional.* An opaque pointer to consumer-provided private data.
+
+ Consumers *MUST NOT* process this member. Lifetime of this member is handled by
+ the producer who created this object, and should be cleaned up if necessary during
+ the call to :c:member:`ArrowArrayTask.get_data`.
+
+The ArrowAsyncProducer structure
+''''''''''''''''''''''''''''''''
+
+This producer-provided and managed object has the following fields:
+
+.. c:member:: void (*ArrowAsyncProducer.request)(struct ArrowAsyncProducer*, uint64_t)
+
+ *Mandatory.* This function must be called by a consumer to start receiving calls to
+ :c:member:`ArrowAsyncDeviceStreamHandler.on_next_task`. It *MUST* be valid to call
+ this synchronously from within :c:member:`ArrowAsyncDeviceStreamHandler.on_next_task`
+ or :c:member:`ArrowAsyncDeviceStreamHandler.on_schema`. As a result, this function
+ *MUST NOT* synchronously call ``on_next_task`` or ``on_error`` to avoid recursive
+ and reentrant callbacks.
+
+ After ``cancel`` is called, additional calls to this function must be a NOP, but allowed.
+
+ While not cancelled, calling this function registers the given number of additional
+ arrays/batches to be produced by the producer. A producer should only call
+ the appropriate ``on_next_task`` callback up to a maximum of the total sum of calls to
+ this method before propagating back-pressure / waiting.
+
+ Any error encountered by calling request must be propagated by calling the ``on_error``
+ callback of the ``ArrowAsyncDeviceStreamHandler``.
+
+ A producer *MUST* support an unbounded number of calls to this callback and *MUST* support
+ a total registered demand (sum requested - sum delivered) of up to ``UINT64_MAX``.
+
+.. c:member:: void (*ArrowAsyncProducer.cancel)(struct ArrowAsyncProducer*)
+
+ *Mandatory.* This function signals to the producer that it must *eventually* stop calling
+ ``on_next_task``. Calls to ``cancel`` must be idempotent and thread-safe. After calling
+ it once, subsequent calls *MUST* be a NOP.
+
+ It is not required that calling ``cancel`` affect the producer *immediately*, only that it
+ must eventually stop calling ``on_next_task`` and then subsequently call ``release``
+ on the async handler object. As such, a consumer *MUST* be prepared to receive one or more
+ calls to ``on_next_task`` or ``on_error`` even after calling ``cancel`` if there are still
+ requested arrays pending.
+
+ Successful cancelling *MUST NOT* result in a producer calling
+ :c:member:`ArrowAsyncDeviceStreamHandler.on_error`, instead it should finish out any remaining
+ tasks (calling ``on_next_task`` accordingly) and eventually just call ``release``.
+
+ Any error encountered during handling a call to cancel must be reported via the ``on_error``
+ callback on the async stream handler.
Review Comment:
Perhaps add that `cancel` is not allowed to call any consumer-side handlers other than `on_error`?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-43631: [C][Format] Add ArrowAsyncDeviceStreamHandler interface [arrow]
Posted by "pitrou (via GitHub)" <gi...@apache.org>.
pitrou commented on code in PR #43632:
URL: https://github.com/apache/arrow/pull/43632#discussion_r1757479276
##########
docs/source/format/CDeviceDataInterface.rst:
##########
@@ -650,6 +652,331 @@ The stream source is not assumed to be thread-safe. Consumers wanting to
call ``get_next`` from several threads should ensure those calls are
serialized.
+Async Device Stream Interface
+=============================
+
+The :ref:`C stream interface <_c-device-stream-interface>` provides a synchronous
+API centered around the consumer calling the callback functions to retrieve
+the next record batch. For some bindings, use cases, and interoperability, a more
+asynchronous, producer-focused interface may be required. These scenarios can utilize
+the ``ArrowAsyncDeviceStreamHandler``.
+
+Semantics
+---------
+
+Rather than the producer providing a structure of callbacks for a consumer to
+call and retrieve records, the Async interface is a consumer allocated structure.
+The consumer allocated struct provides handler callbacks for the producer to call
+when the schema and chunks of data are available, rather than the consumer using
+a blocking pull-style iteration.
+
+In addition to the ``ArrowAsyncDeviceStreamHandler``, there are also two additional
+structs used for the full data flow: ``ArrowAsyncTask`` and ``ArrowAsyncProducer``.
+
+Structure Definition
+--------------------
+
+The C device async stream interface is defined with a single ``struct`` definition:
+
+.. code-block:: c
+
+ #ifndef ARROW_C_ASYNC_STREAM_INTERFACE
+ #define ARROW_C_ASYNC_STREAM_INTERFACE
+
+ struct ArrowAsyncTask {
+ int (*get_data)(struct ArrowArrayTask* self, struct ArrowDeviceArray* out);
+
+ void* private_data;
+ };
+
+ struct ArrowAsyncProducer {
+ void (*request)(struct ArrowAsyncProducer* self, uint64_t n);
+ void (*cancel)(struct ArrowAsyncProducer* self);
+
+ void (*release)(struct ArrowAsyncProducer* self);
+ void* private_data;
+ };
+
+ struct ArrowAsyncDeviceStreamHandler {
+ // handlers
+ int (*on_schema)(struct ArrowAsyncDeviceStreamHandler* self,
+ struct ArrowAsyncProducer* producer,
+ struct ArrowSchema* stream_schema, const char* addl_metadata);
+ int (*on_next_task)(struct ArrowAsyncDeviceStreamHandler* self,
+ struct ArrowAsyncTask* task, const char* metadata);
+ void (*on_error)(struct ArrowAsyncDeviceStreamHandler* self,
+ int code, const char* message, const char* metadata);
+
+ // release callback
+ void (*release)(struct ArrowAsyncDeviceStreamHandler* self);
+
+ // opaque handler-specific data
+ void* private_data;
+ };
+
+ #endif // ARROW_C_ASYNC_STREAM_INTERFACE
+
+.. note::
+ The canonical guard ``ARROW_C_ASYNC_STREAM_INTERFACE`` is meant to avoid
+ duplicate definitions if two projects copy the C async stream interface
+ definitions into their own headers, and a third-party project includes
+ from these two projects. It is therefore important that this guard is kept
+ exactly as-is when these definitions are copied.
+
+The ArrowAsyncDeviceStreamHandler structure
+'''''''''''''''''''''''''''''''''''''''''''
+
+The structure has the following fields:
+
+.. c:member:: int (*ArrowAsyncDeviceStreamHandler.on_schema)(struct ArrowAsyncDeviceStreamHandler*, struct ArrowAsyncProducer*, struct ArrowSchema*, const char*)
+
+ *Mandatory.* Handler for receiving the schema of the stream. All records should
+ match the provided schema. If successful, the function should return 0, otherwise
+ it should return an ``errno``-compatible error code.
+
+ The ``const char*`` parameter exists for producers to provide any extra contextual information
+ they want, such as the total number of rows in the stream, statistics, or otherwise. This is
+ encoded in the same format as :c:member:`ArrowSchema.metadata`. If not ``NULL``,
+ the lifetime is only the scope of the call to this function. A consumer who wants to maintain
+ the additional metadata beyond the lifetime of this call *MUST* copy the value themselves.
+
+ Unless the ``on_error`` handler is called, this will always get called exactly once and will be
+ the first method called on this object. As such the producer *MUST* provide an ``ArrowAsyncProducer``
+ object when calling this function to allow the consumer to manage back-pressure and flow control.
+ The producer maintains ownership of the ``ArrowAsyncProducer`` and must clean it up before or after
+ calling the release callback on this object.
+
+ A producer that receives a non-zero result here must not subsequently call anything other than
+ the release callback on this object.
+
+.. c:member:: int (*ArrowAsyncDeviceStreamHandler.on_next_task)(struct ArrowAsyncDeviceStreamHandler*, struct ArrowAsyncTask*, const char*)
+
+ *Mandatory.* Handler to be called when a new record is available for processing. The
+ schema for each record should be the same as the schema that ``on_schema`` was called with.
+ If successfully handled, the function should return 0, otherwise it should return an
+ ``errno``-compatible error code.
+
+ Rather than passing the record itself it receives an ``ArrowAsyncTask`` instead to facilitate
+ better consumer-focused thread control as far as receiving the data. A call to this function
+ simply indicates that data is available via the provided task.
+
+ The ``const char*`` parameter exists for producers to provide any extra contextual information
+ they want. This is encoded in the same format as :c:member:`ArrowSchema.metadata`. If not ``NULL``,
+ the lifetime is only the scope of the call to this function. A consumer who wants to maintain
+ the additional metadata beyond the lifetime of this call *MUST* copy the value themselves.
+
+ A producer *MUST NOT* call this concurrently from multiple different threads.
+
+ The :c:member:`ArrowAsyncProducer.request` callback must be called to start receiving calls to this
+ handler.
+
+ This function *MUST* be able to be called re-entrantly on the same thread to allow for the
+ :c:member:`ArrowAsyncProducer.request` callback to potentially call this method recursively.
+
+.. c:member:: void (*ArrowAsyncDeviceStreamHandler.on_error)(struct ArrowAsyncDeviceStreamHandler, int, const char*, const char*)
+
+ *Mandatory.* Handler to be called when an error is encountered by the producer. After calling
+ this, the ``release`` callback will be called as the last call on this struct. The parameters
+ are an ``errno``-compatible error code and an optional error message and metadata.
+
+ If the message and metadata are not ``NULL``, their lifetime is only valid during the scope
+ of this call. A consumer who wants to maintain these values past the return of this function
+ *MUST* copy the values themselves.
+
+ If the metadata parameter is not ``NULL``, to provide key-value error metadata, then it should
+ be encoded identically to the way that metadata is encoded in :c:member:`ArrowSchema.metadata`.
+
+ It is valid for this to be called by a producer with or without a preceding call to
+ :c:member:`ArrowAsyncProducer.request`. This callback *MUST NOT* call any methods of an
+ ``ArrowAsyncProducer`` object.
+
+.. c:member:: void (*ArrowAsyncDeviceStreamHandler.release)(struct ArrowAsyncDeviceStreamHandler*)
+
+ *Mandatory.* A pointer to a consumer-provided release callback for the handler.
+
+ It is valid for this to be called by a producer with or without a preceding call to
+ :c:member:`ArrowAsyncProducer.request`. This must not call any methods of an ``ArrowAsyncProducer``
+ object.
+
+.. c:member:: void* ArrowAsyncDeviceStreamHandler.private_data
+
+ *Optional.* An opaque pointer to consumer-provided private data.
+
+ Producers *MUST NOT* process this member. Lifetime of this member is handled by
+ the consumer, and especially by the release callback.
+
+The ArrowAsyncTask structure
+''''''''''''''''''''''''''''
+
+The purpose of using a Task object rather than passing the array directly to the ``on_next``
+callback is to allow for more complex and efficient thread handling. Utilizing a Task
+object allows for a producer to separate the "decoding" logic from the I/O, enabling a
+consumer to avoid transferring data between CPU cores (e.g. from one L1/L2 cache to another).
+
+This producer-provided structure has the following fields:
+
+.. c:member:: int (*ArrowArrayTask.get_data)(struct ArrowArrayTask*, struct ArrowDeviceArray*)
Review Comment:
This is a blocking call while the rest of this API is non-blocking? This seems unexpected.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-43631: [C][Format] Add ArrowAsyncDeviceStreamHandler interface [arrow]
Posted by "zeroshade (via GitHub)" <gi...@apache.org>.
zeroshade commented on code in PR #43632:
URL: https://github.com/apache/arrow/pull/43632#discussion_r1759118508
##########
docs/source/format/CDeviceDataInterface.rst:
##########
@@ -650,6 +652,331 @@ The stream source is not assumed to be thread-safe. Consumers wanting to
call ``get_next`` from several threads should ensure those calls are
serialized.
+Async Device Stream Interface
+=============================
+
+The :ref:`C stream interface <_c-device-stream-interface>` provides a synchronous
+API centered around the consumer calling the callback functions to retrieve
+the next record batch. For some bindings, use cases, and interoperability, a more
+asynchronous, producer-focused interface may be required. These scenarios can utilize
+the ``ArrowAsyncDeviceStreamHandler``.
+
+Semantics
+---------
+
+Rather than the producer providing a structure of callbacks for a consumer to
+call and retrieve records, the Async interface is a consumer allocated structure.
+The consumer allocated struct provides handler callbacks for the producer to call
+when the schema and chunks of data are available, rather than the consumer using
+a blocking pull-style iteration.
+
+In addition to the ``ArrowAsyncDeviceStreamHandler``, there are also two additional
+structs used for the full data flow: ``ArrowAsyncTask`` and ``ArrowAsyncProducer``.
+
+Structure Definition
+--------------------
+
+The C device async stream interface is defined with a single ``struct`` definition:
+
+.. code-block:: c
+
+ #ifndef ARROW_C_ASYNC_STREAM_INTERFACE
+ #define ARROW_C_ASYNC_STREAM_INTERFACE
+
+ struct ArrowAsyncTask {
+ int (*get_data)(struct ArrowArrayTask* self, struct ArrowDeviceArray* out);
+
+ void* private_data;
+ };
+
+ struct ArrowAsyncProducer {
+ void (*request)(struct ArrowAsyncProducer* self, uint64_t n);
+ void (*cancel)(struct ArrowAsyncProducer* self);
+
+ void (*release)(struct ArrowAsyncProducer* self);
+ void* private_data;
+ };
+
+ struct ArrowAsyncDeviceStreamHandler {
+ // handlers
+ int (*on_schema)(struct ArrowAsyncDeviceStreamHandler* self,
+ struct ArrowAsyncProducer* producer,
+ struct ArrowSchema* stream_schema, const char* addl_metadata);
+ int (*on_next_task)(struct ArrowAsyncDeviceStreamHandler* self,
+ struct ArrowAsyncTask* task, const char* metadata);
+ void (*on_error)(struct ArrowAsyncDeviceStreamHandler* self,
+ int code, const char* message, const char* metadata);
+
+ // release callback
+ void (*release)(struct ArrowAsyncDeviceStreamHandler* self);
+
+ // opaque handler-specific data
+ void* private_data;
+ };
+
+ #endif // ARROW_C_ASYNC_STREAM_INTERFACE
+
+.. note::
+ The canonical guard ``ARROW_C_ASYNC_STREAM_INTERFACE`` is meant to avoid
+ duplicate definitions if two projects copy the C async stream interface
+ definitions into their own headers, and a third-party project includes
+ from these two projects. It is therefore important that this guard is kept
+ exactly as-is when these definitions are copied.
+
+The ArrowAsyncDeviceStreamHandler structure
+'''''''''''''''''''''''''''''''''''''''''''
+
+The structure has the following fields:
+
+.. c:member:: int (*ArrowAsyncDeviceStreamHandler.on_schema)(struct ArrowAsyncDeviceStreamHandler*, struct ArrowAsyncProducer*, struct ArrowSchema*, const char*)
+
+ *Mandatory.* Handler for receiving the schema of the stream. All records should
+ match the provided schema. If successful, the function should return 0, otherwise
+ it should return an ``errno``-compatible error code.
+
+ The ``const char*`` parameter exists for producers to provide any extra contextual information
+ they want, such as the total number of rows in the stream, statistics, or otherwise. This is
+ encoded in the same format as :c:member:`ArrowSchema.metadata`. If not ``NULL``,
+ the lifetime is only the scope of the call to this function. A consumer who wants to maintain
+ the additional metadata beyond the lifetime of this call *MUST* copy the value themselves.
+
+ Unless the ``on_error`` handler is called, this will always get called exactly once and will be
+ the first method called on this object. As such the producer *MUST* provide an ``ArrowAsyncProducer``
+ object when calling this function to allow the consumer to manage back-pressure and flow control.
+ The producer maintains ownership of the ``ArrowAsyncProducer`` and must clean it up before or after
+ calling the release callback on this object.
+
+ A producer that receives a non-zero result here must not subsequently call anything other than
+ the release callback on this object.
+
+.. c:member:: int (*ArrowAsyncDeviceStreamHandler.on_next_task)(struct ArrowAsyncDeviceStreamHandler*, struct ArrowAsyncTask*, const char*)
+
+ *Mandatory.* Handler to be called when a new record is available for processing. The
+ schema for each record should be the same as the schema that ``on_schema`` was called with.
+ If successfully handled, the function should return 0, otherwise it should return an
+ ``errno``-compatible error code.
+
+ Rather than passing the record itself it receives an ``ArrowAsyncTask`` instead to facilitate
+ better consumer-focused thread control as far as receiving the data. A call to this function
+ simply indicates that data is available via the provided task.
+
+ The ``const char*`` parameter exists for producers to provide any extra contextual information
+ they want. This is encoded in the same format as :c:member:`ArrowSchema.metadata`. If not ``NULL``,
+ the lifetime is only the scope of the call to this function. A consumer who wants to maintain
+ the additional metadata beyond the lifetime of this call *MUST* copy the value themselves.
+
+ A producer *MUST NOT* call this concurrently from multiple different threads.
+
+ The :c:member:`ArrowAsyncProducer.request` callback must be called to start receiving calls to this
+ handler.
+
+ This function *MUST* be able to be called re-entrantly on the same thread to allow for the
+ :c:member:`ArrowAsyncProducer.request` callback to potentially call this method recursively.
+
+.. c:member:: void (*ArrowAsyncDeviceStreamHandler.on_error)(struct ArrowAsyncDeviceStreamHandler, int, const char*, const char*)
+
+ *Mandatory.* Handler to be called when an error is encountered by the producer. After calling
+ this, the ``release`` callback will be called as the last call on this struct. The parameters
+ are an ``errno``-compatible error code and an optional error message and metadata.
+
+ If the message and metadata are not ``NULL``, their lifetime is only valid during the scope
+ of this call. A consumer who wants to maintain these values past the return of this function
+ *MUST* copy the values themselves.
+
+ If the metadata parameter is not ``NULL``, to provide key-value error metadata, then it should
+ be encoded identically to the way that metadata is encoded in :c:member:`ArrowSchema.metadata`.
+
+ It is valid for this to be called by a producer with or without a preceding call to
+ :c:member:`ArrowAsyncProducer.request`. This callback *MUST NOT* call any methods of an
+ ``ArrowAsyncProducer`` object.
+
+.. c:member:: void (*ArrowAsyncDeviceStreamHandler.release)(struct ArrowAsyncDeviceStreamHandler*)
+
+ *Mandatory.* A pointer to a consumer-provided release callback for the handler.
+
+ It is valid for this to be called by a producer with or without a preceding call to
+ :c:member:`ArrowAsyncProducer.request`. This must not call any methods of an ``ArrowAsyncProducer``
+ object.
+
+.. c:member:: void* ArrowAsyncDeviceStreamHandler.private_data
+
+ *Optional.* An opaque pointer to consumer-provided private data.
+
+ Producers *MUST NOT* process this member. Lifetime of this member is handled by
+ the consumer, and especially by the release callback.
+
+The ArrowAsyncTask structure
+''''''''''''''''''''''''''''
+
+The purpose of using a Task object rather than passing the array directly to the ``on_next``
+callback is to allow for more complex and efficient thread handling. Utilizing a Task
+object allows for a producer to separate the "decoding" logic from the I/O, enabling a
+consumer to avoid transferring data between CPU cores (e.g. from one L1/L2 cache to another).
+
+This producer-provided structure has the following fields:
+
+.. c:member:: int (*ArrowArrayTask.get_data)(struct ArrowArrayTask*, struct ArrowDeviceArray*)
+
+ *Mandatory.* A callback to populate the provided ``ArrowDeviceArray`` with the available data.
+ The order of ArrowAsyncTasks provided by the producer enables a consumer to know the order of
+ the data to process.
+
+ If a non-zero value is returned from this, it should be followed only by the producer calling
+ the ``on_error`` callback of the ``ArrowAsyncDeviceStreamHandler``. Because calling this method
+ is likely to be separate from the current control flow, returning a non-zero value to signal
+ an error occuring allows the current thread to decide handle the case accordingly, while still
+ allowing all error logging and handling to be centralized in the
+ :c:member:`ArrowAsyncDeviceStreamHandler.on_error` callback.
+
+ Rather than having a separate release callback, any required cleanup should be performed as part
+ of the invocation of this callback, with ownership of the Array being passed to the consumer-provided
+ parameter calling this, and must be released separately.
Review Comment:
Clarified and split
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-43631: [C][Format] Add ArrowAsyncDeviceStreamHandler interface [arrow]
Posted by "zeroshade (via GitHub)" <gi...@apache.org>.
zeroshade commented on code in PR #43632:
URL: https://github.com/apache/arrow/pull/43632#discussion_r1765340011
##########
docs/source/format/CDeviceDataInterface.rst:
##########
@@ -650,6 +652,331 @@ The stream source is not assumed to be thread-safe. Consumers wanting to
call ``get_next`` from several threads should ensure those calls are
serialized.
+Async Device Stream Interface
+=============================
+
+The :ref:`C stream interface <_c-device-stream-interface>` provides a synchronous
+API centered around the consumer calling the callback functions to retrieve
+the next record batch. For some bindings, use cases, and interoperability, a more
+asynchronous, producer-focused interface may be required. These scenarios can utilize
+the ``ArrowAsyncDeviceStreamHandler``.
+
+Semantics
+---------
+
+Rather than the producer providing a structure of callbacks for a consumer to
+call and retrieve records, the Async interface is a consumer allocated structure.
+The consumer allocated struct provides handler callbacks for the producer to call
+when the schema and chunks of data are available, rather than the consumer using
+a blocking pull-style iteration.
+
+In addition to the ``ArrowAsyncDeviceStreamHandler``, there are also two additional
+structs used for the full data flow: ``ArrowAsyncTask`` and ``ArrowAsyncProducer``.
+
+Structure Definition
+--------------------
+
+The C device async stream interface is defined with a single ``struct`` definition:
+
+.. code-block:: c
+
+ #ifndef ARROW_C_ASYNC_STREAM_INTERFACE
+ #define ARROW_C_ASYNC_STREAM_INTERFACE
+
+ struct ArrowAsyncTask {
+ int (*get_data)(struct ArrowArrayTask* self, struct ArrowDeviceArray* out);
+
+ void* private_data;
+ };
+
+ struct ArrowAsyncProducer {
+ void (*request)(struct ArrowAsyncProducer* self, uint64_t n);
+ void (*cancel)(struct ArrowAsyncProducer* self);
+
+ void (*release)(struct ArrowAsyncProducer* self);
+ void* private_data;
+ };
+
+ struct ArrowAsyncDeviceStreamHandler {
+ // handlers
+ int (*on_schema)(struct ArrowAsyncDeviceStreamHandler* self,
+ struct ArrowAsyncProducer* producer,
+ struct ArrowSchema* stream_schema, const char* addl_metadata);
+ int (*on_next_task)(struct ArrowAsyncDeviceStreamHandler* self,
+ struct ArrowAsyncTask* task, const char* metadata);
+ void (*on_error)(struct ArrowAsyncDeviceStreamHandler* self,
+ int code, const char* message, const char* metadata);
+
+ // release callback
+ void (*release)(struct ArrowAsyncDeviceStreamHandler* self);
+
+ // opaque handler-specific data
+ void* private_data;
+ };
+
+ #endif // ARROW_C_ASYNC_STREAM_INTERFACE
+
+.. note::
+ The canonical guard ``ARROW_C_ASYNC_STREAM_INTERFACE`` is meant to avoid
+ duplicate definitions if two projects copy the C async stream interface
+ definitions into their own headers, and a third-party project includes
+ from these two projects. It is therefore important that this guard is kept
+ exactly as-is when these definitions are copied.
+
+The ArrowAsyncDeviceStreamHandler structure
+'''''''''''''''''''''''''''''''''''''''''''
+
+The structure has the following fields:
+
+.. c:member:: int (*ArrowAsyncDeviceStreamHandler.on_schema)(struct ArrowAsyncDeviceStreamHandler*, struct ArrowAsyncProducer*, struct ArrowSchema*, const char*)
+
+ *Mandatory.* Handler for receiving the schema of the stream. All records should
+ match the provided schema. If successful, the function should return 0, otherwise
+ it should return an ``errno``-compatible error code.
+
+ The ``const char*`` parameter exists for producers to provide any extra contextual information
+ they want, such as the total number of rows in the stream, statistics, or otherwise. This is
+ encoded in the same format as :c:member:`ArrowSchema.metadata`. If not ``NULL``,
+ the lifetime is only the scope of the call to this function. A consumer who wants to maintain
+ the additional metadata beyond the lifetime of this call *MUST* copy the value themselves.
+
+ Unless the ``on_error`` handler is called, this will always get called exactly once and will be
+ the first method called on this object. As such the producer *MUST* provide an ``ArrowAsyncProducer``
+ object when calling this function to allow the consumer to manage back-pressure and flow control.
+ The producer maintains ownership of the ``ArrowAsyncProducer`` and must clean it up before or after
+ calling the release callback on this object.
+
+ A producer that receives a non-zero result here must not subsequently call anything other than
+ the release callback on this object.
+
+.. c:member:: int (*ArrowAsyncDeviceStreamHandler.on_next_task)(struct ArrowAsyncDeviceStreamHandler*, struct ArrowAsyncTask*, const char*)
+
+ *Mandatory.* Handler to be called when a new record is available for processing. The
+ schema for each record should be the same as the schema that ``on_schema`` was called with.
+ If successfully handled, the function should return 0, otherwise it should return an
+ ``errno``-compatible error code.
+
+ Rather than passing the record itself it receives an ``ArrowAsyncTask`` instead to facilitate
+ better consumer-focused thread control as far as receiving the data. A call to this function
+ simply indicates that data is available via the provided task.
+
+ The ``const char*`` parameter exists for producers to provide any extra contextual information
+ they want. This is encoded in the same format as :c:member:`ArrowSchema.metadata`. If not ``NULL``,
+ the lifetime is only the scope of the call to this function. A consumer who wants to maintain
+ the additional metadata beyond the lifetime of this call *MUST* copy the value themselves.
+
+ A producer *MUST NOT* call this concurrently from multiple different threads.
+
+ The :c:member:`ArrowAsyncProducer.request` callback must be called to start receiving calls to this
+ handler.
+
+ This function *MUST* be able to be called re-entrantly on the same thread to allow for the
+ :c:member:`ArrowAsyncProducer.request` callback to potentially call this method recursively.
+
+.. c:member:: void (*ArrowAsyncDeviceStreamHandler.on_error)(struct ArrowAsyncDeviceStreamHandler, int, const char*, const char*)
+
+ *Mandatory.* Handler to be called when an error is encountered by the producer. After calling
+ this, the ``release`` callback will be called as the last call on this struct. The parameters
+ are an ``errno``-compatible error code and an optional error message and metadata.
+
+ If the message and metadata are not ``NULL``, their lifetime is only valid during the scope
+ of this call. A consumer who wants to maintain these values past the return of this function
+ *MUST* copy the values themselves.
+
+ If the metadata parameter is not ``NULL``, to provide key-value error metadata, then it should
+ be encoded identically to the way that metadata is encoded in :c:member:`ArrowSchema.metadata`.
+
+ It is valid for this to be called by a producer with or without a preceding call to
+ :c:member:`ArrowAsyncProducer.request`. This callback *MUST NOT* call any methods of an
+ ``ArrowAsyncProducer`` object.
+
+.. c:member:: void (*ArrowAsyncDeviceStreamHandler.release)(struct ArrowAsyncDeviceStreamHandler*)
+
+ *Mandatory.* A pointer to a consumer-provided release callback for the handler.
+
+ It is valid for this to be called by a producer with or without a preceding call to
+ :c:member:`ArrowAsyncProducer.request`. This must not call any methods of an ``ArrowAsyncProducer``
+ object.
+
+.. c:member:: void* ArrowAsyncDeviceStreamHandler.private_data
+
+ *Optional.* An opaque pointer to consumer-provided private data.
+
+ Producers *MUST NOT* process this member. Lifetime of this member is handled by
+ the consumer, and especially by the release callback.
+
+The ArrowAsyncTask structure
+''''''''''''''''''''''''''''
+
+The purpose of using a Task object rather than passing the array directly to the ``on_next``
+callback is to allow for more complex and efficient thread handling. Utilizing a Task
+object allows for a producer to separate the "decoding" logic from the I/O, enabling a
+consumer to avoid transferring data between CPU cores (e.g. from one L1/L2 cache to another).
+
+This producer-provided structure has the following fields:
+
+.. c:member:: int (*ArrowArrayTask.get_data)(struct ArrowArrayTask*, struct ArrowDeviceArray*)
+
+ *Mandatory.* A callback to populate the provided ``ArrowDeviceArray`` with the available data.
+ The order of ArrowAsyncTasks provided by the producer enables a consumer to know the order of
+ the data to process.
+
+ If a non-zero value is returned from this, it should be followed only by the producer calling
+ the ``on_error`` callback of the ``ArrowAsyncDeviceStreamHandler``. Because calling this method
+ is likely to be separate from the current control flow, returning a non-zero value to signal
+ an error occuring allows the current thread to decide handle the case accordingly, while still
+ allowing all error logging and handling to be centralized in the
+ :c:member:`ArrowAsyncDeviceStreamHandler.on_error` callback.
+
+ Rather than having a separate release callback, any required cleanup should be performed as part
+ of the invocation of this callback, with ownership of the Array being passed to the consumer-provided
+ parameter calling this, and must be released separately.
+
+ It is only valid to call this method exactly once.
+
+.. c:member:: void* ArrowArrayTask.private_data
+
+ *Optional.* An opaque pointer to consumer-provided private data.
+
+ Consumers *MUST NOT* process this member. Lifetime of this member is handled by
+ the producer who created this object, and should be cleaned up if necessary during
+ the call to :c:member:`ArrowArrayTask.get_data`.
+
+The ArrowAsyncProducer structure
+''''''''''''''''''''''''''''''''
+
+This producer-provided and managed object has the following fields:
+
+.. c:member:: void (*ArrowAsyncProducer.request)(struct ArrowAsyncProducer*, uint64_t)
+
+ *Mandatory.* This function must be called by a consumer to start receiving calls to
+ :c:member:`ArrowAsyncDeviceStreamHandler.on_next_task`. It *MUST* be valid to call
+ this synchronously from within :c:member:`ArrowAsyncDeviceStreamHandler.on_next_task`
+ or :c:member:`ArrowAsyncDeviceStreamHandler.on_schema`. As a result, this function
+ *MUST NOT* synchronously call ``on_next_task`` or ``on_error`` to avoid recursive
+ and reentrant callbacks.
+
+ After ``cancel`` is called, additional calls to this function must be a NOP, but allowed.
+
+ While not cancelled, calling this function registers the given number of additional
+ arrays/batches to be produced by the producer. A producer should only call
+ the appropriate ``on_next_task`` callback up to a maximum of the total sum of calls to
+ this method before propagating back-pressure / waiting.
+
+ Any error encountered by calling request must be propagated by calling the ``on_error``
+ callback of the ``ArrowAsyncDeviceStreamHandler``.
+
+ A producer *MUST* support an unbounded number of calls to this callback and *MUST* support
+ a total registered demand (sum requested - sum delivered) of up to ``UINT64_MAX``.
Review Comment:
I'm fine with removing this requirement. At a worst case scenario it can be added back later on if needed as it wouldn't require an ABI change.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-43631: [C][Format] Add ArrowAsyncDeviceStreamHandler interface [arrow]
Posted by "felipecrv (via GitHub)" <gi...@apache.org>.
felipecrv commented on code in PR #43632:
URL: https://github.com/apache/arrow/pull/43632#discussion_r1767114387
##########
docs/source/format/CDeviceDataInterface.rst:
##########
@@ -650,6 +652,340 @@ The stream source is not assumed to be thread-safe. Consumers wanting to
call ``get_next`` from several threads should ensure those calls are
serialized.
+Async Device Stream Interface
+=============================
+
+The :ref:`C stream interface <_c-device-stream-interface>` provides a synchronous
+API centered around the consumer calling the callback functions to retrieve
Review Comment:
Are these really "call-back" functions? I know the spec and code calls them callbacks, but "producer functions" will be less confusing here.
```suggestion
API centered around the consumer calling the producer functions to retrieve
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-43631: [C][Format] Add ArrowAsyncDeviceStreamHandler interface [arrow]
Posted by "pitrou (via GitHub)" <gi...@apache.org>.
pitrou commented on code in PR #43632:
URL: https://github.com/apache/arrow/pull/43632#discussion_r1764964295
##########
docs/source/format/CDeviceDataInterface.rst:
##########
@@ -650,6 +652,331 @@ The stream source is not assumed to be thread-safe. Consumers wanting to
call ``get_next`` from several threads should ensure those calls are
serialized.
+Async Device Stream Interface
+=============================
+
+The :ref:`C stream interface <_c-device-stream-interface>` provides a synchronous
+API centered around the consumer calling the callback functions to retrieve
+the next record batch. For some bindings, use cases, and interoperability, a more
+asynchronous, producer-focused interface may be required. These scenarios can utilize
+the ``ArrowAsyncDeviceStreamHandler``.
+
+Semantics
+---------
+
+Rather than the producer providing a structure of callbacks for a consumer to
+call and retrieve records, the Async interface is a consumer allocated structure.
+The consumer allocated struct provides handler callbacks for the producer to call
+when the schema and chunks of data are available, rather than the consumer using
+a blocking pull-style iteration.
+
+In addition to the ``ArrowAsyncDeviceStreamHandler``, there are also two additional
+structs used for the full data flow: ``ArrowAsyncTask`` and ``ArrowAsyncProducer``.
+
+Structure Definition
+--------------------
+
+The C device async stream interface is defined with a single ``struct`` definition:
+
+.. code-block:: c
+
+ #ifndef ARROW_C_ASYNC_STREAM_INTERFACE
+ #define ARROW_C_ASYNC_STREAM_INTERFACE
+
+ struct ArrowAsyncTask {
+ int (*get_data)(struct ArrowArrayTask* self, struct ArrowDeviceArray* out);
+
+ void* private_data;
+ };
+
+ struct ArrowAsyncProducer {
+ void (*request)(struct ArrowAsyncProducer* self, uint64_t n);
+ void (*cancel)(struct ArrowAsyncProducer* self);
+
+ void (*release)(struct ArrowAsyncProducer* self);
+ void* private_data;
+ };
+
+ struct ArrowAsyncDeviceStreamHandler {
+ // handlers
+ int (*on_schema)(struct ArrowAsyncDeviceStreamHandler* self,
+ struct ArrowAsyncProducer* producer,
+ struct ArrowSchema* stream_schema, const char* addl_metadata);
+ int (*on_next_task)(struct ArrowAsyncDeviceStreamHandler* self,
+ struct ArrowAsyncTask* task, const char* metadata);
+ void (*on_error)(struct ArrowAsyncDeviceStreamHandler* self,
+ int code, const char* message, const char* metadata);
+
+ // release callback
+ void (*release)(struct ArrowAsyncDeviceStreamHandler* self);
+
+ // opaque handler-specific data
+ void* private_data;
+ };
+
+ #endif // ARROW_C_ASYNC_STREAM_INTERFACE
+
+.. note::
+ The canonical guard ``ARROW_C_ASYNC_STREAM_INTERFACE`` is meant to avoid
+ duplicate definitions if two projects copy the C async stream interface
+ definitions into their own headers, and a third-party project includes
+ from these two projects. It is therefore important that this guard is kept
+ exactly as-is when these definitions are copied.
+
+The ArrowAsyncDeviceStreamHandler structure
+'''''''''''''''''''''''''''''''''''''''''''
+
+The structure has the following fields:
+
+.. c:member:: int (*ArrowAsyncDeviceStreamHandler.on_schema)(struct ArrowAsyncDeviceStreamHandler*, struct ArrowAsyncProducer*, struct ArrowSchema*, const char*)
+
+ *Mandatory.* Handler for receiving the schema of the stream. All records should
+ match the provided schema. If successful, the function should return 0, otherwise
+ it should return an ``errno``-compatible error code.
+
+ The ``const char*`` parameter exists for producers to provide any extra contextual information
+ they want, such as the total number of rows in the stream, statistics, or otherwise. This is
+ encoded in the same format as :c:member:`ArrowSchema.metadata`. If not ``NULL``,
+ the lifetime is only the scope of the call to this function. A consumer who wants to maintain
+ the additional metadata beyond the lifetime of this call *MUST* copy the value themselves.
+
+ Unless the ``on_error`` handler is called, this will always get called exactly once and will be
+ the first method called on this object. As such the producer *MUST* provide an ``ArrowAsyncProducer``
+ object when calling this function to allow the consumer to manage back-pressure and flow control.
+ The producer maintains ownership of the ``ArrowAsyncProducer`` and must clean it up before or after
+ calling the release callback on this object.
+
+ A producer that receives a non-zero result here must not subsequently call anything other than
+ the release callback on this object.
+
+.. c:member:: int (*ArrowAsyncDeviceStreamHandler.on_next_task)(struct ArrowAsyncDeviceStreamHandler*, struct ArrowAsyncTask*, const char*)
+
+ *Mandatory.* Handler to be called when a new record is available for processing. The
+ schema for each record should be the same as the schema that ``on_schema`` was called with.
+ If successfully handled, the function should return 0, otherwise it should return an
+ ``errno``-compatible error code.
+
+ Rather than passing the record itself it receives an ``ArrowAsyncTask`` instead to facilitate
+ better consumer-focused thread control as far as receiving the data. A call to this function
+ simply indicates that data is available via the provided task.
+
+ The ``const char*`` parameter exists for producers to provide any extra contextual information
+ they want. This is encoded in the same format as :c:member:`ArrowSchema.metadata`. If not ``NULL``,
+ the lifetime is only the scope of the call to this function. A consumer who wants to maintain
+ the additional metadata beyond the lifetime of this call *MUST* copy the value themselves.
+
+ A producer *MUST NOT* call this concurrently from multiple different threads.
+
+ The :c:member:`ArrowAsyncProducer.request` callback must be called to start receiving calls to this
+ handler.
+
+ This function *MUST* be able to be called re-entrantly on the same thread to allow for the
+ :c:member:`ArrowAsyncProducer.request` callback to potentially call this method recursively.
+
+.. c:member:: void (*ArrowAsyncDeviceStreamHandler.on_error)(struct ArrowAsyncDeviceStreamHandler, int, const char*, const char*)
+
+ *Mandatory.* Handler to be called when an error is encountered by the producer. After calling
+ this, the ``release`` callback will be called as the last call on this struct. The parameters
+ are an ``errno``-compatible error code and an optional error message and metadata.
+
+ If the message and metadata are not ``NULL``, their lifetime is only valid during the scope
+ of this call. A consumer who wants to maintain these values past the return of this function
+ *MUST* copy the values themselves.
+
+ If the metadata parameter is not ``NULL``, to provide key-value error metadata, then it should
+ be encoded identically to the way that metadata is encoded in :c:member:`ArrowSchema.metadata`.
+
+ It is valid for this to be called by a producer with or without a preceding call to
+ :c:member:`ArrowAsyncProducer.request`. This callback *MUST NOT* call any methods of an
+ ``ArrowAsyncProducer`` object.
+
+.. c:member:: void (*ArrowAsyncDeviceStreamHandler.release)(struct ArrowAsyncDeviceStreamHandler*)
+
+ *Mandatory.* A pointer to a consumer-provided release callback for the handler.
+
+ It is valid for this to be called by a producer with or without a preceding call to
+ :c:member:`ArrowAsyncProducer.request`. This must not call any methods of an ``ArrowAsyncProducer``
+ object.
+
+.. c:member:: void* ArrowAsyncDeviceStreamHandler.private_data
+
+ *Optional.* An opaque pointer to consumer-provided private data.
+
+ Producers *MUST NOT* process this member. Lifetime of this member is handled by
+ the consumer, and especially by the release callback.
+
+The ArrowAsyncTask structure
+''''''''''''''''''''''''''''
+
+The purpose of using a Task object rather than passing the array directly to the ``on_next``
+callback is to allow for more complex and efficient thread handling. Utilizing a Task
+object allows for a producer to separate the "decoding" logic from the I/O, enabling a
+consumer to avoid transferring data between CPU cores (e.g. from one L1/L2 cache to another).
+
+This producer-provided structure has the following fields:
+
+.. c:member:: int (*ArrowArrayTask.get_data)(struct ArrowArrayTask*, struct ArrowDeviceArray*)
+
+ *Mandatory.* A callback to populate the provided ``ArrowDeviceArray`` with the available data.
+ The order of ArrowAsyncTasks provided by the producer enables a consumer to know the order of
+ the data to process.
+
+ If a non-zero value is returned from this, it should be followed only by the producer calling
+ the ``on_error`` callback of the ``ArrowAsyncDeviceStreamHandler``. Because calling this method
+ is likely to be separate from the current control flow, returning a non-zero value to signal
+ an error occuring allows the current thread to decide handle the case accordingly, while still
+ allowing all error logging and handling to be centralized in the
+ :c:member:`ArrowAsyncDeviceStreamHandler.on_error` callback.
+
+ Rather than having a separate release callback, any required cleanup should be performed as part
+ of the invocation of this callback, with ownership of the Array being passed to the consumer-provided
+ parameter calling this, and must be released separately.
+
+ It is only valid to call this method exactly once.
+
+.. c:member:: void* ArrowArrayTask.private_data
+
+ *Optional.* An opaque pointer to consumer-provided private data.
+
+ Consumers *MUST NOT* process this member. Lifetime of this member is handled by
+ the producer who created this object, and should be cleaned up if necessary during
+ the call to :c:member:`ArrowArrayTask.get_data`.
+
+The ArrowAsyncProducer structure
+''''''''''''''''''''''''''''''''
+
+This producer-provided and managed object has the following fields:
+
+.. c:member:: void (*ArrowAsyncProducer.request)(struct ArrowAsyncProducer*, uint64_t)
Review Comment:
Of course it's possible to do so (at worse you can write the required code by hand), but it's more tedious than using existing facilities around signed integers, e.g. https://docs.oracle.com/en/java/javase/22/docs/api/java.base/java/lang/foreign/ValueLayout.html#field-summary
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-43631: [C][Format] Add ArrowAsyncDeviceStreamHandler interface [arrow]
Posted by "ianmcook (via GitHub)" <gi...@apache.org>.
ianmcook commented on PR #43632:
URL: https://github.com/apache/arrow/pull/43632#issuecomment-2440085763
@github-actions crossbow submit preview-docs
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-43631: [C][Format] Add ArrowAsyncDeviceStreamHandler interface [arrow]
Posted by "conbench-apache-arrow[bot] (via GitHub)" <gi...@apache.org>.
conbench-apache-arrow[bot] commented on PR #43632:
URL: https://github.com/apache/arrow/pull/43632#issuecomment-2460386224
After merging your PR, Conbench analyzed the 3 benchmarking runs that have been run so far on merge-commit 40b2fca4742e2692a917755fd8db2939e10fa02d.
There were no benchmark performance regressions. 🎉
The [full Conbench report](https://github.com/apache/arrow/runs/32611565218) has more details. It also includes information about 30 possible false positives for unstable benchmarks that are known to sometimes produce them.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-43631: [C][Format] Add ArrowAsyncDeviceStreamHandler interface [arrow]
Posted by "ianmcook (via GitHub)" <gi...@apache.org>.
ianmcook commented on code in PR #43632:
URL: https://github.com/apache/arrow/pull/43632#discussion_r1813806954
##########
docs/source/format/CDeviceDataInterface.rst:
##########
@@ -650,6 +652,340 @@ The stream source is not assumed to be thread-safe. Consumers wanting to
call ``get_next`` from several threads should ensure those calls are
serialized.
+Async Device Stream Interface
+=============================
+
+The :ref:`C stream interface <_c-device-stream-interface>` provides a synchronous
Review Comment:
The underscore is used in the anchor but not in the ref
```suggestion
The :ref:`C stream interface <c-device-stream-interface>` provides a synchronous
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-43631: [C][Format] Add ArrowAsyncDeviceStreamHandler interface [arrow]
Posted by "zeroshade (via GitHub)" <gi...@apache.org>.
zeroshade commented on code in PR #43632:
URL: https://github.com/apache/arrow/pull/43632#discussion_r1759021550
##########
docs/source/format/CDeviceDataInterface.rst:
##########
@@ -650,6 +652,331 @@ The stream source is not assumed to be thread-safe. Consumers wanting to
call ``get_next`` from several threads should ensure those calls are
serialized.
+Async Device Stream Interface
+=============================
+
+The :ref:`C stream interface <_c-device-stream-interface>` provides a synchronous
+API centered around the consumer calling the callback functions to retrieve
+the next record batch. For some bindings, use cases, and interoperability, a more
+asynchronous, producer-focused interface may be required. These scenarios can utilize
+the ``ArrowAsyncDeviceStreamHandler``.
+
+Semantics
+---------
+
+Rather than the producer providing a structure of callbacks for a consumer to
+call and retrieve records, the Async interface is a consumer allocated structure.
+The consumer allocated struct provides handler callbacks for the producer to call
+when the schema and chunks of data are available, rather than the consumer using
+a blocking pull-style iteration.
+
+In addition to the ``ArrowAsyncDeviceStreamHandler``, there are also two additional
+structs used for the full data flow: ``ArrowAsyncTask`` and ``ArrowAsyncProducer``.
+
+Structure Definition
+--------------------
+
+The C device async stream interface is defined with a single ``struct`` definition:
+
+.. code-block:: c
+
+ #ifndef ARROW_C_ASYNC_STREAM_INTERFACE
+ #define ARROW_C_ASYNC_STREAM_INTERFACE
+
+ struct ArrowAsyncTask {
+ int (*get_data)(struct ArrowArrayTask* self, struct ArrowDeviceArray* out);
+
+ void* private_data;
+ };
+
+ struct ArrowAsyncProducer {
+ void (*request)(struct ArrowAsyncProducer* self, uint64_t n);
+ void (*cancel)(struct ArrowAsyncProducer* self);
+
+ void (*release)(struct ArrowAsyncProducer* self);
+ void* private_data;
+ };
+
+ struct ArrowAsyncDeviceStreamHandler {
+ // handlers
+ int (*on_schema)(struct ArrowAsyncDeviceStreamHandler* self,
+ struct ArrowAsyncProducer* producer,
+ struct ArrowSchema* stream_schema, const char* addl_metadata);
+ int (*on_next_task)(struct ArrowAsyncDeviceStreamHandler* self,
+ struct ArrowAsyncTask* task, const char* metadata);
+ void (*on_error)(struct ArrowAsyncDeviceStreamHandler* self,
+ int code, const char* message, const char* metadata);
+
+ // release callback
+ void (*release)(struct ArrowAsyncDeviceStreamHandler* self);
+
+ // opaque handler-specific data
+ void* private_data;
+ };
+
+ #endif // ARROW_C_ASYNC_STREAM_INTERFACE
+
+.. note::
+ The canonical guard ``ARROW_C_ASYNC_STREAM_INTERFACE`` is meant to avoid
+ duplicate definitions if two projects copy the C async stream interface
+ definitions into their own headers, and a third-party project includes
+ from these two projects. It is therefore important that this guard is kept
+ exactly as-is when these definitions are copied.
+
+The ArrowAsyncDeviceStreamHandler structure
+'''''''''''''''''''''''''''''''''''''''''''
+
+The structure has the following fields:
+
+.. c:member:: int (*ArrowAsyncDeviceStreamHandler.on_schema)(struct ArrowAsyncDeviceStreamHandler*, struct ArrowAsyncProducer*, struct ArrowSchema*, const char*)
+
+ *Mandatory.* Handler for receiving the schema of the stream. All records should
+ match the provided schema. If successful, the function should return 0, otherwise
+ it should return an ``errno``-compatible error code.
+
+ The ``const char*`` parameter exists for producers to provide any extra contextual information
+ they want, such as the total number of rows in the stream, statistics, or otherwise. This is
+ encoded in the same format as :c:member:`ArrowSchema.metadata`. If not ``NULL``,
+ the lifetime is only the scope of the call to this function. A consumer who wants to maintain
+ the additional metadata beyond the lifetime of this call *MUST* copy the value themselves.
+
+ Unless the ``on_error`` handler is called, this will always get called exactly once and will be
+ the first method called on this object. As such the producer *MUST* provide an ``ArrowAsyncProducer``
+ object when calling this function to allow the consumer to manage back-pressure and flow control.
+ The producer maintains ownership of the ``ArrowAsyncProducer`` and must clean it up before or after
+ calling the release callback on this object.
+
+ A producer that receives a non-zero result here must not subsequently call anything other than
+ the release callback on this object.
+
+.. c:member:: int (*ArrowAsyncDeviceStreamHandler.on_next_task)(struct ArrowAsyncDeviceStreamHandler*, struct ArrowAsyncTask*, const char*)
+
+ *Mandatory.* Handler to be called when a new record is available for processing. The
+ schema for each record should be the same as the schema that ``on_schema`` was called with.
+ If successfully handled, the function should return 0, otherwise it should return an
+ ``errno``-compatible error code.
+
+ Rather than passing the record itself it receives an ``ArrowAsyncTask`` instead to facilitate
+ better consumer-focused thread control as far as receiving the data. A call to this function
+ simply indicates that data is available via the provided task.
+
+ The ``const char*`` parameter exists for producers to provide any extra contextual information
+ they want. This is encoded in the same format as :c:member:`ArrowSchema.metadata`. If not ``NULL``,
+ the lifetime is only the scope of the call to this function. A consumer who wants to maintain
+ the additional metadata beyond the lifetime of this call *MUST* copy the value themselves.
+
+ A producer *MUST NOT* call this concurrently from multiple different threads.
+
+ The :c:member:`ArrowAsyncProducer.request` callback must be called to start receiving calls to this
+ handler.
+
+ This function *MUST* be able to be called re-entrantly on the same thread to allow for the
+ :c:member:`ArrowAsyncProducer.request` callback to potentially call this method recursively.
+
+.. c:member:: void (*ArrowAsyncDeviceStreamHandler.on_error)(struct ArrowAsyncDeviceStreamHandler, int, const char*, const char*)
+
+ *Mandatory.* Handler to be called when an error is encountered by the producer. After calling
+ this, the ``release`` callback will be called as the last call on this struct. The parameters
+ are an ``errno``-compatible error code and an optional error message and metadata.
+
+ If the message and metadata are not ``NULL``, their lifetime is only valid during the scope
+ of this call. A consumer who wants to maintain these values past the return of this function
+ *MUST* copy the values themselves.
+
+ If the metadata parameter is not ``NULL``, to provide key-value error metadata, then it should
+ be encoded identically to the way that metadata is encoded in :c:member:`ArrowSchema.metadata`.
+
+ It is valid for this to be called by a producer with or without a preceding call to
+ :c:member:`ArrowAsyncProducer.request`. This callback *MUST NOT* call any methods of an
+ ``ArrowAsyncProducer`` object.
+
+.. c:member:: void (*ArrowAsyncDeviceStreamHandler.release)(struct ArrowAsyncDeviceStreamHandler*)
+
+ *Mandatory.* A pointer to a consumer-provided release callback for the handler.
+
+ It is valid for this to be called by a producer with or without a preceding call to
+ :c:member:`ArrowAsyncProducer.request`. This must not call any methods of an ``ArrowAsyncProducer``
+ object.
+
+.. c:member:: void* ArrowAsyncDeviceStreamHandler.private_data
+
+ *Optional.* An opaque pointer to consumer-provided private data.
+
+ Producers *MUST NOT* process this member. Lifetime of this member is handled by
+ the consumer, and especially by the release callback.
+
+The ArrowAsyncTask structure
+''''''''''''''''''''''''''''
+
+The purpose of using a Task object rather than passing the array directly to the ``on_next``
+callback is to allow for more complex and efficient thread handling. Utilizing a Task
+object allows for a producer to separate the "decoding" logic from the I/O, enabling a
+consumer to avoid transferring data between CPU cores (e.g. from one L1/L2 cache to another).
Review Comment:
It's not required to offload the decoding to consumer threads, it's just a possible idea to reduce the cost of transferring data between CPU threads / improve cache handling as per the comment. (See @westonpace's comments for more info about the origin of this idea). For simplicity, most cases will likely perform the decoding on the producer side and then have `get_data` be nearly instantaneous to just return the data directly.
This aspect merely makes it viable for use cases that require this complexity for efficiency to still be able to use this interface.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-43631: [C][Format] Add ArrowAsyncDeviceStreamHandler interface [arrow]
Posted by "zeroshade (via GitHub)" <gi...@apache.org>.
zeroshade commented on code in PR #43632:
URL: https://github.com/apache/arrow/pull/43632#discussion_r1718785732
##########
docs/source/format/CDeviceDataInterface.rst:
##########
@@ -650,6 +652,137 @@ The stream source is not assumed to be thread-safe. Consumers wanting to
call ``get_next`` from several threads should ensure those calls are
serialized.
+Async Device Stream Interface
+=============================
+
+The :ref:`C stream interface <_c-device-stream-interface>` provides a synchronous
+API centered around the consumer calling the callback functions to retrieve
+the next record batch. For some bindings, use cases, and interoperability, a more
+asynchronous, producer-focused interface may be required. These scenarios can utilize
+the ``ArrowAsyncDeviceStreamHandler``.
+
+Semantics
+---------
+
+Rather than the producer providing a structure of callbacks for a consumer to
+call and retrieve records, the Async interface is a consumer allocated structure.
+The consumer allocated struct provides handler callbacks for the producer to call
+when the schema and chunks of data are available, rather than the consumer using
+a blocking pull-style iteration.
+
+Structure Definition
+--------------------
+
+The C device async stream interface is defined with a single ``struct`` definition:
+
+.. code-block:: c
+
+ #ifndef ARROW_C_ASYNC_STREAM_INTERFACE
+ #define ARROW_C_ASYNC_STREAM_INTERFACE
+
+ struct ArrowAsyncDeviceStreamHandler {
+ // handlers
+ int (*on_schema)(struct ArrowAsyncDeviceStreamHandler* self,
+ struct ArrowSchema* stream_schema, void* extension_param);
+ int (*on_next)(struct ArrowAsyncDeviceStreamHandler* self,
+ struct ArrowDeviceArray* next, void* extension_param);
+ void (*on_error)(struct ArrowAsyncDeviceStreamHandler* self,
+ int code, const char* message, const char* metadata);
+
+ // release callback
+ void (*release)(struct ArrowAsyncDeviceStreamHandler* self);
+
+ // opaque handler-specific data
+ void* private_data;
+ };
+
+ #endif // ARROW_C_ASYNC_STREAM_INTERFACE
+
+.. note::
+ The canonical guard ``ARROW_C_ASYNC_STREAM_INTERFACE`` is meant to avoid
+ duplicate definitions if two projects copy the C async stream interface
+ definitions into their own headers, and a third-party project includes
+ from these two projects. It is therefore important that this guard is kept
+ exactly as-is when these definitions are copied.
+
+The ArrowAsyncDeviceStreamHandler structure
+'''''''''''''''''''''''''''''''''''''''''''
+
+The structure has the following fields:
+
+.. c:member:: int (*ArrowAsyncDeviceStreamHandler.on_schema)(struct ArrowAsyncDeviceStreamHandler*, struct ArrowSchema*, void*)
+
+ *Mandatory.* Handler for receiving the schema of the stream. All records should
+ match the provided schema. If successful, the function should return 0, otherwise
+ it should return an ``errno``-compatible error code.
+
+ The ``void*`` parameter exists for producers to provide any extra contextual information
+ they want, such as the total number of rows in the stream, statistics, or otherwise.
+
+ Unless the ``on_error`` handler is called, this will always get called exactly once.
+
+.. c:member:: int (*ArrowAsyncDeviceStreamHandler.on_next)(struct ArrowAsyncDeviceStreamHandler*, struct ArrowDeviceArray*, void*)
+
+ *Mandatory.* Handler to be called when a new record is available for processing. The
+ schema for each record should be the same as the schema that ``on_schema`` was called with.
+ If successfully handled, the function should return 0, otherwise it should return an
+ ``errno``-compatible error code.
+
+ The ``void*`` parameter exists for producers to provide any extra contextual information
+ they want.
+
+ Unless the ``on_error`` handler is called, this will always get called at least once. The
+ end of the stream is indicated by it being called with an empty or released ``ArrowDeviceArray``.
+
+.. c:member:: void (*ArrowAsyncDeviceStreamHandler.on_error)(struct ArrowAsyncDeviceStreamHandler, int, const char*, const char*)
+
+ *Mandatory.* Handler to be called when an error is encountered by the producer. After calling
+ this, the ``release`` callback will be called as the last call on this struct. The parameters
+ are an ``errno``-compatible error code and an optional error message and metadata.
+
+ If the message and metadata are not ``NULL``, their lifetime is only valid during the scope
+ of this call. A consumer who wants to maintain these values past the return of this function
+ *MUST* copy the values themselves.
+
+ If the metadata parameter is not ``NULL``, to provide key-value error metadata, then it should
+ be encoded identically to the way that metadata is encoded in :c:member:`ArrowSchema.metadata`.
+
+.. c:member:: void (*ArrowAsyncDeviceStreamHandler.release)(struct ArrowAsyncDeviceStreamHandler*)
+
+ *Mandatory.* A pointer to a consumer-provided release callback for the handler.
+
+.. c:member:: void* ArrowAsyncDeviceStreamHandler.private_data
+
+ *Optional.* An opaque pointer to consumer-provided private data.
+
+ Producers *MUST NOT* process this member. Lifetime of this member is handled by
+ the consumer, and especially by the release callback.
+
+Result lifetimes
+''''''''''''''''
+
+The ``ArrowSchema`` and ``ArrowDeviceArray`` that are passed to the ``on_schema`` and
+``on_next`` callbacks must be released independently. Their lifetimes are not tied to
+the ``ArrowAsyncDeviceStreamHandler`` or the handler functions.
+
+The ``const char*`` error ``message`` and ``metadata`` which are passed to ``on_error``
+are only valid within the scope of the ``on_error`` function itself. They must be copied
+if it is necessary for them to exist after it returns.
+
+Stream Handler Lifetime
+'''''''''''''''''''''''
+
+Lifetime of the async stream handler is managed using a release callback with similar
+usage as in :ref:`C data interface <c-data-interface-released>`.
+
+Thread safety
+'''''''''''''
+
+All handler functions should only be called in a serialized manner, but are not guaranteed
+to be called from the same thread every time. A producer should wait for handler callbacks to
+return before calling the next handler callback. As a result, a consumer can manage back-pressure
+simply by sleeping or otherwise waiting to return from the ``on_next`` handler if necessary.
+
Review Comment:
add a clarification sentence regarding this.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-43631: [C][Format] Add ArrowAsyncDeviceStreamHandler interface [arrow]
Posted by "zeroshade (via GitHub)" <gi...@apache.org>.
zeroshade commented on code in PR #43632:
URL: https://github.com/apache/arrow/pull/43632#discussion_r1767468481
##########
docs/source/format/CDeviceDataInterface.rst:
##########
@@ -650,6 +652,340 @@ The stream source is not assumed to be thread-safe. Consumers wanting to
call ``get_next`` from several threads should ensure those calls are
serialized.
+Async Device Stream Interface
+=============================
+
+The :ref:`C stream interface <_c-device-stream-interface>` provides a synchronous
+API centered around the consumer calling the callback functions to retrieve
+the next record batch. For some bindings, use cases, and interoperability, a more
+asynchronous, producer-focused interface may be required. These scenarios can utilize
+the ``ArrowAsyncDeviceStreamHandler``.
+
+Semantics
+---------
+
+Rather than the producer providing a structure of callbacks for a consumer to
+call and retrieve records, the Async interface is a consumer allocated structure.
+The consumer allocated struct provides handler callbacks for the producer to call
+when the schema and chunks of data are available, rather than the consumer using
+a blocking pull-style iteration.
+
+In addition to the ``ArrowAsyncDeviceStreamHandler``, there are also two additional
+structs used for the full data flow: ``ArrowAsyncTask`` and ``ArrowAsyncProducer``.
+
+Structure Definition
+--------------------
+
+The C device async stream interface is defined with a single ``struct`` definition:
+
+.. code-block:: c
+
+ #ifndef ARROW_C_ASYNC_STREAM_INTERFACE
+ #define ARROW_C_ASYNC_STREAM_INTERFACE
+
+ struct ArrowAsyncTask {
+ int (*get_data)(struct ArrowArrayTask* self, struct ArrowDeviceArray* out);
+
+ void* private_data;
+ };
+
+ struct ArrowAsyncProducer {
+ void (*request)(struct ArrowAsyncProducer* self, int64_t n);
+ void (*cancel)(struct ArrowAsyncProducer* self);
+
+ void (*release)(struct ArrowAsyncProducer* self);
+ void* private_data;
+ };
+
+ struct ArrowAsyncDeviceStreamHandler {
+ // handlers
+ int (*on_schema)(struct ArrowAsyncDeviceStreamHandler* self,
+ struct ArrowAsyncProducer* producer,
+ struct ArrowSchema* stream_schema, const char* addl_metadata);
+ int (*on_next_task)(struct ArrowAsyncDeviceStreamHandler* self,
+ struct ArrowAsyncTask* task, const char* metadata);
+ void (*on_error)(struct ArrowAsyncDeviceStreamHandler* self,
+ int code, const char* message, const char* metadata);
+
+ // release callback
+ void (*release)(struct ArrowAsyncDeviceStreamHandler* self);
+
+ // opaque handler-specific data
+ void* private_data;
+ };
+
+ #endif // ARROW_C_ASYNC_STREAM_INTERFACE
+
+.. note::
+ The canonical guard ``ARROW_C_ASYNC_STREAM_INTERFACE`` is meant to avoid
+ duplicate definitions if two projects copy the C async stream interface
+ definitions into their own headers, and a third-party project includes
+ from these two projects. It is therefore important that this guard is kept
+ exactly as-is when these definitions are copied.
+
+The ArrowAsyncDeviceStreamHandler structure
+'''''''''''''''''''''''''''''''''''''''''''
+
+The structure has the following fields:
+
+.. c:member:: int (*ArrowAsyncDeviceStreamHandler.on_schema)(struct ArrowAsyncDeviceStreamHandler*, struct ArrowAsyncProducer*, struct ArrowSchema*, const char*)
+
+ *Mandatory.* Handler for receiving the schema of the stream. All records should
+ match the provided schema. If successful, the function should return 0, otherwise
+ it should return an ``errno``-compatible error code.
+
+ The ``const char*`` parameter exists for producers to provide any extra contextual information
+ they want, such as the total number of rows in the stream, statistics, or otherwise. This is
+ encoded in the same format as :c:member:`ArrowSchema.metadata`. If not ``NULL``,
+ the lifetime is only the scope of the call to this function. A consumer who wants to maintain
+ the additional metadata beyond the lifetime of this call *MUST* copy the value themselves.
+
+ Unless the ``on_error`` handler is called, this will always get called exactly once and will be
+ the first method called on this object. As such the producer *MUST* provide an ``ArrowAsyncProducer``
+ object when calling this function to allow the consumer to manage back-pressure and flow control.
+ The producer maintains ownership of the ``ArrowAsyncProducer`` and must clean it up *after*
+ calling the release callback on this object.
+
+ A producer that receives a non-zero result here must not subsequently call anything other than
+ the release callback on this object.
+
+.. c:member:: int (*ArrowAsyncDeviceStreamHandler.on_next_task)(struct ArrowAsyncDeviceStreamHandler*, struct ArrowAsyncTask*, const char*)
+
+ *Mandatory.* Handler to be called when a new record is available for processing. The
+ schema for each record should be the same as the schema that ``on_schema`` was called with.
+ If successfully handled, the function should return 0, otherwise it should return an
+ ``errno``-compatible error code.
+
+ Rather than passing the record itself it receives an ``ArrowAsyncTask`` instead to facilitate
+ better consumer-focused thread control as far as receiving the data. A call to this function
+ simply indicates that data is available via the provided task.
+
+ The producer signals the end of the stream by passing ``NULL`` for the ``ArrowAsyncTask``
+ pointer instead of a valid address. This task object is only valid during the lifetime of
+ this function call. If the consumer wants to use the task beyond the scope of this method, it
+ must copy or move its contents to a new ArrowAsyncTask object.
Review Comment:
@bkietz suggested renaming it to `extract_data` over [here](https://github.com/apache/arrow/pull/43632#discussion_r1766895138) do you have a preference for `extract_data` vs `consume_data`?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-43631: [C][Format] Add ArrowAsyncDeviceStreamHandler interface [arrow]
Posted by "pitrou (via GitHub)" <gi...@apache.org>.
pitrou commented on code in PR #43632:
URL: https://github.com/apache/arrow/pull/43632#discussion_r1764965580
##########
docs/source/format/CDeviceDataInterface.rst:
##########
@@ -650,6 +652,331 @@ The stream source is not assumed to be thread-safe. Consumers wanting to
call ``get_next`` from several threads should ensure those calls are
serialized.
+Async Device Stream Interface
+=============================
+
+The :ref:`C stream interface <_c-device-stream-interface>` provides a synchronous
+API centered around the consumer calling the callback functions to retrieve
+the next record batch. For some bindings, use cases, and interoperability, a more
+asynchronous, producer-focused interface may be required. These scenarios can utilize
+the ``ArrowAsyncDeviceStreamHandler``.
+
+Semantics
+---------
+
+Rather than the producer providing a structure of callbacks for a consumer to
+call and retrieve records, the Async interface is a consumer allocated structure.
+The consumer allocated struct provides handler callbacks for the producer to call
+when the schema and chunks of data are available, rather than the consumer using
+a blocking pull-style iteration.
+
+In addition to the ``ArrowAsyncDeviceStreamHandler``, there are also two additional
+structs used for the full data flow: ``ArrowAsyncTask`` and ``ArrowAsyncProducer``.
+
+Structure Definition
+--------------------
+
+The C device async stream interface is defined with a single ``struct`` definition:
+
+.. code-block:: c
+
+ #ifndef ARROW_C_ASYNC_STREAM_INTERFACE
+ #define ARROW_C_ASYNC_STREAM_INTERFACE
+
+ struct ArrowAsyncTask {
+ int (*get_data)(struct ArrowArrayTask* self, struct ArrowDeviceArray* out);
+
+ void* private_data;
+ };
+
+ struct ArrowAsyncProducer {
+ void (*request)(struct ArrowAsyncProducer* self, uint64_t n);
+ void (*cancel)(struct ArrowAsyncProducer* self);
+
+ void (*release)(struct ArrowAsyncProducer* self);
+ void* private_data;
+ };
+
+ struct ArrowAsyncDeviceStreamHandler {
+ // handlers
+ int (*on_schema)(struct ArrowAsyncDeviceStreamHandler* self,
+ struct ArrowAsyncProducer* producer,
+ struct ArrowSchema* stream_schema, const char* addl_metadata);
+ int (*on_next_task)(struct ArrowAsyncDeviceStreamHandler* self,
+ struct ArrowAsyncTask* task, const char* metadata);
+ void (*on_error)(struct ArrowAsyncDeviceStreamHandler* self,
+ int code, const char* message, const char* metadata);
+
+ // release callback
+ void (*release)(struct ArrowAsyncDeviceStreamHandler* self);
+
+ // opaque handler-specific data
+ void* private_data;
+ };
+
+ #endif // ARROW_C_ASYNC_STREAM_INTERFACE
+
+.. note::
+ The canonical guard ``ARROW_C_ASYNC_STREAM_INTERFACE`` is meant to avoid
+ duplicate definitions if two projects copy the C async stream interface
+ definitions into their own headers, and a third-party project includes
+ from these two projects. It is therefore important that this guard is kept
+ exactly as-is when these definitions are copied.
+
+The ArrowAsyncDeviceStreamHandler structure
+'''''''''''''''''''''''''''''''''''''''''''
+
+The structure has the following fields:
+
+.. c:member:: int (*ArrowAsyncDeviceStreamHandler.on_schema)(struct ArrowAsyncDeviceStreamHandler*, struct ArrowAsyncProducer*, struct ArrowSchema*, const char*)
+
+ *Mandatory.* Handler for receiving the schema of the stream. All records should
+ match the provided schema. If successful, the function should return 0, otherwise
+ it should return an ``errno``-compatible error code.
+
+ The ``const char*`` parameter exists for producers to provide any extra contextual information
+ they want, such as the total number of rows in the stream, statistics, or otherwise. This is
+ encoded in the same format as :c:member:`ArrowSchema.metadata`. If not ``NULL``,
+ the lifetime is only the scope of the call to this function. A consumer who wants to maintain
+ the additional metadata beyond the lifetime of this call *MUST* copy the value themselves.
+
+ Unless the ``on_error`` handler is called, this will always get called exactly once and will be
+ the first method called on this object. As such the producer *MUST* provide an ``ArrowAsyncProducer``
+ object when calling this function to allow the consumer to manage back-pressure and flow control.
+ The producer maintains ownership of the ``ArrowAsyncProducer`` and must clean it up before or after
+ calling the release callback on this object.
+
+ A producer that receives a non-zero result here must not subsequently call anything other than
+ the release callback on this object.
+
+.. c:member:: int (*ArrowAsyncDeviceStreamHandler.on_next_task)(struct ArrowAsyncDeviceStreamHandler*, struct ArrowAsyncTask*, const char*)
+
+ *Mandatory.* Handler to be called when a new record is available for processing. The
+ schema for each record should be the same as the schema that ``on_schema`` was called with.
+ If successfully handled, the function should return 0, otherwise it should return an
+ ``errno``-compatible error code.
+
+ Rather than passing the record itself it receives an ``ArrowAsyncTask`` instead to facilitate
+ better consumer-focused thread control as far as receiving the data. A call to this function
+ simply indicates that data is available via the provided task.
+
+ The ``const char*`` parameter exists for producers to provide any extra contextual information
+ they want. This is encoded in the same format as :c:member:`ArrowSchema.metadata`. If not ``NULL``,
+ the lifetime is only the scope of the call to this function. A consumer who wants to maintain
+ the additional metadata beyond the lifetime of this call *MUST* copy the value themselves.
+
+ A producer *MUST NOT* call this concurrently from multiple different threads.
+
+ The :c:member:`ArrowAsyncProducer.request` callback must be called to start receiving calls to this
+ handler.
+
+ This function *MUST* be able to be called re-entrantly on the same thread to allow for the
+ :c:member:`ArrowAsyncProducer.request` callback to potentially call this method recursively.
+
+.. c:member:: void (*ArrowAsyncDeviceStreamHandler.on_error)(struct ArrowAsyncDeviceStreamHandler, int, const char*, const char*)
+
+ *Mandatory.* Handler to be called when an error is encountered by the producer. After calling
+ this, the ``release`` callback will be called as the last call on this struct. The parameters
+ are an ``errno``-compatible error code and an optional error message and metadata.
+
+ If the message and metadata are not ``NULL``, their lifetime is only valid during the scope
+ of this call. A consumer who wants to maintain these values past the return of this function
+ *MUST* copy the values themselves.
+
+ If the metadata parameter is not ``NULL``, to provide key-value error metadata, then it should
+ be encoded identically to the way that metadata is encoded in :c:member:`ArrowSchema.metadata`.
+
+ It is valid for this to be called by a producer with or without a preceding call to
+ :c:member:`ArrowAsyncProducer.request`. This callback *MUST NOT* call any methods of an
+ ``ArrowAsyncProducer`` object.
+
+.. c:member:: void (*ArrowAsyncDeviceStreamHandler.release)(struct ArrowAsyncDeviceStreamHandler*)
+
+ *Mandatory.* A pointer to a consumer-provided release callback for the handler.
+
+ It is valid for this to be called by a producer with or without a preceding call to
+ :c:member:`ArrowAsyncProducer.request`. This must not call any methods of an ``ArrowAsyncProducer``
+ object.
+
+.. c:member:: void* ArrowAsyncDeviceStreamHandler.private_data
+
+ *Optional.* An opaque pointer to consumer-provided private data.
+
+ Producers *MUST NOT* process this member. Lifetime of this member is handled by
+ the consumer, and especially by the release callback.
+
+The ArrowAsyncTask structure
+''''''''''''''''''''''''''''
+
+The purpose of using a Task object rather than passing the array directly to the ``on_next``
+callback is to allow for more complex and efficient thread handling. Utilizing a Task
+object allows for a producer to separate the "decoding" logic from the I/O, enabling a
+consumer to avoid transferring data between CPU cores (e.g. from one L1/L2 cache to another).
+
+This producer-provided structure has the following fields:
+
+.. c:member:: int (*ArrowArrayTask.get_data)(struct ArrowArrayTask*, struct ArrowDeviceArray*)
+
+ *Mandatory.* A callback to populate the provided ``ArrowDeviceArray`` with the available data.
+ The order of ArrowAsyncTasks provided by the producer enables a consumer to know the order of
+ the data to process.
+
+ If a non-zero value is returned from this, it should be followed only by the producer calling
+ the ``on_error`` callback of the ``ArrowAsyncDeviceStreamHandler``. Because calling this method
+ is likely to be separate from the current control flow, returning a non-zero value to signal
+ an error occuring allows the current thread to decide handle the case accordingly, while still
+ allowing all error logging and handling to be centralized in the
+ :c:member:`ArrowAsyncDeviceStreamHandler.on_error` callback.
+
+ Rather than having a separate release callback, any required cleanup should be performed as part
+ of the invocation of this callback, with ownership of the Array being passed to the consumer-provided
+ parameter calling this, and must be released separately.
+
+ It is only valid to call this method exactly once.
+
+.. c:member:: void* ArrowArrayTask.private_data
+
+ *Optional.* An opaque pointer to consumer-provided private data.
+
+ Consumers *MUST NOT* process this member. Lifetime of this member is handled by
+ the producer who created this object, and should be cleaned up if necessary during
+ the call to :c:member:`ArrowArrayTask.get_data`.
+
+The ArrowAsyncProducer structure
+''''''''''''''''''''''''''''''''
+
+This producer-provided and managed object has the following fields:
+
+.. c:member:: void (*ArrowAsyncProducer.request)(struct ArrowAsyncProducer*, uint64_t)
Review Comment:
Overall the point still stands:
1) there is no real advantage in using an unsigned integer
2) it breaks our conventions and will make the codebase more error-prone
3) it can make things slightly harder for Java developers
so let's please switch to `int64_t`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-43631: [C][Format] Add ArrowAsyncDeviceStreamHandler interface [arrow]
Posted by "pitrou (via GitHub)" <gi...@apache.org>.
pitrou commented on code in PR #43632:
URL: https://github.com/apache/arrow/pull/43632#discussion_r1765346604
##########
docs/source/format/CDeviceDataInterface.rst:
##########
@@ -650,6 +652,331 @@ The stream source is not assumed to be thread-safe. Consumers wanting to
call ``get_next`` from several threads should ensure those calls are
serialized.
+Async Device Stream Interface
+=============================
+
+The :ref:`C stream interface <_c-device-stream-interface>` provides a synchronous
+API centered around the consumer calling the callback functions to retrieve
+the next record batch. For some bindings, use cases, and interoperability, a more
+asynchronous, producer-focused interface may be required. These scenarios can utilize
+the ``ArrowAsyncDeviceStreamHandler``.
+
+Semantics
+---------
+
+Rather than the producer providing a structure of callbacks for a consumer to
+call and retrieve records, the Async interface is a consumer allocated structure.
+The consumer allocated struct provides handler callbacks for the producer to call
+when the schema and chunks of data are available, rather than the consumer using
+a blocking pull-style iteration.
+
+In addition to the ``ArrowAsyncDeviceStreamHandler``, there are also two additional
+structs used for the full data flow: ``ArrowAsyncTask`` and ``ArrowAsyncProducer``.
+
+Structure Definition
+--------------------
+
+The C device async stream interface is defined with a single ``struct`` definition:
+
+.. code-block:: c
+
+ #ifndef ARROW_C_ASYNC_STREAM_INTERFACE
+ #define ARROW_C_ASYNC_STREAM_INTERFACE
+
+ struct ArrowAsyncTask {
+ int (*get_data)(struct ArrowArrayTask* self, struct ArrowDeviceArray* out);
+
+ void* private_data;
+ };
+
+ struct ArrowAsyncProducer {
+ void (*request)(struct ArrowAsyncProducer* self, uint64_t n);
+ void (*cancel)(struct ArrowAsyncProducer* self);
+
+ void (*release)(struct ArrowAsyncProducer* self);
+ void* private_data;
+ };
+
+ struct ArrowAsyncDeviceStreamHandler {
+ // handlers
+ int (*on_schema)(struct ArrowAsyncDeviceStreamHandler* self,
+ struct ArrowAsyncProducer* producer,
+ struct ArrowSchema* stream_schema, const char* addl_metadata);
+ int (*on_next_task)(struct ArrowAsyncDeviceStreamHandler* self,
+ struct ArrowAsyncTask* task, const char* metadata);
+ void (*on_error)(struct ArrowAsyncDeviceStreamHandler* self,
+ int code, const char* message, const char* metadata);
+
+ // release callback
+ void (*release)(struct ArrowAsyncDeviceStreamHandler* self);
+
+ // opaque handler-specific data
+ void* private_data;
+ };
+
+ #endif // ARROW_C_ASYNC_STREAM_INTERFACE
+
+.. note::
+ The canonical guard ``ARROW_C_ASYNC_STREAM_INTERFACE`` is meant to avoid
+ duplicate definitions if two projects copy the C async stream interface
+ definitions into their own headers, and a third-party project includes
+ from these two projects. It is therefore important that this guard is kept
+ exactly as-is when these definitions are copied.
+
+The ArrowAsyncDeviceStreamHandler structure
+'''''''''''''''''''''''''''''''''''''''''''
+
+The structure has the following fields:
+
+.. c:member:: int (*ArrowAsyncDeviceStreamHandler.on_schema)(struct ArrowAsyncDeviceStreamHandler*, struct ArrowAsyncProducer*, struct ArrowSchema*, const char*)
+
+ *Mandatory.* Handler for receiving the schema of the stream. All records should
+ match the provided schema. If successful, the function should return 0, otherwise
+ it should return an ``errno``-compatible error code.
+
+ The ``const char*`` parameter exists for producers to provide any extra contextual information
+ they want, such as the total number of rows in the stream, statistics, or otherwise. This is
+ encoded in the same format as :c:member:`ArrowSchema.metadata`. If not ``NULL``,
+ the lifetime is only the scope of the call to this function. A consumer who wants to maintain
+ the additional metadata beyond the lifetime of this call *MUST* copy the value themselves.
+
+ Unless the ``on_error`` handler is called, this will always get called exactly once and will be
+ the first method called on this object. As such the producer *MUST* provide an ``ArrowAsyncProducer``
+ object when calling this function to allow the consumer to manage back-pressure and flow control.
+ The producer maintains ownership of the ``ArrowAsyncProducer`` and must clean it up before or after
+ calling the release callback on this object.
+
+ A producer that receives a non-zero result here must not subsequently call anything other than
+ the release callback on this object.
+
+.. c:member:: int (*ArrowAsyncDeviceStreamHandler.on_next_task)(struct ArrowAsyncDeviceStreamHandler*, struct ArrowAsyncTask*, const char*)
+
+ *Mandatory.* Handler to be called when a new record is available for processing. The
+ schema for each record should be the same as the schema that ``on_schema`` was called with.
+ If successfully handled, the function should return 0, otherwise it should return an
+ ``errno``-compatible error code.
+
+ Rather than passing the record itself it receives an ``ArrowAsyncTask`` instead to facilitate
+ better consumer-focused thread control as far as receiving the data. A call to this function
+ simply indicates that data is available via the provided task.
+
+ The ``const char*`` parameter exists for producers to provide any extra contextual information
+ they want. This is encoded in the same format as :c:member:`ArrowSchema.metadata`. If not ``NULL``,
+ the lifetime is only the scope of the call to this function. A consumer who wants to maintain
+ the additional metadata beyond the lifetime of this call *MUST* copy the value themselves.
+
+ A producer *MUST NOT* call this concurrently from multiple different threads.
+
+ The :c:member:`ArrowAsyncProducer.request` callback must be called to start receiving calls to this
+ handler.
+
+ This function *MUST* be able to be called re-entrantly on the same thread to allow for the
+ :c:member:`ArrowAsyncProducer.request` callback to potentially call this method recursively.
+
+.. c:member:: void (*ArrowAsyncDeviceStreamHandler.on_error)(struct ArrowAsyncDeviceStreamHandler, int, const char*, const char*)
+
+ *Mandatory.* Handler to be called when an error is encountered by the producer. After calling
+ this, the ``release`` callback will be called as the last call on this struct. The parameters
+ are an ``errno``-compatible error code and an optional error message and metadata.
+
+ If the message and metadata are not ``NULL``, their lifetime is only valid during the scope
+ of this call. A consumer who wants to maintain these values past the return of this function
+ *MUST* copy the values themselves.
+
+ If the metadata parameter is not ``NULL``, to provide key-value error metadata, then it should
+ be encoded identically to the way that metadata is encoded in :c:member:`ArrowSchema.metadata`.
+
+ It is valid for this to be called by a producer with or without a preceding call to
+ :c:member:`ArrowAsyncProducer.request`. This callback *MUST NOT* call any methods of an
+ ``ArrowAsyncProducer`` object.
+
+.. c:member:: void (*ArrowAsyncDeviceStreamHandler.release)(struct ArrowAsyncDeviceStreamHandler*)
+
+ *Mandatory.* A pointer to a consumer-provided release callback for the handler.
+
+ It is valid for this to be called by a producer with or without a preceding call to
+ :c:member:`ArrowAsyncProducer.request`. This must not call any methods of an ``ArrowAsyncProducer``
+ object.
+
+.. c:member:: void* ArrowAsyncDeviceStreamHandler.private_data
+
+ *Optional.* An opaque pointer to consumer-provided private data.
+
+ Producers *MUST NOT* process this member. Lifetime of this member is handled by
+ the consumer, and especially by the release callback.
+
+The ArrowAsyncTask structure
+''''''''''''''''''''''''''''
+
+The purpose of using a Task object rather than passing the array directly to the ``on_next``
+callback is to allow for more complex and efficient thread handling. Utilizing a Task
+object allows for a producer to separate the "decoding" logic from the I/O, enabling a
+consumer to avoid transferring data between CPU cores (e.g. from one L1/L2 cache to another).
+
+This producer-provided structure has the following fields:
+
+.. c:member:: int (*ArrowArrayTask.get_data)(struct ArrowArrayTask*, struct ArrowDeviceArray*)
Review Comment:
Again, the consumer does not know if there's anything beneficial to offload. Why not let the producer decide?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-43631: [C][Format] Add ArrowAsyncDeviceStreamHandler interface [arrow]
Posted by "zeroshade (via GitHub)" <gi...@apache.org>.
zeroshade commented on code in PR #43632:
URL: https://github.com/apache/arrow/pull/43632#discussion_r1767549318
##########
cpp/src/arrow/c/abi.h:
##########
@@ -228,6 +228,207 @@ struct ArrowDeviceArrayStream {
#endif // ARROW_C_DEVICE_STREAM_INTERFACE
+#ifndef ARROW_C_ASYNC_STREAM_INTERFACE
+#define ARROW_C_ASYNC_STREAM_INTERFACE
+
+// ArrowAsyncTask represents available data from a producer that was passed to
+// an invocation of `on_next_task` on the ArrowAsyncDeviceStreamHandler.
+//
+// The reason for this Task approach instead of the Async interface returning
+// the Array directly is to allow for more complex thread handling and reducing
+// context switching and data transfers between CPU cores (e.g. from one L1/L2
+// cache to another) if desired.
+//
+// For example, the `on_next_task` callback can be called when data is ready, while
+// the producer puts potential "decoding" logic in the `ArrowAsyncTask` object. This
+// allows for the producer to manage the I/O on one thread which calls `on_next_task`
+// and the consumer can determine when the decoding (producer logic in the `get_data`
+// callback of the task) occurs and on which thread, to avoid a CPU core transfer
+// (data staying in the L2 cache).
+struct ArrowAsyncTask {
+ // This callback should populate the ArrowDeviceArray associated with this task.
+ // The order of ArrowAsyncTasks provided by the producer enables a consumer to
+ // ensure the order of data to process.
+ //
+ // This function is expected to be synchronous, but should not perform any blocking
+ // I/O. Ideally it should be as cheap as possible so as to not tie up the consumer
+ // thread unnecessarily.
+ //
+ // Returns: 0 if successful, errno-compatible error otherwise.
+ //
+ // If a non-0 value is returned then it should be followed by a call to `on_error`
+ // on the appropriate ArrowAsyncDeviceStreamHandler. This is because the it's highly
+ // likely that whatever is calling this function may be entirely disconnected from
+ // the current control flow. Indicating an error here with a non-zero return allows
+ // the current flow to be aware of the error occurring, while still allowing any
+ // logging or error handling to still be centralized in the `on_error` callback of
+ // the original Async handler.
+ //
+ // Rather than a release callback, any required cleanup should be performed as part
+ // of the invocation of `get_data`. Ownership of the Array is passed to the consumer
+ // calling this, and so it must be released separately.
+ //
+ // It is only valid to call this method exactly once.
+ int (*get_data)(struct ArrowArrayTask* self, struct ArrowDeviceArray* out);
+
+ // opaque task-specific data
+ void* private_data;
+};
+
+// ArrowAsyncProducer represents a 1-to-1 relationship between an async producer
+// and consumer. This object allows the consumer to perform backpressure and flow
+// control on the asynchronous stream processing. This object must be owned by the
+// producer who creates it, and thus is responsible for cleaning it up.
+struct ArrowAsyncProducer {
+ // A consumer must call this function to start receiving on_next_task calls.
+ //
+ // It *must* be valid to call this synchronously from within `on_next_task` or
+ // `on_schema`, but this function *must not* immediately call `on_next_task` so as
+ // to avoid recursion and reentrant callbacks.
+ //
+ // After cancel has been called, additional calls to this function must be NOPs,
+ // but allowed. While not cancelled, calling this function must register the
+ // given number of additional arrays/batches to be produced with the producer.
+ // The producer should only call `on_next_task` at most the registered number
+ // of arrays before propagating backpressure.
Review Comment:
It would be pretty easy to toss a call to request inside the branch that calls `get_data`, i'll add it there.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-43631: [C][Format] Add ArrowAsyncDeviceStreamHandler interface [arrow]
Posted by "pitrou (via GitHub)" <gi...@apache.org>.
pitrou commented on code in PR #43632:
URL: https://github.com/apache/arrow/pull/43632#discussion_r1764760835
##########
docs/source/format/CDeviceDataInterface.rst:
##########
@@ -650,6 +652,331 @@ The stream source is not assumed to be thread-safe. Consumers wanting to
call ``get_next`` from several threads should ensure those calls are
serialized.
+Async Device Stream Interface
+=============================
+
+The :ref:`C stream interface <_c-device-stream-interface>` provides a synchronous
+API centered around the consumer calling the callback functions to retrieve
+the next record batch. For some bindings, use cases, and interoperability, a more
+asynchronous, producer-focused interface may be required. These scenarios can utilize
+the ``ArrowAsyncDeviceStreamHandler``.
+
+Semantics
+---------
+
+Rather than the producer providing a structure of callbacks for a consumer to
+call and retrieve records, the Async interface is a consumer allocated structure.
+The consumer allocated struct provides handler callbacks for the producer to call
+when the schema and chunks of data are available, rather than the consumer using
+a blocking pull-style iteration.
+
+In addition to the ``ArrowAsyncDeviceStreamHandler``, there are also two additional
+structs used for the full data flow: ``ArrowAsyncTask`` and ``ArrowAsyncProducer``.
+
+Structure Definition
+--------------------
+
+The C device async stream interface is defined with a single ``struct`` definition:
+
+.. code-block:: c
+
+ #ifndef ARROW_C_ASYNC_STREAM_INTERFACE
+ #define ARROW_C_ASYNC_STREAM_INTERFACE
+
+ struct ArrowAsyncTask {
+ int (*get_data)(struct ArrowArrayTask* self, struct ArrowDeviceArray* out);
+
+ void* private_data;
+ };
+
+ struct ArrowAsyncProducer {
+ void (*request)(struct ArrowAsyncProducer* self, uint64_t n);
+ void (*cancel)(struct ArrowAsyncProducer* self);
+
+ void (*release)(struct ArrowAsyncProducer* self);
+ void* private_data;
+ };
+
+ struct ArrowAsyncDeviceStreamHandler {
+ // handlers
+ int (*on_schema)(struct ArrowAsyncDeviceStreamHandler* self,
+ struct ArrowAsyncProducer* producer,
+ struct ArrowSchema* stream_schema, const char* addl_metadata);
+ int (*on_next_task)(struct ArrowAsyncDeviceStreamHandler* self,
+ struct ArrowAsyncTask* task, const char* metadata);
+ void (*on_error)(struct ArrowAsyncDeviceStreamHandler* self,
+ int code, const char* message, const char* metadata);
+
+ // release callback
+ void (*release)(struct ArrowAsyncDeviceStreamHandler* self);
+
+ // opaque handler-specific data
+ void* private_data;
+ };
+
+ #endif // ARROW_C_ASYNC_STREAM_INTERFACE
+
+.. note::
+ The canonical guard ``ARROW_C_ASYNC_STREAM_INTERFACE`` is meant to avoid
+ duplicate definitions if two projects copy the C async stream interface
+ definitions into their own headers, and a third-party project includes
+ from these two projects. It is therefore important that this guard is kept
+ exactly as-is when these definitions are copied.
+
+The ArrowAsyncDeviceStreamHandler structure
+'''''''''''''''''''''''''''''''''''''''''''
+
+The structure has the following fields:
+
+.. c:member:: int (*ArrowAsyncDeviceStreamHandler.on_schema)(struct ArrowAsyncDeviceStreamHandler*, struct ArrowAsyncProducer*, struct ArrowSchema*, const char*)
+
+ *Mandatory.* Handler for receiving the schema of the stream. All records should
+ match the provided schema. If successful, the function should return 0, otherwise
+ it should return an ``errno``-compatible error code.
+
+ The ``const char*`` parameter exists for producers to provide any extra contextual information
+ they want, such as the total number of rows in the stream, statistics, or otherwise. This is
+ encoded in the same format as :c:member:`ArrowSchema.metadata`. If not ``NULL``,
+ the lifetime is only the scope of the call to this function. A consumer who wants to maintain
+ the additional metadata beyond the lifetime of this call *MUST* copy the value themselves.
+
+ Unless the ``on_error`` handler is called, this will always get called exactly once and will be
+ the first method called on this object. As such the producer *MUST* provide an ``ArrowAsyncProducer``
+ object when calling this function to allow the consumer to manage back-pressure and flow control.
+ The producer maintains ownership of the ``ArrowAsyncProducer`` and must clean it up before or after
+ calling the release callback on this object.
+
+ A producer that receives a non-zero result here must not subsequently call anything other than
+ the release callback on this object.
+
+.. c:member:: int (*ArrowAsyncDeviceStreamHandler.on_next_task)(struct ArrowAsyncDeviceStreamHandler*, struct ArrowAsyncTask*, const char*)
+
+ *Mandatory.* Handler to be called when a new record is available for processing. The
+ schema for each record should be the same as the schema that ``on_schema`` was called with.
+ If successfully handled, the function should return 0, otherwise it should return an
+ ``errno``-compatible error code.
+
+ Rather than passing the record itself it receives an ``ArrowAsyncTask`` instead to facilitate
+ better consumer-focused thread control as far as receiving the data. A call to this function
+ simply indicates that data is available via the provided task.
+
+ The ``const char*`` parameter exists for producers to provide any extra contextual information
+ they want. This is encoded in the same format as :c:member:`ArrowSchema.metadata`. If not ``NULL``,
+ the lifetime is only the scope of the call to this function. A consumer who wants to maintain
+ the additional metadata beyond the lifetime of this call *MUST* copy the value themselves.
+
+ A producer *MUST NOT* call this concurrently from multiple different threads.
+
+ The :c:member:`ArrowAsyncProducer.request` callback must be called to start receiving calls to this
+ handler.
+
+ This function *MUST* be able to be called re-entrantly on the same thread to allow for the
+ :c:member:`ArrowAsyncProducer.request` callback to potentially call this method recursively.
+
+.. c:member:: void (*ArrowAsyncDeviceStreamHandler.on_error)(struct ArrowAsyncDeviceStreamHandler, int, const char*, const char*)
+
+ *Mandatory.* Handler to be called when an error is encountered by the producer. After calling
+ this, the ``release`` callback will be called as the last call on this struct. The parameters
+ are an ``errno``-compatible error code and an optional error message and metadata.
+
+ If the message and metadata are not ``NULL``, their lifetime is only valid during the scope
+ of this call. A consumer who wants to maintain these values past the return of this function
+ *MUST* copy the values themselves.
+
+ If the metadata parameter is not ``NULL``, to provide key-value error metadata, then it should
+ be encoded identically to the way that metadata is encoded in :c:member:`ArrowSchema.metadata`.
+
+ It is valid for this to be called by a producer with or without a preceding call to
+ :c:member:`ArrowAsyncProducer.request`. This callback *MUST NOT* call any methods of an
+ ``ArrowAsyncProducer`` object.
+
+.. c:member:: void (*ArrowAsyncDeviceStreamHandler.release)(struct ArrowAsyncDeviceStreamHandler*)
+
+ *Mandatory.* A pointer to a consumer-provided release callback for the handler.
+
+ It is valid for this to be called by a producer with or without a preceding call to
+ :c:member:`ArrowAsyncProducer.request`. This must not call any methods of an ``ArrowAsyncProducer``
+ object.
+
+.. c:member:: void* ArrowAsyncDeviceStreamHandler.private_data
+
+ *Optional.* An opaque pointer to consumer-provided private data.
+
+ Producers *MUST NOT* process this member. Lifetime of this member is handled by
+ the consumer, and especially by the release callback.
+
+The ArrowAsyncTask structure
+''''''''''''''''''''''''''''
+
+The purpose of using a Task object rather than passing the array directly to the ``on_next``
+callback is to allow for more complex and efficient thread handling. Utilizing a Task
+object allows for a producer to separate the "decoding" logic from the I/O, enabling a
+consumer to avoid transferring data between CPU cores (e.g. from one L1/L2 cache to another).
+
+This producer-provided structure has the following fields:
+
+.. c:member:: int (*ArrowArrayTask.get_data)(struct ArrowArrayTask*, struct ArrowDeviceArray*)
+
+ *Mandatory.* A callback to populate the provided ``ArrowDeviceArray`` with the available data.
+ The order of ArrowAsyncTasks provided by the producer enables a consumer to know the order of
+ the data to process.
+
+ If a non-zero value is returned from this, it should be followed only by the producer calling
+ the ``on_error`` callback of the ``ArrowAsyncDeviceStreamHandler``. Because calling this method
+ is likely to be separate from the current control flow, returning a non-zero value to signal
+ an error occuring allows the current thread to decide handle the case accordingly, while still
+ allowing all error logging and handling to be centralized in the
+ :c:member:`ArrowAsyncDeviceStreamHandler.on_error` callback.
+
+ Rather than having a separate release callback, any required cleanup should be performed as part
+ of the invocation of this callback, with ownership of the Array being passed to the consumer-provided
+ parameter calling this, and must be released separately.
+
+ It is only valid to call this method exactly once.
+
+.. c:member:: void* ArrowArrayTask.private_data
+
+ *Optional.* An opaque pointer to consumer-provided private data.
+
+ Consumers *MUST NOT* process this member. Lifetime of this member is handled by
+ the producer who created this object, and should be cleaned up if necessary during
+ the call to :c:member:`ArrowArrayTask.get_data`.
+
+The ArrowAsyncProducer structure
+''''''''''''''''''''''''''''''''
+
+This producer-provided and managed object has the following fields:
+
+.. c:member:: void (*ArrowAsyncProducer.request)(struct ArrowAsyncProducer*, uint64_t)
Review Comment:
For consistency with our usual conventions. Also because AFAIU Java doesn't have unsigned integers.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-43631: [C][Format] Add ArrowAsyncDeviceStreamHandler interface [arrow]
Posted by "zeroshade (via GitHub)" <gi...@apache.org>.
zeroshade commented on code in PR #43632:
URL: https://github.com/apache/arrow/pull/43632#discussion_r1765319156
##########
docs/source/format/CDeviceDataInterface.rst:
##########
@@ -650,6 +652,331 @@ The stream source is not assumed to be thread-safe. Consumers wanting to
call ``get_next`` from several threads should ensure those calls are
serialized.
+Async Device Stream Interface
+=============================
+
+The :ref:`C stream interface <_c-device-stream-interface>` provides a synchronous
+API centered around the consumer calling the callback functions to retrieve
+the next record batch. For some bindings, use cases, and interoperability, a more
+asynchronous, producer-focused interface may be required. These scenarios can utilize
+the ``ArrowAsyncDeviceStreamHandler``.
+
+Semantics
+---------
+
+Rather than the producer providing a structure of callbacks for a consumer to
+call and retrieve records, the Async interface is a consumer allocated structure.
+The consumer allocated struct provides handler callbacks for the producer to call
+when the schema and chunks of data are available, rather than the consumer using
+a blocking pull-style iteration.
+
+In addition to the ``ArrowAsyncDeviceStreamHandler``, there are also two additional
+structs used for the full data flow: ``ArrowAsyncTask`` and ``ArrowAsyncProducer``.
+
+Structure Definition
+--------------------
+
+The C device async stream interface is defined with a single ``struct`` definition:
+
+.. code-block:: c
+
+ #ifndef ARROW_C_ASYNC_STREAM_INTERFACE
+ #define ARROW_C_ASYNC_STREAM_INTERFACE
+
+ struct ArrowAsyncTask {
+ int (*get_data)(struct ArrowArrayTask* self, struct ArrowDeviceArray* out);
+
+ void* private_data;
+ };
+
+ struct ArrowAsyncProducer {
+ void (*request)(struct ArrowAsyncProducer* self, uint64_t n);
+ void (*cancel)(struct ArrowAsyncProducer* self);
+
+ void (*release)(struct ArrowAsyncProducer* self);
+ void* private_data;
+ };
+
+ struct ArrowAsyncDeviceStreamHandler {
+ // handlers
+ int (*on_schema)(struct ArrowAsyncDeviceStreamHandler* self,
+ struct ArrowAsyncProducer* producer,
+ struct ArrowSchema* stream_schema, const char* addl_metadata);
+ int (*on_next_task)(struct ArrowAsyncDeviceStreamHandler* self,
+ struct ArrowAsyncTask* task, const char* metadata);
+ void (*on_error)(struct ArrowAsyncDeviceStreamHandler* self,
+ int code, const char* message, const char* metadata);
+
+ // release callback
+ void (*release)(struct ArrowAsyncDeviceStreamHandler* self);
+
+ // opaque handler-specific data
+ void* private_data;
+ };
+
+ #endif // ARROW_C_ASYNC_STREAM_INTERFACE
+
+.. note::
+ The canonical guard ``ARROW_C_ASYNC_STREAM_INTERFACE`` is meant to avoid
+ duplicate definitions if two projects copy the C async stream interface
+ definitions into their own headers, and a third-party project includes
+ from these two projects. It is therefore important that this guard is kept
+ exactly as-is when these definitions are copied.
+
+The ArrowAsyncDeviceStreamHandler structure
+'''''''''''''''''''''''''''''''''''''''''''
+
+The structure has the following fields:
+
+.. c:member:: int (*ArrowAsyncDeviceStreamHandler.on_schema)(struct ArrowAsyncDeviceStreamHandler*, struct ArrowAsyncProducer*, struct ArrowSchema*, const char*)
+
+ *Mandatory.* Handler for receiving the schema of the stream. All records should
+ match the provided schema. If successful, the function should return 0, otherwise
+ it should return an ``errno``-compatible error code.
+
+ The ``const char*`` parameter exists for producers to provide any extra contextual information
+ they want, such as the total number of rows in the stream, statistics, or otherwise. This is
+ encoded in the same format as :c:member:`ArrowSchema.metadata`. If not ``NULL``,
+ the lifetime is only the scope of the call to this function. A consumer who wants to maintain
+ the additional metadata beyond the lifetime of this call *MUST* copy the value themselves.
+
+ Unless the ``on_error`` handler is called, this will always get called exactly once and will be
+ the first method called on this object. As such the producer *MUST* provide an ``ArrowAsyncProducer``
+ object when calling this function to allow the consumer to manage back-pressure and flow control.
+ The producer maintains ownership of the ``ArrowAsyncProducer`` and must clean it up before or after
+ calling the release callback on this object.
+
+ A producer that receives a non-zero result here must not subsequently call anything other than
+ the release callback on this object.
+
+.. c:member:: int (*ArrowAsyncDeviceStreamHandler.on_next_task)(struct ArrowAsyncDeviceStreamHandler*, struct ArrowAsyncTask*, const char*)
+
+ *Mandatory.* Handler to be called when a new record is available for processing. The
+ schema for each record should be the same as the schema that ``on_schema`` was called with.
+ If successfully handled, the function should return 0, otherwise it should return an
+ ``errno``-compatible error code.
+
+ Rather than passing the record itself it receives an ``ArrowAsyncTask`` instead to facilitate
+ better consumer-focused thread control as far as receiving the data. A call to this function
+ simply indicates that data is available via the provided task.
+
+ The ``const char*`` parameter exists for producers to provide any extra contextual information
+ they want. This is encoded in the same format as :c:member:`ArrowSchema.metadata`. If not ``NULL``,
+ the lifetime is only the scope of the call to this function. A consumer who wants to maintain
+ the additional metadata beyond the lifetime of this call *MUST* copy the value themselves.
+
+ A producer *MUST NOT* call this concurrently from multiple different threads.
+
+ The :c:member:`ArrowAsyncProducer.request` callback must be called to start receiving calls to this
+ handler.
+
+ This function *MUST* be able to be called re-entrantly on the same thread to allow for the
+ :c:member:`ArrowAsyncProducer.request` callback to potentially call this method recursively.
+
+.. c:member:: void (*ArrowAsyncDeviceStreamHandler.on_error)(struct ArrowAsyncDeviceStreamHandler, int, const char*, const char*)
+
+ *Mandatory.* Handler to be called when an error is encountered by the producer. After calling
+ this, the ``release`` callback will be called as the last call on this struct. The parameters
+ are an ``errno``-compatible error code and an optional error message and metadata.
+
+ If the message and metadata are not ``NULL``, their lifetime is only valid during the scope
+ of this call. A consumer who wants to maintain these values past the return of this function
+ *MUST* copy the values themselves.
+
+ If the metadata parameter is not ``NULL``, to provide key-value error metadata, then it should
+ be encoded identically to the way that metadata is encoded in :c:member:`ArrowSchema.metadata`.
+
+ It is valid for this to be called by a producer with or without a preceding call to
+ :c:member:`ArrowAsyncProducer.request`. This callback *MUST NOT* call any methods of an
+ ``ArrowAsyncProducer`` object.
+
+.. c:member:: void (*ArrowAsyncDeviceStreamHandler.release)(struct ArrowAsyncDeviceStreamHandler*)
+
+ *Mandatory.* A pointer to a consumer-provided release callback for the handler.
+
+ It is valid for this to be called by a producer with or without a preceding call to
+ :c:member:`ArrowAsyncProducer.request`. This must not call any methods of an ``ArrowAsyncProducer``
+ object.
+
+.. c:member:: void* ArrowAsyncDeviceStreamHandler.private_data
+
+ *Optional.* An opaque pointer to consumer-provided private data.
+
+ Producers *MUST NOT* process this member. Lifetime of this member is handled by
+ the consumer, and especially by the release callback.
+
+The ArrowAsyncTask structure
+''''''''''''''''''''''''''''
+
+The purpose of using a Task object rather than passing the array directly to the ``on_next``
+callback is to allow for more complex and efficient thread handling. Utilizing a Task
+object allows for a producer to separate the "decoding" logic from the I/O, enabling a
+consumer to avoid transferring data between CPU cores (e.g. from one L1/L2 cache to another).
+
+This producer-provided structure has the following fields:
+
+.. c:member:: int (*ArrowArrayTask.get_data)(struct ArrowArrayTask*, struct ArrowDeviceArray*)
Review Comment:
It's as cheap as possible but it is allowed to leave a potential decoding / decompressing step for this method as long as the method is not io-blocking. This allows offloading a decoding/decompressing step to the consumer threads if desired
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-43631: [C][Format] Add ArrowAsyncDeviceStreamHandler interface [arrow]
Posted by "felipecrv (via GitHub)" <gi...@apache.org>.
felipecrv commented on code in PR #43632:
URL: https://github.com/apache/arrow/pull/43632#discussion_r1767797217
##########
docs/source/format/CDeviceDataInterface.rst:
##########
@@ -650,6 +652,340 @@ The stream source is not assumed to be thread-safe. Consumers wanting to
call ``get_next`` from several threads should ensure those calls are
serialized.
+Async Device Stream Interface
+=============================
+
+The :ref:`C stream interface <_c-device-stream-interface>` provides a synchronous
+API centered around the consumer calling the callback functions to retrieve
+the next record batch. For some bindings, use cases, and interoperability, a more
+asynchronous, producer-focused interface may be required. These scenarios can utilize
+the ``ArrowAsyncDeviceStreamHandler``.
+
+Semantics
+---------
+
+Rather than the producer providing a structure of callbacks for a consumer to
+call and retrieve records, the Async interface is a consumer allocated structure.
+The consumer allocated struct provides handler callbacks for the producer to call
+when the schema and chunks of data are available, rather than the consumer using
+a blocking pull-style iteration.
+
+In addition to the ``ArrowAsyncDeviceStreamHandler``, there are also two additional
+structs used for the full data flow: ``ArrowAsyncTask`` and ``ArrowAsyncProducer``.
+
+Structure Definition
+--------------------
+
+The C device async stream interface is defined with a single ``struct`` definition:
+
+.. code-block:: c
+
+ #ifndef ARROW_C_ASYNC_STREAM_INTERFACE
+ #define ARROW_C_ASYNC_STREAM_INTERFACE
+
+ struct ArrowAsyncTask {
+ int (*get_data)(struct ArrowArrayTask* self, struct ArrowDeviceArray* out);
+
+ void* private_data;
+ };
+
+ struct ArrowAsyncProducer {
+ void (*request)(struct ArrowAsyncProducer* self, int64_t n);
+ void (*cancel)(struct ArrowAsyncProducer* self);
+
+ void (*release)(struct ArrowAsyncProducer* self);
+ void* private_data;
+ };
+
+ struct ArrowAsyncDeviceStreamHandler {
+ // handlers
+ int (*on_schema)(struct ArrowAsyncDeviceStreamHandler* self,
+ struct ArrowAsyncProducer* producer,
+ struct ArrowSchema* stream_schema, const char* addl_metadata);
+ int (*on_next_task)(struct ArrowAsyncDeviceStreamHandler* self,
+ struct ArrowAsyncTask* task, const char* metadata);
+ void (*on_error)(struct ArrowAsyncDeviceStreamHandler* self,
+ int code, const char* message, const char* metadata);
+
+ // release callback
+ void (*release)(struct ArrowAsyncDeviceStreamHandler* self);
+
+ // opaque handler-specific data
+ void* private_data;
+ };
+
+ #endif // ARROW_C_ASYNC_STREAM_INTERFACE
+
+.. note::
+ The canonical guard ``ARROW_C_ASYNC_STREAM_INTERFACE`` is meant to avoid
+ duplicate definitions if two projects copy the C async stream interface
+ definitions into their own headers, and a third-party project includes
+ from these two projects. It is therefore important that this guard is kept
+ exactly as-is when these definitions are copied.
+
+The ArrowAsyncDeviceStreamHandler structure
+'''''''''''''''''''''''''''''''''''''''''''
+
+The structure has the following fields:
+
+.. c:member:: int (*ArrowAsyncDeviceStreamHandler.on_schema)(struct ArrowAsyncDeviceStreamHandler*, struct ArrowAsyncProducer*, struct ArrowSchema*, const char*)
+
+ *Mandatory.* Handler for receiving the schema of the stream. All records should
+ match the provided schema. If successful, the function should return 0, otherwise
+ it should return an ``errno``-compatible error code.
+
+ The ``const char*`` parameter exists for producers to provide any extra contextual information
+ they want, such as the total number of rows in the stream, statistics, or otherwise. This is
+ encoded in the same format as :c:member:`ArrowSchema.metadata`. If not ``NULL``,
+ the lifetime is only the scope of the call to this function. A consumer who wants to maintain
+ the additional metadata beyond the lifetime of this call *MUST* copy the value themselves.
+
+ Unless the ``on_error`` handler is called, this will always get called exactly once and will be
+ the first method called on this object. As such the producer *MUST* provide an ``ArrowAsyncProducer``
+ object when calling this function to allow the consumer to manage back-pressure and flow control.
+ The producer maintains ownership of the ``ArrowAsyncProducer`` and must clean it up *after*
+ calling the release callback on this object.
+
+ A producer that receives a non-zero result here must not subsequently call anything other than
+ the release callback on this object.
+
+.. c:member:: int (*ArrowAsyncDeviceStreamHandler.on_next_task)(struct ArrowAsyncDeviceStreamHandler*, struct ArrowAsyncTask*, const char*)
+
+ *Mandatory.* Handler to be called when a new record is available for processing. The
+ schema for each record should be the same as the schema that ``on_schema`` was called with.
+ If successfully handled, the function should return 0, otherwise it should return an
+ ``errno``-compatible error code.
+
+ Rather than passing the record itself it receives an ``ArrowAsyncTask`` instead to facilitate
+ better consumer-focused thread control as far as receiving the data. A call to this function
+ simply indicates that data is available via the provided task.
+
+ The producer signals the end of the stream by passing ``NULL`` for the ``ArrowAsyncTask``
+ pointer instead of a valid address. This task object is only valid during the lifetime of
+ this function call. If the consumer wants to use the task beyond the scope of this method, it
+ must copy or move its contents to a new ArrowAsyncTask object.
Review Comment:
`extract_data` works as well and has the nice "decompression" connotation. But what does @bkietz think about `consume_data`?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-43631: [C][Format] Add ArrowAsyncDeviceStreamHandler interface [arrow]
Posted by "bkietz (via GitHub)" <gi...@apache.org>.
bkietz commented on code in PR #43632:
URL: https://github.com/apache/arrow/pull/43632#discussion_r1816974645
##########
cpp/src/arrow/c/abi.h:
##########
@@ -356,8 +356,7 @@ struct ArrowAsyncDeviceStreamHandler {
// A producer that receives a non-zero return here should stop producing and eventually
// call release instead.
int (*on_schema)(struct ArrowAsyncDeviceStreamHandler* self,
- struct ArrowAsyncProducer* producer, struct ArrowSchema* stream_schema,
- const char* addl_metadata);
+ struct ArrowSchema* stream_schema, const char* addl_metadata);
Review Comment:
If you already responded elsewhere then sorry for the noise, but I can't find my own first comment on this: we could remove the ambiguity of what this metadata pertains to by making it a member of `ArrowAsyncProducer`. This would simplify this signature and remove the need to clarify `addl_metadata`'s lifetime
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-43631: [C][Format] Add ArrowAsyncDeviceStreamHandler interface [arrow]
Posted by "leventov (via GitHub)" <gi...@apache.org>.
leventov commented on PR #43632:
URL: https://github.com/apache/arrow/pull/43632#issuecomment-2337461521
### Velox
Disclaimer: it's my an attempt to research how control flow works in Velox without knowing anything about its code organisation beforehand.
See [Task doc](https://github.com/facebookincubator/velox/blob/24dd2e9382f74389ae01968a1350a2d01bdb6785/velox/docs/develop/task.rst#splits), [Task.h](https://github.com/facebookincubator/velox/blob/24dd2e9382f74389ae01968a1350a2d01bdb6785/velox/exec/Task.h). What we are calling "task" is called `Split` in Velox, and `Task` is more equivalent to ClickHouse's `IProcessor`.
`Task` is more like a data structure exchange (with queue management inside) than flow control interface. I think the reason is because it's easier to do cross-language control flow with this, as Presto and Hive (Java) should be able to "hand over" the control flow to Velox and back.
Async flow control is shared between [`ExchangeSource`](https://github.com/facebookincubator/velox/blob/24dd2e9382f74389ae01968a1350a2d01bdb6785/velox/exec/ExchangeSource.h#L23) (controls the remote data producer): see `request()` and `close()` methods. However, `ExchangeSource` just adds "pages" to a queue, doesn't call the consumer directly. `Response` struct can indicate that the remote source has ended. [`ExchangeClient`](https://github.com/facebookincubator/velox/blob/24dd2e9382f74389ae01968a1350a2d01bdb6785/velox/exec/ExchangeClient.h) is driven by the consumer, where `next()` takes the "pages" from the queue, and can propagate `close()` to the ExchangeSource. Again, this seems to be driven by Velox's requirement for cross-language flow control.
cc @pedroerp to check I'm messing up something.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-43631: [C][Format] Add ArrowAsyncDeviceStreamHandler interface [arrow]
Posted by "zeroshade (via GitHub)" <gi...@apache.org>.
zeroshade commented on code in PR #43632:
URL: https://github.com/apache/arrow/pull/43632#discussion_r1750442503
##########
cpp/src/arrow/c/abi.h:
##########
@@ -228,6 +228,180 @@ struct ArrowDeviceArrayStream {
#endif // ARROW_C_DEVICE_STREAM_INTERFACE
+#ifndef ARROW_C_ASYNC_STREAM_INTERFACE
+#define ARROW_C_ASYNC_STREAM_INTERFACE
+
+// ArrowAsyncTask represents available data from a producer that was passed to
+// an invocation of `on_next_task` on the ArrowAsyncDeviceStreamHandler.
+struct ArrowAsyncTask {
+ // This callback should populate the ArrowDeviceArray associated with this task.
+ // The order of ArrowAsyncTasks provided by the producer enables a consumer to
+ // ensure the order of data to process.
+ //
+ // Returns: 0 if successful, errno-compatible error otherwise.
+ //
+ // If a non-0 value is returned then it should be followed by a call to `on_error`
+ // on the appropriate ArrowAsyncDeviceStreamHandler.
+ //
+ // Rather than a release callback, any required cleanup should be performed as part
+ // of the invocation of `get_data`. Ownership of the Array is passed to the consumer
+ // calling this, and so it must be released separately.
+ //
+ // It is only valid to call this method exactly once.
+ int (*get_data)(struct ArrowArrayTask* self, struct ArrowDeviceArray* out);
+
+ // opaque task-specific data
+ void* private_data;
+};
+
+// ArrowAsyncProducer represents a 1-to-1 relationship between an async producer
+// and consumer. This object allows the consumer to perform backpressure and flow
+// control on the asynchronous stream processing. This object must be owned by the
+// producer who creates it, and thus is responsible for cleaning it up.
+struct ArrowAsyncProducer {
+ // A consumer must call this function to start receiving on_next_task calls.
+ //
+ // It *must* be valid to call this synchronously from within on_next_task or
+ // on_schema, with a producer placing an upper bound on possible synchronous
+ // recursion between calls of on_next_task -> request -> on_next_task ->....
Review Comment:
Yea, it's likely a thread-local. The reason for the need to be able to immediately recurse is to be able to leverage `on_next_task` to add new requests to the demand / queue for flow control. For example, having a simple internal queue that can call `request(1)` after `on_next_task` processes the task.
While it places a burden on the producer to prevent stack overflow due to recursion, it makes it significantly easier for the consumer who doesn't need to manage the `request` call having to not be synchronous with the `on_next_task` call. Disallowing the recursion as a whole makes it even more difficult on the producer than having to use a thread-local I believe. Allowing the recursion makes implementing `request` easier for a producer, aside from the thread-local to prevent stack overflow, it makes implementing `request` much simpler.
That said, that's just my opinion. I'd be curious what others think and have no problem changing this if people generally agree with you on this.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-43631: [C][Format] Add ArrowAsyncDeviceStreamHandler interface [arrow]
Posted by "zeroshade (via GitHub)" <gi...@apache.org>.
zeroshade commented on code in PR #43632:
URL: https://github.com/apache/arrow/pull/43632#discussion_r1771653564
##########
cpp/src/arrow/c/abi.h:
##########
@@ -228,6 +228,207 @@ struct ArrowDeviceArrayStream {
#endif // ARROW_C_DEVICE_STREAM_INTERFACE
+#ifndef ARROW_C_ASYNC_STREAM_INTERFACE
+#define ARROW_C_ASYNC_STREAM_INTERFACE
+
+// ArrowAsyncTask represents available data from a producer that was passed to
+// an invocation of `on_next_task` on the ArrowAsyncDeviceStreamHandler.
+//
+// The reason for this Task approach instead of the Async interface returning
+// the Array directly is to allow for more complex thread handling and reducing
+// context switching and data transfers between CPU cores (e.g. from one L1/L2
+// cache to another) if desired.
+//
+// For example, the `on_next_task` callback can be called when data is ready, while
+// the producer puts potential "decoding" logic in the `ArrowAsyncTask` object. This
+// allows for the producer to manage the I/O on one thread which calls `on_next_task`
+// and the consumer can determine when the decoding (producer logic in the `get_data`
+// callback of the task) occurs and on which thread, to avoid a CPU core transfer
+// (data staying in the L2 cache).
+struct ArrowAsyncTask {
+ // This callback should populate the ArrowDeviceArray associated with this task.
+ // The order of ArrowAsyncTasks provided by the producer enables a consumer to
+ // ensure the order of data to process.
+ //
+ // This function is expected to be synchronous, but should not perform any blocking
+ // I/O. Ideally it should be as cheap as possible so as to not tie up the consumer
+ // thread unnecessarily.
+ //
+ // Returns: 0 if successful, errno-compatible error otherwise.
+ //
+ // If a non-0 value is returned then it should be followed by a call to `on_error`
+ // on the appropriate ArrowAsyncDeviceStreamHandler. This is because the it's highly
+ // likely that whatever is calling this function may be entirely disconnected from
+ // the current control flow. Indicating an error here with a non-zero return allows
+ // the current flow to be aware of the error occurring, while still allowing any
+ // logging or error handling to still be centralized in the `on_error` callback of
+ // the original Async handler.
+ //
+ // Rather than a release callback, any required cleanup should be performed as part
+ // of the invocation of `get_data`. Ownership of the Array is passed to the consumer
+ // calling this, and so it must be released separately.
+ //
+ // It is only valid to call this method exactly once.
+ int (*get_data)(struct ArrowArrayTask* self, struct ArrowDeviceArray* out);
+
+ // opaque task-specific data
+ void* private_data;
+};
+
+// ArrowAsyncProducer represents a 1-to-1 relationship between an async producer
+// and consumer. This object allows the consumer to perform backpressure and flow
+// control on the asynchronous stream processing. This object must be owned by the
+// producer who creates it, and thus is responsible for cleaning it up.
+struct ArrowAsyncProducer {
+ // A consumer must call this function to start receiving on_next_task calls.
+ //
+ // It *must* be valid to call this synchronously from within `on_next_task` or
+ // `on_schema`, but this function *must not* immediately call `on_next_task` so as
+ // to avoid recursion and reentrant callbacks.
+ //
+ // After cancel has been called, additional calls to this function must be NOPs,
+ // but allowed. While not cancelled, calling this function must register the
+ // given number of additional arrays/batches to be produced with the producer.
+ // The producer should only call `on_next_task` at most the registered number
+ // of arrays before propagating backpressure.
+ //
+ // Any error encountered by calling request must be propagated by calling the `on_error`
+ // callback of the ArrowAsyncDeviceStreamHandler.
+ //
+ // While not cancelled, any subsequent calls to `on_next_task`, `on_error` or
+ // `release` should be scheduled by the producer to be called later rather
+ //
+ // It is invalid for a consumer to call this with a value of n <= 0, producers should
+ // error if given such a value.
+ void (*request)(struct ArrowAsyncProducer* self, int64_t n);
+
+ // This cancel callback signals a producer that it must eventually stop making calls
+ // to on_next_task. It must be idempotent and thread-safe. After calling cancel once,
+ // subsequent calls must be NOPs. This must not call any consumer-side handlers other
+ // than `on_error`.
+ //
+ // It is not required that calling cancel affect the producer immediately, only that it
+ // must eventually stop calling on_next_task and subsequently call release on the
+ // async handler. As such, a consumer must be prepared to receive one or more calls to
+ // `on_next_task` even after calling cancel if there are still requested arrays pending.
+ //
+ // Successful cancellation should *not* result in the producer calling `on_error`, it
+ // should finish out any remaining tasks and eventually call `release`.
+ //
+ // Any error encountered during handling a call to cancel must be reported via the
+ // on_error callback on the async stream handler.
+ void (*cancel)(struct ArrowAsyncProducer* self);
+
+ // producer-specific opaque data.
+ void* private_data;
+};
+
+// Similar to ArrowDeviceArrayStream, except designed for an asynchronous
+// style of interaction. While ArrowDeviceArrayStream provides producer
+// defined callbacks, this is intended to be created by the consumer instead.
+// The consumer passes this handler to the producer, which in turn uses the
+// callbacks to inform the consumer of events in the stream.
+struct ArrowAsyncDeviceStreamHandler {
+ // Handler for receiving a schema. The passed in stream_schema must be
+ // released or moved by the handler (producer is giving ownership of the schema to
+ // the handler, but not ownership of the top level object itself).
+ //
+ // With the exception of an error occurring (on_error), this must be the first
+ // callback function which is called by a producer and must only be called exactly
+ // once. As such, the producer should provide a valid ArrowAsyncProducer instance
+ // so the consumer can control the flow. See the documentation on ArrowAsyncProducer
+ // for how it works. The ArrowAsyncProducer is owned by the producer who calls this
+ // function and thus the producer is responsible for cleaning it up when calling
+ // the release callback of this handler.
+ //
+ // The addl_metadata argument can be null or can be used by a producer
+ // to pass arbitrary extra information to the consumer beyond the metadata in the schema
+ // itself (such as total number of rows, context info, or otherwise). The data should
+ // be passed using the same encoding as the metadata within the ArrowSchema struct
+ // itself (defined in the spec at
+ // https://arrow.apache.org/docs/format/CDataInterface.html#c.ArrowSchema.metadata)
+ //
+ // If addl_metadata is non-null then it only needs to exist for the lifetime of this
+ // call, a consumer who wants it to live after that must copy it to ensure lifetime.
+ //
+ // Return value: 0 if successful, `errno`-compatible error otherwise
+ //
+ // A producer that receives a non-zero return here should stop producing and eventually
+ // call release instead.
+ int (*on_schema)(struct ArrowAsyncDeviceStreamHandler* self,
+ struct ArrowAsyncProducer* producer, struct ArrowSchema* stream_schema,
+ const char* addl_metadata);
Review Comment:
I added the `addl_` prefix here to indicate that it is separate from the metadata that would exist in the `ArrowSchema` object. I didn't want anyone to start throwing metadata in here that should live in the schema itself. But if we don't think that's a concern I can rename it
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-43631: [C][Format] Add ArrowAsyncDeviceStreamHandler interface [arrow]
Posted by "pitrou (via GitHub)" <gi...@apache.org>.
pitrou commented on code in PR #43632:
URL: https://github.com/apache/arrow/pull/43632#discussion_r1757469760
##########
docs/source/format/CDeviceDataInterface.rst:
##########
@@ -650,6 +652,331 @@ The stream source is not assumed to be thread-safe. Consumers wanting to
call ``get_next`` from several threads should ensure those calls are
serialized.
+Async Device Stream Interface
+=============================
+
+The :ref:`C stream interface <_c-device-stream-interface>` provides a synchronous
+API centered around the consumer calling the callback functions to retrieve
+the next record batch. For some bindings, use cases, and interoperability, a more
+asynchronous, producer-focused interface may be required. These scenarios can utilize
+the ``ArrowAsyncDeviceStreamHandler``.
+
+Semantics
+---------
+
+Rather than the producer providing a structure of callbacks for a consumer to
+call and retrieve records, the Async interface is a consumer allocated structure.
+The consumer allocated struct provides handler callbacks for the producer to call
+when the schema and chunks of data are available, rather than the consumer using
+a blocking pull-style iteration.
+
+In addition to the ``ArrowAsyncDeviceStreamHandler``, there are also two additional
+structs used for the full data flow: ``ArrowAsyncTask`` and ``ArrowAsyncProducer``.
+
+Structure Definition
+--------------------
+
+The C device async stream interface is defined with a single ``struct`` definition:
+
+.. code-block:: c
+
+ #ifndef ARROW_C_ASYNC_STREAM_INTERFACE
+ #define ARROW_C_ASYNC_STREAM_INTERFACE
+
+ struct ArrowAsyncTask {
+ int (*get_data)(struct ArrowArrayTask* self, struct ArrowDeviceArray* out);
+
+ void* private_data;
+ };
+
+ struct ArrowAsyncProducer {
+ void (*request)(struct ArrowAsyncProducer* self, uint64_t n);
+ void (*cancel)(struct ArrowAsyncProducer* self);
+
+ void (*release)(struct ArrowAsyncProducer* self);
+ void* private_data;
+ };
+
+ struct ArrowAsyncDeviceStreamHandler {
+ // handlers
+ int (*on_schema)(struct ArrowAsyncDeviceStreamHandler* self,
+ struct ArrowAsyncProducer* producer,
+ struct ArrowSchema* stream_schema, const char* addl_metadata);
+ int (*on_next_task)(struct ArrowAsyncDeviceStreamHandler* self,
+ struct ArrowAsyncTask* task, const char* metadata);
+ void (*on_error)(struct ArrowAsyncDeviceStreamHandler* self,
+ int code, const char* message, const char* metadata);
+
+ // release callback
+ void (*release)(struct ArrowAsyncDeviceStreamHandler* self);
+
+ // opaque handler-specific data
+ void* private_data;
+ };
+
+ #endif // ARROW_C_ASYNC_STREAM_INTERFACE
+
+.. note::
+ The canonical guard ``ARROW_C_ASYNC_STREAM_INTERFACE`` is meant to avoid
+ duplicate definitions if two projects copy the C async stream interface
+ definitions into their own headers, and a third-party project includes
+ from these two projects. It is therefore important that this guard is kept
+ exactly as-is when these definitions are copied.
+
+The ArrowAsyncDeviceStreamHandler structure
+'''''''''''''''''''''''''''''''''''''''''''
+
+The structure has the following fields:
+
+.. c:member:: int (*ArrowAsyncDeviceStreamHandler.on_schema)(struct ArrowAsyncDeviceStreamHandler*, struct ArrowAsyncProducer*, struct ArrowSchema*, const char*)
+
+ *Mandatory.* Handler for receiving the schema of the stream. All records should
+ match the provided schema. If successful, the function should return 0, otherwise
+ it should return an ``errno``-compatible error code.
+
+ The ``const char*`` parameter exists for producers to provide any extra contextual information
+ they want, such as the total number of rows in the stream, statistics, or otherwise. This is
+ encoded in the same format as :c:member:`ArrowSchema.metadata`. If not ``NULL``,
+ the lifetime is only the scope of the call to this function. A consumer who wants to maintain
+ the additional metadata beyond the lifetime of this call *MUST* copy the value themselves.
+
+ Unless the ``on_error`` handler is called, this will always get called exactly once and will be
+ the first method called on this object. As such the producer *MUST* provide an ``ArrowAsyncProducer``
+ object when calling this function to allow the consumer to manage back-pressure and flow control.
+ The producer maintains ownership of the ``ArrowAsyncProducer`` and must clean it up before or after
+ calling the release callback on this object.
+
+ A producer that receives a non-zero result here must not subsequently call anything other than
+ the release callback on this object.
+
+.. c:member:: int (*ArrowAsyncDeviceStreamHandler.on_next_task)(struct ArrowAsyncDeviceStreamHandler*, struct ArrowAsyncTask*, const char*)
Review Comment:
Why is it called a "task"? It's an array, so perhaps `on_next_array`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-43631: [C][Format] Add ArrowAsyncDeviceStreamHandler interface [arrow]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #43632:
URL: https://github.com/apache/arrow/pull/43632#issuecomment-2278671735
:warning: GitHub issue #43631 **has been automatically assigned in GitHub** to PR creator.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-43631: [C][Format] Add ArrowAsyncDeviceStreamHandler interface [arrow]
Posted by "felipecrv (via GitHub)" <gi...@apache.org>.
felipecrv commented on code in PR #43632:
URL: https://github.com/apache/arrow/pull/43632#discussion_r1716170327
##########
cpp/src/arrow/c/abi.h:
##########
@@ -228,6 +228,65 @@ struct ArrowDeviceArrayStream {
#endif // ARROW_C_DEVICE_STREAM_INTERFACE
+#ifndef ARROW_C_ASYNC_STREAM_INTERFACE
+#define ARROW_C_ASYNC_STREAM_INTERFACE
+
+// Similar to ArrowDeviceArrayStream, except designed for an asynchronous
+// style of interaction. While ArrowDeviceArrayStream provides producer
+// defined callbacks, this is intended to be created by the consumer instead.
+// The consumer passes this handler to the producer, which in turn uses the
+// callbacks to inform the consumer of events in the stream.
+struct ArrowAsyncDeviceStreamHandler {
+ // Handler for receiving a schema. The passed in stream_schema should be
+ // released or moved by the handler (producer is giving ownership of it to
+ // the handler).
+ //
+ // The `extension_param` argument can be null or can be used by a producer
+ // to pass arbitrary extra information to the consumer (such as total number
+ // of rows, context info, or otherwise).
+ //
+ // Return value: 0 if successful, `errno`-compatible error otherwise
+ int (*on_schema)(struct ArrowAsyncDeviceStreamHandler* self,
+ struct ArrowSchema* stream_schema, void* extension_param);
+
+ // Handler for receiving an array/record batch. Always called at least once
+ // unless an error is encountered (which would result in calling on_error).
+ // An empty/released array is passed to indicate the end of the stream if no
+ // errors have been encountered.
+ //
+ // The `extension_param` argument can be null or can be used by a producer
+ // to pass arbitrary extra information to the consumer.
+ //
+ // Return value: 0 if successful, `errno`-compatible error otherwise.
+ int (*on_next)(struct ArrowAsyncDeviceStreamHandler* self,
+ struct ArrowDeviceArray* next, void* extension_param);
Review Comment:
@zeroshade is easy of use really a concern for such a low-level interface? I think limiting potential is worse than making it hard to user directly. Very sophisticated users should be able to implement advanced concurrency patterns (e.g. handling out-of-order pieces of the stream).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-43631: [C][Format] Add ArrowAsyncDeviceStreamHandler interface [arrow]
Posted by "westonpace (via GitHub)" <gi...@apache.org>.
westonpace commented on PR #43632:
URL: https://github.com/apache/arrow/pull/43632#issuecomment-2281940063
I've been coding Rust too long now and as a result I must at least offer, for your consideration, a pull based asynchronous stream (which is convenient in that it is nearly identical to the synchronous version and it doesn't require special backpressure signals). Cancellation is handled by releasing the `ArrowAsyncDeviceArrayStream` before it is finished (if canceled while a pending call is in progress then the `Waker` needs to stay alive until `wake` is called and the producer needs to call `wake` even though it has been released).
If parallelism is desired by the consumer then the consumer should launch a new thread task to call `get_next` as soon as the previous call completes (while the old thread processes the data returned by the previous call to `get_next`).
```
struct Waker {
// Signal to the producer that more data is available, the consumer shall release any resources
// associated with the waker after this method is called. The producer shall not call any other
// methods after calling this method.
//
// The producer must always call this method even if an error is encountered (in which case the
// error will be reported on the following call to `get_next`).
void wake(Waker* waker);
void* private_data;
};
struct ArrowAsyncDeviceArrayStream {
// Callback to get the next array
// (if no error and the array is released, the stream has ended)
//
// Return value: 0 if successful, an `errno`-compatible error code otherwise.
//
// If EWOULDBLOCK is returned then the producer is not ready to generate more data. If the
// producer returns this value then the producer takes ownership of `waker` and is responsible
// for calling `wake` when more data is available. If any other value is returned then the producer
// shall ignore `waker` and never call any methods on it.
//
// If successful, the ArrowDeviceArray must be released independently from the stream.
int (*get_next)(struct ArrowAsyncDeviceArrayStream* self, Waker* waker, struct ArrowDeviceArray* out);
// The rest is identical to `ArrowDeviceArrayStream`
ArrowDeviceType device_type;
const char* (*get_last_error)(struct ArrowAsyncDeviceArrayStream* self);
void (*release)(struct ArrowAsyncDeviceArrayStream* self);
void* private_data;
};
```
Or, just for fun, an epoll-styled approach, which has the same pull-based advantages and doesn't require a waker but is less friendly towards coroutine-style asynchronicity.
```
struct ArrowAsyncDeviceArrayStream {
// Callback to get the next array
// (if no error and the array is released, the stream has ended)
//
// Return value: 0 if successful, an `errno`-compatible error code otherwise.
//
// If EWOULDBLOCK is returned then the producer is not ready to generate more data. The
// consumer should call `wait`. `out` will not be valid.
//
// If successful, the ArrowDeviceArray must be released independently from the stream.
int (*get_next)(struct ArrowAsyncDeviceArrayStream* self, struct ArrowDeviceArray* out);
// Blocks until data is available
void (*wait)(struct ArrowAsyncDeviceArrayStream* self, int timeout);
// The rest is identical to `ArrowDeviceArrayStream`
ArrowDeviceType device_type;
const char* (*get_last_error)(struct ArrowAsyncDeviceArrayStream* self);
void (*release)(struct ArrowAsyncDeviceArrayStream* self);
void* private_data;
};
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-43631: [C][Format] Add ArrowAsyncDeviceStreamHandler interface [arrow]
Posted by "lidavidm (via GitHub)" <gi...@apache.org>.
lidavidm commented on PR #43632:
URL: https://github.com/apache/arrow/pull/43632#issuecomment-2285097739
Before I dive into the details of the proposal (thanks Weston for all the feedback!) I'll note that gRPC has both models:
- a lower-level pull based model where the consumer creates a reactor and pushes operations/polls for completion (so more like the epoll model, less like the Rust Waker model), which for a long time was the only async API
- a callback-based model that wraps the pull-based model, which was added later for ease of use.
It might be worthwhile to consider similar wrapping in our case.
Also, it might be worth considering what exactly is easy/efficient to implement for different use cases. I think ADBC drivers written in Go, for instance, would probably not adapt to the pull based model well, since Go doesn't really want or need to give you that much control, but ones written in Rust or C++ would adapt much better.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-43631: [C][Format] Add ArrowAsyncDeviceStreamHandler interface [arrow]
Posted by "bkietz (via GitHub)" <gi...@apache.org>.
bkietz commented on code in PR #43632:
URL: https://github.com/apache/arrow/pull/43632#discussion_r1718857806
##########
cpp/src/arrow/c/abi.h:
##########
@@ -228,6 +228,65 @@ struct ArrowDeviceArrayStream {
#endif // ARROW_C_DEVICE_STREAM_INTERFACE
+#ifndef ARROW_C_ASYNC_STREAM_INTERFACE
+#define ARROW_C_ASYNC_STREAM_INTERFACE
+
+// Similar to ArrowDeviceArrayStream, except designed for an asynchronous
+// style of interaction. While ArrowDeviceArrayStream provides producer
+// defined callbacks, this is intended to be created by the consumer instead.
+// The consumer passes this handler to the producer, which in turn uses the
+// callbacks to inform the consumer of events in the stream.
+struct ArrowAsyncDeviceStreamHandler {
+ // Handler for receiving a schema. The passed in stream_schema should be
+ // released or moved by the handler (producer is giving ownership of it to
+ // the handler).
+ //
+ // The `extension_param` argument can be null or can be used by a producer
Review Comment:
Maybe it could be `const void*` then?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-43631: [C][Format] Add ArrowAsyncDeviceStreamHandler interface [arrow]
Posted by "kou (via GitHub)" <gi...@apache.org>.
kou commented on PR #43632:
URL: https://github.com/apache/arrow/pull/43632#issuecomment-2309579712
> > FWIW, do we need to do anything to accommodate the arrow::ArrayStatistics API that @kou has been working on?
>
> That's a good question, should we add some other callback / method for providing statistics? Or would this be something that we'd add to the `on_next` method?
Ah, new method is better than integrating into `on_next()` method. Because statistics may be needed before an array/record batch is received.
But we may not need the feature in `ArrowAsyncDeviceStreamHandler`. We will not provide a specific API to retrieve statistics for the Arrow C data interface: https://github.com/apache/arrow/pull/43553
We just provide a schema for statistics. The Arrow C data interface users will pass a statistics array like a normal array. We don't provide a statistics specific API for it.
Users may want to use two separated `ArrowAsyncDeviceStreamHandler`s. One `ArrowAsyncDeviceStreamHandler` is for statistics and another is for data. Statistics array can be passed via `on_next()` because it's a normal Arrow array.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-43631: [C][Format] Add ArrowAsyncDeviceStreamHandler interface [arrow]
Posted by "lidavidm (via GitHub)" <gi...@apache.org>.
lidavidm commented on code in PR #43632:
URL: https://github.com/apache/arrow/pull/43632#discussion_r1716190846
##########
cpp/src/arrow/c/abi.h:
##########
@@ -228,6 +228,65 @@ struct ArrowDeviceArrayStream {
#endif // ARROW_C_DEVICE_STREAM_INTERFACE
+#ifndef ARROW_C_ASYNC_STREAM_INTERFACE
+#define ARROW_C_ASYNC_STREAM_INTERFACE
+
+// Similar to ArrowDeviceArrayStream, except designed for an asynchronous
+// style of interaction. While ArrowDeviceArrayStream provides producer
+// defined callbacks, this is intended to be created by the consumer instead.
+// The consumer passes this handler to the producer, which in turn uses the
+// callbacks to inform the consumer of events in the stream.
+struct ArrowAsyncDeviceStreamHandler {
+ // Handler for receiving a schema. The passed in stream_schema should be
+ // released or moved by the handler (producer is giving ownership of it to
+ // the handler).
+ //
+ // The `extension_param` argument can be null or can be used by a producer
+ // to pass arbitrary extra information to the consumer (such as total number
+ // of rows, context info, or otherwise).
+ //
+ // Return value: 0 if successful, `errno`-compatible error otherwise
+ int (*on_schema)(struct ArrowAsyncDeviceStreamHandler* self,
+ struct ArrowSchema* stream_schema, void* extension_param);
+
+ // Handler for receiving an array/record batch. Always called at least once
+ // unless an error is encountered (which would result in calling on_error).
+ // An empty/released array is passed to indicate the end of the stream if no
+ // errors have been encountered.
+ //
+ // The `extension_param` argument can be null or can be used by a producer
+ // to pass arbitrary extra information to the consumer.
+ //
+ // Return value: 0 if successful, `errno`-compatible error otherwise.
+ int (*on_next)(struct ArrowAsyncDeviceStreamHandler* self,
+ struct ArrowDeviceArray* next, void* extension_param);
Review Comment:
IMO, yes, to some degree. A complex spec that few people can feasibly implement won't get used. (Hence my question below about which paradigm will fit actual consumers/producers)
And the scheme described here is pretty much what gRPC does for its callback-based async API which AFAIK has been proved out within Google at least. (Except it's a little more explicit about callback/backpressure: the consumer needs to explicitly request the next message each time. https://github.com/grpc/proposal/blob/master/L67-cpp-callback-api.md)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-43631: [C][Format] Add ArrowAsyncDeviceStreamHandler interface [arrow]
Posted by "zeroshade (via GitHub)" <gi...@apache.org>.
zeroshade commented on code in PR #43632:
URL: https://github.com/apache/arrow/pull/43632#discussion_r1713988979
##########
cpp/src/arrow/c/abi.h:
##########
@@ -228,6 +228,65 @@ struct ArrowDeviceArrayStream {
#endif // ARROW_C_DEVICE_STREAM_INTERFACE
+#ifndef ARROW_C_ASYNC_STREAM_INTERFACE
+#define ARROW_C_ASYNC_STREAM_INTERFACE
+
+// Similar to ArrowDeviceArrayStream, except designed for an asynchronous
+// style of interaction. While ArrowDeviceArrayStream provides producer
+// defined callbacks, this is intended to be created by the consumer instead.
+// The consumer passes this handler to the producer, which in turn uses the
+// callbacks to inform the consumer of events in the stream.
+struct ArrowAsyncDeviceStreamHandler {
+ // Handler for receiving a schema. The passed in stream_schema should be
+ // released or moved by the handler (producer is giving ownership of it to
+ // the handler).
+ //
+ // The `extension_param` argument can be null or can be used by a producer
+ // to pass arbitrary extra information to the consumer (such as total number
+ // of rows, context info, or otherwise).
+ //
+ // Return value: 0 if successful, `errno`-compatible error otherwise
+ int (*on_schema)(struct ArrowAsyncDeviceStreamHandler* self,
+ struct ArrowSchema* stream_schema, void* extension_param);
+
+ // Handler for receiving an array/record batch. Always called at least once
+ // unless an error is encountered (which would result in calling on_error).
+ // An empty/released array is passed to indicate the end of the stream if no
+ // errors have been encountered.
+ //
+ // The `extension_param` argument can be null or can be used by a producer
+ // to pass arbitrary extra information to the consumer.
+ //
+ // Return value: 0 if successful, `errno`-compatible error otherwise.
+ int (*on_next)(struct ArrowAsyncDeviceStreamHandler* self,
+ struct ArrowDeviceArray* next, void* extension_param);
+
+ // Handler for encountering an error. The producer should call release after
+ // this returns to clean up any resources.
+ //
+ // If the message or metadata are non-null, they will only last as long as this
+ // function call. The consumer would need to perform a copy of the data if it is
+ // it is necessary for them live past the lifetime of this call.
+ //
+ // Error metadata should be encoded as with metadata in ArrowSchema, defined in
+ // the spec at
+ // https://arrow.apache.org/docs/format/CDataInterface.html#c.ArrowSchema.metadata
+ //
+ // After this call, producers should follow-up by calling the release callback.
+ void (*on_error)(struct ArrowAsyncDeviceStreamHandler* self, int code,
+ const char* message, const char* metadata);
+
+ // Release callback to release any resources for the handler. Should always be
+ // called by a producer when it is done utilizing a handler. No callbacks should
+ // be called after this is called.
+ void (*release)(struct ArrowAsyncDeviceStreamHandler* self);
Review Comment:
The spec document currently states that the producer *must* wait until a callback returns before calling the next callback. The intent is that a consumer shouldn't have to handle concurrent calls like that, and backpressure is managed by the callbacks themselves (producer won't call the next callback until the current one returns).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-43631: [C][Format] Add ArrowAsyncDeviceStreamHandler interface [arrow]
Posted by "zeroshade (via GitHub)" <gi...@apache.org>.
zeroshade commented on PR #43632:
URL: https://github.com/apache/arrow/pull/43632#issuecomment-2284477412
@westonpace I completely forgot to address the other suggestion where you *did* put the waker directly into the interface lol
```c
struct Waker {
// Signal to the producer that more data is available, the consumer shall release any resources
// associated with the waker after this method is called. The producer shall not call any other
// methods after calling this method.
//
// The producer must always call this method even if an error is encountered (in which case the
// error will be reported on the following call to `get_next`).
void wake(Waker* waker);
void* private_data;
};
struct ArrowAsyncDeviceArrayStream {
// Callback to get the next array
// (if no error and the array is released, the stream has ended)
//
// Return value: 0 if successful, an `errno`-compatible error code otherwise.
//
// If EWOULDBLOCK is returned then the producer is not ready to generate more data. If the
// producer returns this value then the producer takes ownership of `waker` and is responsible
// for calling `wake` when more data is available. If any other value is returned then the producer
// shall ignore `waker` and never call any methods on it.
//
// If successful, the ArrowDeviceArray must be released independently from the stream.
int (*get_next)(struct ArrowAsyncDeviceArrayStream* self, Waker* waker, struct ArrowDeviceArray* out);
// The rest is identical to `ArrowDeviceArrayStream`
ArrowDeviceType device_type;
const char* (*get_last_error)(struct ArrowAsyncDeviceArrayStream* self);
void (*release)(struct ArrowAsyncDeviceArrayStream* self);
void* private_data;
};
```
I guess I still have the same questions as before though. Isn't the waker provided *by* the consumer for the producer to signal to the *consumer* that data is available? not the other way around?
The original idea and semantics here were that the `ArrowAsyncDeviceArrayStream` is constructed by the consumer and given to the producer, given the comments you've put here it seems like this is the other way around now? The producer still provides the struct, and the consumer calls `get_next` providing the `waker`. Do I have that right? What happens if `get_next` is called again before `wake` is called on the waker? Are we defining that `get_next` must not be called until `wake` is called? The semantics are a bit confusing to me here as far as what you're proposing.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-43631: [C][Format] Add ArrowAsyncDeviceStreamHandler interface [arrow]
Posted by "pitrou (via GitHub)" <gi...@apache.org>.
pitrou commented on code in PR #43632:
URL: https://github.com/apache/arrow/pull/43632#discussion_r1718560109
##########
cpp/src/arrow/c/abi.h:
##########
@@ -228,6 +228,65 @@ struct ArrowDeviceArrayStream {
#endif // ARROW_C_DEVICE_STREAM_INTERFACE
+#ifndef ARROW_C_ASYNC_STREAM_INTERFACE
+#define ARROW_C_ASYNC_STREAM_INTERFACE
+
+// Similar to ArrowDeviceArrayStream, except designed for an asynchronous
+// style of interaction. While ArrowDeviceArrayStream provides producer
+// defined callbacks, this is intended to be created by the consumer instead.
+// The consumer passes this handler to the producer, which in turn uses the
+// callbacks to inform the consumer of events in the stream.
Review Comment:
Oops, I just saw @westonpace 's question below. Sorry :-)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-43631: [C][Format] Add ArrowAsyncDeviceStreamHandler interface [arrow]
Posted by "zeroshade (via GitHub)" <gi...@apache.org>.
zeroshade commented on code in PR #43632:
URL: https://github.com/apache/arrow/pull/43632#discussion_r1757091118
##########
cpp/src/arrow/c/abi.h:
##########
@@ -228,6 +228,180 @@ struct ArrowDeviceArrayStream {
#endif // ARROW_C_DEVICE_STREAM_INTERFACE
+#ifndef ARROW_C_ASYNC_STREAM_INTERFACE
+#define ARROW_C_ASYNC_STREAM_INTERFACE
+
+// ArrowAsyncTask represents available data from a producer that was passed to
+// an invocation of `on_next_task` on the ArrowAsyncDeviceStreamHandler.
+struct ArrowAsyncTask {
+ // This callback should populate the ArrowDeviceArray associated with this task.
Review Comment:
Is this expected to be non-blocking? I thought we were giving up the whole `waker` idea and allowing this to be blocking if needed? Though it shouldn't block for any significant amount of time, all of the waiting should have happened before it the producer called `on_next_task` in the first place.
##########
cpp/src/arrow/c/abi.h:
##########
@@ -228,6 +228,180 @@ struct ArrowDeviceArrayStream {
#endif // ARROW_C_DEVICE_STREAM_INTERFACE
+#ifndef ARROW_C_ASYNC_STREAM_INTERFACE
+#define ARROW_C_ASYNC_STREAM_INTERFACE
+
+// ArrowAsyncTask represents available data from a producer that was passed to
Review Comment:
None of the comments in this file use the `///` for doxygen, so I was following the existing pattern. If we want to convert all of these to doxygen, I think we should do it as a separate change to convert *all* the `//` lines to `///` in the file as a whole.
I'll add the rationale, @westonpace can you take a look at what I put here and ensure it's accurate?
##########
cpp/src/arrow/c/abi.h:
##########
@@ -228,6 +228,180 @@ struct ArrowDeviceArrayStream {
#endif // ARROW_C_DEVICE_STREAM_INTERFACE
+#ifndef ARROW_C_ASYNC_STREAM_INTERFACE
+#define ARROW_C_ASYNC_STREAM_INTERFACE
+
+// ArrowAsyncTask represents available data from a producer that was passed to
+// an invocation of `on_next_task` on the ArrowAsyncDeviceStreamHandler.
+struct ArrowAsyncTask {
+ // This callback should populate the ArrowDeviceArray associated with this task.
+ // The order of ArrowAsyncTasks provided by the producer enables a consumer to
+ // ensure the order of data to process.
+ //
+ // Returns: 0 if successful, errno-compatible error otherwise.
+ //
+ // If a non-0 value is returned then it should be followed by a call to `on_error`
+ // on the appropriate ArrowAsyncDeviceStreamHandler.
Review Comment:
The reason for this to be set up like this is precisely *because* it's potentially disconnected from the current control flow.
Returning the non-zero value to indicate an error allows the current control flow to be aware of the failure and act accordingly, while still centralizing the handling in the `on_error` callback. This way, the current control flow can end early and is aware of the error, but the consumer can still rely on its `on_error` callback to manage any logging etc.
##########
cpp/src/arrow/c/abi.h:
##########
@@ -228,6 +228,180 @@ struct ArrowDeviceArrayStream {
#endif // ARROW_C_DEVICE_STREAM_INTERFACE
+#ifndef ARROW_C_ASYNC_STREAM_INTERFACE
+#define ARROW_C_ASYNC_STREAM_INTERFACE
+
+// ArrowAsyncTask represents available data from a producer that was passed to
+// an invocation of `on_next_task` on the ArrowAsyncDeviceStreamHandler.
+struct ArrowAsyncTask {
+ // This callback should populate the ArrowDeviceArray associated with this task.
+ // The order of ArrowAsyncTasks provided by the producer enables a consumer to
+ // ensure the order of data to process.
+ //
+ // Returns: 0 if successful, errno-compatible error otherwise.
+ //
+ // If a non-0 value is returned then it should be followed by a call to `on_error`
+ // on the appropriate ArrowAsyncDeviceStreamHandler.
+ //
+ // Rather than a release callback, any required cleanup should be performed as part
+ // of the invocation of `get_data`. Ownership of the Array is passed to the consumer
+ // calling this, and so it must be released separately.
+ //
+ // It is only valid to call this method exactly once.
+ int (*get_data)(struct ArrowArrayTask* self, struct ArrowDeviceArray* out);
+
+ // opaque task-specific data
+ void* private_data;
+};
+
+// ArrowAsyncProducer represents a 1-to-1 relationship between an async producer
+// and consumer. This object allows the consumer to perform backpressure and flow
+// control on the asynchronous stream processing. This object must be owned by the
+// producer who creates it, and thus is responsible for cleaning it up.
+struct ArrowAsyncProducer {
+ // A consumer must call this function to start receiving on_next_task calls.
+ //
+ // It *must* be valid to call this synchronously from within on_next_task or
+ // on_schema, with a producer placing an upper bound on possible synchronous
+ // recursion between calls of on_next_task -> request -> on_next_task ->....
Review Comment:
that's fair, I'll update the documentation accordingly.
##########
cpp/src/arrow/c/abi.h:
##########
@@ -228,6 +228,180 @@ struct ArrowDeviceArrayStream {
#endif // ARROW_C_DEVICE_STREAM_INTERFACE
+#ifndef ARROW_C_ASYNC_STREAM_INTERFACE
+#define ARROW_C_ASYNC_STREAM_INTERFACE
+
+// ArrowAsyncTask represents available data from a producer that was passed to
+// an invocation of `on_next_task` on the ArrowAsyncDeviceStreamHandler.
+struct ArrowAsyncTask {
+ // This callback should populate the ArrowDeviceArray associated with this task.
+ // The order of ArrowAsyncTasks provided by the producer enables a consumer to
+ // ensure the order of data to process.
+ //
+ // Returns: 0 if successful, errno-compatible error otherwise.
+ //
+ // If a non-0 value is returned then it should be followed by a call to `on_error`
+ // on the appropriate ArrowAsyncDeviceStreamHandler.
+ //
+ // Rather than a release callback, any required cleanup should be performed as part
+ // of the invocation of `get_data`. Ownership of the Array is passed to the consumer
+ // calling this, and so it must be released separately.
+ //
+ // It is only valid to call this method exactly once.
+ int (*get_data)(struct ArrowArrayTask* self, struct ArrowDeviceArray* out);
+
+ // opaque task-specific data
+ void* private_data;
+};
+
+// ArrowAsyncProducer represents a 1-to-1 relationship between an async producer
+// and consumer. This object allows the consumer to perform backpressure and flow
+// control on the asynchronous stream processing. This object must be owned by the
+// producer who creates it, and thus is responsible for cleaning it up.
+struct ArrowAsyncProducer {
+ // A consumer must call this function to start receiving on_next_task calls.
+ //
+ // It *must* be valid to call this synchronously from within on_next_task or
+ // on_schema, with a producer placing an upper bound on possible synchronous
+ // recursion between calls of on_next_task -> request -> on_next_task ->....
+ //
+ // After cancel has been called, additional calls to this function must be NOPs,
+ // but allowed.
+ //
+ // While not cancelled, calling this function must register the given number of
+ // additional arrays/batches to be produced with the producer. The producer should
+ // only call `on_next_task` at most the registered number of arrays before propagating
+ // backpressure.
+ //
+ // While not cancelled, calling request MAY synchronously call `on_next_task`,
+ // `on_error`, or `release` on the ArrowAsyncDeviceStreamHandler.
+ //
+ // Any error encountered by calling request must be propagated by calling the `on_error`
+ // callback of the ArrowAsyncDeviceStreamHandler.
+ //
+ // A producer must support an unbounded number of calls to request and must support
+ // a total registered demand (sum requested - sum delivered) of up to UINT64_MAX.
+ void (*request)(struct ArrowAsyncProducer* self, uint64_t n);
+
+ // This cancel callback signals a producer that it must eventually stop making calls
+ // to on_next_task. It must be idempotent and thread-safe.
+ //
+ // After calling cancel once, subsequent calls must be NOPs.
+ //
+ // It is not required that calling cancel affect the producer immediately, only that it
+ // must eventually stop calling on_next_task and subsequently call release on the
+ // async handler. As such, a consumer must be prepared to receive one or more calls to
+ // `on_next_task` even after calling cancel if there are still requested arrays pending.
+ //
+ // Any error encountered during handling a call to cancel must be reported via the on_error
+ // callback on the async stream handler.
+ void (*cancel)(struct ArrowAsyncProducer* self);
Review Comment:
I'll add a line clarifying this
##########
cpp/src/arrow/c/abi.h:
##########
@@ -228,6 +228,180 @@ struct ArrowDeviceArrayStream {
#endif // ARROW_C_DEVICE_STREAM_INTERFACE
+#ifndef ARROW_C_ASYNC_STREAM_INTERFACE
+#define ARROW_C_ASYNC_STREAM_INTERFACE
+
+// ArrowAsyncTask represents available data from a producer that was passed to
+// an invocation of `on_next_task` on the ArrowAsyncDeviceStreamHandler.
+struct ArrowAsyncTask {
+ // This callback should populate the ArrowDeviceArray associated with this task.
+ // The order of ArrowAsyncTasks provided by the producer enables a consumer to
+ // ensure the order of data to process.
+ //
+ // Returns: 0 if successful, errno-compatible error otherwise.
+ //
+ // If a non-0 value is returned then it should be followed by a call to `on_error`
+ // on the appropriate ArrowAsyncDeviceStreamHandler.
+ //
+ // Rather than a release callback, any required cleanup should be performed as part
+ // of the invocation of `get_data`. Ownership of the Array is passed to the consumer
+ // calling this, and so it must be released separately.
+ //
+ // It is only valid to call this method exactly once.
+ int (*get_data)(struct ArrowArrayTask* self, struct ArrowDeviceArray* out);
+
+ // opaque task-specific data
+ void* private_data;
+};
+
+// ArrowAsyncProducer represents a 1-to-1 relationship between an async producer
+// and consumer. This object allows the consumer to perform backpressure and flow
+// control on the asynchronous stream processing. This object must be owned by the
+// producer who creates it, and thus is responsible for cleaning it up.
+struct ArrowAsyncProducer {
+ // A consumer must call this function to start receiving on_next_task calls.
+ //
+ // It *must* be valid to call this synchronously from within on_next_task or
+ // on_schema, with a producer placing an upper bound on possible synchronous
+ // recursion between calls of on_next_task -> request -> on_next_task ->....
+ //
+ // After cancel has been called, additional calls to this function must be NOPs,
+ // but allowed.
+ //
+ // While not cancelled, calling this function must register the given number of
+ // additional arrays/batches to be produced with the producer. The producer should
+ // only call `on_next_task` at most the registered number of arrays before propagating
+ // backpressure.
+ //
+ // While not cancelled, calling request MAY synchronously call `on_next_task`,
+ // `on_error`, or `release` on the ArrowAsyncDeviceStreamHandler.
+ //
+ // Any error encountered by calling request must be propagated by calling the `on_error`
+ // callback of the ArrowAsyncDeviceStreamHandler.
+ //
+ // A producer must support an unbounded number of calls to request and must support
+ // a total registered demand (sum requested - sum delivered) of up to UINT64_MAX.
+ void (*request)(struct ArrowAsyncProducer* self, uint64_t n);
+
+ // This cancel callback signals a producer that it must eventually stop making calls
+ // to on_next_task. It must be idempotent and thread-safe.
+ //
+ // After calling cancel once, subsequent calls must be NOPs.
+ //
+ // It is not required that calling cancel affect the producer immediately, only that it
+ // must eventually stop calling on_next_task and subsequently call release on the
+ // async handler. As such, a consumer must be prepared to receive one or more calls to
+ // `on_next_task` even after calling cancel if there are still requested arrays pending.
+ //
+ // Any error encountered during handling a call to cancel must be reported via the on_error
+ // callback on the async stream handler.
+ void (*cancel)(struct ArrowAsyncProducer* self);
+
+ // producer-specific opaque data.
+ void* private_data;
+};
+
+// Similar to ArrowDeviceArrayStream, except designed for an asynchronous
+// style of interaction. While ArrowDeviceArrayStream provides producer
+// defined callbacks, this is intended to be created by the consumer instead.
+// The consumer passes this handler to the producer, which in turn uses the
+// callbacks to inform the consumer of events in the stream.
+struct ArrowAsyncDeviceStreamHandler {
+ // Handler for receiving a schema. The passed in stream_schema must be
+ // released or moved by the handler (producer is giving ownership of the schema to
+ // the handler, but not ownership of the top level object itself).
+ //
+ // With the exception of an error occurring (on_error), this must be the first
+ // callback function which is called by a producer and must only be called exactly
+ // once. As such, the producer should provide a valid ArrowAsyncProducer instance
+ // so the consumer can control the flow. See the documentation on ArrowAsyncProducer
+ // for how it works. The ArrowAsyncProducer is owned by the producer who calls this
+ // function and thus the producer is responsible for cleaning it up when calling
+ // the release callback of this handler.
+ //
+ // The addl_metadata argument can be null or can be used by a producer
+ // to pass arbitrary extra information to the consumer beyond the metadata in the schema
+ // itself (such as total number of rows, context info, or otherwise). The data should
+ // be passed using the same encoding as the metadata within the ArrowSchema struct
+ // itself (defined in the spec at
+ // https://arrow.apache.org/docs/format/CDataInterface.html#c.ArrowSchema.metadata)
+ //
+ // If addl_metadata is non-null then it only needs to exist for the lifetime of this call,
+ // a consumer who wants it to live after that must copy it to ensure lifetime.
+ //
+ // Return value: 0 if successful, `errno`-compatible error otherwise
+ //
+ // A producer that receives a non-zero return here should stop producing and eventually
+ // call release instead.
+ int (*on_schema)(struct ArrowAsyncDeviceStreamHandler* self,
+ struct ArrowAsyncProducer* producer, struct ArrowSchema* stream_schema,
+ const char* addl_metadata);
+
+ // Handler for receiving data. This is called when data is available providing an
+ // ArrowAsyncTask struct to signify it. The consumer is responsible for calling
+ // the release callback on the ArrowAsyncTask after retrieving the array/batch.
+ // An empty/released task is passed to indicate the end of the stream if no
+ // errors have been encountered.
+ //
+ // The `request` callback of a provided ArrowAsyncProducer must be called in order
+ // to start receiving calls to this handler.
+ //
+ // The metadata argument can be null or can be used by a producer
+ // to pass arbitrary extra information to the consumer (such as total number
+ // of rows, context info, or otherwise). The data should be passed using the same
+ // encoding as the metadata within the ArrowSchema struct itself (defined in
+ // the spec at
+ // https://arrow.apache.org/docs/format/CDataInterface.html#c.ArrowSchema.metadata)
+ //
+ // If metadata is non-null then it only needs to exist for the lifetime of this call,
+ // a consumer who wants it to live after that must copy it to ensure lifetime.
+ //
+ // A producer *must not* call this concurrently from multiple different threads.
+ //
+ // A consumer must be prepared to received one or more calls to this callback even
+ // after calling cancel on the corresponding ArrowAsyncProducer, as cancel does not
+ // guarantee it happens immediately.
+ //
+ // Return value: 0 if successful, `errno`-compatible error otherwise.
+ int (*on_next_task)(struct ArrowAsyncDeviceStreamHandler* self,
+ struct ArrowAsyncTask* task, const char* metadata);
Review Comment:
fixed.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-43631: [C][Format] Add ArrowAsyncDeviceStreamHandler interface [arrow]
Posted by "pitrou (via GitHub)" <gi...@apache.org>.
pitrou commented on code in PR #43632:
URL: https://github.com/apache/arrow/pull/43632#discussion_r1757477224
##########
docs/source/format/CDeviceDataInterface.rst:
##########
@@ -650,6 +652,331 @@ The stream source is not assumed to be thread-safe. Consumers wanting to
call ``get_next`` from several threads should ensure those calls are
serialized.
+Async Device Stream Interface
+=============================
+
+The :ref:`C stream interface <_c-device-stream-interface>` provides a synchronous
+API centered around the consumer calling the callback functions to retrieve
+the next record batch. For some bindings, use cases, and interoperability, a more
+asynchronous, producer-focused interface may be required. These scenarios can utilize
+the ``ArrowAsyncDeviceStreamHandler``.
+
+Semantics
+---------
+
+Rather than the producer providing a structure of callbacks for a consumer to
+call and retrieve records, the Async interface is a consumer allocated structure.
+The consumer allocated struct provides handler callbacks for the producer to call
+when the schema and chunks of data are available, rather than the consumer using
+a blocking pull-style iteration.
+
+In addition to the ``ArrowAsyncDeviceStreamHandler``, there are also two additional
+structs used for the full data flow: ``ArrowAsyncTask`` and ``ArrowAsyncProducer``.
+
+Structure Definition
+--------------------
+
+The C device async stream interface is defined with a single ``struct`` definition:
+
+.. code-block:: c
+
+ #ifndef ARROW_C_ASYNC_STREAM_INTERFACE
+ #define ARROW_C_ASYNC_STREAM_INTERFACE
+
+ struct ArrowAsyncTask {
+ int (*get_data)(struct ArrowArrayTask* self, struct ArrowDeviceArray* out);
+
+ void* private_data;
+ };
+
+ struct ArrowAsyncProducer {
+ void (*request)(struct ArrowAsyncProducer* self, uint64_t n);
+ void (*cancel)(struct ArrowAsyncProducer* self);
+
+ void (*release)(struct ArrowAsyncProducer* self);
+ void* private_data;
+ };
+
+ struct ArrowAsyncDeviceStreamHandler {
+ // handlers
+ int (*on_schema)(struct ArrowAsyncDeviceStreamHandler* self,
+ struct ArrowAsyncProducer* producer,
+ struct ArrowSchema* stream_schema, const char* addl_metadata);
+ int (*on_next_task)(struct ArrowAsyncDeviceStreamHandler* self,
+ struct ArrowAsyncTask* task, const char* metadata);
+ void (*on_error)(struct ArrowAsyncDeviceStreamHandler* self,
+ int code, const char* message, const char* metadata);
+
+ // release callback
+ void (*release)(struct ArrowAsyncDeviceStreamHandler* self);
+
+ // opaque handler-specific data
+ void* private_data;
+ };
+
+ #endif // ARROW_C_ASYNC_STREAM_INTERFACE
+
+.. note::
+ The canonical guard ``ARROW_C_ASYNC_STREAM_INTERFACE`` is meant to avoid
+ duplicate definitions if two projects copy the C async stream interface
+ definitions into their own headers, and a third-party project includes
+ from these two projects. It is therefore important that this guard is kept
+ exactly as-is when these definitions are copied.
+
+The ArrowAsyncDeviceStreamHandler structure
+'''''''''''''''''''''''''''''''''''''''''''
+
+The structure has the following fields:
+
+.. c:member:: int (*ArrowAsyncDeviceStreamHandler.on_schema)(struct ArrowAsyncDeviceStreamHandler*, struct ArrowAsyncProducer*, struct ArrowSchema*, const char*)
+
+ *Mandatory.* Handler for receiving the schema of the stream. All records should
+ match the provided schema. If successful, the function should return 0, otherwise
+ it should return an ``errno``-compatible error code.
+
+ The ``const char*`` parameter exists for producers to provide any extra contextual information
+ they want, such as the total number of rows in the stream, statistics, or otherwise. This is
+ encoded in the same format as :c:member:`ArrowSchema.metadata`. If not ``NULL``,
+ the lifetime is only the scope of the call to this function. A consumer who wants to maintain
+ the additional metadata beyond the lifetime of this call *MUST* copy the value themselves.
+
+ Unless the ``on_error`` handler is called, this will always get called exactly once and will be
+ the first method called on this object. As such the producer *MUST* provide an ``ArrowAsyncProducer``
+ object when calling this function to allow the consumer to manage back-pressure and flow control.
+ The producer maintains ownership of the ``ArrowAsyncProducer`` and must clean it up before or after
+ calling the release callback on this object.
+
+ A producer that receives a non-zero result here must not subsequently call anything other than
+ the release callback on this object.
+
+.. c:member:: int (*ArrowAsyncDeviceStreamHandler.on_next_task)(struct ArrowAsyncDeviceStreamHandler*, struct ArrowAsyncTask*, const char*)
+
+ *Mandatory.* Handler to be called when a new record is available for processing. The
+ schema for each record should be the same as the schema that ``on_schema`` was called with.
+ If successfully handled, the function should return 0, otherwise it should return an
+ ``errno``-compatible error code.
+
+ Rather than passing the record itself it receives an ``ArrowAsyncTask`` instead to facilitate
+ better consumer-focused thread control as far as receiving the data. A call to this function
+ simply indicates that data is available via the provided task.
+
+ The ``const char*`` parameter exists for producers to provide any extra contextual information
+ they want. This is encoded in the same format as :c:member:`ArrowSchema.metadata`. If not ``NULL``,
+ the lifetime is only the scope of the call to this function. A consumer who wants to maintain
+ the additional metadata beyond the lifetime of this call *MUST* copy the value themselves.
+
+ A producer *MUST NOT* call this concurrently from multiple different threads.
+
+ The :c:member:`ArrowAsyncProducer.request` callback must be called to start receiving calls to this
+ handler.
+
+ This function *MUST* be able to be called re-entrantly on the same thread to allow for the
+ :c:member:`ArrowAsyncProducer.request` callback to potentially call this method recursively.
+
+.. c:member:: void (*ArrowAsyncDeviceStreamHandler.on_error)(struct ArrowAsyncDeviceStreamHandler, int, const char*, const char*)
+
+ *Mandatory.* Handler to be called when an error is encountered by the producer. After calling
+ this, the ``release`` callback will be called as the last call on this struct. The parameters
+ are an ``errno``-compatible error code and an optional error message and metadata.
+
+ If the message and metadata are not ``NULL``, their lifetime is only valid during the scope
+ of this call. A consumer who wants to maintain these values past the return of this function
+ *MUST* copy the values themselves.
+
+ If the metadata parameter is not ``NULL``, to provide key-value error metadata, then it should
+ be encoded identically to the way that metadata is encoded in :c:member:`ArrowSchema.metadata`.
+
+ It is valid for this to be called by a producer with or without a preceding call to
+ :c:member:`ArrowAsyncProducer.request`. This callback *MUST NOT* call any methods of an
+ ``ArrowAsyncProducer`` object.
+
+.. c:member:: void (*ArrowAsyncDeviceStreamHandler.release)(struct ArrowAsyncDeviceStreamHandler*)
+
+ *Mandatory.* A pointer to a consumer-provided release callback for the handler.
+
+ It is valid for this to be called by a producer with or without a preceding call to
+ :c:member:`ArrowAsyncProducer.request`. This must not call any methods of an ``ArrowAsyncProducer``
+ object.
+
+.. c:member:: void* ArrowAsyncDeviceStreamHandler.private_data
+
+ *Optional.* An opaque pointer to consumer-provided private data.
+
+ Producers *MUST NOT* process this member. Lifetime of this member is handled by
+ the consumer, and especially by the release callback.
+
+The ArrowAsyncTask structure
+''''''''''''''''''''''''''''
+
+The purpose of using a Task object rather than passing the array directly to the ``on_next``
+callback is to allow for more complex and efficient thread handling. Utilizing a Task
+object allows for a producer to separate the "decoding" logic from the I/O, enabling a
+consumer to avoid transferring data between CPU cores (e.g. from one L1/L2 cache to another).
+
+This producer-provided structure has the following fields:
+
+.. c:member:: int (*ArrowArrayTask.get_data)(struct ArrowArrayTask*, struct ArrowDeviceArray*)
+
+ *Mandatory.* A callback to populate the provided ``ArrowDeviceArray`` with the available data.
+ The order of ArrowAsyncTasks provided by the producer enables a consumer to know the order of
+ the data to process.
+
+ If a non-zero value is returned from this, it should be followed only by the producer calling
+ the ``on_error`` callback of the ``ArrowAsyncDeviceStreamHandler``. Because calling this method
+ is likely to be separate from the current control flow, returning a non-zero value to signal
+ an error occuring allows the current thread to decide handle the case accordingly, while still
+ allowing all error logging and handling to be centralized in the
+ :c:member:`ArrowAsyncDeviceStreamHandler.on_error` callback.
+
+ Rather than having a separate release callback, any required cleanup should be performed as part
+ of the invocation of this callback, with ownership of the Array being passed to the consumer-provided
+ parameter calling this, and must be released separately.
Review Comment:
What must be released separately? This is difficult to parse, can you split this sentence in two?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-43631: [C][Format] Add ArrowAsyncDeviceStreamHandler interface [arrow]
Posted by "westonpace (via GitHub)" <gi...@apache.org>.
westonpace commented on code in PR #43632:
URL: https://github.com/apache/arrow/pull/43632#discussion_r1766084568
##########
docs/source/format/CDeviceDataInterface.rst:
##########
@@ -650,6 +652,331 @@ The stream source is not assumed to be thread-safe. Consumers wanting to
call ``get_next`` from several threads should ensure those calls are
serialized.
+Async Device Stream Interface
+=============================
+
+The :ref:`C stream interface <_c-device-stream-interface>` provides a synchronous
+API centered around the consumer calling the callback functions to retrieve
+the next record batch. For some bindings, use cases, and interoperability, a more
+asynchronous, producer-focused interface may be required. These scenarios can utilize
+the ``ArrowAsyncDeviceStreamHandler``.
+
+Semantics
+---------
+
+Rather than the producer providing a structure of callbacks for a consumer to
+call and retrieve records, the Async interface is a consumer allocated structure.
+The consumer allocated struct provides handler callbacks for the producer to call
+when the schema and chunks of data are available, rather than the consumer using
+a blocking pull-style iteration.
+
+In addition to the ``ArrowAsyncDeviceStreamHandler``, there are also two additional
+structs used for the full data flow: ``ArrowAsyncTask`` and ``ArrowAsyncProducer``.
+
+Structure Definition
+--------------------
+
+The C device async stream interface is defined with a single ``struct`` definition:
+
+.. code-block:: c
+
+ #ifndef ARROW_C_ASYNC_STREAM_INTERFACE
+ #define ARROW_C_ASYNC_STREAM_INTERFACE
+
+ struct ArrowAsyncTask {
+ int (*get_data)(struct ArrowArrayTask* self, struct ArrowDeviceArray* out);
+
+ void* private_data;
+ };
+
+ struct ArrowAsyncProducer {
+ void (*request)(struct ArrowAsyncProducer* self, uint64_t n);
+ void (*cancel)(struct ArrowAsyncProducer* self);
+
+ void (*release)(struct ArrowAsyncProducer* self);
+ void* private_data;
+ };
+
+ struct ArrowAsyncDeviceStreamHandler {
+ // handlers
+ int (*on_schema)(struct ArrowAsyncDeviceStreamHandler* self,
+ struct ArrowAsyncProducer* producer,
+ struct ArrowSchema* stream_schema, const char* addl_metadata);
+ int (*on_next_task)(struct ArrowAsyncDeviceStreamHandler* self,
+ struct ArrowAsyncTask* task, const char* metadata);
+ void (*on_error)(struct ArrowAsyncDeviceStreamHandler* self,
+ int code, const char* message, const char* metadata);
+
+ // release callback
+ void (*release)(struct ArrowAsyncDeviceStreamHandler* self);
+
+ // opaque handler-specific data
+ void* private_data;
+ };
+
+ #endif // ARROW_C_ASYNC_STREAM_INTERFACE
+
+.. note::
+ The canonical guard ``ARROW_C_ASYNC_STREAM_INTERFACE`` is meant to avoid
+ duplicate definitions if two projects copy the C async stream interface
+ definitions into their own headers, and a third-party project includes
+ from these two projects. It is therefore important that this guard is kept
+ exactly as-is when these definitions are copied.
+
+The ArrowAsyncDeviceStreamHandler structure
+'''''''''''''''''''''''''''''''''''''''''''
+
+The structure has the following fields:
+
+.. c:member:: int (*ArrowAsyncDeviceStreamHandler.on_schema)(struct ArrowAsyncDeviceStreamHandler*, struct ArrowAsyncProducer*, struct ArrowSchema*, const char*)
+
+ *Mandatory.* Handler for receiving the schema of the stream. All records should
+ match the provided schema. If successful, the function should return 0, otherwise
+ it should return an ``errno``-compatible error code.
+
+ The ``const char*`` parameter exists for producers to provide any extra contextual information
+ they want, such as the total number of rows in the stream, statistics, or otherwise. This is
+ encoded in the same format as :c:member:`ArrowSchema.metadata`. If not ``NULL``,
+ the lifetime is only the scope of the call to this function. A consumer who wants to maintain
+ the additional metadata beyond the lifetime of this call *MUST* copy the value themselves.
+
+ Unless the ``on_error`` handler is called, this will always get called exactly once and will be
+ the first method called on this object. As such the producer *MUST* provide an ``ArrowAsyncProducer``
+ object when calling this function to allow the consumer to manage back-pressure and flow control.
+ The producer maintains ownership of the ``ArrowAsyncProducer`` and must clean it up before or after
+ calling the release callback on this object.
+
+ A producer that receives a non-zero result here must not subsequently call anything other than
+ the release callback on this object.
+
+.. c:member:: int (*ArrowAsyncDeviceStreamHandler.on_next_task)(struct ArrowAsyncDeviceStreamHandler*, struct ArrowAsyncTask*, const char*)
+
+ *Mandatory.* Handler to be called when a new record is available for processing. The
+ schema for each record should be the same as the schema that ``on_schema`` was called with.
+ If successfully handled, the function should return 0, otherwise it should return an
+ ``errno``-compatible error code.
+
+ Rather than passing the record itself it receives an ``ArrowAsyncTask`` instead to facilitate
+ better consumer-focused thread control as far as receiving the data. A call to this function
+ simply indicates that data is available via the provided task.
+
+ The ``const char*`` parameter exists for producers to provide any extra contextual information
+ they want. This is encoded in the same format as :c:member:`ArrowSchema.metadata`. If not ``NULL``,
+ the lifetime is only the scope of the call to this function. A consumer who wants to maintain
+ the additional metadata beyond the lifetime of this call *MUST* copy the value themselves.
+
+ A producer *MUST NOT* call this concurrently from multiple different threads.
+
+ The :c:member:`ArrowAsyncProducer.request` callback must be called to start receiving calls to this
+ handler.
+
+ This function *MUST* be able to be called re-entrantly on the same thread to allow for the
+ :c:member:`ArrowAsyncProducer.request` callback to potentially call this method recursively.
+
+.. c:member:: void (*ArrowAsyncDeviceStreamHandler.on_error)(struct ArrowAsyncDeviceStreamHandler, int, const char*, const char*)
+
+ *Mandatory.* Handler to be called when an error is encountered by the producer. After calling
+ this, the ``release`` callback will be called as the last call on this struct. The parameters
+ are an ``errno``-compatible error code and an optional error message and metadata.
+
+ If the message and metadata are not ``NULL``, their lifetime is only valid during the scope
+ of this call. A consumer who wants to maintain these values past the return of this function
+ *MUST* copy the values themselves.
+
+ If the metadata parameter is not ``NULL``, to provide key-value error metadata, then it should
+ be encoded identically to the way that metadata is encoded in :c:member:`ArrowSchema.metadata`.
+
+ It is valid for this to be called by a producer with or without a preceding call to
+ :c:member:`ArrowAsyncProducer.request`. This callback *MUST NOT* call any methods of an
+ ``ArrowAsyncProducer`` object.
+
+.. c:member:: void (*ArrowAsyncDeviceStreamHandler.release)(struct ArrowAsyncDeviceStreamHandler*)
+
+ *Mandatory.* A pointer to a consumer-provided release callback for the handler.
+
+ It is valid for this to be called by a producer with or without a preceding call to
+ :c:member:`ArrowAsyncProducer.request`. This must not call any methods of an ``ArrowAsyncProducer``
+ object.
+
+.. c:member:: void* ArrowAsyncDeviceStreamHandler.private_data
+
+ *Optional.* An opaque pointer to consumer-provided private data.
+
+ Producers *MUST NOT* process this member. Lifetime of this member is handled by
+ the consumer, and especially by the release callback.
+
+The ArrowAsyncTask structure
+''''''''''''''''''''''''''''
+
+The purpose of using a Task object rather than passing the array directly to the ``on_next``
+callback is to allow for more complex and efficient thread handling. Utilizing a Task
+object allows for a producer to separate the "decoding" logic from the I/O, enabling a
+consumer to avoid transferring data between CPU cores (e.g. from one L1/L2 cache to another).
Review Comment:
I will admit there is one wrinkle I'm still ironing out :)
Decompression often happens at different scale than the consumers batch size. For example, there might be 100K values per compression block but consumers might want to operate on batches of 32K.
This can force a thread transfer since three decode tasks share the same compression task and only one thread can do the work. Still, this is something I would handle entirely in the consumer in my case (e.g. two CPU thread pools, one for decompression and one for decoding).
> a simpler consumer which has only one thread. That's also a useful case to support;
Simple cases are maybe more easily handled by the synchronous interface?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-43631: [C][Format] Add ArrowAsyncDeviceStreamHandler interface [arrow]
Posted by "westonpace (via GitHub)" <gi...@apache.org>.
westonpace commented on code in PR #43632:
URL: https://github.com/apache/arrow/pull/43632#discussion_r1767375952
##########
cpp/src/arrow/c/abi.h:
##########
@@ -228,6 +228,207 @@ struct ArrowDeviceArrayStream {
#endif // ARROW_C_DEVICE_STREAM_INTERFACE
+#ifndef ARROW_C_ASYNC_STREAM_INTERFACE
+#define ARROW_C_ASYNC_STREAM_INTERFACE
+
+// ArrowAsyncTask represents available data from a producer that was passed to
+// an invocation of `on_next_task` on the ArrowAsyncDeviceStreamHandler.
+//
+// The reason for this Task approach instead of the Async interface returning
+// the Array directly is to allow for more complex thread handling and reducing
+// context switching and data transfers between CPU cores (e.g. from one L1/L2
+// cache to another) if desired.
+//
+// For example, the `on_next_task` callback can be called when data is ready, while
+// the producer puts potential "decoding" logic in the `ArrowAsyncTask` object. This
+// allows for the producer to manage the I/O on one thread which calls `on_next_task`
+// and the consumer can determine when the decoding (producer logic in the `get_data`
+// callback of the task) occurs and on which thread, to avoid a CPU core transfer
+// (data staying in the L2 cache).
+struct ArrowAsyncTask {
+ // This callback should populate the ArrowDeviceArray associated with this task.
+ // The order of ArrowAsyncTasks provided by the producer enables a consumer to
+ // ensure the order of data to process.
+ //
+ // This function is expected to be synchronous, but should not perform any blocking
+ // I/O. Ideally it should be as cheap as possible so as to not tie up the consumer
+ // thread unnecessarily.
+ //
+ // Returns: 0 if successful, errno-compatible error otherwise.
+ //
+ // If a non-0 value is returned then it should be followed by a call to `on_error`
+ // on the appropriate ArrowAsyncDeviceStreamHandler. This is because the it's highly
+ // likely that whatever is calling this function may be entirely disconnected from
+ // the current control flow. Indicating an error here with a non-zero return allows
+ // the current flow to be aware of the error occurring, while still allowing any
+ // logging or error handling to still be centralized in the `on_error` callback of
+ // the original Async handler.
+ //
+ // Rather than a release callback, any required cleanup should be performed as part
+ // of the invocation of `get_data`. Ownership of the Array is passed to the consumer
+ // calling this, and so it must be released separately.
+ //
+ // It is only valid to call this method exactly once.
+ int (*get_data)(struct ArrowArrayTask* self, struct ArrowDeviceArray* out);
+
+ // opaque task-specific data
+ void* private_data;
+};
+
+// ArrowAsyncProducer represents a 1-to-1 relationship between an async producer
+// and consumer. This object allows the consumer to perform backpressure and flow
+// control on the asynchronous stream processing. This object must be owned by the
+// producer who creates it, and thus is responsible for cleaning it up.
+struct ArrowAsyncProducer {
+ // A consumer must call this function to start receiving on_next_task calls.
+ //
+ // It *must* be valid to call this synchronously from within `on_next_task` or
+ // `on_schema`, but this function *must not* immediately call `on_next_task` so as
+ // to avoid recursion and reentrant callbacks.
+ //
+ // After cancel has been called, additional calls to this function must be NOPs,
+ // but allowed. While not cancelled, calling this function must register the
+ // given number of additional arrays/batches to be produced with the producer.
+ // The producer should only call `on_next_task` at most the registered number
+ // of arrays before propagating backpressure.
Review Comment:
Your mermaid diagram only shows `request` being called once (IIRC) and it might be more helpful if there was someway to demonstrate it was part of a loop too. That being said, the diagram has a lot of info already so maybe it's ok.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-43631: [C][Format] Add ArrowAsyncDeviceStreamHandler interface [arrow]
Posted by "zeroshade (via GitHub)" <gi...@apache.org>.
zeroshade commented on code in PR #43632:
URL: https://github.com/apache/arrow/pull/43632#discussion_r1759048610
##########
docs/source/format/CDeviceDataInterface.rst:
##########
@@ -650,6 +652,331 @@ The stream source is not assumed to be thread-safe. Consumers wanting to
call ``get_next`` from several threads should ensure those calls are
serialized.
+Async Device Stream Interface
+=============================
+
+The :ref:`C stream interface <_c-device-stream-interface>` provides a synchronous
+API centered around the consumer calling the callback functions to retrieve
+the next record batch. For some bindings, use cases, and interoperability, a more
+asynchronous, producer-focused interface may be required. These scenarios can utilize
+the ``ArrowAsyncDeviceStreamHandler``.
+
+Semantics
+---------
+
+Rather than the producer providing a structure of callbacks for a consumer to
+call and retrieve records, the Async interface is a consumer allocated structure.
+The consumer allocated struct provides handler callbacks for the producer to call
+when the schema and chunks of data are available, rather than the consumer using
+a blocking pull-style iteration.
+
+In addition to the ``ArrowAsyncDeviceStreamHandler``, there are also two additional
+structs used for the full data flow: ``ArrowAsyncTask`` and ``ArrowAsyncProducer``.
+
+Structure Definition
+--------------------
+
+The C device async stream interface is defined with a single ``struct`` definition:
+
+.. code-block:: c
+
+ #ifndef ARROW_C_ASYNC_STREAM_INTERFACE
+ #define ARROW_C_ASYNC_STREAM_INTERFACE
+
+ struct ArrowAsyncTask {
+ int (*get_data)(struct ArrowArrayTask* self, struct ArrowDeviceArray* out);
+
+ void* private_data;
+ };
+
+ struct ArrowAsyncProducer {
+ void (*request)(struct ArrowAsyncProducer* self, uint64_t n);
+ void (*cancel)(struct ArrowAsyncProducer* self);
+
+ void (*release)(struct ArrowAsyncProducer* self);
+ void* private_data;
+ };
+
+ struct ArrowAsyncDeviceStreamHandler {
+ // handlers
+ int (*on_schema)(struct ArrowAsyncDeviceStreamHandler* self,
+ struct ArrowAsyncProducer* producer,
+ struct ArrowSchema* stream_schema, const char* addl_metadata);
+ int (*on_next_task)(struct ArrowAsyncDeviceStreamHandler* self,
+ struct ArrowAsyncTask* task, const char* metadata);
+ void (*on_error)(struct ArrowAsyncDeviceStreamHandler* self,
+ int code, const char* message, const char* metadata);
+
+ // release callback
+ void (*release)(struct ArrowAsyncDeviceStreamHandler* self);
+
+ // opaque handler-specific data
+ void* private_data;
+ };
+
+ #endif // ARROW_C_ASYNC_STREAM_INTERFACE
+
+.. note::
+ The canonical guard ``ARROW_C_ASYNC_STREAM_INTERFACE`` is meant to avoid
+ duplicate definitions if two projects copy the C async stream interface
+ definitions into their own headers, and a third-party project includes
+ from these two projects. It is therefore important that this guard is kept
+ exactly as-is when these definitions are copied.
+
+The ArrowAsyncDeviceStreamHandler structure
+'''''''''''''''''''''''''''''''''''''''''''
+
+The structure has the following fields:
+
+.. c:member:: int (*ArrowAsyncDeviceStreamHandler.on_schema)(struct ArrowAsyncDeviceStreamHandler*, struct ArrowAsyncProducer*, struct ArrowSchema*, const char*)
+
+ *Mandatory.* Handler for receiving the schema of the stream. All records should
+ match the provided schema. If successful, the function should return 0, otherwise
+ it should return an ``errno``-compatible error code.
+
+ The ``const char*`` parameter exists for producers to provide any extra contextual information
+ they want, such as the total number of rows in the stream, statistics, or otherwise. This is
+ encoded in the same format as :c:member:`ArrowSchema.metadata`. If not ``NULL``,
+ the lifetime is only the scope of the call to this function. A consumer who wants to maintain
+ the additional metadata beyond the lifetime of this call *MUST* copy the value themselves.
+
+ Unless the ``on_error`` handler is called, this will always get called exactly once and will be
+ the first method called on this object. As such the producer *MUST* provide an ``ArrowAsyncProducer``
+ object when calling this function to allow the consumer to manage back-pressure and flow control.
+ The producer maintains ownership of the ``ArrowAsyncProducer`` and must clean it up before or after
+ calling the release callback on this object.
+
+ A producer that receives a non-zero result here must not subsequently call anything other than
+ the release callback on this object.
+
+.. c:member:: int (*ArrowAsyncDeviceStreamHandler.on_next_task)(struct ArrowAsyncDeviceStreamHandler*, struct ArrowAsyncTask*, const char*)
+
+ *Mandatory.* Handler to be called when a new record is available for processing. The
+ schema for each record should be the same as the schema that ``on_schema`` was called with.
+ If successfully handled, the function should return 0, otherwise it should return an
+ ``errno``-compatible error code.
+
+ Rather than passing the record itself it receives an ``ArrowAsyncTask`` instead to facilitate
+ better consumer-focused thread control as far as receiving the data. A call to this function
+ simply indicates that data is available via the provided task.
+
+ The ``const char*`` parameter exists for producers to provide any extra contextual information
+ they want. This is encoded in the same format as :c:member:`ArrowSchema.metadata`. If not ``NULL``,
+ the lifetime is only the scope of the call to this function. A consumer who wants to maintain
+ the additional metadata beyond the lifetime of this call *MUST* copy the value themselves.
+
+ A producer *MUST NOT* call this concurrently from multiple different threads.
+
+ The :c:member:`ArrowAsyncProducer.request` callback must be called to start receiving calls to this
+ handler.
+
+ This function *MUST* be able to be called re-entrantly on the same thread to allow for the
+ :c:member:`ArrowAsyncProducer.request` callback to potentially call this method recursively.
+
+.. c:member:: void (*ArrowAsyncDeviceStreamHandler.on_error)(struct ArrowAsyncDeviceStreamHandler, int, const char*, const char*)
+
+ *Mandatory.* Handler to be called when an error is encountered by the producer. After calling
+ this, the ``release`` callback will be called as the last call on this struct. The parameters
+ are an ``errno``-compatible error code and an optional error message and metadata.
+
+ If the message and metadata are not ``NULL``, their lifetime is only valid during the scope
+ of this call. A consumer who wants to maintain these values past the return of this function
+ *MUST* copy the values themselves.
+
+ If the metadata parameter is not ``NULL``, to provide key-value error metadata, then it should
+ be encoded identically to the way that metadata is encoded in :c:member:`ArrowSchema.metadata`.
+
+ It is valid for this to be called by a producer with or without a preceding call to
+ :c:member:`ArrowAsyncProducer.request`. This callback *MUST NOT* call any methods of an
+ ``ArrowAsyncProducer`` object.
+
+.. c:member:: void (*ArrowAsyncDeviceStreamHandler.release)(struct ArrowAsyncDeviceStreamHandler*)
+
+ *Mandatory.* A pointer to a consumer-provided release callback for the handler.
+
+ It is valid for this to be called by a producer with or without a preceding call to
+ :c:member:`ArrowAsyncProducer.request`. This must not call any methods of an ``ArrowAsyncProducer``
+ object.
+
+.. c:member:: void* ArrowAsyncDeviceStreamHandler.private_data
+
+ *Optional.* An opaque pointer to consumer-provided private data.
+
+ Producers *MUST NOT* process this member. Lifetime of this member is handled by
+ the consumer, and especially by the release callback.
+
Review Comment:
A producer signals this by calling `on_next_task` with a `NULL` pointer for the `ArrowArrayTask`.
```
// Handler for receiving data. This is called when data is available providing an
// ArrowAsyncTask struct to signify it. The producer indicates the end of the stream
// by passing NULL as the value for the task rather than a valid pointer to a task.
```
I'll add that to the rst file, looks like i missed it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-43631: [C][Format] Add ArrowAsyncDeviceStreamHandler interface [arrow]
Posted by "pitrou (via GitHub)" <gi...@apache.org>.
pitrou commented on code in PR #43632:
URL: https://github.com/apache/arrow/pull/43632#discussion_r1764759345
##########
docs/source/format/CDeviceDataInterface.rst:
##########
@@ -650,6 +652,331 @@ The stream source is not assumed to be thread-safe. Consumers wanting to
call ``get_next`` from several threads should ensure those calls are
serialized.
+Async Device Stream Interface
+=============================
+
+The :ref:`C stream interface <_c-device-stream-interface>` provides a synchronous
+API centered around the consumer calling the callback functions to retrieve
+the next record batch. For some bindings, use cases, and interoperability, a more
+asynchronous, producer-focused interface may be required. These scenarios can utilize
+the ``ArrowAsyncDeviceStreamHandler``.
+
+Semantics
+---------
+
+Rather than the producer providing a structure of callbacks for a consumer to
+call and retrieve records, the Async interface is a consumer allocated structure.
+The consumer allocated struct provides handler callbacks for the producer to call
+when the schema and chunks of data are available, rather than the consumer using
+a blocking pull-style iteration.
+
+In addition to the ``ArrowAsyncDeviceStreamHandler``, there are also two additional
+structs used for the full data flow: ``ArrowAsyncTask`` and ``ArrowAsyncProducer``.
+
+Structure Definition
+--------------------
+
+The C device async stream interface is defined with a single ``struct`` definition:
+
+.. code-block:: c
+
+ #ifndef ARROW_C_ASYNC_STREAM_INTERFACE
+ #define ARROW_C_ASYNC_STREAM_INTERFACE
+
+ struct ArrowAsyncTask {
+ int (*get_data)(struct ArrowArrayTask* self, struct ArrowDeviceArray* out);
+
+ void* private_data;
+ };
+
+ struct ArrowAsyncProducer {
+ void (*request)(struct ArrowAsyncProducer* self, uint64_t n);
+ void (*cancel)(struct ArrowAsyncProducer* self);
+
+ void (*release)(struct ArrowAsyncProducer* self);
+ void* private_data;
+ };
+
+ struct ArrowAsyncDeviceStreamHandler {
+ // handlers
+ int (*on_schema)(struct ArrowAsyncDeviceStreamHandler* self,
+ struct ArrowAsyncProducer* producer,
+ struct ArrowSchema* stream_schema, const char* addl_metadata);
+ int (*on_next_task)(struct ArrowAsyncDeviceStreamHandler* self,
+ struct ArrowAsyncTask* task, const char* metadata);
+ void (*on_error)(struct ArrowAsyncDeviceStreamHandler* self,
+ int code, const char* message, const char* metadata);
+
+ // release callback
+ void (*release)(struct ArrowAsyncDeviceStreamHandler* self);
+
+ // opaque handler-specific data
+ void* private_data;
+ };
+
+ #endif // ARROW_C_ASYNC_STREAM_INTERFACE
+
+.. note::
+ The canonical guard ``ARROW_C_ASYNC_STREAM_INTERFACE`` is meant to avoid
+ duplicate definitions if two projects copy the C async stream interface
+ definitions into their own headers, and a third-party project includes
+ from these two projects. It is therefore important that this guard is kept
+ exactly as-is when these definitions are copied.
+
+The ArrowAsyncDeviceStreamHandler structure
+'''''''''''''''''''''''''''''''''''''''''''
+
+The structure has the following fields:
+
+.. c:member:: int (*ArrowAsyncDeviceStreamHandler.on_schema)(struct ArrowAsyncDeviceStreamHandler*, struct ArrowAsyncProducer*, struct ArrowSchema*, const char*)
+
+ *Mandatory.* Handler for receiving the schema of the stream. All records should
+ match the provided schema. If successful, the function should return 0, otherwise
+ it should return an ``errno``-compatible error code.
+
+ The ``const char*`` parameter exists for producers to provide any extra contextual information
+ they want, such as the total number of rows in the stream, statistics, or otherwise. This is
+ encoded in the same format as :c:member:`ArrowSchema.metadata`. If not ``NULL``,
+ the lifetime is only the scope of the call to this function. A consumer who wants to maintain
+ the additional metadata beyond the lifetime of this call *MUST* copy the value themselves.
+
+ Unless the ``on_error`` handler is called, this will always get called exactly once and will be
+ the first method called on this object. As such the producer *MUST* provide an ``ArrowAsyncProducer``
+ object when calling this function to allow the consumer to manage back-pressure and flow control.
+ The producer maintains ownership of the ``ArrowAsyncProducer`` and must clean it up before or after
+ calling the release callback on this object.
+
+ A producer that receives a non-zero result here must not subsequently call anything other than
+ the release callback on this object.
+
+.. c:member:: int (*ArrowAsyncDeviceStreamHandler.on_next_task)(struct ArrowAsyncDeviceStreamHandler*, struct ArrowAsyncTask*, const char*)
+
+ *Mandatory.* Handler to be called when a new record is available for processing. The
+ schema for each record should be the same as the schema that ``on_schema`` was called with.
+ If successfully handled, the function should return 0, otherwise it should return an
+ ``errno``-compatible error code.
+
+ Rather than passing the record itself it receives an ``ArrowAsyncTask`` instead to facilitate
+ better consumer-focused thread control as far as receiving the data. A call to this function
+ simply indicates that data is available via the provided task.
+
+ The ``const char*`` parameter exists for producers to provide any extra contextual information
+ they want. This is encoded in the same format as :c:member:`ArrowSchema.metadata`. If not ``NULL``,
+ the lifetime is only the scope of the call to this function. A consumer who wants to maintain
+ the additional metadata beyond the lifetime of this call *MUST* copy the value themselves.
+
+ A producer *MUST NOT* call this concurrently from multiple different threads.
+
+ The :c:member:`ArrowAsyncProducer.request` callback must be called to start receiving calls to this
+ handler.
+
+ This function *MUST* be able to be called re-entrantly on the same thread to allow for the
+ :c:member:`ArrowAsyncProducer.request` callback to potentially call this method recursively.
+
+.. c:member:: void (*ArrowAsyncDeviceStreamHandler.on_error)(struct ArrowAsyncDeviceStreamHandler, int, const char*, const char*)
+
+ *Mandatory.* Handler to be called when an error is encountered by the producer. After calling
+ this, the ``release`` callback will be called as the last call on this struct. The parameters
+ are an ``errno``-compatible error code and an optional error message and metadata.
+
+ If the message and metadata are not ``NULL``, their lifetime is only valid during the scope
+ of this call. A consumer who wants to maintain these values past the return of this function
+ *MUST* copy the values themselves.
+
+ If the metadata parameter is not ``NULL``, to provide key-value error metadata, then it should
+ be encoded identically to the way that metadata is encoded in :c:member:`ArrowSchema.metadata`.
+
+ It is valid for this to be called by a producer with or without a preceding call to
+ :c:member:`ArrowAsyncProducer.request`. This callback *MUST NOT* call any methods of an
+ ``ArrowAsyncProducer`` object.
+
+.. c:member:: void (*ArrowAsyncDeviceStreamHandler.release)(struct ArrowAsyncDeviceStreamHandler*)
+
+ *Mandatory.* A pointer to a consumer-provided release callback for the handler.
+
+ It is valid for this to be called by a producer with or without a preceding call to
+ :c:member:`ArrowAsyncProducer.request`. This must not call any methods of an ``ArrowAsyncProducer``
+ object.
+
+.. c:member:: void* ArrowAsyncDeviceStreamHandler.private_data
+
+ *Optional.* An opaque pointer to consumer-provided private data.
+
+ Producers *MUST NOT* process this member. Lifetime of this member is handled by
+ the consumer, and especially by the release callback.
+
+The ArrowAsyncTask structure
+''''''''''''''''''''''''''''
+
+The purpose of using a Task object rather than passing the array directly to the ``on_next``
+callback is to allow for more complex and efficient thread handling. Utilizing a Task
+object allows for a producer to separate the "decoding" logic from the I/O, enabling a
+consumer to avoid transferring data between CPU cores (e.g. from one L1/L2 cache to another).
+
+This producer-provided structure has the following fields:
+
+.. c:member:: int (*ArrowArrayTask.get_data)(struct ArrowArrayTask*, struct ArrowDeviceArray*)
Review Comment:
If it's as cheap as possible, then the whole argument about offloading to consumer threads does not stand, does it?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-43631: [C][Format] Add ArrowAsyncDeviceStreamHandler interface [arrow]
Posted by "zeroshade (via GitHub)" <gi...@apache.org>.
zeroshade merged PR #43632:
URL: https://github.com/apache/arrow/pull/43632
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-43631: [C][Format] Add ArrowAsyncDeviceStreamHandler interface [arrow]
Posted by "leventov (via GitHub)" <gi...@apache.org>.
leventov commented on code in PR #43632:
URL: https://github.com/apache/arrow/pull/43632#discussion_r1721539510
##########
cpp/src/arrow/c/abi.h:
##########
@@ -228,6 +228,65 @@ struct ArrowDeviceArrayStream {
#endif // ARROW_C_DEVICE_STREAM_INTERFACE
+#ifndef ARROW_C_ASYNC_STREAM_INTERFACE
+#define ARROW_C_ASYNC_STREAM_INTERFACE
+
+// Similar to ArrowDeviceArrayStream, except designed for an asynchronous
+// style of interaction. While ArrowDeviceArrayStream provides producer
+// defined callbacks, this is intended to be created by the consumer instead.
+// The consumer passes this handler to the producer, which in turn uses the
+// callbacks to inform the consumer of events in the stream.
+struct ArrowAsyncDeviceStreamHandler {
+ // Handler for receiving a schema. The passed in stream_schema should be
+ // released or moved by the handler (producer is giving ownership of it to
+ // the handler).
+ //
+ // The `extension_param` argument can be null or can be used by a producer
+ // to pass arbitrary extra information to the consumer (such as total number
+ // of rows, context info, or otherwise).
+ //
+ // Return value: 0 if successful, `errno`-compatible error otherwise
Review Comment:
> Right, but it might make more sense to just have a unified cancellation mechanism. And again, what is the producer really supposed to do with the error code? It's just going to drop it on the floor.
I think it can be useful for monitoring on the middleware/transport level, if the error is proxied to a remote server.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-43631: [C][Format] Add ArrowAsyncDeviceStreamHandler interface [arrow]
Posted by "lidavidm (via GitHub)" <gi...@apache.org>.
lidavidm commented on code in PR #43632:
URL: https://github.com/apache/arrow/pull/43632#discussion_r1716290263
##########
cpp/src/arrow/c/abi.h:
##########
@@ -228,6 +228,65 @@ struct ArrowDeviceArrayStream {
#endif // ARROW_C_DEVICE_STREAM_INTERFACE
+#ifndef ARROW_C_ASYNC_STREAM_INTERFACE
+#define ARROW_C_ASYNC_STREAM_INTERFACE
+
+// Similar to ArrowDeviceArrayStream, except designed for an asynchronous
+// style of interaction. While ArrowDeviceArrayStream provides producer
+// defined callbacks, this is intended to be created by the consumer instead.
+// The consumer passes this handler to the producer, which in turn uses the
+// callbacks to inform the consumer of events in the stream.
+struct ArrowAsyncDeviceStreamHandler {
+ // Handler for receiving a schema. The passed in stream_schema should be
+ // released or moved by the handler (producer is giving ownership of it to
+ // the handler).
+ //
+ // The `extension_param` argument can be null or can be used by a producer
+ // to pass arbitrary extra information to the consumer (such as total number
+ // of rows, context info, or otherwise).
+ //
+ // Return value: 0 if successful, `errno`-compatible error otherwise
+ int (*on_schema)(struct ArrowAsyncDeviceStreamHandler* self,
+ struct ArrowSchema* stream_schema, void* extension_param);
+
+ // Handler for receiving an array/record batch. Always called at least once
+ // unless an error is encountered (which would result in calling on_error).
+ // An empty/released array is passed to indicate the end of the stream if no
+ // errors have been encountered.
+ //
+ // The `extension_param` argument can be null or can be used by a producer
+ // to pass arbitrary extra information to the consumer.
+ //
+ // Return value: 0 if successful, `errno`-compatible error otherwise.
+ int (*on_next)(struct ArrowAsyncDeviceStreamHandler* self,
+ struct ArrowDeviceArray* next, void* extension_param);
Review Comment:
For some gRPC implementations, tying up the thread is explicitly not recommended so we may not want to encourage that
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-43631: [C][Format] Add ArrowAsyncDeviceStreamHandler interface [arrow]
Posted by "paleolimbot (via GitHub)" <gi...@apache.org>.
paleolimbot commented on code in PR #43632:
URL: https://github.com/apache/arrow/pull/43632#discussion_r1712469719
##########
cpp/src/arrow/c/abi.h:
##########
@@ -228,6 +228,65 @@ struct ArrowDeviceArrayStream {
#endif // ARROW_C_DEVICE_STREAM_INTERFACE
+#ifndef ARROW_C_ASYNC_STREAM_INTERFACE
+#define ARROW_C_ASYNC_STREAM_INTERFACE
+
+// Similar to ArrowDeviceArrayStream, except designed for an asynchronous
+// style of interaction. While ArrowDeviceArrayStream provides producer
+// defined callbacks, this is intended to be created by the consumer instead.
+// The consumer passes this handler to the producer, which in turn uses the
+// callbacks to inform the consumer of events in the stream.
+struct ArrowAsyncDeviceStreamHandler {
+ // Handler for receiving a schema. The passed in stream_schema should be
+ // released or moved by the handler (producer is giving ownership of it to
+ // the handler).
+ //
+ // The `extension_param` argument can be null or can be used by a producer
Review Comment:
I am lukewarm on a vague extension parameter here...maybe `metadata` or a JSON string if we really need additional key/value information outside the scope of the schema's metadata?
##########
cpp/src/arrow/c/abi.h:
##########
@@ -228,6 +228,65 @@ struct ArrowDeviceArrayStream {
#endif // ARROW_C_DEVICE_STREAM_INTERFACE
+#ifndef ARROW_C_ASYNC_STREAM_INTERFACE
+#define ARROW_C_ASYNC_STREAM_INTERFACE
+
+// Similar to ArrowDeviceArrayStream, except designed for an asynchronous
+// style of interaction. While ArrowDeviceArrayStream provides producer
+// defined callbacks, this is intended to be created by the consumer instead.
+// The consumer passes this handler to the producer, which in turn uses the
+// callbacks to inform the consumer of events in the stream.
+struct ArrowAsyncDeviceStreamHandler {
+ // Handler for receiving a schema. The passed in stream_schema should be
+ // released or moved by the handler (producer is giving ownership of it to
+ // the handler).
+ //
+ // The `extension_param` argument can be null or can be used by a producer
+ // to pass arbitrary extra information to the consumer (such as total number
+ // of rows, context info, or otherwise).
+ //
+ // Return value: 0 if successful, `errno`-compatible error otherwise
Review Comment:
I imagine the consumer should be able to return something to tell the producer to stop (I agree this should be explicit, though, and errno might not be the most precise tool but it might be slightly more informative than a `bool` or invented enum).
For argument's sake: If a non-zero value is returned, the producer must not call any other callbacks except the release callback.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-43631: [C][Format] Add ArrowAsyncDeviceStreamHandler interface [arrow]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #43632:
URL: https://github.com/apache/arrow/pull/43632#issuecomment-2433474627
Revision: 2ff218b337e075baad7859f04db4c11b645c0d31
Submitted crossbow builds: [ursacomputing/crossbow @ actions-71ea1ce117](https://github.com/ursacomputing/crossbow/branches/all?query=actions-71ea1ce117)
|Task|Status|
|----|------|
|preview-docs|[](https://github.com/ursacomputing/crossbow/actions/runs/11488227535/job/31974520158)|
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-43631: [C][Format] Add ArrowAsyncDeviceStreamHandler interface [arrow]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #43632:
URL: https://github.com/apache/arrow/pull/43632#issuecomment-2440086486
Revision: 0e3574b4fb634660a75169622dd8e4aa33599423
Submitted crossbow builds: [ursacomputing/crossbow @ actions-e929f1681c](https://github.com/ursacomputing/crossbow/branches/all?query=actions-e929f1681c)
|Task|Status|
|----|------|
|preview-docs|[](https://github.com/ursacomputing/crossbow/actions/runs/11542042834/job/32124780735)|
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-43631: [C][Format] Add ArrowAsyncDeviceStreamHandler interface [arrow]
Posted by "ianmcook (via GitHub)" <gi...@apache.org>.
ianmcook commented on PR #43632:
URL: https://github.com/apache/arrow/pull/43632#issuecomment-2440482535
Docs preview: http://crossbow.voltrondata.com/pr_docs/43632/format/CDeviceDataInterface.html#async-device-stream-interface
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-43631: [C][Format] Add ArrowAsyncDeviceStreamHandler interface [arrow]
Posted by "zeroshade (via GitHub)" <gi...@apache.org>.
zeroshade commented on code in PR #43632:
URL: https://github.com/apache/arrow/pull/43632#discussion_r1829312370
##########
cpp/src/arrow/c/abi.h:
##########
@@ -356,8 +356,7 @@ struct ArrowAsyncDeviceStreamHandler {
// A producer that receives a non-zero return here should stop producing and eventually
// call release instead.
int (*on_schema)(struct ArrowAsyncDeviceStreamHandler* self,
- struct ArrowAsyncProducer* producer, struct ArrowSchema* stream_schema,
- const char* addl_metadata);
+ struct ArrowSchema* stream_schema, const char* addl_metadata);
Review Comment:
Sounds good, I'll update this and the other two PRs accordingly
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-43631: [C][Format] Add ArrowAsyncDeviceStreamHandler interface [arrow]
Posted by "raulcd (via GitHub)" <gi...@apache.org>.
raulcd commented on code in PR #43632:
URL: https://github.com/apache/arrow/pull/43632#discussion_r1816790916
##########
docs/source/format/CDeviceDataInterface.rst:
##########
@@ -650,6 +652,356 @@ The stream source is not assumed to be thread-safe. Consumers wanting to
call ``get_next`` from several threads should ensure those calls are
serialized.
+Async Device Stream Interface
+=============================
+
+.. warning::
+
+ Experimental: The Aync C Device Stream interface is experimental in its current
Review Comment:
```suggestion
Experimental: The Async C Device Stream interface is experimental in its current
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-43631: [C][Format] Add ArrowAsyncDeviceStreamHandler interface [arrow]
Posted by "kou (via GitHub)" <gi...@apache.org>.
kou commented on code in PR #43632:
URL: https://github.com/apache/arrow/pull/43632#discussion_r1814127878
##########
docs/source/format/CDeviceDataInterface.rst:
##########
@@ -650,6 +652,340 @@ The stream source is not assumed to be thread-safe. Consumers wanting to
call ``get_next`` from several threads should ensure those calls are
serialized.
+Async Device Stream Interface
+=============================
+
+The :ref:`C stream interface <c-device-stream-interface>` provides a synchronous
+API centered around the consumer calling the producer functions to retrieve
+the next record batch. For concurrent communication between producer and consumer,
+the ``ArrowAsyncDeviceStreamHandler`` can be used. This interface is non-opinionated
+and may fit into different concurrent communication models.
+
+Semantics
+---------
+
+Rather than the producer providing a structure of callbacks for a consumer to
+call and retrieve records, the Async interface is a structure allocated and populated by the consumer.
+The consumer allocated struct provides handler callbacks for the producer to call
+when the schema and chunks of data are available.
+
+In addition to the ``ArrowAsyncDeviceStreamHandler``, there are also two additional
+structs used for the full data flow: ``ArrowAsyncTask`` and ``ArrowAsyncProducer``.
+
+Structure Definition
+--------------------
+
+The C device async stream interface consists of three ``struct`` definitions:
+
+.. code-block:: c
+
+ #ifndef ARROW_C_ASYNC_STREAM_INTERFACE
+ #define ARROW_C_ASYNC_STREAM_INTERFACE
+
+ struct ArrowAsyncTask {
+ int (*extract_data)(struct ArrowArrayTask* self, struct ArrowDeviceArray* out);
+
+ void* private_data;
+ };
+
+ struct ArrowAsyncProducer {
+ void (*request)(struct ArrowAsyncProducer* self, int64_t n);
+ void (*cancel)(struct ArrowAsyncProducer* self);
+
+ void (*release)(struct ArrowAsyncProducer* self);
+ void* private_data;
+ };
+
+ struct ArrowAsyncDeviceStreamHandler {
+ // consumer-specific handlers
+ int (*on_schema)(struct ArrowAsyncDeviceStreamHandler* self,
+ struct ArrowAsyncProducer* producer,
+ struct ArrowSchema* stream_schema, const char* addl_metadata);
+ int (*on_next_task)(struct ArrowAsyncDeviceStreamHandler* self,
+ struct ArrowAsyncTask* task, const char* metadata);
+ void (*on_error)(struct ArrowAsyncDeviceStreamHandler* self,
+ int code, const char* message, const char* metadata);
+
+ // release callback
+ void (*release)(struct ArrowAsyncDeviceStreamHandler* self);
+
+ // opaque handler-specific data
+ void* private_data;
+ };
+
+ #endif // ARROW_C_ASYNC_STREAM_INTERFACE
+
+.. note::
+ The canonical guard ``ARROW_C_ASYNC_STREAM_INTERFACE`` is meant to avoid
+ duplicate definitions if two projects copy the C async stream interface
+ definitions into their own headers, and a third-party project includes
+ from these two projects. It is therefore important that this guard is kept
+ exactly as-is when these definitions are copied.
+
+The ArrowAsyncDeviceStreamHandler structure
+'''''''''''''''''''''''''''''''''''''''''''
+
+The structure has the following fields:
+
+.. c:member:: int (*ArrowAsyncDeviceStreamHandler.on_schema)(struct ArrowAsyncDeviceStreamHandler*, struct ArrowAsyncProducer*, struct ArrowSchema*, const char*)
+
+ *Mandatory.* Handler for receiving the schema of the stream. All incoming records should
+ match the provided schema. If successful, the function should return 0, otherwise
+ it should return an ``errno``-compatible error code.
+
+ The ``const char*`` parameter exists for producers to provide any extra contextual information
+ they want, such as the total number of rows in the stream, statistics, or otherwise. This is
+ encoded in the same format as :c:member:`ArrowSchema.metadata`. If not ``NULL``,
+ the lifetime is only the scope of the call to this function. A consumer who wants to maintain
+ the additional metadata beyond the lifetime of this call *MUST* copy the value themselves.
+
+ Unless the ``on_error`` handler is called, this will always get called exactly once and will be
+ the first method called on this object. As such the producer *MUST* provide an ``ArrowAsyncProducer``
+ object when calling this function to allow the consumer to apply back-pressure and control the flow of data.
+ The producer maintains ownership of the ``ArrowAsyncProducer`` and must clean it up *after*
+ calling the release callback on the ``ArrowAsyncDeviceStreamHandler``.
+
+ A producer that receives a non-zero result here must not subsequently call anything other than
+ the release callback on this object.
+
+.. c:member:: int (*ArrowAsyncDeviceStreamHandler.on_next_task)(struct ArrowAsyncDeviceStreamHandler*, struct ArrowAsyncTask*, const char*)
+
+ *Mandatory.* Handler to be called when a new record is available for processing. The
+ schema for each record should be the same as the schema that ``on_schema`` was called with.
+ If successfully handled, the function should return 0, otherwise it should return an
+ ``errno``-compatible error code.
+
+ Rather than passing the record itself it receives an ``ArrowAsyncTask`` instead to facilitate
+ better consumer-focused thread control as far as receiving the data. A call to this function
+ simply indicates that data is available via the provided task.
+
+ The producer signals the end of the stream by passing ``NULL`` for the ``ArrowAsyncTask``
+ pointer instead of a valid address. This task object is only valid during the lifetime of
+ this function call. If the consumer wants to use the task beyond the scope of this method, it
+ must copy or move its contents to a new ArrowAsyncTask object.
+
+ The ``const char*`` parameter exists for producers to provide any extra contextual information
+ they want. This is encoded in the same format as :c:member:`ArrowSchema.metadata`. If not ``NULL``,
+ the lifetime is only the scope of the call to this function. A consumer who wants to maintain
+ the additional metadata beyond the lifetime of this call *MUST* copy the value themselves.
+
+ A producer *MUST NOT* call this concurrently from multiple threads.
+
+ The :c:member:`ArrowAsyncProducer.request` callback must be called to start receiving calls to this
+ handler.
+
+.. c:member:: void (*ArrowAsyncDeviceStreamHandler.on_error)(struct ArrowAsyncDeviceStreamHandler, int, const char*, const char*)
+
+ *Mandatory.* Handler to be called when an error is encountered by the producer. After calling
+ this, the ``release`` callback will be called as the last call on this struct. The parameters
+ are an ``errno``-compatible error code and an optional error message and metadata.
+
+ If the message and metadata are not ``NULL``, their lifetime is only valid during the scope
+ of this call. A consumer who wants to maintain these values past the return of this function
+ *MUST* copy the values themselves.
+
+ If the metadata parameter is not ``NULL``, to provide key-value error metadata, then it should
+ be encoded identically to the way that metadata is encoded in :c:member:`ArrowSchema.metadata`.
+
+ It is valid for this to be called by a producer with or without a preceding call to
+ :c:member:`ArrowAsyncProducer.request`. This callback *MUST NOT* call any methods of an
+ ``ArrowAsyncProducer`` object.
+
+.. c:member:: void (*ArrowAsyncDeviceStreamHandler.release)(struct ArrowAsyncDeviceStreamHandler*)
+
+ *Mandatory.* A pointer to a consumer-provided release callback for the handler.
+
+ It is valid for this to be called by a producer with or without a preceding call to
+ :c:member:`ArrowAsyncProducer.request`. This must not call any methods of an ``ArrowAsyncProducer``
+ object.
+
+.. c:member:: void* ArrowAsyncDeviceStreamHandler.private_data
+
+ *Optional.* An opaque pointer to consumer-provided private data.
+
+ Producers *MUST NOT* process this member. Lifetime of this member is handled by
+ the consumer, and especially by the release callback.
+
+The ArrowAsyncTask structure
+''''''''''''''''''''''''''''
+
+The purpose of using a Task object rather than passing the array directly to the ``on_next``
+callback is to allow for more complex and efficient thread handling. Utilizing a Task
+object allows for a producer to separate the "decoding" logic from the I/O, enabling a
+consumer to avoid transferring data between CPU cores (e.g. from one L1/L2 cache to another).
+
+This producer-provided structure has the following fields:
+
+.. c:member:: int (*ArrowArrayTask.extract_data)(struct ArrowArrayTask*, struct ArrowDeviceArray*)
+
+ *Mandatory.* A callback to populate the provided ``ArrowDeviceArray`` with the available data.
+ The order of ArrowAsyncTasks provided by the producer enables a consumer to know the order of
Review Comment:
```suggestion
The order of ``ArrowAsyncTasks`` provided by the producer enables a consumer to know the order of
```
##########
docs/source/format/CDeviceDataInterface.rst:
##########
@@ -650,6 +652,340 @@ The stream source is not assumed to be thread-safe. Consumers wanting to
call ``get_next`` from several threads should ensure those calls are
serialized.
+Async Device Stream Interface
+=============================
+
+The :ref:`C stream interface <c-device-stream-interface>` provides a synchronous
+API centered around the consumer calling the producer functions to retrieve
+the next record batch. For concurrent communication between producer and consumer,
+the ``ArrowAsyncDeviceStreamHandler`` can be used. This interface is non-opinionated
+and may fit into different concurrent communication models.
+
+Semantics
+---------
+
+Rather than the producer providing a structure of callbacks for a consumer to
+call and retrieve records, the Async interface is a structure allocated and populated by the consumer.
+The consumer allocated struct provides handler callbacks for the producer to call
+when the schema and chunks of data are available.
+
+In addition to the ``ArrowAsyncDeviceStreamHandler``, there are also two additional
+structs used for the full data flow: ``ArrowAsyncTask`` and ``ArrowAsyncProducer``.
+
+Structure Definition
+--------------------
+
+The C device async stream interface consists of three ``struct`` definitions:
+
+.. code-block:: c
+
+ #ifndef ARROW_C_ASYNC_STREAM_INTERFACE
+ #define ARROW_C_ASYNC_STREAM_INTERFACE
+
+ struct ArrowAsyncTask {
+ int (*extract_data)(struct ArrowArrayTask* self, struct ArrowDeviceArray* out);
+
+ void* private_data;
+ };
+
+ struct ArrowAsyncProducer {
+ void (*request)(struct ArrowAsyncProducer* self, int64_t n);
+ void (*cancel)(struct ArrowAsyncProducer* self);
+
+ void (*release)(struct ArrowAsyncProducer* self);
+ void* private_data;
+ };
+
+ struct ArrowAsyncDeviceStreamHandler {
+ // consumer-specific handlers
+ int (*on_schema)(struct ArrowAsyncDeviceStreamHandler* self,
+ struct ArrowAsyncProducer* producer,
+ struct ArrowSchema* stream_schema, const char* addl_metadata);
+ int (*on_next_task)(struct ArrowAsyncDeviceStreamHandler* self,
+ struct ArrowAsyncTask* task, const char* metadata);
+ void (*on_error)(struct ArrowAsyncDeviceStreamHandler* self,
+ int code, const char* message, const char* metadata);
+
+ // release callback
+ void (*release)(struct ArrowAsyncDeviceStreamHandler* self);
+
+ // opaque handler-specific data
+ void* private_data;
+ };
+
+ #endif // ARROW_C_ASYNC_STREAM_INTERFACE
+
+.. note::
+ The canonical guard ``ARROW_C_ASYNC_STREAM_INTERFACE`` is meant to avoid
+ duplicate definitions if two projects copy the C async stream interface
+ definitions into their own headers, and a third-party project includes
+ from these two projects. It is therefore important that this guard is kept
+ exactly as-is when these definitions are copied.
+
+The ArrowAsyncDeviceStreamHandler structure
+'''''''''''''''''''''''''''''''''''''''''''
+
+The structure has the following fields:
+
+.. c:member:: int (*ArrowAsyncDeviceStreamHandler.on_schema)(struct ArrowAsyncDeviceStreamHandler*, struct ArrowAsyncProducer*, struct ArrowSchema*, const char*)
+
+ *Mandatory.* Handler for receiving the schema of the stream. All incoming records should
+ match the provided schema. If successful, the function should return 0, otherwise
+ it should return an ``errno``-compatible error code.
+
+ The ``const char*`` parameter exists for producers to provide any extra contextual information
+ they want, such as the total number of rows in the stream, statistics, or otherwise. This is
+ encoded in the same format as :c:member:`ArrowSchema.metadata`. If not ``NULL``,
+ the lifetime is only the scope of the call to this function. A consumer who wants to maintain
+ the additional metadata beyond the lifetime of this call *MUST* copy the value themselves.
+
+ Unless the ``on_error`` handler is called, this will always get called exactly once and will be
+ the first method called on this object. As such the producer *MUST* provide an ``ArrowAsyncProducer``
+ object when calling this function to allow the consumer to apply back-pressure and control the flow of data.
+ The producer maintains ownership of the ``ArrowAsyncProducer`` and must clean it up *after*
+ calling the release callback on the ``ArrowAsyncDeviceStreamHandler``.
+
+ A producer that receives a non-zero result here must not subsequently call anything other than
+ the release callback on this object.
+
+.. c:member:: int (*ArrowAsyncDeviceStreamHandler.on_next_task)(struct ArrowAsyncDeviceStreamHandler*, struct ArrowAsyncTask*, const char*)
+
+ *Mandatory.* Handler to be called when a new record is available for processing. The
+ schema for each record should be the same as the schema that ``on_schema`` was called with.
+ If successfully handled, the function should return 0, otherwise it should return an
+ ``errno``-compatible error code.
+
+ Rather than passing the record itself it receives an ``ArrowAsyncTask`` instead to facilitate
+ better consumer-focused thread control as far as receiving the data. A call to this function
+ simply indicates that data is available via the provided task.
+
+ The producer signals the end of the stream by passing ``NULL`` for the ``ArrowAsyncTask``
+ pointer instead of a valid address. This task object is only valid during the lifetime of
+ this function call. If the consumer wants to use the task beyond the scope of this method, it
+ must copy or move its contents to a new ArrowAsyncTask object.
+
+ The ``const char*`` parameter exists for producers to provide any extra contextual information
+ they want. This is encoded in the same format as :c:member:`ArrowSchema.metadata`. If not ``NULL``,
+ the lifetime is only the scope of the call to this function. A consumer who wants to maintain
+ the additional metadata beyond the lifetime of this call *MUST* copy the value themselves.
+
+ A producer *MUST NOT* call this concurrently from multiple threads.
+
+ The :c:member:`ArrowAsyncProducer.request` callback must be called to start receiving calls to this
+ handler.
+
+.. c:member:: void (*ArrowAsyncDeviceStreamHandler.on_error)(struct ArrowAsyncDeviceStreamHandler, int, const char*, const char*)
+
+ *Mandatory.* Handler to be called when an error is encountered by the producer. After calling
+ this, the ``release`` callback will be called as the last call on this struct. The parameters
+ are an ``errno``-compatible error code and an optional error message and metadata.
+
+ If the message and metadata are not ``NULL``, their lifetime is only valid during the scope
+ of this call. A consumer who wants to maintain these values past the return of this function
+ *MUST* copy the values themselves.
+
+ If the metadata parameter is not ``NULL``, to provide key-value error metadata, then it should
+ be encoded identically to the way that metadata is encoded in :c:member:`ArrowSchema.metadata`.
+
+ It is valid for this to be called by a producer with or without a preceding call to
+ :c:member:`ArrowAsyncProducer.request`. This callback *MUST NOT* call any methods of an
+ ``ArrowAsyncProducer`` object.
+
+.. c:member:: void (*ArrowAsyncDeviceStreamHandler.release)(struct ArrowAsyncDeviceStreamHandler*)
+
+ *Mandatory.* A pointer to a consumer-provided release callback for the handler.
+
+ It is valid for this to be called by a producer with or without a preceding call to
+ :c:member:`ArrowAsyncProducer.request`. This must not call any methods of an ``ArrowAsyncProducer``
+ object.
+
+.. c:member:: void* ArrowAsyncDeviceStreamHandler.private_data
+
+ *Optional.* An opaque pointer to consumer-provided private data.
+
+ Producers *MUST NOT* process this member. Lifetime of this member is handled by
+ the consumer, and especially by the release callback.
+
+The ArrowAsyncTask structure
+''''''''''''''''''''''''''''
+
+The purpose of using a Task object rather than passing the array directly to the ``on_next``
+callback is to allow for more complex and efficient thread handling. Utilizing a Task
+object allows for a producer to separate the "decoding" logic from the I/O, enabling a
+consumer to avoid transferring data between CPU cores (e.g. from one L1/L2 cache to another).
+
+This producer-provided structure has the following fields:
+
+.. c:member:: int (*ArrowArrayTask.extract_data)(struct ArrowArrayTask*, struct ArrowDeviceArray*)
+
+ *Mandatory.* A callback to populate the provided ``ArrowDeviceArray`` with the available data.
+ The order of ArrowAsyncTasks provided by the producer enables a consumer to know the order of
+ the data to process. If the consumer does not care about the data that is owned by this task,
+ it must still call ``extract_data`` so that the producer can perform any required cleanup. ``NULL``
+ should be passed as the device array pointer to indicate that the consumer doesn't want the
+ actual data, letting the task perform necessary cleanup.
+
+ If a non-zero value is returned from this, it should be followed only by the producer calling
+ the ``on_error`` callback of the ``ArrowAsyncDeviceStreamHandler``. Because calling this method
+ is likely to be separate from the current control flow, returning a non-zero value to signal
+ an error occuring allows the current thread to decide handle the case accordingly, while still
+ allowing all error logging and handling to be centralized in the
+ :c:member:`ArrowAsyncDeviceStreamHandler.on_error` callback.
+
+ Rather than having a separate release callback, any required cleanup should be performed as part
+ of the invocation of this callback. Ownership of the Array is given to the pointer passed in as
+ a parameter, and this array must be released separately.
+
+ It is only valid to call this method exactly once.
+
+.. c:member:: void* ArrowArrayTask.private_data
+
+ *Optional.* An opaque pointer to producer-provided private data.
+
+ Consumers *MUST NOT* process this member. Lifetime of this member is handled by
+ the producer who created this object, and should be cleaned up if necessary during
+ the call to :c:member:`ArrowArrayTask.extract_data`.
+
+The ArrowAsyncProducer structure
+''''''''''''''''''''''''''''''''
+
+This producer-provided and managed object has the following fields:
+
+.. c:member:: void (*ArrowAsyncProducer.request)(struct ArrowAsyncProducer*, uint64_t)
+
+ *Mandatory.* This function must be called by a consumer to start receiving calls to
+ :c:member:`ArrowAsyncDeviceStreamHandler.on_next_task`. It *MUST* be valid to call
+ this synchronously from within :c:member:`ArrowAsyncDeviceStreamHandler.on_next_task`
+ or :c:member:`ArrowAsyncDeviceStreamHandler.on_schema`. As a result, this function
+ *MUST NOT* synchronously call ``on_next_task`` or ``on_error`` to avoid recursive
+ and reentrant callbacks.
+
+ After ``cancel`` is called, additional calls to this function must be a NOP, but allowed.
+
+ While not cancelled, calling this function registers the given number of additional
+ arrays/batches to be produced by the producer. A producer should only call
+ the appropriate ``on_next_task`` callback up to a maximum of the total sum of calls to
+ this method before propagating back-pressure / waiting.
+
+ Any error encountered by calling request must be propagated by calling the ``on_error``
+ callback of the ``ArrowAsyncDeviceStreamHandler``.
+
+ It is invalid to call this function with a value of ``n`` that is ``<= 0``. Producers should
+ error (e.g. call ``on_error``) if receiving such a value for ``n``.
+
+.. c:member:: void (*ArrowAsyncProducer.cancel)(struct ArrowAsyncProducer*)
+
+ *Mandatory.* This function signals to the producer that it must *eventually* stop calling
+ ``on_next_task``. Calls to ``cancel`` must be idempotent and thread-safe. After calling
+ it once, subsequent calls *MUST* be a NOP. This *MUST NOT* call any consumer-side handlers
+ other than ``on_error``.
+
+ It is not required that calling ``cancel`` affect the producer *immediately*, only that it
+ must eventually stop calling ``on_next_task`` and then subsequently call ``release``
+ on the async handler object. As such, a consumer *MUST* be prepared to receive one or more
+ calls to ``on_next_task`` or ``on_error`` even after calling ``cancel`` if there are still
+ requested arrays pending.
+
+ Successful cancelling *MUST NOT* result in a producer calling
+ :c:member:`ArrowAsyncDeviceStreamHandler.on_error`, instead it should finish out any remaining
+ tasks (calling ``on_next_task`` accordingly) and eventually just call ``release``.
+
+ Any error encountered during handling a call to cancel must be reported via the ``on_error``
+ callback on the async stream handler.
+
+.. c:member:: void* ArrowAsyncProducer.private_data
+
+ *Optional.* An opaque pointer to producer-provided specific data.
+
+ Consumers *MUST NOT* process this member, the lifetime is owned by the producer
+ that constructed this object.
+
+Error Handling
+''''''''''''''
+
+Unlike the regular C Stream interface, the Async interface allows for errors to flow in
+both directions. As a result, error handling can be slightly more complex. Thus this spec
+designates the following rules:
+
+* If the producer encounters an error during processing, it should call the ``on_error``
+callback, and then call ``release`` after it returns.
+
+* If ``on_schema`` or ``on_next_task`` returns a non-zero integer value, the producer *should not*
+call the ``on_error`` callback, but instead should eventually call ``release`` at some point
+before or after any logging or processing of the error code.
Review Comment:
```suggestion
call the ``on_error`` callback, but instead should eventually call ``release`` at some point
before or after any logging or processing of the error code.
```
##########
docs/source/format/CDeviceDataInterface.rst:
##########
@@ -650,6 +652,340 @@ The stream source is not assumed to be thread-safe. Consumers wanting to
call ``get_next`` from several threads should ensure those calls are
serialized.
+Async Device Stream Interface
+=============================
+
+The :ref:`C stream interface <c-device-stream-interface>` provides a synchronous
+API centered around the consumer calling the producer functions to retrieve
+the next record batch. For concurrent communication between producer and consumer,
+the ``ArrowAsyncDeviceStreamHandler`` can be used. This interface is non-opinionated
+and may fit into different concurrent communication models.
+
+Semantics
+---------
+
+Rather than the producer providing a structure of callbacks for a consumer to
+call and retrieve records, the Async interface is a structure allocated and populated by the consumer.
+The consumer allocated struct provides handler callbacks for the producer to call
+when the schema and chunks of data are available.
+
+In addition to the ``ArrowAsyncDeviceStreamHandler``, there are also two additional
+structs used for the full data flow: ``ArrowAsyncTask`` and ``ArrowAsyncProducer``.
+
+Structure Definition
+--------------------
+
+The C device async stream interface consists of three ``struct`` definitions:
+
+.. code-block:: c
+
+ #ifndef ARROW_C_ASYNC_STREAM_INTERFACE
+ #define ARROW_C_ASYNC_STREAM_INTERFACE
+
+ struct ArrowAsyncTask {
+ int (*extract_data)(struct ArrowArrayTask* self, struct ArrowDeviceArray* out);
+
+ void* private_data;
+ };
+
+ struct ArrowAsyncProducer {
+ void (*request)(struct ArrowAsyncProducer* self, int64_t n);
+ void (*cancel)(struct ArrowAsyncProducer* self);
+
+ void (*release)(struct ArrowAsyncProducer* self);
+ void* private_data;
+ };
+
+ struct ArrowAsyncDeviceStreamHandler {
+ // consumer-specific handlers
+ int (*on_schema)(struct ArrowAsyncDeviceStreamHandler* self,
+ struct ArrowAsyncProducer* producer,
+ struct ArrowSchema* stream_schema, const char* addl_metadata);
+ int (*on_next_task)(struct ArrowAsyncDeviceStreamHandler* self,
+ struct ArrowAsyncTask* task, const char* metadata);
+ void (*on_error)(struct ArrowAsyncDeviceStreamHandler* self,
+ int code, const char* message, const char* metadata);
+
+ // release callback
+ void (*release)(struct ArrowAsyncDeviceStreamHandler* self);
+
+ // opaque handler-specific data
+ void* private_data;
+ };
+
+ #endif // ARROW_C_ASYNC_STREAM_INTERFACE
+
+.. note::
+ The canonical guard ``ARROW_C_ASYNC_STREAM_INTERFACE`` is meant to avoid
+ duplicate definitions if two projects copy the C async stream interface
+ definitions into their own headers, and a third-party project includes
+ from these two projects. It is therefore important that this guard is kept
+ exactly as-is when these definitions are copied.
+
+The ArrowAsyncDeviceStreamHandler structure
+'''''''''''''''''''''''''''''''''''''''''''
+
+The structure has the following fields:
+
+.. c:member:: int (*ArrowAsyncDeviceStreamHandler.on_schema)(struct ArrowAsyncDeviceStreamHandler*, struct ArrowAsyncProducer*, struct ArrowSchema*, const char*)
+
+ *Mandatory.* Handler for receiving the schema of the stream. All incoming records should
+ match the provided schema. If successful, the function should return 0, otherwise
+ it should return an ``errno``-compatible error code.
+
+ The ``const char*`` parameter exists for producers to provide any extra contextual information
+ they want, such as the total number of rows in the stream, statistics, or otherwise. This is
+ encoded in the same format as :c:member:`ArrowSchema.metadata`. If not ``NULL``,
+ the lifetime is only the scope of the call to this function. A consumer who wants to maintain
+ the additional metadata beyond the lifetime of this call *MUST* copy the value themselves.
+
+ Unless the ``on_error`` handler is called, this will always get called exactly once and will be
+ the first method called on this object. As such the producer *MUST* provide an ``ArrowAsyncProducer``
+ object when calling this function to allow the consumer to apply back-pressure and control the flow of data.
+ The producer maintains ownership of the ``ArrowAsyncProducer`` and must clean it up *after*
+ calling the release callback on the ``ArrowAsyncDeviceStreamHandler``.
+
+ A producer that receives a non-zero result here must not subsequently call anything other than
+ the release callback on this object.
+
+.. c:member:: int (*ArrowAsyncDeviceStreamHandler.on_next_task)(struct ArrowAsyncDeviceStreamHandler*, struct ArrowAsyncTask*, const char*)
+
+ *Mandatory.* Handler to be called when a new record is available for processing. The
+ schema for each record should be the same as the schema that ``on_schema`` was called with.
+ If successfully handled, the function should return 0, otherwise it should return an
+ ``errno``-compatible error code.
+
+ Rather than passing the record itself it receives an ``ArrowAsyncTask`` instead to facilitate
+ better consumer-focused thread control as far as receiving the data. A call to this function
+ simply indicates that data is available via the provided task.
+
+ The producer signals the end of the stream by passing ``NULL`` for the ``ArrowAsyncTask``
+ pointer instead of a valid address. This task object is only valid during the lifetime of
+ this function call. If the consumer wants to use the task beyond the scope of this method, it
+ must copy or move its contents to a new ArrowAsyncTask object.
+
+ The ``const char*`` parameter exists for producers to provide any extra contextual information
+ they want. This is encoded in the same format as :c:member:`ArrowSchema.metadata`. If not ``NULL``,
+ the lifetime is only the scope of the call to this function. A consumer who wants to maintain
+ the additional metadata beyond the lifetime of this call *MUST* copy the value themselves.
+
+ A producer *MUST NOT* call this concurrently from multiple threads.
+
+ The :c:member:`ArrowAsyncProducer.request` callback must be called to start receiving calls to this
+ handler.
+
+.. c:member:: void (*ArrowAsyncDeviceStreamHandler.on_error)(struct ArrowAsyncDeviceStreamHandler, int, const char*, const char*)
+
+ *Mandatory.* Handler to be called when an error is encountered by the producer. After calling
+ this, the ``release`` callback will be called as the last call on this struct. The parameters
+ are an ``errno``-compatible error code and an optional error message and metadata.
+
+ If the message and metadata are not ``NULL``, their lifetime is only valid during the scope
+ of this call. A consumer who wants to maintain these values past the return of this function
+ *MUST* copy the values themselves.
+
+ If the metadata parameter is not ``NULL``, to provide key-value error metadata, then it should
+ be encoded identically to the way that metadata is encoded in :c:member:`ArrowSchema.metadata`.
+
+ It is valid for this to be called by a producer with or without a preceding call to
+ :c:member:`ArrowAsyncProducer.request`. This callback *MUST NOT* call any methods of an
+ ``ArrowAsyncProducer`` object.
+
+.. c:member:: void (*ArrowAsyncDeviceStreamHandler.release)(struct ArrowAsyncDeviceStreamHandler*)
+
+ *Mandatory.* A pointer to a consumer-provided release callback for the handler.
+
+ It is valid for this to be called by a producer with or without a preceding call to
+ :c:member:`ArrowAsyncProducer.request`. This must not call any methods of an ``ArrowAsyncProducer``
+ object.
+
+.. c:member:: void* ArrowAsyncDeviceStreamHandler.private_data
+
+ *Optional.* An opaque pointer to consumer-provided private data.
+
+ Producers *MUST NOT* process this member. Lifetime of this member is handled by
+ the consumer, and especially by the release callback.
+
+The ArrowAsyncTask structure
+''''''''''''''''''''''''''''
+
+The purpose of using a Task object rather than passing the array directly to the ``on_next``
+callback is to allow for more complex and efficient thread handling. Utilizing a Task
+object allows for a producer to separate the "decoding" logic from the I/O, enabling a
+consumer to avoid transferring data between CPU cores (e.g. from one L1/L2 cache to another).
+
+This producer-provided structure has the following fields:
+
+.. c:member:: int (*ArrowArrayTask.extract_data)(struct ArrowArrayTask*, struct ArrowDeviceArray*)
+
+ *Mandatory.* A callback to populate the provided ``ArrowDeviceArray`` with the available data.
+ The order of ArrowAsyncTasks provided by the producer enables a consumer to know the order of
+ the data to process. If the consumer does not care about the data that is owned by this task,
+ it must still call ``extract_data`` so that the producer can perform any required cleanup. ``NULL``
+ should be passed as the device array pointer to indicate that the consumer doesn't want the
+ actual data, letting the task perform necessary cleanup.
+
+ If a non-zero value is returned from this, it should be followed only by the producer calling
+ the ``on_error`` callback of the ``ArrowAsyncDeviceStreamHandler``. Because calling this method
+ is likely to be separate from the current control flow, returning a non-zero value to signal
+ an error occuring allows the current thread to decide handle the case accordingly, while still
+ allowing all error logging and handling to be centralized in the
+ :c:member:`ArrowAsyncDeviceStreamHandler.on_error` callback.
+
+ Rather than having a separate release callback, any required cleanup should be performed as part
+ of the invocation of this callback. Ownership of the Array is given to the pointer passed in as
+ a parameter, and this array must be released separately.
+
+ It is only valid to call this method exactly once.
+
+.. c:member:: void* ArrowArrayTask.private_data
+
+ *Optional.* An opaque pointer to producer-provided private data.
+
+ Consumers *MUST NOT* process this member. Lifetime of this member is handled by
+ the producer who created this object, and should be cleaned up if necessary during
+ the call to :c:member:`ArrowArrayTask.extract_data`.
+
+The ArrowAsyncProducer structure
+''''''''''''''''''''''''''''''''
+
+This producer-provided and managed object has the following fields:
+
+.. c:member:: void (*ArrowAsyncProducer.request)(struct ArrowAsyncProducer*, uint64_t)
+
+ *Mandatory.* This function must be called by a consumer to start receiving calls to
+ :c:member:`ArrowAsyncDeviceStreamHandler.on_next_task`. It *MUST* be valid to call
+ this synchronously from within :c:member:`ArrowAsyncDeviceStreamHandler.on_next_task`
+ or :c:member:`ArrowAsyncDeviceStreamHandler.on_schema`. As a result, this function
+ *MUST NOT* synchronously call ``on_next_task`` or ``on_error`` to avoid recursive
+ and reentrant callbacks.
+
+ After ``cancel`` is called, additional calls to this function must be a NOP, but allowed.
+
+ While not cancelled, calling this function registers the given number of additional
+ arrays/batches to be produced by the producer. A producer should only call
+ the appropriate ``on_next_task`` callback up to a maximum of the total sum of calls to
+ this method before propagating back-pressure / waiting.
+
+ Any error encountered by calling request must be propagated by calling the ``on_error``
+ callback of the ``ArrowAsyncDeviceStreamHandler``.
+
+ It is invalid to call this function with a value of ``n`` that is ``<= 0``. Producers should
+ error (e.g. call ``on_error``) if receiving such a value for ``n``.
+
+.. c:member:: void (*ArrowAsyncProducer.cancel)(struct ArrowAsyncProducer*)
+
+ *Mandatory.* This function signals to the producer that it must *eventually* stop calling
+ ``on_next_task``. Calls to ``cancel`` must be idempotent and thread-safe. After calling
+ it once, subsequent calls *MUST* be a NOP. This *MUST NOT* call any consumer-side handlers
+ other than ``on_error``.
+
+ It is not required that calling ``cancel`` affect the producer *immediately*, only that it
+ must eventually stop calling ``on_next_task`` and then subsequently call ``release``
+ on the async handler object. As such, a consumer *MUST* be prepared to receive one or more
+ calls to ``on_next_task`` or ``on_error`` even after calling ``cancel`` if there are still
+ requested arrays pending.
+
+ Successful cancelling *MUST NOT* result in a producer calling
+ :c:member:`ArrowAsyncDeviceStreamHandler.on_error`, instead it should finish out any remaining
+ tasks (calling ``on_next_task`` accordingly) and eventually just call ``release``.
+
+ Any error encountered during handling a call to cancel must be reported via the ``on_error``
+ callback on the async stream handler.
+
+.. c:member:: void* ArrowAsyncProducer.private_data
+
+ *Optional.* An opaque pointer to producer-provided specific data.
+
+ Consumers *MUST NOT* process this member, the lifetime is owned by the producer
+ that constructed this object.
+
+Error Handling
+''''''''''''''
+
+Unlike the regular C Stream interface, the Async interface allows for errors to flow in
+both directions. As a result, error handling can be slightly more complex. Thus this spec
+designates the following rules:
+
+* If the producer encounters an error during processing, it should call the ``on_error``
+callback, and then call ``release`` after it returns.
Review Comment:
```suggestion
callback, and then call ``release`` after it returns.
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-43631: [C][Format] Add ArrowAsyncDeviceStreamHandler interface [arrow]
Posted by "pitrou (via GitHub)" <gi...@apache.org>.
pitrou commented on code in PR #43632:
URL: https://github.com/apache/arrow/pull/43632#discussion_r1757478217
##########
docs/source/format/CDeviceDataInterface.rst:
##########
@@ -650,6 +652,331 @@ The stream source is not assumed to be thread-safe. Consumers wanting to
call ``get_next`` from several threads should ensure those calls are
serialized.
+Async Device Stream Interface
+=============================
+
+The :ref:`C stream interface <_c-device-stream-interface>` provides a synchronous
+API centered around the consumer calling the callback functions to retrieve
+the next record batch. For some bindings, use cases, and interoperability, a more
+asynchronous, producer-focused interface may be required. These scenarios can utilize
+the ``ArrowAsyncDeviceStreamHandler``.
+
+Semantics
+---------
+
+Rather than the producer providing a structure of callbacks for a consumer to
+call and retrieve records, the Async interface is a consumer allocated structure.
+The consumer allocated struct provides handler callbacks for the producer to call
+when the schema and chunks of data are available, rather than the consumer using
+a blocking pull-style iteration.
+
+In addition to the ``ArrowAsyncDeviceStreamHandler``, there are also two additional
+structs used for the full data flow: ``ArrowAsyncTask`` and ``ArrowAsyncProducer``.
+
+Structure Definition
+--------------------
+
+The C device async stream interface is defined with a single ``struct`` definition:
+
+.. code-block:: c
+
+ #ifndef ARROW_C_ASYNC_STREAM_INTERFACE
+ #define ARROW_C_ASYNC_STREAM_INTERFACE
+
+ struct ArrowAsyncTask {
+ int (*get_data)(struct ArrowArrayTask* self, struct ArrowDeviceArray* out);
+
+ void* private_data;
+ };
+
+ struct ArrowAsyncProducer {
+ void (*request)(struct ArrowAsyncProducer* self, uint64_t n);
+ void (*cancel)(struct ArrowAsyncProducer* self);
+
+ void (*release)(struct ArrowAsyncProducer* self);
+ void* private_data;
+ };
+
+ struct ArrowAsyncDeviceStreamHandler {
+ // handlers
+ int (*on_schema)(struct ArrowAsyncDeviceStreamHandler* self,
+ struct ArrowAsyncProducer* producer,
+ struct ArrowSchema* stream_schema, const char* addl_metadata);
+ int (*on_next_task)(struct ArrowAsyncDeviceStreamHandler* self,
+ struct ArrowAsyncTask* task, const char* metadata);
+ void (*on_error)(struct ArrowAsyncDeviceStreamHandler* self,
+ int code, const char* message, const char* metadata);
+
+ // release callback
+ void (*release)(struct ArrowAsyncDeviceStreamHandler* self);
+
+ // opaque handler-specific data
+ void* private_data;
+ };
+
+ #endif // ARROW_C_ASYNC_STREAM_INTERFACE
+
+.. note::
+ The canonical guard ``ARROW_C_ASYNC_STREAM_INTERFACE`` is meant to avoid
+ duplicate definitions if two projects copy the C async stream interface
+ definitions into their own headers, and a third-party project includes
+ from these two projects. It is therefore important that this guard is kept
+ exactly as-is when these definitions are copied.
+
+The ArrowAsyncDeviceStreamHandler structure
+'''''''''''''''''''''''''''''''''''''''''''
+
+The structure has the following fields:
+
+.. c:member:: int (*ArrowAsyncDeviceStreamHandler.on_schema)(struct ArrowAsyncDeviceStreamHandler*, struct ArrowAsyncProducer*, struct ArrowSchema*, const char*)
+
+ *Mandatory.* Handler for receiving the schema of the stream. All records should
+ match the provided schema. If successful, the function should return 0, otherwise
+ it should return an ``errno``-compatible error code.
+
+ The ``const char*`` parameter exists for producers to provide any extra contextual information
+ they want, such as the total number of rows in the stream, statistics, or otherwise. This is
+ encoded in the same format as :c:member:`ArrowSchema.metadata`. If not ``NULL``,
+ the lifetime is only the scope of the call to this function. A consumer who wants to maintain
+ the additional metadata beyond the lifetime of this call *MUST* copy the value themselves.
+
+ Unless the ``on_error`` handler is called, this will always get called exactly once and will be
+ the first method called on this object. As such the producer *MUST* provide an ``ArrowAsyncProducer``
+ object when calling this function to allow the consumer to manage back-pressure and flow control.
+ The producer maintains ownership of the ``ArrowAsyncProducer`` and must clean it up before or after
+ calling the release callback on this object.
+
+ A producer that receives a non-zero result here must not subsequently call anything other than
+ the release callback on this object.
+
+.. c:member:: int (*ArrowAsyncDeviceStreamHandler.on_next_task)(struct ArrowAsyncDeviceStreamHandler*, struct ArrowAsyncTask*, const char*)
+
+ *Mandatory.* Handler to be called when a new record is available for processing. The
+ schema for each record should be the same as the schema that ``on_schema`` was called with.
+ If successfully handled, the function should return 0, otherwise it should return an
+ ``errno``-compatible error code.
+
+ Rather than passing the record itself it receives an ``ArrowAsyncTask`` instead to facilitate
+ better consumer-focused thread control as far as receiving the data. A call to this function
+ simply indicates that data is available via the provided task.
+
+ The ``const char*`` parameter exists for producers to provide any extra contextual information
+ they want. This is encoded in the same format as :c:member:`ArrowSchema.metadata`. If not ``NULL``,
+ the lifetime is only the scope of the call to this function. A consumer who wants to maintain
+ the additional metadata beyond the lifetime of this call *MUST* copy the value themselves.
+
+ A producer *MUST NOT* call this concurrently from multiple different threads.
+
+ The :c:member:`ArrowAsyncProducer.request` callback must be called to start receiving calls to this
+ handler.
+
+ This function *MUST* be able to be called re-entrantly on the same thread to allow for the
+ :c:member:`ArrowAsyncProducer.request` callback to potentially call this method recursively.
+
+.. c:member:: void (*ArrowAsyncDeviceStreamHandler.on_error)(struct ArrowAsyncDeviceStreamHandler, int, const char*, const char*)
+
+ *Mandatory.* Handler to be called when an error is encountered by the producer. After calling
+ this, the ``release`` callback will be called as the last call on this struct. The parameters
+ are an ``errno``-compatible error code and an optional error message and metadata.
+
+ If the message and metadata are not ``NULL``, their lifetime is only valid during the scope
+ of this call. A consumer who wants to maintain these values past the return of this function
+ *MUST* copy the values themselves.
+
+ If the metadata parameter is not ``NULL``, to provide key-value error metadata, then it should
+ be encoded identically to the way that metadata is encoded in :c:member:`ArrowSchema.metadata`.
+
+ It is valid for this to be called by a producer with or without a preceding call to
+ :c:member:`ArrowAsyncProducer.request`. This callback *MUST NOT* call any methods of an
+ ``ArrowAsyncProducer`` object.
+
+.. c:member:: void (*ArrowAsyncDeviceStreamHandler.release)(struct ArrowAsyncDeviceStreamHandler*)
+
+ *Mandatory.* A pointer to a consumer-provided release callback for the handler.
+
+ It is valid for this to be called by a producer with or without a preceding call to
+ :c:member:`ArrowAsyncProducer.request`. This must not call any methods of an ``ArrowAsyncProducer``
+ object.
+
+.. c:member:: void* ArrowAsyncDeviceStreamHandler.private_data
+
+ *Optional.* An opaque pointer to consumer-provided private data.
+
+ Producers *MUST NOT* process this member. Lifetime of this member is handled by
+ the consumer, and especially by the release callback.
+
+The ArrowAsyncTask structure
+''''''''''''''''''''''''''''
+
+The purpose of using a Task object rather than passing the array directly to the ``on_next``
+callback is to allow for more complex and efficient thread handling. Utilizing a Task
+object allows for a producer to separate the "decoding" logic from the I/O, enabling a
+consumer to avoid transferring data between CPU cores (e.g. from one L1/L2 cache to another).
+
+This producer-provided structure has the following fields:
+
+.. c:member:: int (*ArrowArrayTask.get_data)(struct ArrowArrayTask*, struct ArrowDeviceArray*)
+
+ *Mandatory.* A callback to populate the provided ``ArrowDeviceArray`` with the available data.
+ The order of ArrowAsyncTasks provided by the producer enables a consumer to know the order of
+ the data to process.
+
+ If a non-zero value is returned from this, it should be followed only by the producer calling
+ the ``on_error`` callback of the ``ArrowAsyncDeviceStreamHandler``. Because calling this method
+ is likely to be separate from the current control flow, returning a non-zero value to signal
+ an error occuring allows the current thread to decide handle the case accordingly, while still
+ allowing all error logging and handling to be centralized in the
+ :c:member:`ArrowAsyncDeviceStreamHandler.on_error` callback.
+
+ Rather than having a separate release callback, any required cleanup should be performed as part
+ of the invocation of this callback, with ownership of the Array being passed to the consumer-provided
+ parameter calling this, and must be released separately.
+
+ It is only valid to call this method exactly once.
+
+.. c:member:: void* ArrowArrayTask.private_data
+
+ *Optional.* An opaque pointer to consumer-provided private data.
Review Comment:
Why would a consumer put their own private data in a provider-supplied struct? This seems weird.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-43631: [C][Format] Add ArrowAsyncDeviceStreamHandler interface [arrow]
Posted by "pitrou (via GitHub)" <gi...@apache.org>.
pitrou commented on code in PR #43632:
URL: https://github.com/apache/arrow/pull/43632#discussion_r1757474514
##########
docs/source/format/CDeviceDataInterface.rst:
##########
@@ -650,6 +652,331 @@ The stream source is not assumed to be thread-safe. Consumers wanting to
call ``get_next`` from several threads should ensure those calls are
serialized.
+Async Device Stream Interface
+=============================
+
+The :ref:`C stream interface <_c-device-stream-interface>` provides a synchronous
+API centered around the consumer calling the callback functions to retrieve
+the next record batch. For some bindings, use cases, and interoperability, a more
+asynchronous, producer-focused interface may be required. These scenarios can utilize
+the ``ArrowAsyncDeviceStreamHandler``.
+
+Semantics
+---------
+
+Rather than the producer providing a structure of callbacks for a consumer to
+call and retrieve records, the Async interface is a consumer allocated structure.
+The consumer allocated struct provides handler callbacks for the producer to call
+when the schema and chunks of data are available, rather than the consumer using
+a blocking pull-style iteration.
+
+In addition to the ``ArrowAsyncDeviceStreamHandler``, there are also two additional
+structs used for the full data flow: ``ArrowAsyncTask`` and ``ArrowAsyncProducer``.
+
+Structure Definition
+--------------------
+
+The C device async stream interface is defined with a single ``struct`` definition:
+
+.. code-block:: c
+
+ #ifndef ARROW_C_ASYNC_STREAM_INTERFACE
+ #define ARROW_C_ASYNC_STREAM_INTERFACE
+
+ struct ArrowAsyncTask {
+ int (*get_data)(struct ArrowArrayTask* self, struct ArrowDeviceArray* out);
+
+ void* private_data;
+ };
+
+ struct ArrowAsyncProducer {
+ void (*request)(struct ArrowAsyncProducer* self, uint64_t n);
+ void (*cancel)(struct ArrowAsyncProducer* self);
+
+ void (*release)(struct ArrowAsyncProducer* self);
+ void* private_data;
+ };
+
+ struct ArrowAsyncDeviceStreamHandler {
+ // handlers
+ int (*on_schema)(struct ArrowAsyncDeviceStreamHandler* self,
+ struct ArrowAsyncProducer* producer,
+ struct ArrowSchema* stream_schema, const char* addl_metadata);
+ int (*on_next_task)(struct ArrowAsyncDeviceStreamHandler* self,
+ struct ArrowAsyncTask* task, const char* metadata);
+ void (*on_error)(struct ArrowAsyncDeviceStreamHandler* self,
+ int code, const char* message, const char* metadata);
+
+ // release callback
+ void (*release)(struct ArrowAsyncDeviceStreamHandler* self);
+
+ // opaque handler-specific data
+ void* private_data;
+ };
+
+ #endif // ARROW_C_ASYNC_STREAM_INTERFACE
+
+.. note::
+ The canonical guard ``ARROW_C_ASYNC_STREAM_INTERFACE`` is meant to avoid
+ duplicate definitions if two projects copy the C async stream interface
+ definitions into their own headers, and a third-party project includes
+ from these two projects. It is therefore important that this guard is kept
+ exactly as-is when these definitions are copied.
+
+The ArrowAsyncDeviceStreamHandler structure
+'''''''''''''''''''''''''''''''''''''''''''
+
+The structure has the following fields:
+
+.. c:member:: int (*ArrowAsyncDeviceStreamHandler.on_schema)(struct ArrowAsyncDeviceStreamHandler*, struct ArrowAsyncProducer*, struct ArrowSchema*, const char*)
+
+ *Mandatory.* Handler for receiving the schema of the stream. All records should
+ match the provided schema. If successful, the function should return 0, otherwise
+ it should return an ``errno``-compatible error code.
+
+ The ``const char*`` parameter exists for producers to provide any extra contextual information
+ they want, such as the total number of rows in the stream, statistics, or otherwise. This is
+ encoded in the same format as :c:member:`ArrowSchema.metadata`. If not ``NULL``,
+ the lifetime is only the scope of the call to this function. A consumer who wants to maintain
+ the additional metadata beyond the lifetime of this call *MUST* copy the value themselves.
+
+ Unless the ``on_error`` handler is called, this will always get called exactly once and will be
+ the first method called on this object. As such the producer *MUST* provide an ``ArrowAsyncProducer``
+ object when calling this function to allow the consumer to manage back-pressure and flow control.
+ The producer maintains ownership of the ``ArrowAsyncProducer`` and must clean it up before or after
+ calling the release callback on this object.
+
+ A producer that receives a non-zero result here must not subsequently call anything other than
+ the release callback on this object.
+
+.. c:member:: int (*ArrowAsyncDeviceStreamHandler.on_next_task)(struct ArrowAsyncDeviceStreamHandler*, struct ArrowAsyncTask*, const char*)
+
+ *Mandatory.* Handler to be called when a new record is available for processing. The
+ schema for each record should be the same as the schema that ``on_schema`` was called with.
+ If successfully handled, the function should return 0, otherwise it should return an
+ ``errno``-compatible error code.
+
+ Rather than passing the record itself it receives an ``ArrowAsyncTask`` instead to facilitate
+ better consumer-focused thread control as far as receiving the data. A call to this function
Review Comment:
Ok, I've read the explanation below.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-43631: [C][Format] Add ArrowAsyncDeviceStreamHandler interface [arrow]
Posted by "zeroshade (via GitHub)" <gi...@apache.org>.
zeroshade commented on code in PR #43632:
URL: https://github.com/apache/arrow/pull/43632#discussion_r1765332073
##########
docs/source/format/CDeviceDataInterface.rst:
##########
@@ -650,6 +652,331 @@ The stream source is not assumed to be thread-safe. Consumers wanting to
call ``get_next`` from several threads should ensure those calls are
serialized.
+Async Device Stream Interface
+=============================
+
+The :ref:`C stream interface <_c-device-stream-interface>` provides a synchronous
+API centered around the consumer calling the callback functions to retrieve
+the next record batch. For some bindings, use cases, and interoperability, a more
+asynchronous, producer-focused interface may be required. These scenarios can utilize
+the ``ArrowAsyncDeviceStreamHandler``.
+
+Semantics
+---------
+
+Rather than the producer providing a structure of callbacks for a consumer to
+call and retrieve records, the Async interface is a consumer allocated structure.
+The consumer allocated struct provides handler callbacks for the producer to call
+when the schema and chunks of data are available, rather than the consumer using
+a blocking pull-style iteration.
+
+In addition to the ``ArrowAsyncDeviceStreamHandler``, there are also two additional
+structs used for the full data flow: ``ArrowAsyncTask`` and ``ArrowAsyncProducer``.
+
+Structure Definition
+--------------------
+
+The C device async stream interface is defined with a single ``struct`` definition:
+
+.. code-block:: c
+
+ #ifndef ARROW_C_ASYNC_STREAM_INTERFACE
+ #define ARROW_C_ASYNC_STREAM_INTERFACE
+
+ struct ArrowAsyncTask {
+ int (*get_data)(struct ArrowArrayTask* self, struct ArrowDeviceArray* out);
+
+ void* private_data;
+ };
+
+ struct ArrowAsyncProducer {
+ void (*request)(struct ArrowAsyncProducer* self, uint64_t n);
+ void (*cancel)(struct ArrowAsyncProducer* self);
+
+ void (*release)(struct ArrowAsyncProducer* self);
+ void* private_data;
+ };
+
+ struct ArrowAsyncDeviceStreamHandler {
+ // handlers
+ int (*on_schema)(struct ArrowAsyncDeviceStreamHandler* self,
+ struct ArrowAsyncProducer* producer,
+ struct ArrowSchema* stream_schema, const char* addl_metadata);
+ int (*on_next_task)(struct ArrowAsyncDeviceStreamHandler* self,
+ struct ArrowAsyncTask* task, const char* metadata);
+ void (*on_error)(struct ArrowAsyncDeviceStreamHandler* self,
+ int code, const char* message, const char* metadata);
+
+ // release callback
+ void (*release)(struct ArrowAsyncDeviceStreamHandler* self);
+
+ // opaque handler-specific data
+ void* private_data;
+ };
+
+ #endif // ARROW_C_ASYNC_STREAM_INTERFACE
+
+.. note::
+ The canonical guard ``ARROW_C_ASYNC_STREAM_INTERFACE`` is meant to avoid
+ duplicate definitions if two projects copy the C async stream interface
+ definitions into their own headers, and a third-party project includes
+ from these two projects. It is therefore important that this guard is kept
+ exactly as-is when these definitions are copied.
+
+The ArrowAsyncDeviceStreamHandler structure
+'''''''''''''''''''''''''''''''''''''''''''
+
+The structure has the following fields:
+
+.. c:member:: int (*ArrowAsyncDeviceStreamHandler.on_schema)(struct ArrowAsyncDeviceStreamHandler*, struct ArrowAsyncProducer*, struct ArrowSchema*, const char*)
+
+ *Mandatory.* Handler for receiving the schema of the stream. All records should
+ match the provided schema. If successful, the function should return 0, otherwise
+ it should return an ``errno``-compatible error code.
+
+ The ``const char*`` parameter exists for producers to provide any extra contextual information
+ they want, such as the total number of rows in the stream, statistics, or otherwise. This is
+ encoded in the same format as :c:member:`ArrowSchema.metadata`. If not ``NULL``,
+ the lifetime is only the scope of the call to this function. A consumer who wants to maintain
+ the additional metadata beyond the lifetime of this call *MUST* copy the value themselves.
+
+ Unless the ``on_error`` handler is called, this will always get called exactly once and will be
+ the first method called on this object. As such the producer *MUST* provide an ``ArrowAsyncProducer``
+ object when calling this function to allow the consumer to manage back-pressure and flow control.
+ The producer maintains ownership of the ``ArrowAsyncProducer`` and must clean it up before or after
+ calling the release callback on this object.
+
+ A producer that receives a non-zero result here must not subsequently call anything other than
+ the release callback on this object.
+
+.. c:member:: int (*ArrowAsyncDeviceStreamHandler.on_next_task)(struct ArrowAsyncDeviceStreamHandler*, struct ArrowAsyncTask*, const char*)
+
+ *Mandatory.* Handler to be called when a new record is available for processing. The
+ schema for each record should be the same as the schema that ``on_schema`` was called with.
+ If successfully handled, the function should return 0, otherwise it should return an
+ ``errno``-compatible error code.
+
+ Rather than passing the record itself it receives an ``ArrowAsyncTask`` instead to facilitate
+ better consumer-focused thread control as far as receiving the data. A call to this function
+ simply indicates that data is available via the provided task.
+
+ The ``const char*`` parameter exists for producers to provide any extra contextual information
+ they want. This is encoded in the same format as :c:member:`ArrowSchema.metadata`. If not ``NULL``,
+ the lifetime is only the scope of the call to this function. A consumer who wants to maintain
+ the additional metadata beyond the lifetime of this call *MUST* copy the value themselves.
+
+ A producer *MUST NOT* call this concurrently from multiple different threads.
+
+ The :c:member:`ArrowAsyncProducer.request` callback must be called to start receiving calls to this
+ handler.
+
+ This function *MUST* be able to be called re-entrantly on the same thread to allow for the
+ :c:member:`ArrowAsyncProducer.request` callback to potentially call this method recursively.
+
+.. c:member:: void (*ArrowAsyncDeviceStreamHandler.on_error)(struct ArrowAsyncDeviceStreamHandler, int, const char*, const char*)
+
+ *Mandatory.* Handler to be called when an error is encountered by the producer. After calling
+ this, the ``release`` callback will be called as the last call on this struct. The parameters
+ are an ``errno``-compatible error code and an optional error message and metadata.
+
+ If the message and metadata are not ``NULL``, their lifetime is only valid during the scope
+ of this call. A consumer who wants to maintain these values past the return of this function
+ *MUST* copy the values themselves.
+
+ If the metadata parameter is not ``NULL``, to provide key-value error metadata, then it should
+ be encoded identically to the way that metadata is encoded in :c:member:`ArrowSchema.metadata`.
+
+ It is valid for this to be called by a producer with or without a preceding call to
+ :c:member:`ArrowAsyncProducer.request`. This callback *MUST NOT* call any methods of an
+ ``ArrowAsyncProducer`` object.
+
+.. c:member:: void (*ArrowAsyncDeviceStreamHandler.release)(struct ArrowAsyncDeviceStreamHandler*)
+
+ *Mandatory.* A pointer to a consumer-provided release callback for the handler.
+
+ It is valid for this to be called by a producer with or without a preceding call to
+ :c:member:`ArrowAsyncProducer.request`. This must not call any methods of an ``ArrowAsyncProducer``
+ object.
+
+.. c:member:: void* ArrowAsyncDeviceStreamHandler.private_data
+
+ *Optional.* An opaque pointer to consumer-provided private data.
+
+ Producers *MUST NOT* process this member. Lifetime of this member is handled by
+ the consumer, and especially by the release callback.
+
+The ArrowAsyncTask structure
+''''''''''''''''''''''''''''
+
+The purpose of using a Task object rather than passing the array directly to the ``on_next``
+callback is to allow for more complex and efficient thread handling. Utilizing a Task
+object allows for a producer to separate the "decoding" logic from the I/O, enabling a
+consumer to avoid transferring data between CPU cores (e.g. from one L1/L2 cache to another).
+
+This producer-provided structure has the following fields:
+
+.. c:member:: int (*ArrowArrayTask.get_data)(struct ArrowArrayTask*, struct ArrowDeviceArray*)
+
+ *Mandatory.* A callback to populate the provided ``ArrowDeviceArray`` with the available data.
+ The order of ArrowAsyncTasks provided by the producer enables a consumer to know the order of
+ the data to process.
+
+ If a non-zero value is returned from this, it should be followed only by the producer calling
+ the ``on_error`` callback of the ``ArrowAsyncDeviceStreamHandler``. Because calling this method
+ is likely to be separate from the current control flow, returning a non-zero value to signal
+ an error occuring allows the current thread to decide handle the case accordingly, while still
+ allowing all error logging and handling to be centralized in the
+ :c:member:`ArrowAsyncDeviceStreamHandler.on_error` callback.
+
+ Rather than having a separate release callback, any required cleanup should be performed as part
+ of the invocation of this callback, with ownership of the Array being passed to the consumer-provided
+ parameter calling this, and must be released separately.
+
+ It is only valid to call this method exactly once.
+
+.. c:member:: void* ArrowArrayTask.private_data
+
+ *Optional.* An opaque pointer to consumer-provided private data.
+
+ Consumers *MUST NOT* process this member. Lifetime of this member is handled by
+ the producer who created this object, and should be cleaned up if necessary during
+ the call to :c:member:`ArrowArrayTask.get_data`.
+
+The ArrowAsyncProducer structure
+''''''''''''''''''''''''''''''''
+
+This producer-provided and managed object has the following fields:
+
+.. c:member:: void (*ArrowAsyncProducer.request)(struct ArrowAsyncProducer*, uint64_t)
Review Comment:
The most convincing argument to me here is that it breaks the conventions, so i'll switch it to `int64_t` and update the docs that it should error if n < 0
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-43631: [C][Format] Add ArrowAsyncDeviceStreamHandler interface [arrow]
Posted by "westonpace (via GitHub)" <gi...@apache.org>.
westonpace commented on code in PR #43632:
URL: https://github.com/apache/arrow/pull/43632#discussion_r1766708550
##########
cpp/src/arrow/c/abi.h:
##########
@@ -228,6 +228,207 @@ struct ArrowDeviceArrayStream {
#endif // ARROW_C_DEVICE_STREAM_INTERFACE
+#ifndef ARROW_C_ASYNC_STREAM_INTERFACE
+#define ARROW_C_ASYNC_STREAM_INTERFACE
+
+// ArrowAsyncTask represents available data from a producer that was passed to
+// an invocation of `on_next_task` on the ArrowAsyncDeviceStreamHandler.
+//
+// The reason for this Task approach instead of the Async interface returning
+// the Array directly is to allow for more complex thread handling and reducing
+// context switching and data transfers between CPU cores (e.g. from one L1/L2
+// cache to another) if desired.
+//
+// For example, the `on_next_task` callback can be called when data is ready, while
+// the producer puts potential "decoding" logic in the `ArrowAsyncTask` object. This
+// allows for the producer to manage the I/O on one thread which calls `on_next_task`
+// and the consumer can determine when the decoding (producer logic in the `get_data`
+// callback of the task) occurs and on which thread, to avoid a CPU core transfer
+// (data staying in the L2 cache).
+struct ArrowAsyncTask {
+ // This callback should populate the ArrowDeviceArray associated with this task.
+ // The order of ArrowAsyncTasks provided by the producer enables a consumer to
+ // ensure the order of data to process.
+ //
+ // This function is expected to be synchronous, but should not perform any blocking
+ // I/O. Ideally it should be as cheap as possible so as to not tie up the consumer
+ // thread unnecessarily.
+ //
+ // Returns: 0 if successful, errno-compatible error otherwise.
+ //
+ // If a non-0 value is returned then it should be followed by a call to `on_error`
+ // on the appropriate ArrowAsyncDeviceStreamHandler. This is because the it's highly
+ // likely that whatever is calling this function may be entirely disconnected from
+ // the current control flow. Indicating an error here with a non-zero return allows
+ // the current flow to be aware of the error occurring, while still allowing any
+ // logging or error handling to still be centralized in the `on_error` callback of
+ // the original Async handler.
+ //
+ // Rather than a release callback, any required cleanup should be performed as part
+ // of the invocation of `get_data`. Ownership of the Array is passed to the consumer
+ // calling this, and so it must be released separately.
+ //
+ // It is only valid to call this method exactly once.
+ int (*get_data)(struct ArrowArrayTask* self, struct ArrowDeviceArray* out);
+
+ // opaque task-specific data
+ void* private_data;
+};
+
+// ArrowAsyncProducer represents a 1-to-1 relationship between an async producer
+// and consumer. This object allows the consumer to perform backpressure and flow
+// control on the asynchronous stream processing. This object must be owned by the
+// producer who creates it, and thus is responsible for cleaning it up.
+struct ArrowAsyncProducer {
+ // A consumer must call this function to start receiving on_next_task calls.
+ //
+ // It *must* be valid to call this synchronously from within `on_next_task` or
+ // `on_schema`, but this function *must not* immediately call `on_next_task` so as
+ // to avoid recursion and reentrant callbacks.
+ //
+ // After cancel has been called, additional calls to this function must be NOPs,
+ // but allowed. While not cancelled, calling this function must register the
+ // given number of additional arrays/batches to be produced with the producer.
+ // The producer should only call `on_next_task` at most the registered number
+ // of arrays before propagating backpressure.
Review Comment:
So is the idea that I will call this method many times? In other words, if I want to allow up to 32 concurrent tasks I would call `request(32)` and then, after I complete each task I call `request(1)`?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-43631: [C][Format] Add ArrowAsyncDeviceStreamHandler interface [arrow]
Posted by "lidavidm (via GitHub)" <gi...@apache.org>.
lidavidm commented on code in PR #43632:
URL: https://github.com/apache/arrow/pull/43632#discussion_r1764961329
##########
docs/source/format/CDeviceDataInterface.rst:
##########
@@ -650,6 +652,331 @@ The stream source is not assumed to be thread-safe. Consumers wanting to
call ``get_next`` from several threads should ensure those calls are
serialized.
+Async Device Stream Interface
+=============================
+
+The :ref:`C stream interface <_c-device-stream-interface>` provides a synchronous
+API centered around the consumer calling the callback functions to retrieve
+the next record batch. For some bindings, use cases, and interoperability, a more
+asynchronous, producer-focused interface may be required. These scenarios can utilize
+the ``ArrowAsyncDeviceStreamHandler``.
+
+Semantics
+---------
+
+Rather than the producer providing a structure of callbacks for a consumer to
+call and retrieve records, the Async interface is a consumer allocated structure.
+The consumer allocated struct provides handler callbacks for the producer to call
+when the schema and chunks of data are available, rather than the consumer using
+a blocking pull-style iteration.
+
+In addition to the ``ArrowAsyncDeviceStreamHandler``, there are also two additional
+structs used for the full data flow: ``ArrowAsyncTask`` and ``ArrowAsyncProducer``.
+
+Structure Definition
+--------------------
+
+The C device async stream interface is defined with a single ``struct`` definition:
+
+.. code-block:: c
+
+ #ifndef ARROW_C_ASYNC_STREAM_INTERFACE
+ #define ARROW_C_ASYNC_STREAM_INTERFACE
+
+ struct ArrowAsyncTask {
+ int (*get_data)(struct ArrowArrayTask* self, struct ArrowDeviceArray* out);
+
+ void* private_data;
+ };
+
+ struct ArrowAsyncProducer {
+ void (*request)(struct ArrowAsyncProducer* self, uint64_t n);
+ void (*cancel)(struct ArrowAsyncProducer* self);
+
+ void (*release)(struct ArrowAsyncProducer* self);
+ void* private_data;
+ };
+
+ struct ArrowAsyncDeviceStreamHandler {
+ // handlers
+ int (*on_schema)(struct ArrowAsyncDeviceStreamHandler* self,
+ struct ArrowAsyncProducer* producer,
+ struct ArrowSchema* stream_schema, const char* addl_metadata);
+ int (*on_next_task)(struct ArrowAsyncDeviceStreamHandler* self,
+ struct ArrowAsyncTask* task, const char* metadata);
+ void (*on_error)(struct ArrowAsyncDeviceStreamHandler* self,
+ int code, const char* message, const char* metadata);
+
+ // release callback
+ void (*release)(struct ArrowAsyncDeviceStreamHandler* self);
+
+ // opaque handler-specific data
+ void* private_data;
+ };
+
+ #endif // ARROW_C_ASYNC_STREAM_INTERFACE
+
+.. note::
+ The canonical guard ``ARROW_C_ASYNC_STREAM_INTERFACE`` is meant to avoid
+ duplicate definitions if two projects copy the C async stream interface
+ definitions into their own headers, and a third-party project includes
+ from these two projects. It is therefore important that this guard is kept
+ exactly as-is when these definitions are copied.
+
+The ArrowAsyncDeviceStreamHandler structure
+'''''''''''''''''''''''''''''''''''''''''''
+
+The structure has the following fields:
+
+.. c:member:: int (*ArrowAsyncDeviceStreamHandler.on_schema)(struct ArrowAsyncDeviceStreamHandler*, struct ArrowAsyncProducer*, struct ArrowSchema*, const char*)
+
+ *Mandatory.* Handler for receiving the schema of the stream. All records should
+ match the provided schema. If successful, the function should return 0, otherwise
+ it should return an ``errno``-compatible error code.
+
+ The ``const char*`` parameter exists for producers to provide any extra contextual information
+ they want, such as the total number of rows in the stream, statistics, or otherwise. This is
+ encoded in the same format as :c:member:`ArrowSchema.metadata`. If not ``NULL``,
+ the lifetime is only the scope of the call to this function. A consumer who wants to maintain
+ the additional metadata beyond the lifetime of this call *MUST* copy the value themselves.
+
+ Unless the ``on_error`` handler is called, this will always get called exactly once and will be
+ the first method called on this object. As such the producer *MUST* provide an ``ArrowAsyncProducer``
+ object when calling this function to allow the consumer to manage back-pressure and flow control.
+ The producer maintains ownership of the ``ArrowAsyncProducer`` and must clean it up before or after
+ calling the release callback on this object.
+
+ A producer that receives a non-zero result here must not subsequently call anything other than
+ the release callback on this object.
+
+.. c:member:: int (*ArrowAsyncDeviceStreamHandler.on_next_task)(struct ArrowAsyncDeviceStreamHandler*, struct ArrowAsyncTask*, const char*)
+
+ *Mandatory.* Handler to be called when a new record is available for processing. The
+ schema for each record should be the same as the schema that ``on_schema`` was called with.
+ If successfully handled, the function should return 0, otherwise it should return an
+ ``errno``-compatible error code.
+
+ Rather than passing the record itself it receives an ``ArrowAsyncTask`` instead to facilitate
+ better consumer-focused thread control as far as receiving the data. A call to this function
+ simply indicates that data is available via the provided task.
+
+ The ``const char*`` parameter exists for producers to provide any extra contextual information
+ they want. This is encoded in the same format as :c:member:`ArrowSchema.metadata`. If not ``NULL``,
+ the lifetime is only the scope of the call to this function. A consumer who wants to maintain
+ the additional metadata beyond the lifetime of this call *MUST* copy the value themselves.
+
+ A producer *MUST NOT* call this concurrently from multiple different threads.
+
+ The :c:member:`ArrowAsyncProducer.request` callback must be called to start receiving calls to this
+ handler.
+
+ This function *MUST* be able to be called re-entrantly on the same thread to allow for the
+ :c:member:`ArrowAsyncProducer.request` callback to potentially call this method recursively.
+
+.. c:member:: void (*ArrowAsyncDeviceStreamHandler.on_error)(struct ArrowAsyncDeviceStreamHandler, int, const char*, const char*)
+
+ *Mandatory.* Handler to be called when an error is encountered by the producer. After calling
+ this, the ``release`` callback will be called as the last call on this struct. The parameters
+ are an ``errno``-compatible error code and an optional error message and metadata.
+
+ If the message and metadata are not ``NULL``, their lifetime is only valid during the scope
+ of this call. A consumer who wants to maintain these values past the return of this function
+ *MUST* copy the values themselves.
+
+ If the metadata parameter is not ``NULL``, to provide key-value error metadata, then it should
+ be encoded identically to the way that metadata is encoded in :c:member:`ArrowSchema.metadata`.
+
+ It is valid for this to be called by a producer with or without a preceding call to
+ :c:member:`ArrowAsyncProducer.request`. This callback *MUST NOT* call any methods of an
+ ``ArrowAsyncProducer`` object.
+
+.. c:member:: void (*ArrowAsyncDeviceStreamHandler.release)(struct ArrowAsyncDeviceStreamHandler*)
+
+ *Mandatory.* A pointer to a consumer-provided release callback for the handler.
+
+ It is valid for this to be called by a producer with or without a preceding call to
+ :c:member:`ArrowAsyncProducer.request`. This must not call any methods of an ``ArrowAsyncProducer``
+ object.
+
+.. c:member:: void* ArrowAsyncDeviceStreamHandler.private_data
+
+ *Optional.* An opaque pointer to consumer-provided private data.
+
+ Producers *MUST NOT* process this member. Lifetime of this member is handled by
+ the consumer, and especially by the release callback.
+
+The ArrowAsyncTask structure
+''''''''''''''''''''''''''''
+
+The purpose of using a Task object rather than passing the array directly to the ``on_next``
+callback is to allow for more complex and efficient thread handling. Utilizing a Task
+object allows for a producer to separate the "decoding" logic from the I/O, enabling a
+consumer to avoid transferring data between CPU cores (e.g. from one L1/L2 cache to another).
+
+This producer-provided structure has the following fields:
+
+.. c:member:: int (*ArrowArrayTask.get_data)(struct ArrowArrayTask*, struct ArrowDeviceArray*)
+
+ *Mandatory.* A callback to populate the provided ``ArrowDeviceArray`` with the available data.
+ The order of ArrowAsyncTasks provided by the producer enables a consumer to know the order of
+ the data to process.
+
+ If a non-zero value is returned from this, it should be followed only by the producer calling
+ the ``on_error`` callback of the ``ArrowAsyncDeviceStreamHandler``. Because calling this method
+ is likely to be separate from the current control flow, returning a non-zero value to signal
+ an error occuring allows the current thread to decide handle the case accordingly, while still
+ allowing all error logging and handling to be centralized in the
+ :c:member:`ArrowAsyncDeviceStreamHandler.on_error` callback.
+
+ Rather than having a separate release callback, any required cleanup should be performed as part
+ of the invocation of this callback, with ownership of the Array being passed to the consumer-provided
+ parameter calling this, and must be released separately.
+
+ It is only valid to call this method exactly once.
+
+.. c:member:: void* ArrowArrayTask.private_data
+
+ *Optional.* An opaque pointer to consumer-provided private data.
+
+ Consumers *MUST NOT* process this member. Lifetime of this member is handled by
+ the producer who created this object, and should be cleaned up if necessary during
+ the call to :c:member:`ArrowArrayTask.get_data`.
+
+The ArrowAsyncProducer structure
+''''''''''''''''''''''''''''''''
+
+This producer-provided and managed object has the following fields:
+
+.. c:member:: void (*ArrowAsyncProducer.request)(struct ArrowAsyncProducer*, uint64_t)
Review Comment:
That's mostly false, as has been stated before; modern Java may not have an explicit unsigned type but has methods to manipulate integers as if they were unsigned.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-43631: [C][Format] Add ArrowAsyncDeviceStreamHandler interface [arrow]
Posted by "zeroshade (via GitHub)" <gi...@apache.org>.
zeroshade commented on code in PR #43632:
URL: https://github.com/apache/arrow/pull/43632#discussion_r1767463818
##########
docs/source/format/CDeviceDataInterface.rst:
##########
@@ -650,6 +652,340 @@ The stream source is not assumed to be thread-safe. Consumers wanting to
call ``get_next`` from several threads should ensure those calls are
serialized.
+Async Device Stream Interface
+=============================
+
+The :ref:`C stream interface <_c-device-stream-interface>` provides a synchronous
+API centered around the consumer calling the callback functions to retrieve
+the next record batch. For some bindings, use cases, and interoperability, a more
+asynchronous, producer-focused interface may be required. These scenarios can utilize
+the ``ArrowAsyncDeviceStreamHandler``.
+
+Semantics
+---------
+
+Rather than the producer providing a structure of callbacks for a consumer to
+call and retrieve records, the Async interface is a consumer allocated structure.
+The consumer allocated struct provides handler callbacks for the producer to call
+when the schema and chunks of data are available, rather than the consumer using
+a blocking pull-style iteration.
+
+In addition to the ``ArrowAsyncDeviceStreamHandler``, there are also two additional
+structs used for the full data flow: ``ArrowAsyncTask`` and ``ArrowAsyncProducer``.
+
+Structure Definition
+--------------------
+
+The C device async stream interface is defined with a single ``struct`` definition:
+
+.. code-block:: c
+
+ #ifndef ARROW_C_ASYNC_STREAM_INTERFACE
+ #define ARROW_C_ASYNC_STREAM_INTERFACE
+
+ struct ArrowAsyncTask {
+ int (*get_data)(struct ArrowArrayTask* self, struct ArrowDeviceArray* out);
+
+ void* private_data;
+ };
+
+ struct ArrowAsyncProducer {
+ void (*request)(struct ArrowAsyncProducer* self, int64_t n);
+ void (*cancel)(struct ArrowAsyncProducer* self);
+
+ void (*release)(struct ArrowAsyncProducer* self);
+ void* private_data;
+ };
+
+ struct ArrowAsyncDeviceStreamHandler {
+ // handlers
+ int (*on_schema)(struct ArrowAsyncDeviceStreamHandler* self,
+ struct ArrowAsyncProducer* producer,
+ struct ArrowSchema* stream_schema, const char* addl_metadata);
+ int (*on_next_task)(struct ArrowAsyncDeviceStreamHandler* self,
+ struct ArrowAsyncTask* task, const char* metadata);
+ void (*on_error)(struct ArrowAsyncDeviceStreamHandler* self,
+ int code, const char* message, const char* metadata);
+
+ // release callback
+ void (*release)(struct ArrowAsyncDeviceStreamHandler* self);
+
+ // opaque handler-specific data
+ void* private_data;
+ };
+
+ #endif // ARROW_C_ASYNC_STREAM_INTERFACE
+
+.. note::
+ The canonical guard ``ARROW_C_ASYNC_STREAM_INTERFACE`` is meant to avoid
+ duplicate definitions if two projects copy the C async stream interface
+ definitions into their own headers, and a third-party project includes
+ from these two projects. It is therefore important that this guard is kept
+ exactly as-is when these definitions are copied.
+
+The ArrowAsyncDeviceStreamHandler structure
+'''''''''''''''''''''''''''''''''''''''''''
+
+The structure has the following fields:
+
+.. c:member:: int (*ArrowAsyncDeviceStreamHandler.on_schema)(struct ArrowAsyncDeviceStreamHandler*, struct ArrowAsyncProducer*, struct ArrowSchema*, const char*)
+
+ *Mandatory.* Handler for receiving the schema of the stream. All records should
+ match the provided schema. If successful, the function should return 0, otherwise
+ it should return an ``errno``-compatible error code.
Review Comment:
We don't do that above in any of the previous mentions of errno, so I don't feel like we need to do so. At worst, we should make the first mention of `errno` do this elsewhere in the spec as opposed to as part of this change.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-43631: [C][Format] Add ArrowAsyncDeviceStreamHandler interface [arrow]
Posted by "pitrou (via GitHub)" <gi...@apache.org>.
pitrou commented on code in PR #43632:
URL: https://github.com/apache/arrow/pull/43632#discussion_r1764757633
##########
docs/source/format/CDeviceDataInterface.rst:
##########
@@ -650,6 +652,331 @@ The stream source is not assumed to be thread-safe. Consumers wanting to
call ``get_next`` from several threads should ensure those calls are
serialized.
+Async Device Stream Interface
+=============================
+
+The :ref:`C stream interface <_c-device-stream-interface>` provides a synchronous
+API centered around the consumer calling the callback functions to retrieve
+the next record batch. For some bindings, use cases, and interoperability, a more
+asynchronous, producer-focused interface may be required. These scenarios can utilize
+the ``ArrowAsyncDeviceStreamHandler``.
+
+Semantics
+---------
+
+Rather than the producer providing a structure of callbacks for a consumer to
+call and retrieve records, the Async interface is a consumer allocated structure.
+The consumer allocated struct provides handler callbacks for the producer to call
+when the schema and chunks of data are available, rather than the consumer using
+a blocking pull-style iteration.
+
+In addition to the ``ArrowAsyncDeviceStreamHandler``, there are also two additional
+structs used for the full data flow: ``ArrowAsyncTask`` and ``ArrowAsyncProducer``.
+
+Structure Definition
+--------------------
+
+The C device async stream interface is defined with a single ``struct`` definition:
+
+.. code-block:: c
+
+ #ifndef ARROW_C_ASYNC_STREAM_INTERFACE
+ #define ARROW_C_ASYNC_STREAM_INTERFACE
+
+ struct ArrowAsyncTask {
+ int (*get_data)(struct ArrowArrayTask* self, struct ArrowDeviceArray* out);
+
+ void* private_data;
+ };
+
+ struct ArrowAsyncProducer {
+ void (*request)(struct ArrowAsyncProducer* self, uint64_t n);
+ void (*cancel)(struct ArrowAsyncProducer* self);
+
+ void (*release)(struct ArrowAsyncProducer* self);
+ void* private_data;
+ };
+
+ struct ArrowAsyncDeviceStreamHandler {
+ // handlers
+ int (*on_schema)(struct ArrowAsyncDeviceStreamHandler* self,
+ struct ArrowAsyncProducer* producer,
+ struct ArrowSchema* stream_schema, const char* addl_metadata);
+ int (*on_next_task)(struct ArrowAsyncDeviceStreamHandler* self,
+ struct ArrowAsyncTask* task, const char* metadata);
+ void (*on_error)(struct ArrowAsyncDeviceStreamHandler* self,
+ int code, const char* message, const char* metadata);
+
+ // release callback
+ void (*release)(struct ArrowAsyncDeviceStreamHandler* self);
+
+ // opaque handler-specific data
+ void* private_data;
+ };
+
+ #endif // ARROW_C_ASYNC_STREAM_INTERFACE
+
+.. note::
+ The canonical guard ``ARROW_C_ASYNC_STREAM_INTERFACE`` is meant to avoid
+ duplicate definitions if two projects copy the C async stream interface
+ definitions into their own headers, and a third-party project includes
+ from these two projects. It is therefore important that this guard is kept
+ exactly as-is when these definitions are copied.
+
+The ArrowAsyncDeviceStreamHandler structure
+'''''''''''''''''''''''''''''''''''''''''''
+
+The structure has the following fields:
+
+.. c:member:: int (*ArrowAsyncDeviceStreamHandler.on_schema)(struct ArrowAsyncDeviceStreamHandler*, struct ArrowAsyncProducer*, struct ArrowSchema*, const char*)
+
+ *Mandatory.* Handler for receiving the schema of the stream. All records should
+ match the provided schema. If successful, the function should return 0, otherwise
+ it should return an ``errno``-compatible error code.
+
+ The ``const char*`` parameter exists for producers to provide any extra contextual information
+ they want, such as the total number of rows in the stream, statistics, or otherwise. This is
+ encoded in the same format as :c:member:`ArrowSchema.metadata`. If not ``NULL``,
+ the lifetime is only the scope of the call to this function. A consumer who wants to maintain
+ the additional metadata beyond the lifetime of this call *MUST* copy the value themselves.
+
+ Unless the ``on_error`` handler is called, this will always get called exactly once and will be
+ the first method called on this object. As such the producer *MUST* provide an ``ArrowAsyncProducer``
+ object when calling this function to allow the consumer to manage back-pressure and flow control.
+ The producer maintains ownership of the ``ArrowAsyncProducer`` and must clean it up before or after
+ calling the release callback on this object.
+
+ A producer that receives a non-zero result here must not subsequently call anything other than
+ the release callback on this object.
+
+.. c:member:: int (*ArrowAsyncDeviceStreamHandler.on_next_task)(struct ArrowAsyncDeviceStreamHandler*, struct ArrowAsyncTask*, const char*)
+
+ *Mandatory.* Handler to be called when a new record is available for processing. The
+ schema for each record should be the same as the schema that ``on_schema`` was called with.
+ If successfully handled, the function should return 0, otherwise it should return an
+ ``errno``-compatible error code.
+
+ Rather than passing the record itself it receives an ``ArrowAsyncTask`` instead to facilitate
+ better consumer-focused thread control as far as receiving the data. A call to this function
+ simply indicates that data is available via the provided task.
+
+ The ``const char*`` parameter exists for producers to provide any extra contextual information
+ they want. This is encoded in the same format as :c:member:`ArrowSchema.metadata`. If not ``NULL``,
+ the lifetime is only the scope of the call to this function. A consumer who wants to maintain
+ the additional metadata beyond the lifetime of this call *MUST* copy the value themselves.
+
+ A producer *MUST NOT* call this concurrently from multiple different threads.
+
+ The :c:member:`ArrowAsyncProducer.request` callback must be called to start receiving calls to this
+ handler.
+
+ This function *MUST* be able to be called re-entrantly on the same thread to allow for the
+ :c:member:`ArrowAsyncProducer.request` callback to potentially call this method recursively.
+
+.. c:member:: void (*ArrowAsyncDeviceStreamHandler.on_error)(struct ArrowAsyncDeviceStreamHandler, int, const char*, const char*)
+
+ *Mandatory.* Handler to be called when an error is encountered by the producer. After calling
+ this, the ``release`` callback will be called as the last call on this struct. The parameters
+ are an ``errno``-compatible error code and an optional error message and metadata.
+
+ If the message and metadata are not ``NULL``, their lifetime is only valid during the scope
+ of this call. A consumer who wants to maintain these values past the return of this function
+ *MUST* copy the values themselves.
+
+ If the metadata parameter is not ``NULL``, to provide key-value error metadata, then it should
+ be encoded identically to the way that metadata is encoded in :c:member:`ArrowSchema.metadata`.
+
+ It is valid for this to be called by a producer with or without a preceding call to
+ :c:member:`ArrowAsyncProducer.request`. This callback *MUST NOT* call any methods of an
+ ``ArrowAsyncProducer`` object.
+
+.. c:member:: void (*ArrowAsyncDeviceStreamHandler.release)(struct ArrowAsyncDeviceStreamHandler*)
+
+ *Mandatory.* A pointer to a consumer-provided release callback for the handler.
+
+ It is valid for this to be called by a producer with or without a preceding call to
+ :c:member:`ArrowAsyncProducer.request`. This must not call any methods of an ``ArrowAsyncProducer``
+ object.
+
+.. c:member:: void* ArrowAsyncDeviceStreamHandler.private_data
+
+ *Optional.* An opaque pointer to consumer-provided private data.
+
+ Producers *MUST NOT* process this member. Lifetime of this member is handled by
+ the consumer, and especially by the release callback.
+
+The ArrowAsyncTask structure
+''''''''''''''''''''''''''''
+
+The purpose of using a Task object rather than passing the array directly to the ``on_next``
+callback is to allow for more complex and efficient thread handling. Utilizing a Task
+object allows for a producer to separate the "decoding" logic from the I/O, enabling a
+consumer to avoid transferring data between CPU cores (e.g. from one L1/L2 cache to another).
Review Comment:
But the consumer has zero knowledge whether it would be beneficial to offload `get_data` to separate threads, while the producer does. Why would this responsibility fall on the consumer?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-43631: [C][Format] Add ArrowAsyncDeviceStreamHandler interface [arrow]
Posted by "westonpace (via GitHub)" <gi...@apache.org>.
westonpace commented on code in PR #43632:
URL: https://github.com/apache/arrow/pull/43632#discussion_r1766830856
##########
docs/source/format/CDeviceDataInterface.rst:
##########
@@ -650,6 +652,331 @@ The stream source is not assumed to be thread-safe. Consumers wanting to
call ``get_next`` from several threads should ensure those calls are
serialized.
+Async Device Stream Interface
+=============================
+
+The :ref:`C stream interface <_c-device-stream-interface>` provides a synchronous
+API centered around the consumer calling the callback functions to retrieve
+the next record batch. For some bindings, use cases, and interoperability, a more
+asynchronous, producer-focused interface may be required. These scenarios can utilize
+the ``ArrowAsyncDeviceStreamHandler``.
+
+Semantics
+---------
+
+Rather than the producer providing a structure of callbacks for a consumer to
+call and retrieve records, the Async interface is a consumer allocated structure.
+The consumer allocated struct provides handler callbacks for the producer to call
+when the schema and chunks of data are available, rather than the consumer using
+a blocking pull-style iteration.
+
+In addition to the ``ArrowAsyncDeviceStreamHandler``, there are also two additional
+structs used for the full data flow: ``ArrowAsyncTask`` and ``ArrowAsyncProducer``.
+
+Structure Definition
+--------------------
+
+The C device async stream interface is defined with a single ``struct`` definition:
+
+.. code-block:: c
+
+ #ifndef ARROW_C_ASYNC_STREAM_INTERFACE
+ #define ARROW_C_ASYNC_STREAM_INTERFACE
+
+ struct ArrowAsyncTask {
+ int (*get_data)(struct ArrowArrayTask* self, struct ArrowDeviceArray* out);
+
+ void* private_data;
+ };
+
+ struct ArrowAsyncProducer {
+ void (*request)(struct ArrowAsyncProducer* self, uint64_t n);
+ void (*cancel)(struct ArrowAsyncProducer* self);
+
+ void (*release)(struct ArrowAsyncProducer* self);
+ void* private_data;
+ };
+
+ struct ArrowAsyncDeviceStreamHandler {
+ // handlers
+ int (*on_schema)(struct ArrowAsyncDeviceStreamHandler* self,
+ struct ArrowAsyncProducer* producer,
+ struct ArrowSchema* stream_schema, const char* addl_metadata);
+ int (*on_next_task)(struct ArrowAsyncDeviceStreamHandler* self,
+ struct ArrowAsyncTask* task, const char* metadata);
+ void (*on_error)(struct ArrowAsyncDeviceStreamHandler* self,
+ int code, const char* message, const char* metadata);
+
+ // release callback
+ void (*release)(struct ArrowAsyncDeviceStreamHandler* self);
+
+ // opaque handler-specific data
+ void* private_data;
+ };
+
+ #endif // ARROW_C_ASYNC_STREAM_INTERFACE
+
+.. note::
+ The canonical guard ``ARROW_C_ASYNC_STREAM_INTERFACE`` is meant to avoid
+ duplicate definitions if two projects copy the C async stream interface
+ definitions into their own headers, and a third-party project includes
+ from these two projects. It is therefore important that this guard is kept
+ exactly as-is when these definitions are copied.
+
+The ArrowAsyncDeviceStreamHandler structure
+'''''''''''''''''''''''''''''''''''''''''''
+
+The structure has the following fields:
+
+.. c:member:: int (*ArrowAsyncDeviceStreamHandler.on_schema)(struct ArrowAsyncDeviceStreamHandler*, struct ArrowAsyncProducer*, struct ArrowSchema*, const char*)
+
+ *Mandatory.* Handler for receiving the schema of the stream. All records should
+ match the provided schema. If successful, the function should return 0, otherwise
+ it should return an ``errno``-compatible error code.
+
+ The ``const char*`` parameter exists for producers to provide any extra contextual information
+ they want, such as the total number of rows in the stream, statistics, or otherwise. This is
+ encoded in the same format as :c:member:`ArrowSchema.metadata`. If not ``NULL``,
+ the lifetime is only the scope of the call to this function. A consumer who wants to maintain
+ the additional metadata beyond the lifetime of this call *MUST* copy the value themselves.
+
+ Unless the ``on_error`` handler is called, this will always get called exactly once and will be
+ the first method called on this object. As such the producer *MUST* provide an ``ArrowAsyncProducer``
+ object when calling this function to allow the consumer to manage back-pressure and flow control.
+ The producer maintains ownership of the ``ArrowAsyncProducer`` and must clean it up before or after
+ calling the release callback on this object.
+
+ A producer that receives a non-zero result here must not subsequently call anything other than
+ the release callback on this object.
+
+.. c:member:: int (*ArrowAsyncDeviceStreamHandler.on_next_task)(struct ArrowAsyncDeviceStreamHandler*, struct ArrowAsyncTask*, const char*)
+
+ *Mandatory.* Handler to be called when a new record is available for processing. The
+ schema for each record should be the same as the schema that ``on_schema`` was called with.
+ If successfully handled, the function should return 0, otherwise it should return an
+ ``errno``-compatible error code.
+
+ Rather than passing the record itself it receives an ``ArrowAsyncTask`` instead to facilitate
+ better consumer-focused thread control as far as receiving the data. A call to this function
+ simply indicates that data is available via the provided task.
+
+ The ``const char*`` parameter exists for producers to provide any extra contextual information
+ they want. This is encoded in the same format as :c:member:`ArrowSchema.metadata`. If not ``NULL``,
+ the lifetime is only the scope of the call to this function. A consumer who wants to maintain
+ the additional metadata beyond the lifetime of this call *MUST* copy the value themselves.
+
+ A producer *MUST NOT* call this concurrently from multiple different threads.
+
+ The :c:member:`ArrowAsyncProducer.request` callback must be called to start receiving calls to this
+ handler.
+
+ This function *MUST* be able to be called re-entrantly on the same thread to allow for the
+ :c:member:`ArrowAsyncProducer.request` callback to potentially call this method recursively.
+
+.. c:member:: void (*ArrowAsyncDeviceStreamHandler.on_error)(struct ArrowAsyncDeviceStreamHandler, int, const char*, const char*)
+
+ *Mandatory.* Handler to be called when an error is encountered by the producer. After calling
+ this, the ``release`` callback will be called as the last call on this struct. The parameters
+ are an ``errno``-compatible error code and an optional error message and metadata.
+
+ If the message and metadata are not ``NULL``, their lifetime is only valid during the scope
+ of this call. A consumer who wants to maintain these values past the return of this function
+ *MUST* copy the values themselves.
+
+ If the metadata parameter is not ``NULL``, to provide key-value error metadata, then it should
+ be encoded identically to the way that metadata is encoded in :c:member:`ArrowSchema.metadata`.
+
+ It is valid for this to be called by a producer with or without a preceding call to
+ :c:member:`ArrowAsyncProducer.request`. This callback *MUST NOT* call any methods of an
+ ``ArrowAsyncProducer`` object.
+
+.. c:member:: void (*ArrowAsyncDeviceStreamHandler.release)(struct ArrowAsyncDeviceStreamHandler*)
+
+ *Mandatory.* A pointer to a consumer-provided release callback for the handler.
+
+ It is valid for this to be called by a producer with or without a preceding call to
+ :c:member:`ArrowAsyncProducer.request`. This must not call any methods of an ``ArrowAsyncProducer``
+ object.
+
+.. c:member:: void* ArrowAsyncDeviceStreamHandler.private_data
+
+ *Optional.* An opaque pointer to consumer-provided private data.
+
+ Producers *MUST NOT* process this member. Lifetime of this member is handled by
+ the consumer, and especially by the release callback.
+
+The ArrowAsyncTask structure
+''''''''''''''''''''''''''''
+
+The purpose of using a Task object rather than passing the array directly to the ``on_next``
+callback is to allow for more complex and efficient thread handling. Utilizing a Task
+object allows for a producer to separate the "decoding" logic from the I/O, enabling a
+consumer to avoid transferring data between CPU cores (e.g. from one L1/L2 cache to another).
Review Comment:
> If that's the goal for the spec then it should be clearly spelled out IMHO. However, it means that get_data probably shouldn't use any parallelism because of potential oversubscription. This should also be spelled out: get_data is a purely CPU, single-thread blocking task.
It's a _possible_ goal but not the _only_ goal. For example, all my discussion so far has been "scanner is producer, query engine is consumer". However, I think the "query engine is producer, analytics script is consumer" is a completely valid use case as well. In the latter use case there is no separation of I/O and CPU to worry about, if both those thread pools exist then they both exist on the producer and I would full expect the tasks returned by the producer in this case to be shallow batch wrappers.
I agree that `get_data` should be a purely CPU single-threaded blocking task.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-43631: [C][Format] Add ArrowAsyncDeviceStreamHandler interface [arrow]
Posted by "lidavidm (via GitHub)" <gi...@apache.org>.
lidavidm commented on PR #43632:
URL: https://github.com/apache/arrow/pull/43632#issuecomment-2287830710
> I'm basically modeling this on the current lance file reader. The file reader knows (from the metadata) how many tasks there will be. get_next_task is a synchronous method that creates a task which is an asynchronous task.
I think outside of file readers, we're never going to know how many tasks there will be up front so I'm not sure if we want to choose an interface that bakes in this assumption.
> This gets more and more complicated the more times this mismatch is encountered in a pipeline.
Yes, I think we're going to have to end up with a lowest common denominator style approach. And overall I feel like the callback based approach is most likely to be the most natural between languages.
For the pull based stream here: https://github.com/apache/arrow/pull/43632#issuecomment-2281940063
- Doesn't this still mean the consumer can block the producer's thread on accident (by doing processing inside `wake`)?
For the task based approach here: https://github.com/apache/arrow/pull/43632#issuecomment-2282026639
- Is the reason for a separate task to help optimize cache usage? (Basically because the control flow is on the consumer's side, so it is the consumer thread that eventually calls get_next and can then immediately do processing without having to transfer threads to avoid blocking the producer)
- Would it be sufficient for that use case if we had a callback approach that produced a task instead of directly producing an array?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-43631: [C][Format] Add ArrowAsyncDeviceStreamHandler interface [arrow]
Posted by "lidavidm (via GitHub)" <gi...@apache.org>.
lidavidm commented on PR #43632:
URL: https://github.com/apache/arrow/pull/43632#issuecomment-2303921960
@CurtHagenlocher you were commenting on the old https://github.com/apache/arrow-adbc/issues/811 issue, do you have any thoughts on the proposal/discussion here?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-43631: [C][Format] Add ArrowAsyncDeviceStreamHandler interface [arrow]
Posted by "bkietz (via GitHub)" <gi...@apache.org>.
bkietz commented on code in PR #43632:
URL: https://github.com/apache/arrow/pull/43632#discussion_r1722062113
##########
cpp/src/arrow/c/abi.h:
##########
@@ -228,6 +228,65 @@ struct ArrowDeviceArrayStream {
#endif // ARROW_C_DEVICE_STREAM_INTERFACE
+#ifndef ARROW_C_ASYNC_STREAM_INTERFACE
+#define ARROW_C_ASYNC_STREAM_INTERFACE
+
+// Similar to ArrowDeviceArrayStream, except designed for an asynchronous
+// style of interaction. While ArrowDeviceArrayStream provides producer
+// defined callbacks, this is intended to be created by the consumer instead.
+// The consumer passes this handler to the producer, which in turn uses the
+// callbacks to inform the consumer of events in the stream.
+struct ArrowAsyncDeviceStreamHandler {
+ // Handler for receiving a schema. The passed in stream_schema should be
+ // released or moved by the handler (producer is giving ownership of it to
+ // the handler).
+ //
+ // The `extension_param` argument can be null or can be used by a producer
+ // to pass arbitrary extra information to the consumer (such as total number
+ // of rows, context info, or otherwise).
+ //
+ // Return value: 0 if successful, `errno`-compatible error otherwise
+ int (*on_schema)(struct ArrowAsyncDeviceStreamHandler* self,
+ struct ArrowSchema* stream_schema, void* extension_param);
+
+ // Handler for receiving an array/record batch. Always called at least once
+ // unless an error is encountered (which would result in calling on_error).
+ // An empty/released array is passed to indicate the end of the stream if no
+ // errors have been encountered.
+ //
+ // The `extension_param` argument can be null or can be used by a producer
+ // to pass arbitrary extra information to the consumer.
+ //
+ // Return value: 0 if successful, `errno`-compatible error otherwise.
+ int (*on_next)(struct ArrowAsyncDeviceStreamHandler* self,
+ struct ArrowDeviceArray* next, void* extension_param);
Review Comment:
Perhaps we should make this SPSC structure more explicit? It would be easy for newcomers to the API to miss that and expect full synchronization then get a nasty surprise when multiple threads push to a single ArrowAsyncDeviceStreamHandler
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-43631: [C][Format] Add ArrowAsyncDeviceStreamHandler interface [arrow]
Posted by "zeroshade (via GitHub)" <gi...@apache.org>.
zeroshade commented on code in PR #43632:
URL: https://github.com/apache/arrow/pull/43632#discussion_r1750450890
##########
cpp/src/arrow/c/abi.h:
##########
@@ -228,6 +228,180 @@ struct ArrowDeviceArrayStream {
#endif // ARROW_C_DEVICE_STREAM_INTERFACE
+#ifndef ARROW_C_ASYNC_STREAM_INTERFACE
+#define ARROW_C_ASYNC_STREAM_INTERFACE
+
+// ArrowAsyncTask represents available data from a producer that was passed to
+// an invocation of `on_next_task` on the ArrowAsyncDeviceStreamHandler.
+struct ArrowAsyncTask {
+ // This callback should populate the ArrowDeviceArray associated with this task.
+ // The order of ArrowAsyncTasks provided by the producer enables a consumer to
+ // ensure the order of data to process.
+ //
+ // Returns: 0 if successful, errno-compatible error otherwise.
+ //
+ // If a non-0 value is returned then it should be followed by a call to `on_error`
+ // on the appropriate ArrowAsyncDeviceStreamHandler.
+ //
+ // Rather than a release callback, any required cleanup should be performed as part
+ // of the invocation of `get_data`. Ownership of the Array is passed to the consumer
+ // calling this, and so it must be released separately.
+ //
+ // It is only valid to call this method exactly once.
+ int (*get_data)(struct ArrowArrayTask* self, struct ArrowDeviceArray* out);
+
+ // opaque task-specific data
+ void* private_data;
+};
+
+// ArrowAsyncProducer represents a 1-to-1 relationship between an async producer
+// and consumer. This object allows the consumer to perform backpressure and flow
+// control on the asynchronous stream processing. This object must be owned by the
+// producer who creates it, and thus is responsible for cleaning it up.
+struct ArrowAsyncProducer {
+ // A consumer must call this function to start receiving on_next_task calls.
+ //
+ // It *must* be valid to call this synchronously from within on_next_task or
+ // on_schema, with a producer placing an upper bound on possible synchronous
+ // recursion between calls of on_next_task -> request -> on_next_task ->....
+ //
+ // After cancel has been called, additional calls to this function must be NOPs,
+ // but allowed.
+ //
+ // While not cancelled, calling this function must register the given number of
+ // additional arrays/batches to be produced with the producer. The producer should
+ // only call `on_next_task` at most the registered number of arrays before propagating
+ // backpressure.
+ //
+ // While not cancelled, calling request MAY synchronously call `on_next_task`,
+ // `on_error`, or `release` on the ArrowAsyncDeviceStreamHandler.
+ //
+ // Any error encountered by calling request must be propagated by calling the `on_error`
+ // callback of the ArrowAsyncDeviceStreamHandler.
+ //
+ // A producer must support an unbounded number of calls to request and must support
+ // a total registered demand (sum requested - sum delivered) of up to UINT64_MAX.
+ void (*request)(struct ArrowAsyncProducer* self, uint64_t n);
+
+ // This cancel callback signals a producer that it must eventually stop making calls
+ // to on_next_task. It must be idempotent and thread-safe.
+ //
+ // After calling cancel once, subsequent calls must be NOPs.
+ //
+ // It is not required that calling cancel affect the producer immediately, only that it
+ // must eventually stop calling on_next_task and subsequently call release on the
+ // async handler. As such, a consumer must be prepared to receive one or more calls to
+ // `on_next_task` even after calling cancel if there are still requested arrays pending.
+ //
+ // Any error encountered during handling a call to cancel must be reported via the on_error
+ // callback on the async stream handler.
+ void (*cancel)(struct ArrowAsyncProducer* self);
+
+ // producer-specific opaque data.
+ void* private_data;
+};
+
+// Similar to ArrowDeviceArrayStream, except designed for an asynchronous
+// style of interaction. While ArrowDeviceArrayStream provides producer
+// defined callbacks, this is intended to be created by the consumer instead.
+// The consumer passes this handler to the producer, which in turn uses the
+// callbacks to inform the consumer of events in the stream.
+struct ArrowAsyncDeviceStreamHandler {
+ // Handler for receiving a schema. The passed in stream_schema must be
+ // released or moved by the handler (producer is giving ownership of the schema to
+ // the handler, but not ownership of the top level object itself).
+ //
+ // With the exception of an error occurring (on_error), this must be the first
+ // callback function which is called by a producer and must only be called exactly
+ // once. As such, the producer should provide a valid ArrowAsyncProducer instance
+ // so the consumer can control the flow. See the documentation on ArrowAsyncProducer
+ // for how it works. The ArrowAsyncProducer is owned by the producer who calls this
+ // function and thus the producer is responsible for cleaning it up when calling
+ // the release callback of this handler.
+ //
+ // The addl_metadata argument can be null or can be used by a producer
+ // to pass arbitrary extra information to the consumer beyond the metadata in the schema
+ // itself (such as total number of rows, context info, or otherwise). The data should
+ // be passed using the same encoding as the metadata within the ArrowSchema struct
+ // itself (defined in the spec at
+ // https://arrow.apache.org/docs/format/CDataInterface.html#c.ArrowSchema.metadata)
+ //
+ // If addl_metadata is non-null then it only needs to exist for the lifetime of this call,
+ // a consumer who wants it to live after that must copy it to ensure lifetime.
+ //
+ // Return value: 0 if successful, `errno`-compatible error otherwise
+ //
+ // A producer that receives a non-zero return here should stop producing and eventually
+ // call release instead.
+ int (*on_schema)(struct ArrowAsyncDeviceStreamHandler* self,
+ struct ArrowAsyncProducer* producer, struct ArrowSchema* stream_schema,
+ const char* addl_metadata);
+
+ // Handler for receiving data. This is called when data is available providing an
+ // ArrowAsyncTask struct to signify it. The consumer is responsible for calling
+ // the release callback on the ArrowAsyncTask after retrieving the array/batch.
+ // An empty/released task is passed to indicate the end of the stream if no
+ // errors have been encountered.
+ //
+ // The `request` callback of a provided ArrowAsyncProducer must be called in order
+ // to start receiving calls to this handler.
+ //
+ // The metadata argument can be null or can be used by a producer
+ // to pass arbitrary extra information to the consumer (such as total number
+ // of rows, context info, or otherwise). The data should be passed using the same
+ // encoding as the metadata within the ArrowSchema struct itself (defined in
+ // the spec at
+ // https://arrow.apache.org/docs/format/CDataInterface.html#c.ArrowSchema.metadata)
+ //
+ // If metadata is non-null then it only needs to exist for the lifetime of this call,
+ // a consumer who wants it to live after that must copy it to ensure lifetime.
+ //
+ // A producer *must not* call this concurrently from multiple different threads.
+ //
+ // A consumer must be prepared to received one or more calls to this callback even
+ // after calling cancel on the corresponding ArrowAsyncProducer, as cancel does not
+ // guarantee it happens immediately.
+ //
+ // Return value: 0 if successful, `errno`-compatible error otherwise.
+ int (*on_next_task)(struct ArrowAsyncDeviceStreamHandler* self,
+ struct ArrowAsyncTask* task, const char* metadata);
+
+ // Handler for encountering an error. The producer should call release after
+ // this returns to clean up any resources.
+ //
+ // If the message or metadata are non-null, they will only last as long as this
+ // function call. The consumer would need to perform a copy of the data if it is
+ // it is necessary for them live past the lifetime of this call.
+ //
+ // Error metadata should be encoded as with metadata in ArrowSchema, defined in
+ // the spec at
+ // https://arrow.apache.org/docs/format/CDataInterface.html#c.ArrowSchema.metadata
+ //
+ // After this call, producers should follow-up by calling the release callback.
+ //
+ // It is valid for this to be called by a producer with or without a preceding call
+ // to ArrowAsyncProducer.request.
+ //
+ // This callback must not call any methods of an ArrowAsyncProducer object.
+ void (*on_error)(struct ArrowAsyncDeviceStreamHandler* self, int code,
Review Comment:
Technically it can be anything that the producer wants, but given we have the consumer returning errno compatible codes, we should probably specify the same for this too. I'll do that
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-43631: [C][Format] Add ArrowAsyncDeviceStreamHandler interface [arrow]
Posted by "zeroshade (via GitHub)" <gi...@apache.org>.
zeroshade commented on PR #43632:
URL: https://github.com/apache/arrow/pull/43632#issuecomment-2334796541
The latest commits here now reflect an updated version of the interface for everyone's consideration and comments. Thanks!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-43631: [C][Format] Add ArrowAsyncDeviceStreamHandler interface [arrow]
Posted by "zeroshade (via GitHub)" <gi...@apache.org>.
zeroshade commented on PR #43632:
URL: https://github.com/apache/arrow/pull/43632#issuecomment-2346715576
If I don't get any further comments / requested changes by monday, I'll send an email to the mailing list to call for a vote on adding this to the format officially.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-43631: [C][Format] Add ArrowAsyncDeviceStreamHandler interface [arrow]
Posted by "zeroshade (via GitHub)" <gi...@apache.org>.
zeroshade commented on code in PR #43632:
URL: https://github.com/apache/arrow/pull/43632#discussion_r1759041721
##########
docs/source/format/CDeviceDataInterface.rst:
##########
@@ -650,6 +652,331 @@ The stream source is not assumed to be thread-safe. Consumers wanting to
call ``get_next`` from several threads should ensure those calls are
serialized.
+Async Device Stream Interface
+=============================
+
+The :ref:`C stream interface <_c-device-stream-interface>` provides a synchronous
+API centered around the consumer calling the callback functions to retrieve
+the next record batch. For some bindings, use cases, and interoperability, a more
+asynchronous, producer-focused interface may be required. These scenarios can utilize
+the ``ArrowAsyncDeviceStreamHandler``.
+
+Semantics
+---------
+
+Rather than the producer providing a structure of callbacks for a consumer to
+call and retrieve records, the Async interface is a consumer allocated structure.
+The consumer allocated struct provides handler callbacks for the producer to call
+when the schema and chunks of data are available, rather than the consumer using
+a blocking pull-style iteration.
+
+In addition to the ``ArrowAsyncDeviceStreamHandler``, there are also two additional
+structs used for the full data flow: ``ArrowAsyncTask`` and ``ArrowAsyncProducer``.
+
+Structure Definition
+--------------------
+
+The C device async stream interface is defined with a single ``struct`` definition:
+
+.. code-block:: c
+
+ #ifndef ARROW_C_ASYNC_STREAM_INTERFACE
+ #define ARROW_C_ASYNC_STREAM_INTERFACE
+
+ struct ArrowAsyncTask {
+ int (*get_data)(struct ArrowArrayTask* self, struct ArrowDeviceArray* out);
+
+ void* private_data;
+ };
+
+ struct ArrowAsyncProducer {
+ void (*request)(struct ArrowAsyncProducer* self, uint64_t n);
+ void (*cancel)(struct ArrowAsyncProducer* self);
+
+ void (*release)(struct ArrowAsyncProducer* self);
+ void* private_data;
+ };
+
+ struct ArrowAsyncDeviceStreamHandler {
+ // handlers
+ int (*on_schema)(struct ArrowAsyncDeviceStreamHandler* self,
+ struct ArrowAsyncProducer* producer,
+ struct ArrowSchema* stream_schema, const char* addl_metadata);
+ int (*on_next_task)(struct ArrowAsyncDeviceStreamHandler* self,
+ struct ArrowAsyncTask* task, const char* metadata);
+ void (*on_error)(struct ArrowAsyncDeviceStreamHandler* self,
+ int code, const char* message, const char* metadata);
+
+ // release callback
+ void (*release)(struct ArrowAsyncDeviceStreamHandler* self);
+
+ // opaque handler-specific data
+ void* private_data;
+ };
+
+ #endif // ARROW_C_ASYNC_STREAM_INTERFACE
+
+.. note::
+ The canonical guard ``ARROW_C_ASYNC_STREAM_INTERFACE`` is meant to avoid
+ duplicate definitions if two projects copy the C async stream interface
+ definitions into their own headers, and a third-party project includes
+ from these two projects. It is therefore important that this guard is kept
+ exactly as-is when these definitions are copied.
+
+The ArrowAsyncDeviceStreamHandler structure
+'''''''''''''''''''''''''''''''''''''''''''
+
+The structure has the following fields:
+
+.. c:member:: int (*ArrowAsyncDeviceStreamHandler.on_schema)(struct ArrowAsyncDeviceStreamHandler*, struct ArrowAsyncProducer*, struct ArrowSchema*, const char*)
+
+ *Mandatory.* Handler for receiving the schema of the stream. All records should
+ match the provided schema. If successful, the function should return 0, otherwise
+ it should return an ``errno``-compatible error code.
+
+ The ``const char*`` parameter exists for producers to provide any extra contextual information
+ they want, such as the total number of rows in the stream, statistics, or otherwise. This is
+ encoded in the same format as :c:member:`ArrowSchema.metadata`. If not ``NULL``,
+ the lifetime is only the scope of the call to this function. A consumer who wants to maintain
+ the additional metadata beyond the lifetime of this call *MUST* copy the value themselves.
+
+ Unless the ``on_error`` handler is called, this will always get called exactly once and will be
+ the first method called on this object. As such the producer *MUST* provide an ``ArrowAsyncProducer``
+ object when calling this function to allow the consumer to manage back-pressure and flow control.
+ The producer maintains ownership of the ``ArrowAsyncProducer`` and must clean it up before or after
+ calling the release callback on this object.
+
+ A producer that receives a non-zero result here must not subsequently call anything other than
+ the release callback on this object.
+
+.. c:member:: int (*ArrowAsyncDeviceStreamHandler.on_next_task)(struct ArrowAsyncDeviceStreamHandler*, struct ArrowAsyncTask*, const char*)
+
+ *Mandatory.* Handler to be called when a new record is available for processing. The
+ schema for each record should be the same as the schema that ``on_schema`` was called with.
+ If successfully handled, the function should return 0, otherwise it should return an
+ ``errno``-compatible error code.
+
+ Rather than passing the record itself it receives an ``ArrowAsyncTask`` instead to facilitate
+ better consumer-focused thread control as far as receiving the data. A call to this function
+ simply indicates that data is available via the provided task.
+
+ The ``const char*`` parameter exists for producers to provide any extra contextual information
+ they want. This is encoded in the same format as :c:member:`ArrowSchema.metadata`. If not ``NULL``,
+ the lifetime is only the scope of the call to this function. A consumer who wants to maintain
+ the additional metadata beyond the lifetime of this call *MUST* copy the value themselves.
+
+ A producer *MUST NOT* call this concurrently from multiple different threads.
+
+ The :c:member:`ArrowAsyncProducer.request` callback must be called to start receiving calls to this
+ handler.
+
+ This function *MUST* be able to be called re-entrantly on the same thread to allow for the
+ :c:member:`ArrowAsyncProducer.request` callback to potentially call this method recursively.
+
+.. c:member:: void (*ArrowAsyncDeviceStreamHandler.on_error)(struct ArrowAsyncDeviceStreamHandler, int, const char*, const char*)
+
+ *Mandatory.* Handler to be called when an error is encountered by the producer. After calling
+ this, the ``release`` callback will be called as the last call on this struct. The parameters
+ are an ``errno``-compatible error code and an optional error message and metadata.
+
+ If the message and metadata are not ``NULL``, their lifetime is only valid during the scope
+ of this call. A consumer who wants to maintain these values past the return of this function
+ *MUST* copy the values themselves.
+
+ If the metadata parameter is not ``NULL``, to provide key-value error metadata, then it should
+ be encoded identically to the way that metadata is encoded in :c:member:`ArrowSchema.metadata`.
+
+ It is valid for this to be called by a producer with or without a preceding call to
+ :c:member:`ArrowAsyncProducer.request`. This callback *MUST NOT* call any methods of an
+ ``ArrowAsyncProducer`` object.
+
+.. c:member:: void (*ArrowAsyncDeviceStreamHandler.release)(struct ArrowAsyncDeviceStreamHandler*)
+
+ *Mandatory.* A pointer to a consumer-provided release callback for the handler.
+
+ It is valid for this to be called by a producer with or without a preceding call to
+ :c:member:`ArrowAsyncProducer.request`. This must not call any methods of an ``ArrowAsyncProducer``
+ object.
+
+.. c:member:: void* ArrowAsyncDeviceStreamHandler.private_data
+
+ *Optional.* An opaque pointer to consumer-provided private data.
+
+ Producers *MUST NOT* process this member. Lifetime of this member is handled by
+ the consumer, and especially by the release callback.
+
+The ArrowAsyncTask structure
+''''''''''''''''''''''''''''
+
+The purpose of using a Task object rather than passing the array directly to the ``on_next``
+callback is to allow for more complex and efficient thread handling. Utilizing a Task
+object allows for a producer to separate the "decoding" logic from the I/O, enabling a
+consumer to avoid transferring data between CPU cores (e.g. from one L1/L2 cache to another).
+
+This producer-provided structure has the following fields:
+
+.. c:member:: int (*ArrowArrayTask.get_data)(struct ArrowArrayTask*, struct ArrowDeviceArray*)
+
+ *Mandatory.* A callback to populate the provided ``ArrowDeviceArray`` with the available data.
+ The order of ArrowAsyncTasks provided by the producer enables a consumer to know the order of
+ the data to process.
+
+ If a non-zero value is returned from this, it should be followed only by the producer calling
+ the ``on_error`` callback of the ``ArrowAsyncDeviceStreamHandler``. Because calling this method
+ is likely to be separate from the current control flow, returning a non-zero value to signal
+ an error occuring allows the current thread to decide handle the case accordingly, while still
+ allowing all error logging and handling to be centralized in the
+ :c:member:`ArrowAsyncDeviceStreamHandler.on_error` callback.
+
+ Rather than having a separate release callback, any required cleanup should be performed as part
+ of the invocation of this callback, with ownership of the Array being passed to the consumer-provided
+ parameter calling this, and must be released separately.
+
+ It is only valid to call this method exactly once.
+
+.. c:member:: void* ArrowArrayTask.private_data
+
+ *Optional.* An opaque pointer to consumer-provided private data.
+
+ Consumers *MUST NOT* process this member. Lifetime of this member is handled by
+ the producer who created this object, and should be cleaned up if necessary during
+ the call to :c:member:`ArrowArrayTask.get_data`.
+
+The ArrowAsyncProducer structure
+''''''''''''''''''''''''''''''''
+
+This producer-provided and managed object has the following fields:
+
+.. c:member:: void (*ArrowAsyncProducer.request)(struct ArrowAsyncProducer*, uint64_t)
+
+ *Mandatory.* This function must be called by a consumer to start receiving calls to
+ :c:member:`ArrowAsyncDeviceStreamHandler.on_next_task`. It *MUST* be valid to call
+ this synchronously from within :c:member:`ArrowAsyncDeviceStreamHandler.on_next_task`
+ or :c:member:`ArrowAsyncDeviceStreamHandler.on_schema`. As a result, this function
+ *MUST NOT* synchronously call ``on_next_task`` or ``on_error`` to avoid recursive
+ and reentrant callbacks.
+
+ After ``cancel`` is called, additional calls to this function must be a NOP, but allowed.
+
+ While not cancelled, calling this function registers the given number of additional
+ arrays/batches to be produced by the producer. A producer should only call
+ the appropriate ``on_next_task`` callback up to a maximum of the total sum of calls to
+ this method before propagating back-pressure / waiting.
+
+ Any error encountered by calling request must be propagated by calling the ``on_error``
+ callback of the ``ArrowAsyncDeviceStreamHandler``.
+
+ A producer *MUST* support an unbounded number of calls to this callback and *MUST* support
Review Comment:
It simplifies validation and checking for the consumer, again this is for compatibility with Reactive Streams. It ensures that a consumer should *always* be able to call it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-43631: [C][Format] Add ArrowAsyncDeviceStreamHandler interface [arrow]
Posted by "lidavidm (via GitHub)" <gi...@apache.org>.
lidavidm commented on PR #43632:
URL: https://github.com/apache/arrow/pull/43632#issuecomment-2278940567
> > Do we want to handle cancellation here?
>
> I think that with the given proposal, if the producer sees/wants a cancellation it can call `on_error()` with `ECANCEL`; if the consumer sees or wants a cancellation, it can signal it by returning `ECANCEL`?
You may also want to cancel outside of the consumer, though.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-43631: [C][Format] Add ArrowAsyncDeviceStreamHandler interface [arrow]
Posted by "zeroshade (via GitHub)" <gi...@apache.org>.
zeroshade commented on PR #43632:
URL: https://github.com/apache/arrow/pull/43632#issuecomment-2304890696
> There's the partial compromise option:
Something like `int on_next_task(ArrowAsyncDeviceStreamHandler*, ArrowArrayTask*)` ?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-43631: [C][Format] Add ArrowAsyncDeviceStreamHandler interface [arrow]
Posted by "CurtHagenlocher (via GitHub)" <gi...@apache.org>.
CurtHagenlocher commented on PR #43632:
URL: https://github.com/apache/arrow/pull/43632#issuecomment-2304941744
> @CurtHagenlocher you were commenting on the old [apache/arrow-adbc#811](https://github.com/apache/arrow-adbc/issues/811) issue, do you have any thoughts on the proposal/discussion here?
I need to find some time in the next 2-3 days to dig into this again.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-43631: [C][Format] Add ArrowAsyncDeviceStreamHandler interface [arrow]
Posted by "lidavidm (via GitHub)" <gi...@apache.org>.
lidavidm commented on PR #43632:
URL: https://github.com/apache/arrow/pull/43632#issuecomment-2306060082
> Either remove the extension parameter or change it to be const char* using the metadata encoding like we're doing for the on_error callback
I think the metadata encoding is reasonable. I guess it'll force people to do a lot of string parsing, but that's just how it goes.
FWIW, do we need to do anything to accommodate the arrow::ArrayStatistics API that @kou has been working on?
> Do we want to go ahead with the waker approach?
I'm not quite sold on this if we want to have a callback based API. Though I understand wanting to have the most efficient possible theoretical API. At least from my Flight/gRPC based view, though, it seems difficult to take advantage of.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-43631: [C][Format] Add ArrowAsyncDeviceStreamHandler interface [arrow]
Posted by "bkietz (via GitHub)" <gi...@apache.org>.
bkietz commented on code in PR #43632:
URL: https://github.com/apache/arrow/pull/43632#discussion_r1775883307
##########
cpp/src/arrow/c/abi.h:
##########
@@ -228,6 +228,207 @@ struct ArrowDeviceArrayStream {
#endif // ARROW_C_DEVICE_STREAM_INTERFACE
+#ifndef ARROW_C_ASYNC_STREAM_INTERFACE
+#define ARROW_C_ASYNC_STREAM_INTERFACE
+
+// ArrowAsyncTask represents available data from a producer that was passed to
+// an invocation of `on_next_task` on the ArrowAsyncDeviceStreamHandler.
+//
+// The reason for this Task approach instead of the Async interface returning
+// the Array directly is to allow for more complex thread handling and reducing
+// context switching and data transfers between CPU cores (e.g. from one L1/L2
+// cache to another) if desired.
+//
+// For example, the `on_next_task` callback can be called when data is ready, while
+// the producer puts potential "decoding" logic in the `ArrowAsyncTask` object. This
+// allows for the producer to manage the I/O on one thread which calls `on_next_task`
+// and the consumer can determine when the decoding (producer logic in the `get_data`
+// callback of the task) occurs and on which thread, to avoid a CPU core transfer
+// (data staying in the L2 cache).
+struct ArrowAsyncTask {
+ // This callback should populate the ArrowDeviceArray associated with this task.
+ // The order of ArrowAsyncTasks provided by the producer enables a consumer to
+ // ensure the order of data to process.
+ //
+ // This function is expected to be synchronous, but should not perform any blocking
+ // I/O. Ideally it should be as cheap as possible so as to not tie up the consumer
+ // thread unnecessarily.
+ //
+ // Returns: 0 if successful, errno-compatible error otherwise.
+ //
+ // If a non-0 value is returned then it should be followed by a call to `on_error`
+ // on the appropriate ArrowAsyncDeviceStreamHandler. This is because the it's highly
+ // likely that whatever is calling this function may be entirely disconnected from
+ // the current control flow. Indicating an error here with a non-zero return allows
+ // the current flow to be aware of the error occurring, while still allowing any
+ // logging or error handling to still be centralized in the `on_error` callback of
+ // the original Async handler.
+ //
+ // Rather than a release callback, any required cleanup should be performed as part
+ // of the invocation of `get_data`. Ownership of the Array is passed to the consumer
+ // calling this, and so it must be released separately.
+ //
+ // It is only valid to call this method exactly once.
+ int (*get_data)(struct ArrowArrayTask* self, struct ArrowDeviceArray* out);
+
+ // opaque task-specific data
+ void* private_data;
+};
+
+// ArrowAsyncProducer represents a 1-to-1 relationship between an async producer
+// and consumer. This object allows the consumer to perform backpressure and flow
+// control on the asynchronous stream processing. This object must be owned by the
+// producer who creates it, and thus is responsible for cleaning it up.
+struct ArrowAsyncProducer {
+ // A consumer must call this function to start receiving on_next_task calls.
+ //
+ // It *must* be valid to call this synchronously from within `on_next_task` or
+ // `on_schema`, but this function *must not* immediately call `on_next_task` so as
+ // to avoid recursion and reentrant callbacks.
+ //
+ // After cancel has been called, additional calls to this function must be NOPs,
+ // but allowed. While not cancelled, calling this function must register the
+ // given number of additional arrays/batches to be produced with the producer.
+ // The producer should only call `on_next_task` at most the registered number
+ // of arrays before propagating backpressure.
+ //
+ // Any error encountered by calling request must be propagated by calling the `on_error`
+ // callback of the ArrowAsyncDeviceStreamHandler.
+ //
+ // While not cancelled, any subsequent calls to `on_next_task`, `on_error` or
+ // `release` should be scheduled by the producer to be called later rather
+ //
+ // It is invalid for a consumer to call this with a value of n <= 0, producers should
+ // error if given such a value.
+ void (*request)(struct ArrowAsyncProducer* self, int64_t n);
+
+ // This cancel callback signals a producer that it must eventually stop making calls
+ // to on_next_task. It must be idempotent and thread-safe. After calling cancel once,
+ // subsequent calls must be NOPs. This must not call any consumer-side handlers other
+ // than `on_error`.
+ //
+ // It is not required that calling cancel affect the producer immediately, only that it
+ // must eventually stop calling on_next_task and subsequently call release on the
+ // async handler. As such, a consumer must be prepared to receive one or more calls to
+ // `on_next_task` even after calling cancel if there are still requested arrays pending.
+ //
+ // Successful cancellation should *not* result in the producer calling `on_error`, it
+ // should finish out any remaining tasks and eventually call `release`.
+ //
+ // Any error encountered during handling a call to cancel must be reported via the
+ // on_error callback on the async stream handler.
+ void (*cancel)(struct ArrowAsyncProducer* self);
+
+ // producer-specific opaque data.
+ void* private_data;
+};
+
+// Similar to ArrowDeviceArrayStream, except designed for an asynchronous
+// style of interaction. While ArrowDeviceArrayStream provides producer
+// defined callbacks, this is intended to be created by the consumer instead.
+// The consumer passes this handler to the producer, which in turn uses the
+// callbacks to inform the consumer of events in the stream.
+struct ArrowAsyncDeviceStreamHandler {
+ // Handler for receiving a schema. The passed in stream_schema must be
+ // released or moved by the handler (producer is giving ownership of the schema to
+ // the handler, but not ownership of the top level object itself).
+ //
+ // With the exception of an error occurring (on_error), this must be the first
+ // callback function which is called by a producer and must only be called exactly
+ // once. As such, the producer should provide a valid ArrowAsyncProducer instance
+ // so the consumer can control the flow. See the documentation on ArrowAsyncProducer
+ // for how it works. The ArrowAsyncProducer is owned by the producer who calls this
+ // function and thus the producer is responsible for cleaning it up when calling
+ // the release callback of this handler.
+ //
+ // The addl_metadata argument can be null or can be used by a producer
+ // to pass arbitrary extra information to the consumer beyond the metadata in the schema
+ // itself (such as total number of rows, context info, or otherwise). The data should
+ // be passed using the same encoding as the metadata within the ArrowSchema struct
+ // itself (defined in the spec at
+ // https://arrow.apache.org/docs/format/CDataInterface.html#c.ArrowSchema.metadata)
+ //
+ // If addl_metadata is non-null then it only needs to exist for the lifetime of this
+ // call, a consumer who wants it to live after that must copy it to ensure lifetime.
+ //
+ // Return value: 0 if successful, `errno`-compatible error otherwise
+ //
+ // A producer that receives a non-zero return here should stop producing and eventually
+ // call release instead.
+ int (*on_schema)(struct ArrowAsyncDeviceStreamHandler* self,
+ struct ArrowAsyncProducer* producer, struct ArrowSchema* stream_schema,
+ const char* addl_metadata);
Review Comment:
Maybe this metadata could be `ArrowAsyncProducer::metadata`? That would eliminate the ambiguity
##########
cpp/src/arrow/c/abi.h:
##########
@@ -228,6 +228,207 @@ struct ArrowDeviceArrayStream {
#endif // ARROW_C_DEVICE_STREAM_INTERFACE
+#ifndef ARROW_C_ASYNC_STREAM_INTERFACE
+#define ARROW_C_ASYNC_STREAM_INTERFACE
+
+// ArrowAsyncTask represents available data from a producer that was passed to
+// an invocation of `on_next_task` on the ArrowAsyncDeviceStreamHandler.
+//
+// The reason for this Task approach instead of the Async interface returning
+// the Array directly is to allow for more complex thread handling and reducing
+// context switching and data transfers between CPU cores (e.g. from one L1/L2
+// cache to another) if desired.
+//
+// For example, the `on_next_task` callback can be called when data is ready, while
+// the producer puts potential "decoding" logic in the `ArrowAsyncTask` object. This
+// allows for the producer to manage the I/O on one thread which calls `on_next_task`
+// and the consumer can determine when the decoding (producer logic in the `extract_data`
+// callback of the task) occurs and on which thread, to avoid a CPU core transfer
+// (data staying in the L2 cache).
+struct ArrowAsyncTask {
+ // This callback should populate the ArrowDeviceArray associated with this task.
+ // The order of ArrowAsyncTasks provided by the producer enables a consumer to
+ // ensure the order of data to process.
+ //
+ // This function is expected to be synchronous, but should not perform any blocking
+ // I/O. Ideally it should be as cheap as possible so as to not tie up the consumer
+ // thread unnecessarily.
+ //
+ // Returns: 0 if successful, errno-compatible error otherwise.
+ //
+ // If a non-0 value is returned then it should be followed by a call to `on_error`
+ // on the appropriate ArrowAsyncDeviceStreamHandler. This is because it's highly
+ // likely that whatever is calling this function may be entirely disconnected from
+ // the current control flow. Indicating an error here with a non-zero return allows
+ // the current flow to be aware of the error occurring, while still allowing any
+ // logging or error handling to still be centralized in the `on_error` callback of
+ // the original Async handler.
+ //
+ // Rather than a release callback, any required cleanup should be performed as part
+ // of the invocation of `extract_data`. Ownership of the Array is passed to the consumer
+ // calling this, and so it must be released separately.
+ //
+ // It is only valid to call this method exactly once.
+ int (*extract_data)(struct ArrowArrayTask* self, struct ArrowDeviceArray* out);
+
+ // opaque task-specific data
+ void* private_data;
+};
+
+// ArrowAsyncProducer represents a 1-to-1 relationship between an async producer
+// and consumer. This object allows the consumer to perform backpressure and flow
+// control on the asynchronous stream processing. This object must be owned by the
+// producer who creates it, and thus is responsible for cleaning it up.
Review Comment:
```suggestion
// producer who creates it and the producer is responsible for cleaning it up.
```
ArrowAsyncProducer is effectively a lazily initialized member of ArrowAsyncDeviceStreamHandler since its lifetime extends from handler.on_schema() to handler.release(). Maybe embedding the struct there would make this more obvious? IE, have `ArrowAsyncDeviceStreamHandler::producer` of type `struct ArrowAsyncProducer` which the producer initializes before calling on_schema()?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-43631: [C][Format] Add ArrowAsyncDeviceStreamHandler interface [arrow]
Posted by "westonpace (via GitHub)" <gi...@apache.org>.
westonpace commented on PR #43632:
URL: https://github.com/apache/arrow/pull/43632#issuecomment-2297608446
@lidavidm
> Doesn't this still mean the consumer can block the producer's thread on accident (by doing processing inside wake)?
Yes, implementors would be cautioned to implement `wake` as cheaply as possible. I think rust's implementation is a bunch of lock-free stuff but a short-held mutex to move things from one queue to another is probably fine too.
> Is the reason for a separate task to help optimize cache usage?
Yes.
> Would it be sufficient for that use case if we had a callback approach that produced a task instead of directly producing an array?
Off the top of my head, yes, that should be fine. I think you can separate thread transfer from push/pull in that way.
> I don't have much experience on the performance side, but in the development time/lines-of-code side, trying to make a producer that expects to push its output interact with a consumer that wants to pull is expensive (the reverse is also true). This gets more and more complicated the more times this mismatch is encountered in a pipeline.
I'm not entirely sure I agree. Converting from synchronous pull to asynchronous push is pretty straightforward (need to introduce a polling thread). I also don't think maintenance of a new stable ABI is entirely justified by a usability concern but I won't die on that hill.
That being said, I think there are two main factors in desiring an asynchronous interface over a synchronous interface.
1. Manage the # of threads
2. Control context switching
I would guess that 1 is more common than 2 and I think an ABI that just provides just 1 is probably ok.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-43631: [C][Format] Add ArrowAsyncDeviceStreamHandler interface [arrow]
Posted by "paleolimbot (via GitHub)" <gi...@apache.org>.
paleolimbot commented on PR #43632:
URL: https://github.com/apache/arrow/pull/43632#issuecomment-2286856527
Echoing all the thanks to Weston for the detailed response!
I wonder if it is worth clarifying the goals and non-goals of this proposal. In my mind, this is about rectifying two very different ways engines/APIs operate (push vs. pull). I don't have much experience on the performance side, but in the development time/lines-of-code side, trying to make a producer that expects to push its output interact with a consumer that wants to pull is expensive (the reverse is also true). This gets more and more complicated the more times this mismatch is encountered in a pipeline.
I worry that in the quest for the best possible performance that we loose any development time/lines-of-code advantage that a simpler approach might have enabled! I also worry that an ABI that becomes too opinionated about how a scanner should be implemented will still not be able to express other ("non optimal"?) scanners that, for historical reasons (or because we were wrong about what an optimal scanner looks like), don't work that way. I still think that something like the original proposal (with clear, if imperfect, expectations about what can or should happen in the callbacks) is *a* missing piece (if not *the* missing piece).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-43631: [C][Format] Add ArrowAsyncDeviceStreamHandler interface [arrow]
Posted by "pitrou (via GitHub)" <gi...@apache.org>.
pitrou commented on code in PR #43632:
URL: https://github.com/apache/arrow/pull/43632#discussion_r1757495211
##########
docs/source/format/CDeviceDataInterface.rst:
##########
@@ -650,6 +652,331 @@ The stream source is not assumed to be thread-safe. Consumers wanting to
call ``get_next`` from several threads should ensure those calls are
serialized.
+Async Device Stream Interface
+=============================
+
+The :ref:`C stream interface <_c-device-stream-interface>` provides a synchronous
+API centered around the consumer calling the callback functions to retrieve
+the next record batch. For some bindings, use cases, and interoperability, a more
+asynchronous, producer-focused interface may be required. These scenarios can utilize
+the ``ArrowAsyncDeviceStreamHandler``.
+
+Semantics
+---------
+
+Rather than the producer providing a structure of callbacks for a consumer to
+call and retrieve records, the Async interface is a consumer allocated structure.
+The consumer allocated struct provides handler callbacks for the producer to call
+when the schema and chunks of data are available, rather than the consumer using
+a blocking pull-style iteration.
+
+In addition to the ``ArrowAsyncDeviceStreamHandler``, there are also two additional
+structs used for the full data flow: ``ArrowAsyncTask`` and ``ArrowAsyncProducer``.
+
+Structure Definition
+--------------------
+
+The C device async stream interface is defined with a single ``struct`` definition:
+
+.. code-block:: c
+
+ #ifndef ARROW_C_ASYNC_STREAM_INTERFACE
+ #define ARROW_C_ASYNC_STREAM_INTERFACE
+
+ struct ArrowAsyncTask {
+ int (*get_data)(struct ArrowArrayTask* self, struct ArrowDeviceArray* out);
+
+ void* private_data;
+ };
+
+ struct ArrowAsyncProducer {
+ void (*request)(struct ArrowAsyncProducer* self, uint64_t n);
+ void (*cancel)(struct ArrowAsyncProducer* self);
+
+ void (*release)(struct ArrowAsyncProducer* self);
+ void* private_data;
+ };
+
+ struct ArrowAsyncDeviceStreamHandler {
+ // handlers
+ int (*on_schema)(struct ArrowAsyncDeviceStreamHandler* self,
+ struct ArrowAsyncProducer* producer,
+ struct ArrowSchema* stream_schema, const char* addl_metadata);
+ int (*on_next_task)(struct ArrowAsyncDeviceStreamHandler* self,
+ struct ArrowAsyncTask* task, const char* metadata);
+ void (*on_error)(struct ArrowAsyncDeviceStreamHandler* self,
+ int code, const char* message, const char* metadata);
+
+ // release callback
+ void (*release)(struct ArrowAsyncDeviceStreamHandler* self);
+
+ // opaque handler-specific data
+ void* private_data;
+ };
+
+ #endif // ARROW_C_ASYNC_STREAM_INTERFACE
+
+.. note::
+ The canonical guard ``ARROW_C_ASYNC_STREAM_INTERFACE`` is meant to avoid
+ duplicate definitions if two projects copy the C async stream interface
+ definitions into their own headers, and a third-party project includes
+ from these two projects. It is therefore important that this guard is kept
+ exactly as-is when these definitions are copied.
+
+The ArrowAsyncDeviceStreamHandler structure
+'''''''''''''''''''''''''''''''''''''''''''
+
+The structure has the following fields:
+
+.. c:member:: int (*ArrowAsyncDeviceStreamHandler.on_schema)(struct ArrowAsyncDeviceStreamHandler*, struct ArrowAsyncProducer*, struct ArrowSchema*, const char*)
+
+ *Mandatory.* Handler for receiving the schema of the stream. All records should
+ match the provided schema. If successful, the function should return 0, otherwise
+ it should return an ``errno``-compatible error code.
+
+ The ``const char*`` parameter exists for producers to provide any extra contextual information
+ they want, such as the total number of rows in the stream, statistics, or otherwise. This is
+ encoded in the same format as :c:member:`ArrowSchema.metadata`. If not ``NULL``,
+ the lifetime is only the scope of the call to this function. A consumer who wants to maintain
+ the additional metadata beyond the lifetime of this call *MUST* copy the value themselves.
+
+ Unless the ``on_error`` handler is called, this will always get called exactly once and will be
+ the first method called on this object. As such the producer *MUST* provide an ``ArrowAsyncProducer``
+ object when calling this function to allow the consumer to manage back-pressure and flow control.
+ The producer maintains ownership of the ``ArrowAsyncProducer`` and must clean it up before or after
+ calling the release callback on this object.
+
+ A producer that receives a non-zero result here must not subsequently call anything other than
+ the release callback on this object.
+
+.. c:member:: int (*ArrowAsyncDeviceStreamHandler.on_next_task)(struct ArrowAsyncDeviceStreamHandler*, struct ArrowAsyncTask*, const char*)
+
+ *Mandatory.* Handler to be called when a new record is available for processing. The
+ schema for each record should be the same as the schema that ``on_schema`` was called with.
+ If successfully handled, the function should return 0, otherwise it should return an
+ ``errno``-compatible error code.
+
+ Rather than passing the record itself it receives an ``ArrowAsyncTask`` instead to facilitate
+ better consumer-focused thread control as far as receiving the data. A call to this function
+ simply indicates that data is available via the provided task.
+
+ The ``const char*`` parameter exists for producers to provide any extra contextual information
+ they want. This is encoded in the same format as :c:member:`ArrowSchema.metadata`. If not ``NULL``,
+ the lifetime is only the scope of the call to this function. A consumer who wants to maintain
+ the additional metadata beyond the lifetime of this call *MUST* copy the value themselves.
+
+ A producer *MUST NOT* call this concurrently from multiple different threads.
+
+ The :c:member:`ArrowAsyncProducer.request` callback must be called to start receiving calls to this
+ handler.
+
+ This function *MUST* be able to be called re-entrantly on the same thread to allow for the
+ :c:member:`ArrowAsyncProducer.request` callback to potentially call this method recursively.
+
+.. c:member:: void (*ArrowAsyncDeviceStreamHandler.on_error)(struct ArrowAsyncDeviceStreamHandler, int, const char*, const char*)
+
+ *Mandatory.* Handler to be called when an error is encountered by the producer. After calling
+ this, the ``release`` callback will be called as the last call on this struct. The parameters
+ are an ``errno``-compatible error code and an optional error message and metadata.
+
+ If the message and metadata are not ``NULL``, their lifetime is only valid during the scope
+ of this call. A consumer who wants to maintain these values past the return of this function
+ *MUST* copy the values themselves.
+
+ If the metadata parameter is not ``NULL``, to provide key-value error metadata, then it should
+ be encoded identically to the way that metadata is encoded in :c:member:`ArrowSchema.metadata`.
+
+ It is valid for this to be called by a producer with or without a preceding call to
+ :c:member:`ArrowAsyncProducer.request`. This callback *MUST NOT* call any methods of an
+ ``ArrowAsyncProducer`` object.
+
+.. c:member:: void (*ArrowAsyncDeviceStreamHandler.release)(struct ArrowAsyncDeviceStreamHandler*)
+
+ *Mandatory.* A pointer to a consumer-provided release callback for the handler.
+
+ It is valid for this to be called by a producer with or without a preceding call to
+ :c:member:`ArrowAsyncProducer.request`. This must not call any methods of an ``ArrowAsyncProducer``
+ object.
+
+.. c:member:: void* ArrowAsyncDeviceStreamHandler.private_data
+
+ *Optional.* An opaque pointer to consumer-provided private data.
+
+ Producers *MUST NOT* process this member. Lifetime of this member is handled by
+ the consumer, and especially by the release callback.
+
+The ArrowAsyncTask structure
+''''''''''''''''''''''''''''
+
+The purpose of using a Task object rather than passing the array directly to the ``on_next``
+callback is to allow for more complex and efficient thread handling. Utilizing a Task
+object allows for a producer to separate the "decoding" logic from the I/O, enabling a
+consumer to avoid transferring data between CPU cores (e.g. from one L1/L2 cache to another).
+
+This producer-provided structure has the following fields:
+
+.. c:member:: int (*ArrowArrayTask.get_data)(struct ArrowArrayTask*, struct ArrowDeviceArray*)
+
+ *Mandatory.* A callback to populate the provided ``ArrowDeviceArray`` with the available data.
+ The order of ArrowAsyncTasks provided by the producer enables a consumer to know the order of
+ the data to process.
+
+ If a non-zero value is returned from this, it should be followed only by the producer calling
+ the ``on_error`` callback of the ``ArrowAsyncDeviceStreamHandler``. Because calling this method
+ is likely to be separate from the current control flow, returning a non-zero value to signal
+ an error occuring allows the current thread to decide handle the case accordingly, while still
+ allowing all error logging and handling to be centralized in the
+ :c:member:`ArrowAsyncDeviceStreamHandler.on_error` callback.
+
+ Rather than having a separate release callback, any required cleanup should be performed as part
+ of the invocation of this callback, with ownership of the Array being passed to the consumer-provided
+ parameter calling this, and must be released separately.
+
+ It is only valid to call this method exactly once.
+
+.. c:member:: void* ArrowArrayTask.private_data
+
+ *Optional.* An opaque pointer to consumer-provided private data.
+
+ Consumers *MUST NOT* process this member. Lifetime of this member is handled by
+ the producer who created this object, and should be cleaned up if necessary during
+ the call to :c:member:`ArrowArrayTask.get_data`.
+
+The ArrowAsyncProducer structure
+''''''''''''''''''''''''''''''''
+
+This producer-provided and managed object has the following fields:
+
+.. c:member:: void (*ArrowAsyncProducer.request)(struct ArrowAsyncProducer*, uint64_t)
+
+ *Mandatory.* This function must be called by a consumer to start receiving calls to
+ :c:member:`ArrowAsyncDeviceStreamHandler.on_next_task`. It *MUST* be valid to call
+ this synchronously from within :c:member:`ArrowAsyncDeviceStreamHandler.on_next_task`
+ or :c:member:`ArrowAsyncDeviceStreamHandler.on_schema`. As a result, this function
+ *MUST NOT* synchronously call ``on_next_task`` or ``on_error`` to avoid recursive
+ and reentrant callbacks.
+
+ After ``cancel`` is called, additional calls to this function must be a NOP, but allowed.
+
+ While not cancelled, calling this function registers the given number of additional
+ arrays/batches to be produced by the producer. A producer should only call
+ the appropriate ``on_next_task`` callback up to a maximum of the total sum of calls to
+ this method before propagating back-pressure / waiting.
+
+ Any error encountered by calling request must be propagated by calling the ``on_error``
+ callback of the ``ArrowAsyncDeviceStreamHandler``.
+
+ A producer *MUST* support an unbounded number of calls to this callback and *MUST* support
+ a total registered demand (sum requested - sum delivered) of up to ``UINT64_MAX``.
+
+.. c:member:: void (*ArrowAsyncProducer.cancel)(struct ArrowAsyncProducer*)
+
+ *Mandatory.* This function signals to the producer that it must *eventually* stop calling
+ ``on_next_task``. Calls to ``cancel`` must be idempotent and thread-safe. After calling
+ it once, subsequent calls *MUST* be a NOP.
+
+ It is not required that calling ``cancel`` affect the producer *immediately*, only that it
+ must eventually stop calling ``on_next_task`` and then subsequently call ``release``
+ on the async handler object. As such, a consumer *MUST* be prepared to receive one or more
+ calls to ``on_next_task`` or ``on_error`` even after calling ``cancel`` if there are still
+ requested arrays pending.
+
+ Successful cancelling *MUST NOT* result in a producer calling
+ :c:member:`ArrowAsyncDeviceStreamHandler.on_error`, instead it should finish out any remaining
+ tasks (calling ``on_next_task`` accordingly) and eventually just call ``release``.
+
+ Any error encountered during handling a call to cancel must be reported via the ``on_error``
+ callback on the async stream handler.
+
+.. c:member:: void* ArrowAsyncProducer.private_data
+
+ *Optional.* An opaque pointer to producer-provided specific data.
+
+ Consumers *MUST NOT* process this member, the lifetime is owned by the producer
+ that constructed this object.
+
+Error Handling
+''''''''''''''
+
+Unlike the regular C Stream interface, the Async interface allows for errors to flow in
+both directions. As a result, error handling can be slightly more complex. Thus this spec
+designates the following rules:
+
+* If the producer encounters an error during processing, it should call the ``on_error``
+callback, and then call ``release`` after it returns.
+
+* If ``on_schema`` or ``on_next_task`` returns a non-zero integer value, the producer *should not*
+call the ``on_error`` callback, but instead should eventually call ``release`` at some point
+before or after any logging or processing of the error code.
+
+Result lifetimes
+''''''''''''''''
+
+The ``ArrowSchema`` passed to the ``on_schema`` callback must be released independantly,
+with the object itself needing to be moved to a consumer owned ``ArrowSchema`` object. The
+``ArrowSchema*`` passed as a parameter to the callback *MUST NOT* be stored and kept.
+
+The ``ArrowAsyncTask`` object provided to ``on_next_task`` is owned by the producer and
+will be cleaned up during the invocation of calling ``get_data`` on it. If the consumer
+doesn't care about the data, it should pass ``NULL`` instead of a valid ``ArrowDeviceArray*``.
+
+The ``const char*`` error ``message`` and ``metadata`` which are passed to ``on_error``
+are only valid within the scope of the ``on_error`` function itself. They must be copied
+if it is necessary for them to exist after it returns.
+
+Stream Handler Lifetime
+'''''''''''''''''''''''
+
+Lifetime of the async stream handler is managed using a release callback with similar
+usage as in :ref:`C data interface <c-data-interface-released>`.
+
+ArrowAsyncProducer Lifetime
+'''''''''''''''''''''''''''
+
+The lifetime of the ``ArrowAsyncProducer`` passed to ``on_schema`` is owned by the producer
+itself and should be managed by it. It *MUST* remain valid at least until just before
+calling ``release`` on the stream handler object.
+
+Thread safety
+'''''''''''''
+
+All handler functions should only be called in a serialized manner, but are not guaranteed
+to be called from the same thread every time. A producer should wait for handler callbacks to
+return before calling the next handler callback, and before calling the ``release`` callback.
+As a result, back-pressure is managed by how long it takes for the ``on_next`` handler to return.
Review Comment:
There's no `on_next` handler. Furthermore, I would expect backpressure to be managed by when `get_data` on the "task" is called.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-43631: [C][Format] Add ArrowAsyncDeviceStreamHandler interface [arrow]
Posted by "pitrou (via GitHub)" <gi...@apache.org>.
pitrou commented on code in PR #43632:
URL: https://github.com/apache/arrow/pull/43632#discussion_r1757480636
##########
docs/source/format/CDeviceDataInterface.rst:
##########
@@ -650,6 +652,331 @@ The stream source is not assumed to be thread-safe. Consumers wanting to
call ``get_next`` from several threads should ensure those calls are
serialized.
+Async Device Stream Interface
+=============================
+
+The :ref:`C stream interface <_c-device-stream-interface>` provides a synchronous
+API centered around the consumer calling the callback functions to retrieve
+the next record batch. For some bindings, use cases, and interoperability, a more
+asynchronous, producer-focused interface may be required. These scenarios can utilize
+the ``ArrowAsyncDeviceStreamHandler``.
+
+Semantics
+---------
+
+Rather than the producer providing a structure of callbacks for a consumer to
+call and retrieve records, the Async interface is a consumer allocated structure.
+The consumer allocated struct provides handler callbacks for the producer to call
+when the schema and chunks of data are available, rather than the consumer using
+a blocking pull-style iteration.
+
+In addition to the ``ArrowAsyncDeviceStreamHandler``, there are also two additional
+structs used for the full data flow: ``ArrowAsyncTask`` and ``ArrowAsyncProducer``.
+
+Structure Definition
+--------------------
+
+The C device async stream interface is defined with a single ``struct`` definition:
+
+.. code-block:: c
+
+ #ifndef ARROW_C_ASYNC_STREAM_INTERFACE
+ #define ARROW_C_ASYNC_STREAM_INTERFACE
+
+ struct ArrowAsyncTask {
+ int (*get_data)(struct ArrowArrayTask* self, struct ArrowDeviceArray* out);
+
+ void* private_data;
+ };
+
+ struct ArrowAsyncProducer {
+ void (*request)(struct ArrowAsyncProducer* self, uint64_t n);
+ void (*cancel)(struct ArrowAsyncProducer* self);
+
+ void (*release)(struct ArrowAsyncProducer* self);
+ void* private_data;
+ };
+
+ struct ArrowAsyncDeviceStreamHandler {
+ // handlers
+ int (*on_schema)(struct ArrowAsyncDeviceStreamHandler* self,
+ struct ArrowAsyncProducer* producer,
+ struct ArrowSchema* stream_schema, const char* addl_metadata);
+ int (*on_next_task)(struct ArrowAsyncDeviceStreamHandler* self,
+ struct ArrowAsyncTask* task, const char* metadata);
+ void (*on_error)(struct ArrowAsyncDeviceStreamHandler* self,
+ int code, const char* message, const char* metadata);
+
+ // release callback
+ void (*release)(struct ArrowAsyncDeviceStreamHandler* self);
+
+ // opaque handler-specific data
+ void* private_data;
+ };
+
+ #endif // ARROW_C_ASYNC_STREAM_INTERFACE
+
+.. note::
+ The canonical guard ``ARROW_C_ASYNC_STREAM_INTERFACE`` is meant to avoid
+ duplicate definitions if two projects copy the C async stream interface
+ definitions into their own headers, and a third-party project includes
+ from these two projects. It is therefore important that this guard is kept
+ exactly as-is when these definitions are copied.
+
+The ArrowAsyncDeviceStreamHandler structure
+'''''''''''''''''''''''''''''''''''''''''''
+
+The structure has the following fields:
+
+.. c:member:: int (*ArrowAsyncDeviceStreamHandler.on_schema)(struct ArrowAsyncDeviceStreamHandler*, struct ArrowAsyncProducer*, struct ArrowSchema*, const char*)
+
+ *Mandatory.* Handler for receiving the schema of the stream. All records should
+ match the provided schema. If successful, the function should return 0, otherwise
+ it should return an ``errno``-compatible error code.
+
+ The ``const char*`` parameter exists for producers to provide any extra contextual information
+ they want, such as the total number of rows in the stream, statistics, or otherwise. This is
+ encoded in the same format as :c:member:`ArrowSchema.metadata`. If not ``NULL``,
+ the lifetime is only the scope of the call to this function. A consumer who wants to maintain
+ the additional metadata beyond the lifetime of this call *MUST* copy the value themselves.
+
+ Unless the ``on_error`` handler is called, this will always get called exactly once and will be
+ the first method called on this object. As such the producer *MUST* provide an ``ArrowAsyncProducer``
+ object when calling this function to allow the consumer to manage back-pressure and flow control.
+ The producer maintains ownership of the ``ArrowAsyncProducer`` and must clean it up before or after
+ calling the release callback on this object.
+
+ A producer that receives a non-zero result here must not subsequently call anything other than
+ the release callback on this object.
+
+.. c:member:: int (*ArrowAsyncDeviceStreamHandler.on_next_task)(struct ArrowAsyncDeviceStreamHandler*, struct ArrowAsyncTask*, const char*)
+
+ *Mandatory.* Handler to be called when a new record is available for processing. The
+ schema for each record should be the same as the schema that ``on_schema`` was called with.
+ If successfully handled, the function should return 0, otherwise it should return an
+ ``errno``-compatible error code.
+
+ Rather than passing the record itself it receives an ``ArrowAsyncTask`` instead to facilitate
+ better consumer-focused thread control as far as receiving the data. A call to this function
+ simply indicates that data is available via the provided task.
+
+ The ``const char*`` parameter exists for producers to provide any extra contextual information
+ they want. This is encoded in the same format as :c:member:`ArrowSchema.metadata`. If not ``NULL``,
+ the lifetime is only the scope of the call to this function. A consumer who wants to maintain
+ the additional metadata beyond the lifetime of this call *MUST* copy the value themselves.
+
+ A producer *MUST NOT* call this concurrently from multiple different threads.
+
+ The :c:member:`ArrowAsyncProducer.request` callback must be called to start receiving calls to this
+ handler.
+
+ This function *MUST* be able to be called re-entrantly on the same thread to allow for the
+ :c:member:`ArrowAsyncProducer.request` callback to potentially call this method recursively.
+
+.. c:member:: void (*ArrowAsyncDeviceStreamHandler.on_error)(struct ArrowAsyncDeviceStreamHandler, int, const char*, const char*)
+
+ *Mandatory.* Handler to be called when an error is encountered by the producer. After calling
+ this, the ``release`` callback will be called as the last call on this struct. The parameters
+ are an ``errno``-compatible error code and an optional error message and metadata.
+
+ If the message and metadata are not ``NULL``, their lifetime is only valid during the scope
+ of this call. A consumer who wants to maintain these values past the return of this function
+ *MUST* copy the values themselves.
+
+ If the metadata parameter is not ``NULL``, to provide key-value error metadata, then it should
+ be encoded identically to the way that metadata is encoded in :c:member:`ArrowSchema.metadata`.
+
+ It is valid for this to be called by a producer with or without a preceding call to
+ :c:member:`ArrowAsyncProducer.request`. This callback *MUST NOT* call any methods of an
+ ``ArrowAsyncProducer`` object.
+
+.. c:member:: void (*ArrowAsyncDeviceStreamHandler.release)(struct ArrowAsyncDeviceStreamHandler*)
+
+ *Mandatory.* A pointer to a consumer-provided release callback for the handler.
+
+ It is valid for this to be called by a producer with or without a preceding call to
+ :c:member:`ArrowAsyncProducer.request`. This must not call any methods of an ``ArrowAsyncProducer``
+ object.
+
+.. c:member:: void* ArrowAsyncDeviceStreamHandler.private_data
+
+ *Optional.* An opaque pointer to consumer-provided private data.
+
+ Producers *MUST NOT* process this member. Lifetime of this member is handled by
+ the consumer, and especially by the release callback.
+
+The ArrowAsyncTask structure
+''''''''''''''''''''''''''''
+
+The purpose of using a Task object rather than passing the array directly to the ``on_next``
+callback is to allow for more complex and efficient thread handling. Utilizing a Task
+object allows for a producer to separate the "decoding" logic from the I/O, enabling a
+consumer to avoid transferring data between CPU cores (e.g. from one L1/L2 cache to another).
Review Comment:
Hmm, in real life the producer can have its own threads as well, since it's probably supposed to handle multiple in-flight consumer requests. Why would decoding be offloaded to consumer threads? This introduces a layer of complexity that will be difficult to make efficient.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-43631: [C][Format] Add ArrowAsyncDeviceStreamHandler interface [arrow]
Posted by "zeroshade (via GitHub)" <gi...@apache.org>.
zeroshade commented on code in PR #43632:
URL: https://github.com/apache/arrow/pull/43632#discussion_r1759128647
##########
docs/source/format/CDeviceDataInterface.rst:
##########
@@ -650,6 +652,331 @@ The stream source is not assumed to be thread-safe. Consumers wanting to
call ``get_next`` from several threads should ensure those calls are
serialized.
+Async Device Stream Interface
+=============================
+
+The :ref:`C stream interface <_c-device-stream-interface>` provides a synchronous
+API centered around the consumer calling the callback functions to retrieve
+the next record batch. For some bindings, use cases, and interoperability, a more
+asynchronous, producer-focused interface may be required. These scenarios can utilize
+the ``ArrowAsyncDeviceStreamHandler``.
+
+Semantics
+---------
+
+Rather than the producer providing a structure of callbacks for a consumer to
+call and retrieve records, the Async interface is a consumer allocated structure.
+The consumer allocated struct provides handler callbacks for the producer to call
+when the schema and chunks of data are available, rather than the consumer using
+a blocking pull-style iteration.
+
+In addition to the ``ArrowAsyncDeviceStreamHandler``, there are also two additional
+structs used for the full data flow: ``ArrowAsyncTask`` and ``ArrowAsyncProducer``.
+
+Structure Definition
+--------------------
+
+The C device async stream interface is defined with a single ``struct`` definition:
+
+.. code-block:: c
+
+ #ifndef ARROW_C_ASYNC_STREAM_INTERFACE
+ #define ARROW_C_ASYNC_STREAM_INTERFACE
+
+ struct ArrowAsyncTask {
+ int (*get_data)(struct ArrowArrayTask* self, struct ArrowDeviceArray* out);
+
+ void* private_data;
+ };
+
+ struct ArrowAsyncProducer {
+ void (*request)(struct ArrowAsyncProducer* self, uint64_t n);
+ void (*cancel)(struct ArrowAsyncProducer* self);
+
+ void (*release)(struct ArrowAsyncProducer* self);
+ void* private_data;
+ };
+
+ struct ArrowAsyncDeviceStreamHandler {
+ // handlers
+ int (*on_schema)(struct ArrowAsyncDeviceStreamHandler* self,
+ struct ArrowAsyncProducer* producer,
+ struct ArrowSchema* stream_schema, const char* addl_metadata);
+ int (*on_next_task)(struct ArrowAsyncDeviceStreamHandler* self,
+ struct ArrowAsyncTask* task, const char* metadata);
+ void (*on_error)(struct ArrowAsyncDeviceStreamHandler* self,
+ int code, const char* message, const char* metadata);
+
+ // release callback
+ void (*release)(struct ArrowAsyncDeviceStreamHandler* self);
+
+ // opaque handler-specific data
+ void* private_data;
+ };
+
+ #endif // ARROW_C_ASYNC_STREAM_INTERFACE
+
+.. note::
+ The canonical guard ``ARROW_C_ASYNC_STREAM_INTERFACE`` is meant to avoid
+ duplicate definitions if two projects copy the C async stream interface
+ definitions into their own headers, and a third-party project includes
+ from these two projects. It is therefore important that this guard is kept
+ exactly as-is when these definitions are copied.
+
+The ArrowAsyncDeviceStreamHandler structure
+'''''''''''''''''''''''''''''''''''''''''''
+
+The structure has the following fields:
+
+.. c:member:: int (*ArrowAsyncDeviceStreamHandler.on_schema)(struct ArrowAsyncDeviceStreamHandler*, struct ArrowAsyncProducer*, struct ArrowSchema*, const char*)
+
+ *Mandatory.* Handler for receiving the schema of the stream. All records should
+ match the provided schema. If successful, the function should return 0, otherwise
+ it should return an ``errno``-compatible error code.
+
+ The ``const char*`` parameter exists for producers to provide any extra contextual information
+ they want, such as the total number of rows in the stream, statistics, or otherwise. This is
+ encoded in the same format as :c:member:`ArrowSchema.metadata`. If not ``NULL``,
+ the lifetime is only the scope of the call to this function. A consumer who wants to maintain
+ the additional metadata beyond the lifetime of this call *MUST* copy the value themselves.
+
+ Unless the ``on_error`` handler is called, this will always get called exactly once and will be
+ the first method called on this object. As such the producer *MUST* provide an ``ArrowAsyncProducer``
+ object when calling this function to allow the consumer to manage back-pressure and flow control.
+ The producer maintains ownership of the ``ArrowAsyncProducer`` and must clean it up before or after
+ calling the release callback on this object.
+
+ A producer that receives a non-zero result here must not subsequently call anything other than
+ the release callback on this object.
+
+.. c:member:: int (*ArrowAsyncDeviceStreamHandler.on_next_task)(struct ArrowAsyncDeviceStreamHandler*, struct ArrowAsyncTask*, const char*)
+
+ *Mandatory.* Handler to be called when a new record is available for processing. The
+ schema for each record should be the same as the schema that ``on_schema`` was called with.
+ If successfully handled, the function should return 0, otherwise it should return an
+ ``errno``-compatible error code.
+
+ Rather than passing the record itself it receives an ``ArrowAsyncTask`` instead to facilitate
+ better consumer-focused thread control as far as receiving the data. A call to this function
+ simply indicates that data is available via the provided task.
+
+ The ``const char*`` parameter exists for producers to provide any extra contextual information
+ they want. This is encoded in the same format as :c:member:`ArrowSchema.metadata`. If not ``NULL``,
+ the lifetime is only the scope of the call to this function. A consumer who wants to maintain
+ the additional metadata beyond the lifetime of this call *MUST* copy the value themselves.
+
+ A producer *MUST NOT* call this concurrently from multiple different threads.
+
+ The :c:member:`ArrowAsyncProducer.request` callback must be called to start receiving calls to this
+ handler.
+
+ This function *MUST* be able to be called re-entrantly on the same thread to allow for the
+ :c:member:`ArrowAsyncProducer.request` callback to potentially call this method recursively.
+
+.. c:member:: void (*ArrowAsyncDeviceStreamHandler.on_error)(struct ArrowAsyncDeviceStreamHandler, int, const char*, const char*)
+
+ *Mandatory.* Handler to be called when an error is encountered by the producer. After calling
+ this, the ``release`` callback will be called as the last call on this struct. The parameters
+ are an ``errno``-compatible error code and an optional error message and metadata.
+
+ If the message and metadata are not ``NULL``, their lifetime is only valid during the scope
+ of this call. A consumer who wants to maintain these values past the return of this function
+ *MUST* copy the values themselves.
+
+ If the metadata parameter is not ``NULL``, to provide key-value error metadata, then it should
+ be encoded identically to the way that metadata is encoded in :c:member:`ArrowSchema.metadata`.
+
+ It is valid for this to be called by a producer with or without a preceding call to
+ :c:member:`ArrowAsyncProducer.request`. This callback *MUST NOT* call any methods of an
+ ``ArrowAsyncProducer`` object.
+
+.. c:member:: void (*ArrowAsyncDeviceStreamHandler.release)(struct ArrowAsyncDeviceStreamHandler*)
+
+ *Mandatory.* A pointer to a consumer-provided release callback for the handler.
+
+ It is valid for this to be called by a producer with or without a preceding call to
+ :c:member:`ArrowAsyncProducer.request`. This must not call any methods of an ``ArrowAsyncProducer``
+ object.
+
+.. c:member:: void* ArrowAsyncDeviceStreamHandler.private_data
+
+ *Optional.* An opaque pointer to consumer-provided private data.
+
+ Producers *MUST NOT* process this member. Lifetime of this member is handled by
+ the consumer, and especially by the release callback.
+
+The ArrowAsyncTask structure
+''''''''''''''''''''''''''''
+
+The purpose of using a Task object rather than passing the array directly to the ``on_next``
+callback is to allow for more complex and efficient thread handling. Utilizing a Task
+object allows for a producer to separate the "decoding" logic from the I/O, enabling a
+consumer to avoid transferring data between CPU cores (e.g. from one L1/L2 cache to another).
+
+This producer-provided structure has the following fields:
+
+.. c:member:: int (*ArrowArrayTask.get_data)(struct ArrowArrayTask*, struct ArrowDeviceArray*)
+
+ *Mandatory.* A callback to populate the provided ``ArrowDeviceArray`` with the available data.
+ The order of ArrowAsyncTasks provided by the producer enables a consumer to know the order of
+ the data to process.
+
+ If a non-zero value is returned from this, it should be followed only by the producer calling
+ the ``on_error`` callback of the ``ArrowAsyncDeviceStreamHandler``. Because calling this method
+ is likely to be separate from the current control flow, returning a non-zero value to signal
+ an error occuring allows the current thread to decide handle the case accordingly, while still
+ allowing all error logging and handling to be centralized in the
+ :c:member:`ArrowAsyncDeviceStreamHandler.on_error` callback.
+
+ Rather than having a separate release callback, any required cleanup should be performed as part
+ of the invocation of this callback, with ownership of the Array being passed to the consumer-provided
+ parameter calling this, and must be released separately.
+
+ It is only valid to call this method exactly once.
+
+.. c:member:: void* ArrowArrayTask.private_data
+
+ *Optional.* An opaque pointer to consumer-provided private data.
+
+ Consumers *MUST NOT* process this member. Lifetime of this member is handled by
+ the producer who created this object, and should be cleaned up if necessary during
+ the call to :c:member:`ArrowArrayTask.get_data`.
+
+The ArrowAsyncProducer structure
+''''''''''''''''''''''''''''''''
+
+This producer-provided and managed object has the following fields:
+
+.. c:member:: void (*ArrowAsyncProducer.request)(struct ArrowAsyncProducer*, uint64_t)
+
+ *Mandatory.* This function must be called by a consumer to start receiving calls to
+ :c:member:`ArrowAsyncDeviceStreamHandler.on_next_task`. It *MUST* be valid to call
+ this synchronously from within :c:member:`ArrowAsyncDeviceStreamHandler.on_next_task`
+ or :c:member:`ArrowAsyncDeviceStreamHandler.on_schema`. As a result, this function
+ *MUST NOT* synchronously call ``on_next_task`` or ``on_error`` to avoid recursive
+ and reentrant callbacks.
+
+ After ``cancel`` is called, additional calls to this function must be a NOP, but allowed.
+
+ While not cancelled, calling this function registers the given number of additional
+ arrays/batches to be produced by the producer. A producer should only call
+ the appropriate ``on_next_task`` callback up to a maximum of the total sum of calls to
+ this method before propagating back-pressure / waiting.
+
+ Any error encountered by calling request must be propagated by calling the ``on_error``
+ callback of the ``ArrowAsyncDeviceStreamHandler``.
+
+ A producer *MUST* support an unbounded number of calls to this callback and *MUST* support
+ a total registered demand (sum requested - sum delivered) of up to ``UINT64_MAX``.
+
+.. c:member:: void (*ArrowAsyncProducer.cancel)(struct ArrowAsyncProducer*)
+
+ *Mandatory.* This function signals to the producer that it must *eventually* stop calling
+ ``on_next_task``. Calls to ``cancel`` must be idempotent and thread-safe. After calling
+ it once, subsequent calls *MUST* be a NOP.
+
+ It is not required that calling ``cancel`` affect the producer *immediately*, only that it
+ must eventually stop calling ``on_next_task`` and then subsequently call ``release``
+ on the async handler object. As such, a consumer *MUST* be prepared to receive one or more
+ calls to ``on_next_task`` or ``on_error`` even after calling ``cancel`` if there are still
+ requested arrays pending.
+
+ Successful cancelling *MUST NOT* result in a producer calling
+ :c:member:`ArrowAsyncDeviceStreamHandler.on_error`, instead it should finish out any remaining
+ tasks (calling ``on_next_task`` accordingly) and eventually just call ``release``.
+
+ Any error encountered during handling a call to cancel must be reported via the ``on_error``
+ callback on the async stream handler.
+
+.. c:member:: void* ArrowAsyncProducer.private_data
+
+ *Optional.* An opaque pointer to producer-provided specific data.
+
+ Consumers *MUST NOT* process this member, the lifetime is owned by the producer
+ that constructed this object.
+
+Error Handling
+''''''''''''''
+
+Unlike the regular C Stream interface, the Async interface allows for errors to flow in
+both directions. As a result, error handling can be slightly more complex. Thus this spec
+designates the following rules:
+
+* If the producer encounters an error during processing, it should call the ``on_error``
+callback, and then call ``release`` after it returns.
+
+* If ``on_schema`` or ``on_next_task`` returns a non-zero integer value, the producer *should not*
+call the ``on_error`` callback, but instead should eventually call ``release`` at some point
+before or after any logging or processing of the error code.
+
+Result lifetimes
+''''''''''''''''
+
+The ``ArrowSchema`` passed to the ``on_schema`` callback must be released independantly,
+with the object itself needing to be moved to a consumer owned ``ArrowSchema`` object. The
+``ArrowSchema*`` passed as a parameter to the callback *MUST NOT* be stored and kept.
+
+The ``ArrowAsyncTask`` object provided to ``on_next_task`` is owned by the producer and
+will be cleaned up during the invocation of calling ``get_data`` on it. If the consumer
+doesn't care about the data, it should pass ``NULL`` instead of a valid ``ArrowDeviceArray*``.
+
+The ``const char*`` error ``message`` and ``metadata`` which are passed to ``on_error``
+are only valid within the scope of the ``on_error`` function itself. They must be copied
+if it is necessary for them to exist after it returns.
+
+Stream Handler Lifetime
+'''''''''''''''''''''''
+
+Lifetime of the async stream handler is managed using a release callback with similar
+usage as in :ref:`C data interface <c-data-interface-released>`.
+
+ArrowAsyncProducer Lifetime
+'''''''''''''''''''''''''''
+
+The lifetime of the ``ArrowAsyncProducer`` passed to ``on_schema`` is owned by the producer
+itself and should be managed by it. It *MUST* remain valid at least until just before
+calling ``release`` on the stream handler object.
+
+Thread safety
+'''''''''''''
+
+All handler functions should only be called in a serialized manner, but are not guaranteed
+to be called from the same thread every time. A producer should wait for handler callbacks to
+return before calling the next handler callback, and before calling the ``release`` callback.
Review Comment:
Fixed it with the previous updates. The `on_next_task` doc no longer says it should be reentrant.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-43631: [C][Format] Add ArrowAsyncDeviceStreamHandler interface [arrow]
Posted by "pitrou (via GitHub)" <gi...@apache.org>.
pitrou commented on code in PR #43632:
URL: https://github.com/apache/arrow/pull/43632#discussion_r1757468639
##########
docs/source/format/CDeviceDataInterface.rst:
##########
@@ -650,6 +652,331 @@ The stream source is not assumed to be thread-safe. Consumers wanting to
call ``get_next`` from several threads should ensure those calls are
serialized.
+Async Device Stream Interface
+=============================
+
+The :ref:`C stream interface <_c-device-stream-interface>` provides a synchronous
+API centered around the consumer calling the callback functions to retrieve
+the next record batch. For some bindings, use cases, and interoperability, a more
+asynchronous, producer-focused interface may be required. These scenarios can utilize
+the ``ArrowAsyncDeviceStreamHandler``.
+
+Semantics
+---------
+
+Rather than the producer providing a structure of callbacks for a consumer to
+call and retrieve records, the Async interface is a consumer allocated structure.
+The consumer allocated struct provides handler callbacks for the producer to call
+when the schema and chunks of data are available, rather than the consumer using
+a blocking pull-style iteration.
+
+In addition to the ``ArrowAsyncDeviceStreamHandler``, there are also two additional
+structs used for the full data flow: ``ArrowAsyncTask`` and ``ArrowAsyncProducer``.
+
+Structure Definition
+--------------------
+
+The C device async stream interface is defined with a single ``struct`` definition:
+
+.. code-block:: c
+
+ #ifndef ARROW_C_ASYNC_STREAM_INTERFACE
+ #define ARROW_C_ASYNC_STREAM_INTERFACE
+
+ struct ArrowAsyncTask {
+ int (*get_data)(struct ArrowArrayTask* self, struct ArrowDeviceArray* out);
+
+ void* private_data;
+ };
+
+ struct ArrowAsyncProducer {
+ void (*request)(struct ArrowAsyncProducer* self, uint64_t n);
+ void (*cancel)(struct ArrowAsyncProducer* self);
+
+ void (*release)(struct ArrowAsyncProducer* self);
+ void* private_data;
+ };
+
+ struct ArrowAsyncDeviceStreamHandler {
+ // handlers
+ int (*on_schema)(struct ArrowAsyncDeviceStreamHandler* self,
+ struct ArrowAsyncProducer* producer,
+ struct ArrowSchema* stream_schema, const char* addl_metadata);
+ int (*on_next_task)(struct ArrowAsyncDeviceStreamHandler* self,
+ struct ArrowAsyncTask* task, const char* metadata);
+ void (*on_error)(struct ArrowAsyncDeviceStreamHandler* self,
+ int code, const char* message, const char* metadata);
+
+ // release callback
+ void (*release)(struct ArrowAsyncDeviceStreamHandler* self);
+
+ // opaque handler-specific data
+ void* private_data;
+ };
+
+ #endif // ARROW_C_ASYNC_STREAM_INTERFACE
+
+.. note::
+ The canonical guard ``ARROW_C_ASYNC_STREAM_INTERFACE`` is meant to avoid
+ duplicate definitions if two projects copy the C async stream interface
+ definitions into their own headers, and a third-party project includes
+ from these two projects. It is therefore important that this guard is kept
+ exactly as-is when these definitions are copied.
+
+The ArrowAsyncDeviceStreamHandler structure
+'''''''''''''''''''''''''''''''''''''''''''
+
+The structure has the following fields:
+
+.. c:member:: int (*ArrowAsyncDeviceStreamHandler.on_schema)(struct ArrowAsyncDeviceStreamHandler*, struct ArrowAsyncProducer*, struct ArrowSchema*, const char*)
+
+ *Mandatory.* Handler for receiving the schema of the stream. All records should
+ match the provided schema. If successful, the function should return 0, otherwise
+ it should return an ``errno``-compatible error code.
+
+ The ``const char*`` parameter exists for producers to provide any extra contextual information
+ they want, such as the total number of rows in the stream, statistics, or otherwise. This is
+ encoded in the same format as :c:member:`ArrowSchema.metadata`. If not ``NULL``,
+ the lifetime is only the scope of the call to this function. A consumer who wants to maintain
+ the additional metadata beyond the lifetime of this call *MUST* copy the value themselves.
+
+ Unless the ``on_error`` handler is called, this will always get called exactly once and will be
+ the first method called on this object. As such the producer *MUST* provide an ``ArrowAsyncProducer``
+ object when calling this function to allow the consumer to manage back-pressure and flow control.
+ The producer maintains ownership of the ``ArrowAsyncProducer`` and must clean it up before or after
+ calling the release callback on this object.
Review Comment:
Before or after? We should perhaps be authoritative here. After sounds safer.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-43631: [C][Format] Add ArrowAsyncDeviceStreamHandler interface [arrow]
Posted by "zeroshade (via GitHub)" <gi...@apache.org>.
zeroshade commented on code in PR #43632:
URL: https://github.com/apache/arrow/pull/43632#discussion_r1759024918
##########
docs/source/format/CDeviceDataInterface.rst:
##########
@@ -650,6 +652,331 @@ The stream source is not assumed to be thread-safe. Consumers wanting to
call ``get_next`` from several threads should ensure those calls are
serialized.
+Async Device Stream Interface
+=============================
+
+The :ref:`C stream interface <_c-device-stream-interface>` provides a synchronous
+API centered around the consumer calling the callback functions to retrieve
+the next record batch. For some bindings, use cases, and interoperability, a more
+asynchronous, producer-focused interface may be required. These scenarios can utilize
+the ``ArrowAsyncDeviceStreamHandler``.
+
+Semantics
+---------
+
+Rather than the producer providing a structure of callbacks for a consumer to
+call and retrieve records, the Async interface is a consumer allocated structure.
+The consumer allocated struct provides handler callbacks for the producer to call
+when the schema and chunks of data are available, rather than the consumer using
+a blocking pull-style iteration.
+
+In addition to the ``ArrowAsyncDeviceStreamHandler``, there are also two additional
+structs used for the full data flow: ``ArrowAsyncTask`` and ``ArrowAsyncProducer``.
+
+Structure Definition
+--------------------
+
+The C device async stream interface is defined with a single ``struct`` definition:
+
+.. code-block:: c
+
+ #ifndef ARROW_C_ASYNC_STREAM_INTERFACE
+ #define ARROW_C_ASYNC_STREAM_INTERFACE
+
+ struct ArrowAsyncTask {
+ int (*get_data)(struct ArrowArrayTask* self, struct ArrowDeviceArray* out);
+
+ void* private_data;
+ };
+
+ struct ArrowAsyncProducer {
+ void (*request)(struct ArrowAsyncProducer* self, uint64_t n);
+ void (*cancel)(struct ArrowAsyncProducer* self);
+
+ void (*release)(struct ArrowAsyncProducer* self);
+ void* private_data;
+ };
+
+ struct ArrowAsyncDeviceStreamHandler {
+ // handlers
+ int (*on_schema)(struct ArrowAsyncDeviceStreamHandler* self,
+ struct ArrowAsyncProducer* producer,
+ struct ArrowSchema* stream_schema, const char* addl_metadata);
+ int (*on_next_task)(struct ArrowAsyncDeviceStreamHandler* self,
+ struct ArrowAsyncTask* task, const char* metadata);
+ void (*on_error)(struct ArrowAsyncDeviceStreamHandler* self,
+ int code, const char* message, const char* metadata);
+
+ // release callback
+ void (*release)(struct ArrowAsyncDeviceStreamHandler* self);
+
+ // opaque handler-specific data
+ void* private_data;
+ };
+
+ #endif // ARROW_C_ASYNC_STREAM_INTERFACE
+
+.. note::
+ The canonical guard ``ARROW_C_ASYNC_STREAM_INTERFACE`` is meant to avoid
+ duplicate definitions if two projects copy the C async stream interface
+ definitions into their own headers, and a third-party project includes
+ from these two projects. It is therefore important that this guard is kept
+ exactly as-is when these definitions are copied.
+
+The ArrowAsyncDeviceStreamHandler structure
+'''''''''''''''''''''''''''''''''''''''''''
+
+The structure has the following fields:
+
+.. c:member:: int (*ArrowAsyncDeviceStreamHandler.on_schema)(struct ArrowAsyncDeviceStreamHandler*, struct ArrowAsyncProducer*, struct ArrowSchema*, const char*)
+
+ *Mandatory.* Handler for receiving the schema of the stream. All records should
+ match the provided schema. If successful, the function should return 0, otherwise
+ it should return an ``errno``-compatible error code.
+
+ The ``const char*`` parameter exists for producers to provide any extra contextual information
+ they want, such as the total number of rows in the stream, statistics, or otherwise. This is
+ encoded in the same format as :c:member:`ArrowSchema.metadata`. If not ``NULL``,
+ the lifetime is only the scope of the call to this function. A consumer who wants to maintain
+ the additional metadata beyond the lifetime of this call *MUST* copy the value themselves.
+
+ Unless the ``on_error`` handler is called, this will always get called exactly once and will be
+ the first method called on this object. As such the producer *MUST* provide an ``ArrowAsyncProducer``
+ object when calling this function to allow the consumer to manage back-pressure and flow control.
+ The producer maintains ownership of the ``ArrowAsyncProducer`` and must clean it up before or after
+ calling the release callback on this object.
+
+ A producer that receives a non-zero result here must not subsequently call anything other than
+ the release callback on this object.
+
+.. c:member:: int (*ArrowAsyncDeviceStreamHandler.on_next_task)(struct ArrowAsyncDeviceStreamHandler*, struct ArrowAsyncTask*, const char*)
+
+ *Mandatory.* Handler to be called when a new record is available for processing. The
+ schema for each record should be the same as the schema that ``on_schema`` was called with.
+ If successfully handled, the function should return 0, otherwise it should return an
+ ``errno``-compatible error code.
+
+ Rather than passing the record itself it receives an ``ArrowAsyncTask`` instead to facilitate
+ better consumer-focused thread control as far as receiving the data. A call to this function
+ simply indicates that data is available via the provided task.
+
+ The ``const char*`` parameter exists for producers to provide any extra contextual information
+ they want. This is encoded in the same format as :c:member:`ArrowSchema.metadata`. If not ``NULL``,
+ the lifetime is only the scope of the call to this function. A consumer who wants to maintain
+ the additional metadata beyond the lifetime of this call *MUST* copy the value themselves.
+
+ A producer *MUST NOT* call this concurrently from multiple different threads.
+
+ The :c:member:`ArrowAsyncProducer.request` callback must be called to start receiving calls to this
+ handler.
+
+ This function *MUST* be able to be called re-entrantly on the same thread to allow for the
+ :c:member:`ArrowAsyncProducer.request` callback to potentially call this method recursively.
+
+.. c:member:: void (*ArrowAsyncDeviceStreamHandler.on_error)(struct ArrowAsyncDeviceStreamHandler, int, const char*, const char*)
+
+ *Mandatory.* Handler to be called when an error is encountered by the producer. After calling
+ this, the ``release`` callback will be called as the last call on this struct. The parameters
+ are an ``errno``-compatible error code and an optional error message and metadata.
+
+ If the message and metadata are not ``NULL``, their lifetime is only valid during the scope
+ of this call. A consumer who wants to maintain these values past the return of this function
+ *MUST* copy the values themselves.
+
+ If the metadata parameter is not ``NULL``, to provide key-value error metadata, then it should
+ be encoded identically to the way that metadata is encoded in :c:member:`ArrowSchema.metadata`.
+
+ It is valid for this to be called by a producer with or without a preceding call to
+ :c:member:`ArrowAsyncProducer.request`. This callback *MUST NOT* call any methods of an
+ ``ArrowAsyncProducer`` object.
+
+.. c:member:: void (*ArrowAsyncDeviceStreamHandler.release)(struct ArrowAsyncDeviceStreamHandler*)
+
+ *Mandatory.* A pointer to a consumer-provided release callback for the handler.
+
+ It is valid for this to be called by a producer with or without a preceding call to
+ :c:member:`ArrowAsyncProducer.request`. This must not call any methods of an ``ArrowAsyncProducer``
+ object.
+
+.. c:member:: void* ArrowAsyncDeviceStreamHandler.private_data
+
+ *Optional.* An opaque pointer to consumer-provided private data.
+
+ Producers *MUST NOT* process this member. Lifetime of this member is handled by
+ the consumer, and especially by the release callback.
+
+The ArrowAsyncTask structure
+''''''''''''''''''''''''''''
+
+The purpose of using a Task object rather than passing the array directly to the ``on_next``
+callback is to allow for more complex and efficient thread handling. Utilizing a Task
+object allows for a producer to separate the "decoding" logic from the I/O, enabling a
+consumer to avoid transferring data between CPU cores (e.g. from one L1/L2 cache to another).
+
+This producer-provided structure has the following fields:
+
+.. c:member:: int (*ArrowArrayTask.get_data)(struct ArrowArrayTask*, struct ArrowDeviceArray*)
+
+ *Mandatory.* A callback to populate the provided ``ArrowDeviceArray`` with the available data.
+ The order of ArrowAsyncTasks provided by the producer enables a consumer to know the order of
+ the data to process.
+
+ If a non-zero value is returned from this, it should be followed only by the producer calling
+ the ``on_error`` callback of the ``ArrowAsyncDeviceStreamHandler``. Because calling this method
+ is likely to be separate from the current control flow, returning a non-zero value to signal
+ an error occuring allows the current thread to decide handle the case accordingly, while still
+ allowing all error logging and handling to be centralized in the
+ :c:member:`ArrowAsyncDeviceStreamHandler.on_error` callback.
+
+ Rather than having a separate release callback, any required cleanup should be performed as part
+ of the invocation of this callback, with ownership of the Array being passed to the consumer-provided
+ parameter calling this, and must be released separately.
+
+ It is only valid to call this method exactly once.
+
+.. c:member:: void* ArrowArrayTask.private_data
+
+ *Optional.* An opaque pointer to consumer-provided private data.
+
+ Consumers *MUST NOT* process this member. Lifetime of this member is handled by
+ the producer who created this object, and should be cleaned up if necessary during
+ the call to :c:member:`ArrowArrayTask.get_data`.
+
+The ArrowAsyncProducer structure
+''''''''''''''''''''''''''''''''
+
+This producer-provided and managed object has the following fields:
+
+.. c:member:: void (*ArrowAsyncProducer.request)(struct ArrowAsyncProducer*, uint64_t)
+
+ *Mandatory.* This function must be called by a consumer to start receiving calls to
+ :c:member:`ArrowAsyncDeviceStreamHandler.on_next_task`. It *MUST* be valid to call
+ this synchronously from within :c:member:`ArrowAsyncDeviceStreamHandler.on_next_task`
+ or :c:member:`ArrowAsyncDeviceStreamHandler.on_schema`. As a result, this function
+ *MUST NOT* synchronously call ``on_next_task`` or ``on_error`` to avoid recursive
+ and reentrant callbacks.
Review Comment:
Looks like I missed a spot when I updated these, I'll fix that
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-43631: [C][Format] Add ArrowAsyncDeviceStreamHandler interface [arrow]
Posted by "zeroshade (via GitHub)" <gi...@apache.org>.
zeroshade commented on code in PR #43632:
URL: https://github.com/apache/arrow/pull/43632#discussion_r1759004657
##########
docs/source/format/CDeviceDataInterface.rst:
##########
@@ -650,6 +652,331 @@ The stream source is not assumed to be thread-safe. Consumers wanting to
call ``get_next`` from several threads should ensure those calls are
serialized.
+Async Device Stream Interface
+=============================
+
+The :ref:`C stream interface <_c-device-stream-interface>` provides a synchronous
+API centered around the consumer calling the callback functions to retrieve
+the next record batch. For some bindings, use cases, and interoperability, a more
+asynchronous, producer-focused interface may be required. These scenarios can utilize
+the ``ArrowAsyncDeviceStreamHandler``.
+
+Semantics
+---------
+
+Rather than the producer providing a structure of callbacks for a consumer to
+call and retrieve records, the Async interface is a consumer allocated structure.
+The consumer allocated struct provides handler callbacks for the producer to call
+when the schema and chunks of data are available, rather than the consumer using
+a blocking pull-style iteration.
+
+In addition to the ``ArrowAsyncDeviceStreamHandler``, there are also two additional
+structs used for the full data flow: ``ArrowAsyncTask`` and ``ArrowAsyncProducer``.
+
+Structure Definition
+--------------------
+
+The C device async stream interface is defined with a single ``struct`` definition:
+
+.. code-block:: c
+
+ #ifndef ARROW_C_ASYNC_STREAM_INTERFACE
+ #define ARROW_C_ASYNC_STREAM_INTERFACE
+
+ struct ArrowAsyncTask {
+ int (*get_data)(struct ArrowArrayTask* self, struct ArrowDeviceArray* out);
+
+ void* private_data;
+ };
+
+ struct ArrowAsyncProducer {
+ void (*request)(struct ArrowAsyncProducer* self, uint64_t n);
+ void (*cancel)(struct ArrowAsyncProducer* self);
+
+ void (*release)(struct ArrowAsyncProducer* self);
+ void* private_data;
+ };
+
+ struct ArrowAsyncDeviceStreamHandler {
+ // handlers
+ int (*on_schema)(struct ArrowAsyncDeviceStreamHandler* self,
+ struct ArrowAsyncProducer* producer,
+ struct ArrowSchema* stream_schema, const char* addl_metadata);
+ int (*on_next_task)(struct ArrowAsyncDeviceStreamHandler* self,
+ struct ArrowAsyncTask* task, const char* metadata);
+ void (*on_error)(struct ArrowAsyncDeviceStreamHandler* self,
+ int code, const char* message, const char* metadata);
+
+ // release callback
+ void (*release)(struct ArrowAsyncDeviceStreamHandler* self);
+
+ // opaque handler-specific data
+ void* private_data;
+ };
+
+ #endif // ARROW_C_ASYNC_STREAM_INTERFACE
+
+.. note::
+ The canonical guard ``ARROW_C_ASYNC_STREAM_INTERFACE`` is meant to avoid
+ duplicate definitions if two projects copy the C async stream interface
+ definitions into their own headers, and a third-party project includes
+ from these two projects. It is therefore important that this guard is kept
+ exactly as-is when these definitions are copied.
+
+The ArrowAsyncDeviceStreamHandler structure
+'''''''''''''''''''''''''''''''''''''''''''
+
+The structure has the following fields:
+
+.. c:member:: int (*ArrowAsyncDeviceStreamHandler.on_schema)(struct ArrowAsyncDeviceStreamHandler*, struct ArrowAsyncProducer*, struct ArrowSchema*, const char*)
+
+ *Mandatory.* Handler for receiving the schema of the stream. All records should
+ match the provided schema. If successful, the function should return 0, otherwise
+ it should return an ``errno``-compatible error code.
+
+ The ``const char*`` parameter exists for producers to provide any extra contextual information
+ they want, such as the total number of rows in the stream, statistics, or otherwise. This is
+ encoded in the same format as :c:member:`ArrowSchema.metadata`. If not ``NULL``,
+ the lifetime is only the scope of the call to this function. A consumer who wants to maintain
+ the additional metadata beyond the lifetime of this call *MUST* copy the value themselves.
+
+ Unless the ``on_error`` handler is called, this will always get called exactly once and will be
+ the first method called on this object. As such the producer *MUST* provide an ``ArrowAsyncProducer``
+ object when calling this function to allow the consumer to manage back-pressure and flow control.
+ The producer maintains ownership of the ``ArrowAsyncProducer`` and must clean it up before or after
+ calling the release callback on this object.
+
+ A producer that receives a non-zero result here must not subsequently call anything other than
+ the release callback on this object.
+
+.. c:member:: int (*ArrowAsyncDeviceStreamHandler.on_next_task)(struct ArrowAsyncDeviceStreamHandler*, struct ArrowAsyncTask*, const char*)
Review Comment:
Because we are passing an `ArrowAsyncTask` not passing an array directly for the reasons documented. It would be inaccurate to call it `on_next_array` but I'm open to other names.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-43631: [C][Format] Add ArrowAsyncDeviceStreamHandler interface [arrow]
Posted by "zeroshade (via GitHub)" <gi...@apache.org>.
zeroshade commented on code in PR #43632:
URL: https://github.com/apache/arrow/pull/43632#discussion_r1759061087
##########
docs/source/format/CDeviceDataInterface.rst:
##########
@@ -650,6 +652,331 @@ The stream source is not assumed to be thread-safe. Consumers wanting to
call ``get_next`` from several threads should ensure those calls are
serialized.
+Async Device Stream Interface
+=============================
+
+The :ref:`C stream interface <_c-device-stream-interface>` provides a synchronous
+API centered around the consumer calling the callback functions to retrieve
+the next record batch. For some bindings, use cases, and interoperability, a more
+asynchronous, producer-focused interface may be required. These scenarios can utilize
+the ``ArrowAsyncDeviceStreamHandler``.
+
+Semantics
+---------
+
+Rather than the producer providing a structure of callbacks for a consumer to
+call and retrieve records, the Async interface is a consumer allocated structure.
+The consumer allocated struct provides handler callbacks for the producer to call
+when the schema and chunks of data are available, rather than the consumer using
+a blocking pull-style iteration.
+
+In addition to the ``ArrowAsyncDeviceStreamHandler``, there are also two additional
+structs used for the full data flow: ``ArrowAsyncTask`` and ``ArrowAsyncProducer``.
+
+Structure Definition
+--------------------
+
+The C device async stream interface is defined with a single ``struct`` definition:
+
+.. code-block:: c
+
+ #ifndef ARROW_C_ASYNC_STREAM_INTERFACE
+ #define ARROW_C_ASYNC_STREAM_INTERFACE
+
+ struct ArrowAsyncTask {
+ int (*get_data)(struct ArrowArrayTask* self, struct ArrowDeviceArray* out);
+
+ void* private_data;
+ };
+
+ struct ArrowAsyncProducer {
+ void (*request)(struct ArrowAsyncProducer* self, uint64_t n);
+ void (*cancel)(struct ArrowAsyncProducer* self);
+
+ void (*release)(struct ArrowAsyncProducer* self);
+ void* private_data;
+ };
+
+ struct ArrowAsyncDeviceStreamHandler {
+ // handlers
+ int (*on_schema)(struct ArrowAsyncDeviceStreamHandler* self,
+ struct ArrowAsyncProducer* producer,
+ struct ArrowSchema* stream_schema, const char* addl_metadata);
+ int (*on_next_task)(struct ArrowAsyncDeviceStreamHandler* self,
+ struct ArrowAsyncTask* task, const char* metadata);
+ void (*on_error)(struct ArrowAsyncDeviceStreamHandler* self,
+ int code, const char* message, const char* metadata);
+
+ // release callback
+ void (*release)(struct ArrowAsyncDeviceStreamHandler* self);
+
+ // opaque handler-specific data
+ void* private_data;
+ };
+
+ #endif // ARROW_C_ASYNC_STREAM_INTERFACE
+
+.. note::
+ The canonical guard ``ARROW_C_ASYNC_STREAM_INTERFACE`` is meant to avoid
+ duplicate definitions if two projects copy the C async stream interface
+ definitions into their own headers, and a third-party project includes
+ from these two projects. It is therefore important that this guard is kept
+ exactly as-is when these definitions are copied.
+
+The ArrowAsyncDeviceStreamHandler structure
+'''''''''''''''''''''''''''''''''''''''''''
+
+The structure has the following fields:
+
+.. c:member:: int (*ArrowAsyncDeviceStreamHandler.on_schema)(struct ArrowAsyncDeviceStreamHandler*, struct ArrowAsyncProducer*, struct ArrowSchema*, const char*)
+
+ *Mandatory.* Handler for receiving the schema of the stream. All records should
+ match the provided schema. If successful, the function should return 0, otherwise
+ it should return an ``errno``-compatible error code.
+
+ The ``const char*`` parameter exists for producers to provide any extra contextual information
+ they want, such as the total number of rows in the stream, statistics, or otherwise. This is
+ encoded in the same format as :c:member:`ArrowSchema.metadata`. If not ``NULL``,
+ the lifetime is only the scope of the call to this function. A consumer who wants to maintain
+ the additional metadata beyond the lifetime of this call *MUST* copy the value themselves.
+
+ Unless the ``on_error`` handler is called, this will always get called exactly once and will be
+ the first method called on this object. As such the producer *MUST* provide an ``ArrowAsyncProducer``
+ object when calling this function to allow the consumer to manage back-pressure and flow control.
+ The producer maintains ownership of the ``ArrowAsyncProducer`` and must clean it up before or after
+ calling the release callback on this object.
+
+ A producer that receives a non-zero result here must not subsequently call anything other than
+ the release callback on this object.
+
+.. c:member:: int (*ArrowAsyncDeviceStreamHandler.on_next_task)(struct ArrowAsyncDeviceStreamHandler*, struct ArrowAsyncTask*, const char*)
+
+ *Mandatory.* Handler to be called when a new record is available for processing. The
+ schema for each record should be the same as the schema that ``on_schema`` was called with.
+ If successfully handled, the function should return 0, otherwise it should return an
+ ``errno``-compatible error code.
+
+ Rather than passing the record itself it receives an ``ArrowAsyncTask`` instead to facilitate
+ better consumer-focused thread control as far as receiving the data. A call to this function
+ simply indicates that data is available via the provided task.
+
+ The ``const char*`` parameter exists for producers to provide any extra contextual information
+ they want. This is encoded in the same format as :c:member:`ArrowSchema.metadata`. If not ``NULL``,
+ the lifetime is only the scope of the call to this function. A consumer who wants to maintain
+ the additional metadata beyond the lifetime of this call *MUST* copy the value themselves.
+
+ A producer *MUST NOT* call this concurrently from multiple different threads.
+
+ The :c:member:`ArrowAsyncProducer.request` callback must be called to start receiving calls to this
+ handler.
+
+ This function *MUST* be able to be called re-entrantly on the same thread to allow for the
+ :c:member:`ArrowAsyncProducer.request` callback to potentially call this method recursively.
+
+.. c:member:: void (*ArrowAsyncDeviceStreamHandler.on_error)(struct ArrowAsyncDeviceStreamHandler, int, const char*, const char*)
+
+ *Mandatory.* Handler to be called when an error is encountered by the producer. After calling
+ this, the ``release`` callback will be called as the last call on this struct. The parameters
+ are an ``errno``-compatible error code and an optional error message and metadata.
+
+ If the message and metadata are not ``NULL``, their lifetime is only valid during the scope
+ of this call. A consumer who wants to maintain these values past the return of this function
+ *MUST* copy the values themselves.
+
+ If the metadata parameter is not ``NULL``, to provide key-value error metadata, then it should
+ be encoded identically to the way that metadata is encoded in :c:member:`ArrowSchema.metadata`.
+
+ It is valid for this to be called by a producer with or without a preceding call to
+ :c:member:`ArrowAsyncProducer.request`. This callback *MUST NOT* call any methods of an
+ ``ArrowAsyncProducer`` object.
+
+.. c:member:: void (*ArrowAsyncDeviceStreamHandler.release)(struct ArrowAsyncDeviceStreamHandler*)
+
+ *Mandatory.* A pointer to a consumer-provided release callback for the handler.
+
+ It is valid for this to be called by a producer with or without a preceding call to
+ :c:member:`ArrowAsyncProducer.request`. This must not call any methods of an ``ArrowAsyncProducer``
+ object.
+
+.. c:member:: void* ArrowAsyncDeviceStreamHandler.private_data
+
+ *Optional.* An opaque pointer to consumer-provided private data.
+
+ Producers *MUST NOT* process this member. Lifetime of this member is handled by
+ the consumer, and especially by the release callback.
+
+The ArrowAsyncTask structure
+''''''''''''''''''''''''''''
+
+The purpose of using a Task object rather than passing the array directly to the ``on_next``
+callback is to allow for more complex and efficient thread handling. Utilizing a Task
+object allows for a producer to separate the "decoding" logic from the I/O, enabling a
+consumer to avoid transferring data between CPU cores (e.g. from one L1/L2 cache to another).
+
+This producer-provided structure has the following fields:
+
+.. c:member:: int (*ArrowArrayTask.get_data)(struct ArrowArrayTask*, struct ArrowDeviceArray*)
+
+ *Mandatory.* A callback to populate the provided ``ArrowDeviceArray`` with the available data.
+ The order of ArrowAsyncTasks provided by the producer enables a consumer to know the order of
+ the data to process.
+
+ If a non-zero value is returned from this, it should be followed only by the producer calling
+ the ``on_error`` callback of the ``ArrowAsyncDeviceStreamHandler``. Because calling this method
+ is likely to be separate from the current control flow, returning a non-zero value to signal
+ an error occuring allows the current thread to decide handle the case accordingly, while still
+ allowing all error logging and handling to be centralized in the
+ :c:member:`ArrowAsyncDeviceStreamHandler.on_error` callback.
+
+ Rather than having a separate release callback, any required cleanup should be performed as part
+ of the invocation of this callback, with ownership of the Array being passed to the consumer-provided
+ parameter calling this, and must be released separately.
+
+ It is only valid to call this method exactly once.
+
+.. c:member:: void* ArrowArrayTask.private_data
+
+ *Optional.* An opaque pointer to consumer-provided private data.
+
+ Consumers *MUST NOT* process this member. Lifetime of this member is handled by
+ the producer who created this object, and should be cleaned up if necessary during
+ the call to :c:member:`ArrowArrayTask.get_data`.
+
+The ArrowAsyncProducer structure
+''''''''''''''''''''''''''''''''
+
+This producer-provided and managed object has the following fields:
+
+.. c:member:: void (*ArrowAsyncProducer.request)(struct ArrowAsyncProducer*, uint64_t)
+
+ *Mandatory.* This function must be called by a consumer to start receiving calls to
+ :c:member:`ArrowAsyncDeviceStreamHandler.on_next_task`. It *MUST* be valid to call
+ this synchronously from within :c:member:`ArrowAsyncDeviceStreamHandler.on_next_task`
+ or :c:member:`ArrowAsyncDeviceStreamHandler.on_schema`. As a result, this function
+ *MUST NOT* synchronously call ``on_next_task`` or ``on_error`` to avoid recursive
+ and reentrant callbacks.
+
+ After ``cancel`` is called, additional calls to this function must be a NOP, but allowed.
+
+ While not cancelled, calling this function registers the given number of additional
+ arrays/batches to be produced by the producer. A producer should only call
+ the appropriate ``on_next_task`` callback up to a maximum of the total sum of calls to
+ this method before propagating back-pressure / waiting.
+
+ Any error encountered by calling request must be propagated by calling the ``on_error``
+ callback of the ``ArrowAsyncDeviceStreamHandler``.
+
+ A producer *MUST* support an unbounded number of calls to this callback and *MUST* support
+ a total registered demand (sum requested - sum delivered) of up to ``UINT64_MAX``.
+
+.. c:member:: void (*ArrowAsyncProducer.cancel)(struct ArrowAsyncProducer*)
+
+ *Mandatory.* This function signals to the producer that it must *eventually* stop calling
+ ``on_next_task``. Calls to ``cancel`` must be idempotent and thread-safe. After calling
+ it once, subsequent calls *MUST* be a NOP.
+
+ It is not required that calling ``cancel`` affect the producer *immediately*, only that it
+ must eventually stop calling ``on_next_task`` and then subsequently call ``release``
+ on the async handler object. As such, a consumer *MUST* be prepared to receive one or more
+ calls to ``on_next_task`` or ``on_error`` even after calling ``cancel`` if there are still
+ requested arrays pending.
+
+ Successful cancelling *MUST NOT* result in a producer calling
+ :c:member:`ArrowAsyncDeviceStreamHandler.on_error`, instead it should finish out any remaining
+ tasks (calling ``on_next_task`` accordingly) and eventually just call ``release``.
+
+ Any error encountered during handling a call to cancel must be reported via the ``on_error``
+ callback on the async stream handler.
+
+.. c:member:: void* ArrowAsyncProducer.private_data
+
+ *Optional.* An opaque pointer to producer-provided specific data.
+
+ Consumers *MUST NOT* process this member, the lifetime is owned by the producer
+ that constructed this object.
+
+Error Handling
+''''''''''''''
+
+Unlike the regular C Stream interface, the Async interface allows for errors to flow in
+both directions. As a result, error handling can be slightly more complex. Thus this spec
+designates the following rules:
+
+* If the producer encounters an error during processing, it should call the ``on_error``
+callback, and then call ``release`` after it returns.
+
+* If ``on_schema`` or ``on_next_task`` returns a non-zero integer value, the producer *should not*
+call the ``on_error`` callback, but instead should eventually call ``release`` at some point
+before or after any logging or processing of the error code.
+
+Result lifetimes
+''''''''''''''''
+
+The ``ArrowSchema`` passed to the ``on_schema`` callback must be released independantly,
+with the object itself needing to be moved to a consumer owned ``ArrowSchema`` object. The
+``ArrowSchema*`` passed as a parameter to the callback *MUST NOT* be stored and kept.
+
+The ``ArrowAsyncTask`` object provided to ``on_next_task`` is owned by the producer and
+will be cleaned up during the invocation of calling ``get_data`` on it. If the consumer
+doesn't care about the data, it should pass ``NULL`` instead of a valid ``ArrowDeviceArray*``.
+
+The ``const char*`` error ``message`` and ``metadata`` which are passed to ``on_error``
+are only valid within the scope of the ``on_error`` function itself. They must be copied
+if it is necessary for them to exist after it returns.
+
+Stream Handler Lifetime
+'''''''''''''''''''''''
+
+Lifetime of the async stream handler is managed using a release callback with similar
+usage as in :ref:`C data interface <c-data-interface-released>`.
+
+ArrowAsyncProducer Lifetime
+'''''''''''''''''''''''''''
+
+The lifetime of the ``ArrowAsyncProducer`` passed to ``on_schema`` is owned by the producer
+itself and should be managed by it. It *MUST* remain valid at least until just before
+calling ``release`` on the stream handler object.
+
+Thread safety
+'''''''''''''
+
+All handler functions should only be called in a serialized manner, but are not guaranteed
+to be called from the same thread every time. A producer should wait for handler callbacks to
+return before calling the next handler callback, and before calling the ``release`` callback.
+As a result, back-pressure is managed by how long it takes for the ``on_next`` handler to return.
+
+The ``ArrowAsyncDeviceStreamHandler`` object should be able to handle callbacks as soon as
+it is passed to the producer, any initialization should be performed before it is provided.
+
+Possible Sequence Diagram
+-------------------------
+
+.. mermaid::
Review Comment:
You can use https://mermaid.live/edit to see a rendered version. You can also follow [this link](https://mermaid.live/edit#pako:eNqdU01vwjAM_StRToXR_oAeKqFx2GXaJHashExiICJNunxsQ4j_PqddKRsbh_nktM_Pzy_OkQsrkZfc42tEI3ChYOugqQ2juLfGxwZdXlV3z87KKNCVbO6cfZ_7gxELfFMCl8EhNA9gpEY37SsHdJ5KB5qSWbPyYocNZCPJAJ3OeuZlB5hOriT8puDcp3JpAB8yQ4V9aQuuT1Joa1sWWxYsMyyoBv3476fgb3oNfoRVAL-_kPxCx0FgCjSyP5AHbGMdQxA7lopGzHmQPyZJnMUWw0pCgKwoii_-jrtP1-Tznj21QVkDWh9-Ia-q_LZNAuiSdXZN3hnUWu_VWiNzdAXKKLMde_zboYsmo44qHzluLlS6WY3gMZvwGSc8CZO0sMfEWPNAy4I1LymVuIGoQ81rcyIoxGCXxMnL4CLOeGzJ2WG_h48oVbDusX8D3VM4fQI_2wk4) which should already have the diagram in it to view :)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-43631: [C][Format] Add ArrowAsyncDeviceStreamHandler interface [arrow]
Posted by "zeroshade (via GitHub)" <gi...@apache.org>.
zeroshade commented on code in PR #43632:
URL: https://github.com/apache/arrow/pull/43632#discussion_r1767474678
##########
docs/source/format/CDeviceDataInterface.rst:
##########
@@ -650,6 +652,340 @@ The stream source is not assumed to be thread-safe. Consumers wanting to
call ``get_next`` from several threads should ensure those calls are
serialized.
+Async Device Stream Interface
+=============================
+
+The :ref:`C stream interface <_c-device-stream-interface>` provides a synchronous
+API centered around the consumer calling the callback functions to retrieve
+the next record batch. For some bindings, use cases, and interoperability, a more
+asynchronous, producer-focused interface may be required. These scenarios can utilize
+the ``ArrowAsyncDeviceStreamHandler``.
+
+Semantics
+---------
+
+Rather than the producer providing a structure of callbacks for a consumer to
+call and retrieve records, the Async interface is a consumer allocated structure.
+The consumer allocated struct provides handler callbacks for the producer to call
+when the schema and chunks of data are available, rather than the consumer using
+a blocking pull-style iteration.
+
+In addition to the ``ArrowAsyncDeviceStreamHandler``, there are also two additional
+structs used for the full data flow: ``ArrowAsyncTask`` and ``ArrowAsyncProducer``.
+
+Structure Definition
+--------------------
+
+The C device async stream interface is defined with a single ``struct`` definition:
+
+.. code-block:: c
+
+ #ifndef ARROW_C_ASYNC_STREAM_INTERFACE
+ #define ARROW_C_ASYNC_STREAM_INTERFACE
+
+ struct ArrowAsyncTask {
+ int (*get_data)(struct ArrowArrayTask* self, struct ArrowDeviceArray* out);
+
+ void* private_data;
+ };
+
+ struct ArrowAsyncProducer {
+ void (*request)(struct ArrowAsyncProducer* self, int64_t n);
+ void (*cancel)(struct ArrowAsyncProducer* self);
+
+ void (*release)(struct ArrowAsyncProducer* self);
+ void* private_data;
+ };
+
+ struct ArrowAsyncDeviceStreamHandler {
+ // handlers
+ int (*on_schema)(struct ArrowAsyncDeviceStreamHandler* self,
+ struct ArrowAsyncProducer* producer,
+ struct ArrowSchema* stream_schema, const char* addl_metadata);
+ int (*on_next_task)(struct ArrowAsyncDeviceStreamHandler* self,
+ struct ArrowAsyncTask* task, const char* metadata);
+ void (*on_error)(struct ArrowAsyncDeviceStreamHandler* self,
+ int code, const char* message, const char* metadata);
+
+ // release callback
+ void (*release)(struct ArrowAsyncDeviceStreamHandler* self);
+
+ // opaque handler-specific data
+ void* private_data;
+ };
+
+ #endif // ARROW_C_ASYNC_STREAM_INTERFACE
+
+.. note::
+ The canonical guard ``ARROW_C_ASYNC_STREAM_INTERFACE`` is meant to avoid
+ duplicate definitions if two projects copy the C async stream interface
+ definitions into their own headers, and a third-party project includes
+ from these two projects. It is therefore important that this guard is kept
+ exactly as-is when these definitions are copied.
+
+The ArrowAsyncDeviceStreamHandler structure
+'''''''''''''''''''''''''''''''''''''''''''
+
+The structure has the following fields:
+
+.. c:member:: int (*ArrowAsyncDeviceStreamHandler.on_schema)(struct ArrowAsyncDeviceStreamHandler*, struct ArrowAsyncProducer*, struct ArrowSchema*, const char*)
+
+ *Mandatory.* Handler for receiving the schema of the stream. All records should
+ match the provided schema. If successful, the function should return 0, otherwise
+ it should return an ``errno``-compatible error code.
+
+ The ``const char*`` parameter exists for producers to provide any extra contextual information
+ they want, such as the total number of rows in the stream, statistics, or otherwise. This is
+ encoded in the same format as :c:member:`ArrowSchema.metadata`. If not ``NULL``,
+ the lifetime is only the scope of the call to this function. A consumer who wants to maintain
+ the additional metadata beyond the lifetime of this call *MUST* copy the value themselves.
+
+ Unless the ``on_error`` handler is called, this will always get called exactly once and will be
+ the first method called on this object. As such the producer *MUST* provide an ``ArrowAsyncProducer``
+ object when calling this function to allow the consumer to manage back-pressure and flow control.
+ The producer maintains ownership of the ``ArrowAsyncProducer`` and must clean it up *after*
+ calling the release callback on this object.
+
+ A producer that receives a non-zero result here must not subsequently call anything other than
+ the release callback on this object.
+
+.. c:member:: int (*ArrowAsyncDeviceStreamHandler.on_next_task)(struct ArrowAsyncDeviceStreamHandler*, struct ArrowAsyncTask*, const char*)
+
+ *Mandatory.* Handler to be called when a new record is available for processing. The
+ schema for each record should be the same as the schema that ``on_schema`` was called with.
+ If successfully handled, the function should return 0, otherwise it should return an
+ ``errno``-compatible error code.
+
+ Rather than passing the record itself it receives an ``ArrowAsyncTask`` instead to facilitate
+ better consumer-focused thread control as far as receiving the data. A call to this function
+ simply indicates that data is available via the provided task.
+
+ The producer signals the end of the stream by passing ``NULL`` for the ``ArrowAsyncTask``
+ pointer instead of a valid address. This task object is only valid during the lifetime of
+ this function call. If the consumer wants to use the task beyond the scope of this method, it
+ must copy or move its contents to a new ArrowAsyncTask object.
+
+ The ``const char*`` parameter exists for producers to provide any extra contextual information
+ they want. This is encoded in the same format as :c:member:`ArrowSchema.metadata`. If not ``NULL``,
+ the lifetime is only the scope of the call to this function. A consumer who wants to maintain
+ the additional metadata beyond the lifetime of this call *MUST* copy the value themselves.
+
+ A producer *MUST NOT* call this concurrently from multiple threads.
+
+ The :c:member:`ArrowAsyncProducer.request` callback must be called to start receiving calls to this
+ handler.
+
+.. c:member:: void (*ArrowAsyncDeviceStreamHandler.on_error)(struct ArrowAsyncDeviceStreamHandler, int, const char*, const char*)
+
+ *Mandatory.* Handler to be called when an error is encountered by the producer. After calling
+ this, the ``release`` callback will be called as the last call on this struct. The parameters
+ are an ``errno``-compatible error code and an optional error message and metadata.
+
+ If the message and metadata are not ``NULL``, their lifetime is only valid during the scope
+ of this call. A consumer who wants to maintain these values past the return of this function
+ *MUST* copy the values themselves.
+
+ If the metadata parameter is not ``NULL``, to provide key-value error metadata, then it should
+ be encoded identically to the way that metadata is encoded in :c:member:`ArrowSchema.metadata`.
+
+ It is valid for this to be called by a producer with or without a preceding call to
+ :c:member:`ArrowAsyncProducer.request`. This callback *MUST NOT* call any methods of an
+ ``ArrowAsyncProducer`` object.
+
+.. c:member:: void (*ArrowAsyncDeviceStreamHandler.release)(struct ArrowAsyncDeviceStreamHandler*)
+
+ *Mandatory.* A pointer to a consumer-provided release callback for the handler.
+
+ It is valid for this to be called by a producer with or without a preceding call to
+ :c:member:`ArrowAsyncProducer.request`. This must not call any methods of an ``ArrowAsyncProducer``
+ object.
+
+.. c:member:: void* ArrowAsyncDeviceStreamHandler.private_data
+
+ *Optional.* An opaque pointer to consumer-provided private data.
+
+ Producers *MUST NOT* process this member. Lifetime of this member is handled by
+ the consumer, and especially by the release callback.
+
+The ArrowAsyncTask structure
+''''''''''''''''''''''''''''
+
+The purpose of using a Task object rather than passing the array directly to the ``on_next``
+callback is to allow for more complex and efficient thread handling. Utilizing a Task
+object allows for a producer to separate the "decoding" logic from the I/O, enabling a
+consumer to avoid transferring data between CPU cores (e.g. from one L1/L2 cache to another).
+
+This producer-provided structure has the following fields:
+
+.. c:member:: int (*ArrowArrayTask.get_data)(struct ArrowArrayTask*, struct ArrowDeviceArray*)
+
+ *Mandatory.* A callback to populate the provided ``ArrowDeviceArray`` with the available data.
+ The order of ArrowAsyncTasks provided by the producer enables a consumer to know the order of
+ the data to process. If the consumer does not care about the data that is owned by this task,
+ it must still call ``get_data`` so that the producer can perform any required cleanup. ``NULL``
+ should be passed as the device array pointer to indicate that the consumer doesn't want the
+ actual data, letting the task perform necessary cleanup.
+
+ If a non-zero value is returned from this, it should be followed only by the producer calling
+ the ``on_error`` callback of the ``ArrowAsyncDeviceStreamHandler``. Because calling this method
Review Comment:
not quite. The function that retrieves the data from the task is called *by the consumer* and returns an `errno`-compatible int to indicate whether there was any issue with providing that data to the consumer from the task.
As such, the consumer can propagate that error if they receive it, but getting more context and information about the error for logging beyond the errno value is still centralized to the `on_error` callback. As such, this does not force anything to be infallible.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-43631: [C][Format] Add ArrowAsyncDeviceStreamHandler interface [arrow]
Posted by "zeroshade (via GitHub)" <gi...@apache.org>.
zeroshade commented on code in PR #43632:
URL: https://github.com/apache/arrow/pull/43632#discussion_r1767770010
##########
docs/source/format/CDeviceDataInterface.rst:
##########
@@ -650,6 +652,340 @@ The stream source is not assumed to be thread-safe. Consumers wanting to
call ``get_next`` from several threads should ensure those calls are
serialized.
+Async Device Stream Interface
+=============================
+
+The :ref:`C stream interface <_c-device-stream-interface>` provides a synchronous
+API centered around the consumer calling the callback functions to retrieve
+the next record batch. For some bindings, use cases, and interoperability, a more
+asynchronous, producer-focused interface may be required. These scenarios can utilize
+the ``ArrowAsyncDeviceStreamHandler``.
+
+Semantics
+---------
+
+Rather than the producer providing a structure of callbacks for a consumer to
+call and retrieve records, the Async interface is a consumer allocated structure.
+The consumer allocated struct provides handler callbacks for the producer to call
+when the schema and chunks of data are available, rather than the consumer using
+a blocking pull-style iteration.
+
+In addition to the ``ArrowAsyncDeviceStreamHandler``, there are also two additional
+structs used for the full data flow: ``ArrowAsyncTask`` and ``ArrowAsyncProducer``.
+
+Structure Definition
+--------------------
+
+The C device async stream interface is defined with a single ``struct`` definition:
+
+.. code-block:: c
+
+ #ifndef ARROW_C_ASYNC_STREAM_INTERFACE
+ #define ARROW_C_ASYNC_STREAM_INTERFACE
+
+ struct ArrowAsyncTask {
+ int (*get_data)(struct ArrowArrayTask* self, struct ArrowDeviceArray* out);
+
+ void* private_data;
+ };
+
+ struct ArrowAsyncProducer {
+ void (*request)(struct ArrowAsyncProducer* self, int64_t n);
+ void (*cancel)(struct ArrowAsyncProducer* self);
+
+ void (*release)(struct ArrowAsyncProducer* self);
+ void* private_data;
+ };
+
+ struct ArrowAsyncDeviceStreamHandler {
+ // handlers
+ int (*on_schema)(struct ArrowAsyncDeviceStreamHandler* self,
+ struct ArrowAsyncProducer* producer,
+ struct ArrowSchema* stream_schema, const char* addl_metadata);
+ int (*on_next_task)(struct ArrowAsyncDeviceStreamHandler* self,
+ struct ArrowAsyncTask* task, const char* metadata);
+ void (*on_error)(struct ArrowAsyncDeviceStreamHandler* self,
+ int code, const char* message, const char* metadata);
+
+ // release callback
+ void (*release)(struct ArrowAsyncDeviceStreamHandler* self);
+
+ // opaque handler-specific data
+ void* private_data;
+ };
+
+ #endif // ARROW_C_ASYNC_STREAM_INTERFACE
+
+.. note::
+ The canonical guard ``ARROW_C_ASYNC_STREAM_INTERFACE`` is meant to avoid
+ duplicate definitions if two projects copy the C async stream interface
+ definitions into their own headers, and a third-party project includes
+ from these two projects. It is therefore important that this guard is kept
+ exactly as-is when these definitions are copied.
+
+The ArrowAsyncDeviceStreamHandler structure
+'''''''''''''''''''''''''''''''''''''''''''
+
+The structure has the following fields:
+
+.. c:member:: int (*ArrowAsyncDeviceStreamHandler.on_schema)(struct ArrowAsyncDeviceStreamHandler*, struct ArrowAsyncProducer*, struct ArrowSchema*, const char*)
+
+ *Mandatory.* Handler for receiving the schema of the stream. All records should
+ match the provided schema. If successful, the function should return 0, otherwise
+ it should return an ``errno``-compatible error code.
+
+ The ``const char*`` parameter exists for producers to provide any extra contextual information
+ they want, such as the total number of rows in the stream, statistics, or otherwise. This is
+ encoded in the same format as :c:member:`ArrowSchema.metadata`. If not ``NULL``,
+ the lifetime is only the scope of the call to this function. A consumer who wants to maintain
+ the additional metadata beyond the lifetime of this call *MUST* copy the value themselves.
+
+ Unless the ``on_error`` handler is called, this will always get called exactly once and will be
+ the first method called on this object. As such the producer *MUST* provide an ``ArrowAsyncProducer``
+ object when calling this function to allow the consumer to manage back-pressure and flow control.
+ The producer maintains ownership of the ``ArrowAsyncProducer`` and must clean it up *after*
+ calling the release callback on this object.
+
+ A producer that receives a non-zero result here must not subsequently call anything other than
+ the release callback on this object.
+
+.. c:member:: int (*ArrowAsyncDeviceStreamHandler.on_next_task)(struct ArrowAsyncDeviceStreamHandler*, struct ArrowAsyncTask*, const char*)
+
+ *Mandatory.* Handler to be called when a new record is available for processing. The
+ schema for each record should be the same as the schema that ``on_schema`` was called with.
+ If successfully handled, the function should return 0, otherwise it should return an
+ ``errno``-compatible error code.
+
+ Rather than passing the record itself it receives an ``ArrowAsyncTask`` instead to facilitate
+ better consumer-focused thread control as far as receiving the data. A call to this function
+ simply indicates that data is available via the provided task.
+
+ The producer signals the end of the stream by passing ``NULL`` for the ``ArrowAsyncTask``
+ pointer instead of a valid address. This task object is only valid during the lifetime of
+ this function call. If the consumer wants to use the task beyond the scope of this method, it
+ must copy or move its contents to a new ArrowAsyncTask object.
+
+ The ``const char*`` parameter exists for producers to provide any extra contextual information
+ they want. This is encoded in the same format as :c:member:`ArrowSchema.metadata`. If not ``NULL``,
+ the lifetime is only the scope of the call to this function. A consumer who wants to maintain
+ the additional metadata beyond the lifetime of this call *MUST* copy the value themselves.
+
+ A producer *MUST NOT* call this concurrently from multiple threads.
+
+ The :c:member:`ArrowAsyncProducer.request` callback must be called to start receiving calls to this
+ handler.
+
+.. c:member:: void (*ArrowAsyncDeviceStreamHandler.on_error)(struct ArrowAsyncDeviceStreamHandler, int, const char*, const char*)
+
+ *Mandatory.* Handler to be called when an error is encountered by the producer. After calling
+ this, the ``release`` callback will be called as the last call on this struct. The parameters
+ are an ``errno``-compatible error code and an optional error message and metadata.
+
+ If the message and metadata are not ``NULL``, their lifetime is only valid during the scope
+ of this call. A consumer who wants to maintain these values past the return of this function
+ *MUST* copy the values themselves.
+
+ If the metadata parameter is not ``NULL``, to provide key-value error metadata, then it should
+ be encoded identically to the way that metadata is encoded in :c:member:`ArrowSchema.metadata`.
+
+ It is valid for this to be called by a producer with or without a preceding call to
+ :c:member:`ArrowAsyncProducer.request`. This callback *MUST NOT* call any methods of an
+ ``ArrowAsyncProducer`` object.
+
+.. c:member:: void (*ArrowAsyncDeviceStreamHandler.release)(struct ArrowAsyncDeviceStreamHandler*)
+
+ *Mandatory.* A pointer to a consumer-provided release callback for the handler.
+
+ It is valid for this to be called by a producer with or without a preceding call to
+ :c:member:`ArrowAsyncProducer.request`. This must not call any methods of an ``ArrowAsyncProducer``
+ object.
+
+.. c:member:: void* ArrowAsyncDeviceStreamHandler.private_data
+
+ *Optional.* An opaque pointer to consumer-provided private data.
+
+ Producers *MUST NOT* process this member. Lifetime of this member is handled by
+ the consumer, and especially by the release callback.
+
+The ArrowAsyncTask structure
+''''''''''''''''''''''''''''
+
+The purpose of using a Task object rather than passing the array directly to the ``on_next``
+callback is to allow for more complex and efficient thread handling. Utilizing a Task
+object allows for a producer to separate the "decoding" logic from the I/O, enabling a
+consumer to avoid transferring data between CPU cores (e.g. from one L1/L2 cache to another).
+
+This producer-provided structure has the following fields:
+
+.. c:member:: int (*ArrowArrayTask.get_data)(struct ArrowArrayTask*, struct ArrowDeviceArray*)
+
+ *Mandatory.* A callback to populate the provided ``ArrowDeviceArray`` with the available data.
+ The order of ArrowAsyncTasks provided by the producer enables a consumer to know the order of
+ the data to process. If the consumer does not care about the data that is owned by this task,
+ it must still call ``get_data`` so that the producer can perform any required cleanup. ``NULL``
+ should be passed as the device array pointer to indicate that the consumer doesn't want the
+ actual data, letting the task perform necessary cleanup.
+
+ If a non-zero value is returned from this, it should be followed only by the producer calling
+ the ``on_error`` callback of the ``ArrowAsyncDeviceStreamHandler``. Because calling this method
Review Comment:
alternately, the producer can keep a reference to the `ArrowAsyncStreamHandler` in the `private_data` of the task, so it can easily call `on_error` without having to keep track of the errors produced
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-43631: [C][Format] Add ArrowAsyncDeviceStreamHandler interface [arrow]
Posted by "zeroshade (via GitHub)" <gi...@apache.org>.
zeroshade commented on PR #43632:
URL: https://github.com/apache/arrow/pull/43632#issuecomment-2338448929
@leventov Unless i'm mistaken, it seems like both Velox and Datafusion should be able to easily slot into and utilize the updated API described and designed here. Is there anything specific that you're seeing wouldn't work with those interfaces?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-43631: [C][Format] Add ArrowAsyncDeviceStreamHandler interface [arrow]
Posted by "zeroshade (via GitHub)" <gi...@apache.org>.
zeroshade commented on code in PR #43632:
URL: https://github.com/apache/arrow/pull/43632#discussion_r1829476978
##########
cpp/src/arrow/c/abi.h:
##########
@@ -356,8 +356,7 @@ struct ArrowAsyncDeviceStreamHandler {
// A producer that receives a non-zero return here should stop producing and eventually
// call release instead.
int (*on_schema)(struct ArrowAsyncDeviceStreamHandler* self,
- struct ArrowAsyncProducer* producer, struct ArrowSchema* stream_schema,
- const char* addl_metadata);
+ struct ArrowSchema* stream_schema, const char* addl_metadata);
Review Comment:
I've updated this, shifting the `additional_metadata` member according to this suggestion.
@bkietz if this is all good can I get an approval? I'll merge this after that.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-43631: [C][Format] Add ArrowAsyncDeviceStreamHandler interface [arrow]
Posted by "bkietz (via GitHub)" <gi...@apache.org>.
bkietz commented on code in PR #43632:
URL: https://github.com/apache/arrow/pull/43632#discussion_r1822736474
##########
cpp/src/arrow/c/abi.h:
##########
@@ -356,8 +356,7 @@ struct ArrowAsyncDeviceStreamHandler {
// A producer that receives a non-zero return here should stop producing and eventually
// call release instead.
int (*on_schema)(struct ArrowAsyncDeviceStreamHandler* self,
- struct ArrowAsyncProducer* producer, struct ArrowSchema* stream_schema,
- const char* addl_metadata);
+ struct ArrowSchema* stream_schema, const char* addl_metadata);
Review Comment:
After `on_schema` all members of `self->producer` are initialized, right? So that would include `self->producer->metadata`. And all members of `self->producer` are expected to stay valid from then until `self` is destroyed, so that includes `self->producer->metadata`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-43631: [C][Format] Add ArrowAsyncDeviceStreamHandler interface [arrow]
Posted by "westonpace (via GitHub)" <gi...@apache.org>.
westonpace commented on PR #43632:
URL: https://github.com/apache/arrow/pull/43632#issuecomment-2282026639
Ah, one more potential approach. This one is actually my favorite for a file(s) scanner. It has all the advantages of a pull-based approach plus it puts the consumer in complete control of deciding how much decode parallelism there should be (by calling `next_task` before the previous call to `next_task` has completed). On the other hand, it can make it more difficult if the producer does not have any control over the parallelism or know the size of the stream ahead-of-time (e.g. maybe it is receiving data from a TCP stream):
```
struct Waker {
// Signal to the producer that more data is available, the consumer shall release any resources
// associated with the waker after this method is called. The producer shall not call any other
// methods after calling this method.
//
// The producer must always call this method even if an error is encountered (in which case the
// error will be reported on the following call to `get_next`).
void wake(Waker* waker);
void* private_data;
};
struct ArrowArrayTask {
// Callback to get the array
// (if no error and the array is released, the stream has ended)
//
// Return value: 0 if successful, an `errno`-compatible error code otherwise.
//
// If EWOULDBLOCK is returned then the array is not ready. If the
// producer returns this value then the producer takes ownership of `waker` and is responsible
// for calling `wake` when more data is available. If any other value is returned then the producer
// shall ignore `waker` and never call any methods on it.
//
// If successful, the ArrowDeviceArray must be released independently from the stream.
int (*get_next)(struct ArrowArrayTask* self, Waker* waker, struct ArrowDeviceArray* out);
void (*release)(struct ArrowArrayTask* self);
void* ArrowArrayTask;
};
struct ArrowAsyncDeviceArrayStream {
// Callback to get the next array task
// (if no error and the task is released, the stream has ended)
//
// Return value: 0 if successful, an `errno`-compatible error code otherwise.
//
// The consumer is allowed to call this method again even if the tasks returned by a previous
// call have not completed. However, the consumer shall not call this re-entrantly.
//
// If successful, the ArrowDeviceArray must be released independently from the stream.
int (*get_next_task)(struct ArrowAsyncDeviceArrayStream* self, struct ArrowArrayTask* task);
// The rest is identical to `ArrowDeviceArrayStream`
ArrowDeviceType device_type;
const char* (*get_last_error)(struct ArrowAsyncDeviceArrayStream* self);
void (*release)(struct ArrowAsyncDeviceArrayStream* self);
void* private_data;
};
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-43631: [C][Format] Add ArrowAsyncDeviceStreamHandler interface [arrow]
Posted by "pitrou (via GitHub)" <gi...@apache.org>.
pitrou commented on code in PR #43632:
URL: https://github.com/apache/arrow/pull/43632#discussion_r1764971441
##########
docs/source/format/CDeviceDataInterface.rst:
##########
@@ -650,6 +652,331 @@ The stream source is not assumed to be thread-safe. Consumers wanting to
call ``get_next`` from several threads should ensure those calls are
serialized.
+Async Device Stream Interface
+=============================
+
+The :ref:`C stream interface <_c-device-stream-interface>` provides a synchronous
+API centered around the consumer calling the callback functions to retrieve
+the next record batch. For some bindings, use cases, and interoperability, a more
+asynchronous, producer-focused interface may be required. These scenarios can utilize
+the ``ArrowAsyncDeviceStreamHandler``.
+
+Semantics
+---------
+
+Rather than the producer providing a structure of callbacks for a consumer to
+call and retrieve records, the Async interface is a consumer allocated structure.
+The consumer allocated struct provides handler callbacks for the producer to call
+when the schema and chunks of data are available, rather than the consumer using
+a blocking pull-style iteration.
+
+In addition to the ``ArrowAsyncDeviceStreamHandler``, there are also two additional
+structs used for the full data flow: ``ArrowAsyncTask`` and ``ArrowAsyncProducer``.
+
+Structure Definition
+--------------------
+
+The C device async stream interface is defined with a single ``struct`` definition:
+
+.. code-block:: c
+
+ #ifndef ARROW_C_ASYNC_STREAM_INTERFACE
+ #define ARROW_C_ASYNC_STREAM_INTERFACE
+
+ struct ArrowAsyncTask {
+ int (*get_data)(struct ArrowArrayTask* self, struct ArrowDeviceArray* out);
+
+ void* private_data;
+ };
+
+ struct ArrowAsyncProducer {
+ void (*request)(struct ArrowAsyncProducer* self, uint64_t n);
+ void (*cancel)(struct ArrowAsyncProducer* self);
+
+ void (*release)(struct ArrowAsyncProducer* self);
+ void* private_data;
+ };
+
+ struct ArrowAsyncDeviceStreamHandler {
+ // handlers
+ int (*on_schema)(struct ArrowAsyncDeviceStreamHandler* self,
+ struct ArrowAsyncProducer* producer,
+ struct ArrowSchema* stream_schema, const char* addl_metadata);
+ int (*on_next_task)(struct ArrowAsyncDeviceStreamHandler* self,
+ struct ArrowAsyncTask* task, const char* metadata);
+ void (*on_error)(struct ArrowAsyncDeviceStreamHandler* self,
+ int code, const char* message, const char* metadata);
+
+ // release callback
+ void (*release)(struct ArrowAsyncDeviceStreamHandler* self);
+
+ // opaque handler-specific data
+ void* private_data;
+ };
+
+ #endif // ARROW_C_ASYNC_STREAM_INTERFACE
+
+.. note::
+ The canonical guard ``ARROW_C_ASYNC_STREAM_INTERFACE`` is meant to avoid
+ duplicate definitions if two projects copy the C async stream interface
+ definitions into their own headers, and a third-party project includes
+ from these two projects. It is therefore important that this guard is kept
+ exactly as-is when these definitions are copied.
+
+The ArrowAsyncDeviceStreamHandler structure
+'''''''''''''''''''''''''''''''''''''''''''
+
+The structure has the following fields:
+
+.. c:member:: int (*ArrowAsyncDeviceStreamHandler.on_schema)(struct ArrowAsyncDeviceStreamHandler*, struct ArrowAsyncProducer*, struct ArrowSchema*, const char*)
+
+ *Mandatory.* Handler for receiving the schema of the stream. All records should
+ match the provided schema. If successful, the function should return 0, otherwise
+ it should return an ``errno``-compatible error code.
+
+ The ``const char*`` parameter exists for producers to provide any extra contextual information
+ they want, such as the total number of rows in the stream, statistics, or otherwise. This is
+ encoded in the same format as :c:member:`ArrowSchema.metadata`. If not ``NULL``,
+ the lifetime is only the scope of the call to this function. A consumer who wants to maintain
+ the additional metadata beyond the lifetime of this call *MUST* copy the value themselves.
+
+ Unless the ``on_error`` handler is called, this will always get called exactly once and will be
+ the first method called on this object. As such the producer *MUST* provide an ``ArrowAsyncProducer``
+ object when calling this function to allow the consumer to manage back-pressure and flow control.
+ The producer maintains ownership of the ``ArrowAsyncProducer`` and must clean it up before or after
+ calling the release callback on this object.
+
+ A producer that receives a non-zero result here must not subsequently call anything other than
+ the release callback on this object.
+
+.. c:member:: int (*ArrowAsyncDeviceStreamHandler.on_next_task)(struct ArrowAsyncDeviceStreamHandler*, struct ArrowAsyncTask*, const char*)
+
+ *Mandatory.* Handler to be called when a new record is available for processing. The
+ schema for each record should be the same as the schema that ``on_schema`` was called with.
+ If successfully handled, the function should return 0, otherwise it should return an
+ ``errno``-compatible error code.
+
+ Rather than passing the record itself it receives an ``ArrowAsyncTask`` instead to facilitate
+ better consumer-focused thread control as far as receiving the data. A call to this function
+ simply indicates that data is available via the provided task.
+
+ The ``const char*`` parameter exists for producers to provide any extra contextual information
+ they want. This is encoded in the same format as :c:member:`ArrowSchema.metadata`. If not ``NULL``,
+ the lifetime is only the scope of the call to this function. A consumer who wants to maintain
+ the additional metadata beyond the lifetime of this call *MUST* copy the value themselves.
+
+ A producer *MUST NOT* call this concurrently from multiple different threads.
+
+ The :c:member:`ArrowAsyncProducer.request` callback must be called to start receiving calls to this
+ handler.
+
+ This function *MUST* be able to be called re-entrantly on the same thread to allow for the
+ :c:member:`ArrowAsyncProducer.request` callback to potentially call this method recursively.
+
+.. c:member:: void (*ArrowAsyncDeviceStreamHandler.on_error)(struct ArrowAsyncDeviceStreamHandler, int, const char*, const char*)
+
+ *Mandatory.* Handler to be called when an error is encountered by the producer. After calling
+ this, the ``release`` callback will be called as the last call on this struct. The parameters
+ are an ``errno``-compatible error code and an optional error message and metadata.
+
+ If the message and metadata are not ``NULL``, their lifetime is only valid during the scope
+ of this call. A consumer who wants to maintain these values past the return of this function
+ *MUST* copy the values themselves.
+
+ If the metadata parameter is not ``NULL``, to provide key-value error metadata, then it should
+ be encoded identically to the way that metadata is encoded in :c:member:`ArrowSchema.metadata`.
+
+ It is valid for this to be called by a producer with or without a preceding call to
+ :c:member:`ArrowAsyncProducer.request`. This callback *MUST NOT* call any methods of an
+ ``ArrowAsyncProducer`` object.
+
+.. c:member:: void (*ArrowAsyncDeviceStreamHandler.release)(struct ArrowAsyncDeviceStreamHandler*)
+
+ *Mandatory.* A pointer to a consumer-provided release callback for the handler.
+
+ It is valid for this to be called by a producer with or without a preceding call to
+ :c:member:`ArrowAsyncProducer.request`. This must not call any methods of an ``ArrowAsyncProducer``
+ object.
+
+.. c:member:: void* ArrowAsyncDeviceStreamHandler.private_data
+
+ *Optional.* An opaque pointer to consumer-provided private data.
+
+ Producers *MUST NOT* process this member. Lifetime of this member is handled by
+ the consumer, and especially by the release callback.
+
+The ArrowAsyncTask structure
+''''''''''''''''''''''''''''
+
+The purpose of using a Task object rather than passing the array directly to the ``on_next``
+callback is to allow for more complex and efficient thread handling. Utilizing a Task
+object allows for a producer to separate the "decoding" logic from the I/O, enabling a
+consumer to avoid transferring data between CPU cores (e.g. from one L1/L2 cache to another).
+
+This producer-provided structure has the following fields:
+
+.. c:member:: int (*ArrowArrayTask.get_data)(struct ArrowArrayTask*, struct ArrowDeviceArray*)
+
+ *Mandatory.* A callback to populate the provided ``ArrowDeviceArray`` with the available data.
+ The order of ArrowAsyncTasks provided by the producer enables a consumer to know the order of
+ the data to process.
+
+ If a non-zero value is returned from this, it should be followed only by the producer calling
+ the ``on_error`` callback of the ``ArrowAsyncDeviceStreamHandler``. Because calling this method
+ is likely to be separate from the current control flow, returning a non-zero value to signal
+ an error occuring allows the current thread to decide handle the case accordingly, while still
+ allowing all error logging and handling to be centralized in the
+ :c:member:`ArrowAsyncDeviceStreamHandler.on_error` callback.
+
+ Rather than having a separate release callback, any required cleanup should be performed as part
+ of the invocation of this callback, with ownership of the Array being passed to the consumer-provided
+ parameter calling this, and must be released separately.
+
+ It is only valid to call this method exactly once.
+
+.. c:member:: void* ArrowArrayTask.private_data
+
+ *Optional.* An opaque pointer to consumer-provided private data.
+
+ Consumers *MUST NOT* process this member. Lifetime of this member is handled by
+ the producer who created this object, and should be cleaned up if necessary during
+ the call to :c:member:`ArrowArrayTask.get_data`.
+
+The ArrowAsyncProducer structure
+''''''''''''''''''''''''''''''''
+
+This producer-provided and managed object has the following fields:
+
+.. c:member:: void (*ArrowAsyncProducer.request)(struct ArrowAsyncProducer*, uint64_t)
+
+ *Mandatory.* This function must be called by a consumer to start receiving calls to
+ :c:member:`ArrowAsyncDeviceStreamHandler.on_next_task`. It *MUST* be valid to call
+ this synchronously from within :c:member:`ArrowAsyncDeviceStreamHandler.on_next_task`
+ or :c:member:`ArrowAsyncDeviceStreamHandler.on_schema`. As a result, this function
+ *MUST NOT* synchronously call ``on_next_task`` or ``on_error`` to avoid recursive
+ and reentrant callbacks.
+
+ After ``cancel`` is called, additional calls to this function must be a NOP, but allowed.
+
+ While not cancelled, calling this function registers the given number of additional
+ arrays/batches to be produced by the producer. A producer should only call
+ the appropriate ``on_next_task`` callback up to a maximum of the total sum of calls to
+ this method before propagating back-pressure / waiting.
+
+ Any error encountered by calling request must be propagated by calling the ``on_error``
+ callback of the ``ArrowAsyncDeviceStreamHandler``.
+
+ A producer *MUST* support an unbounded number of calls to this callback and *MUST* support
Review Comment:
Consumers will have to handle errors anyway, why would this simplify anything?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-43631: [C][Format] Add ArrowAsyncDeviceStreamHandler interface [arrow]
Posted by "lidavidm (via GitHub)" <gi...@apache.org>.
lidavidm commented on code in PR #43632:
URL: https://github.com/apache/arrow/pull/43632#discussion_r1821689061
##########
cpp/src/arrow/c/abi.h:
##########
@@ -356,8 +356,7 @@ struct ArrowAsyncDeviceStreamHandler {
// A producer that receives a non-zero return here should stop producing and eventually
// call release instead.
int (*on_schema)(struct ArrowAsyncDeviceStreamHandler* self,
- struct ArrowAsyncProducer* producer, struct ArrowSchema* stream_schema,
- const char* addl_metadata);
+ struct ArrowSchema* stream_schema, const char* addl_metadata);
Review Comment:
The metadata might not be available until after the callback anyways so there'd instead be the question of when it's valid/invalid
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-43631: [C][Format] Add ArrowAsyncDeviceStreamHandler interface [arrow]
Posted by "zeroshade (via GitHub)" <gi...@apache.org>.
zeroshade commented on PR #43632:
URL: https://github.com/apache/arrow/pull/43632#issuecomment-2307300731
> I'm not quite sold on this if we want to have a callback based API. Though I understand wanting to have the most efficient possible theoretical API. At least from my Flight/gRPC based view, though, it seems difficult to take advantage of.
Personally I agree here. But I'm willing to accommodate it if we believe it is the best way forward, @westonpace @paleolimbot @pitrou Thoughts?
> FWIW, do we need to do anything to accommodate the arrow::ArrayStatistics API that @kou has been working on?
That's a good question, should we add some other callback / method for providing statistics? Or would this be something that we'd add to the `on_next` method? @kou?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-43631: [C][Format] Add ArrowAsyncDeviceStreamHandler interface [arrow]
Posted by "zeroshade (via GitHub)" <gi...@apache.org>.
zeroshade commented on code in PR #43632:
URL: https://github.com/apache/arrow/pull/43632#discussion_r1750452544
##########
cpp/src/arrow/c/abi.h:
##########
@@ -228,6 +228,180 @@ struct ArrowDeviceArrayStream {
#endif // ARROW_C_DEVICE_STREAM_INTERFACE
+#ifndef ARROW_C_ASYNC_STREAM_INTERFACE
+#define ARROW_C_ASYNC_STREAM_INTERFACE
+
+// ArrowAsyncTask represents available data from a producer that was passed to
+// an invocation of `on_next_task` on the ArrowAsyncDeviceStreamHandler.
+struct ArrowAsyncTask {
+ // This callback should populate the ArrowDeviceArray associated with this task.
+ // The order of ArrowAsyncTasks provided by the producer enables a consumer to
+ // ensure the order of data to process.
+ //
+ // Returns: 0 if successful, errno-compatible error otherwise.
+ //
+ // If a non-0 value is returned then it should be followed by a call to `on_error`
+ // on the appropriate ArrowAsyncDeviceStreamHandler.
+ //
+ // Rather than a release callback, any required cleanup should be performed as part
+ // of the invocation of `get_data`. Ownership of the Array is passed to the consumer
+ // calling this, and so it must be released separately.
+ //
+ // It is only valid to call this method exactly once.
+ int (*get_data)(struct ArrowArrayTask* self, struct ArrowDeviceArray* out);
+
+ // opaque task-specific data
+ void* private_data;
+};
+
+// ArrowAsyncProducer represents a 1-to-1 relationship between an async producer
+// and consumer. This object allows the consumer to perform backpressure and flow
+// control on the asynchronous stream processing. This object must be owned by the
+// producer who creates it, and thus is responsible for cleaning it up.
+struct ArrowAsyncProducer {
+ // A consumer must call this function to start receiving on_next_task calls.
+ //
+ // It *must* be valid to call this synchronously from within on_next_task or
+ // on_schema, with a producer placing an upper bound on possible synchronous
+ // recursion between calls of on_next_task -> request -> on_next_task ->....
+ //
+ // After cancel has been called, additional calls to this function must be NOPs,
+ // but allowed.
+ //
+ // While not cancelled, calling this function must register the given number of
+ // additional arrays/batches to be produced with the producer. The producer should
+ // only call `on_next_task` at most the registered number of arrays before propagating
+ // backpressure.
+ //
+ // While not cancelled, calling request MAY synchronously call `on_next_task`,
+ // `on_error`, or `release` on the ArrowAsyncDeviceStreamHandler.
+ //
+ // Any error encountered by calling request must be propagated by calling the `on_error`
+ // callback of the ArrowAsyncDeviceStreamHandler.
+ //
+ // A producer must support an unbounded number of calls to request and must support
+ // a total registered demand (sum requested - sum delivered) of up to UINT64_MAX.
+ void (*request)(struct ArrowAsyncProducer* self, uint64_t n);
+
+ // This cancel callback signals a producer that it must eventually stop making calls
+ // to on_next_task. It must be idempotent and thread-safe.
+ //
+ // After calling cancel once, subsequent calls must be NOPs.
+ //
+ // It is not required that calling cancel affect the producer immediately, only that it
+ // must eventually stop calling on_next_task and subsequently call release on the
+ // async handler. As such, a consumer must be prepared to receive one or more calls to
+ // `on_next_task` even after calling cancel if there are still requested arrays pending.
+ //
+ // Any error encountered during handling a call to cancel must be reported via the on_error
+ // callback on the async stream handler.
+ void (*cancel)(struct ArrowAsyncProducer* self);
Review Comment:
successful cancellation should lead only to `release`. The producer should only call `on_error` if an error occurs during processing of `cancel`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-43631: [C][Format] Add ArrowAsyncDeviceStreamHandler interface [arrow]
Posted by "pitrou (via GitHub)" <gi...@apache.org>.
pitrou commented on code in PR #43632:
URL: https://github.com/apache/arrow/pull/43632#discussion_r1757486596
##########
docs/source/format/CDeviceDataInterface.rst:
##########
@@ -650,6 +652,331 @@ The stream source is not assumed to be thread-safe. Consumers wanting to
call ``get_next`` from several threads should ensure those calls are
serialized.
+Async Device Stream Interface
+=============================
+
+The :ref:`C stream interface <_c-device-stream-interface>` provides a synchronous
+API centered around the consumer calling the callback functions to retrieve
+the next record batch. For some bindings, use cases, and interoperability, a more
+asynchronous, producer-focused interface may be required. These scenarios can utilize
+the ``ArrowAsyncDeviceStreamHandler``.
+
+Semantics
+---------
+
+Rather than the producer providing a structure of callbacks for a consumer to
+call and retrieve records, the Async interface is a consumer allocated structure.
+The consumer allocated struct provides handler callbacks for the producer to call
+when the schema and chunks of data are available, rather than the consumer using
+a blocking pull-style iteration.
+
+In addition to the ``ArrowAsyncDeviceStreamHandler``, there are also two additional
+structs used for the full data flow: ``ArrowAsyncTask`` and ``ArrowAsyncProducer``.
+
+Structure Definition
+--------------------
+
+The C device async stream interface is defined with a single ``struct`` definition:
+
+.. code-block:: c
+
+ #ifndef ARROW_C_ASYNC_STREAM_INTERFACE
+ #define ARROW_C_ASYNC_STREAM_INTERFACE
+
+ struct ArrowAsyncTask {
+ int (*get_data)(struct ArrowArrayTask* self, struct ArrowDeviceArray* out);
+
+ void* private_data;
+ };
+
+ struct ArrowAsyncProducer {
+ void (*request)(struct ArrowAsyncProducer* self, uint64_t n);
+ void (*cancel)(struct ArrowAsyncProducer* self);
+
+ void (*release)(struct ArrowAsyncProducer* self);
+ void* private_data;
+ };
+
+ struct ArrowAsyncDeviceStreamHandler {
+ // handlers
+ int (*on_schema)(struct ArrowAsyncDeviceStreamHandler* self,
+ struct ArrowAsyncProducer* producer,
+ struct ArrowSchema* stream_schema, const char* addl_metadata);
+ int (*on_next_task)(struct ArrowAsyncDeviceStreamHandler* self,
+ struct ArrowAsyncTask* task, const char* metadata);
+ void (*on_error)(struct ArrowAsyncDeviceStreamHandler* self,
+ int code, const char* message, const char* metadata);
+
+ // release callback
+ void (*release)(struct ArrowAsyncDeviceStreamHandler* self);
+
+ // opaque handler-specific data
+ void* private_data;
+ };
+
+ #endif // ARROW_C_ASYNC_STREAM_INTERFACE
+
+.. note::
+ The canonical guard ``ARROW_C_ASYNC_STREAM_INTERFACE`` is meant to avoid
+ duplicate definitions if two projects copy the C async stream interface
+ definitions into their own headers, and a third-party project includes
+ from these two projects. It is therefore important that this guard is kept
+ exactly as-is when these definitions are copied.
+
+The ArrowAsyncDeviceStreamHandler structure
+'''''''''''''''''''''''''''''''''''''''''''
+
+The structure has the following fields:
+
+.. c:member:: int (*ArrowAsyncDeviceStreamHandler.on_schema)(struct ArrowAsyncDeviceStreamHandler*, struct ArrowAsyncProducer*, struct ArrowSchema*, const char*)
+
+ *Mandatory.* Handler for receiving the schema of the stream. All records should
+ match the provided schema. If successful, the function should return 0, otherwise
+ it should return an ``errno``-compatible error code.
+
+ The ``const char*`` parameter exists for producers to provide any extra contextual information
+ they want, such as the total number of rows in the stream, statistics, or otherwise. This is
+ encoded in the same format as :c:member:`ArrowSchema.metadata`. If not ``NULL``,
+ the lifetime is only the scope of the call to this function. A consumer who wants to maintain
+ the additional metadata beyond the lifetime of this call *MUST* copy the value themselves.
+
+ Unless the ``on_error`` handler is called, this will always get called exactly once and will be
+ the first method called on this object. As such the producer *MUST* provide an ``ArrowAsyncProducer``
+ object when calling this function to allow the consumer to manage back-pressure and flow control.
+ The producer maintains ownership of the ``ArrowAsyncProducer`` and must clean it up before or after
+ calling the release callback on this object.
+
+ A producer that receives a non-zero result here must not subsequently call anything other than
+ the release callback on this object.
+
+.. c:member:: int (*ArrowAsyncDeviceStreamHandler.on_next_task)(struct ArrowAsyncDeviceStreamHandler*, struct ArrowAsyncTask*, const char*)
+
+ *Mandatory.* Handler to be called when a new record is available for processing. The
+ schema for each record should be the same as the schema that ``on_schema`` was called with.
+ If successfully handled, the function should return 0, otherwise it should return an
+ ``errno``-compatible error code.
+
+ Rather than passing the record itself it receives an ``ArrowAsyncTask`` instead to facilitate
+ better consumer-focused thread control as far as receiving the data. A call to this function
+ simply indicates that data is available via the provided task.
+
+ The ``const char*`` parameter exists for producers to provide any extra contextual information
+ they want. This is encoded in the same format as :c:member:`ArrowSchema.metadata`. If not ``NULL``,
+ the lifetime is only the scope of the call to this function. A consumer who wants to maintain
+ the additional metadata beyond the lifetime of this call *MUST* copy the value themselves.
+
+ A producer *MUST NOT* call this concurrently from multiple different threads.
+
+ The :c:member:`ArrowAsyncProducer.request` callback must be called to start receiving calls to this
+ handler.
+
+ This function *MUST* be able to be called re-entrantly on the same thread to allow for the
+ :c:member:`ArrowAsyncProducer.request` callback to potentially call this method recursively.
+
+.. c:member:: void (*ArrowAsyncDeviceStreamHandler.on_error)(struct ArrowAsyncDeviceStreamHandler, int, const char*, const char*)
+
+ *Mandatory.* Handler to be called when an error is encountered by the producer. After calling
+ this, the ``release`` callback will be called as the last call on this struct. The parameters
+ are an ``errno``-compatible error code and an optional error message and metadata.
+
+ If the message and metadata are not ``NULL``, their lifetime is only valid during the scope
+ of this call. A consumer who wants to maintain these values past the return of this function
+ *MUST* copy the values themselves.
+
+ If the metadata parameter is not ``NULL``, to provide key-value error metadata, then it should
+ be encoded identically to the way that metadata is encoded in :c:member:`ArrowSchema.metadata`.
+
+ It is valid for this to be called by a producer with or without a preceding call to
+ :c:member:`ArrowAsyncProducer.request`. This callback *MUST NOT* call any methods of an
+ ``ArrowAsyncProducer`` object.
+
+.. c:member:: void (*ArrowAsyncDeviceStreamHandler.release)(struct ArrowAsyncDeviceStreamHandler*)
+
+ *Mandatory.* A pointer to a consumer-provided release callback for the handler.
+
+ It is valid for this to be called by a producer with or without a preceding call to
+ :c:member:`ArrowAsyncProducer.request`. This must not call any methods of an ``ArrowAsyncProducer``
+ object.
+
+.. c:member:: void* ArrowAsyncDeviceStreamHandler.private_data
+
+ *Optional.* An opaque pointer to consumer-provided private data.
+
+ Producers *MUST NOT* process this member. Lifetime of this member is handled by
+ the consumer, and especially by the release callback.
+
+The ArrowAsyncTask structure
+''''''''''''''''''''''''''''
+
+The purpose of using a Task object rather than passing the array directly to the ``on_next``
+callback is to allow for more complex and efficient thread handling. Utilizing a Task
+object allows for a producer to separate the "decoding" logic from the I/O, enabling a
+consumer to avoid transferring data between CPU cores (e.g. from one L1/L2 cache to another).
+
+This producer-provided structure has the following fields:
+
+.. c:member:: int (*ArrowArrayTask.get_data)(struct ArrowArrayTask*, struct ArrowDeviceArray*)
+
+ *Mandatory.* A callback to populate the provided ``ArrowDeviceArray`` with the available data.
+ The order of ArrowAsyncTasks provided by the producer enables a consumer to know the order of
+ the data to process.
+
+ If a non-zero value is returned from this, it should be followed only by the producer calling
+ the ``on_error`` callback of the ``ArrowAsyncDeviceStreamHandler``. Because calling this method
+ is likely to be separate from the current control flow, returning a non-zero value to signal
+ an error occuring allows the current thread to decide handle the case accordingly, while still
+ allowing all error logging and handling to be centralized in the
+ :c:member:`ArrowAsyncDeviceStreamHandler.on_error` callback.
+
+ Rather than having a separate release callback, any required cleanup should be performed as part
+ of the invocation of this callback, with ownership of the Array being passed to the consumer-provided
+ parameter calling this, and must be released separately.
+
+ It is only valid to call this method exactly once.
+
+.. c:member:: void* ArrowArrayTask.private_data
+
+ *Optional.* An opaque pointer to consumer-provided private data.
+
+ Consumers *MUST NOT* process this member. Lifetime of this member is handled by
+ the producer who created this object, and should be cleaned up if necessary during
+ the call to :c:member:`ArrowArrayTask.get_data`.
+
+The ArrowAsyncProducer structure
+''''''''''''''''''''''''''''''''
+
+This producer-provided and managed object has the following fields:
+
+.. c:member:: void (*ArrowAsyncProducer.request)(struct ArrowAsyncProducer*, uint64_t)
+
+ *Mandatory.* This function must be called by a consumer to start receiving calls to
+ :c:member:`ArrowAsyncDeviceStreamHandler.on_next_task`. It *MUST* be valid to call
+ this synchronously from within :c:member:`ArrowAsyncDeviceStreamHandler.on_next_task`
+ or :c:member:`ArrowAsyncDeviceStreamHandler.on_schema`. As a result, this function
+ *MUST NOT* synchronously call ``on_next_task`` or ``on_error`` to avoid recursive
+ and reentrant callbacks.
+
+ After ``cancel`` is called, additional calls to this function must be a NOP, but allowed.
+
+ While not cancelled, calling this function registers the given number of additional
+ arrays/batches to be produced by the producer. A producer should only call
+ the appropriate ``on_next_task`` callback up to a maximum of the total sum of calls to
+ this method before propagating back-pressure / waiting.
+
+ Any error encountered by calling request must be propagated by calling the ``on_error``
+ callback of the ``ArrowAsyncDeviceStreamHandler``.
+
+ A producer *MUST* support an unbounded number of calls to this callback and *MUST* support
+ a total registered demand (sum requested - sum delivered) of up to ``UINT64_MAX``.
Review Comment:
Uh, what? A producer can't place a smaller upper limit? Why is that?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-43631: [C][Format] Add ArrowAsyncDeviceStreamHandler interface [arrow]
Posted by "pitrou (via GitHub)" <gi...@apache.org>.
pitrou commented on code in PR #43632:
URL: https://github.com/apache/arrow/pull/43632#discussion_r1757487825
##########
docs/source/format/CDeviceDataInterface.rst:
##########
@@ -650,6 +652,331 @@ The stream source is not assumed to be thread-safe. Consumers wanting to
call ``get_next`` from several threads should ensure those calls are
serialized.
+Async Device Stream Interface
+=============================
+
+The :ref:`C stream interface <_c-device-stream-interface>` provides a synchronous
+API centered around the consumer calling the callback functions to retrieve
+the next record batch. For some bindings, use cases, and interoperability, a more
+asynchronous, producer-focused interface may be required. These scenarios can utilize
+the ``ArrowAsyncDeviceStreamHandler``.
+
+Semantics
+---------
+
+Rather than the producer providing a structure of callbacks for a consumer to
+call and retrieve records, the Async interface is a consumer allocated structure.
+The consumer allocated struct provides handler callbacks for the producer to call
+when the schema and chunks of data are available, rather than the consumer using
+a blocking pull-style iteration.
+
+In addition to the ``ArrowAsyncDeviceStreamHandler``, there are also two additional
+structs used for the full data flow: ``ArrowAsyncTask`` and ``ArrowAsyncProducer``.
+
+Structure Definition
+--------------------
+
+The C device async stream interface is defined with a single ``struct`` definition:
+
+.. code-block:: c
+
+ #ifndef ARROW_C_ASYNC_STREAM_INTERFACE
+ #define ARROW_C_ASYNC_STREAM_INTERFACE
+
+ struct ArrowAsyncTask {
+ int (*get_data)(struct ArrowArrayTask* self, struct ArrowDeviceArray* out);
+
+ void* private_data;
+ };
+
+ struct ArrowAsyncProducer {
+ void (*request)(struct ArrowAsyncProducer* self, uint64_t n);
+ void (*cancel)(struct ArrowAsyncProducer* self);
+
+ void (*release)(struct ArrowAsyncProducer* self);
+ void* private_data;
+ };
+
+ struct ArrowAsyncDeviceStreamHandler {
+ // handlers
+ int (*on_schema)(struct ArrowAsyncDeviceStreamHandler* self,
+ struct ArrowAsyncProducer* producer,
+ struct ArrowSchema* stream_schema, const char* addl_metadata);
+ int (*on_next_task)(struct ArrowAsyncDeviceStreamHandler* self,
+ struct ArrowAsyncTask* task, const char* metadata);
+ void (*on_error)(struct ArrowAsyncDeviceStreamHandler* self,
+ int code, const char* message, const char* metadata);
+
+ // release callback
+ void (*release)(struct ArrowAsyncDeviceStreamHandler* self);
+
+ // opaque handler-specific data
+ void* private_data;
+ };
+
+ #endif // ARROW_C_ASYNC_STREAM_INTERFACE
+
+.. note::
+ The canonical guard ``ARROW_C_ASYNC_STREAM_INTERFACE`` is meant to avoid
+ duplicate definitions if two projects copy the C async stream interface
+ definitions into their own headers, and a third-party project includes
+ from these two projects. It is therefore important that this guard is kept
+ exactly as-is when these definitions are copied.
+
+The ArrowAsyncDeviceStreamHandler structure
+'''''''''''''''''''''''''''''''''''''''''''
+
+The structure has the following fields:
+
+.. c:member:: int (*ArrowAsyncDeviceStreamHandler.on_schema)(struct ArrowAsyncDeviceStreamHandler*, struct ArrowAsyncProducer*, struct ArrowSchema*, const char*)
+
+ *Mandatory.* Handler for receiving the schema of the stream. All records should
+ match the provided schema. If successful, the function should return 0, otherwise
+ it should return an ``errno``-compatible error code.
+
+ The ``const char*`` parameter exists for producers to provide any extra contextual information
+ they want, such as the total number of rows in the stream, statistics, or otherwise. This is
+ encoded in the same format as :c:member:`ArrowSchema.metadata`. If not ``NULL``,
+ the lifetime is only the scope of the call to this function. A consumer who wants to maintain
+ the additional metadata beyond the lifetime of this call *MUST* copy the value themselves.
+
+ Unless the ``on_error`` handler is called, this will always get called exactly once and will be
+ the first method called on this object. As such the producer *MUST* provide an ``ArrowAsyncProducer``
+ object when calling this function to allow the consumer to manage back-pressure and flow control.
+ The producer maintains ownership of the ``ArrowAsyncProducer`` and must clean it up before or after
+ calling the release callback on this object.
+
+ A producer that receives a non-zero result here must not subsequently call anything other than
+ the release callback on this object.
+
+.. c:member:: int (*ArrowAsyncDeviceStreamHandler.on_next_task)(struct ArrowAsyncDeviceStreamHandler*, struct ArrowAsyncTask*, const char*)
+
+ *Mandatory.* Handler to be called when a new record is available for processing. The
+ schema for each record should be the same as the schema that ``on_schema`` was called with.
+ If successfully handled, the function should return 0, otherwise it should return an
+ ``errno``-compatible error code.
+
+ Rather than passing the record itself it receives an ``ArrowAsyncTask`` instead to facilitate
+ better consumer-focused thread control as far as receiving the data. A call to this function
+ simply indicates that data is available via the provided task.
+
+ The ``const char*`` parameter exists for producers to provide any extra contextual information
+ they want. This is encoded in the same format as :c:member:`ArrowSchema.metadata`. If not ``NULL``,
+ the lifetime is only the scope of the call to this function. A consumer who wants to maintain
+ the additional metadata beyond the lifetime of this call *MUST* copy the value themselves.
+
+ A producer *MUST NOT* call this concurrently from multiple different threads.
+
+ The :c:member:`ArrowAsyncProducer.request` callback must be called to start receiving calls to this
+ handler.
+
+ This function *MUST* be able to be called re-entrantly on the same thread to allow for the
+ :c:member:`ArrowAsyncProducer.request` callback to potentially call this method recursively.
+
+.. c:member:: void (*ArrowAsyncDeviceStreamHandler.on_error)(struct ArrowAsyncDeviceStreamHandler, int, const char*, const char*)
+
+ *Mandatory.* Handler to be called when an error is encountered by the producer. After calling
+ this, the ``release`` callback will be called as the last call on this struct. The parameters
+ are an ``errno``-compatible error code and an optional error message and metadata.
+
+ If the message and metadata are not ``NULL``, their lifetime is only valid during the scope
+ of this call. A consumer who wants to maintain these values past the return of this function
+ *MUST* copy the values themselves.
+
+ If the metadata parameter is not ``NULL``, to provide key-value error metadata, then it should
+ be encoded identically to the way that metadata is encoded in :c:member:`ArrowSchema.metadata`.
+
+ It is valid for this to be called by a producer with or without a preceding call to
+ :c:member:`ArrowAsyncProducer.request`. This callback *MUST NOT* call any methods of an
+ ``ArrowAsyncProducer`` object.
+
+.. c:member:: void (*ArrowAsyncDeviceStreamHandler.release)(struct ArrowAsyncDeviceStreamHandler*)
+
+ *Mandatory.* A pointer to a consumer-provided release callback for the handler.
+
+ It is valid for this to be called by a producer with or without a preceding call to
+ :c:member:`ArrowAsyncProducer.request`. This must not call any methods of an ``ArrowAsyncProducer``
+ object.
+
+.. c:member:: void* ArrowAsyncDeviceStreamHandler.private_data
+
+ *Optional.* An opaque pointer to consumer-provided private data.
+
+ Producers *MUST NOT* process this member. Lifetime of this member is handled by
+ the consumer, and especially by the release callback.
+
Review Comment:
Also, how does a producer signal that the stream has ended? Shouldn't there be a dedicated callback on `ArrowAsyncDeviceStreamHandler`?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-43631: [C][Format] Add ArrowAsyncDeviceStreamHandler interface [arrow]
Posted by "pitrou (via GitHub)" <gi...@apache.org>.
pitrou commented on code in PR #43632:
URL: https://github.com/apache/arrow/pull/43632#discussion_r1757470498
##########
docs/source/format/CDeviceDataInterface.rst:
##########
@@ -650,6 +652,331 @@ The stream source is not assumed to be thread-safe. Consumers wanting to
call ``get_next`` from several threads should ensure those calls are
serialized.
+Async Device Stream Interface
+=============================
+
+The :ref:`C stream interface <_c-device-stream-interface>` provides a synchronous
+API centered around the consumer calling the callback functions to retrieve
+the next record batch. For some bindings, use cases, and interoperability, a more
+asynchronous, producer-focused interface may be required. These scenarios can utilize
+the ``ArrowAsyncDeviceStreamHandler``.
+
+Semantics
+---------
+
+Rather than the producer providing a structure of callbacks for a consumer to
+call and retrieve records, the Async interface is a consumer allocated structure.
+The consumer allocated struct provides handler callbacks for the producer to call
+when the schema and chunks of data are available, rather than the consumer using
+a blocking pull-style iteration.
+
+In addition to the ``ArrowAsyncDeviceStreamHandler``, there are also two additional
+structs used for the full data flow: ``ArrowAsyncTask`` and ``ArrowAsyncProducer``.
+
+Structure Definition
+--------------------
+
+The C device async stream interface is defined with a single ``struct`` definition:
+
+.. code-block:: c
+
+ #ifndef ARROW_C_ASYNC_STREAM_INTERFACE
+ #define ARROW_C_ASYNC_STREAM_INTERFACE
+
+ struct ArrowAsyncTask {
+ int (*get_data)(struct ArrowArrayTask* self, struct ArrowDeviceArray* out);
+
+ void* private_data;
+ };
+
+ struct ArrowAsyncProducer {
+ void (*request)(struct ArrowAsyncProducer* self, uint64_t n);
+ void (*cancel)(struct ArrowAsyncProducer* self);
+
+ void (*release)(struct ArrowAsyncProducer* self);
+ void* private_data;
+ };
+
+ struct ArrowAsyncDeviceStreamHandler {
+ // handlers
+ int (*on_schema)(struct ArrowAsyncDeviceStreamHandler* self,
+ struct ArrowAsyncProducer* producer,
+ struct ArrowSchema* stream_schema, const char* addl_metadata);
+ int (*on_next_task)(struct ArrowAsyncDeviceStreamHandler* self,
+ struct ArrowAsyncTask* task, const char* metadata);
+ void (*on_error)(struct ArrowAsyncDeviceStreamHandler* self,
+ int code, const char* message, const char* metadata);
+
+ // release callback
+ void (*release)(struct ArrowAsyncDeviceStreamHandler* self);
+
+ // opaque handler-specific data
+ void* private_data;
+ };
+
+ #endif // ARROW_C_ASYNC_STREAM_INTERFACE
+
+.. note::
+ The canonical guard ``ARROW_C_ASYNC_STREAM_INTERFACE`` is meant to avoid
+ duplicate definitions if two projects copy the C async stream interface
+ definitions into their own headers, and a third-party project includes
+ from these two projects. It is therefore important that this guard is kept
+ exactly as-is when these definitions are copied.
+
+The ArrowAsyncDeviceStreamHandler structure
+'''''''''''''''''''''''''''''''''''''''''''
+
+The structure has the following fields:
+
+.. c:member:: int (*ArrowAsyncDeviceStreamHandler.on_schema)(struct ArrowAsyncDeviceStreamHandler*, struct ArrowAsyncProducer*, struct ArrowSchema*, const char*)
+
+ *Mandatory.* Handler for receiving the schema of the stream. All records should
+ match the provided schema. If successful, the function should return 0, otherwise
+ it should return an ``errno``-compatible error code.
+
+ The ``const char*`` parameter exists for producers to provide any extra contextual information
+ they want, such as the total number of rows in the stream, statistics, or otherwise. This is
+ encoded in the same format as :c:member:`ArrowSchema.metadata`. If not ``NULL``,
+ the lifetime is only the scope of the call to this function. A consumer who wants to maintain
+ the additional metadata beyond the lifetime of this call *MUST* copy the value themselves.
+
+ Unless the ``on_error`` handler is called, this will always get called exactly once and will be
+ the first method called on this object. As such the producer *MUST* provide an ``ArrowAsyncProducer``
+ object when calling this function to allow the consumer to manage back-pressure and flow control.
+ The producer maintains ownership of the ``ArrowAsyncProducer`` and must clean it up before or after
+ calling the release callback on this object.
+
+ A producer that receives a non-zero result here must not subsequently call anything other than
+ the release callback on this object.
+
+.. c:member:: int (*ArrowAsyncDeviceStreamHandler.on_next_task)(struct ArrowAsyncDeviceStreamHandler*, struct ArrowAsyncTask*, const char*)
+
+ *Mandatory.* Handler to be called when a new record is available for processing. The
+ schema for each record should be the same as the schema that ``on_schema`` was called with.
+ If successfully handled, the function should return 0, otherwise it should return an
+ ``errno``-compatible error code.
+
+ Rather than passing the record itself it receives an ``ArrowAsyncTask`` instead to facilitate
+ better consumer-focused thread control as far as receiving the data. A call to this function
Review Comment:
Why does it facilitate enough in that regard? This only seems to complicate things: the data is available already, why not give it already to the consumer?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-43631: [C][Format] Add ArrowAsyncDeviceStreamHandler interface [arrow]
Posted by "ianmcook (via GitHub)" <gi...@apache.org>.
ianmcook commented on PR #43632:
URL: https://github.com/apache/arrow/pull/43632#issuecomment-2433471180
@github-actions crossbow submit preview-docs
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-43631: [C][Format] Add ArrowAsyncDeviceStreamHandler interface [arrow]
Posted by "zeroshade (via GitHub)" <gi...@apache.org>.
zeroshade commented on code in PR #43632:
URL: https://github.com/apache/arrow/pull/43632#discussion_r1824635253
##########
cpp/src/arrow/c/abi.h:
##########
@@ -356,8 +356,7 @@ struct ArrowAsyncDeviceStreamHandler {
// A producer that receives a non-zero return here should stop producing and eventually
// call release instead.
int (*on_schema)(struct ArrowAsyncDeviceStreamHandler* self,
- struct ArrowAsyncProducer* producer, struct ArrowSchema* stream_schema,
- const char* addl_metadata);
+ struct ArrowSchema* stream_schema, const char* addl_metadata);
Review Comment:
I might be amenable to this idea, though I'm still unsure about forcing the keeping of the metadata around.
Using the suggestion from @bkietz that answers the question of lifetime and when it's valid/invalid. What do you think @lidavidm?
this is the last question in play before we can merge this :)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-43631: [C][Format] Add ArrowAsyncDeviceStreamHandler interface [arrow]
Posted by "lidavidm (via GitHub)" <gi...@apache.org>.
lidavidm commented on code in PR #43632:
URL: https://github.com/apache/arrow/pull/43632#discussion_r1830217290
##########
cpp/src/arrow/c/abi.h:
##########
@@ -319,6 +319,11 @@ struct ArrowAsyncProducer {
// on_error callback on the async stream handler.
void (*cancel)(struct ArrowAsyncProducer* self);
+ // Any additional metadata tied to a specific stream of data. This must either be NULL
+ // or a valid pointer to metadata which is encoded in the same way schema metadata
+ // would be. Non-null metadata must be valid for the lifetime of this object.
Review Comment:
nit: can we keep the context about what kind of metadata might be useful to pass, and the encoding of the string?
##########
cpp/src/arrow/c/abi.h:
##########
@@ -319,6 +319,11 @@ struct ArrowAsyncProducer {
// on_error callback on the async stream handler.
void (*cancel)(struct ArrowAsyncProducer* self);
+ // Any additional metadata tied to a specific stream of data. This must either be NULL
+ // or a valid pointer to metadata which is encoded in the same way schema metadata
+ // would be. Non-null metadata must be valid for the lifetime of this object.
Review Comment:
Also we should note it may only be valid after the schema callback?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-43631: [C][Format] Add ArrowAsyncDeviceStreamHandler interface [arrow]
Posted by "zeroshade (via GitHub)" <gi...@apache.org>.
zeroshade commented on code in PR #43632:
URL: https://github.com/apache/arrow/pull/43632#discussion_r1830566926
##########
cpp/src/arrow/c/abi.h:
##########
@@ -319,6 +319,11 @@ struct ArrowAsyncProducer {
// on_error callback on the async stream handler.
void (*cancel)(struct ArrowAsyncProducer* self);
+ // Any additional metadata tied to a specific stream of data. This must either be NULL
+ // or a valid pointer to metadata which is encoded in the same way schema metadata
+ // would be. Non-null metadata must be valid for the lifetime of this object.
Review Comment:
This entire struct must be valid *before* the `on_schema` call, not after.
I'll add the comment about the context etc.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-43631: [C][Format] Add ArrowAsyncDeviceStreamHandler interface [arrow]
Posted by "westonpace (via GitHub)" <gi...@apache.org>.
westonpace commented on code in PR #43632:
URL: https://github.com/apache/arrow/pull/43632#discussion_r1712642429
##########
cpp/src/arrow/c/abi.h:
##########
@@ -228,6 +228,65 @@ struct ArrowDeviceArrayStream {
#endif // ARROW_C_DEVICE_STREAM_INTERFACE
+#ifndef ARROW_C_ASYNC_STREAM_INTERFACE
+#define ARROW_C_ASYNC_STREAM_INTERFACE
+
+// Similar to ArrowDeviceArrayStream, except designed for an asynchronous
+// style of interaction. While ArrowDeviceArrayStream provides producer
+// defined callbacks, this is intended to be created by the consumer instead.
+// The consumer passes this handler to the producer, which in turn uses the
+// callbacks to inform the consumer of events in the stream.
+struct ArrowAsyncDeviceStreamHandler {
+ // Handler for receiving a schema. The passed in stream_schema should be
+ // released or moved by the handler (producer is giving ownership of it to
+ // the handler).
+ //
+ // The `extension_param` argument can be null or can be used by a producer
+ // to pass arbitrary extra information to the consumer (such as total number
+ // of rows, context info, or otherwise).
+ //
+ // Return value: 0 if successful, `errno`-compatible error otherwise
+ int (*on_schema)(struct ArrowAsyncDeviceStreamHandler* self,
+ struct ArrowSchema* stream_schema, void* extension_param);
+
+ // Handler for receiving an array/record batch. Always called at least once
+ // unless an error is encountered (which would result in calling on_error).
+ // An empty/released array is passed to indicate the end of the stream if no
+ // errors have been encountered.
+ //
+ // The `extension_param` argument can be null or can be used by a producer
+ // to pass arbitrary extra information to the consumer.
+ //
+ // Return value: 0 if successful, `errno`-compatible error otherwise.
+ int (*on_next)(struct ArrowAsyncDeviceStreamHandler* self,
+ struct ArrowDeviceArray* next, void* extension_param);
+
+ // Handler for encountering an error. The producer should call release after
+ // this returns to clean up any resources.
+ //
+ // If the message or metadata are non-null, they will only last as long as this
+ // function call. The consumer would need to perform a copy of the data if it is
+ // it is necessary for them live past the lifetime of this call.
+ //
+ // Error metadata should be encoded as with metadata in ArrowSchema, defined in
+ // the spec at
+ // https://arrow.apache.org/docs/format/CDataInterface.html#c.ArrowSchema.metadata
+ //
+ // After this call, producers should follow-up by calling the release callback.
+ void (*on_error)(struct ArrowAsyncDeviceStreamHandler* self, int code,
+ const char* message, const char* metadata);
+
+ // Release callback to release any resources for the handler. Should always be
+ // called by a producer when it is done utilizing a handler. No callbacks should
+ // be called after this is called.
+ void (*release)(struct ArrowAsyncDeviceStreamHandler* self);
Review Comment:
What should the consumer do if there are calls to `on_error` or `on_next` that are still in progress when this is called?
##########
cpp/src/arrow/c/abi.h:
##########
@@ -228,6 +228,65 @@ struct ArrowDeviceArrayStream {
#endif // ARROW_C_DEVICE_STREAM_INTERFACE
+#ifndef ARROW_C_ASYNC_STREAM_INTERFACE
+#define ARROW_C_ASYNC_STREAM_INTERFACE
+
+// Similar to ArrowDeviceArrayStream, except designed for an asynchronous
+// style of interaction. While ArrowDeviceArrayStream provides producer
+// defined callbacks, this is intended to be created by the consumer instead.
+// The consumer passes this handler to the producer, which in turn uses the
+// callbacks to inform the consumer of events in the stream.
+struct ArrowAsyncDeviceStreamHandler {
+ // Handler for receiving a schema. The passed in stream_schema should be
+ // released or moved by the handler (producer is giving ownership of it to
+ // the handler).
+ //
+ // The `extension_param` argument can be null or can be used by a producer
+ // to pass arbitrary extra information to the consumer (such as total number
+ // of rows, context info, or otherwise).
+ //
+ // Return value: 0 if successful, `errno`-compatible error otherwise
+ int (*on_schema)(struct ArrowAsyncDeviceStreamHandler* self,
+ struct ArrowSchema* stream_schema, void* extension_param);
+
+ // Handler for receiving an array/record batch. Always called at least once
+ // unless an error is encountered (which would result in calling on_error).
+ // An empty/released array is passed to indicate the end of the stream if no
+ // errors have been encountered.
+ //
+ // The `extension_param` argument can be null or can be used by a producer
+ // to pass arbitrary extra information to the consumer.
+ //
+ // Return value: 0 if successful, `errno`-compatible error otherwise.
+ int (*on_next)(struct ArrowAsyncDeviceStreamHandler* self,
+ struct ArrowDeviceArray* next, void* extension_param);
Review Comment:
You're explicit about ownership elsewhere so it might be nice to just add a statement "after this call the consumer is responsible for releasing the array provided by `next`"
##########
cpp/src/arrow/c/abi.h:
##########
@@ -228,6 +228,65 @@ struct ArrowDeviceArrayStream {
#endif // ARROW_C_DEVICE_STREAM_INTERFACE
+#ifndef ARROW_C_ASYNC_STREAM_INTERFACE
+#define ARROW_C_ASYNC_STREAM_INTERFACE
+
+// Similar to ArrowDeviceArrayStream, except designed for an asynchronous
+// style of interaction. While ArrowDeviceArrayStream provides producer
+// defined callbacks, this is intended to be created by the consumer instead.
+// The consumer passes this handler to the producer, which in turn uses the
+// callbacks to inform the consumer of events in the stream.
+struct ArrowAsyncDeviceStreamHandler {
+ // Handler for receiving a schema. The passed in stream_schema should be
+ // released or moved by the handler (producer is giving ownership of it to
+ // the handler).
+ //
+ // The `extension_param` argument can be null or can be used by a producer
+ // to pass arbitrary extra information to the consumer (such as total number
+ // of rows, context info, or otherwise).
+ //
+ // Return value: 0 if successful, `errno`-compatible error otherwise
Review Comment:
Callbacks returning an error like this is pretty common in Rust. Normally what the producer does is:
* Abort the reader
* Return an error "External error: the producer received error code <X> from a call to `on_schema`: <errno-description>"
I think it's nice to be able to return an error code here. For example, maybe the consumer can't handle one of the data types reported by the schema.
Either way though, if an error is returned or if the consumer explicitly cancels in some way, I assume the producer needs to keep accepting calls to the callbacks right? (e.g. returning an error here won't prevent `on_next` from being called once or twice while the producer is cleaning up)
##########
cpp/src/arrow/c/abi.h:
##########
@@ -228,6 +228,65 @@ struct ArrowDeviceArrayStream {
#endif // ARROW_C_DEVICE_STREAM_INTERFACE
+#ifndef ARROW_C_ASYNC_STREAM_INTERFACE
+#define ARROW_C_ASYNC_STREAM_INTERFACE
+
+// Similar to ArrowDeviceArrayStream, except designed for an asynchronous
+// style of interaction. While ArrowDeviceArrayStream provides producer
+// defined callbacks, this is intended to be created by the consumer instead.
+// The consumer passes this handler to the producer, which in turn uses the
+// callbacks to inform the consumer of events in the stream.
+struct ArrowAsyncDeviceStreamHandler {
+ // Handler for receiving a schema. The passed in stream_schema should be
+ // released or moved by the handler (producer is giving ownership of it to
+ // the handler).
+ //
+ // The `extension_param` argument can be null or can be used by a producer
Review Comment:
I read this as "the contents of `stream_schema` are only valid during the call. Copy off any information that you need. The `ArrowSchema` will be deleted by the producer at some point after the call completes." Or, to put it another way "the consumer shall not call `release` on the `ArrowSchema`"
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-43631: [C][Format] Add ArrowAsyncDeviceStreamHandler interface [arrow]
Posted by "pitrou (via GitHub)" <gi...@apache.org>.
pitrou commented on code in PR #43632:
URL: https://github.com/apache/arrow/pull/43632#discussion_r1718563592
##########
docs/source/format/CDeviceDataInterface.rst:
##########
@@ -650,6 +652,137 @@ The stream source is not assumed to be thread-safe. Consumers wanting to
call ``get_next`` from several threads should ensure those calls are
serialized.
+Async Device Stream Interface
+=============================
+
+The :ref:`C stream interface <_c-device-stream-interface>` provides a synchronous
+API centered around the consumer calling the callback functions to retrieve
+the next record batch. For some bindings, use cases, and interoperability, a more
+asynchronous, producer-focused interface may be required. These scenarios can utilize
+the ``ArrowAsyncDeviceStreamHandler``.
+
+Semantics
+---------
+
+Rather than the producer providing a structure of callbacks for a consumer to
+call and retrieve records, the Async interface is a consumer allocated structure.
+The consumer allocated struct provides handler callbacks for the producer to call
+when the schema and chunks of data are available, rather than the consumer using
+a blocking pull-style iteration.
+
+Structure Definition
+--------------------
+
+The C device async stream interface is defined with a single ``struct`` definition:
+
+.. code-block:: c
+
+ #ifndef ARROW_C_ASYNC_STREAM_INTERFACE
+ #define ARROW_C_ASYNC_STREAM_INTERFACE
+
+ struct ArrowAsyncDeviceStreamHandler {
+ // handlers
+ int (*on_schema)(struct ArrowAsyncDeviceStreamHandler* self,
+ struct ArrowSchema* stream_schema, void* extension_param);
+ int (*on_next)(struct ArrowAsyncDeviceStreamHandler* self,
+ struct ArrowDeviceArray* next, void* extension_param);
+ void (*on_error)(struct ArrowAsyncDeviceStreamHandler* self,
+ int code, const char* message, const char* metadata);
+
+ // release callback
+ void (*release)(struct ArrowAsyncDeviceStreamHandler* self);
+
+ // opaque handler-specific data
+ void* private_data;
+ };
+
+ #endif // ARROW_C_ASYNC_STREAM_INTERFACE
+
+.. note::
+ The canonical guard ``ARROW_C_ASYNC_STREAM_INTERFACE`` is meant to avoid
+ duplicate definitions if two projects copy the C async stream interface
+ definitions into their own headers, and a third-party project includes
+ from these two projects. It is therefore important that this guard is kept
+ exactly as-is when these definitions are copied.
+
+The ArrowAsyncDeviceStreamHandler structure
+'''''''''''''''''''''''''''''''''''''''''''
+
+The structure has the following fields:
+
+.. c:member:: int (*ArrowAsyncDeviceStreamHandler.on_schema)(struct ArrowAsyncDeviceStreamHandler*, struct ArrowSchema*, void*)
+
+ *Mandatory.* Handler for receiving the schema of the stream. All records should
+ match the provided schema. If successful, the function should return 0, otherwise
+ it should return an ``errno``-compatible error code.
+
+ The ``void*`` parameter exists for producers to provide any extra contextual information
+ they want, such as the total number of rows in the stream, statistics, or otherwise.
+
+ Unless the ``on_error`` handler is called, this will always get called exactly once.
+
+.. c:member:: int (*ArrowAsyncDeviceStreamHandler.on_next)(struct ArrowAsyncDeviceStreamHandler*, struct ArrowDeviceArray*, void*)
+
+ *Mandatory.* Handler to be called when a new record is available for processing. The
+ schema for each record should be the same as the schema that ``on_schema`` was called with.
+ If successfully handled, the function should return 0, otherwise it should return an
+ ``errno``-compatible error code.
+
+ The ``void*`` parameter exists for producers to provide any extra contextual information
+ they want.
+
+ Unless the ``on_error`` handler is called, this will always get called at least once. The
+ end of the stream is indicated by it being called with an empty or released ``ArrowDeviceArray``.
+
+.. c:member:: void (*ArrowAsyncDeviceStreamHandler.on_error)(struct ArrowAsyncDeviceStreamHandler, int, const char*, const char*)
+
+ *Mandatory.* Handler to be called when an error is encountered by the producer. After calling
+ this, the ``release`` callback will be called as the last call on this struct. The parameters
+ are an ``errno``-compatible error code and an optional error message and metadata.
+
+ If the message and metadata are not ``NULL``, their lifetime is only valid during the scope
+ of this call. A consumer who wants to maintain these values past the return of this function
+ *MUST* copy the values themselves.
+
+ If the metadata parameter is not ``NULL``, to provide key-value error metadata, then it should
+ be encoded identically to the way that metadata is encoded in :c:member:`ArrowSchema.metadata`.
+
+.. c:member:: void (*ArrowAsyncDeviceStreamHandler.release)(struct ArrowAsyncDeviceStreamHandler*)
+
+ *Mandatory.* A pointer to a consumer-provided release callback for the handler.
+
+.. c:member:: void* ArrowAsyncDeviceStreamHandler.private_data
+
+ *Optional.* An opaque pointer to consumer-provided private data.
+
+ Producers *MUST NOT* process this member. Lifetime of this member is handled by
+ the consumer, and especially by the release callback.
+
+Result lifetimes
Review Comment:
Should add a section about error handling as well? Errors can flow in both directions here, which is a bit more complex that the regular C stream interface.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-43631: [C][Format] Add ArrowAsyncDeviceStreamHandler interface [arrow]
Posted by "zeroshade (via GitHub)" <gi...@apache.org>.
zeroshade commented on code in PR #43632:
URL: https://github.com/apache/arrow/pull/43632#discussion_r1714031538
##########
cpp/src/arrow/c/abi.h:
##########
@@ -228,6 +228,65 @@ struct ArrowDeviceArrayStream {
#endif // ARROW_C_DEVICE_STREAM_INTERFACE
+#ifndef ARROW_C_ASYNC_STREAM_INTERFACE
+#define ARROW_C_ASYNC_STREAM_INTERFACE
+
+// Similar to ArrowDeviceArrayStream, except designed for an asynchronous
+// style of interaction. While ArrowDeviceArrayStream provides producer
+// defined callbacks, this is intended to be created by the consumer instead.
+// The consumer passes this handler to the producer, which in turn uses the
+// callbacks to inform the consumer of events in the stream.
+struct ArrowAsyncDeviceStreamHandler {
+ // Handler for receiving a schema. The passed in stream_schema should be
+ // released or moved by the handler (producer is giving ownership of it to
+ // the handler).
+ //
+ // The `extension_param` argument can be null or can be used by a producer
+ // to pass arbitrary extra information to the consumer (such as total number
+ // of rows, context info, or otherwise).
+ //
+ // Return value: 0 if successful, `errno`-compatible error otherwise
+ int (*on_schema)(struct ArrowAsyncDeviceStreamHandler* self,
+ struct ArrowSchema* stream_schema, void* extension_param);
+
+ // Handler for receiving an array/record batch. Always called at least once
+ // unless an error is encountered (which would result in calling on_error).
+ // An empty/released array is passed to indicate the end of the stream if no
+ // errors have been encountered.
+ //
+ // The `extension_param` argument can be null or can be used by a producer
+ // to pass arbitrary extra information to the consumer.
+ //
+ // Return value: 0 if successful, `errno`-compatible error otherwise.
+ int (*on_next)(struct ArrowAsyncDeviceStreamHandler* self,
+ struct ArrowDeviceArray* next, void* extension_param);
Review Comment:
I guess the question is how much complexity we want this to have and who should own that complexity.
As described, the intent was that a consumer doesn't need to handle concurrency by default because a producer should only call the `on_next` when the previous one returns. But in the scenario you describe, where the processing inside of `on_next` might be lengthy, I think it's fine for the consumer to have to make the decision as to whether or not it should create a new thread task (if it wants to handle things in parallel) or do it inside of `on_next` (and potentially tie up the producer, but that's basically just backpressure essentially).
I think this is a good balance and tradeoff of complexity vs ease of use.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-43631: [C][Format] Add ArrowAsyncDeviceStreamHandler interface [arrow]
Posted by "zeroshade (via GitHub)" <gi...@apache.org>.
zeroshade commented on code in PR #43632:
URL: https://github.com/apache/arrow/pull/43632#discussion_r1713968078
##########
cpp/src/arrow/c/abi.h:
##########
@@ -228,6 +228,65 @@ struct ArrowDeviceArrayStream {
#endif // ARROW_C_DEVICE_STREAM_INTERFACE
+#ifndef ARROW_C_ASYNC_STREAM_INTERFACE
+#define ARROW_C_ASYNC_STREAM_INTERFACE
+
+// Similar to ArrowDeviceArrayStream, except designed for an asynchronous
+// style of interaction. While ArrowDeviceArrayStream provides producer
+// defined callbacks, this is intended to be created by the consumer instead.
+// The consumer passes this handler to the producer, which in turn uses the
+// callbacks to inform the consumer of events in the stream.
+struct ArrowAsyncDeviceStreamHandler {
+ // Handler for receiving a schema. The passed in stream_schema should be
+ // released or moved by the handler (producer is giving ownership of it to
+ // the handler).
+ //
+ // The `extension_param` argument can be null or can be used by a producer
Review Comment:
I guess I should reword it a bit then, the intent was that the ownership of the `ArrowSchema` is being given to the consumer from the Producer, thus the consumer *MUST* call `release` on the `ArrowSchema` unless it needs to keep it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-43631: [C][Format] Add ArrowAsyncDeviceStreamHandler interface [arrow]
Posted by "leventov (via GitHub)" <gi...@apache.org>.
leventov commented on code in PR #43632:
URL: https://github.com/apache/arrow/pull/43632#discussion_r1721539510
##########
cpp/src/arrow/c/abi.h:
##########
@@ -228,6 +228,65 @@ struct ArrowDeviceArrayStream {
#endif // ARROW_C_DEVICE_STREAM_INTERFACE
+#ifndef ARROW_C_ASYNC_STREAM_INTERFACE
+#define ARROW_C_ASYNC_STREAM_INTERFACE
+
+// Similar to ArrowDeviceArrayStream, except designed for an asynchronous
+// style of interaction. While ArrowDeviceArrayStream provides producer
+// defined callbacks, this is intended to be created by the consumer instead.
+// The consumer passes this handler to the producer, which in turn uses the
+// callbacks to inform the consumer of events in the stream.
+struct ArrowAsyncDeviceStreamHandler {
+ // Handler for receiving a schema. The passed in stream_schema should be
+ // released or moved by the handler (producer is giving ownership of it to
+ // the handler).
+ //
+ // The `extension_param` argument can be null or can be used by a producer
+ // to pass arbitrary extra information to the consumer (such as total number
+ // of rows, context info, or otherwise).
+ //
+ // Return value: 0 if successful, `errno`-compatible error otherwise
Review Comment:
> Right, but it might make more sense to just have a unified cancellation mechanism. And again, what is the producer really supposed to do with the error code? It's just going to drop it on the floor.
I think it can be useful for monitoring on the middleware/transport level, if the API wraps communication with a remote server. See APPLICATION_ERROR in [RSocket protocol](https://rsocket.io/about/protocol/).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-43631: [C][Format] Add ArrowAsyncDeviceStreamHandler interface [arrow]
Posted by "zeroshade (via GitHub)" <gi...@apache.org>.
zeroshade commented on code in PR #43632:
URL: https://github.com/apache/arrow/pull/43632#discussion_r1771647516
##########
docs/source/format/CDeviceDataInterface.rst:
##########
@@ -650,6 +652,331 @@ The stream source is not assumed to be thread-safe. Consumers wanting to
call ``get_next`` from several threads should ensure those calls are
serialized.
+Async Device Stream Interface
+=============================
+
+The :ref:`C stream interface <_c-device-stream-interface>` provides a synchronous
+API centered around the consumer calling the callback functions to retrieve
+the next record batch. For some bindings, use cases, and interoperability, a more
+asynchronous, producer-focused interface may be required. These scenarios can utilize
+the ``ArrowAsyncDeviceStreamHandler``.
+
+Semantics
+---------
+
+Rather than the producer providing a structure of callbacks for a consumer to
+call and retrieve records, the Async interface is a consumer allocated structure.
+The consumer allocated struct provides handler callbacks for the producer to call
+when the schema and chunks of data are available, rather than the consumer using
+a blocking pull-style iteration.
+
+In addition to the ``ArrowAsyncDeviceStreamHandler``, there are also two additional
+structs used for the full data flow: ``ArrowAsyncTask`` and ``ArrowAsyncProducer``.
+
+Structure Definition
+--------------------
+
+The C device async stream interface is defined with a single ``struct`` definition:
+
+.. code-block:: c
+
+ #ifndef ARROW_C_ASYNC_STREAM_INTERFACE
+ #define ARROW_C_ASYNC_STREAM_INTERFACE
+
+ struct ArrowAsyncTask {
+ int (*get_data)(struct ArrowArrayTask* self, struct ArrowDeviceArray* out);
+
+ void* private_data;
+ };
+
+ struct ArrowAsyncProducer {
+ void (*request)(struct ArrowAsyncProducer* self, uint64_t n);
+ void (*cancel)(struct ArrowAsyncProducer* self);
+
+ void (*release)(struct ArrowAsyncProducer* self);
+ void* private_data;
+ };
+
+ struct ArrowAsyncDeviceStreamHandler {
+ // handlers
+ int (*on_schema)(struct ArrowAsyncDeviceStreamHandler* self,
+ struct ArrowAsyncProducer* producer,
+ struct ArrowSchema* stream_schema, const char* addl_metadata);
+ int (*on_next_task)(struct ArrowAsyncDeviceStreamHandler* self,
+ struct ArrowAsyncTask* task, const char* metadata);
+ void (*on_error)(struct ArrowAsyncDeviceStreamHandler* self,
+ int code, const char* message, const char* metadata);
+
+ // release callback
+ void (*release)(struct ArrowAsyncDeviceStreamHandler* self);
+
+ // opaque handler-specific data
+ void* private_data;
+ };
+
+ #endif // ARROW_C_ASYNC_STREAM_INTERFACE
+
+.. note::
+ The canonical guard ``ARROW_C_ASYNC_STREAM_INTERFACE`` is meant to avoid
+ duplicate definitions if two projects copy the C async stream interface
+ definitions into their own headers, and a third-party project includes
+ from these two projects. It is therefore important that this guard is kept
+ exactly as-is when these definitions are copied.
+
+The ArrowAsyncDeviceStreamHandler structure
+'''''''''''''''''''''''''''''''''''''''''''
+
+The structure has the following fields:
+
+.. c:member:: int (*ArrowAsyncDeviceStreamHandler.on_schema)(struct ArrowAsyncDeviceStreamHandler*, struct ArrowAsyncProducer*, struct ArrowSchema*, const char*)
+
+ *Mandatory.* Handler for receiving the schema of the stream. All records should
+ match the provided schema. If successful, the function should return 0, otherwise
+ it should return an ``errno``-compatible error code.
+
+ The ``const char*`` parameter exists for producers to provide any extra contextual information
+ they want, such as the total number of rows in the stream, statistics, or otherwise. This is
+ encoded in the same format as :c:member:`ArrowSchema.metadata`. If not ``NULL``,
+ the lifetime is only the scope of the call to this function. A consumer who wants to maintain
+ the additional metadata beyond the lifetime of this call *MUST* copy the value themselves.
+
+ Unless the ``on_error`` handler is called, this will always get called exactly once and will be
+ the first method called on this object. As such the producer *MUST* provide an ``ArrowAsyncProducer``
+ object when calling this function to allow the consumer to manage back-pressure and flow control.
+ The producer maintains ownership of the ``ArrowAsyncProducer`` and must clean it up before or after
+ calling the release callback on this object.
+
+ A producer that receives a non-zero result here must not subsequently call anything other than
+ the release callback on this object.
+
+.. c:member:: int (*ArrowAsyncDeviceStreamHandler.on_next_task)(struct ArrowAsyncDeviceStreamHandler*, struct ArrowAsyncTask*, const char*)
+
+ *Mandatory.* Handler to be called when a new record is available for processing. The
+ schema for each record should be the same as the schema that ``on_schema`` was called with.
+ If successfully handled, the function should return 0, otherwise it should return an
+ ``errno``-compatible error code.
+
+ Rather than passing the record itself it receives an ``ArrowAsyncTask`` instead to facilitate
+ better consumer-focused thread control as far as receiving the data. A call to this function
+ simply indicates that data is available via the provided task.
+
+ The ``const char*`` parameter exists for producers to provide any extra contextual information
+ they want. This is encoded in the same format as :c:member:`ArrowSchema.metadata`. If not ``NULL``,
+ the lifetime is only the scope of the call to this function. A consumer who wants to maintain
+ the additional metadata beyond the lifetime of this call *MUST* copy the value themselves.
+
+ A producer *MUST NOT* call this concurrently from multiple different threads.
+
+ The :c:member:`ArrowAsyncProducer.request` callback must be called to start receiving calls to this
+ handler.
+
+ This function *MUST* be able to be called re-entrantly on the same thread to allow for the
+ :c:member:`ArrowAsyncProducer.request` callback to potentially call this method recursively.
+
+.. c:member:: void (*ArrowAsyncDeviceStreamHandler.on_error)(struct ArrowAsyncDeviceStreamHandler, int, const char*, const char*)
+
+ *Mandatory.* Handler to be called when an error is encountered by the producer. After calling
+ this, the ``release`` callback will be called as the last call on this struct. The parameters
+ are an ``errno``-compatible error code and an optional error message and metadata.
+
+ If the message and metadata are not ``NULL``, their lifetime is only valid during the scope
+ of this call. A consumer who wants to maintain these values past the return of this function
+ *MUST* copy the values themselves.
+
+ If the metadata parameter is not ``NULL``, to provide key-value error metadata, then it should
+ be encoded identically to the way that metadata is encoded in :c:member:`ArrowSchema.metadata`.
+
+ It is valid for this to be called by a producer with or without a preceding call to
+ :c:member:`ArrowAsyncProducer.request`. This callback *MUST NOT* call any methods of an
+ ``ArrowAsyncProducer`` object.
+
+.. c:member:: void (*ArrowAsyncDeviceStreamHandler.release)(struct ArrowAsyncDeviceStreamHandler*)
+
+ *Mandatory.* A pointer to a consumer-provided release callback for the handler.
+
+ It is valid for this to be called by a producer with or without a preceding call to
+ :c:member:`ArrowAsyncProducer.request`. This must not call any methods of an ``ArrowAsyncProducer``
+ object.
+
+.. c:member:: void* ArrowAsyncDeviceStreamHandler.private_data
+
+ *Optional.* An opaque pointer to consumer-provided private data.
+
+ Producers *MUST NOT* process this member. Lifetime of this member is handled by
+ the consumer, and especially by the release callback.
+
+The ArrowAsyncTask structure
+''''''''''''''''''''''''''''
+
+The purpose of using a Task object rather than passing the array directly to the ``on_next``
+callback is to allow for more complex and efficient thread handling. Utilizing a Task
+object allows for a producer to separate the "decoding" logic from the I/O, enabling a
+consumer to avoid transferring data between CPU cores (e.g. from one L1/L2 cache to another).
Review Comment:
@pitrou any further comments here? Did we reach a consensus?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-43631: [C][Format] Add ArrowAsyncDeviceStreamHandler interface [arrow]
Posted by "felipecrv (via GitHub)" <gi...@apache.org>.
felipecrv commented on code in PR #43632:
URL: https://github.com/apache/arrow/pull/43632#discussion_r1767708644
##########
docs/source/format/CDeviceDataInterface.rst:
##########
@@ -650,6 +652,340 @@ The stream source is not assumed to be thread-safe. Consumers wanting to
call ``get_next`` from several threads should ensure those calls are
serialized.
+Async Device Stream Interface
+=============================
+
+The :ref:`C stream interface <_c-device-stream-interface>` provides a synchronous
+API centered around the consumer calling the callback functions to retrieve
+the next record batch. For some bindings, use cases, and interoperability, a more
+asynchronous, producer-focused interface may be required. These scenarios can utilize
+the ``ArrowAsyncDeviceStreamHandler``.
+
+Semantics
+---------
+
+Rather than the producer providing a structure of callbacks for a consumer to
+call and retrieve records, the Async interface is a consumer allocated structure.
+The consumer allocated struct provides handler callbacks for the producer to call
+when the schema and chunks of data are available, rather than the consumer using
+a blocking pull-style iteration.
+
+In addition to the ``ArrowAsyncDeviceStreamHandler``, there are also two additional
+structs used for the full data flow: ``ArrowAsyncTask`` and ``ArrowAsyncProducer``.
+
+Structure Definition
+--------------------
+
+The C device async stream interface is defined with a single ``struct`` definition:
+
+.. code-block:: c
+
+ #ifndef ARROW_C_ASYNC_STREAM_INTERFACE
+ #define ARROW_C_ASYNC_STREAM_INTERFACE
+
+ struct ArrowAsyncTask {
+ int (*get_data)(struct ArrowArrayTask* self, struct ArrowDeviceArray* out);
+
+ void* private_data;
+ };
+
+ struct ArrowAsyncProducer {
+ void (*request)(struct ArrowAsyncProducer* self, int64_t n);
+ void (*cancel)(struct ArrowAsyncProducer* self);
+
+ void (*release)(struct ArrowAsyncProducer* self);
+ void* private_data;
+ };
+
+ struct ArrowAsyncDeviceStreamHandler {
+ // handlers
+ int (*on_schema)(struct ArrowAsyncDeviceStreamHandler* self,
+ struct ArrowAsyncProducer* producer,
+ struct ArrowSchema* stream_schema, const char* addl_metadata);
+ int (*on_next_task)(struct ArrowAsyncDeviceStreamHandler* self,
+ struct ArrowAsyncTask* task, const char* metadata);
+ void (*on_error)(struct ArrowAsyncDeviceStreamHandler* self,
+ int code, const char* message, const char* metadata);
+
+ // release callback
+ void (*release)(struct ArrowAsyncDeviceStreamHandler* self);
+
+ // opaque handler-specific data
+ void* private_data;
+ };
+
+ #endif // ARROW_C_ASYNC_STREAM_INTERFACE
+
+.. note::
+ The canonical guard ``ARROW_C_ASYNC_STREAM_INTERFACE`` is meant to avoid
+ duplicate definitions if two projects copy the C async stream interface
+ definitions into their own headers, and a third-party project includes
+ from these two projects. It is therefore important that this guard is kept
+ exactly as-is when these definitions are copied.
+
+The ArrowAsyncDeviceStreamHandler structure
+'''''''''''''''''''''''''''''''''''''''''''
+
+The structure has the following fields:
+
+.. c:member:: int (*ArrowAsyncDeviceStreamHandler.on_schema)(struct ArrowAsyncDeviceStreamHandler*, struct ArrowAsyncProducer*, struct ArrowSchema*, const char*)
+
+ *Mandatory.* Handler for receiving the schema of the stream. All records should
+ match the provided schema. If successful, the function should return 0, otherwise
+ it should return an ``errno``-compatible error code.
+
+ The ``const char*`` parameter exists for producers to provide any extra contextual information
+ they want, such as the total number of rows in the stream, statistics, or otherwise. This is
+ encoded in the same format as :c:member:`ArrowSchema.metadata`. If not ``NULL``,
+ the lifetime is only the scope of the call to this function. A consumer who wants to maintain
+ the additional metadata beyond the lifetime of this call *MUST* copy the value themselves.
+
+ Unless the ``on_error`` handler is called, this will always get called exactly once and will be
+ the first method called on this object. As such the producer *MUST* provide an ``ArrowAsyncProducer``
+ object when calling this function to allow the consumer to manage back-pressure and flow control.
+ The producer maintains ownership of the ``ArrowAsyncProducer`` and must clean it up *after*
+ calling the release callback on this object.
+
+ A producer that receives a non-zero result here must not subsequently call anything other than
+ the release callback on this object.
+
+.. c:member:: int (*ArrowAsyncDeviceStreamHandler.on_next_task)(struct ArrowAsyncDeviceStreamHandler*, struct ArrowAsyncTask*, const char*)
+
+ *Mandatory.* Handler to be called when a new record is available for processing. The
+ schema for each record should be the same as the schema that ``on_schema`` was called with.
+ If successfully handled, the function should return 0, otherwise it should return an
+ ``errno``-compatible error code.
+
+ Rather than passing the record itself it receives an ``ArrowAsyncTask`` instead to facilitate
+ better consumer-focused thread control as far as receiving the data. A call to this function
+ simply indicates that data is available via the provided task.
+
+ The producer signals the end of the stream by passing ``NULL`` for the ``ArrowAsyncTask``
+ pointer instead of a valid address. This task object is only valid during the lifetime of
+ this function call. If the consumer wants to use the task beyond the scope of this method, it
+ must copy or move its contents to a new ArrowAsyncTask object.
+
+ The ``const char*`` parameter exists for producers to provide any extra contextual information
+ they want. This is encoded in the same format as :c:member:`ArrowSchema.metadata`. If not ``NULL``,
+ the lifetime is only the scope of the call to this function. A consumer who wants to maintain
+ the additional metadata beyond the lifetime of this call *MUST* copy the value themselves.
+
+ A producer *MUST NOT* call this concurrently from multiple threads.
+
+ The :c:member:`ArrowAsyncProducer.request` callback must be called to start receiving calls to this
+ handler.
+
+.. c:member:: void (*ArrowAsyncDeviceStreamHandler.on_error)(struct ArrowAsyncDeviceStreamHandler, int, const char*, const char*)
+
+ *Mandatory.* Handler to be called when an error is encountered by the producer. After calling
+ this, the ``release`` callback will be called as the last call on this struct. The parameters
+ are an ``errno``-compatible error code and an optional error message and metadata.
+
+ If the message and metadata are not ``NULL``, their lifetime is only valid during the scope
+ of this call. A consumer who wants to maintain these values past the return of this function
+ *MUST* copy the values themselves.
+
+ If the metadata parameter is not ``NULL``, to provide key-value error metadata, then it should
+ be encoded identically to the way that metadata is encoded in :c:member:`ArrowSchema.metadata`.
+
+ It is valid for this to be called by a producer with or without a preceding call to
+ :c:member:`ArrowAsyncProducer.request`. This callback *MUST NOT* call any methods of an
+ ``ArrowAsyncProducer`` object.
+
+.. c:member:: void (*ArrowAsyncDeviceStreamHandler.release)(struct ArrowAsyncDeviceStreamHandler*)
+
+ *Mandatory.* A pointer to a consumer-provided release callback for the handler.
+
+ It is valid for this to be called by a producer with or without a preceding call to
+ :c:member:`ArrowAsyncProducer.request`. This must not call any methods of an ``ArrowAsyncProducer``
+ object.
+
+.. c:member:: void* ArrowAsyncDeviceStreamHandler.private_data
+
+ *Optional.* An opaque pointer to consumer-provided private data.
+
+ Producers *MUST NOT* process this member. Lifetime of this member is handled by
+ the consumer, and especially by the release callback.
+
+The ArrowAsyncTask structure
+''''''''''''''''''''''''''''
+
+The purpose of using a Task object rather than passing the array directly to the ``on_next``
+callback is to allow for more complex and efficient thread handling. Utilizing a Task
+object allows for a producer to separate the "decoding" logic from the I/O, enabling a
+consumer to avoid transferring data between CPU cores (e.g. from one L1/L2 cache to another).
+
+This producer-provided structure has the following fields:
+
+.. c:member:: int (*ArrowArrayTask.get_data)(struct ArrowArrayTask*, struct ArrowDeviceArray*)
+
+ *Mandatory.* A callback to populate the provided ``ArrowDeviceArray`` with the available data.
+ The order of ArrowAsyncTasks provided by the producer enables a consumer to know the order of
+ the data to process. If the consumer does not care about the data that is owned by this task,
+ it must still call ``get_data`` so that the producer can perform any required cleanup. ``NULL``
+ should be passed as the device array pointer to indicate that the consumer doesn't want the
+ actual data, letting the task perform necessary cleanup.
+
+ If a non-zero value is returned from this, it should be followed only by the producer calling
+ the ``on_error`` callback of the ``ArrowAsyncDeviceStreamHandler``. Because calling this method
Review Comment:
> If a non-zero value is returned from this, it should be followed only by the producer calling
> the ``on_error`` callback
Is the producer then required to keep track of errors produced by the returned tasks?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-43631: [C][Format] Add ArrowAsyncDeviceStreamHandler interface [arrow]
Posted by "felipecrv (via GitHub)" <gi...@apache.org>.
felipecrv commented on code in PR #43632:
URL: https://github.com/apache/arrow/pull/43632#discussion_r1767030294
##########
cpp/src/arrow/c/abi.h:
##########
@@ -228,6 +228,207 @@ struct ArrowDeviceArrayStream {
#endif // ARROW_C_DEVICE_STREAM_INTERFACE
+#ifndef ARROW_C_ASYNC_STREAM_INTERFACE
+#define ARROW_C_ASYNC_STREAM_INTERFACE
+
+// ArrowAsyncTask represents available data from a producer that was passed to
+// an invocation of `on_next_task` on the ArrowAsyncDeviceStreamHandler.
+//
+// The reason for this Task approach instead of the Async interface returning
+// the Array directly is to allow for more complex thread handling and reducing
+// context switching and data transfers between CPU cores (e.g. from one L1/L2
+// cache to another) if desired.
Review Comment:
This is a consequence of consumer being able to maintain an upper bound on memory consumption in terms of the number record batches in memory at any point in time.
Readers might be unable to make this causal inference. I don't really have a suggestion on how to reword this, just saying that the connection might be unclear to many readers.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-43631: [C][Format] Add ArrowAsyncDeviceStreamHandler interface [arrow]
Posted by "pitrou (via GitHub)" <gi...@apache.org>.
pitrou commented on code in PR #43632:
URL: https://github.com/apache/arrow/pull/43632#discussion_r1766370604
##########
docs/source/format/CDeviceDataInterface.rst:
##########
@@ -650,6 +652,331 @@ The stream source is not assumed to be thread-safe. Consumers wanting to
call ``get_next`` from several threads should ensure those calls are
serialized.
+Async Device Stream Interface
+=============================
+
+The :ref:`C stream interface <_c-device-stream-interface>` provides a synchronous
+API centered around the consumer calling the callback functions to retrieve
+the next record batch. For some bindings, use cases, and interoperability, a more
+asynchronous, producer-focused interface may be required. These scenarios can utilize
+the ``ArrowAsyncDeviceStreamHandler``.
+
+Semantics
+---------
+
+Rather than the producer providing a structure of callbacks for a consumer to
+call and retrieve records, the Async interface is a consumer allocated structure.
+The consumer allocated struct provides handler callbacks for the producer to call
+when the schema and chunks of data are available, rather than the consumer using
+a blocking pull-style iteration.
+
+In addition to the ``ArrowAsyncDeviceStreamHandler``, there are also two additional
+structs used for the full data flow: ``ArrowAsyncTask`` and ``ArrowAsyncProducer``.
+
+Structure Definition
+--------------------
+
+The C device async stream interface is defined with a single ``struct`` definition:
+
+.. code-block:: c
+
+ #ifndef ARROW_C_ASYNC_STREAM_INTERFACE
+ #define ARROW_C_ASYNC_STREAM_INTERFACE
+
+ struct ArrowAsyncTask {
+ int (*get_data)(struct ArrowArrayTask* self, struct ArrowDeviceArray* out);
+
+ void* private_data;
+ };
+
+ struct ArrowAsyncProducer {
+ void (*request)(struct ArrowAsyncProducer* self, uint64_t n);
+ void (*cancel)(struct ArrowAsyncProducer* self);
+
+ void (*release)(struct ArrowAsyncProducer* self);
+ void* private_data;
+ };
+
+ struct ArrowAsyncDeviceStreamHandler {
+ // handlers
+ int (*on_schema)(struct ArrowAsyncDeviceStreamHandler* self,
+ struct ArrowAsyncProducer* producer,
+ struct ArrowSchema* stream_schema, const char* addl_metadata);
+ int (*on_next_task)(struct ArrowAsyncDeviceStreamHandler* self,
+ struct ArrowAsyncTask* task, const char* metadata);
+ void (*on_error)(struct ArrowAsyncDeviceStreamHandler* self,
+ int code, const char* message, const char* metadata);
+
+ // release callback
+ void (*release)(struct ArrowAsyncDeviceStreamHandler* self);
+
+ // opaque handler-specific data
+ void* private_data;
+ };
+
+ #endif // ARROW_C_ASYNC_STREAM_INTERFACE
+
+.. note::
+ The canonical guard ``ARROW_C_ASYNC_STREAM_INTERFACE`` is meant to avoid
+ duplicate definitions if two projects copy the C async stream interface
+ definitions into their own headers, and a third-party project includes
+ from these two projects. It is therefore important that this guard is kept
+ exactly as-is when these definitions are copied.
+
+The ArrowAsyncDeviceStreamHandler structure
+'''''''''''''''''''''''''''''''''''''''''''
+
+The structure has the following fields:
+
+.. c:member:: int (*ArrowAsyncDeviceStreamHandler.on_schema)(struct ArrowAsyncDeviceStreamHandler*, struct ArrowAsyncProducer*, struct ArrowSchema*, const char*)
+
+ *Mandatory.* Handler for receiving the schema of the stream. All records should
+ match the provided schema. If successful, the function should return 0, otherwise
+ it should return an ``errno``-compatible error code.
+
+ The ``const char*`` parameter exists for producers to provide any extra contextual information
+ they want, such as the total number of rows in the stream, statistics, or otherwise. This is
+ encoded in the same format as :c:member:`ArrowSchema.metadata`. If not ``NULL``,
+ the lifetime is only the scope of the call to this function. A consumer who wants to maintain
+ the additional metadata beyond the lifetime of this call *MUST* copy the value themselves.
+
+ Unless the ``on_error`` handler is called, this will always get called exactly once and will be
+ the first method called on this object. As such the producer *MUST* provide an ``ArrowAsyncProducer``
+ object when calling this function to allow the consumer to manage back-pressure and flow control.
+ The producer maintains ownership of the ``ArrowAsyncProducer`` and must clean it up before or after
+ calling the release callback on this object.
+
+ A producer that receives a non-zero result here must not subsequently call anything other than
+ the release callback on this object.
+
+.. c:member:: int (*ArrowAsyncDeviceStreamHandler.on_next_task)(struct ArrowAsyncDeviceStreamHandler*, struct ArrowAsyncTask*, const char*)
+
+ *Mandatory.* Handler to be called when a new record is available for processing. The
+ schema for each record should be the same as the schema that ``on_schema`` was called with.
+ If successfully handled, the function should return 0, otherwise it should return an
+ ``errno``-compatible error code.
+
+ Rather than passing the record itself it receives an ``ArrowAsyncTask`` instead to facilitate
+ better consumer-focused thread control as far as receiving the data. A call to this function
+ simply indicates that data is available via the provided task.
+
+ The ``const char*`` parameter exists for producers to provide any extra contextual information
+ they want. This is encoded in the same format as :c:member:`ArrowSchema.metadata`. If not ``NULL``,
+ the lifetime is only the scope of the call to this function. A consumer who wants to maintain
+ the additional metadata beyond the lifetime of this call *MUST* copy the value themselves.
+
+ A producer *MUST NOT* call this concurrently from multiple different threads.
+
+ The :c:member:`ArrowAsyncProducer.request` callback must be called to start receiving calls to this
+ handler.
+
+ This function *MUST* be able to be called re-entrantly on the same thread to allow for the
+ :c:member:`ArrowAsyncProducer.request` callback to potentially call this method recursively.
+
+.. c:member:: void (*ArrowAsyncDeviceStreamHandler.on_error)(struct ArrowAsyncDeviceStreamHandler, int, const char*, const char*)
+
+ *Mandatory.* Handler to be called when an error is encountered by the producer. After calling
+ this, the ``release`` callback will be called as the last call on this struct. The parameters
+ are an ``errno``-compatible error code and an optional error message and metadata.
+
+ If the message and metadata are not ``NULL``, their lifetime is only valid during the scope
+ of this call. A consumer who wants to maintain these values past the return of this function
+ *MUST* copy the values themselves.
+
+ If the metadata parameter is not ``NULL``, to provide key-value error metadata, then it should
+ be encoded identically to the way that metadata is encoded in :c:member:`ArrowSchema.metadata`.
+
+ It is valid for this to be called by a producer with or without a preceding call to
+ :c:member:`ArrowAsyncProducer.request`. This callback *MUST NOT* call any methods of an
+ ``ArrowAsyncProducer`` object.
+
+.. c:member:: void (*ArrowAsyncDeviceStreamHandler.release)(struct ArrowAsyncDeviceStreamHandler*)
+
+ *Mandatory.* A pointer to a consumer-provided release callback for the handler.
+
+ It is valid for this to be called by a producer with or without a preceding call to
+ :c:member:`ArrowAsyncProducer.request`. This must not call any methods of an ``ArrowAsyncProducer``
+ object.
+
+.. c:member:: void* ArrowAsyncDeviceStreamHandler.private_data
+
+ *Optional.* An opaque pointer to consumer-provided private data.
+
+ Producers *MUST NOT* process this member. Lifetime of this member is handled by
+ the consumer, and especially by the release callback.
+
+The ArrowAsyncTask structure
+''''''''''''''''''''''''''''
+
+The purpose of using a Task object rather than passing the array directly to the ``on_next``
+callback is to allow for more complex and efficient thread handling. Utilizing a Task
+object allows for a producer to separate the "decoding" logic from the I/O, enabling a
+consumer to avoid transferring data between CPU cores (e.g. from one L1/L2 cache to another).
Review Comment:
> That is my goal. In my current design the I/O thread pool belongs to the producer and the CPU thread pool belongs to the consumer. Hence the need for the task interface.
If that's the goal for the spec then it should be clearly spelled out IMHO. _However_, it means that `get_data` probably shouldn't use any parallelism because of potential oversubscription. This should also be spelled out: `get_data` is a purely CPU, single-thread blocking task.
> If the producer wants to do the decompression then it can do so. The "task" that gets delivered in this case is just a thin wrapper around the decoded data and doesn't do any work.
That's a good point.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-43631: [C][Format] Add ArrowAsyncDeviceStreamHandler interface [arrow]
Posted by "westonpace (via GitHub)" <gi...@apache.org>.
westonpace commented on code in PR #43632:
URL: https://github.com/apache/arrow/pull/43632#discussion_r1766830856
##########
docs/source/format/CDeviceDataInterface.rst:
##########
@@ -650,6 +652,331 @@ The stream source is not assumed to be thread-safe. Consumers wanting to
call ``get_next`` from several threads should ensure those calls are
serialized.
+Async Device Stream Interface
+=============================
+
+The :ref:`C stream interface <_c-device-stream-interface>` provides a synchronous
+API centered around the consumer calling the callback functions to retrieve
+the next record batch. For some bindings, use cases, and interoperability, a more
+asynchronous, producer-focused interface may be required. These scenarios can utilize
+the ``ArrowAsyncDeviceStreamHandler``.
+
+Semantics
+---------
+
+Rather than the producer providing a structure of callbacks for a consumer to
+call and retrieve records, the Async interface is a consumer allocated structure.
+The consumer allocated struct provides handler callbacks for the producer to call
+when the schema and chunks of data are available, rather than the consumer using
+a blocking pull-style iteration.
+
+In addition to the ``ArrowAsyncDeviceStreamHandler``, there are also two additional
+structs used for the full data flow: ``ArrowAsyncTask`` and ``ArrowAsyncProducer``.
+
+Structure Definition
+--------------------
+
+The C device async stream interface is defined with a single ``struct`` definition:
+
+.. code-block:: c
+
+ #ifndef ARROW_C_ASYNC_STREAM_INTERFACE
+ #define ARROW_C_ASYNC_STREAM_INTERFACE
+
+ struct ArrowAsyncTask {
+ int (*get_data)(struct ArrowArrayTask* self, struct ArrowDeviceArray* out);
+
+ void* private_data;
+ };
+
+ struct ArrowAsyncProducer {
+ void (*request)(struct ArrowAsyncProducer* self, uint64_t n);
+ void (*cancel)(struct ArrowAsyncProducer* self);
+
+ void (*release)(struct ArrowAsyncProducer* self);
+ void* private_data;
+ };
+
+ struct ArrowAsyncDeviceStreamHandler {
+ // handlers
+ int (*on_schema)(struct ArrowAsyncDeviceStreamHandler* self,
+ struct ArrowAsyncProducer* producer,
+ struct ArrowSchema* stream_schema, const char* addl_metadata);
+ int (*on_next_task)(struct ArrowAsyncDeviceStreamHandler* self,
+ struct ArrowAsyncTask* task, const char* metadata);
+ void (*on_error)(struct ArrowAsyncDeviceStreamHandler* self,
+ int code, const char* message, const char* metadata);
+
+ // release callback
+ void (*release)(struct ArrowAsyncDeviceStreamHandler* self);
+
+ // opaque handler-specific data
+ void* private_data;
+ };
+
+ #endif // ARROW_C_ASYNC_STREAM_INTERFACE
+
+.. note::
+ The canonical guard ``ARROW_C_ASYNC_STREAM_INTERFACE`` is meant to avoid
+ duplicate definitions if two projects copy the C async stream interface
+ definitions into their own headers, and a third-party project includes
+ from these two projects. It is therefore important that this guard is kept
+ exactly as-is when these definitions are copied.
+
+The ArrowAsyncDeviceStreamHandler structure
+'''''''''''''''''''''''''''''''''''''''''''
+
+The structure has the following fields:
+
+.. c:member:: int (*ArrowAsyncDeviceStreamHandler.on_schema)(struct ArrowAsyncDeviceStreamHandler*, struct ArrowAsyncProducer*, struct ArrowSchema*, const char*)
+
+ *Mandatory.* Handler for receiving the schema of the stream. All records should
+ match the provided schema. If successful, the function should return 0, otherwise
+ it should return an ``errno``-compatible error code.
+
+ The ``const char*`` parameter exists for producers to provide any extra contextual information
+ they want, such as the total number of rows in the stream, statistics, or otherwise. This is
+ encoded in the same format as :c:member:`ArrowSchema.metadata`. If not ``NULL``,
+ the lifetime is only the scope of the call to this function. A consumer who wants to maintain
+ the additional metadata beyond the lifetime of this call *MUST* copy the value themselves.
+
+ Unless the ``on_error`` handler is called, this will always get called exactly once and will be
+ the first method called on this object. As such the producer *MUST* provide an ``ArrowAsyncProducer``
+ object when calling this function to allow the consumer to manage back-pressure and flow control.
+ The producer maintains ownership of the ``ArrowAsyncProducer`` and must clean it up before or after
+ calling the release callback on this object.
+
+ A producer that receives a non-zero result here must not subsequently call anything other than
+ the release callback on this object.
+
+.. c:member:: int (*ArrowAsyncDeviceStreamHandler.on_next_task)(struct ArrowAsyncDeviceStreamHandler*, struct ArrowAsyncTask*, const char*)
+
+ *Mandatory.* Handler to be called when a new record is available for processing. The
+ schema for each record should be the same as the schema that ``on_schema`` was called with.
+ If successfully handled, the function should return 0, otherwise it should return an
+ ``errno``-compatible error code.
+
+ Rather than passing the record itself it receives an ``ArrowAsyncTask`` instead to facilitate
+ better consumer-focused thread control as far as receiving the data. A call to this function
+ simply indicates that data is available via the provided task.
+
+ The ``const char*`` parameter exists for producers to provide any extra contextual information
+ they want. This is encoded in the same format as :c:member:`ArrowSchema.metadata`. If not ``NULL``,
+ the lifetime is only the scope of the call to this function. A consumer who wants to maintain
+ the additional metadata beyond the lifetime of this call *MUST* copy the value themselves.
+
+ A producer *MUST NOT* call this concurrently from multiple different threads.
+
+ The :c:member:`ArrowAsyncProducer.request` callback must be called to start receiving calls to this
+ handler.
+
+ This function *MUST* be able to be called re-entrantly on the same thread to allow for the
+ :c:member:`ArrowAsyncProducer.request` callback to potentially call this method recursively.
+
+.. c:member:: void (*ArrowAsyncDeviceStreamHandler.on_error)(struct ArrowAsyncDeviceStreamHandler, int, const char*, const char*)
+
+ *Mandatory.* Handler to be called when an error is encountered by the producer. After calling
+ this, the ``release`` callback will be called as the last call on this struct. The parameters
+ are an ``errno``-compatible error code and an optional error message and metadata.
+
+ If the message and metadata are not ``NULL``, their lifetime is only valid during the scope
+ of this call. A consumer who wants to maintain these values past the return of this function
+ *MUST* copy the values themselves.
+
+ If the metadata parameter is not ``NULL``, to provide key-value error metadata, then it should
+ be encoded identically to the way that metadata is encoded in :c:member:`ArrowSchema.metadata`.
+
+ It is valid for this to be called by a producer with or without a preceding call to
+ :c:member:`ArrowAsyncProducer.request`. This callback *MUST NOT* call any methods of an
+ ``ArrowAsyncProducer`` object.
+
+.. c:member:: void (*ArrowAsyncDeviceStreamHandler.release)(struct ArrowAsyncDeviceStreamHandler*)
+
+ *Mandatory.* A pointer to a consumer-provided release callback for the handler.
+
+ It is valid for this to be called by a producer with or without a preceding call to
+ :c:member:`ArrowAsyncProducer.request`. This must not call any methods of an ``ArrowAsyncProducer``
+ object.
+
+.. c:member:: void* ArrowAsyncDeviceStreamHandler.private_data
+
+ *Optional.* An opaque pointer to consumer-provided private data.
+
+ Producers *MUST NOT* process this member. Lifetime of this member is handled by
+ the consumer, and especially by the release callback.
+
+The ArrowAsyncTask structure
+''''''''''''''''''''''''''''
+
+The purpose of using a Task object rather than passing the array directly to the ``on_next``
+callback is to allow for more complex and efficient thread handling. Utilizing a Task
+object allows for a producer to separate the "decoding" logic from the I/O, enabling a
+consumer to avoid transferring data between CPU cores (e.g. from one L1/L2 cache to another).
Review Comment:
> If that's the goal for the spec then it should be clearly spelled out IMHO. However, it means that get_data probably shouldn't use any parallelism because of potential oversubscription. This should also be spelled out: get_data is a purely CPU, single-thread blocking task.
It's a _possible_ goal but not _the_ goal. For example, all my discussion so far has been "scanner is producer, query engine is consumer". However, I think the "query engine is producer, analytics script is consumer" is a completely valid use case as well. In the latter use case there is no separation of I/O and CPU to worry about, if both those thread pools exist then they both exist on the producer and I would full expect the tasks returned by the producer in this case to be shallow batch wrappers.
I agree that `get_data` should be a purely CPU single-threaded blocking task.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-43631: [C][Format] Add ArrowAsyncDeviceStreamHandler interface [arrow]
Posted by "lidavidm (via GitHub)" <gi...@apache.org>.
lidavidm commented on code in PR #43632:
URL: https://github.com/apache/arrow/pull/43632#discussion_r1712459774
##########
cpp/src/arrow/c/abi.h:
##########
@@ -228,6 +228,65 @@ struct ArrowDeviceArrayStream {
#endif // ARROW_C_DEVICE_STREAM_INTERFACE
+#ifndef ARROW_C_ASYNC_STREAM_INTERFACE
+#define ARROW_C_ASYNC_STREAM_INTERFACE
+
+// Similar to ArrowDeviceArrayStream, except designed for an asynchronous
+// style of interaction. While ArrowDeviceArrayStream provides producer
+// defined callbacks, this is intended to be created by the consumer instead.
+// The consumer passes this handler to the producer, which in turn uses the
+// callbacks to inform the consumer of events in the stream.
+struct ArrowAsyncDeviceStreamHandler {
+ // Handler for receiving a schema. The passed in stream_schema should be
+ // released or moved by the handler (producer is giving ownership of it to
+ // the handler).
+ //
+ // The `extension_param` argument can be null or can be used by a producer
Review Comment:
How should it be freed/what is the lifetime + ownership? (Especially in the case where the consumer may not understand the format of the extension and won't know how to free it?)
##########
cpp/src/arrow/c/abi.h:
##########
@@ -228,6 +228,65 @@ struct ArrowDeviceArrayStream {
#endif // ARROW_C_DEVICE_STREAM_INTERFACE
+#ifndef ARROW_C_ASYNC_STREAM_INTERFACE
+#define ARROW_C_ASYNC_STREAM_INTERFACE
+
+// Similar to ArrowDeviceArrayStream, except designed for an asynchronous
+// style of interaction. While ArrowDeviceArrayStream provides producer
+// defined callbacks, this is intended to be created by the consumer instead.
+// The consumer passes this handler to the producer, which in turn uses the
+// callbacks to inform the consumer of events in the stream.
+struct ArrowAsyncDeviceStreamHandler {
+ // Handler for receiving a schema. The passed in stream_schema should be
+ // released or moved by the handler (producer is giving ownership of it to
+ // the handler).
+ //
+ // The `extension_param` argument can be null or can be used by a producer
+ // to pass arbitrary extra information to the consumer (such as total number
+ // of rows, context info, or otherwise).
+ //
+ // Return value: 0 if successful, `errno`-compatible error otherwise
Review Comment:
What is the producer supposed to do with an error? (FWIW, gRPC just uses void for its callbacks)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-43631: [C][Format] Add ArrowAsyncDeviceStreamHandler interface [arrow]
Posted by "westonpace (via GitHub)" <gi...@apache.org>.
westonpace commented on code in PR #43632:
URL: https://github.com/apache/arrow/pull/43632#discussion_r1712647337
##########
cpp/src/arrow/c/abi.h:
##########
@@ -228,6 +228,65 @@ struct ArrowDeviceArrayStream {
#endif // ARROW_C_DEVICE_STREAM_INTERFACE
+#ifndef ARROW_C_ASYNC_STREAM_INTERFACE
+#define ARROW_C_ASYNC_STREAM_INTERFACE
+
+// Similar to ArrowDeviceArrayStream, except designed for an asynchronous
+// style of interaction. While ArrowDeviceArrayStream provides producer
+// defined callbacks, this is intended to be created by the consumer instead.
+// The consumer passes this handler to the producer, which in turn uses the
+// callbacks to inform the consumer of events in the stream.
+struct ArrowAsyncDeviceStreamHandler {
+ // Handler for receiving a schema. The passed in stream_schema should be
+ // released or moved by the handler (producer is giving ownership of it to
+ // the handler).
+ //
+ // The `extension_param` argument can be null or can be used by a producer
+ // to pass arbitrary extra information to the consumer (such as total number
+ // of rows, context info, or otherwise).
+ //
+ // Return value: 0 if successful, `errno`-compatible error otherwise
+ int (*on_schema)(struct ArrowAsyncDeviceStreamHandler* self,
+ struct ArrowSchema* stream_schema, void* extension_param);
+
+ // Handler for receiving an array/record batch. Always called at least once
+ // unless an error is encountered (which would result in calling on_error).
+ // An empty/released array is passed to indicate the end of the stream if no
+ // errors have been encountered.
+ //
+ // The `extension_param` argument can be null or can be used by a producer
+ // to pass arbitrary extra information to the consumer.
+ //
+ // Return value: 0 if successful, `errno`-compatible error otherwise.
+ int (*on_next)(struct ArrowAsyncDeviceStreamHandler* self,
+ struct ArrowDeviceArray* next, void* extension_param);
Review Comment:
What should happen if the consumer needs to do some lengthy processing of `next` (e.g. maybe the producer is a simple scanner and the consumer is running a complex analysis query). Should it force the creation of a new thread task (which is annoying because the array is probably in the CPU cache already and we don't want to have to process it on a new thread) or should it do the analysis as part of the call to `on_next` (which is weird because now we are tying up a producer's thread and would that interfere with the production of new data?).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-43631: [C][Format] Add ArrowAsyncDeviceStreamHandler interface [arrow]
Posted by "leventov (via GitHub)" <gi...@apache.org>.
leventov commented on code in PR #43632:
URL: https://github.com/apache/arrow/pull/43632#discussion_r1721539510
##########
cpp/src/arrow/c/abi.h:
##########
@@ -228,6 +228,65 @@ struct ArrowDeviceArrayStream {
#endif // ARROW_C_DEVICE_STREAM_INTERFACE
+#ifndef ARROW_C_ASYNC_STREAM_INTERFACE
+#define ARROW_C_ASYNC_STREAM_INTERFACE
+
+// Similar to ArrowDeviceArrayStream, except designed for an asynchronous
+// style of interaction. While ArrowDeviceArrayStream provides producer
+// defined callbacks, this is intended to be created by the consumer instead.
+// The consumer passes this handler to the producer, which in turn uses the
+// callbacks to inform the consumer of events in the stream.
+struct ArrowAsyncDeviceStreamHandler {
+ // Handler for receiving a schema. The passed in stream_schema should be
+ // released or moved by the handler (producer is giving ownership of it to
+ // the handler).
+ //
+ // The `extension_param` argument can be null or can be used by a producer
+ // to pass arbitrary extra information to the consumer (such as total number
+ // of rows, context info, or otherwise).
+ //
+ // Return value: 0 if successful, `errno`-compatible error otherwise
Review Comment:
> Right, but it might make more sense to just have a unified cancellation mechanism. And again, what is the producer really supposed to do with the error code? It's just going to drop it on the floor.
I think it can be useful for monitoring on the middleware/transport level, if the API wraps communication with a remote server.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-43631: [C][Format] Add ArrowAsyncDeviceStreamHandler interface [arrow]
Posted by "leventov (via GitHub)" <gi...@apache.org>.
leventov commented on code in PR #43632:
URL: https://github.com/apache/arrow/pull/43632#discussion_r1721409104
##########
cpp/src/arrow/c/abi.h:
##########
@@ -228,6 +228,65 @@ struct ArrowDeviceArrayStream {
#endif // ARROW_C_DEVICE_STREAM_INTERFACE
+#ifndef ARROW_C_ASYNC_STREAM_INTERFACE
+#define ARROW_C_ASYNC_STREAM_INTERFACE
+
+// Similar to ArrowDeviceArrayStream, except designed for an asynchronous
+// style of interaction. While ArrowDeviceArrayStream provides producer
+// defined callbacks, this is intended to be created by the consumer instead.
+// The consumer passes this handler to the producer, which in turn uses the
+// callbacks to inform the consumer of events in the stream.
+struct ArrowAsyncDeviceStreamHandler {
+ // Handler for receiving a schema. The passed in stream_schema should be
+ // released or moved by the handler (producer is giving ownership of it to
+ // the handler).
+ //
+ // The `extension_param` argument can be null or can be used by a producer
+ // to pass arbitrary extra information to the consumer (such as total number
+ // of rows, context info, or otherwise).
+ //
+ // Return value: 0 if successful, `errno`-compatible error otherwise
+ int (*on_schema)(struct ArrowAsyncDeviceStreamHandler* self,
+ struct ArrowSchema* stream_schema, void* extension_param);
+
+ // Handler for receiving an array/record batch. Always called at least once
+ // unless an error is encountered (which would result in calling on_error).
+ // An empty/released array is passed to indicate the end of the stream if no
+ // errors have been encountered.
+ //
+ // The `extension_param` argument can be null or can be used by a producer
+ // to pass arbitrary extra information to the consumer.
+ //
+ // Return value: 0 if successful, `errno`-compatible error otherwise.
+ int (*on_next)(struct ArrowAsyncDeviceStreamHandler* self,
+ struct ArrowDeviceArray* next, void* extension_param);
Review Comment:
> Very sophisticated users should be able to implement advanced concurrency patterns (e.g. handling out-of-order pieces of the stream).
@felipecrv IMO this API the wrong level to assume out-of-order delivery. Out of order arrival assumes producer's concurrency. If the producer is capable to produce arrays concurrently, and the consumer consume them concurrently or out-of-order, too, it's better to simply create multiple streams and make multiple of consumer's threads/green threads/coroutines/async tasks consume them on the single producer--single consumer basis. It would be *much* less tricky to implement both for consumers and producers, and quite likely even be more performant, too, because there would be no write contention and concurrent queues on the producer's side. Cf. [single writer principle](https://mechanical-sympathy.blogspot.com/2011/09/single-writer-principle.html).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-43631: [C][Format] Add ArrowAsyncDeviceStreamHandler interface [arrow]
Posted by "westonpace (via GitHub)" <gi...@apache.org>.
westonpace commented on code in PR #43632:
URL: https://github.com/apache/arrow/pull/43632#discussion_r1722458512
##########
docs/source/format/CDeviceDataInterface.rst:
##########
@@ -650,6 +652,137 @@ The stream source is not assumed to be thread-safe. Consumers wanting to
call ``get_next`` from several threads should ensure those calls are
serialized.
+Async Device Stream Interface
+=============================
+
+The :ref:`C stream interface <_c-device-stream-interface>` provides a synchronous
+API centered around the consumer calling the callback functions to retrieve
+the next record batch. For some bindings, use cases, and interoperability, a more
+asynchronous, producer-focused interface may be required. These scenarios can utilize
+the ``ArrowAsyncDeviceStreamHandler``.
+
+Semantics
+---------
+
+Rather than the producer providing a structure of callbacks for a consumer to
+call and retrieve records, the Async interface is a consumer allocated structure.
+The consumer allocated struct provides handler callbacks for the producer to call
+when the schema and chunks of data are available, rather than the consumer using
+a blocking pull-style iteration.
+
+Structure Definition
+--------------------
+
+The C device async stream interface is defined with a single ``struct`` definition:
+
+.. code-block:: c
+
+ #ifndef ARROW_C_ASYNC_STREAM_INTERFACE
+ #define ARROW_C_ASYNC_STREAM_INTERFACE
+
+ struct ArrowAsyncDeviceStreamHandler {
+ // handlers
+ int (*on_schema)(struct ArrowAsyncDeviceStreamHandler* self,
+ struct ArrowSchema* stream_schema, void* extension_param);
+ int (*on_next)(struct ArrowAsyncDeviceStreamHandler* self,
+ struct ArrowDeviceArray* next, void* extension_param);
+ void (*on_error)(struct ArrowAsyncDeviceStreamHandler* self,
+ int code, const char* message, const char* metadata);
+
+ // release callback
+ void (*release)(struct ArrowAsyncDeviceStreamHandler* self);
+
+ // opaque handler-specific data
+ void* private_data;
+ };
+
+ #endif // ARROW_C_ASYNC_STREAM_INTERFACE
+
+.. note::
+ The canonical guard ``ARROW_C_ASYNC_STREAM_INTERFACE`` is meant to avoid
+ duplicate definitions if two projects copy the C async stream interface
+ definitions into their own headers, and a third-party project includes
+ from these two projects. It is therefore important that this guard is kept
+ exactly as-is when these definitions are copied.
+
+The ArrowAsyncDeviceStreamHandler structure
+'''''''''''''''''''''''''''''''''''''''''''
+
+The structure has the following fields:
+
+.. c:member:: int (*ArrowAsyncDeviceStreamHandler.on_schema)(struct ArrowAsyncDeviceStreamHandler*, struct ArrowSchema*, void*)
+
+ *Mandatory.* Handler for receiving the schema of the stream. All records should
+ match the provided schema. If successful, the function should return 0, otherwise
+ it should return an ``errno``-compatible error code.
+
+ The ``void*`` parameter exists for producers to provide any extra contextual information
+ they want, such as the total number of rows in the stream, statistics, or otherwise.
+
+ Unless the ``on_error`` handler is called, this will always get called exactly once.
+
+.. c:member:: int (*ArrowAsyncDeviceStreamHandler.on_next)(struct ArrowAsyncDeviceStreamHandler*, struct ArrowDeviceArray*, void*)
+
+ *Mandatory.* Handler to be called when a new record is available for processing. The
+ schema for each record should be the same as the schema that ``on_schema`` was called with.
+ If successfully handled, the function should return 0, otherwise it should return an
+ ``errno``-compatible error code.
+
+ The ``void*`` parameter exists for producers to provide any extra contextual information
+ they want.
+
+ Unless the ``on_error`` handler is called, this will always get called at least once. The
+ end of the stream is indicated by it being called with an empty or released ``ArrowDeviceArray``.
+
+.. c:member:: void (*ArrowAsyncDeviceStreamHandler.on_error)(struct ArrowAsyncDeviceStreamHandler, int, const char*, const char*)
+
+ *Mandatory.* Handler to be called when an error is encountered by the producer. After calling
+ this, the ``release`` callback will be called as the last call on this struct. The parameters
+ are an ``errno``-compatible error code and an optional error message and metadata.
+
+ If the message and metadata are not ``NULL``, their lifetime is only valid during the scope
+ of this call. A consumer who wants to maintain these values past the return of this function
+ *MUST* copy the values themselves.
+
+ If the metadata parameter is not ``NULL``, to provide key-value error metadata, then it should
+ be encoded identically to the way that metadata is encoded in :c:member:`ArrowSchema.metadata`.
+
+.. c:member:: void (*ArrowAsyncDeviceStreamHandler.release)(struct ArrowAsyncDeviceStreamHandler*)
+
+ *Mandatory.* A pointer to a consumer-provided release callback for the handler.
+
+.. c:member:: void* ArrowAsyncDeviceStreamHandler.private_data
+
+ *Optional.* An opaque pointer to consumer-provided private data.
+
+ Producers *MUST NOT* process this member. Lifetime of this member is handled by
+ the consumer, and especially by the release callback.
+
+Result lifetimes
+''''''''''''''''
+
+The ``ArrowSchema`` and ``ArrowDeviceArray`` that are passed to the ``on_schema`` and
+``on_next`` callbacks must be released independently. Their lifetimes are not tied to
+the ``ArrowAsyncDeviceStreamHandler`` or the handler functions.
+
+The ``const char*`` error ``message`` and ``metadata`` which are passed to ``on_error``
+are only valid within the scope of the ``on_error`` function itself. They must be copied
+if it is necessary for them to exist after it returns.
+
+Stream Handler Lifetime
+'''''''''''''''''''''''
+
+Lifetime of the async stream handler is managed using a release callback with similar
+usage as in :ref:`C data interface <c-data-interface-released>`.
+
+Thread safety
+'''''''''''''
+
+All handler functions should only be called in a serialized manner, but are not guaranteed
+to be called from the same thread every time. A producer should wait for handler callbacks to
+return before calling the next handler callback. As a result, a consumer can manage back-pressure
+simply by sleeping or otherwise waiting to return from the ``on_next`` handler if necessary.
+
Review Comment:
By "as soon as" do you mean "during the call?" In other words, if there is a...
```
dataset.scan(handler);
```
Then is your question "can the producer call the callback functions before `scan` returns"?
I'd personally be ok either way (I don't think it's too much burden on the consumer to have `handler` ready before the call starts) but agree it would be good to clarify which it is.
Or is your question something different (e.g. should there be a `set_handler` and a `start_producing` method?)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-43631: [C][Format] Add ArrowAsyncDeviceStreamHandler interface [arrow]
Posted by "zeroshade (via GitHub)" <gi...@apache.org>.
zeroshade commented on code in PR #43632:
URL: https://github.com/apache/arrow/pull/43632#discussion_r1766963168
##########
docs/source/format/CDeviceDataInterface.rst:
##########
@@ -650,6 +652,331 @@ The stream source is not assumed to be thread-safe. Consumers wanting to
call ``get_next`` from several threads should ensure those calls are
serialized.
+Async Device Stream Interface
+=============================
+
+The :ref:`C stream interface <_c-device-stream-interface>` provides a synchronous
+API centered around the consumer calling the callback functions to retrieve
+the next record batch. For some bindings, use cases, and interoperability, a more
+asynchronous, producer-focused interface may be required. These scenarios can utilize
+the ``ArrowAsyncDeviceStreamHandler``.
+
+Semantics
+---------
+
+Rather than the producer providing a structure of callbacks for a consumer to
+call and retrieve records, the Async interface is a consumer allocated structure.
+The consumer allocated struct provides handler callbacks for the producer to call
+when the schema and chunks of data are available, rather than the consumer using
+a blocking pull-style iteration.
+
+In addition to the ``ArrowAsyncDeviceStreamHandler``, there are also two additional
+structs used for the full data flow: ``ArrowAsyncTask`` and ``ArrowAsyncProducer``.
+
+Structure Definition
+--------------------
+
+The C device async stream interface is defined with a single ``struct`` definition:
+
+.. code-block:: c
+
+ #ifndef ARROW_C_ASYNC_STREAM_INTERFACE
+ #define ARROW_C_ASYNC_STREAM_INTERFACE
+
+ struct ArrowAsyncTask {
+ int (*get_data)(struct ArrowArrayTask* self, struct ArrowDeviceArray* out);
+
+ void* private_data;
+ };
+
+ struct ArrowAsyncProducer {
+ void (*request)(struct ArrowAsyncProducer* self, uint64_t n);
+ void (*cancel)(struct ArrowAsyncProducer* self);
+
+ void (*release)(struct ArrowAsyncProducer* self);
+ void* private_data;
+ };
+
+ struct ArrowAsyncDeviceStreamHandler {
+ // handlers
+ int (*on_schema)(struct ArrowAsyncDeviceStreamHandler* self,
+ struct ArrowAsyncProducer* producer,
+ struct ArrowSchema* stream_schema, const char* addl_metadata);
+ int (*on_next_task)(struct ArrowAsyncDeviceStreamHandler* self,
+ struct ArrowAsyncTask* task, const char* metadata);
+ void (*on_error)(struct ArrowAsyncDeviceStreamHandler* self,
+ int code, const char* message, const char* metadata);
+
+ // release callback
+ void (*release)(struct ArrowAsyncDeviceStreamHandler* self);
+
+ // opaque handler-specific data
+ void* private_data;
+ };
+
+ #endif // ARROW_C_ASYNC_STREAM_INTERFACE
+
+.. note::
+ The canonical guard ``ARROW_C_ASYNC_STREAM_INTERFACE`` is meant to avoid
+ duplicate definitions if two projects copy the C async stream interface
+ definitions into their own headers, and a third-party project includes
+ from these two projects. It is therefore important that this guard is kept
+ exactly as-is when these definitions are copied.
+
+The ArrowAsyncDeviceStreamHandler structure
+'''''''''''''''''''''''''''''''''''''''''''
+
+The structure has the following fields:
+
+.. c:member:: int (*ArrowAsyncDeviceStreamHandler.on_schema)(struct ArrowAsyncDeviceStreamHandler*, struct ArrowAsyncProducer*, struct ArrowSchema*, const char*)
+
+ *Mandatory.* Handler for receiving the schema of the stream. All records should
+ match the provided schema. If successful, the function should return 0, otherwise
+ it should return an ``errno``-compatible error code.
+
+ The ``const char*`` parameter exists for producers to provide any extra contextual information
+ they want, such as the total number of rows in the stream, statistics, or otherwise. This is
+ encoded in the same format as :c:member:`ArrowSchema.metadata`. If not ``NULL``,
+ the lifetime is only the scope of the call to this function. A consumer who wants to maintain
+ the additional metadata beyond the lifetime of this call *MUST* copy the value themselves.
+
+ Unless the ``on_error`` handler is called, this will always get called exactly once and will be
+ the first method called on this object. As such the producer *MUST* provide an ``ArrowAsyncProducer``
+ object when calling this function to allow the consumer to manage back-pressure and flow control.
+ The producer maintains ownership of the ``ArrowAsyncProducer`` and must clean it up before or after
+ calling the release callback on this object.
+
+ A producer that receives a non-zero result here must not subsequently call anything other than
+ the release callback on this object.
+
+.. c:member:: int (*ArrowAsyncDeviceStreamHandler.on_next_task)(struct ArrowAsyncDeviceStreamHandler*, struct ArrowAsyncTask*, const char*)
+
+ *Mandatory.* Handler to be called when a new record is available for processing. The
+ schema for each record should be the same as the schema that ``on_schema`` was called with.
+ If successfully handled, the function should return 0, otherwise it should return an
+ ``errno``-compatible error code.
+
+ Rather than passing the record itself it receives an ``ArrowAsyncTask`` instead to facilitate
+ better consumer-focused thread control as far as receiving the data. A call to this function
+ simply indicates that data is available via the provided task.
+
+ The ``const char*`` parameter exists for producers to provide any extra contextual information
+ they want. This is encoded in the same format as :c:member:`ArrowSchema.metadata`. If not ``NULL``,
+ the lifetime is only the scope of the call to this function. A consumer who wants to maintain
+ the additional metadata beyond the lifetime of this call *MUST* copy the value themselves.
+
+ A producer *MUST NOT* call this concurrently from multiple different threads.
+
+ The :c:member:`ArrowAsyncProducer.request` callback must be called to start receiving calls to this
+ handler.
+
+ This function *MUST* be able to be called re-entrantly on the same thread to allow for the
+ :c:member:`ArrowAsyncProducer.request` callback to potentially call this method recursively.
+
+.. c:member:: void (*ArrowAsyncDeviceStreamHandler.on_error)(struct ArrowAsyncDeviceStreamHandler, int, const char*, const char*)
+
+ *Mandatory.* Handler to be called when an error is encountered by the producer. After calling
+ this, the ``release`` callback will be called as the last call on this struct. The parameters
+ are an ``errno``-compatible error code and an optional error message and metadata.
+
+ If the message and metadata are not ``NULL``, their lifetime is only valid during the scope
+ of this call. A consumer who wants to maintain these values past the return of this function
+ *MUST* copy the values themselves.
+
+ If the metadata parameter is not ``NULL``, to provide key-value error metadata, then it should
+ be encoded identically to the way that metadata is encoded in :c:member:`ArrowSchema.metadata`.
+
+ It is valid for this to be called by a producer with or without a preceding call to
+ :c:member:`ArrowAsyncProducer.request`. This callback *MUST NOT* call any methods of an
+ ``ArrowAsyncProducer`` object.
+
+.. c:member:: void (*ArrowAsyncDeviceStreamHandler.release)(struct ArrowAsyncDeviceStreamHandler*)
+
+ *Mandatory.* A pointer to a consumer-provided release callback for the handler.
+
+ It is valid for this to be called by a producer with or without a preceding call to
+ :c:member:`ArrowAsyncProducer.request`. This must not call any methods of an ``ArrowAsyncProducer``
+ object.
+
+.. c:member:: void* ArrowAsyncDeviceStreamHandler.private_data
+
+ *Optional.* An opaque pointer to consumer-provided private data.
+
+ Producers *MUST NOT* process this member. Lifetime of this member is handled by
+ the consumer, and especially by the release callback.
+
+The ArrowAsyncTask structure
+''''''''''''''''''''''''''''
+
+The purpose of using a Task object rather than passing the array directly to the ``on_next``
+callback is to allow for more complex and efficient thread handling. Utilizing a Task
+object allows for a producer to separate the "decoding" logic from the I/O, enabling a
+consumer to avoid transferring data between CPU cores (e.g. from one L1/L2 cache to another).
Review Comment:
It seems that we've got somewhat of a consensus here and I'll update the comments and docs accordingly as far as `get_data` being a purely CPU, single-threaded blocking task.
I'm fine with renaming it `extract_data` if others agree with @bkietz's suggestion. Thoughts?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-43631: [C][Format] Add ArrowAsyncDeviceStreamHandler interface [arrow]
Posted by "zeroshade (via GitHub)" <gi...@apache.org>.
zeroshade commented on code in PR #43632:
URL: https://github.com/apache/arrow/pull/43632#discussion_r1766972235
##########
cpp/src/arrow/c/abi.h:
##########
@@ -228,6 +228,207 @@ struct ArrowDeviceArrayStream {
#endif // ARROW_C_DEVICE_STREAM_INTERFACE
+#ifndef ARROW_C_ASYNC_STREAM_INTERFACE
+#define ARROW_C_ASYNC_STREAM_INTERFACE
+
+// ArrowAsyncTask represents available data from a producer that was passed to
+// an invocation of `on_next_task` on the ArrowAsyncDeviceStreamHandler.
+//
+// The reason for this Task approach instead of the Async interface returning
+// the Array directly is to allow for more complex thread handling and reducing
+// context switching and data transfers between CPU cores (e.g. from one L1/L2
+// cache to another) if desired.
+//
+// For example, the `on_next_task` callback can be called when data is ready, while
+// the producer puts potential "decoding" logic in the `ArrowAsyncTask` object. This
+// allows for the producer to manage the I/O on one thread which calls `on_next_task`
+// and the consumer can determine when the decoding (producer logic in the `get_data`
+// callback of the task) occurs and on which thread, to avoid a CPU core transfer
+// (data staying in the L2 cache).
+struct ArrowAsyncTask {
+ // This callback should populate the ArrowDeviceArray associated with this task.
+ // The order of ArrowAsyncTasks provided by the producer enables a consumer to
+ // ensure the order of data to process.
+ //
+ // This function is expected to be synchronous, but should not perform any blocking
+ // I/O. Ideally it should be as cheap as possible so as to not tie up the consumer
+ // thread unnecessarily.
+ //
+ // Returns: 0 if successful, errno-compatible error otherwise.
+ //
+ // If a non-0 value is returned then it should be followed by a call to `on_error`
+ // on the appropriate ArrowAsyncDeviceStreamHandler. This is because the it's highly
+ // likely that whatever is calling this function may be entirely disconnected from
+ // the current control flow. Indicating an error here with a non-zero return allows
+ // the current flow to be aware of the error occurring, while still allowing any
+ // logging or error handling to still be centralized in the `on_error` callback of
+ // the original Async handler.
+ //
+ // Rather than a release callback, any required cleanup should be performed as part
+ // of the invocation of `get_data`. Ownership of the Array is passed to the consumer
+ // calling this, and so it must be released separately.
+ //
+ // It is only valid to call this method exactly once.
+ int (*get_data)(struct ArrowArrayTask* self, struct ArrowDeviceArray* out);
+
+ // opaque task-specific data
+ void* private_data;
+};
+
+// ArrowAsyncProducer represents a 1-to-1 relationship between an async producer
+// and consumer. This object allows the consumer to perform backpressure and flow
+// control on the asynchronous stream processing. This object must be owned by the
+// producer who creates it, and thus is responsible for cleaning it up.
+struct ArrowAsyncProducer {
+ // A consumer must call this function to start receiving on_next_task calls.
+ //
+ // It *must* be valid to call this synchronously from within `on_next_task` or
+ // `on_schema`, but this function *must not* immediately call `on_next_task` so as
+ // to avoid recursion and reentrant callbacks.
+ //
+ // After cancel has been called, additional calls to this function must be NOPs,
+ // but allowed. While not cancelled, calling this function must register the
+ // given number of additional arrays/batches to be produced with the producer.
+ // The producer should only call `on_next_task` at most the registered number
+ // of arrays before propagating backpressure.
Review Comment:
That's definitely one way you could do it which would be valid.
Another possibility would be that you could call `request(32)` and you track available memory usage and only call `request(N)` for more when the available memory increases after calls to `get_data` on the tasks and completing whatever processing you wanted on the resulting batches, or some other strategy.
The idea is that backpressure is managed by the consumer signalling the producer "hey, I can handle up to N more batches of data right now".
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-43631: [C][Format] Add ArrowAsyncDeviceStreamHandler interface [arrow]
Posted by "zeroshade (via GitHub)" <gi...@apache.org>.
zeroshade commented on code in PR #43632:
URL: https://github.com/apache/arrow/pull/43632#discussion_r1765329980
##########
docs/source/format/CDeviceDataInterface.rst:
##########
@@ -650,6 +652,331 @@ The stream source is not assumed to be thread-safe. Consumers wanting to
call ``get_next`` from several threads should ensure those calls are
serialized.
+Async Device Stream Interface
+=============================
+
+The :ref:`C stream interface <_c-device-stream-interface>` provides a synchronous
+API centered around the consumer calling the callback functions to retrieve
+the next record batch. For some bindings, use cases, and interoperability, a more
+asynchronous, producer-focused interface may be required. These scenarios can utilize
+the ``ArrowAsyncDeviceStreamHandler``.
+
+Semantics
+---------
+
+Rather than the producer providing a structure of callbacks for a consumer to
+call and retrieve records, the Async interface is a consumer allocated structure.
+The consumer allocated struct provides handler callbacks for the producer to call
+when the schema and chunks of data are available, rather than the consumer using
+a blocking pull-style iteration.
+
+In addition to the ``ArrowAsyncDeviceStreamHandler``, there are also two additional
+structs used for the full data flow: ``ArrowAsyncTask`` and ``ArrowAsyncProducer``.
+
+Structure Definition
+--------------------
+
+The C device async stream interface is defined with a single ``struct`` definition:
+
+.. code-block:: c
+
+ #ifndef ARROW_C_ASYNC_STREAM_INTERFACE
+ #define ARROW_C_ASYNC_STREAM_INTERFACE
+
+ struct ArrowAsyncTask {
+ int (*get_data)(struct ArrowArrayTask* self, struct ArrowDeviceArray* out);
+
+ void* private_data;
+ };
+
+ struct ArrowAsyncProducer {
+ void (*request)(struct ArrowAsyncProducer* self, uint64_t n);
+ void (*cancel)(struct ArrowAsyncProducer* self);
+
+ void (*release)(struct ArrowAsyncProducer* self);
+ void* private_data;
+ };
+
+ struct ArrowAsyncDeviceStreamHandler {
+ // handlers
+ int (*on_schema)(struct ArrowAsyncDeviceStreamHandler* self,
+ struct ArrowAsyncProducer* producer,
+ struct ArrowSchema* stream_schema, const char* addl_metadata);
+ int (*on_next_task)(struct ArrowAsyncDeviceStreamHandler* self,
+ struct ArrowAsyncTask* task, const char* metadata);
+ void (*on_error)(struct ArrowAsyncDeviceStreamHandler* self,
+ int code, const char* message, const char* metadata);
+
+ // release callback
+ void (*release)(struct ArrowAsyncDeviceStreamHandler* self);
+
+ // opaque handler-specific data
+ void* private_data;
+ };
+
+ #endif // ARROW_C_ASYNC_STREAM_INTERFACE
+
+.. note::
+ The canonical guard ``ARROW_C_ASYNC_STREAM_INTERFACE`` is meant to avoid
+ duplicate definitions if two projects copy the C async stream interface
+ definitions into their own headers, and a third-party project includes
+ from these two projects. It is therefore important that this guard is kept
+ exactly as-is when these definitions are copied.
+
+The ArrowAsyncDeviceStreamHandler structure
+'''''''''''''''''''''''''''''''''''''''''''
+
+The structure has the following fields:
+
+.. c:member:: int (*ArrowAsyncDeviceStreamHandler.on_schema)(struct ArrowAsyncDeviceStreamHandler*, struct ArrowAsyncProducer*, struct ArrowSchema*, const char*)
+
+ *Mandatory.* Handler for receiving the schema of the stream. All records should
+ match the provided schema. If successful, the function should return 0, otherwise
+ it should return an ``errno``-compatible error code.
+
+ The ``const char*`` parameter exists for producers to provide any extra contextual information
+ they want, such as the total number of rows in the stream, statistics, or otherwise. This is
+ encoded in the same format as :c:member:`ArrowSchema.metadata`. If not ``NULL``,
+ the lifetime is only the scope of the call to this function. A consumer who wants to maintain
+ the additional metadata beyond the lifetime of this call *MUST* copy the value themselves.
+
+ Unless the ``on_error`` handler is called, this will always get called exactly once and will be
+ the first method called on this object. As such the producer *MUST* provide an ``ArrowAsyncProducer``
+ object when calling this function to allow the consumer to manage back-pressure and flow control.
+ The producer maintains ownership of the ``ArrowAsyncProducer`` and must clean it up before or after
+ calling the release callback on this object.
+
+ A producer that receives a non-zero result here must not subsequently call anything other than
+ the release callback on this object.
+
+.. c:member:: int (*ArrowAsyncDeviceStreamHandler.on_next_task)(struct ArrowAsyncDeviceStreamHandler*, struct ArrowAsyncTask*, const char*)
+
+ *Mandatory.* Handler to be called when a new record is available for processing. The
+ schema for each record should be the same as the schema that ``on_schema`` was called with.
+ If successfully handled, the function should return 0, otherwise it should return an
+ ``errno``-compatible error code.
+
+ Rather than passing the record itself it receives an ``ArrowAsyncTask`` instead to facilitate
+ better consumer-focused thread control as far as receiving the data. A call to this function
+ simply indicates that data is available via the provided task.
+
+ The ``const char*`` parameter exists for producers to provide any extra contextual information
+ they want. This is encoded in the same format as :c:member:`ArrowSchema.metadata`. If not ``NULL``,
+ the lifetime is only the scope of the call to this function. A consumer who wants to maintain
+ the additional metadata beyond the lifetime of this call *MUST* copy the value themselves.
+
+ A producer *MUST NOT* call this concurrently from multiple different threads.
+
+ The :c:member:`ArrowAsyncProducer.request` callback must be called to start receiving calls to this
+ handler.
+
+ This function *MUST* be able to be called re-entrantly on the same thread to allow for the
+ :c:member:`ArrowAsyncProducer.request` callback to potentially call this method recursively.
+
+.. c:member:: void (*ArrowAsyncDeviceStreamHandler.on_error)(struct ArrowAsyncDeviceStreamHandler, int, const char*, const char*)
+
+ *Mandatory.* Handler to be called when an error is encountered by the producer. After calling
+ this, the ``release`` callback will be called as the last call on this struct. The parameters
+ are an ``errno``-compatible error code and an optional error message and metadata.
+
+ If the message and metadata are not ``NULL``, their lifetime is only valid during the scope
+ of this call. A consumer who wants to maintain these values past the return of this function
+ *MUST* copy the values themselves.
+
+ If the metadata parameter is not ``NULL``, to provide key-value error metadata, then it should
+ be encoded identically to the way that metadata is encoded in :c:member:`ArrowSchema.metadata`.
+
+ It is valid for this to be called by a producer with or without a preceding call to
+ :c:member:`ArrowAsyncProducer.request`. This callback *MUST NOT* call any methods of an
+ ``ArrowAsyncProducer`` object.
+
+.. c:member:: void (*ArrowAsyncDeviceStreamHandler.release)(struct ArrowAsyncDeviceStreamHandler*)
+
+ *Mandatory.* A pointer to a consumer-provided release callback for the handler.
+
+ It is valid for this to be called by a producer with or without a preceding call to
+ :c:member:`ArrowAsyncProducer.request`. This must not call any methods of an ``ArrowAsyncProducer``
+ object.
+
+.. c:member:: void* ArrowAsyncDeviceStreamHandler.private_data
+
+ *Optional.* An opaque pointer to consumer-provided private data.
+
+ Producers *MUST NOT* process this member. Lifetime of this member is handled by
+ the consumer, and especially by the release callback.
+
+The ArrowAsyncTask structure
+''''''''''''''''''''''''''''
+
+The purpose of using a Task object rather than passing the array directly to the ``on_next``
+callback is to allow for more complex and efficient thread handling. Utilizing a Task
+object allows for a producer to separate the "decoding" logic from the I/O, enabling a
+consumer to avoid transferring data between CPU cores (e.g. from one L1/L2 cache to another).
Review Comment:
It's to allow a potential efficiency. This is adapted and based on the discussion that started [here](https://github.com/apache/arrow/pull/43632#issuecomment-2284669419) with @westonpace's comment.
But rather than the exact workflow that was proposed there I've adapted it to the following:
* Consumer calls `request` on the `ArrowAsyncProducer`
* Producer schedules IO
* When IO completes and data is available, Producer creates task to decode/decompress the data and calls `on_next_task` to hand it to the consumer
* Once consumer has a free thread it assigns it the task. Consumer runs task, decodes data (the decode logic is producer logic), and then processes the decoded data (consumer logic) all without a CPU core transfer (data stays in L2 cache).
In this way, the consumer determines when it has a free thread to process the data, it just potentially runs the producer logic to perform a decode/decompress before it processes the data. If this efficiency of managing the processor caches isn't necessary then the producer task could just return the data without any additional work if desired.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-43631: [C][Format] Add ArrowAsyncDeviceStreamHandler interface [arrow]
Posted by "zeroshade (via GitHub)" <gi...@apache.org>.
zeroshade commented on code in PR #43632:
URL: https://github.com/apache/arrow/pull/43632#discussion_r1765354142
##########
docs/source/format/CDeviceDataInterface.rst:
##########
@@ -650,6 +652,331 @@ The stream source is not assumed to be thread-safe. Consumers wanting to
call ``get_next`` from several threads should ensure those calls are
serialized.
+Async Device Stream Interface
+=============================
+
+The :ref:`C stream interface <_c-device-stream-interface>` provides a synchronous
+API centered around the consumer calling the callback functions to retrieve
+the next record batch. For some bindings, use cases, and interoperability, a more
+asynchronous, producer-focused interface may be required. These scenarios can utilize
+the ``ArrowAsyncDeviceStreamHandler``.
+
+Semantics
+---------
+
+Rather than the producer providing a structure of callbacks for a consumer to
+call and retrieve records, the Async interface is a consumer allocated structure.
+The consumer allocated struct provides handler callbacks for the producer to call
+when the schema and chunks of data are available, rather than the consumer using
+a blocking pull-style iteration.
+
+In addition to the ``ArrowAsyncDeviceStreamHandler``, there are also two additional
+structs used for the full data flow: ``ArrowAsyncTask`` and ``ArrowAsyncProducer``.
+
+Structure Definition
+--------------------
+
+The C device async stream interface is defined with a single ``struct`` definition:
+
+.. code-block:: c
+
+ #ifndef ARROW_C_ASYNC_STREAM_INTERFACE
+ #define ARROW_C_ASYNC_STREAM_INTERFACE
+
+ struct ArrowAsyncTask {
+ int (*get_data)(struct ArrowArrayTask* self, struct ArrowDeviceArray* out);
+
+ void* private_data;
+ };
+
+ struct ArrowAsyncProducer {
+ void (*request)(struct ArrowAsyncProducer* self, uint64_t n);
+ void (*cancel)(struct ArrowAsyncProducer* self);
+
+ void (*release)(struct ArrowAsyncProducer* self);
+ void* private_data;
+ };
+
+ struct ArrowAsyncDeviceStreamHandler {
+ // handlers
+ int (*on_schema)(struct ArrowAsyncDeviceStreamHandler* self,
+ struct ArrowAsyncProducer* producer,
+ struct ArrowSchema* stream_schema, const char* addl_metadata);
+ int (*on_next_task)(struct ArrowAsyncDeviceStreamHandler* self,
+ struct ArrowAsyncTask* task, const char* metadata);
+ void (*on_error)(struct ArrowAsyncDeviceStreamHandler* self,
+ int code, const char* message, const char* metadata);
+
+ // release callback
+ void (*release)(struct ArrowAsyncDeviceStreamHandler* self);
+
+ // opaque handler-specific data
+ void* private_data;
+ };
+
+ #endif // ARROW_C_ASYNC_STREAM_INTERFACE
+
+.. note::
+ The canonical guard ``ARROW_C_ASYNC_STREAM_INTERFACE`` is meant to avoid
+ duplicate definitions if two projects copy the C async stream interface
+ definitions into their own headers, and a third-party project includes
+ from these two projects. It is therefore important that this guard is kept
+ exactly as-is when these definitions are copied.
+
+The ArrowAsyncDeviceStreamHandler structure
+'''''''''''''''''''''''''''''''''''''''''''
+
+The structure has the following fields:
+
+.. c:member:: int (*ArrowAsyncDeviceStreamHandler.on_schema)(struct ArrowAsyncDeviceStreamHandler*, struct ArrowAsyncProducer*, struct ArrowSchema*, const char*)
+
+ *Mandatory.* Handler for receiving the schema of the stream. All records should
+ match the provided schema. If successful, the function should return 0, otherwise
+ it should return an ``errno``-compatible error code.
+
+ The ``const char*`` parameter exists for producers to provide any extra contextual information
+ they want, such as the total number of rows in the stream, statistics, or otherwise. This is
+ encoded in the same format as :c:member:`ArrowSchema.metadata`. If not ``NULL``,
+ the lifetime is only the scope of the call to this function. A consumer who wants to maintain
+ the additional metadata beyond the lifetime of this call *MUST* copy the value themselves.
+
+ Unless the ``on_error`` handler is called, this will always get called exactly once and will be
+ the first method called on this object. As such the producer *MUST* provide an ``ArrowAsyncProducer``
+ object when calling this function to allow the consumer to manage back-pressure and flow control.
+ The producer maintains ownership of the ``ArrowAsyncProducer`` and must clean it up before or after
+ calling the release callback on this object.
+
+ A producer that receives a non-zero result here must not subsequently call anything other than
+ the release callback on this object.
+
+.. c:member:: int (*ArrowAsyncDeviceStreamHandler.on_next_task)(struct ArrowAsyncDeviceStreamHandler*, struct ArrowAsyncTask*, const char*)
+
+ *Mandatory.* Handler to be called when a new record is available for processing. The
+ schema for each record should be the same as the schema that ``on_schema`` was called with.
+ If successfully handled, the function should return 0, otherwise it should return an
+ ``errno``-compatible error code.
+
+ Rather than passing the record itself it receives an ``ArrowAsyncTask`` instead to facilitate
+ better consumer-focused thread control as far as receiving the data. A call to this function
+ simply indicates that data is available via the provided task.
+
+ The ``const char*`` parameter exists for producers to provide any extra contextual information
+ they want. This is encoded in the same format as :c:member:`ArrowSchema.metadata`. If not ``NULL``,
+ the lifetime is only the scope of the call to this function. A consumer who wants to maintain
+ the additional metadata beyond the lifetime of this call *MUST* copy the value themselves.
+
+ A producer *MUST NOT* call this concurrently from multiple different threads.
+
+ The :c:member:`ArrowAsyncProducer.request` callback must be called to start receiving calls to this
+ handler.
+
+ This function *MUST* be able to be called re-entrantly on the same thread to allow for the
+ :c:member:`ArrowAsyncProducer.request` callback to potentially call this method recursively.
+
+.. c:member:: void (*ArrowAsyncDeviceStreamHandler.on_error)(struct ArrowAsyncDeviceStreamHandler, int, const char*, const char*)
+
+ *Mandatory.* Handler to be called when an error is encountered by the producer. After calling
+ this, the ``release`` callback will be called as the last call on this struct. The parameters
+ are an ``errno``-compatible error code and an optional error message and metadata.
+
+ If the message and metadata are not ``NULL``, their lifetime is only valid during the scope
+ of this call. A consumer who wants to maintain these values past the return of this function
+ *MUST* copy the values themselves.
+
+ If the metadata parameter is not ``NULL``, to provide key-value error metadata, then it should
+ be encoded identically to the way that metadata is encoded in :c:member:`ArrowSchema.metadata`.
+
+ It is valid for this to be called by a producer with or without a preceding call to
+ :c:member:`ArrowAsyncProducer.request`. This callback *MUST NOT* call any methods of an
+ ``ArrowAsyncProducer`` object.
+
+.. c:member:: void (*ArrowAsyncDeviceStreamHandler.release)(struct ArrowAsyncDeviceStreamHandler*)
+
+ *Mandatory.* A pointer to a consumer-provided release callback for the handler.
+
+ It is valid for this to be called by a producer with or without a preceding call to
+ :c:member:`ArrowAsyncProducer.request`. This must not call any methods of an ``ArrowAsyncProducer``
+ object.
+
+.. c:member:: void* ArrowAsyncDeviceStreamHandler.private_data
+
+ *Optional.* An opaque pointer to consumer-provided private data.
+
+ Producers *MUST NOT* process this member. Lifetime of this member is handled by
+ the consumer, and especially by the release callback.
+
+The ArrowAsyncTask structure
+''''''''''''''''''''''''''''
+
+The purpose of using a Task object rather than passing the array directly to the ``on_next``
+callback is to allow for more complex and efficient thread handling. Utilizing a Task
+object allows for a producer to separate the "decoding" logic from the I/O, enabling a
+consumer to avoid transferring data between CPU cores (e.g. from one L1/L2 cache to another).
+
+This producer-provided structure has the following fields:
+
+.. c:member:: int (*ArrowArrayTask.get_data)(struct ArrowArrayTask*, struct ArrowDeviceArray*)
Review Comment:
The producer *is* deciding. The producer is what populates the `get_data` callback. If there's nothing beneficial to offload, then the function just returns the data directly. If there *is* something beneficial to offload then that logic is encapsulated in the `get_data` function. This pattern just allows the producer to make that decision.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-43631: [C][Format] Add ArrowAsyncDeviceStreamHandler interface [arrow]
Posted by "zeroshade (via GitHub)" <gi...@apache.org>.
zeroshade commented on PR #43632:
URL: https://github.com/apache/arrow/pull/43632#issuecomment-2372249270
Do we have any further comments on here? I'll give this till EOD tomorrow and if no one objects I'll put the vote on the mailing list at that point.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-43631: [C][Format] Add ArrowAsyncDeviceStreamHandler interface [arrow]
Posted by "lidavidm (via GitHub)" <gi...@apache.org>.
lidavidm commented on code in PR #43632:
URL: https://github.com/apache/arrow/pull/43632#discussion_r1755932219
##########
cpp/src/arrow/c/abi.h:
##########
@@ -228,6 +228,180 @@ struct ArrowDeviceArrayStream {
#endif // ARROW_C_DEVICE_STREAM_INTERFACE
+#ifndef ARROW_C_ASYNC_STREAM_INTERFACE
+#define ARROW_C_ASYNC_STREAM_INTERFACE
+
+// ArrowAsyncTask represents available data from a producer that was passed to
+// an invocation of `on_next_task` on the ArrowAsyncDeviceStreamHandler.
+struct ArrowAsyncTask {
+ // This callback should populate the ArrowDeviceArray associated with this task.
+ // The order of ArrowAsyncTasks provided by the producer enables a consumer to
+ // ensure the order of data to process.
+ //
+ // Returns: 0 if successful, errno-compatible error otherwise.
+ //
+ // If a non-0 value is returned then it should be followed by a call to `on_error`
+ // on the appropriate ArrowAsyncDeviceStreamHandler.
Review Comment:
Actually, (1) do we want this to return an error at all? (2) if we do, would it be better to deliver it as a callback (which is potentially/probably entirely disconnected from the current control flow) or have the regular get_last_error method?
##########
cpp/src/arrow/c/abi.h:
##########
@@ -228,6 +228,180 @@ struct ArrowDeviceArrayStream {
#endif // ARROW_C_DEVICE_STREAM_INTERFACE
+#ifndef ARROW_C_ASYNC_STREAM_INTERFACE
+#define ARROW_C_ASYNC_STREAM_INTERFACE
+
+// ArrowAsyncTask represents available data from a producer that was passed to
+// an invocation of `on_next_task` on the ArrowAsyncDeviceStreamHandler.
+struct ArrowAsyncTask {
+ // This callback should populate the ArrowDeviceArray associated with this task.
+ // The order of ArrowAsyncTasks provided by the producer enables a consumer to
+ // ensure the order of data to process.
+ //
+ // Returns: 0 if successful, errno-compatible error otherwise.
+ //
+ // If a non-0 value is returned then it should be followed by a call to `on_error`
+ // on the appropriate ArrowAsyncDeviceStreamHandler.
+ //
+ // Rather than a release callback, any required cleanup should be performed as part
+ // of the invocation of `get_data`. Ownership of the Array is passed to the consumer
+ // calling this, and so it must be released separately.
+ //
+ // It is only valid to call this method exactly once.
+ int (*get_data)(struct ArrowArrayTask* self, struct ArrowDeviceArray* out);
+
+ // opaque task-specific data
+ void* private_data;
+};
+
+// ArrowAsyncProducer represents a 1-to-1 relationship between an async producer
+// and consumer. This object allows the consumer to perform backpressure and flow
+// control on the asynchronous stream processing. This object must be owned by the
+// producer who creates it, and thus is responsible for cleaning it up.
+struct ArrowAsyncProducer {
+ // A consumer must call this function to start receiving on_next_task calls.
+ //
+ // It *must* be valid to call this synchronously from within on_next_task or
+ // on_schema, with a producer placing an upper bound on possible synchronous
+ // recursion between calls of on_next_task -> request -> on_next_task ->....
+ //
+ // After cancel has been called, additional calls to this function must be NOPs,
+ // but allowed.
+ //
+ // While not cancelled, calling this function must register the given number of
+ // additional arrays/batches to be produced with the producer. The producer should
+ // only call `on_next_task` at most the registered number of arrays before propagating
+ // backpressure.
+ //
+ // While not cancelled, calling request MAY synchronously call `on_next_task`,
+ // `on_error`, or `release` on the ArrowAsyncDeviceStreamHandler.
+ //
+ // Any error encountered by calling request must be propagated by calling the `on_error`
+ // callback of the ArrowAsyncDeviceStreamHandler.
+ //
+ // A producer must support an unbounded number of calls to request and must support
+ // a total registered demand (sum requested - sum delivered) of up to UINT64_MAX.
+ void (*request)(struct ArrowAsyncProducer* self, uint64_t n);
+
+ // This cancel callback signals a producer that it must eventually stop making calls
+ // to on_next_task. It must be idempotent and thread-safe.
+ //
+ // After calling cancel once, subsequent calls must be NOPs.
+ //
+ // It is not required that calling cancel affect the producer immediately, only that it
+ // must eventually stop calling on_next_task and subsequently call release on the
+ // async handler. As such, a consumer must be prepared to receive one or more calls to
+ // `on_next_task` even after calling cancel if there are still requested arrays pending.
+ //
+ // Any error encountered during handling a call to cancel must be reported via the on_error
+ // callback on the async stream handler.
+ void (*cancel)(struct ArrowAsyncProducer* self);
+
+ // producer-specific opaque data.
+ void* private_data;
+};
+
+// Similar to ArrowDeviceArrayStream, except designed for an asynchronous
+// style of interaction. While ArrowDeviceArrayStream provides producer
+// defined callbacks, this is intended to be created by the consumer instead.
+// The consumer passes this handler to the producer, which in turn uses the
+// callbacks to inform the consumer of events in the stream.
+struct ArrowAsyncDeviceStreamHandler {
+ // Handler for receiving a schema. The passed in stream_schema must be
+ // released or moved by the handler (producer is giving ownership of the schema to
+ // the handler, but not ownership of the top level object itself).
+ //
+ // With the exception of an error occurring (on_error), this must be the first
+ // callback function which is called by a producer and must only be called exactly
+ // once. As such, the producer should provide a valid ArrowAsyncProducer instance
+ // so the consumer can control the flow. See the documentation on ArrowAsyncProducer
+ // for how it works. The ArrowAsyncProducer is owned by the producer who calls this
+ // function and thus the producer is responsible for cleaning it up when calling
+ // the release callback of this handler.
+ //
+ // The addl_metadata argument can be null or can be used by a producer
+ // to pass arbitrary extra information to the consumer beyond the metadata in the schema
+ // itself (such as total number of rows, context info, or otherwise). The data should
+ // be passed using the same encoding as the metadata within the ArrowSchema struct
+ // itself (defined in the spec at
+ // https://arrow.apache.org/docs/format/CDataInterface.html#c.ArrowSchema.metadata)
+ //
+ // If addl_metadata is non-null then it only needs to exist for the lifetime of this call,
+ // a consumer who wants it to live after that must copy it to ensure lifetime.
+ //
+ // Return value: 0 if successful, `errno`-compatible error otherwise
+ //
+ // A producer that receives a non-zero return here should stop producing and eventually
+ // call release instead.
+ int (*on_schema)(struct ArrowAsyncDeviceStreamHandler* self,
+ struct ArrowAsyncProducer* producer, struct ArrowSchema* stream_schema,
+ const char* addl_metadata);
+
+ // Handler for receiving data. This is called when data is available providing an
+ // ArrowAsyncTask struct to signify it. The consumer is responsible for calling
+ // the release callback on the ArrowAsyncTask after retrieving the array/batch.
+ // An empty/released task is passed to indicate the end of the stream if no
+ // errors have been encountered.
+ //
+ // The `request` callback of a provided ArrowAsyncProducer must be called in order
+ // to start receiving calls to this handler.
+ //
+ // The metadata argument can be null or can be used by a producer
+ // to pass arbitrary extra information to the consumer (such as total number
+ // of rows, context info, or otherwise). The data should be passed using the same
+ // encoding as the metadata within the ArrowSchema struct itself (defined in
+ // the spec at
+ // https://arrow.apache.org/docs/format/CDataInterface.html#c.ArrowSchema.metadata)
+ //
+ // If metadata is non-null then it only needs to exist for the lifetime of this call,
+ // a consumer who wants it to live after that must copy it to ensure lifetime.
+ //
+ // A producer *must not* call this concurrently from multiple different threads.
+ //
+ // A consumer must be prepared to received one or more calls to this callback even
+ // after calling cancel on the corresponding ArrowAsyncProducer, as cancel does not
+ // guarantee it happens immediately.
+ //
+ // Return value: 0 if successful, `errno`-compatible error otherwise.
+ int (*on_next_task)(struct ArrowAsyncDeviceStreamHandler* self,
+ struct ArrowAsyncTask* task, const char* metadata);
Review Comment:
ping
##########
cpp/src/arrow/c/abi.h:
##########
@@ -228,6 +228,180 @@ struct ArrowDeviceArrayStream {
#endif // ARROW_C_DEVICE_STREAM_INTERFACE
+#ifndef ARROW_C_ASYNC_STREAM_INTERFACE
+#define ARROW_C_ASYNC_STREAM_INTERFACE
+
+// ArrowAsyncTask represents available data from a producer that was passed to
Review Comment:
(by the way, I think you need `///` for Doxygen to pick this up as a doc comment.)
##########
cpp/src/arrow/c/abi.h:
##########
@@ -228,6 +228,180 @@ struct ArrowDeviceArrayStream {
#endif // ARROW_C_DEVICE_STREAM_INTERFACE
+#ifndef ARROW_C_ASYNC_STREAM_INTERFACE
+#define ARROW_C_ASYNC_STREAM_INTERFACE
+
+// ArrowAsyncTask represents available data from a producer that was passed to
+// an invocation of `on_next_task` on the ArrowAsyncDeviceStreamHandler.
+struct ArrowAsyncTask {
+ // This callback should populate the ArrowDeviceArray associated with this task.
+ // The order of ArrowAsyncTasks provided by the producer enables a consumer to
+ // ensure the order of data to process.
+ //
+ // Returns: 0 if successful, errno-compatible error otherwise.
+ //
+ // If a non-0 value is returned then it should be followed by a call to `on_error`
+ // on the appropriate ArrowAsyncDeviceStreamHandler.
+ //
+ // Rather than a release callback, any required cleanup should be performed as part
+ // of the invocation of `get_data`. Ownership of the Array is passed to the consumer
+ // calling this, and so it must be released separately.
+ //
+ // It is only valid to call this method exactly once.
+ int (*get_data)(struct ArrowArrayTask* self, struct ArrowDeviceArray* out);
+
+ // opaque task-specific data
+ void* private_data;
+};
+
+// ArrowAsyncProducer represents a 1-to-1 relationship between an async producer
+// and consumer. This object allows the consumer to perform backpressure and flow
+// control on the asynchronous stream processing. This object must be owned by the
+// producer who creates it, and thus is responsible for cleaning it up.
+struct ArrowAsyncProducer {
+ // A consumer must call this function to start receiving on_next_task calls.
+ //
+ // It *must* be valid to call this synchronously from within on_next_task or
+ // on_schema, with a producer placing an upper bound on possible synchronous
+ // recursion between calls of on_next_task -> request -> on_next_task ->....
Review Comment:
I'm not saying that the consumer isn't allowed to call it, just that the producer should schedule the task later instead of immediately recursing. It would also avoid reentrant callbacks.
##########
cpp/src/arrow/c/abi.h:
##########
@@ -228,6 +228,180 @@ struct ArrowDeviceArrayStream {
#endif // ARROW_C_DEVICE_STREAM_INTERFACE
+#ifndef ARROW_C_ASYNC_STREAM_INTERFACE
+#define ARROW_C_ASYNC_STREAM_INTERFACE
+
+// ArrowAsyncTask represents available data from a producer that was passed to
+// an invocation of `on_next_task` on the ArrowAsyncDeviceStreamHandler.
+struct ArrowAsyncTask {
+ // This callback should populate the ArrowDeviceArray associated with this task.
Review Comment:
Ping
##########
cpp/src/arrow/c/abi.h:
##########
@@ -228,6 +228,180 @@ struct ArrowDeviceArrayStream {
#endif // ARROW_C_DEVICE_STREAM_INTERFACE
+#ifndef ARROW_C_ASYNC_STREAM_INTERFACE
+#define ARROW_C_ASYNC_STREAM_INTERFACE
+
+// ArrowAsyncTask represents available data from a producer that was passed to
+// an invocation of `on_next_task` on the ArrowAsyncDeviceStreamHandler.
+struct ArrowAsyncTask {
+ // This callback should populate the ArrowDeviceArray associated with this task.
+ // The order of ArrowAsyncTasks provided by the producer enables a consumer to
+ // ensure the order of data to process.
+ //
+ // Returns: 0 if successful, errno-compatible error otherwise.
+ //
+ // If a non-0 value is returned then it should be followed by a call to `on_error`
+ // on the appropriate ArrowAsyncDeviceStreamHandler.
+ //
+ // Rather than a release callback, any required cleanup should be performed as part
+ // of the invocation of `get_data`. Ownership of the Array is passed to the consumer
+ // calling this, and so it must be released separately.
+ //
+ // It is only valid to call this method exactly once.
+ int (*get_data)(struct ArrowArrayTask* self, struct ArrowDeviceArray* out);
+
+ // opaque task-specific data
+ void* private_data;
+};
+
+// ArrowAsyncProducer represents a 1-to-1 relationship between an async producer
+// and consumer. This object allows the consumer to perform backpressure and flow
+// control on the asynchronous stream processing. This object must be owned by the
+// producer who creates it, and thus is responsible for cleaning it up.
+struct ArrowAsyncProducer {
+ // A consumer must call this function to start receiving on_next_task calls.
+ //
+ // It *must* be valid to call this synchronously from within on_next_task or
+ // on_schema, with a producer placing an upper bound on possible synchronous
+ // recursion between calls of on_next_task -> request -> on_next_task ->....
+ //
+ // After cancel has been called, additional calls to this function must be NOPs,
+ // but allowed.
+ //
+ // While not cancelled, calling this function must register the given number of
+ // additional arrays/batches to be produced with the producer. The producer should
+ // only call `on_next_task` at most the registered number of arrays before propagating
+ // backpressure.
+ //
+ // While not cancelled, calling request MAY synchronously call `on_next_task`,
+ // `on_error`, or `release` on the ArrowAsyncDeviceStreamHandler.
+ //
+ // Any error encountered by calling request must be propagated by calling the `on_error`
+ // callback of the ArrowAsyncDeviceStreamHandler.
+ //
+ // A producer must support an unbounded number of calls to request and must support
+ // a total registered demand (sum requested - sum delivered) of up to UINT64_MAX.
+ void (*request)(struct ArrowAsyncProducer* self, uint64_t n);
+
+ // This cancel callback signals a producer that it must eventually stop making calls
+ // to on_next_task. It must be idempotent and thread-safe.
+ //
+ // After calling cancel once, subsequent calls must be NOPs.
+ //
+ // It is not required that calling cancel affect the producer immediately, only that it
+ // must eventually stop calling on_next_task and subsequently call release on the
+ // async handler. As such, a consumer must be prepared to receive one or more calls to
+ // `on_next_task` even after calling cancel if there are still requested arrays pending.
+ //
+ // Any error encountered during handling a call to cancel must be reported via the on_error
+ // callback on the async stream handler.
+ void (*cancel)(struct ArrowAsyncProducer* self);
Review Comment:
I don't see this explicitly documented?
##########
cpp/src/arrow/c/abi.h:
##########
@@ -228,6 +228,180 @@ struct ArrowDeviceArrayStream {
#endif // ARROW_C_DEVICE_STREAM_INTERFACE
+#ifndef ARROW_C_ASYNC_STREAM_INTERFACE
+#define ARROW_C_ASYNC_STREAM_INTERFACE
+
+// ArrowAsyncTask represents available data from a producer that was passed to
Review Comment:
Can we document the rationale for having the task vs just passing the array to the callback directly?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-43631: [C][Format] Add ArrowAsyncDeviceStreamHandler interface [arrow]
Posted by "pedroerp (via GitHub)" <gi...@apache.org>.
pedroerp commented on PR #43632:
URL: https://github.com/apache/arrow/pull/43632#issuecomment-2339503509
@leventov the Velox part seems mostly accurate, with the exception we don't do a lot of cross-language flow control. The common pattern is to delegate the execution of an entire query fragment to Velox, and let Velox use exchanges to shuffle data across different hosts running the same Velox process. But then, Exchange* is just one implementation based on Presto's requirements; one could provide their own shuffle operators implementing specialized behavior.
Also, in Velox parlance, Task is how you execute query plans; Drivers are the threads of execution inside Tasks; Splits are the unit of processing (one split can be a segment of a Parquet file, for example), and it ends up being processed by one Driver.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-43631: [C][Format] Add ArrowAsyncDeviceStreamHandler interface [arrow]
Posted by "zeroshade (via GitHub)" <gi...@apache.org>.
zeroshade commented on code in PR #43632:
URL: https://github.com/apache/arrow/pull/43632#discussion_r1759023061
##########
docs/source/format/CDeviceDataInterface.rst:
##########
@@ -650,6 +652,331 @@ The stream source is not assumed to be thread-safe. Consumers wanting to
call ``get_next`` from several threads should ensure those calls are
serialized.
+Async Device Stream Interface
+=============================
+
+The :ref:`C stream interface <_c-device-stream-interface>` provides a synchronous
+API centered around the consumer calling the callback functions to retrieve
+the next record batch. For some bindings, use cases, and interoperability, a more
+asynchronous, producer-focused interface may be required. These scenarios can utilize
+the ``ArrowAsyncDeviceStreamHandler``.
+
+Semantics
+---------
+
+Rather than the producer providing a structure of callbacks for a consumer to
+call and retrieve records, the Async interface is a consumer allocated structure.
+The consumer allocated struct provides handler callbacks for the producer to call
+when the schema and chunks of data are available, rather than the consumer using
+a blocking pull-style iteration.
+
+In addition to the ``ArrowAsyncDeviceStreamHandler``, there are also two additional
+structs used for the full data flow: ``ArrowAsyncTask`` and ``ArrowAsyncProducer``.
+
+Structure Definition
+--------------------
+
+The C device async stream interface is defined with a single ``struct`` definition:
+
+.. code-block:: c
+
+ #ifndef ARROW_C_ASYNC_STREAM_INTERFACE
+ #define ARROW_C_ASYNC_STREAM_INTERFACE
+
+ struct ArrowAsyncTask {
+ int (*get_data)(struct ArrowArrayTask* self, struct ArrowDeviceArray* out);
+
+ void* private_data;
+ };
+
+ struct ArrowAsyncProducer {
+ void (*request)(struct ArrowAsyncProducer* self, uint64_t n);
+ void (*cancel)(struct ArrowAsyncProducer* self);
+
+ void (*release)(struct ArrowAsyncProducer* self);
+ void* private_data;
+ };
+
+ struct ArrowAsyncDeviceStreamHandler {
+ // handlers
+ int (*on_schema)(struct ArrowAsyncDeviceStreamHandler* self,
+ struct ArrowAsyncProducer* producer,
+ struct ArrowSchema* stream_schema, const char* addl_metadata);
+ int (*on_next_task)(struct ArrowAsyncDeviceStreamHandler* self,
+ struct ArrowAsyncTask* task, const char* metadata);
+ void (*on_error)(struct ArrowAsyncDeviceStreamHandler* self,
+ int code, const char* message, const char* metadata);
+
+ // release callback
+ void (*release)(struct ArrowAsyncDeviceStreamHandler* self);
+
+ // opaque handler-specific data
+ void* private_data;
+ };
+
+ #endif // ARROW_C_ASYNC_STREAM_INTERFACE
+
+.. note::
+ The canonical guard ``ARROW_C_ASYNC_STREAM_INTERFACE`` is meant to avoid
+ duplicate definitions if two projects copy the C async stream interface
+ definitions into their own headers, and a third-party project includes
+ from these two projects. It is therefore important that this guard is kept
+ exactly as-is when these definitions are copied.
+
+The ArrowAsyncDeviceStreamHandler structure
+'''''''''''''''''''''''''''''''''''''''''''
+
+The structure has the following fields:
+
+.. c:member:: int (*ArrowAsyncDeviceStreamHandler.on_schema)(struct ArrowAsyncDeviceStreamHandler*, struct ArrowAsyncProducer*, struct ArrowSchema*, const char*)
+
+ *Mandatory.* Handler for receiving the schema of the stream. All records should
+ match the provided schema. If successful, the function should return 0, otherwise
+ it should return an ``errno``-compatible error code.
+
+ The ``const char*`` parameter exists for producers to provide any extra contextual information
+ they want, such as the total number of rows in the stream, statistics, or otherwise. This is
+ encoded in the same format as :c:member:`ArrowSchema.metadata`. If not ``NULL``,
+ the lifetime is only the scope of the call to this function. A consumer who wants to maintain
+ the additional metadata beyond the lifetime of this call *MUST* copy the value themselves.
+
+ Unless the ``on_error`` handler is called, this will always get called exactly once and will be
+ the first method called on this object. As such the producer *MUST* provide an ``ArrowAsyncProducer``
+ object when calling this function to allow the consumer to manage back-pressure and flow control.
+ The producer maintains ownership of the ``ArrowAsyncProducer`` and must clean it up before or after
+ calling the release callback on this object.
+
+ A producer that receives a non-zero result here must not subsequently call anything other than
+ the release callback on this object.
+
+.. c:member:: int (*ArrowAsyncDeviceStreamHandler.on_next_task)(struct ArrowAsyncDeviceStreamHandler*, struct ArrowAsyncTask*, const char*)
+
+ *Mandatory.* Handler to be called when a new record is available for processing. The
+ schema for each record should be the same as the schema that ``on_schema`` was called with.
+ If successfully handled, the function should return 0, otherwise it should return an
+ ``errno``-compatible error code.
+
+ Rather than passing the record itself it receives an ``ArrowAsyncTask`` instead to facilitate
+ better consumer-focused thread control as far as receiving the data. A call to this function
+ simply indicates that data is available via the provided task.
+
+ The ``const char*`` parameter exists for producers to provide any extra contextual information
+ they want. This is encoded in the same format as :c:member:`ArrowSchema.metadata`. If not ``NULL``,
+ the lifetime is only the scope of the call to this function. A consumer who wants to maintain
+ the additional metadata beyond the lifetime of this call *MUST* copy the value themselves.
+
+ A producer *MUST NOT* call this concurrently from multiple different threads.
+
+ The :c:member:`ArrowAsyncProducer.request` callback must be called to start receiving calls to this
+ handler.
+
+ This function *MUST* be able to be called re-entrantly on the same thread to allow for the
+ :c:member:`ArrowAsyncProducer.request` callback to potentially call this method recursively.
+
+.. c:member:: void (*ArrowAsyncDeviceStreamHandler.on_error)(struct ArrowAsyncDeviceStreamHandler, int, const char*, const char*)
+
+ *Mandatory.* Handler to be called when an error is encountered by the producer. After calling
+ this, the ``release`` callback will be called as the last call on this struct. The parameters
+ are an ``errno``-compatible error code and an optional error message and metadata.
+
+ If the message and metadata are not ``NULL``, their lifetime is only valid during the scope
+ of this call. A consumer who wants to maintain these values past the return of this function
+ *MUST* copy the values themselves.
+
+ If the metadata parameter is not ``NULL``, to provide key-value error metadata, then it should
+ be encoded identically to the way that metadata is encoded in :c:member:`ArrowSchema.metadata`.
+
+ It is valid for this to be called by a producer with or without a preceding call to
+ :c:member:`ArrowAsyncProducer.request`. This callback *MUST NOT* call any methods of an
+ ``ArrowAsyncProducer`` object.
+
+.. c:member:: void (*ArrowAsyncDeviceStreamHandler.release)(struct ArrowAsyncDeviceStreamHandler*)
+
+ *Mandatory.* A pointer to a consumer-provided release callback for the handler.
+
+ It is valid for this to be called by a producer with or without a preceding call to
+ :c:member:`ArrowAsyncProducer.request`. This must not call any methods of an ``ArrowAsyncProducer``
+ object.
+
+.. c:member:: void* ArrowAsyncDeviceStreamHandler.private_data
+
+ *Optional.* An opaque pointer to consumer-provided private data.
+
+ Producers *MUST NOT* process this member. Lifetime of this member is handled by
+ the consumer, and especially by the release callback.
+
+The ArrowAsyncTask structure
+''''''''''''''''''''''''''''
+
+The purpose of using a Task object rather than passing the array directly to the ``on_next``
+callback is to allow for more complex and efficient thread handling. Utilizing a Task
+object allows for a producer to separate the "decoding" logic from the I/O, enabling a
+consumer to avoid transferring data between CPU cores (e.g. from one L1/L2 cache to another).
+
+This producer-provided structure has the following fields:
+
+.. c:member:: int (*ArrowArrayTask.get_data)(struct ArrowArrayTask*, struct ArrowDeviceArray*)
+
+ *Mandatory.* A callback to populate the provided ``ArrowDeviceArray`` with the available data.
+ The order of ArrowAsyncTasks provided by the producer enables a consumer to know the order of
+ the data to process.
+
+ If a non-zero value is returned from this, it should be followed only by the producer calling
+ the ``on_error`` callback of the ``ArrowAsyncDeviceStreamHandler``. Because calling this method
+ is likely to be separate from the current control flow, returning a non-zero value to signal
+ an error occuring allows the current thread to decide handle the case accordingly, while still
+ allowing all error logging and handling to be centralized in the
+ :c:member:`ArrowAsyncDeviceStreamHandler.on_error` callback.
+
+ Rather than having a separate release callback, any required cleanup should be performed as part
+ of the invocation of this callback, with ownership of the Array being passed to the consumer-provided
+ parameter calling this, and must be released separately.
+
+ It is only valid to call this method exactly once.
+
+.. c:member:: void* ArrowArrayTask.private_data
+
+ *Optional.* An opaque pointer to consumer-provided private data.
+
+ Consumers *MUST NOT* process this member. Lifetime of this member is handled by
+ the producer who created this object, and should be cleaned up if necessary during
+ the call to :c:member:`ArrowArrayTask.get_data`.
+
+The ArrowAsyncProducer structure
+''''''''''''''''''''''''''''''''
+
+This producer-provided and managed object has the following fields:
+
+.. c:member:: void (*ArrowAsyncProducer.request)(struct ArrowAsyncProducer*, uint64_t)
Review Comment:
any particular reason? The purpose of using `uint64_t` here was so that implementers don't need to manually validate that the value is > 0
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-43631: [C][Format] Add ArrowAsyncDeviceStreamHandler interface [arrow]
Posted by "zeroshade (via GitHub)" <gi...@apache.org>.
zeroshade commented on code in PR #43632:
URL: https://github.com/apache/arrow/pull/43632#discussion_r1759124776
##########
docs/source/format/CDeviceDataInterface.rst:
##########
@@ -650,6 +652,331 @@ The stream source is not assumed to be thread-safe. Consumers wanting to
call ``get_next`` from several threads should ensure those calls are
serialized.
+Async Device Stream Interface
+=============================
+
+The :ref:`C stream interface <_c-device-stream-interface>` provides a synchronous
+API centered around the consumer calling the callback functions to retrieve
+the next record batch. For some bindings, use cases, and interoperability, a more
+asynchronous, producer-focused interface may be required. These scenarios can utilize
+the ``ArrowAsyncDeviceStreamHandler``.
+
+Semantics
+---------
+
+Rather than the producer providing a structure of callbacks for a consumer to
+call and retrieve records, the Async interface is a consumer allocated structure.
+The consumer allocated struct provides handler callbacks for the producer to call
+when the schema and chunks of data are available, rather than the consumer using
+a blocking pull-style iteration.
+
+In addition to the ``ArrowAsyncDeviceStreamHandler``, there are also two additional
+structs used for the full data flow: ``ArrowAsyncTask`` and ``ArrowAsyncProducer``.
+
+Structure Definition
+--------------------
+
+The C device async stream interface is defined with a single ``struct`` definition:
+
+.. code-block:: c
+
+ #ifndef ARROW_C_ASYNC_STREAM_INTERFACE
+ #define ARROW_C_ASYNC_STREAM_INTERFACE
+
+ struct ArrowAsyncTask {
+ int (*get_data)(struct ArrowArrayTask* self, struct ArrowDeviceArray* out);
+
+ void* private_data;
+ };
+
+ struct ArrowAsyncProducer {
+ void (*request)(struct ArrowAsyncProducer* self, uint64_t n);
+ void (*cancel)(struct ArrowAsyncProducer* self);
+
+ void (*release)(struct ArrowAsyncProducer* self);
+ void* private_data;
+ };
+
+ struct ArrowAsyncDeviceStreamHandler {
+ // handlers
+ int (*on_schema)(struct ArrowAsyncDeviceStreamHandler* self,
+ struct ArrowAsyncProducer* producer,
+ struct ArrowSchema* stream_schema, const char* addl_metadata);
+ int (*on_next_task)(struct ArrowAsyncDeviceStreamHandler* self,
+ struct ArrowAsyncTask* task, const char* metadata);
+ void (*on_error)(struct ArrowAsyncDeviceStreamHandler* self,
+ int code, const char* message, const char* metadata);
+
+ // release callback
+ void (*release)(struct ArrowAsyncDeviceStreamHandler* self);
+
+ // opaque handler-specific data
+ void* private_data;
+ };
+
+ #endif // ARROW_C_ASYNC_STREAM_INTERFACE
+
+.. note::
+ The canonical guard ``ARROW_C_ASYNC_STREAM_INTERFACE`` is meant to avoid
+ duplicate definitions if two projects copy the C async stream interface
+ definitions into their own headers, and a third-party project includes
+ from these two projects. It is therefore important that this guard is kept
+ exactly as-is when these definitions are copied.
+
+The ArrowAsyncDeviceStreamHandler structure
+'''''''''''''''''''''''''''''''''''''''''''
+
+The structure has the following fields:
+
+.. c:member:: int (*ArrowAsyncDeviceStreamHandler.on_schema)(struct ArrowAsyncDeviceStreamHandler*, struct ArrowAsyncProducer*, struct ArrowSchema*, const char*)
+
+ *Mandatory.* Handler for receiving the schema of the stream. All records should
+ match the provided schema. If successful, the function should return 0, otherwise
+ it should return an ``errno``-compatible error code.
+
+ The ``const char*`` parameter exists for producers to provide any extra contextual information
+ they want, such as the total number of rows in the stream, statistics, or otherwise. This is
+ encoded in the same format as :c:member:`ArrowSchema.metadata`. If not ``NULL``,
+ the lifetime is only the scope of the call to this function. A consumer who wants to maintain
+ the additional metadata beyond the lifetime of this call *MUST* copy the value themselves.
+
+ Unless the ``on_error`` handler is called, this will always get called exactly once and will be
+ the first method called on this object. As such the producer *MUST* provide an ``ArrowAsyncProducer``
+ object when calling this function to allow the consumer to manage back-pressure and flow control.
+ The producer maintains ownership of the ``ArrowAsyncProducer`` and must clean it up before or after
+ calling the release callback on this object.
+
+ A producer that receives a non-zero result here must not subsequently call anything other than
+ the release callback on this object.
+
+.. c:member:: int (*ArrowAsyncDeviceStreamHandler.on_next_task)(struct ArrowAsyncDeviceStreamHandler*, struct ArrowAsyncTask*, const char*)
+
+ *Mandatory.* Handler to be called when a new record is available for processing. The
+ schema for each record should be the same as the schema that ``on_schema`` was called with.
+ If successfully handled, the function should return 0, otherwise it should return an
+ ``errno``-compatible error code.
+
+ Rather than passing the record itself it receives an ``ArrowAsyncTask`` instead to facilitate
+ better consumer-focused thread control as far as receiving the data. A call to this function
+ simply indicates that data is available via the provided task.
+
+ The ``const char*`` parameter exists for producers to provide any extra contextual information
+ they want. This is encoded in the same format as :c:member:`ArrowSchema.metadata`. If not ``NULL``,
+ the lifetime is only the scope of the call to this function. A consumer who wants to maintain
+ the additional metadata beyond the lifetime of this call *MUST* copy the value themselves.
+
+ A producer *MUST NOT* call this concurrently from multiple different threads.
+
+ The :c:member:`ArrowAsyncProducer.request` callback must be called to start receiving calls to this
+ handler.
+
+ This function *MUST* be able to be called re-entrantly on the same thread to allow for the
+ :c:member:`ArrowAsyncProducer.request` callback to potentially call this method recursively.
+
+.. c:member:: void (*ArrowAsyncDeviceStreamHandler.on_error)(struct ArrowAsyncDeviceStreamHandler, int, const char*, const char*)
+
+ *Mandatory.* Handler to be called when an error is encountered by the producer. After calling
+ this, the ``release`` callback will be called as the last call on this struct. The parameters
+ are an ``errno``-compatible error code and an optional error message and metadata.
+
+ If the message and metadata are not ``NULL``, their lifetime is only valid during the scope
+ of this call. A consumer who wants to maintain these values past the return of this function
+ *MUST* copy the values themselves.
+
+ If the metadata parameter is not ``NULL``, to provide key-value error metadata, then it should
+ be encoded identically to the way that metadata is encoded in :c:member:`ArrowSchema.metadata`.
+
+ It is valid for this to be called by a producer with or without a preceding call to
+ :c:member:`ArrowAsyncProducer.request`. This callback *MUST NOT* call any methods of an
+ ``ArrowAsyncProducer`` object.
+
+.. c:member:: void (*ArrowAsyncDeviceStreamHandler.release)(struct ArrowAsyncDeviceStreamHandler*)
+
+ *Mandatory.* A pointer to a consumer-provided release callback for the handler.
+
+ It is valid for this to be called by a producer with or without a preceding call to
+ :c:member:`ArrowAsyncProducer.request`. This must not call any methods of an ``ArrowAsyncProducer``
+ object.
+
+.. c:member:: void* ArrowAsyncDeviceStreamHandler.private_data
+
+ *Optional.* An opaque pointer to consumer-provided private data.
+
+ Producers *MUST NOT* process this member. Lifetime of this member is handled by
+ the consumer, and especially by the release callback.
+
+The ArrowAsyncTask structure
+''''''''''''''''''''''''''''
+
+The purpose of using a Task object rather than passing the array directly to the ``on_next``
+callback is to allow for more complex and efficient thread handling. Utilizing a Task
+object allows for a producer to separate the "decoding" logic from the I/O, enabling a
+consumer to avoid transferring data between CPU cores (e.g. from one L1/L2 cache to another).
+
+This producer-provided structure has the following fields:
+
+.. c:member:: int (*ArrowArrayTask.get_data)(struct ArrowArrayTask*, struct ArrowDeviceArray*)
+
+ *Mandatory.* A callback to populate the provided ``ArrowDeviceArray`` with the available data.
+ The order of ArrowAsyncTasks provided by the producer enables a consumer to know the order of
+ the data to process.
+
+ If a non-zero value is returned from this, it should be followed only by the producer calling
+ the ``on_error`` callback of the ``ArrowAsyncDeviceStreamHandler``. Because calling this method
+ is likely to be separate from the current control flow, returning a non-zero value to signal
+ an error occuring allows the current thread to decide handle the case accordingly, while still
+ allowing all error logging and handling to be centralized in the
+ :c:member:`ArrowAsyncDeviceStreamHandler.on_error` callback.
+
+ Rather than having a separate release callback, any required cleanup should be performed as part
+ of the invocation of this callback, with ownership of the Array being passed to the consumer-provided
+ parameter calling this, and must be released separately.
+
+ It is only valid to call this method exactly once.
+
+.. c:member:: void* ArrowArrayTask.private_data
+
+ *Optional.* An opaque pointer to consumer-provided private data.
+
+ Consumers *MUST NOT* process this member. Lifetime of this member is handled by
+ the producer who created this object, and should be cleaned up if necessary during
+ the call to :c:member:`ArrowArrayTask.get_data`.
+
+The ArrowAsyncProducer structure
+''''''''''''''''''''''''''''''''
+
+This producer-provided and managed object has the following fields:
+
+.. c:member:: void (*ArrowAsyncProducer.request)(struct ArrowAsyncProducer*, uint64_t)
+
+ *Mandatory.* This function must be called by a consumer to start receiving calls to
+ :c:member:`ArrowAsyncDeviceStreamHandler.on_next_task`. It *MUST* be valid to call
+ this synchronously from within :c:member:`ArrowAsyncDeviceStreamHandler.on_next_task`
+ or :c:member:`ArrowAsyncDeviceStreamHandler.on_schema`. As a result, this function
+ *MUST NOT* synchronously call ``on_next_task`` or ``on_error`` to avoid recursive
+ and reentrant callbacks.
+
+ After ``cancel`` is called, additional calls to this function must be a NOP, but allowed.
+
+ While not cancelled, calling this function registers the given number of additional
+ arrays/batches to be produced by the producer. A producer should only call
+ the appropriate ``on_next_task`` callback up to a maximum of the total sum of calls to
+ this method before propagating back-pressure / waiting.
+
+ Any error encountered by calling request must be propagated by calling the ``on_error``
+ callback of the ``ArrowAsyncDeviceStreamHandler``.
+
+ A producer *MUST* support an unbounded number of calls to this callback and *MUST* support
+ a total registered demand (sum requested - sum delivered) of up to ``UINT64_MAX``.
+
+.. c:member:: void (*ArrowAsyncProducer.cancel)(struct ArrowAsyncProducer*)
+
+ *Mandatory.* This function signals to the producer that it must *eventually* stop calling
+ ``on_next_task``. Calls to ``cancel`` must be idempotent and thread-safe. After calling
+ it once, subsequent calls *MUST* be a NOP.
+
+ It is not required that calling ``cancel`` affect the producer *immediately*, only that it
+ must eventually stop calling ``on_next_task`` and then subsequently call ``release``
+ on the async handler object. As such, a consumer *MUST* be prepared to receive one or more
+ calls to ``on_next_task`` or ``on_error`` even after calling ``cancel`` if there are still
+ requested arrays pending.
+
+ Successful cancelling *MUST NOT* result in a producer calling
+ :c:member:`ArrowAsyncDeviceStreamHandler.on_error`, instead it should finish out any remaining
+ tasks (calling ``on_next_task`` accordingly) and eventually just call ``release``.
+
+ Any error encountered during handling a call to cancel must be reported via the ``on_error``
+ callback on the async stream handler.
+
+.. c:member:: void* ArrowAsyncProducer.private_data
+
+ *Optional.* An opaque pointer to producer-provided specific data.
+
+ Consumers *MUST NOT* process this member, the lifetime is owned by the producer
+ that constructed this object.
+
+Error Handling
+''''''''''''''
+
+Unlike the regular C Stream interface, the Async interface allows for errors to flow in
+both directions. As a result, error handling can be slightly more complex. Thus this spec
+designates the following rules:
+
+* If the producer encounters an error during processing, it should call the ``on_error``
+callback, and then call ``release`` after it returns.
+
+* If ``on_schema`` or ``on_next_task`` returns a non-zero integer value, the producer *should not*
+call the ``on_error`` callback, but instead should eventually call ``release`` at some point
+before or after any logging or processing of the error code.
+
+Result lifetimes
+''''''''''''''''
+
+The ``ArrowSchema`` passed to the ``on_schema`` callback must be released independantly,
+with the object itself needing to be moved to a consumer owned ``ArrowSchema`` object. The
+``ArrowSchema*`` passed as a parameter to the callback *MUST NOT* be stored and kept.
+
+The ``ArrowAsyncTask`` object provided to ``on_next_task`` is owned by the producer and
+will be cleaned up during the invocation of calling ``get_data`` on it. If the consumer
+doesn't care about the data, it should pass ``NULL`` instead of a valid ``ArrowDeviceArray*``.
Review Comment:
Added the documentation of this
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-43631: [C][Format] Add ArrowAsyncDeviceStreamHandler interface [arrow]
Posted by "zeroshade (via GitHub)" <gi...@apache.org>.
zeroshade commented on code in PR #43632:
URL: https://github.com/apache/arrow/pull/43632#discussion_r1759120270
##########
docs/source/format/CDeviceDataInterface.rst:
##########
@@ -650,6 +652,331 @@ The stream source is not assumed to be thread-safe. Consumers wanting to
call ``get_next`` from several threads should ensure those calls are
serialized.
+Async Device Stream Interface
+=============================
+
+The :ref:`C stream interface <_c-device-stream-interface>` provides a synchronous
+API centered around the consumer calling the callback functions to retrieve
+the next record batch. For some bindings, use cases, and interoperability, a more
+asynchronous, producer-focused interface may be required. These scenarios can utilize
+the ``ArrowAsyncDeviceStreamHandler``.
+
+Semantics
+---------
+
+Rather than the producer providing a structure of callbacks for a consumer to
+call and retrieve records, the Async interface is a consumer allocated structure.
+The consumer allocated struct provides handler callbacks for the producer to call
+when the schema and chunks of data are available, rather than the consumer using
+a blocking pull-style iteration.
+
+In addition to the ``ArrowAsyncDeviceStreamHandler``, there are also two additional
+structs used for the full data flow: ``ArrowAsyncTask`` and ``ArrowAsyncProducer``.
+
+Structure Definition
+--------------------
+
+The C device async stream interface is defined with a single ``struct`` definition:
+
+.. code-block:: c
+
+ #ifndef ARROW_C_ASYNC_STREAM_INTERFACE
+ #define ARROW_C_ASYNC_STREAM_INTERFACE
+
+ struct ArrowAsyncTask {
+ int (*get_data)(struct ArrowArrayTask* self, struct ArrowDeviceArray* out);
+
+ void* private_data;
+ };
+
+ struct ArrowAsyncProducer {
+ void (*request)(struct ArrowAsyncProducer* self, uint64_t n);
+ void (*cancel)(struct ArrowAsyncProducer* self);
+
+ void (*release)(struct ArrowAsyncProducer* self);
+ void* private_data;
+ };
+
+ struct ArrowAsyncDeviceStreamHandler {
+ // handlers
+ int (*on_schema)(struct ArrowAsyncDeviceStreamHandler* self,
+ struct ArrowAsyncProducer* producer,
+ struct ArrowSchema* stream_schema, const char* addl_metadata);
+ int (*on_next_task)(struct ArrowAsyncDeviceStreamHandler* self,
+ struct ArrowAsyncTask* task, const char* metadata);
+ void (*on_error)(struct ArrowAsyncDeviceStreamHandler* self,
+ int code, const char* message, const char* metadata);
+
+ // release callback
+ void (*release)(struct ArrowAsyncDeviceStreamHandler* self);
+
+ // opaque handler-specific data
+ void* private_data;
+ };
+
+ #endif // ARROW_C_ASYNC_STREAM_INTERFACE
+
+.. note::
+ The canonical guard ``ARROW_C_ASYNC_STREAM_INTERFACE`` is meant to avoid
+ duplicate definitions if two projects copy the C async stream interface
+ definitions into their own headers, and a third-party project includes
+ from these two projects. It is therefore important that this guard is kept
+ exactly as-is when these definitions are copied.
+
+The ArrowAsyncDeviceStreamHandler structure
+'''''''''''''''''''''''''''''''''''''''''''
+
+The structure has the following fields:
+
+.. c:member:: int (*ArrowAsyncDeviceStreamHandler.on_schema)(struct ArrowAsyncDeviceStreamHandler*, struct ArrowAsyncProducer*, struct ArrowSchema*, const char*)
+
+ *Mandatory.* Handler for receiving the schema of the stream. All records should
+ match the provided schema. If successful, the function should return 0, otherwise
+ it should return an ``errno``-compatible error code.
+
+ The ``const char*`` parameter exists for producers to provide any extra contextual information
+ they want, such as the total number of rows in the stream, statistics, or otherwise. This is
+ encoded in the same format as :c:member:`ArrowSchema.metadata`. If not ``NULL``,
+ the lifetime is only the scope of the call to this function. A consumer who wants to maintain
+ the additional metadata beyond the lifetime of this call *MUST* copy the value themselves.
+
+ Unless the ``on_error`` handler is called, this will always get called exactly once and will be
+ the first method called on this object. As such the producer *MUST* provide an ``ArrowAsyncProducer``
+ object when calling this function to allow the consumer to manage back-pressure and flow control.
+ The producer maintains ownership of the ``ArrowAsyncProducer`` and must clean it up before or after
+ calling the release callback on this object.
+
+ A producer that receives a non-zero result here must not subsequently call anything other than
+ the release callback on this object.
+
+.. c:member:: int (*ArrowAsyncDeviceStreamHandler.on_next_task)(struct ArrowAsyncDeviceStreamHandler*, struct ArrowAsyncTask*, const char*)
+
+ *Mandatory.* Handler to be called when a new record is available for processing. The
+ schema for each record should be the same as the schema that ``on_schema`` was called with.
+ If successfully handled, the function should return 0, otherwise it should return an
+ ``errno``-compatible error code.
+
+ Rather than passing the record itself it receives an ``ArrowAsyncTask`` instead to facilitate
+ better consumer-focused thread control as far as receiving the data. A call to this function
+ simply indicates that data is available via the provided task.
+
+ The ``const char*`` parameter exists for producers to provide any extra contextual information
+ they want. This is encoded in the same format as :c:member:`ArrowSchema.metadata`. If not ``NULL``,
+ the lifetime is only the scope of the call to this function. A consumer who wants to maintain
+ the additional metadata beyond the lifetime of this call *MUST* copy the value themselves.
+
+ A producer *MUST NOT* call this concurrently from multiple different threads.
+
+ The :c:member:`ArrowAsyncProducer.request` callback must be called to start receiving calls to this
+ handler.
+
+ This function *MUST* be able to be called re-entrantly on the same thread to allow for the
+ :c:member:`ArrowAsyncProducer.request` callback to potentially call this method recursively.
+
+.. c:member:: void (*ArrowAsyncDeviceStreamHandler.on_error)(struct ArrowAsyncDeviceStreamHandler, int, const char*, const char*)
+
+ *Mandatory.* Handler to be called when an error is encountered by the producer. After calling
+ this, the ``release`` callback will be called as the last call on this struct. The parameters
+ are an ``errno``-compatible error code and an optional error message and metadata.
+
+ If the message and metadata are not ``NULL``, their lifetime is only valid during the scope
+ of this call. A consumer who wants to maintain these values past the return of this function
+ *MUST* copy the values themselves.
+
+ If the metadata parameter is not ``NULL``, to provide key-value error metadata, then it should
+ be encoded identically to the way that metadata is encoded in :c:member:`ArrowSchema.metadata`.
+
+ It is valid for this to be called by a producer with or without a preceding call to
+ :c:member:`ArrowAsyncProducer.request`. This callback *MUST NOT* call any methods of an
+ ``ArrowAsyncProducer`` object.
+
+.. c:member:: void (*ArrowAsyncDeviceStreamHandler.release)(struct ArrowAsyncDeviceStreamHandler*)
+
+ *Mandatory.* A pointer to a consumer-provided release callback for the handler.
+
+ It is valid for this to be called by a producer with or without a preceding call to
+ :c:member:`ArrowAsyncProducer.request`. This must not call any methods of an ``ArrowAsyncProducer``
+ object.
+
+.. c:member:: void* ArrowAsyncDeviceStreamHandler.private_data
+
+ *Optional.* An opaque pointer to consumer-provided private data.
+
+ Producers *MUST NOT* process this member. Lifetime of this member is handled by
+ the consumer, and especially by the release callback.
+
+The ArrowAsyncTask structure
+''''''''''''''''''''''''''''
+
+The purpose of using a Task object rather than passing the array directly to the ``on_next``
+callback is to allow for more complex and efficient thread handling. Utilizing a Task
+object allows for a producer to separate the "decoding" logic from the I/O, enabling a
+consumer to avoid transferring data between CPU cores (e.g. from one L1/L2 cache to another).
+
+This producer-provided structure has the following fields:
+
+.. c:member:: int (*ArrowArrayTask.get_data)(struct ArrowArrayTask*, struct ArrowDeviceArray*)
Review Comment:
Any and all I/O blocking should have been handled before the producer called the `on_next_task` callback, this function should not perform any I/O and should be as cheap as possible.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-43631: [C][Format] Add ArrowAsyncDeviceStreamHandler interface [arrow]
Posted by "pitrou (via GitHub)" <gi...@apache.org>.
pitrou commented on code in PR #43632:
URL: https://github.com/apache/arrow/pull/43632#discussion_r1757484935
##########
docs/source/format/CDeviceDataInterface.rst:
##########
@@ -650,6 +652,331 @@ The stream source is not assumed to be thread-safe. Consumers wanting to
call ``get_next`` from several threads should ensure those calls are
serialized.
+Async Device Stream Interface
+=============================
+
+The :ref:`C stream interface <_c-device-stream-interface>` provides a synchronous
+API centered around the consumer calling the callback functions to retrieve
+the next record batch. For some bindings, use cases, and interoperability, a more
+asynchronous, producer-focused interface may be required. These scenarios can utilize
+the ``ArrowAsyncDeviceStreamHandler``.
+
+Semantics
+---------
+
+Rather than the producer providing a structure of callbacks for a consumer to
+call and retrieve records, the Async interface is a consumer allocated structure.
+The consumer allocated struct provides handler callbacks for the producer to call
+when the schema and chunks of data are available, rather than the consumer using
+a blocking pull-style iteration.
+
+In addition to the ``ArrowAsyncDeviceStreamHandler``, there are also two additional
+structs used for the full data flow: ``ArrowAsyncTask`` and ``ArrowAsyncProducer``.
+
+Structure Definition
+--------------------
+
+The C device async stream interface is defined with a single ``struct`` definition:
+
+.. code-block:: c
+
+ #ifndef ARROW_C_ASYNC_STREAM_INTERFACE
+ #define ARROW_C_ASYNC_STREAM_INTERFACE
+
+ struct ArrowAsyncTask {
+ int (*get_data)(struct ArrowArrayTask* self, struct ArrowDeviceArray* out);
+
+ void* private_data;
+ };
+
+ struct ArrowAsyncProducer {
+ void (*request)(struct ArrowAsyncProducer* self, uint64_t n);
+ void (*cancel)(struct ArrowAsyncProducer* self);
+
+ void (*release)(struct ArrowAsyncProducer* self);
+ void* private_data;
+ };
+
+ struct ArrowAsyncDeviceStreamHandler {
+ // handlers
+ int (*on_schema)(struct ArrowAsyncDeviceStreamHandler* self,
+ struct ArrowAsyncProducer* producer,
+ struct ArrowSchema* stream_schema, const char* addl_metadata);
+ int (*on_next_task)(struct ArrowAsyncDeviceStreamHandler* self,
+ struct ArrowAsyncTask* task, const char* metadata);
+ void (*on_error)(struct ArrowAsyncDeviceStreamHandler* self,
+ int code, const char* message, const char* metadata);
+
+ // release callback
+ void (*release)(struct ArrowAsyncDeviceStreamHandler* self);
+
+ // opaque handler-specific data
+ void* private_data;
+ };
+
+ #endif // ARROW_C_ASYNC_STREAM_INTERFACE
+
+.. note::
+ The canonical guard ``ARROW_C_ASYNC_STREAM_INTERFACE`` is meant to avoid
+ duplicate definitions if two projects copy the C async stream interface
+ definitions into their own headers, and a third-party project includes
+ from these two projects. It is therefore important that this guard is kept
+ exactly as-is when these definitions are copied.
+
+The ArrowAsyncDeviceStreamHandler structure
+'''''''''''''''''''''''''''''''''''''''''''
+
+The structure has the following fields:
+
+.. c:member:: int (*ArrowAsyncDeviceStreamHandler.on_schema)(struct ArrowAsyncDeviceStreamHandler*, struct ArrowAsyncProducer*, struct ArrowSchema*, const char*)
+
+ *Mandatory.* Handler for receiving the schema of the stream. All records should
+ match the provided schema. If successful, the function should return 0, otherwise
+ it should return an ``errno``-compatible error code.
+
+ The ``const char*`` parameter exists for producers to provide any extra contextual information
+ they want, such as the total number of rows in the stream, statistics, or otherwise. This is
+ encoded in the same format as :c:member:`ArrowSchema.metadata`. If not ``NULL``,
+ the lifetime is only the scope of the call to this function. A consumer who wants to maintain
+ the additional metadata beyond the lifetime of this call *MUST* copy the value themselves.
+
+ Unless the ``on_error`` handler is called, this will always get called exactly once and will be
+ the first method called on this object. As such the producer *MUST* provide an ``ArrowAsyncProducer``
+ object when calling this function to allow the consumer to manage back-pressure and flow control.
+ The producer maintains ownership of the ``ArrowAsyncProducer`` and must clean it up before or after
+ calling the release callback on this object.
+
+ A producer that receives a non-zero result here must not subsequently call anything other than
+ the release callback on this object.
+
+.. c:member:: int (*ArrowAsyncDeviceStreamHandler.on_next_task)(struct ArrowAsyncDeviceStreamHandler*, struct ArrowAsyncTask*, const char*)
+
+ *Mandatory.* Handler to be called when a new record is available for processing. The
+ schema for each record should be the same as the schema that ``on_schema`` was called with.
+ If successfully handled, the function should return 0, otherwise it should return an
+ ``errno``-compatible error code.
+
+ Rather than passing the record itself it receives an ``ArrowAsyncTask`` instead to facilitate
+ better consumer-focused thread control as far as receiving the data. A call to this function
+ simply indicates that data is available via the provided task.
+
+ The ``const char*`` parameter exists for producers to provide any extra contextual information
+ they want. This is encoded in the same format as :c:member:`ArrowSchema.metadata`. If not ``NULL``,
+ the lifetime is only the scope of the call to this function. A consumer who wants to maintain
+ the additional metadata beyond the lifetime of this call *MUST* copy the value themselves.
+
+ A producer *MUST NOT* call this concurrently from multiple different threads.
+
+ The :c:member:`ArrowAsyncProducer.request` callback must be called to start receiving calls to this
+ handler.
+
+ This function *MUST* be able to be called re-entrantly on the same thread to allow for the
+ :c:member:`ArrowAsyncProducer.request` callback to potentially call this method recursively.
+
+.. c:member:: void (*ArrowAsyncDeviceStreamHandler.on_error)(struct ArrowAsyncDeviceStreamHandler, int, const char*, const char*)
+
+ *Mandatory.* Handler to be called when an error is encountered by the producer. After calling
+ this, the ``release`` callback will be called as the last call on this struct. The parameters
+ are an ``errno``-compatible error code and an optional error message and metadata.
+
+ If the message and metadata are not ``NULL``, their lifetime is only valid during the scope
+ of this call. A consumer who wants to maintain these values past the return of this function
+ *MUST* copy the values themselves.
+
+ If the metadata parameter is not ``NULL``, to provide key-value error metadata, then it should
+ be encoded identically to the way that metadata is encoded in :c:member:`ArrowSchema.metadata`.
+
+ It is valid for this to be called by a producer with or without a preceding call to
+ :c:member:`ArrowAsyncProducer.request`. This callback *MUST NOT* call any methods of an
+ ``ArrowAsyncProducer`` object.
+
+.. c:member:: void (*ArrowAsyncDeviceStreamHandler.release)(struct ArrowAsyncDeviceStreamHandler*)
+
+ *Mandatory.* A pointer to a consumer-provided release callback for the handler.
+
+ It is valid for this to be called by a producer with or without a preceding call to
+ :c:member:`ArrowAsyncProducer.request`. This must not call any methods of an ``ArrowAsyncProducer``
+ object.
+
+.. c:member:: void* ArrowAsyncDeviceStreamHandler.private_data
+
+ *Optional.* An opaque pointer to consumer-provided private data.
+
+ Producers *MUST NOT* process this member. Lifetime of this member is handled by
+ the consumer, and especially by the release callback.
+
+The ArrowAsyncTask structure
+''''''''''''''''''''''''''''
+
+The purpose of using a Task object rather than passing the array directly to the ``on_next``
+callback is to allow for more complex and efficient thread handling. Utilizing a Task
+object allows for a producer to separate the "decoding" logic from the I/O, enabling a
+consumer to avoid transferring data between CPU cores (e.g. from one L1/L2 cache to another).
+
+This producer-provided structure has the following fields:
+
+.. c:member:: int (*ArrowArrayTask.get_data)(struct ArrowArrayTask*, struct ArrowDeviceArray*)
+
+ *Mandatory.* A callback to populate the provided ``ArrowDeviceArray`` with the available data.
+ The order of ArrowAsyncTasks provided by the producer enables a consumer to know the order of
+ the data to process.
+
+ If a non-zero value is returned from this, it should be followed only by the producer calling
+ the ``on_error`` callback of the ``ArrowAsyncDeviceStreamHandler``. Because calling this method
+ is likely to be separate from the current control flow, returning a non-zero value to signal
+ an error occuring allows the current thread to decide handle the case accordingly, while still
+ allowing all error logging and handling to be centralized in the
+ :c:member:`ArrowAsyncDeviceStreamHandler.on_error` callback.
+
+ Rather than having a separate release callback, any required cleanup should be performed as part
+ of the invocation of this callback, with ownership of the Array being passed to the consumer-provided
+ parameter calling this, and must be released separately.
+
+ It is only valid to call this method exactly once.
+
+.. c:member:: void* ArrowArrayTask.private_data
+
+ *Optional.* An opaque pointer to consumer-provided private data.
+
+ Consumers *MUST NOT* process this member. Lifetime of this member is handled by
+ the producer who created this object, and should be cleaned up if necessary during
+ the call to :c:member:`ArrowArrayTask.get_data`.
+
+The ArrowAsyncProducer structure
+''''''''''''''''''''''''''''''''
+
+This producer-provided and managed object has the following fields:
+
+.. c:member:: void (*ArrowAsyncProducer.request)(struct ArrowAsyncProducer*, uint64_t)
+
+ *Mandatory.* This function must be called by a consumer to start receiving calls to
+ :c:member:`ArrowAsyncDeviceStreamHandler.on_next_task`. It *MUST* be valid to call
+ this synchronously from within :c:member:`ArrowAsyncDeviceStreamHandler.on_next_task`
+ or :c:member:`ArrowAsyncDeviceStreamHandler.on_schema`. As a result, this function
+ *MUST NOT* synchronously call ``on_next_task`` or ``on_error`` to avoid recursive
+ and reentrant callbacks.
+
+ After ``cancel`` is called, additional calls to this function must be a NOP, but allowed.
+
+ While not cancelled, calling this function registers the given number of additional
+ arrays/batches to be produced by the producer. A producer should only call
+ the appropriate ``on_next_task`` callback up to a maximum of the total sum of calls to
+ this method before propagating back-pressure / waiting.
Review Comment:
Why does this talk about "propagating back-pressure"? If a consumer requests N arrays, it should receive N arrays.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-43631: [C][Format] Add ArrowAsyncDeviceStreamHandler interface [arrow]
Posted by "pitrou (via GitHub)" <gi...@apache.org>.
pitrou commented on code in PR #43632:
URL: https://github.com/apache/arrow/pull/43632#discussion_r1757492890
##########
docs/source/format/CDeviceDataInterface.rst:
##########
@@ -650,6 +652,331 @@ The stream source is not assumed to be thread-safe. Consumers wanting to
call ``get_next`` from several threads should ensure those calls are
serialized.
+Async Device Stream Interface
+=============================
+
+The :ref:`C stream interface <_c-device-stream-interface>` provides a synchronous
+API centered around the consumer calling the callback functions to retrieve
+the next record batch. For some bindings, use cases, and interoperability, a more
+asynchronous, producer-focused interface may be required. These scenarios can utilize
+the ``ArrowAsyncDeviceStreamHandler``.
+
+Semantics
+---------
+
+Rather than the producer providing a structure of callbacks for a consumer to
+call and retrieve records, the Async interface is a consumer allocated structure.
+The consumer allocated struct provides handler callbacks for the producer to call
+when the schema and chunks of data are available, rather than the consumer using
+a blocking pull-style iteration.
+
+In addition to the ``ArrowAsyncDeviceStreamHandler``, there are also two additional
+structs used for the full data flow: ``ArrowAsyncTask`` and ``ArrowAsyncProducer``.
+
+Structure Definition
+--------------------
+
+The C device async stream interface is defined with a single ``struct`` definition:
+
+.. code-block:: c
+
+ #ifndef ARROW_C_ASYNC_STREAM_INTERFACE
+ #define ARROW_C_ASYNC_STREAM_INTERFACE
+
+ struct ArrowAsyncTask {
+ int (*get_data)(struct ArrowArrayTask* self, struct ArrowDeviceArray* out);
+
+ void* private_data;
+ };
+
+ struct ArrowAsyncProducer {
+ void (*request)(struct ArrowAsyncProducer* self, uint64_t n);
+ void (*cancel)(struct ArrowAsyncProducer* self);
+
+ void (*release)(struct ArrowAsyncProducer* self);
+ void* private_data;
+ };
+
+ struct ArrowAsyncDeviceStreamHandler {
+ // handlers
+ int (*on_schema)(struct ArrowAsyncDeviceStreamHandler* self,
+ struct ArrowAsyncProducer* producer,
+ struct ArrowSchema* stream_schema, const char* addl_metadata);
+ int (*on_next_task)(struct ArrowAsyncDeviceStreamHandler* self,
+ struct ArrowAsyncTask* task, const char* metadata);
+ void (*on_error)(struct ArrowAsyncDeviceStreamHandler* self,
+ int code, const char* message, const char* metadata);
+
+ // release callback
+ void (*release)(struct ArrowAsyncDeviceStreamHandler* self);
+
+ // opaque handler-specific data
+ void* private_data;
+ };
+
+ #endif // ARROW_C_ASYNC_STREAM_INTERFACE
+
+.. note::
+ The canonical guard ``ARROW_C_ASYNC_STREAM_INTERFACE`` is meant to avoid
+ duplicate definitions if two projects copy the C async stream interface
+ definitions into their own headers, and a third-party project includes
+ from these two projects. It is therefore important that this guard is kept
+ exactly as-is when these definitions are copied.
+
+The ArrowAsyncDeviceStreamHandler structure
+'''''''''''''''''''''''''''''''''''''''''''
+
+The structure has the following fields:
+
+.. c:member:: int (*ArrowAsyncDeviceStreamHandler.on_schema)(struct ArrowAsyncDeviceStreamHandler*, struct ArrowAsyncProducer*, struct ArrowSchema*, const char*)
+
+ *Mandatory.* Handler for receiving the schema of the stream. All records should
+ match the provided schema. If successful, the function should return 0, otherwise
+ it should return an ``errno``-compatible error code.
+
+ The ``const char*`` parameter exists for producers to provide any extra contextual information
+ they want, such as the total number of rows in the stream, statistics, or otherwise. This is
+ encoded in the same format as :c:member:`ArrowSchema.metadata`. If not ``NULL``,
+ the lifetime is only the scope of the call to this function. A consumer who wants to maintain
+ the additional metadata beyond the lifetime of this call *MUST* copy the value themselves.
+
+ Unless the ``on_error`` handler is called, this will always get called exactly once and will be
+ the first method called on this object. As such the producer *MUST* provide an ``ArrowAsyncProducer``
+ object when calling this function to allow the consumer to manage back-pressure and flow control.
+ The producer maintains ownership of the ``ArrowAsyncProducer`` and must clean it up before or after
+ calling the release callback on this object.
+
+ A producer that receives a non-zero result here must not subsequently call anything other than
+ the release callback on this object.
+
+.. c:member:: int (*ArrowAsyncDeviceStreamHandler.on_next_task)(struct ArrowAsyncDeviceStreamHandler*, struct ArrowAsyncTask*, const char*)
+
+ *Mandatory.* Handler to be called when a new record is available for processing. The
+ schema for each record should be the same as the schema that ``on_schema`` was called with.
+ If successfully handled, the function should return 0, otherwise it should return an
+ ``errno``-compatible error code.
+
+ Rather than passing the record itself it receives an ``ArrowAsyncTask`` instead to facilitate
+ better consumer-focused thread control as far as receiving the data. A call to this function
+ simply indicates that data is available via the provided task.
+
+ The ``const char*`` parameter exists for producers to provide any extra contextual information
+ they want. This is encoded in the same format as :c:member:`ArrowSchema.metadata`. If not ``NULL``,
+ the lifetime is only the scope of the call to this function. A consumer who wants to maintain
+ the additional metadata beyond the lifetime of this call *MUST* copy the value themselves.
+
+ A producer *MUST NOT* call this concurrently from multiple different threads.
+
+ The :c:member:`ArrowAsyncProducer.request` callback must be called to start receiving calls to this
+ handler.
+
+ This function *MUST* be able to be called re-entrantly on the same thread to allow for the
+ :c:member:`ArrowAsyncProducer.request` callback to potentially call this method recursively.
+
+.. c:member:: void (*ArrowAsyncDeviceStreamHandler.on_error)(struct ArrowAsyncDeviceStreamHandler, int, const char*, const char*)
+
+ *Mandatory.* Handler to be called when an error is encountered by the producer. After calling
+ this, the ``release`` callback will be called as the last call on this struct. The parameters
+ are an ``errno``-compatible error code and an optional error message and metadata.
+
+ If the message and metadata are not ``NULL``, their lifetime is only valid during the scope
+ of this call. A consumer who wants to maintain these values past the return of this function
+ *MUST* copy the values themselves.
+
+ If the metadata parameter is not ``NULL``, to provide key-value error metadata, then it should
+ be encoded identically to the way that metadata is encoded in :c:member:`ArrowSchema.metadata`.
+
+ It is valid for this to be called by a producer with or without a preceding call to
+ :c:member:`ArrowAsyncProducer.request`. This callback *MUST NOT* call any methods of an
+ ``ArrowAsyncProducer`` object.
+
+.. c:member:: void (*ArrowAsyncDeviceStreamHandler.release)(struct ArrowAsyncDeviceStreamHandler*)
+
+ *Mandatory.* A pointer to a consumer-provided release callback for the handler.
+
+ It is valid for this to be called by a producer with or without a preceding call to
+ :c:member:`ArrowAsyncProducer.request`. This must not call any methods of an ``ArrowAsyncProducer``
+ object.
+
+.. c:member:: void* ArrowAsyncDeviceStreamHandler.private_data
+
+ *Optional.* An opaque pointer to consumer-provided private data.
+
+ Producers *MUST NOT* process this member. Lifetime of this member is handled by
+ the consumer, and especially by the release callback.
+
+The ArrowAsyncTask structure
+''''''''''''''''''''''''''''
+
+The purpose of using a Task object rather than passing the array directly to the ``on_next``
+callback is to allow for more complex and efficient thread handling. Utilizing a Task
+object allows for a producer to separate the "decoding" logic from the I/O, enabling a
+consumer to avoid transferring data between CPU cores (e.g. from one L1/L2 cache to another).
+
+This producer-provided structure has the following fields:
+
+.. c:member:: int (*ArrowArrayTask.get_data)(struct ArrowArrayTask*, struct ArrowDeviceArray*)
+
+ *Mandatory.* A callback to populate the provided ``ArrowDeviceArray`` with the available data.
+ The order of ArrowAsyncTasks provided by the producer enables a consumer to know the order of
+ the data to process.
+
+ If a non-zero value is returned from this, it should be followed only by the producer calling
+ the ``on_error`` callback of the ``ArrowAsyncDeviceStreamHandler``. Because calling this method
+ is likely to be separate from the current control flow, returning a non-zero value to signal
+ an error occuring allows the current thread to decide handle the case accordingly, while still
+ allowing all error logging and handling to be centralized in the
+ :c:member:`ArrowAsyncDeviceStreamHandler.on_error` callback.
+
+ Rather than having a separate release callback, any required cleanup should be performed as part
+ of the invocation of this callback, with ownership of the Array being passed to the consumer-provided
+ parameter calling this, and must be released separately.
+
+ It is only valid to call this method exactly once.
+
+.. c:member:: void* ArrowArrayTask.private_data
+
+ *Optional.* An opaque pointer to consumer-provided private data.
+
+ Consumers *MUST NOT* process this member. Lifetime of this member is handled by
+ the producer who created this object, and should be cleaned up if necessary during
+ the call to :c:member:`ArrowArrayTask.get_data`.
+
+The ArrowAsyncProducer structure
+''''''''''''''''''''''''''''''''
+
+This producer-provided and managed object has the following fields:
+
+.. c:member:: void (*ArrowAsyncProducer.request)(struct ArrowAsyncProducer*, uint64_t)
+
+ *Mandatory.* This function must be called by a consumer to start receiving calls to
+ :c:member:`ArrowAsyncDeviceStreamHandler.on_next_task`. It *MUST* be valid to call
+ this synchronously from within :c:member:`ArrowAsyncDeviceStreamHandler.on_next_task`
+ or :c:member:`ArrowAsyncDeviceStreamHandler.on_schema`. As a result, this function
+ *MUST NOT* synchronously call ``on_next_task`` or ``on_error`` to avoid recursive
+ and reentrant callbacks.
+
+ After ``cancel`` is called, additional calls to this function must be a NOP, but allowed.
+
+ While not cancelled, calling this function registers the given number of additional
+ arrays/batches to be produced by the producer. A producer should only call
+ the appropriate ``on_next_task`` callback up to a maximum of the total sum of calls to
+ this method before propagating back-pressure / waiting.
+
+ Any error encountered by calling request must be propagated by calling the ``on_error``
+ callback of the ``ArrowAsyncDeviceStreamHandler``.
+
+ A producer *MUST* support an unbounded number of calls to this callback and *MUST* support
+ a total registered demand (sum requested - sum delivered) of up to ``UINT64_MAX``.
+
+.. c:member:: void (*ArrowAsyncProducer.cancel)(struct ArrowAsyncProducer*)
+
+ *Mandatory.* This function signals to the producer that it must *eventually* stop calling
+ ``on_next_task``. Calls to ``cancel`` must be idempotent and thread-safe. After calling
+ it once, subsequent calls *MUST* be a NOP.
+
+ It is not required that calling ``cancel`` affect the producer *immediately*, only that it
+ must eventually stop calling ``on_next_task`` and then subsequently call ``release``
+ on the async handler object. As such, a consumer *MUST* be prepared to receive one or more
+ calls to ``on_next_task`` or ``on_error`` even after calling ``cancel`` if there are still
+ requested arrays pending.
+
+ Successful cancelling *MUST NOT* result in a producer calling
+ :c:member:`ArrowAsyncDeviceStreamHandler.on_error`, instead it should finish out any remaining
+ tasks (calling ``on_next_task`` accordingly) and eventually just call ``release``.
+
+ Any error encountered during handling a call to cancel must be reported via the ``on_error``
+ callback on the async stream handler.
+
+.. c:member:: void* ArrowAsyncProducer.private_data
+
+ *Optional.* An opaque pointer to producer-provided specific data.
+
+ Consumers *MUST NOT* process this member, the lifetime is owned by the producer
+ that constructed this object.
+
+Error Handling
+''''''''''''''
+
+Unlike the regular C Stream interface, the Async interface allows for errors to flow in
+both directions. As a result, error handling can be slightly more complex. Thus this spec
+designates the following rules:
+
+* If the producer encounters an error during processing, it should call the ``on_error``
+callback, and then call ``release`` after it returns.
+
+* If ``on_schema`` or ``on_next_task`` returns a non-zero integer value, the producer *should not*
+call the ``on_error`` callback, but instead should eventually call ``release`` at some point
+before or after any logging or processing of the error code.
+
+Result lifetimes
+''''''''''''''''
+
+The ``ArrowSchema`` passed to the ``on_schema`` callback must be released independantly,
+with the object itself needing to be moved to a consumer owned ``ArrowSchema`` object. The
+``ArrowSchema*`` passed as a parameter to the callback *MUST NOT* be stored and kept.
+
+The ``ArrowAsyncTask`` object provided to ``on_next_task`` is owned by the producer and
+will be cleaned up during the invocation of calling ``get_data`` on it. If the consumer
+doesn't care about the data, it should pass ``NULL`` instead of a valid ``ArrowDeviceArray*``.
+
+The ``const char*`` error ``message`` and ``metadata`` which are passed to ``on_error``
+are only valid within the scope of the ``on_error`` function itself. They must be copied
+if it is necessary for them to exist after it returns.
+
+Stream Handler Lifetime
+'''''''''''''''''''''''
+
+Lifetime of the async stream handler is managed using a release callback with similar
+usage as in :ref:`C data interface <c-data-interface-released>`.
+
+ArrowAsyncProducer Lifetime
+'''''''''''''''''''''''''''
+
+The lifetime of the ``ArrowAsyncProducer`` passed to ``on_schema`` is owned by the producer
+itself and should be managed by it. It *MUST* remain valid at least until just before
+calling ``release`` on the stream handler object.
+
+Thread safety
+'''''''''''''
+
+All handler functions should only be called in a serialized manner, but are not guaranteed
Review Comment:
You mean consumer-side handler functions, or methods in `ArrowAsyncDeviceStreamHandler`? Let's be extra explicit.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-43631: [C][Format] Add ArrowAsyncDeviceStreamHandler interface [arrow]
Posted by "lidavidm (via GitHub)" <gi...@apache.org>.
lidavidm commented on code in PR #43632:
URL: https://github.com/apache/arrow/pull/43632#discussion_r1828551784
##########
cpp/src/arrow/c/abi.h:
##########
@@ -356,8 +356,7 @@ struct ArrowAsyncDeviceStreamHandler {
// A producer that receives a non-zero return here should stop producing and eventually
// call release instead.
int (*on_schema)(struct ArrowAsyncDeviceStreamHandler* self,
- struct ArrowAsyncProducer* producer, struct ArrowSchema* stream_schema,
- const char* addl_metadata);
+ struct ArrowSchema* stream_schema, const char* addl_metadata);
Review Comment:
I suppose I don't mind then. As @kou notes let's spell it out too (`additional_metadata`)?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-43631: [C][Format] Add ArrowAsyncDeviceStreamHandler interface [arrow]
Posted by "bkietz (via GitHub)" <gi...@apache.org>.
bkietz commented on code in PR #43632:
URL: https://github.com/apache/arrow/pull/43632#discussion_r1830334771
##########
cpp/src/arrow/c/abi.h:
##########
@@ -319,6 +319,11 @@ struct ArrowAsyncProducer {
// on_error callback on the async stream handler.
void (*cancel)(struct ArrowAsyncProducer* self);
+ // Any additional metadata tied to a specific stream of data. This must either be NULL
+ // or a valid pointer to metadata which is encoded in the same way schema metadata
+ // would be. Non-null metadata must be valid for the lifetime of this object.
Review Comment:
This entire struct is only valid after the schema callback, so noting that again for just this field seems redundant. What kind of context are you thinking of? An example like "a producer might use this to communicate the total number of rows and/or batches in the stream if that is known? "
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-43631: [C][Format] Add ArrowAsyncDeviceStreamHandler interface [arrow]
Posted by "zeroshade (via GitHub)" <gi...@apache.org>.
zeroshade commented on PR #43632:
URL: https://github.com/apache/arrow/pull/43632#issuecomment-2302898364
I just want to consolidate the discussion here, please let me know if I missed anything or got anything wrong.
## Changes to the proposal
* Either remove the `extension` parameter or change it to be `const char*` using the metadata encoding like we're doing for the `on_error` callback
* Change `on_next` to instead be `get_next_task` and create an `ArrowArrayTask` struct that populates the `ArrowDeviceArray`
* Do we want to go ahead with the `waker` approach?
I'm unsure about the `ArrowArrayTask` approach since the majority of use cases we're going to have aren't going to know how many tasks there will be ahead of time (file readers are the only cases I can think of) so I agree with @lidavidm that I don't think we want to bake in an assumption of knowing how many tasks will be there ahead of time.
Do we need the entirely separate task struct? Could `ArrowAsyncDeviceArrayStream` instead have the `get_next` directly take a `waker` and have the whole `EWOULDBLOCK` semantics for handling the stream?
My big concern is the case of an IPC stream where we don't know beforehand how many record batches are in the stream. If `get_next_task` is called multiple times in a row, historically we've used a return of 0 + null for the Array to indicate the end of the stream. So the only reasonable way to handle `get_next_task` being called multiple times in a row while there are tasks pending is to return a valid task for each call until the producer *knows* that the stream has ended and then we define semantics for the task struct to indicate that the stream has ended successfully.
Does that make sense?
## Next steps
After confirming that everyone is on-board with the above ideas, I'll update this PR with the changes for more feedback.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-43631: [C][Format] Add ArrowAsyncDeviceStreamHandler interface [arrow]
Posted by "zeroshade (via GitHub)" <gi...@apache.org>.
zeroshade commented on PR #43632:
URL: https://github.com/apache/arrow/pull/43632#issuecomment-2332382933
After looking into the `IProcessor` stuff from ClickHouse that @leventov referred to, there's a couple of points I've figured out:
* The `IProcessor` specifies that none of the methods on a single object can be executed in parallel, avoiding the entire concurrency issue in the same way as we avoid it with the non-async version of the stream struct.
* The `prepare()` method that @leventov mentioned is even more strict, it cannot be executed in parallel even for different objects that are in any-way connected (even indirectly).
* If we go with the Task approach, this means that the `get_next_task` function would have to explicitly state that it is not allowed to be called in parallel, while the resulting `Task` objects could be utilized in parallel.
* Going with the partial compromise where it's still a callback but it produces a task rather than the array directly via `on_next_task` means that we don't have to worry about a consumer calling it in parallel and merely need to define if a producer is allowed to call it in parallel or not (and thus whether the consumer who creates the callback needs to handle that)
* By using the Task-based approach it means that the producer just calls the `on_next_task` callback when data is *ready* and the consumer can then choose when to actually *retrieve* that data and on what thread etc. by choosing when to call the `get_array` method. This also eliminates the need for the `waker` pattern as the callback is *only* called when there is data ready (calling `on_error` if an error is encountered)
* The `Reactive Streams` specification that was mentioned has an explicit `onComplete` method which I guess could be analagous to our `release` callback here, or simply calling `on_next_task` with a nullptr.
* Following this reactive stream concept, it looks like backpressure and cancellation is managed by an object passed from the producer back to the consumer via `onSubscribe` which in our case could be an `on_start` method which is called before any `on_next_task` calls happen.
* The object that the producer gives the consumer has two methods: `request(n)` and `cancel`. The `request` method signals the producer that the consumer is able to receive *up to* `n` more calls of `on_next_task` before having to wait. Calling `cancel` on the object signals to the producer the cancellation, which must *eventually* stop calling `on_next_task`. among other guarantees. All in all, I think I actually like this model.
I'm gonna try to put something together taking into account the feedback so far, looking at the Reactive Streams specification, and any other examples I can come across (i'm open to suggestions from people if anyone has an idea of an existing C/C++ based Async api that might be worthwhile to look at for prior art.
I plan to have something up here tomorrow, in the meantime please feel free to chime in on my above statements and what you think of it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-43631: [C][Format] Add ArrowAsyncDeviceStreamHandler interface [arrow]
Posted by "lidavidm (via GitHub)" <gi...@apache.org>.
lidavidm commented on code in PR #43632:
URL: https://github.com/apache/arrow/pull/43632#discussion_r1749586187
##########
cpp/src/arrow/c/abi.h:
##########
@@ -228,6 +228,180 @@ struct ArrowDeviceArrayStream {
#endif // ARROW_C_DEVICE_STREAM_INTERFACE
+#ifndef ARROW_C_ASYNC_STREAM_INTERFACE
+#define ARROW_C_ASYNC_STREAM_INTERFACE
+
+// ArrowAsyncTask represents available data from a producer that was passed to
+// an invocation of `on_next_task` on the ArrowAsyncDeviceStreamHandler.
+struct ArrowAsyncTask {
+ // This callback should populate the ArrowDeviceArray associated with this task.
Review Comment:
Should we also clarify that it is expected to be nonblocking?
##########
cpp/src/arrow/c/abi.h:
##########
@@ -228,6 +228,180 @@ struct ArrowDeviceArrayStream {
#endif // ARROW_C_DEVICE_STREAM_INTERFACE
+#ifndef ARROW_C_ASYNC_STREAM_INTERFACE
+#define ARROW_C_ASYNC_STREAM_INTERFACE
+
+// ArrowAsyncTask represents available data from a producer that was passed to
+// an invocation of `on_next_task` on the ArrowAsyncDeviceStreamHandler.
+struct ArrowAsyncTask {
+ // This callback should populate the ArrowDeviceArray associated with this task.
+ // The order of ArrowAsyncTasks provided by the producer enables a consumer to
+ // ensure the order of data to process.
+ //
+ // Returns: 0 if successful, errno-compatible error otherwise.
+ //
+ // If a non-0 value is returned then it should be followed by a call to `on_error`
+ // on the appropriate ArrowAsyncDeviceStreamHandler.
+ //
+ // Rather than a release callback, any required cleanup should be performed as part
+ // of the invocation of `get_data`. Ownership of the Array is passed to the consumer
+ // calling this, and so it must be released separately.
+ //
+ // It is only valid to call this method exactly once.
+ int (*get_data)(struct ArrowArrayTask* self, struct ArrowDeviceArray* out);
+
+ // opaque task-specific data
+ void* private_data;
+};
+
+// ArrowAsyncProducer represents a 1-to-1 relationship between an async producer
+// and consumer. This object allows the consumer to perform backpressure and flow
+// control on the asynchronous stream processing. This object must be owned by the
+// producer who creates it, and thus is responsible for cleaning it up.
+struct ArrowAsyncProducer {
+ // A consumer must call this function to start receiving on_next_task calls.
+ //
+ // It *must* be valid to call this synchronously from within on_next_task or
+ // on_schema, with a producer placing an upper bound on possible synchronous
+ // recursion between calls of on_next_task -> request -> on_next_task ->....
+ //
+ // After cancel has been called, additional calls to this function must be NOPs,
+ // but allowed.
+ //
+ // While not cancelled, calling this function must register the given number of
+ // additional arrays/batches to be produced with the producer. The producer should
+ // only call `on_next_task` at most the registered number of arrays before propagating
+ // backpressure.
+ //
+ // While not cancelled, calling request MAY synchronously call `on_next_task`,
+ // `on_error`, or `release` on the ArrowAsyncDeviceStreamHandler.
+ //
+ // Any error encountered by calling request must be propagated by calling the `on_error`
+ // callback of the ArrowAsyncDeviceStreamHandler.
+ //
+ // A producer must support an unbounded number of calls to request and must support
+ // a total registered demand (sum requested - sum delivered) of up to UINT64_MAX.
+ void (*request)(struct ArrowAsyncProducer* self, uint64_t n);
+
+ // This cancel callback signals a producer that it must eventually stop making calls
+ // to on_next_task. It must be idempotent and thread-safe.
+ //
+ // After calling cancel once, subsequent calls must be NOPs.
+ //
+ // It is not required that calling cancel affect the producer immediately, only that it
+ // must eventually stop calling on_next_task and subsequently call release on the
+ // async handler. As such, a consumer must be prepared to receive one or more calls to
+ // `on_next_task` even after calling cancel if there are still requested arrays pending.
+ //
+ // Any error encountered during handling a call to cancel must be reported via the on_error
+ // callback on the async stream handler.
+ void (*cancel)(struct ArrowAsyncProducer* self);
+
+ // producer-specific opaque data.
+ void* private_data;
+};
+
+// Similar to ArrowDeviceArrayStream, except designed for an asynchronous
+// style of interaction. While ArrowDeviceArrayStream provides producer
+// defined callbacks, this is intended to be created by the consumer instead.
+// The consumer passes this handler to the producer, which in turn uses the
+// callbacks to inform the consumer of events in the stream.
+struct ArrowAsyncDeviceStreamHandler {
+ // Handler for receiving a schema. The passed in stream_schema must be
+ // released or moved by the handler (producer is giving ownership of the schema to
+ // the handler, but not ownership of the top level object itself).
+ //
+ // With the exception of an error occurring (on_error), this must be the first
+ // callback function which is called by a producer and must only be called exactly
+ // once. As such, the producer should provide a valid ArrowAsyncProducer instance
+ // so the consumer can control the flow. See the documentation on ArrowAsyncProducer
+ // for how it works. The ArrowAsyncProducer is owned by the producer who calls this
+ // function and thus the producer is responsible for cleaning it up when calling
+ // the release callback of this handler.
+ //
+ // The addl_metadata argument can be null or can be used by a producer
+ // to pass arbitrary extra information to the consumer beyond the metadata in the schema
+ // itself (such as total number of rows, context info, or otherwise). The data should
+ // be passed using the same encoding as the metadata within the ArrowSchema struct
+ // itself (defined in the spec at
+ // https://arrow.apache.org/docs/format/CDataInterface.html#c.ArrowSchema.metadata)
+ //
+ // If addl_metadata is non-null then it only needs to exist for the lifetime of this call,
+ // a consumer who wants it to live after that must copy it to ensure lifetime.
+ //
+ // Return value: 0 if successful, `errno`-compatible error otherwise
+ //
+ // A producer that receives a non-zero return here should stop producing and eventually
+ // call release instead.
+ int (*on_schema)(struct ArrowAsyncDeviceStreamHandler* self,
+ struct ArrowAsyncProducer* producer, struct ArrowSchema* stream_schema,
+ const char* addl_metadata);
+
+ // Handler for receiving data. This is called when data is available providing an
+ // ArrowAsyncTask struct to signify it. The consumer is responsible for calling
+ // the release callback on the ArrowAsyncTask after retrieving the array/batch.
+ // An empty/released task is passed to indicate the end of the stream if no
+ // errors have been encountered.
+ //
+ // The `request` callback of a provided ArrowAsyncProducer must be called in order
+ // to start receiving calls to this handler.
+ //
+ // The metadata argument can be null or can be used by a producer
+ // to pass arbitrary extra information to the consumer (such as total number
+ // of rows, context info, or otherwise). The data should be passed using the same
+ // encoding as the metadata within the ArrowSchema struct itself (defined in
+ // the spec at
+ // https://arrow.apache.org/docs/format/CDataInterface.html#c.ArrowSchema.metadata)
+ //
+ // If metadata is non-null then it only needs to exist for the lifetime of this call,
+ // a consumer who wants it to live after that must copy it to ensure lifetime.
+ //
+ // A producer *must not* call this concurrently from multiple different threads.
+ //
+ // A consumer must be prepared to received one or more calls to this callback even
+ // after calling cancel on the corresponding ArrowAsyncProducer, as cancel does not
+ // guarantee it happens immediately.
+ //
+ // Return value: 0 if successful, `errno`-compatible error otherwise.
+ int (*on_next_task)(struct ArrowAsyncDeviceStreamHandler* self,
+ struct ArrowAsyncTask* task, const char* metadata);
Review Comment:
Presumably the consumer needs to copy the contents of `task` if it intends to use them after this returns?
##########
cpp/src/arrow/c/abi.h:
##########
@@ -228,6 +228,180 @@ struct ArrowDeviceArrayStream {
#endif // ARROW_C_DEVICE_STREAM_INTERFACE
+#ifndef ARROW_C_ASYNC_STREAM_INTERFACE
+#define ARROW_C_ASYNC_STREAM_INTERFACE
+
+// ArrowAsyncTask represents available data from a producer that was passed to
+// an invocation of `on_next_task` on the ArrowAsyncDeviceStreamHandler.
+struct ArrowAsyncTask {
+ // This callback should populate the ArrowDeviceArray associated with this task.
+ // The order of ArrowAsyncTasks provided by the producer enables a consumer to
+ // ensure the order of data to process.
+ //
+ // Returns: 0 if successful, errno-compatible error otherwise.
+ //
+ // If a non-0 value is returned then it should be followed by a call to `on_error`
+ // on the appropriate ArrowAsyncDeviceStreamHandler.
+ //
+ // Rather than a release callback, any required cleanup should be performed as part
+ // of the invocation of `get_data`. Ownership of the Array is passed to the consumer
+ // calling this, and so it must be released separately.
+ //
+ // It is only valid to call this method exactly once.
+ int (*get_data)(struct ArrowArrayTask* self, struct ArrowDeviceArray* out);
+
+ // opaque task-specific data
+ void* private_data;
+};
+
+// ArrowAsyncProducer represents a 1-to-1 relationship between an async producer
+// and consumer. This object allows the consumer to perform backpressure and flow
+// control on the asynchronous stream processing. This object must be owned by the
+// producer who creates it, and thus is responsible for cleaning it up.
+struct ArrowAsyncProducer {
+ // A consumer must call this function to start receiving on_next_task calls.
+ //
+ // It *must* be valid to call this synchronously from within on_next_task or
+ // on_schema, with a producer placing an upper bound on possible synchronous
+ // recursion between calls of on_next_task -> request -> on_next_task ->....
+ //
+ // After cancel has been called, additional calls to this function must be NOPs,
+ // but allowed.
+ //
+ // While not cancelled, calling this function must register the given number of
+ // additional arrays/batches to be produced with the producer. The producer should
+ // only call `on_next_task` at most the registered number of arrays before propagating
+ // backpressure.
+ //
+ // While not cancelled, calling request MAY synchronously call `on_next_task`,
+ // `on_error`, or `release` on the ArrowAsyncDeviceStreamHandler.
+ //
+ // Any error encountered by calling request must be propagated by calling the `on_error`
+ // callback of the ArrowAsyncDeviceStreamHandler.
+ //
+ // A producer must support an unbounded number of calls to request and must support
+ // a total registered demand (sum requested - sum delivered) of up to UINT64_MAX.
+ void (*request)(struct ArrowAsyncProducer* self, uint64_t n);
+
+ // This cancel callback signals a producer that it must eventually stop making calls
+ // to on_next_task. It must be idempotent and thread-safe.
+ //
+ // After calling cancel once, subsequent calls must be NOPs.
+ //
+ // It is not required that calling cancel affect the producer immediately, only that it
+ // must eventually stop calling on_next_task and subsequently call release on the
+ // async handler. As such, a consumer must be prepared to receive one or more calls to
+ // `on_next_task` even after calling cancel if there are still requested arrays pending.
+ //
+ // Any error encountered during handling a call to cancel must be reported via the on_error
+ // callback on the async stream handler.
+ void (*cancel)(struct ArrowAsyncProducer* self);
Review Comment:
Will cancellation lead to `on_error` being called or only `release`?
##########
cpp/src/arrow/c/abi.h:
##########
@@ -228,6 +228,180 @@ struct ArrowDeviceArrayStream {
#endif // ARROW_C_DEVICE_STREAM_INTERFACE
+#ifndef ARROW_C_ASYNC_STREAM_INTERFACE
+#define ARROW_C_ASYNC_STREAM_INTERFACE
+
+// ArrowAsyncTask represents available data from a producer that was passed to
+// an invocation of `on_next_task` on the ArrowAsyncDeviceStreamHandler.
+struct ArrowAsyncTask {
+ // This callback should populate the ArrowDeviceArray associated with this task.
+ // The order of ArrowAsyncTasks provided by the producer enables a consumer to
+ // ensure the order of data to process.
+ //
+ // Returns: 0 if successful, errno-compatible error otherwise.
+ //
+ // If a non-0 value is returned then it should be followed by a call to `on_error`
+ // on the appropriate ArrowAsyncDeviceStreamHandler.
+ //
+ // Rather than a release callback, any required cleanup should be performed as part
+ // of the invocation of `get_data`. Ownership of the Array is passed to the consumer
+ // calling this, and so it must be released separately.
+ //
+ // It is only valid to call this method exactly once.
+ int (*get_data)(struct ArrowArrayTask* self, struct ArrowDeviceArray* out);
+
+ // opaque task-specific data
+ void* private_data;
+};
+
+// ArrowAsyncProducer represents a 1-to-1 relationship between an async producer
+// and consumer. This object allows the consumer to perform backpressure and flow
+// control on the asynchronous stream processing. This object must be owned by the
+// producer who creates it, and thus is responsible for cleaning it up.
+struct ArrowAsyncProducer {
+ // A consumer must call this function to start receiving on_next_task calls.
+ //
+ // It *must* be valid to call this synchronously from within on_next_task or
+ // on_schema, with a producer placing an upper bound on possible synchronous
+ // recursion between calls of on_next_task -> request -> on_next_task ->....
Review Comment:
Can the producer really track this easily? (A thread-local?) I'd rather stipulate that the producer should not immediately recurse
##########
cpp/src/arrow/c/abi.h:
##########
@@ -228,6 +228,180 @@ struct ArrowDeviceArrayStream {
#endif // ARROW_C_DEVICE_STREAM_INTERFACE
+#ifndef ARROW_C_ASYNC_STREAM_INTERFACE
+#define ARROW_C_ASYNC_STREAM_INTERFACE
+
+// ArrowAsyncTask represents available data from a producer that was passed to
+// an invocation of `on_next_task` on the ArrowAsyncDeviceStreamHandler.
+struct ArrowAsyncTask {
+ // This callback should populate the ArrowDeviceArray associated with this task.
+ // The order of ArrowAsyncTasks provided by the producer enables a consumer to
+ // ensure the order of data to process.
+ //
+ // Returns: 0 if successful, errno-compatible error otherwise.
+ //
+ // If a non-0 value is returned then it should be followed by a call to `on_error`
+ // on the appropriate ArrowAsyncDeviceStreamHandler.
+ //
+ // Rather than a release callback, any required cleanup should be performed as part
+ // of the invocation of `get_data`. Ownership of the Array is passed to the consumer
+ // calling this, and so it must be released separately.
+ //
+ // It is only valid to call this method exactly once.
+ int (*get_data)(struct ArrowArrayTask* self, struct ArrowDeviceArray* out);
+
+ // opaque task-specific data
+ void* private_data;
+};
+
+// ArrowAsyncProducer represents a 1-to-1 relationship between an async producer
+// and consumer. This object allows the consumer to perform backpressure and flow
+// control on the asynchronous stream processing. This object must be owned by the
+// producer who creates it, and thus is responsible for cleaning it up.
+struct ArrowAsyncProducer {
+ // A consumer must call this function to start receiving on_next_task calls.
+ //
+ // It *must* be valid to call this synchronously from within on_next_task or
+ // on_schema, with a producer placing an upper bound on possible synchronous
+ // recursion between calls of on_next_task -> request -> on_next_task ->....
+ //
+ // After cancel has been called, additional calls to this function must be NOPs,
+ // but allowed.
+ //
+ // While not cancelled, calling this function must register the given number of
+ // additional arrays/batches to be produced with the producer. The producer should
+ // only call `on_next_task` at most the registered number of arrays before propagating
+ // backpressure.
+ //
+ // While not cancelled, calling request MAY synchronously call `on_next_task`,
+ // `on_error`, or `release` on the ArrowAsyncDeviceStreamHandler.
+ //
+ // Any error encountered by calling request must be propagated by calling the `on_error`
+ // callback of the ArrowAsyncDeviceStreamHandler.
+ //
+ // A producer must support an unbounded number of calls to request and must support
+ // a total registered demand (sum requested - sum delivered) of up to UINT64_MAX.
+ void (*request)(struct ArrowAsyncProducer* self, uint64_t n);
+
+ // This cancel callback signals a producer that it must eventually stop making calls
+ // to on_next_task. It must be idempotent and thread-safe.
+ //
+ // After calling cancel once, subsequent calls must be NOPs.
+ //
+ // It is not required that calling cancel affect the producer immediately, only that it
+ // must eventually stop calling on_next_task and subsequently call release on the
+ // async handler. As such, a consumer must be prepared to receive one or more calls to
+ // `on_next_task` even after calling cancel if there are still requested arrays pending.
+ //
+ // Any error encountered during handling a call to cancel must be reported via the on_error
+ // callback on the async stream handler.
+ void (*cancel)(struct ArrowAsyncProducer* self);
+
+ // producer-specific opaque data.
+ void* private_data;
+};
+
+// Similar to ArrowDeviceArrayStream, except designed for an asynchronous
+// style of interaction. While ArrowDeviceArrayStream provides producer
+// defined callbacks, this is intended to be created by the consumer instead.
+// The consumer passes this handler to the producer, which in turn uses the
+// callbacks to inform the consumer of events in the stream.
+struct ArrowAsyncDeviceStreamHandler {
+ // Handler for receiving a schema. The passed in stream_schema must be
+ // released or moved by the handler (producer is giving ownership of the schema to
+ // the handler, but not ownership of the top level object itself).
+ //
+ // With the exception of an error occurring (on_error), this must be the first
+ // callback function which is called by a producer and must only be called exactly
+ // once. As such, the producer should provide a valid ArrowAsyncProducer instance
+ // so the consumer can control the flow. See the documentation on ArrowAsyncProducer
+ // for how it works. The ArrowAsyncProducer is owned by the producer who calls this
+ // function and thus the producer is responsible for cleaning it up when calling
+ // the release callback of this handler.
+ //
+ // The addl_metadata argument can be null or can be used by a producer
+ // to pass arbitrary extra information to the consumer beyond the metadata in the schema
+ // itself (such as total number of rows, context info, or otherwise). The data should
+ // be passed using the same encoding as the metadata within the ArrowSchema struct
+ // itself (defined in the spec at
+ // https://arrow.apache.org/docs/format/CDataInterface.html#c.ArrowSchema.metadata)
+ //
+ // If addl_metadata is non-null then it only needs to exist for the lifetime of this call,
+ // a consumer who wants it to live after that must copy it to ensure lifetime.
+ //
+ // Return value: 0 if successful, `errno`-compatible error otherwise
+ //
+ // A producer that receives a non-zero return here should stop producing and eventually
+ // call release instead.
+ int (*on_schema)(struct ArrowAsyncDeviceStreamHandler* self,
+ struct ArrowAsyncProducer* producer, struct ArrowSchema* stream_schema,
+ const char* addl_metadata);
+
+ // Handler for receiving data. This is called when data is available providing an
+ // ArrowAsyncTask struct to signify it. The consumer is responsible for calling
+ // the release callback on the ArrowAsyncTask after retrieving the array/batch.
+ // An empty/released task is passed to indicate the end of the stream if no
+ // errors have been encountered.
+ //
+ // The `request` callback of a provided ArrowAsyncProducer must be called in order
+ // to start receiving calls to this handler.
+ //
+ // The metadata argument can be null or can be used by a producer
+ // to pass arbitrary extra information to the consumer (such as total number
+ // of rows, context info, or otherwise). The data should be passed using the same
+ // encoding as the metadata within the ArrowSchema struct itself (defined in
+ // the spec at
+ // https://arrow.apache.org/docs/format/CDataInterface.html#c.ArrowSchema.metadata)
+ //
+ // If metadata is non-null then it only needs to exist for the lifetime of this call,
+ // a consumer who wants it to live after that must copy it to ensure lifetime.
+ //
+ // A producer *must not* call this concurrently from multiple different threads.
+ //
+ // A consumer must be prepared to received one or more calls to this callback even
+ // after calling cancel on the corresponding ArrowAsyncProducer, as cancel does not
+ // guarantee it happens immediately.
+ //
+ // Return value: 0 if successful, `errno`-compatible error otherwise.
+ int (*on_next_task)(struct ArrowAsyncDeviceStreamHandler* self,
+ struct ArrowAsyncTask* task, const char* metadata);
+
+ // Handler for encountering an error. The producer should call release after
+ // this returns to clean up any resources.
+ //
+ // If the message or metadata are non-null, they will only last as long as this
+ // function call. The consumer would need to perform a copy of the data if it is
+ // it is necessary for them live past the lifetime of this call.
+ //
+ // Error metadata should be encoded as with metadata in ArrowSchema, defined in
+ // the spec at
+ // https://arrow.apache.org/docs/format/CDataInterface.html#c.ArrowSchema.metadata
+ //
+ // After this call, producers should follow-up by calling the release callback.
+ //
+ // It is valid for this to be called by a producer with or without a preceding call
+ // to ArrowAsyncProducer.request.
+ //
+ // This callback must not call any methods of an ArrowAsyncProducer object.
+ void (*on_error)(struct ArrowAsyncDeviceStreamHandler* self, int code,
Review Comment:
Is `code` an errno?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-43631: [C][Format] Add ArrowAsyncDeviceStreamHandler interface [arrow]
Posted by "zeroshade (via GitHub)" <gi...@apache.org>.
zeroshade commented on code in PR #43632:
URL: https://github.com/apache/arrow/pull/43632#discussion_r1750449460
##########
cpp/src/arrow/c/abi.h:
##########
@@ -228,6 +228,180 @@ struct ArrowDeviceArrayStream {
#endif // ARROW_C_DEVICE_STREAM_INTERFACE
+#ifndef ARROW_C_ASYNC_STREAM_INTERFACE
+#define ARROW_C_ASYNC_STREAM_INTERFACE
+
+// ArrowAsyncTask represents available data from a producer that was passed to
+// an invocation of `on_next_task` on the ArrowAsyncDeviceStreamHandler.
+struct ArrowAsyncTask {
+ // This callback should populate the ArrowDeviceArray associated with this task.
+ // The order of ArrowAsyncTasks provided by the producer enables a consumer to
+ // ensure the order of data to process.
+ //
+ // Returns: 0 if successful, errno-compatible error otherwise.
+ //
+ // If a non-0 value is returned then it should be followed by a call to `on_error`
+ // on the appropriate ArrowAsyncDeviceStreamHandler.
+ //
+ // Rather than a release callback, any required cleanup should be performed as part
+ // of the invocation of `get_data`. Ownership of the Array is passed to the consumer
+ // calling this, and so it must be released separately.
+ //
+ // It is only valid to call this method exactly once.
+ int (*get_data)(struct ArrowArrayTask* self, struct ArrowDeviceArray* out);
+
+ // opaque task-specific data
+ void* private_data;
+};
+
+// ArrowAsyncProducer represents a 1-to-1 relationship between an async producer
+// and consumer. This object allows the consumer to perform backpressure and flow
+// control on the asynchronous stream processing. This object must be owned by the
+// producer who creates it, and thus is responsible for cleaning it up.
+struct ArrowAsyncProducer {
+ // A consumer must call this function to start receiving on_next_task calls.
+ //
+ // It *must* be valid to call this synchronously from within on_next_task or
+ // on_schema, with a producer placing an upper bound on possible synchronous
+ // recursion between calls of on_next_task -> request -> on_next_task ->....
+ //
+ // After cancel has been called, additional calls to this function must be NOPs,
+ // but allowed.
+ //
+ // While not cancelled, calling this function must register the given number of
+ // additional arrays/batches to be produced with the producer. The producer should
+ // only call `on_next_task` at most the registered number of arrays before propagating
+ // backpressure.
+ //
+ // While not cancelled, calling request MAY synchronously call `on_next_task`,
+ // `on_error`, or `release` on the ArrowAsyncDeviceStreamHandler.
+ //
+ // Any error encountered by calling request must be propagated by calling the `on_error`
+ // callback of the ArrowAsyncDeviceStreamHandler.
+ //
+ // A producer must support an unbounded number of calls to request and must support
+ // a total registered demand (sum requested - sum delivered) of up to UINT64_MAX.
+ void (*request)(struct ArrowAsyncProducer* self, uint64_t n);
+
+ // This cancel callback signals a producer that it must eventually stop making calls
+ // to on_next_task. It must be idempotent and thread-safe.
+ //
+ // After calling cancel once, subsequent calls must be NOPs.
+ //
+ // It is not required that calling cancel affect the producer immediately, only that it
+ // must eventually stop calling on_next_task and subsequently call release on the
+ // async handler. As such, a consumer must be prepared to receive one or more calls to
+ // `on_next_task` even after calling cancel if there are still requested arrays pending.
+ //
+ // Any error encountered during handling a call to cancel must be reported via the on_error
+ // callback on the async stream handler.
+ void (*cancel)(struct ArrowAsyncProducer* self);
+
+ // producer-specific opaque data.
+ void* private_data;
+};
+
+// Similar to ArrowDeviceArrayStream, except designed for an asynchronous
+// style of interaction. While ArrowDeviceArrayStream provides producer
+// defined callbacks, this is intended to be created by the consumer instead.
+// The consumer passes this handler to the producer, which in turn uses the
+// callbacks to inform the consumer of events in the stream.
+struct ArrowAsyncDeviceStreamHandler {
+ // Handler for receiving a schema. The passed in stream_schema must be
+ // released or moved by the handler (producer is giving ownership of the schema to
+ // the handler, but not ownership of the top level object itself).
+ //
+ // With the exception of an error occurring (on_error), this must be the first
+ // callback function which is called by a producer and must only be called exactly
+ // once. As such, the producer should provide a valid ArrowAsyncProducer instance
+ // so the consumer can control the flow. See the documentation on ArrowAsyncProducer
+ // for how it works. The ArrowAsyncProducer is owned by the producer who calls this
+ // function and thus the producer is responsible for cleaning it up when calling
+ // the release callback of this handler.
+ //
+ // The addl_metadata argument can be null or can be used by a producer
+ // to pass arbitrary extra information to the consumer beyond the metadata in the schema
+ // itself (such as total number of rows, context info, or otherwise). The data should
+ // be passed using the same encoding as the metadata within the ArrowSchema struct
+ // itself (defined in the spec at
+ // https://arrow.apache.org/docs/format/CDataInterface.html#c.ArrowSchema.metadata)
+ //
+ // If addl_metadata is non-null then it only needs to exist for the lifetime of this call,
+ // a consumer who wants it to live after that must copy it to ensure lifetime.
+ //
+ // Return value: 0 if successful, `errno`-compatible error otherwise
+ //
+ // A producer that receives a non-zero return here should stop producing and eventually
+ // call release instead.
+ int (*on_schema)(struct ArrowAsyncDeviceStreamHandler* self,
+ struct ArrowAsyncProducer* producer, struct ArrowSchema* stream_schema,
+ const char* addl_metadata);
+
+ // Handler for receiving data. This is called when data is available providing an
+ // ArrowAsyncTask struct to signify it. The consumer is responsible for calling
+ // the release callback on the ArrowAsyncTask after retrieving the array/batch.
+ // An empty/released task is passed to indicate the end of the stream if no
+ // errors have been encountered.
+ //
+ // The `request` callback of a provided ArrowAsyncProducer must be called in order
+ // to start receiving calls to this handler.
+ //
+ // The metadata argument can be null or can be used by a producer
+ // to pass arbitrary extra information to the consumer (such as total number
+ // of rows, context info, or otherwise). The data should be passed using the same
+ // encoding as the metadata within the ArrowSchema struct itself (defined in
+ // the spec at
+ // https://arrow.apache.org/docs/format/CDataInterface.html#c.ArrowSchema.metadata)
+ //
+ // If metadata is non-null then it only needs to exist for the lifetime of this call,
+ // a consumer who wants it to live after that must copy it to ensure lifetime.
+ //
+ // A producer *must not* call this concurrently from multiple different threads.
+ //
+ // A consumer must be prepared to received one or more calls to this callback even
+ // after calling cancel on the corresponding ArrowAsyncProducer, as cancel does not
+ // guarantee it happens immediately.
+ //
+ // Return value: 0 if successful, `errno`-compatible error otherwise.
+ int (*on_next_task)(struct ArrowAsyncDeviceStreamHandler* self,
+ struct ArrowAsyncTask* task, const char* metadata);
Review Comment:
Yea, that's one of the reasons I've tried to make the task as small as possible: just the get_data callback + private_data member.
I'll update the comment to specify that.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-43631: [C][Format] Add ArrowAsyncDeviceStreamHandler interface [arrow]
Posted by "pitrou (via GitHub)" <gi...@apache.org>.
pitrou commented on code in PR #43632:
URL: https://github.com/apache/arrow/pull/43632#discussion_r1718565581
##########
docs/source/format/CDeviceDataInterface.rst:
##########
@@ -650,6 +652,137 @@ The stream source is not assumed to be thread-safe. Consumers wanting to
call ``get_next`` from several threads should ensure those calls are
serialized.
+Async Device Stream Interface
+=============================
+
+The :ref:`C stream interface <_c-device-stream-interface>` provides a synchronous
+API centered around the consumer calling the callback functions to retrieve
+the next record batch. For some bindings, use cases, and interoperability, a more
+asynchronous, producer-focused interface may be required. These scenarios can utilize
+the ``ArrowAsyncDeviceStreamHandler``.
+
+Semantics
+---------
+
+Rather than the producer providing a structure of callbacks for a consumer to
+call and retrieve records, the Async interface is a consumer allocated structure.
+The consumer allocated struct provides handler callbacks for the producer to call
+when the schema and chunks of data are available, rather than the consumer using
+a blocking pull-style iteration.
+
+Structure Definition
+--------------------
+
+The C device async stream interface is defined with a single ``struct`` definition:
+
+.. code-block:: c
+
+ #ifndef ARROW_C_ASYNC_STREAM_INTERFACE
+ #define ARROW_C_ASYNC_STREAM_INTERFACE
+
+ struct ArrowAsyncDeviceStreamHandler {
+ // handlers
+ int (*on_schema)(struct ArrowAsyncDeviceStreamHandler* self,
+ struct ArrowSchema* stream_schema, void* extension_param);
+ int (*on_next)(struct ArrowAsyncDeviceStreamHandler* self,
+ struct ArrowDeviceArray* next, void* extension_param);
+ void (*on_error)(struct ArrowAsyncDeviceStreamHandler* self,
+ int code, const char* message, const char* metadata);
+
+ // release callback
+ void (*release)(struct ArrowAsyncDeviceStreamHandler* self);
+
+ // opaque handler-specific data
+ void* private_data;
+ };
+
+ #endif // ARROW_C_ASYNC_STREAM_INTERFACE
+
+.. note::
+ The canonical guard ``ARROW_C_ASYNC_STREAM_INTERFACE`` is meant to avoid
+ duplicate definitions if two projects copy the C async stream interface
+ definitions into their own headers, and a third-party project includes
+ from these two projects. It is therefore important that this guard is kept
+ exactly as-is when these definitions are copied.
+
+The ArrowAsyncDeviceStreamHandler structure
+'''''''''''''''''''''''''''''''''''''''''''
+
+The structure has the following fields:
+
+.. c:member:: int (*ArrowAsyncDeviceStreamHandler.on_schema)(struct ArrowAsyncDeviceStreamHandler*, struct ArrowSchema*, void*)
+
+ *Mandatory.* Handler for receiving the schema of the stream. All records should
+ match the provided schema. If successful, the function should return 0, otherwise
+ it should return an ``errno``-compatible error code.
+
+ The ``void*`` parameter exists for producers to provide any extra contextual information
+ they want, such as the total number of rows in the stream, statistics, or otherwise.
+
+ Unless the ``on_error`` handler is called, this will always get called exactly once.
+
+.. c:member:: int (*ArrowAsyncDeviceStreamHandler.on_next)(struct ArrowAsyncDeviceStreamHandler*, struct ArrowDeviceArray*, void*)
+
+ *Mandatory.* Handler to be called when a new record is available for processing. The
+ schema for each record should be the same as the schema that ``on_schema`` was called with.
+ If successfully handled, the function should return 0, otherwise it should return an
+ ``errno``-compatible error code.
+
+ The ``void*`` parameter exists for producers to provide any extra contextual information
+ they want.
+
+ Unless the ``on_error`` handler is called, this will always get called at least once. The
+ end of the stream is indicated by it being called with an empty or released ``ArrowDeviceArray``.
+
+.. c:member:: void (*ArrowAsyncDeviceStreamHandler.on_error)(struct ArrowAsyncDeviceStreamHandler, int, const char*, const char*)
+
+ *Mandatory.* Handler to be called when an error is encountered by the producer. After calling
+ this, the ``release`` callback will be called as the last call on this struct. The parameters
+ are an ``errno``-compatible error code and an optional error message and metadata.
+
+ If the message and metadata are not ``NULL``, their lifetime is only valid during the scope
+ of this call. A consumer who wants to maintain these values past the return of this function
+ *MUST* copy the values themselves.
+
+ If the metadata parameter is not ``NULL``, to provide key-value error metadata, then it should
+ be encoded identically to the way that metadata is encoded in :c:member:`ArrowSchema.metadata`.
+
+.. c:member:: void (*ArrowAsyncDeviceStreamHandler.release)(struct ArrowAsyncDeviceStreamHandler*)
+
+ *Mandatory.* A pointer to a consumer-provided release callback for the handler.
+
+.. c:member:: void* ArrowAsyncDeviceStreamHandler.private_data
+
+ *Optional.* An opaque pointer to consumer-provided private data.
+
+ Producers *MUST NOT* process this member. Lifetime of this member is handled by
+ the consumer, and especially by the release callback.
+
+Result lifetimes
+''''''''''''''''
+
+The ``ArrowSchema`` and ``ArrowDeviceArray`` that are passed to the ``on_schema`` and
+``on_next`` callbacks must be released independently. Their lifetimes are not tied to
+the ``ArrowAsyncDeviceStreamHandler`` or the handler functions.
+
+The ``const char*`` error ``message`` and ``metadata`` which are passed to ``on_error``
+are only valid within the scope of the ``on_error`` function itself. They must be copied
+if it is necessary for them to exist after it returns.
+
+Stream Handler Lifetime
+'''''''''''''''''''''''
+
+Lifetime of the async stream handler is managed using a release callback with similar
+usage as in :ref:`C data interface <c-data-interface-released>`.
+
+Thread safety
+'''''''''''''
+
+All handler functions should only be called in a serialized manner, but are not guaranteed
+to be called from the same thread every time. A producer should wait for handler callbacks to
+return before calling the next handler callback. As a result, a consumer can manage back-pressure
+simply by sleeping or otherwise waiting to return from the ``on_next`` handler if necessary.
Review Comment:
Do we really want to encourage consumers to handle backpressure by sleeping? This forces the producer to invoke consumer callbacks from a dedicated thread, adding complexity and overhead.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-43631: [C][Format] Add ArrowAsyncDeviceStreamHandler interface [arrow]
Posted by "pitrou (via GitHub)" <gi...@apache.org>.
pitrou commented on code in PR #43632:
URL: https://github.com/apache/arrow/pull/43632#discussion_r1718564422
##########
docs/source/format/CDeviceDataInterface.rst:
##########
@@ -650,6 +652,137 @@ The stream source is not assumed to be thread-safe. Consumers wanting to
call ``get_next`` from several threads should ensure those calls are
serialized.
+Async Device Stream Interface
+=============================
+
+The :ref:`C stream interface <_c-device-stream-interface>` provides a synchronous
+API centered around the consumer calling the callback functions to retrieve
+the next record batch. For some bindings, use cases, and interoperability, a more
+asynchronous, producer-focused interface may be required. These scenarios can utilize
+the ``ArrowAsyncDeviceStreamHandler``.
+
+Semantics
+---------
+
+Rather than the producer providing a structure of callbacks for a consumer to
+call and retrieve records, the Async interface is a consumer allocated structure.
+The consumer allocated struct provides handler callbacks for the producer to call
+when the schema and chunks of data are available, rather than the consumer using
+a blocking pull-style iteration.
+
+Structure Definition
+--------------------
+
+The C device async stream interface is defined with a single ``struct`` definition:
+
+.. code-block:: c
+
+ #ifndef ARROW_C_ASYNC_STREAM_INTERFACE
+ #define ARROW_C_ASYNC_STREAM_INTERFACE
+
+ struct ArrowAsyncDeviceStreamHandler {
+ // handlers
+ int (*on_schema)(struct ArrowAsyncDeviceStreamHandler* self,
+ struct ArrowSchema* stream_schema, void* extension_param);
+ int (*on_next)(struct ArrowAsyncDeviceStreamHandler* self,
+ struct ArrowDeviceArray* next, void* extension_param);
+ void (*on_error)(struct ArrowAsyncDeviceStreamHandler* self,
+ int code, const char* message, const char* metadata);
+
+ // release callback
+ void (*release)(struct ArrowAsyncDeviceStreamHandler* self);
+
+ // opaque handler-specific data
+ void* private_data;
+ };
+
+ #endif // ARROW_C_ASYNC_STREAM_INTERFACE
+
+.. note::
+ The canonical guard ``ARROW_C_ASYNC_STREAM_INTERFACE`` is meant to avoid
+ duplicate definitions if two projects copy the C async stream interface
+ definitions into their own headers, and a third-party project includes
+ from these two projects. It is therefore important that this guard is kept
+ exactly as-is when these definitions are copied.
+
+The ArrowAsyncDeviceStreamHandler structure
+'''''''''''''''''''''''''''''''''''''''''''
+
+The structure has the following fields:
+
+.. c:member:: int (*ArrowAsyncDeviceStreamHandler.on_schema)(struct ArrowAsyncDeviceStreamHandler*, struct ArrowSchema*, void*)
+
+ *Mandatory.* Handler for receiving the schema of the stream. All records should
+ match the provided schema. If successful, the function should return 0, otherwise
+ it should return an ``errno``-compatible error code.
+
+ The ``void*`` parameter exists for producers to provide any extra contextual information
+ they want, such as the total number of rows in the stream, statistics, or otherwise.
+
+ Unless the ``on_error`` handler is called, this will always get called exactly once.
+
+.. c:member:: int (*ArrowAsyncDeviceStreamHandler.on_next)(struct ArrowAsyncDeviceStreamHandler*, struct ArrowDeviceArray*, void*)
+
+ *Mandatory.* Handler to be called when a new record is available for processing. The
+ schema for each record should be the same as the schema that ``on_schema`` was called with.
+ If successfully handled, the function should return 0, otherwise it should return an
+ ``errno``-compatible error code.
+
+ The ``void*`` parameter exists for producers to provide any extra contextual information
+ they want.
+
+ Unless the ``on_error`` handler is called, this will always get called at least once. The
+ end of the stream is indicated by it being called with an empty or released ``ArrowDeviceArray``.
+
+.. c:member:: void (*ArrowAsyncDeviceStreamHandler.on_error)(struct ArrowAsyncDeviceStreamHandler, int, const char*, const char*)
+
+ *Mandatory.* Handler to be called when an error is encountered by the producer. After calling
+ this, the ``release`` callback will be called as the last call on this struct. The parameters
+ are an ``errno``-compatible error code and an optional error message and metadata.
+
+ If the message and metadata are not ``NULL``, their lifetime is only valid during the scope
+ of this call. A consumer who wants to maintain these values past the return of this function
+ *MUST* copy the values themselves.
+
+ If the metadata parameter is not ``NULL``, to provide key-value error metadata, then it should
+ be encoded identically to the way that metadata is encoded in :c:member:`ArrowSchema.metadata`.
+
+.. c:member:: void (*ArrowAsyncDeviceStreamHandler.release)(struct ArrowAsyncDeviceStreamHandler*)
+
+ *Mandatory.* A pointer to a consumer-provided release callback for the handler.
+
+.. c:member:: void* ArrowAsyncDeviceStreamHandler.private_data
+
+ *Optional.* An opaque pointer to consumer-provided private data.
+
+ Producers *MUST NOT* process this member. Lifetime of this member is handled by
+ the consumer, and especially by the release callback.
+
+Result lifetimes
+''''''''''''''''
+
+The ``ArrowSchema`` and ``ArrowDeviceArray`` that are passed to the ``on_schema`` and
+``on_next`` callbacks must be released independently. Their lifetimes are not tied to
+the ``ArrowAsyncDeviceStreamHandler`` or the handler functions.
+
+The ``const char*`` error ``message`` and ``metadata`` which are passed to ``on_error``
+are only valid within the scope of the ``on_error`` function itself. They must be copied
+if it is necessary for them to exist after it returns.
+
+Stream Handler Lifetime
+'''''''''''''''''''''''
+
+Lifetime of the async stream handler is managed using a release callback with similar
+usage as in :ref:`C data interface <c-data-interface-released>`.
+
+Thread safety
+'''''''''''''
+
+All handler functions should only be called in a serialized manner, but are not guaranteed
+to be called from the same thread every time. A producer should wait for handler callbacks to
+return before calling the next handler callback. As a result, a consumer can manage back-pressure
Review Comment:
```suggestion
return before calling the next handler callback, and before calling the `release` callback.
As a result, a consumer can manage back-pressure
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-43631: [C][Format] Add ArrowAsyncDeviceStreamHandler interface [arrow]
Posted by "pitrou (via GitHub)" <gi...@apache.org>.
pitrou commented on code in PR #43632:
URL: https://github.com/apache/arrow/pull/43632#discussion_r1718557385
##########
cpp/src/arrow/c/abi.h:
##########
@@ -228,6 +228,65 @@ struct ArrowDeviceArrayStream {
#endif // ARROW_C_DEVICE_STREAM_INTERFACE
+#ifndef ARROW_C_ASYNC_STREAM_INTERFACE
+#define ARROW_C_ASYNC_STREAM_INTERFACE
+
+// Similar to ArrowDeviceArrayStream, except designed for an asynchronous
+// style of interaction. While ArrowDeviceArrayStream provides producer
+// defined callbacks, this is intended to be created by the consumer instead.
+// The consumer passes this handler to the producer, which in turn uses the
+// callbacks to inform the consumer of events in the stream.
Review Comment:
What this doesn't state is whether the producer must serialize calls to these callbacks, or if, for example, it is allowed to invoke `on_next` from several threads at once.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-43631: [C][Format] Add ArrowAsyncDeviceStreamHandler interface [arrow]
Posted by "pitrou (via GitHub)" <gi...@apache.org>.
pitrou commented on code in PR #43632:
URL: https://github.com/apache/arrow/pull/43632#discussion_r1718570454
##########
docs/source/format/CDeviceDataInterface.rst:
##########
@@ -650,6 +652,137 @@ The stream source is not assumed to be thread-safe. Consumers wanting to
call ``get_next`` from several threads should ensure those calls are
serialized.
+Async Device Stream Interface
+=============================
+
+The :ref:`C stream interface <_c-device-stream-interface>` provides a synchronous
+API centered around the consumer calling the callback functions to retrieve
+the next record batch. For some bindings, use cases, and interoperability, a more
+asynchronous, producer-focused interface may be required. These scenarios can utilize
+the ``ArrowAsyncDeviceStreamHandler``.
+
+Semantics
+---------
+
+Rather than the producer providing a structure of callbacks for a consumer to
+call and retrieve records, the Async interface is a consumer allocated structure.
+The consumer allocated struct provides handler callbacks for the producer to call
+when the schema and chunks of data are available, rather than the consumer using
+a blocking pull-style iteration.
+
+Structure Definition
+--------------------
+
+The C device async stream interface is defined with a single ``struct`` definition:
+
+.. code-block:: c
+
+ #ifndef ARROW_C_ASYNC_STREAM_INTERFACE
+ #define ARROW_C_ASYNC_STREAM_INTERFACE
+
+ struct ArrowAsyncDeviceStreamHandler {
+ // handlers
+ int (*on_schema)(struct ArrowAsyncDeviceStreamHandler* self,
+ struct ArrowSchema* stream_schema, void* extension_param);
+ int (*on_next)(struct ArrowAsyncDeviceStreamHandler* self,
+ struct ArrowDeviceArray* next, void* extension_param);
+ void (*on_error)(struct ArrowAsyncDeviceStreamHandler* self,
+ int code, const char* message, const char* metadata);
+
+ // release callback
+ void (*release)(struct ArrowAsyncDeviceStreamHandler* self);
+
+ // opaque handler-specific data
+ void* private_data;
+ };
+
+ #endif // ARROW_C_ASYNC_STREAM_INTERFACE
+
+.. note::
+ The canonical guard ``ARROW_C_ASYNC_STREAM_INTERFACE`` is meant to avoid
+ duplicate definitions if two projects copy the C async stream interface
+ definitions into their own headers, and a third-party project includes
+ from these two projects. It is therefore important that this guard is kept
+ exactly as-is when these definitions are copied.
+
+The ArrowAsyncDeviceStreamHandler structure
+'''''''''''''''''''''''''''''''''''''''''''
+
+The structure has the following fields:
+
+.. c:member:: int (*ArrowAsyncDeviceStreamHandler.on_schema)(struct ArrowAsyncDeviceStreamHandler*, struct ArrowSchema*, void*)
+
+ *Mandatory.* Handler for receiving the schema of the stream. All records should
+ match the provided schema. If successful, the function should return 0, otherwise
+ it should return an ``errno``-compatible error code.
+
+ The ``void*`` parameter exists for producers to provide any extra contextual information
+ they want, such as the total number of rows in the stream, statistics, or otherwise.
+
+ Unless the ``on_error`` handler is called, this will always get called exactly once.
+
+.. c:member:: int (*ArrowAsyncDeviceStreamHandler.on_next)(struct ArrowAsyncDeviceStreamHandler*, struct ArrowDeviceArray*, void*)
+
+ *Mandatory.* Handler to be called when a new record is available for processing. The
+ schema for each record should be the same as the schema that ``on_schema`` was called with.
+ If successfully handled, the function should return 0, otherwise it should return an
+ ``errno``-compatible error code.
+
+ The ``void*`` parameter exists for producers to provide any extra contextual information
+ they want.
+
+ Unless the ``on_error`` handler is called, this will always get called at least once. The
+ end of the stream is indicated by it being called with an empty or released ``ArrowDeviceArray``.
+
+.. c:member:: void (*ArrowAsyncDeviceStreamHandler.on_error)(struct ArrowAsyncDeviceStreamHandler, int, const char*, const char*)
+
+ *Mandatory.* Handler to be called when an error is encountered by the producer. After calling
+ this, the ``release`` callback will be called as the last call on this struct. The parameters
+ are an ``errno``-compatible error code and an optional error message and metadata.
+
+ If the message and metadata are not ``NULL``, their lifetime is only valid during the scope
+ of this call. A consumer who wants to maintain these values past the return of this function
+ *MUST* copy the values themselves.
+
+ If the metadata parameter is not ``NULL``, to provide key-value error metadata, then it should
+ be encoded identically to the way that metadata is encoded in :c:member:`ArrowSchema.metadata`.
+
+.. c:member:: void (*ArrowAsyncDeviceStreamHandler.release)(struct ArrowAsyncDeviceStreamHandler*)
+
+ *Mandatory.* A pointer to a consumer-provided release callback for the handler.
+
+.. c:member:: void* ArrowAsyncDeviceStreamHandler.private_data
+
+ *Optional.* An opaque pointer to consumer-provided private data.
+
+ Producers *MUST NOT* process this member. Lifetime of this member is handled by
+ the consumer, and especially by the release callback.
+
+Result lifetimes
+''''''''''''''''
+
+The ``ArrowSchema`` and ``ArrowDeviceArray`` that are passed to the ``on_schema`` and
+``on_next`` callbacks must be released independently. Their lifetimes are not tied to
+the ``ArrowAsyncDeviceStreamHandler`` or the handler functions.
+
+The ``const char*`` error ``message`` and ``metadata`` which are passed to ``on_error``
+are only valid within the scope of the ``on_error`` function itself. They must be copied
+if it is necessary for them to exist after it returns.
+
+Stream Handler Lifetime
+'''''''''''''''''''''''
+
+Lifetime of the async stream handler is managed using a release callback with similar
+usage as in :ref:`C data interface <c-data-interface-released>`.
+
+Thread safety
+'''''''''''''
+
+All handler functions should only be called in a serialized manner, but are not guaranteed
+to be called from the same thread every time. A producer should wait for handler callbacks to
+return before calling the next handler callback. As a result, a consumer can manage back-pressure
+simply by sleeping or otherwise waiting to return from the ``on_next`` handler if necessary.
+
Review Comment:
Yet another subsection may be useful, to give guidance about serialization between the initial `ArrowAsyncDeviceStreamHandler` handoff (when the consumer passes the struct to the producer) and the first callback invocations (probably `on_schema` or `on_error`). Is the producer allowed to call `on_schema` as soon as the consumer passes the `ArrowAsyncDeviceStreamHandler` to it (which makes things harder for the client)? Or must the producer defer callback invocation after that (which is probably easier for the client, but perhaps harder for the producer)?
@westonpace Thoughts?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-43631: [C][Format] Add ArrowAsyncDeviceStreamHandler interface [arrow]
Posted by "zeroshade (via GitHub)" <gi...@apache.org>.
zeroshade commented on code in PR #43632:
URL: https://github.com/apache/arrow/pull/43632#discussion_r1718727206
##########
cpp/src/arrow/c/abi.h:
##########
@@ -228,6 +228,65 @@ struct ArrowDeviceArrayStream {
#endif // ARROW_C_DEVICE_STREAM_INTERFACE
+#ifndef ARROW_C_ASYNC_STREAM_INTERFACE
+#define ARROW_C_ASYNC_STREAM_INTERFACE
+
+// Similar to ArrowDeviceArrayStream, except designed for an asynchronous
+// style of interaction. While ArrowDeviceArrayStream provides producer
+// defined callbacks, this is intended to be created by the consumer instead.
+// The consumer passes this handler to the producer, which in turn uses the
+// callbacks to inform the consumer of events in the stream.
+struct ArrowAsyncDeviceStreamHandler {
+ // Handler for receiving a schema. The passed in stream_schema should be
+ // released or moved by the handler (producer is giving ownership of it to
+ // the handler).
+ //
+ // The `extension_param` argument can be null or can be used by a producer
+ // to pass arbitrary extra information to the consumer (such as total number
+ // of rows, context info, or otherwise).
+ //
+ // Return value: 0 if successful, `errno`-compatible error otherwise
+ int (*on_schema)(struct ArrowAsyncDeviceStreamHandler* self,
+ struct ArrowSchema* stream_schema, void* extension_param);
+
+ // Handler for receiving an array/record batch. Always called at least once
+ // unless an error is encountered (which would result in calling on_error).
+ // An empty/released array is passed to indicate the end of the stream if no
+ // errors have been encountered.
+ //
+ // The `extension_param` argument can be null or can be used by a producer
+ // to pass arbitrary extra information to the consumer.
+ //
+ // Return value: 0 if successful, `errno`-compatible error otherwise.
+ int (*on_next)(struct ArrowAsyncDeviceStreamHandler* self,
+ struct ArrowDeviceArray* next, void* extension_param);
+
+ // Handler for encountering an error. The producer should call release after
+ // this returns to clean up any resources.
+ //
+ // If the message or metadata are non-null, they will only last as long as this
+ // function call. The consumer would need to perform a copy of the data if it is
+ // it is necessary for them live past the lifetime of this call.
+ //
+ // Error metadata should be encoded as with metadata in ArrowSchema, defined in
+ // the spec at
+ // https://arrow.apache.org/docs/format/CDataInterface.html#c.ArrowSchema.metadata
+ //
+ // After this call, producers should follow-up by calling the release callback.
+ void (*on_error)(struct ArrowAsyncDeviceStreamHandler* self, int code,
+ const char* message, const char* metadata);
Review Comment:
This came out of the way that ADBC handles error context information. Rather than the case like `on_schema` / `on_next` where the `extension_param` can be anything depending on context and producer / consumer contracts. Error metadata is almost universally key-value pairs to provide contextual information either as strings or as interpretable bytes (i.e. grpc error codes/objects etc.) Rather than requiring everyone figure out how to handle this key-value management of error information themselves, it was suggested that we simply re-utilize the existing metadata encoding.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-43631: [C][Format] Add ArrowAsyncDeviceStreamHandler interface [arrow]
Posted by "zeroshade (via GitHub)" <gi...@apache.org>.
zeroshade commented on code in PR #43632:
URL: https://github.com/apache/arrow/pull/43632#discussion_r1718785140
##########
docs/source/format/CDeviceDataInterface.rst:
##########
@@ -650,6 +652,137 @@ The stream source is not assumed to be thread-safe. Consumers wanting to
call ``get_next`` from several threads should ensure those calls are
serialized.
+Async Device Stream Interface
+=============================
+
+The :ref:`C stream interface <_c-device-stream-interface>` provides a synchronous
+API centered around the consumer calling the callback functions to retrieve
+the next record batch. For some bindings, use cases, and interoperability, a more
+asynchronous, producer-focused interface may be required. These scenarios can utilize
+the ``ArrowAsyncDeviceStreamHandler``.
+
+Semantics
+---------
+
+Rather than the producer providing a structure of callbacks for a consumer to
+call and retrieve records, the Async interface is a consumer allocated structure.
+The consumer allocated struct provides handler callbacks for the producer to call
+when the schema and chunks of data are available, rather than the consumer using
+a blocking pull-style iteration.
+
+Structure Definition
+--------------------
+
+The C device async stream interface is defined with a single ``struct`` definition:
+
+.. code-block:: c
+
+ #ifndef ARROW_C_ASYNC_STREAM_INTERFACE
+ #define ARROW_C_ASYNC_STREAM_INTERFACE
+
+ struct ArrowAsyncDeviceStreamHandler {
+ // handlers
+ int (*on_schema)(struct ArrowAsyncDeviceStreamHandler* self,
+ struct ArrowSchema* stream_schema, void* extension_param);
+ int (*on_next)(struct ArrowAsyncDeviceStreamHandler* self,
+ struct ArrowDeviceArray* next, void* extension_param);
+ void (*on_error)(struct ArrowAsyncDeviceStreamHandler* self,
+ int code, const char* message, const char* metadata);
+
+ // release callback
+ void (*release)(struct ArrowAsyncDeviceStreamHandler* self);
+
+ // opaque handler-specific data
+ void* private_data;
+ };
+
+ #endif // ARROW_C_ASYNC_STREAM_INTERFACE
+
+.. note::
+ The canonical guard ``ARROW_C_ASYNC_STREAM_INTERFACE`` is meant to avoid
+ duplicate definitions if two projects copy the C async stream interface
+ definitions into their own headers, and a third-party project includes
+ from these two projects. It is therefore important that this guard is kept
+ exactly as-is when these definitions are copied.
+
+The ArrowAsyncDeviceStreamHandler structure
+'''''''''''''''''''''''''''''''''''''''''''
+
+The structure has the following fields:
+
+.. c:member:: int (*ArrowAsyncDeviceStreamHandler.on_schema)(struct ArrowAsyncDeviceStreamHandler*, struct ArrowSchema*, void*)
+
+ *Mandatory.* Handler for receiving the schema of the stream. All records should
+ match the provided schema. If successful, the function should return 0, otherwise
+ it should return an ``errno``-compatible error code.
+
+ The ``void*`` parameter exists for producers to provide any extra contextual information
+ they want, such as the total number of rows in the stream, statistics, or otherwise.
+
+ Unless the ``on_error`` handler is called, this will always get called exactly once.
+
+.. c:member:: int (*ArrowAsyncDeviceStreamHandler.on_next)(struct ArrowAsyncDeviceStreamHandler*, struct ArrowDeviceArray*, void*)
+
+ *Mandatory.* Handler to be called when a new record is available for processing. The
+ schema for each record should be the same as the schema that ``on_schema`` was called with.
+ If successfully handled, the function should return 0, otherwise it should return an
+ ``errno``-compatible error code.
+
+ The ``void*`` parameter exists for producers to provide any extra contextual information
+ they want.
+
+ Unless the ``on_error`` handler is called, this will always get called at least once. The
+ end of the stream is indicated by it being called with an empty or released ``ArrowDeviceArray``.
+
+.. c:member:: void (*ArrowAsyncDeviceStreamHandler.on_error)(struct ArrowAsyncDeviceStreamHandler, int, const char*, const char*)
+
+ *Mandatory.* Handler to be called when an error is encountered by the producer. After calling
+ this, the ``release`` callback will be called as the last call on this struct. The parameters
+ are an ``errno``-compatible error code and an optional error message and metadata.
+
+ If the message and metadata are not ``NULL``, their lifetime is only valid during the scope
+ of this call. A consumer who wants to maintain these values past the return of this function
+ *MUST* copy the values themselves.
+
+ If the metadata parameter is not ``NULL``, to provide key-value error metadata, then it should
+ be encoded identically to the way that metadata is encoded in :c:member:`ArrowSchema.metadata`.
+
+.. c:member:: void (*ArrowAsyncDeviceStreamHandler.release)(struct ArrowAsyncDeviceStreamHandler*)
+
+ *Mandatory.* A pointer to a consumer-provided release callback for the handler.
+
+.. c:member:: void* ArrowAsyncDeviceStreamHandler.private_data
+
+ *Optional.* An opaque pointer to consumer-provided private data.
+
+ Producers *MUST NOT* process this member. Lifetime of this member is handled by
+ the consumer, and especially by the release callback.
+
+Result lifetimes
Review Comment:
added a section on error handling
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-43631: [C][Format] Add ArrowAsyncDeviceStreamHandler interface [arrow]
Posted by "bkietz (via GitHub)" <gi...@apache.org>.
bkietz commented on code in PR #43632:
URL: https://github.com/apache/arrow/pull/43632#discussion_r1718884653
##########
cpp/src/arrow/c/abi.h:
##########
@@ -228,6 +228,65 @@ struct ArrowDeviceArrayStream {
#endif // ARROW_C_DEVICE_STREAM_INTERFACE
+#ifndef ARROW_C_ASYNC_STREAM_INTERFACE
+#define ARROW_C_ASYNC_STREAM_INTERFACE
+
+// Similar to ArrowDeviceArrayStream, except designed for an asynchronous
+// style of interaction. While ArrowDeviceArrayStream provides producer
+// defined callbacks, this is intended to be created by the consumer instead.
+// The consumer passes this handler to the producer, which in turn uses the
+// callbacks to inform the consumer of events in the stream.
+struct ArrowAsyncDeviceStreamHandler {
+ // Handler for receiving a schema. The passed in stream_schema should be
+ // released or moved by the handler (producer is giving ownership of it to
+ // the handler).
+ //
+ // The `extension_param` argument can be null or can be used by a producer
Review Comment:
The fact that it's `void*` implies that there's out-of-band communication between the producer and consumer. For example:
- the consumer is a GUI and the producer is a dataset library
- the consumer initiates an interaction by passing
- a query string
- an `ArrowAsyncDeviceStreamHandler`
- `ExtensionParamKind::TOTAL_NUMBER_OF_ROWS`
Since the producer received that last argument, it will ensure that `extension_param` points to an `int64_t` (or fail, or leave it null to indicate that capability is not available, or ...)
Since this scenario already assumes the producer and consumer can communicate what the extension parameter should be without recourse to the arrow C ABI, it seems odd to give them another more restrictive channel to communicate non-arrow objects.
I think this parameter should either be removed as out of scope or replaced with something accessible to consumers without out-of-band typeinfo (as @paleolimbot suggested, [metadata](https://arrow.apache.org/docs/format/CDataInterface.html#c.ArrowSchema.metadata) would work)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-43631: [C][Format] Add ArrowAsyncDeviceStreamHandler interface [arrow]
Posted by "zeroshade (via GitHub)" <gi...@apache.org>.
zeroshade commented on code in PR #43632:
URL: https://github.com/apache/arrow/pull/43632#discussion_r1713974576
##########
cpp/src/arrow/c/abi.h:
##########
@@ -228,6 +228,65 @@ struct ArrowDeviceArrayStream {
#endif // ARROW_C_DEVICE_STREAM_INTERFACE
+#ifndef ARROW_C_ASYNC_STREAM_INTERFACE
+#define ARROW_C_ASYNC_STREAM_INTERFACE
+
+// Similar to ArrowDeviceArrayStream, except designed for an asynchronous
+// style of interaction. While ArrowDeviceArrayStream provides producer
+// defined callbacks, this is intended to be created by the consumer instead.
+// The consumer passes this handler to the producer, which in turn uses the
+// callbacks to inform the consumer of events in the stream.
+struct ArrowAsyncDeviceStreamHandler {
+ // Handler for receiving a schema. The passed in stream_schema should be
+ // released or moved by the handler (producer is giving ownership of it to
+ // the handler).
+ //
+ // The `extension_param` argument can be null or can be used by a producer
+ // to pass arbitrary extra information to the consumer (such as total number
+ // of rows, context info, or otherwise).
+ //
+ // Return value: 0 if successful, `errno`-compatible error otherwise
Review Comment:
> Either way though, if an error is returned or if the consumer explicitly cancels in some way, I assume the producer needs to keep accepting calls to the callbacks right? (e.g. returning an error here won't prevent on_next from being called once or twice while the producer is cleaning up)
It won't prevent it, but the docs explicitly state that if an error is returned from the callback, the producer should *only* call release after that and not call further callbacks. though there isn't anything to functionally prevent it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-43631: [C][Format] Add ArrowAsyncDeviceStreamHandler interface [arrow]
Posted by "zeroshade (via GitHub)" <gi...@apache.org>.
zeroshade commented on code in PR #43632:
URL: https://github.com/apache/arrow/pull/43632#discussion_r1713969745
##########
cpp/src/arrow/c/abi.h:
##########
@@ -228,6 +228,65 @@ struct ArrowDeviceArrayStream {
#endif // ARROW_C_DEVICE_STREAM_INTERFACE
+#ifndef ARROW_C_ASYNC_STREAM_INTERFACE
+#define ARROW_C_ASYNC_STREAM_INTERFACE
+
+// Similar to ArrowDeviceArrayStream, except designed for an asynchronous
+// style of interaction. While ArrowDeviceArrayStream provides producer
+// defined callbacks, this is intended to be created by the consumer instead.
+// The consumer passes this handler to the producer, which in turn uses the
+// callbacks to inform the consumer of events in the stream.
+struct ArrowAsyncDeviceStreamHandler {
+ // Handler for receiving a schema. The passed in stream_schema should be
+ // released or moved by the handler (producer is giving ownership of it to
+ // the handler).
+ //
+ // The `extension_param` argument can be null or can be used by a producer
Review Comment:
> How should it be freed/what is the lifetime + ownership? (Especially in the case where the consumer may not understand the format of the extension and won't know how to free it?)
The intent was that the `extension_param` is entirely owned by the producer, the consumer shouldn't care or need to manage the lifetime of the extension at all.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-43631: [C][Format] Add ArrowAsyncDeviceStreamHandler interface [arrow]
Posted by "lidavidm (via GitHub)" <gi...@apache.org>.
lidavidm commented on code in PR #43632:
URL: https://github.com/apache/arrow/pull/43632#discussion_r1716191223
##########
cpp/src/arrow/c/abi.h:
##########
@@ -228,6 +228,65 @@ struct ArrowDeviceArrayStream {
#endif // ARROW_C_DEVICE_STREAM_INTERFACE
+#ifndef ARROW_C_ASYNC_STREAM_INTERFACE
+#define ARROW_C_ASYNC_STREAM_INTERFACE
+
+// Similar to ArrowDeviceArrayStream, except designed for an asynchronous
+// style of interaction. While ArrowDeviceArrayStream provides producer
+// defined callbacks, this is intended to be created by the consumer instead.
+// The consumer passes this handler to the producer, which in turn uses the
+// callbacks to inform the consumer of events in the stream.
+struct ArrowAsyncDeviceStreamHandler {
+ // Handler for receiving a schema. The passed in stream_schema should be
+ // released or moved by the handler (producer is giving ownership of it to
+ // the handler).
+ //
+ // The `extension_param` argument can be null or can be used by a producer
+ // to pass arbitrary extra information to the consumer (such as total number
+ // of rows, context info, or otherwise).
+ //
+ // Return value: 0 if successful, `errno`-compatible error otherwise
+ int (*on_schema)(struct ArrowAsyncDeviceStreamHandler* self,
+ struct ArrowSchema* stream_schema, void* extension_param);
+
+ // Handler for receiving an array/record batch. Always called at least once
+ // unless an error is encountered (which would result in calling on_error).
+ // An empty/released array is passed to indicate the end of the stream if no
+ // errors have been encountered.
+ //
+ // The `extension_param` argument can be null or can be used by a producer
+ // to pass arbitrary extra information to the consumer.
+ //
+ // Return value: 0 if successful, `errno`-compatible error otherwise.
+ int (*on_next)(struct ArrowAsyncDeviceStreamHandler* self,
+ struct ArrowDeviceArray* next, void* extension_param);
Review Comment:
So actually it's probably closer to Weston's proposal below.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-43631: [C][Format] Add ArrowAsyncDeviceStreamHandler interface [arrow]
Posted by "zeroshade (via GitHub)" <gi...@apache.org>.
zeroshade commented on code in PR #43632:
URL: https://github.com/apache/arrow/pull/43632#discussion_r1821320534
##########
cpp/src/arrow/c/abi.h:
##########
@@ -356,8 +356,7 @@ struct ArrowAsyncDeviceStreamHandler {
// A producer that receives a non-zero return here should stop producing and eventually
// call release instead.
int (*on_schema)(struct ArrowAsyncDeviceStreamHandler* self,
- struct ArrowAsyncProducer* producer, struct ArrowSchema* stream_schema,
- const char* addl_metadata);
+ struct ArrowSchema* stream_schema, const char* addl_metadata);
Review Comment:
While it would simplify the signature I'm not sure I necessarily like the idea as far as the lifetime handling of it.
@pitrou @lidavidm @paleolimbot what are your thoughts?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-43631: [C][Format] Add ArrowAsyncDeviceStreamHandler interface [arrow]
Posted by "zeroshade (via GitHub)" <gi...@apache.org>.
zeroshade commented on code in PR #43632:
URL: https://github.com/apache/arrow/pull/43632#discussion_r1816600519
##########
cpp/src/arrow/c/abi.h:
##########
@@ -228,6 +228,207 @@ struct ArrowDeviceArrayStream {
#endif // ARROW_C_DEVICE_STREAM_INTERFACE
+#ifndef ARROW_C_ASYNC_STREAM_INTERFACE
+#define ARROW_C_ASYNC_STREAM_INTERFACE
+
+// ArrowAsyncTask represents available data from a producer that was passed to
+// an invocation of `on_next_task` on the ArrowAsyncDeviceStreamHandler.
+//
+// The reason for this Task approach instead of the Async interface returning
+// the Array directly is to allow for more complex thread handling and reducing
+// context switching and data transfers between CPU cores (e.g. from one L1/L2
+// cache to another) if desired.
+//
+// For example, the `on_next_task` callback can be called when data is ready, while
+// the producer puts potential "decoding" logic in the `ArrowAsyncTask` object. This
+// allows for the producer to manage the I/O on one thread which calls `on_next_task`
+// and the consumer can determine when the decoding (producer logic in the `extract_data`
+// callback of the task) occurs and on which thread, to avoid a CPU core transfer
+// (data staying in the L2 cache).
+struct ArrowAsyncTask {
+ // This callback should populate the ArrowDeviceArray associated with this task.
+ // The order of ArrowAsyncTasks provided by the producer enables a consumer to
+ // ensure the order of data to process.
+ //
+ // This function is expected to be synchronous, but should not perform any blocking
+ // I/O. Ideally it should be as cheap as possible so as to not tie up the consumer
+ // thread unnecessarily.
+ //
+ // Returns: 0 if successful, errno-compatible error otherwise.
+ //
+ // If a non-0 value is returned then it should be followed by a call to `on_error`
+ // on the appropriate ArrowAsyncDeviceStreamHandler. This is because it's highly
+ // likely that whatever is calling this function may be entirely disconnected from
+ // the current control flow. Indicating an error here with a non-zero return allows
+ // the current flow to be aware of the error occurring, while still allowing any
+ // logging or error handling to still be centralized in the `on_error` callback of
+ // the original Async handler.
+ //
+ // Rather than a release callback, any required cleanup should be performed as part
+ // of the invocation of `extract_data`. Ownership of the Array is passed to the consumer
+ // calling this, and so it must be released separately.
+ //
+ // It is only valid to call this method exactly once.
+ int (*extract_data)(struct ArrowArrayTask* self, struct ArrowDeviceArray* out);
+
+ // opaque task-specific data
+ void* private_data;
+};
+
+// ArrowAsyncProducer represents a 1-to-1 relationship between an async producer
+// and consumer. This object allows the consumer to perform backpressure and flow
+// control on the asynchronous stream processing. This object must be owned by the
+// producer who creates it, and thus is responsible for cleaning it up.
Review Comment:
fun story, I took this advice and the implementation examples do this and I completely forgot to update the docs here. I'll update this appropriately. I agree that it should be an embedded member which should be initialized before calling `on_schema`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-43631: [C][Format] Add ArrowAsyncDeviceStreamHandler interface [arrow]
Posted by "bkietz (via GitHub)" <gi...@apache.org>.
bkietz commented on code in PR #43632:
URL: https://github.com/apache/arrow/pull/43632#discussion_r1766895138
##########
docs/source/format/CDeviceDataInterface.rst:
##########
@@ -650,6 +652,331 @@ The stream source is not assumed to be thread-safe. Consumers wanting to
call ``get_next`` from several threads should ensure those calls are
serialized.
+Async Device Stream Interface
+=============================
+
+The :ref:`C stream interface <_c-device-stream-interface>` provides a synchronous
+API centered around the consumer calling the callback functions to retrieve
+the next record batch. For some bindings, use cases, and interoperability, a more
+asynchronous, producer-focused interface may be required. These scenarios can utilize
+the ``ArrowAsyncDeviceStreamHandler``.
+
+Semantics
+---------
+
+Rather than the producer providing a structure of callbacks for a consumer to
+call and retrieve records, the Async interface is a consumer allocated structure.
+The consumer allocated struct provides handler callbacks for the producer to call
+when the schema and chunks of data are available, rather than the consumer using
+a blocking pull-style iteration.
+
+In addition to the ``ArrowAsyncDeviceStreamHandler``, there are also two additional
+structs used for the full data flow: ``ArrowAsyncTask`` and ``ArrowAsyncProducer``.
+
+Structure Definition
+--------------------
+
+The C device async stream interface is defined with a single ``struct`` definition:
+
+.. code-block:: c
+
+ #ifndef ARROW_C_ASYNC_STREAM_INTERFACE
+ #define ARROW_C_ASYNC_STREAM_INTERFACE
+
+ struct ArrowAsyncTask {
+ int (*get_data)(struct ArrowArrayTask* self, struct ArrowDeviceArray* out);
+
+ void* private_data;
+ };
+
+ struct ArrowAsyncProducer {
+ void (*request)(struct ArrowAsyncProducer* self, uint64_t n);
+ void (*cancel)(struct ArrowAsyncProducer* self);
+
+ void (*release)(struct ArrowAsyncProducer* self);
+ void* private_data;
+ };
+
+ struct ArrowAsyncDeviceStreamHandler {
+ // handlers
+ int (*on_schema)(struct ArrowAsyncDeviceStreamHandler* self,
+ struct ArrowAsyncProducer* producer,
+ struct ArrowSchema* stream_schema, const char* addl_metadata);
+ int (*on_next_task)(struct ArrowAsyncDeviceStreamHandler* self,
+ struct ArrowAsyncTask* task, const char* metadata);
+ void (*on_error)(struct ArrowAsyncDeviceStreamHandler* self,
+ int code, const char* message, const char* metadata);
+
+ // release callback
+ void (*release)(struct ArrowAsyncDeviceStreamHandler* self);
+
+ // opaque handler-specific data
+ void* private_data;
+ };
+
+ #endif // ARROW_C_ASYNC_STREAM_INTERFACE
+
+.. note::
+ The canonical guard ``ARROW_C_ASYNC_STREAM_INTERFACE`` is meant to avoid
+ duplicate definitions if two projects copy the C async stream interface
+ definitions into their own headers, and a third-party project includes
+ from these two projects. It is therefore important that this guard is kept
+ exactly as-is when these definitions are copied.
+
+The ArrowAsyncDeviceStreamHandler structure
+'''''''''''''''''''''''''''''''''''''''''''
+
+The structure has the following fields:
+
+.. c:member:: int (*ArrowAsyncDeviceStreamHandler.on_schema)(struct ArrowAsyncDeviceStreamHandler*, struct ArrowAsyncProducer*, struct ArrowSchema*, const char*)
+
+ *Mandatory.* Handler for receiving the schema of the stream. All records should
+ match the provided schema. If successful, the function should return 0, otherwise
+ it should return an ``errno``-compatible error code.
+
+ The ``const char*`` parameter exists for producers to provide any extra contextual information
+ they want, such as the total number of rows in the stream, statistics, or otherwise. This is
+ encoded in the same format as :c:member:`ArrowSchema.metadata`. If not ``NULL``,
+ the lifetime is only the scope of the call to this function. A consumer who wants to maintain
+ the additional metadata beyond the lifetime of this call *MUST* copy the value themselves.
+
+ Unless the ``on_error`` handler is called, this will always get called exactly once and will be
+ the first method called on this object. As such the producer *MUST* provide an ``ArrowAsyncProducer``
+ object when calling this function to allow the consumer to manage back-pressure and flow control.
+ The producer maintains ownership of the ``ArrowAsyncProducer`` and must clean it up before or after
+ calling the release callback on this object.
+
+ A producer that receives a non-zero result here must not subsequently call anything other than
+ the release callback on this object.
+
+.. c:member:: int (*ArrowAsyncDeviceStreamHandler.on_next_task)(struct ArrowAsyncDeviceStreamHandler*, struct ArrowAsyncTask*, const char*)
+
+ *Mandatory.* Handler to be called when a new record is available for processing. The
+ schema for each record should be the same as the schema that ``on_schema`` was called with.
+ If successfully handled, the function should return 0, otherwise it should return an
+ ``errno``-compatible error code.
+
+ Rather than passing the record itself it receives an ``ArrowAsyncTask`` instead to facilitate
+ better consumer-focused thread control as far as receiving the data. A call to this function
+ simply indicates that data is available via the provided task.
+
+ The ``const char*`` parameter exists for producers to provide any extra contextual information
+ they want. This is encoded in the same format as :c:member:`ArrowSchema.metadata`. If not ``NULL``,
+ the lifetime is only the scope of the call to this function. A consumer who wants to maintain
+ the additional metadata beyond the lifetime of this call *MUST* copy the value themselves.
+
+ A producer *MUST NOT* call this concurrently from multiple different threads.
+
+ The :c:member:`ArrowAsyncProducer.request` callback must be called to start receiving calls to this
+ handler.
+
+ This function *MUST* be able to be called re-entrantly on the same thread to allow for the
+ :c:member:`ArrowAsyncProducer.request` callback to potentially call this method recursively.
+
+.. c:member:: void (*ArrowAsyncDeviceStreamHandler.on_error)(struct ArrowAsyncDeviceStreamHandler, int, const char*, const char*)
+
+ *Mandatory.* Handler to be called when an error is encountered by the producer. After calling
+ this, the ``release`` callback will be called as the last call on this struct. The parameters
+ are an ``errno``-compatible error code and an optional error message and metadata.
+
+ If the message and metadata are not ``NULL``, their lifetime is only valid during the scope
+ of this call. A consumer who wants to maintain these values past the return of this function
+ *MUST* copy the values themselves.
+
+ If the metadata parameter is not ``NULL``, to provide key-value error metadata, then it should
+ be encoded identically to the way that metadata is encoded in :c:member:`ArrowSchema.metadata`.
+
+ It is valid for this to be called by a producer with or without a preceding call to
+ :c:member:`ArrowAsyncProducer.request`. This callback *MUST NOT* call any methods of an
+ ``ArrowAsyncProducer`` object.
+
+.. c:member:: void (*ArrowAsyncDeviceStreamHandler.release)(struct ArrowAsyncDeviceStreamHandler*)
+
+ *Mandatory.* A pointer to a consumer-provided release callback for the handler.
+
+ It is valid for this to be called by a producer with or without a preceding call to
+ :c:member:`ArrowAsyncProducer.request`. This must not call any methods of an ``ArrowAsyncProducer``
+ object.
+
+.. c:member:: void* ArrowAsyncDeviceStreamHandler.private_data
+
+ *Optional.* An opaque pointer to consumer-provided private data.
+
+ Producers *MUST NOT* process this member. Lifetime of this member is handled by
+ the consumer, and especially by the release callback.
+
+The ArrowAsyncTask structure
+''''''''''''''''''''''''''''
+
+The purpose of using a Task object rather than passing the array directly to the ``on_next``
+callback is to allow for more complex and efficient thread handling. Utilizing a Task
+object allows for a producer to separate the "decoding" logic from the I/O, enabling a
+consumer to avoid transferring data between CPU cores (e.g. from one L1/L2 cache to another).
Review Comment:
Maybe to emphasize that it is potentially doing extra work this should be renamed to `extract_data`?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-43631: [C][Format] Add ArrowAsyncDeviceStreamHandler interface [arrow]
Posted by "danepitkin (via GitHub)" <gi...@apache.org>.
danepitkin commented on code in PR #43632:
URL: https://github.com/apache/arrow/pull/43632#discussion_r1769122794
##########
cpp/src/arrow/c/abi.h:
##########
@@ -228,6 +228,207 @@ struct ArrowDeviceArrayStream {
#endif // ARROW_C_DEVICE_STREAM_INTERFACE
+#ifndef ARROW_C_ASYNC_STREAM_INTERFACE
+#define ARROW_C_ASYNC_STREAM_INTERFACE
+
+// ArrowAsyncTask represents available data from a producer that was passed to
+// an invocation of `on_next_task` on the ArrowAsyncDeviceStreamHandler.
+//
+// The reason for this Task approach instead of the Async interface returning
+// the Array directly is to allow for more complex thread handling and reducing
+// context switching and data transfers between CPU cores (e.g. from one L1/L2
+// cache to another) if desired.
+//
+// For example, the `on_next_task` callback can be called when data is ready, while
+// the producer puts potential "decoding" logic in the `ArrowAsyncTask` object. This
+// allows for the producer to manage the I/O on one thread which calls `on_next_task`
+// and the consumer can determine when the decoding (producer logic in the `get_data`
+// callback of the task) occurs and on which thread, to avoid a CPU core transfer
+// (data staying in the L2 cache).
+struct ArrowAsyncTask {
+ // This callback should populate the ArrowDeviceArray associated with this task.
+ // The order of ArrowAsyncTasks provided by the producer enables a consumer to
+ // ensure the order of data to process.
+ //
+ // This function is expected to be synchronous, but should not perform any blocking
+ // I/O. Ideally it should be as cheap as possible so as to not tie up the consumer
+ // thread unnecessarily.
+ //
+ // Returns: 0 if successful, errno-compatible error otherwise.
+ //
+ // If a non-0 value is returned then it should be followed by a call to `on_error`
+ // on the appropriate ArrowAsyncDeviceStreamHandler. This is because the it's highly
Review Comment:
```suggestion
// on the appropriate ArrowAsyncDeviceStreamHandler. This is because it's highly
```
##########
cpp/src/arrow/c/abi.h:
##########
@@ -228,6 +228,207 @@ struct ArrowDeviceArrayStream {
#endif // ARROW_C_DEVICE_STREAM_INTERFACE
+#ifndef ARROW_C_ASYNC_STREAM_INTERFACE
+#define ARROW_C_ASYNC_STREAM_INTERFACE
+
+// ArrowAsyncTask represents available data from a producer that was passed to
+// an invocation of `on_next_task` on the ArrowAsyncDeviceStreamHandler.
+//
+// The reason for this Task approach instead of the Async interface returning
+// the Array directly is to allow for more complex thread handling and reducing
+// context switching and data transfers between CPU cores (e.g. from one L1/L2
+// cache to another) if desired.
+//
+// For example, the `on_next_task` callback can be called when data is ready, while
+// the producer puts potential "decoding" logic in the `ArrowAsyncTask` object. This
+// allows for the producer to manage the I/O on one thread which calls `on_next_task`
+// and the consumer can determine when the decoding (producer logic in the `get_data`
+// callback of the task) occurs and on which thread, to avoid a CPU core transfer
+// (data staying in the L2 cache).
+struct ArrowAsyncTask {
+ // This callback should populate the ArrowDeviceArray associated with this task.
+ // The order of ArrowAsyncTasks provided by the producer enables a consumer to
+ // ensure the order of data to process.
+ //
+ // This function is expected to be synchronous, but should not perform any blocking
+ // I/O. Ideally it should be as cheap as possible so as to not tie up the consumer
+ // thread unnecessarily.
+ //
+ // Returns: 0 if successful, errno-compatible error otherwise.
+ //
+ // If a non-0 value is returned then it should be followed by a call to `on_error`
+ // on the appropriate ArrowAsyncDeviceStreamHandler. This is because the it's highly
+ // likely that whatever is calling this function may be entirely disconnected from
+ // the current control flow. Indicating an error here with a non-zero return allows
+ // the current flow to be aware of the error occurring, while still allowing any
+ // logging or error handling to still be centralized in the `on_error` callback of
+ // the original Async handler.
+ //
+ // Rather than a release callback, any required cleanup should be performed as part
+ // of the invocation of `get_data`. Ownership of the Array is passed to the consumer
+ // calling this, and so it must be released separately.
+ //
+ // It is only valid to call this method exactly once.
+ int (*get_data)(struct ArrowArrayTask* self, struct ArrowDeviceArray* out);
+
+ // opaque task-specific data
+ void* private_data;
+};
+
+// ArrowAsyncProducer represents a 1-to-1 relationship between an async producer
+// and consumer. This object allows the consumer to perform backpressure and flow
+// control on the asynchronous stream processing. This object must be owned by the
+// producer who creates it, and thus is responsible for cleaning it up.
+struct ArrowAsyncProducer {
+ // A consumer must call this function to start receiving on_next_task calls.
+ //
+ // It *must* be valid to call this synchronously from within `on_next_task` or
+ // `on_schema`, but this function *must not* immediately call `on_next_task` so as
+ // to avoid recursion and reentrant callbacks.
+ //
+ // After cancel has been called, additional calls to this function must be NOPs,
+ // but allowed. While not cancelled, calling this function must register the
+ // given number of additional arrays/batches to be produced with the producer.
+ // The producer should only call `on_next_task` at most the registered number
+ // of arrays before propagating backpressure.
+ //
+ // Any error encountered by calling request must be propagated by calling the `on_error`
+ // callback of the ArrowAsyncDeviceStreamHandler.
+ //
+ // While not cancelled, any subsequent calls to `on_next_task`, `on_error` or
+ // `release` should be scheduled by the producer to be called later rather
+ //
+ // It is invalid for a consumer to call this with a value of n <= 0, producers should
+ // error if given such a value.
+ void (*request)(struct ArrowAsyncProducer* self, int64_t n);
+
+ // This cancel callback signals a producer that it must eventually stop making calls
+ // to on_next_task. It must be idempotent and thread-safe. After calling cancel once,
+ // subsequent calls must be NOPs. This must not call any consumer-side handlers other
+ // than `on_error`.
+ //
+ // It is not required that calling cancel affect the producer immediately, only that it
+ // must eventually stop calling on_next_task and subsequently call release on the
+ // async handler. As such, a consumer must be prepared to receive one or more calls to
+ // `on_next_task` even after calling cancel if there are still requested arrays pending.
+ //
+ // Successful cancellation should *not* result in the producer calling `on_error`, it
+ // should finish out any remaining tasks and eventually call `release`.
+ //
+ // Any error encountered during handling a call to cancel must be reported via the
+ // on_error callback on the async stream handler.
+ void (*cancel)(struct ArrowAsyncProducer* self);
+
+ // producer-specific opaque data.
+ void* private_data;
+};
+
+// Similar to ArrowDeviceArrayStream, except designed for an asynchronous
+// style of interaction. While ArrowDeviceArrayStream provides producer
+// defined callbacks, this is intended to be created by the consumer instead.
+// The consumer passes this handler to the producer, which in turn uses the
+// callbacks to inform the consumer of events in the stream.
+struct ArrowAsyncDeviceStreamHandler {
+ // Handler for receiving a schema. The passed in stream_schema must be
+ // released or moved by the handler (producer is giving ownership of the schema to
+ // the handler, but not ownership of the top level object itself).
+ //
+ // With the exception of an error occurring (on_error), this must be the first
+ // callback function which is called by a producer and must only be called exactly
+ // once. As such, the producer should provide a valid ArrowAsyncProducer instance
+ // so the consumer can control the flow. See the documentation on ArrowAsyncProducer
+ // for how it works. The ArrowAsyncProducer is owned by the producer who calls this
+ // function and thus the producer is responsible for cleaning it up when calling
+ // the release callback of this handler.
+ //
+ // The addl_metadata argument can be null or can be used by a producer
+ // to pass arbitrary extra information to the consumer beyond the metadata in the schema
+ // itself (such as total number of rows, context info, or otherwise). The data should
+ // be passed using the same encoding as the metadata within the ArrowSchema struct
+ // itself (defined in the spec at
+ // https://arrow.apache.org/docs/format/CDataInterface.html#c.ArrowSchema.metadata)
+ //
+ // If addl_metadata is non-null then it only needs to exist for the lifetime of this
+ // call, a consumer who wants it to live after that must copy it to ensure lifetime.
+ //
+ // Return value: 0 if successful, `errno`-compatible error otherwise
+ //
+ // A producer that receives a non-zero return here should stop producing and eventually
+ // call release instead.
+ int (*on_schema)(struct ArrowAsyncDeviceStreamHandler* self,
+ struct ArrowAsyncProducer* producer, struct ArrowSchema* stream_schema,
+ const char* addl_metadata);
Review Comment:
nit: should we rename `addl_metadata` -> `metadata` to match the other APIs introduced here? Removing the acronym can makes it more readable IMO.
##########
cpp/src/arrow/c/abi.h:
##########
@@ -228,6 +228,207 @@ struct ArrowDeviceArrayStream {
#endif // ARROW_C_DEVICE_STREAM_INTERFACE
+#ifndef ARROW_C_ASYNC_STREAM_INTERFACE
+#define ARROW_C_ASYNC_STREAM_INTERFACE
+
+// ArrowAsyncTask represents available data from a producer that was passed to
+// an invocation of `on_next_task` on the ArrowAsyncDeviceStreamHandler.
+//
+// The reason for this Task approach instead of the Async interface returning
+// the Array directly is to allow for more complex thread handling and reducing
+// context switching and data transfers between CPU cores (e.g. from one L1/L2
+// cache to another) if desired.
+//
+// For example, the `on_next_task` callback can be called when data is ready, while
+// the producer puts potential "decoding" logic in the `ArrowAsyncTask` object. This
+// allows for the producer to manage the I/O on one thread which calls `on_next_task`
+// and the consumer can determine when the decoding (producer logic in the `get_data`
+// callback of the task) occurs and on which thread, to avoid a CPU core transfer
+// (data staying in the L2 cache).
+struct ArrowAsyncTask {
+ // This callback should populate the ArrowDeviceArray associated with this task.
+ // The order of ArrowAsyncTasks provided by the producer enables a consumer to
+ // ensure the order of data to process.
+ //
+ // This function is expected to be synchronous, but should not perform any blocking
+ // I/O. Ideally it should be as cheap as possible so as to not tie up the consumer
+ // thread unnecessarily.
+ //
+ // Returns: 0 if successful, errno-compatible error otherwise.
+ //
+ // If a non-0 value is returned then it should be followed by a call to `on_error`
+ // on the appropriate ArrowAsyncDeviceStreamHandler. This is because the it's highly
+ // likely that whatever is calling this function may be entirely disconnected from
+ // the current control flow. Indicating an error here with a non-zero return allows
+ // the current flow to be aware of the error occurring, while still allowing any
+ // logging or error handling to still be centralized in the `on_error` callback of
+ // the original Async handler.
+ //
+ // Rather than a release callback, any required cleanup should be performed as part
+ // of the invocation of `get_data`. Ownership of the Array is passed to the consumer
+ // calling this, and so it must be released separately.
+ //
+ // It is only valid to call this method exactly once.
+ int (*get_data)(struct ArrowArrayTask* self, struct ArrowDeviceArray* out);
+
+ // opaque task-specific data
+ void* private_data;
+};
+
+// ArrowAsyncProducer represents a 1-to-1 relationship between an async producer
+// and consumer. This object allows the consumer to perform backpressure and flow
+// control on the asynchronous stream processing. This object must be owned by the
+// producer who creates it, and thus is responsible for cleaning it up.
+struct ArrowAsyncProducer {
+ // A consumer must call this function to start receiving on_next_task calls.
+ //
+ // It *must* be valid to call this synchronously from within `on_next_task` or
+ // `on_schema`, but this function *must not* immediately call `on_next_task` so as
+ // to avoid recursion and reentrant callbacks.
+ //
+ // After cancel has been called, additional calls to this function must be NOPs,
+ // but allowed. While not cancelled, calling this function must register the
+ // given number of additional arrays/batches to be produced with the producer.
+ // The producer should only call `on_next_task` at most the registered number
+ // of arrays before propagating backpressure.
+ //
+ // Any error encountered by calling request must be propagated by calling the `on_error`
+ // callback of the ArrowAsyncDeviceStreamHandler.
+ //
+ // While not cancelled, any subsequent calls to `on_next_task`, `on_error` or
+ // `release` should be scheduled by the producer to be called later rather
+ //
+ // It is invalid for a consumer to call this with a value of n <= 0, producers should
+ // error if given such a value.
+ void (*request)(struct ArrowAsyncProducer* self, int64_t n);
+
+ // This cancel callback signals a producer that it must eventually stop making calls
+ // to on_next_task. It must be idempotent and thread-safe. After calling cancel once,
+ // subsequent calls must be NOPs. This must not call any consumer-side handlers other
+ // than `on_error`.
+ //
+ // It is not required that calling cancel affect the producer immediately, only that it
+ // must eventually stop calling on_next_task and subsequently call release on the
+ // async handler. As such, a consumer must be prepared to receive one or more calls to
+ // `on_next_task` even after calling cancel if there are still requested arrays pending.
+ //
+ // Successful cancellation should *not* result in the producer calling `on_error`, it
+ // should finish out any remaining tasks and eventually call `release`.
+ //
+ // Any error encountered during handling a call to cancel must be reported via the
+ // on_error callback on the async stream handler.
+ void (*cancel)(struct ArrowAsyncProducer* self);
+
+ // producer-specific opaque data.
+ void* private_data;
+};
+
+// Similar to ArrowDeviceArrayStream, except designed for an asynchronous
+// style of interaction. While ArrowDeviceArrayStream provides producer
+// defined callbacks, this is intended to be created by the consumer instead.
+// The consumer passes this handler to the producer, which in turn uses the
+// callbacks to inform the consumer of events in the stream.
+struct ArrowAsyncDeviceStreamHandler {
+ // Handler for receiving a schema. The passed in stream_schema must be
+ // released or moved by the handler (producer is giving ownership of the schema to
+ // the handler, but not ownership of the top level object itself).
+ //
+ // With the exception of an error occurring (on_error), this must be the first
+ // callback function which is called by a producer and must only be called exactly
+ // once. As such, the producer should provide a valid ArrowAsyncProducer instance
+ // so the consumer can control the flow. See the documentation on ArrowAsyncProducer
+ // for how it works. The ArrowAsyncProducer is owned by the producer who calls this
+ // function and thus the producer is responsible for cleaning it up when calling
+ // the release callback of this handler.
+ //
+ // The addl_metadata argument can be null or can be used by a producer
+ // to pass arbitrary extra information to the consumer beyond the metadata in the schema
+ // itself (such as total number of rows, context info, or otherwise). The data should
+ // be passed using the same encoding as the metadata within the ArrowSchema struct
+ // itself (defined in the spec at
+ // https://arrow.apache.org/docs/format/CDataInterface.html#c.ArrowSchema.metadata)
+ //
+ // If addl_metadata is non-null then it only needs to exist for the lifetime of this
+ // call, a consumer who wants it to live after that must copy it to ensure lifetime.
+ //
+ // Return value: 0 if successful, `errno`-compatible error otherwise
+ //
+ // A producer that receives a non-zero return here should stop producing and eventually
+ // call release instead.
+ int (*on_schema)(struct ArrowAsyncDeviceStreamHandler* self,
+ struct ArrowAsyncProducer* producer, struct ArrowSchema* stream_schema,
+ const char* addl_metadata);
+
+ // Handler for receiving data. This is called when data is available providing an
+ // ArrowAsyncTask struct to signify it. The producer indicates the end of the stream
+ // by passing NULL as the value for the task rather than a valid pointer to a task.
+ // The task object is only valid for the lifetime of this function call, if a consumer
+ // wants to utilize it after this function returns, it must copy or move the contents
+ // of it to a new ArrowAsyncTask object.
+ //
+ // The `request` callback of a provided ArrowAsyncProducer must be called in order
+ // to start receiving calls to this handler.
+ //
+ // The metadata argument can be null or can be used by a producer
+ // to pass arbitrary extra information to the consumer (such as total number
+ // of rows, context info, or otherwise). The data should be passed using the same
+ // encoding as the metadata within the ArrowSchema struct itself (defined in
+ // the spec at
+ // https://arrow.apache.org/docs/format/CDataInterface.html#c.ArrowSchema.metadata)
+ //
+ // If metadata is non-null then it only needs to exist for the lifetime of this call,
+ // a consumer who wants it to live after that must copy it to ensure lifetime.
+ //
+ // A producer *must not* call this concurrently from multiple different threads.
+ //
+ // A consumer must be prepared to received one or more calls to this callback even
Review Comment:
```suggestion
// A consumer must be prepared to receive one or more calls to this callback even
```
##########
cpp/src/arrow/c/abi.h:
##########
@@ -228,6 +228,207 @@ struct ArrowDeviceArrayStream {
#endif // ARROW_C_DEVICE_STREAM_INTERFACE
+#ifndef ARROW_C_ASYNC_STREAM_INTERFACE
+#define ARROW_C_ASYNC_STREAM_INTERFACE
+
+// ArrowAsyncTask represents available data from a producer that was passed to
+// an invocation of `on_next_task` on the ArrowAsyncDeviceStreamHandler.
+//
+// The reason for this Task approach instead of the Async interface returning
+// the Array directly is to allow for more complex thread handling and reducing
+// context switching and data transfers between CPU cores (e.g. from one L1/L2
+// cache to another) if desired.
+//
+// For example, the `on_next_task` callback can be called when data is ready, while
+// the producer puts potential "decoding" logic in the `ArrowAsyncTask` object. This
+// allows for the producer to manage the I/O on one thread which calls `on_next_task`
+// and the consumer can determine when the decoding (producer logic in the `get_data`
+// callback of the task) occurs and on which thread, to avoid a CPU core transfer
+// (data staying in the L2 cache).
+struct ArrowAsyncTask {
+ // This callback should populate the ArrowDeviceArray associated with this task.
+ // The order of ArrowAsyncTasks provided by the producer enables a consumer to
+ // ensure the order of data to process.
+ //
+ // This function is expected to be synchronous, but should not perform any blocking
+ // I/O. Ideally it should be as cheap as possible so as to not tie up the consumer
+ // thread unnecessarily.
+ //
+ // Returns: 0 if successful, errno-compatible error otherwise.
+ //
+ // If a non-0 value is returned then it should be followed by a call to `on_error`
+ // on the appropriate ArrowAsyncDeviceStreamHandler. This is because the it's highly
+ // likely that whatever is calling this function may be entirely disconnected from
+ // the current control flow. Indicating an error here with a non-zero return allows
+ // the current flow to be aware of the error occurring, while still allowing any
+ // logging or error handling to still be centralized in the `on_error` callback of
+ // the original Async handler.
+ //
+ // Rather than a release callback, any required cleanup should be performed as part
+ // of the invocation of `get_data`. Ownership of the Array is passed to the consumer
+ // calling this, and so it must be released separately.
+ //
+ // It is only valid to call this method exactly once.
+ int (*get_data)(struct ArrowArrayTask* self, struct ArrowDeviceArray* out);
+
+ // opaque task-specific data
+ void* private_data;
+};
+
+// ArrowAsyncProducer represents a 1-to-1 relationship between an async producer
+// and consumer. This object allows the consumer to perform backpressure and flow
+// control on the asynchronous stream processing. This object must be owned by the
+// producer who creates it, and thus is responsible for cleaning it up.
+struct ArrowAsyncProducer {
+ // A consumer must call this function to start receiving on_next_task calls.
+ //
+ // It *must* be valid to call this synchronously from within `on_next_task` or
+ // `on_schema`, but this function *must not* immediately call `on_next_task` so as
+ // to avoid recursion and reentrant callbacks.
+ //
+ // After cancel has been called, additional calls to this function must be NOPs,
+ // but allowed. While not cancelled, calling this function must register the
+ // given number of additional arrays/batches to be produced with the producer.
+ // The producer should only call `on_next_task` at most the registered number
+ // of arrays before propagating backpressure.
+ //
+ // Any error encountered by calling request must be propagated by calling the `on_error`
+ // callback of the ArrowAsyncDeviceStreamHandler.
+ //
+ // While not cancelled, any subsequent calls to `on_next_task`, `on_error` or
+ // `release` should be scheduled by the producer to be called later rather
Review Comment:
```suggestion
// `release` should be scheduled by the producer to be called later.
```
##########
cpp/src/arrow/c/abi.h:
##########
@@ -228,6 +228,207 @@ struct ArrowDeviceArrayStream {
#endif // ARROW_C_DEVICE_STREAM_INTERFACE
+#ifndef ARROW_C_ASYNC_STREAM_INTERFACE
+#define ARROW_C_ASYNC_STREAM_INTERFACE
+
+// ArrowAsyncTask represents available data from a producer that was passed to
+// an invocation of `on_next_task` on the ArrowAsyncDeviceStreamHandler.
+//
+// The reason for this Task approach instead of the Async interface returning
+// the Array directly is to allow for more complex thread handling and reducing
+// context switching and data transfers between CPU cores (e.g. from one L1/L2
+// cache to another) if desired.
+//
+// For example, the `on_next_task` callback can be called when data is ready, while
+// the producer puts potential "decoding" logic in the `ArrowAsyncTask` object. This
+// allows for the producer to manage the I/O on one thread which calls `on_next_task`
+// and the consumer can determine when the decoding (producer logic in the `get_data`
+// callback of the task) occurs and on which thread, to avoid a CPU core transfer
+// (data staying in the L2 cache).
+struct ArrowAsyncTask {
+ // This callback should populate the ArrowDeviceArray associated with this task.
+ // The order of ArrowAsyncTasks provided by the producer enables a consumer to
+ // ensure the order of data to process.
+ //
+ // This function is expected to be synchronous, but should not perform any blocking
+ // I/O. Ideally it should be as cheap as possible so as to not tie up the consumer
+ // thread unnecessarily.
+ //
+ // Returns: 0 if successful, errno-compatible error otherwise.
+ //
+ // If a non-0 value is returned then it should be followed by a call to `on_error`
+ // on the appropriate ArrowAsyncDeviceStreamHandler. This is because the it's highly
+ // likely that whatever is calling this function may be entirely disconnected from
+ // the current control flow. Indicating an error here with a non-zero return allows
+ // the current flow to be aware of the error occurring, while still allowing any
+ // logging or error handling to still be centralized in the `on_error` callback of
+ // the original Async handler.
+ //
+ // Rather than a release callback, any required cleanup should be performed as part
+ // of the invocation of `get_data`. Ownership of the Array is passed to the consumer
+ // calling this, and so it must be released separately.
+ //
+ // It is only valid to call this method exactly once.
+ int (*get_data)(struct ArrowArrayTask* self, struct ArrowDeviceArray* out);
+
+ // opaque task-specific data
+ void* private_data;
+};
+
+// ArrowAsyncProducer represents a 1-to-1 relationship between an async producer
+// and consumer. This object allows the consumer to perform backpressure and flow
+// control on the asynchronous stream processing. This object must be owned by the
+// producer who creates it, and thus is responsible for cleaning it up.
+struct ArrowAsyncProducer {
+ // A consumer must call this function to start receiving on_next_task calls.
+ //
+ // It *must* be valid to call this synchronously from within `on_next_task` or
+ // `on_schema`, but this function *must not* immediately call `on_next_task` so as
+ // to avoid recursion and reentrant callbacks.
+ //
+ // After cancel has been called, additional calls to this function must be NOPs,
+ // but allowed. While not cancelled, calling this function must register the
+ // given number of additional arrays/batches to be produced with the producer.
+ // The producer should only call `on_next_task` at most the registered number
+ // of arrays before propagating backpressure.
+ //
+ // Any error encountered by calling request must be propagated by calling the `on_error`
+ // callback of the ArrowAsyncDeviceStreamHandler.
+ //
+ // While not cancelled, any subsequent calls to `on_next_task`, `on_error` or
+ // `release` should be scheduled by the producer to be called later rather
+ //
+ // It is invalid for a consumer to call this with a value of n <= 0, producers should
+ // error if given such a value.
+ void (*request)(struct ArrowAsyncProducer* self, int64_t n);
+
+ // This cancel callback signals a producer that it must eventually stop making calls
+ // to on_next_task. It must be idempotent and thread-safe. After calling cancel once,
+ // subsequent calls must be NOPs. This must not call any consumer-side handlers other
+ // than `on_error`.
+ //
+ // It is not required that calling cancel affect the producer immediately, only that it
+ // must eventually stop calling on_next_task and subsequently call release on the
+ // async handler. As such, a consumer must be prepared to receive one or more calls to
+ // `on_next_task` even after calling cancel if there are still requested arrays pending.
+ //
+ // Successful cancellation should *not* result in the producer calling `on_error`, it
+ // should finish out any remaining tasks and eventually call `release`.
+ //
+ // Any error encountered during handling a call to cancel must be reported via the
+ // on_error callback on the async stream handler.
+ void (*cancel)(struct ArrowAsyncProducer* self);
+
+ // producer-specific opaque data.
+ void* private_data;
+};
+
+// Similar to ArrowDeviceArrayStream, except designed for an asynchronous
+// style of interaction. While ArrowDeviceArrayStream provides producer
+// defined callbacks, this is intended to be created by the consumer instead.
+// The consumer passes this handler to the producer, which in turn uses the
+// callbacks to inform the consumer of events in the stream.
+struct ArrowAsyncDeviceStreamHandler {
+ // Handler for receiving a schema. The passed in stream_schema must be
+ // released or moved by the handler (producer is giving ownership of the schema to
+ // the handler, but not ownership of the top level object itself).
+ //
+ // With the exception of an error occurring (on_error), this must be the first
+ // callback function which is called by a producer and must only be called exactly
+ // once. As such, the producer should provide a valid ArrowAsyncProducer instance
+ // so the consumer can control the flow. See the documentation on ArrowAsyncProducer
+ // for how it works. The ArrowAsyncProducer is owned by the producer who calls this
+ // function and thus the producer is responsible for cleaning it up when calling
+ // the release callback of this handler.
+ //
+ // The addl_metadata argument can be null or can be used by a producer
+ // to pass arbitrary extra information to the consumer beyond the metadata in the schema
+ // itself (such as total number of rows, context info, or otherwise). The data should
+ // be passed using the same encoding as the metadata within the ArrowSchema struct
+ // itself (defined in the spec at
+ // https://arrow.apache.org/docs/format/CDataInterface.html#c.ArrowSchema.metadata)
+ //
+ // If addl_metadata is non-null then it only needs to exist for the lifetime of this
+ // call, a consumer who wants it to live after that must copy it to ensure lifetime.
+ //
+ // Return value: 0 if successful, `errno`-compatible error otherwise
+ //
+ // A producer that receives a non-zero return here should stop producing and eventually
+ // call release instead.
+ int (*on_schema)(struct ArrowAsyncDeviceStreamHandler* self,
+ struct ArrowAsyncProducer* producer, struct ArrowSchema* stream_schema,
+ const char* addl_metadata);
+
+ // Handler for receiving data. This is called when data is available providing an
+ // ArrowAsyncTask struct to signify it. The producer indicates the end of the stream
+ // by passing NULL as the value for the task rather than a valid pointer to a task.
+ // The task object is only valid for the lifetime of this function call, if a consumer
+ // wants to utilize it after this function returns, it must copy or move the contents
+ // of it to a new ArrowAsyncTask object.
+ //
+ // The `request` callback of a provided ArrowAsyncProducer must be called in order
+ // to start receiving calls to this handler.
+ //
+ // The metadata argument can be null or can be used by a producer
+ // to pass arbitrary extra information to the consumer (such as total number
+ // of rows, context info, or otherwise). The data should be passed using the same
+ // encoding as the metadata within the ArrowSchema struct itself (defined in
+ // the spec at
+ // https://arrow.apache.org/docs/format/CDataInterface.html#c.ArrowSchema.metadata)
+ //
+ // If metadata is non-null then it only needs to exist for the lifetime of this call,
+ // a consumer who wants it to live after that must copy it to ensure lifetime.
+ //
+ // A producer *must not* call this concurrently from multiple different threads.
+ //
+ // A consumer must be prepared to received one or more calls to this callback even
+ // after calling cancel on the corresponding ArrowAsyncProducer, as cancel does not
+ // guarantee it happens immediately.
+ //
+ // Return value: 0 if successful, `errno`-compatible error otherwise.
+ //
+ // If the consumer returns a non-zero return from this method, that indicates to the
+ // producer that it should stop propagating data as an error occurred. After receiving
+ // such a return, the only interaction with this object is for the producer to call
+ // the `release` callback.
+ int (*on_next_task)(struct ArrowAsyncDeviceStreamHandler* self,
+ struct ArrowAsyncTask* task, const char* metadata);
+
+ // Handler for encountering an error. The producer should call release after
+ // this returns to clean up any resources. The `code` passed in can be any error
+ // code that a producer wants, but should be errno-compatible for consistency.
+ //
+ // If the message or metadata are non-null, they will only last as long as this
+ // function call. The consumer would need to perform a copy of the data if it is
+ // it is necessary for them live past the lifetime of this call.
Review Comment:
```suggestion
// necessary for them to live past the lifetime of this call.
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-43631: [C][Format] Add ArrowAsyncDeviceStreamHandler interface [arrow]
Posted by "bkietz (via GitHub)" <gi...@apache.org>.
bkietz commented on code in PR #43632:
URL: https://github.com/apache/arrow/pull/43632#discussion_r1769371343
##########
docs/source/format/CDeviceDataInterface.rst:
##########
@@ -650,6 +652,340 @@ The stream source is not assumed to be thread-safe. Consumers wanting to
call ``get_next`` from several threads should ensure those calls are
serialized.
+Async Device Stream Interface
+=============================
+
+The :ref:`C stream interface <_c-device-stream-interface>` provides a synchronous
+API centered around the consumer calling the callback functions to retrieve
+the next record batch. For some bindings, use cases, and interoperability, a more
+asynchronous, producer-focused interface may be required. These scenarios can utilize
+the ``ArrowAsyncDeviceStreamHandler``.
+
+Semantics
+---------
+
+Rather than the producer providing a structure of callbacks for a consumer to
+call and retrieve records, the Async interface is a consumer allocated structure.
+The consumer allocated struct provides handler callbacks for the producer to call
+when the schema and chunks of data are available, rather than the consumer using
+a blocking pull-style iteration.
+
+In addition to the ``ArrowAsyncDeviceStreamHandler``, there are also two additional
+structs used for the full data flow: ``ArrowAsyncTask`` and ``ArrowAsyncProducer``.
+
+Structure Definition
+--------------------
+
+The C device async stream interface is defined with a single ``struct`` definition:
+
+.. code-block:: c
+
+ #ifndef ARROW_C_ASYNC_STREAM_INTERFACE
+ #define ARROW_C_ASYNC_STREAM_INTERFACE
+
+ struct ArrowAsyncTask {
+ int (*get_data)(struct ArrowArrayTask* self, struct ArrowDeviceArray* out);
+
+ void* private_data;
+ };
+
+ struct ArrowAsyncProducer {
+ void (*request)(struct ArrowAsyncProducer* self, int64_t n);
+ void (*cancel)(struct ArrowAsyncProducer* self);
+
+ void (*release)(struct ArrowAsyncProducer* self);
+ void* private_data;
+ };
+
+ struct ArrowAsyncDeviceStreamHandler {
+ // handlers
+ int (*on_schema)(struct ArrowAsyncDeviceStreamHandler* self,
+ struct ArrowAsyncProducer* producer,
+ struct ArrowSchema* stream_schema, const char* addl_metadata);
+ int (*on_next_task)(struct ArrowAsyncDeviceStreamHandler* self,
+ struct ArrowAsyncTask* task, const char* metadata);
+ void (*on_error)(struct ArrowAsyncDeviceStreamHandler* self,
+ int code, const char* message, const char* metadata);
+
+ // release callback
+ void (*release)(struct ArrowAsyncDeviceStreamHandler* self);
+
+ // opaque handler-specific data
+ void* private_data;
+ };
+
+ #endif // ARROW_C_ASYNC_STREAM_INTERFACE
+
+.. note::
+ The canonical guard ``ARROW_C_ASYNC_STREAM_INTERFACE`` is meant to avoid
+ duplicate definitions if two projects copy the C async stream interface
+ definitions into their own headers, and a third-party project includes
+ from these two projects. It is therefore important that this guard is kept
+ exactly as-is when these definitions are copied.
+
+The ArrowAsyncDeviceStreamHandler structure
+'''''''''''''''''''''''''''''''''''''''''''
+
+The structure has the following fields:
+
+.. c:member:: int (*ArrowAsyncDeviceStreamHandler.on_schema)(struct ArrowAsyncDeviceStreamHandler*, struct ArrowAsyncProducer*, struct ArrowSchema*, const char*)
+
+ *Mandatory.* Handler for receiving the schema of the stream. All records should
+ match the provided schema. If successful, the function should return 0, otherwise
+ it should return an ``errno``-compatible error code.
+
+ The ``const char*`` parameter exists for producers to provide any extra contextual information
+ they want, such as the total number of rows in the stream, statistics, or otherwise. This is
+ encoded in the same format as :c:member:`ArrowSchema.metadata`. If not ``NULL``,
+ the lifetime is only the scope of the call to this function. A consumer who wants to maintain
+ the additional metadata beyond the lifetime of this call *MUST* copy the value themselves.
+
+ Unless the ``on_error`` handler is called, this will always get called exactly once and will be
+ the first method called on this object. As such the producer *MUST* provide an ``ArrowAsyncProducer``
+ object when calling this function to allow the consumer to manage back-pressure and flow control.
+ The producer maintains ownership of the ``ArrowAsyncProducer`` and must clean it up *after*
+ calling the release callback on this object.
+
+ A producer that receives a non-zero result here must not subsequently call anything other than
+ the release callback on this object.
+
+.. c:member:: int (*ArrowAsyncDeviceStreamHandler.on_next_task)(struct ArrowAsyncDeviceStreamHandler*, struct ArrowAsyncTask*, const char*)
+
+ *Mandatory.* Handler to be called when a new record is available for processing. The
+ schema for each record should be the same as the schema that ``on_schema`` was called with.
+ If successfully handled, the function should return 0, otherwise it should return an
+ ``errno``-compatible error code.
+
+ Rather than passing the record itself it receives an ``ArrowAsyncTask`` instead to facilitate
+ better consumer-focused thread control as far as receiving the data. A call to this function
+ simply indicates that data is available via the provided task.
+
+ The producer signals the end of the stream by passing ``NULL`` for the ``ArrowAsyncTask``
+ pointer instead of a valid address. This task object is only valid during the lifetime of
+ this function call. If the consumer wants to use the task beyond the scope of this method, it
+ must copy or move its contents to a new ArrowAsyncTask object.
Review Comment:
`consume_data` sounds like a function to which data is passed rather than a function from which data is returned
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-43631: [C][Format] Add ArrowAsyncDeviceStreamHandler interface [arrow]
Posted by "zeroshade (via GitHub)" <gi...@apache.org>.
zeroshade commented on code in PR #43632:
URL: https://github.com/apache/arrow/pull/43632#discussion_r1770572224
##########
docs/source/format/CDeviceDataInterface.rst:
##########
@@ -650,6 +652,340 @@ The stream source is not assumed to be thread-safe. Consumers wanting to
call ``get_next`` from several threads should ensure those calls are
serialized.
+Async Device Stream Interface
+=============================
+
+The :ref:`C stream interface <_c-device-stream-interface>` provides a synchronous
+API centered around the consumer calling the callback functions to retrieve
+the next record batch. For some bindings, use cases, and interoperability, a more
+asynchronous, producer-focused interface may be required. These scenarios can utilize
+the ``ArrowAsyncDeviceStreamHandler``.
+
+Semantics
+---------
+
+Rather than the producer providing a structure of callbacks for a consumer to
+call and retrieve records, the Async interface is a consumer allocated structure.
+The consumer allocated struct provides handler callbacks for the producer to call
+when the schema and chunks of data are available, rather than the consumer using
+a blocking pull-style iteration.
+
+In addition to the ``ArrowAsyncDeviceStreamHandler``, there are also two additional
+structs used for the full data flow: ``ArrowAsyncTask`` and ``ArrowAsyncProducer``.
+
+Structure Definition
+--------------------
+
+The C device async stream interface is defined with a single ``struct`` definition:
+
+.. code-block:: c
+
+ #ifndef ARROW_C_ASYNC_STREAM_INTERFACE
+ #define ARROW_C_ASYNC_STREAM_INTERFACE
+
+ struct ArrowAsyncTask {
+ int (*get_data)(struct ArrowArrayTask* self, struct ArrowDeviceArray* out);
+
+ void* private_data;
+ };
+
+ struct ArrowAsyncProducer {
+ void (*request)(struct ArrowAsyncProducer* self, int64_t n);
+ void (*cancel)(struct ArrowAsyncProducer* self);
+
+ void (*release)(struct ArrowAsyncProducer* self);
+ void* private_data;
+ };
+
+ struct ArrowAsyncDeviceStreamHandler {
+ // handlers
+ int (*on_schema)(struct ArrowAsyncDeviceStreamHandler* self,
+ struct ArrowAsyncProducer* producer,
+ struct ArrowSchema* stream_schema, const char* addl_metadata);
+ int (*on_next_task)(struct ArrowAsyncDeviceStreamHandler* self,
+ struct ArrowAsyncTask* task, const char* metadata);
+ void (*on_error)(struct ArrowAsyncDeviceStreamHandler* self,
+ int code, const char* message, const char* metadata);
+
+ // release callback
+ void (*release)(struct ArrowAsyncDeviceStreamHandler* self);
+
+ // opaque handler-specific data
+ void* private_data;
+ };
+
+ #endif // ARROW_C_ASYNC_STREAM_INTERFACE
+
+.. note::
+ The canonical guard ``ARROW_C_ASYNC_STREAM_INTERFACE`` is meant to avoid
+ duplicate definitions if two projects copy the C async stream interface
+ definitions into their own headers, and a third-party project includes
+ from these two projects. It is therefore important that this guard is kept
+ exactly as-is when these definitions are copied.
+
+The ArrowAsyncDeviceStreamHandler structure
+'''''''''''''''''''''''''''''''''''''''''''
+
+The structure has the following fields:
+
+.. c:member:: int (*ArrowAsyncDeviceStreamHandler.on_schema)(struct ArrowAsyncDeviceStreamHandler*, struct ArrowAsyncProducer*, struct ArrowSchema*, const char*)
+
+ *Mandatory.* Handler for receiving the schema of the stream. All records should
+ match the provided schema. If successful, the function should return 0, otherwise
+ it should return an ``errno``-compatible error code.
+
+ The ``const char*`` parameter exists for producers to provide any extra contextual information
+ they want, such as the total number of rows in the stream, statistics, or otherwise. This is
+ encoded in the same format as :c:member:`ArrowSchema.metadata`. If not ``NULL``,
+ the lifetime is only the scope of the call to this function. A consumer who wants to maintain
+ the additional metadata beyond the lifetime of this call *MUST* copy the value themselves.
+
+ Unless the ``on_error`` handler is called, this will always get called exactly once and will be
+ the first method called on this object. As such the producer *MUST* provide an ``ArrowAsyncProducer``
+ object when calling this function to allow the consumer to manage back-pressure and flow control.
+ The producer maintains ownership of the ``ArrowAsyncProducer`` and must clean it up *after*
+ calling the release callback on this object.
+
+ A producer that receives a non-zero result here must not subsequently call anything other than
+ the release callback on this object.
+
+.. c:member:: int (*ArrowAsyncDeviceStreamHandler.on_next_task)(struct ArrowAsyncDeviceStreamHandler*, struct ArrowAsyncTask*, const char*)
+
+ *Mandatory.* Handler to be called when a new record is available for processing. The
+ schema for each record should be the same as the schema that ``on_schema`` was called with.
+ If successfully handled, the function should return 0, otherwise it should return an
+ ``errno``-compatible error code.
+
+ Rather than passing the record itself it receives an ``ArrowAsyncTask`` instead to facilitate
+ better consumer-focused thread control as far as receiving the data. A call to this function
+ simply indicates that data is available via the provided task.
+
+ The producer signals the end of the stream by passing ``NULL`` for the ``ArrowAsyncTask``
+ pointer instead of a valid address. This task object is only valid during the lifetime of
+ this function call. If the consumer wants to use the task beyond the scope of this method, it
+ must copy or move its contents to a new ArrowAsyncTask object.
Review Comment:
`extract_data` it is then! :)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-43631: [C][Format] Add ArrowAsyncDeviceStreamHandler interface [arrow]
Posted by "westonpace (via GitHub)" <gi...@apache.org>.
westonpace commented on code in PR #43632:
URL: https://github.com/apache/arrow/pull/43632#discussion_r1766078825
##########
docs/source/format/CDeviceDataInterface.rst:
##########
@@ -650,6 +652,331 @@ The stream source is not assumed to be thread-safe. Consumers wanting to
call ``get_next`` from several threads should ensure those calls are
serialized.
+Async Device Stream Interface
+=============================
+
+The :ref:`C stream interface <_c-device-stream-interface>` provides a synchronous
+API centered around the consumer calling the callback functions to retrieve
+the next record batch. For some bindings, use cases, and interoperability, a more
+asynchronous, producer-focused interface may be required. These scenarios can utilize
+the ``ArrowAsyncDeviceStreamHandler``.
+
+Semantics
+---------
+
+Rather than the producer providing a structure of callbacks for a consumer to
+call and retrieve records, the Async interface is a consumer allocated structure.
+The consumer allocated struct provides handler callbacks for the producer to call
+when the schema and chunks of data are available, rather than the consumer using
+a blocking pull-style iteration.
+
+In addition to the ``ArrowAsyncDeviceStreamHandler``, there are also two additional
+structs used for the full data flow: ``ArrowAsyncTask`` and ``ArrowAsyncProducer``.
+
+Structure Definition
+--------------------
+
+The C device async stream interface is defined with a single ``struct`` definition:
+
+.. code-block:: c
+
+ #ifndef ARROW_C_ASYNC_STREAM_INTERFACE
+ #define ARROW_C_ASYNC_STREAM_INTERFACE
+
+ struct ArrowAsyncTask {
+ int (*get_data)(struct ArrowArrayTask* self, struct ArrowDeviceArray* out);
+
+ void* private_data;
+ };
+
+ struct ArrowAsyncProducer {
+ void (*request)(struct ArrowAsyncProducer* self, uint64_t n);
+ void (*cancel)(struct ArrowAsyncProducer* self);
+
+ void (*release)(struct ArrowAsyncProducer* self);
+ void* private_data;
+ };
+
+ struct ArrowAsyncDeviceStreamHandler {
+ // handlers
+ int (*on_schema)(struct ArrowAsyncDeviceStreamHandler* self,
+ struct ArrowAsyncProducer* producer,
+ struct ArrowSchema* stream_schema, const char* addl_metadata);
+ int (*on_next_task)(struct ArrowAsyncDeviceStreamHandler* self,
+ struct ArrowAsyncTask* task, const char* metadata);
+ void (*on_error)(struct ArrowAsyncDeviceStreamHandler* self,
+ int code, const char* message, const char* metadata);
+
+ // release callback
+ void (*release)(struct ArrowAsyncDeviceStreamHandler* self);
+
+ // opaque handler-specific data
+ void* private_data;
+ };
+
+ #endif // ARROW_C_ASYNC_STREAM_INTERFACE
+
+.. note::
+ The canonical guard ``ARROW_C_ASYNC_STREAM_INTERFACE`` is meant to avoid
+ duplicate definitions if two projects copy the C async stream interface
+ definitions into their own headers, and a third-party project includes
+ from these two projects. It is therefore important that this guard is kept
+ exactly as-is when these definitions are copied.
+
+The ArrowAsyncDeviceStreamHandler structure
+'''''''''''''''''''''''''''''''''''''''''''
+
+The structure has the following fields:
+
+.. c:member:: int (*ArrowAsyncDeviceStreamHandler.on_schema)(struct ArrowAsyncDeviceStreamHandler*, struct ArrowAsyncProducer*, struct ArrowSchema*, const char*)
+
+ *Mandatory.* Handler for receiving the schema of the stream. All records should
+ match the provided schema. If successful, the function should return 0, otherwise
+ it should return an ``errno``-compatible error code.
+
+ The ``const char*`` parameter exists for producers to provide any extra contextual information
+ they want, such as the total number of rows in the stream, statistics, or otherwise. This is
+ encoded in the same format as :c:member:`ArrowSchema.metadata`. If not ``NULL``,
+ the lifetime is only the scope of the call to this function. A consumer who wants to maintain
+ the additional metadata beyond the lifetime of this call *MUST* copy the value themselves.
+
+ Unless the ``on_error`` handler is called, this will always get called exactly once and will be
+ the first method called on this object. As such the producer *MUST* provide an ``ArrowAsyncProducer``
+ object when calling this function to allow the consumer to manage back-pressure and flow control.
+ The producer maintains ownership of the ``ArrowAsyncProducer`` and must clean it up before or after
+ calling the release callback on this object.
+
+ A producer that receives a non-zero result here must not subsequently call anything other than
+ the release callback on this object.
+
+.. c:member:: int (*ArrowAsyncDeviceStreamHandler.on_next_task)(struct ArrowAsyncDeviceStreamHandler*, struct ArrowAsyncTask*, const char*)
+
+ *Mandatory.* Handler to be called when a new record is available for processing. The
+ schema for each record should be the same as the schema that ``on_schema`` was called with.
+ If successfully handled, the function should return 0, otherwise it should return an
+ ``errno``-compatible error code.
+
+ Rather than passing the record itself it receives an ``ArrowAsyncTask`` instead to facilitate
+ better consumer-focused thread control as far as receiving the data. A call to this function
+ simply indicates that data is available via the provided task.
+
+ The ``const char*`` parameter exists for producers to provide any extra contextual information
+ they want. This is encoded in the same format as :c:member:`ArrowSchema.metadata`. If not ``NULL``,
+ the lifetime is only the scope of the call to this function. A consumer who wants to maintain
+ the additional metadata beyond the lifetime of this call *MUST* copy the value themselves.
+
+ A producer *MUST NOT* call this concurrently from multiple different threads.
+
+ The :c:member:`ArrowAsyncProducer.request` callback must be called to start receiving calls to this
+ handler.
+
+ This function *MUST* be able to be called re-entrantly on the same thread to allow for the
+ :c:member:`ArrowAsyncProducer.request` callback to potentially call this method recursively.
+
+.. c:member:: void (*ArrowAsyncDeviceStreamHandler.on_error)(struct ArrowAsyncDeviceStreamHandler, int, const char*, const char*)
+
+ *Mandatory.* Handler to be called when an error is encountered by the producer. After calling
+ this, the ``release`` callback will be called as the last call on this struct. The parameters
+ are an ``errno``-compatible error code and an optional error message and metadata.
+
+ If the message and metadata are not ``NULL``, their lifetime is only valid during the scope
+ of this call. A consumer who wants to maintain these values past the return of this function
+ *MUST* copy the values themselves.
+
+ If the metadata parameter is not ``NULL``, to provide key-value error metadata, then it should
+ be encoded identically to the way that metadata is encoded in :c:member:`ArrowSchema.metadata`.
+
+ It is valid for this to be called by a producer with or without a preceding call to
+ :c:member:`ArrowAsyncProducer.request`. This callback *MUST NOT* call any methods of an
+ ``ArrowAsyncProducer`` object.
+
+.. c:member:: void (*ArrowAsyncDeviceStreamHandler.release)(struct ArrowAsyncDeviceStreamHandler*)
+
+ *Mandatory.* A pointer to a consumer-provided release callback for the handler.
+
+ It is valid for this to be called by a producer with or without a preceding call to
+ :c:member:`ArrowAsyncProducer.request`. This must not call any methods of an ``ArrowAsyncProducer``
+ object.
+
+.. c:member:: void* ArrowAsyncDeviceStreamHandler.private_data
+
+ *Optional.* An opaque pointer to consumer-provided private data.
+
+ Producers *MUST NOT* process this member. Lifetime of this member is handled by
+ the consumer, and especially by the release callback.
+
+The ArrowAsyncTask structure
+''''''''''''''''''''''''''''
+
+The purpose of using a Task object rather than passing the array directly to the ``on_next``
+callback is to allow for more complex and efficient thread handling. Utilizing a Task
+object allows for a producer to separate the "decoding" logic from the I/O, enabling a
+consumer to avoid transferring data between CPU cores (e.g. from one L1/L2 cache to another).
Review Comment:
Note: all answers obviously biased by my current designs.
> The reason not to do I/O and decoding on the same thread is that frequently we have separate thread pools for each, and we don't want to block for IO on a work thread nor do we want to do anything in an IO thread other than queue IO as immediately as we can.
That is my goal. In my current design the I/O thread pool belongs to the producer and the CPU thread pool belongs to the consumer. Hence the need for the task interface.
> what's the story for how a consumer would pass use_threads=True to a producer?
What does this mean? No I/O threads or no CPU threads?
Re I/O threads:
This is something negotiated outside the protocol.
Re CPU threads:
For the latter, in the "producer does not decode" case then the producer is not using CPU threads and so it's a non-issue.
In the "producer does decode" case then this would be something negotiated outside the protocol (e.g. as part of the call that starts producing).
In my current design the producer is created with a filesystem which has a "preferred parallelism" which is what controls the number of I/O threads. The consumer controls the number of CPU threads by choosing how many tasks to run in parallel.
> Is that out of scope for this protocol
IMO, yes.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-43631: [C][Format] Add ArrowAsyncDeviceStreamHandler interface [arrow]
Posted by "leventov (via GitHub)" <gi...@apache.org>.
leventov commented on PR #43632:
URL: https://github.com/apache/arrow/pull/43632#issuecomment-2341590731
@zeroshade I didn't imply there are incompatibilities, just brought examples of prior art for reference.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-43631: [C][Format] Add ArrowAsyncDeviceStreamHandler interface [arrow]
Posted by "pitrou (via GitHub)" <gi...@apache.org>.
pitrou commented on code in PR #43632:
URL: https://github.com/apache/arrow/pull/43632#discussion_r1757494343
##########
docs/source/format/CDeviceDataInterface.rst:
##########
@@ -650,6 +652,331 @@ The stream source is not assumed to be thread-safe. Consumers wanting to
call ``get_next`` from several threads should ensure those calls are
serialized.
+Async Device Stream Interface
+=============================
+
+The :ref:`C stream interface <_c-device-stream-interface>` provides a synchronous
+API centered around the consumer calling the callback functions to retrieve
+the next record batch. For some bindings, use cases, and interoperability, a more
+asynchronous, producer-focused interface may be required. These scenarios can utilize
+the ``ArrowAsyncDeviceStreamHandler``.
+
+Semantics
+---------
+
+Rather than the producer providing a structure of callbacks for a consumer to
+call and retrieve records, the Async interface is a consumer allocated structure.
+The consumer allocated struct provides handler callbacks for the producer to call
+when the schema and chunks of data are available, rather than the consumer using
+a blocking pull-style iteration.
+
+In addition to the ``ArrowAsyncDeviceStreamHandler``, there are also two additional
+structs used for the full data flow: ``ArrowAsyncTask`` and ``ArrowAsyncProducer``.
+
+Structure Definition
+--------------------
+
+The C device async stream interface is defined with a single ``struct`` definition:
+
+.. code-block:: c
+
+ #ifndef ARROW_C_ASYNC_STREAM_INTERFACE
+ #define ARROW_C_ASYNC_STREAM_INTERFACE
+
+ struct ArrowAsyncTask {
+ int (*get_data)(struct ArrowArrayTask* self, struct ArrowDeviceArray* out);
+
+ void* private_data;
+ };
+
+ struct ArrowAsyncProducer {
+ void (*request)(struct ArrowAsyncProducer* self, uint64_t n);
+ void (*cancel)(struct ArrowAsyncProducer* self);
+
+ void (*release)(struct ArrowAsyncProducer* self);
+ void* private_data;
+ };
+
+ struct ArrowAsyncDeviceStreamHandler {
+ // handlers
+ int (*on_schema)(struct ArrowAsyncDeviceStreamHandler* self,
+ struct ArrowAsyncProducer* producer,
+ struct ArrowSchema* stream_schema, const char* addl_metadata);
+ int (*on_next_task)(struct ArrowAsyncDeviceStreamHandler* self,
+ struct ArrowAsyncTask* task, const char* metadata);
+ void (*on_error)(struct ArrowAsyncDeviceStreamHandler* self,
+ int code, const char* message, const char* metadata);
+
+ // release callback
+ void (*release)(struct ArrowAsyncDeviceStreamHandler* self);
+
+ // opaque handler-specific data
+ void* private_data;
+ };
+
+ #endif // ARROW_C_ASYNC_STREAM_INTERFACE
+
+.. note::
+ The canonical guard ``ARROW_C_ASYNC_STREAM_INTERFACE`` is meant to avoid
+ duplicate definitions if two projects copy the C async stream interface
+ definitions into their own headers, and a third-party project includes
+ from these two projects. It is therefore important that this guard is kept
+ exactly as-is when these definitions are copied.
+
+The ArrowAsyncDeviceStreamHandler structure
+'''''''''''''''''''''''''''''''''''''''''''
+
+The structure has the following fields:
+
+.. c:member:: int (*ArrowAsyncDeviceStreamHandler.on_schema)(struct ArrowAsyncDeviceStreamHandler*, struct ArrowAsyncProducer*, struct ArrowSchema*, const char*)
+
+ *Mandatory.* Handler for receiving the schema of the stream. All records should
+ match the provided schema. If successful, the function should return 0, otherwise
+ it should return an ``errno``-compatible error code.
+
+ The ``const char*`` parameter exists for producers to provide any extra contextual information
+ they want, such as the total number of rows in the stream, statistics, or otherwise. This is
+ encoded in the same format as :c:member:`ArrowSchema.metadata`. If not ``NULL``,
+ the lifetime is only the scope of the call to this function. A consumer who wants to maintain
+ the additional metadata beyond the lifetime of this call *MUST* copy the value themselves.
+
+ Unless the ``on_error`` handler is called, this will always get called exactly once and will be
+ the first method called on this object. As such the producer *MUST* provide an ``ArrowAsyncProducer``
+ object when calling this function to allow the consumer to manage back-pressure and flow control.
+ The producer maintains ownership of the ``ArrowAsyncProducer`` and must clean it up before or after
+ calling the release callback on this object.
+
+ A producer that receives a non-zero result here must not subsequently call anything other than
+ the release callback on this object.
+
+.. c:member:: int (*ArrowAsyncDeviceStreamHandler.on_next_task)(struct ArrowAsyncDeviceStreamHandler*, struct ArrowAsyncTask*, const char*)
+
+ *Mandatory.* Handler to be called when a new record is available for processing. The
+ schema for each record should be the same as the schema that ``on_schema`` was called with.
+ If successfully handled, the function should return 0, otherwise it should return an
+ ``errno``-compatible error code.
+
+ Rather than passing the record itself it receives an ``ArrowAsyncTask`` instead to facilitate
+ better consumer-focused thread control as far as receiving the data. A call to this function
+ simply indicates that data is available via the provided task.
+
+ The ``const char*`` parameter exists for producers to provide any extra contextual information
+ they want. This is encoded in the same format as :c:member:`ArrowSchema.metadata`. If not ``NULL``,
+ the lifetime is only the scope of the call to this function. A consumer who wants to maintain
+ the additional metadata beyond the lifetime of this call *MUST* copy the value themselves.
+
+ A producer *MUST NOT* call this concurrently from multiple different threads.
+
+ The :c:member:`ArrowAsyncProducer.request` callback must be called to start receiving calls to this
+ handler.
+
+ This function *MUST* be able to be called re-entrantly on the same thread to allow for the
+ :c:member:`ArrowAsyncProducer.request` callback to potentially call this method recursively.
+
+.. c:member:: void (*ArrowAsyncDeviceStreamHandler.on_error)(struct ArrowAsyncDeviceStreamHandler, int, const char*, const char*)
+
+ *Mandatory.* Handler to be called when an error is encountered by the producer. After calling
+ this, the ``release`` callback will be called as the last call on this struct. The parameters
+ are an ``errno``-compatible error code and an optional error message and metadata.
+
+ If the message and metadata are not ``NULL``, their lifetime is only valid during the scope
+ of this call. A consumer who wants to maintain these values past the return of this function
+ *MUST* copy the values themselves.
+
+ If the metadata parameter is not ``NULL``, to provide key-value error metadata, then it should
+ be encoded identically to the way that metadata is encoded in :c:member:`ArrowSchema.metadata`.
+
+ It is valid for this to be called by a producer with or without a preceding call to
+ :c:member:`ArrowAsyncProducer.request`. This callback *MUST NOT* call any methods of an
+ ``ArrowAsyncProducer`` object.
+
+.. c:member:: void (*ArrowAsyncDeviceStreamHandler.release)(struct ArrowAsyncDeviceStreamHandler*)
+
+ *Mandatory.* A pointer to a consumer-provided release callback for the handler.
+
+ It is valid for this to be called by a producer with or without a preceding call to
+ :c:member:`ArrowAsyncProducer.request`. This must not call any methods of an ``ArrowAsyncProducer``
+ object.
+
+.. c:member:: void* ArrowAsyncDeviceStreamHandler.private_data
+
+ *Optional.* An opaque pointer to consumer-provided private data.
+
+ Producers *MUST NOT* process this member. Lifetime of this member is handled by
+ the consumer, and especially by the release callback.
+
+The ArrowAsyncTask structure
+''''''''''''''''''''''''''''
+
+The purpose of using a Task object rather than passing the array directly to the ``on_next``
+callback is to allow for more complex and efficient thread handling. Utilizing a Task
+object allows for a producer to separate the "decoding" logic from the I/O, enabling a
+consumer to avoid transferring data between CPU cores (e.g. from one L1/L2 cache to another).
+
+This producer-provided structure has the following fields:
+
+.. c:member:: int (*ArrowArrayTask.get_data)(struct ArrowArrayTask*, struct ArrowDeviceArray*)
+
+ *Mandatory.* A callback to populate the provided ``ArrowDeviceArray`` with the available data.
+ The order of ArrowAsyncTasks provided by the producer enables a consumer to know the order of
+ the data to process.
+
+ If a non-zero value is returned from this, it should be followed only by the producer calling
+ the ``on_error`` callback of the ``ArrowAsyncDeviceStreamHandler``. Because calling this method
+ is likely to be separate from the current control flow, returning a non-zero value to signal
+ an error occuring allows the current thread to decide handle the case accordingly, while still
+ allowing all error logging and handling to be centralized in the
+ :c:member:`ArrowAsyncDeviceStreamHandler.on_error` callback.
+
+ Rather than having a separate release callback, any required cleanup should be performed as part
+ of the invocation of this callback, with ownership of the Array being passed to the consumer-provided
+ parameter calling this, and must be released separately.
+
+ It is only valid to call this method exactly once.
+
+.. c:member:: void* ArrowArrayTask.private_data
+
+ *Optional.* An opaque pointer to consumer-provided private data.
+
+ Consumers *MUST NOT* process this member. Lifetime of this member is handled by
+ the producer who created this object, and should be cleaned up if necessary during
+ the call to :c:member:`ArrowArrayTask.get_data`.
+
+The ArrowAsyncProducer structure
+''''''''''''''''''''''''''''''''
+
+This producer-provided and managed object has the following fields:
+
+.. c:member:: void (*ArrowAsyncProducer.request)(struct ArrowAsyncProducer*, uint64_t)
+
+ *Mandatory.* This function must be called by a consumer to start receiving calls to
+ :c:member:`ArrowAsyncDeviceStreamHandler.on_next_task`. It *MUST* be valid to call
+ this synchronously from within :c:member:`ArrowAsyncDeviceStreamHandler.on_next_task`
+ or :c:member:`ArrowAsyncDeviceStreamHandler.on_schema`. As a result, this function
+ *MUST NOT* synchronously call ``on_next_task`` or ``on_error`` to avoid recursive
+ and reentrant callbacks.
+
+ After ``cancel`` is called, additional calls to this function must be a NOP, but allowed.
+
+ While not cancelled, calling this function registers the given number of additional
+ arrays/batches to be produced by the producer. A producer should only call
+ the appropriate ``on_next_task`` callback up to a maximum of the total sum of calls to
+ this method before propagating back-pressure / waiting.
+
+ Any error encountered by calling request must be propagated by calling the ``on_error``
+ callback of the ``ArrowAsyncDeviceStreamHandler``.
+
+ A producer *MUST* support an unbounded number of calls to this callback and *MUST* support
+ a total registered demand (sum requested - sum delivered) of up to ``UINT64_MAX``.
+
+.. c:member:: void (*ArrowAsyncProducer.cancel)(struct ArrowAsyncProducer*)
+
+ *Mandatory.* This function signals to the producer that it must *eventually* stop calling
+ ``on_next_task``. Calls to ``cancel`` must be idempotent and thread-safe. After calling
+ it once, subsequent calls *MUST* be a NOP.
+
+ It is not required that calling ``cancel`` affect the producer *immediately*, only that it
+ must eventually stop calling ``on_next_task`` and then subsequently call ``release``
+ on the async handler object. As such, a consumer *MUST* be prepared to receive one or more
+ calls to ``on_next_task`` or ``on_error`` even after calling ``cancel`` if there are still
+ requested arrays pending.
+
+ Successful cancelling *MUST NOT* result in a producer calling
+ :c:member:`ArrowAsyncDeviceStreamHandler.on_error`, instead it should finish out any remaining
+ tasks (calling ``on_next_task`` accordingly) and eventually just call ``release``.
+
+ Any error encountered during handling a call to cancel must be reported via the ``on_error``
+ callback on the async stream handler.
+
+.. c:member:: void* ArrowAsyncProducer.private_data
+
+ *Optional.* An opaque pointer to producer-provided specific data.
+
+ Consumers *MUST NOT* process this member, the lifetime is owned by the producer
+ that constructed this object.
+
+Error Handling
+''''''''''''''
+
+Unlike the regular C Stream interface, the Async interface allows for errors to flow in
+both directions. As a result, error handling can be slightly more complex. Thus this spec
+designates the following rules:
+
+* If the producer encounters an error during processing, it should call the ``on_error``
+callback, and then call ``release`` after it returns.
+
+* If ``on_schema`` or ``on_next_task`` returns a non-zero integer value, the producer *should not*
+call the ``on_error`` callback, but instead should eventually call ``release`` at some point
+before or after any logging or processing of the error code.
+
+Result lifetimes
+''''''''''''''''
+
+The ``ArrowSchema`` passed to the ``on_schema`` callback must be released independantly,
+with the object itself needing to be moved to a consumer owned ``ArrowSchema`` object. The
+``ArrowSchema*`` passed as a parameter to the callback *MUST NOT* be stored and kept.
+
+The ``ArrowAsyncTask`` object provided to ``on_next_task`` is owned by the producer and
+will be cleaned up during the invocation of calling ``get_data`` on it. If the consumer
+doesn't care about the data, it should pass ``NULL`` instead of a valid ``ArrowDeviceArray*``.
+
+The ``const char*`` error ``message`` and ``metadata`` which are passed to ``on_error``
+are only valid within the scope of the ``on_error`` function itself. They must be copied
+if it is necessary for them to exist after it returns.
+
+Stream Handler Lifetime
+'''''''''''''''''''''''
+
+Lifetime of the async stream handler is managed using a release callback with similar
+usage as in :ref:`C data interface <c-data-interface-released>`.
+
+ArrowAsyncProducer Lifetime
+'''''''''''''''''''''''''''
+
+The lifetime of the ``ArrowAsyncProducer`` passed to ``on_schema`` is owned by the producer
+itself and should be managed by it. It *MUST* remain valid at least until just before
+calling ``release`` on the stream handler object.
+
+Thread safety
+'''''''''''''
+
+All handler functions should only be called in a serialized manner, but are not guaranteed
+to be called from the same thread every time. A producer should wait for handler callbacks to
+return before calling the next handler callback, and before calling the ``release`` callback.
Review Comment:
But the `on_next_task` doc above says it should be reentrant.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-43631: [C][Format] Add ArrowAsyncDeviceStreamHandler interface [arrow]
Posted by "zeroshade (via GitHub)" <gi...@apache.org>.
zeroshade commented on code in PR #43632:
URL: https://github.com/apache/arrow/pull/43632#discussion_r1765341344
##########
docs/source/format/CDeviceDataInterface.rst:
##########
@@ -650,6 +652,331 @@ The stream source is not assumed to be thread-safe. Consumers wanting to
call ``get_next`` from several threads should ensure those calls are
serialized.
+Async Device Stream Interface
+=============================
+
+The :ref:`C stream interface <_c-device-stream-interface>` provides a synchronous
+API centered around the consumer calling the callback functions to retrieve
+the next record batch. For some bindings, use cases, and interoperability, a more
+asynchronous, producer-focused interface may be required. These scenarios can utilize
+the ``ArrowAsyncDeviceStreamHandler``.
+
+Semantics
+---------
+
+Rather than the producer providing a structure of callbacks for a consumer to
+call and retrieve records, the Async interface is a consumer allocated structure.
+The consumer allocated struct provides handler callbacks for the producer to call
+when the schema and chunks of data are available, rather than the consumer using
+a blocking pull-style iteration.
+
+In addition to the ``ArrowAsyncDeviceStreamHandler``, there are also two additional
+structs used for the full data flow: ``ArrowAsyncTask`` and ``ArrowAsyncProducer``.
+
+Structure Definition
+--------------------
+
+The C device async stream interface is defined with a single ``struct`` definition:
+
+.. code-block:: c
+
+ #ifndef ARROW_C_ASYNC_STREAM_INTERFACE
+ #define ARROW_C_ASYNC_STREAM_INTERFACE
+
+ struct ArrowAsyncTask {
+ int (*get_data)(struct ArrowArrayTask* self, struct ArrowDeviceArray* out);
+
+ void* private_data;
+ };
+
+ struct ArrowAsyncProducer {
+ void (*request)(struct ArrowAsyncProducer* self, uint64_t n);
+ void (*cancel)(struct ArrowAsyncProducer* self);
+
+ void (*release)(struct ArrowAsyncProducer* self);
+ void* private_data;
+ };
+
+ struct ArrowAsyncDeviceStreamHandler {
+ // handlers
+ int (*on_schema)(struct ArrowAsyncDeviceStreamHandler* self,
+ struct ArrowAsyncProducer* producer,
+ struct ArrowSchema* stream_schema, const char* addl_metadata);
+ int (*on_next_task)(struct ArrowAsyncDeviceStreamHandler* self,
+ struct ArrowAsyncTask* task, const char* metadata);
+ void (*on_error)(struct ArrowAsyncDeviceStreamHandler* self,
+ int code, const char* message, const char* metadata);
+
+ // release callback
+ void (*release)(struct ArrowAsyncDeviceStreamHandler* self);
+
+ // opaque handler-specific data
+ void* private_data;
+ };
+
+ #endif // ARROW_C_ASYNC_STREAM_INTERFACE
+
+.. note::
+ The canonical guard ``ARROW_C_ASYNC_STREAM_INTERFACE`` is meant to avoid
+ duplicate definitions if two projects copy the C async stream interface
+ definitions into their own headers, and a third-party project includes
+ from these two projects. It is therefore important that this guard is kept
+ exactly as-is when these definitions are copied.
+
+The ArrowAsyncDeviceStreamHandler structure
+'''''''''''''''''''''''''''''''''''''''''''
+
+The structure has the following fields:
+
+.. c:member:: int (*ArrowAsyncDeviceStreamHandler.on_schema)(struct ArrowAsyncDeviceStreamHandler*, struct ArrowAsyncProducer*, struct ArrowSchema*, const char*)
+
+ *Mandatory.* Handler for receiving the schema of the stream. All records should
+ match the provided schema. If successful, the function should return 0, otherwise
+ it should return an ``errno``-compatible error code.
+
+ The ``const char*`` parameter exists for producers to provide any extra contextual information
+ they want, such as the total number of rows in the stream, statistics, or otherwise. This is
+ encoded in the same format as :c:member:`ArrowSchema.metadata`. If not ``NULL``,
+ the lifetime is only the scope of the call to this function. A consumer who wants to maintain
+ the additional metadata beyond the lifetime of this call *MUST* copy the value themselves.
+
+ Unless the ``on_error`` handler is called, this will always get called exactly once and will be
+ the first method called on this object. As such the producer *MUST* provide an ``ArrowAsyncProducer``
+ object when calling this function to allow the consumer to manage back-pressure and flow control.
+ The producer maintains ownership of the ``ArrowAsyncProducer`` and must clean it up before or after
+ calling the release callback on this object.
+
+ A producer that receives a non-zero result here must not subsequently call anything other than
+ the release callback on this object.
+
+.. c:member:: int (*ArrowAsyncDeviceStreamHandler.on_next_task)(struct ArrowAsyncDeviceStreamHandler*, struct ArrowAsyncTask*, const char*)
+
+ *Mandatory.* Handler to be called when a new record is available for processing. The
+ schema for each record should be the same as the schema that ``on_schema`` was called with.
+ If successfully handled, the function should return 0, otherwise it should return an
+ ``errno``-compatible error code.
+
+ Rather than passing the record itself it receives an ``ArrowAsyncTask`` instead to facilitate
+ better consumer-focused thread control as far as receiving the data. A call to this function
+ simply indicates that data is available via the provided task.
+
+ The ``const char*`` parameter exists for producers to provide any extra contextual information
+ they want. This is encoded in the same format as :c:member:`ArrowSchema.metadata`. If not ``NULL``,
+ the lifetime is only the scope of the call to this function. A consumer who wants to maintain
+ the additional metadata beyond the lifetime of this call *MUST* copy the value themselves.
+
+ A producer *MUST NOT* call this concurrently from multiple different threads.
+
+ The :c:member:`ArrowAsyncProducer.request` callback must be called to start receiving calls to this
+ handler.
+
+ This function *MUST* be able to be called re-entrantly on the same thread to allow for the
+ :c:member:`ArrowAsyncProducer.request` callback to potentially call this method recursively.
+
+.. c:member:: void (*ArrowAsyncDeviceStreamHandler.on_error)(struct ArrowAsyncDeviceStreamHandler, int, const char*, const char*)
+
+ *Mandatory.* Handler to be called when an error is encountered by the producer. After calling
+ this, the ``release`` callback will be called as the last call on this struct. The parameters
+ are an ``errno``-compatible error code and an optional error message and metadata.
+
+ If the message and metadata are not ``NULL``, their lifetime is only valid during the scope
+ of this call. A consumer who wants to maintain these values past the return of this function
+ *MUST* copy the values themselves.
+
+ If the metadata parameter is not ``NULL``, to provide key-value error metadata, then it should
+ be encoded identically to the way that metadata is encoded in :c:member:`ArrowSchema.metadata`.
+
+ It is valid for this to be called by a producer with or without a preceding call to
+ :c:member:`ArrowAsyncProducer.request`. This callback *MUST NOT* call any methods of an
+ ``ArrowAsyncProducer`` object.
+
+.. c:member:: void (*ArrowAsyncDeviceStreamHandler.release)(struct ArrowAsyncDeviceStreamHandler*)
+
+ *Mandatory.* A pointer to a consumer-provided release callback for the handler.
+
+ It is valid for this to be called by a producer with or without a preceding call to
+ :c:member:`ArrowAsyncProducer.request`. This must not call any methods of an ``ArrowAsyncProducer``
+ object.
+
+.. c:member:: void* ArrowAsyncDeviceStreamHandler.private_data
+
+ *Optional.* An opaque pointer to consumer-provided private data.
+
+ Producers *MUST NOT* process this member. Lifetime of this member is handled by
+ the consumer, and especially by the release callback.
+
+The ArrowAsyncTask structure
+''''''''''''''''''''''''''''
+
+The purpose of using a Task object rather than passing the array directly to the ``on_next``
+callback is to allow for more complex and efficient thread handling. Utilizing a Task
+object allows for a producer to separate the "decoding" logic from the I/O, enabling a
+consumer to avoid transferring data between CPU cores (e.g. from one L1/L2 cache to another).
+
+This producer-provided structure has the following fields:
+
+.. c:member:: int (*ArrowArrayTask.get_data)(struct ArrowArrayTask*, struct ArrowDeviceArray*)
+
+ *Mandatory.* A callback to populate the provided ``ArrowDeviceArray`` with the available data.
+ The order of ArrowAsyncTasks provided by the producer enables a consumer to know the order of
+ the data to process.
+
+ If a non-zero value is returned from this, it should be followed only by the producer calling
+ the ``on_error`` callback of the ``ArrowAsyncDeviceStreamHandler``. Because calling this method
+ is likely to be separate from the current control flow, returning a non-zero value to signal
+ an error occuring allows the current thread to decide handle the case accordingly, while still
+ allowing all error logging and handling to be centralized in the
+ :c:member:`ArrowAsyncDeviceStreamHandler.on_error` callback.
+
+ Rather than having a separate release callback, any required cleanup should be performed as part
+ of the invocation of this callback, with ownership of the Array being passed to the consumer-provided
+ parameter calling this, and must be released separately.
+
+ It is only valid to call this method exactly once.
+
+.. c:member:: void* ArrowArrayTask.private_data
+
+ *Optional.* An opaque pointer to consumer-provided private data.
+
+ Consumers *MUST NOT* process this member. Lifetime of this member is handled by
+ the producer who created this object, and should be cleaned up if necessary during
+ the call to :c:member:`ArrowArrayTask.get_data`.
+
+The ArrowAsyncProducer structure
+''''''''''''''''''''''''''''''''
+
+This producer-provided and managed object has the following fields:
+
+.. c:member:: void (*ArrowAsyncProducer.request)(struct ArrowAsyncProducer*, uint64_t)
+
+ *Mandatory.* This function must be called by a consumer to start receiving calls to
+ :c:member:`ArrowAsyncDeviceStreamHandler.on_next_task`. It *MUST* be valid to call
+ this synchronously from within :c:member:`ArrowAsyncDeviceStreamHandler.on_next_task`
+ or :c:member:`ArrowAsyncDeviceStreamHandler.on_schema`. As a result, this function
+ *MUST NOT* synchronously call ``on_next_task`` or ``on_error`` to avoid recursive
+ and reentrant callbacks.
+
+ After ``cancel`` is called, additional calls to this function must be a NOP, but allowed.
+
+ While not cancelled, calling this function registers the given number of additional
+ arrays/batches to be produced by the producer. A producer should only call
+ the appropriate ``on_next_task`` callback up to a maximum of the total sum of calls to
+ this method before propagating back-pressure / waiting.
+
+ Any error encountered by calling request must be propagated by calling the ``on_error``
+ callback of the ``ArrowAsyncDeviceStreamHandler``.
+
+ A producer *MUST* support an unbounded number of calls to this callback and *MUST* support
Review Comment:
I'm fine with removing this requirement, at a worst case it can be added back without an ABI change if desired.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-43631: [C][Format] Add ArrowAsyncDeviceStreamHandler interface [arrow]
Posted by "pitrou (via GitHub)" <gi...@apache.org>.
pitrou commented on code in PR #43632:
URL: https://github.com/apache/arrow/pull/43632#discussion_r1765351052
##########
docs/source/format/CDeviceDataInterface.rst:
##########
@@ -650,6 +652,331 @@ The stream source is not assumed to be thread-safe. Consumers wanting to
call ``get_next`` from several threads should ensure those calls are
serialized.
+Async Device Stream Interface
+=============================
+
+The :ref:`C stream interface <_c-device-stream-interface>` provides a synchronous
+API centered around the consumer calling the callback functions to retrieve
+the next record batch. For some bindings, use cases, and interoperability, a more
+asynchronous, producer-focused interface may be required. These scenarios can utilize
+the ``ArrowAsyncDeviceStreamHandler``.
+
+Semantics
+---------
+
+Rather than the producer providing a structure of callbacks for a consumer to
+call and retrieve records, the Async interface is a consumer allocated structure.
+The consumer allocated struct provides handler callbacks for the producer to call
+when the schema and chunks of data are available, rather than the consumer using
+a blocking pull-style iteration.
+
+In addition to the ``ArrowAsyncDeviceStreamHandler``, there are also two additional
+structs used for the full data flow: ``ArrowAsyncTask`` and ``ArrowAsyncProducer``.
+
+Structure Definition
+--------------------
+
+The C device async stream interface is defined with a single ``struct`` definition:
+
+.. code-block:: c
+
+ #ifndef ARROW_C_ASYNC_STREAM_INTERFACE
+ #define ARROW_C_ASYNC_STREAM_INTERFACE
+
+ struct ArrowAsyncTask {
+ int (*get_data)(struct ArrowArrayTask* self, struct ArrowDeviceArray* out);
+
+ void* private_data;
+ };
+
+ struct ArrowAsyncProducer {
+ void (*request)(struct ArrowAsyncProducer* self, uint64_t n);
+ void (*cancel)(struct ArrowAsyncProducer* self);
+
+ void (*release)(struct ArrowAsyncProducer* self);
+ void* private_data;
+ };
+
+ struct ArrowAsyncDeviceStreamHandler {
+ // handlers
+ int (*on_schema)(struct ArrowAsyncDeviceStreamHandler* self,
+ struct ArrowAsyncProducer* producer,
+ struct ArrowSchema* stream_schema, const char* addl_metadata);
+ int (*on_next_task)(struct ArrowAsyncDeviceStreamHandler* self,
+ struct ArrowAsyncTask* task, const char* metadata);
+ void (*on_error)(struct ArrowAsyncDeviceStreamHandler* self,
+ int code, const char* message, const char* metadata);
+
+ // release callback
+ void (*release)(struct ArrowAsyncDeviceStreamHandler* self);
+
+ // opaque handler-specific data
+ void* private_data;
+ };
+
+ #endif // ARROW_C_ASYNC_STREAM_INTERFACE
+
+.. note::
+ The canonical guard ``ARROW_C_ASYNC_STREAM_INTERFACE`` is meant to avoid
+ duplicate definitions if two projects copy the C async stream interface
+ definitions into their own headers, and a third-party project includes
+ from these two projects. It is therefore important that this guard is kept
+ exactly as-is when these definitions are copied.
+
+The ArrowAsyncDeviceStreamHandler structure
+'''''''''''''''''''''''''''''''''''''''''''
+
+The structure has the following fields:
+
+.. c:member:: int (*ArrowAsyncDeviceStreamHandler.on_schema)(struct ArrowAsyncDeviceStreamHandler*, struct ArrowAsyncProducer*, struct ArrowSchema*, const char*)
+
+ *Mandatory.* Handler for receiving the schema of the stream. All records should
+ match the provided schema. If successful, the function should return 0, otherwise
+ it should return an ``errno``-compatible error code.
+
+ The ``const char*`` parameter exists for producers to provide any extra contextual information
+ they want, such as the total number of rows in the stream, statistics, or otherwise. This is
+ encoded in the same format as :c:member:`ArrowSchema.metadata`. If not ``NULL``,
+ the lifetime is only the scope of the call to this function. A consumer who wants to maintain
+ the additional metadata beyond the lifetime of this call *MUST* copy the value themselves.
+
+ Unless the ``on_error`` handler is called, this will always get called exactly once and will be
+ the first method called on this object. As such the producer *MUST* provide an ``ArrowAsyncProducer``
+ object when calling this function to allow the consumer to manage back-pressure and flow control.
+ The producer maintains ownership of the ``ArrowAsyncProducer`` and must clean it up before or after
+ calling the release callback on this object.
+
+ A producer that receives a non-zero result here must not subsequently call anything other than
+ the release callback on this object.
+
+.. c:member:: int (*ArrowAsyncDeviceStreamHandler.on_next_task)(struct ArrowAsyncDeviceStreamHandler*, struct ArrowAsyncTask*, const char*)
+
+ *Mandatory.* Handler to be called when a new record is available for processing. The
+ schema for each record should be the same as the schema that ``on_schema`` was called with.
+ If successfully handled, the function should return 0, otherwise it should return an
+ ``errno``-compatible error code.
+
+ Rather than passing the record itself it receives an ``ArrowAsyncTask`` instead to facilitate
+ better consumer-focused thread control as far as receiving the data. A call to this function
+ simply indicates that data is available via the provided task.
+
+ The ``const char*`` parameter exists for producers to provide any extra contextual information
+ they want. This is encoded in the same format as :c:member:`ArrowSchema.metadata`. If not ``NULL``,
+ the lifetime is only the scope of the call to this function. A consumer who wants to maintain
+ the additional metadata beyond the lifetime of this call *MUST* copy the value themselves.
+
+ A producer *MUST NOT* call this concurrently from multiple different threads.
+
+ The :c:member:`ArrowAsyncProducer.request` callback must be called to start receiving calls to this
+ handler.
+
+ This function *MUST* be able to be called re-entrantly on the same thread to allow for the
+ :c:member:`ArrowAsyncProducer.request` callback to potentially call this method recursively.
+
+.. c:member:: void (*ArrowAsyncDeviceStreamHandler.on_error)(struct ArrowAsyncDeviceStreamHandler, int, const char*, const char*)
+
+ *Mandatory.* Handler to be called when an error is encountered by the producer. After calling
+ this, the ``release`` callback will be called as the last call on this struct. The parameters
+ are an ``errno``-compatible error code and an optional error message and metadata.
+
+ If the message and metadata are not ``NULL``, their lifetime is only valid during the scope
+ of this call. A consumer who wants to maintain these values past the return of this function
+ *MUST* copy the values themselves.
+
+ If the metadata parameter is not ``NULL``, to provide key-value error metadata, then it should
+ be encoded identically to the way that metadata is encoded in :c:member:`ArrowSchema.metadata`.
+
+ It is valid for this to be called by a producer with or without a preceding call to
+ :c:member:`ArrowAsyncProducer.request`. This callback *MUST NOT* call any methods of an
+ ``ArrowAsyncProducer`` object.
+
+.. c:member:: void (*ArrowAsyncDeviceStreamHandler.release)(struct ArrowAsyncDeviceStreamHandler*)
+
+ *Mandatory.* A pointer to a consumer-provided release callback for the handler.
+
+ It is valid for this to be called by a producer with or without a preceding call to
+ :c:member:`ArrowAsyncProducer.request`. This must not call any methods of an ``ArrowAsyncProducer``
+ object.
+
+.. c:member:: void* ArrowAsyncDeviceStreamHandler.private_data
+
+ *Optional.* An opaque pointer to consumer-provided private data.
+
+ Producers *MUST NOT* process this member. Lifetime of this member is handled by
+ the consumer, and especially by the release callback.
+
+The ArrowAsyncTask structure
+''''''''''''''''''''''''''''
+
+The purpose of using a Task object rather than passing the array directly to the ``on_next``
+callback is to allow for more complex and efficient thread handling. Utilizing a Task
+object allows for a producer to separate the "decoding" logic from the I/O, enabling a
+consumer to avoid transferring data between CPU cores (e.g. from one L1/L2 cache to another).
Review Comment:
As an existing real-world example, if the producer is reading from an IPC file, for each new RecordBatch it is able to decompress its buffers on multiple threads, which is good for latency.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-43631: [C][Format] Add ArrowAsyncDeviceStreamHandler interface [arrow]
Posted by "westonpace (via GitHub)" <gi...@apache.org>.
westonpace commented on code in PR #43632:
URL: https://github.com/apache/arrow/pull/43632#discussion_r1765851427
##########
docs/source/format/CDeviceDataInterface.rst:
##########
@@ -650,6 +652,331 @@ The stream source is not assumed to be thread-safe. Consumers wanting to
call ``get_next`` from several threads should ensure those calls are
serialized.
+Async Device Stream Interface
+=============================
+
+The :ref:`C stream interface <_c-device-stream-interface>` provides a synchronous
+API centered around the consumer calling the callback functions to retrieve
+the next record batch. For some bindings, use cases, and interoperability, a more
+asynchronous, producer-focused interface may be required. These scenarios can utilize
+the ``ArrowAsyncDeviceStreamHandler``.
+
+Semantics
+---------
+
+Rather than the producer providing a structure of callbacks for a consumer to
+call and retrieve records, the Async interface is a consumer allocated structure.
+The consumer allocated struct provides handler callbacks for the producer to call
+when the schema and chunks of data are available, rather than the consumer using
+a blocking pull-style iteration.
+
+In addition to the ``ArrowAsyncDeviceStreamHandler``, there are also two additional
+structs used for the full data flow: ``ArrowAsyncTask`` and ``ArrowAsyncProducer``.
+
+Structure Definition
+--------------------
+
+The C device async stream interface is defined with a single ``struct`` definition:
+
+.. code-block:: c
+
+ #ifndef ARROW_C_ASYNC_STREAM_INTERFACE
+ #define ARROW_C_ASYNC_STREAM_INTERFACE
+
+ struct ArrowAsyncTask {
+ int (*get_data)(struct ArrowArrayTask* self, struct ArrowDeviceArray* out);
+
+ void* private_data;
+ };
+
+ struct ArrowAsyncProducer {
+ void (*request)(struct ArrowAsyncProducer* self, uint64_t n);
+ void (*cancel)(struct ArrowAsyncProducer* self);
+
+ void (*release)(struct ArrowAsyncProducer* self);
+ void* private_data;
+ };
+
+ struct ArrowAsyncDeviceStreamHandler {
+ // handlers
+ int (*on_schema)(struct ArrowAsyncDeviceStreamHandler* self,
+ struct ArrowAsyncProducer* producer,
+ struct ArrowSchema* stream_schema, const char* addl_metadata);
+ int (*on_next_task)(struct ArrowAsyncDeviceStreamHandler* self,
+ struct ArrowAsyncTask* task, const char* metadata);
+ void (*on_error)(struct ArrowAsyncDeviceStreamHandler* self,
+ int code, const char* message, const char* metadata);
+
+ // release callback
+ void (*release)(struct ArrowAsyncDeviceStreamHandler* self);
+
+ // opaque handler-specific data
+ void* private_data;
+ };
+
+ #endif // ARROW_C_ASYNC_STREAM_INTERFACE
+
+.. note::
+ The canonical guard ``ARROW_C_ASYNC_STREAM_INTERFACE`` is meant to avoid
+ duplicate definitions if two projects copy the C async stream interface
+ definitions into their own headers, and a third-party project includes
+ from these two projects. It is therefore important that this guard is kept
+ exactly as-is when these definitions are copied.
+
+The ArrowAsyncDeviceStreamHandler structure
+'''''''''''''''''''''''''''''''''''''''''''
+
+The structure has the following fields:
+
+.. c:member:: int (*ArrowAsyncDeviceStreamHandler.on_schema)(struct ArrowAsyncDeviceStreamHandler*, struct ArrowAsyncProducer*, struct ArrowSchema*, const char*)
+
+ *Mandatory.* Handler for receiving the schema of the stream. All records should
+ match the provided schema. If successful, the function should return 0, otherwise
+ it should return an ``errno``-compatible error code.
+
+ The ``const char*`` parameter exists for producers to provide any extra contextual information
+ they want, such as the total number of rows in the stream, statistics, or otherwise. This is
+ encoded in the same format as :c:member:`ArrowSchema.metadata`. If not ``NULL``,
+ the lifetime is only the scope of the call to this function. A consumer who wants to maintain
+ the additional metadata beyond the lifetime of this call *MUST* copy the value themselves.
+
+ Unless the ``on_error`` handler is called, this will always get called exactly once and will be
+ the first method called on this object. As such the producer *MUST* provide an ``ArrowAsyncProducer``
+ object when calling this function to allow the consumer to manage back-pressure and flow control.
+ The producer maintains ownership of the ``ArrowAsyncProducer`` and must clean it up before or after
+ calling the release callback on this object.
+
+ A producer that receives a non-zero result here must not subsequently call anything other than
+ the release callback on this object.
+
+.. c:member:: int (*ArrowAsyncDeviceStreamHandler.on_next_task)(struct ArrowAsyncDeviceStreamHandler*, struct ArrowAsyncTask*, const char*)
+
+ *Mandatory.* Handler to be called when a new record is available for processing. The
+ schema for each record should be the same as the schema that ``on_schema`` was called with.
+ If successfully handled, the function should return 0, otherwise it should return an
+ ``errno``-compatible error code.
+
+ Rather than passing the record itself it receives an ``ArrowAsyncTask`` instead to facilitate
+ better consumer-focused thread control as far as receiving the data. A call to this function
+ simply indicates that data is available via the provided task.
+
+ The ``const char*`` parameter exists for producers to provide any extra contextual information
+ they want. This is encoded in the same format as :c:member:`ArrowSchema.metadata`. If not ``NULL``,
+ the lifetime is only the scope of the call to this function. A consumer who wants to maintain
+ the additional metadata beyond the lifetime of this call *MUST* copy the value themselves.
+
+ A producer *MUST NOT* call this concurrently from multiple different threads.
+
+ The :c:member:`ArrowAsyncProducer.request` callback must be called to start receiving calls to this
+ handler.
+
+ This function *MUST* be able to be called re-entrantly on the same thread to allow for the
+ :c:member:`ArrowAsyncProducer.request` callback to potentially call this method recursively.
+
+.. c:member:: void (*ArrowAsyncDeviceStreamHandler.on_error)(struct ArrowAsyncDeviceStreamHandler, int, const char*, const char*)
+
+ *Mandatory.* Handler to be called when an error is encountered by the producer. After calling
+ this, the ``release`` callback will be called as the last call on this struct. The parameters
+ are an ``errno``-compatible error code and an optional error message and metadata.
+
+ If the message and metadata are not ``NULL``, their lifetime is only valid during the scope
+ of this call. A consumer who wants to maintain these values past the return of this function
+ *MUST* copy the values themselves.
+
+ If the metadata parameter is not ``NULL``, to provide key-value error metadata, then it should
+ be encoded identically to the way that metadata is encoded in :c:member:`ArrowSchema.metadata`.
+
+ It is valid for this to be called by a producer with or without a preceding call to
+ :c:member:`ArrowAsyncProducer.request`. This callback *MUST NOT* call any methods of an
+ ``ArrowAsyncProducer`` object.
+
+.. c:member:: void (*ArrowAsyncDeviceStreamHandler.release)(struct ArrowAsyncDeviceStreamHandler*)
+
+ *Mandatory.* A pointer to a consumer-provided release callback for the handler.
+
+ It is valid for this to be called by a producer with or without a preceding call to
+ :c:member:`ArrowAsyncProducer.request`. This must not call any methods of an ``ArrowAsyncProducer``
+ object.
+
+.. c:member:: void* ArrowAsyncDeviceStreamHandler.private_data
+
+ *Optional.* An opaque pointer to consumer-provided private data.
+
+ Producers *MUST NOT* process this member. Lifetime of this member is handled by
+ the consumer, and especially by the release callback.
+
+The ArrowAsyncTask structure
+''''''''''''''''''''''''''''
+
+The purpose of using a Task object rather than passing the array directly to the ``on_next``
+callback is to allow for more complex and efficient thread handling. Utilizing a Task
+object allows for a producer to separate the "decoding" logic from the I/O, enabling a
+consumer to avoid transferring data between CPU cores (e.g. from one L1/L2 cache to another).
Review Comment:
If the producer wants to do the decompression then it can do so. The "task" that gets delivered in this case is just a thin wrapper around the decoded data and doesn't do any work.
If the producer wants to leave decompression for the consumer then it can do so. The "task" that gets delivered in this case actually does decompression.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-43631: [C][Format] Add ArrowAsyncDeviceStreamHandler interface [arrow]
Posted by "zeroshade (via GitHub)" <gi...@apache.org>.
zeroshade commented on code in PR #43632:
URL: https://github.com/apache/arrow/pull/43632#discussion_r1759006080
##########
docs/source/format/CDeviceDataInterface.rst:
##########
@@ -650,6 +652,331 @@ The stream source is not assumed to be thread-safe. Consumers wanting to
call ``get_next`` from several threads should ensure those calls are
serialized.
+Async Device Stream Interface
+=============================
+
+The :ref:`C stream interface <_c-device-stream-interface>` provides a synchronous
+API centered around the consumer calling the callback functions to retrieve
+the next record batch. For some bindings, use cases, and interoperability, a more
+asynchronous, producer-focused interface may be required. These scenarios can utilize
+the ``ArrowAsyncDeviceStreamHandler``.
+
+Semantics
+---------
+
+Rather than the producer providing a structure of callbacks for a consumer to
+call and retrieve records, the Async interface is a consumer allocated structure.
+The consumer allocated struct provides handler callbacks for the producer to call
+when the schema and chunks of data are available, rather than the consumer using
+a blocking pull-style iteration.
+
+In addition to the ``ArrowAsyncDeviceStreamHandler``, there are also two additional
+structs used for the full data flow: ``ArrowAsyncTask`` and ``ArrowAsyncProducer``.
+
+Structure Definition
+--------------------
+
+The C device async stream interface is defined with a single ``struct`` definition:
+
+.. code-block:: c
+
+ #ifndef ARROW_C_ASYNC_STREAM_INTERFACE
+ #define ARROW_C_ASYNC_STREAM_INTERFACE
+
+ struct ArrowAsyncTask {
+ int (*get_data)(struct ArrowArrayTask* self, struct ArrowDeviceArray* out);
+
+ void* private_data;
+ };
+
+ struct ArrowAsyncProducer {
+ void (*request)(struct ArrowAsyncProducer* self, uint64_t n);
+ void (*cancel)(struct ArrowAsyncProducer* self);
+
+ void (*release)(struct ArrowAsyncProducer* self);
+ void* private_data;
+ };
+
+ struct ArrowAsyncDeviceStreamHandler {
+ // handlers
+ int (*on_schema)(struct ArrowAsyncDeviceStreamHandler* self,
+ struct ArrowAsyncProducer* producer,
+ struct ArrowSchema* stream_schema, const char* addl_metadata);
+ int (*on_next_task)(struct ArrowAsyncDeviceStreamHandler* self,
+ struct ArrowAsyncTask* task, const char* metadata);
+ void (*on_error)(struct ArrowAsyncDeviceStreamHandler* self,
+ int code, const char* message, const char* metadata);
+
+ // release callback
+ void (*release)(struct ArrowAsyncDeviceStreamHandler* self);
+
+ // opaque handler-specific data
+ void* private_data;
+ };
+
+ #endif // ARROW_C_ASYNC_STREAM_INTERFACE
+
+.. note::
+ The canonical guard ``ARROW_C_ASYNC_STREAM_INTERFACE`` is meant to avoid
+ duplicate definitions if two projects copy the C async stream interface
+ definitions into their own headers, and a third-party project includes
+ from these two projects. It is therefore important that this guard is kept
+ exactly as-is when these definitions are copied.
+
+The ArrowAsyncDeviceStreamHandler structure
+'''''''''''''''''''''''''''''''''''''''''''
+
+The structure has the following fields:
+
+.. c:member:: int (*ArrowAsyncDeviceStreamHandler.on_schema)(struct ArrowAsyncDeviceStreamHandler*, struct ArrowAsyncProducer*, struct ArrowSchema*, const char*)
+
+ *Mandatory.* Handler for receiving the schema of the stream. All records should
+ match the provided schema. If successful, the function should return 0, otherwise
+ it should return an ``errno``-compatible error code.
+
+ The ``const char*`` parameter exists for producers to provide any extra contextual information
+ they want, such as the total number of rows in the stream, statistics, or otherwise. This is
+ encoded in the same format as :c:member:`ArrowSchema.metadata`. If not ``NULL``,
+ the lifetime is only the scope of the call to this function. A consumer who wants to maintain
+ the additional metadata beyond the lifetime of this call *MUST* copy the value themselves.
+
+ Unless the ``on_error`` handler is called, this will always get called exactly once and will be
+ the first method called on this object. As such the producer *MUST* provide an ``ArrowAsyncProducer``
+ object when calling this function to allow the consumer to manage back-pressure and flow control.
+ The producer maintains ownership of the ``ArrowAsyncProducer`` and must clean it up before or after
+ calling the release callback on this object.
+
+ A producer that receives a non-zero result here must not subsequently call anything other than
+ the release callback on this object.
+
+.. c:member:: int (*ArrowAsyncDeviceStreamHandler.on_next_task)(struct ArrowAsyncDeviceStreamHandler*, struct ArrowAsyncTask*, const char*)
+
+ *Mandatory.* Handler to be called when a new record is available for processing. The
+ schema for each record should be the same as the schema that ``on_schema`` was called with.
+ If successfully handled, the function should return 0, otherwise it should return an
+ ``errno``-compatible error code.
+
+ Rather than passing the record itself it receives an ``ArrowAsyncTask`` instead to facilitate
+ better consumer-focused thread control as far as receiving the data. A call to this function
+ simply indicates that data is available via the provided task.
+
+ The ``const char*`` parameter exists for producers to provide any extra contextual information
+ they want. This is encoded in the same format as :c:member:`ArrowSchema.metadata`. If not ``NULL``,
+ the lifetime is only the scope of the call to this function. A consumer who wants to maintain
+ the additional metadata beyond the lifetime of this call *MUST* copy the value themselves.
+
+ A producer *MUST NOT* call this concurrently from multiple different threads.
+
+ The :c:member:`ArrowAsyncProducer.request` callback must be called to start receiving calls to this
+ handler.
+
+ This function *MUST* be able to be called re-entrantly on the same thread to allow for the
+ :c:member:`ArrowAsyncProducer.request` callback to potentially call this method recursively.
+
+.. c:member:: void (*ArrowAsyncDeviceStreamHandler.on_error)(struct ArrowAsyncDeviceStreamHandler, int, const char*, const char*)
+
+ *Mandatory.* Handler to be called when an error is encountered by the producer. After calling
+ this, the ``release`` callback will be called as the last call on this struct. The parameters
+ are an ``errno``-compatible error code and an optional error message and metadata.
+
+ If the message and metadata are not ``NULL``, their lifetime is only valid during the scope
+ of this call. A consumer who wants to maintain these values past the return of this function
+ *MUST* copy the values themselves.
+
+ If the metadata parameter is not ``NULL``, to provide key-value error metadata, then it should
+ be encoded identically to the way that metadata is encoded in :c:member:`ArrowSchema.metadata`.
+
+ It is valid for this to be called by a producer with or without a preceding call to
+ :c:member:`ArrowAsyncProducer.request`. This callback *MUST NOT* call any methods of an
+ ``ArrowAsyncProducer`` object.
+
+.. c:member:: void (*ArrowAsyncDeviceStreamHandler.release)(struct ArrowAsyncDeviceStreamHandler*)
+
+ *Mandatory.* A pointer to a consumer-provided release callback for the handler.
+
+ It is valid for this to be called by a producer with or without a preceding call to
+ :c:member:`ArrowAsyncProducer.request`. This must not call any methods of an ``ArrowAsyncProducer``
+ object.
+
+.. c:member:: void* ArrowAsyncDeviceStreamHandler.private_data
+
+ *Optional.* An opaque pointer to consumer-provided private data.
+
+ Producers *MUST NOT* process this member. Lifetime of this member is handled by
+ the consumer, and especially by the release callback.
+
+The ArrowAsyncTask structure
+''''''''''''''''''''''''''''
+
+The purpose of using a Task object rather than passing the array directly to the ``on_next``
+callback is to allow for more complex and efficient thread handling. Utilizing a Task
+object allows for a producer to separate the "decoding" logic from the I/O, enabling a
+consumer to avoid transferring data between CPU cores (e.g. from one L1/L2 cache to another).
+
+This producer-provided structure has the following fields:
+
+.. c:member:: int (*ArrowArrayTask.get_data)(struct ArrowArrayTask*, struct ArrowDeviceArray*)
+
+ *Mandatory.* A callback to populate the provided ``ArrowDeviceArray`` with the available data.
+ The order of ArrowAsyncTasks provided by the producer enables a consumer to know the order of
+ the data to process.
+
+ If a non-zero value is returned from this, it should be followed only by the producer calling
+ the ``on_error`` callback of the ``ArrowAsyncDeviceStreamHandler``. Because calling this method
+ is likely to be separate from the current control flow, returning a non-zero value to signal
+ an error occuring allows the current thread to decide handle the case accordingly, while still
+ allowing all error logging and handling to be centralized in the
+ :c:member:`ArrowAsyncDeviceStreamHandler.on_error` callback.
+
+ Rather than having a separate release callback, any required cleanup should be performed as part
+ of the invocation of this callback, with ownership of the Array being passed to the consumer-provided
+ parameter calling this, and must be released separately.
+
+ It is only valid to call this method exactly once.
+
+.. c:member:: void* ArrowArrayTask.private_data
+
+ *Optional.* An opaque pointer to consumer-provided private data.
Review Comment:
that's a typo, i'll fix that
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-43631: [C][Format] Add ArrowAsyncDeviceStreamHandler interface [arrow]
Posted by "zeroshade (via GitHub)" <gi...@apache.org>.
zeroshade commented on code in PR #43632:
URL: https://github.com/apache/arrow/pull/43632#discussion_r1759037667
##########
docs/source/format/CDeviceDataInterface.rst:
##########
@@ -650,6 +652,331 @@ The stream source is not assumed to be thread-safe. Consumers wanting to
call ``get_next`` from several threads should ensure those calls are
serialized.
+Async Device Stream Interface
+=============================
+
+The :ref:`C stream interface <_c-device-stream-interface>` provides a synchronous
+API centered around the consumer calling the callback functions to retrieve
+the next record batch. For some bindings, use cases, and interoperability, a more
+asynchronous, producer-focused interface may be required. These scenarios can utilize
+the ``ArrowAsyncDeviceStreamHandler``.
+
+Semantics
+---------
+
+Rather than the producer providing a structure of callbacks for a consumer to
+call and retrieve records, the Async interface is a consumer allocated structure.
+The consumer allocated struct provides handler callbacks for the producer to call
+when the schema and chunks of data are available, rather than the consumer using
+a blocking pull-style iteration.
+
+In addition to the ``ArrowAsyncDeviceStreamHandler``, there are also two additional
+structs used for the full data flow: ``ArrowAsyncTask`` and ``ArrowAsyncProducer``.
+
+Structure Definition
+--------------------
+
+The C device async stream interface is defined with a single ``struct`` definition:
+
+.. code-block:: c
+
+ #ifndef ARROW_C_ASYNC_STREAM_INTERFACE
+ #define ARROW_C_ASYNC_STREAM_INTERFACE
+
+ struct ArrowAsyncTask {
+ int (*get_data)(struct ArrowArrayTask* self, struct ArrowDeviceArray* out);
+
+ void* private_data;
+ };
+
+ struct ArrowAsyncProducer {
+ void (*request)(struct ArrowAsyncProducer* self, uint64_t n);
+ void (*cancel)(struct ArrowAsyncProducer* self);
+
+ void (*release)(struct ArrowAsyncProducer* self);
+ void* private_data;
+ };
+
+ struct ArrowAsyncDeviceStreamHandler {
+ // handlers
+ int (*on_schema)(struct ArrowAsyncDeviceStreamHandler* self,
+ struct ArrowAsyncProducer* producer,
+ struct ArrowSchema* stream_schema, const char* addl_metadata);
+ int (*on_next_task)(struct ArrowAsyncDeviceStreamHandler* self,
+ struct ArrowAsyncTask* task, const char* metadata);
+ void (*on_error)(struct ArrowAsyncDeviceStreamHandler* self,
+ int code, const char* message, const char* metadata);
+
+ // release callback
+ void (*release)(struct ArrowAsyncDeviceStreamHandler* self);
+
+ // opaque handler-specific data
+ void* private_data;
+ };
+
+ #endif // ARROW_C_ASYNC_STREAM_INTERFACE
+
+.. note::
+ The canonical guard ``ARROW_C_ASYNC_STREAM_INTERFACE`` is meant to avoid
+ duplicate definitions if two projects copy the C async stream interface
+ definitions into their own headers, and a third-party project includes
+ from these two projects. It is therefore important that this guard is kept
+ exactly as-is when these definitions are copied.
+
+The ArrowAsyncDeviceStreamHandler structure
+'''''''''''''''''''''''''''''''''''''''''''
+
+The structure has the following fields:
+
+.. c:member:: int (*ArrowAsyncDeviceStreamHandler.on_schema)(struct ArrowAsyncDeviceStreamHandler*, struct ArrowAsyncProducer*, struct ArrowSchema*, const char*)
+
+ *Mandatory.* Handler for receiving the schema of the stream. All records should
+ match the provided schema. If successful, the function should return 0, otherwise
+ it should return an ``errno``-compatible error code.
+
+ The ``const char*`` parameter exists for producers to provide any extra contextual information
+ they want, such as the total number of rows in the stream, statistics, or otherwise. This is
+ encoded in the same format as :c:member:`ArrowSchema.metadata`. If not ``NULL``,
+ the lifetime is only the scope of the call to this function. A consumer who wants to maintain
+ the additional metadata beyond the lifetime of this call *MUST* copy the value themselves.
+
+ Unless the ``on_error`` handler is called, this will always get called exactly once and will be
+ the first method called on this object. As such the producer *MUST* provide an ``ArrowAsyncProducer``
+ object when calling this function to allow the consumer to manage back-pressure and flow control.
+ The producer maintains ownership of the ``ArrowAsyncProducer`` and must clean it up before or after
+ calling the release callback on this object.
+
+ A producer that receives a non-zero result here must not subsequently call anything other than
+ the release callback on this object.
+
+.. c:member:: int (*ArrowAsyncDeviceStreamHandler.on_next_task)(struct ArrowAsyncDeviceStreamHandler*, struct ArrowAsyncTask*, const char*)
+
+ *Mandatory.* Handler to be called when a new record is available for processing. The
+ schema for each record should be the same as the schema that ``on_schema`` was called with.
+ If successfully handled, the function should return 0, otherwise it should return an
+ ``errno``-compatible error code.
+
+ Rather than passing the record itself it receives an ``ArrowAsyncTask`` instead to facilitate
+ better consumer-focused thread control as far as receiving the data. A call to this function
+ simply indicates that data is available via the provided task.
+
+ The ``const char*`` parameter exists for producers to provide any extra contextual information
+ they want. This is encoded in the same format as :c:member:`ArrowSchema.metadata`. If not ``NULL``,
+ the lifetime is only the scope of the call to this function. A consumer who wants to maintain
+ the additional metadata beyond the lifetime of this call *MUST* copy the value themselves.
+
+ A producer *MUST NOT* call this concurrently from multiple different threads.
+
+ The :c:member:`ArrowAsyncProducer.request` callback must be called to start receiving calls to this
+ handler.
+
+ This function *MUST* be able to be called re-entrantly on the same thread to allow for the
+ :c:member:`ArrowAsyncProducer.request` callback to potentially call this method recursively.
+
+.. c:member:: void (*ArrowAsyncDeviceStreamHandler.on_error)(struct ArrowAsyncDeviceStreamHandler, int, const char*, const char*)
+
+ *Mandatory.* Handler to be called when an error is encountered by the producer. After calling
+ this, the ``release`` callback will be called as the last call on this struct. The parameters
+ are an ``errno``-compatible error code and an optional error message and metadata.
+
+ If the message and metadata are not ``NULL``, their lifetime is only valid during the scope
+ of this call. A consumer who wants to maintain these values past the return of this function
+ *MUST* copy the values themselves.
+
+ If the metadata parameter is not ``NULL``, to provide key-value error metadata, then it should
+ be encoded identically to the way that metadata is encoded in :c:member:`ArrowSchema.metadata`.
+
+ It is valid for this to be called by a producer with or without a preceding call to
+ :c:member:`ArrowAsyncProducer.request`. This callback *MUST NOT* call any methods of an
+ ``ArrowAsyncProducer`` object.
+
+.. c:member:: void (*ArrowAsyncDeviceStreamHandler.release)(struct ArrowAsyncDeviceStreamHandler*)
+
+ *Mandatory.* A pointer to a consumer-provided release callback for the handler.
+
+ It is valid for this to be called by a producer with or without a preceding call to
+ :c:member:`ArrowAsyncProducer.request`. This must not call any methods of an ``ArrowAsyncProducer``
+ object.
+
+.. c:member:: void* ArrowAsyncDeviceStreamHandler.private_data
+
+ *Optional.* An opaque pointer to consumer-provided private data.
+
+ Producers *MUST NOT* process this member. Lifetime of this member is handled by
+ the consumer, and especially by the release callback.
+
+The ArrowAsyncTask structure
+''''''''''''''''''''''''''''
+
+The purpose of using a Task object rather than passing the array directly to the ``on_next``
+callback is to allow for more complex and efficient thread handling. Utilizing a Task
+object allows for a producer to separate the "decoding" logic from the I/O, enabling a
+consumer to avoid transferring data between CPU cores (e.g. from one L1/L2 cache to another).
+
+This producer-provided structure has the following fields:
+
+.. c:member:: int (*ArrowArrayTask.get_data)(struct ArrowArrayTask*, struct ArrowDeviceArray*)
+
+ *Mandatory.* A callback to populate the provided ``ArrowDeviceArray`` with the available data.
+ The order of ArrowAsyncTasks provided by the producer enables a consumer to know the order of
+ the data to process.
+
+ If a non-zero value is returned from this, it should be followed only by the producer calling
+ the ``on_error`` callback of the ``ArrowAsyncDeviceStreamHandler``. Because calling this method
+ is likely to be separate from the current control flow, returning a non-zero value to signal
+ an error occuring allows the current thread to decide handle the case accordingly, while still
+ allowing all error logging and handling to be centralized in the
+ :c:member:`ArrowAsyncDeviceStreamHandler.on_error` callback.
+
+ Rather than having a separate release callback, any required cleanup should be performed as part
+ of the invocation of this callback, with ownership of the Array being passed to the consumer-provided
+ parameter calling this, and must be released separately.
+
+ It is only valid to call this method exactly once.
+
+.. c:member:: void* ArrowArrayTask.private_data
+
+ *Optional.* An opaque pointer to consumer-provided private data.
+
+ Consumers *MUST NOT* process this member. Lifetime of this member is handled by
+ the producer who created this object, and should be cleaned up if necessary during
+ the call to :c:member:`ArrowArrayTask.get_data`.
+
+The ArrowAsyncProducer structure
+''''''''''''''''''''''''''''''''
+
+This producer-provided and managed object has the following fields:
+
+.. c:member:: void (*ArrowAsyncProducer.request)(struct ArrowAsyncProducer*, uint64_t)
+
+ *Mandatory.* This function must be called by a consumer to start receiving calls to
+ :c:member:`ArrowAsyncDeviceStreamHandler.on_next_task`. It *MUST* be valid to call
+ this synchronously from within :c:member:`ArrowAsyncDeviceStreamHandler.on_next_task`
+ or :c:member:`ArrowAsyncDeviceStreamHandler.on_schema`. As a result, this function
+ *MUST NOT* synchronously call ``on_next_task`` or ``on_error`` to avoid recursive
+ and reentrant callbacks.
+
+ After ``cancel`` is called, additional calls to this function must be a NOP, but allowed.
+
+ While not cancelled, calling this function registers the given number of additional
+ arrays/batches to be produced by the producer. A producer should only call
+ the appropriate ``on_next_task`` callback up to a maximum of the total sum of calls to
+ this method before propagating back-pressure / waiting.
+
+ Any error encountered by calling request must be propagated by calling the ``on_error``
+ callback of the ``ArrowAsyncDeviceStreamHandler``.
+
+ A producer *MUST* support an unbounded number of calls to this callback and *MUST* support
+ a total registered demand (sum requested - sum delivered) of up to ``UINT64_MAX``.
Review Comment:
for compatibility with the Reactive Streams protocols and implementation. It also simplifies validation and error checking. Obviously if the demand is higher than the number of arrays in the stream, the consumer would eventually just receive notice that the stream is over (by receiving a NULL for the `ArrowAsyncTask` in the `on_next_task` callback)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-43631: [C][Format] Add ArrowAsyncDeviceStreamHandler interface [arrow]
Posted by "zeroshade (via GitHub)" <gi...@apache.org>.
zeroshade commented on code in PR #43632:
URL: https://github.com/apache/arrow/pull/43632#discussion_r1759127202
##########
docs/source/format/CDeviceDataInterface.rst:
##########
@@ -650,6 +652,331 @@ The stream source is not assumed to be thread-safe. Consumers wanting to
call ``get_next`` from several threads should ensure those calls are
serialized.
+Async Device Stream Interface
+=============================
+
+The :ref:`C stream interface <_c-device-stream-interface>` provides a synchronous
+API centered around the consumer calling the callback functions to retrieve
+the next record batch. For some bindings, use cases, and interoperability, a more
+asynchronous, producer-focused interface may be required. These scenarios can utilize
+the ``ArrowAsyncDeviceStreamHandler``.
+
+Semantics
+---------
+
+Rather than the producer providing a structure of callbacks for a consumer to
+call and retrieve records, the Async interface is a consumer allocated structure.
+The consumer allocated struct provides handler callbacks for the producer to call
+when the schema and chunks of data are available, rather than the consumer using
+a blocking pull-style iteration.
+
+In addition to the ``ArrowAsyncDeviceStreamHandler``, there are also two additional
+structs used for the full data flow: ``ArrowAsyncTask`` and ``ArrowAsyncProducer``.
+
+Structure Definition
+--------------------
+
+The C device async stream interface is defined with a single ``struct`` definition:
+
+.. code-block:: c
+
+ #ifndef ARROW_C_ASYNC_STREAM_INTERFACE
+ #define ARROW_C_ASYNC_STREAM_INTERFACE
+
+ struct ArrowAsyncTask {
+ int (*get_data)(struct ArrowArrayTask* self, struct ArrowDeviceArray* out);
+
+ void* private_data;
+ };
+
+ struct ArrowAsyncProducer {
+ void (*request)(struct ArrowAsyncProducer* self, uint64_t n);
+ void (*cancel)(struct ArrowAsyncProducer* self);
+
+ void (*release)(struct ArrowAsyncProducer* self);
+ void* private_data;
+ };
+
+ struct ArrowAsyncDeviceStreamHandler {
+ // handlers
+ int (*on_schema)(struct ArrowAsyncDeviceStreamHandler* self,
+ struct ArrowAsyncProducer* producer,
+ struct ArrowSchema* stream_schema, const char* addl_metadata);
+ int (*on_next_task)(struct ArrowAsyncDeviceStreamHandler* self,
+ struct ArrowAsyncTask* task, const char* metadata);
+ void (*on_error)(struct ArrowAsyncDeviceStreamHandler* self,
+ int code, const char* message, const char* metadata);
+
+ // release callback
+ void (*release)(struct ArrowAsyncDeviceStreamHandler* self);
+
+ // opaque handler-specific data
+ void* private_data;
+ };
+
+ #endif // ARROW_C_ASYNC_STREAM_INTERFACE
+
+.. note::
+ The canonical guard ``ARROW_C_ASYNC_STREAM_INTERFACE`` is meant to avoid
+ duplicate definitions if two projects copy the C async stream interface
+ definitions into their own headers, and a third-party project includes
+ from these two projects. It is therefore important that this guard is kept
+ exactly as-is when these definitions are copied.
+
+The ArrowAsyncDeviceStreamHandler structure
+'''''''''''''''''''''''''''''''''''''''''''
+
+The structure has the following fields:
+
+.. c:member:: int (*ArrowAsyncDeviceStreamHandler.on_schema)(struct ArrowAsyncDeviceStreamHandler*, struct ArrowAsyncProducer*, struct ArrowSchema*, const char*)
+
+ *Mandatory.* Handler for receiving the schema of the stream. All records should
+ match the provided schema. If successful, the function should return 0, otherwise
+ it should return an ``errno``-compatible error code.
+
+ The ``const char*`` parameter exists for producers to provide any extra contextual information
+ they want, such as the total number of rows in the stream, statistics, or otherwise. This is
+ encoded in the same format as :c:member:`ArrowSchema.metadata`. If not ``NULL``,
+ the lifetime is only the scope of the call to this function. A consumer who wants to maintain
+ the additional metadata beyond the lifetime of this call *MUST* copy the value themselves.
+
+ Unless the ``on_error`` handler is called, this will always get called exactly once and will be
+ the first method called on this object. As such the producer *MUST* provide an ``ArrowAsyncProducer``
+ object when calling this function to allow the consumer to manage back-pressure and flow control.
+ The producer maintains ownership of the ``ArrowAsyncProducer`` and must clean it up before or after
+ calling the release callback on this object.
+
+ A producer that receives a non-zero result here must not subsequently call anything other than
+ the release callback on this object.
+
+.. c:member:: int (*ArrowAsyncDeviceStreamHandler.on_next_task)(struct ArrowAsyncDeviceStreamHandler*, struct ArrowAsyncTask*, const char*)
+
+ *Mandatory.* Handler to be called when a new record is available for processing. The
+ schema for each record should be the same as the schema that ``on_schema`` was called with.
+ If successfully handled, the function should return 0, otherwise it should return an
+ ``errno``-compatible error code.
+
+ Rather than passing the record itself it receives an ``ArrowAsyncTask`` instead to facilitate
+ better consumer-focused thread control as far as receiving the data. A call to this function
+ simply indicates that data is available via the provided task.
+
+ The ``const char*`` parameter exists for producers to provide any extra contextual information
+ they want. This is encoded in the same format as :c:member:`ArrowSchema.metadata`. If not ``NULL``,
+ the lifetime is only the scope of the call to this function. A consumer who wants to maintain
+ the additional metadata beyond the lifetime of this call *MUST* copy the value themselves.
+
+ A producer *MUST NOT* call this concurrently from multiple different threads.
+
+ The :c:member:`ArrowAsyncProducer.request` callback must be called to start receiving calls to this
+ handler.
+
+ This function *MUST* be able to be called re-entrantly on the same thread to allow for the
+ :c:member:`ArrowAsyncProducer.request` callback to potentially call this method recursively.
+
+.. c:member:: void (*ArrowAsyncDeviceStreamHandler.on_error)(struct ArrowAsyncDeviceStreamHandler, int, const char*, const char*)
+
+ *Mandatory.* Handler to be called when an error is encountered by the producer. After calling
+ this, the ``release`` callback will be called as the last call on this struct. The parameters
+ are an ``errno``-compatible error code and an optional error message and metadata.
+
+ If the message and metadata are not ``NULL``, their lifetime is only valid during the scope
+ of this call. A consumer who wants to maintain these values past the return of this function
+ *MUST* copy the values themselves.
+
+ If the metadata parameter is not ``NULL``, to provide key-value error metadata, then it should
+ be encoded identically to the way that metadata is encoded in :c:member:`ArrowSchema.metadata`.
+
+ It is valid for this to be called by a producer with or without a preceding call to
+ :c:member:`ArrowAsyncProducer.request`. This callback *MUST NOT* call any methods of an
+ ``ArrowAsyncProducer`` object.
+
+.. c:member:: void (*ArrowAsyncDeviceStreamHandler.release)(struct ArrowAsyncDeviceStreamHandler*)
+
+ *Mandatory.* A pointer to a consumer-provided release callback for the handler.
+
+ It is valid for this to be called by a producer with or without a preceding call to
+ :c:member:`ArrowAsyncProducer.request`. This must not call any methods of an ``ArrowAsyncProducer``
+ object.
+
+.. c:member:: void* ArrowAsyncDeviceStreamHandler.private_data
+
+ *Optional.* An opaque pointer to consumer-provided private data.
+
+ Producers *MUST NOT* process this member. Lifetime of this member is handled by
+ the consumer, and especially by the release callback.
+
+The ArrowAsyncTask structure
+''''''''''''''''''''''''''''
+
+The purpose of using a Task object rather than passing the array directly to the ``on_next``
+callback is to allow for more complex and efficient thread handling. Utilizing a Task
+object allows for a producer to separate the "decoding" logic from the I/O, enabling a
+consumer to avoid transferring data between CPU cores (e.g. from one L1/L2 cache to another).
+
+This producer-provided structure has the following fields:
+
+.. c:member:: int (*ArrowArrayTask.get_data)(struct ArrowArrayTask*, struct ArrowDeviceArray*)
+
+ *Mandatory.* A callback to populate the provided ``ArrowDeviceArray`` with the available data.
+ The order of ArrowAsyncTasks provided by the producer enables a consumer to know the order of
+ the data to process.
+
+ If a non-zero value is returned from this, it should be followed only by the producer calling
+ the ``on_error`` callback of the ``ArrowAsyncDeviceStreamHandler``. Because calling this method
+ is likely to be separate from the current control flow, returning a non-zero value to signal
+ an error occuring allows the current thread to decide handle the case accordingly, while still
+ allowing all error logging and handling to be centralized in the
+ :c:member:`ArrowAsyncDeviceStreamHandler.on_error` callback.
+
+ Rather than having a separate release callback, any required cleanup should be performed as part
+ of the invocation of this callback, with ownership of the Array being passed to the consumer-provided
+ parameter calling this, and must be released separately.
+
+ It is only valid to call this method exactly once.
+
+.. c:member:: void* ArrowArrayTask.private_data
+
+ *Optional.* An opaque pointer to consumer-provided private data.
+
+ Consumers *MUST NOT* process this member. Lifetime of this member is handled by
+ the producer who created this object, and should be cleaned up if necessary during
+ the call to :c:member:`ArrowArrayTask.get_data`.
+
+The ArrowAsyncProducer structure
+''''''''''''''''''''''''''''''''
+
+This producer-provided and managed object has the following fields:
+
+.. c:member:: void (*ArrowAsyncProducer.request)(struct ArrowAsyncProducer*, uint64_t)
+
+ *Mandatory.* This function must be called by a consumer to start receiving calls to
+ :c:member:`ArrowAsyncDeviceStreamHandler.on_next_task`. It *MUST* be valid to call
+ this synchronously from within :c:member:`ArrowAsyncDeviceStreamHandler.on_next_task`
+ or :c:member:`ArrowAsyncDeviceStreamHandler.on_schema`. As a result, this function
+ *MUST NOT* synchronously call ``on_next_task`` or ``on_error`` to avoid recursive
+ and reentrant callbacks.
+
+ After ``cancel`` is called, additional calls to this function must be a NOP, but allowed.
+
+ While not cancelled, calling this function registers the given number of additional
+ arrays/batches to be produced by the producer. A producer should only call
+ the appropriate ``on_next_task`` callback up to a maximum of the total sum of calls to
+ this method before propagating back-pressure / waiting.
+
+ Any error encountered by calling request must be propagated by calling the ``on_error``
+ callback of the ``ArrowAsyncDeviceStreamHandler``.
+
+ A producer *MUST* support an unbounded number of calls to this callback and *MUST* support
+ a total registered demand (sum requested - sum delivered) of up to ``UINT64_MAX``.
+
+.. c:member:: void (*ArrowAsyncProducer.cancel)(struct ArrowAsyncProducer*)
+
+ *Mandatory.* This function signals to the producer that it must *eventually* stop calling
+ ``on_next_task``. Calls to ``cancel`` must be idempotent and thread-safe. After calling
+ it once, subsequent calls *MUST* be a NOP.
+
+ It is not required that calling ``cancel`` affect the producer *immediately*, only that it
+ must eventually stop calling ``on_next_task`` and then subsequently call ``release``
+ on the async handler object. As such, a consumer *MUST* be prepared to receive one or more
+ calls to ``on_next_task`` or ``on_error`` even after calling ``cancel`` if there are still
+ requested arrays pending.
+
+ Successful cancelling *MUST NOT* result in a producer calling
+ :c:member:`ArrowAsyncDeviceStreamHandler.on_error`, instead it should finish out any remaining
+ tasks (calling ``on_next_task`` accordingly) and eventually just call ``release``.
+
+ Any error encountered during handling a call to cancel must be reported via the ``on_error``
+ callback on the async stream handler.
+
+.. c:member:: void* ArrowAsyncProducer.private_data
+
+ *Optional.* An opaque pointer to producer-provided specific data.
+
+ Consumers *MUST NOT* process this member, the lifetime is owned by the producer
+ that constructed this object.
+
+Error Handling
+''''''''''''''
+
+Unlike the regular C Stream interface, the Async interface allows for errors to flow in
+both directions. As a result, error handling can be slightly more complex. Thus this spec
+designates the following rules:
+
+* If the producer encounters an error during processing, it should call the ``on_error``
+callback, and then call ``release`` after it returns.
+
+* If ``on_schema`` or ``on_next_task`` returns a non-zero integer value, the producer *should not*
+call the ``on_error`` callback, but instead should eventually call ``release`` at some point
+before or after any logging or processing of the error code.
+
+Result lifetimes
+''''''''''''''''
+
+The ``ArrowSchema`` passed to the ``on_schema`` callback must be released independantly,
+with the object itself needing to be moved to a consumer owned ``ArrowSchema`` object. The
+``ArrowSchema*`` passed as a parameter to the callback *MUST NOT* be stored and kept.
+
+The ``ArrowAsyncTask`` object provided to ``on_next_task`` is owned by the producer and
+will be cleaned up during the invocation of calling ``get_data`` on it. If the consumer
+doesn't care about the data, it should pass ``NULL`` instead of a valid ``ArrowDeviceArray*``.
+
+The ``const char*`` error ``message`` and ``metadata`` which are passed to ``on_error``
+are only valid within the scope of the ``on_error`` function itself. They must be copied
+if it is necessary for them to exist after it returns.
+
+Stream Handler Lifetime
+'''''''''''''''''''''''
+
+Lifetime of the async stream handler is managed using a release callback with similar
+usage as in :ref:`C data interface <c-data-interface-released>`.
+
+ArrowAsyncProducer Lifetime
+'''''''''''''''''''''''''''
+
+The lifetime of the ``ArrowAsyncProducer`` passed to ``on_schema`` is owned by the producer
+itself and should be managed by it. It *MUST* remain valid at least until just before
+calling ``release`` on the stream handler object.
+
+Thread safety
+'''''''''''''
+
+All handler functions should only be called in a serialized manner, but are not guaranteed
Review Comment:
fixed
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-43631: [C][Format] Add ArrowAsyncDeviceStreamHandler interface [arrow]
Posted by "lidavidm (via GitHub)" <gi...@apache.org>.
lidavidm commented on code in PR #43632:
URL: https://github.com/apache/arrow/pull/43632#discussion_r1830411074
##########
cpp/src/arrow/c/abi.h:
##########
@@ -319,6 +319,11 @@ struct ArrowAsyncProducer {
// on_error callback on the async stream handler.
void (*cancel)(struct ArrowAsyncProducer* self);
+ // Any additional metadata tied to a specific stream of data. This must either be NULL
+ // or a valid pointer to metadata which is encoded in the same way schema metadata
+ // would be. Non-null metadata must be valid for the lifetime of this object.
Review Comment:
Basically the context we had earlier, yes
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org