[GitHub] [arrow] zeroshade opened a new pull request, #34972: GH-34971: [Format] Enhance C-Data API to support non-cpu cases

["zeroshade (via GitHub)" <gi...@apache.org>]

zeroshade opened a new pull request, #34972:
URL: https://github.com/apache/arrow/pull/34972

   <!--
   Thanks for opening a pull request!
   If this is your first pull request you can find detailed information on how 
   to contribute here:
     * [New Contributor's Guide](https://arrow.apache.org/docs/dev/developers/guide/step_by_step/pr_lifecycle.html#reviews-and-merge-of-the-pull-request)
     * [Contributing Overview](https://arrow.apache.org/docs/dev/developers/overview.html)
   
   
   If this is not a [minor PR](https://github.com/apache/arrow/blob/main/CONTRIBUTING.md#Minor-Fixes). Could you open an issue for this pull request on GitHub? https://github.com/apache/arrow/issues/new/choose
   
   Opening GitHub issues ahead of time contributes to the [Openness](http://theapacheway.com/open/#:~:text=Openness%20allows%20new%20users%20the,must%20happen%20in%20the%20open.) of the Apache Arrow project.
   
   Then could you also rename the pull request title in the following format?
   
       GH-${GITHUB_ISSUE_ID}: [${COMPONENT}] ${SUMMARY}
   
   or
   
       MINOR: [${COMPONENT}] ${SUMMARY}
   
   In the case of PARQUET issues on JIRA the title also supports:
   
       PARQUET-${JIRA_ISSUE_ID}: [${COMPONENT}] ${SUMMARY}
   
   -->
   
   ### Rationale for this change
   In order to support non-cpu devices and memory usage, we can add new `ArrowDeviceArray` and `ArrowDeviceArrayStream` structs to the C-Data API in order to allow for handling these types of memory without breaking the existing ABI.
   
   <!--
    Why are you proposing this change? If this is already explained clearly in the issue then this section is not needed.
    Explaining clearly why changes are proposed helps reviewers understand your changes and offer better suggestions for fixes.  
   -->
   
   ### What changes are included in this PR?
   Definitions for a new `ArrowDeviceArray`, `ArrowDeviceArrayStream` and `ArrowDeviceType` enums.
   
   <!--
   There is no need to duplicate the description in the issue here but it is sometimes worth providing a summary of the individual changes in this PR.
   -->
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [arrow] pitrou commented on a diff in pull request #34972: GH-34971: [Format] Add non-CPU version of C Data Interface

["pitrou (via GitHub)" <gi...@apache.org>]

pitrou commented on code in PR #34972:
URL: https://github.com/apache/arrow/pull/34972#discussion_r1219640910


##########
docs/source/format/CDeviceDataInterface.rst:
##########
@@ -0,0 +1,688 @@
+.. Licensed to the Apache Software Foundation (ASF) under one
+.. or more contributor license agreements.  See the NOTICE file
+.. distributed with this work for additional information
+.. regarding copyright ownership.  The ASF licenses this file
+.. to you under the Apache License, Version 2.0 (the
+.. "License"); you may not use this file except in compliance
+.. with the License.  You may obtain a copy of the License at
+
+..   http://www.apache.org/licenses/LICENSE-2.0
+
+.. Unless required by applicable law or agreed to in writing,
+.. software distributed under the License is distributed on an
+.. "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+.. KIND, either express or implied.  See the License for the
+.. specific language governing permissions and limitations
+.. under the License.
+
+.. highlight:: c
+
+.. _c-device-data-interface:
+
+=================================
+The Arrow C Device data interface
+=================================
+
+.. warning:: The Arrow C Device Data Interface should be considered experimental
+
+Rationale
+=========
+
+The current :ref:`C Data Interface <c-data-interface>`, and most
+implementations of it, make the assumption that all data buffers provided
+are CPU buffers. Since Apache Arrow is designed to be a universal in-memory
+format for representing tabular ("columnar") data, there will be the desire
+to leverage this data on non-CPU hardware such as GPUs. One example of such
+a case is the `RAPIDS cuDF library`_ which uses the Arrow memory format with
+CUDA for NVIDIA GPUs. Since copying data from host to device and back is
+expensive, the ideal would be to be able to leave the data on the device
+for as long as possible, even when passing it between runtimes and
+libraries.
+
+The Arrow C Device data interface builds on the existing C data interface
+by adding a very small, stable set of C definitions to it. These definitions
+are equivalents to the ``ArrowArray`` and ``ArrowArrayStream`` structures
+from the C Data Interface which add members to allow specifying the device
+type and pass necessary information to synchronize with the producer.
+For non-C/C++ languages and runtimes, translating the C definitions to
+corresponding C FFI declarations should be just as simple as with the
+current C data interface.
+
+Applications and libraries can then use Arrow schemas and Arrow formatted
+memory on non-CPU devices to exchange data just as easily as they do
+now with CPU data. This will enable leaving data on those devices longer
+and avoiding costly copies back and forth between the host and device
+just to leverage new libraries and runtimes.
+
+Goals
+-----
+
+* Expose an ABI-stable interface built on the existing C data interface.
+* Make it easy for third-party projects to implement support with little
+  initial investment.
+* Allow zero-copy sharing of Arrow formatted device memory between
+  independant runtimes and components running in the same process.
+* Avoid the need for one-to-one adaptation layers such as the
+  `CUDA Array Interface`_ for Python processes to pass CUDA data.
+* Enable integration without explicit dependencies (either at compile-time
+  or runtime) on the Arrow software project itself.
+
+The intent is for the Arrow C Device data interface to expand the reach
+of the current C data interface, allowing it to also become the standard
+low-level building block for columnar processing on devices like GPUs or
+FPGAs.
+
+Structure definitions
+=====================
+
+Because this is built on the C data interface, the C Device data interface
+uses the ``ArrowSchema`` and ``ArrowArray`` structures as defined in the
+:ref:`C data interface spec <c-data-interface-struct-defs>`. It then adds the
+following free-standing definitions. Like the rest of the Arrow project,
+they are available under the Apache License 2.0.
+
+.. code-block:: c
+
+    #ifndef ARROW_C_DEVICE_DATA_INTERFACE
+    #define ARROW_C_DEVICE_DATA_INTERFACE
+
+    // Device type for the allocated memory
+    typedef int32_t ArrowDeviceType;
+
+    // CPU device, same as using ArrowArray directly
+    #define ARROW_DEVICE_CPU 1
+    // CUDA GPU Device
+    #define ARROW_DEVICE_CUDA 2
+    // Pinned CUDA CPU memory by cudaMallocHost
+    #define ARROW_DEVICE_CUDA_HOST 3
+    // OpenCL Device
+    #define ARROW_DEVICE_OPENCL 4
+    // Vulkan buffer for next-gen graphics
+    #define ARROW_DEVICE_VULKAN 7
+    // Metal for Apple GPU
+    #define ARROW_DEVICE_METAL 8
+    // Verilog simulator buffer
+    #define ARROW_DEVICE_VPI 9
+    // ROCm GPUs for AMD GPUs
+    #define ARROW_DEVICE_ROCM 10
+    // Pinned ROCm CPU memory allocated by hipMallocHost
+    #define ARROW_DEVICE_ROCM_HOST 11
+    // Reserved for extension
+    //
+    // used to quickly test extension devices, semantics
+    // can differ based on implementation
+    #define ARROW_DEVICE_EXT_DEV 12
+    // CUDA managed/unified memory allocated by cudaMallocManaged
+    #define ARROW_DEVICE_CUDA_MANAGED 13
+    // Unified shared memory allocated on a oneAPI
+    // non-partitioned device.
+    //
+    // A call to the oneAPI runtime is required to determine the
+    // device type, the USM allocation type and the sycl context
+    // that it is bound to.
+    #define ARROW_DEVICE_ONEAPI 14
+    // GPU support for next-gen WebGPU standard
+    #define ARROW_DEVICE_WEBGPU 15
+    // Qualcomm Hexagon DSP
+    #define ARROW_DEVICE_HEXAGON 16
+
+    struct ArrowDeviceArray {
+      struct ArrowArray array;
+      int64_t device_id;
+      ArrowDeviceType device_type;
+      void* sync_event;
+
+      // reserved bytes for future expansion
+      int64_t reserved[3];
+    };
+
+    #endif  // ARROW_C_DEVICE_DATA_INTERFACE
+
+.. note::
+   The canonical guard ``ARROW_C_DEVICE_DATA_INTERFACE`` is meant to avoid
+   duplicate definitions if two projects copy the definitions in their own
+   headers, and a third-party project includes from these two projects. It
+   is therefore important that this guard is kept exactly as-is when these
+   definitions are copied.
+
+ArrowDeviceType
+---------------
+
+The ``ArrowDeviceType`` typedef is used to indicate what type of device the
+provided memory buffers were allocated on. This, in conjunction with the
+``device_id``, should be sufficient to reference the correct data buffers.
+
+We then use macros to define values for different device types. The provided
+macro values are compatible with the widely used `dlpack`_ ``DLDeviceType``
+definition values, using the same value for each as the equivalent
+``kDL<type>`` enum from dlpack.h. The list will be kept in sync with those
+equivalent enum values over time to ensure compatibility, rather than
+potentially diverging. To avoid the Arrow project having to be in the
+position of vetting new hardware devices, new additions should first be
+added to dlpack before we add a corresponding macro here.
+
+To ensure predictability with the ABI, we use macros instead of an ``enum``
+so the storage type is not compiler dependent.
+
+.. c:macro:: ARROW_DEVICE_CPU
+
+    CPU Device, equivalent to just using ``ArrowArray`` directly instead of
+    using ``ArrowDeviceArray``.
+
+.. c:macro:: ARROW_DEVICE_CUDA
+
+    A `CUDA`_ GPU Device. This could represent data allocated either with the
+    runtime library (``cudaMalloc``) or the device driver (``cuMemAlloc``).
+
+.. c:macro:: ARROW_DEVICE_CUDA_HOST
+
+    CPU memory that was pinned and page-locked by CUDA by using
+    ``cudaMallocHost`` or ``cuMemAllocHost``.
+
+.. c:macro:: ARROW_DEVICE_OPENCL
+
+    Data allocated on the device by using the `OpenCL (Open Computing Language)`_
+    framework.
+
+.. c:macro:: ARROW_DEVICE_VULKAN
+
+    Data allocated by the `Vulkan`_ framework and libraries.
+
+.. c:macro:: ARROW_DEVICE_METAL
+
+    Data on Apple GPU devices using the `Metal`_ framework and libraries.
+
+.. c:macro:: ARROW_DEVICE_VPI
+
+    Indicates usage of a Verilog simulator buffer.
+
+.. c:macro:: ARROW_DEVICE_ROCM
+
+    An AMD device using the `ROCm`_ stack.
+
+.. c:macro:: ARROW_DEVICE_ROCM_HOST
+
+    CPU memory pinned and page-locked allocated ``hipMallocHost``.
+
+.. c:macro:: ARROW_DEVICE_EXT_DEV
+
+    This value is an escape-hatch for devices to extend which aren't
+    currently represented otherwise. Producers would need to provide
+    additional information/context specific to the device if using
+    this device type. This is used to quickly test extension devices
+    and semantics can differ based on the implementation.
+
+.. c:macro:: ARROW_DEVICE_CUDA_MANAGED
+
+    CUDA managed/unified memory which is allocated by ``cudaMallocManaged``.
+
+.. c:macro:: ARROW_DEVICE_ONEAPI
+
+    Unified shared memory allocated on an Intel `oneAPI`_ non-partitioned
+    device. A call to the ``oneAPI`` runtime is required to determine
+    the specific device type, the USM allocation type and the sycl context
+    that it is bound to.
+
+.. c:macro:: ARROW_DEVICE_WEBGPU
+
+    GPU support for next-gen WebGPU standards
+
+.. c:macro:: ARROW_DEVICE_HEXAGON
+
+    Data allocated on a Qualcomm Hexagon DSP device.
+
+The ArrowDeviceArray structure
+------------------------------
+
+The ``ArrowDeviceArray`` structure embeds the C data ``ArrowArray`` structure
+and adds additional information necessary for consumers to use the data. It
+has the following fields:
+
+.. c:member:: struct ArrowArray ArrowDeviceArray.array
+
+    The allocated array data. The values in the ``void**`` buffers (along
+    with the buffers of any children) are what is allocated on the device.
+    The buffer values should be device pointers. The rest of the structure
+    should be accessible to the CPU.
+
+    The ``private_data`` and ``release`` callback of this structure should
+    contain any necessary information and structures related to freeing
+    the array according to the device it is allocated on, rather than
+    having a separate release callback and ``private_data`` pointer here.
+
+.. c:member:: int64_t ArrowDeviceArray.device_id
+
+    The device id to identify a specific device if multiple devices of this
+    type are on the system. The semantics of the id will be hardware dependent,
+    but we use an ``int64_t`` to future-proof the id as devices change over time.
+
+.. c:member:: ArrowDeviceType ArrowDeviceArray.device_type
+
+    The type of the device which can access the buffers in the array.
+
+.. c:member:: void* ArrowDeviceArray.sync_event
+
+    Optional. An event-like object to synchronize on if needed.
+
+    Many devices, like GPUs, are primarily asynchronous with respect to
+    CPU processing. As such in order to safely access memory, it is often

Review Comment:
   ```suggestion
       CPU processing. As such, in order to safely access device memory, it is often
   ```



##########
docs/source/format/CDeviceDataInterface.rst:
##########
@@ -0,0 +1,688 @@
+.. Licensed to the Apache Software Foundation (ASF) under one
+.. or more contributor license agreements.  See the NOTICE file
+.. distributed with this work for additional information
+.. regarding copyright ownership.  The ASF licenses this file
+.. to you under the Apache License, Version 2.0 (the
+.. "License"); you may not use this file except in compliance
+.. with the License.  You may obtain a copy of the License at
+
+..   http://www.apache.org/licenses/LICENSE-2.0
+
+.. Unless required by applicable law or agreed to in writing,
+.. software distributed under the License is distributed on an
+.. "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+.. KIND, either express or implied.  See the License for the
+.. specific language governing permissions and limitations
+.. under the License.
+
+.. highlight:: c
+
+.. _c-device-data-interface:
+
+=================================
+The Arrow C Device data interface
+=================================
+
+.. warning:: The Arrow C Device Data Interface should be considered experimental
+
+Rationale
+=========
+
+The current :ref:`C Data Interface <c-data-interface>`, and most
+implementations of it, make the assumption that all data buffers provided
+are CPU buffers. Since Apache Arrow is designed to be a universal in-memory
+format for representing tabular ("columnar") data, there will be the desire
+to leverage this data on non-CPU hardware such as GPUs. One example of such
+a case is the `RAPIDS cuDF library`_ which uses the Arrow memory format with
+CUDA for NVIDIA GPUs. Since copying data from host to device and back is
+expensive, the ideal would be to be able to leave the data on the device
+for as long as possible, even when passing it between runtimes and
+libraries.
+
+The Arrow C Device data interface builds on the existing C data interface
+by adding a very small, stable set of C definitions to it. These definitions
+are equivalents to the ``ArrowArray`` and ``ArrowArrayStream`` structures
+from the C Data Interface which add members to allow specifying the device
+type and pass necessary information to synchronize with the producer.
+For non-C/C++ languages and runtimes, translating the C definitions to
+corresponding C FFI declarations should be just as simple as with the
+current C data interface.
+
+Applications and libraries can then use Arrow schemas and Arrow formatted
+memory on non-CPU devices to exchange data just as easily as they do
+now with CPU data. This will enable leaving data on those devices longer
+and avoiding costly copies back and forth between the host and device
+just to leverage new libraries and runtimes.
+
+Goals
+-----
+
+* Expose an ABI-stable interface built on the existing C data interface.
+* Make it easy for third-party projects to implement support with little
+  initial investment.
+* Allow zero-copy sharing of Arrow formatted device memory between
+  independant runtimes and components running in the same process.
+* Avoid the need for one-to-one adaptation layers such as the
+  `CUDA Array Interface`_ for Python processes to pass CUDA data.
+* Enable integration without explicit dependencies (either at compile-time
+  or runtime) on the Arrow software project itself.
+
+The intent is for the Arrow C Device data interface to expand the reach
+of the current C data interface, allowing it to also become the standard
+low-level building block for columnar processing on devices like GPUs or
+FPGAs.
+
+Structure definitions
+=====================
+
+Because this is built on the C data interface, the C Device data interface
+uses the ``ArrowSchema`` and ``ArrowArray`` structures as defined in the
+:ref:`C data interface spec <c-data-interface-struct-defs>`. It then adds the
+following free-standing definitions. Like the rest of the Arrow project,
+they are available under the Apache License 2.0.
+
+.. code-block:: c
+
+    #ifndef ARROW_C_DEVICE_DATA_INTERFACE
+    #define ARROW_C_DEVICE_DATA_INTERFACE
+
+    // Device type for the allocated memory
+    typedef int32_t ArrowDeviceType;
+
+    // CPU device, same as using ArrowArray directly
+    #define ARROW_DEVICE_CPU 1
+    // CUDA GPU Device
+    #define ARROW_DEVICE_CUDA 2
+    // Pinned CUDA CPU memory by cudaMallocHost
+    #define ARROW_DEVICE_CUDA_HOST 3
+    // OpenCL Device
+    #define ARROW_DEVICE_OPENCL 4
+    // Vulkan buffer for next-gen graphics
+    #define ARROW_DEVICE_VULKAN 7
+    // Metal for Apple GPU
+    #define ARROW_DEVICE_METAL 8
+    // Verilog simulator buffer
+    #define ARROW_DEVICE_VPI 9
+    // ROCm GPUs for AMD GPUs
+    #define ARROW_DEVICE_ROCM 10
+    // Pinned ROCm CPU memory allocated by hipMallocHost
+    #define ARROW_DEVICE_ROCM_HOST 11
+    // Reserved for extension
+    //
+    // used to quickly test extension devices, semantics
+    // can differ based on implementation
+    #define ARROW_DEVICE_EXT_DEV 12
+    // CUDA managed/unified memory allocated by cudaMallocManaged
+    #define ARROW_DEVICE_CUDA_MANAGED 13
+    // Unified shared memory allocated on a oneAPI
+    // non-partitioned device.
+    //
+    // A call to the oneAPI runtime is required to determine the
+    // device type, the USM allocation type and the sycl context
+    // that it is bound to.
+    #define ARROW_DEVICE_ONEAPI 14
+    // GPU support for next-gen WebGPU standard
+    #define ARROW_DEVICE_WEBGPU 15
+    // Qualcomm Hexagon DSP
+    #define ARROW_DEVICE_HEXAGON 16
+
+    struct ArrowDeviceArray {
+      struct ArrowArray array;
+      int64_t device_id;
+      ArrowDeviceType device_type;
+      void* sync_event;
+
+      // reserved bytes for future expansion
+      int64_t reserved[3];
+    };
+
+    #endif  // ARROW_C_DEVICE_DATA_INTERFACE
+
+.. note::
+   The canonical guard ``ARROW_C_DEVICE_DATA_INTERFACE`` is meant to avoid
+   duplicate definitions if two projects copy the definitions in their own
+   headers, and a third-party project includes from these two projects. It
+   is therefore important that this guard is kept exactly as-is when these
+   definitions are copied.
+
+ArrowDeviceType
+---------------
+
+The ``ArrowDeviceType`` typedef is used to indicate what type of device the
+provided memory buffers were allocated on. This, in conjunction with the
+``device_id``, should be sufficient to reference the correct data buffers.
+
+We then use macros to define values for different device types. The provided
+macro values are compatible with the widely used `dlpack`_ ``DLDeviceType``
+definition values, using the same value for each as the equivalent
+``kDL<type>`` enum from dlpack.h. The list will be kept in sync with those

Review Comment:
   ```suggestion
   ``kDL<type>`` enum from ``dlpack.h``. The list will be kept in sync with those
   ```



##########
docs/source/format/CDeviceDataInterface.rst:
##########
@@ -0,0 +1,688 @@
+.. Licensed to the Apache Software Foundation (ASF) under one
+.. or more contributor license agreements.  See the NOTICE file
+.. distributed with this work for additional information
+.. regarding copyright ownership.  The ASF licenses this file
+.. to you under the Apache License, Version 2.0 (the
+.. "License"); you may not use this file except in compliance
+.. with the License.  You may obtain a copy of the License at
+
+..   http://www.apache.org/licenses/LICENSE-2.0
+
+.. Unless required by applicable law or agreed to in writing,
+.. software distributed under the License is distributed on an
+.. "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+.. KIND, either express or implied.  See the License for the
+.. specific language governing permissions and limitations
+.. under the License.
+
+.. highlight:: c
+
+.. _c-device-data-interface:
+
+=================================
+The Arrow C Device data interface
+=================================
+
+.. warning:: The Arrow C Device Data Interface should be considered experimental
+
+Rationale
+=========
+
+The current :ref:`C Data Interface <c-data-interface>`, and most
+implementations of it, make the assumption that all data buffers provided
+are CPU buffers. Since Apache Arrow is designed to be a universal in-memory
+format for representing tabular ("columnar") data, there will be the desire
+to leverage this data on non-CPU hardware such as GPUs. One example of such
+a case is the `RAPIDS cuDF library`_ which uses the Arrow memory format with
+CUDA for NVIDIA GPUs. Since copying data from host to device and back is
+expensive, the ideal would be to be able to leave the data on the device
+for as long as possible, even when passing it between runtimes and
+libraries.
+
+The Arrow C Device data interface builds on the existing C data interface
+by adding a very small, stable set of C definitions to it. These definitions
+are equivalents to the ``ArrowArray`` and ``ArrowArrayStream`` structures
+from the C Data Interface which add members to allow specifying the device
+type and pass necessary information to synchronize with the producer.
+For non-C/C++ languages and runtimes, translating the C definitions to
+corresponding C FFI declarations should be just as simple as with the
+current C data interface.
+
+Applications and libraries can then use Arrow schemas and Arrow formatted
+memory on non-CPU devices to exchange data just as easily as they do
+now with CPU data. This will enable leaving data on those devices longer
+and avoiding costly copies back and forth between the host and device
+just to leverage new libraries and runtimes.
+
+Goals
+-----
+
+* Expose an ABI-stable interface built on the existing C data interface.
+* Make it easy for third-party projects to implement support with little
+  initial investment.
+* Allow zero-copy sharing of Arrow formatted device memory between
+  independant runtimes and components running in the same process.
+* Avoid the need for one-to-one adaptation layers such as the
+  `CUDA Array Interface`_ for Python processes to pass CUDA data.
+* Enable integration without explicit dependencies (either at compile-time
+  or runtime) on the Arrow software project itself.
+
+The intent is for the Arrow C Device data interface to expand the reach
+of the current C data interface, allowing it to also become the standard
+low-level building block for columnar processing on devices like GPUs or
+FPGAs.
+
+Structure definitions
+=====================
+
+Because this is built on the C data interface, the C Device data interface
+uses the ``ArrowSchema`` and ``ArrowArray`` structures as defined in the
+:ref:`C data interface spec <c-data-interface-struct-defs>`. It then adds the
+following free-standing definitions. Like the rest of the Arrow project,
+they are available under the Apache License 2.0.
+
+.. code-block:: c
+
+    #ifndef ARROW_C_DEVICE_DATA_INTERFACE
+    #define ARROW_C_DEVICE_DATA_INTERFACE
+
+    // Device type for the allocated memory
+    typedef int32_t ArrowDeviceType;
+
+    // CPU device, same as using ArrowArray directly
+    #define ARROW_DEVICE_CPU 1
+    // CUDA GPU Device
+    #define ARROW_DEVICE_CUDA 2
+    // Pinned CUDA CPU memory by cudaMallocHost
+    #define ARROW_DEVICE_CUDA_HOST 3
+    // OpenCL Device
+    #define ARROW_DEVICE_OPENCL 4
+    // Vulkan buffer for next-gen graphics
+    #define ARROW_DEVICE_VULKAN 7
+    // Metal for Apple GPU
+    #define ARROW_DEVICE_METAL 8
+    // Verilog simulator buffer
+    #define ARROW_DEVICE_VPI 9
+    // ROCm GPUs for AMD GPUs
+    #define ARROW_DEVICE_ROCM 10
+    // Pinned ROCm CPU memory allocated by hipMallocHost
+    #define ARROW_DEVICE_ROCM_HOST 11
+    // Reserved for extension
+    //
+    // used to quickly test extension devices, semantics
+    // can differ based on implementation
+    #define ARROW_DEVICE_EXT_DEV 12
+    // CUDA managed/unified memory allocated by cudaMallocManaged
+    #define ARROW_DEVICE_CUDA_MANAGED 13
+    // Unified shared memory allocated on a oneAPI
+    // non-partitioned device.
+    //
+    // A call to the oneAPI runtime is required to determine the
+    // device type, the USM allocation type and the sycl context
+    // that it is bound to.
+    #define ARROW_DEVICE_ONEAPI 14
+    // GPU support for next-gen WebGPU standard
+    #define ARROW_DEVICE_WEBGPU 15
+    // Qualcomm Hexagon DSP
+    #define ARROW_DEVICE_HEXAGON 16
+
+    struct ArrowDeviceArray {
+      struct ArrowArray array;
+      int64_t device_id;
+      ArrowDeviceType device_type;
+      void* sync_event;
+
+      // reserved bytes for future expansion
+      int64_t reserved[3];
+    };
+
+    #endif  // ARROW_C_DEVICE_DATA_INTERFACE
+
+.. note::
+   The canonical guard ``ARROW_C_DEVICE_DATA_INTERFACE`` is meant to avoid
+   duplicate definitions if two projects copy the definitions in their own
+   headers, and a third-party project includes from these two projects. It
+   is therefore important that this guard is kept exactly as-is when these
+   definitions are copied.
+
+ArrowDeviceType
+---------------
+
+The ``ArrowDeviceType`` typedef is used to indicate what type of device the
+provided memory buffers were allocated on. This, in conjunction with the
+``device_id``, should be sufficient to reference the correct data buffers.
+
+We then use macros to define values for different device types. The provided
+macro values are compatible with the widely used `dlpack`_ ``DLDeviceType``
+definition values, using the same value for each as the equivalent
+``kDL<type>`` enum from dlpack.h. The list will be kept in sync with those
+equivalent enum values over time to ensure compatibility, rather than
+potentially diverging. To avoid the Arrow project having to be in the
+position of vetting new hardware devices, new additions should first be
+added to dlpack before we add a corresponding macro here.
+
+To ensure predictability with the ABI, we use macros instead of an ``enum``
+so the storage type is not compiler dependent.
+
+.. c:macro:: ARROW_DEVICE_CPU
+
+    CPU Device, equivalent to just using ``ArrowArray`` directly instead of
+    using ``ArrowDeviceArray``.
+
+.. c:macro:: ARROW_DEVICE_CUDA
+
+    A `CUDA`_ GPU Device. This could represent data allocated either with the
+    runtime library (``cudaMalloc``) or the device driver (``cuMemAlloc``).
+
+.. c:macro:: ARROW_DEVICE_CUDA_HOST
+
+    CPU memory that was pinned and page-locked by CUDA by using
+    ``cudaMallocHost`` or ``cuMemAllocHost``.
+
+.. c:macro:: ARROW_DEVICE_OPENCL
+
+    Data allocated on the device by using the `OpenCL (Open Computing Language)`_
+    framework.
+
+.. c:macro:: ARROW_DEVICE_VULKAN
+
+    Data allocated by the `Vulkan`_ framework and libraries.
+
+.. c:macro:: ARROW_DEVICE_METAL
+
+    Data on Apple GPU devices using the `Metal`_ framework and libraries.
+
+.. c:macro:: ARROW_DEVICE_VPI
+
+    Indicates usage of a Verilog simulator buffer.
+
+.. c:macro:: ARROW_DEVICE_ROCM
+
+    An AMD device using the `ROCm`_ stack.
+
+.. c:macro:: ARROW_DEVICE_ROCM_HOST
+
+    CPU memory pinned and page-locked allocated ``hipMallocHost``.
+
+.. c:macro:: ARROW_DEVICE_EXT_DEV
+
+    This value is an escape-hatch for devices to extend which aren't
+    currently represented otherwise. Producers would need to provide
+    additional information/context specific to the device if using
+    this device type. This is used to quickly test extension devices
+    and semantics can differ based on the implementation.
+
+.. c:macro:: ARROW_DEVICE_CUDA_MANAGED
+
+    CUDA managed/unified memory which is allocated by ``cudaMallocManaged``.
+
+.. c:macro:: ARROW_DEVICE_ONEAPI
+
+    Unified shared memory allocated on an Intel `oneAPI`_ non-partitioned
+    device. A call to the ``oneAPI`` runtime is required to determine
+    the specific device type, the USM allocation type and the sycl context
+    that it is bound to.
+
+.. c:macro:: ARROW_DEVICE_WEBGPU
+
+    GPU support for next-gen WebGPU standards
+
+.. c:macro:: ARROW_DEVICE_HEXAGON
+
+    Data allocated on a Qualcomm Hexagon DSP device.
+
+The ArrowDeviceArray structure
+------------------------------
+
+The ``ArrowDeviceArray`` structure embeds the C data ``ArrowArray`` structure
+and adds additional information necessary for consumers to use the data. It
+has the following fields:
+
+.. c:member:: struct ArrowArray ArrowDeviceArray.array
+
+    The allocated array data. The values in the ``void**`` buffers (along
+    with the buffers of any children) are what is allocated on the device.
+    The buffer values should be device pointers. The rest of the structure
+    should be accessible to the CPU.
+
+    The ``private_data`` and ``release`` callback of this structure should
+    contain any necessary information and structures related to freeing
+    the array according to the device it is allocated on, rather than
+    having a separate release callback and ``private_data`` pointer here.
+
+.. c:member:: int64_t ArrowDeviceArray.device_id
+
+    The device id to identify a specific device if multiple devices of this
+    type are on the system. The semantics of the id will be hardware dependent,
+    but we use an ``int64_t`` to future-proof the id as devices change over time.
+
+.. c:member:: ArrowDeviceType ArrowDeviceArray.device_type
+
+    The type of the device which can access the buffers in the array.
+
+.. c:member:: void* ArrowDeviceArray.sync_event
+
+    Optional. An event-like object to synchronize on if needed.
+
+    Many devices, like GPUs, are primarily asynchronous with respect to
+    CPU processing. As such in order to safely access memory, it is often
+    necessary to have an object to synchronize processing with. Since
+    different devices will use different types to specify this, we use a
+    void* which can be coerced into a pointer to whatever the device

Review Comment:
   ```suggestion
       ``void*`` which can be coerced into a pointer to whatever the device
   ```



##########
docs/source/format/CDeviceDataInterface.rst:
##########
@@ -0,0 +1,688 @@
+.. Licensed to the Apache Software Foundation (ASF) under one
+.. or more contributor license agreements.  See the NOTICE file
+.. distributed with this work for additional information
+.. regarding copyright ownership.  The ASF licenses this file
+.. to you under the Apache License, Version 2.0 (the
+.. "License"); you may not use this file except in compliance
+.. with the License.  You may obtain a copy of the License at
+
+..   http://www.apache.org/licenses/LICENSE-2.0
+
+.. Unless required by applicable law or agreed to in writing,
+.. software distributed under the License is distributed on an
+.. "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+.. KIND, either express or implied.  See the License for the
+.. specific language governing permissions and limitations
+.. under the License.
+
+.. highlight:: c
+
+.. _c-device-data-interface:
+
+=================================
+The Arrow C Device data interface
+=================================
+
+.. warning:: The Arrow C Device Data Interface should be considered experimental
+
+Rationale
+=========
+
+The current :ref:`C Data Interface <c-data-interface>`, and most
+implementations of it, make the assumption that all data buffers provided
+are CPU buffers. Since Apache Arrow is designed to be a universal in-memory
+format for representing tabular ("columnar") data, there will be the desire
+to leverage this data on non-CPU hardware such as GPUs. One example of such
+a case is the `RAPIDS cuDF library`_ which uses the Arrow memory format with
+CUDA for NVIDIA GPUs. Since copying data from host to device and back is
+expensive, the ideal would be to be able to leave the data on the device
+for as long as possible, even when passing it between runtimes and
+libraries.
+
+The Arrow C Device data interface builds on the existing C data interface
+by adding a very small, stable set of C definitions to it. These definitions
+are equivalents to the ``ArrowArray`` and ``ArrowArrayStream`` structures
+from the C Data Interface which add members to allow specifying the device
+type and pass necessary information to synchronize with the producer.
+For non-C/C++ languages and runtimes, translating the C definitions to
+corresponding C FFI declarations should be just as simple as with the
+current C data interface.
+
+Applications and libraries can then use Arrow schemas and Arrow formatted
+memory on non-CPU devices to exchange data just as easily as they do
+now with CPU data. This will enable leaving data on those devices longer
+and avoiding costly copies back and forth between the host and device
+just to leverage new libraries and runtimes.
+
+Goals
+-----
+
+* Expose an ABI-stable interface built on the existing C data interface.
+* Make it easy for third-party projects to implement support with little
+  initial investment.
+* Allow zero-copy sharing of Arrow formatted device memory between
+  independant runtimes and components running in the same process.
+* Avoid the need for one-to-one adaptation layers such as the
+  `CUDA Array Interface`_ for Python processes to pass CUDA data.
+* Enable integration without explicit dependencies (either at compile-time
+  or runtime) on the Arrow software project itself.
+
+The intent is for the Arrow C Device data interface to expand the reach
+of the current C data interface, allowing it to also become the standard
+low-level building block for columnar processing on devices like GPUs or
+FPGAs.
+
+Structure definitions
+=====================
+
+Because this is built on the C data interface, the C Device data interface
+uses the ``ArrowSchema`` and ``ArrowArray`` structures as defined in the
+:ref:`C data interface spec <c-data-interface-struct-defs>`. It then adds the
+following free-standing definitions. Like the rest of the Arrow project,
+they are available under the Apache License 2.0.
+
+.. code-block:: c
+
+    #ifndef ARROW_C_DEVICE_DATA_INTERFACE
+    #define ARROW_C_DEVICE_DATA_INTERFACE
+
+    // Device type for the allocated memory
+    typedef int32_t ArrowDeviceType;
+
+    // CPU device, same as using ArrowArray directly
+    #define ARROW_DEVICE_CPU 1
+    // CUDA GPU Device
+    #define ARROW_DEVICE_CUDA 2
+    // Pinned CUDA CPU memory by cudaMallocHost
+    #define ARROW_DEVICE_CUDA_HOST 3
+    // OpenCL Device
+    #define ARROW_DEVICE_OPENCL 4
+    // Vulkan buffer for next-gen graphics
+    #define ARROW_DEVICE_VULKAN 7
+    // Metal for Apple GPU
+    #define ARROW_DEVICE_METAL 8
+    // Verilog simulator buffer
+    #define ARROW_DEVICE_VPI 9
+    // ROCm GPUs for AMD GPUs
+    #define ARROW_DEVICE_ROCM 10
+    // Pinned ROCm CPU memory allocated by hipMallocHost
+    #define ARROW_DEVICE_ROCM_HOST 11
+    // Reserved for extension
+    //
+    // used to quickly test extension devices, semantics
+    // can differ based on implementation
+    #define ARROW_DEVICE_EXT_DEV 12
+    // CUDA managed/unified memory allocated by cudaMallocManaged
+    #define ARROW_DEVICE_CUDA_MANAGED 13
+    // Unified shared memory allocated on a oneAPI
+    // non-partitioned device.
+    //
+    // A call to the oneAPI runtime is required to determine the
+    // device type, the USM allocation type and the sycl context
+    // that it is bound to.
+    #define ARROW_DEVICE_ONEAPI 14
+    // GPU support for next-gen WebGPU standard
+    #define ARROW_DEVICE_WEBGPU 15
+    // Qualcomm Hexagon DSP
+    #define ARROW_DEVICE_HEXAGON 16
+
+    struct ArrowDeviceArray {
+      struct ArrowArray array;
+      int64_t device_id;
+      ArrowDeviceType device_type;
+      void* sync_event;
+
+      // reserved bytes for future expansion
+      int64_t reserved[3];
+    };
+
+    #endif  // ARROW_C_DEVICE_DATA_INTERFACE
+
+.. note::
+   The canonical guard ``ARROW_C_DEVICE_DATA_INTERFACE`` is meant to avoid
+   duplicate definitions if two projects copy the definitions in their own
+   headers, and a third-party project includes from these two projects. It
+   is therefore important that this guard is kept exactly as-is when these
+   definitions are copied.
+
+ArrowDeviceType
+---------------
+
+The ``ArrowDeviceType`` typedef is used to indicate what type of device the
+provided memory buffers were allocated on. This, in conjunction with the
+``device_id``, should be sufficient to reference the correct data buffers.
+
+We then use macros to define values for different device types. The provided
+macro values are compatible with the widely used `dlpack`_ ``DLDeviceType``
+definition values, using the same value for each as the equivalent
+``kDL<type>`` enum from dlpack.h. The list will be kept in sync with those
+equivalent enum values over time to ensure compatibility, rather than
+potentially diverging. To avoid the Arrow project having to be in the
+position of vetting new hardware devices, new additions should first be
+added to dlpack before we add a corresponding macro here.
+
+To ensure predictability with the ABI, we use macros instead of an ``enum``
+so the storage type is not compiler dependent.
+
+.. c:macro:: ARROW_DEVICE_CPU
+
+    CPU Device, equivalent to just using ``ArrowArray`` directly instead of
+    using ``ArrowDeviceArray``.
+
+.. c:macro:: ARROW_DEVICE_CUDA
+
+    A `CUDA`_ GPU Device. This could represent data allocated either with the
+    runtime library (``cudaMalloc``) or the device driver (``cuMemAlloc``).
+
+.. c:macro:: ARROW_DEVICE_CUDA_HOST
+
+    CPU memory that was pinned and page-locked by CUDA by using
+    ``cudaMallocHost`` or ``cuMemAllocHost``.
+
+.. c:macro:: ARROW_DEVICE_OPENCL
+
+    Data allocated on the device by using the `OpenCL (Open Computing Language)`_
+    framework.
+
+.. c:macro:: ARROW_DEVICE_VULKAN
+
+    Data allocated by the `Vulkan`_ framework and libraries.
+
+.. c:macro:: ARROW_DEVICE_METAL
+
+    Data on Apple GPU devices using the `Metal`_ framework and libraries.
+
+.. c:macro:: ARROW_DEVICE_VPI
+
+    Indicates usage of a Verilog simulator buffer.
+
+.. c:macro:: ARROW_DEVICE_ROCM
+
+    An AMD device using the `ROCm`_ stack.
+
+.. c:macro:: ARROW_DEVICE_ROCM_HOST
+
+    CPU memory pinned and page-locked allocated ``hipMallocHost``.
+
+.. c:macro:: ARROW_DEVICE_EXT_DEV
+
+    This value is an escape-hatch for devices to extend which aren't
+    currently represented otherwise. Producers would need to provide
+    additional information/context specific to the device if using
+    this device type. This is used to quickly test extension devices
+    and semantics can differ based on the implementation.
+
+.. c:macro:: ARROW_DEVICE_CUDA_MANAGED
+
+    CUDA managed/unified memory which is allocated by ``cudaMallocManaged``.
+
+.. c:macro:: ARROW_DEVICE_ONEAPI
+
+    Unified shared memory allocated on an Intel `oneAPI`_ non-partitioned
+    device. A call to the ``oneAPI`` runtime is required to determine
+    the specific device type, the USM allocation type and the sycl context
+    that it is bound to.
+
+.. c:macro:: ARROW_DEVICE_WEBGPU
+
+    GPU support for next-gen WebGPU standards
+
+.. c:macro:: ARROW_DEVICE_HEXAGON
+
+    Data allocated on a Qualcomm Hexagon DSP device.
+
+The ArrowDeviceArray structure
+------------------------------
+
+The ``ArrowDeviceArray`` structure embeds the C data ``ArrowArray`` structure
+and adds additional information necessary for consumers to use the data. It
+has the following fields:
+
+.. c:member:: struct ArrowArray ArrowDeviceArray.array
+
+    The allocated array data. The values in the ``void**`` buffers (along
+    with the buffers of any children) are what is allocated on the device.
+    The buffer values should be device pointers. The rest of the structure
+    should be accessible to the CPU.
+
+    The ``private_data`` and ``release`` callback of this structure should
+    contain any necessary information and structures related to freeing
+    the array according to the device it is allocated on, rather than
+    having a separate release callback and ``private_data`` pointer here.
+
+.. c:member:: int64_t ArrowDeviceArray.device_id
+
+    The device id to identify a specific device if multiple devices of this
+    type are on the system. The semantics of the id will be hardware dependent,
+    but we use an ``int64_t`` to future-proof the id as devices change over time.
+
+.. c:member:: ArrowDeviceType ArrowDeviceArray.device_type
+
+    The type of the device which can access the buffers in the array.
+
+.. c:member:: void* ArrowDeviceArray.sync_event
+
+    Optional. An event-like object to synchronize on if needed.
+
+    Many devices, like GPUs, are primarily asynchronous with respect to
+    CPU processing. As such in order to safely access memory, it is often
+    necessary to have an object to synchronize processing with. Since
+    different devices will use different types to specify this, we use a
+    void* which can be coerced into a pointer to whatever the device
+    appropriate type is.
+
+    If synchronization is not needed, this can be null. If this is non-null
+    then it MUST be used to call the appropriate sync method for the device
+    (e.g. ``cudaStreamWaitEvent`` or ``hipStreamWaitEvent``) before attempting
+    to access the memory in the buffers.
+
+    Expected types to coerce this ``void*`` to depending on the reported
+    device type:
+
+    * CUDA: ``cudaEvent_t*``
+    * ROCm: ``hipEvent_t*``
+    * OpenCL: ``cl_event*``
+    * Vulkan: ``VkEvent*``
+    * Metal: ``MTLEvent*``
+    * OneAPI: ``sycl::event*``
+
+    If an event is provided, then the producer MUST ensure that the exported
+    data is available on the device before the event is triggered. The 
+    consumer SHOULD wait on the event before trying to access the exported
+    data.
+
+.. c:member:: int64_t ArrowDeviceArray.reserved[3]
+
+    As non-CPU development expands, there may be a need to expand this
+    structure. In order to do so without potentially breaking ABI changes,
+    we reserve 24 bytes at the end of the object. This also has the added
+    benefit of bringing the total size of this structure to exactly 128
+    bytes (a power of 2) on 64-bit systems. These bytes MUST be zero'd
+    out after initialization by the producer in order to ensure safe 
+    evolution of the ABI in the future.
+
+.. note::
+    Rather than store the shape / types of the data alongside the
+    ``ArrowDeviceArray``, users should utilize the existing ``ArrowSchema``
+    structure to pass any data type and shape information.
+
+Synchronization event types
+---------------------------
+
+The table below lists the expected event types for each device type.
+If no event type is supported ("N/A"), then the ``sync_event`` member
+should always be null. 
+
+Remember that the event *CAN* be null if synchronization is not needed
+to access the data.
+
++---------------------------+--------------------+---------+
+| Device Type               | Actual Event Type  | Notes   |
++===========================+====================+=========+
+| ARROW_DEVICE_CPU          | N/A                |         |
++---------------------------+--------------------+---------+
+| ARROW_DEVICE_CUDA         | ``cudaEvent_t*``   |         |
++---------------------------+--------------------+---------+
+| ARROW_DEVICE_CUDA_HOST    | ``cudaEvent_t*``   |         |
++---------------------------+--------------------+---------+
+| ARROW_DEVICE_OPENCL       | ``cl_event*``      |         |
++---------------------------+--------------------+---------+
+| ARROW_DEVICE_VULKAN       | ``VkEvent*``       |         |
++---------------------------+--------------------+---------+
+| ARROW_DEVICE_METAL        | ``MTLEvent*``      |         |
++---------------------------+--------------------+---------+
+| ARROW_DEVICE_VPI          | N/A                | (1)     |
++---------------------------+--------------------+---------+
+| ARROW_DEVICE_ROCM         | ``hipEvent_t*``    |         |
++---------------------------+--------------------+---------+
+| ARROW_DEVICE_ROCM_HOST    | ``hipEvent_t*``    |         |
++---------------------------+--------------------+---------+
+| ARROW_DEVICE_EXT_DEV      |                    | (2)     |
++---------------------------+--------------------+---------+
+| ARROW_DEVICE_CUDA_MANAGED | ``cudaEvent_t*``   |         |
++---------------------------+--------------------+---------+
+| ARROW_DEVICE_ONEAPI       | ``sycl::event*``   |         |
++---------------------------+--------------------+---------+
+| ARROW_DEVICE_WEBGPU       | N/A                | (1)     |
++---------------------------+--------------------+---------+
+| ARROW_DEVICE_HEXAGON      | N/A                | (1)     |
++---------------------------+--------------------+---------+
+
+Notes:
+
+* \(1) Currently unknown if framework has an event type to support.
+* \(2) Extension Device has producer defined semantics and thus if
+       synchronization is needed for an extension device, the producer
+       should document the type.
+
+
+Semantics
+=========

Review Comment:
   Perhaps we should make clear somewhere that, amongst all the information exported by the C Device Data Interface, only the data buffers reside in device memory, all the rest is in CPU memory.



##########
docs/source/format/CDeviceDataInterface.rst:
##########
@@ -0,0 +1,688 @@
+.. Licensed to the Apache Software Foundation (ASF) under one
+.. or more contributor license agreements.  See the NOTICE file
+.. distributed with this work for additional information
+.. regarding copyright ownership.  The ASF licenses this file
+.. to you under the Apache License, Version 2.0 (the
+.. "License"); you may not use this file except in compliance
+.. with the License.  You may obtain a copy of the License at
+
+..   http://www.apache.org/licenses/LICENSE-2.0
+
+.. Unless required by applicable law or agreed to in writing,
+.. software distributed under the License is distributed on an
+.. "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+.. KIND, either express or implied.  See the License for the
+.. specific language governing permissions and limitations
+.. under the License.
+
+.. highlight:: c
+
+.. _c-device-data-interface:
+
+=================================
+The Arrow C Device data interface
+=================================
+
+.. warning:: The Arrow C Device Data Interface should be considered experimental
+
+Rationale
+=========
+
+The current :ref:`C Data Interface <c-data-interface>`, and most
+implementations of it, make the assumption that all data buffers provided
+are CPU buffers. Since Apache Arrow is designed to be a universal in-memory
+format for representing tabular ("columnar") data, there will be the desire
+to leverage this data on non-CPU hardware such as GPUs. One example of such
+a case is the `RAPIDS cuDF library`_ which uses the Arrow memory format with
+CUDA for NVIDIA GPUs. Since copying data from host to device and back is
+expensive, the ideal would be to be able to leave the data on the device
+for as long as possible, even when passing it between runtimes and
+libraries.
+
+The Arrow C Device data interface builds on the existing C data interface
+by adding a very small, stable set of C definitions to it. These definitions
+are equivalents to the ``ArrowArray`` and ``ArrowArrayStream`` structures
+from the C Data Interface which add members to allow specifying the device
+type and pass necessary information to synchronize with the producer.
+For non-C/C++ languages and runtimes, translating the C definitions to
+corresponding C FFI declarations should be just as simple as with the
+current C data interface.
+
+Applications and libraries can then use Arrow schemas and Arrow formatted
+memory on non-CPU devices to exchange data just as easily as they do
+now with CPU data. This will enable leaving data on those devices longer
+and avoiding costly copies back and forth between the host and device
+just to leverage new libraries and runtimes.
+
+Goals
+-----
+
+* Expose an ABI-stable interface built on the existing C data interface.
+* Make it easy for third-party projects to implement support with little
+  initial investment.
+* Allow zero-copy sharing of Arrow formatted device memory between
+  independant runtimes and components running in the same process.
+* Avoid the need for one-to-one adaptation layers such as the
+  `CUDA Array Interface`_ for Python processes to pass CUDA data.
+* Enable integration without explicit dependencies (either at compile-time
+  or runtime) on the Arrow software project itself.
+
+The intent is for the Arrow C Device data interface to expand the reach
+of the current C data interface, allowing it to also become the standard
+low-level building block for columnar processing on devices like GPUs or
+FPGAs.
+
+Structure definitions
+=====================
+
+Because this is built on the C data interface, the C Device data interface
+uses the ``ArrowSchema`` and ``ArrowArray`` structures as defined in the
+:ref:`C data interface spec <c-data-interface-struct-defs>`. It then adds the
+following free-standing definitions. Like the rest of the Arrow project,
+they are available under the Apache License 2.0.
+
+.. code-block:: c
+
+    #ifndef ARROW_C_DEVICE_DATA_INTERFACE
+    #define ARROW_C_DEVICE_DATA_INTERFACE
+
+    // Device type for the allocated memory
+    typedef int32_t ArrowDeviceType;
+
+    // CPU device, same as using ArrowArray directly
+    #define ARROW_DEVICE_CPU 1
+    // CUDA GPU Device
+    #define ARROW_DEVICE_CUDA 2
+    // Pinned CUDA CPU memory by cudaMallocHost
+    #define ARROW_DEVICE_CUDA_HOST 3
+    // OpenCL Device
+    #define ARROW_DEVICE_OPENCL 4
+    // Vulkan buffer for next-gen graphics
+    #define ARROW_DEVICE_VULKAN 7
+    // Metal for Apple GPU
+    #define ARROW_DEVICE_METAL 8
+    // Verilog simulator buffer
+    #define ARROW_DEVICE_VPI 9
+    // ROCm GPUs for AMD GPUs
+    #define ARROW_DEVICE_ROCM 10
+    // Pinned ROCm CPU memory allocated by hipMallocHost
+    #define ARROW_DEVICE_ROCM_HOST 11
+    // Reserved for extension
+    //
+    // used to quickly test extension devices, semantics
+    // can differ based on implementation
+    #define ARROW_DEVICE_EXT_DEV 12
+    // CUDA managed/unified memory allocated by cudaMallocManaged
+    #define ARROW_DEVICE_CUDA_MANAGED 13
+    // Unified shared memory allocated on a oneAPI
+    // non-partitioned device.
+    //
+    // A call to the oneAPI runtime is required to determine the
+    // device type, the USM allocation type and the sycl context
+    // that it is bound to.
+    #define ARROW_DEVICE_ONEAPI 14
+    // GPU support for next-gen WebGPU standard
+    #define ARROW_DEVICE_WEBGPU 15
+    // Qualcomm Hexagon DSP
+    #define ARROW_DEVICE_HEXAGON 16
+
+    struct ArrowDeviceArray {
+      struct ArrowArray array;
+      int64_t device_id;
+      ArrowDeviceType device_type;
+      void* sync_event;
+
+      // reserved bytes for future expansion
+      int64_t reserved[3];
+    };
+
+    #endif  // ARROW_C_DEVICE_DATA_INTERFACE
+
+.. note::
+   The canonical guard ``ARROW_C_DEVICE_DATA_INTERFACE`` is meant to avoid
+   duplicate definitions if two projects copy the definitions in their own
+   headers, and a third-party project includes from these two projects. It
+   is therefore important that this guard is kept exactly as-is when these
+   definitions are copied.
+
+ArrowDeviceType
+---------------
+
+The ``ArrowDeviceType`` typedef is used to indicate what type of device the
+provided memory buffers were allocated on. This, in conjunction with the
+``device_id``, should be sufficient to reference the correct data buffers.
+
+We then use macros to define values for different device types. The provided
+macro values are compatible with the widely used `dlpack`_ ``DLDeviceType``
+definition values, using the same value for each as the equivalent
+``kDL<type>`` enum from dlpack.h. The list will be kept in sync with those
+equivalent enum values over time to ensure compatibility, rather than
+potentially diverging. To avoid the Arrow project having to be in the
+position of vetting new hardware devices, new additions should first be
+added to dlpack before we add a corresponding macro here.
+
+To ensure predictability with the ABI, we use macros instead of an ``enum``
+so the storage type is not compiler dependent.
+
+.. c:macro:: ARROW_DEVICE_CPU
+
+    CPU Device, equivalent to just using ``ArrowArray`` directly instead of
+    using ``ArrowDeviceArray``.
+
+.. c:macro:: ARROW_DEVICE_CUDA
+
+    A `CUDA`_ GPU Device. This could represent data allocated either with the
+    runtime library (``cudaMalloc``) or the device driver (``cuMemAlloc``).
+
+.. c:macro:: ARROW_DEVICE_CUDA_HOST
+
+    CPU memory that was pinned and page-locked by CUDA by using
+    ``cudaMallocHost`` or ``cuMemAllocHost``.
+
+.. c:macro:: ARROW_DEVICE_OPENCL
+
+    Data allocated on the device by using the `OpenCL (Open Computing Language)`_
+    framework.
+
+.. c:macro:: ARROW_DEVICE_VULKAN
+
+    Data allocated by the `Vulkan`_ framework and libraries.
+
+.. c:macro:: ARROW_DEVICE_METAL
+
+    Data on Apple GPU devices using the `Metal`_ framework and libraries.
+
+.. c:macro:: ARROW_DEVICE_VPI
+
+    Indicates usage of a Verilog simulator buffer.
+
+.. c:macro:: ARROW_DEVICE_ROCM
+
+    An AMD device using the `ROCm`_ stack.
+
+.. c:macro:: ARROW_DEVICE_ROCM_HOST
+
+    CPU memory pinned and page-locked allocated ``hipMallocHost``.
+
+.. c:macro:: ARROW_DEVICE_EXT_DEV
+
+    This value is an escape-hatch for devices to extend which aren't
+    currently represented otherwise. Producers would need to provide
+    additional information/context specific to the device if using
+    this device type. This is used to quickly test extension devices
+    and semantics can differ based on the implementation.
+
+.. c:macro:: ARROW_DEVICE_CUDA_MANAGED
+
+    CUDA managed/unified memory which is allocated by ``cudaMallocManaged``.
+
+.. c:macro:: ARROW_DEVICE_ONEAPI
+
+    Unified shared memory allocated on an Intel `oneAPI`_ non-partitioned
+    device. A call to the ``oneAPI`` runtime is required to determine
+    the specific device type, the USM allocation type and the sycl context
+    that it is bound to.
+
+.. c:macro:: ARROW_DEVICE_WEBGPU
+
+    GPU support for next-gen WebGPU standards
+
+.. c:macro:: ARROW_DEVICE_HEXAGON
+
+    Data allocated on a Qualcomm Hexagon DSP device.
+
+The ArrowDeviceArray structure
+------------------------------
+
+The ``ArrowDeviceArray`` structure embeds the C data ``ArrowArray`` structure
+and adds additional information necessary for consumers to use the data. It
+has the following fields:
+
+.. c:member:: struct ArrowArray ArrowDeviceArray.array
+
+    The allocated array data. The values in the ``void**`` buffers (along
+    with the buffers of any children) are what is allocated on the device.
+    The buffer values should be device pointers. The rest of the structure
+    should be accessible to the CPU.
+
+    The ``private_data`` and ``release`` callback of this structure should
+    contain any necessary information and structures related to freeing
+    the array according to the device it is allocated on, rather than
+    having a separate release callback and ``private_data`` pointer here.
+
+.. c:member:: int64_t ArrowDeviceArray.device_id
+
+    The device id to identify a specific device if multiple devices of this
+    type are on the system. The semantics of the id will be hardware dependent,
+    but we use an ``int64_t`` to future-proof the id as devices change over time.
+
+.. c:member:: ArrowDeviceType ArrowDeviceArray.device_type
+
+    The type of the device which can access the buffers in the array.
+
+.. c:member:: void* ArrowDeviceArray.sync_event
+
+    Optional. An event-like object to synchronize on if needed.
+
+    Many devices, like GPUs, are primarily asynchronous with respect to
+    CPU processing. As such in order to safely access memory, it is often
+    necessary to have an object to synchronize processing with. Since
+    different devices will use different types to specify this, we use a
+    void* which can be coerced into a pointer to whatever the device
+    appropriate type is.
+
+    If synchronization is not needed, this can be null. If this is non-null
+    then it MUST be used to call the appropriate sync method for the device
+    (e.g. ``cudaStreamWaitEvent`` or ``hipStreamWaitEvent``) before attempting
+    to access the memory in the buffers.
+
+    Expected types to coerce this ``void*`` to depending on the reported
+    device type:
+
+    * CUDA: ``cudaEvent_t*``
+    * ROCm: ``hipEvent_t*``
+    * OpenCL: ``cl_event*``
+    * Vulkan: ``VkEvent*``
+    * Metal: ``MTLEvent*``
+    * OneAPI: ``sycl::event*``
+
+    If an event is provided, then the producer MUST ensure that the exported
+    data is available on the device before the event is triggered. The 
+    consumer SHOULD wait on the event before trying to access the exported
+    data.
+
+.. c:member:: int64_t ArrowDeviceArray.reserved[3]
+
+    As non-CPU development expands, there may be a need to expand this
+    structure. In order to do so without potentially breaking ABI changes,
+    we reserve 24 bytes at the end of the object. This also has the added
+    benefit of bringing the total size of this structure to exactly 128
+    bytes (a power of 2) on 64-bit systems. These bytes MUST be zero'd
+    out after initialization by the producer in order to ensure safe 
+    evolution of the ABI in the future.
+
+.. note::
+    Rather than store the shape / types of the data alongside the
+    ``ArrowDeviceArray``, users should utilize the existing ``ArrowSchema``
+    structure to pass any data type and shape information.
+
+Synchronization event types
+---------------------------
+
+The table below lists the expected event types for each device type.
+If no event type is supported ("N/A"), then the ``sync_event`` member
+should always be null. 
+
+Remember that the event *CAN* be null if synchronization is not needed
+to access the data.
+
++---------------------------+--------------------+---------+
+| Device Type               | Actual Event Type  | Notes   |
++===========================+====================+=========+
+| ARROW_DEVICE_CPU          | N/A                |         |
++---------------------------+--------------------+---------+
+| ARROW_DEVICE_CUDA         | ``cudaEvent_t*``   |         |
++---------------------------+--------------------+---------+
+| ARROW_DEVICE_CUDA_HOST    | ``cudaEvent_t*``   |         |
++---------------------------+--------------------+---------+
+| ARROW_DEVICE_OPENCL       | ``cl_event*``      |         |
++---------------------------+--------------------+---------+
+| ARROW_DEVICE_VULKAN       | ``VkEvent*``       |         |
++---------------------------+--------------------+---------+
+| ARROW_DEVICE_METAL        | ``MTLEvent*``      |         |
++---------------------------+--------------------+---------+
+| ARROW_DEVICE_VPI          | N/A                | (1)     |
++---------------------------+--------------------+---------+
+| ARROW_DEVICE_ROCM         | ``hipEvent_t*``    |         |
++---------------------------+--------------------+---------+
+| ARROW_DEVICE_ROCM_HOST    | ``hipEvent_t*``    |         |
++---------------------------+--------------------+---------+
+| ARROW_DEVICE_EXT_DEV      |                    | (2)     |
++---------------------------+--------------------+---------+
+| ARROW_DEVICE_CUDA_MANAGED | ``cudaEvent_t*``   |         |
++---------------------------+--------------------+---------+
+| ARROW_DEVICE_ONEAPI       | ``sycl::event*``   |         |
++---------------------------+--------------------+---------+
+| ARROW_DEVICE_WEBGPU       | N/A                | (1)     |
++---------------------------+--------------------+---------+
+| ARROW_DEVICE_HEXAGON      | N/A                | (1)     |
++---------------------------+--------------------+---------+
+
+Notes:
+
+* \(1) Currently unknown if framework has an event type to support.
+* \(2) Extension Device has producer defined semantics and thus if
+       synchronization is needed for an extension device, the producer
+       should document the type.
+
+
+Semantics
+=========
+
+Memory management
+-----------------
+
+The ``ArrowDeviceArray`` structure contains an ``ArrowArray`` object which
+itself has :ref:`specific semantics <c-data-interface-semantics>` for releasing
+memory. The term *"base structure"* below refers to the ``ArrowDeviceArray``
+object that is passed directly between the producer and consumer -- not any
+child structure thereof.
+
+It is intended for the base structure to be stack- or heap-allocated by the
+*consumer*. In this case, the producer API should take a pointer to the
+consumer-allocated structure.
+
+However, any data pointed to by the struct MUST be allocated and maintained
+by the producer. This includes the ``sync_event`` member if it is not null,
+along with any pointers in the ``ArrowArray`` object as usual. Data lifetime
+is managed through the ``release`` callback of the ``ArrowArray`` member.
+
+For an ``ArrowDeviceArray``, the semantics of a released structure and the
+callback semantics are identical to those for
+:ref:`ArrowArray itself <c-data-interface-released>`. Any producer specific context
+information necessary for releasing the device data buffers, in addition to
+any allocated event, should be stored in the ``private_data`` member of
+the ``ArrowArray`` and managed by the ``release`` callback.
+
+Moving an array
+'''''''''''''''
+
+The consumer can *move* the ``ArrowDeviceArray`` structure by bitwise copying
+or shallow member-wise copying. Then it MUST mark the source structure released
+by setting the ``release`` member of the embedded ``ArrowArray`` structure to
+``NULL``, but *without* calling that release callback. This ensures that only
+one live copy of the struct is active at any given time and that lifetime is
+correctly communicated to the producer.
+
+As usual, the release callback will be called on the destination structure
+when it is not needed anymore.
+
+Record batches
+--------------
+As with the C data interface itself, a record batch can be trivially considered
+as an equivalent struct array. In this case the metadata of the top-level
+``ArrowSchema`` can be used for schema-level metadata of the record batch.
+
+Mutability
+----------
+
+Both the producer and the consumer SHOULD consider the exported data (that
+is, the data reachable on the device through the ``buffers`` member of
+the embedded ``ArrowArray``) to be immutable, as either party could otherwise
+see inconsistent data while the other is mutating it.
+
+Synchronization
+---------------
+
+If the ``sync_event`` member is non-NULL, the consumer should not attempt 
+to access or read the data until they have synchronized on that event. If
+the ``sync_event`` member is NULL, then it MUST be safe to access the data
+without any synchronization necessary on the part of the consumer.
+
+C producer example
+====================
+
+Exporting a simple ``int32`` device array
+-----------------------------------------
+
+Export a non-nullable ``int32`` type with empty metadata. An example of this
+can be seen in the :ref:`C data interface docs directly <c-data-interface-export-int32-schema>`.
+
+To export the data itself, we transfer ownership to the consumer through
+the release callback. This example will use CUDA, but the equivalent calls
+could be used for any device:
+
+.. code-block:: c
+
+    static void release_int32_device_array(struct ArrowArray* array) {
+        assert(array->n_buffers == 2);
+        // destroy the event
+        cudaEvent_t* ev_ptr = (cudaEvent_t*)(array->private_data);
+        cudaError_t status = cudaEventDestroy(*ev_ptr);
+        assert(status == cudaSuccess);
+        free(ev_ptr);
+
+        // free the buffers and the buffers array
+        status = cudaFree(array->buffers[1]);
+        assert(status == cudaSuccess);
+        free(array->buffers);
+
+        // mark released
+        array->release = NULL;
+    }
+
+    void export_int32_device_array(void* cudaAllocdPtr,
+                                   cudaStream_t stream,
+                                   int64_t length,
+                                   struct ArrowDeviceArray* array) {
+        // get device id
+        int device;
+        cudaError_t status;
+        status = cudaGetDevice(&device);
+        assert(status == cudaSuccess);
+
+        cudaEvent_t* ev_ptr = (cudaEvent_t*)malloc(sizeof(cudaEvent_t));
+        assert(ev_ptr != NULL);
+        status = cudaEventCreate(ev_ptr);
+        assert(status == cudaSuccess);
+
+        // record event on the stream, assuming that the passed in
+        // stream is where the work to produce the data will be processing.
+        status = cudaEventRecord(*ev_ptr, stream);
+        assert(status == cudaSuccess);
+
+        // initialize fields
+        *array = (struct ArrowDeviceArray) {
+            .array = (struct ArrowArray) {
+                .length = length,
+                .null_count = 0,
+                .offset = 0,
+                .n_buffers = 2,
+                .n_children = 0,
+                .children = NULL,
+                .dictionary = NULL,
+                // bookeeping
+                .release = &release_int32_device_array,
+                // store the event pointer as private data in the array
+                // so that we can access it in the release callback.
+                .private_data = (void*)(ev_ptr),
+            },
+            .device_id = (int64_t)(device),
+            .device_type = ARROW_DEVICE_CUDA,
+            // pass the event pointer to the consumer
+            .sync_event = (void*)(ev_ptr),
+        };
+
+        // allocate list of buffers
+        array->array.buffers = (const void**)malloc(sizeof(void*) * array->array.n_buffers);
+        assert(array->array.buffers != NULL);
+        array->array.buffers[0] = NULL;
+        array->array.buffers[1] = cudaAllocdPtr;
+    }
+
+    // calling the release callback should be done using the array member
+    // of the device array.
+    static void release_device_array_helper(struct ArrowDeviceArray* arr) {
+        arr->array.release(&arr->array);
+    }
+
+=======================
+Device Stream Interface
+=======================
+
+Like the :ref:`C stream interface <c-stream-interface>`, the C Device data
+interface also specifies a higher-level structure for easing communication
+of streaming data within a single process. Defining an ``ArrowDeviceArrayStream``
+structure.

Review Comment:
   Is there a leftover sentence here?
   ```suggestion
   of streaming data within a single process.
   ```



##########
docs/source/format/CDeviceDataInterface.rst:
##########
@@ -0,0 +1,628 @@
+.. Licensed to the Apache Software Foundation (ASF) under one
+.. or more contributor license agreements.  See the NOTICE file
+.. distributed with this work for additional information
+.. regarding copyright ownership.  The ASF licenses this file
+.. to you under the Apache License, Version 2.0 (the
+.. "License"); you may not use this file except in compliance
+.. with the License.  You may obtain a copy of the License at
+
+..   http://www.apache.org/licenses/LICENSE-2.0
+
+.. Unless required by applicable law or agreed to in writing,
+.. software distributed under the License is distributed on an
+.. "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+.. KIND, either express or implied.  See the License for the
+.. specific language governing permissions and limitations
+.. under the License.
+
+.. highlight:: c
+
+.. _c-device-data-interface:
+
+=================================
+The Arrow C Device data interface
+=================================
+
+.. note:: The Arrow C Device Data Interface should be considered experimental
+
+Rationale
+=========
+
+The current :ref:`C Data Interface <c-data-interface>`, and most
+implementations of it, make the assumption that all data buffers provided
+are CPU buffers. Since Apache Arrow is designed to be a universal in-memory
+format for representing tabular ("columnar") data, there will be the desire
+to leverage this data on non-CPU hardware such as GPUs. One example of such
+a case is the `RAPIDS cuDF library`_ which uses the Arrow memory format with
+CUDA for NVIDIA GPUs. Since copying data from host to device and back is
+expensive, the ideal would be to be able to leave the data on the device
+for as long as possible, even when passing it between runtimes and
+libraries.
+
+The Arrow C Device data interface builds on the existing C data interface
+by adding a very small, stable set of C definitions to it. These definitions
+are equivalents to the ``ArrowArray`` and ``ArrowArrayStream`` structures
+from the C Data Interface which add members to allow specifying the device
+type and pass necessary information to synchronize with the producer.
+For non-C/C++ languages and runtimes, translating the C definitions to
+corresponding C FFI declarations should be just as simple as with the
+current C data interface.
+
+Applications and libraries can then use Arrow schemas and Arrow formatted
+memory on non-CPU devices to exchange data just as easily as they do
+now with CPU data. This will enable leaving data on those devices longer
+and avoiding costly copies back and forth between the host and device
+just to leverage new libraries and runtimes.
+
+Goals
+-----
+
+* Expose an ABI-stable interface built on the existing C data interface.
+* Make it easy for third-party projects to implement support with little
+  initial investment.
+* Allow zero-copy sharing of Arrow formatted device memory between
+  independant runtimes and components running in the same process.
+* Avoid the need for one-to-one adaptation layers such as the
+  `CUDA Array Interface`_ for Python processes to pass CUDA data.
+* Enable integration without explicit dependencies (either at compile-time
+  or runtime) on the Arrow software project itself.
+
+The intent is for the Arrow C Device data interface to expand the reach
+of the current C data interface, allowing it to also become the standard
+low-level building block for columnar processing on devices like GPUs or
+FPGAs.
+
+Structure definitions
+=====================
+
+Because this is built on the C data interface, the C Device data interface
+uses the ``ArrowSchema`` and ``ArrowArray`` structures as defined in the
+:ref:`C data interface spec <c-data-interface-struct-defs>`. It then adds the
+following free-standing definitions. Like the rest of the Arrow project,
+they are available under the Apache License 2.0.
+
+.. code-block:: c
+
+    #ifndef ARROW_C_DEVICE_DATA_INTERFACE
+    #define ARROW_C_DEVICE_DATA_INTERFACE
+
+    // Device type for the allocated memory
+    typedef int32_t ArrowDeviceType;
+
+    // CPU device, same as using ArrowArray directly
+    #define ARROW_DEVICE_CPU 1
+    // CUDA GPU Device
+    #define ARROW_DEVICE_CUDA 2
+    // Pinned CUDA CPU memory by cudaMallocHost
+    #define ARROW_DEVICE_CUDA_HOST 3
+    // OpenCL Device
+    #define ARROW_DEVICE_OPENCL 4
+    // Vulkan buffer for next-gen graphics
+    #define ARROW_DEVICE_VULKAN 7
+    // Metal for Apple GPU
+    #define ARROW_DEVICE_METAL 8
+    // Verilog simulator buffer
+    #define ARROW_DEVICE_VPI 9
+    // ROCm GPUs for AMD GPUs
+    #define ARROW_DEVICE_ROCM 10
+    // Pinned ROCm CPU memory allocated by hipMallocHost
+    #define ARROW_DEVICE_ROCM_HOST 11
+    // Reserved for extension
+    //
+    // used to quickly test extension devices, semantics
+    // can differ based on implementation
+    #define ARROW_DEVICE_EXT_DEV 12
+    // CUDA managed/unified memory allocated by cudaMallocManaged
+    #define ARROW_DEVICE_CUDA_MANAGED 13
+    // Unified shared memory allocated on a oneAPI
+    // non-partitioned device.
+    //
+    // A call to the oneAPI runtime is required to determine the
+    // device type, the USM allocation type and the sycl context
+    // that it is bound to.
+    #define ARROW_DEVICE_ONEAPI 14
+    // GPU support for next-gen WebGPU standard
+    #define ARROW_DEVICE_WEBGPU 15
+    // Qualcomm Hexagon DSP
+    #define ARROW_DEVICE_HEXAGON 16
+
+    struct ArrowDeviceArray {
+      struct ArrowArray array;
+      int64_t device_id;
+      ArrowDeviceType device_type;
+      void* sync_event;
+
+      // reserved bytes for future expansion
+      int64_t reserved[3];
+    };
+
+    #endif  // ARROW_C_DEVICE_DATA_INTERFACE
+
+.. note::
+   The canonical guard ``ARROW_C_DEVICE_DATA_INTERFACE`` is meant to avoid
+   duplicate definitions if two projects copy the definitions in their own
+   headers, and a third-party project includes from these two projects. It
+   is therefore important that this guard is kept exactly as-is when these
+   definitions are copied.
+
+ArrowDeviceType
+---------------
+
+The ``ArrowDeviceType`` typedef is used to indicate what type of device the
+provided memory buffers were allocated on. This, in conjunction with the
+``device_id``, should be sufficient to reference the correct data buffers.
+
+We then use macros to define values for different device types. The provided
+macro values are compatible with the widely used `dlpack`_ ``DLDeviceType``
+definition values, using the same value for each as the equivalent
+``kDL<type>`` enum from dlpack.h. The list will be kept in sync with those
+equivalent enum values over time to ensure compatibility, rather than
+potentially diverging. To avoid the Arrow project having to be in the
+position of vetting new hardware devices, new additions should first be
+added to dlpack before we add a corresponding macro here.
+
+To ensure predictability with the ABI, we use macros instead of an ``enum``
+so the storage type is not compiler dependent.
+
+.. c:macro:: ARROW_DEVICE_CPU
+
+    CPU Device, equivalent to just using ``ArrowArray`` directly instead of
+    using ``ArrowDeviceArray``.
+
+.. c:macro:: ARROW_DEVICE_CUDA
+
+    A `CUDA`_ GPU Device. This could represent data allocated either with the
+    runtime library (``cudaMalloc``) or the device driver (``cuMemAlloc``).
+
+.. c:macro:: ARROW_DEVICE_CUDA_HOST
+
+    CPU memory that was pinned and page-locked by CUDA by using
+    ``cudaMallocHost`` or ``cuMemAllocHost``.
+
+.. c:macro:: ARROW_DEVICE_OPENCL
+
+    Data allocated on the device by using the `OpenCL (Open Computing Language)`_
+    framework.
+
+.. c:macro:: ARROW_DEVICE_VULKAN
+
+    Data allocated by the `Vulkan`_ framework and libraries.
+
+.. c:macro:: ARROW_DEVICE_METAL
+
+    Data on Apple GPU devices using the `Metal`_ framework and libraries.
+
+.. c:macro:: ARROW_DEVICE_VPI
+
+    Indicates usage of a Verilog simulator buffer.
+
+.. c:macro:: ARROW_DEVICE_ROCM
+
+    An AMD device using the `ROCm`_ stack.
+
+.. c:macro:: ARROW_DEVICE_ROCM_HOST
+
+    CPU memory pinned and page-locked allocated ``hipMallocHost``.
+
+.. c:macro:: ARROW_DEVICE_EXT_DEV
+
+    This value is an escape-hatch for devices to extend which aren't
+    currently represented otherwise. Producers would need to provide
+    additional information/context specific to the device if using
+    this device type. This is used to quickly test extension devices
+    and semantics can differ based on the implementation.
+
+.. c:macro:: ARROW_DEVICE_CUDA_MANAGED
+
+    CUDA managed/unified memory which is allocated by ``cudaMallocManaged``.
+
+.. c:macro:: ARROW_DEVICE_ONEAPI
+
+    Unified shared memory allocated on an Intel `oneAPI`_ non-partitioned
+    device. A call to the ``oneAPI`` runtime is required to determine
+    the specific device type, the USM allocation type and the sycl context
+    that it is bound to.
+
+.. c:macro:: ARROW_DEVICE_WEBGPU
+
+    GPU support for next-gen WebGPU standards
+
+.. c:macro:: ARROW_DEVICE_HEXAGON
+
+    Data allocated on a Qualcomm Hexagon DSP device.
+
+The ArrowDeviceArray structure
+------------------------------
+
+The ``ArrowDeviceArray`` structure embeds the C data ``ArrowArray`` structure
+and adds additional information necessary for consumers to use the data. It
+has the following fields:
+
+.. c:member:: struct ArrowArray ArrowDeviceArray.array
+
+    The allocated array data. The values in the ``void**`` buffers (along
+    with the buffers of any children) are what is allocated on the device.
+    The buffer values should be device pointers. The rest of the structure
+    should be accessible to the CPU.
+
+    The ``private_data`` and ``release`` callback of this structure should
+    contain any necessary information and structures related to freeing
+    the array according to the device it is allocated on, rather than
+    having a separate release callback and ``private_data`` pointer here.
+
+.. c:member:: int64_t ArrowDeviceArray.device_id
+
+    The device id to identify a specific device if multiple devices of this
+    type are on the system. The semantics of the id will be hardware dependent,
+    but we use an ``int64_t`` to future-proof the id as devices change over time.
+
+.. c:member:: ArrowDeviceType ArrowDeviceArray.device_type
+
+    The type of the device which can access the buffers in the array.
+
+.. c:member:: void* ArrowDeviceArray.sync_event
+
+    Optional. An event-like object to synchronize on if needed.
+
+    Many devices, like GPUs, are primarily asynchronous with respect to
+    CPU processing. As such in order to safely access memory, it is often
+    necessary to have an object to synchronize processing with. Since
+    different devices will use different types to specify this, we use a
+    void* which can be coerced into a pointer to whatever the device
+    appropriate type is.
+
+    If synchronization is not needed, this can be null. If this is non-null
+    then it MUST be used to call the appropriate sync method for the device
+    (e.g. ``cudaStreamWaitEvent`` or ``hipStreamWaitEvent``) before attempting
+    to access the memory in the buffers.
+
+    Expected types to coerce this ``void*`` to depending on the reported
+    device type:
+
+    * CUDA: ``cudaEvent_t*``
+    * ROCm: ``hipEvent_t*``
+    * OpenCL: ``cl_event*``
+    * Vulkan: ``VkEvent*``
+    * Metal: ``MTLEvent*``
+    * OneAPI: ``sycl::event*``
+
+    If an event is provided, then the producer MUST ensure that the event
+    is triggered/recorded at the end of the processing stream once the data
+    is considered available for use.
+
+
+.. c:member:: int64_t ArrowDeviceArray.reserved[3]
+
+    As non-CPU development expands, there may be a need to expand this
+    structure. In order to do so without potentially breaking ABI changes,
+    we reserve 24 bytes at the end of the object. This also has the added
+    benefit of bringing the total size of this structure to exactly 128
+    bytes (a power of 2) on 64-bit systems. These bytes should be zero'd
+    out after allocation in order to ensure safe evolution of the ABI in
+    the future.
+
+.. note::
+    Rather than store the shape / types of the data alongside the

Review Comment:
   Then you should clearly state the context of this note, i.e. a comparison with the Cuda Array Interface. Otherwise it's confusing.
   
   Perhaps you also want to move this at the end into a "Interoperability with other interchange formats" paragraph?



##########
docs/source/format/CDeviceDataInterface.rst:
##########
@@ -0,0 +1,628 @@
+.. Licensed to the Apache Software Foundation (ASF) under one
+.. or more contributor license agreements.  See the NOTICE file
+.. distributed with this work for additional information
+.. regarding copyright ownership.  The ASF licenses this file
+.. to you under the Apache License, Version 2.0 (the
+.. "License"); you may not use this file except in compliance
+.. with the License.  You may obtain a copy of the License at
+
+..   http://www.apache.org/licenses/LICENSE-2.0
+
+.. Unless required by applicable law or agreed to in writing,
+.. software distributed under the License is distributed on an
+.. "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+.. KIND, either express or implied.  See the License for the
+.. specific language governing permissions and limitations
+.. under the License.
+
+.. highlight:: c
+
+.. _c-device-data-interface:
+
+=================================
+The Arrow C Device data interface
+=================================
+
+.. note:: The Arrow C Device Data Interface should be considered experimental
+
+Rationale
+=========
+
+The current :ref:`C Data Interface <c-data-interface>`, and most
+implementations of it, make the assumption that all data buffers provided
+are CPU buffers. Since Apache Arrow is designed to be a universal in-memory
+format for representing tabular ("columnar") data, there will be the desire
+to leverage this data on non-CPU hardware such as GPUs. One example of such
+a case is the `RAPIDS cuDF library`_ which uses the Arrow memory format with
+CUDA for NVIDIA GPUs. Since copying data from host to device and back is
+expensive, the ideal would be to be able to leave the data on the device
+for as long as possible, even when passing it between runtimes and
+libraries.
+
+The Arrow C Device data interface builds on the existing C data interface
+by adding a very small, stable set of C definitions to it. These definitions
+are equivalents to the ``ArrowArray`` and ``ArrowArrayStream`` structures
+from the C Data Interface which add members to allow specifying the device
+type and pass necessary information to synchronize with the producer.
+For non-C/C++ languages and runtimes, translating the C definitions to
+corresponding C FFI declarations should be just as simple as with the
+current C data interface.
+
+Applications and libraries can then use Arrow schemas and Arrow formatted
+memory on non-CPU devices to exchange data just as easily as they do
+now with CPU data. This will enable leaving data on those devices longer
+and avoiding costly copies back and forth between the host and device
+just to leverage new libraries and runtimes.
+
+Goals
+-----
+
+* Expose an ABI-stable interface built on the existing C data interface.
+* Make it easy for third-party projects to implement support with little
+  initial investment.
+* Allow zero-copy sharing of Arrow formatted device memory between
+  independant runtimes and components running in the same process.
+* Avoid the need for one-to-one adaptation layers such as the
+  `CUDA Array Interface`_ for Python processes to pass CUDA data.
+* Enable integration without explicit dependencies (either at compile-time
+  or runtime) on the Arrow software project itself.
+
+The intent is for the Arrow C Device data interface to expand the reach
+of the current C data interface, allowing it to also become the standard
+low-level building block for columnar processing on devices like GPUs or
+FPGAs.
+
+Structure definitions
+=====================
+
+Because this is built on the C data interface, the C Device data interface
+uses the ``ArrowSchema`` and ``ArrowArray`` structures as defined in the
+:ref:`C data interface spec <c-data-interface-struct-defs>`. It then adds the
+following free-standing definitions. Like the rest of the Arrow project,
+they are available under the Apache License 2.0.
+
+.. code-block:: c
+
+    #ifndef ARROW_C_DEVICE_DATA_INTERFACE
+    #define ARROW_C_DEVICE_DATA_INTERFACE
+
+    // Device type for the allocated memory
+    typedef int32_t ArrowDeviceType;
+
+    // CPU device, same as using ArrowArray directly
+    #define ARROW_DEVICE_CPU 1
+    // CUDA GPU Device
+    #define ARROW_DEVICE_CUDA 2
+    // Pinned CUDA CPU memory by cudaMallocHost
+    #define ARROW_DEVICE_CUDA_HOST 3
+    // OpenCL Device
+    #define ARROW_DEVICE_OPENCL 4
+    // Vulkan buffer for next-gen graphics
+    #define ARROW_DEVICE_VULKAN 7
+    // Metal for Apple GPU
+    #define ARROW_DEVICE_METAL 8
+    // Verilog simulator buffer
+    #define ARROW_DEVICE_VPI 9
+    // ROCm GPUs for AMD GPUs
+    #define ARROW_DEVICE_ROCM 10
+    // Pinned ROCm CPU memory allocated by hipMallocHost
+    #define ARROW_DEVICE_ROCM_HOST 11
+    // Reserved for extension
+    //
+    // used to quickly test extension devices, semantics
+    // can differ based on implementation
+    #define ARROW_DEVICE_EXT_DEV 12
+    // CUDA managed/unified memory allocated by cudaMallocManaged
+    #define ARROW_DEVICE_CUDA_MANAGED 13
+    // Unified shared memory allocated on a oneAPI
+    // non-partitioned device.
+    //
+    // A call to the oneAPI runtime is required to determine the
+    // device type, the USM allocation type and the sycl context
+    // that it is bound to.
+    #define ARROW_DEVICE_ONEAPI 14
+    // GPU support for next-gen WebGPU standard
+    #define ARROW_DEVICE_WEBGPU 15
+    // Qualcomm Hexagon DSP
+    #define ARROW_DEVICE_HEXAGON 16
+
+    struct ArrowDeviceArray {
+      struct ArrowArray array;
+      int64_t device_id;
+      ArrowDeviceType device_type;
+      void* sync_event;
+
+      // reserved bytes for future expansion
+      int64_t reserved[3];
+    };
+
+    #endif  // ARROW_C_DEVICE_DATA_INTERFACE
+
+.. note::
+   The canonical guard ``ARROW_C_DEVICE_DATA_INTERFACE`` is meant to avoid
+   duplicate definitions if two projects copy the definitions in their own
+   headers, and a third-party project includes from these two projects. It
+   is therefore important that this guard is kept exactly as-is when these
+   definitions are copied.
+
+ArrowDeviceType
+---------------
+
+The ``ArrowDeviceType`` typedef is used to indicate what type of device the
+provided memory buffers were allocated on. This, in conjunction with the
+``device_id``, should be sufficient to reference the correct data buffers.
+
+We then use macros to define values for different device types. The provided
+macro values are compatible with the widely used `dlpack`_ ``DLDeviceType``
+definition values, using the same value for each as the equivalent
+``kDL<type>`` enum from dlpack.h. The list will be kept in sync with those
+equivalent enum values over time to ensure compatibility, rather than
+potentially diverging. To avoid the Arrow project having to be in the
+position of vetting new hardware devices, new additions should first be
+added to dlpack before we add a corresponding macro here.
+
+To ensure predictability with the ABI, we use macros instead of an ``enum``
+so the storage type is not compiler dependent.
+
+.. c:macro:: ARROW_DEVICE_CPU
+
+    CPU Device, equivalent to just using ``ArrowArray`` directly instead of
+    using ``ArrowDeviceArray``.
+
+.. c:macro:: ARROW_DEVICE_CUDA
+
+    A `CUDA`_ GPU Device. This could represent data allocated either with the
+    runtime library (``cudaMalloc``) or the device driver (``cuMemAlloc``).
+
+.. c:macro:: ARROW_DEVICE_CUDA_HOST
+
+    CPU memory that was pinned and page-locked by CUDA by using
+    ``cudaMallocHost`` or ``cuMemAllocHost``.
+
+.. c:macro:: ARROW_DEVICE_OPENCL
+
+    Data allocated on the device by using the `OpenCL (Open Computing Language)`_
+    framework.
+
+.. c:macro:: ARROW_DEVICE_VULKAN
+
+    Data allocated by the `Vulkan`_ framework and libraries.
+
+.. c:macro:: ARROW_DEVICE_METAL
+
+    Data on Apple GPU devices using the `Metal`_ framework and libraries.
+
+.. c:macro:: ARROW_DEVICE_VPI
+
+    Indicates usage of a Verilog simulator buffer.
+
+.. c:macro:: ARROW_DEVICE_ROCM
+
+    An AMD device using the `ROCm`_ stack.
+
+.. c:macro:: ARROW_DEVICE_ROCM_HOST
+
+    CPU memory pinned and page-locked allocated ``hipMallocHost``.
+
+.. c:macro:: ARROW_DEVICE_EXT_DEV
+
+    This value is an escape-hatch for devices to extend which aren't
+    currently represented otherwise. Producers would need to provide
+    additional information/context specific to the device if using
+    this device type. This is used to quickly test extension devices
+    and semantics can differ based on the implementation.
+
+.. c:macro:: ARROW_DEVICE_CUDA_MANAGED
+
+    CUDA managed/unified memory which is allocated by ``cudaMallocManaged``.
+
+.. c:macro:: ARROW_DEVICE_ONEAPI
+
+    Unified shared memory allocated on an Intel `oneAPI`_ non-partitioned
+    device. A call to the ``oneAPI`` runtime is required to determine
+    the specific device type, the USM allocation type and the sycl context
+    that it is bound to.
+
+.. c:macro:: ARROW_DEVICE_WEBGPU
+
+    GPU support for next-gen WebGPU standards
+
+.. c:macro:: ARROW_DEVICE_HEXAGON
+
+    Data allocated on a Qualcomm Hexagon DSP device.
+
+The ArrowDeviceArray structure
+------------------------------
+
+The ``ArrowDeviceArray`` structure embeds the C data ``ArrowArray`` structure
+and adds additional information necessary for consumers to use the data. It
+has the following fields:
+
+.. c:member:: struct ArrowArray ArrowDeviceArray.array
+
+    The allocated array data. The values in the ``void**`` buffers (along
+    with the buffers of any children) are what is allocated on the device.
+    The buffer values should be device pointers. The rest of the structure
+    should be accessible to the CPU.
+
+    The ``private_data`` and ``release`` callback of this structure should
+    contain any necessary information and structures related to freeing
+    the array according to the device it is allocated on, rather than
+    having a separate release callback and ``private_data`` pointer here.
+
+.. c:member:: int64_t ArrowDeviceArray.device_id
+
+    The device id to identify a specific device if multiple devices of this
+    type are on the system. The semantics of the id will be hardware dependent,
+    but we use an ``int64_t`` to future-proof the id as devices change over time.
+
+.. c:member:: ArrowDeviceType ArrowDeviceArray.device_type
+
+    The type of the device which can access the buffers in the array.
+
+.. c:member:: void* ArrowDeviceArray.sync_event
+
+    Optional. An event-like object to synchronize on if needed.
+
+    Many devices, like GPUs, are primarily asynchronous with respect to
+    CPU processing. As such in order to safely access memory, it is often
+    necessary to have an object to synchronize processing with. Since
+    different devices will use different types to specify this, we use a
+    void* which can be coerced into a pointer to whatever the device
+    appropriate type is.
+
+    If synchronization is not needed, this can be null. If this is non-null
+    then it MUST be used to call the appropriate sync method for the device
+    (e.g. ``cudaStreamWaitEvent`` or ``hipStreamWaitEvent``) before attempting
+    to access the memory in the buffers.
+
+    Expected types to coerce this ``void*`` to depending on the reported
+    device type:
+
+    * CUDA: ``cudaEvent_t*``
+    * ROCm: ``hipEvent_t*``
+    * OpenCL: ``cl_event*``
+    * Vulkan: ``VkEvent*``
+    * Metal: ``MTLEvent*``
+    * OneAPI: ``sycl::event*``
+
+    If an event is provided, then the producer MUST ensure that the event
+    is triggered/recorded at the end of the processing stream once the data
+    is considered available for use.
+
+
+.. c:member:: int64_t ArrowDeviceArray.reserved[3]
+
+    As non-CPU development expands, there may be a need to expand this
+    structure. In order to do so without potentially breaking ABI changes,
+    we reserve 24 bytes at the end of the object. This also has the added
+    benefit of bringing the total size of this structure to exactly 128
+    bytes (a power of 2) on 64-bit systems. These bytes should be zero'd
+    out after allocation in order to ensure safe evolution of the ABI in
+    the future.
+
+.. note::
+    Rather than store the shape / types of the data alongside the
+    ``ArrowDeviceArray``, users should utilize the existing ``ArrowSchema``
+    structure to pass any data type and shape information.
+
+Semantics
+=========
+
+Memory management
+-----------------
+
+The ``ArrowDeviceArray`` structure contains an ``ArrowArray`` object which
+itself has :ref:`specific semantics <c-data-interface-semantics>` for releasing
+memory. The term *"base structure"* below refers to the ``ArrowDeviceArray``
+object that is passed directly between the producer and consumer -- not any
+child structure thereof.
+
+It is intended for the base structure to be stack- or heap-allocated by the
+*consumer*. In this case, the producer API should take a pointer to the
+consumer-allocated structure.
+
+However, any data pointed to by the struct MUST be allocated and maintained
+by the producer. This includes the ``sync_event`` member if it is not null,
+along with any pointers in the ``ArrowArray`` object as usual. Data lifetime
+is managed through the ``release`` callback of the ``ArrowArray`` member.
+
+For an ``ArrowDeviceArray``, the semantics of a released structure and the
+callback semantics are identical to those for
+:ref:`ArrowArray itself <c-data-interface-released>`. Any producer specific context
+information necessary for releasing the device data buffers, in addition to
+any allocated event, should be stored in the ``private_data`` member of
+the ``ArrowArray`` and managed by the ``release`` callback.
+
+Moving an array
+'''''''''''''''
+
+The consumer can *move* the ``ArrowDeviceArray`` structure by bitwise copying
+or shallow member-wise copying. Then it MUST mark the source structure released
+by setting the ``release`` member of the embedded ``ArrowArray`` structure to
+``NULL``, but *without* calling that release callback. This ensures that only
+one live copy of the struct is active at any given time and that lifetime is
+correctly communicated to the producer.
+
+As usual, the release callback will be called on the destination structure
+when it is not needed anymore.
+
+Record batches
+--------------
+As with the C data interface itself, a record batch can be trivially considered
+as an equivalent struct array. In this case the metadata of the top-level
+``ArrowSchema`` can be used for schema-level metadata of the record batch.
+
+Mutability
+----------
+
+Both the producer and the consumer SHOULD consider the exported data (that
+is, the data reachable on the device through the ``buffers`` member of
+the embedded ``ArrowArray``) to be immutable, as either party could otherwise
+see inconsistent data while the other is mutating it.
+
+Likewise, if the ``sync_event`` member is non-NULL, the consumer should not
+attempt to access or read the data until they have synchronized on that event.
+
+C producer example
+====================
+
+Exporting a simple ``int32`` device array
+-----------------------------------------
+
+Export a non-nullable ``int32`` type with empty metadata. An example of this
+can be seen in the :ref:`C data interface docs directly <c-data-interface-export-int32-schema>`.
+
+To export the data itself, we transfer ownership to the consumer through
+the release callback. This example will use CUDA, but the equivalent calls
+could be used for any device:
+
+.. code-block:: c
+
+    static void release_int32_device_array(struct ArrowArray* array) {
+        assert(array->n_buffers == 2);
+        // destroy the event
+        cudaEvent_t* ev_ptr = reinterpret_cast<cudaEvent_t*>(array->private_data);
+        cudaError_t status = cudaEventDestroy(*ev_ptr);
+        assert(status == cudaSuccess);
+        free(ev_ptr);
+
+        // free the buffers and the buffers array
+        status = cudaFree(array->buffers[1]);
+        assert(status == cudaSuccess);
+        free(array->buffers);
+
+        // mark released
+        array->release = NULL;
+    }
+
+    __host__ void export_int32_device_array(void* cudaAllocdPtr,
+                                            cudaStream_t stream,
+                                            int64_t length,
+                                            struct ArrowDeviceArray* array) {
+        // get device id
+        int device;
+        cudaError_t status;
+        status = cudaGetDevice(&device);
+        assert(status == cudaSuccess);
+
+        cudaEvent_t* ev_ptr = (cudaEvent_t*)malloc(sizeof(cudaEvent_t));
+        assert(ev_ptr != NULL);
+        status = cudaEventCreate(ev_ptr);
+        assert(status == cudaSuccess);
+
+        // record event on the stream, assuming that the passed in
+        // stream is where the work to produce the data will be processing.
+        status = cudaEventRecord(*ev_ptr, stream);
+        assert(status == cudaSuccess);
+
+        // initialize fields
+        *array = (struct ArrowDeviceArray) {
+            .array = (struct ArrowArray) {
+                .length = length,
+                .null_count = 0,
+                .offset = 0,
+                .n_buffers = 2,
+                .n_children = 0,
+                .children = NULL,
+                .dictionary = NULL,
+                // bookeeping
+                .release = &release_int32_device_array,
+                .private_data = reinterpret_cast<void*>(ev_ptr),
+            },
+            .device_id = static_cast<int64_t>(device),
+            .device_type = ARROW_DEVICE_CUDA,
+            // pass the event pointer to the consumer
+            .sync_event = reinterpret_cast<void*>(ev_ptr),
+        };
+
+        // allocate list of buffers
+        array->array.buffers = (const void**)malloc(sizeof(void*) * array->array.n_buffers);
+        assert(array->array.buffers != NULL);
+        array->array.buffers[0] = NULL;
+        array->array.buffers[1] = cudaAllocdPtr;
+    }
+
+================
+Device Stream Interface
+================
+
+Like the :ref:`C stream interface <c-stream-interface>`, the C Device data
+interface also specifies a higher-level structure for easing communication
+of streaming data within a single process. Defining an ``ArrowDeviceArrayStream``
+structure.
+
+Semantics
+=========
+
+An Arrow C device stream exposes a streaming source of data chunks, each with
+the same schema. Chunks are obtained by calling a blocking pull-style iteration
+function. It is expected that all chunks should be providing data on the same
+device type (but not necessarily the same device id). If it is necessary
+to provide a stream of data on multiple device types, a producer should
+provide a separate stream object for each device type.
+
+Structure definition
+====================
+
+The C device stream interface is defined by a single ``struct`` definition:
+
+.. code-block:: c
+
+    #ifndef ARROW_C_DEVICE_STREAM_INTERFACE
+    #define ARROW_C_DEVICE_STREAM_INTERFACE
+
+    struct ArrowDeviceArrayStream {
+        // device type that all arrays will be accessible from
+        ArrowDeviceType device_type;
+        // callbacks
+        int (*get_schema)(struct ArrowDeviceArrayStream*, struct ArrowSchema*);
+        int (*get_next)(struct ArrowDeviceArrayStream*, struct ArrowDeviceArray*);
+        const char* (*get_last_error)(struct ArrowDeviceArrayStream*);
+
+        // release callback
+        void (*release)(struct ArrowDeviceArrayStream*);
+
+        // opaque producer-specific data
+        void* private_data;
+    };
+
+    #endif  // ARROW_C_DEVICE_STREAM_INTERFACE
+
+.. note::
+    The canonical guard ``ARROW_C_DEVICE_STREAM_INTERFACE`` is meant to avoid
+    duplicate definitions if two projects copy the C device stream interface
+    definitions into their own headers, and a third-party project includes
+    from these two projects. It is therefore important that this guard is
+    kept exactly as-is when these definitions are copied.
+
+The ArrowDeviceArrayStream structure
+------------------------------------
+
+The ``ArrowDeviceArrayStream`` provides a device type that can access the
+resulting data along with the required callbacks to interact with a
+streaming source of Arrow arrays. It has the following fields:
+
+.. c:member:: ArrowDeviceType device_type
+
+    The device type that this stream produces data on. All
+    ``ArrowDeviceArray``s that are produced by this stream should have the
+    same device type as is set here. This is a convenience for the consumer
+    to not have to check every array that is retrieved and instead allows
+    higher-level coding constructs for streams.
+
+.. c:member:: int (*ArrowDeviceArrayStream.get_schema)(struct ArrowDeviceArrayStream*, struct ArrowSchema* out)
+
+    *Mandatory.* This callback allows the consumer to query the schema of
+    the chunks of data in the stream. The schema is the same for all data
+    chunks.
+
+    This callback must NOT be called on a released ``ArrowDeviceArrayStream``.
+
+    *Return value:* 0 on success, a non-zero
+    :ref:`error code <c-stream-interface-error-codes>` otherwise.
+
+.. c:member:: int (*ArrowDeviceArrayStream.get_next)(struct ArrowDeviceArrayStream*, struct ArrowDeviceArray* out)
+
+    *Mandatory.* This callback allows the consumer to get the next chunk of
+    data in the stream.
+
+    This callback must NOT be called on a released ``ArrowDeviceArrayStream``.
+
+    The next chunk of data MUST be accessible from a device type matching the
+    :c:member:`ArrowDeviceArrayStream.device_type`.
+
+    *Return value:* 0 on success, a non-zero
+    :ref:`error code <c-stream-interface-error-codes>` otherwise.
+
+    On success, the consumer must check whether the ``ArrowDeviceArray``'s
+    embedded ``ArrowArray`` is marked :ref:`released <c-data-interface-released>`.
+    If the embedded ``ArrowDeviceArray.array`` is released, then the end of the
+    stream has been reached. Otherwise, the ``ArrowDeviceArray`` contains a
+    valid data chunk.
+
+.. c:member:: const char* (*ArrowDeviceArrayStream.get_last_error)(struct ArrowDeviceArrayStream*)
+
+    *Mandatory.* This callback allows the consumer to get a textual description
+    of the last error.
+
+    This callback must ONLY be called if the last operation on the
+    ``ArrowDeviceArrayStream`` returned an error. It must NOT be called on a
+    released ``ArrowDeviceArrayStream``.
+
+    *Return value:* a pointer to a NULL-terminated character string
+    (UTF8-encoded). NULL can also be returned if no detailed description is
+    available.
+
+    The returned pointer is only guaranteed to be valid until the next call
+    of one of the stream's callbacks. The character string it points to should
+    be copied to consumer-managed storage if it is intended to survive longer.
+
+.. c:member:: void (*ArrowDeviceArrayStream.release)(struct ArrowDeviceArrayStream*)
+
+    *Mandatory.* A pointer to a producer-provided release callback.
+
+.. c:member:: void* ArrowDeviceArrayStream.private_data
+
+    *Optional.* An opaque pointer to producer-provided private data.
+
+    Consumers MUST NOT process this member. Lifetime of this member is
+    handled by the producer, and especially by the release callback.
+
+Result lifetimes
+----------------
+
+The data returned by the ``get_schema`` and ``get_next`` callbacks must be
+released independantly. Their lifetimes are not tied to that of
+``ArrowDeviceArrayStream``.
+
+Stream lifetime
+---------------
+
+Lifetime of the C stream is managed using a release callback with similar
+usage as in :ref:`C data interface <c-data-interface-released>`.
+
+Thread safety
+-------------
+
+The stream source is not assumed to be thread-safe. Consumers wanting to
+call ``get_next`` from several threads should ensure those calls are
+serialized.
+
+Updating this specification
+===========================
+
+.. note::
+    Since this specification is still considered experimental, there is the
+    (still very low) possibility it might change slightly. Once it is
+    supported in an official Arrow release and the "experimental" tag is
+    removed from it, this section will apply and the ABI will be frozen.
+    
+    The reason for the "experimental" tag is because we don't know what we
+    don't know. While it was attempted to ensure this is generic enough to
+    work with a multitude of different frameworks, it's also possible that
+    something was missed. Once there is some usage of this and we are
+    confident there isn't any necessary modifications, the "experimental"
+    tag will be removed and the ABI frozen.

Review Comment:
   Seems good to me, thanks!



##########
docs/source/format/CDeviceDataInterface.rst:
##########
@@ -0,0 +1,688 @@
+.. Licensed to the Apache Software Foundation (ASF) under one
+.. or more contributor license agreements.  See the NOTICE file
+.. distributed with this work for additional information
+.. regarding copyright ownership.  The ASF licenses this file
+.. to you under the Apache License, Version 2.0 (the
+.. "License"); you may not use this file except in compliance
+.. with the License.  You may obtain a copy of the License at
+
+..   http://www.apache.org/licenses/LICENSE-2.0
+
+.. Unless required by applicable law or agreed to in writing,
+.. software distributed under the License is distributed on an
+.. "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+.. KIND, either express or implied.  See the License for the
+.. specific language governing permissions and limitations
+.. under the License.
+
+.. highlight:: c
+
+.. _c-device-data-interface:
+
+=================================
+The Arrow C Device data interface
+=================================
+
+.. warning:: The Arrow C Device Data Interface should be considered experimental
+
+Rationale
+=========
+
+The current :ref:`C Data Interface <c-data-interface>`, and most
+implementations of it, make the assumption that all data buffers provided
+are CPU buffers. Since Apache Arrow is designed to be a universal in-memory
+format for representing tabular ("columnar") data, there will be the desire
+to leverage this data on non-CPU hardware such as GPUs. One example of such
+a case is the `RAPIDS cuDF library`_ which uses the Arrow memory format with
+CUDA for NVIDIA GPUs. Since copying data from host to device and back is
+expensive, the ideal would be to be able to leave the data on the device
+for as long as possible, even when passing it between runtimes and
+libraries.
+
+The Arrow C Device data interface builds on the existing C data interface
+by adding a very small, stable set of C definitions to it. These definitions
+are equivalents to the ``ArrowArray`` and ``ArrowArrayStream`` structures
+from the C Data Interface which add members to allow specifying the device
+type and pass necessary information to synchronize with the producer.
+For non-C/C++ languages and runtimes, translating the C definitions to
+corresponding C FFI declarations should be just as simple as with the
+current C data interface.
+
+Applications and libraries can then use Arrow schemas and Arrow formatted
+memory on non-CPU devices to exchange data just as easily as they do
+now with CPU data. This will enable leaving data on those devices longer
+and avoiding costly copies back and forth between the host and device
+just to leverage new libraries and runtimes.
+
+Goals
+-----
+
+* Expose an ABI-stable interface built on the existing C data interface.
+* Make it easy for third-party projects to implement support with little
+  initial investment.
+* Allow zero-copy sharing of Arrow formatted device memory between
+  independant runtimes and components running in the same process.
+* Avoid the need for one-to-one adaptation layers such as the
+  `CUDA Array Interface`_ for Python processes to pass CUDA data.
+* Enable integration without explicit dependencies (either at compile-time
+  or runtime) on the Arrow software project itself.
+
+The intent is for the Arrow C Device data interface to expand the reach
+of the current C data interface, allowing it to also become the standard
+low-level building block for columnar processing on devices like GPUs or
+FPGAs.
+
+Structure definitions
+=====================
+
+Because this is built on the C data interface, the C Device data interface
+uses the ``ArrowSchema`` and ``ArrowArray`` structures as defined in the
+:ref:`C data interface spec <c-data-interface-struct-defs>`. It then adds the
+following free-standing definitions. Like the rest of the Arrow project,
+they are available under the Apache License 2.0.
+
+.. code-block:: c
+
+    #ifndef ARROW_C_DEVICE_DATA_INTERFACE
+    #define ARROW_C_DEVICE_DATA_INTERFACE
+
+    // Device type for the allocated memory
+    typedef int32_t ArrowDeviceType;
+
+    // CPU device, same as using ArrowArray directly
+    #define ARROW_DEVICE_CPU 1
+    // CUDA GPU Device
+    #define ARROW_DEVICE_CUDA 2
+    // Pinned CUDA CPU memory by cudaMallocHost
+    #define ARROW_DEVICE_CUDA_HOST 3
+    // OpenCL Device
+    #define ARROW_DEVICE_OPENCL 4
+    // Vulkan buffer for next-gen graphics
+    #define ARROW_DEVICE_VULKAN 7
+    // Metal for Apple GPU
+    #define ARROW_DEVICE_METAL 8
+    // Verilog simulator buffer
+    #define ARROW_DEVICE_VPI 9
+    // ROCm GPUs for AMD GPUs
+    #define ARROW_DEVICE_ROCM 10
+    // Pinned ROCm CPU memory allocated by hipMallocHost
+    #define ARROW_DEVICE_ROCM_HOST 11
+    // Reserved for extension
+    //
+    // used to quickly test extension devices, semantics
+    // can differ based on implementation
+    #define ARROW_DEVICE_EXT_DEV 12
+    // CUDA managed/unified memory allocated by cudaMallocManaged
+    #define ARROW_DEVICE_CUDA_MANAGED 13
+    // Unified shared memory allocated on a oneAPI
+    // non-partitioned device.
+    //
+    // A call to the oneAPI runtime is required to determine the
+    // device type, the USM allocation type and the sycl context
+    // that it is bound to.
+    #define ARROW_DEVICE_ONEAPI 14
+    // GPU support for next-gen WebGPU standard
+    #define ARROW_DEVICE_WEBGPU 15
+    // Qualcomm Hexagon DSP
+    #define ARROW_DEVICE_HEXAGON 16
+
+    struct ArrowDeviceArray {
+      struct ArrowArray array;
+      int64_t device_id;
+      ArrowDeviceType device_type;
+      void* sync_event;
+
+      // reserved bytes for future expansion
+      int64_t reserved[3];
+    };
+
+    #endif  // ARROW_C_DEVICE_DATA_INTERFACE
+
+.. note::
+   The canonical guard ``ARROW_C_DEVICE_DATA_INTERFACE`` is meant to avoid
+   duplicate definitions if two projects copy the definitions in their own
+   headers, and a third-party project includes from these two projects. It
+   is therefore important that this guard is kept exactly as-is when these
+   definitions are copied.
+
+ArrowDeviceType
+---------------
+
+The ``ArrowDeviceType`` typedef is used to indicate what type of device the
+provided memory buffers were allocated on. This, in conjunction with the
+``device_id``, should be sufficient to reference the correct data buffers.
+
+We then use macros to define values for different device types. The provided
+macro values are compatible with the widely used `dlpack`_ ``DLDeviceType``
+definition values, using the same value for each as the equivalent
+``kDL<type>`` enum from dlpack.h. The list will be kept in sync with those
+equivalent enum values over time to ensure compatibility, rather than
+potentially diverging. To avoid the Arrow project having to be in the
+position of vetting new hardware devices, new additions should first be
+added to dlpack before we add a corresponding macro here.
+
+To ensure predictability with the ABI, we use macros instead of an ``enum``
+so the storage type is not compiler dependent.
+
+.. c:macro:: ARROW_DEVICE_CPU
+
+    CPU Device, equivalent to just using ``ArrowArray`` directly instead of
+    using ``ArrowDeviceArray``.
+
+.. c:macro:: ARROW_DEVICE_CUDA
+
+    A `CUDA`_ GPU Device. This could represent data allocated either with the
+    runtime library (``cudaMalloc``) or the device driver (``cuMemAlloc``).
+
+.. c:macro:: ARROW_DEVICE_CUDA_HOST
+
+    CPU memory that was pinned and page-locked by CUDA by using
+    ``cudaMallocHost`` or ``cuMemAllocHost``.
+
+.. c:macro:: ARROW_DEVICE_OPENCL
+
+    Data allocated on the device by using the `OpenCL (Open Computing Language)`_
+    framework.
+
+.. c:macro:: ARROW_DEVICE_VULKAN
+
+    Data allocated by the `Vulkan`_ framework and libraries.
+
+.. c:macro:: ARROW_DEVICE_METAL
+
+    Data on Apple GPU devices using the `Metal`_ framework and libraries.
+
+.. c:macro:: ARROW_DEVICE_VPI
+
+    Indicates usage of a Verilog simulator buffer.
+
+.. c:macro:: ARROW_DEVICE_ROCM
+
+    An AMD device using the `ROCm`_ stack.
+
+.. c:macro:: ARROW_DEVICE_ROCM_HOST
+
+    CPU memory pinned and page-locked allocated ``hipMallocHost``.

Review Comment:
   ```suggestion
       CPU memory that was pinned and page-locked by ROCm by using ``hipMallocHost``.
   ```



##########
docs/source/format/CDeviceDataInterface.rst:
##########
@@ -0,0 +1,688 @@
+.. Licensed to the Apache Software Foundation (ASF) under one
+.. or more contributor license agreements.  See the NOTICE file
+.. distributed with this work for additional information
+.. regarding copyright ownership.  The ASF licenses this file
+.. to you under the Apache License, Version 2.0 (the
+.. "License"); you may not use this file except in compliance
+.. with the License.  You may obtain a copy of the License at
+
+..   http://www.apache.org/licenses/LICENSE-2.0
+
+.. Unless required by applicable law or agreed to in writing,
+.. software distributed under the License is distributed on an
+.. "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+.. KIND, either express or implied.  See the License for the
+.. specific language governing permissions and limitations
+.. under the License.
+
+.. highlight:: c
+
+.. _c-device-data-interface:
+
+=================================
+The Arrow C Device data interface
+=================================
+
+.. warning:: The Arrow C Device Data Interface should be considered experimental
+
+Rationale
+=========
+
+The current :ref:`C Data Interface <c-data-interface>`, and most
+implementations of it, make the assumption that all data buffers provided
+are CPU buffers. Since Apache Arrow is designed to be a universal in-memory
+format for representing tabular ("columnar") data, there will be the desire
+to leverage this data on non-CPU hardware such as GPUs. One example of such
+a case is the `RAPIDS cuDF library`_ which uses the Arrow memory format with
+CUDA for NVIDIA GPUs. Since copying data from host to device and back is
+expensive, the ideal would be to be able to leave the data on the device
+for as long as possible, even when passing it between runtimes and
+libraries.
+
+The Arrow C Device data interface builds on the existing C data interface
+by adding a very small, stable set of C definitions to it. These definitions
+are equivalents to the ``ArrowArray`` and ``ArrowArrayStream`` structures
+from the C Data Interface which add members to allow specifying the device
+type and pass necessary information to synchronize with the producer.
+For non-C/C++ languages and runtimes, translating the C definitions to
+corresponding C FFI declarations should be just as simple as with the
+current C data interface.
+
+Applications and libraries can then use Arrow schemas and Arrow formatted
+memory on non-CPU devices to exchange data just as easily as they do
+now with CPU data. This will enable leaving data on those devices longer
+and avoiding costly copies back and forth between the host and device
+just to leverage new libraries and runtimes.
+
+Goals
+-----
+
+* Expose an ABI-stable interface built on the existing C data interface.
+* Make it easy for third-party projects to implement support with little
+  initial investment.
+* Allow zero-copy sharing of Arrow formatted device memory between
+  independant runtimes and components running in the same process.
+* Avoid the need for one-to-one adaptation layers such as the
+  `CUDA Array Interface`_ for Python processes to pass CUDA data.
+* Enable integration without explicit dependencies (either at compile-time
+  or runtime) on the Arrow software project itself.
+
+The intent is for the Arrow C Device data interface to expand the reach
+of the current C data interface, allowing it to also become the standard
+low-level building block for columnar processing on devices like GPUs or
+FPGAs.
+
+Structure definitions
+=====================
+
+Because this is built on the C data interface, the C Device data interface
+uses the ``ArrowSchema`` and ``ArrowArray`` structures as defined in the
+:ref:`C data interface spec <c-data-interface-struct-defs>`. It then adds the
+following free-standing definitions. Like the rest of the Arrow project,
+they are available under the Apache License 2.0.
+
+.. code-block:: c
+
+    #ifndef ARROW_C_DEVICE_DATA_INTERFACE
+    #define ARROW_C_DEVICE_DATA_INTERFACE
+
+    // Device type for the allocated memory
+    typedef int32_t ArrowDeviceType;
+
+    // CPU device, same as using ArrowArray directly
+    #define ARROW_DEVICE_CPU 1
+    // CUDA GPU Device
+    #define ARROW_DEVICE_CUDA 2
+    // Pinned CUDA CPU memory by cudaMallocHost
+    #define ARROW_DEVICE_CUDA_HOST 3
+    // OpenCL Device
+    #define ARROW_DEVICE_OPENCL 4
+    // Vulkan buffer for next-gen graphics
+    #define ARROW_DEVICE_VULKAN 7
+    // Metal for Apple GPU
+    #define ARROW_DEVICE_METAL 8
+    // Verilog simulator buffer
+    #define ARROW_DEVICE_VPI 9
+    // ROCm GPUs for AMD GPUs
+    #define ARROW_DEVICE_ROCM 10
+    // Pinned ROCm CPU memory allocated by hipMallocHost
+    #define ARROW_DEVICE_ROCM_HOST 11
+    // Reserved for extension
+    //
+    // used to quickly test extension devices, semantics
+    // can differ based on implementation
+    #define ARROW_DEVICE_EXT_DEV 12
+    // CUDA managed/unified memory allocated by cudaMallocManaged
+    #define ARROW_DEVICE_CUDA_MANAGED 13
+    // Unified shared memory allocated on a oneAPI
+    // non-partitioned device.
+    //
+    // A call to the oneAPI runtime is required to determine the
+    // device type, the USM allocation type and the sycl context
+    // that it is bound to.
+    #define ARROW_DEVICE_ONEAPI 14
+    // GPU support for next-gen WebGPU standard
+    #define ARROW_DEVICE_WEBGPU 15
+    // Qualcomm Hexagon DSP
+    #define ARROW_DEVICE_HEXAGON 16
+
+    struct ArrowDeviceArray {
+      struct ArrowArray array;
+      int64_t device_id;
+      ArrowDeviceType device_type;
+      void* sync_event;
+
+      // reserved bytes for future expansion
+      int64_t reserved[3];
+    };
+
+    #endif  // ARROW_C_DEVICE_DATA_INTERFACE
+
+.. note::
+   The canonical guard ``ARROW_C_DEVICE_DATA_INTERFACE`` is meant to avoid
+   duplicate definitions if two projects copy the definitions in their own
+   headers, and a third-party project includes from these two projects. It
+   is therefore important that this guard is kept exactly as-is when these
+   definitions are copied.
+
+ArrowDeviceType
+---------------
+
+The ``ArrowDeviceType`` typedef is used to indicate what type of device the
+provided memory buffers were allocated on. This, in conjunction with the
+``device_id``, should be sufficient to reference the correct data buffers.
+
+We then use macros to define values for different device types. The provided
+macro values are compatible with the widely used `dlpack`_ ``DLDeviceType``
+definition values, using the same value for each as the equivalent
+``kDL<type>`` enum from dlpack.h. The list will be kept in sync with those
+equivalent enum values over time to ensure compatibility, rather than
+potentially diverging. To avoid the Arrow project having to be in the
+position of vetting new hardware devices, new additions should first be
+added to dlpack before we add a corresponding macro here.
+
+To ensure predictability with the ABI, we use macros instead of an ``enum``
+so the storage type is not compiler dependent.
+
+.. c:macro:: ARROW_DEVICE_CPU
+
+    CPU Device, equivalent to just using ``ArrowArray`` directly instead of
+    using ``ArrowDeviceArray``.
+
+.. c:macro:: ARROW_DEVICE_CUDA
+
+    A `CUDA`_ GPU Device. This could represent data allocated either with the
+    runtime library (``cudaMalloc``) or the device driver (``cuMemAlloc``).
+
+.. c:macro:: ARROW_DEVICE_CUDA_HOST
+
+    CPU memory that was pinned and page-locked by CUDA by using
+    ``cudaMallocHost`` or ``cuMemAllocHost``.
+
+.. c:macro:: ARROW_DEVICE_OPENCL
+
+    Data allocated on the device by using the `OpenCL (Open Computing Language)`_
+    framework.
+
+.. c:macro:: ARROW_DEVICE_VULKAN
+
+    Data allocated by the `Vulkan`_ framework and libraries.
+
+.. c:macro:: ARROW_DEVICE_METAL
+
+    Data on Apple GPU devices using the `Metal`_ framework and libraries.
+
+.. c:macro:: ARROW_DEVICE_VPI
+
+    Indicates usage of a Verilog simulator buffer.
+
+.. c:macro:: ARROW_DEVICE_ROCM
+
+    An AMD device using the `ROCm`_ stack.
+
+.. c:macro:: ARROW_DEVICE_ROCM_HOST
+
+    CPU memory pinned and page-locked allocated ``hipMallocHost``.
+
+.. c:macro:: ARROW_DEVICE_EXT_DEV
+
+    This value is an escape-hatch for devices to extend which aren't
+    currently represented otherwise. Producers would need to provide
+    additional information/context specific to the device if using
+    this device type. This is used to quickly test extension devices
+    and semantics can differ based on the implementation.
+
+.. c:macro:: ARROW_DEVICE_CUDA_MANAGED
+
+    CUDA managed/unified memory which is allocated by ``cudaMallocManaged``.
+
+.. c:macro:: ARROW_DEVICE_ONEAPI
+
+    Unified shared memory allocated on an Intel `oneAPI`_ non-partitioned
+    device. A call to the ``oneAPI`` runtime is required to determine
+    the specific device type, the USM allocation type and the sycl context
+    that it is bound to.
+
+.. c:macro:: ARROW_DEVICE_WEBGPU
+
+    GPU support for next-gen WebGPU standards
+
+.. c:macro:: ARROW_DEVICE_HEXAGON
+
+    Data allocated on a Qualcomm Hexagon DSP device.
+
+The ArrowDeviceArray structure
+------------------------------
+
+The ``ArrowDeviceArray`` structure embeds the C data ``ArrowArray`` structure
+and adds additional information necessary for consumers to use the data. It
+has the following fields:
+
+.. c:member:: struct ArrowArray ArrowDeviceArray.array
+
+    The allocated array data. The values in the ``void**`` buffers (along
+    with the buffers of any children) are what is allocated on the device.
+    The buffer values should be device pointers. The rest of the structure
+    should be accessible to the CPU.
+
+    The ``private_data`` and ``release`` callback of this structure should
+    contain any necessary information and structures related to freeing
+    the array according to the device it is allocated on, rather than
+    having a separate release callback and ``private_data`` pointer here.
+
+.. c:member:: int64_t ArrowDeviceArray.device_id
+
+    The device id to identify a specific device if multiple devices of this
+    type are on the system. The semantics of the id will be hardware dependent,
+    but we use an ``int64_t`` to future-proof the id as devices change over time.
+
+.. c:member:: ArrowDeviceType ArrowDeviceArray.device_type
+
+    The type of the device which can access the buffers in the array.
+
+.. c:member:: void* ArrowDeviceArray.sync_event
+
+    Optional. An event-like object to synchronize on if needed.
+
+    Many devices, like GPUs, are primarily asynchronous with respect to
+    CPU processing. As such in order to safely access memory, it is often
+    necessary to have an object to synchronize processing with. Since
+    different devices will use different types to specify this, we use a
+    void* which can be coerced into a pointer to whatever the device
+    appropriate type is.
+
+    If synchronization is not needed, this can be null. If this is non-null
+    then it MUST be used to call the appropriate sync method for the device
+    (e.g. ``cudaStreamWaitEvent`` or ``hipStreamWaitEvent``) before attempting
+    to access the memory in the buffers.
+
+    Expected types to coerce this ``void*`` to depending on the reported
+    device type:
+
+    * CUDA: ``cudaEvent_t*``
+    * ROCm: ``hipEvent_t*``
+    * OpenCL: ``cl_event*``
+    * Vulkan: ``VkEvent*``
+    * Metal: ``MTLEvent*``
+    * OneAPI: ``sycl::event*``
+
+    If an event is provided, then the producer MUST ensure that the exported
+    data is available on the device before the event is triggered. The 
+    consumer SHOULD wait on the event before trying to access the exported
+    data.
+
+.. c:member:: int64_t ArrowDeviceArray.reserved[3]
+
+    As non-CPU development expands, there may be a need to expand this
+    structure. In order to do so without potentially breaking ABI changes,
+    we reserve 24 bytes at the end of the object. This also has the added
+    benefit of bringing the total size of this structure to exactly 128
+    bytes (a power of 2) on 64-bit systems. These bytes MUST be zero'd
+    out after initialization by the producer in order to ensure safe 
+    evolution of the ABI in the future.

Review Comment:
   I would not mention the dubious "added benefit" of making the size a power of two.



##########
docs/source/format/CDeviceDataInterface.rst:
##########
@@ -0,0 +1,688 @@
+.. Licensed to the Apache Software Foundation (ASF) under one
+.. or more contributor license agreements.  See the NOTICE file
+.. distributed with this work for additional information
+.. regarding copyright ownership.  The ASF licenses this file
+.. to you under the Apache License, Version 2.0 (the
+.. "License"); you may not use this file except in compliance
+.. with the License.  You may obtain a copy of the License at
+
+..   http://www.apache.org/licenses/LICENSE-2.0
+
+.. Unless required by applicable law or agreed to in writing,
+.. software distributed under the License is distributed on an
+.. "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+.. KIND, either express or implied.  See the License for the
+.. specific language governing permissions and limitations
+.. under the License.
+
+.. highlight:: c
+
+.. _c-device-data-interface:
+
+=================================
+The Arrow C Device data interface
+=================================
+
+.. warning:: The Arrow C Device Data Interface should be considered experimental
+
+Rationale
+=========
+
+The current :ref:`C Data Interface <c-data-interface>`, and most
+implementations of it, make the assumption that all data buffers provided
+are CPU buffers. Since Apache Arrow is designed to be a universal in-memory
+format for representing tabular ("columnar") data, there will be the desire
+to leverage this data on non-CPU hardware such as GPUs. One example of such
+a case is the `RAPIDS cuDF library`_ which uses the Arrow memory format with
+CUDA for NVIDIA GPUs. Since copying data from host to device and back is
+expensive, the ideal would be to be able to leave the data on the device
+for as long as possible, even when passing it between runtimes and
+libraries.
+
+The Arrow C Device data interface builds on the existing C data interface
+by adding a very small, stable set of C definitions to it. These definitions
+are equivalents to the ``ArrowArray`` and ``ArrowArrayStream`` structures
+from the C Data Interface which add members to allow specifying the device
+type and pass necessary information to synchronize with the producer.
+For non-C/C++ languages and runtimes, translating the C definitions to
+corresponding C FFI declarations should be just as simple as with the
+current C data interface.
+
+Applications and libraries can then use Arrow schemas and Arrow formatted
+memory on non-CPU devices to exchange data just as easily as they do
+now with CPU data. This will enable leaving data on those devices longer
+and avoiding costly copies back and forth between the host and device
+just to leverage new libraries and runtimes.
+
+Goals
+-----
+
+* Expose an ABI-stable interface built on the existing C data interface.
+* Make it easy for third-party projects to implement support with little
+  initial investment.
+* Allow zero-copy sharing of Arrow formatted device memory between
+  independant runtimes and components running in the same process.
+* Avoid the need for one-to-one adaptation layers such as the
+  `CUDA Array Interface`_ for Python processes to pass CUDA data.
+* Enable integration without explicit dependencies (either at compile-time
+  or runtime) on the Arrow software project itself.
+
+The intent is for the Arrow C Device data interface to expand the reach
+of the current C data interface, allowing it to also become the standard
+low-level building block for columnar processing on devices like GPUs or
+FPGAs.
+
+Structure definitions
+=====================
+
+Because this is built on the C data interface, the C Device data interface
+uses the ``ArrowSchema`` and ``ArrowArray`` structures as defined in the
+:ref:`C data interface spec <c-data-interface-struct-defs>`. It then adds the
+following free-standing definitions. Like the rest of the Arrow project,
+they are available under the Apache License 2.0.
+
+.. code-block:: c
+
+    #ifndef ARROW_C_DEVICE_DATA_INTERFACE
+    #define ARROW_C_DEVICE_DATA_INTERFACE
+
+    // Device type for the allocated memory
+    typedef int32_t ArrowDeviceType;
+
+    // CPU device, same as using ArrowArray directly
+    #define ARROW_DEVICE_CPU 1
+    // CUDA GPU Device
+    #define ARROW_DEVICE_CUDA 2
+    // Pinned CUDA CPU memory by cudaMallocHost
+    #define ARROW_DEVICE_CUDA_HOST 3
+    // OpenCL Device
+    #define ARROW_DEVICE_OPENCL 4
+    // Vulkan buffer for next-gen graphics
+    #define ARROW_DEVICE_VULKAN 7
+    // Metal for Apple GPU
+    #define ARROW_DEVICE_METAL 8
+    // Verilog simulator buffer
+    #define ARROW_DEVICE_VPI 9
+    // ROCm GPUs for AMD GPUs
+    #define ARROW_DEVICE_ROCM 10
+    // Pinned ROCm CPU memory allocated by hipMallocHost
+    #define ARROW_DEVICE_ROCM_HOST 11
+    // Reserved for extension
+    //
+    // used to quickly test extension devices, semantics
+    // can differ based on implementation
+    #define ARROW_DEVICE_EXT_DEV 12
+    // CUDA managed/unified memory allocated by cudaMallocManaged
+    #define ARROW_DEVICE_CUDA_MANAGED 13
+    // Unified shared memory allocated on a oneAPI
+    // non-partitioned device.
+    //
+    // A call to the oneAPI runtime is required to determine the
+    // device type, the USM allocation type and the sycl context
+    // that it is bound to.
+    #define ARROW_DEVICE_ONEAPI 14
+    // GPU support for next-gen WebGPU standard
+    #define ARROW_DEVICE_WEBGPU 15
+    // Qualcomm Hexagon DSP
+    #define ARROW_DEVICE_HEXAGON 16
+
+    struct ArrowDeviceArray {
+      struct ArrowArray array;
+      int64_t device_id;
+      ArrowDeviceType device_type;
+      void* sync_event;
+
+      // reserved bytes for future expansion
+      int64_t reserved[3];
+    };
+
+    #endif  // ARROW_C_DEVICE_DATA_INTERFACE
+
+.. note::
+   The canonical guard ``ARROW_C_DEVICE_DATA_INTERFACE`` is meant to avoid
+   duplicate definitions if two projects copy the definitions in their own
+   headers, and a third-party project includes from these two projects. It
+   is therefore important that this guard is kept exactly as-is when these
+   definitions are copied.
+
+ArrowDeviceType
+---------------
+
+The ``ArrowDeviceType`` typedef is used to indicate what type of device the
+provided memory buffers were allocated on. This, in conjunction with the
+``device_id``, should be sufficient to reference the correct data buffers.
+
+We then use macros to define values for different device types. The provided
+macro values are compatible with the widely used `dlpack`_ ``DLDeviceType``
+definition values, using the same value for each as the equivalent
+``kDL<type>`` enum from dlpack.h. The list will be kept in sync with those
+equivalent enum values over time to ensure compatibility, rather than
+potentially diverging. To avoid the Arrow project having to be in the
+position of vetting new hardware devices, new additions should first be
+added to dlpack before we add a corresponding macro here.
+
+To ensure predictability with the ABI, we use macros instead of an ``enum``
+so the storage type is not compiler dependent.
+
+.. c:macro:: ARROW_DEVICE_CPU
+
+    CPU Device, equivalent to just using ``ArrowArray`` directly instead of
+    using ``ArrowDeviceArray``.
+
+.. c:macro:: ARROW_DEVICE_CUDA
+
+    A `CUDA`_ GPU Device. This could represent data allocated either with the
+    runtime library (``cudaMalloc``) or the device driver (``cuMemAlloc``).
+
+.. c:macro:: ARROW_DEVICE_CUDA_HOST
+
+    CPU memory that was pinned and page-locked by CUDA by using
+    ``cudaMallocHost`` or ``cuMemAllocHost``.
+
+.. c:macro:: ARROW_DEVICE_OPENCL
+
+    Data allocated on the device by using the `OpenCL (Open Computing Language)`_
+    framework.
+
+.. c:macro:: ARROW_DEVICE_VULKAN
+
+    Data allocated by the `Vulkan`_ framework and libraries.
+
+.. c:macro:: ARROW_DEVICE_METAL
+
+    Data on Apple GPU devices using the `Metal`_ framework and libraries.
+
+.. c:macro:: ARROW_DEVICE_VPI
+
+    Indicates usage of a Verilog simulator buffer.
+
+.. c:macro:: ARROW_DEVICE_ROCM
+
+    An AMD device using the `ROCm`_ stack.
+
+.. c:macro:: ARROW_DEVICE_ROCM_HOST
+
+    CPU memory pinned and page-locked allocated ``hipMallocHost``.
+
+.. c:macro:: ARROW_DEVICE_EXT_DEV
+
+    This value is an escape-hatch for devices to extend which aren't
+    currently represented otherwise. Producers would need to provide
+    additional information/context specific to the device if using
+    this device type. This is used to quickly test extension devices
+    and semantics can differ based on the implementation.
+
+.. c:macro:: ARROW_DEVICE_CUDA_MANAGED
+
+    CUDA managed/unified memory which is allocated by ``cudaMallocManaged``.
+
+.. c:macro:: ARROW_DEVICE_ONEAPI
+
+    Unified shared memory allocated on an Intel `oneAPI`_ non-partitioned
+    device. A call to the ``oneAPI`` runtime is required to determine
+    the specific device type, the USM allocation type and the sycl context
+    that it is bound to.
+
+.. c:macro:: ARROW_DEVICE_WEBGPU
+
+    GPU support for next-gen WebGPU standards
+
+.. c:macro:: ARROW_DEVICE_HEXAGON
+
+    Data allocated on a Qualcomm Hexagon DSP device.
+
+The ArrowDeviceArray structure
+------------------------------
+
+The ``ArrowDeviceArray`` structure embeds the C data ``ArrowArray`` structure
+and adds additional information necessary for consumers to use the data. It
+has the following fields:
+
+.. c:member:: struct ArrowArray ArrowDeviceArray.array
+
+    The allocated array data. The values in the ``void**`` buffers (along
+    with the buffers of any children) are what is allocated on the device.
+    The buffer values should be device pointers. The rest of the structure
+    should be accessible to the CPU.
+
+    The ``private_data`` and ``release`` callback of this structure should
+    contain any necessary information and structures related to freeing
+    the array according to the device it is allocated on, rather than
+    having a separate release callback and ``private_data`` pointer here.
+
+.. c:member:: int64_t ArrowDeviceArray.device_id
+
+    The device id to identify a specific device if multiple devices of this
+    type are on the system. The semantics of the id will be hardware dependent,
+    but we use an ``int64_t`` to future-proof the id as devices change over time.
+
+.. c:member:: ArrowDeviceType ArrowDeviceArray.device_type
+
+    The type of the device which can access the buffers in the array.
+
+.. c:member:: void* ArrowDeviceArray.sync_event
+
+    Optional. An event-like object to synchronize on if needed.
+
+    Many devices, like GPUs, are primarily asynchronous with respect to
+    CPU processing. As such in order to safely access memory, it is often
+    necessary to have an object to synchronize processing with. Since
+    different devices will use different types to specify this, we use a
+    void* which can be coerced into a pointer to whatever the device
+    appropriate type is.
+
+    If synchronization is not needed, this can be null. If this is non-null
+    then it MUST be used to call the appropriate sync method for the device
+    (e.g. ``cudaStreamWaitEvent`` or ``hipStreamWaitEvent``) before attempting
+    to access the memory in the buffers.
+
+    Expected types to coerce this ``void*`` to depending on the reported
+    device type:
+
+    * CUDA: ``cudaEvent_t*``
+    * ROCm: ``hipEvent_t*``
+    * OpenCL: ``cl_event*``
+    * Vulkan: ``VkEvent*``
+    * Metal: ``MTLEvent*``
+    * OneAPI: ``sycl::event*``
+
+    If an event is provided, then the producer MUST ensure that the exported
+    data is available on the device before the event is triggered. The 
+    consumer SHOULD wait on the event before trying to access the exported
+    data.
+
+.. c:member:: int64_t ArrowDeviceArray.reserved[3]
+
+    As non-CPU development expands, there may be a need to expand this
+    structure. In order to do so without potentially breaking ABI changes,
+    we reserve 24 bytes at the end of the object. This also has the added
+    benefit of bringing the total size of this structure to exactly 128
+    bytes (a power of 2) on 64-bit systems. These bytes MUST be zero'd
+    out after initialization by the producer in order to ensure safe 
+    evolution of the ABI in the future.
+
+.. note::
+    Rather than store the shape / types of the data alongside the
+    ``ArrowDeviceArray``, users should utilize the existing ``ArrowSchema``
+    structure to pass any data type and shape information.
+
+Synchronization event types
+---------------------------
+
+The table below lists the expected event types for each device type.
+If no event type is supported ("N/A"), then the ``sync_event`` member
+should always be null. 
+
+Remember that the event *CAN* be null if synchronization is not needed
+to access the data.
+
++---------------------------+--------------------+---------+
+| Device Type               | Actual Event Type  | Notes   |
++===========================+====================+=========+
+| ARROW_DEVICE_CPU          | N/A                |         |
++---------------------------+--------------------+---------+
+| ARROW_DEVICE_CUDA         | ``cudaEvent_t*``   |         |
++---------------------------+--------------------+---------+
+| ARROW_DEVICE_CUDA_HOST    | ``cudaEvent_t*``   |         |
++---------------------------+--------------------+---------+
+| ARROW_DEVICE_OPENCL       | ``cl_event*``      |         |
++---------------------------+--------------------+---------+
+| ARROW_DEVICE_VULKAN       | ``VkEvent*``       |         |
++---------------------------+--------------------+---------+
+| ARROW_DEVICE_METAL        | ``MTLEvent*``      |         |
++---------------------------+--------------------+---------+
+| ARROW_DEVICE_VPI          | N/A                | (1)     |
++---------------------------+--------------------+---------+
+| ARROW_DEVICE_ROCM         | ``hipEvent_t*``    |         |
++---------------------------+--------------------+---------+
+| ARROW_DEVICE_ROCM_HOST    | ``hipEvent_t*``    |         |
++---------------------------+--------------------+---------+
+| ARROW_DEVICE_EXT_DEV      |                    | (2)     |
++---------------------------+--------------------+---------+
+| ARROW_DEVICE_CUDA_MANAGED | ``cudaEvent_t*``   |         |
++---------------------------+--------------------+---------+
+| ARROW_DEVICE_ONEAPI       | ``sycl::event*``   |         |
++---------------------------+--------------------+---------+
+| ARROW_DEVICE_WEBGPU       | N/A                | (1)     |
++---------------------------+--------------------+---------+
+| ARROW_DEVICE_HEXAGON      | N/A                | (1)     |
++---------------------------+--------------------+---------+
+
+Notes:
+
+* \(1) Currently unknown if framework has an event type to support.
+* \(2) Extension Device has producer defined semantics and thus if
+       synchronization is needed for an extension device, the producer
+       should document the type.
+
+
+Semantics
+=========
+
+Memory management
+-----------------
+
+The ``ArrowDeviceArray`` structure contains an ``ArrowArray`` object which
+itself has :ref:`specific semantics <c-data-interface-semantics>` for releasing
+memory. The term *"base structure"* below refers to the ``ArrowDeviceArray``
+object that is passed directly between the producer and consumer -- not any
+child structure thereof.
+
+It is intended for the base structure to be stack- or heap-allocated by the
+*consumer*. In this case, the producer API should take a pointer to the
+consumer-allocated structure.
+
+However, any data pointed to by the struct MUST be allocated and maintained
+by the producer. This includes the ``sync_event`` member if it is not null,
+along with any pointers in the ``ArrowArray`` object as usual. Data lifetime
+is managed through the ``release`` callback of the ``ArrowArray`` member.
+
+For an ``ArrowDeviceArray``, the semantics of a released structure and the
+callback semantics are identical to those for
+:ref:`ArrowArray itself <c-data-interface-released>`. Any producer specific context
+information necessary for releasing the device data buffers, in addition to
+any allocated event, should be stored in the ``private_data`` member of
+the ``ArrowArray`` and managed by the ``release`` callback.
+
+Moving an array
+'''''''''''''''
+
+The consumer can *move* the ``ArrowDeviceArray`` structure by bitwise copying
+or shallow member-wise copying. Then it MUST mark the source structure released
+by setting the ``release`` member of the embedded ``ArrowArray`` structure to
+``NULL``, but *without* calling that release callback. This ensures that only
+one live copy of the struct is active at any given time and that lifetime is
+correctly communicated to the producer.
+
+As usual, the release callback will be called on the destination structure
+when it is not needed anymore.
+
+Record batches
+--------------
+As with the C data interface itself, a record batch can be trivially considered
+as an equivalent struct array. In this case the metadata of the top-level
+``ArrowSchema`` can be used for schema-level metadata of the record batch.
+
+Mutability
+----------
+
+Both the producer and the consumer SHOULD consider the exported data (that
+is, the data reachable on the device through the ``buffers`` member of
+the embedded ``ArrowArray``) to be immutable, as either party could otherwise
+see inconsistent data while the other is mutating it.
+
+Synchronization
+---------------
+
+If the ``sync_event`` member is non-NULL, the consumer should not attempt 
+to access or read the data until they have synchronized on that event. If
+the ``sync_event`` member is NULL, then it MUST be safe to access the data
+without any synchronization necessary on the part of the consumer.
+
+C producer example
+====================
+
+Exporting a simple ``int32`` device array
+-----------------------------------------
+
+Export a non-nullable ``int32`` type with empty metadata. An example of this
+can be seen in the :ref:`C data interface docs directly <c-data-interface-export-int32-schema>`.
+
+To export the data itself, we transfer ownership to the consumer through
+the release callback. This example will use CUDA, but the equivalent calls
+could be used for any device:
+
+.. code-block:: c
+
+    static void release_int32_device_array(struct ArrowArray* array) {
+        assert(array->n_buffers == 2);
+        // destroy the event
+        cudaEvent_t* ev_ptr = (cudaEvent_t*)(array->private_data);
+        cudaError_t status = cudaEventDestroy(*ev_ptr);
+        assert(status == cudaSuccess);
+        free(ev_ptr);
+
+        // free the buffers and the buffers array
+        status = cudaFree(array->buffers[1]);
+        assert(status == cudaSuccess);
+        free(array->buffers);
+
+        // mark released
+        array->release = NULL;
+    }
+
+    void export_int32_device_array(void* cudaAllocdPtr,
+                                   cudaStream_t stream,
+                                   int64_t length,
+                                   struct ArrowDeviceArray* array) {
+        // get device id
+        int device;
+        cudaError_t status;
+        status = cudaGetDevice(&device);
+        assert(status == cudaSuccess);
+
+        cudaEvent_t* ev_ptr = (cudaEvent_t*)malloc(sizeof(cudaEvent_t));
+        assert(ev_ptr != NULL);
+        status = cudaEventCreate(ev_ptr);
+        assert(status == cudaSuccess);
+
+        // record event on the stream, assuming that the passed in
+        // stream is where the work to produce the data will be processing.
+        status = cudaEventRecord(*ev_ptr, stream);
+        assert(status == cudaSuccess);
+
+        // initialize fields
+        *array = (struct ArrowDeviceArray) {

Review Comment:
   Should `memset` `*array` before, or explicitly zero out the reserved area.



##########
docs/source/format/CDeviceDataInterface.rst:
##########
@@ -0,0 +1,688 @@
+.. Licensed to the Apache Software Foundation (ASF) under one
+.. or more contributor license agreements.  See the NOTICE file
+.. distributed with this work for additional information
+.. regarding copyright ownership.  The ASF licenses this file
+.. to you under the Apache License, Version 2.0 (the
+.. "License"); you may not use this file except in compliance
+.. with the License.  You may obtain a copy of the License at
+
+..   http://www.apache.org/licenses/LICENSE-2.0
+
+.. Unless required by applicable law or agreed to in writing,
+.. software distributed under the License is distributed on an
+.. "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+.. KIND, either express or implied.  See the License for the
+.. specific language governing permissions and limitations
+.. under the License.
+
+.. highlight:: c
+
+.. _c-device-data-interface:
+
+=================================
+The Arrow C Device data interface
+=================================
+
+.. warning:: The Arrow C Device Data Interface should be considered experimental
+
+Rationale
+=========
+
+The current :ref:`C Data Interface <c-data-interface>`, and most
+implementations of it, make the assumption that all data buffers provided
+are CPU buffers. Since Apache Arrow is designed to be a universal in-memory
+format for representing tabular ("columnar") data, there will be the desire
+to leverage this data on non-CPU hardware such as GPUs. One example of such
+a case is the `RAPIDS cuDF library`_ which uses the Arrow memory format with
+CUDA for NVIDIA GPUs. Since copying data from host to device and back is
+expensive, the ideal would be to be able to leave the data on the device
+for as long as possible, even when passing it between runtimes and
+libraries.
+
+The Arrow C Device data interface builds on the existing C data interface
+by adding a very small, stable set of C definitions to it. These definitions
+are equivalents to the ``ArrowArray`` and ``ArrowArrayStream`` structures
+from the C Data Interface which add members to allow specifying the device
+type and pass necessary information to synchronize with the producer.
+For non-C/C++ languages and runtimes, translating the C definitions to
+corresponding C FFI declarations should be just as simple as with the
+current C data interface.
+
+Applications and libraries can then use Arrow schemas and Arrow formatted
+memory on non-CPU devices to exchange data just as easily as they do
+now with CPU data. This will enable leaving data on those devices longer
+and avoiding costly copies back and forth between the host and device
+just to leverage new libraries and runtimes.
+
+Goals
+-----
+
+* Expose an ABI-stable interface built on the existing C data interface.
+* Make it easy for third-party projects to implement support with little
+  initial investment.
+* Allow zero-copy sharing of Arrow formatted device memory between
+  independant runtimes and components running in the same process.
+* Avoid the need for one-to-one adaptation layers such as the
+  `CUDA Array Interface`_ for Python processes to pass CUDA data.
+* Enable integration without explicit dependencies (either at compile-time
+  or runtime) on the Arrow software project itself.
+
+The intent is for the Arrow C Device data interface to expand the reach
+of the current C data interface, allowing it to also become the standard
+low-level building block for columnar processing on devices like GPUs or
+FPGAs.
+
+Structure definitions
+=====================
+
+Because this is built on the C data interface, the C Device data interface
+uses the ``ArrowSchema`` and ``ArrowArray`` structures as defined in the
+:ref:`C data interface spec <c-data-interface-struct-defs>`. It then adds the
+following free-standing definitions. Like the rest of the Arrow project,
+they are available under the Apache License 2.0.
+
+.. code-block:: c
+
+    #ifndef ARROW_C_DEVICE_DATA_INTERFACE
+    #define ARROW_C_DEVICE_DATA_INTERFACE
+
+    // Device type for the allocated memory
+    typedef int32_t ArrowDeviceType;
+
+    // CPU device, same as using ArrowArray directly
+    #define ARROW_DEVICE_CPU 1
+    // CUDA GPU Device
+    #define ARROW_DEVICE_CUDA 2
+    // Pinned CUDA CPU memory by cudaMallocHost
+    #define ARROW_DEVICE_CUDA_HOST 3
+    // OpenCL Device
+    #define ARROW_DEVICE_OPENCL 4
+    // Vulkan buffer for next-gen graphics
+    #define ARROW_DEVICE_VULKAN 7
+    // Metal for Apple GPU
+    #define ARROW_DEVICE_METAL 8
+    // Verilog simulator buffer
+    #define ARROW_DEVICE_VPI 9
+    // ROCm GPUs for AMD GPUs
+    #define ARROW_DEVICE_ROCM 10
+    // Pinned ROCm CPU memory allocated by hipMallocHost
+    #define ARROW_DEVICE_ROCM_HOST 11
+    // Reserved for extension
+    //
+    // used to quickly test extension devices, semantics
+    // can differ based on implementation
+    #define ARROW_DEVICE_EXT_DEV 12
+    // CUDA managed/unified memory allocated by cudaMallocManaged
+    #define ARROW_DEVICE_CUDA_MANAGED 13
+    // Unified shared memory allocated on a oneAPI
+    // non-partitioned device.
+    //
+    // A call to the oneAPI runtime is required to determine the
+    // device type, the USM allocation type and the sycl context
+    // that it is bound to.
+    #define ARROW_DEVICE_ONEAPI 14
+    // GPU support for next-gen WebGPU standard
+    #define ARROW_DEVICE_WEBGPU 15
+    // Qualcomm Hexagon DSP
+    #define ARROW_DEVICE_HEXAGON 16
+
+    struct ArrowDeviceArray {
+      struct ArrowArray array;
+      int64_t device_id;
+      ArrowDeviceType device_type;
+      void* sync_event;
+
+      // reserved bytes for future expansion
+      int64_t reserved[3];
+    };
+
+    #endif  // ARROW_C_DEVICE_DATA_INTERFACE
+
+.. note::
+   The canonical guard ``ARROW_C_DEVICE_DATA_INTERFACE`` is meant to avoid
+   duplicate definitions if two projects copy the definitions in their own
+   headers, and a third-party project includes from these two projects. It
+   is therefore important that this guard is kept exactly as-is when these
+   definitions are copied.
+
+ArrowDeviceType
+---------------
+
+The ``ArrowDeviceType`` typedef is used to indicate what type of device the
+provided memory buffers were allocated on. This, in conjunction with the
+``device_id``, should be sufficient to reference the correct data buffers.
+
+We then use macros to define values for different device types. The provided
+macro values are compatible with the widely used `dlpack`_ ``DLDeviceType``
+definition values, using the same value for each as the equivalent
+``kDL<type>`` enum from dlpack.h. The list will be kept in sync with those
+equivalent enum values over time to ensure compatibility, rather than
+potentially diverging. To avoid the Arrow project having to be in the
+position of vetting new hardware devices, new additions should first be
+added to dlpack before we add a corresponding macro here.
+
+To ensure predictability with the ABI, we use macros instead of an ``enum``
+so the storage type is not compiler dependent.
+
+.. c:macro:: ARROW_DEVICE_CPU
+
+    CPU Device, equivalent to just using ``ArrowArray`` directly instead of
+    using ``ArrowDeviceArray``.
+
+.. c:macro:: ARROW_DEVICE_CUDA
+
+    A `CUDA`_ GPU Device. This could represent data allocated either with the
+    runtime library (``cudaMalloc``) or the device driver (``cuMemAlloc``).
+
+.. c:macro:: ARROW_DEVICE_CUDA_HOST
+
+    CPU memory that was pinned and page-locked by CUDA by using
+    ``cudaMallocHost`` or ``cuMemAllocHost``.
+
+.. c:macro:: ARROW_DEVICE_OPENCL
+
+    Data allocated on the device by using the `OpenCL (Open Computing Language)`_
+    framework.
+
+.. c:macro:: ARROW_DEVICE_VULKAN
+
+    Data allocated by the `Vulkan`_ framework and libraries.
+
+.. c:macro:: ARROW_DEVICE_METAL
+
+    Data on Apple GPU devices using the `Metal`_ framework and libraries.
+
+.. c:macro:: ARROW_DEVICE_VPI
+
+    Indicates usage of a Verilog simulator buffer.
+
+.. c:macro:: ARROW_DEVICE_ROCM
+
+    An AMD device using the `ROCm`_ stack.
+
+.. c:macro:: ARROW_DEVICE_ROCM_HOST
+
+    CPU memory pinned and page-locked allocated ``hipMallocHost``.
+
+.. c:macro:: ARROW_DEVICE_EXT_DEV
+
+    This value is an escape-hatch for devices to extend which aren't
+    currently represented otherwise. Producers would need to provide
+    additional information/context specific to the device if using
+    this device type. This is used to quickly test extension devices
+    and semantics can differ based on the implementation.
+
+.. c:macro:: ARROW_DEVICE_CUDA_MANAGED
+
+    CUDA managed/unified memory which is allocated by ``cudaMallocManaged``.
+
+.. c:macro:: ARROW_DEVICE_ONEAPI
+
+    Unified shared memory allocated on an Intel `oneAPI`_ non-partitioned
+    device. A call to the ``oneAPI`` runtime is required to determine
+    the specific device type, the USM allocation type and the sycl context
+    that it is bound to.
+
+.. c:macro:: ARROW_DEVICE_WEBGPU
+
+    GPU support for next-gen WebGPU standards
+
+.. c:macro:: ARROW_DEVICE_HEXAGON
+
+    Data allocated on a Qualcomm Hexagon DSP device.
+
+The ArrowDeviceArray structure
+------------------------------
+
+The ``ArrowDeviceArray`` structure embeds the C data ``ArrowArray`` structure
+and adds additional information necessary for consumers to use the data. It
+has the following fields:
+
+.. c:member:: struct ArrowArray ArrowDeviceArray.array
+
+    The allocated array data. The values in the ``void**`` buffers (along
+    with the buffers of any children) are what is allocated on the device.
+    The buffer values should be device pointers. The rest of the structure
+    should be accessible to the CPU.
+
+    The ``private_data`` and ``release`` callback of this structure should
+    contain any necessary information and structures related to freeing
+    the array according to the device it is allocated on, rather than
+    having a separate release callback and ``private_data`` pointer here.
+
+.. c:member:: int64_t ArrowDeviceArray.device_id
+
+    The device id to identify a specific device if multiple devices of this
+    type are on the system. The semantics of the id will be hardware dependent,
+    but we use an ``int64_t`` to future-proof the id as devices change over time.
+
+.. c:member:: ArrowDeviceType ArrowDeviceArray.device_type
+
+    The type of the device which can access the buffers in the array.
+
+.. c:member:: void* ArrowDeviceArray.sync_event
+
+    Optional. An event-like object to synchronize on if needed.
+
+    Many devices, like GPUs, are primarily asynchronous with respect to
+    CPU processing. As such in order to safely access memory, it is often
+    necessary to have an object to synchronize processing with. Since
+    different devices will use different types to specify this, we use a
+    void* which can be coerced into a pointer to whatever the device
+    appropriate type is.
+
+    If synchronization is not needed, this can be null. If this is non-null
+    then it MUST be used to call the appropriate sync method for the device
+    (e.g. ``cudaStreamWaitEvent`` or ``hipStreamWaitEvent``) before attempting
+    to access the memory in the buffers.
+
+    Expected types to coerce this ``void*`` to depending on the reported
+    device type:
+
+    * CUDA: ``cudaEvent_t*``
+    * ROCm: ``hipEvent_t*``
+    * OpenCL: ``cl_event*``
+    * Vulkan: ``VkEvent*``
+    * Metal: ``MTLEvent*``
+    * OneAPI: ``sycl::event*``
+
+    If an event is provided, then the producer MUST ensure that the exported
+    data is available on the device before the event is triggered. The 
+    consumer SHOULD wait on the event before trying to access the exported
+    data.
+
+.. c:member:: int64_t ArrowDeviceArray.reserved[3]
+
+    As non-CPU development expands, there may be a need to expand this
+    structure. In order to do so without potentially breaking ABI changes,
+    we reserve 24 bytes at the end of the object. This also has the added
+    benefit of bringing the total size of this structure to exactly 128
+    bytes (a power of 2) on 64-bit systems. These bytes MUST be zero'd
+    out after initialization by the producer in order to ensure safe 
+    evolution of the ABI in the future.
+
+.. note::
+    Rather than store the shape / types of the data alongside the
+    ``ArrowDeviceArray``, users should utilize the existing ``ArrowSchema``
+    structure to pass any data type and shape information.
+
+Synchronization event types
+---------------------------
+
+The table below lists the expected event types for each device type.
+If no event type is supported ("N/A"), then the ``sync_event`` member
+should always be null. 
+
+Remember that the event *CAN* be null if synchronization is not needed
+to access the data.
+
++---------------------------+--------------------+---------+
+| Device Type               | Actual Event Type  | Notes   |
++===========================+====================+=========+
+| ARROW_DEVICE_CPU          | N/A                |         |
++---------------------------+--------------------+---------+
+| ARROW_DEVICE_CUDA         | ``cudaEvent_t*``   |         |
++---------------------------+--------------------+---------+
+| ARROW_DEVICE_CUDA_HOST    | ``cudaEvent_t*``   |         |
++---------------------------+--------------------+---------+
+| ARROW_DEVICE_OPENCL       | ``cl_event*``      |         |
++---------------------------+--------------------+---------+
+| ARROW_DEVICE_VULKAN       | ``VkEvent*``       |         |
++---------------------------+--------------------+---------+
+| ARROW_DEVICE_METAL        | ``MTLEvent*``      |         |
++---------------------------+--------------------+---------+
+| ARROW_DEVICE_VPI          | N/A                | (1)     |
++---------------------------+--------------------+---------+
+| ARROW_DEVICE_ROCM         | ``hipEvent_t*``    |         |
++---------------------------+--------------------+---------+
+| ARROW_DEVICE_ROCM_HOST    | ``hipEvent_t*``    |         |
++---------------------------+--------------------+---------+
+| ARROW_DEVICE_EXT_DEV      |                    | (2)     |
++---------------------------+--------------------+---------+
+| ARROW_DEVICE_CUDA_MANAGED | ``cudaEvent_t*``   |         |
++---------------------------+--------------------+---------+
+| ARROW_DEVICE_ONEAPI       | ``sycl::event*``   |         |
++---------------------------+--------------------+---------+
+| ARROW_DEVICE_WEBGPU       | N/A                | (1)     |
++---------------------------+--------------------+---------+
+| ARROW_DEVICE_HEXAGON      | N/A                | (1)     |
++---------------------------+--------------------+---------+
+
+Notes:
+
+* \(1) Currently unknown if framework has an event type to support.
+* \(2) Extension Device has producer defined semantics and thus if
+       synchronization is needed for an extension device, the producer
+       should document the type.
+
+
+Semantics
+=========
+
+Memory management
+-----------------
+
+The ``ArrowDeviceArray`` structure contains an ``ArrowArray`` object which
+itself has :ref:`specific semantics <c-data-interface-semantics>` for releasing
+memory. The term *"base structure"* below refers to the ``ArrowDeviceArray``
+object that is passed directly between the producer and consumer -- not any
+child structure thereof.
+
+It is intended for the base structure to be stack- or heap-allocated by the
+*consumer*. In this case, the producer API should take a pointer to the
+consumer-allocated structure.
+
+However, any data pointed to by the struct MUST be allocated and maintained
+by the producer. This includes the ``sync_event`` member if it is not null,
+along with any pointers in the ``ArrowArray`` object as usual. Data lifetime
+is managed through the ``release`` callback of the ``ArrowArray`` member.
+
+For an ``ArrowDeviceArray``, the semantics of a released structure and the
+callback semantics are identical to those for
+:ref:`ArrowArray itself <c-data-interface-released>`. Any producer specific context
+information necessary for releasing the device data buffers, in addition to
+any allocated event, should be stored in the ``private_data`` member of
+the ``ArrowArray`` and managed by the ``release`` callback.
+
+Moving an array
+'''''''''''''''
+
+The consumer can *move* the ``ArrowDeviceArray`` structure by bitwise copying
+or shallow member-wise copying. Then it MUST mark the source structure released
+by setting the ``release`` member of the embedded ``ArrowArray`` structure to
+``NULL``, but *without* calling that release callback. This ensures that only
+one live copy of the struct is active at any given time and that lifetime is
+correctly communicated to the producer.
+
+As usual, the release callback will be called on the destination structure
+when it is not needed anymore.
+
+Record batches
+--------------
+As with the C data interface itself, a record batch can be trivially considered
+as an equivalent struct array. In this case the metadata of the top-level
+``ArrowSchema`` can be used for schema-level metadata of the record batch.
+
+Mutability
+----------
+
+Both the producer and the consumer SHOULD consider the exported data (that
+is, the data reachable on the device through the ``buffers`` member of
+the embedded ``ArrowArray``) to be immutable, as either party could otherwise
+see inconsistent data while the other is mutating it.
+
+Synchronization
+---------------
+
+If the ``sync_event`` member is non-NULL, the consumer should not attempt 
+to access or read the data until they have synchronized on that event. If
+the ``sync_event`` member is NULL, then it MUST be safe to access the data
+without any synchronization necessary on the part of the consumer.
+
+C producer example
+====================
+
+Exporting a simple ``int32`` device array
+-----------------------------------------
+
+Export a non-nullable ``int32`` type with empty metadata. An example of this
+can be seen in the :ref:`C data interface docs directly <c-data-interface-export-int32-schema>`.
+
+To export the data itself, we transfer ownership to the consumer through
+the release callback. This example will use CUDA, but the equivalent calls
+could be used for any device:
+
+.. code-block:: c
+
+    static void release_int32_device_array(struct ArrowArray* array) {
+        assert(array->n_buffers == 2);
+        // destroy the event
+        cudaEvent_t* ev_ptr = (cudaEvent_t*)(array->private_data);
+        cudaError_t status = cudaEventDestroy(*ev_ptr);
+        assert(status == cudaSuccess);
+        free(ev_ptr);
+
+        // free the buffers and the buffers array
+        status = cudaFree(array->buffers[1]);
+        assert(status == cudaSuccess);
+        free(array->buffers);
+
+        // mark released
+        array->release = NULL;
+    }
+
+    void export_int32_device_array(void* cudaAllocdPtr,
+                                   cudaStream_t stream,
+                                   int64_t length,
+                                   struct ArrowDeviceArray* array) {
+        // get device id
+        int device;
+        cudaError_t status;
+        status = cudaGetDevice(&device);
+        assert(status == cudaSuccess);
+
+        cudaEvent_t* ev_ptr = (cudaEvent_t*)malloc(sizeof(cudaEvent_t));
+        assert(ev_ptr != NULL);
+        status = cudaEventCreate(ev_ptr);
+        assert(status == cudaSuccess);
+
+        // record event on the stream, assuming that the passed in
+        // stream is where the work to produce the data will be processing.
+        status = cudaEventRecord(*ev_ptr, stream);
+        assert(status == cudaSuccess);
+
+        // initialize fields
+        *array = (struct ArrowDeviceArray) {
+            .array = (struct ArrowArray) {
+                .length = length,
+                .null_count = 0,
+                .offset = 0,
+                .n_buffers = 2,
+                .n_children = 0,
+                .children = NULL,
+                .dictionary = NULL,
+                // bookeeping
+                .release = &release_int32_device_array,
+                // store the event pointer as private data in the array
+                // so that we can access it in the release callback.
+                .private_data = (void*)(ev_ptr),
+            },
+            .device_id = (int64_t)(device),
+            .device_type = ARROW_DEVICE_CUDA,
+            // pass the event pointer to the consumer
+            .sync_event = (void*)(ev_ptr),
+        };
+
+        // allocate list of buffers
+        array->array.buffers = (const void**)malloc(sizeof(void*) * array->array.n_buffers);
+        assert(array->array.buffers != NULL);
+        array->array.buffers[0] = NULL;
+        array->array.buffers[1] = cudaAllocdPtr;
+    }
+
+    // calling the release callback should be done using the array member
+    // of the device array.
+    static void release_device_array_helper(struct ArrowDeviceArray* arr) {
+        arr->array.release(&arr->array);
+    }
+
+=======================
+Device Stream Interface
+=======================
+
+Like the :ref:`C stream interface <c-stream-interface>`, the C Device data
+interface also specifies a higher-level structure for easing communication
+of streaming data within a single process. Defining an ``ArrowDeviceArrayStream``
+structure.
+
+Semantics
+=========
+
+An Arrow C device stream exposes a streaming source of data chunks, each with
+the same schema. Chunks are obtained by calling a blocking pull-style iteration
+function. It is expected that all chunks should be providing data on the same
+device type (but not necessarily the same device id). If it is necessary
+to provide a stream of data on multiple device types, a producer should
+provide a separate stream object for each device type.
+
+Structure definition
+====================
+
+The C device stream interface is defined by a single ``struct`` definition:
+
+.. code-block:: c
+
+    #ifndef ARROW_C_DEVICE_STREAM_INTERFACE
+    #define ARROW_C_DEVICE_STREAM_INTERFACE
+
+    struct ArrowDeviceArrayStream {
+        // device type that all arrays will be accessible from
+        ArrowDeviceType device_type;
+        // callbacks
+        int (*get_schema)(struct ArrowDeviceArrayStream*, struct ArrowSchema*);
+        int (*get_next)(struct ArrowDeviceArrayStream*, struct ArrowDeviceArray*);
+        const char* (*get_last_error)(struct ArrowDeviceArrayStream*);
+
+        // release callback
+        void (*release)(struct ArrowDeviceArrayStream*);
+
+        // opaque producer-specific data
+        void* private_data;
+    };
+
+    #endif  // ARROW_C_DEVICE_STREAM_INTERFACE
+
+.. note::
+    The canonical guard ``ARROW_C_DEVICE_STREAM_INTERFACE`` is meant to avoid
+    duplicate definitions if two projects copy the C device stream interface
+    definitions into their own headers, and a third-party project includes
+    from these two projects. It is therefore important that this guard is
+    kept exactly as-is when these definitions are copied.
+
+The ArrowDeviceArrayStream structure
+------------------------------------
+
+The ``ArrowDeviceArrayStream`` provides a device type that can access the
+resulting data along with the required callbacks to interact with a
+streaming source of Arrow arrays. It has the following fields:
+
+.. c:member:: ArrowDeviceType device_type
+
+    The device type that this stream produces data on. All

Review Comment:
   Mandatory or optional?



##########
docs/source/format/CDeviceDataInterface.rst:
##########
@@ -0,0 +1,628 @@
+.. Licensed to the Apache Software Foundation (ASF) under one
+.. or more contributor license agreements.  See the NOTICE file
+.. distributed with this work for additional information
+.. regarding copyright ownership.  The ASF licenses this file
+.. to you under the Apache License, Version 2.0 (the
+.. "License"); you may not use this file except in compliance
+.. with the License.  You may obtain a copy of the License at
+
+..   http://www.apache.org/licenses/LICENSE-2.0
+
+.. Unless required by applicable law or agreed to in writing,
+.. software distributed under the License is distributed on an
+.. "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+.. KIND, either express or implied.  See the License for the
+.. specific language governing permissions and limitations
+.. under the License.
+
+.. highlight:: c
+
+.. _c-device-data-interface:
+
+=================================
+The Arrow C Device data interface
+=================================
+
+.. note:: The Arrow C Device Data Interface should be considered experimental
+
+Rationale
+=========
+
+The current :ref:`C Data Interface <c-data-interface>`, and most
+implementations of it, make the assumption that all data buffers provided
+are CPU buffers. Since Apache Arrow is designed to be a universal in-memory
+format for representing tabular ("columnar") data, there will be the desire
+to leverage this data on non-CPU hardware such as GPUs. One example of such
+a case is the `RAPIDS cuDF library`_ which uses the Arrow memory format with
+CUDA for NVIDIA GPUs. Since copying data from host to device and back is
+expensive, the ideal would be to be able to leave the data on the device
+for as long as possible, even when passing it between runtimes and
+libraries.
+
+The Arrow C Device data interface builds on the existing C data interface
+by adding a very small, stable set of C definitions to it. These definitions
+are equivalents to the ``ArrowArray`` and ``ArrowArrayStream`` structures
+from the C Data Interface which add members to allow specifying the device
+type and pass necessary information to synchronize with the producer.
+For non-C/C++ languages and runtimes, translating the C definitions to
+corresponding C FFI declarations should be just as simple as with the
+current C data interface.
+
+Applications and libraries can then use Arrow schemas and Arrow formatted
+memory on non-CPU devices to exchange data just as easily as they do
+now with CPU data. This will enable leaving data on those devices longer
+and avoiding costly copies back and forth between the host and device
+just to leverage new libraries and runtimes.
+
+Goals
+-----
+
+* Expose an ABI-stable interface built on the existing C data interface.
+* Make it easy for third-party projects to implement support with little
+  initial investment.
+* Allow zero-copy sharing of Arrow formatted device memory between
+  independant runtimes and components running in the same process.
+* Avoid the need for one-to-one adaptation layers such as the
+  `CUDA Array Interface`_ for Python processes to pass CUDA data.
+* Enable integration without explicit dependencies (either at compile-time
+  or runtime) on the Arrow software project itself.
+
+The intent is for the Arrow C Device data interface to expand the reach
+of the current C data interface, allowing it to also become the standard
+low-level building block for columnar processing on devices like GPUs or
+FPGAs.
+
+Structure definitions
+=====================
+
+Because this is built on the C data interface, the C Device data interface
+uses the ``ArrowSchema`` and ``ArrowArray`` structures as defined in the
+:ref:`C data interface spec <c-data-interface-struct-defs>`. It then adds the
+following free-standing definitions. Like the rest of the Arrow project,
+they are available under the Apache License 2.0.
+
+.. code-block:: c
+
+    #ifndef ARROW_C_DEVICE_DATA_INTERFACE
+    #define ARROW_C_DEVICE_DATA_INTERFACE
+
+    // Device type for the allocated memory
+    typedef int32_t ArrowDeviceType;
+
+    // CPU device, same as using ArrowArray directly
+    #define ARROW_DEVICE_CPU 1
+    // CUDA GPU Device
+    #define ARROW_DEVICE_CUDA 2
+    // Pinned CUDA CPU memory by cudaMallocHost
+    #define ARROW_DEVICE_CUDA_HOST 3
+    // OpenCL Device
+    #define ARROW_DEVICE_OPENCL 4
+    // Vulkan buffer for next-gen graphics
+    #define ARROW_DEVICE_VULKAN 7
+    // Metal for Apple GPU
+    #define ARROW_DEVICE_METAL 8
+    // Verilog simulator buffer
+    #define ARROW_DEVICE_VPI 9
+    // ROCm GPUs for AMD GPUs
+    #define ARROW_DEVICE_ROCM 10
+    // Pinned ROCm CPU memory allocated by hipMallocHost
+    #define ARROW_DEVICE_ROCM_HOST 11
+    // Reserved for extension
+    //
+    // used to quickly test extension devices, semantics
+    // can differ based on implementation
+    #define ARROW_DEVICE_EXT_DEV 12
+    // CUDA managed/unified memory allocated by cudaMallocManaged
+    #define ARROW_DEVICE_CUDA_MANAGED 13
+    // Unified shared memory allocated on a oneAPI
+    // non-partitioned device.
+    //
+    // A call to the oneAPI runtime is required to determine the
+    // device type, the USM allocation type and the sycl context
+    // that it is bound to.
+    #define ARROW_DEVICE_ONEAPI 14
+    // GPU support for next-gen WebGPU standard
+    #define ARROW_DEVICE_WEBGPU 15
+    // Qualcomm Hexagon DSP
+    #define ARROW_DEVICE_HEXAGON 16
+
+    struct ArrowDeviceArray {
+      struct ArrowArray array;
+      int64_t device_id;
+      ArrowDeviceType device_type;
+      void* sync_event;
+
+      // reserved bytes for future expansion
+      int64_t reserved[3];
+    };
+
+    #endif  // ARROW_C_DEVICE_DATA_INTERFACE
+
+.. note::
+   The canonical guard ``ARROW_C_DEVICE_DATA_INTERFACE`` is meant to avoid
+   duplicate definitions if two projects copy the definitions in their own
+   headers, and a third-party project includes from these two projects. It
+   is therefore important that this guard is kept exactly as-is when these
+   definitions are copied.
+
+ArrowDeviceType
+---------------
+
+The ``ArrowDeviceType`` typedef is used to indicate what type of device the
+provided memory buffers were allocated on. This, in conjunction with the
+``device_id``, should be sufficient to reference the correct data buffers.
+
+We then use macros to define values for different device types. The provided
+macro values are compatible with the widely used `dlpack`_ ``DLDeviceType``
+definition values, using the same value for each as the equivalent
+``kDL<type>`` enum from dlpack.h. The list will be kept in sync with those
+equivalent enum values over time to ensure compatibility, rather than
+potentially diverging. To avoid the Arrow project having to be in the
+position of vetting new hardware devices, new additions should first be
+added to dlpack before we add a corresponding macro here.
+
+To ensure predictability with the ABI, we use macros instead of an ``enum``
+so the storage type is not compiler dependent.
+
+.. c:macro:: ARROW_DEVICE_CPU
+
+    CPU Device, equivalent to just using ``ArrowArray`` directly instead of
+    using ``ArrowDeviceArray``.
+
+.. c:macro:: ARROW_DEVICE_CUDA
+
+    A `CUDA`_ GPU Device. This could represent data allocated either with the
+    runtime library (``cudaMalloc``) or the device driver (``cuMemAlloc``).
+
+.. c:macro:: ARROW_DEVICE_CUDA_HOST
+
+    CPU memory that was pinned and page-locked by CUDA by using
+    ``cudaMallocHost`` or ``cuMemAllocHost``.
+
+.. c:macro:: ARROW_DEVICE_OPENCL
+
+    Data allocated on the device by using the `OpenCL (Open Computing Language)`_
+    framework.
+
+.. c:macro:: ARROW_DEVICE_VULKAN
+
+    Data allocated by the `Vulkan`_ framework and libraries.
+
+.. c:macro:: ARROW_DEVICE_METAL
+
+    Data on Apple GPU devices using the `Metal`_ framework and libraries.
+
+.. c:macro:: ARROW_DEVICE_VPI
+
+    Indicates usage of a Verilog simulator buffer.
+
+.. c:macro:: ARROW_DEVICE_ROCM
+
+    An AMD device using the `ROCm`_ stack.
+
+.. c:macro:: ARROW_DEVICE_ROCM_HOST
+
+    CPU memory pinned and page-locked allocated ``hipMallocHost``.
+
+.. c:macro:: ARROW_DEVICE_EXT_DEV
+
+    This value is an escape-hatch for devices to extend which aren't
+    currently represented otherwise. Producers would need to provide
+    additional information/context specific to the device if using
+    this device type. This is used to quickly test extension devices
+    and semantics can differ based on the implementation.
+
+.. c:macro:: ARROW_DEVICE_CUDA_MANAGED
+
+    CUDA managed/unified memory which is allocated by ``cudaMallocManaged``.
+
+.. c:macro:: ARROW_DEVICE_ONEAPI
+
+    Unified shared memory allocated on an Intel `oneAPI`_ non-partitioned
+    device. A call to the ``oneAPI`` runtime is required to determine
+    the specific device type, the USM allocation type and the sycl context
+    that it is bound to.
+
+.. c:macro:: ARROW_DEVICE_WEBGPU
+
+    GPU support for next-gen WebGPU standards
+
+.. c:macro:: ARROW_DEVICE_HEXAGON
+
+    Data allocated on a Qualcomm Hexagon DSP device.
+
+The ArrowDeviceArray structure
+------------------------------
+
+The ``ArrowDeviceArray`` structure embeds the C data ``ArrowArray`` structure
+and adds additional information necessary for consumers to use the data. It
+has the following fields:
+
+.. c:member:: struct ArrowArray ArrowDeviceArray.array
+
+    The allocated array data. The values in the ``void**`` buffers (along
+    with the buffers of any children) are what is allocated on the device.
+    The buffer values should be device pointers. The rest of the structure
+    should be accessible to the CPU.
+
+    The ``private_data`` and ``release`` callback of this structure should
+    contain any necessary information and structures related to freeing
+    the array according to the device it is allocated on, rather than
+    having a separate release callback and ``private_data`` pointer here.
+
+.. c:member:: int64_t ArrowDeviceArray.device_id
+
+    The device id to identify a specific device if multiple devices of this
+    type are on the system. The semantics of the id will be hardware dependent,
+    but we use an ``int64_t`` to future-proof the id as devices change over time.
+
+.. c:member:: ArrowDeviceType ArrowDeviceArray.device_type
+
+    The type of the device which can access the buffers in the array.
+
+.. c:member:: void* ArrowDeviceArray.sync_event
+
+    Optional. An event-like object to synchronize on if needed.
+
+    Many devices, like GPUs, are primarily asynchronous with respect to
+    CPU processing. As such in order to safely access memory, it is often
+    necessary to have an object to synchronize processing with. Since
+    different devices will use different types to specify this, we use a
+    void* which can be coerced into a pointer to whatever the device
+    appropriate type is.
+
+    If synchronization is not needed, this can be null. If this is non-null
+    then it MUST be used to call the appropriate sync method for the device
+    (e.g. ``cudaStreamWaitEvent`` or ``hipStreamWaitEvent``) before attempting
+    to access the memory in the buffers.
+
+    Expected types to coerce this ``void*`` to depending on the reported

Review Comment:
   Ok, but do we still want the bullet list below?



##########
docs/source/format/CDeviceDataInterface.rst:
##########
@@ -0,0 +1,688 @@
+.. Licensed to the Apache Software Foundation (ASF) under one
+.. or more contributor license agreements.  See the NOTICE file
+.. distributed with this work for additional information
+.. regarding copyright ownership.  The ASF licenses this file
+.. to you under the Apache License, Version 2.0 (the
+.. "License"); you may not use this file except in compliance
+.. with the License.  You may obtain a copy of the License at
+
+..   http://www.apache.org/licenses/LICENSE-2.0
+
+.. Unless required by applicable law or agreed to in writing,
+.. software distributed under the License is distributed on an
+.. "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+.. KIND, either express or implied.  See the License for the
+.. specific language governing permissions and limitations
+.. under the License.
+
+.. highlight:: c
+
+.. _c-device-data-interface:
+
+=================================
+The Arrow C Device data interface
+=================================
+
+.. warning:: The Arrow C Device Data Interface should be considered experimental
+
+Rationale
+=========
+
+The current :ref:`C Data Interface <c-data-interface>`, and most
+implementations of it, make the assumption that all data buffers provided
+are CPU buffers. Since Apache Arrow is designed to be a universal in-memory
+format for representing tabular ("columnar") data, there will be the desire
+to leverage this data on non-CPU hardware such as GPUs. One example of such
+a case is the `RAPIDS cuDF library`_ which uses the Arrow memory format with
+CUDA for NVIDIA GPUs. Since copying data from host to device and back is
+expensive, the ideal would be to be able to leave the data on the device
+for as long as possible, even when passing it between runtimes and
+libraries.
+
+The Arrow C Device data interface builds on the existing C data interface
+by adding a very small, stable set of C definitions to it. These definitions
+are equivalents to the ``ArrowArray`` and ``ArrowArrayStream`` structures
+from the C Data Interface which add members to allow specifying the device
+type and pass necessary information to synchronize with the producer.
+For non-C/C++ languages and runtimes, translating the C definitions to
+corresponding C FFI declarations should be just as simple as with the
+current C data interface.
+
+Applications and libraries can then use Arrow schemas and Arrow formatted
+memory on non-CPU devices to exchange data just as easily as they do
+now with CPU data. This will enable leaving data on those devices longer
+and avoiding costly copies back and forth between the host and device
+just to leverage new libraries and runtimes.
+
+Goals
+-----
+
+* Expose an ABI-stable interface built on the existing C data interface.
+* Make it easy for third-party projects to implement support with little
+  initial investment.
+* Allow zero-copy sharing of Arrow formatted device memory between
+  independant runtimes and components running in the same process.
+* Avoid the need for one-to-one adaptation layers such as the
+  `CUDA Array Interface`_ for Python processes to pass CUDA data.
+* Enable integration without explicit dependencies (either at compile-time
+  or runtime) on the Arrow software project itself.
+
+The intent is for the Arrow C Device data interface to expand the reach
+of the current C data interface, allowing it to also become the standard
+low-level building block for columnar processing on devices like GPUs or
+FPGAs.
+
+Structure definitions
+=====================
+
+Because this is built on the C data interface, the C Device data interface
+uses the ``ArrowSchema`` and ``ArrowArray`` structures as defined in the
+:ref:`C data interface spec <c-data-interface-struct-defs>`. It then adds the
+following free-standing definitions. Like the rest of the Arrow project,
+they are available under the Apache License 2.0.
+
+.. code-block:: c
+
+    #ifndef ARROW_C_DEVICE_DATA_INTERFACE
+    #define ARROW_C_DEVICE_DATA_INTERFACE
+
+    // Device type for the allocated memory
+    typedef int32_t ArrowDeviceType;
+
+    // CPU device, same as using ArrowArray directly
+    #define ARROW_DEVICE_CPU 1
+    // CUDA GPU Device
+    #define ARROW_DEVICE_CUDA 2
+    // Pinned CUDA CPU memory by cudaMallocHost
+    #define ARROW_DEVICE_CUDA_HOST 3
+    // OpenCL Device
+    #define ARROW_DEVICE_OPENCL 4
+    // Vulkan buffer for next-gen graphics
+    #define ARROW_DEVICE_VULKAN 7
+    // Metal for Apple GPU
+    #define ARROW_DEVICE_METAL 8
+    // Verilog simulator buffer
+    #define ARROW_DEVICE_VPI 9
+    // ROCm GPUs for AMD GPUs
+    #define ARROW_DEVICE_ROCM 10
+    // Pinned ROCm CPU memory allocated by hipMallocHost
+    #define ARROW_DEVICE_ROCM_HOST 11
+    // Reserved for extension
+    //
+    // used to quickly test extension devices, semantics
+    // can differ based on implementation
+    #define ARROW_DEVICE_EXT_DEV 12
+    // CUDA managed/unified memory allocated by cudaMallocManaged
+    #define ARROW_DEVICE_CUDA_MANAGED 13
+    // Unified shared memory allocated on a oneAPI
+    // non-partitioned device.
+    //
+    // A call to the oneAPI runtime is required to determine the
+    // device type, the USM allocation type and the sycl context
+    // that it is bound to.
+    #define ARROW_DEVICE_ONEAPI 14
+    // GPU support for next-gen WebGPU standard
+    #define ARROW_DEVICE_WEBGPU 15
+    // Qualcomm Hexagon DSP
+    #define ARROW_DEVICE_HEXAGON 16
+
+    struct ArrowDeviceArray {
+      struct ArrowArray array;
+      int64_t device_id;
+      ArrowDeviceType device_type;
+      void* sync_event;
+
+      // reserved bytes for future expansion
+      int64_t reserved[3];
+    };
+
+    #endif  // ARROW_C_DEVICE_DATA_INTERFACE
+
+.. note::
+   The canonical guard ``ARROW_C_DEVICE_DATA_INTERFACE`` is meant to avoid
+   duplicate definitions if two projects copy the definitions in their own
+   headers, and a third-party project includes from these two projects. It
+   is therefore important that this guard is kept exactly as-is when these
+   definitions are copied.
+
+ArrowDeviceType
+---------------
+
+The ``ArrowDeviceType`` typedef is used to indicate what type of device the
+provided memory buffers were allocated on. This, in conjunction with the
+``device_id``, should be sufficient to reference the correct data buffers.
+
+We then use macros to define values for different device types. The provided
+macro values are compatible with the widely used `dlpack`_ ``DLDeviceType``
+definition values, using the same value for each as the equivalent
+``kDL<type>`` enum from dlpack.h. The list will be kept in sync with those
+equivalent enum values over time to ensure compatibility, rather than
+potentially diverging. To avoid the Arrow project having to be in the
+position of vetting new hardware devices, new additions should first be
+added to dlpack before we add a corresponding macro here.
+
+To ensure predictability with the ABI, we use macros instead of an ``enum``
+so the storage type is not compiler dependent.
+
+.. c:macro:: ARROW_DEVICE_CPU
+
+    CPU Device, equivalent to just using ``ArrowArray`` directly instead of
+    using ``ArrowDeviceArray``.
+
+.. c:macro:: ARROW_DEVICE_CUDA
+
+    A `CUDA`_ GPU Device. This could represent data allocated either with the
+    runtime library (``cudaMalloc``) or the device driver (``cuMemAlloc``).
+
+.. c:macro:: ARROW_DEVICE_CUDA_HOST
+
+    CPU memory that was pinned and page-locked by CUDA by using
+    ``cudaMallocHost`` or ``cuMemAllocHost``.
+
+.. c:macro:: ARROW_DEVICE_OPENCL
+
+    Data allocated on the device by using the `OpenCL (Open Computing Language)`_
+    framework.
+
+.. c:macro:: ARROW_DEVICE_VULKAN
+
+    Data allocated by the `Vulkan`_ framework and libraries.
+
+.. c:macro:: ARROW_DEVICE_METAL
+
+    Data on Apple GPU devices using the `Metal`_ framework and libraries.
+
+.. c:macro:: ARROW_DEVICE_VPI
+
+    Indicates usage of a Verilog simulator buffer.
+
+.. c:macro:: ARROW_DEVICE_ROCM
+
+    An AMD device using the `ROCm`_ stack.
+
+.. c:macro:: ARROW_DEVICE_ROCM_HOST
+
+    CPU memory pinned and page-locked allocated ``hipMallocHost``.
+
+.. c:macro:: ARROW_DEVICE_EXT_DEV
+
+    This value is an escape-hatch for devices to extend which aren't
+    currently represented otherwise. Producers would need to provide
+    additional information/context specific to the device if using
+    this device type. This is used to quickly test extension devices
+    and semantics can differ based on the implementation.
+
+.. c:macro:: ARROW_DEVICE_CUDA_MANAGED
+
+    CUDA managed/unified memory which is allocated by ``cudaMallocManaged``.
+
+.. c:macro:: ARROW_DEVICE_ONEAPI
+
+    Unified shared memory allocated on an Intel `oneAPI`_ non-partitioned
+    device. A call to the ``oneAPI`` runtime is required to determine
+    the specific device type, the USM allocation type and the sycl context
+    that it is bound to.
+
+.. c:macro:: ARROW_DEVICE_WEBGPU
+
+    GPU support for next-gen WebGPU standards
+
+.. c:macro:: ARROW_DEVICE_HEXAGON
+
+    Data allocated on a Qualcomm Hexagon DSP device.
+
+The ArrowDeviceArray structure
+------------------------------
+
+The ``ArrowDeviceArray`` structure embeds the C data ``ArrowArray`` structure
+and adds additional information necessary for consumers to use the data. It
+has the following fields:
+
+.. c:member:: struct ArrowArray ArrowDeviceArray.array

Review Comment:
   For each of these, define if it's mandatory or optional.



##########
docs/source/format/CDeviceDataInterface.rst:
##########
@@ -0,0 +1,688 @@
+.. Licensed to the Apache Software Foundation (ASF) under one
+.. or more contributor license agreements.  See the NOTICE file
+.. distributed with this work for additional information
+.. regarding copyright ownership.  The ASF licenses this file
+.. to you under the Apache License, Version 2.0 (the
+.. "License"); you may not use this file except in compliance
+.. with the License.  You may obtain a copy of the License at
+
+..   http://www.apache.org/licenses/LICENSE-2.0
+
+.. Unless required by applicable law or agreed to in writing,
+.. software distributed under the License is distributed on an
+.. "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+.. KIND, either express or implied.  See the License for the
+.. specific language governing permissions and limitations
+.. under the License.
+
+.. highlight:: c
+
+.. _c-device-data-interface:
+
+=================================
+The Arrow C Device data interface
+=================================
+
+.. warning:: The Arrow C Device Data Interface should be considered experimental
+
+Rationale
+=========
+
+The current :ref:`C Data Interface <c-data-interface>`, and most
+implementations of it, make the assumption that all data buffers provided
+are CPU buffers. Since Apache Arrow is designed to be a universal in-memory
+format for representing tabular ("columnar") data, there will be the desire
+to leverage this data on non-CPU hardware such as GPUs. One example of such
+a case is the `RAPIDS cuDF library`_ which uses the Arrow memory format with
+CUDA for NVIDIA GPUs. Since copying data from host to device and back is
+expensive, the ideal would be to be able to leave the data on the device
+for as long as possible, even when passing it between runtimes and
+libraries.
+
+The Arrow C Device data interface builds on the existing C data interface
+by adding a very small, stable set of C definitions to it. These definitions
+are equivalents to the ``ArrowArray`` and ``ArrowArrayStream`` structures
+from the C Data Interface which add members to allow specifying the device
+type and pass necessary information to synchronize with the producer.
+For non-C/C++ languages and runtimes, translating the C definitions to
+corresponding C FFI declarations should be just as simple as with the
+current C data interface.
+
+Applications and libraries can then use Arrow schemas and Arrow formatted
+memory on non-CPU devices to exchange data just as easily as they do
+now with CPU data. This will enable leaving data on those devices longer
+and avoiding costly copies back and forth between the host and device
+just to leverage new libraries and runtimes.
+
+Goals
+-----
+
+* Expose an ABI-stable interface built on the existing C data interface.
+* Make it easy for third-party projects to implement support with little
+  initial investment.
+* Allow zero-copy sharing of Arrow formatted device memory between
+  independant runtimes and components running in the same process.
+* Avoid the need for one-to-one adaptation layers such as the
+  `CUDA Array Interface`_ for Python processes to pass CUDA data.
+* Enable integration without explicit dependencies (either at compile-time
+  or runtime) on the Arrow software project itself.
+
+The intent is for the Arrow C Device data interface to expand the reach
+of the current C data interface, allowing it to also become the standard
+low-level building block for columnar processing on devices like GPUs or
+FPGAs.
+
+Structure definitions
+=====================
+
+Because this is built on the C data interface, the C Device data interface
+uses the ``ArrowSchema`` and ``ArrowArray`` structures as defined in the
+:ref:`C data interface spec <c-data-interface-struct-defs>`. It then adds the
+following free-standing definitions. Like the rest of the Arrow project,
+they are available under the Apache License 2.0.
+
+.. code-block:: c
+
+    #ifndef ARROW_C_DEVICE_DATA_INTERFACE
+    #define ARROW_C_DEVICE_DATA_INTERFACE
+
+    // Device type for the allocated memory
+    typedef int32_t ArrowDeviceType;
+
+    // CPU device, same as using ArrowArray directly
+    #define ARROW_DEVICE_CPU 1
+    // CUDA GPU Device
+    #define ARROW_DEVICE_CUDA 2
+    // Pinned CUDA CPU memory by cudaMallocHost
+    #define ARROW_DEVICE_CUDA_HOST 3
+    // OpenCL Device
+    #define ARROW_DEVICE_OPENCL 4
+    // Vulkan buffer for next-gen graphics
+    #define ARROW_DEVICE_VULKAN 7
+    // Metal for Apple GPU
+    #define ARROW_DEVICE_METAL 8
+    // Verilog simulator buffer
+    #define ARROW_DEVICE_VPI 9
+    // ROCm GPUs for AMD GPUs
+    #define ARROW_DEVICE_ROCM 10
+    // Pinned ROCm CPU memory allocated by hipMallocHost
+    #define ARROW_DEVICE_ROCM_HOST 11
+    // Reserved for extension
+    //
+    // used to quickly test extension devices, semantics
+    // can differ based on implementation
+    #define ARROW_DEVICE_EXT_DEV 12
+    // CUDA managed/unified memory allocated by cudaMallocManaged
+    #define ARROW_DEVICE_CUDA_MANAGED 13
+    // Unified shared memory allocated on a oneAPI
+    // non-partitioned device.
+    //
+    // A call to the oneAPI runtime is required to determine the
+    // device type, the USM allocation type and the sycl context
+    // that it is bound to.
+    #define ARROW_DEVICE_ONEAPI 14
+    // GPU support for next-gen WebGPU standard
+    #define ARROW_DEVICE_WEBGPU 15
+    // Qualcomm Hexagon DSP
+    #define ARROW_DEVICE_HEXAGON 16
+
+    struct ArrowDeviceArray {
+      struct ArrowArray array;
+      int64_t device_id;
+      ArrowDeviceType device_type;
+      void* sync_event;
+
+      // reserved bytes for future expansion
+      int64_t reserved[3];
+    };
+
+    #endif  // ARROW_C_DEVICE_DATA_INTERFACE
+
+.. note::
+   The canonical guard ``ARROW_C_DEVICE_DATA_INTERFACE`` is meant to avoid
+   duplicate definitions if two projects copy the definitions in their own
+   headers, and a third-party project includes from these two projects. It
+   is therefore important that this guard is kept exactly as-is when these
+   definitions are copied.
+
+ArrowDeviceType
+---------------
+
+The ``ArrowDeviceType`` typedef is used to indicate what type of device the
+provided memory buffers were allocated on. This, in conjunction with the
+``device_id``, should be sufficient to reference the correct data buffers.
+
+We then use macros to define values for different device types. The provided
+macro values are compatible with the widely used `dlpack`_ ``DLDeviceType``
+definition values, using the same value for each as the equivalent
+``kDL<type>`` enum from dlpack.h. The list will be kept in sync with those
+equivalent enum values over time to ensure compatibility, rather than
+potentially diverging. To avoid the Arrow project having to be in the
+position of vetting new hardware devices, new additions should first be
+added to dlpack before we add a corresponding macro here.
+
+To ensure predictability with the ABI, we use macros instead of an ``enum``
+so the storage type is not compiler dependent.
+
+.. c:macro:: ARROW_DEVICE_CPU
+
+    CPU Device, equivalent to just using ``ArrowArray`` directly instead of
+    using ``ArrowDeviceArray``.
+
+.. c:macro:: ARROW_DEVICE_CUDA
+
+    A `CUDA`_ GPU Device. This could represent data allocated either with the
+    runtime library (``cudaMalloc``) or the device driver (``cuMemAlloc``).
+
+.. c:macro:: ARROW_DEVICE_CUDA_HOST
+
+    CPU memory that was pinned and page-locked by CUDA by using
+    ``cudaMallocHost`` or ``cuMemAllocHost``.
+
+.. c:macro:: ARROW_DEVICE_OPENCL
+
+    Data allocated on the device by using the `OpenCL (Open Computing Language)`_
+    framework.
+
+.. c:macro:: ARROW_DEVICE_VULKAN
+
+    Data allocated by the `Vulkan`_ framework and libraries.
+
+.. c:macro:: ARROW_DEVICE_METAL
+
+    Data on Apple GPU devices using the `Metal`_ framework and libraries.
+
+.. c:macro:: ARROW_DEVICE_VPI
+
+    Indicates usage of a Verilog simulator buffer.
+
+.. c:macro:: ARROW_DEVICE_ROCM
+
+    An AMD device using the `ROCm`_ stack.
+
+.. c:macro:: ARROW_DEVICE_ROCM_HOST
+
+    CPU memory pinned and page-locked allocated ``hipMallocHost``.
+
+.. c:macro:: ARROW_DEVICE_EXT_DEV
+
+    This value is an escape-hatch for devices to extend which aren't
+    currently represented otherwise. Producers would need to provide
+    additional information/context specific to the device if using
+    this device type. This is used to quickly test extension devices
+    and semantics can differ based on the implementation.
+
+.. c:macro:: ARROW_DEVICE_CUDA_MANAGED
+
+    CUDA managed/unified memory which is allocated by ``cudaMallocManaged``.
+
+.. c:macro:: ARROW_DEVICE_ONEAPI
+
+    Unified shared memory allocated on an Intel `oneAPI`_ non-partitioned
+    device. A call to the ``oneAPI`` runtime is required to determine
+    the specific device type, the USM allocation type and the sycl context
+    that it is bound to.
+
+.. c:macro:: ARROW_DEVICE_WEBGPU
+
+    GPU support for next-gen WebGPU standards
+
+.. c:macro:: ARROW_DEVICE_HEXAGON
+
+    Data allocated on a Qualcomm Hexagon DSP device.
+
+The ArrowDeviceArray structure
+------------------------------
+
+The ``ArrowDeviceArray`` structure embeds the C data ``ArrowArray`` structure
+and adds additional information necessary for consumers to use the data. It
+has the following fields:
+
+.. c:member:: struct ArrowArray ArrowDeviceArray.array
+
+    The allocated array data. The values in the ``void**`` buffers (along
+    with the buffers of any children) are what is allocated on the device.
+    The buffer values should be device pointers. The rest of the structure
+    should be accessible to the CPU.
+
+    The ``private_data`` and ``release`` callback of this structure should
+    contain any necessary information and structures related to freeing
+    the array according to the device it is allocated on, rather than
+    having a separate release callback and ``private_data`` pointer here.
+
+.. c:member:: int64_t ArrowDeviceArray.device_id
+
+    The device id to identify a specific device if multiple devices of this
+    type are on the system. The semantics of the id will be hardware dependent,
+    but we use an ``int64_t`` to future-proof the id as devices change over time.
+
+.. c:member:: ArrowDeviceType ArrowDeviceArray.device_type
+
+    The type of the device which can access the buffers in the array.
+
+.. c:member:: void* ArrowDeviceArray.sync_event
+
+    Optional. An event-like object to synchronize on if needed.
+
+    Many devices, like GPUs, are primarily asynchronous with respect to
+    CPU processing. As such in order to safely access memory, it is often
+    necessary to have an object to synchronize processing with. Since
+    different devices will use different types to specify this, we use a
+    void* which can be coerced into a pointer to whatever the device
+    appropriate type is.
+
+    If synchronization is not needed, this can be null. If this is non-null
+    then it MUST be used to call the appropriate sync method for the device
+    (e.g. ``cudaStreamWaitEvent`` or ``hipStreamWaitEvent``) before attempting
+    to access the memory in the buffers.
+
+    Expected types to coerce this ``void*`` to depending on the reported
+    device type:
+
+    * CUDA: ``cudaEvent_t*``
+    * ROCm: ``hipEvent_t*``
+    * OpenCL: ``cl_event*``
+    * Vulkan: ``VkEvent*``
+    * Metal: ``MTLEvent*``
+    * OneAPI: ``sycl::event*``
+
+    If an event is provided, then the producer MUST ensure that the exported
+    data is available on the device before the event is triggered. The 
+    consumer SHOULD wait on the event before trying to access the exported
+    data.
+
+.. c:member:: int64_t ArrowDeviceArray.reserved[3]
+
+    As non-CPU development expands, there may be a need to expand this
+    structure. In order to do so without potentially breaking ABI changes,
+    we reserve 24 bytes at the end of the object. This also has the added
+    benefit of bringing the total size of this structure to exactly 128
+    bytes (a power of 2) on 64-bit systems. These bytes MUST be zero'd
+    out after initialization by the producer in order to ensure safe 
+    evolution of the ABI in the future.
+
+.. note::
+    Rather than store the shape / types of the data alongside the
+    ``ArrowDeviceArray``, users should utilize the existing ``ArrowSchema``
+    structure to pass any data type and shape information.
+
+Synchronization event types
+---------------------------
+
+The table below lists the expected event types for each device type.
+If no event type is supported ("N/A"), then the ``sync_event`` member
+should always be null. 
+
+Remember that the event *CAN* be null if synchronization is not needed
+to access the data.
+
++---------------------------+--------------------+---------+
+| Device Type               | Actual Event Type  | Notes   |
++===========================+====================+=========+
+| ARROW_DEVICE_CPU          | N/A                |         |
++---------------------------+--------------------+---------+
+| ARROW_DEVICE_CUDA         | ``cudaEvent_t*``   |         |
++---------------------------+--------------------+---------+
+| ARROW_DEVICE_CUDA_HOST    | ``cudaEvent_t*``   |         |
++---------------------------+--------------------+---------+
+| ARROW_DEVICE_OPENCL       | ``cl_event*``      |         |
++---------------------------+--------------------+---------+
+| ARROW_DEVICE_VULKAN       | ``VkEvent*``       |         |
++---------------------------+--------------------+---------+
+| ARROW_DEVICE_METAL        | ``MTLEvent*``      |         |
++---------------------------+--------------------+---------+
+| ARROW_DEVICE_VPI          | N/A                | (1)     |
++---------------------------+--------------------+---------+
+| ARROW_DEVICE_ROCM         | ``hipEvent_t*``    |         |
++---------------------------+--------------------+---------+
+| ARROW_DEVICE_ROCM_HOST    | ``hipEvent_t*``    |         |
++---------------------------+--------------------+---------+
+| ARROW_DEVICE_EXT_DEV      |                    | (2)     |
++---------------------------+--------------------+---------+
+| ARROW_DEVICE_CUDA_MANAGED | ``cudaEvent_t*``   |         |
++---------------------------+--------------------+---------+
+| ARROW_DEVICE_ONEAPI       | ``sycl::event*``   |         |
++---------------------------+--------------------+---------+
+| ARROW_DEVICE_WEBGPU       | N/A                | (1)     |
++---------------------------+--------------------+---------+
+| ARROW_DEVICE_HEXAGON      | N/A                | (1)     |
++---------------------------+--------------------+---------+
+
+Notes:
+
+* \(1) Currently unknown if framework has an event type to support.
+* \(2) Extension Device has producer defined semantics and thus if
+       synchronization is needed for an extension device, the producer
+       should document the type.
+
+
+Semantics
+=========
+
+Memory management
+-----------------
+
+The ``ArrowDeviceArray`` structure contains an ``ArrowArray`` object which
+itself has :ref:`specific semantics <c-data-interface-semantics>` for releasing
+memory. The term *"base structure"* below refers to the ``ArrowDeviceArray``
+object that is passed directly between the producer and consumer -- not any
+child structure thereof.
+
+It is intended for the base structure to be stack- or heap-allocated by the
+*consumer*. In this case, the producer API should take a pointer to the
+consumer-allocated structure.
+
+However, any data pointed to by the struct MUST be allocated and maintained
+by the producer. This includes the ``sync_event`` member if it is not null,
+along with any pointers in the ``ArrowArray`` object as usual. Data lifetime
+is managed through the ``release`` callback of the ``ArrowArray`` member.
+
+For an ``ArrowDeviceArray``, the semantics of a released structure and the
+callback semantics are identical to those for
+:ref:`ArrowArray itself <c-data-interface-released>`. Any producer specific context
+information necessary for releasing the device data buffers, in addition to
+any allocated event, should be stored in the ``private_data`` member of
+the ``ArrowArray`` and managed by the ``release`` callback.
+
+Moving an array
+'''''''''''''''
+
+The consumer can *move* the ``ArrowDeviceArray`` structure by bitwise copying
+or shallow member-wise copying. Then it MUST mark the source structure released
+by setting the ``release`` member of the embedded ``ArrowArray`` structure to
+``NULL``, but *without* calling that release callback. This ensures that only
+one live copy of the struct is active at any given time and that lifetime is
+correctly communicated to the producer.
+
+As usual, the release callback will be called on the destination structure
+when it is not needed anymore.
+
+Record batches
+--------------
+As with the C data interface itself, a record batch can be trivially considered
+as an equivalent struct array. In this case the metadata of the top-level
+``ArrowSchema`` can be used for schema-level metadata of the record batch.
+
+Mutability
+----------
+
+Both the producer and the consumer SHOULD consider the exported data (that
+is, the data reachable on the device through the ``buffers`` member of
+the embedded ``ArrowArray``) to be immutable, as either party could otherwise
+see inconsistent data while the other is mutating it.
+
+Synchronization
+---------------
+
+If the ``sync_event`` member is non-NULL, the consumer should not attempt 
+to access or read the data until they have synchronized on that event. If
+the ``sync_event`` member is NULL, then it MUST be safe to access the data
+without any synchronization necessary on the part of the consumer.
+
+C producer example
+====================
+
+Exporting a simple ``int32`` device array
+-----------------------------------------
+
+Export a non-nullable ``int32`` type with empty metadata. An example of this
+can be seen in the :ref:`C data interface docs directly <c-data-interface-export-int32-schema>`.
+
+To export the data itself, we transfer ownership to the consumer through
+the release callback. This example will use CUDA, but the equivalent calls
+could be used for any device:
+
+.. code-block:: c
+
+    static void release_int32_device_array(struct ArrowArray* array) {
+        assert(array->n_buffers == 2);
+        // destroy the event
+        cudaEvent_t* ev_ptr = (cudaEvent_t*)(array->private_data);
+        cudaError_t status = cudaEventDestroy(*ev_ptr);
+        assert(status == cudaSuccess);
+        free(ev_ptr);
+
+        // free the buffers and the buffers array
+        status = cudaFree(array->buffers[1]);
+        assert(status == cudaSuccess);
+        free(array->buffers);
+
+        // mark released
+        array->release = NULL;
+    }
+
+    void export_int32_device_array(void* cudaAllocdPtr,
+                                   cudaStream_t stream,
+                                   int64_t length,
+                                   struct ArrowDeviceArray* array) {
+        // get device id
+        int device;
+        cudaError_t status;
+        status = cudaGetDevice(&device);
+        assert(status == cudaSuccess);
+
+        cudaEvent_t* ev_ptr = (cudaEvent_t*)malloc(sizeof(cudaEvent_t));
+        assert(ev_ptr != NULL);
+        status = cudaEventCreate(ev_ptr);
+        assert(status == cudaSuccess);
+
+        // record event on the stream, assuming that the passed in
+        // stream is where the work to produce the data will be processing.
+        status = cudaEventRecord(*ev_ptr, stream);
+        assert(status == cudaSuccess);
+
+        // initialize fields
+        *array = (struct ArrowDeviceArray) {
+            .array = (struct ArrowArray) {
+                .length = length,
+                .null_count = 0,
+                .offset = 0,
+                .n_buffers = 2,
+                .n_children = 0,
+                .children = NULL,
+                .dictionary = NULL,
+                // bookeeping

Review Comment:
   ```suggestion
                   // bookkeeping
   ```



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [arrow] zeroshade merged pull request #34972: GH-34971: [Format] Add non-CPU version of C Data Interface

["zeroshade (via GitHub)" <gi...@apache.org>]

zeroshade merged PR #34972:
URL: https://github.com/apache/arrow/pull/34972


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [arrow] wgtmac commented on a diff in pull request #34972: GH-34971: [Format] Enhance C-Data API to support non-cpu cases

["wgtmac (via GitHub)" <gi...@apache.org>]

wgtmac commented on code in PR #34972:
URL: https://github.com/apache/arrow/pull/34972#discussion_r1212810710


##########
cpp/src/arrow/c/abi.h:
##########
@@ -65,6 +85,119 @@ struct ArrowArray {
 
 #endif  // ARROW_C_DATA_INTERFACE
 
+#ifndef ARROW_C_DEVICE_DATA_INTERFACE
+#define ARROW_C_DEVICE_DATA_INTERFACE
+
+/// \defgroup arrow-device-types Device Types
+/// These macros are compatible with the dlpack DLDeviceType values,
+/// using the same value for each enum as the equivalent kDL<type>
+/// from dlpack.h. This list should continue to be kept in sync with
+/// the equivalent dlpack.h enum values over time to ensure
+/// compatibility, rather than potentially diverging.
+///
+/// To ensure predictability with the ABI we use macros instead of
+/// an enum so the storage type is not compiler dependent.
+///
+/// @{
+
+/// \brief DeviceType for the allocated memory
+typedef int32_t ArrowDeviceType;
+
+/// \brief CPU device, same as using ArrowArray directly
+#define ARROW_DEVICE_CPU 1
+/// \brief CUDA GPU Device
+#define ARROW_DEVICE_CUDA 2
+/// \brief Pinned CUDA CPU memory by cudaMallocHost
+#define ARROW_DEVICE_CUDA_HOST 3
+/// \brief OpenCL Device
+#define ARROW_DEVICE_OPENCL 4
+/// \brief Vulkan buffer for next-gen graphics
+#define ARROW_DEVICE_VULKAN 7
+/// \brief Metal for Apple GPU
+#define ARROW_DEVICE_METAL 8
+/// \brief Verilog simulator buffer
+#define ARROW_DEVICE_VPI 9
+/// \brief ROCm GPUs for AMD GPUs
+#define ARROW_DEVICE_ROCM 10
+/// \brief Pinned ROCm CPU memory allocated by hipMallocHost
+#define ARROW_DEVICE_ROCM_HOST 11
+/// \brief Reserved for extension
+///
+/// used to quickly test extension devices, semantics
+/// can differ based on the implementation
+#define ARROW_DEVICE_EXT_DEV 12
+/// \brief CUDA managed/unified memory allocated by cudaMallocManaged
+#define ARROW_DEVICE_CUDA_MANAGED 13
+/// \brief unified shared memory allocated on a oneAPI
+/// non-partitioned device.
+///
+/// A call to the oneAPI runtime is required to determine the device
+/// type, the USM allocation type, and the sycl context it is bound to.
+#define ARROW_DEVICE_ONEAPI 14
+/// \brief GPU support for next-gen WebGPU standard
+#define ARROW_DEVICE_WEBGPU 15
+/// \brief Qualcomm Hexagon DSP
+#define ARROW_DEVICE_HEXAGON 16
+
+/// @}
+
+/// \brief Struct for passing an Arrow Array alongside
+/// device memory information.
+struct ArrowDeviceArray {
+  /// \brief the Allocated Array
+  ///
+  /// the buffers in the array (along with the buffers of any
+  /// children) are what is allocated on the device.
+  ///
+  /// the private_data and release callback of the arrow array
+  /// should contain any necessary information and structures
+  /// related to freeing the array according to the device it
+  /// is allocated on, rather than having a separate release
+  /// callback embedded here.
+  struct ArrowArray array;
+  /// \brief The device id to identify a specific device
+  /// if multiple of this type are on the system.
+  ///
+  /// the semantics of the id will be hardware dependant.

Review Comment:
   ```suggestion
     /// the semantics of the id will be hardware dependent.
   ```



##########
cpp/src/arrow/c/abi.h:
##########
@@ -65,6 +85,119 @@ struct ArrowArray {
 
 #endif  // ARROW_C_DATA_INTERFACE
 
+#ifndef ARROW_C_DEVICE_DATA_INTERFACE
+#define ARROW_C_DEVICE_DATA_INTERFACE
+
+/// \defgroup arrow-device-types Device Types
+/// These macros are compatible with the dlpack DLDeviceType values,
+/// using the same value for each enum as the equivalent kDL<type>
+/// from dlpack.h. This list should continue to be kept in sync with
+/// the equivalent dlpack.h enum values over time to ensure
+/// compatibility, rather than potentially diverging.
+///
+/// To ensure predictability with the ABI we use macros instead of
+/// an enum so the storage type is not compiler dependent.
+///
+/// @{
+
+/// \brief DeviceType for the allocated memory
+typedef int32_t ArrowDeviceType;
+
+/// \brief CPU device, same as using ArrowArray directly
+#define ARROW_DEVICE_CPU 1
+/// \brief CUDA GPU Device
+#define ARROW_DEVICE_CUDA 2
+/// \brief Pinned CUDA CPU memory by cudaMallocHost
+#define ARROW_DEVICE_CUDA_HOST 3
+/// \brief OpenCL Device
+#define ARROW_DEVICE_OPENCL 4
+/// \brief Vulkan buffer for next-gen graphics
+#define ARROW_DEVICE_VULKAN 7
+/// \brief Metal for Apple GPU
+#define ARROW_DEVICE_METAL 8
+/// \brief Verilog simulator buffer
+#define ARROW_DEVICE_VPI 9
+/// \brief ROCm GPUs for AMD GPUs
+#define ARROW_DEVICE_ROCM 10
+/// \brief Pinned ROCm CPU memory allocated by hipMallocHost
+#define ARROW_DEVICE_ROCM_HOST 11
+/// \brief Reserved for extension
+///
+/// used to quickly test extension devices, semantics
+/// can differ based on the implementation
+#define ARROW_DEVICE_EXT_DEV 12
+/// \brief CUDA managed/unified memory allocated by cudaMallocManaged
+#define ARROW_DEVICE_CUDA_MANAGED 13
+/// \brief unified shared memory allocated on a oneAPI
+/// non-partitioned device.
+///
+/// A call to the oneAPI runtime is required to determine the device
+/// type, the USM allocation type, and the sycl context it is bound to.
+#define ARROW_DEVICE_ONEAPI 14
+/// \brief GPU support for next-gen WebGPU standard
+#define ARROW_DEVICE_WEBGPU 15
+/// \brief Qualcomm Hexagon DSP
+#define ARROW_DEVICE_HEXAGON 16
+
+/// @}
+
+/// \brief Struct for passing an Arrow Array alongside
+/// device memory information.
+struct ArrowDeviceArray {
+  /// \brief the Allocated Array
+  ///
+  /// the buffers in the array (along with the buffers of any
+  /// children) are what is allocated on the device.
+  ///
+  /// the private_data and release callback of the arrow array
+  /// should contain any necessary information and structures
+  /// related to freeing the array according to the device it
+  /// is allocated on, rather than having a separate release
+  /// callback embedded here.
+  struct ArrowArray array;
+  /// \brief The device id to identify a specific device
+  /// if multiple of this type are on the system.
+  ///
+  /// the semantics of the id will be hardware dependant.
+  int64_t device_id;
+  /// \brief The type of device which can access this memory.
+  ArrowDeviceType device_type;
+  /// \brief An event-like object to synchronize on if needed.
+  ///
+  /// Many devices, like GPUs, are primarily asynchronous with
+  /// respect to CPU processing. As such in order to safely access
+  /// memory, it is often necessary to have an object to synchronize
+  /// processing on. Since different devices will use different types
+  /// to specify this we use a void* that can be coerced into
+  /// whatever the device appropriate type is (e.g. cudaEvent_t for
+  /// CUDA and hipEvent_t for HIP).
+  ///
+  /// If synchronization is not needed this can be null. If this is
+  /// non-null, then it should be used to call the appropriate sync
+  /// method for the device (e.g. cudaStreamWaitEvent / hipStreamWaitEvent).
+  ///
+  /// Expected type to coerce this void* to depending on device type:
+  ///   cuda: cudaEvent_t*

Review Comment:
   ```suggestion
     ///   CUDA: cudaEvent_t*
   ```



##########
cpp/src/arrow/c/abi.h:
##########
@@ -106,6 +239,75 @@ struct ArrowArrayStream {
 
 #endif  // ARROW_C_STREAM_INTERFACE
 
+#ifndef ARROW_C_DEVICE_STREAM_INTERFACE
+#define ARROW_C_DEVICE_STREAM_INTERFACE
+
+/// \brief Equivalent to ArrowArrayStream, but for ArrowDeviceArrays.
+///
+/// This stream is intended to provide a stream of data on a single
+/// device, if a producer wants data to be produced on multiple devices
+/// then multiple streams should be provided. One per device.
+struct ArrowDeviceArrayStream {
+  /// \brief The device that this stream produces data on.
+  ///
+  /// All ArrowDeviceArrays that are produced by this
+  /// stream should have the same device_type as set
+  /// here. Including it here in the stream object is
+  /// a convenience to allow consumers simpler processing
+  /// since they can assume all arrays that result from
+  /// this stream to be on this device type.
+  ArrowDeviceType device_type;
+
+  /// \brief Callback to get the stream schema
+  /// (will be the same for all arrays in the stream).
+  ///
+  /// If successful, the ArrowSchema must be released independantly from the stream.

Review Comment:
   ```suggestion
     /// If successful, the ArrowSchema must be released independently from the stream.
   ```



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [arrow] kkraus14 commented on a diff in pull request #34972: GH-34971: [Format] Enhance C-Data API to support non-cpu cases

["kkraus14 (via GitHub)" <gi...@apache.org>]

kkraus14 commented on code in PR #34972:
URL: https://github.com/apache/arrow/pull/34972#discussion_r1164215682


##########
cpp/src/arrow/c/abi.h:
##########
@@ -106,6 +212,98 @@ struct ArrowArrayStream {
 
 #endif  // ARROW_C_STREAM_INTERFACE
 
+#ifndef ARROW_C_DEVICE_STREAM_INTERFACE
+#define ARROW_C_DEVICE_STREAM_INTERFACE
+
+/// \brief Equivalent to ArrowArrayStream, but for ArrowDeviceArrays.
+///
+/// This stream is intended to provide a stream of data on a single
+/// device, if a producer wants data to be produced on multiple devices
+/// then multiple streams should be provided. One per device.
+struct ArrowDeviceArrayStream {
+  /// \brief The device that this stream produces data on.
+  ///
+  /// All ArrowDeviceArrays that are produced by this
+  /// stream should have the same device_type as set
+  /// here. The device_type needs to be provided here
+  /// so that consumers can provide the correct type
+  /// of queue_ptr when calling get_next.
+  ArrowDeviceType device_type;
+
+  /// \brief Callback to get the stream schema
+  /// (will be the same for all arrays in the stream).
+  ///
+  /// If successful, the ArrowSchema must be released independantly from the stream.
+  /// The schema should be accessible via CPU memory.
+  ///
+  /// \param[in] self The ArrowDeviceArrayStream object itself
+  /// \param[out] out C struct to export the schema to
+  /// \return 0 if successful, an `errno`-compatible error code otherwise.
+  int (*get_schema)(struct ArrowDeviceArrayStream* self, struct ArrowSchema* out);
+
+  /// \brief Callback to get the device id for the next array.
+  ///
+  /// This is necessary so that the proper/correct stream pointer can be provided
+  /// to get_next.
+  ///
+  /// The next call to `get_next` should provide an ArrowDeviceArray whose
+  /// device_id matches what is provided here, and whose device_type is the
+  /// same as the device_type member of this stream.
+  ///
+  /// \param[in] self The ArrowDeviceArrayStream object itself
+  /// \param[out] out_device_id Pointer to be populated with the device id, must not be
+  /// null \return 0 if successful, an `errno`-compatible error code otherwise.
+  int (*get_next_device_id)(struct ArrowDeviceArrayStream* self, int64_t* out_device_id);
+
+  /// \brief Callback to get the next array
+  ///
+  /// If there is no error and the returned array has been released, the stream
+  /// has ended. If successful, the ArrowArray must be released independently
+  /// from the stream.
+  ///
+  /// Because different frameworks use different types to represent this, we
+  /// accept a void* which should then be reinterpreted into whatever the
+  /// appropriate type is (e.g. cudaStream_t) for use by the producer.
+  ///
+  /// \param[in] self The ArrowDeviceArrayStream object itself
+  /// \param[in] queue_ptr The appropriate queue, stream, or
+  /// equivalent object for the device that the data is allocated on
+  /// to indicate where the consumer wants the data to be accessible.
+  /// If queue_ptr is NULL then the default stream (e.g. CUDA stream 0)
+  /// should be used to ensure that the memory is accessible from any stream.

Review Comment:
   > can the user request that Tensorflow data be tied to a specific CUDA stream?
   
   No a user can't request that.
   
   > Otherwise, how can we implement the "consumer-provided CUDA stream" semantics?
   
   Tensorflow can handle the consumer provided stream internally as part of it's implementation in producing dlpack or Arrow C Data Interface structs.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [arrow] zeroshade commented on a diff in pull request #34972: GH-34971: [Format] Enhance C-Data API to support non-cpu cases

["zeroshade (via GitHub)" <gi...@apache.org>]

zeroshade commented on code in PR #34972:
URL: https://github.com/apache/arrow/pull/34972#discussion_r1161999635


##########
cpp/src/arrow/c/abi.h:
##########
@@ -106,6 +212,98 @@ struct ArrowArrayStream {
 
 #endif  // ARROW_C_STREAM_INTERFACE
 
+#ifndef ARROW_C_DEVICE_STREAM_INTERFACE
+#define ARROW_C_DEVICE_STREAM_INTERFACE
+
+/// \brief Equivalent to ArrowArrayStream, but for ArrowDeviceArrays.
+///
+/// This stream is intended to provide a stream of data on a single
+/// device, if a producer wants data to be produced on multiple devices
+/// then multiple streams should be provided. One per device.
+struct ArrowDeviceArrayStream {
+  /// \brief The device that this stream produces data on.
+  ///
+  /// All ArrowDeviceArrays that are produced by this
+  /// stream should have the same device_type as set
+  /// here. The device_type needs to be provided here
+  /// so that consumers can provide the correct type
+  /// of queue_ptr when calling get_next.
+  ArrowDeviceType device_type;
+
+  /// \brief Callback to get the stream schema
+  /// (will be the same for all arrays in the stream).
+  ///
+  /// If successful, the ArrowSchema must be released independantly from the stream.
+  /// The schema should be accessible via CPU memory.
+  ///
+  /// \param[in] self The ArrowDeviceArrayStream object itself
+  /// \param[out] out C struct to export the schema to
+  /// \return 0 if successful, an `errno`-compatible error code otherwise.
+  int (*get_schema)(struct ArrowDeviceArrayStream* self, struct ArrowSchema* out);
+
+  /// \brief Callback to get the device id for the next array.
+  ///
+  /// This is necessary so that the proper/correct stream pointer can be provided
+  /// to get_next.
+  ///
+  /// The next call to `get_next` should provide an ArrowDeviceArray whose
+  /// device_id matches what is provided here, and whose device_type is the
+  /// same as the device_type member of this stream.

Review Comment:
   ultimately this is a consequence of the fact that the existing frameworks and APIs don't provide any good way to manage the stream's lifetime easily which makes having the consumer pass the stream be the safest route to take.
   
   I'm absolutely open to suggestions to make this better as long as the consumer is able to pass in the desired stream.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [arrow] pitrou commented on a diff in pull request #34972: GH-34971: [Format] Enhance C-Data API to support non-cpu cases

["pitrou (via GitHub)" <gi...@apache.org>]

pitrou commented on code in PR #34972:
URL: https://github.com/apache/arrow/pull/34972#discussion_r1161087835


##########
cpp/src/arrow/c/abi.h:
##########
@@ -106,6 +169,77 @@ struct ArrowArrayStream {
 
 #endif  // ARROW_C_STREAM_INTERFACE
 
+#ifndef ARROW_C_DEVICE_STREAM_INTERFACE
+#define ARROW_C_DEVICE_STREAM_INTERFACE
+
+struct ArrowDeviceArrayStream {
+  // The device that this stream produces data on.
+  // All ArrowDeviceArrays that are produced by this
+  // stream should have the same device_type as set
+  // here. The device_type needs to be provided here
+  // so that consumers can provide the correct type
+  // of stream_ptr when calling get_next.
+  ArrowDeviceType device_type;
+
+  // Callback to get the stream schema
+  // (will be the same for all arrays in the stream).
+  //
+  // Return value: 0 if successful, an `errno`-compatible error code otherwise.
+  //
+  // If successful, the ArrowSchema must be released independently from the stream.
+  int (*get_schema)(struct ArrowDeviceArrayStream*, struct ArrowSchema* out);
+
+  // Callback to get the device id for the next array.
+  // This is necessary so that the proper/correct stream pointer can be provided
+  // to get_next. The parameter provided must not be null.
+  //
+  // Return value: 0 if successful, an `errno`-compatible error code otherwise.
+  //
+  // The next call to `get_next` should provide an ArrowDeviceArray whose
+  // device_id matches what is provided here, and whose device_type is the
+  // same as the device_type member of this stream.
+  int (*get_next_device_id)(struct ArrowDeviceArrayStream*, int* out_device_id);
+
+  // Callback to get the next array
+  // (if no error and the array is released, the stream has ended)
+  //
+  // the provided stream_ptr should be the appropriate stream, or
+  // equivalent object, for the device that the data is allocated on
+  // to indicate where the consumer wants the data to be accessible.

Review Comment:
   In the CUDA Array Interface, it's the producer that optionally fills in the stream that must be synchronized on to access the data:
   https://numba.readthedocs.io/en/stable/cuda/cuda_array_interface.html#synchronization
   
   Besides, there's an interesting statement about stream lifetime here:
   https://numba.readthedocs.io/en/stable/cuda/cuda_array_interface.html#streams
   



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [arrow] zeroshade commented on a diff in pull request #34972: GH-34971: [Format] Enhance C-Data API to support non-cpu cases

["zeroshade (via GitHub)" <gi...@apache.org>]

zeroshade commented on code in PR #34972:
URL: https://github.com/apache/arrow/pull/34972#discussion_r1161301156


##########
cpp/src/arrow/c/abi.h:
##########
@@ -106,6 +169,77 @@ struct ArrowArrayStream {
 
 #endif  // ARROW_C_STREAM_INTERFACE
 
+#ifndef ARROW_C_DEVICE_STREAM_INTERFACE
+#define ARROW_C_DEVICE_STREAM_INTERFACE
+
+struct ArrowDeviceArrayStream {
+  // The device that this stream produces data on.
+  // All ArrowDeviceArrays that are produced by this
+  // stream should have the same device_type as set
+  // here. The device_type needs to be provided here
+  // so that consumers can provide the correct type
+  // of stream_ptr when calling get_next.
+  ArrowDeviceType device_type;
+
+  // Callback to get the stream schema
+  // (will be the same for all arrays in the stream).
+  //
+  // Return value: 0 if successful, an `errno`-compatible error code otherwise.
+  //
+  // If successful, the ArrowSchema must be released independently from the stream.
+  int (*get_schema)(struct ArrowDeviceArrayStream*, struct ArrowSchema* out);
+
+  // Callback to get the device id for the next array.
+  // This is necessary so that the proper/correct stream pointer can be provided
+  // to get_next. The parameter provided must not be null.
+  //
+  // Return value: 0 if successful, an `errno`-compatible error code otherwise.
+  //
+  // The next call to `get_next` should provide an ArrowDeviceArray whose
+  // device_id matches what is provided here, and whose device_type is the
+  // same as the device_type member of this stream.
+  int (*get_next_device_id)(struct ArrowDeviceArrayStream*, int* out_device_id);
+
+  // Callback to get the next array
+  // (if no error and the array is released, the stream has ended)
+  //
+  // the provided stream_ptr should be the appropriate stream, or

Review Comment:
   This is referring to whichever type is used by the appropriate backend to represent a stream (or should be null if the backend doesn't utilize this concept). 
   
   I'll update the comments to be more explicit about this.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [arrow] github-actions[bot] commented on pull request #34972: GH-34971: [Format] Enhance C-Data API to support non-cpu cases

["github-actions[bot] (via GitHub)" <gi...@apache.org>]

github-actions[bot] commented on PR #34972:
URL: https://github.com/apache/arrow/pull/34972#issuecomment-1500457484

   * Closes: #34971


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [arrow] kkraus14 commented on a diff in pull request #34972: GH-34971: [Format] Enhance C-Data API to support non-cpu cases

["kkraus14 (via GitHub)" <gi...@apache.org>]

kkraus14 commented on code in PR #34972:
URL: https://github.com/apache/arrow/pull/34972#discussion_r1163122203


##########
cpp/src/arrow/c/abi.h:
##########
@@ -106,6 +212,98 @@ struct ArrowArrayStream {
 
 #endif  // ARROW_C_STREAM_INTERFACE
 
+#ifndef ARROW_C_DEVICE_STREAM_INTERFACE
+#define ARROW_C_DEVICE_STREAM_INTERFACE
+
+/// \brief Equivalent to ArrowArrayStream, but for ArrowDeviceArrays.
+///
+/// This stream is intended to provide a stream of data on a single
+/// device, if a producer wants data to be produced on multiple devices
+/// then multiple streams should be provided. One per device.
+struct ArrowDeviceArrayStream {
+  /// \brief The device that this stream produces data on.
+  ///
+  /// All ArrowDeviceArrays that are produced by this
+  /// stream should have the same device_type as set
+  /// here. The device_type needs to be provided here
+  /// so that consumers can provide the correct type
+  /// of queue_ptr when calling get_next.
+  ArrowDeviceType device_type;
+
+  /// \brief Callback to get the stream schema
+  /// (will be the same for all arrays in the stream).
+  ///
+  /// If successful, the ArrowSchema must be released independantly from the stream.
+  /// The schema should be accessible via CPU memory.
+  ///
+  /// \param[in] self The ArrowDeviceArrayStream object itself
+  /// \param[out] out C struct to export the schema to
+  /// \return 0 if successful, an `errno`-compatible error code otherwise.
+  int (*get_schema)(struct ArrowDeviceArrayStream* self, struct ArrowSchema* out);
+
+  /// \brief Callback to get the device id for the next array.
+  ///
+  /// This is necessary so that the proper/correct stream pointer can be provided
+  /// to get_next.
+  ///
+  /// The next call to `get_next` should provide an ArrowDeviceArray whose
+  /// device_id matches what is provided here, and whose device_type is the
+  /// same as the device_type member of this stream.
+  ///
+  /// \param[in] self The ArrowDeviceArrayStream object itself
+  /// \param[out] out_device_id Pointer to be populated with the device id, must not be
+  /// null \return 0 if successful, an `errno`-compatible error code otherwise.
+  int (*get_next_device_id)(struct ArrowDeviceArrayStream* self, int64_t* out_device_id);
+
+  /// \brief Callback to get the next array
+  ///
+  /// If there is no error and the returned array has been released, the stream
+  /// has ended. If successful, the ArrowArray must be released independently
+  /// from the stream.
+  ///
+  /// Because different frameworks use different types to represent this, we
+  /// accept a void* which should then be reinterpreted into whatever the
+  /// appropriate type is (e.g. cudaStream_t) for use by the producer.
+  ///
+  /// \param[in] self The ArrowDeviceArrayStream object itself
+  /// \param[in] queue_ptr The appropriate queue, stream, or
+  /// equivalent object for the device that the data is allocated on
+  /// to indicate where the consumer wants the data to be accessible.
+  /// If queue_ptr is NULL then the default stream (e.g. CUDA stream 0)
+  /// should be used to ensure that the memory is accessible from any stream.

Review Comment:
   > Consumer gets a C Data Interface export of the previously produced data; the ArrowArray's release callback is responsible for handling the lifetime of stream 3, in addition to the lifetime of the allocated data
   
   This isn't possible for many frameworks as they manage streams internally and don't have an easy way to release or share ownership of a stream to be controlled in the release callback.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [arrow] pitrou commented on a diff in pull request #34972: GH-34971: [Format] Enhance C-Data API to support non-cpu cases

["pitrou (via GitHub)" <gi...@apache.org>]

pitrou commented on code in PR #34972:
URL: https://github.com/apache/arrow/pull/34972#discussion_r1163123179


##########
cpp/src/arrow/c/abi.h:
##########
@@ -106,6 +212,98 @@ struct ArrowArrayStream {
 
 #endif  // ARROW_C_STREAM_INTERFACE
 
+#ifndef ARROW_C_DEVICE_STREAM_INTERFACE
+#define ARROW_C_DEVICE_STREAM_INTERFACE
+
+/// \brief Equivalent to ArrowArrayStream, but for ArrowDeviceArrays.
+///
+/// This stream is intended to provide a stream of data on a single
+/// device, if a producer wants data to be produced on multiple devices
+/// then multiple streams should be provided. One per device.
+struct ArrowDeviceArrayStream {
+  /// \brief The device that this stream produces data on.
+  ///
+  /// All ArrowDeviceArrays that are produced by this
+  /// stream should have the same device_type as set
+  /// here. The device_type needs to be provided here
+  /// so that consumers can provide the correct type
+  /// of queue_ptr when calling get_next.
+  ArrowDeviceType device_type;
+
+  /// \brief Callback to get the stream schema
+  /// (will be the same for all arrays in the stream).
+  ///
+  /// If successful, the ArrowSchema must be released independantly from the stream.
+  /// The schema should be accessible via CPU memory.
+  ///
+  /// \param[in] self The ArrowDeviceArrayStream object itself
+  /// \param[out] out C struct to export the schema to
+  /// \return 0 if successful, an `errno`-compatible error code otherwise.
+  int (*get_schema)(struct ArrowDeviceArrayStream* self, struct ArrowSchema* out);
+
+  /// \brief Callback to get the device id for the next array.
+  ///
+  /// This is necessary so that the proper/correct stream pointer can be provided
+  /// to get_next.
+  ///
+  /// The next call to `get_next` should provide an ArrowDeviceArray whose
+  /// device_id matches what is provided here, and whose device_type is the
+  /// same as the device_type member of this stream.
+  ///
+  /// \param[in] self The ArrowDeviceArrayStream object itself
+  /// \param[out] out_device_id Pointer to be populated with the device id, must not be
+  /// null \return 0 if successful, an `errno`-compatible error code otherwise.
+  int (*get_next_device_id)(struct ArrowDeviceArrayStream* self, int64_t* out_device_id);
+
+  /// \brief Callback to get the next array
+  ///
+  /// If there is no error and the returned array has been released, the stream
+  /// has ended. If successful, the ArrowArray must be released independently
+  /// from the stream.
+  ///
+  /// Because different frameworks use different types to represent this, we
+  /// accept a void* which should then be reinterpreted into whatever the
+  /// appropriate type is (e.g. cudaStream_t) for use by the producer.
+  ///
+  /// \param[in] self The ArrowDeviceArrayStream object itself
+  /// \param[in] queue_ptr The appropriate queue, stream, or
+  /// equivalent object for the device that the data is allocated on
+  /// to indicate where the consumer wants the data to be accessible.
+  /// If queue_ptr is NULL then the default stream (e.g. CUDA stream 0)
+  /// should be used to ensure that the memory is accessible from any stream.

Review Comment:
   > This isn't possible for many frameworks as they manage streams internally and don't have an easy way to release or share ownership of a stream to be controlled in the release callback.
   
   Sorry, but why not? Which kind of framework do you have in mind? 
   



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [arrow] zeroshade commented on a diff in pull request #34972: GH-34971: [Format] Enhance C-Data API to support non-cpu cases

["zeroshade (via GitHub)" <gi...@apache.org>]

zeroshade commented on code in PR #34972:
URL: https://github.com/apache/arrow/pull/34972#discussion_r1162000832


##########
cpp/src/arrow/c/abi.h:
##########
@@ -106,6 +212,98 @@ struct ArrowArrayStream {
 
 #endif  // ARROW_C_STREAM_INTERFACE
 
+#ifndef ARROW_C_DEVICE_STREAM_INTERFACE
+#define ARROW_C_DEVICE_STREAM_INTERFACE
+
+/// \brief Equivalent to ArrowArrayStream, but for ArrowDeviceArrays.
+///
+/// This stream is intended to provide a stream of data on a single
+/// device, if a producer wants data to be produced on multiple devices
+/// then multiple streams should be provided. One per device.
+struct ArrowDeviceArrayStream {
+  /// \brief The device that this stream produces data on.
+  ///
+  /// All ArrowDeviceArrays that are produced by this
+  /// stream should have the same device_type as set
+  /// here. The device_type needs to be provided here
+  /// so that consumers can provide the correct type
+  /// of queue_ptr when calling get_next.
+  ArrowDeviceType device_type;
+
+  /// \brief Callback to get the stream schema
+  /// (will be the same for all arrays in the stream).
+  ///
+  /// If successful, the ArrowSchema must be released independantly from the stream.
+  /// The schema should be accessible via CPU memory.
+  ///
+  /// \param[in] self The ArrowDeviceArrayStream object itself
+  /// \param[out] out C struct to export the schema to
+  /// \return 0 if successful, an `errno`-compatible error code otherwise.
+  int (*get_schema)(struct ArrowDeviceArrayStream* self, struct ArrowSchema* out);
+
+  /// \brief Callback to get the device id for the next array.
+  ///
+  /// This is necessary so that the proper/correct stream pointer can be provided
+  /// to get_next.
+  ///
+  /// The next call to `get_next` should provide an ArrowDeviceArray whose
+  /// device_id matches what is provided here, and whose device_type is the
+  /// same as the device_type member of this stream.
+  ///
+  /// \param[in] self The ArrowDeviceArrayStream object itself
+  /// \param[out] out_device_id Pointer to be populated with the device id, must not be
+  /// null \return 0 if successful, an `errno`-compatible error code otherwise.
+  int (*get_next_device_id)(struct ArrowDeviceArrayStream* self, int64_t* out_device_id);
+
+  /// \brief Callback to get the next array
+  ///
+  /// If there is no error and the returned array has been released, the stream
+  /// has ended. If successful, the ArrowArray must be released independently
+  /// from the stream.
+  ///
+  /// Because different frameworks use different types to represent this, we
+  /// accept a void* which should then be reinterpreted into whatever the
+  /// appropriate type is (e.g. cudaStream_t) for use by the producer.
+  ///
+  /// \param[in] self The ArrowDeviceArrayStream object itself
+  /// \param[in] queue_ptr The appropriate queue, stream, or
+  /// equivalent object for the device that the data is allocated on
+  /// to indicate where the consumer wants the data to be accessible.
+  /// If queue_ptr is NULL then the default stream (e.g. CUDA stream 0)
+  /// should be used to ensure that the memory is accessible from any stream.

Review Comment:
   @pitrou the issue is that there isn't a mechanism that you could call in the release callback (to my knowledge) to cleanly control the lifetime. (@kkraus14 correct me if I'm wrong and this isn't what you meant)



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [arrow] kkraus14 commented on a diff in pull request #34972: GH-34971: [Format] Enhance C-Data API to support non-cpu cases

["kkraus14 (via GitHub)" <gi...@apache.org>]

kkraus14 commented on code in PR #34972:
URL: https://github.com/apache/arrow/pull/34972#discussion_r1191511616


##########
cpp/src/arrow/c/abi.h:
##########
@@ -65,6 +85,111 @@ struct ArrowArray {
 
 #endif  // ARROW_C_DATA_INTERFACE
 
+#ifndef ARROW_C_DEVICE_DATA_INTERFACE
+#define ARROW_C_DEVICE_DATA_INTERFACE
+
+/// \defgroup arrow-device-types Device Types
+/// These macros are compatible with the dlpack DLDeviceType values,
+/// using the same value for each enum as the equivalent kDL<type>
+/// from dlpack.h. This list should continue to be kept in sync with
+/// the equivalent dlpack.h enum values over time to ensure 
+/// compatibility, rather than potentially diverging.
+///
+/// To ensure predictability with the ABI we use macros instead of
+/// an enum so the storage type is not compiler dependent.
+///
+/// @{
+
+/// \brief DeviceType for the allocated memory
+typedef int32_t ArrowDeviceType;
+
+/// \brief CPU device, same as using ArrowArray directly
+#define ARROW_DEVICE_CPU = 1
+/// \brief CUDA GPU Device
+#define ARROW_DEVICE_CUDA = 2
+/// \brief Pinned CUDA CPU memory by cudaMallocHost
+#define ARROW_DEVICE_CUDA_HOST = 3
+/// \brief OpenCL Device
+#define ARROW_DEVICE_OPENCL = 4
+/// \brief Vulkan buffer for next-gen graphics
+#define ARROW_DEVICE_VULKAN = 7
+/// \brief Metal for Apple GPU
+#define ARROW_DEVICE_METAL = 8
+/// \brief Verilog simulator buffer
+#define ARROW_DEVICE_VPI = 9
+/// \brief ROCm GPUs for AMD GPUs
+#define ARROW_DEVICE_ROCM = 10
+/// \brief Pinned ROCm CPU memory allocated by hipMallocHost
+#define ARROW_DEVICE_ROCMHOST = 11
+/// \brief Reserved for extension
+///
+/// used to quickly test extension devices, semantics
+/// can differ based on the implementation
+#define ARROW_DEVICE_EXT_DEV = 12
+/// \brief CUDA managed/unified memory allocated by cudaMallocManaged
+#define ARROW_DEVICE_CUDA_MANAGED = 13
+/// \brief unified shared memory allocated on a oneAPI
+/// non-partitioned device.
+///
+/// A call to the oneAPI runtime is required to determine the device
+/// type, the USM allocation type, and the sycl context it is bound to.
+#define ARROW_DEVICE_ONEAPI = 14
+/// \brief GPU support for next-gen WebGPU standard
+#define ARROW_DEVICE_WEBGPU = 15
+/// \brief Qualcomm Hexagon DSP
+#define ARROW_DEVICE_HEXAGON = 16
+
+/// @}
+
+/// \brief Struct for passing an Arrow Array alongside
+/// device memory information.
+struct ArrowDeviceArray {
+  /// \brief the Allocated Array
+  ///
+  /// the buffers in the array (along with the buffers of any
+  /// children) are what is allocated on the device.
+  ///
+  /// the private_data and release callback of the arrow array
+  /// should contain any necessary information and structures
+  /// related to freeing the array according to the device it
+  /// is allocated on, rather than having a separate release
+  /// callback embedded here.
+  struct ArrowArray array;
+  /// \brief The device id to identify a specific device
+  /// if multiple of this type are on the system.
+  ///
+  /// the semantics of the id will be hardware dependant.
+  int64_t device_id;
+  /// \brief The type of device which can access this memory.
+  ArrowDeviceType device_type;
+  /// \brief An event-like object to synchronize on if needed.
+  ///
+  /// Many devices, like GPUs, are primarily asynchronous with
+  /// respect to CPU processing. As such in order to safely access
+  /// memory, it is often necessary to have an object to synchronize
+  /// processing on. Since different devices will use different types

Review Comment:
   Beyond the `cuda` and `hip` paths which you referenced below I'm unclear on what other devices would pass here



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [arrow] zeroshade commented on pull request #34972: GH-34971: [Format] Enhance C-Data API to support non-cpu cases

["zeroshade (via GitHub)" <gi...@apache.org>]

zeroshade commented on PR #34972:
URL: https://github.com/apache/arrow/pull/34972#issuecomment-1560010548

   @pitrou I've added a full prose spec writeup for the documentation. Please have a look at your earliest convenience


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [arrow] zeroshade commented on a diff in pull request #34972: GH-34971: [Format] Enhance C-Data API to support non-cpu cases

["zeroshade (via GitHub)" <gi...@apache.org>]

zeroshade commented on code in PR #34972:
URL: https://github.com/apache/arrow/pull/34972#discussion_r1204346824


##########
docs/source/format/CDeviceDataInterface.rst:
##########
@@ -0,0 +1,613 @@
+.. Licensed to the Apache Software Foundation (ASF) under one
+.. or more contributor license agreements.  See the NOTICE file
+.. distributed with this work for additional information
+.. regarding copyright ownership.  The ASF licenses this file
+.. to you under the Apache License, Version 2.0 (the
+.. "License"); you may not use this file except in compliance
+.. with the License.  You may obtain a copy of the License at
+
+..   http://www.apache.org/licenses/LICENSE-2.0
+
+.. Unless required by applicable law or agreed to in writing,
+.. software distributed under the License is distributed on an
+.. "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+.. KIND, either express or implied.  See the License for the
+.. specific language governing permissions and limitations
+.. under the License.
+
+.. highlight:: c
+
+.. _c-device-data-interface:
+
+=================================
+The Arrow C Device data interface
+=================================
+
+.. note:: The Arrow C Device Data Interface should be considered experimental
+
+Rationale
+=========
+
+The current :ref:`C Data Interface <c-data-interface>`, and most
+implementations of it, make the assumption that all data buffers provided
+are CPU buffers. Since Apache Arrow is designed to be a universal in-memory
+format for representing tabular ("columnar") data, there will be the desire
+to leverage this data on Non-CPU hardware such as GPUs. One example of such
+a case is the `RAPIDS cuDF library` which uses the Arrow memory format with

Review Comment:
   sorry, not `:ref:`, but adding the `_` to indicate it. I'm still more used to md than rst lol



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [arrow] lidavidm commented on a diff in pull request #34972: GH-34971: [Format] Enhance C-Data API to support non-cpu cases

["lidavidm (via GitHub)" <gi...@apache.org>]

lidavidm commented on code in PR #34972:
URL: https://github.com/apache/arrow/pull/34972#discussion_r1202894556


##########
docs/source/format/CDeviceDataInterface.rst:
##########
@@ -0,0 +1,613 @@
+.. Licensed to the Apache Software Foundation (ASF) under one
+.. or more contributor license agreements.  See the NOTICE file
+.. distributed with this work for additional information
+.. regarding copyright ownership.  The ASF licenses this file
+.. to you under the Apache License, Version 2.0 (the
+.. "License"); you may not use this file except in compliance
+.. with the License.  You may obtain a copy of the License at
+
+..   http://www.apache.org/licenses/LICENSE-2.0
+
+.. Unless required by applicable law or agreed to in writing,
+.. software distributed under the License is distributed on an
+.. "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+.. KIND, either express or implied.  See the License for the
+.. specific language governing permissions and limitations
+.. under the License.
+
+.. highlight:: c
+
+.. _c-device-data-interface:
+
+=================================
+The Arrow C Device data interface
+=================================
+
+.. note:: The Arrow C Device Data Interface should be considered experimental
+
+Rationale
+=========
+
+The current :ref:`C Data Interface <c-data-interface>`, and most
+implementations of it, make the assumption that all data buffers provided
+are CPU buffers. Since Apache Arrow is designed to be a universal in-memory
+format for representing tabular ("columnar") data, there will be the desire
+to leverage this data on Non-CPU hardware such as GPUs. One example of such
+a case is the `RAPIDS cuDF library` which uses the Arrow memory format with

Review Comment:
   ```suggestion
   a case is the ``RAPIDS cuDF library`` which uses the Arrow memory format with
   ```
   
   or really, link to RAPIDS instead?



##########
docs/source/format/CDeviceDataInterface.rst:
##########
@@ -0,0 +1,613 @@
+.. Licensed to the Apache Software Foundation (ASF) under one
+.. or more contributor license agreements.  See the NOTICE file
+.. distributed with this work for additional information
+.. regarding copyright ownership.  The ASF licenses this file
+.. to you under the Apache License, Version 2.0 (the
+.. "License"); you may not use this file except in compliance
+.. with the License.  You may obtain a copy of the License at
+
+..   http://www.apache.org/licenses/LICENSE-2.0
+
+.. Unless required by applicable law or agreed to in writing,
+.. software distributed under the License is distributed on an
+.. "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+.. KIND, either express or implied.  See the License for the
+.. specific language governing permissions and limitations
+.. under the License.
+
+.. highlight:: c
+
+.. _c-device-data-interface:
+
+=================================
+The Arrow C Device data interface
+=================================
+
+.. note:: The Arrow C Device Data Interface should be considered experimental
+
+Rationale
+=========
+
+The current :ref:`C Data Interface <c-data-interface>`, and most
+implementations of it, make the assumption that all data buffers provided
+are CPU buffers. Since Apache Arrow is designed to be a universal in-memory
+format for representing tabular ("columnar") data, there will be the desire
+to leverage this data on Non-CPU hardware such as GPUs. One example of such
+a case is the `RAPIDS cuDF library` which uses the Arrow memory format with
+CUDA for NVIDIA GPUs. Since copying data from host to device and back is
+expensive, the ideal would be to be able to leave the data on the device
+for as long as possible, even when passing it between runtimes and
+libraries.
+
+The Arrow C Device data interface builds on the existing C data interface
+by adding a very small, stable set of C definitions to it. For non-C/C++
+languages and runtimes, translating the C definitions to corresponding
+C FFI declarations should be just as simple as with the current C data
+interface.
+
+Applications and libraries can then use Arrow schemas and Arrow formatted
+memory on non-cpu devices to exchange data just as easily as they do

Review Comment:
   ```suggestion
   memory on non-CPU devices to exchange data just as easily as they do
   ```



##########
docs/source/format/CDeviceDataInterface.rst:
##########
@@ -0,0 +1,613 @@
+.. Licensed to the Apache Software Foundation (ASF) under one
+.. or more contributor license agreements.  See the NOTICE file
+.. distributed with this work for additional information
+.. regarding copyright ownership.  The ASF licenses this file
+.. to you under the Apache License, Version 2.0 (the
+.. "License"); you may not use this file except in compliance
+.. with the License.  You may obtain a copy of the License at
+
+..   http://www.apache.org/licenses/LICENSE-2.0
+
+.. Unless required by applicable law or agreed to in writing,
+.. software distributed under the License is distributed on an
+.. "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+.. KIND, either express or implied.  See the License for the
+.. specific language governing permissions and limitations
+.. under the License.
+
+.. highlight:: c
+
+.. _c-device-data-interface:
+
+=================================
+The Arrow C Device data interface
+=================================
+
+.. note:: The Arrow C Device Data Interface should be considered experimental
+
+Rationale
+=========
+
+The current :ref:`C Data Interface <c-data-interface>`, and most
+implementations of it, make the assumption that all data buffers provided
+are CPU buffers. Since Apache Arrow is designed to be a universal in-memory
+format for representing tabular ("columnar") data, there will be the desire
+to leverage this data on Non-CPU hardware such as GPUs. One example of such

Review Comment:
   ```suggestion
   to leverage this data on non-CPU hardware such as GPUs. One example of such
   ```



##########
docs/source/format/CDeviceDataInterface.rst:
##########
@@ -0,0 +1,613 @@
+.. Licensed to the Apache Software Foundation (ASF) under one
+.. or more contributor license agreements.  See the NOTICE file
+.. distributed with this work for additional information
+.. regarding copyright ownership.  The ASF licenses this file
+.. to you under the Apache License, Version 2.0 (the
+.. "License"); you may not use this file except in compliance
+.. with the License.  You may obtain a copy of the License at
+
+..   http://www.apache.org/licenses/LICENSE-2.0
+
+.. Unless required by applicable law or agreed to in writing,
+.. software distributed under the License is distributed on an
+.. "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+.. KIND, either express or implied.  See the License for the
+.. specific language governing permissions and limitations
+.. under the License.
+
+.. highlight:: c
+
+.. _c-device-data-interface:
+
+=================================
+The Arrow C Device data interface
+=================================
+
+.. note:: The Arrow C Device Data Interface should be considered experimental
+
+Rationale
+=========
+
+The current :ref:`C Data Interface <c-data-interface>`, and most
+implementations of it, make the assumption that all data buffers provided
+are CPU buffers. Since Apache Arrow is designed to be a universal in-memory
+format for representing tabular ("columnar") data, there will be the desire
+to leverage this data on Non-CPU hardware such as GPUs. One example of such
+a case is the `RAPIDS cuDF library` which uses the Arrow memory format with
+CUDA for NVIDIA GPUs. Since copying data from host to device and back is
+expensive, the ideal would be to be able to leave the data on the device
+for as long as possible, even when passing it between runtimes and
+libraries.
+
+The Arrow C Device data interface builds on the existing C data interface
+by adding a very small, stable set of C definitions to it. For non-C/C++
+languages and runtimes, translating the C definitions to corresponding
+C FFI declarations should be just as simple as with the current C data
+interface.
+
+Applications and libraries can then use Arrow schemas and Arrow formatted
+memory on non-cpu devices to exchange data just as easily as they do
+now with CPU data. This will enable leaving data on those devices longer
+and avoiding costly copies back and forth between the host and device
+just to leverage new libraries and runtimes.
+
+Goals
+-----
+
+* Expose an ABI-stable interface built on the existing C data interface.
+* Make it easy for third-party projects to implement support with little
+  initial investment.
+* Allow zero-copy sharing of Arrow formatted device memory between
+  independant runtimes and components running in the same process.
+* Avoid the need for one-to-one adaptation layers such as the
+  `CUDA Array Interface` for Python processes to pass CUDA data.
+* Enable integration without explicit dependencies (either at compile-time
+  or runtime) on the Arrow software project itself.
+
+The intent is for the Arrow C Device data interface to expand the reach
+of the current C data interface, allowing it to also become the standard
+low-level building block for columnar processing on devices like GPUs or
+FPGAs.
+
+Structure definitions
+=====================
+
+Because this is built on the C data interface, the C Device data interface
+uses the ``ArrowSchema`` and ``ArrowArray`` structures as defined in the
+:ref:`C data interface spec <c-data-interface-struct-defs>`. It then adds the
+following free-standing definitions. Like the rest of the Arrow project,
+they are available under the Apache License 2.0.
+
+.. code-block:: c
+
+    #ifndef ARROW_C_DEVICE_DATA_INTERFACE
+    #define ARROW_C_DEVICE_DATA_INTERFACE
+
+    // Device type for the allocated memory
+    typedef int32_t ArrowDeviceType;
+
+    // CPU device, same as using ArrowArray directly
+    #define ARROW_DEVICE_CPU 1
+    // CUDA GPU Device
+    #define ARROW_DEVICE_CUDA 2
+    // Pinned CUDA CPU memory by cudaMallocHost
+    #define ARROW_DEVICE_CUDA_HOST 3
+    // OpenCL Device
+    #define ARROW_DEVICE_OPENCL 4
+    // Vulkan buffer for next-gen graphics
+    #define ARROW_DEVICE_VULKAN 7
+    // Metal for Apple GPU
+    #define ARROW_DEVICE_METAL 8
+    // Verilog simulator buffer
+    #define ARROW_DEVICE_VPI 9
+    // ROCm GPUs for AMD GPUs
+    #define ARROW_DEVICE_ROCM 10
+    // Pinned ROCm CPU memory allocated by hipMallocHost
+    #define ARROW_DEVICE_ROCM_HOST 11
+    // Reserved for extension
+    //
+    // used to quickly test extension devices, semantics
+    // can differ based on implementation
+    #define ARROW_DEVICE_EXT_DEV 12
+    // CUDA managed/unified memory allocated by cudaMallocManaged
+    #define ARROW_DEVICE_CUDA_MANAGED 13
+    // Unified shared memory allocated on a oneAPI
+    // non-partitioned device.
+    //
+    // A call to the oneAPI runtime is required to determine the
+    // device type, the USM allocation type and the sycl context
+    // that it is bound to.
+    #define ARROW_DEVICE_ONEAPI 14
+    // GPU support for next-gen WebGPU standard
+    #define ARROW_DEVICE_WEBGPU 15
+    // Qualcomm Hexagon DSP
+    #define ARROW_DEVICE_HEXAGON 16
+
+    struct ArrowDeviceArray {
+      struct ArrowArray array;
+      int64_t device_id;
+      ArrowDeviceType device_type;
+      void* sync_event;
+
+      // reserved bytes for future expansion
+      int64_t reserved[3];
+    };
+
+    #endif  // ARROW_C_DEVICE_DATA_INTERFACE
+
+.. note::
+   The canonical guard ``ARROW_C_DEVICE_DATA_INTERFACE`` is meant to avoid
+   duplicate definitions if two projects copy the definitions in their own
+   headers, and a third-party project includes from these two projects. It
+   is therefore important that this guard is kept exactly as-is when these
+   definitions are copied.
+
+ArrowDeviceType
+---------------
+
+The ``ArrowDeviceType`` typedef is used to indicate what type of device the
+provided memory buffers were allocated on. This, in conjunction with the
+``device_id``, should be sufficient to reference the correct data buffers.
+
+We then use macros to define values for different device types. The provided
+macro values are compatible with the widely used `dlpack` ``DLDeviceType``
+definition values, using the same value for each as the equivalent
+``kDL<type>`` enum from dlpack.h. The list will be kept in sync with those
+equivalent enum values over time to ensure compatibility, rather than
+potentially diverging. To avoid the Arrow project having to be in the
+position of vetting new hardware devices, new additions should first be
+added to dlpack before we add a corresponding macro here.
+
+To ensure predictability with the ABI, we use macros instead of an ``enum``
+so the storage type is not compiler dependent.
+
+.. c:macro:: ARROW_DEVICE_CPU
+
+    CPU Device, equivalent to just using ``ArrowArray`` directly instead of
+    using ``ArrowDeviceArray``.
+
+.. c:macro:: ARROW_DEVICE_CUDA
+
+    A `CUDA` GPU Device. This could represent data allocated either with the
+    runtime library (``cudaMalloc``) or the device driver (``cuMemAlloc``).
+
+.. c:macro:: ARROW_DEVICE_CUDA_HOST
+
+    CPU memory that was pinned and page-locked by CUDA by using
+    ``cudaMallocHost`` or ``cuMemAllocHost``.
+
+.. c:macro:: ARROW_DEVICE_OPENCL
+
+    Data allocated on the device by using the `OpenCL (Open Computing Language)`
+    framework.
+
+.. c:macro:: ARROW_DEVICE_VULKAN
+
+    Data allocated by the `Vulkan` framework and libraries.
+
+.. c:macro:: ARROW_DEVICE_METAL
+
+    Data on Apple GPU devices using the `Metal` framework and libraries.
+
+.. c:macro:: ARROW_DEVICE_VPI
+
+    Indicates usage of a Verilog simulator buffer.
+
+.. c:macro:: ARROW_DEVICE_ROCM
+
+    An AMD device using the `ROCm` stack.
+
+.. c:macro:: ARROW_DEVICE_ROCM_HOST
+
+    CPU memory pinned and page-locked allocated ``hipMallocHost``.
+
+.. c:macro:: ARROW_DEVICE_EXT_DEV
+
+    This value is an escape-hatch for devices to extend which aren't
+    currently represented otherwise. Producers would need to provide
+    additional information/context specific to the device if using
+    this device type. This is used to quickly test extension devices
+    and semantics can differ based on the implementation.
+
+.. c:macro:: ARROW_DEVICE_CUDA_MANAGED
+
+    CUDA managed/unified memory which is allocated by ``cudaMallocManaged``.
+
+.. c:macro:: ARROW_DEVICE_ONEAPI
+
+    Unified shared memory allocated on an Intel `oneAPI` non-partitioned
+    device. A call to the ``oneAPI`` runtime is required to determine
+    the specific device type, the USM allocation type and the sycl context
+    that it is bound to.
+
+.. c:macro:: ARROW_DEVICE_WEBGPU
+
+    GPU support for next-gen WebGPU standards
+
+.. c:macro:: ARROW_DEVICE_HEXAGON
+
+    Data allocated on a Qualcomm Hexagon DSP device.
+
+The ArrowDeviceArray structure
+------------------------------
+
+The ``ArrowDeviceArray`` structure embeds the C data ``ArrowArray`` structure
+and adds additional information necessary for consumers to use the data. It
+has the following fields:
+
+.. c:member:: struct ArrowArray ArrowDeviceArray.array
+
+    The allocated array data. The values in the ``void**`` buffers (along
+    with the buffers of any children) are what is allocated on the device.
+    The buffer values should be device pointers. The rest of the structure
+    should be accessible to the CPU.
+
+    The ``private_data`` and ``release`` callback of this structure should
+    contain any necessary information and structures related to freeing
+    the array according to the device it is allocated on, rather than
+    having a separate release callback and ``private_data`` pointer here.
+
+.. c:member:: int64_t ArrowDeviceArray.device_id
+
+    The device id to identify a specific device if multiple devices of this
+    type are on the system. The semantics of the id will be hardware dependent,
+    but we use an ``int64_t`` to future-proof the id as devices change over time.
+
+.. c:member:: ArrowDeviceType ArrowDeviceArray.device_type
+
+    The type of the device which can access the buffers in the array.
+
+.. c:member:: void* ArrowDeviceArray.sync_event
+
+    Optional. An event-like object to synchronize on if needed.
+
+    Many devices, like GPUs, are primarily asynchronous with respect to
+    CPU processing. As such in order to safely access memory, it is often
+    necessary to have an object to synchronize processing with. Since
+    different devices will use different types to specify this, we use a
+    void* which can be coerced into a pointer to whatever the device
+    appropriate type is.
+
+    If synchronization is not needed, this can be null. If this is non-null
+    then it MUST be used to call the appropriate sync method for the device
+    (e.g. ``cudaStreamWaitEvent`` or ``hipStreamWaitEvent``) before attempting
+    to access the memory in the buffers.
+
+    Expected types to coerce this ``void*`` to depending on the reported
+    device type:
+
+    * CUDA: ``cudaEvent_t*``
+    * ROCm: ``hipEvent_t*``
+    * OpenCL: ``cl_event*``
+    * Vulkan: ``VkEvent*``
+    * Metal: ``MTLEvent*``
+    * OneAPI: ``sycl::event*``
+
+    If an event is provided, then the producer MUST ensure that the event
+    is triggered/recorded at the end of the processing stream once the data
+    is considered available for use.
+
+
+.. c:member:: int64_t ArrowDeviceArray.reserved[3]
+
+    As non-CPU development expands, there may be a need to expand this
+    structure. In order to do so without potentially breaking ABI changes,
+    we reserve 24 bytes at the end of the object. This also has the added
+    benefit of bringing the total size of this structure to exactly 128
+    bytes (a power of 2) on 64-bit systems. These bytes should be zero'd
+    out after allocation in order to ensure safe evolution of the ABI in
+    the future.
+
+.. note::
+    Rather than store the shape / types of the data alongside the
+    ``ArrowDeviceArray``, users should utilize the existing ``ArrowSchema``
+    structure to pass any data type and shape information.
+
+Semantics
+=========
+
+Memory management
+-----------------
+
+The ``ArrowDeviceArray`` structure contains an ``ArrowArray`` object which
+itself has :ref:`specific semantics <c-data-interface-semantics>` for releasing
+memory. The term *"base structure"* below refers to the ``ArrowDeviceArray``
+object that is passed directly between the producer and consumer -- not any
+child structure thereof.
+
+It is intended for the base structure to be stack- or heap-allocated by the
+*consumer*. In this case, the producer API should take a pointer to the
+consumer-allocated structure.
+
+However, any data pointed to by the struct MUST be allocated and maintained
+by the producer. This includes the ``sync_event`` member if it is not null,
+along with any pointers in the ``ArrowArray`` object as usual. Data lifetime
+is managed through the ``release`` callback of the ``ArrowArray`` member.
+
+For an ``ArrowDeviceArray``, the semantics of a released structure and the
+callback semantics are identical to those for
+:ref:`ArrowArray itself <c-data-interface-released>`. Any producer specific context
+information necessary for releasing the device data buffers, in addition to
+any allocated event, should be stored in the ``private_data`` member of
+the ``ArrowArray`` and managed by the ``release`` callback.
+
+Moving an array
+'''''''''''''''
+
+The consumer can *move* the ``ArrowDeviceArray`` structure by bitwise copying
+or shallow member-wise copying. Then it MUST mark the source structure released
+by setting the ``release`` member of the embedded ``ArrowArray`` structure to
+``NULL``, but *without* calling that release callback. This ensures that only
+one live copy of the struct is active at any given time and that lifetime is
+correctly communicated to the producer.
+
+As usual, the release callback will be called on the destination structure
+when it is not needed anymore.
+
+Record batches
+--------------
+As with the C data interface itself, a record batch can be trivially considered
+as an equivalent struct array. In this case the metadata of the top-level
+``ArrowSchema`` can be used for schema-level metadata of the record batch.
+
+Mutability
+----------
+
+Both the producer and the consumer SHOULD consider the exported data (that
+is, the data reachable on the device through the ``buffers`` member of
+the embedded ``ArrowArray``) to be immutable, as either party could otherwise
+see inconsistent data while the other is mutating it.
+
+Likewise, if the ``sync_event`` member is non-NULL, the consumer should not
+attempt to access or read the data until they have synchronized on that event.
+
+C producer example
+====================
+
+Exporting a simple ``int32`` device array
+-----------------------------------------
+
+Export a non-nullable ``int32`` type with empty metadata. An example of this
+can be seen in the :ref:`C data interface docs directly <c-data-interface-export-int32-schema>`.
+
+To export the data itself, we transfer ownership to the consumer through
+the release callback. This example will use CUDA, but the equivalent calls
+could be used for any device:
+
+.. code-block:: c
+
+    static void release_int32_device_array(struct ArrowArray* array) {
+        assert(array->n_buffers == 2);
+        // destroy the event
+        cudaEvent_t* ev_ptr = reinterpret_cast<cudaEvent_t*>(array->private_data);
+        cudaError_t status = cudaEventDestroy(*ev_ptr);
+        assert(status == cudaSuccess);
+        free(ev_ptr);
+
+        // free the buffers and the buffers array
+        status = cudaFree(array->buffers[1]);
+        assert(status == cudaSuccess);
+        free(array->buffers);
+
+        // mark released
+        array->release = NULL;
+    }
+
+    __host__ void export_int32_device_array(void* cudaAllocdPtr,
+                                            cudaStream_t stream,
+                                            int64_t length,
+                                            struct ArrowDeviceArray* array) {
+        // get device id
+        int device;
+        cudaError_t status;
+        status = cudaGetDevice(&device);
+        assert(status == cudaSuccess);
+
+        cudaEvent_t* ev_ptr = (cudaEvent_t*)malloc(sizeof(cudaEvent_t));
+        assert(ev_ptr != NULL);
+        status = cudaEventCreate(ev_ptr);
+        assert(status == cudaSuccess);
+
+        // record event on the stream, assuming that the passed in
+        // stream is where the work to produce the data will be processing.        
+        status = cudaEventRecord(*ev_ptr, stream);
+        assert(status == cudaSuccess);
+
+        // initialize fields
+        *array = (struct ArrowDeviceArray) {
+            .array = (struct ArrowArray) {
+                .length = length,
+                .null_count = 0,
+                .offset = 0,
+                .n_buffers = 2,
+                .n_children = 0,
+                .children = NULL,
+                .dictionary = NULL,
+                // bookeeping
+                .release = &release_int32_device_array,
+                .private_data = reinterpret_cast<void*>(ev_ptr),
+            },
+            .device_id = static_cast<int64_t>(device),
+            .device_type = ARROW_DEVICE_CUDA,
+            // pass the event pointer to the consumer
+            .sync_event = reinterpret_cast<void*>(ev_ptr),
+        };
+
+        // allocate list of buffers
+        array->array.buffers = (const void**)malloc(sizeof(void*) * array->array.n_buffers);
+        assert(array->array.buffers != NULL);
+        array->array.buffers[0] = NULL;
+        array->array.buffers[1] = cudaAllocdPtr;
+    }
+
+================
+Device Stream Interface
+================
+
+Like the :ref:`C stream interface <c-stream-interface>`, the C Device data 
+interface also specifies a higher-level structure for easing communication
+of streaming data within a single process. Defining an ``ArrowDeviceArrayStream``
+structure.
+
+Semantics
+=========
+
+An Arrow C device stream exposes a streaming source of data chunks, each with
+the same schema. Chunks are obtained by calling a blocking pull-style iteration
+function. It is expected that all chunks should be providing data on the same
+device type (but not necessarily the same device id). If it is necessary
+to provide a stream of data on multiple device types, a producer should 
+provide a separate stream object for each device type.
+
+Structure definition
+====================
+
+The C device stream interface is defined by a single ``struct`` definition:
+
+.. code-block:: c
+
+    #ifndef ARROW_C_DEVICE_STREAM_INTERFACE
+    #define ARROW_C_DEVICE_STREAM_INTERFACE
+
+    struct ArrowDeviceArrayStream {
+        // device type that all arrays will be accessible from
+        ArrowDeviceType device_type;
+        // callbacks
+        int (*get_schema)(struct ArrowDeviceArrayStream*, struct ArrowSchema*);
+        int (*get_next)(struct ArrowDeviceArrayStream*, struct ArrowDeviceArray*);
+        const char* (*get_last_error)(struct ArrowDeviceArrayStream*);
+
+        // release callback
+        void (*release)(struct ArrowDeviceArrayStream*);
+
+        // opaque producer-specific data
+        void* private_data;
+    };
+
+    #endif  // ARROW_C_DEVICE_STREAM_INTERFACE
+
+.. note::
+    The canonical guard ``ARROW_C_DEVICE_STREAM_INTERFACE`` is meant to avoid
+    duplicate definitions if two projects copy the C device stream interface
+    definitions into their own headers, and a third-party project includes
+    from these two projects. It is therefore important that this guard is
+    kept exactly as-is when these definitions are copied.
+
+The ArrowDeviceArrayStream structure
+------------------------------------
+
+The ``ArrowDeviceArrayStream`` provides a device type that can access the
+resulting data along with the required callbacks to interact with a
+streaming source of Arrow arrays. It has the following fields:
+
+.. c:member:: ArrowDeviceType device_type
+
+    The device type that this stream produces data on. All 
+    ``ArrowDeviceArray``s that are produced by this stream should have the
+    same device type as is set here. This is a convenience for the consumer
+    to not have to check every array that is retrieved and instead allows
+    higher-level coding constructs for streams.
+
+.. c:member:: int (*ArrowDeviceArrayStream.get_schema)(struct ArrowDeviceArrayStream*, struct ArrowSchema* out)
+
+    *Mandatory.* This callback allows the consumer to query the schema of
+    the chunks of data in the stream. The schema is the same for all data
+    chunks.
+
+    This callback must NOT be called on a released ``ArrowDeviceArrayStream``.
+
+    *Return value:* 0 on success, a non-zero
+    :ref:`error code <c-stream-interface-error-codes>` otherwise.
+
+.. c:member:: int (*ArrowDeviceArrayStream.get_next)(struct ArrowDeviceArrayStream*, struct ArrowDeviceArray* out)
+
+    *Mandatory.* This callback allows the consumer to get the next chunk of
+    data in the stream.
+
+    This callback must NOT be called on a released ``ArrowDeviceArrayStream``.
+
+    The next chunk of data MUST be accessible from a device type matching the
+    :c:member:`ArrowDeviceArrayStream.device_type`.
+
+    *Return value:* 0 on success, a non-zero
+    :ref:`error code <c-stream-interface-error-codes>` otherwise.
+
+    On success, the consumer must check whether the ``ArrowDeviceArray``'s
+    embedded ``ArrowArray`` is marked :ref:`released <c-data-interface-released>`.
+    If the embedded ``ArrowDeviceArray.array`` is released, then the end of the 
+    stream has been reached. Otherwise, the ``ArrowDeviceArray`` contains a
+    valid data chunk.
+
+.. c:member:: const char* (*ArrowDeviceArrayStream.get_last_error)(struct ArrowDeviceArrayStream*)
+
+    *Mandatory.* This callback allows the consumer to get a textual description
+    of the last error.
+
+    This callback must ONLY be called if the last operation on the 
+    ``ArrowDeviceArrayStream`` returned an error. It must NOT be called on a
+    released ``ArrowDeviceArrayStream``.
+
+    *Return value:* a pointer to a NULL-terminated character string 
+    (UTF8-encoded). NULL can also be returned if no detailed description is
+    available.
+
+    The returned pointer is only guaranteed to be valid until the next call
+    of one of the stream's callbacks. The character string it points to should
+    be copied to consumer-managed storage if it is intended to survive longer.
+
+.. c:member:: void (*ArrowDeviceArrayStream.release)(struct ArrowDeviceArrayStream*)
+
+    *Mandatory.* A pointer to a producer-provided release callback.
+
+.. c:member:: void* ArrowDeviceArrayStream.private_data
+
+    *Optional.* An opaque pointer to producer-provided private data.
+
+    Consumers MUST NOT process this member. Lifetime of this member is
+    handled by the producer, and especially by the release callback.
+
+Result lifetimes
+----------------
+
+The data returned by the ``get_schema`` and ``get_next`` callbacks must be
+released independantly. Their lifetimes are not tied to that of 
+``ArrowDeviceArrayStream``.
+
+Stream lifetime
+---------------
+
+Lifetime of the C stream is managed using a release callback with similar 
+usage as in :ref:`C data interface <c-data-interface-released>`.
+
+Thread safety
+-------------
+
+The stream source is not assumed to be thread-safe. Consumers wanting to
+call ``get_next`` from several threads should ensure those calls are 
+serialized.
+
+Updating this specification
+===========================
+
+Once this specification is supported in an official Arrow release, the C ABI
+is frozen. This means that the ``ArrowDeviceArray`` structure definition 

Review Comment:
   What does this mean in terms of this specification also being 'experimental'?



##########
docs/source/format/CDeviceDataInterface.rst:
##########
@@ -0,0 +1,613 @@
+.. Licensed to the Apache Software Foundation (ASF) under one
+.. or more contributor license agreements.  See the NOTICE file
+.. distributed with this work for additional information
+.. regarding copyright ownership.  The ASF licenses this file
+.. to you under the Apache License, Version 2.0 (the
+.. "License"); you may not use this file except in compliance
+.. with the License.  You may obtain a copy of the License at
+
+..   http://www.apache.org/licenses/LICENSE-2.0
+
+.. Unless required by applicable law or agreed to in writing,
+.. software distributed under the License is distributed on an
+.. "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+.. KIND, either express or implied.  See the License for the
+.. specific language governing permissions and limitations
+.. under the License.
+
+.. highlight:: c
+
+.. _c-device-data-interface:
+
+=================================
+The Arrow C Device data interface
+=================================
+
+.. note:: The Arrow C Device Data Interface should be considered experimental
+
+Rationale
+=========
+
+The current :ref:`C Data Interface <c-data-interface>`, and most
+implementations of it, make the assumption that all data buffers provided
+are CPU buffers. Since Apache Arrow is designed to be a universal in-memory
+format for representing tabular ("columnar") data, there will be the desire
+to leverage this data on Non-CPU hardware such as GPUs. One example of such
+a case is the `RAPIDS cuDF library` which uses the Arrow memory format with
+CUDA for NVIDIA GPUs. Since copying data from host to device and back is
+expensive, the ideal would be to be able to leave the data on the device
+for as long as possible, even when passing it between runtimes and
+libraries.
+
+The Arrow C Device data interface builds on the existing C data interface
+by adding a very small, stable set of C definitions to it. For non-C/C++
+languages and runtimes, translating the C definitions to corresponding
+C FFI declarations should be just as simple as with the current C data
+interface.

Review Comment:
   Maybe describe what the definitions _are_? (Equivalents of the array/stream objects that specify the device type and allow for synchronizing with the producer?)



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [arrow] aregm commented on a diff in pull request #34972: GH-34971: [Format] Enhance C-Data API to support non-cpu cases

["aregm (via GitHub)" <gi...@apache.org>]

aregm commented on code in PR #34972:
URL: https://github.com/apache/arrow/pull/34972#discussion_r1164473963


##########
cpp/src/arrow/c/abi.h:
##########
@@ -65,6 +65,69 @@ struct ArrowArray {
 
 #endif  // ARROW_C_DATA_INTERFACE
 
+#ifndef ARROW_C_DEVICE_DATA_INTERFACE
+#define ARROW_C_DEVICE_DATA_INTERFACE
+
+// ArrowDeviceType is compatible with dlpack DLDeviceType for portability
+// it uses the same values for each enum as the equivalent kDL<type> from dlpack.h
+#ifdef __cplusplus
+typedef enum : int32_t {
+#else
+typedef enum {
+#endif
+  // CPU device, same as using ArrowArray directly
+  kArrowCPU = 1,
+  // CUDA GPU Device
+  kArrowCUDA = 2,
+  // Pinned CUDA CPU memory by cudaMallocHost
+  kArrowCUDAHost = 3,
+  // OpenCL Device
+  kArrowOpenCL = 4,
+  // Vulkan buffer for next-gen graphics
+  kArrowVulkan = 7,
+  // Metal for Apple GPU
+  kArrowMetal = 8,
+  // Verilog simulator buffer
+  kArrowVPI = 9,
+  // ROCm GPUs for AMD GPUs
+  kArrowROCM = 10,
+  // Pinned ROCm CPU memory allocated by hipMallocHost
+  kArrowROCMHost = 11,
+  // Reserved for extension
+  // used to quickly test extension devices,
+  // semantics can differ based on the implementation
+  kArrowExtDev = 12,
+  // CUDA managed/unified memory allocated by cudaMallocManaged
+  kArrowCUDAManaged = 13,
+  // unified shared memory allocated on a oneAPI non-partitioned
+  // device. call to oneAPI runtime is required to determine the
+  // device type, the USM allocation type and the sycl context it
+  // is bound to
+  kArrowOneAPI = 14,
+  // GPU support for next-gen WebGPU standard
+  kArrowWebGPU = 15,
+  // Qualcomm Hexagon DSP
+  kArrowHexagon = 16,
+} ArrowDeviceType;
+
+struct ArrowDeviceArray {
+  // the private_date and release callback of the arrow array
+  // should contain any necessary information and structures
+  // related to freeing the array according to the device it
+  // is allocated on, rather than having a separate release
+  // callback embedded here.
+  struct ArrowArray* array;
+  int device_id;

Review Comment:
   Once remark, though - the assumption is that Arrow will not go beyond a single host, anyone who decided to go low-level knows how to map [UUID](https://spec.oneapi.io/level-zero/latest/core/api.html#ze-uuid-t) to host/device. But with the upcoming CXL capability, this can be a problem. Let's leave it int64 and address the issue when it comes. So +1.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [arrow] pitrou commented on a diff in pull request #34972: GH-34971: [Format] Enhance C-Data API to support non-cpu cases

["pitrou (via GitHub)" <gi...@apache.org>]

pitrou commented on code in PR #34972:
URL: https://github.com/apache/arrow/pull/34972#discussion_r1164283118


##########
cpp/src/arrow/c/abi.h:
##########
@@ -106,6 +212,98 @@ struct ArrowArrayStream {
 
 #endif  // ARROW_C_STREAM_INTERFACE
 
+#ifndef ARROW_C_DEVICE_STREAM_INTERFACE
+#define ARROW_C_DEVICE_STREAM_INTERFACE
+
+/// \brief Equivalent to ArrowArrayStream, but for ArrowDeviceArrays.
+///
+/// This stream is intended to provide a stream of data on a single
+/// device, if a producer wants data to be produced on multiple devices
+/// then multiple streams should be provided. One per device.
+struct ArrowDeviceArrayStream {
+  /// \brief The device that this stream produces data on.
+  ///
+  /// All ArrowDeviceArrays that are produced by this
+  /// stream should have the same device_type as set
+  /// here. The device_type needs to be provided here
+  /// so that consumers can provide the correct type
+  /// of queue_ptr when calling get_next.
+  ArrowDeviceType device_type;
+
+  /// \brief Callback to get the stream schema
+  /// (will be the same for all arrays in the stream).
+  ///
+  /// If successful, the ArrowSchema must be released independantly from the stream.
+  /// The schema should be accessible via CPU memory.
+  ///
+  /// \param[in] self The ArrowDeviceArrayStream object itself
+  /// \param[out] out C struct to export the schema to
+  /// \return 0 if successful, an `errno`-compatible error code otherwise.
+  int (*get_schema)(struct ArrowDeviceArrayStream* self, struct ArrowSchema* out);
+
+  /// \brief Callback to get the device id for the next array.
+  ///
+  /// This is necessary so that the proper/correct stream pointer can be provided
+  /// to get_next.
+  ///
+  /// The next call to `get_next` should provide an ArrowDeviceArray whose
+  /// device_id matches what is provided here, and whose device_type is the
+  /// same as the device_type member of this stream.
+  ///
+  /// \param[in] self The ArrowDeviceArrayStream object itself
+  /// \param[out] out_device_id Pointer to be populated with the device id, must not be
+  /// null \return 0 if successful, an `errno`-compatible error code otherwise.
+  int (*get_next_device_id)(struct ArrowDeviceArrayStream* self, int64_t* out_device_id);
+
+  /// \brief Callback to get the next array
+  ///
+  /// If there is no error and the returned array has been released, the stream
+  /// has ended. If successful, the ArrowArray must be released independently
+  /// from the stream.
+  ///
+  /// Because different frameworks use different types to represent this, we
+  /// accept a void* which should then be reinterpreted into whatever the
+  /// appropriate type is (e.g. cudaStream_t) for use by the producer.
+  ///
+  /// \param[in] self The ArrowDeviceArrayStream object itself
+  /// \param[in] queue_ptr The appropriate queue, stream, or
+  /// equivalent object for the device that the data is allocated on
+  /// to indicate where the consumer wants the data to be accessible.
+  /// If queue_ptr is NULL then the default stream (e.g. CUDA stream 0)
+  /// should be used to ensure that the memory is accessible from any stream.

Review Comment:
   Why wouldn't they? They can easily refcount the usage of their own CUDA streams.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [arrow] kkraus14 commented on a diff in pull request #34972: GH-34971: [Format] Enhance C-Data API to support non-cpu cases

["kkraus14 (via GitHub)" <gi...@apache.org>]

kkraus14 commented on code in PR #34972:
URL: https://github.com/apache/arrow/pull/34972#discussion_r1164526679


##########
cpp/src/arrow/c/abi.h:
##########
@@ -106,6 +212,98 @@ struct ArrowArrayStream {
 
 #endif  // ARROW_C_STREAM_INTERFACE
 
+#ifndef ARROW_C_DEVICE_STREAM_INTERFACE
+#define ARROW_C_DEVICE_STREAM_INTERFACE
+
+/// \brief Equivalent to ArrowArrayStream, but for ArrowDeviceArrays.
+///
+/// This stream is intended to provide a stream of data on a single
+/// device, if a producer wants data to be produced on multiple devices
+/// then multiple streams should be provided. One per device.
+struct ArrowDeviceArrayStream {
+  /// \brief The device that this stream produces data on.
+  ///
+  /// All ArrowDeviceArrays that are produced by this
+  /// stream should have the same device_type as set
+  /// here. The device_type needs to be provided here
+  /// so that consumers can provide the correct type
+  /// of queue_ptr when calling get_next.
+  ArrowDeviceType device_type;
+
+  /// \brief Callback to get the stream schema
+  /// (will be the same for all arrays in the stream).
+  ///
+  /// If successful, the ArrowSchema must be released independantly from the stream.
+  /// The schema should be accessible via CPU memory.
+  ///
+  /// \param[in] self The ArrowDeviceArrayStream object itself
+  /// \param[out] out C struct to export the schema to
+  /// \return 0 if successful, an `errno`-compatible error code otherwise.
+  int (*get_schema)(struct ArrowDeviceArrayStream* self, struct ArrowSchema* out);
+
+  /// \brief Callback to get the device id for the next array.
+  ///
+  /// This is necessary so that the proper/correct stream pointer can be provided
+  /// to get_next.
+  ///
+  /// The next call to `get_next` should provide an ArrowDeviceArray whose
+  /// device_id matches what is provided here, and whose device_type is the
+  /// same as the device_type member of this stream.
+  ///
+  /// \param[in] self The ArrowDeviceArrayStream object itself
+  /// \param[out] out_device_id Pointer to be populated with the device id, must not be
+  /// null \return 0 if successful, an `errno`-compatible error code otherwise.
+  int (*get_next_device_id)(struct ArrowDeviceArrayStream* self, int64_t* out_device_id);
+
+  /// \brief Callback to get the next array
+  ///
+  /// If there is no error and the returned array has been released, the stream
+  /// has ended. If successful, the ArrowArray must be released independently
+  /// from the stream.
+  ///
+  /// Because different frameworks use different types to represent this, we
+  /// accept a void* which should then be reinterpreted into whatever the
+  /// appropriate type is (e.g. cudaStream_t) for use by the producer.
+  ///
+  /// \param[in] self The ArrowDeviceArrayStream object itself
+  /// \param[in] queue_ptr The appropriate queue, stream, or
+  /// equivalent object for the device that the data is allocated on
+  /// to indicate where the consumer wants the data to be accessible.
+  /// If queue_ptr is NULL then the default stream (e.g. CUDA stream 0)
+  /// should be used to ensure that the memory is accessible from any stream.

Review Comment:
   Okay, regardless if we take the producer provided path then I think it makes a lot more sense for the producer to share an Event than a Stream.
   
   An Event can be waited on via `cudaStreamWaitEvent` / `hipStreamWaitEvent` which does a device side wait which would have minimal overhead if it's the same stream or `cudaEventSynchronize` / `hipEventSynchronize` if blocking host code is desired.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [arrow] kkraus14 commented on pull request #34972: GH-34971: [Format] Enhance C-Data API to support non-cpu cases

["kkraus14 (via GitHub)" <gi...@apache.org>]

kkraus14 commented on PR #34972:
URL: https://github.com/apache/arrow/pull/34972#issuecomment-1511730719

   For AMD GPUs there's HIP Events (https://docs.amd.com/bundle/HIP-API-Guide-v5.2/page/group___event.html) which operate the same as CUDA Events.
   
   There is OpenCL / SYCL events that appear similar as well: https://developer.codeplay.com/products/computecpp/ce/2.11.0/guides/sycl-for-cuda-developers/migrating-from-cuda-to-sycl#platform-api-equivalence.
   
   For SYCL, there is the ability to automatically handles data dependencies: https://developer.codeplay.com/products/computecpp/ce/2.11.0/guides/sycl-guide/multiple-kernels. Though I suspect the raw pointers that we use for passing the data aren't sufficient for this.
   
   I'm not sure of other programming models that we'd like to capture.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [arrow] zeroshade commented on pull request #34972: GH-34971: [Format] Enhance C-Data API to support non-cpu cases

["zeroshade (via GitHub)" <gi...@apache.org>]

zeroshade commented on PR #34972:
URL: https://github.com/apache/arrow/pull/34972#issuecomment-1542690743

   @pitrou @lidavidm @paleolimbot @oleksandr-pavlyk @westonpace @leofang @gmarkall
   
   I've updated the header and comments based on the discussions here for the past few weeks. If possible I'd like to reach a consensus here before the end of the week at which point I'll update the arrow dev mailing list for more discussion and hopefully a vote.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [arrow] kkraus14 commented on a diff in pull request #34972: GH-34971: [Format] Enhance C-Data API to support non-cpu cases

["kkraus14 (via GitHub)" <gi...@apache.org>]

kkraus14 commented on code in PR #34972:
URL: https://github.com/apache/arrow/pull/34972#discussion_r1191661868


##########
cpp/src/arrow/c/abi.h:
##########
@@ -65,6 +85,111 @@ struct ArrowArray {
 
 #endif  // ARROW_C_DATA_INTERFACE
 
+#ifndef ARROW_C_DEVICE_DATA_INTERFACE
+#define ARROW_C_DEVICE_DATA_INTERFACE
+
+/// \defgroup arrow-device-types Device Types
+/// These macros are compatible with the dlpack DLDeviceType values,
+/// using the same value for each enum as the equivalent kDL<type>
+/// from dlpack.h. This list should continue to be kept in sync with
+/// the equivalent dlpack.h enum values over time to ensure 
+/// compatibility, rather than potentially diverging.
+///
+/// To ensure predictability with the ABI we use macros instead of
+/// an enum so the storage type is not compiler dependent.
+///
+/// @{
+
+/// \brief DeviceType for the allocated memory
+typedef int32_t ArrowDeviceType;
+
+/// \brief CPU device, same as using ArrowArray directly
+#define ARROW_DEVICE_CPU = 1
+/// \brief CUDA GPU Device
+#define ARROW_DEVICE_CUDA = 2
+/// \brief Pinned CUDA CPU memory by cudaMallocHost
+#define ARROW_DEVICE_CUDA_HOST = 3
+/// \brief OpenCL Device
+#define ARROW_DEVICE_OPENCL = 4
+/// \brief Vulkan buffer for next-gen graphics
+#define ARROW_DEVICE_VULKAN = 7
+/// \brief Metal for Apple GPU
+#define ARROW_DEVICE_METAL = 8
+/// \brief Verilog simulator buffer
+#define ARROW_DEVICE_VPI = 9
+/// \brief ROCm GPUs for AMD GPUs
+#define ARROW_DEVICE_ROCM = 10
+/// \brief Pinned ROCm CPU memory allocated by hipMallocHost
+#define ARROW_DEVICE_ROCMHOST = 11
+/// \brief Reserved for extension
+///
+/// used to quickly test extension devices, semantics
+/// can differ based on the implementation
+#define ARROW_DEVICE_EXT_DEV = 12
+/// \brief CUDA managed/unified memory allocated by cudaMallocManaged
+#define ARROW_DEVICE_CUDA_MANAGED = 13
+/// \brief unified shared memory allocated on a oneAPI
+/// non-partitioned device.
+///
+/// A call to the oneAPI runtime is required to determine the device
+/// type, the USM allocation type, and the sycl context it is bound to.
+#define ARROW_DEVICE_ONEAPI = 14
+/// \brief GPU support for next-gen WebGPU standard
+#define ARROW_DEVICE_WEBGPU = 15
+/// \brief Qualcomm Hexagon DSP
+#define ARROW_DEVICE_HEXAGON = 16
+
+/// @}
+
+/// \brief Struct for passing an Arrow Array alongside
+/// device memory information.
+struct ArrowDeviceArray {
+  /// \brief the Allocated Array
+  ///
+  /// the buffers in the array (along with the buffers of any
+  /// children) are what is allocated on the device.
+  ///
+  /// the private_data and release callback of the arrow array
+  /// should contain any necessary information and structures
+  /// related to freeing the array according to the device it
+  /// is allocated on, rather than having a separate release
+  /// callback embedded here.
+  struct ArrowArray array;
+  /// \brief The device id to identify a specific device
+  /// if multiple of this type are on the system.
+  ///
+  /// the semantics of the id will be hardware dependant.
+  int64_t device_id;
+  /// \brief The type of device which can access this memory.
+  ArrowDeviceType device_type;
+  /// \brief An event-like object to synchronize on if needed.
+  ///
+  /// Many devices, like GPUs, are primarily asynchronous with
+  /// respect to CPU processing. As such in order to safely access
+  /// memory, it is often necessary to have an object to synchronize
+  /// processing on. Since different devices will use different types

Review Comment:
   - OpenCL: http://people.cs.bris.ac.uk/~simonm/workshops/BSC_2013/opencl:course:bsc/Slides/OpenCL_events.pdf
   - Vulkan: https://registry.khronos.org/vulkan/specs/1.3-extensions/man/html/VkEvent.html
   - Metal: https://developer.apple.com/documentation/metal/mtlevent
   - ROCM is HIP
   - OneAPI: There's SYCL events: https://developer.codeplay.com/products/computecpp/ce/2.11.0/api-reference/classcl_1_1sycl_1_1event but I'm not clear if that directly translates to OneAPI or not



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [arrow] gmarkall commented on pull request #34972: GH-34971: [Format] Enhance C-Data API to support non-cpu cases

["gmarkall (via GitHub)" <gi...@apache.org>]

gmarkall commented on PR #34972:
URL: https://github.com/apache/arrow/pull/34972#issuecomment-1507003526

   > @gmarkall @stuartarchibald Not sure you want to read the entire discussion here, but I or @kkraus14 can provide a quick summary if needed :-)
   
   I've read the entire discussion and I believe I've understood it - please do let me know if I appear to be going off-course in the following though :slightly_smiling_face: 
   
   > Hmm, since the primary example of a producer-supplied CUDA stream is the [CUDA Array Interface](https://numba.readthedocs.io/en/stable/cuda/cuda_array_interface.html#streams), perhaps it would be nice to have the Numba devs' feedback on consumer-supplied vs. producer-supplied.
   
   The stream and synchronization parts of the CAI specification aren't really originated solely in Numba / from Numba developers, but evolved though the discussion mostly on https://github.com/numba/numba/pull/5162 - there was input from quite a few people on that PR and the issues referred to therein (I won't ping them all here) so I'd say the design that landed there was much more of a community agreement than a Numba-derived specification / feature.
   
   Like @kkraus14, I haven't really seen it used in practice, except in Numba which I suppose provides a reference example for a "safe by default" implementation. I think the specification arose from how we expected people would want to use it rather than based on experience prior to the definition. Then, following the agreement and updating of the specification, I think most that had been involved in the discussions found no high-priority need for this particular feature (e.g. changing priorities / directions) so I think this particular set of design decisions hasn't really been tested out as much as it might appear.
   
   I also think it would also have been more difficult to have the consumer supply the stream for CAI, because it is just a property containing a dict - this might have steered thinking away from even giving consideration to the idea of consumer-supplied streams because some more radical change to the interface would have been needed to support it.
   
   Given the above, I think I wouldn't assume too much about the amount of experience that drove the choices made for the CAI, and would suggest tending towards what seems the most reasonable choice for Arrow, rather than what might provide the most common ground with the CAI.
   
   (As a side note - Numba doesn't support DLPack because it didn't seem very important to support it - anyone wanting to use DLPack with Numba device arrays could just convert to CuPy using CAI then to DLPack from there.)


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [arrow] pitrou commented on a diff in pull request #34972: GH-34971: [Format] Enhance C-Data API to support non-cpu cases

["pitrou (via GitHub)" <gi...@apache.org>]

pitrou commented on code in PR #34972:
URL: https://github.com/apache/arrow/pull/34972#discussion_r1161079933


##########
cpp/src/arrow/c/abi.h:
##########
@@ -65,6 +65,69 @@ struct ArrowArray {
 
 #endif  // ARROW_C_DATA_INTERFACE
 
+#ifndef ARROW_C_DEVICE_DATA_INTERFACE
+#define ARROW_C_DEVICE_DATA_INTERFACE
+
+// ArrowDeviceType is compatible with dlpack DLDeviceType for portability
+// it uses the same values for each enum as the equivalent kDL<type> from dlpack.h
+#ifdef __cplusplus
+typedef enum : int32_t {
+#else
+typedef enum {
+#endif
+  // CPU device, same as using ArrowArray directly
+  kArrowCPU = 1,
+  // CUDA GPU Device
+  kArrowCUDA = 2,
+  // Pinned CUDA CPU memory by cudaMallocHost
+  kArrowCUDAHost = 3,
+  // OpenCL Device
+  kArrowOpenCL = 4,
+  // Vulkan buffer for next-gen graphics
+  kArrowVulkan = 7,
+  // Metal for Apple GPU
+  kArrowMetal = 8,
+  // Verilog simulator buffer
+  kArrowVPI = 9,
+  // ROCm GPUs for AMD GPUs
+  kArrowROCM = 10,
+  // Pinned ROCm CPU memory allocated by hipMallocHost
+  kArrowROCMHost = 11,
+  // Reserved for extension
+  // used to quickly test extension devices,
+  // semantics can differ based on the implementation
+  kArrowExtDev = 12,
+  // CUDA managed/unified memory allocated by cudaMallocManaged
+  kArrowCUDAManaged = 13,
+  // unified shared memory allocated on a oneAPI non-partitioned
+  // device. call to oneAPI runtime is required to determine the
+  // device type, the USM allocation type and the sycl context it
+  // is bound to
+  kArrowOneAPI = 14,
+  // GPU support for next-gen WebGPU standard
+  kArrowWebGPU = 15,
+  // Qualcomm Hexagon DSP
+  kArrowHexagon = 16,
+} ArrowDeviceType;
+
+struct ArrowDeviceArray {
+  // the private_date and release callback of the arrow array
+  // should contain any necessary information and structures
+  // related to freeing the array according to the device it
+  // is allocated on, rather than having a separate release
+  // callback embedded here.
+  struct ArrowArray* array;

Review Comment:
   If you make this a pointer, then who owns the `struct ArrowArray` area?
   You should instead embed the struct.
   ```suggestion
     struct ArrowArray array;
   ```



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [arrow] pitrou commented on a diff in pull request #34972: GH-34971: [Format] Enhance C-Data API to support non-cpu cases

["pitrou (via GitHub)" <gi...@apache.org>]

pitrou commented on code in PR #34972:
URL: https://github.com/apache/arrow/pull/34972#discussion_r1161080235


##########
cpp/src/arrow/c/abi.h:
##########
@@ -106,6 +169,77 @@ struct ArrowArrayStream {
 
 #endif  // ARROW_C_STREAM_INTERFACE
 
+#ifndef ARROW_C_DEVICE_STREAM_INTERFACE
+#define ARROW_C_DEVICE_STREAM_INTERFACE
+
+struct ArrowDeviceArrayStream {
+  // The device that this stream produces data on.
+  // All ArrowDeviceArrays that are produced by this
+  // stream should have the same device_type as set
+  // here. The device_type needs to be provided here
+  // so that consumers can provide the correct type
+  // of stream_ptr when calling get_next.
+  ArrowDeviceType device_type;
+
+  // Callback to get the stream schema
+  // (will be the same for all arrays in the stream).
+  //
+  // Return value: 0 if successful, an `errno`-compatible error code otherwise.
+  //
+  // If successful, the ArrowSchema must be released independently from the stream.
+  int (*get_schema)(struct ArrowDeviceArrayStream*, struct ArrowSchema* out);
+
+  // Callback to get the device id for the next array.
+  // This is necessary so that the proper/correct stream pointer can be provided
+  // to get_next. The parameter provided must not be null.
+  //
+  // Return value: 0 if successful, an `errno`-compatible error code otherwise.
+  //
+  // The next call to `get_next` should provide an ArrowDeviceArray whose
+  // device_id matches what is provided here, and whose device_type is the
+  // same as the device_type member of this stream.
+  int (*get_next_device_id)(struct ArrowDeviceArrayStream*, int* out_device_id);
+
+  // Callback to get the next array
+  // (if no error and the array is released, the stream has ended)
+  //
+  // the provided stream_ptr should be the appropriate stream, or
+  // equivalent object, for the device that the data is allocated on
+  // to indicate where the consumer wants the data to be accessible.

Review Comment:
   I'm not sure I understand, why would the consumer be able to choose where the data is accessible? If the data is already allocated (which is certainly a common case), how the producer make it accessible to a new stream?



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [arrow] paleolimbot commented on a diff in pull request #34972: GH-34971: [Format] Enhance C-Data API to support non-cpu cases

["paleolimbot (via GitHub)" <gi...@apache.org>]

paleolimbot commented on code in PR #34972:
URL: https://github.com/apache/arrow/pull/34972#discussion_r1161175691


##########
cpp/src/arrow/c/abi.h:
##########
@@ -65,6 +65,69 @@ struct ArrowArray {
 
 #endif  // ARROW_C_DATA_INTERFACE
 
+#ifndef ARROW_C_DEVICE_DATA_INTERFACE
+#define ARROW_C_DEVICE_DATA_INTERFACE
+
+// ArrowDeviceType is compatible with dlpack DLDeviceType for portability
+// it uses the same values for each enum as the equivalent kDL<type> from dlpack.h
+#ifdef __cplusplus
+typedef enum : int32_t {
+#else
+typedef enum {
+#endif

Review Comment:
   It seems goofy that one might be able to get a different enum storage type depending on whether or not this is included in a C or C++ source file. The ADBC header specifically avoids enums and I wonder if it would be more predictable to `#define ARROW_DEVICE_TYPE_CPU 1` etc. and use `int32_t device_type` below.



##########
cpp/src/arrow/c/abi.h:
##########
@@ -65,6 +65,69 @@ struct ArrowArray {
 
 #endif  // ARROW_C_DATA_INTERFACE
 
+#ifndef ARROW_C_DEVICE_DATA_INTERFACE
+#define ARROW_C_DEVICE_DATA_INTERFACE
+
+// ArrowDeviceType is compatible with dlpack DLDeviceType for portability
+// it uses the same values for each enum as the equivalent kDL<type> from dlpack.h
+#ifdef __cplusplus
+typedef enum : int32_t {
+#else
+typedef enum {
+#endif
+  // CPU device, same as using ArrowArray directly
+  kArrowCPU = 1,
+  // CUDA GPU Device
+  kArrowCUDA = 2,
+  // Pinned CUDA CPU memory by cudaMallocHost
+  kArrowCUDAHost = 3,
+  // OpenCL Device
+  kArrowOpenCL = 4,
+  // Vulkan buffer for next-gen graphics
+  kArrowVulkan = 7,
+  // Metal for Apple GPU
+  kArrowMetal = 8,
+  // Verilog simulator buffer
+  kArrowVPI = 9,
+  // ROCm GPUs for AMD GPUs
+  kArrowROCM = 10,
+  // Pinned ROCm CPU memory allocated by hipMallocHost
+  kArrowROCMHost = 11,
+  // Reserved for extension
+  // used to quickly test extension devices,
+  // semantics can differ based on the implementation
+  kArrowExtDev = 12,
+  // CUDA managed/unified memory allocated by cudaMallocManaged
+  kArrowCUDAManaged = 13,
+  // unified shared memory allocated on a oneAPI non-partitioned
+  // device. call to oneAPI runtime is required to determine the
+  // device type, the USM allocation type and the sycl context it
+  // is bound to
+  kArrowOneAPI = 14,
+  // GPU support for next-gen WebGPU standard
+  kArrowWebGPU = 15,
+  // Qualcomm Hexagon DSP
+  kArrowHexagon = 16,
+} ArrowDeviceType;
+
+struct ArrowDeviceArray {
+  // the private_date and release callback of the arrow array
+  // should contain any necessary information and structures
+  // related to freeing the array according to the device it
+  // is allocated on, rather than having a separate release
+  // callback embedded here.
+  struct ArrowArray* array;
+  int device_id;

Review Comment:
   (Or if `int` is sufficient, `int32_t` would be more consistent with the existing C data interface)



##########
cpp/src/arrow/c/abi.h:
##########
@@ -106,6 +169,77 @@ struct ArrowArrayStream {
 
 #endif  // ARROW_C_STREAM_INTERFACE
 
+#ifndef ARROW_C_DEVICE_STREAM_INTERFACE
+#define ARROW_C_DEVICE_STREAM_INTERFACE
+
+struct ArrowDeviceArrayStream {
+  // The device that this stream produces data on.
+  // All ArrowDeviceArrays that are produced by this
+  // stream should have the same device_type as set
+  // here. The device_type needs to be provided here
+  // so that consumers can provide the correct type
+  // of stream_ptr when calling get_next.
+  ArrowDeviceType device_type;
+
+  // Callback to get the stream schema
+  // (will be the same for all arrays in the stream).
+  //
+  // Return value: 0 if successful, an `errno`-compatible error code otherwise.
+  //
+  // If successful, the ArrowSchema must be released independently from the stream.
+  int (*get_schema)(struct ArrowDeviceArrayStream*, struct ArrowSchema* out);
+
+  // Callback to get the device id for the next array.
+  // This is necessary so that the proper/correct stream pointer can be provided
+  // to get_next. The parameter provided must not be null.
+  //
+  // Return value: 0 if successful, an `errno`-compatible error code otherwise.
+  //
+  // The next call to `get_next` should provide an ArrowDeviceArray whose
+  // device_id matches what is provided here, and whose device_type is the
+  // same as the device_type member of this stream.
+  int (*get_next_device_id)(struct ArrowDeviceArrayStream*, int* out_device_id);
+
+  // Callback to get the next array
+  // (if no error and the array is released, the stream has ended)
+  //
+  // the provided stream_ptr should be the appropriate stream, or
+  // equivalent object, for the device that the data is allocated on
+  // to indicate where the consumer wants the data to be accessible.

Review Comment:
   Should this `void*` instead be a member of `ArrowDeviceArray`? It seems like that pointer might be useful to anybody attempting to access data in those buffers?



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [arrow] pitrou commented on a diff in pull request #34972: GH-34971: [Format] Enhance C-Data API to support non-cpu cases

["pitrou (via GitHub)" <gi...@apache.org>]

pitrou commented on code in PR #34972:
URL: https://github.com/apache/arrow/pull/34972#discussion_r1161988488


##########
cpp/src/arrow/c/abi.h:
##########
@@ -106,6 +169,77 @@ struct ArrowArrayStream {
 
 #endif  // ARROW_C_STREAM_INTERFACE
 
+#ifndef ARROW_C_DEVICE_STREAM_INTERFACE
+#define ARROW_C_DEVICE_STREAM_INTERFACE
+
+struct ArrowDeviceArrayStream {
+  // The device that this stream produces data on.
+  // All ArrowDeviceArrays that are produced by this
+  // stream should have the same device_type as set
+  // here. The device_type needs to be provided here
+  // so that consumers can provide the correct type
+  // of stream_ptr when calling get_next.
+  ArrowDeviceType device_type;
+
+  // Callback to get the stream schema
+  // (will be the same for all arrays in the stream).
+  //
+  // Return value: 0 if successful, an `errno`-compatible error code otherwise.
+  //
+  // If successful, the ArrowSchema must be released independently from the stream.
+  int (*get_schema)(struct ArrowDeviceArrayStream*, struct ArrowSchema* out);
+
+  // Callback to get the device id for the next array.
+  // This is necessary so that the proper/correct stream pointer can be provided
+  // to get_next. The parameter provided must not be null.
+  //
+  // Return value: 0 if successful, an `errno`-compatible error code otherwise.
+  //
+  // The next call to `get_next` should provide an ArrowDeviceArray whose
+  // device_id matches what is provided here, and whose device_type is the
+  // same as the device_type member of this stream.
+  int (*get_next_device_id)(struct ArrowDeviceArrayStream*, int* out_device_id);
+
+  // Callback to get the next array
+  // (if no error and the array is released, the stream has ended)
+  //
+  // the provided stream_ptr should be the appropriate stream, or
+  // equivalent object, for the device that the data is allocated on
+  // to indicate where the consumer wants the data to be accessible.

Review Comment:
   > The idea of a consumer of the interface passing a stream to the producer of the interface is that the producer can guarantee that the data is safe to consume on the passed in stream.
   
   This means that the `ArrowDeviceArray` struct itself doesn't hold any information about the CUDA stream the array can be accessed from, and the consumer _has_ to go through the `ArrowDeviceStream` to pass its own stream information.
   



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [arrow] zeroshade commented on a diff in pull request #34972: GH-34971: [Format] Enhance C-Data API to support non-cpu cases

["zeroshade (via GitHub)" <gi...@apache.org>]

zeroshade commented on code in PR #34972:
URL: https://github.com/apache/arrow/pull/34972#discussion_r1161301469


##########
cpp/src/arrow/c/abi.h:
##########
@@ -106,6 +169,77 @@ struct ArrowArrayStream {
 
 #endif  // ARROW_C_STREAM_INTERFACE
 
+#ifndef ARROW_C_DEVICE_STREAM_INTERFACE
+#define ARROW_C_DEVICE_STREAM_INTERFACE
+
+struct ArrowDeviceArrayStream {
+  // The device that this stream produces data on.
+  // All ArrowDeviceArrays that are produced by this
+  // stream should have the same device_type as set
+  // here. The device_type needs to be provided here
+  // so that consumers can provide the correct type
+  // of stream_ptr when calling get_next.
+  ArrowDeviceType device_type;
+
+  // Callback to get the stream schema
+  // (will be the same for all arrays in the stream).
+  //
+  // Return value: 0 if successful, an `errno`-compatible error code otherwise.
+  //
+  // If successful, the ArrowSchema must be released independently from the stream.
+  int (*get_schema)(struct ArrowDeviceArrayStream*, struct ArrowSchema* out);
+
+  // Callback to get the device id for the next array.
+  // This is necessary so that the proper/correct stream pointer can be provided
+  // to get_next. The parameter provided must not be null.
+  //
+  // Return value: 0 if successful, an `errno`-compatible error code otherwise.
+  //
+  // The next call to `get_next` should provide an ArrowDeviceArray whose
+  // device_id matches what is provided here, and whose device_type is the
+  // same as the device_type member of this stream.
+  int (*get_next_device_id)(struct ArrowDeviceArrayStream*, int* out_device_id);
+
+  // Callback to get the next array
+  // (if no error and the array is released, the stream has ended)
+  //
+  // the provided stream_ptr should be the appropriate stream, or
+  // equivalent object, for the device that the data is allocated on
+  // to indicate where the consumer wants the data to be accessible.
+  // if stream_ptr is NULL then the default stream (e.g. CUDA stream 0)
+  // should be used to ensure that the memory is accessible from any stream.

Review Comment:
   As far as I'm aware, this should make sense for all backends. If they utilize the concept of a stream, they all have some "default" stream that can be used if this is null (the example given being stream 0 for CUDA), so I'm fairly certain this isn't device-specific behavior being encoded. @kkraus14 can you fact-check me on this?



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [arrow] pitrou commented on a diff in pull request #34972: GH-34971: [Format] Enhance C-Data API to support non-cpu cases

["pitrou (via GitHub)" <gi...@apache.org>]

pitrou commented on code in PR #34972:
URL: https://github.com/apache/arrow/pull/34972#discussion_r1161305089


##########
cpp/src/arrow/c/abi.h:
##########
@@ -106,6 +169,77 @@ struct ArrowArrayStream {
 
 #endif  // ARROW_C_STREAM_INTERFACE
 
+#ifndef ARROW_C_DEVICE_STREAM_INTERFACE
+#define ARROW_C_DEVICE_STREAM_INTERFACE
+
+struct ArrowDeviceArrayStream {
+  // The device that this stream produces data on.
+  // All ArrowDeviceArrays that are produced by this
+  // stream should have the same device_type as set
+  // here. The device_type needs to be provided here
+  // so that consumers can provide the correct type
+  // of stream_ptr when calling get_next.
+  ArrowDeviceType device_type;
+
+  // Callback to get the stream schema
+  // (will be the same for all arrays in the stream).
+  //
+  // Return value: 0 if successful, an `errno`-compatible error code otherwise.
+  //
+  // If successful, the ArrowSchema must be released independently from the stream.
+  int (*get_schema)(struct ArrowDeviceArrayStream*, struct ArrowSchema* out);
+
+  // Callback to get the device id for the next array.
+  // This is necessary so that the proper/correct stream pointer can be provided
+  // to get_next. The parameter provided must not be null.
+  //
+  // Return value: 0 if successful, an `errno`-compatible error code otherwise.
+  //
+  // The next call to `get_next` should provide an ArrowDeviceArray whose
+  // device_id matches what is provided here, and whose device_type is the
+  // same as the device_type member of this stream.
+  int (*get_next_device_id)(struct ArrowDeviceArrayStream*, int* out_device_id);
+
+  // Callback to get the next array
+  // (if no error and the array is released, the stream has ended)
+  //
+  // the provided stream_ptr should be the appropriate stream, or
+  // equivalent object, for the device that the data is allocated on
+  // to indicate where the consumer wants the data to be accessible.
+  // if stream_ptr is NULL then the default stream (e.g. CUDA stream 0)
+  // should be used to ensure that the memory is accessible from any stream.

Review Comment:
   Also sidenote, it would be nice if we could find a less ambiguous term, as "stream" already refers to something else in this interface (though I have no concrete suggestion).



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [arrow] kkraus14 commented on pull request #34972: GH-34971: [Format] Enhance C-Data API to support non-cpu cases

["kkraus14 (via GitHub)" <gi...@apache.org>]

kkraus14 commented on PR #34972:
URL: https://github.com/apache/arrow/pull/34972#issuecomment-1505773870

   > @kkraus14 @pitrou Did you two reach a consensus on the question concerning the consumer passing of the stream/queue pointer into `get_next` or is this still an open question that needs to be discussed further?
   
   I am still in favor of passing the stream/queue, but if @pitrou has a strong preference for returning it from the producer instead I would just ask that we use an Event instead of a Stream/Queue.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [arrow] pitrou commented on pull request #34972: GH-34971: [Format] Enhance C-Data API to support non-cpu cases

["pitrou (via GitHub)" <gi...@apache.org>]

pitrou commented on PR #34972:
URL: https://github.com/apache/arrow/pull/34972#issuecomment-1511654356

   In that case, a bit more research is needed to find out what the equivalent of a CUDA `Event` would be for other backends.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [arrow] zeroshade commented on a diff in pull request #34972: GH-34971: [Format] Enhance C-Data API to support non-cpu cases

["zeroshade (via GitHub)" <gi...@apache.org>]

zeroshade commented on code in PR #34972:
URL: https://github.com/apache/arrow/pull/34972#discussion_r1204369289


##########
docs/source/format/CDeviceDataInterface.rst:
##########
@@ -0,0 +1,613 @@
+.. Licensed to the Apache Software Foundation (ASF) under one
+.. or more contributor license agreements.  See the NOTICE file
+.. distributed with this work for additional information
+.. regarding copyright ownership.  The ASF licenses this file
+.. to you under the Apache License, Version 2.0 (the
+.. "License"); you may not use this file except in compliance
+.. with the License.  You may obtain a copy of the License at
+
+..   http://www.apache.org/licenses/LICENSE-2.0
+
+.. Unless required by applicable law or agreed to in writing,
+.. software distributed under the License is distributed on an
+.. "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+.. KIND, either express or implied.  See the License for the
+.. specific language governing permissions and limitations
+.. under the License.
+
+.. highlight:: c
+
+.. _c-device-data-interface:
+
+=================================
+The Arrow C Device data interface
+=================================
+
+.. note:: The Arrow C Device Data Interface should be considered experimental
+
+Rationale
+=========
+
+The current :ref:`C Data Interface <c-data-interface>`, and most
+implementations of it, make the assumption that all data buffers provided
+are CPU buffers. Since Apache Arrow is designed to be a universal in-memory
+format for representing tabular ("columnar") data, there will be the desire
+to leverage this data on Non-CPU hardware such as GPUs. One example of such
+a case is the `RAPIDS cuDF library` which uses the Arrow memory format with
+CUDA for NVIDIA GPUs. Since copying data from host to device and back is
+expensive, the ideal would be to be able to leave the data on the device
+for as long as possible, even when passing it between runtimes and
+libraries.
+
+The Arrow C Device data interface builds on the existing C data interface
+by adding a very small, stable set of C definitions to it. For non-C/C++
+languages and runtimes, translating the C definitions to corresponding
+C FFI declarations should be just as simple as with the current C data
+interface.
+
+Applications and libraries can then use Arrow schemas and Arrow formatted
+memory on non-cpu devices to exchange data just as easily as they do
+now with CPU data. This will enable leaving data on those devices longer
+and avoiding costly copies back and forth between the host and device
+just to leverage new libraries and runtimes.
+
+Goals
+-----
+
+* Expose an ABI-stable interface built on the existing C data interface.
+* Make it easy for third-party projects to implement support with little
+  initial investment.
+* Allow zero-copy sharing of Arrow formatted device memory between
+  independant runtimes and components running in the same process.
+* Avoid the need for one-to-one adaptation layers such as the
+  `CUDA Array Interface` for Python processes to pass CUDA data.
+* Enable integration without explicit dependencies (either at compile-time
+  or runtime) on the Arrow software project itself.
+
+The intent is for the Arrow C Device data interface to expand the reach
+of the current C data interface, allowing it to also become the standard
+low-level building block for columnar processing on devices like GPUs or
+FPGAs.
+
+Structure definitions
+=====================
+
+Because this is built on the C data interface, the C Device data interface
+uses the ``ArrowSchema`` and ``ArrowArray`` structures as defined in the
+:ref:`C data interface spec <c-data-interface-struct-defs>`. It then adds the
+following free-standing definitions. Like the rest of the Arrow project,
+they are available under the Apache License 2.0.
+
+.. code-block:: c
+
+    #ifndef ARROW_C_DEVICE_DATA_INTERFACE
+    #define ARROW_C_DEVICE_DATA_INTERFACE
+
+    // Device type for the allocated memory
+    typedef int32_t ArrowDeviceType;
+
+    // CPU device, same as using ArrowArray directly
+    #define ARROW_DEVICE_CPU 1
+    // CUDA GPU Device
+    #define ARROW_DEVICE_CUDA 2
+    // Pinned CUDA CPU memory by cudaMallocHost
+    #define ARROW_DEVICE_CUDA_HOST 3
+    // OpenCL Device
+    #define ARROW_DEVICE_OPENCL 4
+    // Vulkan buffer for next-gen graphics
+    #define ARROW_DEVICE_VULKAN 7
+    // Metal for Apple GPU
+    #define ARROW_DEVICE_METAL 8
+    // Verilog simulator buffer
+    #define ARROW_DEVICE_VPI 9
+    // ROCm GPUs for AMD GPUs
+    #define ARROW_DEVICE_ROCM 10
+    // Pinned ROCm CPU memory allocated by hipMallocHost
+    #define ARROW_DEVICE_ROCM_HOST 11
+    // Reserved for extension
+    //
+    // used to quickly test extension devices, semantics
+    // can differ based on implementation
+    #define ARROW_DEVICE_EXT_DEV 12
+    // CUDA managed/unified memory allocated by cudaMallocManaged
+    #define ARROW_DEVICE_CUDA_MANAGED 13
+    // Unified shared memory allocated on a oneAPI
+    // non-partitioned device.
+    //
+    // A call to the oneAPI runtime is required to determine the
+    // device type, the USM allocation type and the sycl context
+    // that it is bound to.
+    #define ARROW_DEVICE_ONEAPI 14
+    // GPU support for next-gen WebGPU standard
+    #define ARROW_DEVICE_WEBGPU 15
+    // Qualcomm Hexagon DSP
+    #define ARROW_DEVICE_HEXAGON 16
+
+    struct ArrowDeviceArray {
+      struct ArrowArray array;
+      int64_t device_id;
+      ArrowDeviceType device_type;
+      void* sync_event;
+
+      // reserved bytes for future expansion
+      int64_t reserved[3];
+    };
+
+    #endif  // ARROW_C_DEVICE_DATA_INTERFACE
+
+.. note::
+   The canonical guard ``ARROW_C_DEVICE_DATA_INTERFACE`` is meant to avoid
+   duplicate definitions if two projects copy the definitions in their own
+   headers, and a third-party project includes from these two projects. It
+   is therefore important that this guard is kept exactly as-is when these
+   definitions are copied.
+
+ArrowDeviceType
+---------------
+
+The ``ArrowDeviceType`` typedef is used to indicate what type of device the
+provided memory buffers were allocated on. This, in conjunction with the
+``device_id``, should be sufficient to reference the correct data buffers.
+
+We then use macros to define values for different device types. The provided
+macro values are compatible with the widely used `dlpack` ``DLDeviceType``
+definition values, using the same value for each as the equivalent
+``kDL<type>`` enum from dlpack.h. The list will be kept in sync with those
+equivalent enum values over time to ensure compatibility, rather than
+potentially diverging. To avoid the Arrow project having to be in the
+position of vetting new hardware devices, new additions should first be
+added to dlpack before we add a corresponding macro here.
+
+To ensure predictability with the ABI, we use macros instead of an ``enum``
+so the storage type is not compiler dependent.
+
+.. c:macro:: ARROW_DEVICE_CPU
+
+    CPU Device, equivalent to just using ``ArrowArray`` directly instead of
+    using ``ArrowDeviceArray``.
+
+.. c:macro:: ARROW_DEVICE_CUDA
+
+    A `CUDA` GPU Device. This could represent data allocated either with the
+    runtime library (``cudaMalloc``) or the device driver (``cuMemAlloc``).
+
+.. c:macro:: ARROW_DEVICE_CUDA_HOST
+
+    CPU memory that was pinned and page-locked by CUDA by using
+    ``cudaMallocHost`` or ``cuMemAllocHost``.
+
+.. c:macro:: ARROW_DEVICE_OPENCL
+
+    Data allocated on the device by using the `OpenCL (Open Computing Language)`
+    framework.
+
+.. c:macro:: ARROW_DEVICE_VULKAN
+
+    Data allocated by the `Vulkan` framework and libraries.
+
+.. c:macro:: ARROW_DEVICE_METAL
+
+    Data on Apple GPU devices using the `Metal` framework and libraries.
+
+.. c:macro:: ARROW_DEVICE_VPI
+
+    Indicates usage of a Verilog simulator buffer.
+
+.. c:macro:: ARROW_DEVICE_ROCM
+
+    An AMD device using the `ROCm` stack.
+
+.. c:macro:: ARROW_DEVICE_ROCM_HOST
+
+    CPU memory pinned and page-locked allocated ``hipMallocHost``.
+
+.. c:macro:: ARROW_DEVICE_EXT_DEV
+
+    This value is an escape-hatch for devices to extend which aren't
+    currently represented otherwise. Producers would need to provide
+    additional information/context specific to the device if using
+    this device type. This is used to quickly test extension devices
+    and semantics can differ based on the implementation.
+
+.. c:macro:: ARROW_DEVICE_CUDA_MANAGED
+
+    CUDA managed/unified memory which is allocated by ``cudaMallocManaged``.
+
+.. c:macro:: ARROW_DEVICE_ONEAPI
+
+    Unified shared memory allocated on an Intel `oneAPI` non-partitioned
+    device. A call to the ``oneAPI`` runtime is required to determine
+    the specific device type, the USM allocation type and the sycl context
+    that it is bound to.
+
+.. c:macro:: ARROW_DEVICE_WEBGPU
+
+    GPU support for next-gen WebGPU standards
+
+.. c:macro:: ARROW_DEVICE_HEXAGON
+
+    Data allocated on a Qualcomm Hexagon DSP device.
+
+The ArrowDeviceArray structure
+------------------------------
+
+The ``ArrowDeviceArray`` structure embeds the C data ``ArrowArray`` structure
+and adds additional information necessary for consumers to use the data. It
+has the following fields:
+
+.. c:member:: struct ArrowArray ArrowDeviceArray.array
+
+    The allocated array data. The values in the ``void**`` buffers (along
+    with the buffers of any children) are what is allocated on the device.
+    The buffer values should be device pointers. The rest of the structure
+    should be accessible to the CPU.
+
+    The ``private_data`` and ``release`` callback of this structure should
+    contain any necessary information and structures related to freeing
+    the array according to the device it is allocated on, rather than
+    having a separate release callback and ``private_data`` pointer here.
+
+.. c:member:: int64_t ArrowDeviceArray.device_id
+
+    The device id to identify a specific device if multiple devices of this
+    type are on the system. The semantics of the id will be hardware dependent,
+    but we use an ``int64_t`` to future-proof the id as devices change over time.
+
+.. c:member:: ArrowDeviceType ArrowDeviceArray.device_type
+
+    The type of the device which can access the buffers in the array.
+
+.. c:member:: void* ArrowDeviceArray.sync_event
+
+    Optional. An event-like object to synchronize on if needed.
+
+    Many devices, like GPUs, are primarily asynchronous with respect to
+    CPU processing. As such in order to safely access memory, it is often
+    necessary to have an object to synchronize processing with. Since
+    different devices will use different types to specify this, we use a
+    void* which can be coerced into a pointer to whatever the device
+    appropriate type is.
+
+    If synchronization is not needed, this can be null. If this is non-null
+    then it MUST be used to call the appropriate sync method for the device
+    (e.g. ``cudaStreamWaitEvent`` or ``hipStreamWaitEvent``) before attempting
+    to access the memory in the buffers.
+
+    Expected types to coerce this ``void*`` to depending on the reported
+    device type:
+
+    * CUDA: ``cudaEvent_t*``
+    * ROCm: ``hipEvent_t*``
+    * OpenCL: ``cl_event*``
+    * Vulkan: ``VkEvent*``
+    * Metal: ``MTLEvent*``
+    * OneAPI: ``sycl::event*``
+
+    If an event is provided, then the producer MUST ensure that the event
+    is triggered/recorded at the end of the processing stream once the data
+    is considered available for use.
+
+
+.. c:member:: int64_t ArrowDeviceArray.reserved[3]
+
+    As non-CPU development expands, there may be a need to expand this
+    structure. In order to do so without potentially breaking ABI changes,
+    we reserve 24 bytes at the end of the object. This also has the added
+    benefit of bringing the total size of this structure to exactly 128
+    bytes (a power of 2) on 64-bit systems. These bytes should be zero'd
+    out after allocation in order to ensure safe evolution of the ABI in
+    the future.
+
+.. note::
+    Rather than store the shape / types of the data alongside the
+    ``ArrowDeviceArray``, users should utilize the existing ``ArrowSchema``
+    structure to pass any data type and shape information.
+
+Semantics
+=========
+
+Memory management
+-----------------
+
+The ``ArrowDeviceArray`` structure contains an ``ArrowArray`` object which
+itself has :ref:`specific semantics <c-data-interface-semantics>` for releasing
+memory. The term *"base structure"* below refers to the ``ArrowDeviceArray``
+object that is passed directly between the producer and consumer -- not any
+child structure thereof.
+
+It is intended for the base structure to be stack- or heap-allocated by the
+*consumer*. In this case, the producer API should take a pointer to the
+consumer-allocated structure.
+
+However, any data pointed to by the struct MUST be allocated and maintained
+by the producer. This includes the ``sync_event`` member if it is not null,
+along with any pointers in the ``ArrowArray`` object as usual. Data lifetime
+is managed through the ``release`` callback of the ``ArrowArray`` member.
+
+For an ``ArrowDeviceArray``, the semantics of a released structure and the
+callback semantics are identical to those for
+:ref:`ArrowArray itself <c-data-interface-released>`. Any producer specific context
+information necessary for releasing the device data buffers, in addition to
+any allocated event, should be stored in the ``private_data`` member of
+the ``ArrowArray`` and managed by the ``release`` callback.
+
+Moving an array
+'''''''''''''''
+
+The consumer can *move* the ``ArrowDeviceArray`` structure by bitwise copying
+or shallow member-wise copying. Then it MUST mark the source structure released
+by setting the ``release`` member of the embedded ``ArrowArray`` structure to
+``NULL``, but *without* calling that release callback. This ensures that only
+one live copy of the struct is active at any given time and that lifetime is
+correctly communicated to the producer.
+
+As usual, the release callback will be called on the destination structure
+when it is not needed anymore.
+
+Record batches
+--------------
+As with the C data interface itself, a record batch can be trivially considered
+as an equivalent struct array. In this case the metadata of the top-level
+``ArrowSchema`` can be used for schema-level metadata of the record batch.
+
+Mutability
+----------
+
+Both the producer and the consumer SHOULD consider the exported data (that
+is, the data reachable on the device through the ``buffers`` member of
+the embedded ``ArrowArray``) to be immutable, as either party could otherwise
+see inconsistent data while the other is mutating it.
+
+Likewise, if the ``sync_event`` member is non-NULL, the consumer should not
+attempt to access or read the data until they have synchronized on that event.
+
+C producer example
+====================
+
+Exporting a simple ``int32`` device array
+-----------------------------------------
+
+Export a non-nullable ``int32`` type with empty metadata. An example of this
+can be seen in the :ref:`C data interface docs directly <c-data-interface-export-int32-schema>`.
+
+To export the data itself, we transfer ownership to the consumer through
+the release callback. This example will use CUDA, but the equivalent calls
+could be used for any device:
+
+.. code-block:: c
+
+    static void release_int32_device_array(struct ArrowArray* array) {
+        assert(array->n_buffers == 2);
+        // destroy the event
+        cudaEvent_t* ev_ptr = reinterpret_cast<cudaEvent_t*>(array->private_data);
+        cudaError_t status = cudaEventDestroy(*ev_ptr);
+        assert(status == cudaSuccess);
+        free(ev_ptr);
+
+        // free the buffers and the buffers array
+        status = cudaFree(array->buffers[1]);
+        assert(status == cudaSuccess);
+        free(array->buffers);
+
+        // mark released
+        array->release = NULL;
+    }
+
+    __host__ void export_int32_device_array(void* cudaAllocdPtr,
+                                            cudaStream_t stream,
+                                            int64_t length,
+                                            struct ArrowDeviceArray* array) {
+        // get device id
+        int device;
+        cudaError_t status;
+        status = cudaGetDevice(&device);
+        assert(status == cudaSuccess);
+
+        cudaEvent_t* ev_ptr = (cudaEvent_t*)malloc(sizeof(cudaEvent_t));
+        assert(ev_ptr != NULL);
+        status = cudaEventCreate(ev_ptr);
+        assert(status == cudaSuccess);
+
+        // record event on the stream, assuming that the passed in
+        // stream is where the work to produce the data will be processing.        
+        status = cudaEventRecord(*ev_ptr, stream);
+        assert(status == cudaSuccess);
+
+        // initialize fields
+        *array = (struct ArrowDeviceArray) {
+            .array = (struct ArrowArray) {
+                .length = length,
+                .null_count = 0,
+                .offset = 0,
+                .n_buffers = 2,
+                .n_children = 0,
+                .children = NULL,
+                .dictionary = NULL,
+                // bookeeping
+                .release = &release_int32_device_array,
+                .private_data = reinterpret_cast<void*>(ev_ptr),
+            },
+            .device_id = static_cast<int64_t>(device),
+            .device_type = ARROW_DEVICE_CUDA,
+            // pass the event pointer to the consumer
+            .sync_event = reinterpret_cast<void*>(ev_ptr),
+        };
+
+        // allocate list of buffers
+        array->array.buffers = (const void**)malloc(sizeof(void*) * array->array.n_buffers);
+        assert(array->array.buffers != NULL);
+        array->array.buffers[0] = NULL;
+        array->array.buffers[1] = cudaAllocdPtr;
+    }
+
+================
+Device Stream Interface
+================
+
+Like the :ref:`C stream interface <c-stream-interface>`, the C Device data 
+interface also specifies a higher-level structure for easing communication
+of streaming data within a single process. Defining an ``ArrowDeviceArrayStream``
+structure.
+
+Semantics
+=========
+
+An Arrow C device stream exposes a streaming source of data chunks, each with
+the same schema. Chunks are obtained by calling a blocking pull-style iteration
+function. It is expected that all chunks should be providing data on the same
+device type (but not necessarily the same device id). If it is necessary
+to provide a stream of data on multiple device types, a producer should 
+provide a separate stream object for each device type.
+
+Structure definition
+====================
+
+The C device stream interface is defined by a single ``struct`` definition:
+
+.. code-block:: c
+
+    #ifndef ARROW_C_DEVICE_STREAM_INTERFACE
+    #define ARROW_C_DEVICE_STREAM_INTERFACE
+
+    struct ArrowDeviceArrayStream {
+        // device type that all arrays will be accessible from
+        ArrowDeviceType device_type;
+        // callbacks
+        int (*get_schema)(struct ArrowDeviceArrayStream*, struct ArrowSchema*);
+        int (*get_next)(struct ArrowDeviceArrayStream*, struct ArrowDeviceArray*);
+        const char* (*get_last_error)(struct ArrowDeviceArrayStream*);
+
+        // release callback
+        void (*release)(struct ArrowDeviceArrayStream*);
+
+        // opaque producer-specific data
+        void* private_data;
+    };
+
+    #endif  // ARROW_C_DEVICE_STREAM_INTERFACE
+
+.. note::
+    The canonical guard ``ARROW_C_DEVICE_STREAM_INTERFACE`` is meant to avoid
+    duplicate definitions if two projects copy the C device stream interface
+    definitions into their own headers, and a third-party project includes
+    from these two projects. It is therefore important that this guard is
+    kept exactly as-is when these definitions are copied.
+
+The ArrowDeviceArrayStream structure
+------------------------------------
+
+The ``ArrowDeviceArrayStream`` provides a device type that can access the
+resulting data along with the required callbacks to interact with a
+streaming source of Arrow arrays. It has the following fields:
+
+.. c:member:: ArrowDeviceType device_type
+
+    The device type that this stream produces data on. All 
+    ``ArrowDeviceArray``s that are produced by this stream should have the
+    same device type as is set here. This is a convenience for the consumer
+    to not have to check every array that is retrieved and instead allows
+    higher-level coding constructs for streams.
+
+.. c:member:: int (*ArrowDeviceArrayStream.get_schema)(struct ArrowDeviceArrayStream*, struct ArrowSchema* out)
+
+    *Mandatory.* This callback allows the consumer to query the schema of
+    the chunks of data in the stream. The schema is the same for all data
+    chunks.
+
+    This callback must NOT be called on a released ``ArrowDeviceArrayStream``.
+
+    *Return value:* 0 on success, a non-zero
+    :ref:`error code <c-stream-interface-error-codes>` otherwise.
+
+.. c:member:: int (*ArrowDeviceArrayStream.get_next)(struct ArrowDeviceArrayStream*, struct ArrowDeviceArray* out)
+
+    *Mandatory.* This callback allows the consumer to get the next chunk of
+    data in the stream.
+
+    This callback must NOT be called on a released ``ArrowDeviceArrayStream``.
+
+    The next chunk of data MUST be accessible from a device type matching the
+    :c:member:`ArrowDeviceArrayStream.device_type`.
+
+    *Return value:* 0 on success, a non-zero
+    :ref:`error code <c-stream-interface-error-codes>` otherwise.
+
+    On success, the consumer must check whether the ``ArrowDeviceArray``'s
+    embedded ``ArrowArray`` is marked :ref:`released <c-data-interface-released>`.
+    If the embedded ``ArrowDeviceArray.array`` is released, then the end of the 
+    stream has been reached. Otherwise, the ``ArrowDeviceArray`` contains a
+    valid data chunk.
+
+.. c:member:: const char* (*ArrowDeviceArrayStream.get_last_error)(struct ArrowDeviceArrayStream*)
+
+    *Mandatory.* This callback allows the consumer to get a textual description
+    of the last error.
+
+    This callback must ONLY be called if the last operation on the 
+    ``ArrowDeviceArrayStream`` returned an error. It must NOT be called on a
+    released ``ArrowDeviceArrayStream``.
+
+    *Return value:* a pointer to a NULL-terminated character string 
+    (UTF8-encoded). NULL can also be returned if no detailed description is
+    available.
+
+    The returned pointer is only guaranteed to be valid until the next call
+    of one of the stream's callbacks. The character string it points to should
+    be copied to consumer-managed storage if it is intended to survive longer.
+
+.. c:member:: void (*ArrowDeviceArrayStream.release)(struct ArrowDeviceArrayStream*)
+
+    *Mandatory.* A pointer to a producer-provided release callback.
+
+.. c:member:: void* ArrowDeviceArrayStream.private_data
+
+    *Optional.* An opaque pointer to producer-provided private data.
+
+    Consumers MUST NOT process this member. Lifetime of this member is
+    handled by the producer, and especially by the release callback.
+
+Result lifetimes
+----------------
+
+The data returned by the ``get_schema`` and ``get_next`` callbacks must be
+released independantly. Their lifetimes are not tied to that of 
+``ArrowDeviceArrayStream``.
+
+Stream lifetime
+---------------
+
+Lifetime of the C stream is managed using a release callback with similar 
+usage as in :ref:`C data interface <c-data-interface-released>`.
+
+Thread safety
+-------------
+
+The stream source is not assumed to be thread-safe. Consumers wanting to
+call ``get_next`` from several threads should ensure those calls are 
+serialized.
+
+Updating this specification
+===========================
+
+Once this specification is supported in an official Arrow release, the C ABI
+is frozen. This means that the ``ArrowDeviceArray`` structure definition 

Review Comment:
   Added a note here to clarify the language. Let me know your thoughts, thanks!



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [arrow] pitrou commented on a diff in pull request #34972: GH-34971: [Format] Add non-CPU version of C Data Interface

["pitrou (via GitHub)" <gi...@apache.org>]

pitrou commented on code in PR #34972:
URL: https://github.com/apache/arrow/pull/34972#discussion_r1213457088


##########
docs/source/format/CDeviceDataInterface.rst:
##########
@@ -0,0 +1,628 @@
+.. Licensed to the Apache Software Foundation (ASF) under one
+.. or more contributor license agreements.  See the NOTICE file
+.. distributed with this work for additional information
+.. regarding copyright ownership.  The ASF licenses this file
+.. to you under the Apache License, Version 2.0 (the
+.. "License"); you may not use this file except in compliance
+.. with the License.  You may obtain a copy of the License at
+
+..   http://www.apache.org/licenses/LICENSE-2.0
+
+.. Unless required by applicable law or agreed to in writing,
+.. software distributed under the License is distributed on an
+.. "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+.. KIND, either express or implied.  See the License for the
+.. specific language governing permissions and limitations
+.. under the License.
+
+.. highlight:: c
+
+.. _c-device-data-interface:
+
+=================================
+The Arrow C Device data interface
+=================================
+
+.. note:: The Arrow C Device Data Interface should be considered experimental
+
+Rationale
+=========
+
+The current :ref:`C Data Interface <c-data-interface>`, and most
+implementations of it, make the assumption that all data buffers provided
+are CPU buffers. Since Apache Arrow is designed to be a universal in-memory
+format for representing tabular ("columnar") data, there will be the desire
+to leverage this data on non-CPU hardware such as GPUs. One example of such
+a case is the `RAPIDS cuDF library`_ which uses the Arrow memory format with
+CUDA for NVIDIA GPUs. Since copying data from host to device and back is
+expensive, the ideal would be to be able to leave the data on the device
+for as long as possible, even when passing it between runtimes and
+libraries.
+
+The Arrow C Device data interface builds on the existing C data interface
+by adding a very small, stable set of C definitions to it. These definitions
+are equivalents to the ``ArrowArray`` and ``ArrowArrayStream`` structures
+from the C Data Interface which add members to allow specifying the device
+type and pass necessary information to synchronize with the producer.
+For non-C/C++ languages and runtimes, translating the C definitions to
+corresponding C FFI declarations should be just as simple as with the
+current C data interface.
+
+Applications and libraries can then use Arrow schemas and Arrow formatted
+memory on non-CPU devices to exchange data just as easily as they do
+now with CPU data. This will enable leaving data on those devices longer
+and avoiding costly copies back and forth between the host and device
+just to leverage new libraries and runtimes.
+
+Goals
+-----
+
+* Expose an ABI-stable interface built on the existing C data interface.
+* Make it easy for third-party projects to implement support with little
+  initial investment.
+* Allow zero-copy sharing of Arrow formatted device memory between
+  independant runtimes and components running in the same process.
+* Avoid the need for one-to-one adaptation layers such as the
+  `CUDA Array Interface`_ for Python processes to pass CUDA data.
+* Enable integration without explicit dependencies (either at compile-time
+  or runtime) on the Arrow software project itself.
+
+The intent is for the Arrow C Device data interface to expand the reach
+of the current C data interface, allowing it to also become the standard
+low-level building block for columnar processing on devices like GPUs or
+FPGAs.
+
+Structure definitions
+=====================
+
+Because this is built on the C data interface, the C Device data interface
+uses the ``ArrowSchema`` and ``ArrowArray`` structures as defined in the
+:ref:`C data interface spec <c-data-interface-struct-defs>`. It then adds the
+following free-standing definitions. Like the rest of the Arrow project,
+they are available under the Apache License 2.0.
+
+.. code-block:: c
+
+    #ifndef ARROW_C_DEVICE_DATA_INTERFACE
+    #define ARROW_C_DEVICE_DATA_INTERFACE
+
+    // Device type for the allocated memory
+    typedef int32_t ArrowDeviceType;
+
+    // CPU device, same as using ArrowArray directly
+    #define ARROW_DEVICE_CPU 1
+    // CUDA GPU Device
+    #define ARROW_DEVICE_CUDA 2
+    // Pinned CUDA CPU memory by cudaMallocHost
+    #define ARROW_DEVICE_CUDA_HOST 3
+    // OpenCL Device
+    #define ARROW_DEVICE_OPENCL 4
+    // Vulkan buffer for next-gen graphics
+    #define ARROW_DEVICE_VULKAN 7
+    // Metal for Apple GPU
+    #define ARROW_DEVICE_METAL 8
+    // Verilog simulator buffer
+    #define ARROW_DEVICE_VPI 9
+    // ROCm GPUs for AMD GPUs
+    #define ARROW_DEVICE_ROCM 10
+    // Pinned ROCm CPU memory allocated by hipMallocHost
+    #define ARROW_DEVICE_ROCM_HOST 11
+    // Reserved for extension
+    //
+    // used to quickly test extension devices, semantics
+    // can differ based on implementation
+    #define ARROW_DEVICE_EXT_DEV 12
+    // CUDA managed/unified memory allocated by cudaMallocManaged
+    #define ARROW_DEVICE_CUDA_MANAGED 13
+    // Unified shared memory allocated on a oneAPI
+    // non-partitioned device.
+    //
+    // A call to the oneAPI runtime is required to determine the
+    // device type, the USM allocation type and the sycl context
+    // that it is bound to.
+    #define ARROW_DEVICE_ONEAPI 14
+    // GPU support for next-gen WebGPU standard
+    #define ARROW_DEVICE_WEBGPU 15
+    // Qualcomm Hexagon DSP
+    #define ARROW_DEVICE_HEXAGON 16
+
+    struct ArrowDeviceArray {
+      struct ArrowArray array;
+      int64_t device_id;
+      ArrowDeviceType device_type;
+      void* sync_event;
+
+      // reserved bytes for future expansion
+      int64_t reserved[3];
+    };
+
+    #endif  // ARROW_C_DEVICE_DATA_INTERFACE
+
+.. note::
+   The canonical guard ``ARROW_C_DEVICE_DATA_INTERFACE`` is meant to avoid
+   duplicate definitions if two projects copy the definitions in their own
+   headers, and a third-party project includes from these two projects. It
+   is therefore important that this guard is kept exactly as-is when these
+   definitions are copied.
+
+ArrowDeviceType
+---------------
+
+The ``ArrowDeviceType`` typedef is used to indicate what type of device the
+provided memory buffers were allocated on. This, in conjunction with the
+``device_id``, should be sufficient to reference the correct data buffers.
+
+We then use macros to define values for different device types. The provided
+macro values are compatible with the widely used `dlpack`_ ``DLDeviceType``
+definition values, using the same value for each as the equivalent
+``kDL<type>`` enum from dlpack.h. The list will be kept in sync with those
+equivalent enum values over time to ensure compatibility, rather than
+potentially diverging. To avoid the Arrow project having to be in the
+position of vetting new hardware devices, new additions should first be
+added to dlpack before we add a corresponding macro here.
+
+To ensure predictability with the ABI, we use macros instead of an ``enum``
+so the storage type is not compiler dependent.
+
+.. c:macro:: ARROW_DEVICE_CPU
+
+    CPU Device, equivalent to just using ``ArrowArray`` directly instead of
+    using ``ArrowDeviceArray``.
+
+.. c:macro:: ARROW_DEVICE_CUDA
+
+    A `CUDA`_ GPU Device. This could represent data allocated either with the
+    runtime library (``cudaMalloc``) or the device driver (``cuMemAlloc``).
+
+.. c:macro:: ARROW_DEVICE_CUDA_HOST
+
+    CPU memory that was pinned and page-locked by CUDA by using
+    ``cudaMallocHost`` or ``cuMemAllocHost``.
+
+.. c:macro:: ARROW_DEVICE_OPENCL
+
+    Data allocated on the device by using the `OpenCL (Open Computing Language)`_
+    framework.
+
+.. c:macro:: ARROW_DEVICE_VULKAN
+
+    Data allocated by the `Vulkan`_ framework and libraries.
+
+.. c:macro:: ARROW_DEVICE_METAL
+
+    Data on Apple GPU devices using the `Metal`_ framework and libraries.
+
+.. c:macro:: ARROW_DEVICE_VPI
+
+    Indicates usage of a Verilog simulator buffer.
+
+.. c:macro:: ARROW_DEVICE_ROCM
+
+    An AMD device using the `ROCm`_ stack.
+
+.. c:macro:: ARROW_DEVICE_ROCM_HOST
+
+    CPU memory pinned and page-locked allocated ``hipMallocHost``.
+
+.. c:macro:: ARROW_DEVICE_EXT_DEV
+
+    This value is an escape-hatch for devices to extend which aren't
+    currently represented otherwise. Producers would need to provide
+    additional information/context specific to the device if using
+    this device type. This is used to quickly test extension devices
+    and semantics can differ based on the implementation.
+
+.. c:macro:: ARROW_DEVICE_CUDA_MANAGED
+
+    CUDA managed/unified memory which is allocated by ``cudaMallocManaged``.
+
+.. c:macro:: ARROW_DEVICE_ONEAPI
+
+    Unified shared memory allocated on an Intel `oneAPI`_ non-partitioned
+    device. A call to the ``oneAPI`` runtime is required to determine
+    the specific device type, the USM allocation type and the sycl context
+    that it is bound to.
+
+.. c:macro:: ARROW_DEVICE_WEBGPU
+
+    GPU support for next-gen WebGPU standards
+
+.. c:macro:: ARROW_DEVICE_HEXAGON
+
+    Data allocated on a Qualcomm Hexagon DSP device.
+
+The ArrowDeviceArray structure
+------------------------------
+
+The ``ArrowDeviceArray`` structure embeds the C data ``ArrowArray`` structure
+and adds additional information necessary for consumers to use the data. It
+has the following fields:
+
+.. c:member:: struct ArrowArray ArrowDeviceArray.array
+
+    The allocated array data. The values in the ``void**`` buffers (along
+    with the buffers of any children) are what is allocated on the device.
+    The buffer values should be device pointers. The rest of the structure
+    should be accessible to the CPU.
+
+    The ``private_data`` and ``release`` callback of this structure should
+    contain any necessary information and structures related to freeing
+    the array according to the device it is allocated on, rather than
+    having a separate release callback and ``private_data`` pointer here.
+
+.. c:member:: int64_t ArrowDeviceArray.device_id
+
+    The device id to identify a specific device if multiple devices of this
+    type are on the system. The semantics of the id will be hardware dependent,
+    but we use an ``int64_t`` to future-proof the id as devices change over time.
+
+.. c:member:: ArrowDeviceType ArrowDeviceArray.device_type
+
+    The type of the device which can access the buffers in the array.
+
+.. c:member:: void* ArrowDeviceArray.sync_event
+
+    Optional. An event-like object to synchronize on if needed.
+
+    Many devices, like GPUs, are primarily asynchronous with respect to
+    CPU processing. As such in order to safely access memory, it is often
+    necessary to have an object to synchronize processing with. Since
+    different devices will use different types to specify this, we use a
+    void* which can be coerced into a pointer to whatever the device
+    appropriate type is.
+
+    If synchronization is not needed, this can be null. If this is non-null
+    then it MUST be used to call the appropriate sync method for the device
+    (e.g. ``cudaStreamWaitEvent`` or ``hipStreamWaitEvent``) before attempting
+    to access the memory in the buffers.
+
+    Expected types to coerce this ``void*`` to depending on the reported
+    device type:
+
+    * CUDA: ``cudaEvent_t*``
+    * ROCm: ``hipEvent_t*``
+    * OpenCL: ``cl_event*``
+    * Vulkan: ``VkEvent*``
+    * Metal: ``MTLEvent*``
+    * OneAPI: ``sycl::event*``
+
+    If an event is provided, then the producer MUST ensure that the event
+    is triggered/recorded at the end of the processing stream once the data
+    is considered available for use.
+
+
+.. c:member:: int64_t ArrowDeviceArray.reserved[3]
+
+    As non-CPU development expands, there may be a need to expand this
+    structure. In order to do so without potentially breaking ABI changes,
+    we reserve 24 bytes at the end of the object. This also has the added
+    benefit of bringing the total size of this structure to exactly 128
+    bytes (a power of 2) on 64-bit systems. These bytes should be zero'd
+    out after allocation in order to ensure safe evolution of the ABI in
+    the future.
+
+.. note::
+    Rather than store the shape / types of the data alongside the
+    ``ArrowDeviceArray``, users should utilize the existing ``ArrowSchema``
+    structure to pass any data type and shape information.
+
+Semantics
+=========
+
+Memory management
+-----------------
+
+The ``ArrowDeviceArray`` structure contains an ``ArrowArray`` object which
+itself has :ref:`specific semantics <c-data-interface-semantics>` for releasing
+memory. The term *"base structure"* below refers to the ``ArrowDeviceArray``
+object that is passed directly between the producer and consumer -- not any
+child structure thereof.
+
+It is intended for the base structure to be stack- or heap-allocated by the
+*consumer*. In this case, the producer API should take a pointer to the
+consumer-allocated structure.
+
+However, any data pointed to by the struct MUST be allocated and maintained
+by the producer. This includes the ``sync_event`` member if it is not null,
+along with any pointers in the ``ArrowArray`` object as usual. Data lifetime
+is managed through the ``release`` callback of the ``ArrowArray`` member.
+
+For an ``ArrowDeviceArray``, the semantics of a released structure and the
+callback semantics are identical to those for
+:ref:`ArrowArray itself <c-data-interface-released>`. Any producer specific context
+information necessary for releasing the device data buffers, in addition to
+any allocated event, should be stored in the ``private_data`` member of
+the ``ArrowArray`` and managed by the ``release`` callback.
+
+Moving an array
+'''''''''''''''
+
+The consumer can *move* the ``ArrowDeviceArray`` structure by bitwise copying
+or shallow member-wise copying. Then it MUST mark the source structure released
+by setting the ``release`` member of the embedded ``ArrowArray`` structure to
+``NULL``, but *without* calling that release callback. This ensures that only
+one live copy of the struct is active at any given time and that lifetime is
+correctly communicated to the producer.
+
+As usual, the release callback will be called on the destination structure
+when it is not needed anymore.
+
+Record batches
+--------------
+As with the C data interface itself, a record batch can be trivially considered
+as an equivalent struct array. In this case the metadata of the top-level
+``ArrowSchema`` can be used for schema-level metadata of the record batch.
+
+Mutability
+----------
+
+Both the producer and the consumer SHOULD consider the exported data (that
+is, the data reachable on the device through the ``buffers`` member of
+the embedded ``ArrowArray``) to be immutable, as either party could otherwise
+see inconsistent data while the other is mutating it.
+
+Likewise, if the ``sync_event`` member is non-NULL, the consumer should not
+attempt to access or read the data until they have synchronized on that event.
+
+C producer example
+====================
+
+Exporting a simple ``int32`` device array
+-----------------------------------------
+
+Export a non-nullable ``int32`` type with empty metadata. An example of this
+can be seen in the :ref:`C data interface docs directly <c-data-interface-export-int32-schema>`.
+
+To export the data itself, we transfer ownership to the consumer through
+the release callback. This example will use CUDA, but the equivalent calls
+could be used for any device:
+
+.. code-block:: c
+
+    static void release_int32_device_array(struct ArrowArray* array) {
+        assert(array->n_buffers == 2);
+        // destroy the event
+        cudaEvent_t* ev_ptr = reinterpret_cast<cudaEvent_t*>(array->private_data);

Review Comment:
   Oh, I misread. Thanks for the clarification!



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [arrow] pitrou commented on a diff in pull request #34972: GH-34971: [Format] Add non-CPU version of C Data Interface

["pitrou (via GitHub)" <gi...@apache.org>]

pitrou commented on code in PR #34972:
URL: https://github.com/apache/arrow/pull/34972#discussion_r1219877919


##########
cpp/src/arrow/c/abi.h:
##########
@@ -15,10 +15,27 @@
 // specific language governing permissions and limitations
 // under the License.
 
+/// \file abi.h Arrow C-Data Interface
+///
+/// The Arrow C-Data interface defines a very small, stable set

Review Comment:
   Nit
   ```suggestion
   /// \file abi.h Arrow C Data Interface
   ///
   /// The Arrow C Data interface defines a very small, stable set
   ```



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [arrow] pitrou commented on a diff in pull request #34972: GH-34971: [Format] Enhance C-Data API to support non-cpu cases

["pitrou (via GitHub)" <gi...@apache.org>]

pitrou commented on code in PR #34972:
URL: https://github.com/apache/arrow/pull/34972#discussion_r1211981587


##########
cpp/src/arrow/c/abi.h:
##########
@@ -15,10 +15,30 @@
 // specific language governing permissions and limitations
 // under the License.
 
+/// \file abi.h Arrow C-Data Interface
+///
+/// The Arrow C-Data interface defines a very small, stable set
+/// of C definitions which can be easily copied into any project's
+/// source code and vendored to be used for columnar data interchange
+/// in the Arrow format. For non-C/C++ languages and runtimes,
+/// it should be almost as easy to translate the C definitions into
+/// the corresponding C FFI declarations.
+///
+/// Applications and libraries can therefore work with Arrow memory
+/// without necessarily using the Arrow libraries or reinventing
+/// the wheel. Developers can choose between tight integration
+/// with the Arrow software project or minimal integration with
+/// the Arrow format only.
+
 #pragma once
 
 #include <stdint.h>
 
+/// \defgroup Arrow C-Data Interface

Review Comment:
   It's "C Data Interface", not C-Data or whatever else.



##########
docs/source/format/CDeviceDataInterface.rst:
##########
@@ -0,0 +1,628 @@
+.. Licensed to the Apache Software Foundation (ASF) under one
+.. or more contributor license agreements.  See the NOTICE file
+.. distributed with this work for additional information
+.. regarding copyright ownership.  The ASF licenses this file
+.. to you under the Apache License, Version 2.0 (the
+.. "License"); you may not use this file except in compliance
+.. with the License.  You may obtain a copy of the License at
+
+..   http://www.apache.org/licenses/LICENSE-2.0
+
+.. Unless required by applicable law or agreed to in writing,
+.. software distributed under the License is distributed on an
+.. "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+.. KIND, either express or implied.  See the License for the
+.. specific language governing permissions and limitations
+.. under the License.
+
+.. highlight:: c
+
+.. _c-device-data-interface:
+
+=================================
+The Arrow C Device data interface
+=================================
+
+.. note:: The Arrow C Device Data Interface should be considered experimental
+
+Rationale
+=========
+
+The current :ref:`C Data Interface <c-data-interface>`, and most
+implementations of it, make the assumption that all data buffers provided
+are CPU buffers. Since Apache Arrow is designed to be a universal in-memory
+format for representing tabular ("columnar") data, there will be the desire
+to leverage this data on non-CPU hardware such as GPUs. One example of such
+a case is the `RAPIDS cuDF library`_ which uses the Arrow memory format with
+CUDA for NVIDIA GPUs. Since copying data from host to device and back is
+expensive, the ideal would be to be able to leave the data on the device
+for as long as possible, even when passing it between runtimes and
+libraries.
+
+The Arrow C Device data interface builds on the existing C data interface
+by adding a very small, stable set of C definitions to it. These definitions
+are equivalents to the ``ArrowArray`` and ``ArrowArrayStream`` structures
+from the C Data Interface which add members to allow specifying the device
+type and pass necessary information to synchronize with the producer.
+For non-C/C++ languages and runtimes, translating the C definitions to
+corresponding C FFI declarations should be just as simple as with the
+current C data interface.
+
+Applications and libraries can then use Arrow schemas and Arrow formatted
+memory on non-CPU devices to exchange data just as easily as they do
+now with CPU data. This will enable leaving data on those devices longer
+and avoiding costly copies back and forth between the host and device
+just to leverage new libraries and runtimes.
+
+Goals
+-----
+
+* Expose an ABI-stable interface built on the existing C data interface.
+* Make it easy for third-party projects to implement support with little
+  initial investment.
+* Allow zero-copy sharing of Arrow formatted device memory between
+  independant runtimes and components running in the same process.
+* Avoid the need for one-to-one adaptation layers such as the
+  `CUDA Array Interface`_ for Python processes to pass CUDA data.
+* Enable integration without explicit dependencies (either at compile-time
+  or runtime) on the Arrow software project itself.
+
+The intent is for the Arrow C Device data interface to expand the reach
+of the current C data interface, allowing it to also become the standard
+low-level building block for columnar processing on devices like GPUs or
+FPGAs.
+
+Structure definitions
+=====================
+
+Because this is built on the C data interface, the C Device data interface
+uses the ``ArrowSchema`` and ``ArrowArray`` structures as defined in the
+:ref:`C data interface spec <c-data-interface-struct-defs>`. It then adds the
+following free-standing definitions. Like the rest of the Arrow project,
+they are available under the Apache License 2.0.
+
+.. code-block:: c
+
+    #ifndef ARROW_C_DEVICE_DATA_INTERFACE
+    #define ARROW_C_DEVICE_DATA_INTERFACE
+
+    // Device type for the allocated memory
+    typedef int32_t ArrowDeviceType;
+
+    // CPU device, same as using ArrowArray directly
+    #define ARROW_DEVICE_CPU 1
+    // CUDA GPU Device
+    #define ARROW_DEVICE_CUDA 2
+    // Pinned CUDA CPU memory by cudaMallocHost
+    #define ARROW_DEVICE_CUDA_HOST 3
+    // OpenCL Device
+    #define ARROW_DEVICE_OPENCL 4
+    // Vulkan buffer for next-gen graphics
+    #define ARROW_DEVICE_VULKAN 7
+    // Metal for Apple GPU
+    #define ARROW_DEVICE_METAL 8
+    // Verilog simulator buffer
+    #define ARROW_DEVICE_VPI 9
+    // ROCm GPUs for AMD GPUs
+    #define ARROW_DEVICE_ROCM 10
+    // Pinned ROCm CPU memory allocated by hipMallocHost
+    #define ARROW_DEVICE_ROCM_HOST 11
+    // Reserved for extension
+    //
+    // used to quickly test extension devices, semantics
+    // can differ based on implementation
+    #define ARROW_DEVICE_EXT_DEV 12
+    // CUDA managed/unified memory allocated by cudaMallocManaged
+    #define ARROW_DEVICE_CUDA_MANAGED 13
+    // Unified shared memory allocated on a oneAPI
+    // non-partitioned device.
+    //
+    // A call to the oneAPI runtime is required to determine the
+    // device type, the USM allocation type and the sycl context
+    // that it is bound to.
+    #define ARROW_DEVICE_ONEAPI 14
+    // GPU support for next-gen WebGPU standard
+    #define ARROW_DEVICE_WEBGPU 15
+    // Qualcomm Hexagon DSP
+    #define ARROW_DEVICE_HEXAGON 16
+
+    struct ArrowDeviceArray {
+      struct ArrowArray array;
+      int64_t device_id;
+      ArrowDeviceType device_type;
+      void* sync_event;
+
+      // reserved bytes for future expansion
+      int64_t reserved[3];
+    };
+
+    #endif  // ARROW_C_DEVICE_DATA_INTERFACE
+
+.. note::
+   The canonical guard ``ARROW_C_DEVICE_DATA_INTERFACE`` is meant to avoid
+   duplicate definitions if two projects copy the definitions in their own
+   headers, and a third-party project includes from these two projects. It
+   is therefore important that this guard is kept exactly as-is when these
+   definitions are copied.
+
+ArrowDeviceType
+---------------
+
+The ``ArrowDeviceType`` typedef is used to indicate what type of device the
+provided memory buffers were allocated on. This, in conjunction with the
+``device_id``, should be sufficient to reference the correct data buffers.
+
+We then use macros to define values for different device types. The provided
+macro values are compatible with the widely used `dlpack`_ ``DLDeviceType``
+definition values, using the same value for each as the equivalent
+``kDL<type>`` enum from dlpack.h. The list will be kept in sync with those
+equivalent enum values over time to ensure compatibility, rather than
+potentially diverging. To avoid the Arrow project having to be in the
+position of vetting new hardware devices, new additions should first be
+added to dlpack before we add a corresponding macro here.
+
+To ensure predictability with the ABI, we use macros instead of an ``enum``
+so the storage type is not compiler dependent.
+
+.. c:macro:: ARROW_DEVICE_CPU
+
+    CPU Device, equivalent to just using ``ArrowArray`` directly instead of
+    using ``ArrowDeviceArray``.
+
+.. c:macro:: ARROW_DEVICE_CUDA
+
+    A `CUDA`_ GPU Device. This could represent data allocated either with the
+    runtime library (``cudaMalloc``) or the device driver (``cuMemAlloc``).
+
+.. c:macro:: ARROW_DEVICE_CUDA_HOST
+
+    CPU memory that was pinned and page-locked by CUDA by using
+    ``cudaMallocHost`` or ``cuMemAllocHost``.
+
+.. c:macro:: ARROW_DEVICE_OPENCL
+
+    Data allocated on the device by using the `OpenCL (Open Computing Language)`_
+    framework.
+
+.. c:macro:: ARROW_DEVICE_VULKAN
+
+    Data allocated by the `Vulkan`_ framework and libraries.
+
+.. c:macro:: ARROW_DEVICE_METAL
+
+    Data on Apple GPU devices using the `Metal`_ framework and libraries.
+
+.. c:macro:: ARROW_DEVICE_VPI
+
+    Indicates usage of a Verilog simulator buffer.
+
+.. c:macro:: ARROW_DEVICE_ROCM
+
+    An AMD device using the `ROCm`_ stack.
+
+.. c:macro:: ARROW_DEVICE_ROCM_HOST
+
+    CPU memory pinned and page-locked allocated ``hipMallocHost``.
+
+.. c:macro:: ARROW_DEVICE_EXT_DEV
+
+    This value is an escape-hatch for devices to extend which aren't
+    currently represented otherwise. Producers would need to provide
+    additional information/context specific to the device if using
+    this device type. This is used to quickly test extension devices
+    and semantics can differ based on the implementation.
+
+.. c:macro:: ARROW_DEVICE_CUDA_MANAGED
+
+    CUDA managed/unified memory which is allocated by ``cudaMallocManaged``.
+
+.. c:macro:: ARROW_DEVICE_ONEAPI
+
+    Unified shared memory allocated on an Intel `oneAPI`_ non-partitioned
+    device. A call to the ``oneAPI`` runtime is required to determine
+    the specific device type, the USM allocation type and the sycl context
+    that it is bound to.
+
+.. c:macro:: ARROW_DEVICE_WEBGPU
+
+    GPU support for next-gen WebGPU standards
+
+.. c:macro:: ARROW_DEVICE_HEXAGON
+
+    Data allocated on a Qualcomm Hexagon DSP device.
+
+The ArrowDeviceArray structure
+------------------------------
+
+The ``ArrowDeviceArray`` structure embeds the C data ``ArrowArray`` structure
+and adds additional information necessary for consumers to use the data. It
+has the following fields:
+
+.. c:member:: struct ArrowArray ArrowDeviceArray.array
+
+    The allocated array data. The values in the ``void**`` buffers (along
+    with the buffers of any children) are what is allocated on the device.
+    The buffer values should be device pointers. The rest of the structure
+    should be accessible to the CPU.
+
+    The ``private_data`` and ``release`` callback of this structure should
+    contain any necessary information and structures related to freeing
+    the array according to the device it is allocated on, rather than
+    having a separate release callback and ``private_data`` pointer here.
+
+.. c:member:: int64_t ArrowDeviceArray.device_id
+
+    The device id to identify a specific device if multiple devices of this
+    type are on the system. The semantics of the id will be hardware dependent,
+    but we use an ``int64_t`` to future-proof the id as devices change over time.
+
+.. c:member:: ArrowDeviceType ArrowDeviceArray.device_type
+
+    The type of the device which can access the buffers in the array.
+
+.. c:member:: void* ArrowDeviceArray.sync_event
+
+    Optional. An event-like object to synchronize on if needed.
+
+    Many devices, like GPUs, are primarily asynchronous with respect to
+    CPU processing. As such in order to safely access memory, it is often
+    necessary to have an object to synchronize processing with. Since
+    different devices will use different types to specify this, we use a
+    void* which can be coerced into a pointer to whatever the device
+    appropriate type is.
+
+    If synchronization is not needed, this can be null. If this is non-null
+    then it MUST be used to call the appropriate sync method for the device
+    (e.g. ``cudaStreamWaitEvent`` or ``hipStreamWaitEvent``) before attempting
+    to access the memory in the buffers.
+
+    Expected types to coerce this ``void*`` to depending on the reported
+    device type:
+
+    * CUDA: ``cudaEvent_t*``
+    * ROCm: ``hipEvent_t*``
+    * OpenCL: ``cl_event*``
+    * Vulkan: ``VkEvent*``
+    * Metal: ``MTLEvent*``
+    * OneAPI: ``sycl::event*``
+
+    If an event is provided, then the producer MUST ensure that the event
+    is triggered/recorded at the end of the processing stream once the data
+    is considered available for use.
+
+
+.. c:member:: int64_t ArrowDeviceArray.reserved[3]
+
+    As non-CPU development expands, there may be a need to expand this
+    structure. In order to do so without potentially breaking ABI changes,
+    we reserve 24 bytes at the end of the object. This also has the added
+    benefit of bringing the total size of this structure to exactly 128
+    bytes (a power of 2) on 64-bit systems. These bytes should be zero'd
+    out after allocation in order to ensure safe evolution of the ABI in
+    the future.
+
+.. note::
+    Rather than store the shape / types of the data alongside the
+    ``ArrowDeviceArray``, users should utilize the existing ``ArrowSchema``
+    structure to pass any data type and shape information.
+
+Semantics
+=========
+
+Memory management
+-----------------
+
+The ``ArrowDeviceArray`` structure contains an ``ArrowArray`` object which
+itself has :ref:`specific semantics <c-data-interface-semantics>` for releasing
+memory. The term *"base structure"* below refers to the ``ArrowDeviceArray``
+object that is passed directly between the producer and consumer -- not any
+child structure thereof.
+
+It is intended for the base structure to be stack- or heap-allocated by the
+*consumer*. In this case, the producer API should take a pointer to the
+consumer-allocated structure.
+
+However, any data pointed to by the struct MUST be allocated and maintained
+by the producer. This includes the ``sync_event`` member if it is not null,
+along with any pointers in the ``ArrowArray`` object as usual. Data lifetime
+is managed through the ``release`` callback of the ``ArrowArray`` member.
+
+For an ``ArrowDeviceArray``, the semantics of a released structure and the
+callback semantics are identical to those for
+:ref:`ArrowArray itself <c-data-interface-released>`. Any producer specific context
+information necessary for releasing the device data buffers, in addition to
+any allocated event, should be stored in the ``private_data`` member of
+the ``ArrowArray`` and managed by the ``release`` callback.
+
+Moving an array
+'''''''''''''''
+
+The consumer can *move* the ``ArrowDeviceArray`` structure by bitwise copying
+or shallow member-wise copying. Then it MUST mark the source structure released
+by setting the ``release`` member of the embedded ``ArrowArray`` structure to
+``NULL``, but *without* calling that release callback. This ensures that only
+one live copy of the struct is active at any given time and that lifetime is
+correctly communicated to the producer.
+
+As usual, the release callback will be called on the destination structure
+when it is not needed anymore.
+
+Record batches
+--------------
+As with the C data interface itself, a record batch can be trivially considered
+as an equivalent struct array. In this case the metadata of the top-level
+``ArrowSchema`` can be used for schema-level metadata of the record batch.
+
+Mutability
+----------
+
+Both the producer and the consumer SHOULD consider the exported data (that
+is, the data reachable on the device through the ``buffers`` member of
+the embedded ``ArrowArray``) to be immutable, as either party could otherwise
+see inconsistent data while the other is mutating it.
+
+Likewise, if the ``sync_event`` member is non-NULL, the consumer should not

Review Comment:
   Add a "synchronization" paragraph instead of tackling this in the mutability para?



##########
docs/source/format/CDeviceDataInterface.rst:
##########
@@ -0,0 +1,628 @@
+.. Licensed to the Apache Software Foundation (ASF) under one
+.. or more contributor license agreements.  See the NOTICE file
+.. distributed with this work for additional information
+.. regarding copyright ownership.  The ASF licenses this file
+.. to you under the Apache License, Version 2.0 (the
+.. "License"); you may not use this file except in compliance
+.. with the License.  You may obtain a copy of the License at
+
+..   http://www.apache.org/licenses/LICENSE-2.0
+
+.. Unless required by applicable law or agreed to in writing,
+.. software distributed under the License is distributed on an
+.. "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+.. KIND, either express or implied.  See the License for the
+.. specific language governing permissions and limitations
+.. under the License.
+
+.. highlight:: c
+
+.. _c-device-data-interface:
+
+=================================
+The Arrow C Device data interface
+=================================
+
+.. note:: The Arrow C Device Data Interface should be considered experimental
+
+Rationale
+=========
+
+The current :ref:`C Data Interface <c-data-interface>`, and most
+implementations of it, make the assumption that all data buffers provided
+are CPU buffers. Since Apache Arrow is designed to be a universal in-memory
+format for representing tabular ("columnar") data, there will be the desire
+to leverage this data on non-CPU hardware such as GPUs. One example of such
+a case is the `RAPIDS cuDF library`_ which uses the Arrow memory format with
+CUDA for NVIDIA GPUs. Since copying data from host to device and back is
+expensive, the ideal would be to be able to leave the data on the device
+for as long as possible, even when passing it between runtimes and
+libraries.
+
+The Arrow C Device data interface builds on the existing C data interface
+by adding a very small, stable set of C definitions to it. These definitions
+are equivalents to the ``ArrowArray`` and ``ArrowArrayStream`` structures
+from the C Data Interface which add members to allow specifying the device
+type and pass necessary information to synchronize with the producer.
+For non-C/C++ languages and runtimes, translating the C definitions to
+corresponding C FFI declarations should be just as simple as with the
+current C data interface.
+
+Applications and libraries can then use Arrow schemas and Arrow formatted
+memory on non-CPU devices to exchange data just as easily as they do
+now with CPU data. This will enable leaving data on those devices longer
+and avoiding costly copies back and forth between the host and device
+just to leverage new libraries and runtimes.
+
+Goals
+-----
+
+* Expose an ABI-stable interface built on the existing C data interface.
+* Make it easy for third-party projects to implement support with little
+  initial investment.
+* Allow zero-copy sharing of Arrow formatted device memory between
+  independant runtimes and components running in the same process.
+* Avoid the need for one-to-one adaptation layers such as the
+  `CUDA Array Interface`_ for Python processes to pass CUDA data.
+* Enable integration without explicit dependencies (either at compile-time
+  or runtime) on the Arrow software project itself.
+
+The intent is for the Arrow C Device data interface to expand the reach
+of the current C data interface, allowing it to also become the standard
+low-level building block for columnar processing on devices like GPUs or
+FPGAs.
+
+Structure definitions
+=====================
+
+Because this is built on the C data interface, the C Device data interface
+uses the ``ArrowSchema`` and ``ArrowArray`` structures as defined in the
+:ref:`C data interface spec <c-data-interface-struct-defs>`. It then adds the
+following free-standing definitions. Like the rest of the Arrow project,
+they are available under the Apache License 2.0.
+
+.. code-block:: c
+
+    #ifndef ARROW_C_DEVICE_DATA_INTERFACE
+    #define ARROW_C_DEVICE_DATA_INTERFACE
+
+    // Device type for the allocated memory
+    typedef int32_t ArrowDeviceType;
+
+    // CPU device, same as using ArrowArray directly
+    #define ARROW_DEVICE_CPU 1
+    // CUDA GPU Device
+    #define ARROW_DEVICE_CUDA 2
+    // Pinned CUDA CPU memory by cudaMallocHost
+    #define ARROW_DEVICE_CUDA_HOST 3
+    // OpenCL Device
+    #define ARROW_DEVICE_OPENCL 4
+    // Vulkan buffer for next-gen graphics
+    #define ARROW_DEVICE_VULKAN 7
+    // Metal for Apple GPU
+    #define ARROW_DEVICE_METAL 8
+    // Verilog simulator buffer
+    #define ARROW_DEVICE_VPI 9
+    // ROCm GPUs for AMD GPUs
+    #define ARROW_DEVICE_ROCM 10
+    // Pinned ROCm CPU memory allocated by hipMallocHost
+    #define ARROW_DEVICE_ROCM_HOST 11
+    // Reserved for extension
+    //
+    // used to quickly test extension devices, semantics
+    // can differ based on implementation
+    #define ARROW_DEVICE_EXT_DEV 12
+    // CUDA managed/unified memory allocated by cudaMallocManaged
+    #define ARROW_DEVICE_CUDA_MANAGED 13
+    // Unified shared memory allocated on a oneAPI
+    // non-partitioned device.
+    //
+    // A call to the oneAPI runtime is required to determine the
+    // device type, the USM allocation type and the sycl context
+    // that it is bound to.
+    #define ARROW_DEVICE_ONEAPI 14
+    // GPU support for next-gen WebGPU standard
+    #define ARROW_DEVICE_WEBGPU 15
+    // Qualcomm Hexagon DSP
+    #define ARROW_DEVICE_HEXAGON 16
+
+    struct ArrowDeviceArray {
+      struct ArrowArray array;
+      int64_t device_id;
+      ArrowDeviceType device_type;
+      void* sync_event;
+
+      // reserved bytes for future expansion
+      int64_t reserved[3];
+    };
+
+    #endif  // ARROW_C_DEVICE_DATA_INTERFACE
+
+.. note::
+   The canonical guard ``ARROW_C_DEVICE_DATA_INTERFACE`` is meant to avoid
+   duplicate definitions if two projects copy the definitions in their own
+   headers, and a third-party project includes from these two projects. It
+   is therefore important that this guard is kept exactly as-is when these
+   definitions are copied.
+
+ArrowDeviceType
+---------------
+
+The ``ArrowDeviceType`` typedef is used to indicate what type of device the
+provided memory buffers were allocated on. This, in conjunction with the
+``device_id``, should be sufficient to reference the correct data buffers.
+
+We then use macros to define values for different device types. The provided
+macro values are compatible with the widely used `dlpack`_ ``DLDeviceType``
+definition values, using the same value for each as the equivalent
+``kDL<type>`` enum from dlpack.h. The list will be kept in sync with those
+equivalent enum values over time to ensure compatibility, rather than
+potentially diverging. To avoid the Arrow project having to be in the
+position of vetting new hardware devices, new additions should first be
+added to dlpack before we add a corresponding macro here.
+
+To ensure predictability with the ABI, we use macros instead of an ``enum``
+so the storage type is not compiler dependent.
+
+.. c:macro:: ARROW_DEVICE_CPU
+
+    CPU Device, equivalent to just using ``ArrowArray`` directly instead of
+    using ``ArrowDeviceArray``.
+
+.. c:macro:: ARROW_DEVICE_CUDA
+
+    A `CUDA`_ GPU Device. This could represent data allocated either with the
+    runtime library (``cudaMalloc``) or the device driver (``cuMemAlloc``).
+
+.. c:macro:: ARROW_DEVICE_CUDA_HOST
+
+    CPU memory that was pinned and page-locked by CUDA by using
+    ``cudaMallocHost`` or ``cuMemAllocHost``.
+
+.. c:macro:: ARROW_DEVICE_OPENCL
+
+    Data allocated on the device by using the `OpenCL (Open Computing Language)`_
+    framework.
+
+.. c:macro:: ARROW_DEVICE_VULKAN
+
+    Data allocated by the `Vulkan`_ framework and libraries.
+
+.. c:macro:: ARROW_DEVICE_METAL
+
+    Data on Apple GPU devices using the `Metal`_ framework and libraries.
+
+.. c:macro:: ARROW_DEVICE_VPI
+
+    Indicates usage of a Verilog simulator buffer.
+
+.. c:macro:: ARROW_DEVICE_ROCM
+
+    An AMD device using the `ROCm`_ stack.
+
+.. c:macro:: ARROW_DEVICE_ROCM_HOST
+
+    CPU memory pinned and page-locked allocated ``hipMallocHost``.
+
+.. c:macro:: ARROW_DEVICE_EXT_DEV
+
+    This value is an escape-hatch for devices to extend which aren't
+    currently represented otherwise. Producers would need to provide
+    additional information/context specific to the device if using
+    this device type. This is used to quickly test extension devices
+    and semantics can differ based on the implementation.
+
+.. c:macro:: ARROW_DEVICE_CUDA_MANAGED
+
+    CUDA managed/unified memory which is allocated by ``cudaMallocManaged``.
+
+.. c:macro:: ARROW_DEVICE_ONEAPI
+
+    Unified shared memory allocated on an Intel `oneAPI`_ non-partitioned
+    device. A call to the ``oneAPI`` runtime is required to determine
+    the specific device type, the USM allocation type and the sycl context
+    that it is bound to.
+
+.. c:macro:: ARROW_DEVICE_WEBGPU
+
+    GPU support for next-gen WebGPU standards
+
+.. c:macro:: ARROW_DEVICE_HEXAGON
+
+    Data allocated on a Qualcomm Hexagon DSP device.
+
+The ArrowDeviceArray structure
+------------------------------
+
+The ``ArrowDeviceArray`` structure embeds the C data ``ArrowArray`` structure
+and adds additional information necessary for consumers to use the data. It
+has the following fields:
+
+.. c:member:: struct ArrowArray ArrowDeviceArray.array
+
+    The allocated array data. The values in the ``void**`` buffers (along
+    with the buffers of any children) are what is allocated on the device.
+    The buffer values should be device pointers. The rest of the structure
+    should be accessible to the CPU.
+
+    The ``private_data`` and ``release`` callback of this structure should
+    contain any necessary information and structures related to freeing
+    the array according to the device it is allocated on, rather than
+    having a separate release callback and ``private_data`` pointer here.
+
+.. c:member:: int64_t ArrowDeviceArray.device_id
+
+    The device id to identify a specific device if multiple devices of this
+    type are on the system. The semantics of the id will be hardware dependent,
+    but we use an ``int64_t`` to future-proof the id as devices change over time.
+
+.. c:member:: ArrowDeviceType ArrowDeviceArray.device_type
+
+    The type of the device which can access the buffers in the array.
+
+.. c:member:: void* ArrowDeviceArray.sync_event
+
+    Optional. An event-like object to synchronize on if needed.
+
+    Many devices, like GPUs, are primarily asynchronous with respect to
+    CPU processing. As such in order to safely access memory, it is often
+    necessary to have an object to synchronize processing with. Since
+    different devices will use different types to specify this, we use a
+    void* which can be coerced into a pointer to whatever the device
+    appropriate type is.
+
+    If synchronization is not needed, this can be null. If this is non-null
+    then it MUST be used to call the appropriate sync method for the device
+    (e.g. ``cudaStreamWaitEvent`` or ``hipStreamWaitEvent``) before attempting
+    to access the memory in the buffers.
+
+    Expected types to coerce this ``void*`` to depending on the reported
+    device type:
+
+    * CUDA: ``cudaEvent_t*``
+    * ROCm: ``hipEvent_t*``
+    * OpenCL: ``cl_event*``
+    * Vulkan: ``VkEvent*``
+    * Metal: ``MTLEvent*``
+    * OneAPI: ``sycl::event*``
+
+    If an event is provided, then the producer MUST ensure that the event
+    is triggered/recorded at the end of the processing stream once the data
+    is considered available for use.
+
+
+.. c:member:: int64_t ArrowDeviceArray.reserved[3]
+
+    As non-CPU development expands, there may be a need to expand this
+    structure. In order to do so without potentially breaking ABI changes,
+    we reserve 24 bytes at the end of the object. This also has the added
+    benefit of bringing the total size of this structure to exactly 128
+    bytes (a power of 2) on 64-bit systems. These bytes should be zero'd
+    out after allocation in order to ensure safe evolution of the ABI in
+    the future.
+
+.. note::
+    Rather than store the shape / types of the data alongside the
+    ``ArrowDeviceArray``, users should utilize the existing ``ArrowSchema``
+    structure to pass any data type and shape information.
+
+Semantics
+=========
+
+Memory management
+-----------------
+
+The ``ArrowDeviceArray`` structure contains an ``ArrowArray`` object which
+itself has :ref:`specific semantics <c-data-interface-semantics>` for releasing
+memory. The term *"base structure"* below refers to the ``ArrowDeviceArray``
+object that is passed directly between the producer and consumer -- not any
+child structure thereof.
+
+It is intended for the base structure to be stack- or heap-allocated by the
+*consumer*. In this case, the producer API should take a pointer to the
+consumer-allocated structure.
+
+However, any data pointed to by the struct MUST be allocated and maintained
+by the producer. This includes the ``sync_event`` member if it is not null,
+along with any pointers in the ``ArrowArray`` object as usual. Data lifetime
+is managed through the ``release`` callback of the ``ArrowArray`` member.
+
+For an ``ArrowDeviceArray``, the semantics of a released structure and the
+callback semantics are identical to those for
+:ref:`ArrowArray itself <c-data-interface-released>`. Any producer specific context
+information necessary for releasing the device data buffers, in addition to
+any allocated event, should be stored in the ``private_data`` member of
+the ``ArrowArray`` and managed by the ``release`` callback.
+
+Moving an array
+'''''''''''''''
+
+The consumer can *move* the ``ArrowDeviceArray`` structure by bitwise copying
+or shallow member-wise copying. Then it MUST mark the source structure released
+by setting the ``release`` member of the embedded ``ArrowArray`` structure to
+``NULL``, but *without* calling that release callback. This ensures that only
+one live copy of the struct is active at any given time and that lifetime is
+correctly communicated to the producer.
+
+As usual, the release callback will be called on the destination structure
+when it is not needed anymore.
+
+Record batches
+--------------
+As with the C data interface itself, a record batch can be trivially considered
+as an equivalent struct array. In this case the metadata of the top-level
+``ArrowSchema`` can be used for schema-level metadata of the record batch.
+
+Mutability
+----------
+
+Both the producer and the consumer SHOULD consider the exported data (that
+is, the data reachable on the device through the ``buffers`` member of
+the embedded ``ArrowArray``) to be immutable, as either party could otherwise
+see inconsistent data while the other is mutating it.
+
+Likewise, if the ``sync_event`` member is non-NULL, the consumer should not
+attempt to access or read the data until they have synchronized on that event.
+
+C producer example
+====================
+
+Exporting a simple ``int32`` device array
+-----------------------------------------
+
+Export a non-nullable ``int32`` type with empty metadata. An example of this
+can be seen in the :ref:`C data interface docs directly <c-data-interface-export-int32-schema>`.
+
+To export the data itself, we transfer ownership to the consumer through
+the release callback. This example will use CUDA, but the equivalent calls
+could be used for any device:
+
+.. code-block:: c
+
+    static void release_int32_device_array(struct ArrowArray* array) {
+        assert(array->n_buffers == 2);
+        // destroy the event
+        cudaEvent_t* ev_ptr = reinterpret_cast<cudaEvent_t*>(array->private_data);
+        cudaError_t status = cudaEventDestroy(*ev_ptr);
+        assert(status == cudaSuccess);
+        free(ev_ptr);
+
+        // free the buffers and the buffers array
+        status = cudaFree(array->buffers[1]);
+        assert(status == cudaSuccess);
+        free(array->buffers);
+
+        // mark released
+        array->release = NULL;
+    }
+
+    __host__ void export_int32_device_array(void* cudaAllocdPtr,
+                                            cudaStream_t stream,
+                                            int64_t length,
+                                            struct ArrowDeviceArray* array) {
+        // get device id
+        int device;
+        cudaError_t status;
+        status = cudaGetDevice(&device);
+        assert(status == cudaSuccess);
+
+        cudaEvent_t* ev_ptr = (cudaEvent_t*)malloc(sizeof(cudaEvent_t));
+        assert(ev_ptr != NULL);
+        status = cudaEventCreate(ev_ptr);
+        assert(status == cudaSuccess);
+
+        // record event on the stream, assuming that the passed in
+        // stream is where the work to produce the data will be processing.
+        status = cudaEventRecord(*ev_ptr, stream);
+        assert(status == cudaSuccess);
+
+        // initialize fields
+        *array = (struct ArrowDeviceArray) {
+            .array = (struct ArrowArray) {
+                .length = length,
+                .null_count = 0,
+                .offset = 0,
+                .n_buffers = 2,
+                .n_children = 0,
+                .children = NULL,
+                .dictionary = NULL,
+                // bookeeping
+                .release = &release_int32_device_array,
+                .private_data = reinterpret_cast<void*>(ev_ptr),
+            },
+            .device_id = static_cast<int64_t>(device),
+            .device_type = ARROW_DEVICE_CUDA,
+            // pass the event pointer to the consumer
+            .sync_event = reinterpret_cast<void*>(ev_ptr),
+        };
+
+        // allocate list of buffers
+        array->array.buffers = (const void**)malloc(sizeof(void*) * array->array.n_buffers);
+        assert(array->array.buffers != NULL);
+        array->array.buffers[0] = NULL;
+        array->array.buffers[1] = cudaAllocdPtr;
+    }
+
+================
+Device Stream Interface
+================
+
+Like the :ref:`C stream interface <c-stream-interface>`, the C Device data
+interface also specifies a higher-level structure for easing communication
+of streaming data within a single process. Defining an ``ArrowDeviceArrayStream``
+structure.
+
+Semantics
+=========
+
+An Arrow C device stream exposes a streaming source of data chunks, each with
+the same schema. Chunks are obtained by calling a blocking pull-style iteration
+function. It is expected that all chunks should be providing data on the same
+device type (but not necessarily the same device id). If it is necessary
+to provide a stream of data on multiple device types, a producer should
+provide a separate stream object for each device type.
+
+Structure definition
+====================
+
+The C device stream interface is defined by a single ``struct`` definition:
+
+.. code-block:: c
+
+    #ifndef ARROW_C_DEVICE_STREAM_INTERFACE
+    #define ARROW_C_DEVICE_STREAM_INTERFACE
+
+    struct ArrowDeviceArrayStream {
+        // device type that all arrays will be accessible from
+        ArrowDeviceType device_type;
+        // callbacks
+        int (*get_schema)(struct ArrowDeviceArrayStream*, struct ArrowSchema*);
+        int (*get_next)(struct ArrowDeviceArrayStream*, struct ArrowDeviceArray*);
+        const char* (*get_last_error)(struct ArrowDeviceArrayStream*);
+
+        // release callback
+        void (*release)(struct ArrowDeviceArrayStream*);
+
+        // opaque producer-specific data
+        void* private_data;
+    };
+
+    #endif  // ARROW_C_DEVICE_STREAM_INTERFACE
+
+.. note::
+    The canonical guard ``ARROW_C_DEVICE_STREAM_INTERFACE`` is meant to avoid
+    duplicate definitions if two projects copy the C device stream interface
+    definitions into their own headers, and a third-party project includes
+    from these two projects. It is therefore important that this guard is
+    kept exactly as-is when these definitions are copied.
+
+The ArrowDeviceArrayStream structure
+------------------------------------
+
+The ``ArrowDeviceArrayStream`` provides a device type that can access the
+resulting data along with the required callbacks to interact with a
+streaming source of Arrow arrays. It has the following fields:
+
+.. c:member:: ArrowDeviceType device_type
+
+    The device type that this stream produces data on. All
+    ``ArrowDeviceArray``s that are produced by this stream should have the
+    same device type as is set here. This is a convenience for the consumer
+    to not have to check every array that is retrieved and instead allows
+    higher-level coding constructs for streams.
+
+.. c:member:: int (*ArrowDeviceArrayStream.get_schema)(struct ArrowDeviceArrayStream*, struct ArrowSchema* out)
+
+    *Mandatory.* This callback allows the consumer to query the schema of
+    the chunks of data in the stream. The schema is the same for all data
+    chunks.
+
+    This callback must NOT be called on a released ``ArrowDeviceArrayStream``.
+
+    *Return value:* 0 on success, a non-zero
+    :ref:`error code <c-stream-interface-error-codes>` otherwise.
+
+.. c:member:: int (*ArrowDeviceArrayStream.get_next)(struct ArrowDeviceArrayStream*, struct ArrowDeviceArray* out)
+
+    *Mandatory.* This callback allows the consumer to get the next chunk of
+    data in the stream.
+
+    This callback must NOT be called on a released ``ArrowDeviceArrayStream``.
+
+    The next chunk of data MUST be accessible from a device type matching the
+    :c:member:`ArrowDeviceArrayStream.device_type`.
+
+    *Return value:* 0 on success, a non-zero
+    :ref:`error code <c-stream-interface-error-codes>` otherwise.
+
+    On success, the consumer must check whether the ``ArrowDeviceArray``'s
+    embedded ``ArrowArray`` is marked :ref:`released <c-data-interface-released>`.
+    If the embedded ``ArrowDeviceArray.array`` is released, then the end of the
+    stream has been reached. Otherwise, the ``ArrowDeviceArray`` contains a
+    valid data chunk.
+
+.. c:member:: const char* (*ArrowDeviceArrayStream.get_last_error)(struct ArrowDeviceArrayStream*)
+
+    *Mandatory.* This callback allows the consumer to get a textual description
+    of the last error.
+
+    This callback must ONLY be called if the last operation on the
+    ``ArrowDeviceArrayStream`` returned an error. It must NOT be called on a
+    released ``ArrowDeviceArrayStream``.
+
+    *Return value:* a pointer to a NULL-terminated character string
+    (UTF8-encoded). NULL can also be returned if no detailed description is
+    available.
+
+    The returned pointer is only guaranteed to be valid until the next call
+    of one of the stream's callbacks. The character string it points to should
+    be copied to consumer-managed storage if it is intended to survive longer.
+
+.. c:member:: void (*ArrowDeviceArrayStream.release)(struct ArrowDeviceArrayStream*)
+
+    *Mandatory.* A pointer to a producer-provided release callback.
+
+.. c:member:: void* ArrowDeviceArrayStream.private_data
+
+    *Optional.* An opaque pointer to producer-provided private data.
+
+    Consumers MUST NOT process this member. Lifetime of this member is
+    handled by the producer, and especially by the release callback.
+
+Result lifetimes
+----------------
+
+The data returned by the ``get_schema`` and ``get_next`` callbacks must be
+released independantly. Their lifetimes are not tied to that of
+``ArrowDeviceArrayStream``.
+
+Stream lifetime
+---------------
+
+Lifetime of the C stream is managed using a release callback with similar
+usage as in :ref:`C data interface <c-data-interface-released>`.
+
+Thread safety
+-------------
+
+The stream source is not assumed to be thread-safe. Consumers wanting to
+call ``get_next`` from several threads should ensure those calls are
+serialized.
+
+Updating this specification
+===========================
+
+.. note::
+    Since this specification is still considered experimental, there is the
+    (still very low) possibility it might change slightly. Once it is
+    supported in an official Arrow release and the "experimental" tag is
+    removed from it, this section will apply and the ABI will be frozen.
+    
+    The reason for the "experimental" tag is because we don't know what we
+    don't know. While it was attempted to ensure this is generic enough to
+    work with a multitude of different frameworks, it's also possible that
+    something was missed. Once there is some usage of this and we are
+    confident there isn't any necessary modifications, the "experimental"
+    tag will be removed and the ABI frozen.

Review Comment:
   There is some repetition here. Consolidate and make this note a bit shorter?



##########
docs/source/format/CDeviceDataInterface.rst:
##########
@@ -0,0 +1,628 @@
+.. Licensed to the Apache Software Foundation (ASF) under one
+.. or more contributor license agreements.  See the NOTICE file
+.. distributed with this work for additional information
+.. regarding copyright ownership.  The ASF licenses this file
+.. to you under the Apache License, Version 2.0 (the
+.. "License"); you may not use this file except in compliance
+.. with the License.  You may obtain a copy of the License at
+
+..   http://www.apache.org/licenses/LICENSE-2.0
+
+.. Unless required by applicable law or agreed to in writing,
+.. software distributed under the License is distributed on an
+.. "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+.. KIND, either express or implied.  See the License for the
+.. specific language governing permissions and limitations
+.. under the License.
+
+.. highlight:: c
+
+.. _c-device-data-interface:
+
+=================================
+The Arrow C Device data interface
+=================================
+
+.. note:: The Arrow C Device Data Interface should be considered experimental
+
+Rationale
+=========
+
+The current :ref:`C Data Interface <c-data-interface>`, and most
+implementations of it, make the assumption that all data buffers provided
+are CPU buffers. Since Apache Arrow is designed to be a universal in-memory
+format for representing tabular ("columnar") data, there will be the desire
+to leverage this data on non-CPU hardware such as GPUs. One example of such
+a case is the `RAPIDS cuDF library`_ which uses the Arrow memory format with
+CUDA for NVIDIA GPUs. Since copying data from host to device and back is
+expensive, the ideal would be to be able to leave the data on the device
+for as long as possible, even when passing it between runtimes and
+libraries.
+
+The Arrow C Device data interface builds on the existing C data interface
+by adding a very small, stable set of C definitions to it. These definitions
+are equivalents to the ``ArrowArray`` and ``ArrowArrayStream`` structures
+from the C Data Interface which add members to allow specifying the device
+type and pass necessary information to synchronize with the producer.
+For non-C/C++ languages and runtimes, translating the C definitions to
+corresponding C FFI declarations should be just as simple as with the
+current C data interface.
+
+Applications and libraries can then use Arrow schemas and Arrow formatted
+memory on non-CPU devices to exchange data just as easily as they do
+now with CPU data. This will enable leaving data on those devices longer
+and avoiding costly copies back and forth between the host and device
+just to leverage new libraries and runtimes.
+
+Goals
+-----
+
+* Expose an ABI-stable interface built on the existing C data interface.
+* Make it easy for third-party projects to implement support with little
+  initial investment.
+* Allow zero-copy sharing of Arrow formatted device memory between
+  independant runtimes and components running in the same process.
+* Avoid the need for one-to-one adaptation layers such as the
+  `CUDA Array Interface`_ for Python processes to pass CUDA data.
+* Enable integration without explicit dependencies (either at compile-time
+  or runtime) on the Arrow software project itself.
+
+The intent is for the Arrow C Device data interface to expand the reach
+of the current C data interface, allowing it to also become the standard
+low-level building block for columnar processing on devices like GPUs or
+FPGAs.
+
+Structure definitions
+=====================
+
+Because this is built on the C data interface, the C Device data interface
+uses the ``ArrowSchema`` and ``ArrowArray`` structures as defined in the
+:ref:`C data interface spec <c-data-interface-struct-defs>`. It then adds the
+following free-standing definitions. Like the rest of the Arrow project,
+they are available under the Apache License 2.0.
+
+.. code-block:: c
+
+    #ifndef ARROW_C_DEVICE_DATA_INTERFACE
+    #define ARROW_C_DEVICE_DATA_INTERFACE
+
+    // Device type for the allocated memory
+    typedef int32_t ArrowDeviceType;
+
+    // CPU device, same as using ArrowArray directly
+    #define ARROW_DEVICE_CPU 1
+    // CUDA GPU Device
+    #define ARROW_DEVICE_CUDA 2
+    // Pinned CUDA CPU memory by cudaMallocHost
+    #define ARROW_DEVICE_CUDA_HOST 3
+    // OpenCL Device
+    #define ARROW_DEVICE_OPENCL 4
+    // Vulkan buffer for next-gen graphics
+    #define ARROW_DEVICE_VULKAN 7
+    // Metal for Apple GPU
+    #define ARROW_DEVICE_METAL 8
+    // Verilog simulator buffer
+    #define ARROW_DEVICE_VPI 9
+    // ROCm GPUs for AMD GPUs
+    #define ARROW_DEVICE_ROCM 10
+    // Pinned ROCm CPU memory allocated by hipMallocHost
+    #define ARROW_DEVICE_ROCM_HOST 11
+    // Reserved for extension
+    //
+    // used to quickly test extension devices, semantics
+    // can differ based on implementation
+    #define ARROW_DEVICE_EXT_DEV 12
+    // CUDA managed/unified memory allocated by cudaMallocManaged
+    #define ARROW_DEVICE_CUDA_MANAGED 13
+    // Unified shared memory allocated on a oneAPI
+    // non-partitioned device.
+    //
+    // A call to the oneAPI runtime is required to determine the
+    // device type, the USM allocation type and the sycl context
+    // that it is bound to.
+    #define ARROW_DEVICE_ONEAPI 14
+    // GPU support for next-gen WebGPU standard
+    #define ARROW_DEVICE_WEBGPU 15
+    // Qualcomm Hexagon DSP
+    #define ARROW_DEVICE_HEXAGON 16
+
+    struct ArrowDeviceArray {
+      struct ArrowArray array;
+      int64_t device_id;
+      ArrowDeviceType device_type;
+      void* sync_event;
+
+      // reserved bytes for future expansion
+      int64_t reserved[3];
+    };
+
+    #endif  // ARROW_C_DEVICE_DATA_INTERFACE
+
+.. note::
+   The canonical guard ``ARROW_C_DEVICE_DATA_INTERFACE`` is meant to avoid
+   duplicate definitions if two projects copy the definitions in their own
+   headers, and a third-party project includes from these two projects. It
+   is therefore important that this guard is kept exactly as-is when these
+   definitions are copied.
+
+ArrowDeviceType
+---------------
+
+The ``ArrowDeviceType`` typedef is used to indicate what type of device the
+provided memory buffers were allocated on. This, in conjunction with the
+``device_id``, should be sufficient to reference the correct data buffers.
+
+We then use macros to define values for different device types. The provided
+macro values are compatible with the widely used `dlpack`_ ``DLDeviceType``
+definition values, using the same value for each as the equivalent
+``kDL<type>`` enum from dlpack.h. The list will be kept in sync with those
+equivalent enum values over time to ensure compatibility, rather than
+potentially diverging. To avoid the Arrow project having to be in the
+position of vetting new hardware devices, new additions should first be
+added to dlpack before we add a corresponding macro here.
+
+To ensure predictability with the ABI, we use macros instead of an ``enum``
+so the storage type is not compiler dependent.
+
+.. c:macro:: ARROW_DEVICE_CPU
+
+    CPU Device, equivalent to just using ``ArrowArray`` directly instead of
+    using ``ArrowDeviceArray``.
+
+.. c:macro:: ARROW_DEVICE_CUDA
+
+    A `CUDA`_ GPU Device. This could represent data allocated either with the
+    runtime library (``cudaMalloc``) or the device driver (``cuMemAlloc``).
+
+.. c:macro:: ARROW_DEVICE_CUDA_HOST
+
+    CPU memory that was pinned and page-locked by CUDA by using
+    ``cudaMallocHost`` or ``cuMemAllocHost``.
+
+.. c:macro:: ARROW_DEVICE_OPENCL
+
+    Data allocated on the device by using the `OpenCL (Open Computing Language)`_
+    framework.
+
+.. c:macro:: ARROW_DEVICE_VULKAN
+
+    Data allocated by the `Vulkan`_ framework and libraries.
+
+.. c:macro:: ARROW_DEVICE_METAL
+
+    Data on Apple GPU devices using the `Metal`_ framework and libraries.
+
+.. c:macro:: ARROW_DEVICE_VPI
+
+    Indicates usage of a Verilog simulator buffer.
+
+.. c:macro:: ARROW_DEVICE_ROCM
+
+    An AMD device using the `ROCm`_ stack.
+
+.. c:macro:: ARROW_DEVICE_ROCM_HOST
+
+    CPU memory pinned and page-locked allocated ``hipMallocHost``.
+
+.. c:macro:: ARROW_DEVICE_EXT_DEV
+
+    This value is an escape-hatch for devices to extend which aren't
+    currently represented otherwise. Producers would need to provide
+    additional information/context specific to the device if using
+    this device type. This is used to quickly test extension devices
+    and semantics can differ based on the implementation.
+
+.. c:macro:: ARROW_DEVICE_CUDA_MANAGED
+
+    CUDA managed/unified memory which is allocated by ``cudaMallocManaged``.
+
+.. c:macro:: ARROW_DEVICE_ONEAPI
+
+    Unified shared memory allocated on an Intel `oneAPI`_ non-partitioned
+    device. A call to the ``oneAPI`` runtime is required to determine
+    the specific device type, the USM allocation type and the sycl context
+    that it is bound to.
+
+.. c:macro:: ARROW_DEVICE_WEBGPU
+
+    GPU support for next-gen WebGPU standards
+
+.. c:macro:: ARROW_DEVICE_HEXAGON
+
+    Data allocated on a Qualcomm Hexagon DSP device.
+
+The ArrowDeviceArray structure
+------------------------------
+
+The ``ArrowDeviceArray`` structure embeds the C data ``ArrowArray`` structure
+and adds additional information necessary for consumers to use the data. It
+has the following fields:
+
+.. c:member:: struct ArrowArray ArrowDeviceArray.array
+
+    The allocated array data. The values in the ``void**`` buffers (along
+    with the buffers of any children) are what is allocated on the device.
+    The buffer values should be device pointers. The rest of the structure
+    should be accessible to the CPU.
+
+    The ``private_data`` and ``release`` callback of this structure should
+    contain any necessary information and structures related to freeing
+    the array according to the device it is allocated on, rather than
+    having a separate release callback and ``private_data`` pointer here.
+
+.. c:member:: int64_t ArrowDeviceArray.device_id
+
+    The device id to identify a specific device if multiple devices of this
+    type are on the system. The semantics of the id will be hardware dependent,
+    but we use an ``int64_t`` to future-proof the id as devices change over time.
+
+.. c:member:: ArrowDeviceType ArrowDeviceArray.device_type
+
+    The type of the device which can access the buffers in the array.
+
+.. c:member:: void* ArrowDeviceArray.sync_event
+
+    Optional. An event-like object to synchronize on if needed.
+
+    Many devices, like GPUs, are primarily asynchronous with respect to
+    CPU processing. As such in order to safely access memory, it is often
+    necessary to have an object to synchronize processing with. Since
+    different devices will use different types to specify this, we use a
+    void* which can be coerced into a pointer to whatever the device
+    appropriate type is.
+
+    If synchronization is not needed, this can be null. If this is non-null
+    then it MUST be used to call the appropriate sync method for the device
+    (e.g. ``cudaStreamWaitEvent`` or ``hipStreamWaitEvent``) before attempting
+    to access the memory in the buffers.
+
+    Expected types to coerce this ``void*`` to depending on the reported
+    device type:
+
+    * CUDA: ``cudaEvent_t*``
+    * ROCm: ``hipEvent_t*``
+    * OpenCL: ``cl_event*``
+    * Vulkan: ``VkEvent*``
+    * Metal: ``MTLEvent*``
+    * OneAPI: ``sycl::event*``
+
+    If an event is provided, then the producer MUST ensure that the event
+    is triggered/recorded at the end of the processing stream once the data
+    is considered available for use.
+
+
+.. c:member:: int64_t ArrowDeviceArray.reserved[3]
+
+    As non-CPU development expands, there may be a need to expand this
+    structure. In order to do so without potentially breaking ABI changes,
+    we reserve 24 bytes at the end of the object. This also has the added
+    benefit of bringing the total size of this structure to exactly 128
+    bytes (a power of 2) on 64-bit systems. These bytes should be zero'd
+    out after allocation in order to ensure safe evolution of the ABI in
+    the future.

Review Comment:
   I'm still skeptical about this, but if we want to keep this, replace "should" with "MUST"?



##########
docs/source/format/CDeviceDataInterface.rst:
##########
@@ -0,0 +1,628 @@
+.. Licensed to the Apache Software Foundation (ASF) under one
+.. or more contributor license agreements.  See the NOTICE file
+.. distributed with this work for additional information
+.. regarding copyright ownership.  The ASF licenses this file
+.. to you under the Apache License, Version 2.0 (the
+.. "License"); you may not use this file except in compliance
+.. with the License.  You may obtain a copy of the License at
+
+..   http://www.apache.org/licenses/LICENSE-2.0
+
+.. Unless required by applicable law or agreed to in writing,
+.. software distributed under the License is distributed on an
+.. "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+.. KIND, either express or implied.  See the License for the
+.. specific language governing permissions and limitations
+.. under the License.
+
+.. highlight:: c
+
+.. _c-device-data-interface:
+
+=================================
+The Arrow C Device data interface
+=================================
+
+.. note:: The Arrow C Device Data Interface should be considered experimental

Review Comment:
   ```suggestion
   .. warning:: The Arrow C Device Data Interface should be considered experimental
   ```



##########
docs/source/format/CDeviceDataInterface.rst:
##########
@@ -0,0 +1,628 @@
+.. Licensed to the Apache Software Foundation (ASF) under one
+.. or more contributor license agreements.  See the NOTICE file
+.. distributed with this work for additional information
+.. regarding copyright ownership.  The ASF licenses this file
+.. to you under the Apache License, Version 2.0 (the
+.. "License"); you may not use this file except in compliance
+.. with the License.  You may obtain a copy of the License at
+
+..   http://www.apache.org/licenses/LICENSE-2.0
+
+.. Unless required by applicable law or agreed to in writing,
+.. software distributed under the License is distributed on an
+.. "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+.. KIND, either express or implied.  See the License for the
+.. specific language governing permissions and limitations
+.. under the License.
+
+.. highlight:: c
+
+.. _c-device-data-interface:
+
+=================================
+The Arrow C Device data interface
+=================================
+
+.. note:: The Arrow C Device Data Interface should be considered experimental
+
+Rationale
+=========
+
+The current :ref:`C Data Interface <c-data-interface>`, and most
+implementations of it, make the assumption that all data buffers provided
+are CPU buffers. Since Apache Arrow is designed to be a universal in-memory
+format for representing tabular ("columnar") data, there will be the desire
+to leverage this data on non-CPU hardware such as GPUs. One example of such
+a case is the `RAPIDS cuDF library`_ which uses the Arrow memory format with
+CUDA for NVIDIA GPUs. Since copying data from host to device and back is
+expensive, the ideal would be to be able to leave the data on the device
+for as long as possible, even when passing it between runtimes and
+libraries.
+
+The Arrow C Device data interface builds on the existing C data interface
+by adding a very small, stable set of C definitions to it. These definitions
+are equivalents to the ``ArrowArray`` and ``ArrowArrayStream`` structures
+from the C Data Interface which add members to allow specifying the device
+type and pass necessary information to synchronize with the producer.
+For non-C/C++ languages and runtimes, translating the C definitions to
+corresponding C FFI declarations should be just as simple as with the
+current C data interface.
+
+Applications and libraries can then use Arrow schemas and Arrow formatted
+memory on non-CPU devices to exchange data just as easily as they do
+now with CPU data. This will enable leaving data on those devices longer
+and avoiding costly copies back and forth between the host and device
+just to leverage new libraries and runtimes.
+
+Goals
+-----
+
+* Expose an ABI-stable interface built on the existing C data interface.
+* Make it easy for third-party projects to implement support with little
+  initial investment.
+* Allow zero-copy sharing of Arrow formatted device memory between
+  independant runtimes and components running in the same process.
+* Avoid the need for one-to-one adaptation layers such as the
+  `CUDA Array Interface`_ for Python processes to pass CUDA data.
+* Enable integration without explicit dependencies (either at compile-time
+  or runtime) on the Arrow software project itself.
+
+The intent is for the Arrow C Device data interface to expand the reach
+of the current C data interface, allowing it to also become the standard
+low-level building block for columnar processing on devices like GPUs or
+FPGAs.
+
+Structure definitions
+=====================
+
+Because this is built on the C data interface, the C Device data interface
+uses the ``ArrowSchema`` and ``ArrowArray`` structures as defined in the
+:ref:`C data interface spec <c-data-interface-struct-defs>`. It then adds the
+following free-standing definitions. Like the rest of the Arrow project,
+they are available under the Apache License 2.0.
+
+.. code-block:: c
+
+    #ifndef ARROW_C_DEVICE_DATA_INTERFACE
+    #define ARROW_C_DEVICE_DATA_INTERFACE
+
+    // Device type for the allocated memory
+    typedef int32_t ArrowDeviceType;
+
+    // CPU device, same as using ArrowArray directly
+    #define ARROW_DEVICE_CPU 1
+    // CUDA GPU Device
+    #define ARROW_DEVICE_CUDA 2
+    // Pinned CUDA CPU memory by cudaMallocHost
+    #define ARROW_DEVICE_CUDA_HOST 3
+    // OpenCL Device
+    #define ARROW_DEVICE_OPENCL 4
+    // Vulkan buffer for next-gen graphics
+    #define ARROW_DEVICE_VULKAN 7
+    // Metal for Apple GPU
+    #define ARROW_DEVICE_METAL 8
+    // Verilog simulator buffer
+    #define ARROW_DEVICE_VPI 9
+    // ROCm GPUs for AMD GPUs
+    #define ARROW_DEVICE_ROCM 10
+    // Pinned ROCm CPU memory allocated by hipMallocHost
+    #define ARROW_DEVICE_ROCM_HOST 11
+    // Reserved for extension
+    //
+    // used to quickly test extension devices, semantics
+    // can differ based on implementation
+    #define ARROW_DEVICE_EXT_DEV 12
+    // CUDA managed/unified memory allocated by cudaMallocManaged
+    #define ARROW_DEVICE_CUDA_MANAGED 13
+    // Unified shared memory allocated on a oneAPI
+    // non-partitioned device.
+    //
+    // A call to the oneAPI runtime is required to determine the
+    // device type, the USM allocation type and the sycl context
+    // that it is bound to.
+    #define ARROW_DEVICE_ONEAPI 14
+    // GPU support for next-gen WebGPU standard
+    #define ARROW_DEVICE_WEBGPU 15
+    // Qualcomm Hexagon DSP
+    #define ARROW_DEVICE_HEXAGON 16
+
+    struct ArrowDeviceArray {
+      struct ArrowArray array;
+      int64_t device_id;
+      ArrowDeviceType device_type;
+      void* sync_event;
+
+      // reserved bytes for future expansion
+      int64_t reserved[3];
+    };
+
+    #endif  // ARROW_C_DEVICE_DATA_INTERFACE
+
+.. note::
+   The canonical guard ``ARROW_C_DEVICE_DATA_INTERFACE`` is meant to avoid
+   duplicate definitions if two projects copy the definitions in their own
+   headers, and a third-party project includes from these two projects. It
+   is therefore important that this guard is kept exactly as-is when these
+   definitions are copied.
+
+ArrowDeviceType
+---------------
+
+The ``ArrowDeviceType`` typedef is used to indicate what type of device the
+provided memory buffers were allocated on. This, in conjunction with the
+``device_id``, should be sufficient to reference the correct data buffers.
+
+We then use macros to define values for different device types. The provided
+macro values are compatible with the widely used `dlpack`_ ``DLDeviceType``
+definition values, using the same value for each as the equivalent
+``kDL<type>`` enum from dlpack.h. The list will be kept in sync with those
+equivalent enum values over time to ensure compatibility, rather than
+potentially diverging. To avoid the Arrow project having to be in the
+position of vetting new hardware devices, new additions should first be
+added to dlpack before we add a corresponding macro here.
+
+To ensure predictability with the ABI, we use macros instead of an ``enum``
+so the storage type is not compiler dependent.
+
+.. c:macro:: ARROW_DEVICE_CPU
+
+    CPU Device, equivalent to just using ``ArrowArray`` directly instead of
+    using ``ArrowDeviceArray``.
+
+.. c:macro:: ARROW_DEVICE_CUDA
+
+    A `CUDA`_ GPU Device. This could represent data allocated either with the
+    runtime library (``cudaMalloc``) or the device driver (``cuMemAlloc``).
+
+.. c:macro:: ARROW_DEVICE_CUDA_HOST
+
+    CPU memory that was pinned and page-locked by CUDA by using
+    ``cudaMallocHost`` or ``cuMemAllocHost``.
+
+.. c:macro:: ARROW_DEVICE_OPENCL
+
+    Data allocated on the device by using the `OpenCL (Open Computing Language)`_
+    framework.
+
+.. c:macro:: ARROW_DEVICE_VULKAN
+
+    Data allocated by the `Vulkan`_ framework and libraries.
+
+.. c:macro:: ARROW_DEVICE_METAL
+
+    Data on Apple GPU devices using the `Metal`_ framework and libraries.
+
+.. c:macro:: ARROW_DEVICE_VPI
+
+    Indicates usage of a Verilog simulator buffer.
+
+.. c:macro:: ARROW_DEVICE_ROCM
+
+    An AMD device using the `ROCm`_ stack.
+
+.. c:macro:: ARROW_DEVICE_ROCM_HOST
+
+    CPU memory pinned and page-locked allocated ``hipMallocHost``.
+
+.. c:macro:: ARROW_DEVICE_EXT_DEV
+
+    This value is an escape-hatch for devices to extend which aren't
+    currently represented otherwise. Producers would need to provide
+    additional information/context specific to the device if using
+    this device type. This is used to quickly test extension devices
+    and semantics can differ based on the implementation.
+
+.. c:macro:: ARROW_DEVICE_CUDA_MANAGED
+
+    CUDA managed/unified memory which is allocated by ``cudaMallocManaged``.
+
+.. c:macro:: ARROW_DEVICE_ONEAPI
+
+    Unified shared memory allocated on an Intel `oneAPI`_ non-partitioned
+    device. A call to the ``oneAPI`` runtime is required to determine
+    the specific device type, the USM allocation type and the sycl context
+    that it is bound to.
+
+.. c:macro:: ARROW_DEVICE_WEBGPU
+
+    GPU support for next-gen WebGPU standards
+
+.. c:macro:: ARROW_DEVICE_HEXAGON
+
+    Data allocated on a Qualcomm Hexagon DSP device.
+
+The ArrowDeviceArray structure
+------------------------------
+
+The ``ArrowDeviceArray`` structure embeds the C data ``ArrowArray`` structure
+and adds additional information necessary for consumers to use the data. It
+has the following fields:
+
+.. c:member:: struct ArrowArray ArrowDeviceArray.array
+
+    The allocated array data. The values in the ``void**`` buffers (along
+    with the buffers of any children) are what is allocated on the device.
+    The buffer values should be device pointers. The rest of the structure
+    should be accessible to the CPU.
+
+    The ``private_data`` and ``release`` callback of this structure should
+    contain any necessary information and structures related to freeing
+    the array according to the device it is allocated on, rather than
+    having a separate release callback and ``private_data`` pointer here.
+
+.. c:member:: int64_t ArrowDeviceArray.device_id
+
+    The device id to identify a specific device if multiple devices of this
+    type are on the system. The semantics of the id will be hardware dependent,
+    but we use an ``int64_t`` to future-proof the id as devices change over time.
+
+.. c:member:: ArrowDeviceType ArrowDeviceArray.device_type
+
+    The type of the device which can access the buffers in the array.
+
+.. c:member:: void* ArrowDeviceArray.sync_event
+
+    Optional. An event-like object to synchronize on if needed.
+
+    Many devices, like GPUs, are primarily asynchronous with respect to
+    CPU processing. As such in order to safely access memory, it is often
+    necessary to have an object to synchronize processing with. Since
+    different devices will use different types to specify this, we use a
+    void* which can be coerced into a pointer to whatever the device
+    appropriate type is.
+
+    If synchronization is not needed, this can be null. If this is non-null
+    then it MUST be used to call the appropriate sync method for the device
+    (e.g. ``cudaStreamWaitEvent`` or ``hipStreamWaitEvent``) before attempting
+    to access the memory in the buffers.
+
+    Expected types to coerce this ``void*`` to depending on the reported
+    device type:
+
+    * CUDA: ``cudaEvent_t*``
+    * ROCm: ``hipEvent_t*``
+    * OpenCL: ``cl_event*``
+    * Vulkan: ``VkEvent*``
+    * Metal: ``MTLEvent*``
+    * OneAPI: ``sycl::event*``
+
+    If an event is provided, then the producer MUST ensure that the event
+    is triggered/recorded at the end of the processing stream once the data
+    is considered available for use.
+
+
+.. c:member:: int64_t ArrowDeviceArray.reserved[3]
+
+    As non-CPU development expands, there may be a need to expand this
+    structure. In order to do so without potentially breaking ABI changes,
+    we reserve 24 bytes at the end of the object. This also has the added
+    benefit of bringing the total size of this structure to exactly 128
+    bytes (a power of 2) on 64-bit systems. These bytes should be zero'd
+    out after allocation in order to ensure safe evolution of the ABI in
+    the future.
+
+.. note::
+    Rather than store the shape / types of the data alongside the

Review Comment:
   Not sure this note is useful?



##########
docs/source/format/CDeviceDataInterface.rst:
##########
@@ -0,0 +1,628 @@
+.. Licensed to the Apache Software Foundation (ASF) under one
+.. or more contributor license agreements.  See the NOTICE file
+.. distributed with this work for additional information
+.. regarding copyright ownership.  The ASF licenses this file
+.. to you under the Apache License, Version 2.0 (the
+.. "License"); you may not use this file except in compliance
+.. with the License.  You may obtain a copy of the License at
+
+..   http://www.apache.org/licenses/LICENSE-2.0
+
+.. Unless required by applicable law or agreed to in writing,
+.. software distributed under the License is distributed on an
+.. "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+.. KIND, either express or implied.  See the License for the
+.. specific language governing permissions and limitations
+.. under the License.
+
+.. highlight:: c
+
+.. _c-device-data-interface:
+
+=================================
+The Arrow C Device data interface
+=================================
+
+.. note:: The Arrow C Device Data Interface should be considered experimental
+
+Rationale
+=========
+
+The current :ref:`C Data Interface <c-data-interface>`, and most
+implementations of it, make the assumption that all data buffers provided
+are CPU buffers. Since Apache Arrow is designed to be a universal in-memory
+format for representing tabular ("columnar") data, there will be the desire
+to leverage this data on non-CPU hardware such as GPUs. One example of such
+a case is the `RAPIDS cuDF library`_ which uses the Arrow memory format with
+CUDA for NVIDIA GPUs. Since copying data from host to device and back is
+expensive, the ideal would be to be able to leave the data on the device
+for as long as possible, even when passing it between runtimes and
+libraries.
+
+The Arrow C Device data interface builds on the existing C data interface
+by adding a very small, stable set of C definitions to it. These definitions
+are equivalents to the ``ArrowArray`` and ``ArrowArrayStream`` structures
+from the C Data Interface which add members to allow specifying the device
+type and pass necessary information to synchronize with the producer.
+For non-C/C++ languages and runtimes, translating the C definitions to
+corresponding C FFI declarations should be just as simple as with the
+current C data interface.
+
+Applications and libraries can then use Arrow schemas and Arrow formatted
+memory on non-CPU devices to exchange data just as easily as they do
+now with CPU data. This will enable leaving data on those devices longer
+and avoiding costly copies back and forth between the host and device
+just to leverage new libraries and runtimes.
+
+Goals
+-----
+
+* Expose an ABI-stable interface built on the existing C data interface.
+* Make it easy for third-party projects to implement support with little
+  initial investment.
+* Allow zero-copy sharing of Arrow formatted device memory between
+  independant runtimes and components running in the same process.
+* Avoid the need for one-to-one adaptation layers such as the
+  `CUDA Array Interface`_ for Python processes to pass CUDA data.
+* Enable integration without explicit dependencies (either at compile-time
+  or runtime) on the Arrow software project itself.
+
+The intent is for the Arrow C Device data interface to expand the reach
+of the current C data interface, allowing it to also become the standard
+low-level building block for columnar processing on devices like GPUs or
+FPGAs.
+
+Structure definitions
+=====================
+
+Because this is built on the C data interface, the C Device data interface
+uses the ``ArrowSchema`` and ``ArrowArray`` structures as defined in the
+:ref:`C data interface spec <c-data-interface-struct-defs>`. It then adds the
+following free-standing definitions. Like the rest of the Arrow project,
+they are available under the Apache License 2.0.
+
+.. code-block:: c
+
+    #ifndef ARROW_C_DEVICE_DATA_INTERFACE
+    #define ARROW_C_DEVICE_DATA_INTERFACE
+
+    // Device type for the allocated memory
+    typedef int32_t ArrowDeviceType;
+
+    // CPU device, same as using ArrowArray directly
+    #define ARROW_DEVICE_CPU 1
+    // CUDA GPU Device
+    #define ARROW_DEVICE_CUDA 2
+    // Pinned CUDA CPU memory by cudaMallocHost
+    #define ARROW_DEVICE_CUDA_HOST 3
+    // OpenCL Device
+    #define ARROW_DEVICE_OPENCL 4
+    // Vulkan buffer for next-gen graphics
+    #define ARROW_DEVICE_VULKAN 7
+    // Metal for Apple GPU
+    #define ARROW_DEVICE_METAL 8
+    // Verilog simulator buffer
+    #define ARROW_DEVICE_VPI 9
+    // ROCm GPUs for AMD GPUs
+    #define ARROW_DEVICE_ROCM 10
+    // Pinned ROCm CPU memory allocated by hipMallocHost
+    #define ARROW_DEVICE_ROCM_HOST 11
+    // Reserved for extension
+    //
+    // used to quickly test extension devices, semantics
+    // can differ based on implementation
+    #define ARROW_DEVICE_EXT_DEV 12
+    // CUDA managed/unified memory allocated by cudaMallocManaged
+    #define ARROW_DEVICE_CUDA_MANAGED 13
+    // Unified shared memory allocated on a oneAPI
+    // non-partitioned device.
+    //
+    // A call to the oneAPI runtime is required to determine the
+    // device type, the USM allocation type and the sycl context
+    // that it is bound to.
+    #define ARROW_DEVICE_ONEAPI 14
+    // GPU support for next-gen WebGPU standard
+    #define ARROW_DEVICE_WEBGPU 15
+    // Qualcomm Hexagon DSP
+    #define ARROW_DEVICE_HEXAGON 16
+
+    struct ArrowDeviceArray {
+      struct ArrowArray array;
+      int64_t device_id;
+      ArrowDeviceType device_type;
+      void* sync_event;
+
+      // reserved bytes for future expansion
+      int64_t reserved[3];
+    };
+
+    #endif  // ARROW_C_DEVICE_DATA_INTERFACE
+
+.. note::
+   The canonical guard ``ARROW_C_DEVICE_DATA_INTERFACE`` is meant to avoid
+   duplicate definitions if two projects copy the definitions in their own
+   headers, and a third-party project includes from these two projects. It
+   is therefore important that this guard is kept exactly as-is when these
+   definitions are copied.
+
+ArrowDeviceType
+---------------
+
+The ``ArrowDeviceType`` typedef is used to indicate what type of device the
+provided memory buffers were allocated on. This, in conjunction with the
+``device_id``, should be sufficient to reference the correct data buffers.
+
+We then use macros to define values for different device types. The provided
+macro values are compatible with the widely used `dlpack`_ ``DLDeviceType``
+definition values, using the same value for each as the equivalent
+``kDL<type>`` enum from dlpack.h. The list will be kept in sync with those
+equivalent enum values over time to ensure compatibility, rather than
+potentially diverging. To avoid the Arrow project having to be in the
+position of vetting new hardware devices, new additions should first be
+added to dlpack before we add a corresponding macro here.
+
+To ensure predictability with the ABI, we use macros instead of an ``enum``
+so the storage type is not compiler dependent.
+
+.. c:macro:: ARROW_DEVICE_CPU
+
+    CPU Device, equivalent to just using ``ArrowArray`` directly instead of
+    using ``ArrowDeviceArray``.
+
+.. c:macro:: ARROW_DEVICE_CUDA
+
+    A `CUDA`_ GPU Device. This could represent data allocated either with the
+    runtime library (``cudaMalloc``) or the device driver (``cuMemAlloc``).
+
+.. c:macro:: ARROW_DEVICE_CUDA_HOST
+
+    CPU memory that was pinned and page-locked by CUDA by using
+    ``cudaMallocHost`` or ``cuMemAllocHost``.
+
+.. c:macro:: ARROW_DEVICE_OPENCL
+
+    Data allocated on the device by using the `OpenCL (Open Computing Language)`_
+    framework.
+
+.. c:macro:: ARROW_DEVICE_VULKAN
+
+    Data allocated by the `Vulkan`_ framework and libraries.
+
+.. c:macro:: ARROW_DEVICE_METAL
+
+    Data on Apple GPU devices using the `Metal`_ framework and libraries.
+
+.. c:macro:: ARROW_DEVICE_VPI
+
+    Indicates usage of a Verilog simulator buffer.
+
+.. c:macro:: ARROW_DEVICE_ROCM
+
+    An AMD device using the `ROCm`_ stack.
+
+.. c:macro:: ARROW_DEVICE_ROCM_HOST
+
+    CPU memory pinned and page-locked allocated ``hipMallocHost``.
+
+.. c:macro:: ARROW_DEVICE_EXT_DEV
+
+    This value is an escape-hatch for devices to extend which aren't
+    currently represented otherwise. Producers would need to provide
+    additional information/context specific to the device if using
+    this device type. This is used to quickly test extension devices
+    and semantics can differ based on the implementation.
+
+.. c:macro:: ARROW_DEVICE_CUDA_MANAGED
+
+    CUDA managed/unified memory which is allocated by ``cudaMallocManaged``.
+
+.. c:macro:: ARROW_DEVICE_ONEAPI
+
+    Unified shared memory allocated on an Intel `oneAPI`_ non-partitioned
+    device. A call to the ``oneAPI`` runtime is required to determine
+    the specific device type, the USM allocation type and the sycl context
+    that it is bound to.
+
+.. c:macro:: ARROW_DEVICE_WEBGPU
+
+    GPU support for next-gen WebGPU standards
+
+.. c:macro:: ARROW_DEVICE_HEXAGON
+
+    Data allocated on a Qualcomm Hexagon DSP device.
+
+The ArrowDeviceArray structure
+------------------------------
+
+The ``ArrowDeviceArray`` structure embeds the C data ``ArrowArray`` structure
+and adds additional information necessary for consumers to use the data. It
+has the following fields:
+
+.. c:member:: struct ArrowArray ArrowDeviceArray.array
+
+    The allocated array data. The values in the ``void**`` buffers (along
+    with the buffers of any children) are what is allocated on the device.
+    The buffer values should be device pointers. The rest of the structure
+    should be accessible to the CPU.
+
+    The ``private_data`` and ``release`` callback of this structure should
+    contain any necessary information and structures related to freeing
+    the array according to the device it is allocated on, rather than
+    having a separate release callback and ``private_data`` pointer here.
+
+.. c:member:: int64_t ArrowDeviceArray.device_id
+
+    The device id to identify a specific device if multiple devices of this
+    type are on the system. The semantics of the id will be hardware dependent,
+    but we use an ``int64_t`` to future-proof the id as devices change over time.
+
+.. c:member:: ArrowDeviceType ArrowDeviceArray.device_type
+
+    The type of the device which can access the buffers in the array.
+
+.. c:member:: void* ArrowDeviceArray.sync_event
+
+    Optional. An event-like object to synchronize on if needed.
+
+    Many devices, like GPUs, are primarily asynchronous with respect to
+    CPU processing. As such in order to safely access memory, it is often
+    necessary to have an object to synchronize processing with. Since
+    different devices will use different types to specify this, we use a
+    void* which can be coerced into a pointer to whatever the device
+    appropriate type is.
+
+    If synchronization is not needed, this can be null. If this is non-null
+    then it MUST be used to call the appropriate sync method for the device
+    (e.g. ``cudaStreamWaitEvent`` or ``hipStreamWaitEvent``) before attempting
+    to access the memory in the buffers.
+
+    Expected types to coerce this ``void*`` to depending on the reported

Review Comment:
   Perhaps detail this in a separate paragraph about event types?
   For example:
   ```restructuredtext
   Synchronization event types
   ---------------------------
   
   The table below lists the event types supported for each the device type.
   If no event type is supported ("N/A"), then the ``sync_event`` member
   should always be null.
   
   +--------------------------+----------------------+--------+
   | Device type              | Actual event type    | Notes  |
   +==========================+======================+========+
   | ARROW_DEVICE_CPU         | N/A                  |        |
   +--------------------------+----------------------+--------+
   | ARROW_DEVICE_CUDA        | ``cudaEvent_t*``     |        |
   +--------------------------+----------------------+--------+
   | ...                      | ...                  |        |
   +--------------------------+----------------------+--------+
   ```



##########
docs/source/format/CDeviceDataInterface.rst:
##########
@@ -0,0 +1,628 @@
+.. Licensed to the Apache Software Foundation (ASF) under one
+.. or more contributor license agreements.  See the NOTICE file
+.. distributed with this work for additional information
+.. regarding copyright ownership.  The ASF licenses this file
+.. to you under the Apache License, Version 2.0 (the
+.. "License"); you may not use this file except in compliance
+.. with the License.  You may obtain a copy of the License at
+
+..   http://www.apache.org/licenses/LICENSE-2.0
+
+.. Unless required by applicable law or agreed to in writing,
+.. software distributed under the License is distributed on an
+.. "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+.. KIND, either express or implied.  See the License for the
+.. specific language governing permissions and limitations
+.. under the License.
+
+.. highlight:: c
+
+.. _c-device-data-interface:
+
+=================================
+The Arrow C Device data interface
+=================================
+
+.. note:: The Arrow C Device Data Interface should be considered experimental
+
+Rationale
+=========
+
+The current :ref:`C Data Interface <c-data-interface>`, and most
+implementations of it, make the assumption that all data buffers provided
+are CPU buffers. Since Apache Arrow is designed to be a universal in-memory
+format for representing tabular ("columnar") data, there will be the desire
+to leverage this data on non-CPU hardware such as GPUs. One example of such
+a case is the `RAPIDS cuDF library`_ which uses the Arrow memory format with
+CUDA for NVIDIA GPUs. Since copying data from host to device and back is
+expensive, the ideal would be to be able to leave the data on the device
+for as long as possible, even when passing it between runtimes and
+libraries.
+
+The Arrow C Device data interface builds on the existing C data interface
+by adding a very small, stable set of C definitions to it. These definitions
+are equivalents to the ``ArrowArray`` and ``ArrowArrayStream`` structures
+from the C Data Interface which add members to allow specifying the device
+type and pass necessary information to synchronize with the producer.
+For non-C/C++ languages and runtimes, translating the C definitions to
+corresponding C FFI declarations should be just as simple as with the
+current C data interface.
+
+Applications and libraries can then use Arrow schemas and Arrow formatted
+memory on non-CPU devices to exchange data just as easily as they do
+now with CPU data. This will enable leaving data on those devices longer
+and avoiding costly copies back and forth between the host and device
+just to leverage new libraries and runtimes.
+
+Goals
+-----
+
+* Expose an ABI-stable interface built on the existing C data interface.
+* Make it easy for third-party projects to implement support with little
+  initial investment.
+* Allow zero-copy sharing of Arrow formatted device memory between
+  independant runtimes and components running in the same process.
+* Avoid the need for one-to-one adaptation layers such as the
+  `CUDA Array Interface`_ for Python processes to pass CUDA data.
+* Enable integration without explicit dependencies (either at compile-time
+  or runtime) on the Arrow software project itself.
+
+The intent is for the Arrow C Device data interface to expand the reach
+of the current C data interface, allowing it to also become the standard
+low-level building block for columnar processing on devices like GPUs or
+FPGAs.
+
+Structure definitions
+=====================
+
+Because this is built on the C data interface, the C Device data interface
+uses the ``ArrowSchema`` and ``ArrowArray`` structures as defined in the
+:ref:`C data interface spec <c-data-interface-struct-defs>`. It then adds the
+following free-standing definitions. Like the rest of the Arrow project,
+they are available under the Apache License 2.0.
+
+.. code-block:: c
+
+    #ifndef ARROW_C_DEVICE_DATA_INTERFACE
+    #define ARROW_C_DEVICE_DATA_INTERFACE
+
+    // Device type for the allocated memory
+    typedef int32_t ArrowDeviceType;
+
+    // CPU device, same as using ArrowArray directly
+    #define ARROW_DEVICE_CPU 1
+    // CUDA GPU Device
+    #define ARROW_DEVICE_CUDA 2
+    // Pinned CUDA CPU memory by cudaMallocHost
+    #define ARROW_DEVICE_CUDA_HOST 3
+    // OpenCL Device
+    #define ARROW_DEVICE_OPENCL 4
+    // Vulkan buffer for next-gen graphics
+    #define ARROW_DEVICE_VULKAN 7
+    // Metal for Apple GPU
+    #define ARROW_DEVICE_METAL 8
+    // Verilog simulator buffer
+    #define ARROW_DEVICE_VPI 9
+    // ROCm GPUs for AMD GPUs
+    #define ARROW_DEVICE_ROCM 10
+    // Pinned ROCm CPU memory allocated by hipMallocHost
+    #define ARROW_DEVICE_ROCM_HOST 11
+    // Reserved for extension
+    //
+    // used to quickly test extension devices, semantics
+    // can differ based on implementation
+    #define ARROW_DEVICE_EXT_DEV 12
+    // CUDA managed/unified memory allocated by cudaMallocManaged
+    #define ARROW_DEVICE_CUDA_MANAGED 13
+    // Unified shared memory allocated on a oneAPI
+    // non-partitioned device.
+    //
+    // A call to the oneAPI runtime is required to determine the
+    // device type, the USM allocation type and the sycl context
+    // that it is bound to.
+    #define ARROW_DEVICE_ONEAPI 14
+    // GPU support for next-gen WebGPU standard
+    #define ARROW_DEVICE_WEBGPU 15
+    // Qualcomm Hexagon DSP
+    #define ARROW_DEVICE_HEXAGON 16
+
+    struct ArrowDeviceArray {
+      struct ArrowArray array;
+      int64_t device_id;
+      ArrowDeviceType device_type;
+      void* sync_event;
+
+      // reserved bytes for future expansion
+      int64_t reserved[3];
+    };
+
+    #endif  // ARROW_C_DEVICE_DATA_INTERFACE
+
+.. note::
+   The canonical guard ``ARROW_C_DEVICE_DATA_INTERFACE`` is meant to avoid
+   duplicate definitions if two projects copy the definitions in their own
+   headers, and a third-party project includes from these two projects. It
+   is therefore important that this guard is kept exactly as-is when these
+   definitions are copied.
+
+ArrowDeviceType
+---------------
+
+The ``ArrowDeviceType`` typedef is used to indicate what type of device the
+provided memory buffers were allocated on. This, in conjunction with the
+``device_id``, should be sufficient to reference the correct data buffers.
+
+We then use macros to define values for different device types. The provided
+macro values are compatible with the widely used `dlpack`_ ``DLDeviceType``
+definition values, using the same value for each as the equivalent
+``kDL<type>`` enum from dlpack.h. The list will be kept in sync with those
+equivalent enum values over time to ensure compatibility, rather than
+potentially diverging. To avoid the Arrow project having to be in the
+position of vetting new hardware devices, new additions should first be
+added to dlpack before we add a corresponding macro here.
+
+To ensure predictability with the ABI, we use macros instead of an ``enum``
+so the storage type is not compiler dependent.
+
+.. c:macro:: ARROW_DEVICE_CPU
+
+    CPU Device, equivalent to just using ``ArrowArray`` directly instead of
+    using ``ArrowDeviceArray``.
+
+.. c:macro:: ARROW_DEVICE_CUDA
+
+    A `CUDA`_ GPU Device. This could represent data allocated either with the
+    runtime library (``cudaMalloc``) or the device driver (``cuMemAlloc``).
+
+.. c:macro:: ARROW_DEVICE_CUDA_HOST
+
+    CPU memory that was pinned and page-locked by CUDA by using
+    ``cudaMallocHost`` or ``cuMemAllocHost``.
+
+.. c:macro:: ARROW_DEVICE_OPENCL
+
+    Data allocated on the device by using the `OpenCL (Open Computing Language)`_
+    framework.
+
+.. c:macro:: ARROW_DEVICE_VULKAN
+
+    Data allocated by the `Vulkan`_ framework and libraries.
+
+.. c:macro:: ARROW_DEVICE_METAL
+
+    Data on Apple GPU devices using the `Metal`_ framework and libraries.
+
+.. c:macro:: ARROW_DEVICE_VPI
+
+    Indicates usage of a Verilog simulator buffer.
+
+.. c:macro:: ARROW_DEVICE_ROCM
+
+    An AMD device using the `ROCm`_ stack.
+
+.. c:macro:: ARROW_DEVICE_ROCM_HOST
+
+    CPU memory pinned and page-locked allocated ``hipMallocHost``.
+
+.. c:macro:: ARROW_DEVICE_EXT_DEV
+
+    This value is an escape-hatch for devices to extend which aren't
+    currently represented otherwise. Producers would need to provide
+    additional information/context specific to the device if using
+    this device type. This is used to quickly test extension devices
+    and semantics can differ based on the implementation.
+
+.. c:macro:: ARROW_DEVICE_CUDA_MANAGED
+
+    CUDA managed/unified memory which is allocated by ``cudaMallocManaged``.
+
+.. c:macro:: ARROW_DEVICE_ONEAPI
+
+    Unified shared memory allocated on an Intel `oneAPI`_ non-partitioned
+    device. A call to the ``oneAPI`` runtime is required to determine
+    the specific device type, the USM allocation type and the sycl context
+    that it is bound to.
+
+.. c:macro:: ARROW_DEVICE_WEBGPU
+
+    GPU support for next-gen WebGPU standards
+
+.. c:macro:: ARROW_DEVICE_HEXAGON
+
+    Data allocated on a Qualcomm Hexagon DSP device.
+
+The ArrowDeviceArray structure
+------------------------------
+
+The ``ArrowDeviceArray`` structure embeds the C data ``ArrowArray`` structure
+and adds additional information necessary for consumers to use the data. It
+has the following fields:
+
+.. c:member:: struct ArrowArray ArrowDeviceArray.array
+
+    The allocated array data. The values in the ``void**`` buffers (along
+    with the buffers of any children) are what is allocated on the device.
+    The buffer values should be device pointers. The rest of the structure
+    should be accessible to the CPU.
+
+    The ``private_data`` and ``release`` callback of this structure should
+    contain any necessary information and structures related to freeing
+    the array according to the device it is allocated on, rather than
+    having a separate release callback and ``private_data`` pointer here.
+
+.. c:member:: int64_t ArrowDeviceArray.device_id
+
+    The device id to identify a specific device if multiple devices of this
+    type are on the system. The semantics of the id will be hardware dependent,
+    but we use an ``int64_t`` to future-proof the id as devices change over time.
+
+.. c:member:: ArrowDeviceType ArrowDeviceArray.device_type
+
+    The type of the device which can access the buffers in the array.
+
+.. c:member:: void* ArrowDeviceArray.sync_event
+
+    Optional. An event-like object to synchronize on if needed.
+
+    Many devices, like GPUs, are primarily asynchronous with respect to
+    CPU processing. As such in order to safely access memory, it is often
+    necessary to have an object to synchronize processing with. Since
+    different devices will use different types to specify this, we use a
+    void* which can be coerced into a pointer to whatever the device
+    appropriate type is.
+
+    If synchronization is not needed, this can be null. If this is non-null
+    then it MUST be used to call the appropriate sync method for the device
+    (e.g. ``cudaStreamWaitEvent`` or ``hipStreamWaitEvent``) before attempting
+    to access the memory in the buffers.
+
+    Expected types to coerce this ``void*`` to depending on the reported
+    device type:
+
+    * CUDA: ``cudaEvent_t*``
+    * ROCm: ``hipEvent_t*``
+    * OpenCL: ``cl_event*``
+    * Vulkan: ``VkEvent*``
+    * Metal: ``MTLEvent*``
+    * OneAPI: ``sycl::event*``
+
+    If an event is provided, then the producer MUST ensure that the event
+    is triggered/recorded at the end of the processing stream once the data
+    is considered available for use.

Review Comment:
   This is not clear to me. What is the "processing stream"? Can we make this simpler? e.g.:
   > If an event is provided, then the producer MUST ensure that the exported data is available on the device before the even becomes ready. The consumer SHOULD wait on the event before trying to access the exported data.



##########
docs/source/format/CDeviceDataInterface.rst:
##########
@@ -0,0 +1,628 @@
+.. Licensed to the Apache Software Foundation (ASF) under one
+.. or more contributor license agreements.  See the NOTICE file
+.. distributed with this work for additional information
+.. regarding copyright ownership.  The ASF licenses this file
+.. to you under the Apache License, Version 2.0 (the
+.. "License"); you may not use this file except in compliance
+.. with the License.  You may obtain a copy of the License at
+
+..   http://www.apache.org/licenses/LICENSE-2.0
+
+.. Unless required by applicable law or agreed to in writing,
+.. software distributed under the License is distributed on an
+.. "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+.. KIND, either express or implied.  See the License for the
+.. specific language governing permissions and limitations
+.. under the License.
+
+.. highlight:: c
+
+.. _c-device-data-interface:
+
+=================================
+The Arrow C Device data interface
+=================================
+
+.. note:: The Arrow C Device Data Interface should be considered experimental
+
+Rationale
+=========
+
+The current :ref:`C Data Interface <c-data-interface>`, and most
+implementations of it, make the assumption that all data buffers provided
+are CPU buffers. Since Apache Arrow is designed to be a universal in-memory
+format for representing tabular ("columnar") data, there will be the desire
+to leverage this data on non-CPU hardware such as GPUs. One example of such
+a case is the `RAPIDS cuDF library`_ which uses the Arrow memory format with
+CUDA for NVIDIA GPUs. Since copying data from host to device and back is
+expensive, the ideal would be to be able to leave the data on the device
+for as long as possible, even when passing it between runtimes and
+libraries.
+
+The Arrow C Device data interface builds on the existing C data interface
+by adding a very small, stable set of C definitions to it. These definitions
+are equivalents to the ``ArrowArray`` and ``ArrowArrayStream`` structures
+from the C Data Interface which add members to allow specifying the device
+type and pass necessary information to synchronize with the producer.
+For non-C/C++ languages and runtimes, translating the C definitions to
+corresponding C FFI declarations should be just as simple as with the
+current C data interface.
+
+Applications and libraries can then use Arrow schemas and Arrow formatted
+memory on non-CPU devices to exchange data just as easily as they do
+now with CPU data. This will enable leaving data on those devices longer
+and avoiding costly copies back and forth between the host and device
+just to leverage new libraries and runtimes.
+
+Goals
+-----
+
+* Expose an ABI-stable interface built on the existing C data interface.
+* Make it easy for third-party projects to implement support with little
+  initial investment.
+* Allow zero-copy sharing of Arrow formatted device memory between
+  independant runtimes and components running in the same process.
+* Avoid the need for one-to-one adaptation layers such as the
+  `CUDA Array Interface`_ for Python processes to pass CUDA data.
+* Enable integration without explicit dependencies (either at compile-time
+  or runtime) on the Arrow software project itself.
+
+The intent is for the Arrow C Device data interface to expand the reach
+of the current C data interface, allowing it to also become the standard
+low-level building block for columnar processing on devices like GPUs or
+FPGAs.
+
+Structure definitions
+=====================
+
+Because this is built on the C data interface, the C Device data interface
+uses the ``ArrowSchema`` and ``ArrowArray`` structures as defined in the
+:ref:`C data interface spec <c-data-interface-struct-defs>`. It then adds the
+following free-standing definitions. Like the rest of the Arrow project,
+they are available under the Apache License 2.0.
+
+.. code-block:: c
+
+    #ifndef ARROW_C_DEVICE_DATA_INTERFACE
+    #define ARROW_C_DEVICE_DATA_INTERFACE
+
+    // Device type for the allocated memory
+    typedef int32_t ArrowDeviceType;
+
+    // CPU device, same as using ArrowArray directly
+    #define ARROW_DEVICE_CPU 1
+    // CUDA GPU Device
+    #define ARROW_DEVICE_CUDA 2
+    // Pinned CUDA CPU memory by cudaMallocHost
+    #define ARROW_DEVICE_CUDA_HOST 3
+    // OpenCL Device
+    #define ARROW_DEVICE_OPENCL 4
+    // Vulkan buffer for next-gen graphics
+    #define ARROW_DEVICE_VULKAN 7
+    // Metal for Apple GPU
+    #define ARROW_DEVICE_METAL 8
+    // Verilog simulator buffer
+    #define ARROW_DEVICE_VPI 9
+    // ROCm GPUs for AMD GPUs
+    #define ARROW_DEVICE_ROCM 10
+    // Pinned ROCm CPU memory allocated by hipMallocHost
+    #define ARROW_DEVICE_ROCM_HOST 11
+    // Reserved for extension
+    //
+    // used to quickly test extension devices, semantics
+    // can differ based on implementation
+    #define ARROW_DEVICE_EXT_DEV 12
+    // CUDA managed/unified memory allocated by cudaMallocManaged
+    #define ARROW_DEVICE_CUDA_MANAGED 13
+    // Unified shared memory allocated on a oneAPI
+    // non-partitioned device.
+    //
+    // A call to the oneAPI runtime is required to determine the
+    // device type, the USM allocation type and the sycl context
+    // that it is bound to.
+    #define ARROW_DEVICE_ONEAPI 14
+    // GPU support for next-gen WebGPU standard
+    #define ARROW_DEVICE_WEBGPU 15
+    // Qualcomm Hexagon DSP
+    #define ARROW_DEVICE_HEXAGON 16
+
+    struct ArrowDeviceArray {
+      struct ArrowArray array;
+      int64_t device_id;
+      ArrowDeviceType device_type;
+      void* sync_event;
+
+      // reserved bytes for future expansion
+      int64_t reserved[3];
+    };
+
+    #endif  // ARROW_C_DEVICE_DATA_INTERFACE
+
+.. note::
+   The canonical guard ``ARROW_C_DEVICE_DATA_INTERFACE`` is meant to avoid
+   duplicate definitions if two projects copy the definitions in their own
+   headers, and a third-party project includes from these two projects. It
+   is therefore important that this guard is kept exactly as-is when these
+   definitions are copied.
+
+ArrowDeviceType
+---------------
+
+The ``ArrowDeviceType`` typedef is used to indicate what type of device the
+provided memory buffers were allocated on. This, in conjunction with the
+``device_id``, should be sufficient to reference the correct data buffers.
+
+We then use macros to define values for different device types. The provided
+macro values are compatible with the widely used `dlpack`_ ``DLDeviceType``
+definition values, using the same value for each as the equivalent
+``kDL<type>`` enum from dlpack.h. The list will be kept in sync with those
+equivalent enum values over time to ensure compatibility, rather than
+potentially diverging. To avoid the Arrow project having to be in the
+position of vetting new hardware devices, new additions should first be
+added to dlpack before we add a corresponding macro here.
+
+To ensure predictability with the ABI, we use macros instead of an ``enum``
+so the storage type is not compiler dependent.
+
+.. c:macro:: ARROW_DEVICE_CPU
+
+    CPU Device, equivalent to just using ``ArrowArray`` directly instead of
+    using ``ArrowDeviceArray``.
+
+.. c:macro:: ARROW_DEVICE_CUDA
+
+    A `CUDA`_ GPU Device. This could represent data allocated either with the
+    runtime library (``cudaMalloc``) or the device driver (``cuMemAlloc``).
+
+.. c:macro:: ARROW_DEVICE_CUDA_HOST
+
+    CPU memory that was pinned and page-locked by CUDA by using
+    ``cudaMallocHost`` or ``cuMemAllocHost``.
+
+.. c:macro:: ARROW_DEVICE_OPENCL
+
+    Data allocated on the device by using the `OpenCL (Open Computing Language)`_
+    framework.
+
+.. c:macro:: ARROW_DEVICE_VULKAN
+
+    Data allocated by the `Vulkan`_ framework and libraries.
+
+.. c:macro:: ARROW_DEVICE_METAL
+
+    Data on Apple GPU devices using the `Metal`_ framework and libraries.
+
+.. c:macro:: ARROW_DEVICE_VPI
+
+    Indicates usage of a Verilog simulator buffer.
+
+.. c:macro:: ARROW_DEVICE_ROCM
+
+    An AMD device using the `ROCm`_ stack.
+
+.. c:macro:: ARROW_DEVICE_ROCM_HOST
+
+    CPU memory pinned and page-locked allocated ``hipMallocHost``.
+
+.. c:macro:: ARROW_DEVICE_EXT_DEV
+
+    This value is an escape-hatch for devices to extend which aren't
+    currently represented otherwise. Producers would need to provide
+    additional information/context specific to the device if using
+    this device type. This is used to quickly test extension devices
+    and semantics can differ based on the implementation.
+
+.. c:macro:: ARROW_DEVICE_CUDA_MANAGED
+
+    CUDA managed/unified memory which is allocated by ``cudaMallocManaged``.
+
+.. c:macro:: ARROW_DEVICE_ONEAPI
+
+    Unified shared memory allocated on an Intel `oneAPI`_ non-partitioned
+    device. A call to the ``oneAPI`` runtime is required to determine
+    the specific device type, the USM allocation type and the sycl context
+    that it is bound to.
+
+.. c:macro:: ARROW_DEVICE_WEBGPU
+
+    GPU support for next-gen WebGPU standards
+
+.. c:macro:: ARROW_DEVICE_HEXAGON
+
+    Data allocated on a Qualcomm Hexagon DSP device.
+
+The ArrowDeviceArray structure
+------------------------------
+
+The ``ArrowDeviceArray`` structure embeds the C data ``ArrowArray`` structure
+and adds additional information necessary for consumers to use the data. It
+has the following fields:
+
+.. c:member:: struct ArrowArray ArrowDeviceArray.array
+
+    The allocated array data. The values in the ``void**`` buffers (along
+    with the buffers of any children) are what is allocated on the device.
+    The buffer values should be device pointers. The rest of the structure
+    should be accessible to the CPU.
+
+    The ``private_data`` and ``release`` callback of this structure should
+    contain any necessary information and structures related to freeing
+    the array according to the device it is allocated on, rather than
+    having a separate release callback and ``private_data`` pointer here.
+
+.. c:member:: int64_t ArrowDeviceArray.device_id
+
+    The device id to identify a specific device if multiple devices of this
+    type are on the system. The semantics of the id will be hardware dependent,
+    but we use an ``int64_t`` to future-proof the id as devices change over time.
+
+.. c:member:: ArrowDeviceType ArrowDeviceArray.device_type
+
+    The type of the device which can access the buffers in the array.
+
+.. c:member:: void* ArrowDeviceArray.sync_event
+
+    Optional. An event-like object to synchronize on if needed.
+
+    Many devices, like GPUs, are primarily asynchronous with respect to
+    CPU processing. As such in order to safely access memory, it is often
+    necessary to have an object to synchronize processing with. Since
+    different devices will use different types to specify this, we use a
+    void* which can be coerced into a pointer to whatever the device
+    appropriate type is.
+
+    If synchronization is not needed, this can be null. If this is non-null
+    then it MUST be used to call the appropriate sync method for the device
+    (e.g. ``cudaStreamWaitEvent`` or ``hipStreamWaitEvent``) before attempting
+    to access the memory in the buffers.
+
+    Expected types to coerce this ``void*`` to depending on the reported
+    device type:
+
+    * CUDA: ``cudaEvent_t*``
+    * ROCm: ``hipEvent_t*``
+    * OpenCL: ``cl_event*``
+    * Vulkan: ``VkEvent*``
+    * Metal: ``MTLEvent*``
+    * OneAPI: ``sycl::event*``
+
+    If an event is provided, then the producer MUST ensure that the event
+    is triggered/recorded at the end of the processing stream once the data
+    is considered available for use.
+
+
+.. c:member:: int64_t ArrowDeviceArray.reserved[3]
+
+    As non-CPU development expands, there may be a need to expand this
+    structure. In order to do so without potentially breaking ABI changes,
+    we reserve 24 bytes at the end of the object. This also has the added
+    benefit of bringing the total size of this structure to exactly 128
+    bytes (a power of 2) on 64-bit systems. These bytes should be zero'd
+    out after allocation in order to ensure safe evolution of the ABI in
+    the future.
+
+.. note::
+    Rather than store the shape / types of the data alongside the
+    ``ArrowDeviceArray``, users should utilize the existing ``ArrowSchema``
+    structure to pass any data type and shape information.
+
+Semantics
+=========
+
+Memory management
+-----------------
+
+The ``ArrowDeviceArray`` structure contains an ``ArrowArray`` object which
+itself has :ref:`specific semantics <c-data-interface-semantics>` for releasing
+memory. The term *"base structure"* below refers to the ``ArrowDeviceArray``
+object that is passed directly between the producer and consumer -- not any
+child structure thereof.
+
+It is intended for the base structure to be stack- or heap-allocated by the
+*consumer*. In this case, the producer API should take a pointer to the
+consumer-allocated structure.
+
+However, any data pointed to by the struct MUST be allocated and maintained
+by the producer. This includes the ``sync_event`` member if it is not null,
+along with any pointers in the ``ArrowArray`` object as usual. Data lifetime
+is managed through the ``release`` callback of the ``ArrowArray`` member.
+
+For an ``ArrowDeviceArray``, the semantics of a released structure and the
+callback semantics are identical to those for
+:ref:`ArrowArray itself <c-data-interface-released>`. Any producer specific context
+information necessary for releasing the device data buffers, in addition to
+any allocated event, should be stored in the ``private_data`` member of
+the ``ArrowArray`` and managed by the ``release`` callback.
+
+Moving an array
+'''''''''''''''
+
+The consumer can *move* the ``ArrowDeviceArray`` structure by bitwise copying
+or shallow member-wise copying. Then it MUST mark the source structure released
+by setting the ``release`` member of the embedded ``ArrowArray`` structure to
+``NULL``, but *without* calling that release callback. This ensures that only
+one live copy of the struct is active at any given time and that lifetime is
+correctly communicated to the producer.
+
+As usual, the release callback will be called on the destination structure
+when it is not needed anymore.
+
+Record batches
+--------------
+As with the C data interface itself, a record batch can be trivially considered
+as an equivalent struct array. In this case the metadata of the top-level
+``ArrowSchema`` can be used for schema-level metadata of the record batch.
+
+Mutability
+----------
+
+Both the producer and the consumer SHOULD consider the exported data (that
+is, the data reachable on the device through the ``buffers`` member of
+the embedded ``ArrowArray``) to be immutable, as either party could otherwise
+see inconsistent data while the other is mutating it.
+
+Likewise, if the ``sync_event`` member is non-NULL, the consumer should not
+attempt to access or read the data until they have synchronized on that event.
+
+C producer example
+====================
+
+Exporting a simple ``int32`` device array
+-----------------------------------------
+
+Export a non-nullable ``int32`` type with empty metadata. An example of this
+can be seen in the :ref:`C data interface docs directly <c-data-interface-export-int32-schema>`.
+
+To export the data itself, we transfer ownership to the consumer through
+the release callback. This example will use CUDA, but the equivalent calls
+could be used for any device:
+
+.. code-block:: c
+
+    static void release_int32_device_array(struct ArrowArray* array) {
+        assert(array->n_buffers == 2);
+        // destroy the event
+        cudaEvent_t* ev_ptr = reinterpret_cast<cudaEvent_t*>(array->private_data);
+        cudaError_t status = cudaEventDestroy(*ev_ptr);
+        assert(status == cudaSuccess);
+        free(ev_ptr);
+
+        // free the buffers and the buffers array
+        status = cudaFree(array->buffers[1]);
+        assert(status == cudaSuccess);
+        free(array->buffers);
+
+        // mark released
+        array->release = NULL;
+    }
+
+    __host__ void export_int32_device_array(void* cudaAllocdPtr,
+                                            cudaStream_t stream,
+                                            int64_t length,
+                                            struct ArrowDeviceArray* array) {
+        // get device id
+        int device;
+        cudaError_t status;
+        status = cudaGetDevice(&device);
+        assert(status == cudaSuccess);
+
+        cudaEvent_t* ev_ptr = (cudaEvent_t*)malloc(sizeof(cudaEvent_t));
+        assert(ev_ptr != NULL);
+        status = cudaEventCreate(ev_ptr);
+        assert(status == cudaSuccess);
+
+        // record event on the stream, assuming that the passed in
+        // stream is where the work to produce the data will be processing.
+        status = cudaEventRecord(*ev_ptr, stream);
+        assert(status == cudaSuccess);
+
+        // initialize fields
+        *array = (struct ArrowDeviceArray) {
+            .array = (struct ArrowArray) {
+                .length = length,
+                .null_count = 0,
+                .offset = 0,
+                .n_buffers = 2,
+                .n_children = 0,
+                .children = NULL,
+                .dictionary = NULL,
+                // bookeeping
+                .release = &release_int32_device_array,
+                .private_data = reinterpret_cast<void*>(ev_ptr),
+            },
+            .device_id = static_cast<int64_t>(device),
+            .device_type = ARROW_DEVICE_CUDA,
+            // pass the event pointer to the consumer
+            .sync_event = reinterpret_cast<void*>(ev_ptr),

Review Comment:
   Use C-style cast here.



##########
docs/source/format/CDeviceDataInterface.rst:
##########
@@ -0,0 +1,628 @@
+.. Licensed to the Apache Software Foundation (ASF) under one
+.. or more contributor license agreements.  See the NOTICE file
+.. distributed with this work for additional information
+.. regarding copyright ownership.  The ASF licenses this file
+.. to you under the Apache License, Version 2.0 (the
+.. "License"); you may not use this file except in compliance
+.. with the License.  You may obtain a copy of the License at
+
+..   http://www.apache.org/licenses/LICENSE-2.0
+
+.. Unless required by applicable law or agreed to in writing,
+.. software distributed under the License is distributed on an
+.. "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+.. KIND, either express or implied.  See the License for the
+.. specific language governing permissions and limitations
+.. under the License.
+
+.. highlight:: c
+
+.. _c-device-data-interface:
+
+=================================
+The Arrow C Device data interface
+=================================
+
+.. note:: The Arrow C Device Data Interface should be considered experimental
+
+Rationale
+=========
+
+The current :ref:`C Data Interface <c-data-interface>`, and most
+implementations of it, make the assumption that all data buffers provided
+are CPU buffers. Since Apache Arrow is designed to be a universal in-memory
+format for representing tabular ("columnar") data, there will be the desire
+to leverage this data on non-CPU hardware such as GPUs. One example of such
+a case is the `RAPIDS cuDF library`_ which uses the Arrow memory format with
+CUDA for NVIDIA GPUs. Since copying data from host to device and back is
+expensive, the ideal would be to be able to leave the data on the device
+for as long as possible, even when passing it between runtimes and
+libraries.
+
+The Arrow C Device data interface builds on the existing C data interface
+by adding a very small, stable set of C definitions to it. These definitions
+are equivalents to the ``ArrowArray`` and ``ArrowArrayStream`` structures
+from the C Data Interface which add members to allow specifying the device
+type and pass necessary information to synchronize with the producer.
+For non-C/C++ languages and runtimes, translating the C definitions to
+corresponding C FFI declarations should be just as simple as with the
+current C data interface.
+
+Applications and libraries can then use Arrow schemas and Arrow formatted
+memory on non-CPU devices to exchange data just as easily as they do
+now with CPU data. This will enable leaving data on those devices longer
+and avoiding costly copies back and forth between the host and device
+just to leverage new libraries and runtimes.
+
+Goals
+-----
+
+* Expose an ABI-stable interface built on the existing C data interface.
+* Make it easy for third-party projects to implement support with little
+  initial investment.
+* Allow zero-copy sharing of Arrow formatted device memory between
+  independant runtimes and components running in the same process.
+* Avoid the need for one-to-one adaptation layers such as the
+  `CUDA Array Interface`_ for Python processes to pass CUDA data.
+* Enable integration without explicit dependencies (either at compile-time
+  or runtime) on the Arrow software project itself.
+
+The intent is for the Arrow C Device data interface to expand the reach
+of the current C data interface, allowing it to also become the standard
+low-level building block for columnar processing on devices like GPUs or
+FPGAs.
+
+Structure definitions
+=====================
+
+Because this is built on the C data interface, the C Device data interface
+uses the ``ArrowSchema`` and ``ArrowArray`` structures as defined in the
+:ref:`C data interface spec <c-data-interface-struct-defs>`. It then adds the
+following free-standing definitions. Like the rest of the Arrow project,
+they are available under the Apache License 2.0.
+
+.. code-block:: c
+
+    #ifndef ARROW_C_DEVICE_DATA_INTERFACE
+    #define ARROW_C_DEVICE_DATA_INTERFACE
+
+    // Device type for the allocated memory
+    typedef int32_t ArrowDeviceType;
+
+    // CPU device, same as using ArrowArray directly
+    #define ARROW_DEVICE_CPU 1
+    // CUDA GPU Device
+    #define ARROW_DEVICE_CUDA 2
+    // Pinned CUDA CPU memory by cudaMallocHost
+    #define ARROW_DEVICE_CUDA_HOST 3
+    // OpenCL Device
+    #define ARROW_DEVICE_OPENCL 4
+    // Vulkan buffer for next-gen graphics
+    #define ARROW_DEVICE_VULKAN 7
+    // Metal for Apple GPU
+    #define ARROW_DEVICE_METAL 8
+    // Verilog simulator buffer
+    #define ARROW_DEVICE_VPI 9
+    // ROCm GPUs for AMD GPUs
+    #define ARROW_DEVICE_ROCM 10
+    // Pinned ROCm CPU memory allocated by hipMallocHost
+    #define ARROW_DEVICE_ROCM_HOST 11
+    // Reserved for extension
+    //
+    // used to quickly test extension devices, semantics
+    // can differ based on implementation
+    #define ARROW_DEVICE_EXT_DEV 12
+    // CUDA managed/unified memory allocated by cudaMallocManaged
+    #define ARROW_DEVICE_CUDA_MANAGED 13
+    // Unified shared memory allocated on a oneAPI
+    // non-partitioned device.
+    //
+    // A call to the oneAPI runtime is required to determine the
+    // device type, the USM allocation type and the sycl context
+    // that it is bound to.
+    #define ARROW_DEVICE_ONEAPI 14
+    // GPU support for next-gen WebGPU standard
+    #define ARROW_DEVICE_WEBGPU 15
+    // Qualcomm Hexagon DSP
+    #define ARROW_DEVICE_HEXAGON 16
+
+    struct ArrowDeviceArray {
+      struct ArrowArray array;
+      int64_t device_id;
+      ArrowDeviceType device_type;
+      void* sync_event;
+
+      // reserved bytes for future expansion
+      int64_t reserved[3];
+    };
+
+    #endif  // ARROW_C_DEVICE_DATA_INTERFACE
+
+.. note::
+   The canonical guard ``ARROW_C_DEVICE_DATA_INTERFACE`` is meant to avoid
+   duplicate definitions if two projects copy the definitions in their own
+   headers, and a third-party project includes from these two projects. It
+   is therefore important that this guard is kept exactly as-is when these
+   definitions are copied.
+
+ArrowDeviceType
+---------------
+
+The ``ArrowDeviceType`` typedef is used to indicate what type of device the
+provided memory buffers were allocated on. This, in conjunction with the
+``device_id``, should be sufficient to reference the correct data buffers.
+
+We then use macros to define values for different device types. The provided
+macro values are compatible with the widely used `dlpack`_ ``DLDeviceType``
+definition values, using the same value for each as the equivalent
+``kDL<type>`` enum from dlpack.h. The list will be kept in sync with those
+equivalent enum values over time to ensure compatibility, rather than
+potentially diverging. To avoid the Arrow project having to be in the
+position of vetting new hardware devices, new additions should first be
+added to dlpack before we add a corresponding macro here.
+
+To ensure predictability with the ABI, we use macros instead of an ``enum``
+so the storage type is not compiler dependent.
+
+.. c:macro:: ARROW_DEVICE_CPU
+
+    CPU Device, equivalent to just using ``ArrowArray`` directly instead of
+    using ``ArrowDeviceArray``.
+
+.. c:macro:: ARROW_DEVICE_CUDA
+
+    A `CUDA`_ GPU Device. This could represent data allocated either with the
+    runtime library (``cudaMalloc``) or the device driver (``cuMemAlloc``).
+
+.. c:macro:: ARROW_DEVICE_CUDA_HOST
+
+    CPU memory that was pinned and page-locked by CUDA by using
+    ``cudaMallocHost`` or ``cuMemAllocHost``.
+
+.. c:macro:: ARROW_DEVICE_OPENCL
+
+    Data allocated on the device by using the `OpenCL (Open Computing Language)`_
+    framework.
+
+.. c:macro:: ARROW_DEVICE_VULKAN
+
+    Data allocated by the `Vulkan`_ framework and libraries.
+
+.. c:macro:: ARROW_DEVICE_METAL
+
+    Data on Apple GPU devices using the `Metal`_ framework and libraries.
+
+.. c:macro:: ARROW_DEVICE_VPI
+
+    Indicates usage of a Verilog simulator buffer.
+
+.. c:macro:: ARROW_DEVICE_ROCM
+
+    An AMD device using the `ROCm`_ stack.
+
+.. c:macro:: ARROW_DEVICE_ROCM_HOST
+
+    CPU memory pinned and page-locked allocated ``hipMallocHost``.
+
+.. c:macro:: ARROW_DEVICE_EXT_DEV
+
+    This value is an escape-hatch for devices to extend which aren't
+    currently represented otherwise. Producers would need to provide
+    additional information/context specific to the device if using
+    this device type. This is used to quickly test extension devices
+    and semantics can differ based on the implementation.
+
+.. c:macro:: ARROW_DEVICE_CUDA_MANAGED
+
+    CUDA managed/unified memory which is allocated by ``cudaMallocManaged``.
+
+.. c:macro:: ARROW_DEVICE_ONEAPI
+
+    Unified shared memory allocated on an Intel `oneAPI`_ non-partitioned
+    device. A call to the ``oneAPI`` runtime is required to determine
+    the specific device type, the USM allocation type and the sycl context
+    that it is bound to.
+
+.. c:macro:: ARROW_DEVICE_WEBGPU
+
+    GPU support for next-gen WebGPU standards
+
+.. c:macro:: ARROW_DEVICE_HEXAGON
+
+    Data allocated on a Qualcomm Hexagon DSP device.
+
+The ArrowDeviceArray structure
+------------------------------
+
+The ``ArrowDeviceArray`` structure embeds the C data ``ArrowArray`` structure
+and adds additional information necessary for consumers to use the data. It
+has the following fields:
+
+.. c:member:: struct ArrowArray ArrowDeviceArray.array
+
+    The allocated array data. The values in the ``void**`` buffers (along
+    with the buffers of any children) are what is allocated on the device.
+    The buffer values should be device pointers. The rest of the structure
+    should be accessible to the CPU.
+
+    The ``private_data`` and ``release`` callback of this structure should
+    contain any necessary information and structures related to freeing
+    the array according to the device it is allocated on, rather than
+    having a separate release callback and ``private_data`` pointer here.
+
+.. c:member:: int64_t ArrowDeviceArray.device_id
+
+    The device id to identify a specific device if multiple devices of this
+    type are on the system. The semantics of the id will be hardware dependent,
+    but we use an ``int64_t`` to future-proof the id as devices change over time.
+
+.. c:member:: ArrowDeviceType ArrowDeviceArray.device_type
+
+    The type of the device which can access the buffers in the array.
+
+.. c:member:: void* ArrowDeviceArray.sync_event
+
+    Optional. An event-like object to synchronize on if needed.
+
+    Many devices, like GPUs, are primarily asynchronous with respect to
+    CPU processing. As such in order to safely access memory, it is often
+    necessary to have an object to synchronize processing with. Since
+    different devices will use different types to specify this, we use a
+    void* which can be coerced into a pointer to whatever the device
+    appropriate type is.
+
+    If synchronization is not needed, this can be null. If this is non-null
+    then it MUST be used to call the appropriate sync method for the device
+    (e.g. ``cudaStreamWaitEvent`` or ``hipStreamWaitEvent``) before attempting
+    to access the memory in the buffers.
+
+    Expected types to coerce this ``void*`` to depending on the reported
+    device type:
+
+    * CUDA: ``cudaEvent_t*``
+    * ROCm: ``hipEvent_t*``
+    * OpenCL: ``cl_event*``
+    * Vulkan: ``VkEvent*``
+    * Metal: ``MTLEvent*``
+    * OneAPI: ``sycl::event*``
+
+    If an event is provided, then the producer MUST ensure that the event
+    is triggered/recorded at the end of the processing stream once the data
+    is considered available for use.
+
+
+.. c:member:: int64_t ArrowDeviceArray.reserved[3]
+
+    As non-CPU development expands, there may be a need to expand this
+    structure. In order to do so without potentially breaking ABI changes,
+    we reserve 24 bytes at the end of the object. This also has the added
+    benefit of bringing the total size of this structure to exactly 128
+    bytes (a power of 2) on 64-bit systems. These bytes should be zero'd
+    out after allocation in order to ensure safe evolution of the ABI in
+    the future.
+
+.. note::
+    Rather than store the shape / types of the data alongside the
+    ``ArrowDeviceArray``, users should utilize the existing ``ArrowSchema``
+    structure to pass any data type and shape information.
+
+Semantics
+=========
+
+Memory management
+-----------------
+
+The ``ArrowDeviceArray`` structure contains an ``ArrowArray`` object which
+itself has :ref:`specific semantics <c-data-interface-semantics>` for releasing
+memory. The term *"base structure"* below refers to the ``ArrowDeviceArray``
+object that is passed directly between the producer and consumer -- not any
+child structure thereof.
+
+It is intended for the base structure to be stack- or heap-allocated by the
+*consumer*. In this case, the producer API should take a pointer to the
+consumer-allocated structure.
+
+However, any data pointed to by the struct MUST be allocated and maintained
+by the producer. This includes the ``sync_event`` member if it is not null,
+along with any pointers in the ``ArrowArray`` object as usual. Data lifetime
+is managed through the ``release`` callback of the ``ArrowArray`` member.
+
+For an ``ArrowDeviceArray``, the semantics of a released structure and the
+callback semantics are identical to those for
+:ref:`ArrowArray itself <c-data-interface-released>`. Any producer specific context
+information necessary for releasing the device data buffers, in addition to
+any allocated event, should be stored in the ``private_data`` member of
+the ``ArrowArray`` and managed by the ``release`` callback.
+
+Moving an array
+'''''''''''''''
+
+The consumer can *move* the ``ArrowDeviceArray`` structure by bitwise copying
+or shallow member-wise copying. Then it MUST mark the source structure released
+by setting the ``release`` member of the embedded ``ArrowArray`` structure to
+``NULL``, but *without* calling that release callback. This ensures that only
+one live copy of the struct is active at any given time and that lifetime is
+correctly communicated to the producer.
+
+As usual, the release callback will be called on the destination structure
+when it is not needed anymore.
+
+Record batches
+--------------
+As with the C data interface itself, a record batch can be trivially considered
+as an equivalent struct array. In this case the metadata of the top-level
+``ArrowSchema`` can be used for schema-level metadata of the record batch.
+
+Mutability
+----------
+
+Both the producer and the consumer SHOULD consider the exported data (that
+is, the data reachable on the device through the ``buffers`` member of
+the embedded ``ArrowArray``) to be immutable, as either party could otherwise
+see inconsistent data while the other is mutating it.
+
+Likewise, if the ``sync_event`` member is non-NULL, the consumer should not
+attempt to access or read the data until they have synchronized on that event.
+
+C producer example
+====================
+
+Exporting a simple ``int32`` device array
+-----------------------------------------
+
+Export a non-nullable ``int32`` type with empty metadata. An example of this
+can be seen in the :ref:`C data interface docs directly <c-data-interface-export-int32-schema>`.
+
+To export the data itself, we transfer ownership to the consumer through
+the release callback. This example will use CUDA, but the equivalent calls
+could be used for any device:
+
+.. code-block:: c
+
+    static void release_int32_device_array(struct ArrowArray* array) {
+        assert(array->n_buffers == 2);
+        // destroy the event
+        cudaEvent_t* ev_ptr = reinterpret_cast<cudaEvent_t*>(array->private_data);

Review Comment:
   This wouldn't work since `array` is a `struct ArrowArray*` here? Need to cast to `struct ArrowDeviceArray*` first.
   
   (note the cast might technically be UB...)



##########
docs/source/format/CDeviceDataInterface.rst:
##########
@@ -0,0 +1,628 @@
+.. Licensed to the Apache Software Foundation (ASF) under one
+.. or more contributor license agreements.  See the NOTICE file
+.. distributed with this work for additional information
+.. regarding copyright ownership.  The ASF licenses this file
+.. to you under the Apache License, Version 2.0 (the
+.. "License"); you may not use this file except in compliance
+.. with the License.  You may obtain a copy of the License at
+
+..   http://www.apache.org/licenses/LICENSE-2.0
+
+.. Unless required by applicable law or agreed to in writing,
+.. software distributed under the License is distributed on an
+.. "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+.. KIND, either express or implied.  See the License for the
+.. specific language governing permissions and limitations
+.. under the License.
+
+.. highlight:: c
+
+.. _c-device-data-interface:
+
+=================================
+The Arrow C Device data interface
+=================================
+
+.. note:: The Arrow C Device Data Interface should be considered experimental
+
+Rationale
+=========
+
+The current :ref:`C Data Interface <c-data-interface>`, and most
+implementations of it, make the assumption that all data buffers provided
+are CPU buffers. Since Apache Arrow is designed to be a universal in-memory
+format for representing tabular ("columnar") data, there will be the desire
+to leverage this data on non-CPU hardware such as GPUs. One example of such
+a case is the `RAPIDS cuDF library`_ which uses the Arrow memory format with
+CUDA for NVIDIA GPUs. Since copying data from host to device and back is
+expensive, the ideal would be to be able to leave the data on the device
+for as long as possible, even when passing it between runtimes and
+libraries.
+
+The Arrow C Device data interface builds on the existing C data interface
+by adding a very small, stable set of C definitions to it. These definitions
+are equivalents to the ``ArrowArray`` and ``ArrowArrayStream`` structures
+from the C Data Interface which add members to allow specifying the device
+type and pass necessary information to synchronize with the producer.
+For non-C/C++ languages and runtimes, translating the C definitions to
+corresponding C FFI declarations should be just as simple as with the
+current C data interface.
+
+Applications and libraries can then use Arrow schemas and Arrow formatted
+memory on non-CPU devices to exchange data just as easily as they do
+now with CPU data. This will enable leaving data on those devices longer
+and avoiding costly copies back and forth between the host and device
+just to leverage new libraries and runtimes.
+
+Goals
+-----
+
+* Expose an ABI-stable interface built on the existing C data interface.
+* Make it easy for third-party projects to implement support with little
+  initial investment.
+* Allow zero-copy sharing of Arrow formatted device memory between
+  independant runtimes and components running in the same process.
+* Avoid the need for one-to-one adaptation layers such as the
+  `CUDA Array Interface`_ for Python processes to pass CUDA data.
+* Enable integration without explicit dependencies (either at compile-time
+  or runtime) on the Arrow software project itself.
+
+The intent is for the Arrow C Device data interface to expand the reach
+of the current C data interface, allowing it to also become the standard
+low-level building block for columnar processing on devices like GPUs or
+FPGAs.
+
+Structure definitions
+=====================
+
+Because this is built on the C data interface, the C Device data interface
+uses the ``ArrowSchema`` and ``ArrowArray`` structures as defined in the
+:ref:`C data interface spec <c-data-interface-struct-defs>`. It then adds the
+following free-standing definitions. Like the rest of the Arrow project,
+they are available under the Apache License 2.0.
+
+.. code-block:: c
+
+    #ifndef ARROW_C_DEVICE_DATA_INTERFACE
+    #define ARROW_C_DEVICE_DATA_INTERFACE
+
+    // Device type for the allocated memory
+    typedef int32_t ArrowDeviceType;
+
+    // CPU device, same as using ArrowArray directly
+    #define ARROW_DEVICE_CPU 1
+    // CUDA GPU Device
+    #define ARROW_DEVICE_CUDA 2
+    // Pinned CUDA CPU memory by cudaMallocHost
+    #define ARROW_DEVICE_CUDA_HOST 3
+    // OpenCL Device
+    #define ARROW_DEVICE_OPENCL 4
+    // Vulkan buffer for next-gen graphics
+    #define ARROW_DEVICE_VULKAN 7
+    // Metal for Apple GPU
+    #define ARROW_DEVICE_METAL 8
+    // Verilog simulator buffer
+    #define ARROW_DEVICE_VPI 9
+    // ROCm GPUs for AMD GPUs
+    #define ARROW_DEVICE_ROCM 10
+    // Pinned ROCm CPU memory allocated by hipMallocHost
+    #define ARROW_DEVICE_ROCM_HOST 11
+    // Reserved for extension
+    //
+    // used to quickly test extension devices, semantics
+    // can differ based on implementation
+    #define ARROW_DEVICE_EXT_DEV 12
+    // CUDA managed/unified memory allocated by cudaMallocManaged
+    #define ARROW_DEVICE_CUDA_MANAGED 13
+    // Unified shared memory allocated on a oneAPI
+    // non-partitioned device.
+    //
+    // A call to the oneAPI runtime is required to determine the
+    // device type, the USM allocation type and the sycl context
+    // that it is bound to.
+    #define ARROW_DEVICE_ONEAPI 14
+    // GPU support for next-gen WebGPU standard
+    #define ARROW_DEVICE_WEBGPU 15
+    // Qualcomm Hexagon DSP
+    #define ARROW_DEVICE_HEXAGON 16
+
+    struct ArrowDeviceArray {
+      struct ArrowArray array;
+      int64_t device_id;
+      ArrowDeviceType device_type;
+      void* sync_event;
+
+      // reserved bytes for future expansion
+      int64_t reserved[3];
+    };
+
+    #endif  // ARROW_C_DEVICE_DATA_INTERFACE
+
+.. note::
+   The canonical guard ``ARROW_C_DEVICE_DATA_INTERFACE`` is meant to avoid
+   duplicate definitions if two projects copy the definitions in their own
+   headers, and a third-party project includes from these two projects. It
+   is therefore important that this guard is kept exactly as-is when these
+   definitions are copied.
+
+ArrowDeviceType
+---------------
+
+The ``ArrowDeviceType`` typedef is used to indicate what type of device the
+provided memory buffers were allocated on. This, in conjunction with the
+``device_id``, should be sufficient to reference the correct data buffers.
+
+We then use macros to define values for different device types. The provided
+macro values are compatible with the widely used `dlpack`_ ``DLDeviceType``
+definition values, using the same value for each as the equivalent
+``kDL<type>`` enum from dlpack.h. The list will be kept in sync with those
+equivalent enum values over time to ensure compatibility, rather than
+potentially diverging. To avoid the Arrow project having to be in the
+position of vetting new hardware devices, new additions should first be
+added to dlpack before we add a corresponding macro here.
+
+To ensure predictability with the ABI, we use macros instead of an ``enum``
+so the storage type is not compiler dependent.
+
+.. c:macro:: ARROW_DEVICE_CPU
+
+    CPU Device, equivalent to just using ``ArrowArray`` directly instead of
+    using ``ArrowDeviceArray``.
+
+.. c:macro:: ARROW_DEVICE_CUDA
+
+    A `CUDA`_ GPU Device. This could represent data allocated either with the
+    runtime library (``cudaMalloc``) or the device driver (``cuMemAlloc``).
+
+.. c:macro:: ARROW_DEVICE_CUDA_HOST
+
+    CPU memory that was pinned and page-locked by CUDA by using
+    ``cudaMallocHost`` or ``cuMemAllocHost``.
+
+.. c:macro:: ARROW_DEVICE_OPENCL
+
+    Data allocated on the device by using the `OpenCL (Open Computing Language)`_
+    framework.
+
+.. c:macro:: ARROW_DEVICE_VULKAN
+
+    Data allocated by the `Vulkan`_ framework and libraries.
+
+.. c:macro:: ARROW_DEVICE_METAL
+
+    Data on Apple GPU devices using the `Metal`_ framework and libraries.
+
+.. c:macro:: ARROW_DEVICE_VPI
+
+    Indicates usage of a Verilog simulator buffer.
+
+.. c:macro:: ARROW_DEVICE_ROCM
+
+    An AMD device using the `ROCm`_ stack.
+
+.. c:macro:: ARROW_DEVICE_ROCM_HOST
+
+    CPU memory pinned and page-locked allocated ``hipMallocHost``.
+
+.. c:macro:: ARROW_DEVICE_EXT_DEV
+
+    This value is an escape-hatch for devices to extend which aren't
+    currently represented otherwise. Producers would need to provide
+    additional information/context specific to the device if using
+    this device type. This is used to quickly test extension devices
+    and semantics can differ based on the implementation.
+
+.. c:macro:: ARROW_DEVICE_CUDA_MANAGED
+
+    CUDA managed/unified memory which is allocated by ``cudaMallocManaged``.
+
+.. c:macro:: ARROW_DEVICE_ONEAPI
+
+    Unified shared memory allocated on an Intel `oneAPI`_ non-partitioned
+    device. A call to the ``oneAPI`` runtime is required to determine
+    the specific device type, the USM allocation type and the sycl context
+    that it is bound to.
+
+.. c:macro:: ARROW_DEVICE_WEBGPU
+
+    GPU support for next-gen WebGPU standards
+
+.. c:macro:: ARROW_DEVICE_HEXAGON
+
+    Data allocated on a Qualcomm Hexagon DSP device.
+
+The ArrowDeviceArray structure
+------------------------------
+
+The ``ArrowDeviceArray`` structure embeds the C data ``ArrowArray`` structure
+and adds additional information necessary for consumers to use the data. It
+has the following fields:
+
+.. c:member:: struct ArrowArray ArrowDeviceArray.array
+
+    The allocated array data. The values in the ``void**`` buffers (along
+    with the buffers of any children) are what is allocated on the device.
+    The buffer values should be device pointers. The rest of the structure
+    should be accessible to the CPU.
+
+    The ``private_data`` and ``release`` callback of this structure should
+    contain any necessary information and structures related to freeing
+    the array according to the device it is allocated on, rather than
+    having a separate release callback and ``private_data`` pointer here.
+
+.. c:member:: int64_t ArrowDeviceArray.device_id
+
+    The device id to identify a specific device if multiple devices of this
+    type are on the system. The semantics of the id will be hardware dependent,
+    but we use an ``int64_t`` to future-proof the id as devices change over time.
+
+.. c:member:: ArrowDeviceType ArrowDeviceArray.device_type
+
+    The type of the device which can access the buffers in the array.
+
+.. c:member:: void* ArrowDeviceArray.sync_event
+
+    Optional. An event-like object to synchronize on if needed.
+
+    Many devices, like GPUs, are primarily asynchronous with respect to
+    CPU processing. As such in order to safely access memory, it is often
+    necessary to have an object to synchronize processing with. Since
+    different devices will use different types to specify this, we use a
+    void* which can be coerced into a pointer to whatever the device
+    appropriate type is.
+
+    If synchronization is not needed, this can be null. If this is non-null
+    then it MUST be used to call the appropriate sync method for the device
+    (e.g. ``cudaStreamWaitEvent`` or ``hipStreamWaitEvent``) before attempting
+    to access the memory in the buffers.
+
+    Expected types to coerce this ``void*`` to depending on the reported
+    device type:
+
+    * CUDA: ``cudaEvent_t*``
+    * ROCm: ``hipEvent_t*``
+    * OpenCL: ``cl_event*``
+    * Vulkan: ``VkEvent*``
+    * Metal: ``MTLEvent*``
+    * OneAPI: ``sycl::event*``
+
+    If an event is provided, then the producer MUST ensure that the event
+    is triggered/recorded at the end of the processing stream once the data
+    is considered available for use.
+
+
+.. c:member:: int64_t ArrowDeviceArray.reserved[3]
+
+    As non-CPU development expands, there may be a need to expand this
+    structure. In order to do so without potentially breaking ABI changes,
+    we reserve 24 bytes at the end of the object. This also has the added
+    benefit of bringing the total size of this structure to exactly 128
+    bytes (a power of 2) on 64-bit systems. These bytes should be zero'd
+    out after allocation in order to ensure safe evolution of the ABI in
+    the future.
+
+.. note::
+    Rather than store the shape / types of the data alongside the
+    ``ArrowDeviceArray``, users should utilize the existing ``ArrowSchema``
+    structure to pass any data type and shape information.
+
+Semantics
+=========
+
+Memory management
+-----------------
+
+The ``ArrowDeviceArray`` structure contains an ``ArrowArray`` object which
+itself has :ref:`specific semantics <c-data-interface-semantics>` for releasing
+memory. The term *"base structure"* below refers to the ``ArrowDeviceArray``
+object that is passed directly between the producer and consumer -- not any
+child structure thereof.
+
+It is intended for the base structure to be stack- or heap-allocated by the
+*consumer*. In this case, the producer API should take a pointer to the
+consumer-allocated structure.
+
+However, any data pointed to by the struct MUST be allocated and maintained
+by the producer. This includes the ``sync_event`` member if it is not null,
+along with any pointers in the ``ArrowArray`` object as usual. Data lifetime
+is managed through the ``release`` callback of the ``ArrowArray`` member.
+
+For an ``ArrowDeviceArray``, the semantics of a released structure and the
+callback semantics are identical to those for
+:ref:`ArrowArray itself <c-data-interface-released>`. Any producer specific context
+information necessary for releasing the device data buffers, in addition to
+any allocated event, should be stored in the ``private_data`` member of
+the ``ArrowArray`` and managed by the ``release`` callback.
+
+Moving an array
+'''''''''''''''
+
+The consumer can *move* the ``ArrowDeviceArray`` structure by bitwise copying
+or shallow member-wise copying. Then it MUST mark the source structure released
+by setting the ``release`` member of the embedded ``ArrowArray`` structure to
+``NULL``, but *without* calling that release callback. This ensures that only
+one live copy of the struct is active at any given time and that lifetime is
+correctly communicated to the producer.
+
+As usual, the release callback will be called on the destination structure
+when it is not needed anymore.
+
+Record batches
+--------------
+As with the C data interface itself, a record batch can be trivially considered
+as an equivalent struct array. In this case the metadata of the top-level
+``ArrowSchema`` can be used for schema-level metadata of the record batch.
+
+Mutability
+----------
+
+Both the producer and the consumer SHOULD consider the exported data (that
+is, the data reachable on the device through the ``buffers`` member of
+the embedded ``ArrowArray``) to be immutable, as either party could otherwise
+see inconsistent data while the other is mutating it.
+
+Likewise, if the ``sync_event`` member is non-NULL, the consumer should not
+attempt to access or read the data until they have synchronized on that event.
+
+C producer example
+====================
+
+Exporting a simple ``int32`` device array
+-----------------------------------------
+
+Export a non-nullable ``int32`` type with empty metadata. An example of this
+can be seen in the :ref:`C data interface docs directly <c-data-interface-export-int32-schema>`.
+
+To export the data itself, we transfer ownership to the consumer through
+the release callback. This example will use CUDA, but the equivalent calls
+could be used for any device:
+
+.. code-block:: c
+
+    static void release_int32_device_array(struct ArrowArray* array) {
+        assert(array->n_buffers == 2);
+        // destroy the event
+        cudaEvent_t* ev_ptr = reinterpret_cast<cudaEvent_t*>(array->private_data);
+        cudaError_t status = cudaEventDestroy(*ev_ptr);
+        assert(status == cudaSuccess);
+        free(ev_ptr);
+
+        // free the buffers and the buffers array
+        status = cudaFree(array->buffers[1]);
+        assert(status == cudaSuccess);
+        free(array->buffers);
+
+        // mark released
+        array->release = NULL;
+    }
+
+    __host__ void export_int32_device_array(void* cudaAllocdPtr,

Review Comment:
   If `__host__` required? If so, it should also be added to the release function above.



##########
docs/source/format/CDeviceDataInterface.rst:
##########
@@ -0,0 +1,628 @@
+.. Licensed to the Apache Software Foundation (ASF) under one
+.. or more contributor license agreements.  See the NOTICE file
+.. distributed with this work for additional information
+.. regarding copyright ownership.  The ASF licenses this file
+.. to you under the Apache License, Version 2.0 (the
+.. "License"); you may not use this file except in compliance
+.. with the License.  You may obtain a copy of the License at
+
+..   http://www.apache.org/licenses/LICENSE-2.0
+
+.. Unless required by applicable law or agreed to in writing,
+.. software distributed under the License is distributed on an
+.. "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+.. KIND, either express or implied.  See the License for the
+.. specific language governing permissions and limitations
+.. under the License.
+
+.. highlight:: c
+
+.. _c-device-data-interface:
+
+=================================
+The Arrow C Device data interface
+=================================
+
+.. note:: The Arrow C Device Data Interface should be considered experimental
+
+Rationale
+=========
+
+The current :ref:`C Data Interface <c-data-interface>`, and most
+implementations of it, make the assumption that all data buffers provided
+are CPU buffers. Since Apache Arrow is designed to be a universal in-memory
+format for representing tabular ("columnar") data, there will be the desire
+to leverage this data on non-CPU hardware such as GPUs. One example of such
+a case is the `RAPIDS cuDF library`_ which uses the Arrow memory format with
+CUDA for NVIDIA GPUs. Since copying data from host to device and back is
+expensive, the ideal would be to be able to leave the data on the device
+for as long as possible, even when passing it between runtimes and
+libraries.
+
+The Arrow C Device data interface builds on the existing C data interface
+by adding a very small, stable set of C definitions to it. These definitions
+are equivalents to the ``ArrowArray`` and ``ArrowArrayStream`` structures
+from the C Data Interface which add members to allow specifying the device
+type and pass necessary information to synchronize with the producer.
+For non-C/C++ languages and runtimes, translating the C definitions to
+corresponding C FFI declarations should be just as simple as with the
+current C data interface.
+
+Applications and libraries can then use Arrow schemas and Arrow formatted
+memory on non-CPU devices to exchange data just as easily as they do
+now with CPU data. This will enable leaving data on those devices longer
+and avoiding costly copies back and forth between the host and device
+just to leverage new libraries and runtimes.
+
+Goals
+-----
+
+* Expose an ABI-stable interface built on the existing C data interface.
+* Make it easy for third-party projects to implement support with little
+  initial investment.
+* Allow zero-copy sharing of Arrow formatted device memory between
+  independant runtimes and components running in the same process.
+* Avoid the need for one-to-one adaptation layers such as the
+  `CUDA Array Interface`_ for Python processes to pass CUDA data.
+* Enable integration without explicit dependencies (either at compile-time
+  or runtime) on the Arrow software project itself.
+
+The intent is for the Arrow C Device data interface to expand the reach
+of the current C data interface, allowing it to also become the standard
+low-level building block for columnar processing on devices like GPUs or
+FPGAs.
+
+Structure definitions
+=====================
+
+Because this is built on the C data interface, the C Device data interface
+uses the ``ArrowSchema`` and ``ArrowArray`` structures as defined in the
+:ref:`C data interface spec <c-data-interface-struct-defs>`. It then adds the
+following free-standing definitions. Like the rest of the Arrow project,
+they are available under the Apache License 2.0.
+
+.. code-block:: c
+
+    #ifndef ARROW_C_DEVICE_DATA_INTERFACE
+    #define ARROW_C_DEVICE_DATA_INTERFACE
+
+    // Device type for the allocated memory
+    typedef int32_t ArrowDeviceType;
+
+    // CPU device, same as using ArrowArray directly
+    #define ARROW_DEVICE_CPU 1
+    // CUDA GPU Device
+    #define ARROW_DEVICE_CUDA 2
+    // Pinned CUDA CPU memory by cudaMallocHost
+    #define ARROW_DEVICE_CUDA_HOST 3
+    // OpenCL Device
+    #define ARROW_DEVICE_OPENCL 4
+    // Vulkan buffer for next-gen graphics
+    #define ARROW_DEVICE_VULKAN 7
+    // Metal for Apple GPU
+    #define ARROW_DEVICE_METAL 8
+    // Verilog simulator buffer
+    #define ARROW_DEVICE_VPI 9
+    // ROCm GPUs for AMD GPUs
+    #define ARROW_DEVICE_ROCM 10
+    // Pinned ROCm CPU memory allocated by hipMallocHost
+    #define ARROW_DEVICE_ROCM_HOST 11
+    // Reserved for extension
+    //
+    // used to quickly test extension devices, semantics
+    // can differ based on implementation
+    #define ARROW_DEVICE_EXT_DEV 12
+    // CUDA managed/unified memory allocated by cudaMallocManaged
+    #define ARROW_DEVICE_CUDA_MANAGED 13
+    // Unified shared memory allocated on a oneAPI
+    // non-partitioned device.
+    //
+    // A call to the oneAPI runtime is required to determine the
+    // device type, the USM allocation type and the sycl context
+    // that it is bound to.
+    #define ARROW_DEVICE_ONEAPI 14
+    // GPU support for next-gen WebGPU standard
+    #define ARROW_DEVICE_WEBGPU 15
+    // Qualcomm Hexagon DSP
+    #define ARROW_DEVICE_HEXAGON 16
+
+    struct ArrowDeviceArray {
+      struct ArrowArray array;
+      int64_t device_id;
+      ArrowDeviceType device_type;
+      void* sync_event;
+
+      // reserved bytes for future expansion
+      int64_t reserved[3];
+    };
+
+    #endif  // ARROW_C_DEVICE_DATA_INTERFACE
+
+.. note::
+   The canonical guard ``ARROW_C_DEVICE_DATA_INTERFACE`` is meant to avoid
+   duplicate definitions if two projects copy the definitions in their own
+   headers, and a third-party project includes from these two projects. It
+   is therefore important that this guard is kept exactly as-is when these
+   definitions are copied.
+
+ArrowDeviceType
+---------------
+
+The ``ArrowDeviceType`` typedef is used to indicate what type of device the
+provided memory buffers were allocated on. This, in conjunction with the
+``device_id``, should be sufficient to reference the correct data buffers.
+
+We then use macros to define values for different device types. The provided
+macro values are compatible with the widely used `dlpack`_ ``DLDeviceType``
+definition values, using the same value for each as the equivalent
+``kDL<type>`` enum from dlpack.h. The list will be kept in sync with those
+equivalent enum values over time to ensure compatibility, rather than
+potentially diverging. To avoid the Arrow project having to be in the
+position of vetting new hardware devices, new additions should first be
+added to dlpack before we add a corresponding macro here.
+
+To ensure predictability with the ABI, we use macros instead of an ``enum``
+so the storage type is not compiler dependent.
+
+.. c:macro:: ARROW_DEVICE_CPU
+
+    CPU Device, equivalent to just using ``ArrowArray`` directly instead of
+    using ``ArrowDeviceArray``.
+
+.. c:macro:: ARROW_DEVICE_CUDA
+
+    A `CUDA`_ GPU Device. This could represent data allocated either with the
+    runtime library (``cudaMalloc``) or the device driver (``cuMemAlloc``).
+
+.. c:macro:: ARROW_DEVICE_CUDA_HOST
+
+    CPU memory that was pinned and page-locked by CUDA by using
+    ``cudaMallocHost`` or ``cuMemAllocHost``.
+
+.. c:macro:: ARROW_DEVICE_OPENCL
+
+    Data allocated on the device by using the `OpenCL (Open Computing Language)`_
+    framework.
+
+.. c:macro:: ARROW_DEVICE_VULKAN
+
+    Data allocated by the `Vulkan`_ framework and libraries.
+
+.. c:macro:: ARROW_DEVICE_METAL
+
+    Data on Apple GPU devices using the `Metal`_ framework and libraries.
+
+.. c:macro:: ARROW_DEVICE_VPI
+
+    Indicates usage of a Verilog simulator buffer.
+
+.. c:macro:: ARROW_DEVICE_ROCM
+
+    An AMD device using the `ROCm`_ stack.
+
+.. c:macro:: ARROW_DEVICE_ROCM_HOST
+
+    CPU memory pinned and page-locked allocated ``hipMallocHost``.
+
+.. c:macro:: ARROW_DEVICE_EXT_DEV
+
+    This value is an escape-hatch for devices to extend which aren't
+    currently represented otherwise. Producers would need to provide
+    additional information/context specific to the device if using
+    this device type. This is used to quickly test extension devices
+    and semantics can differ based on the implementation.
+
+.. c:macro:: ARROW_DEVICE_CUDA_MANAGED
+
+    CUDA managed/unified memory which is allocated by ``cudaMallocManaged``.
+
+.. c:macro:: ARROW_DEVICE_ONEAPI
+
+    Unified shared memory allocated on an Intel `oneAPI`_ non-partitioned
+    device. A call to the ``oneAPI`` runtime is required to determine
+    the specific device type, the USM allocation type and the sycl context
+    that it is bound to.
+
+.. c:macro:: ARROW_DEVICE_WEBGPU
+
+    GPU support for next-gen WebGPU standards
+
+.. c:macro:: ARROW_DEVICE_HEXAGON
+
+    Data allocated on a Qualcomm Hexagon DSP device.
+
+The ArrowDeviceArray structure
+------------------------------
+
+The ``ArrowDeviceArray`` structure embeds the C data ``ArrowArray`` structure
+and adds additional information necessary for consumers to use the data. It
+has the following fields:
+
+.. c:member:: struct ArrowArray ArrowDeviceArray.array
+
+    The allocated array data. The values in the ``void**`` buffers (along
+    with the buffers of any children) are what is allocated on the device.
+    The buffer values should be device pointers. The rest of the structure
+    should be accessible to the CPU.
+
+    The ``private_data`` and ``release`` callback of this structure should
+    contain any necessary information and structures related to freeing
+    the array according to the device it is allocated on, rather than
+    having a separate release callback and ``private_data`` pointer here.
+
+.. c:member:: int64_t ArrowDeviceArray.device_id
+
+    The device id to identify a specific device if multiple devices of this
+    type are on the system. The semantics of the id will be hardware dependent,
+    but we use an ``int64_t`` to future-proof the id as devices change over time.
+
+.. c:member:: ArrowDeviceType ArrowDeviceArray.device_type
+
+    The type of the device which can access the buffers in the array.
+
+.. c:member:: void* ArrowDeviceArray.sync_event
+
+    Optional. An event-like object to synchronize on if needed.
+
+    Many devices, like GPUs, are primarily asynchronous with respect to
+    CPU processing. As such in order to safely access memory, it is often
+    necessary to have an object to synchronize processing with. Since
+    different devices will use different types to specify this, we use a
+    void* which can be coerced into a pointer to whatever the device
+    appropriate type is.
+
+    If synchronization is not needed, this can be null. If this is non-null
+    then it MUST be used to call the appropriate sync method for the device
+    (e.g. ``cudaStreamWaitEvent`` or ``hipStreamWaitEvent``) before attempting
+    to access the memory in the buffers.
+
+    Expected types to coerce this ``void*`` to depending on the reported
+    device type:
+
+    * CUDA: ``cudaEvent_t*``
+    * ROCm: ``hipEvent_t*``
+    * OpenCL: ``cl_event*``
+    * Vulkan: ``VkEvent*``
+    * Metal: ``MTLEvent*``
+    * OneAPI: ``sycl::event*``
+
+    If an event is provided, then the producer MUST ensure that the event
+    is triggered/recorded at the end of the processing stream once the data
+    is considered available for use.
+
+
+.. c:member:: int64_t ArrowDeviceArray.reserved[3]
+
+    As non-CPU development expands, there may be a need to expand this
+    structure. In order to do so without potentially breaking ABI changes,
+    we reserve 24 bytes at the end of the object. This also has the added
+    benefit of bringing the total size of this structure to exactly 128
+    bytes (a power of 2) on 64-bit systems. These bytes should be zero'd
+    out after allocation in order to ensure safe evolution of the ABI in
+    the future.
+
+.. note::
+    Rather than store the shape / types of the data alongside the
+    ``ArrowDeviceArray``, users should utilize the existing ``ArrowSchema``
+    structure to pass any data type and shape information.
+
+Semantics
+=========
+
+Memory management
+-----------------
+
+The ``ArrowDeviceArray`` structure contains an ``ArrowArray`` object which
+itself has :ref:`specific semantics <c-data-interface-semantics>` for releasing
+memory. The term *"base structure"* below refers to the ``ArrowDeviceArray``
+object that is passed directly between the producer and consumer -- not any
+child structure thereof.
+
+It is intended for the base structure to be stack- or heap-allocated by the
+*consumer*. In this case, the producer API should take a pointer to the
+consumer-allocated structure.
+
+However, any data pointed to by the struct MUST be allocated and maintained
+by the producer. This includes the ``sync_event`` member if it is not null,
+along with any pointers in the ``ArrowArray`` object as usual. Data lifetime
+is managed through the ``release`` callback of the ``ArrowArray`` member.
+
+For an ``ArrowDeviceArray``, the semantics of a released structure and the
+callback semantics are identical to those for
+:ref:`ArrowArray itself <c-data-interface-released>`. Any producer specific context
+information necessary for releasing the device data buffers, in addition to
+any allocated event, should be stored in the ``private_data`` member of
+the ``ArrowArray`` and managed by the ``release`` callback.
+
+Moving an array
+'''''''''''''''
+
+The consumer can *move* the ``ArrowDeviceArray`` structure by bitwise copying
+or shallow member-wise copying. Then it MUST mark the source structure released
+by setting the ``release`` member of the embedded ``ArrowArray`` structure to
+``NULL``, but *without* calling that release callback. This ensures that only
+one live copy of the struct is active at any given time and that lifetime is
+correctly communicated to the producer.
+
+As usual, the release callback will be called on the destination structure
+when it is not needed anymore.
+
+Record batches
+--------------
+As with the C data interface itself, a record batch can be trivially considered
+as an equivalent struct array. In this case the metadata of the top-level
+``ArrowSchema`` can be used for schema-level metadata of the record batch.
+
+Mutability
+----------
+
+Both the producer and the consumer SHOULD consider the exported data (that
+is, the data reachable on the device through the ``buffers`` member of
+the embedded ``ArrowArray``) to be immutable, as either party could otherwise
+see inconsistent data while the other is mutating it.
+
+Likewise, if the ``sync_event`` member is non-NULL, the consumer should not
+attempt to access or read the data until they have synchronized on that event.
+
+C producer example
+====================
+
+Exporting a simple ``int32`` device array
+-----------------------------------------
+
+Export a non-nullable ``int32`` type with empty metadata. An example of this
+can be seen in the :ref:`C data interface docs directly <c-data-interface-export-int32-schema>`.
+
+To export the data itself, we transfer ownership to the consumer through
+the release callback. This example will use CUDA, but the equivalent calls
+could be used for any device:
+
+.. code-block:: c
+
+    static void release_int32_device_array(struct ArrowArray* array) {
+        assert(array->n_buffers == 2);
+        // destroy the event
+        cudaEvent_t* ev_ptr = reinterpret_cast<cudaEvent_t*>(array->private_data);
+        cudaError_t status = cudaEventDestroy(*ev_ptr);
+        assert(status == cudaSuccess);
+        free(ev_ptr);
+
+        // free the buffers and the buffers array
+        status = cudaFree(array->buffers[1]);
+        assert(status == cudaSuccess);
+        free(array->buffers);
+
+        // mark released
+        array->release = NULL;
+    }
+
+    __host__ void export_int32_device_array(void* cudaAllocdPtr,
+                                            cudaStream_t stream,
+                                            int64_t length,
+                                            struct ArrowDeviceArray* array) {
+        // get device id
+        int device;
+        cudaError_t status;
+        status = cudaGetDevice(&device);
+        assert(status == cudaSuccess);
+
+        cudaEvent_t* ev_ptr = (cudaEvent_t*)malloc(sizeof(cudaEvent_t));
+        assert(ev_ptr != NULL);
+        status = cudaEventCreate(ev_ptr);
+        assert(status == cudaSuccess);
+
+        // record event on the stream, assuming that the passed in
+        // stream is where the work to produce the data will be processing.
+        status = cudaEventRecord(*ev_ptr, stream);
+        assert(status == cudaSuccess);
+
+        // initialize fields
+        *array = (struct ArrowDeviceArray) {
+            .array = (struct ArrowArray) {
+                .length = length,
+                .null_count = 0,
+                .offset = 0,
+                .n_buffers = 2,
+                .n_children = 0,
+                .children = NULL,
+                .dictionary = NULL,
+                // bookeeping
+                .release = &release_int32_device_array,
+                .private_data = reinterpret_cast<void*>(ev_ptr),
+            },
+            .device_id = static_cast<int64_t>(device),
+            .device_type = ARROW_DEVICE_CUDA,
+            // pass the event pointer to the consumer
+            .sync_event = reinterpret_cast<void*>(ev_ptr),
+        };
+
+        // allocate list of buffers
+        array->array.buffers = (const void**)malloc(sizeof(void*) * array->array.n_buffers);
+        assert(array->array.buffers != NULL);
+        array->array.buffers[0] = NULL;
+        array->array.buffers[1] = cudaAllocdPtr;
+    }
+
+================

Review Comment:
   Formatting nit: should match over/underlining length with title length, otherwise I think reST won't like you :-)



##########
docs/source/format/CDeviceDataInterface.rst:
##########
@@ -0,0 +1,628 @@
+.. Licensed to the Apache Software Foundation (ASF) under one
+.. or more contributor license agreements.  See the NOTICE file
+.. distributed with this work for additional information
+.. regarding copyright ownership.  The ASF licenses this file
+.. to you under the Apache License, Version 2.0 (the
+.. "License"); you may not use this file except in compliance
+.. with the License.  You may obtain a copy of the License at
+
+..   http://www.apache.org/licenses/LICENSE-2.0
+
+.. Unless required by applicable law or agreed to in writing,
+.. software distributed under the License is distributed on an
+.. "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+.. KIND, either express or implied.  See the License for the
+.. specific language governing permissions and limitations
+.. under the License.
+
+.. highlight:: c
+
+.. _c-device-data-interface:
+
+=================================
+The Arrow C Device data interface
+=================================
+
+.. note:: The Arrow C Device Data Interface should be considered experimental
+
+Rationale
+=========
+
+The current :ref:`C Data Interface <c-data-interface>`, and most
+implementations of it, make the assumption that all data buffers provided
+are CPU buffers. Since Apache Arrow is designed to be a universal in-memory
+format for representing tabular ("columnar") data, there will be the desire
+to leverage this data on non-CPU hardware such as GPUs. One example of such
+a case is the `RAPIDS cuDF library`_ which uses the Arrow memory format with
+CUDA for NVIDIA GPUs. Since copying data from host to device and back is
+expensive, the ideal would be to be able to leave the data on the device
+for as long as possible, even when passing it between runtimes and
+libraries.
+
+The Arrow C Device data interface builds on the existing C data interface
+by adding a very small, stable set of C definitions to it. These definitions
+are equivalents to the ``ArrowArray`` and ``ArrowArrayStream`` structures
+from the C Data Interface which add members to allow specifying the device
+type and pass necessary information to synchronize with the producer.
+For non-C/C++ languages and runtimes, translating the C definitions to
+corresponding C FFI declarations should be just as simple as with the
+current C data interface.
+
+Applications and libraries can then use Arrow schemas and Arrow formatted
+memory on non-CPU devices to exchange data just as easily as they do
+now with CPU data. This will enable leaving data on those devices longer
+and avoiding costly copies back and forth between the host and device
+just to leverage new libraries and runtimes.
+
+Goals
+-----
+
+* Expose an ABI-stable interface built on the existing C data interface.
+* Make it easy for third-party projects to implement support with little
+  initial investment.
+* Allow zero-copy sharing of Arrow formatted device memory between
+  independant runtimes and components running in the same process.
+* Avoid the need for one-to-one adaptation layers such as the
+  `CUDA Array Interface`_ for Python processes to pass CUDA data.
+* Enable integration without explicit dependencies (either at compile-time
+  or runtime) on the Arrow software project itself.
+
+The intent is for the Arrow C Device data interface to expand the reach
+of the current C data interface, allowing it to also become the standard
+low-level building block for columnar processing on devices like GPUs or
+FPGAs.
+
+Structure definitions
+=====================
+
+Because this is built on the C data interface, the C Device data interface
+uses the ``ArrowSchema`` and ``ArrowArray`` structures as defined in the
+:ref:`C data interface spec <c-data-interface-struct-defs>`. It then adds the
+following free-standing definitions. Like the rest of the Arrow project,
+they are available under the Apache License 2.0.
+
+.. code-block:: c
+
+    #ifndef ARROW_C_DEVICE_DATA_INTERFACE
+    #define ARROW_C_DEVICE_DATA_INTERFACE
+
+    // Device type for the allocated memory
+    typedef int32_t ArrowDeviceType;
+
+    // CPU device, same as using ArrowArray directly
+    #define ARROW_DEVICE_CPU 1
+    // CUDA GPU Device
+    #define ARROW_DEVICE_CUDA 2
+    // Pinned CUDA CPU memory by cudaMallocHost
+    #define ARROW_DEVICE_CUDA_HOST 3
+    // OpenCL Device
+    #define ARROW_DEVICE_OPENCL 4
+    // Vulkan buffer for next-gen graphics
+    #define ARROW_DEVICE_VULKAN 7
+    // Metal for Apple GPU
+    #define ARROW_DEVICE_METAL 8
+    // Verilog simulator buffer
+    #define ARROW_DEVICE_VPI 9
+    // ROCm GPUs for AMD GPUs
+    #define ARROW_DEVICE_ROCM 10
+    // Pinned ROCm CPU memory allocated by hipMallocHost
+    #define ARROW_DEVICE_ROCM_HOST 11
+    // Reserved for extension
+    //
+    // used to quickly test extension devices, semantics
+    // can differ based on implementation
+    #define ARROW_DEVICE_EXT_DEV 12
+    // CUDA managed/unified memory allocated by cudaMallocManaged
+    #define ARROW_DEVICE_CUDA_MANAGED 13
+    // Unified shared memory allocated on a oneAPI
+    // non-partitioned device.
+    //
+    // A call to the oneAPI runtime is required to determine the
+    // device type, the USM allocation type and the sycl context
+    // that it is bound to.
+    #define ARROW_DEVICE_ONEAPI 14
+    // GPU support for next-gen WebGPU standard
+    #define ARROW_DEVICE_WEBGPU 15
+    // Qualcomm Hexagon DSP
+    #define ARROW_DEVICE_HEXAGON 16
+
+    struct ArrowDeviceArray {
+      struct ArrowArray array;
+      int64_t device_id;
+      ArrowDeviceType device_type;
+      void* sync_event;
+
+      // reserved bytes for future expansion
+      int64_t reserved[3];
+    };
+
+    #endif  // ARROW_C_DEVICE_DATA_INTERFACE
+
+.. note::
+   The canonical guard ``ARROW_C_DEVICE_DATA_INTERFACE`` is meant to avoid
+   duplicate definitions if two projects copy the definitions in their own
+   headers, and a third-party project includes from these two projects. It
+   is therefore important that this guard is kept exactly as-is when these
+   definitions are copied.
+
+ArrowDeviceType
+---------------
+
+The ``ArrowDeviceType`` typedef is used to indicate what type of device the
+provided memory buffers were allocated on. This, in conjunction with the
+``device_id``, should be sufficient to reference the correct data buffers.
+
+We then use macros to define values for different device types. The provided
+macro values are compatible with the widely used `dlpack`_ ``DLDeviceType``
+definition values, using the same value for each as the equivalent
+``kDL<type>`` enum from dlpack.h. The list will be kept in sync with those
+equivalent enum values over time to ensure compatibility, rather than
+potentially diverging. To avoid the Arrow project having to be in the
+position of vetting new hardware devices, new additions should first be
+added to dlpack before we add a corresponding macro here.
+
+To ensure predictability with the ABI, we use macros instead of an ``enum``
+so the storage type is not compiler dependent.
+
+.. c:macro:: ARROW_DEVICE_CPU
+
+    CPU Device, equivalent to just using ``ArrowArray`` directly instead of
+    using ``ArrowDeviceArray``.
+
+.. c:macro:: ARROW_DEVICE_CUDA
+
+    A `CUDA`_ GPU Device. This could represent data allocated either with the
+    runtime library (``cudaMalloc``) or the device driver (``cuMemAlloc``).
+
+.. c:macro:: ARROW_DEVICE_CUDA_HOST
+
+    CPU memory that was pinned and page-locked by CUDA by using
+    ``cudaMallocHost`` or ``cuMemAllocHost``.
+
+.. c:macro:: ARROW_DEVICE_OPENCL
+
+    Data allocated on the device by using the `OpenCL (Open Computing Language)`_
+    framework.
+
+.. c:macro:: ARROW_DEVICE_VULKAN
+
+    Data allocated by the `Vulkan`_ framework and libraries.
+
+.. c:macro:: ARROW_DEVICE_METAL
+
+    Data on Apple GPU devices using the `Metal`_ framework and libraries.
+
+.. c:macro:: ARROW_DEVICE_VPI
+
+    Indicates usage of a Verilog simulator buffer.
+
+.. c:macro:: ARROW_DEVICE_ROCM
+
+    An AMD device using the `ROCm`_ stack.
+
+.. c:macro:: ARROW_DEVICE_ROCM_HOST
+
+    CPU memory pinned and page-locked allocated ``hipMallocHost``.
+
+.. c:macro:: ARROW_DEVICE_EXT_DEV
+
+    This value is an escape-hatch for devices to extend which aren't
+    currently represented otherwise. Producers would need to provide
+    additional information/context specific to the device if using
+    this device type. This is used to quickly test extension devices
+    and semantics can differ based on the implementation.
+
+.. c:macro:: ARROW_DEVICE_CUDA_MANAGED
+
+    CUDA managed/unified memory which is allocated by ``cudaMallocManaged``.
+
+.. c:macro:: ARROW_DEVICE_ONEAPI
+
+    Unified shared memory allocated on an Intel `oneAPI`_ non-partitioned
+    device. A call to the ``oneAPI`` runtime is required to determine
+    the specific device type, the USM allocation type and the sycl context
+    that it is bound to.
+
+.. c:macro:: ARROW_DEVICE_WEBGPU
+
+    GPU support for next-gen WebGPU standards
+
+.. c:macro:: ARROW_DEVICE_HEXAGON
+
+    Data allocated on a Qualcomm Hexagon DSP device.
+
+The ArrowDeviceArray structure
+------------------------------
+
+The ``ArrowDeviceArray`` structure embeds the C data ``ArrowArray`` structure
+and adds additional information necessary for consumers to use the data. It
+has the following fields:
+
+.. c:member:: struct ArrowArray ArrowDeviceArray.array
+
+    The allocated array data. The values in the ``void**`` buffers (along
+    with the buffers of any children) are what is allocated on the device.
+    The buffer values should be device pointers. The rest of the structure
+    should be accessible to the CPU.
+
+    The ``private_data`` and ``release`` callback of this structure should
+    contain any necessary information and structures related to freeing
+    the array according to the device it is allocated on, rather than
+    having a separate release callback and ``private_data`` pointer here.
+
+.. c:member:: int64_t ArrowDeviceArray.device_id
+
+    The device id to identify a specific device if multiple devices of this
+    type are on the system. The semantics of the id will be hardware dependent,
+    but we use an ``int64_t`` to future-proof the id as devices change over time.
+
+.. c:member:: ArrowDeviceType ArrowDeviceArray.device_type
+
+    The type of the device which can access the buffers in the array.
+
+.. c:member:: void* ArrowDeviceArray.sync_event
+
+    Optional. An event-like object to synchronize on if needed.
+
+    Many devices, like GPUs, are primarily asynchronous with respect to
+    CPU processing. As such in order to safely access memory, it is often
+    necessary to have an object to synchronize processing with. Since
+    different devices will use different types to specify this, we use a
+    void* which can be coerced into a pointer to whatever the device
+    appropriate type is.
+
+    If synchronization is not needed, this can be null. If this is non-null
+    then it MUST be used to call the appropriate sync method for the device
+    (e.g. ``cudaStreamWaitEvent`` or ``hipStreamWaitEvent``) before attempting
+    to access the memory in the buffers.
+
+    Expected types to coerce this ``void*`` to depending on the reported
+    device type:
+
+    * CUDA: ``cudaEvent_t*``
+    * ROCm: ``hipEvent_t*``
+    * OpenCL: ``cl_event*``
+    * Vulkan: ``VkEvent*``
+    * Metal: ``MTLEvent*``
+    * OneAPI: ``sycl::event*``
+
+    If an event is provided, then the producer MUST ensure that the event
+    is triggered/recorded at the end of the processing stream once the data
+    is considered available for use.
+
+
+.. c:member:: int64_t ArrowDeviceArray.reserved[3]
+
+    As non-CPU development expands, there may be a need to expand this
+    structure. In order to do so without potentially breaking ABI changes,
+    we reserve 24 bytes at the end of the object. This also has the added
+    benefit of bringing the total size of this structure to exactly 128
+    bytes (a power of 2) on 64-bit systems. These bytes should be zero'd
+    out after allocation in order to ensure safe evolution of the ABI in
+    the future.
+
+.. note::
+    Rather than store the shape / types of the data alongside the
+    ``ArrowDeviceArray``, users should utilize the existing ``ArrowSchema``
+    structure to pass any data type and shape information.
+
+Semantics
+=========
+
+Memory management
+-----------------
+
+The ``ArrowDeviceArray`` structure contains an ``ArrowArray`` object which
+itself has :ref:`specific semantics <c-data-interface-semantics>` for releasing
+memory. The term *"base structure"* below refers to the ``ArrowDeviceArray``
+object that is passed directly between the producer and consumer -- not any
+child structure thereof.
+
+It is intended for the base structure to be stack- or heap-allocated by the
+*consumer*. In this case, the producer API should take a pointer to the
+consumer-allocated structure.
+
+However, any data pointed to by the struct MUST be allocated and maintained
+by the producer. This includes the ``sync_event`` member if it is not null,
+along with any pointers in the ``ArrowArray`` object as usual. Data lifetime
+is managed through the ``release`` callback of the ``ArrowArray`` member.
+
+For an ``ArrowDeviceArray``, the semantics of a released structure and the
+callback semantics are identical to those for
+:ref:`ArrowArray itself <c-data-interface-released>`. Any producer specific context
+information necessary for releasing the device data buffers, in addition to
+any allocated event, should be stored in the ``private_data`` member of
+the ``ArrowArray`` and managed by the ``release`` callback.
+
+Moving an array
+'''''''''''''''
+
+The consumer can *move* the ``ArrowDeviceArray`` structure by bitwise copying
+or shallow member-wise copying. Then it MUST mark the source structure released
+by setting the ``release`` member of the embedded ``ArrowArray`` structure to
+``NULL``, but *without* calling that release callback. This ensures that only
+one live copy of the struct is active at any given time and that lifetime is
+correctly communicated to the producer.
+
+As usual, the release callback will be called on the destination structure
+when it is not needed anymore.
+
+Record batches
+--------------
+As with the C data interface itself, a record batch can be trivially considered
+as an equivalent struct array. In this case the metadata of the top-level
+``ArrowSchema`` can be used for schema-level metadata of the record batch.
+
+Mutability
+----------
+
+Both the producer and the consumer SHOULD consider the exported data (that
+is, the data reachable on the device through the ``buffers`` member of
+the embedded ``ArrowArray``) to be immutable, as either party could otherwise
+see inconsistent data while the other is mutating it.
+
+Likewise, if the ``sync_event`` member is non-NULL, the consumer should not
+attempt to access or read the data until they have synchronized on that event.
+
+C producer example
+====================
+
+Exporting a simple ``int32`` device array
+-----------------------------------------
+
+Export a non-nullable ``int32`` type with empty metadata. An example of this
+can be seen in the :ref:`C data interface docs directly <c-data-interface-export-int32-schema>`.
+
+To export the data itself, we transfer ownership to the consumer through
+the release callback. This example will use CUDA, but the equivalent calls
+could be used for any device:
+
+.. code-block:: c
+
+    static void release_int32_device_array(struct ArrowArray* array) {
+        assert(array->n_buffers == 2);
+        // destroy the event
+        cudaEvent_t* ev_ptr = reinterpret_cast<cudaEvent_t*>(array->private_data);
+        cudaError_t status = cudaEventDestroy(*ev_ptr);
+        assert(status == cudaSuccess);
+        free(ev_ptr);
+
+        // free the buffers and the buffers array
+        status = cudaFree(array->buffers[1]);
+        assert(status == cudaSuccess);
+        free(array->buffers);
+
+        // mark released
+        array->release = NULL;
+    }
+
+    __host__ void export_int32_device_array(void* cudaAllocdPtr,
+                                            cudaStream_t stream,
+                                            int64_t length,
+                                            struct ArrowDeviceArray* array) {
+        // get device id
+        int device;
+        cudaError_t status;
+        status = cudaGetDevice(&device);
+        assert(status == cudaSuccess);
+
+        cudaEvent_t* ev_ptr = (cudaEvent_t*)malloc(sizeof(cudaEvent_t));
+        assert(ev_ptr != NULL);
+        status = cudaEventCreate(ev_ptr);
+        assert(status == cudaSuccess);
+
+        // record event on the stream, assuming that the passed in
+        // stream is where the work to produce the data will be processing.
+        status = cudaEventRecord(*ev_ptr, stream);
+        assert(status == cudaSuccess);
+
+        // initialize fields
+        *array = (struct ArrowDeviceArray) {
+            .array = (struct ArrowArray) {
+                .length = length,
+                .null_count = 0,
+                .offset = 0,
+                .n_buffers = 2,
+                .n_children = 0,
+                .children = NULL,
+                .dictionary = NULL,
+                // bookeeping
+                .release = &release_int32_device_array,
+                .private_data = reinterpret_cast<void*>(ev_ptr),
+            },
+            .device_id = static_cast<int64_t>(device),
+            .device_type = ARROW_DEVICE_CUDA,
+            // pass the event pointer to the consumer
+            .sync_event = reinterpret_cast<void*>(ev_ptr),
+        };
+
+        // allocate list of buffers
+        array->array.buffers = (const void**)malloc(sizeof(void*) * array->array.n_buffers);
+        assert(array->array.buffers != NULL);
+        array->array.buffers[0] = NULL;
+        array->array.buffers[1] = cudaAllocdPtr;
+    }
+
+================
+Device Stream Interface
+================
+
+Like the :ref:`C stream interface <c-stream-interface>`, the C Device data
+interface also specifies a higher-level structure for easing communication
+of streaming data within a single process. Defining an ``ArrowDeviceArrayStream``
+structure.
+
+Semantics
+=========
+
+An Arrow C device stream exposes a streaming source of data chunks, each with
+the same schema. Chunks are obtained by calling a blocking pull-style iteration
+function. It is expected that all chunks should be providing data on the same
+device type (but not necessarily the same device id). If it is necessary
+to provide a stream of data on multiple device types, a producer should
+provide a separate stream object for each device type.
+
+Structure definition
+====================
+
+The C device stream interface is defined by a single ``struct`` definition:
+
+.. code-block:: c
+
+    #ifndef ARROW_C_DEVICE_STREAM_INTERFACE
+    #define ARROW_C_DEVICE_STREAM_INTERFACE
+
+    struct ArrowDeviceArrayStream {
+        // device type that all arrays will be accessible from
+        ArrowDeviceType device_type;
+        // callbacks
+        int (*get_schema)(struct ArrowDeviceArrayStream*, struct ArrowSchema*);
+        int (*get_next)(struct ArrowDeviceArrayStream*, struct ArrowDeviceArray*);
+        const char* (*get_last_error)(struct ArrowDeviceArrayStream*);
+
+        // release callback
+        void (*release)(struct ArrowDeviceArrayStream*);
+
+        // opaque producer-specific data
+        void* private_data;
+    };
+
+    #endif  // ARROW_C_DEVICE_STREAM_INTERFACE
+
+.. note::
+    The canonical guard ``ARROW_C_DEVICE_STREAM_INTERFACE`` is meant to avoid
+    duplicate definitions if two projects copy the C device stream interface
+    definitions into their own headers, and a third-party project includes
+    from these two projects. It is therefore important that this guard is
+    kept exactly as-is when these definitions are copied.
+
+The ArrowDeviceArrayStream structure
+------------------------------------
+
+The ``ArrowDeviceArrayStream`` provides a device type that can access the
+resulting data along with the required callbacks to interact with a
+streaming source of Arrow arrays. It has the following fields:
+
+.. c:member:: ArrowDeviceType device_type
+
+    The device type that this stream produces data on. All
+    ``ArrowDeviceArray``s that are produced by this stream should have the
+    same device type as is set here. This is a convenience for the consumer
+    to not have to check every array that is retrieved and instead allows
+    higher-level coding constructs for streams.
+
+.. c:member:: int (*ArrowDeviceArrayStream.get_schema)(struct ArrowDeviceArrayStream*, struct ArrowSchema* out)
+
+    *Mandatory.* This callback allows the consumer to query the schema of

Review Comment:
   Do we want to just redirect to the equivalent definitions in the C Stream Interface spec?



##########
docs/source/format/CDeviceDataInterface.rst:
##########
@@ -0,0 +1,628 @@
+.. Licensed to the Apache Software Foundation (ASF) under one
+.. or more contributor license agreements.  See the NOTICE file
+.. distributed with this work for additional information
+.. regarding copyright ownership.  The ASF licenses this file
+.. to you under the Apache License, Version 2.0 (the
+.. "License"); you may not use this file except in compliance
+.. with the License.  You may obtain a copy of the License at
+
+..   http://www.apache.org/licenses/LICENSE-2.0
+
+.. Unless required by applicable law or agreed to in writing,
+.. software distributed under the License is distributed on an
+.. "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+.. KIND, either express or implied.  See the License for the
+.. specific language governing permissions and limitations
+.. under the License.
+
+.. highlight:: c
+
+.. _c-device-data-interface:
+
+=================================
+The Arrow C Device data interface
+=================================
+
+.. note:: The Arrow C Device Data Interface should be considered experimental
+
+Rationale
+=========
+
+The current :ref:`C Data Interface <c-data-interface>`, and most
+implementations of it, make the assumption that all data buffers provided
+are CPU buffers. Since Apache Arrow is designed to be a universal in-memory
+format for representing tabular ("columnar") data, there will be the desire
+to leverage this data on non-CPU hardware such as GPUs. One example of such
+a case is the `RAPIDS cuDF library`_ which uses the Arrow memory format with
+CUDA for NVIDIA GPUs. Since copying data from host to device and back is
+expensive, the ideal would be to be able to leave the data on the device
+for as long as possible, even when passing it between runtimes and
+libraries.
+
+The Arrow C Device data interface builds on the existing C data interface
+by adding a very small, stable set of C definitions to it. These definitions
+are equivalents to the ``ArrowArray`` and ``ArrowArrayStream`` structures
+from the C Data Interface which add members to allow specifying the device
+type and pass necessary information to synchronize with the producer.
+For non-C/C++ languages and runtimes, translating the C definitions to
+corresponding C FFI declarations should be just as simple as with the
+current C data interface.
+
+Applications and libraries can then use Arrow schemas and Arrow formatted
+memory on non-CPU devices to exchange data just as easily as they do
+now with CPU data. This will enable leaving data on those devices longer
+and avoiding costly copies back and forth between the host and device
+just to leverage new libraries and runtimes.
+
+Goals
+-----
+
+* Expose an ABI-stable interface built on the existing C data interface.
+* Make it easy for third-party projects to implement support with little
+  initial investment.
+* Allow zero-copy sharing of Arrow formatted device memory between
+  independant runtimes and components running in the same process.
+* Avoid the need for one-to-one adaptation layers such as the
+  `CUDA Array Interface`_ for Python processes to pass CUDA data.
+* Enable integration without explicit dependencies (either at compile-time
+  or runtime) on the Arrow software project itself.
+
+The intent is for the Arrow C Device data interface to expand the reach
+of the current C data interface, allowing it to also become the standard
+low-level building block for columnar processing on devices like GPUs or
+FPGAs.
+
+Structure definitions
+=====================
+
+Because this is built on the C data interface, the C Device data interface
+uses the ``ArrowSchema`` and ``ArrowArray`` structures as defined in the
+:ref:`C data interface spec <c-data-interface-struct-defs>`. It then adds the
+following free-standing definitions. Like the rest of the Arrow project,
+they are available under the Apache License 2.0.
+
+.. code-block:: c
+
+    #ifndef ARROW_C_DEVICE_DATA_INTERFACE
+    #define ARROW_C_DEVICE_DATA_INTERFACE
+
+    // Device type for the allocated memory
+    typedef int32_t ArrowDeviceType;
+
+    // CPU device, same as using ArrowArray directly
+    #define ARROW_DEVICE_CPU 1
+    // CUDA GPU Device
+    #define ARROW_DEVICE_CUDA 2
+    // Pinned CUDA CPU memory by cudaMallocHost
+    #define ARROW_DEVICE_CUDA_HOST 3
+    // OpenCL Device
+    #define ARROW_DEVICE_OPENCL 4
+    // Vulkan buffer for next-gen graphics
+    #define ARROW_DEVICE_VULKAN 7
+    // Metal for Apple GPU
+    #define ARROW_DEVICE_METAL 8
+    // Verilog simulator buffer
+    #define ARROW_DEVICE_VPI 9
+    // ROCm GPUs for AMD GPUs
+    #define ARROW_DEVICE_ROCM 10
+    // Pinned ROCm CPU memory allocated by hipMallocHost
+    #define ARROW_DEVICE_ROCM_HOST 11
+    // Reserved for extension
+    //
+    // used to quickly test extension devices, semantics
+    // can differ based on implementation
+    #define ARROW_DEVICE_EXT_DEV 12
+    // CUDA managed/unified memory allocated by cudaMallocManaged
+    #define ARROW_DEVICE_CUDA_MANAGED 13
+    // Unified shared memory allocated on a oneAPI
+    // non-partitioned device.
+    //
+    // A call to the oneAPI runtime is required to determine the
+    // device type, the USM allocation type and the sycl context
+    // that it is bound to.
+    #define ARROW_DEVICE_ONEAPI 14
+    // GPU support for next-gen WebGPU standard
+    #define ARROW_DEVICE_WEBGPU 15
+    // Qualcomm Hexagon DSP
+    #define ARROW_DEVICE_HEXAGON 16
+
+    struct ArrowDeviceArray {
+      struct ArrowArray array;
+      int64_t device_id;
+      ArrowDeviceType device_type;
+      void* sync_event;
+
+      // reserved bytes for future expansion
+      int64_t reserved[3];
+    };
+
+    #endif  // ARROW_C_DEVICE_DATA_INTERFACE
+
+.. note::
+   The canonical guard ``ARROW_C_DEVICE_DATA_INTERFACE`` is meant to avoid
+   duplicate definitions if two projects copy the definitions in their own
+   headers, and a third-party project includes from these two projects. It
+   is therefore important that this guard is kept exactly as-is when these
+   definitions are copied.
+
+ArrowDeviceType
+---------------
+
+The ``ArrowDeviceType`` typedef is used to indicate what type of device the
+provided memory buffers were allocated on. This, in conjunction with the
+``device_id``, should be sufficient to reference the correct data buffers.
+
+We then use macros to define values for different device types. The provided
+macro values are compatible with the widely used `dlpack`_ ``DLDeviceType``
+definition values, using the same value for each as the equivalent
+``kDL<type>`` enum from dlpack.h. The list will be kept in sync with those
+equivalent enum values over time to ensure compatibility, rather than
+potentially diverging. To avoid the Arrow project having to be in the
+position of vetting new hardware devices, new additions should first be
+added to dlpack before we add a corresponding macro here.
+
+To ensure predictability with the ABI, we use macros instead of an ``enum``
+so the storage type is not compiler dependent.
+
+.. c:macro:: ARROW_DEVICE_CPU
+
+    CPU Device, equivalent to just using ``ArrowArray`` directly instead of
+    using ``ArrowDeviceArray``.
+
+.. c:macro:: ARROW_DEVICE_CUDA
+
+    A `CUDA`_ GPU Device. This could represent data allocated either with the
+    runtime library (``cudaMalloc``) or the device driver (``cuMemAlloc``).
+
+.. c:macro:: ARROW_DEVICE_CUDA_HOST
+
+    CPU memory that was pinned and page-locked by CUDA by using
+    ``cudaMallocHost`` or ``cuMemAllocHost``.
+
+.. c:macro:: ARROW_DEVICE_OPENCL
+
+    Data allocated on the device by using the `OpenCL (Open Computing Language)`_
+    framework.
+
+.. c:macro:: ARROW_DEVICE_VULKAN
+
+    Data allocated by the `Vulkan`_ framework and libraries.
+
+.. c:macro:: ARROW_DEVICE_METAL
+
+    Data on Apple GPU devices using the `Metal`_ framework and libraries.
+
+.. c:macro:: ARROW_DEVICE_VPI
+
+    Indicates usage of a Verilog simulator buffer.
+
+.. c:macro:: ARROW_DEVICE_ROCM
+
+    An AMD device using the `ROCm`_ stack.
+
+.. c:macro:: ARROW_DEVICE_ROCM_HOST
+
+    CPU memory pinned and page-locked allocated ``hipMallocHost``.
+
+.. c:macro:: ARROW_DEVICE_EXT_DEV
+
+    This value is an escape-hatch for devices to extend which aren't
+    currently represented otherwise. Producers would need to provide
+    additional information/context specific to the device if using
+    this device type. This is used to quickly test extension devices
+    and semantics can differ based on the implementation.
+
+.. c:macro:: ARROW_DEVICE_CUDA_MANAGED
+
+    CUDA managed/unified memory which is allocated by ``cudaMallocManaged``.
+
+.. c:macro:: ARROW_DEVICE_ONEAPI
+
+    Unified shared memory allocated on an Intel `oneAPI`_ non-partitioned
+    device. A call to the ``oneAPI`` runtime is required to determine
+    the specific device type, the USM allocation type and the sycl context
+    that it is bound to.
+
+.. c:macro:: ARROW_DEVICE_WEBGPU
+
+    GPU support for next-gen WebGPU standards
+
+.. c:macro:: ARROW_DEVICE_HEXAGON
+
+    Data allocated on a Qualcomm Hexagon DSP device.
+
+The ArrowDeviceArray structure
+------------------------------
+
+The ``ArrowDeviceArray`` structure embeds the C data ``ArrowArray`` structure
+and adds additional information necessary for consumers to use the data. It
+has the following fields:
+
+.. c:member:: struct ArrowArray ArrowDeviceArray.array
+
+    The allocated array data. The values in the ``void**`` buffers (along
+    with the buffers of any children) are what is allocated on the device.
+    The buffer values should be device pointers. The rest of the structure
+    should be accessible to the CPU.
+
+    The ``private_data`` and ``release`` callback of this structure should
+    contain any necessary information and structures related to freeing
+    the array according to the device it is allocated on, rather than
+    having a separate release callback and ``private_data`` pointer here.
+
+.. c:member:: int64_t ArrowDeviceArray.device_id
+
+    The device id to identify a specific device if multiple devices of this
+    type are on the system. The semantics of the id will be hardware dependent,
+    but we use an ``int64_t`` to future-proof the id as devices change over time.
+
+.. c:member:: ArrowDeviceType ArrowDeviceArray.device_type
+
+    The type of the device which can access the buffers in the array.
+
+.. c:member:: void* ArrowDeviceArray.sync_event
+
+    Optional. An event-like object to synchronize on if needed.
+
+    Many devices, like GPUs, are primarily asynchronous with respect to
+    CPU processing. As such in order to safely access memory, it is often
+    necessary to have an object to synchronize processing with. Since
+    different devices will use different types to specify this, we use a
+    void* which can be coerced into a pointer to whatever the device
+    appropriate type is.
+
+    If synchronization is not needed, this can be null. If this is non-null
+    then it MUST be used to call the appropriate sync method for the device
+    (e.g. ``cudaStreamWaitEvent`` or ``hipStreamWaitEvent``) before attempting
+    to access the memory in the buffers.
+
+    Expected types to coerce this ``void*`` to depending on the reported
+    device type:
+
+    * CUDA: ``cudaEvent_t*``
+    * ROCm: ``hipEvent_t*``
+    * OpenCL: ``cl_event*``
+    * Vulkan: ``VkEvent*``
+    * Metal: ``MTLEvent*``
+    * OneAPI: ``sycl::event*``
+
+    If an event is provided, then the producer MUST ensure that the event
+    is triggered/recorded at the end of the processing stream once the data
+    is considered available for use.
+
+
+.. c:member:: int64_t ArrowDeviceArray.reserved[3]
+
+    As non-CPU development expands, there may be a need to expand this
+    structure. In order to do so without potentially breaking ABI changes,
+    we reserve 24 bytes at the end of the object. This also has the added
+    benefit of bringing the total size of this structure to exactly 128
+    bytes (a power of 2) on 64-bit systems. These bytes should be zero'd
+    out after allocation in order to ensure safe evolution of the ABI in
+    the future.

Review Comment:
   Also, to be clear, those should be zeroed by the producer (which is responsible for initializing the whole struct).



##########
cpp/src/arrow/c/abi.h:
##########
@@ -15,10 +15,30 @@
 // specific language governing permissions and limitations
 // under the License.
 
+/// \file abi.h Arrow C-Data Interface
+///
+/// The Arrow C-Data interface defines a very small, stable set
+/// of C definitions which can be easily copied into any project's
+/// source code and vendored to be used for columnar data interchange
+/// in the Arrow format. For non-C/C++ languages and runtimes,
+/// it should be almost as easy to translate the C definitions into
+/// the corresponding C FFI declarations.
+///
+/// Applications and libraries can therefore work with Arrow memory
+/// without necessarily using the Arrow libraries or reinventing
+/// the wheel. Developers can choose between tight integration
+/// with the Arrow software project or minimal integration with
+/// the Arrow format only.
+
 #pragma once
 
 #include <stdint.h>
 
+/// \defgroup Arrow C-Data Interface

Review Comment:
   Also, I'm not sure doxygen definitions are useful here, since the spec is described in prose. Perhaps give a link to the spec instead of detailed docstrings?



##########
docs/source/index.rst:
##########
@@ -79,6 +79,7 @@ target environment.**
    format/FlightSql
    format/Integration
    format/CDataInterface
+   format/CDeviceDataInterface

Review Comment:
   Would move this below as it includes the C device stream interface as well.



##########
docs/source/format/CDeviceDataInterface.rst:
##########
@@ -0,0 +1,628 @@
+.. Licensed to the Apache Software Foundation (ASF) under one
+.. or more contributor license agreements.  See the NOTICE file
+.. distributed with this work for additional information
+.. regarding copyright ownership.  The ASF licenses this file
+.. to you under the Apache License, Version 2.0 (the
+.. "License"); you may not use this file except in compliance
+.. with the License.  You may obtain a copy of the License at
+
+..   http://www.apache.org/licenses/LICENSE-2.0
+
+.. Unless required by applicable law or agreed to in writing,
+.. software distributed under the License is distributed on an
+.. "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+.. KIND, either express or implied.  See the License for the
+.. specific language governing permissions and limitations
+.. under the License.
+
+.. highlight:: c
+
+.. _c-device-data-interface:
+
+=================================
+The Arrow C Device data interface
+=================================
+
+.. note:: The Arrow C Device Data Interface should be considered experimental
+
+Rationale
+=========
+
+The current :ref:`C Data Interface <c-data-interface>`, and most
+implementations of it, make the assumption that all data buffers provided
+are CPU buffers. Since Apache Arrow is designed to be a universal in-memory
+format for representing tabular ("columnar") data, there will be the desire
+to leverage this data on non-CPU hardware such as GPUs. One example of such
+a case is the `RAPIDS cuDF library`_ which uses the Arrow memory format with
+CUDA for NVIDIA GPUs. Since copying data from host to device and back is
+expensive, the ideal would be to be able to leave the data on the device
+for as long as possible, even when passing it between runtimes and
+libraries.
+
+The Arrow C Device data interface builds on the existing C data interface
+by adding a very small, stable set of C definitions to it. These definitions
+are equivalents to the ``ArrowArray`` and ``ArrowArrayStream`` structures
+from the C Data Interface which add members to allow specifying the device
+type and pass necessary information to synchronize with the producer.
+For non-C/C++ languages and runtimes, translating the C definitions to
+corresponding C FFI declarations should be just as simple as with the
+current C data interface.
+
+Applications and libraries can then use Arrow schemas and Arrow formatted
+memory on non-CPU devices to exchange data just as easily as they do
+now with CPU data. This will enable leaving data on those devices longer
+and avoiding costly copies back and forth between the host and device
+just to leverage new libraries and runtimes.
+
+Goals
+-----
+
+* Expose an ABI-stable interface built on the existing C data interface.
+* Make it easy for third-party projects to implement support with little
+  initial investment.
+* Allow zero-copy sharing of Arrow formatted device memory between
+  independant runtimes and components running in the same process.
+* Avoid the need for one-to-one adaptation layers such as the
+  `CUDA Array Interface`_ for Python processes to pass CUDA data.
+* Enable integration without explicit dependencies (either at compile-time
+  or runtime) on the Arrow software project itself.
+
+The intent is for the Arrow C Device data interface to expand the reach
+of the current C data interface, allowing it to also become the standard
+low-level building block for columnar processing on devices like GPUs or
+FPGAs.
+
+Structure definitions
+=====================
+
+Because this is built on the C data interface, the C Device data interface
+uses the ``ArrowSchema`` and ``ArrowArray`` structures as defined in the
+:ref:`C data interface spec <c-data-interface-struct-defs>`. It then adds the
+following free-standing definitions. Like the rest of the Arrow project,
+they are available under the Apache License 2.0.
+
+.. code-block:: c
+
+    #ifndef ARROW_C_DEVICE_DATA_INTERFACE
+    #define ARROW_C_DEVICE_DATA_INTERFACE
+
+    // Device type for the allocated memory
+    typedef int32_t ArrowDeviceType;
+
+    // CPU device, same as using ArrowArray directly
+    #define ARROW_DEVICE_CPU 1
+    // CUDA GPU Device
+    #define ARROW_DEVICE_CUDA 2
+    // Pinned CUDA CPU memory by cudaMallocHost
+    #define ARROW_DEVICE_CUDA_HOST 3
+    // OpenCL Device
+    #define ARROW_DEVICE_OPENCL 4
+    // Vulkan buffer for next-gen graphics
+    #define ARROW_DEVICE_VULKAN 7
+    // Metal for Apple GPU
+    #define ARROW_DEVICE_METAL 8
+    // Verilog simulator buffer
+    #define ARROW_DEVICE_VPI 9
+    // ROCm GPUs for AMD GPUs
+    #define ARROW_DEVICE_ROCM 10
+    // Pinned ROCm CPU memory allocated by hipMallocHost
+    #define ARROW_DEVICE_ROCM_HOST 11
+    // Reserved for extension
+    //
+    // used to quickly test extension devices, semantics
+    // can differ based on implementation
+    #define ARROW_DEVICE_EXT_DEV 12
+    // CUDA managed/unified memory allocated by cudaMallocManaged
+    #define ARROW_DEVICE_CUDA_MANAGED 13
+    // Unified shared memory allocated on a oneAPI
+    // non-partitioned device.
+    //
+    // A call to the oneAPI runtime is required to determine the
+    // device type, the USM allocation type and the sycl context
+    // that it is bound to.
+    #define ARROW_DEVICE_ONEAPI 14
+    // GPU support for next-gen WebGPU standard
+    #define ARROW_DEVICE_WEBGPU 15
+    // Qualcomm Hexagon DSP
+    #define ARROW_DEVICE_HEXAGON 16
+
+    struct ArrowDeviceArray {
+      struct ArrowArray array;
+      int64_t device_id;
+      ArrowDeviceType device_type;
+      void* sync_event;
+
+      // reserved bytes for future expansion
+      int64_t reserved[3];
+    };
+
+    #endif  // ARROW_C_DEVICE_DATA_INTERFACE
+
+.. note::
+   The canonical guard ``ARROW_C_DEVICE_DATA_INTERFACE`` is meant to avoid
+   duplicate definitions if two projects copy the definitions in their own
+   headers, and a third-party project includes from these two projects. It
+   is therefore important that this guard is kept exactly as-is when these
+   definitions are copied.
+
+ArrowDeviceType
+---------------
+
+The ``ArrowDeviceType`` typedef is used to indicate what type of device the
+provided memory buffers were allocated on. This, in conjunction with the
+``device_id``, should be sufficient to reference the correct data buffers.
+
+We then use macros to define values for different device types. The provided
+macro values are compatible with the widely used `dlpack`_ ``DLDeviceType``
+definition values, using the same value for each as the equivalent
+``kDL<type>`` enum from dlpack.h. The list will be kept in sync with those
+equivalent enum values over time to ensure compatibility, rather than
+potentially diverging. To avoid the Arrow project having to be in the
+position of vetting new hardware devices, new additions should first be
+added to dlpack before we add a corresponding macro here.
+
+To ensure predictability with the ABI, we use macros instead of an ``enum``
+so the storage type is not compiler dependent.
+
+.. c:macro:: ARROW_DEVICE_CPU
+
+    CPU Device, equivalent to just using ``ArrowArray`` directly instead of
+    using ``ArrowDeviceArray``.
+
+.. c:macro:: ARROW_DEVICE_CUDA
+
+    A `CUDA`_ GPU Device. This could represent data allocated either with the
+    runtime library (``cudaMalloc``) or the device driver (``cuMemAlloc``).
+
+.. c:macro:: ARROW_DEVICE_CUDA_HOST
+
+    CPU memory that was pinned and page-locked by CUDA by using
+    ``cudaMallocHost`` or ``cuMemAllocHost``.
+
+.. c:macro:: ARROW_DEVICE_OPENCL
+
+    Data allocated on the device by using the `OpenCL (Open Computing Language)`_
+    framework.
+
+.. c:macro:: ARROW_DEVICE_VULKAN
+
+    Data allocated by the `Vulkan`_ framework and libraries.
+
+.. c:macro:: ARROW_DEVICE_METAL
+
+    Data on Apple GPU devices using the `Metal`_ framework and libraries.
+
+.. c:macro:: ARROW_DEVICE_VPI
+
+    Indicates usage of a Verilog simulator buffer.
+
+.. c:macro:: ARROW_DEVICE_ROCM
+
+    An AMD device using the `ROCm`_ stack.
+
+.. c:macro:: ARROW_DEVICE_ROCM_HOST
+
+    CPU memory pinned and page-locked allocated ``hipMallocHost``.
+
+.. c:macro:: ARROW_DEVICE_EXT_DEV
+
+    This value is an escape-hatch for devices to extend which aren't
+    currently represented otherwise. Producers would need to provide
+    additional information/context specific to the device if using
+    this device type. This is used to quickly test extension devices
+    and semantics can differ based on the implementation.
+
+.. c:macro:: ARROW_DEVICE_CUDA_MANAGED
+
+    CUDA managed/unified memory which is allocated by ``cudaMallocManaged``.
+
+.. c:macro:: ARROW_DEVICE_ONEAPI
+
+    Unified shared memory allocated on an Intel `oneAPI`_ non-partitioned
+    device. A call to the ``oneAPI`` runtime is required to determine
+    the specific device type, the USM allocation type and the sycl context
+    that it is bound to.
+
+.. c:macro:: ARROW_DEVICE_WEBGPU
+
+    GPU support for next-gen WebGPU standards
+
+.. c:macro:: ARROW_DEVICE_HEXAGON
+
+    Data allocated on a Qualcomm Hexagon DSP device.
+
+The ArrowDeviceArray structure
+------------------------------
+
+The ``ArrowDeviceArray`` structure embeds the C data ``ArrowArray`` structure
+and adds additional information necessary for consumers to use the data. It
+has the following fields:
+
+.. c:member:: struct ArrowArray ArrowDeviceArray.array
+
+    The allocated array data. The values in the ``void**`` buffers (along
+    with the buffers of any children) are what is allocated on the device.
+    The buffer values should be device pointers. The rest of the structure
+    should be accessible to the CPU.
+
+    The ``private_data`` and ``release`` callback of this structure should
+    contain any necessary information and structures related to freeing
+    the array according to the device it is allocated on, rather than
+    having a separate release callback and ``private_data`` pointer here.
+
+.. c:member:: int64_t ArrowDeviceArray.device_id
+
+    The device id to identify a specific device if multiple devices of this
+    type are on the system. The semantics of the id will be hardware dependent,
+    but we use an ``int64_t`` to future-proof the id as devices change over time.
+
+.. c:member:: ArrowDeviceType ArrowDeviceArray.device_type
+
+    The type of the device which can access the buffers in the array.
+
+.. c:member:: void* ArrowDeviceArray.sync_event
+
+    Optional. An event-like object to synchronize on if needed.
+
+    Many devices, like GPUs, are primarily asynchronous with respect to
+    CPU processing. As such in order to safely access memory, it is often
+    necessary to have an object to synchronize processing with. Since
+    different devices will use different types to specify this, we use a
+    void* which can be coerced into a pointer to whatever the device
+    appropriate type is.
+
+    If synchronization is not needed, this can be null. If this is non-null
+    then it MUST be used to call the appropriate sync method for the device
+    (e.g. ``cudaStreamWaitEvent`` or ``hipStreamWaitEvent``) before attempting
+    to access the memory in the buffers.
+
+    Expected types to coerce this ``void*`` to depending on the reported
+    device type:
+
+    * CUDA: ``cudaEvent_t*``
+    * ROCm: ``hipEvent_t*``
+    * OpenCL: ``cl_event*``
+    * Vulkan: ``VkEvent*``
+    * Metal: ``MTLEvent*``
+    * OneAPI: ``sycl::event*``
+
+    If an event is provided, then the producer MUST ensure that the event
+    is triggered/recorded at the end of the processing stream once the data
+    is considered available for use.
+
+
+.. c:member:: int64_t ArrowDeviceArray.reserved[3]
+
+    As non-CPU development expands, there may be a need to expand this
+    structure. In order to do so without potentially breaking ABI changes,
+    we reserve 24 bytes at the end of the object. This also has the added
+    benefit of bringing the total size of this structure to exactly 128
+    bytes (a power of 2) on 64-bit systems. These bytes should be zero'd
+    out after allocation in order to ensure safe evolution of the ABI in
+    the future.
+
+.. note::
+    Rather than store the shape / types of the data alongside the
+    ``ArrowDeviceArray``, users should utilize the existing ``ArrowSchema``
+    structure to pass any data type and shape information.
+
+Semantics
+=========
+
+Memory management
+-----------------
+
+The ``ArrowDeviceArray`` structure contains an ``ArrowArray`` object which
+itself has :ref:`specific semantics <c-data-interface-semantics>` for releasing
+memory. The term *"base structure"* below refers to the ``ArrowDeviceArray``
+object that is passed directly between the producer and consumer -- not any
+child structure thereof.
+
+It is intended for the base structure to be stack- or heap-allocated by the
+*consumer*. In this case, the producer API should take a pointer to the
+consumer-allocated structure.
+
+However, any data pointed to by the struct MUST be allocated and maintained
+by the producer. This includes the ``sync_event`` member if it is not null,
+along with any pointers in the ``ArrowArray`` object as usual. Data lifetime
+is managed through the ``release`` callback of the ``ArrowArray`` member.
+
+For an ``ArrowDeviceArray``, the semantics of a released structure and the
+callback semantics are identical to those for
+:ref:`ArrowArray itself <c-data-interface-released>`. Any producer specific context
+information necessary for releasing the device data buffers, in addition to
+any allocated event, should be stored in the ``private_data`` member of
+the ``ArrowArray`` and managed by the ``release`` callback.
+
+Moving an array
+'''''''''''''''
+
+The consumer can *move* the ``ArrowDeviceArray`` structure by bitwise copying
+or shallow member-wise copying. Then it MUST mark the source structure released
+by setting the ``release`` member of the embedded ``ArrowArray`` structure to
+``NULL``, but *without* calling that release callback. This ensures that only
+one live copy of the struct is active at any given time and that lifetime is
+correctly communicated to the producer.
+
+As usual, the release callback will be called on the destination structure
+when it is not needed anymore.
+
+Record batches
+--------------
+As with the C data interface itself, a record batch can be trivially considered
+as an equivalent struct array. In this case the metadata of the top-level
+``ArrowSchema`` can be used for schema-level metadata of the record batch.
+
+Mutability
+----------
+
+Both the producer and the consumer SHOULD consider the exported data (that
+is, the data reachable on the device through the ``buffers`` member of
+the embedded ``ArrowArray``) to be immutable, as either party could otherwise
+see inconsistent data while the other is mutating it.
+
+Likewise, if the ``sync_event`` member is non-NULL, the consumer should not
+attempt to access or read the data until they have synchronized on that event.
+
+C producer example
+====================
+
+Exporting a simple ``int32`` device array
+-----------------------------------------
+
+Export a non-nullable ``int32`` type with empty metadata. An example of this
+can be seen in the :ref:`C data interface docs directly <c-data-interface-export-int32-schema>`.
+
+To export the data itself, we transfer ownership to the consumer through
+the release callback. This example will use CUDA, but the equivalent calls
+could be used for any device:
+
+.. code-block:: c
+
+    static void release_int32_device_array(struct ArrowArray* array) {
+        assert(array->n_buffers == 2);
+        // destroy the event
+        cudaEvent_t* ev_ptr = reinterpret_cast<cudaEvent_t*>(array->private_data);

Review Comment:
   Also, this is sample C code, not C++, so should only use C-style casts (not `reinterpret_cast`).



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [arrow] pitrou commented on a diff in pull request #34972: GH-34971: [Format] Enhance C-Data API to support non-cpu cases

["pitrou (via GitHub)" <gi...@apache.org>]

pitrou commented on code in PR #34972:
URL: https://github.com/apache/arrow/pull/34972#discussion_r1163123179


##########
cpp/src/arrow/c/abi.h:
##########
@@ -106,6 +212,98 @@ struct ArrowArrayStream {
 
 #endif  // ARROW_C_STREAM_INTERFACE
 
+#ifndef ARROW_C_DEVICE_STREAM_INTERFACE
+#define ARROW_C_DEVICE_STREAM_INTERFACE
+
+/// \brief Equivalent to ArrowArrayStream, but for ArrowDeviceArrays.
+///
+/// This stream is intended to provide a stream of data on a single
+/// device, if a producer wants data to be produced on multiple devices
+/// then multiple streams should be provided. One per device.
+struct ArrowDeviceArrayStream {
+  /// \brief The device that this stream produces data on.
+  ///
+  /// All ArrowDeviceArrays that are produced by this
+  /// stream should have the same device_type as set
+  /// here. The device_type needs to be provided here
+  /// so that consumers can provide the correct type
+  /// of queue_ptr when calling get_next.
+  ArrowDeviceType device_type;
+
+  /// \brief Callback to get the stream schema
+  /// (will be the same for all arrays in the stream).
+  ///
+  /// If successful, the ArrowSchema must be released independantly from the stream.
+  /// The schema should be accessible via CPU memory.
+  ///
+  /// \param[in] self The ArrowDeviceArrayStream object itself
+  /// \param[out] out C struct to export the schema to
+  /// \return 0 if successful, an `errno`-compatible error code otherwise.
+  int (*get_schema)(struct ArrowDeviceArrayStream* self, struct ArrowSchema* out);
+
+  /// \brief Callback to get the device id for the next array.
+  ///
+  /// This is necessary so that the proper/correct stream pointer can be provided
+  /// to get_next.
+  ///
+  /// The next call to `get_next` should provide an ArrowDeviceArray whose
+  /// device_id matches what is provided here, and whose device_type is the
+  /// same as the device_type member of this stream.
+  ///
+  /// \param[in] self The ArrowDeviceArrayStream object itself
+  /// \param[out] out_device_id Pointer to be populated with the device id, must not be
+  /// null \return 0 if successful, an `errno`-compatible error code otherwise.
+  int (*get_next_device_id)(struct ArrowDeviceArrayStream* self, int64_t* out_device_id);
+
+  /// \brief Callback to get the next array
+  ///
+  /// If there is no error and the returned array has been released, the stream
+  /// has ended. If successful, the ArrowArray must be released independently
+  /// from the stream.
+  ///
+  /// Because different frameworks use different types to represent this, we
+  /// accept a void* which should then be reinterpreted into whatever the
+  /// appropriate type is (e.g. cudaStream_t) for use by the producer.
+  ///
+  /// \param[in] self The ArrowDeviceArrayStream object itself
+  /// \param[in] queue_ptr The appropriate queue, stream, or
+  /// equivalent object for the device that the data is allocated on
+  /// to indicate where the consumer wants the data to be accessible.
+  /// If queue_ptr is NULL then the default stream (e.g. CUDA stream 0)
+  /// should be used to ensure that the memory is accessible from any stream.

Review Comment:
   > This isn't possible for many frameworks as they manage streams internally and don't have an easy way to release or share ownership of a stream to be controlled in the release callback.
   
   Sorry, but why not?
   



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [arrow] zeroshade commented on a diff in pull request #34972: GH-34971: [Format] Enhance C-Data API to support non-cpu cases

["zeroshade (via GitHub)" <gi...@apache.org>]

zeroshade commented on code in PR #34972:
URL: https://github.com/apache/arrow/pull/34972#discussion_r1161909615


##########
cpp/src/arrow/c/abi.h:
##########
@@ -65,6 +65,69 @@ struct ArrowArray {
 
 #endif  // ARROW_C_DATA_INTERFACE
 
+#ifndef ARROW_C_DEVICE_DATA_INTERFACE
+#define ARROW_C_DEVICE_DATA_INTERFACE
+
+// ArrowDeviceType is compatible with dlpack DLDeviceType for portability
+// it uses the same values for each enum as the equivalent kDL<type> from dlpack.h
+#ifdef __cplusplus
+typedef enum : int32_t {
+#else
+typedef enum {
+#endif
+  // CPU device, same as using ArrowArray directly
+  kArrowCPU = 1,

Review Comment:
   I don't think so, it would be invalid for the device_type to be any value that is *not* one of the macro values, an `unset` should most likely be the CPU value. I can't think of a valid use case to have an `unset` value. 
   
   If someone can think of a valid use case for an "unset" value, i'm happy to add one. But i'd rather not add one for the sake of having one if it would technically be invalid for it to ever be used.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [arrow] zeroshade commented on a diff in pull request #34972: GH-34971: [Format] Enhance C-Data API to support non-cpu cases

["zeroshade (via GitHub)" <gi...@apache.org>]

zeroshade commented on code in PR #34972:
URL: https://github.com/apache/arrow/pull/34972#discussion_r1161907729


##########
cpp/src/arrow/c/abi.h:
##########
@@ -65,6 +65,69 @@ struct ArrowArray {
 
 #endif  // ARROW_C_DATA_INTERFACE
 
+#ifndef ARROW_C_DEVICE_DATA_INTERFACE
+#define ARROW_C_DEVICE_DATA_INTERFACE
+
+// ArrowDeviceType is compatible with dlpack DLDeviceType for portability
+// it uses the same values for each enum as the equivalent kDL<type> from dlpack.h
+#ifdef __cplusplus
+typedef enum : int32_t {
+#else
+typedef enum {
+#endif

Review Comment:
   updated to use macros



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [arrow] zeroshade commented on pull request #34972: GH-34971: [Format] Enhance C-Data API to support non-cpu cases

["zeroshade (via GitHub)" <gi...@apache.org>]

zeroshade commented on PR #34972:
URL: https://github.com/apache/arrow/pull/34972#issuecomment-1505735524

   @kkraus14 @pitrou Did you two reach a consensus on the question concerning the consumer passing of the stream/queue pointer into `get_next` or is this still an open question that needs to be discussed further?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [arrow] pitrou commented on pull request #34972: GH-34971: [Format] Enhance C-Data API to support non-cpu cases

["pitrou (via GitHub)" <gi...@apache.org>]

pitrou commented on PR #34972:
URL: https://github.com/apache/arrow/pull/34972#issuecomment-1505863408

   Hmm, since the primary example of a producer-supplied CUDA stream is the [CUDA Array Interface](https://numba.readthedocs.io/en/stable/cuda/cuda_array_interface.html#streams), perhaps it would be nice to have the Numba devs' feedback on consumer-supplied vs. producer-supplied.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [arrow] pitrou commented on a diff in pull request #34972: GH-34971: [Format] Enhance C-Data API to support non-cpu cases

["pitrou (via GitHub)" <gi...@apache.org>]

pitrou commented on code in PR #34972:
URL: https://github.com/apache/arrow/pull/34972#discussion_r1164398631


##########
cpp/src/arrow/c/abi.h:
##########
@@ -106,6 +212,98 @@ struct ArrowArrayStream {
 
 #endif  // ARROW_C_STREAM_INTERFACE
 
+#ifndef ARROW_C_DEVICE_STREAM_INTERFACE
+#define ARROW_C_DEVICE_STREAM_INTERFACE
+
+/// \brief Equivalent to ArrowArrayStream, but for ArrowDeviceArrays.
+///
+/// This stream is intended to provide a stream of data on a single
+/// device, if a producer wants data to be produced on multiple devices
+/// then multiple streams should be provided. One per device.
+struct ArrowDeviceArrayStream {
+  /// \brief The device that this stream produces data on.
+  ///
+  /// All ArrowDeviceArrays that are produced by this
+  /// stream should have the same device_type as set
+  /// here. The device_type needs to be provided here
+  /// so that consumers can provide the correct type
+  /// of queue_ptr when calling get_next.
+  ArrowDeviceType device_type;
+
+  /// \brief Callback to get the stream schema
+  /// (will be the same for all arrays in the stream).
+  ///
+  /// If successful, the ArrowSchema must be released independantly from the stream.
+  /// The schema should be accessible via CPU memory.
+  ///
+  /// \param[in] self The ArrowDeviceArrayStream object itself
+  /// \param[out] out C struct to export the schema to
+  /// \return 0 if successful, an `errno`-compatible error code otherwise.
+  int (*get_schema)(struct ArrowDeviceArrayStream* self, struct ArrowSchema* out);
+
+  /// \brief Callback to get the device id for the next array.
+  ///
+  /// This is necessary so that the proper/correct stream pointer can be provided
+  /// to get_next.
+  ///
+  /// The next call to `get_next` should provide an ArrowDeviceArray whose
+  /// device_id matches what is provided here, and whose device_type is the
+  /// same as the device_type member of this stream.
+  ///
+  /// \param[in] self The ArrowDeviceArrayStream object itself
+  /// \param[out] out_device_id Pointer to be populated with the device id, must not be
+  /// null \return 0 if successful, an `errno`-compatible error code otherwise.
+  int (*get_next_device_id)(struct ArrowDeviceArrayStream* self, int64_t* out_device_id);
+
+  /// \brief Callback to get the next array
+  ///
+  /// If there is no error and the returned array has been released, the stream
+  /// has ended. If successful, the ArrowArray must be released independently
+  /// from the stream.
+  ///
+  /// Because different frameworks use different types to represent this, we
+  /// accept a void* which should then be reinterpreted into whatever the
+  /// appropriate type is (e.g. cudaStream_t) for use by the producer.
+  ///
+  /// \param[in] self The ArrowDeviceArrayStream object itself
+  /// \param[in] queue_ptr The appropriate queue, stream, or
+  /// equivalent object for the device that the data is allocated on
+  /// to indicate where the consumer wants the data to be accessible.
+  /// If queue_ptr is NULL then the default stream (e.g. CUDA stream 0)
+  /// should be used to ensure that the memory is accessible from any stream.

Review Comment:
   > I tried to dig through Tensorflow's code to figure exactly how they're managing the lifetime of their streams but I'm not confident
   
   The fact that they're handling those lifetimes should be enough to plug a refcounting mechanism (or interact with the GC, in case of a managed language). This is already necessary to manage the lifetime of data exported through the C Data Interface.
   
   I understand that they might not _have_ a refcounting mechanism in place already, but that's basic engineering anyway.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [arrow] zeroshade commented on a diff in pull request #34972: GH-34971: [Format] Enhance C-Data API to support non-cpu cases

["zeroshade (via GitHub)" <gi...@apache.org>]

zeroshade commented on code in PR #34972:
URL: https://github.com/apache/arrow/pull/34972#discussion_r1191601631


##########
cpp/src/arrow/c/abi.h:
##########
@@ -65,6 +85,111 @@ struct ArrowArray {
 
 #endif  // ARROW_C_DATA_INTERFACE
 
+#ifndef ARROW_C_DEVICE_DATA_INTERFACE
+#define ARROW_C_DEVICE_DATA_INTERFACE
+
+/// \defgroup arrow-device-types Device Types
+/// These macros are compatible with the dlpack DLDeviceType values,
+/// using the same value for each enum as the equivalent kDL<type>
+/// from dlpack.h. This list should continue to be kept in sync with
+/// the equivalent dlpack.h enum values over time to ensure 
+/// compatibility, rather than potentially diverging.
+///
+/// To ensure predictability with the ABI we use macros instead of
+/// an enum so the storage type is not compiler dependent.
+///
+/// @{
+
+/// \brief DeviceType for the allocated memory
+typedef int32_t ArrowDeviceType;
+
+/// \brief CPU device, same as using ArrowArray directly
+#define ARROW_DEVICE_CPU = 1
+/// \brief CUDA GPU Device
+#define ARROW_DEVICE_CUDA = 2
+/// \brief Pinned CUDA CPU memory by cudaMallocHost
+#define ARROW_DEVICE_CUDA_HOST = 3
+/// \brief OpenCL Device
+#define ARROW_DEVICE_OPENCL = 4
+/// \brief Vulkan buffer for next-gen graphics
+#define ARROW_DEVICE_VULKAN = 7
+/// \brief Metal for Apple GPU
+#define ARROW_DEVICE_METAL = 8
+/// \brief Verilog simulator buffer
+#define ARROW_DEVICE_VPI = 9
+/// \brief ROCm GPUs for AMD GPUs
+#define ARROW_DEVICE_ROCM = 10
+/// \brief Pinned ROCm CPU memory allocated by hipMallocHost
+#define ARROW_DEVICE_ROCMHOST = 11
+/// \brief Reserved for extension
+///
+/// used to quickly test extension devices, semantics
+/// can differ based on the implementation
+#define ARROW_DEVICE_EXT_DEV = 12
+/// \brief CUDA managed/unified memory allocated by cudaMallocManaged
+#define ARROW_DEVICE_CUDA_MANAGED = 13
+/// \brief unified shared memory allocated on a oneAPI
+/// non-partitioned device.
+///
+/// A call to the oneAPI runtime is required to determine the device
+/// type, the USM allocation type, and the sycl context it is bound to.
+#define ARROW_DEVICE_ONEAPI = 14
+/// \brief GPU support for next-gen WebGPU standard
+#define ARROW_DEVICE_WEBGPU = 15
+/// \brief Qualcomm Hexagon DSP
+#define ARROW_DEVICE_HEXAGON = 16
+
+/// @}
+
+/// \brief Struct for passing an Arrow Array alongside
+/// device memory information.
+struct ArrowDeviceArray {
+  /// \brief the Allocated Array
+  ///
+  /// the buffers in the array (along with the buffers of any
+  /// children) are what is allocated on the device.
+  ///
+  /// the private_data and release callback of the arrow array
+  /// should contain any necessary information and structures
+  /// related to freeing the array according to the device it
+  /// is allocated on, rather than having a separate release
+  /// callback embedded here.
+  struct ArrowArray array;
+  /// \brief The device id to identify a specific device
+  /// if multiple of this type are on the system.
+  ///
+  /// the semantics of the id will be hardware dependant.
+  int64_t device_id;
+  /// \brief The type of device which can access this memory.
+  ArrowDeviceType device_type;
+  /// \brief An event-like object to synchronize on if needed.
+  ///
+  /// Many devices, like GPUs, are primarily asynchronous with
+  /// respect to CPU processing. As such in order to safely access
+  /// memory, it is often necessary to have an object to synchronize
+  /// processing on. Since different devices will use different types

Review Comment:
   currently about three lines below where this thread is (on line 172-173) I specify `cudaEvent_t` for CUDA and `hipEvent_t` for HIP. 
   
   @kkraus14 Do OpenCL/Vulkan/Metal/ROCM/OneAPI have equivalent structures that we could reference? (or any other device in the device list of enums)



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [arrow] zeroshade commented on a diff in pull request #34972: GH-34971: [Format] Enhance C-Data API to support non-cpu cases

["zeroshade (via GitHub)" <gi...@apache.org>]

zeroshade commented on code in PR #34972:
URL: https://github.com/apache/arrow/pull/34972#discussion_r1191319782


##########
cpp/src/arrow/c/abi.h:
##########
@@ -65,6 +85,111 @@ struct ArrowArray {
 
 #endif  // ARROW_C_DATA_INTERFACE
 
+#ifndef ARROW_C_DEVICE_DATA_INTERFACE
+#define ARROW_C_DEVICE_DATA_INTERFACE
+
+/// \defgroup arrow-device-types Device Types
+/// These macros are compatible with the dlpack DLDeviceType values,
+/// using the same value for each enum as the equivalent kDL<type>
+/// from dlpack.h. This list should continue to be kept in sync with
+/// the equivalent dlpack.h enum values over time to ensure 
+/// compatibility, rather than potentially diverging.
+///
+/// To ensure predictability with the ABI we use macros instead of
+/// an enum so the storage type is not compiler dependent.
+///
+/// @{
+
+/// \brief DeviceType for the allocated memory
+typedef int32_t ArrowDeviceType;
+
+/// \brief CPU device, same as using ArrowArray directly
+#define ARROW_DEVICE_CPU = 1
+/// \brief CUDA GPU Device
+#define ARROW_DEVICE_CUDA = 2
+/// \brief Pinned CUDA CPU memory by cudaMallocHost
+#define ARROW_DEVICE_CUDA_HOST = 3
+/// \brief OpenCL Device
+#define ARROW_DEVICE_OPENCL = 4
+/// \brief Vulkan buffer for next-gen graphics
+#define ARROW_DEVICE_VULKAN = 7
+/// \brief Metal for Apple GPU
+#define ARROW_DEVICE_METAL = 8
+/// \brief Verilog simulator buffer
+#define ARROW_DEVICE_VPI = 9
+/// \brief ROCm GPUs for AMD GPUs
+#define ARROW_DEVICE_ROCM = 10
+/// \brief Pinned ROCm CPU memory allocated by hipMallocHost
+#define ARROW_DEVICE_ROCMHOST = 11
+/// \brief Reserved for extension
+///
+/// used to quickly test extension devices, semantics
+/// can differ based on the implementation
+#define ARROW_DEVICE_EXT_DEV = 12
+/// \brief CUDA managed/unified memory allocated by cudaMallocManaged
+#define ARROW_DEVICE_CUDA_MANAGED = 13
+/// \brief unified shared memory allocated on a oneAPI
+/// non-partitioned device.
+///
+/// A call to the oneAPI runtime is required to determine the device
+/// type, the USM allocation type, and the sycl context it is bound to.
+#define ARROW_DEVICE_ONEAPI = 14
+/// \brief GPU support for next-gen WebGPU standard
+#define ARROW_DEVICE_WEBGPU = 15
+/// \brief Qualcomm Hexagon DSP
+#define ARROW_DEVICE_HEXAGON = 16
+
+/// @}
+
+/// \brief Struct for passing an Arrow Array alongside
+/// device memory information.
+struct ArrowDeviceArray {
+  /// \brief the Allocated Array
+  ///
+  /// the buffers in the array (along with the buffers of any
+  /// children) are what is allocated on the device.
+  ///
+  /// the private_data and release callback of the arrow array
+  /// should contain any necessary information and structures
+  /// related to freeing the array according to the device it
+  /// is allocated on, rather than having a separate release
+  /// callback embedded here.
+  struct ArrowArray array;
+  /// \brief The device id to identify a specific device
+  /// if multiple of this type are on the system.
+  ///
+  /// the semantics of the id will be hardware dependant.
+  int64_t device_id;
+  /// \brief The type of device which can access this memory.
+  ArrowDeviceType device_type;
+  /// \brief An event-like object to synchronize on if needed.
+  ///
+  /// Many devices, like GPUs, are primarily asynchronous with
+  /// respect to CPU processing. As such in order to safely access
+  /// memory, it is often necessary to have an object to synchronize
+  /// processing on. Since different devices will use different types

Review Comment:
   I don't think it's worth it to do so because the types might change and will likely not be able to be kept in sync with the proliferation of devices / frameworks. It might be better to let the producer confirm the type they are using? I'm not sure.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [arrow] zeroshade commented on a diff in pull request #34972: GH-34971: [Format] Enhance C-Data API to support non-cpu cases

["zeroshade (via GitHub)" <gi...@apache.org>]

zeroshade commented on code in PR #34972:
URL: https://github.com/apache/arrow/pull/34972#discussion_r1191402941


##########
cpp/src/arrow/c/abi.h:
##########
@@ -65,6 +85,111 @@ struct ArrowArray {
 
 #endif  // ARROW_C_DATA_INTERFACE
 
+#ifndef ARROW_C_DEVICE_DATA_INTERFACE
+#define ARROW_C_DEVICE_DATA_INTERFACE
+
+/// \defgroup arrow-device-types Device Types
+/// These macros are compatible with the dlpack DLDeviceType values,
+/// using the same value for each enum as the equivalent kDL<type>
+/// from dlpack.h. This list should continue to be kept in sync with
+/// the equivalent dlpack.h enum values over time to ensure 
+/// compatibility, rather than potentially diverging.
+///
+/// To ensure predictability with the ABI we use macros instead of
+/// an enum so the storage type is not compiler dependent.
+///
+/// @{
+
+/// \brief DeviceType for the allocated memory
+typedef int32_t ArrowDeviceType;
+
+/// \brief CPU device, same as using ArrowArray directly
+#define ARROW_DEVICE_CPU = 1
+/// \brief CUDA GPU Device
+#define ARROW_DEVICE_CUDA = 2
+/// \brief Pinned CUDA CPU memory by cudaMallocHost
+#define ARROW_DEVICE_CUDA_HOST = 3
+/// \brief OpenCL Device
+#define ARROW_DEVICE_OPENCL = 4
+/// \brief Vulkan buffer for next-gen graphics
+#define ARROW_DEVICE_VULKAN = 7
+/// \brief Metal for Apple GPU
+#define ARROW_DEVICE_METAL = 8
+/// \brief Verilog simulator buffer
+#define ARROW_DEVICE_VPI = 9
+/// \brief ROCm GPUs for AMD GPUs
+#define ARROW_DEVICE_ROCM = 10
+/// \brief Pinned ROCm CPU memory allocated by hipMallocHost
+#define ARROW_DEVICE_ROCMHOST = 11
+/// \brief Reserved for extension
+///
+/// used to quickly test extension devices, semantics
+/// can differ based on the implementation
+#define ARROW_DEVICE_EXT_DEV = 12
+/// \brief CUDA managed/unified memory allocated by cudaMallocManaged
+#define ARROW_DEVICE_CUDA_MANAGED = 13
+/// \brief unified shared memory allocated on a oneAPI
+/// non-partitioned device.
+///
+/// A call to the oneAPI runtime is required to determine the device
+/// type, the USM allocation type, and the sycl context it is bound to.
+#define ARROW_DEVICE_ONEAPI = 14
+/// \brief GPU support for next-gen WebGPU standard
+#define ARROW_DEVICE_WEBGPU = 15
+/// \brief Qualcomm Hexagon DSP
+#define ARROW_DEVICE_HEXAGON = 16
+
+/// @}
+
+/// \brief Struct for passing an Arrow Array alongside
+/// device memory information.
+struct ArrowDeviceArray {
+  /// \brief the Allocated Array
+  ///
+  /// the buffers in the array (along with the buffers of any
+  /// children) are what is allocated on the device.
+  ///
+  /// the private_data and release callback of the arrow array
+  /// should contain any necessary information and structures
+  /// related to freeing the array according to the device it
+  /// is allocated on, rather than having a separate release
+  /// callback embedded here.
+  struct ArrowArray array;
+  /// \brief The device id to identify a specific device
+  /// if multiple of this type are on the system.
+  ///
+  /// the semantics of the id will be hardware dependant.
+  int64_t device_id;
+  /// \brief The type of device which can access this memory.
+  ArrowDeviceType device_type;
+  /// \brief An event-like object to synchronize on if needed.
+  ///
+  /// Many devices, like GPUs, are primarily asynchronous with
+  /// respect to CPU processing. As such in order to safely access
+  /// memory, it is often necessary to have an object to synchronize
+  /// processing on. Since different devices will use different types

Review Comment:
   @kkraus14 you'd know better than I on this, what do you think here? Is it worthwhile to attempt to spell out the expected type for each device (that we can)? Do we have enough information about the devices that it makes sense to do so?



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [arrow] pitrou commented on a diff in pull request #34972: GH-34971: [Format] Enhance C-Data API to support non-cpu cases

["pitrou (via GitHub)" <gi...@apache.org>]

pitrou commented on code in PR #34972:
URL: https://github.com/apache/arrow/pull/34972#discussion_r1161079976


##########
cpp/src/arrow/c/abi.h:
##########
@@ -65,6 +65,69 @@ struct ArrowArray {
 
 #endif  // ARROW_C_DATA_INTERFACE
 
+#ifndef ARROW_C_DEVICE_DATA_INTERFACE
+#define ARROW_C_DEVICE_DATA_INTERFACE
+
+// ArrowDeviceType is compatible with dlpack DLDeviceType for portability
+// it uses the same values for each enum as the equivalent kDL<type> from dlpack.h
+#ifdef __cplusplus
+typedef enum : int32_t {
+#else
+typedef enum {
+#endif
+  // CPU device, same as using ArrowArray directly
+  kArrowCPU = 1,
+  // CUDA GPU Device
+  kArrowCUDA = 2,
+  // Pinned CUDA CPU memory by cudaMallocHost
+  kArrowCUDAHost = 3,
+  // OpenCL Device
+  kArrowOpenCL = 4,
+  // Vulkan buffer for next-gen graphics
+  kArrowVulkan = 7,
+  // Metal for Apple GPU
+  kArrowMetal = 8,
+  // Verilog simulator buffer
+  kArrowVPI = 9,
+  // ROCm GPUs for AMD GPUs
+  kArrowROCM = 10,
+  // Pinned ROCm CPU memory allocated by hipMallocHost
+  kArrowROCMHost = 11,
+  // Reserved for extension
+  // used to quickly test extension devices,
+  // semantics can differ based on the implementation
+  kArrowExtDev = 12,
+  // CUDA managed/unified memory allocated by cudaMallocManaged
+  kArrowCUDAManaged = 13,
+  // unified shared memory allocated on a oneAPI non-partitioned
+  // device. call to oneAPI runtime is required to determine the
+  // device type, the USM allocation type and the sycl context it
+  // is bound to
+  kArrowOneAPI = 14,
+  // GPU support for next-gen WebGPU standard
+  kArrowWebGPU = 15,
+  // Qualcomm Hexagon DSP
+  kArrowHexagon = 16,
+} ArrowDeviceType;
+
+struct ArrowDeviceArray {
+  // the private_date and release callback of the arrow array
+  // should contain any necessary information and structures
+  // related to freeing the array according to the device it
+  // is allocated on, rather than having a separate release
+  // callback embedded here.
+  struct ArrowArray* array;
+  int device_id;

Review Comment:
   Is an `int` sufficient, or do we want to be future-proof and make this `int64_t`?



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [arrow] kkraus14 commented on a diff in pull request #34972: GH-34971: [Format] Enhance C-Data API to support non-cpu cases

["kkraus14 (via GitHub)" <gi...@apache.org>]

kkraus14 commented on code in PR #34972:
URL: https://github.com/apache/arrow/pull/34972#discussion_r1164281700


##########
cpp/src/arrow/c/abi.h:
##########
@@ -106,6 +212,98 @@ struct ArrowArrayStream {
 
 #endif  // ARROW_C_STREAM_INTERFACE
 
+#ifndef ARROW_C_DEVICE_STREAM_INTERFACE
+#define ARROW_C_DEVICE_STREAM_INTERFACE
+
+/// \brief Equivalent to ArrowArrayStream, but for ArrowDeviceArrays.
+///
+/// This stream is intended to provide a stream of data on a single
+/// device, if a producer wants data to be produced on multiple devices
+/// then multiple streams should be provided. One per device.
+struct ArrowDeviceArrayStream {
+  /// \brief The device that this stream produces data on.
+  ///
+  /// All ArrowDeviceArrays that are produced by this
+  /// stream should have the same device_type as set
+  /// here. The device_type needs to be provided here
+  /// so that consumers can provide the correct type
+  /// of queue_ptr when calling get_next.
+  ArrowDeviceType device_type;
+
+  /// \brief Callback to get the stream schema
+  /// (will be the same for all arrays in the stream).
+  ///
+  /// If successful, the ArrowSchema must be released independantly from the stream.
+  /// The schema should be accessible via CPU memory.
+  ///
+  /// \param[in] self The ArrowDeviceArrayStream object itself
+  /// \param[out] out C struct to export the schema to
+  /// \return 0 if successful, an `errno`-compatible error code otherwise.
+  int (*get_schema)(struct ArrowDeviceArrayStream* self, struct ArrowSchema* out);
+
+  /// \brief Callback to get the device id for the next array.
+  ///
+  /// This is necessary so that the proper/correct stream pointer can be provided
+  /// to get_next.
+  ///
+  /// The next call to `get_next` should provide an ArrowDeviceArray whose
+  /// device_id matches what is provided here, and whose device_type is the
+  /// same as the device_type member of this stream.
+  ///
+  /// \param[in] self The ArrowDeviceArrayStream object itself
+  /// \param[out] out_device_id Pointer to be populated with the device id, must not be
+  /// null \return 0 if successful, an `errno`-compatible error code otherwise.
+  int (*get_next_device_id)(struct ArrowDeviceArrayStream* self, int64_t* out_device_id);
+
+  /// \brief Callback to get the next array
+  ///
+  /// If there is no error and the returned array has been released, the stream
+  /// has ended. If successful, the ArrowArray must be released independently
+  /// from the stream.
+  ///
+  /// Because different frameworks use different types to represent this, we
+  /// accept a void* which should then be reinterpreted into whatever the
+  /// appropriate type is (e.g. cudaStream_t) for use by the producer.
+  ///
+  /// \param[in] self The ArrowDeviceArrayStream object itself
+  /// \param[in] queue_ptr The appropriate queue, stream, or
+  /// equivalent object for the device that the data is allocated on
+  /// to indicate where the consumer wants the data to be accessible.
+  /// If queue_ptr is NULL then the default stream (e.g. CUDA stream 0)
+  /// should be used to ensure that the memory is accessible from any stream.

Review Comment:
   > Hmm, wait :-) If it's Tensorflow implementing the C Data Interface (rather than a third-party using Tensorflow as a backend), they can also implement the "producer-supplied CUDA stream" semantics, since they can access their own internals.
   
   They don't have a way to release / share ownership of their CUDA streams into the release callback.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [arrow] zeroshade commented on pull request #34972: GH-34971: [Format] Enhance C-Data API to support non-cpu cases

["zeroshade (via GitHub)" <gi...@apache.org>]

zeroshade commented on PR #34972:
URL: https://github.com/apache/arrow/pull/34972#issuecomment-1509208903

   so, @gmarkall @pitrou @kkraus14 do we have a consensus on the best way forward here? You three definitely have a lot more insight into this than I do, so I'll move forward with whichever approach you all can agree on :smile:


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [arrow] kkraus14 commented on a diff in pull request #34972: GH-34971: [Format] Enhance C-Data API to support non-cpu cases

["kkraus14 (via GitHub)" <gi...@apache.org>]

kkraus14 commented on code in PR #34972:
URL: https://github.com/apache/arrow/pull/34972#discussion_r1161855696


##########
cpp/src/arrow/c/abi.h:
##########
@@ -106,6 +169,77 @@ struct ArrowArrayStream {
 
 #endif  // ARROW_C_STREAM_INTERFACE
 
+#ifndef ARROW_C_DEVICE_STREAM_INTERFACE
+#define ARROW_C_DEVICE_STREAM_INTERFACE
+
+struct ArrowDeviceArrayStream {
+  // The device that this stream produces data on.
+  // All ArrowDeviceArrays that are produced by this
+  // stream should have the same device_type as set
+  // here. The device_type needs to be provided here
+  // so that consumers can provide the correct type
+  // of stream_ptr when calling get_next.
+  ArrowDeviceType device_type;
+
+  // Callback to get the stream schema
+  // (will be the same for all arrays in the stream).
+  //
+  // Return value: 0 if successful, an `errno`-compatible error code otherwise.
+  //
+  // If successful, the ArrowSchema must be released independently from the stream.
+  int (*get_schema)(struct ArrowDeviceArrayStream*, struct ArrowSchema* out);
+
+  // Callback to get the device id for the next array.
+  // This is necessary so that the proper/correct stream pointer can be provided
+  // to get_next. The parameter provided must not be null.
+  //
+  // Return value: 0 if successful, an `errno`-compatible error code otherwise.
+  //
+  // The next call to `get_next` should provide an ArrowDeviceArray whose
+  // device_id matches what is provided here, and whose device_type is the
+  // same as the device_type member of this stream.
+  int (*get_next_device_id)(struct ArrowDeviceArrayStream*, int* out_device_id);
+
+  // Callback to get the next array
+  // (if no error and the array is released, the stream has ended)
+  //
+  // the provided stream_ptr should be the appropriate stream, or

Review Comment:
   Same conversation going on here? https://github.com/apache/arrow/pull/34972/files#r1161080258



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [arrow] pitrou commented on a diff in pull request #34972: GH-34971: [Format] Enhance C-Data API to support non-cpu cases

["pitrou (via GitHub)" <gi...@apache.org>]

pitrou commented on code in PR #34972:
URL: https://github.com/apache/arrow/pull/34972#discussion_r1163063212


##########
cpp/src/arrow/c/abi.h:
##########
@@ -65,6 +65,69 @@ struct ArrowArray {
 
 #endif  // ARROW_C_DATA_INTERFACE
 
+#ifndef ARROW_C_DEVICE_DATA_INTERFACE
+#define ARROW_C_DEVICE_DATA_INTERFACE
+
+// ArrowDeviceType is compatible with dlpack DLDeviceType for portability
+// it uses the same values for each enum as the equivalent kDL<type> from dlpack.h
+#ifdef __cplusplus
+typedef enum : int32_t {
+#else
+typedef enum {
+#endif
+  // CPU device, same as using ArrowArray directly
+  kArrowCPU = 1,
+  // CUDA GPU Device
+  kArrowCUDA = 2,
+  // Pinned CUDA CPU memory by cudaMallocHost
+  kArrowCUDAHost = 3,
+  // OpenCL Device
+  kArrowOpenCL = 4,
+  // Vulkan buffer for next-gen graphics
+  kArrowVulkan = 7,
+  // Metal for Apple GPU
+  kArrowMetal = 8,
+  // Verilog simulator buffer
+  kArrowVPI = 9,
+  // ROCm GPUs for AMD GPUs
+  kArrowROCM = 10,
+  // Pinned ROCm CPU memory allocated by hipMallocHost
+  kArrowROCMHost = 11,
+  // Reserved for extension
+  // used to quickly test extension devices,
+  // semantics can differ based on the implementation
+  kArrowExtDev = 12,
+  // CUDA managed/unified memory allocated by cudaMallocManaged
+  kArrowCUDAManaged = 13,
+  // unified shared memory allocated on a oneAPI non-partitioned
+  // device. call to oneAPI runtime is required to determine the
+  // device type, the USM allocation type and the sycl context it
+  // is bound to
+  kArrowOneAPI = 14,
+  // GPU support for next-gen WebGPU standard
+  kArrowWebGPU = 15,
+  // Qualcomm Hexagon DSP
+  kArrowHexagon = 16,
+} ArrowDeviceType;
+
+struct ArrowDeviceArray {
+  // the private_date and release callback of the arrow array
+  // should contain any necessary information and structures
+  // related to freeing the array according to the device it
+  // is allocated on, rather than having a separate release
+  // callback embedded here.
+  struct ArrowArray* array;
+  int device_id;

Review Comment:
   +1 for `int64_t`. We don't know what future device APIs may choose here.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [arrow] pitrou commented on pull request #34972: GH-34971: [Format] Enhance C-Data API to support non-cpu cases

["pitrou (via GitHub)" <gi...@apache.org>]

pitrou commented on PR #34972:
URL: https://github.com/apache/arrow/pull/34972#issuecomment-1507166234

   Thanks @gmarkall for the detailed insight! (also, hi :-))
   
   > I also think it would also have been more difficult to have the consumer supply the stream for CAI, because it is just a property containing a dict
   
   This is also how the C Data Interface works, though of course it's a C struct rather than a Python dict.
   This is also why I would originally prefer the CUDA stream to be provider-supplied, because it can be part of that interface...
   
   So, to sum it up, we have:
   * in favour of the producer-supplied CUDA stream approach: allows to expose the CUDA stream in the C Data Interface (not only the C Stream Interface, which is more specialized - for example, it's sync not async)
   * in favour of the consumer-supplied CUDA stream approach: allows the consumer to ignore synchronization issues entirely, especially by passing a default stream
   
   Am I missing something in this bullet list?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [arrow] zeroshade commented on a diff in pull request #34972: GH-34971: [Format] Enhance C-Data API to support non-cpu cases

["zeroshade (via GitHub)" <gi...@apache.org>]

zeroshade commented on code in PR #34972:
URL: https://github.com/apache/arrow/pull/34972#discussion_r1161877839


##########
cpp/src/arrow/c/abi.h:
##########
@@ -65,6 +65,69 @@ struct ArrowArray {
 
 #endif  // ARROW_C_DATA_INTERFACE
 
+#ifndef ARROW_C_DEVICE_DATA_INTERFACE
+#define ARROW_C_DEVICE_DATA_INTERFACE
+
+// ArrowDeviceType is compatible with dlpack DLDeviceType for portability
+// it uses the same values for each enum as the equivalent kDL<type> from dlpack.h
+#ifdef __cplusplus
+typedef enum : int32_t {
+#else
+typedef enum {
+#endif

Review Comment:
   Ah, you're referring to the macros for the FLAGs. Fair enough, I'll switch these to macros so that the storage isn't compiler or language dependant.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [arrow] kkraus14 commented on a diff in pull request #34972: GH-34971: [Format] Enhance C-Data API to support non-cpu cases

["kkraus14 (via GitHub)" <gi...@apache.org>]

kkraus14 commented on code in PR #34972:
URL: https://github.com/apache/arrow/pull/34972#discussion_r1164340819


##########
cpp/src/arrow/c/abi.h:
##########
@@ -106,6 +212,98 @@ struct ArrowArrayStream {
 
 #endif  // ARROW_C_STREAM_INTERFACE
 
+#ifndef ARROW_C_DEVICE_STREAM_INTERFACE
+#define ARROW_C_DEVICE_STREAM_INTERFACE
+
+/// \brief Equivalent to ArrowArrayStream, but for ArrowDeviceArrays.
+///
+/// This stream is intended to provide a stream of data on a single
+/// device, if a producer wants data to be produced on multiple devices
+/// then multiple streams should be provided. One per device.
+struct ArrowDeviceArrayStream {
+  /// \brief The device that this stream produces data on.
+  ///
+  /// All ArrowDeviceArrays that are produced by this
+  /// stream should have the same device_type as set
+  /// here. The device_type needs to be provided here
+  /// so that consumers can provide the correct type
+  /// of queue_ptr when calling get_next.
+  ArrowDeviceType device_type;
+
+  /// \brief Callback to get the stream schema
+  /// (will be the same for all arrays in the stream).
+  ///
+  /// If successful, the ArrowSchema must be released independantly from the stream.
+  /// The schema should be accessible via CPU memory.
+  ///
+  /// \param[in] self The ArrowDeviceArrayStream object itself
+  /// \param[out] out C struct to export the schema to
+  /// \return 0 if successful, an `errno`-compatible error code otherwise.
+  int (*get_schema)(struct ArrowDeviceArrayStream* self, struct ArrowSchema* out);
+
+  /// \brief Callback to get the device id for the next array.
+  ///
+  /// This is necessary so that the proper/correct stream pointer can be provided
+  /// to get_next.
+  ///
+  /// The next call to `get_next` should provide an ArrowDeviceArray whose
+  /// device_id matches what is provided here, and whose device_type is the
+  /// same as the device_type member of this stream.
+  ///
+  /// \param[in] self The ArrowDeviceArrayStream object itself
+  /// \param[out] out_device_id Pointer to be populated with the device id, must not be
+  /// null \return 0 if successful, an `errno`-compatible error code otherwise.
+  int (*get_next_device_id)(struct ArrowDeviceArrayStream* self, int64_t* out_device_id);
+
+  /// \brief Callback to get the next array
+  ///
+  /// If there is no error and the returned array has been released, the stream
+  /// has ended. If successful, the ArrowArray must be released independently
+  /// from the stream.
+  ///
+  /// Because different frameworks use different types to represent this, we
+  /// accept a void* which should then be reinterpreted into whatever the
+  /// appropriate type is (e.g. cudaStream_t) for use by the producer.
+  ///
+  /// \param[in] self The ArrowDeviceArrayStream object itself
+  /// \param[in] queue_ptr The appropriate queue, stream, or
+  /// equivalent object for the device that the data is allocated on
+  /// to indicate where the consumer wants the data to be accessible.
+  /// If queue_ptr is NULL then the default stream (e.g. CUDA stream 0)
+  /// should be used to ensure that the memory is accessible from any stream.

Review Comment:
   > Why wouldn't they? They can easily refcount the usage of their own CUDA streams.
   
   I think that is making a lot of assumptions about how folks use and manage CUDA streams 😄. Again, some places use them similarly to thread pools and only control the lifetime of the pool.
   
   I tried to dig through Tensorflow's code to figure exactly how they're managing the lifetime of their streams but I'm not confident, everything I say below may not be correct:
   
   - Something eventually calls down to `AllocateStream` and `DeallocateStream` (https://github.com/tensorflow/tensorflow/blob/b9fc6a9b611ec373c02e5b5ab432b1d7aff9392e/tensorflow/compiler/xla/stream_executor/cuda/cuda_gpu_executor.cc#L759-L774) to create and destroy CUDA streams.
   - These operate on raw ptrs and it looks like there's a class that wraps these, `Stream` (https://github.com/tensorflow/tensorflow/blob/b9fc6a9b611ec373c02e5b5ab432b1d7aff9392e/tensorflow/compiler/xla/stream_executor/stream.cc#L262-L286) which has constructor, destructor, and init functions roughly of what you'd expect.
   - I believe these `Stream` objects are managed in a `StreamPool` (https://github.com/tensorflow/tensorflow/blob/b9fc6a9b611ec373c02e5b5ab432b1d7aff9392e/tensorflow/compiler/xla/service/stream_pool.h#L27-L59) which then allows "borrowing" the streams using unique ptrs.
   
   I guess in theory that if they ultimately have `Stream` objects being used that it could be moved into the private data being used by the release callback.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [arrow] oleksandr-pavlyk commented on pull request #34972: GH-34971: [Format] Enhance C-Data API to support non-cpu cases

["oleksandr-pavlyk (via GitHub)" <gi...@apache.org>]

oleksandr-pavlyk commented on PR #34972:
URL: https://github.com/apache/arrow/pull/34972#issuecomment-1520799743

   I assume we are talking about USM-based chunks. Dereferencing a USM pointer, generically, is only safe in kernels executed on the same device and using the same context that were used to perform the allocation. USM pointers of certain type (USM-shared and USM-host) can also be accessed from host. Deallocation of USM allocations requires a context.
   
   SYCL provides a mechanism to recover the device where the allocation was made from the pointer and the context (see `sycl::get_pointer_device`, [sec. 4.8.4](https://registry.khronos.org/SYCL/specs/sycl-2020/html/sycl-2020.html#_unified_shared_memory_pointer_queries)). Therefore, a USM allocation should be thought of as a tuple of a pointer and the associated SYCL context. 
   
   To share USM allocations across libraries (modules) in the same process space, the same context must be used. Two context instances constructed from the same arguments (either SYCL device, or device selector callable) are going to be different.
   
   To facilitate sharing USM allocations Intel has proposed an [extension](https://github.com/intel/llvm/blob/sycl/sycl/doc/extensions/supported/sycl_ext_oneapi_default_context.asciidoc) and implemented it in oneAPI DPC++ to define a canonical default context for each SYCL platform.
   
   This extension allows to use USM pointer and a SYCL device to describe USM allocations bound to the default platform context. 
   The SYCL device in this tuple can be represented by an integer, which is the position of the device in the vector of all devices known to SYCL runtime, obtained using `sycl::device::get_devices()`. This position is guaranteed to be stable (same for different modules in a process, and the same between process runs).
   
   Hence USM allocations (only if bound to the default platform context) can be shared using DLPack and represented in DLPack struct using a pointer and a integral device identifier. (DLPack support for USM allocations is implemented in [IntelPython/dpctl](https://github.com/IntelPython/dpctl/blob/master/dpctl/tensor/_dlpack.pyx)).
   
   I would therefore not recommend deleting information to reconstruct context from `ArrowDeviceArray` unless a Arrow were to restrict itself to heterogeneous systems in which SYCL only sees a single platform (i.e. only CUDA devices, only Level-Zero devices, or only HIP devices).
   
   To support sharing of USM allocations which may still be due to be accessed by scheduled, but as yet unfinished kernels, one can use SYCL events (one may need to support providing more than one events). Consumer would then gate kernels accessing shared allocations to avoid race conditions.
   
   @tchen suggested in https://github.com/dmlc/dlpack/issues/57#issuecomment-753696812 that instead of using events, a consumer would give a process a stream (or `sycl::queue`) that it would use to process shared USM allocation and the produce would insert a barrier gated by the events from submitted tasks that may yet modified the shared allocation. 
   
   Support for such barriers is yet to make it into the SYCL standard (see [this oneAPI extension](https://github.com/intel/llvm/blob/sycl/sycl/doc/extensions/supported/sycl_ext_oneapi_enqueue_barrier.asciidoc) which is implemented in oneAPI DPC++ compiler). Such mechanism is implemented in IntelPython/dpctl (see [here](https://github.com/IntelPython/dpctl/blob/master/dpctl/tensor/_usmarray.pyx#L934-L935)).
   
    


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [arrow] pitrou commented on a diff in pull request #34972: GH-34971: [Format] Enhance C-Data API to support non-cpu cases

["pitrou (via GitHub)" <gi...@apache.org>]

pitrou commented on code in PR #34972:
URL: https://github.com/apache/arrow/pull/34972#discussion_r1161080235


##########
cpp/src/arrow/c/abi.h:
##########
@@ -106,6 +169,77 @@ struct ArrowArrayStream {
 
 #endif  // ARROW_C_STREAM_INTERFACE
 
+#ifndef ARROW_C_DEVICE_STREAM_INTERFACE
+#define ARROW_C_DEVICE_STREAM_INTERFACE
+
+struct ArrowDeviceArrayStream {
+  // The device that this stream produces data on.
+  // All ArrowDeviceArrays that are produced by this
+  // stream should have the same device_type as set
+  // here. The device_type needs to be provided here
+  // so that consumers can provide the correct type
+  // of stream_ptr when calling get_next.
+  ArrowDeviceType device_type;
+
+  // Callback to get the stream schema
+  // (will be the same for all arrays in the stream).
+  //
+  // Return value: 0 if successful, an `errno`-compatible error code otherwise.
+  //
+  // If successful, the ArrowSchema must be released independently from the stream.
+  int (*get_schema)(struct ArrowDeviceArrayStream*, struct ArrowSchema* out);
+
+  // Callback to get the device id for the next array.
+  // This is necessary so that the proper/correct stream pointer can be provided
+  // to get_next. The parameter provided must not be null.
+  //
+  // Return value: 0 if successful, an `errno`-compatible error code otherwise.
+  //
+  // The next call to `get_next` should provide an ArrowDeviceArray whose
+  // device_id matches what is provided here, and whose device_type is the
+  // same as the device_type member of this stream.
+  int (*get_next_device_id)(struct ArrowDeviceArrayStream*, int* out_device_id);
+
+  // Callback to get the next array
+  // (if no error and the array is released, the stream has ended)
+  //
+  // the provided stream_ptr should be the appropriate stream, or
+  // equivalent object, for the device that the data is allocated on
+  // to indicate where the consumer wants the data to be accessible.

Review Comment:
   I'm not sure I understand, why would the consumer be able to choose where the data is accessible? If the data is already allocated (which is certainly a common case), how can the producer make it accessible to a new stream?



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [arrow] zeroshade commented on a diff in pull request #34972: GH-34971: [Format] Enhance C-Data API to support non-cpu cases

["zeroshade (via GitHub)" <gi...@apache.org>]

zeroshade commented on code in PR #34972:
URL: https://github.com/apache/arrow/pull/34972#discussion_r1161302439


##########
cpp/src/arrow/c/abi.h:
##########
@@ -65,6 +65,69 @@ struct ArrowArray {
 
 #endif  // ARROW_C_DATA_INTERFACE
 
+#ifndef ARROW_C_DEVICE_DATA_INTERFACE
+#define ARROW_C_DEVICE_DATA_INTERFACE
+
+// ArrowDeviceType is compatible with dlpack DLDeviceType for portability
+// it uses the same values for each enum as the equivalent kDL<type> from dlpack.h
+#ifdef __cplusplus
+typedef enum : int32_t {
+#else
+typedef enum {
+#endif
+  // CPU device, same as using ArrowArray directly
+  kArrowCPU = 1,
+  // CUDA GPU Device
+  kArrowCUDA = 2,
+  // Pinned CUDA CPU memory by cudaMallocHost
+  kArrowCUDAHost = 3,
+  // OpenCL Device
+  kArrowOpenCL = 4,
+  // Vulkan buffer for next-gen graphics
+  kArrowVulkan = 7,
+  // Metal for Apple GPU
+  kArrowMetal = 8,
+  // Verilog simulator buffer
+  kArrowVPI = 9,
+  // ROCm GPUs for AMD GPUs
+  kArrowROCM = 10,
+  // Pinned ROCm CPU memory allocated by hipMallocHost
+  kArrowROCMHost = 11,
+  // Reserved for extension
+  // used to quickly test extension devices,
+  // semantics can differ based on the implementation
+  kArrowExtDev = 12,
+  // CUDA managed/unified memory allocated by cudaMallocManaged
+  kArrowCUDAManaged = 13,
+  // unified shared memory allocated on a oneAPI non-partitioned
+  // device. call to oneAPI runtime is required to determine the
+  // device type, the USM allocation type and the sycl context it
+  // is bound to
+  kArrowOneAPI = 14,
+  // GPU support for next-gen WebGPU standard
+  kArrowWebGPU = 15,
+  // Qualcomm Hexagon DSP
+  kArrowHexagon = 16,
+} ArrowDeviceType;
+
+struct ArrowDeviceArray {
+  // the private_date and release callback of the arrow array
+  // should contain any necessary information and structures
+  // related to freeing the array according to the device it
+  // is allocated on, rather than having a separate release
+  // callback embedded here.
+  struct ArrowArray* array;
+  int device_id;
+  ArrowDeviceType device_type;
+  // reserve 128 bytes for future expansion
+  // of this struct as non-CPU development expands
+  // so that we can update without ABI breaking
+  // changes.

Review Comment:
   I'll update the comment to specify that they should be zeroed after allocating an instance of this struct 



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [arrow] lidavidm commented on a diff in pull request #34972: GH-34971: [Format] Enhance C-Data API to support non-cpu cases

["lidavidm (via GitHub)" <gi...@apache.org>]

lidavidm commented on code in PR #34972:
URL: https://github.com/apache/arrow/pull/34972#discussion_r1161362455


##########
cpp/src/arrow/c/abi.h:
##########
@@ -65,6 +65,69 @@ struct ArrowArray {
 
 #endif  // ARROW_C_DATA_INTERFACE
 
+#ifndef ARROW_C_DEVICE_DATA_INTERFACE
+#define ARROW_C_DEVICE_DATA_INTERFACE
+
+// ArrowDeviceType is compatible with dlpack DLDeviceType for portability
+// it uses the same values for each enum as the equivalent kDL<type> from dlpack.h
+#ifdef __cplusplus
+typedef enum : int32_t {
+#else
+typedef enum {
+#endif

Review Comment:
   I wasn't thinking ADBC - the C Data Interface already uses macros.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [arrow] ursabot commented on pull request #34972: GH-34971: [Format] Add non-CPU version of C Data Interface

["ursabot (via GitHub)" <gi...@apache.org>]

ursabot commented on PR #34972:
URL: https://github.com/apache/arrow/pull/34972#issuecomment-1580157148

   Benchmark runs are scheduled for baseline = 9fb8697dcb442f63317c7d6046393fb74842e0ae and contender = 105b9df0d504f25901cb89f49d365ec46b6de87b. 105b9df0d504f25901cb89f49d365ec46b6de87b is a master commit associated with this PR. Results will be available as each benchmark for each run completes.
   Conbench compare runs links:
   [Finished :arrow_down:0.0% :arrow_up:0.0%] [ec2-t3-xlarge-us-east-2](https://conbench.ursa.dev/compare/runs/ba2b77b590c14b0a87fca5aaffe31a80...a0931803d5d54a71aac36401a55b1400/)
   [Finished :arrow_down:0.62% :arrow_up:0.03%] [test-mac-arm](https://conbench.ursa.dev/compare/runs/9d19a8a4cef94079831e07155d024021...0d271d3f2c7c4c868e17056a95ada71e/)
   [Finished :arrow_down:0.33% :arrow_up:0.0%] [ursa-i9-9960x](https://conbench.ursa.dev/compare/runs/6e80e329cdee4813b155c5f4aa8c0d92...9fcfde8e2e27443e93939e74a9be1f6b/)
   [Failed :arrow_down:0.3% :arrow_up:0.0%] [ursa-thinkcentre-m75q](https://conbench.ursa.dev/compare/runs/7e34abd0f46d4a3296d0776a30bcf10f...78cb2217290e4fcdaba607c2ae16282d/)
   Buildkite builds:
   [Finished] [`105b9df0` ec2-t3-xlarge-us-east-2](https://buildkite.com/apache-arrow/arrow-bci-benchmark-on-ec2-t3-xlarge-us-east-2/builds/2989)
   [Finished] [`105b9df0` test-mac-arm](https://buildkite.com/apache-arrow/arrow-bci-benchmark-on-test-mac-arm/builds/3025)
   [Finished] [`105b9df0` ursa-i9-9960x](https://buildkite.com/apache-arrow/arrow-bci-benchmark-on-ursa-i9-9960x/builds/2990)
   [Failed] [`105b9df0` ursa-thinkcentre-m75q](https://buildkite.com/apache-arrow/arrow-bci-benchmark-on-ursa-thinkcentre-m75q/builds/3015)
   [Finished] [`9fb8697d` ec2-t3-xlarge-us-east-2](https://buildkite.com/apache-arrow/arrow-bci-benchmark-on-ec2-t3-xlarge-us-east-2/builds/2988)
   [Finished] [`9fb8697d` test-mac-arm](https://buildkite.com/apache-arrow/arrow-bci-benchmark-on-test-mac-arm/builds/3024)
   [Finished] [`9fb8697d` ursa-i9-9960x](https://buildkite.com/apache-arrow/arrow-bci-benchmark-on-ursa-i9-9960x/builds/2989)
   [Failed] [`9fb8697d` ursa-thinkcentre-m75q](https://buildkite.com/apache-arrow/arrow-bci-benchmark-on-ursa-thinkcentre-m75q/builds/3014)
   Supported benchmarks:
   ec2-t3-xlarge-us-east-2: Supported benchmark langs: Python, R. Runs only benchmarks with cloud = True
   test-mac-arm: Supported benchmark langs: C++, Python, R
   ursa-i9-9960x: Supported benchmark langs: Python, R, JavaScript
   ursa-thinkcentre-m75q: Supported benchmark langs: C++, Java
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [arrow] zeroshade commented on a diff in pull request #34972: GH-34971: [Format] Add non-CPU version of C Data Interface

["zeroshade (via GitHub)" <gi...@apache.org>]

zeroshade commented on code in PR #34972:
URL: https://github.com/apache/arrow/pull/34972#discussion_r1213454135


##########
docs/source/format/CDeviceDataInterface.rst:
##########
@@ -0,0 +1,628 @@
+.. Licensed to the Apache Software Foundation (ASF) under one
+.. or more contributor license agreements.  See the NOTICE file
+.. distributed with this work for additional information
+.. regarding copyright ownership.  The ASF licenses this file
+.. to you under the Apache License, Version 2.0 (the
+.. "License"); you may not use this file except in compliance
+.. with the License.  You may obtain a copy of the License at
+
+..   http://www.apache.org/licenses/LICENSE-2.0
+
+.. Unless required by applicable law or agreed to in writing,
+.. software distributed under the License is distributed on an
+.. "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+.. KIND, either express or implied.  See the License for the
+.. specific language governing permissions and limitations
+.. under the License.
+
+.. highlight:: c
+
+.. _c-device-data-interface:
+
+=================================
+The Arrow C Device data interface
+=================================
+
+.. note:: The Arrow C Device Data Interface should be considered experimental
+
+Rationale
+=========
+
+The current :ref:`C Data Interface <c-data-interface>`, and most
+implementations of it, make the assumption that all data buffers provided
+are CPU buffers. Since Apache Arrow is designed to be a universal in-memory
+format for representing tabular ("columnar") data, there will be the desire
+to leverage this data on non-CPU hardware such as GPUs. One example of such
+a case is the `RAPIDS cuDF library`_ which uses the Arrow memory format with
+CUDA for NVIDIA GPUs. Since copying data from host to device and back is
+expensive, the ideal would be to be able to leave the data on the device
+for as long as possible, even when passing it between runtimes and
+libraries.
+
+The Arrow C Device data interface builds on the existing C data interface
+by adding a very small, stable set of C definitions to it. These definitions
+are equivalents to the ``ArrowArray`` and ``ArrowArrayStream`` structures
+from the C Data Interface which add members to allow specifying the device
+type and pass necessary information to synchronize with the producer.
+For non-C/C++ languages and runtimes, translating the C definitions to
+corresponding C FFI declarations should be just as simple as with the
+current C data interface.
+
+Applications and libraries can then use Arrow schemas and Arrow formatted
+memory on non-CPU devices to exchange data just as easily as they do
+now with CPU data. This will enable leaving data on those devices longer
+and avoiding costly copies back and forth between the host and device
+just to leverage new libraries and runtimes.
+
+Goals
+-----
+
+* Expose an ABI-stable interface built on the existing C data interface.
+* Make it easy for third-party projects to implement support with little
+  initial investment.
+* Allow zero-copy sharing of Arrow formatted device memory between
+  independant runtimes and components running in the same process.
+* Avoid the need for one-to-one adaptation layers such as the
+  `CUDA Array Interface`_ for Python processes to pass CUDA data.
+* Enable integration without explicit dependencies (either at compile-time
+  or runtime) on the Arrow software project itself.
+
+The intent is for the Arrow C Device data interface to expand the reach
+of the current C data interface, allowing it to also become the standard
+low-level building block for columnar processing on devices like GPUs or
+FPGAs.
+
+Structure definitions
+=====================
+
+Because this is built on the C data interface, the C Device data interface
+uses the ``ArrowSchema`` and ``ArrowArray`` structures as defined in the
+:ref:`C data interface spec <c-data-interface-struct-defs>`. It then adds the
+following free-standing definitions. Like the rest of the Arrow project,
+they are available under the Apache License 2.0.
+
+.. code-block:: c
+
+    #ifndef ARROW_C_DEVICE_DATA_INTERFACE
+    #define ARROW_C_DEVICE_DATA_INTERFACE
+
+    // Device type for the allocated memory
+    typedef int32_t ArrowDeviceType;
+
+    // CPU device, same as using ArrowArray directly
+    #define ARROW_DEVICE_CPU 1
+    // CUDA GPU Device
+    #define ARROW_DEVICE_CUDA 2
+    // Pinned CUDA CPU memory by cudaMallocHost
+    #define ARROW_DEVICE_CUDA_HOST 3
+    // OpenCL Device
+    #define ARROW_DEVICE_OPENCL 4
+    // Vulkan buffer for next-gen graphics
+    #define ARROW_DEVICE_VULKAN 7
+    // Metal for Apple GPU
+    #define ARROW_DEVICE_METAL 8
+    // Verilog simulator buffer
+    #define ARROW_DEVICE_VPI 9
+    // ROCm GPUs for AMD GPUs
+    #define ARROW_DEVICE_ROCM 10
+    // Pinned ROCm CPU memory allocated by hipMallocHost
+    #define ARROW_DEVICE_ROCM_HOST 11
+    // Reserved for extension
+    //
+    // used to quickly test extension devices, semantics
+    // can differ based on implementation
+    #define ARROW_DEVICE_EXT_DEV 12
+    // CUDA managed/unified memory allocated by cudaMallocManaged
+    #define ARROW_DEVICE_CUDA_MANAGED 13
+    // Unified shared memory allocated on a oneAPI
+    // non-partitioned device.
+    //
+    // A call to the oneAPI runtime is required to determine the
+    // device type, the USM allocation type and the sycl context
+    // that it is bound to.
+    #define ARROW_DEVICE_ONEAPI 14
+    // GPU support for next-gen WebGPU standard
+    #define ARROW_DEVICE_WEBGPU 15
+    // Qualcomm Hexagon DSP
+    #define ARROW_DEVICE_HEXAGON 16
+
+    struct ArrowDeviceArray {
+      struct ArrowArray array;
+      int64_t device_id;
+      ArrowDeviceType device_type;
+      void* sync_event;
+
+      // reserved bytes for future expansion
+      int64_t reserved[3];
+    };
+
+    #endif  // ARROW_C_DEVICE_DATA_INTERFACE
+
+.. note::
+   The canonical guard ``ARROW_C_DEVICE_DATA_INTERFACE`` is meant to avoid
+   duplicate definitions if two projects copy the definitions in their own
+   headers, and a third-party project includes from these two projects. It
+   is therefore important that this guard is kept exactly as-is when these
+   definitions are copied.
+
+ArrowDeviceType
+---------------
+
+The ``ArrowDeviceType`` typedef is used to indicate what type of device the
+provided memory buffers were allocated on. This, in conjunction with the
+``device_id``, should be sufficient to reference the correct data buffers.
+
+We then use macros to define values for different device types. The provided
+macro values are compatible with the widely used `dlpack`_ ``DLDeviceType``
+definition values, using the same value for each as the equivalent
+``kDL<type>`` enum from dlpack.h. The list will be kept in sync with those
+equivalent enum values over time to ensure compatibility, rather than
+potentially diverging. To avoid the Arrow project having to be in the
+position of vetting new hardware devices, new additions should first be
+added to dlpack before we add a corresponding macro here.
+
+To ensure predictability with the ABI, we use macros instead of an ``enum``
+so the storage type is not compiler dependent.
+
+.. c:macro:: ARROW_DEVICE_CPU
+
+    CPU Device, equivalent to just using ``ArrowArray`` directly instead of
+    using ``ArrowDeviceArray``.
+
+.. c:macro:: ARROW_DEVICE_CUDA
+
+    A `CUDA`_ GPU Device. This could represent data allocated either with the
+    runtime library (``cudaMalloc``) or the device driver (``cuMemAlloc``).
+
+.. c:macro:: ARROW_DEVICE_CUDA_HOST
+
+    CPU memory that was pinned and page-locked by CUDA by using
+    ``cudaMallocHost`` or ``cuMemAllocHost``.
+
+.. c:macro:: ARROW_DEVICE_OPENCL
+
+    Data allocated on the device by using the `OpenCL (Open Computing Language)`_
+    framework.
+
+.. c:macro:: ARROW_DEVICE_VULKAN
+
+    Data allocated by the `Vulkan`_ framework and libraries.
+
+.. c:macro:: ARROW_DEVICE_METAL
+
+    Data on Apple GPU devices using the `Metal`_ framework and libraries.
+
+.. c:macro:: ARROW_DEVICE_VPI
+
+    Indicates usage of a Verilog simulator buffer.
+
+.. c:macro:: ARROW_DEVICE_ROCM
+
+    An AMD device using the `ROCm`_ stack.
+
+.. c:macro:: ARROW_DEVICE_ROCM_HOST
+
+    CPU memory pinned and page-locked allocated ``hipMallocHost``.
+
+.. c:macro:: ARROW_DEVICE_EXT_DEV
+
+    This value is an escape-hatch for devices to extend which aren't
+    currently represented otherwise. Producers would need to provide
+    additional information/context specific to the device if using
+    this device type. This is used to quickly test extension devices
+    and semantics can differ based on the implementation.
+
+.. c:macro:: ARROW_DEVICE_CUDA_MANAGED
+
+    CUDA managed/unified memory which is allocated by ``cudaMallocManaged``.
+
+.. c:macro:: ARROW_DEVICE_ONEAPI
+
+    Unified shared memory allocated on an Intel `oneAPI`_ non-partitioned
+    device. A call to the ``oneAPI`` runtime is required to determine
+    the specific device type, the USM allocation type and the sycl context
+    that it is bound to.
+
+.. c:macro:: ARROW_DEVICE_WEBGPU
+
+    GPU support for next-gen WebGPU standards
+
+.. c:macro:: ARROW_DEVICE_HEXAGON
+
+    Data allocated on a Qualcomm Hexagon DSP device.
+
+The ArrowDeviceArray structure
+------------------------------
+
+The ``ArrowDeviceArray`` structure embeds the C data ``ArrowArray`` structure
+and adds additional information necessary for consumers to use the data. It
+has the following fields:
+
+.. c:member:: struct ArrowArray ArrowDeviceArray.array
+
+    The allocated array data. The values in the ``void**`` buffers (along
+    with the buffers of any children) are what is allocated on the device.
+    The buffer values should be device pointers. The rest of the structure
+    should be accessible to the CPU.
+
+    The ``private_data`` and ``release`` callback of this structure should
+    contain any necessary information and structures related to freeing
+    the array according to the device it is allocated on, rather than
+    having a separate release callback and ``private_data`` pointer here.
+
+.. c:member:: int64_t ArrowDeviceArray.device_id
+
+    The device id to identify a specific device if multiple devices of this
+    type are on the system. The semantics of the id will be hardware dependent,
+    but we use an ``int64_t`` to future-proof the id as devices change over time.
+
+.. c:member:: ArrowDeviceType ArrowDeviceArray.device_type
+
+    The type of the device which can access the buffers in the array.
+
+.. c:member:: void* ArrowDeviceArray.sync_event
+
+    Optional. An event-like object to synchronize on if needed.
+
+    Many devices, like GPUs, are primarily asynchronous with respect to
+    CPU processing. As such in order to safely access memory, it is often
+    necessary to have an object to synchronize processing with. Since
+    different devices will use different types to specify this, we use a
+    void* which can be coerced into a pointer to whatever the device
+    appropriate type is.
+
+    If synchronization is not needed, this can be null. If this is non-null
+    then it MUST be used to call the appropriate sync method for the device
+    (e.g. ``cudaStreamWaitEvent`` or ``hipStreamWaitEvent``) before attempting
+    to access the memory in the buffers.
+
+    Expected types to coerce this ``void*`` to depending on the reported
+    device type:
+
+    * CUDA: ``cudaEvent_t*``
+    * ROCm: ``hipEvent_t*``
+    * OpenCL: ``cl_event*``
+    * Vulkan: ``VkEvent*``
+    * Metal: ``MTLEvent*``
+    * OneAPI: ``sycl::event*``
+
+    If an event is provided, then the producer MUST ensure that the event
+    is triggered/recorded at the end of the processing stream once the data
+    is considered available for use.
+
+
+.. c:member:: int64_t ArrowDeviceArray.reserved[3]
+
+    As non-CPU development expands, there may be a need to expand this
+    structure. In order to do so without potentially breaking ABI changes,
+    we reserve 24 bytes at the end of the object. This also has the added
+    benefit of bringing the total size of this structure to exactly 128
+    bytes (a power of 2) on 64-bit systems. These bytes should be zero'd
+    out after allocation in order to ensure safe evolution of the ABI in
+    the future.
+
+.. note::
+    Rather than store the shape / types of the data alongside the
+    ``ArrowDeviceArray``, users should utilize the existing ``ArrowSchema``
+    structure to pass any data type and shape information.
+
+Semantics
+=========
+
+Memory management
+-----------------
+
+The ``ArrowDeviceArray`` structure contains an ``ArrowArray`` object which
+itself has :ref:`specific semantics <c-data-interface-semantics>` for releasing
+memory. The term *"base structure"* below refers to the ``ArrowDeviceArray``
+object that is passed directly between the producer and consumer -- not any
+child structure thereof.
+
+It is intended for the base structure to be stack- or heap-allocated by the
+*consumer*. In this case, the producer API should take a pointer to the
+consumer-allocated structure.
+
+However, any data pointed to by the struct MUST be allocated and maintained
+by the producer. This includes the ``sync_event`` member if it is not null,
+along with any pointers in the ``ArrowArray`` object as usual. Data lifetime
+is managed through the ``release`` callback of the ``ArrowArray`` member.
+
+For an ``ArrowDeviceArray``, the semantics of a released structure and the
+callback semantics are identical to those for
+:ref:`ArrowArray itself <c-data-interface-released>`. Any producer specific context
+information necessary for releasing the device data buffers, in addition to
+any allocated event, should be stored in the ``private_data`` member of
+the ``ArrowArray`` and managed by the ``release`` callback.
+
+Moving an array
+'''''''''''''''
+
+The consumer can *move* the ``ArrowDeviceArray`` structure by bitwise copying
+or shallow member-wise copying. Then it MUST mark the source structure released
+by setting the ``release`` member of the embedded ``ArrowArray`` structure to
+``NULL``, but *without* calling that release callback. This ensures that only
+one live copy of the struct is active at any given time and that lifetime is
+correctly communicated to the producer.
+
+As usual, the release callback will be called on the destination structure
+when it is not needed anymore.
+
+Record batches
+--------------
+As with the C data interface itself, a record batch can be trivially considered
+as an equivalent struct array. In this case the metadata of the top-level
+``ArrowSchema`` can be used for schema-level metadata of the record batch.
+
+Mutability
+----------
+
+Both the producer and the consumer SHOULD consider the exported data (that
+is, the data reachable on the device through the ``buffers`` member of
+the embedded ``ArrowArray``) to be immutable, as either party could otherwise
+see inconsistent data while the other is mutating it.
+
+Likewise, if the ``sync_event`` member is non-NULL, the consumer should not
+attempt to access or read the data until they have synchronized on that event.
+
+C producer example
+====================
+
+Exporting a simple ``int32`` device array
+-----------------------------------------
+
+Export a non-nullable ``int32`` type with empty metadata. An example of this
+can be seen in the :ref:`C data interface docs directly <c-data-interface-export-int32-schema>`.
+
+To export the data itself, we transfer ownership to the consumer through
+the release callback. This example will use CUDA, but the equivalent calls
+could be used for any device:
+
+.. code-block:: c
+
+    static void release_int32_device_array(struct ArrowArray* array) {
+        assert(array->n_buffers == 2);
+        // destroy the event
+        cudaEvent_t* ev_ptr = reinterpret_cast<cudaEvent_t*>(array->private_data);

Review Comment:
   In this example, you can see that we store the event pointer in the `private_data` member of the `ArrowArray` member inside of `ArrowDeviceArray` and release should be called using `device_array->array.release(&device_array->array)`. That ensures that there's no UB.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [arrow] kkraus14 commented on a diff in pull request #34972: GH-34971: [Format] Add non-CPU version of C Data Interface

["kkraus14 (via GitHub)" <gi...@apache.org>]

kkraus14 commented on code in PR #34972:
URL: https://github.com/apache/arrow/pull/34972#discussion_r1213464120


##########
docs/source/format/CDeviceDataInterface.rst:
##########
@@ -0,0 +1,628 @@
+.. Licensed to the Apache Software Foundation (ASF) under one
+.. or more contributor license agreements.  See the NOTICE file
+.. distributed with this work for additional information
+.. regarding copyright ownership.  The ASF licenses this file
+.. to you under the Apache License, Version 2.0 (the
+.. "License"); you may not use this file except in compliance
+.. with the License.  You may obtain a copy of the License at
+
+..   http://www.apache.org/licenses/LICENSE-2.0
+
+.. Unless required by applicable law or agreed to in writing,
+.. software distributed under the License is distributed on an
+.. "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+.. KIND, either express or implied.  See the License for the
+.. specific language governing permissions and limitations
+.. under the License.
+
+.. highlight:: c
+
+.. _c-device-data-interface:
+
+=================================
+The Arrow C Device data interface
+=================================
+
+.. note:: The Arrow C Device Data Interface should be considered experimental
+
+Rationale
+=========
+
+The current :ref:`C Data Interface <c-data-interface>`, and most
+implementations of it, make the assumption that all data buffers provided
+are CPU buffers. Since Apache Arrow is designed to be a universal in-memory
+format for representing tabular ("columnar") data, there will be the desire
+to leverage this data on non-CPU hardware such as GPUs. One example of such
+a case is the `RAPIDS cuDF library`_ which uses the Arrow memory format with
+CUDA for NVIDIA GPUs. Since copying data from host to device and back is
+expensive, the ideal would be to be able to leave the data on the device
+for as long as possible, even when passing it between runtimes and
+libraries.
+
+The Arrow C Device data interface builds on the existing C data interface
+by adding a very small, stable set of C definitions to it. These definitions
+are equivalents to the ``ArrowArray`` and ``ArrowArrayStream`` structures
+from the C Data Interface which add members to allow specifying the device
+type and pass necessary information to synchronize with the producer.
+For non-C/C++ languages and runtimes, translating the C definitions to
+corresponding C FFI declarations should be just as simple as with the
+current C data interface.
+
+Applications and libraries can then use Arrow schemas and Arrow formatted
+memory on non-CPU devices to exchange data just as easily as they do
+now with CPU data. This will enable leaving data on those devices longer
+and avoiding costly copies back and forth between the host and device
+just to leverage new libraries and runtimes.
+
+Goals
+-----
+
+* Expose an ABI-stable interface built on the existing C data interface.
+* Make it easy for third-party projects to implement support with little
+  initial investment.
+* Allow zero-copy sharing of Arrow formatted device memory between
+  independant runtimes and components running in the same process.
+* Avoid the need for one-to-one adaptation layers such as the
+  `CUDA Array Interface`_ for Python processes to pass CUDA data.
+* Enable integration without explicit dependencies (either at compile-time
+  or runtime) on the Arrow software project itself.
+
+The intent is for the Arrow C Device data interface to expand the reach
+of the current C data interface, allowing it to also become the standard
+low-level building block for columnar processing on devices like GPUs or
+FPGAs.
+
+Structure definitions
+=====================
+
+Because this is built on the C data interface, the C Device data interface
+uses the ``ArrowSchema`` and ``ArrowArray`` structures as defined in the
+:ref:`C data interface spec <c-data-interface-struct-defs>`. It then adds the
+following free-standing definitions. Like the rest of the Arrow project,
+they are available under the Apache License 2.0.
+
+.. code-block:: c
+
+    #ifndef ARROW_C_DEVICE_DATA_INTERFACE
+    #define ARROW_C_DEVICE_DATA_INTERFACE
+
+    // Device type for the allocated memory
+    typedef int32_t ArrowDeviceType;
+
+    // CPU device, same as using ArrowArray directly
+    #define ARROW_DEVICE_CPU 1
+    // CUDA GPU Device
+    #define ARROW_DEVICE_CUDA 2
+    // Pinned CUDA CPU memory by cudaMallocHost
+    #define ARROW_DEVICE_CUDA_HOST 3
+    // OpenCL Device
+    #define ARROW_DEVICE_OPENCL 4
+    // Vulkan buffer for next-gen graphics
+    #define ARROW_DEVICE_VULKAN 7
+    // Metal for Apple GPU
+    #define ARROW_DEVICE_METAL 8
+    // Verilog simulator buffer
+    #define ARROW_DEVICE_VPI 9
+    // ROCm GPUs for AMD GPUs
+    #define ARROW_DEVICE_ROCM 10
+    // Pinned ROCm CPU memory allocated by hipMallocHost
+    #define ARROW_DEVICE_ROCM_HOST 11
+    // Reserved for extension
+    //
+    // used to quickly test extension devices, semantics
+    // can differ based on implementation
+    #define ARROW_DEVICE_EXT_DEV 12
+    // CUDA managed/unified memory allocated by cudaMallocManaged
+    #define ARROW_DEVICE_CUDA_MANAGED 13
+    // Unified shared memory allocated on a oneAPI
+    // non-partitioned device.
+    //
+    // A call to the oneAPI runtime is required to determine the
+    // device type, the USM allocation type and the sycl context
+    // that it is bound to.
+    #define ARROW_DEVICE_ONEAPI 14
+    // GPU support for next-gen WebGPU standard
+    #define ARROW_DEVICE_WEBGPU 15
+    // Qualcomm Hexagon DSP
+    #define ARROW_DEVICE_HEXAGON 16
+
+    struct ArrowDeviceArray {
+      struct ArrowArray array;
+      int64_t device_id;
+      ArrowDeviceType device_type;
+      void* sync_event;
+
+      // reserved bytes for future expansion
+      int64_t reserved[3];
+    };
+
+    #endif  // ARROW_C_DEVICE_DATA_INTERFACE
+
+.. note::
+   The canonical guard ``ARROW_C_DEVICE_DATA_INTERFACE`` is meant to avoid
+   duplicate definitions if two projects copy the definitions in their own
+   headers, and a third-party project includes from these two projects. It
+   is therefore important that this guard is kept exactly as-is when these
+   definitions are copied.
+
+ArrowDeviceType
+---------------
+
+The ``ArrowDeviceType`` typedef is used to indicate what type of device the
+provided memory buffers were allocated on. This, in conjunction with the
+``device_id``, should be sufficient to reference the correct data buffers.
+
+We then use macros to define values for different device types. The provided
+macro values are compatible with the widely used `dlpack`_ ``DLDeviceType``
+definition values, using the same value for each as the equivalent
+``kDL<type>`` enum from dlpack.h. The list will be kept in sync with those
+equivalent enum values over time to ensure compatibility, rather than
+potentially diverging. To avoid the Arrow project having to be in the
+position of vetting new hardware devices, new additions should first be
+added to dlpack before we add a corresponding macro here.
+
+To ensure predictability with the ABI, we use macros instead of an ``enum``
+so the storage type is not compiler dependent.
+
+.. c:macro:: ARROW_DEVICE_CPU
+
+    CPU Device, equivalent to just using ``ArrowArray`` directly instead of
+    using ``ArrowDeviceArray``.
+
+.. c:macro:: ARROW_DEVICE_CUDA
+
+    A `CUDA`_ GPU Device. This could represent data allocated either with the
+    runtime library (``cudaMalloc``) or the device driver (``cuMemAlloc``).
+
+.. c:macro:: ARROW_DEVICE_CUDA_HOST
+
+    CPU memory that was pinned and page-locked by CUDA by using
+    ``cudaMallocHost`` or ``cuMemAllocHost``.
+
+.. c:macro:: ARROW_DEVICE_OPENCL
+
+    Data allocated on the device by using the `OpenCL (Open Computing Language)`_
+    framework.
+
+.. c:macro:: ARROW_DEVICE_VULKAN
+
+    Data allocated by the `Vulkan`_ framework and libraries.
+
+.. c:macro:: ARROW_DEVICE_METAL
+
+    Data on Apple GPU devices using the `Metal`_ framework and libraries.
+
+.. c:macro:: ARROW_DEVICE_VPI
+
+    Indicates usage of a Verilog simulator buffer.
+
+.. c:macro:: ARROW_DEVICE_ROCM
+
+    An AMD device using the `ROCm`_ stack.
+
+.. c:macro:: ARROW_DEVICE_ROCM_HOST
+
+    CPU memory pinned and page-locked allocated ``hipMallocHost``.
+
+.. c:macro:: ARROW_DEVICE_EXT_DEV
+
+    This value is an escape-hatch for devices to extend which aren't
+    currently represented otherwise. Producers would need to provide
+    additional information/context specific to the device if using
+    this device type. This is used to quickly test extension devices
+    and semantics can differ based on the implementation.
+
+.. c:macro:: ARROW_DEVICE_CUDA_MANAGED
+
+    CUDA managed/unified memory which is allocated by ``cudaMallocManaged``.
+
+.. c:macro:: ARROW_DEVICE_ONEAPI
+
+    Unified shared memory allocated on an Intel `oneAPI`_ non-partitioned
+    device. A call to the ``oneAPI`` runtime is required to determine
+    the specific device type, the USM allocation type and the sycl context
+    that it is bound to.
+
+.. c:macro:: ARROW_DEVICE_WEBGPU
+
+    GPU support for next-gen WebGPU standards
+
+.. c:macro:: ARROW_DEVICE_HEXAGON
+
+    Data allocated on a Qualcomm Hexagon DSP device.
+
+The ArrowDeviceArray structure
+------------------------------
+
+The ``ArrowDeviceArray`` structure embeds the C data ``ArrowArray`` structure
+and adds additional information necessary for consumers to use the data. It
+has the following fields:
+
+.. c:member:: struct ArrowArray ArrowDeviceArray.array
+
+    The allocated array data. The values in the ``void**`` buffers (along
+    with the buffers of any children) are what is allocated on the device.
+    The buffer values should be device pointers. The rest of the structure
+    should be accessible to the CPU.
+
+    The ``private_data`` and ``release`` callback of this structure should
+    contain any necessary information and structures related to freeing
+    the array according to the device it is allocated on, rather than
+    having a separate release callback and ``private_data`` pointer here.
+
+.. c:member:: int64_t ArrowDeviceArray.device_id
+
+    The device id to identify a specific device if multiple devices of this
+    type are on the system. The semantics of the id will be hardware dependent,
+    but we use an ``int64_t`` to future-proof the id as devices change over time.
+
+.. c:member:: ArrowDeviceType ArrowDeviceArray.device_type
+
+    The type of the device which can access the buffers in the array.
+
+.. c:member:: void* ArrowDeviceArray.sync_event
+
+    Optional. An event-like object to synchronize on if needed.
+
+    Many devices, like GPUs, are primarily asynchronous with respect to
+    CPU processing. As such in order to safely access memory, it is often
+    necessary to have an object to synchronize processing with. Since
+    different devices will use different types to specify this, we use a
+    void* which can be coerced into a pointer to whatever the device
+    appropriate type is.
+
+    If synchronization is not needed, this can be null. If this is non-null
+    then it MUST be used to call the appropriate sync method for the device
+    (e.g. ``cudaStreamWaitEvent`` or ``hipStreamWaitEvent``) before attempting
+    to access the memory in the buffers.
+
+    Expected types to coerce this ``void*`` to depending on the reported
+    device type:
+
+    * CUDA: ``cudaEvent_t*``
+    * ROCm: ``hipEvent_t*``
+    * OpenCL: ``cl_event*``
+    * Vulkan: ``VkEvent*``
+    * Metal: ``MTLEvent*``
+    * OneAPI: ``sycl::event*``
+
+    If an event is provided, then the producer MUST ensure that the event
+    is triggered/recorded at the end of the processing stream once the data
+    is considered available for use.
+
+
+.. c:member:: int64_t ArrowDeviceArray.reserved[3]
+
+    As non-CPU development expands, there may be a need to expand this
+    structure. In order to do so without potentially breaking ABI changes,
+    we reserve 24 bytes at the end of the object. This also has the added
+    benefit of bringing the total size of this structure to exactly 128
+    bytes (a power of 2) on 64-bit systems. These bytes should be zero'd
+    out after allocation in order to ensure safe evolution of the ABI in
+    the future.
+
+.. note::
+    Rather than store the shape / types of the data alongside the
+    ``ArrowDeviceArray``, users should utilize the existing ``ArrowSchema``
+    structure to pass any data type and shape information.
+
+Semantics
+=========
+
+Memory management
+-----------------
+
+The ``ArrowDeviceArray`` structure contains an ``ArrowArray`` object which
+itself has :ref:`specific semantics <c-data-interface-semantics>` for releasing
+memory. The term *"base structure"* below refers to the ``ArrowDeviceArray``
+object that is passed directly between the producer and consumer -- not any
+child structure thereof.
+
+It is intended for the base structure to be stack- or heap-allocated by the
+*consumer*. In this case, the producer API should take a pointer to the
+consumer-allocated structure.
+
+However, any data pointed to by the struct MUST be allocated and maintained
+by the producer. This includes the ``sync_event`` member if it is not null,
+along with any pointers in the ``ArrowArray`` object as usual. Data lifetime
+is managed through the ``release`` callback of the ``ArrowArray`` member.
+
+For an ``ArrowDeviceArray``, the semantics of a released structure and the
+callback semantics are identical to those for
+:ref:`ArrowArray itself <c-data-interface-released>`. Any producer specific context
+information necessary for releasing the device data buffers, in addition to
+any allocated event, should be stored in the ``private_data`` member of
+the ``ArrowArray`` and managed by the ``release`` callback.
+
+Moving an array
+'''''''''''''''
+
+The consumer can *move* the ``ArrowDeviceArray`` structure by bitwise copying
+or shallow member-wise copying. Then it MUST mark the source structure released
+by setting the ``release`` member of the embedded ``ArrowArray`` structure to
+``NULL``, but *without* calling that release callback. This ensures that only
+one live copy of the struct is active at any given time and that lifetime is
+correctly communicated to the producer.
+
+As usual, the release callback will be called on the destination structure
+when it is not needed anymore.
+
+Record batches
+--------------
+As with the C data interface itself, a record batch can be trivially considered
+as an equivalent struct array. In this case the metadata of the top-level
+``ArrowSchema`` can be used for schema-level metadata of the record batch.
+
+Mutability
+----------
+
+Both the producer and the consumer SHOULD consider the exported data (that
+is, the data reachable on the device through the ``buffers`` member of
+the embedded ``ArrowArray``) to be immutable, as either party could otherwise
+see inconsistent data while the other is mutating it.
+
+Likewise, if the ``sync_event`` member is non-NULL, the consumer should not
+attempt to access or read the data until they have synchronized on that event.
+
+C producer example
+====================
+
+Exporting a simple ``int32`` device array
+-----------------------------------------
+
+Export a non-nullable ``int32`` type with empty metadata. An example of this
+can be seen in the :ref:`C data interface docs directly <c-data-interface-export-int32-schema>`.
+
+To export the data itself, we transfer ownership to the consumer through
+the release callback. This example will use CUDA, but the equivalent calls
+could be used for any device:
+
+.. code-block:: c
+
+    static void release_int32_device_array(struct ArrowArray* array) {
+        assert(array->n_buffers == 2);
+        // destroy the event
+        cudaEvent_t* ev_ptr = reinterpret_cast<cudaEvent_t*>(array->private_data);
+        cudaError_t status = cudaEventDestroy(*ev_ptr);
+        assert(status == cudaSuccess);
+        free(ev_ptr);
+
+        // free the buffers and the buffers array
+        status = cudaFree(array->buffers[1]);
+        assert(status == cudaSuccess);
+        free(array->buffers);
+
+        // mark released
+        array->release = NULL;
+    }
+
+    __host__ void export_int32_device_array(void* cudaAllocdPtr,

Review Comment:
   Given this code block is marked as `c` and there's no kernel code here we should definitely remove this as it's not needed (and likely wouldn't work unless using `nvcc` and would still likely warn).



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [arrow] zeroshade commented on a diff in pull request #34972: GH-34971: [Format] Add non-CPU version of C Data Interface

["zeroshade (via GitHub)" <gi...@apache.org>]

zeroshade commented on code in PR #34972:
URL: https://github.com/apache/arrow/pull/34972#discussion_r1213449532


##########
docs/source/format/CDeviceDataInterface.rst:
##########
@@ -0,0 +1,628 @@
+.. Licensed to the Apache Software Foundation (ASF) under one
+.. or more contributor license agreements.  See the NOTICE file
+.. distributed with this work for additional information
+.. regarding copyright ownership.  The ASF licenses this file
+.. to you under the Apache License, Version 2.0 (the
+.. "License"); you may not use this file except in compliance
+.. with the License.  You may obtain a copy of the License at
+
+..   http://www.apache.org/licenses/LICENSE-2.0
+
+.. Unless required by applicable law or agreed to in writing,
+.. software distributed under the License is distributed on an
+.. "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+.. KIND, either express or implied.  See the License for the
+.. specific language governing permissions and limitations
+.. under the License.
+
+.. highlight:: c
+
+.. _c-device-data-interface:
+
+=================================
+The Arrow C Device data interface
+=================================
+
+.. note:: The Arrow C Device Data Interface should be considered experimental
+
+Rationale
+=========
+
+The current :ref:`C Data Interface <c-data-interface>`, and most
+implementations of it, make the assumption that all data buffers provided
+are CPU buffers. Since Apache Arrow is designed to be a universal in-memory
+format for representing tabular ("columnar") data, there will be the desire
+to leverage this data on non-CPU hardware such as GPUs. One example of such
+a case is the `RAPIDS cuDF library`_ which uses the Arrow memory format with
+CUDA for NVIDIA GPUs. Since copying data from host to device and back is
+expensive, the ideal would be to be able to leave the data on the device
+for as long as possible, even when passing it between runtimes and
+libraries.
+
+The Arrow C Device data interface builds on the existing C data interface
+by adding a very small, stable set of C definitions to it. These definitions
+are equivalents to the ``ArrowArray`` and ``ArrowArrayStream`` structures
+from the C Data Interface which add members to allow specifying the device
+type and pass necessary information to synchronize with the producer.
+For non-C/C++ languages and runtimes, translating the C definitions to
+corresponding C FFI declarations should be just as simple as with the
+current C data interface.
+
+Applications and libraries can then use Arrow schemas and Arrow formatted
+memory on non-CPU devices to exchange data just as easily as they do
+now with CPU data. This will enable leaving data on those devices longer
+and avoiding costly copies back and forth between the host and device
+just to leverage new libraries and runtimes.
+
+Goals
+-----
+
+* Expose an ABI-stable interface built on the existing C data interface.
+* Make it easy for third-party projects to implement support with little
+  initial investment.
+* Allow zero-copy sharing of Arrow formatted device memory between
+  independant runtimes and components running in the same process.
+* Avoid the need for one-to-one adaptation layers such as the
+  `CUDA Array Interface`_ for Python processes to pass CUDA data.
+* Enable integration without explicit dependencies (either at compile-time
+  or runtime) on the Arrow software project itself.
+
+The intent is for the Arrow C Device data interface to expand the reach
+of the current C data interface, allowing it to also become the standard
+low-level building block for columnar processing on devices like GPUs or
+FPGAs.
+
+Structure definitions
+=====================
+
+Because this is built on the C data interface, the C Device data interface
+uses the ``ArrowSchema`` and ``ArrowArray`` structures as defined in the
+:ref:`C data interface spec <c-data-interface-struct-defs>`. It then adds the
+following free-standing definitions. Like the rest of the Arrow project,
+they are available under the Apache License 2.0.
+
+.. code-block:: c
+
+    #ifndef ARROW_C_DEVICE_DATA_INTERFACE
+    #define ARROW_C_DEVICE_DATA_INTERFACE
+
+    // Device type for the allocated memory
+    typedef int32_t ArrowDeviceType;
+
+    // CPU device, same as using ArrowArray directly
+    #define ARROW_DEVICE_CPU 1
+    // CUDA GPU Device
+    #define ARROW_DEVICE_CUDA 2
+    // Pinned CUDA CPU memory by cudaMallocHost
+    #define ARROW_DEVICE_CUDA_HOST 3
+    // OpenCL Device
+    #define ARROW_DEVICE_OPENCL 4
+    // Vulkan buffer for next-gen graphics
+    #define ARROW_DEVICE_VULKAN 7
+    // Metal for Apple GPU
+    #define ARROW_DEVICE_METAL 8
+    // Verilog simulator buffer
+    #define ARROW_DEVICE_VPI 9
+    // ROCm GPUs for AMD GPUs
+    #define ARROW_DEVICE_ROCM 10
+    // Pinned ROCm CPU memory allocated by hipMallocHost
+    #define ARROW_DEVICE_ROCM_HOST 11
+    // Reserved for extension
+    //
+    // used to quickly test extension devices, semantics
+    // can differ based on implementation
+    #define ARROW_DEVICE_EXT_DEV 12
+    // CUDA managed/unified memory allocated by cudaMallocManaged
+    #define ARROW_DEVICE_CUDA_MANAGED 13
+    // Unified shared memory allocated on a oneAPI
+    // non-partitioned device.
+    //
+    // A call to the oneAPI runtime is required to determine the
+    // device type, the USM allocation type and the sycl context
+    // that it is bound to.
+    #define ARROW_DEVICE_ONEAPI 14
+    // GPU support for next-gen WebGPU standard
+    #define ARROW_DEVICE_WEBGPU 15
+    // Qualcomm Hexagon DSP
+    #define ARROW_DEVICE_HEXAGON 16
+
+    struct ArrowDeviceArray {
+      struct ArrowArray array;
+      int64_t device_id;
+      ArrowDeviceType device_type;
+      void* sync_event;
+
+      // reserved bytes for future expansion
+      int64_t reserved[3];
+    };
+
+    #endif  // ARROW_C_DEVICE_DATA_INTERFACE
+
+.. note::
+   The canonical guard ``ARROW_C_DEVICE_DATA_INTERFACE`` is meant to avoid
+   duplicate definitions if two projects copy the definitions in their own
+   headers, and a third-party project includes from these two projects. It
+   is therefore important that this guard is kept exactly as-is when these
+   definitions are copied.
+
+ArrowDeviceType
+---------------
+
+The ``ArrowDeviceType`` typedef is used to indicate what type of device the
+provided memory buffers were allocated on. This, in conjunction with the
+``device_id``, should be sufficient to reference the correct data buffers.
+
+We then use macros to define values for different device types. The provided
+macro values are compatible with the widely used `dlpack`_ ``DLDeviceType``
+definition values, using the same value for each as the equivalent
+``kDL<type>`` enum from dlpack.h. The list will be kept in sync with those
+equivalent enum values over time to ensure compatibility, rather than
+potentially diverging. To avoid the Arrow project having to be in the
+position of vetting new hardware devices, new additions should first be
+added to dlpack before we add a corresponding macro here.
+
+To ensure predictability with the ABI, we use macros instead of an ``enum``
+so the storage type is not compiler dependent.
+
+.. c:macro:: ARROW_DEVICE_CPU
+
+    CPU Device, equivalent to just using ``ArrowArray`` directly instead of
+    using ``ArrowDeviceArray``.
+
+.. c:macro:: ARROW_DEVICE_CUDA
+
+    A `CUDA`_ GPU Device. This could represent data allocated either with the
+    runtime library (``cudaMalloc``) or the device driver (``cuMemAlloc``).
+
+.. c:macro:: ARROW_DEVICE_CUDA_HOST
+
+    CPU memory that was pinned and page-locked by CUDA by using
+    ``cudaMallocHost`` or ``cuMemAllocHost``.
+
+.. c:macro:: ARROW_DEVICE_OPENCL
+
+    Data allocated on the device by using the `OpenCL (Open Computing Language)`_
+    framework.
+
+.. c:macro:: ARROW_DEVICE_VULKAN
+
+    Data allocated by the `Vulkan`_ framework and libraries.
+
+.. c:macro:: ARROW_DEVICE_METAL
+
+    Data on Apple GPU devices using the `Metal`_ framework and libraries.
+
+.. c:macro:: ARROW_DEVICE_VPI
+
+    Indicates usage of a Verilog simulator buffer.
+
+.. c:macro:: ARROW_DEVICE_ROCM
+
+    An AMD device using the `ROCm`_ stack.
+
+.. c:macro:: ARROW_DEVICE_ROCM_HOST
+
+    CPU memory pinned and page-locked allocated ``hipMallocHost``.
+
+.. c:macro:: ARROW_DEVICE_EXT_DEV
+
+    This value is an escape-hatch for devices to extend which aren't
+    currently represented otherwise. Producers would need to provide
+    additional information/context specific to the device if using
+    this device type. This is used to quickly test extension devices
+    and semantics can differ based on the implementation.
+
+.. c:macro:: ARROW_DEVICE_CUDA_MANAGED
+
+    CUDA managed/unified memory which is allocated by ``cudaMallocManaged``.
+
+.. c:macro:: ARROW_DEVICE_ONEAPI
+
+    Unified shared memory allocated on an Intel `oneAPI`_ non-partitioned
+    device. A call to the ``oneAPI`` runtime is required to determine
+    the specific device type, the USM allocation type and the sycl context
+    that it is bound to.
+
+.. c:macro:: ARROW_DEVICE_WEBGPU
+
+    GPU support for next-gen WebGPU standards
+
+.. c:macro:: ARROW_DEVICE_HEXAGON
+
+    Data allocated on a Qualcomm Hexagon DSP device.
+
+The ArrowDeviceArray structure
+------------------------------
+
+The ``ArrowDeviceArray`` structure embeds the C data ``ArrowArray`` structure
+and adds additional information necessary for consumers to use the data. It
+has the following fields:
+
+.. c:member:: struct ArrowArray ArrowDeviceArray.array
+
+    The allocated array data. The values in the ``void**`` buffers (along
+    with the buffers of any children) are what is allocated on the device.
+    The buffer values should be device pointers. The rest of the structure
+    should be accessible to the CPU.
+
+    The ``private_data`` and ``release`` callback of this structure should
+    contain any necessary information and structures related to freeing
+    the array according to the device it is allocated on, rather than
+    having a separate release callback and ``private_data`` pointer here.
+
+.. c:member:: int64_t ArrowDeviceArray.device_id
+
+    The device id to identify a specific device if multiple devices of this
+    type are on the system. The semantics of the id will be hardware dependent,
+    but we use an ``int64_t`` to future-proof the id as devices change over time.
+
+.. c:member:: ArrowDeviceType ArrowDeviceArray.device_type
+
+    The type of the device which can access the buffers in the array.
+
+.. c:member:: void* ArrowDeviceArray.sync_event
+
+    Optional. An event-like object to synchronize on if needed.
+
+    Many devices, like GPUs, are primarily asynchronous with respect to
+    CPU processing. As such in order to safely access memory, it is often
+    necessary to have an object to synchronize processing with. Since
+    different devices will use different types to specify this, we use a
+    void* which can be coerced into a pointer to whatever the device
+    appropriate type is.
+
+    If synchronization is not needed, this can be null. If this is non-null
+    then it MUST be used to call the appropriate sync method for the device
+    (e.g. ``cudaStreamWaitEvent`` or ``hipStreamWaitEvent``) before attempting
+    to access the memory in the buffers.
+
+    Expected types to coerce this ``void*`` to depending on the reported
+    device type:
+
+    * CUDA: ``cudaEvent_t*``
+    * ROCm: ``hipEvent_t*``
+    * OpenCL: ``cl_event*``
+    * Vulkan: ``VkEvent*``
+    * Metal: ``MTLEvent*``
+    * OneAPI: ``sycl::event*``
+
+    If an event is provided, then the producer MUST ensure that the event
+    is triggered/recorded at the end of the processing stream once the data
+    is considered available for use.
+
+
+.. c:member:: int64_t ArrowDeviceArray.reserved[3]
+
+    As non-CPU development expands, there may be a need to expand this
+    structure. In order to do so without potentially breaking ABI changes,
+    we reserve 24 bytes at the end of the object. This also has the added
+    benefit of bringing the total size of this structure to exactly 128
+    bytes (a power of 2) on 64-bit systems. These bytes should be zero'd
+    out after allocation in order to ensure safe evolution of the ABI in
+    the future.
+
+.. note::
+    Rather than store the shape / types of the data alongside the

Review Comment:
   One of the reasons for the creation of things such as the `Cuda Array Interface` that numba uses was in order to pass the shape and type of the data buffer along with the pointers. Since that kind of interface is what we're targeting here and we aren't putting that information into the `ArrowDeviceArray` struct, I felt it was good to explicitly state that producers should use the `ArrowSchema` struct to pass this information. But I'm open to removing this note if people feel it is unnecessary.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [arrow] zeroshade commented on a diff in pull request #34972: GH-34971: [Format] Add non-CPU version of C Data Interface

["zeroshade (via GitHub)" <gi...@apache.org>]

zeroshade commented on code in PR #34972:
URL: https://github.com/apache/arrow/pull/34972#discussion_r1213606997


##########
cpp/src/arrow/c/abi.h:
##########
@@ -15,10 +15,30 @@
 // specific language governing permissions and limitations
 // under the License.
 
+/// \file abi.h Arrow C-Data Interface
+///
+/// The Arrow C-Data interface defines a very small, stable set
+/// of C definitions which can be easily copied into any project's
+/// source code and vendored to be used for columnar data interchange
+/// in the Arrow format. For non-C/C++ languages and runtimes,
+/// it should be almost as easy to translate the C definitions into
+/// the corresponding C FFI declarations.
+///
+/// Applications and libraries can therefore work with Arrow memory
+/// without necessarily using the Arrow libraries or reinventing
+/// the wheel. Developers can choose between tight integration
+/// with the Arrow software project or minimal integration with
+/// the Arrow format only.
+
 #pragma once
 
 #include <stdint.h>
 
+/// \defgroup Arrow C-Data Interface

Review Comment:
   Makes sense, I've added links to the specs and reduced the detail of the comments.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [arrow] kkraus14 commented on a diff in pull request #34972: GH-34971: [Format] Enhance C-Data API to support non-cpu cases

["kkraus14 (via GitHub)" <gi...@apache.org>]

kkraus14 commented on code in PR #34972:
URL: https://github.com/apache/arrow/pull/34972#discussion_r1163373562


##########
cpp/src/arrow/c/abi.h:
##########
@@ -106,6 +212,98 @@ struct ArrowArrayStream {
 
 #endif  // ARROW_C_STREAM_INTERFACE
 
+#ifndef ARROW_C_DEVICE_STREAM_INTERFACE
+#define ARROW_C_DEVICE_STREAM_INTERFACE
+
+/// \brief Equivalent to ArrowArrayStream, but for ArrowDeviceArrays.
+///
+/// This stream is intended to provide a stream of data on a single
+/// device, if a producer wants data to be produced on multiple devices
+/// then multiple streams should be provided. One per device.
+struct ArrowDeviceArrayStream {
+  /// \brief The device that this stream produces data on.
+  ///
+  /// All ArrowDeviceArrays that are produced by this
+  /// stream should have the same device_type as set
+  /// here. The device_type needs to be provided here
+  /// so that consumers can provide the correct type
+  /// of queue_ptr when calling get_next.
+  ArrowDeviceType device_type;
+
+  /// \brief Callback to get the stream schema
+  /// (will be the same for all arrays in the stream).
+  ///
+  /// If successful, the ArrowSchema must be released independantly from the stream.
+  /// The schema should be accessible via CPU memory.
+  ///
+  /// \param[in] self The ArrowDeviceArrayStream object itself
+  /// \param[out] out C struct to export the schema to
+  /// \return 0 if successful, an `errno`-compatible error code otherwise.
+  int (*get_schema)(struct ArrowDeviceArrayStream* self, struct ArrowSchema* out);
+
+  /// \brief Callback to get the device id for the next array.
+  ///
+  /// This is necessary so that the proper/correct stream pointer can be provided
+  /// to get_next.
+  ///
+  /// The next call to `get_next` should provide an ArrowDeviceArray whose
+  /// device_id matches what is provided here, and whose device_type is the
+  /// same as the device_type member of this stream.

Review Comment:
   > Yes, I read this. It looks like solution S1, which is also the one I'm proposing, is considered the most flexible (I don't understand the "harder for compilers" comment, though).
   
   From: https://github.com/dmlc/dlpack/issues/57#issuecomment-751715168
   > It also brings extra burden to the compilers themselves. The compiler will need to generate optional synchronization code based on the streams, which is non-trivial.
   
   I believe the compilers being referred to here are deep learning compilers like XLA which do things like kernel fusion and set up execution graphs of kernels that use streams internally to parallelize the execution of said graphs.
   
   > But you have to actually synchronize on the right stream before being able to use the object, right?
   
   Something / someone needs to guarantee that there isn't a data race with regards to using multiple non-blocking streams, yes. That could be done with events, stream synchronization, or device synchronization.
   
   > How does the user know which stream to synchronize on, if they didn't produce the data themselves?
   
   If you're staying within your framework / library then the expectation is for the framework / library to handle things for the user. If crossing framework / library boundaries, then the expectation is to be reliant on things like interchange protocols to handle the synchronization semantics.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [arrow] kkraus14 commented on a diff in pull request #34972: GH-34971: [Format] Enhance C-Data API to support non-cpu cases

["kkraus14 (via GitHub)" <gi...@apache.org>]

kkraus14 commented on code in PR #34972:
URL: https://github.com/apache/arrow/pull/34972#discussion_r1163373562


##########
cpp/src/arrow/c/abi.h:
##########
@@ -106,6 +212,98 @@ struct ArrowArrayStream {
 
 #endif  // ARROW_C_STREAM_INTERFACE
 
+#ifndef ARROW_C_DEVICE_STREAM_INTERFACE
+#define ARROW_C_DEVICE_STREAM_INTERFACE
+
+/// \brief Equivalent to ArrowArrayStream, but for ArrowDeviceArrays.
+///
+/// This stream is intended to provide a stream of data on a single
+/// device, if a producer wants data to be produced on multiple devices
+/// then multiple streams should be provided. One per device.
+struct ArrowDeviceArrayStream {
+  /// \brief The device that this stream produces data on.
+  ///
+  /// All ArrowDeviceArrays that are produced by this
+  /// stream should have the same device_type as set
+  /// here. The device_type needs to be provided here
+  /// so that consumers can provide the correct type
+  /// of queue_ptr when calling get_next.
+  ArrowDeviceType device_type;
+
+  /// \brief Callback to get the stream schema
+  /// (will be the same for all arrays in the stream).
+  ///
+  /// If successful, the ArrowSchema must be released independantly from the stream.
+  /// The schema should be accessible via CPU memory.
+  ///
+  /// \param[in] self The ArrowDeviceArrayStream object itself
+  /// \param[out] out C struct to export the schema to
+  /// \return 0 if successful, an `errno`-compatible error code otherwise.
+  int (*get_schema)(struct ArrowDeviceArrayStream* self, struct ArrowSchema* out);
+
+  /// \brief Callback to get the device id for the next array.
+  ///
+  /// This is necessary so that the proper/correct stream pointer can be provided
+  /// to get_next.
+  ///
+  /// The next call to `get_next` should provide an ArrowDeviceArray whose
+  /// device_id matches what is provided here, and whose device_type is the
+  /// same as the device_type member of this stream.

Review Comment:
   > Yes, I read this. It looks like solution S1, which is also the one I'm proposing, is considered the most flexible (I don't understand the "harder for compilers" comment, though).
   
   From: https://github.com/dmlc/dlpack/issues/57#issuecomment-751715168
   > It also brings extra burden to the compilers themselves. The compiler will need to generate optional synchronization code based on the streams, which is non-trivial.
   
   > But you have to actually synchronize on the right stream before being able to use the object, right?
   
   Something / someone needs to guarantee that there isn't a data race with regards to using multiple non-blocking streams, yes. That could be done with events, stream synchronization, or device synchronization.
   
   > How does the user know which stream to synchronize on, if they didn't produce the data themselves?
   
   If you're staying within your framework / library then the expectation is for the framework / library to handle things for the user. If crossing framework / library boundaries, then the expectation is to be reliant on things like interchange protocols to handle the synchronization semantics.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [arrow] GregoryKimball commented on a diff in pull request #34972: GH-34971: [Format] Enhance C-Data API to support non-cpu cases

["GregoryKimball (via GitHub)" <gi...@apache.org>]

GregoryKimball commented on code in PR #34972:
URL: https://github.com/apache/arrow/pull/34972#discussion_r1192642045


##########
cpp/src/arrow/c/abi.h:
##########
@@ -65,6 +85,110 @@ struct ArrowArray {
 
 #endif  // ARROW_C_DATA_INTERFACE
 
+#ifndef ARROW_C_DEVICE_DATA_INTERFACE
+#define ARROW_C_DEVICE_DATA_INTERFACE
+
+/// \defgroup arrow-device-types Device Types
+/// These macros are compatible with the dlpack DLDeviceType values,
+/// using the same value for each enum as the equivalent kDL<type>
+/// from dlpack.h. This list should continue to be kept in sync with
+/// the equivalent dlpack.h enum values over time to ensure
+/// compatibility, rather than potentially diverging.
+///
+/// To ensure predictability with the ABI we use macros instead of
+/// an enum so the storage type is not compiler dependent.
+///
+/// @{
+
+/// \brief DeviceType for the allocated memory
+typedef int32_t ArrowDeviceType;
+
+/// \brief CPU device, same as using ArrowArray directly
+#define ARROW_DEVICE_CPU = 1
+/// \brief CUDA GPU Device
+#define ARROW_DEVICE_CUDA = 2
+/// \brief Pinned CUDA CPU memory by cudaMallocHost
+#define ARROW_DEVICE_CUDA_HOST = 3
+/// \brief OpenCL Device
+#define ARROW_DEVICE_OPENCL = 4
+/// \brief Vulkan buffer for next-gen graphics
+#define ARROW_DEVICE_VULKAN = 7
+/// \brief Metal for Apple GPU
+#define ARROW_DEVICE_METAL = 8
+/// \brief Verilog simulator buffer
+#define ARROW_DEVICE_VPI = 9
+/// \brief ROCm GPUs for AMD GPUs
+#define ARROW_DEVICE_ROCM = 10
+/// \brief Pinned ROCm CPU memory allocated by hipMallocHost
+#define ARROW_DEVICE_ROCMHOST = 11
+/// \brief Reserved for extension
+///
+/// used to quickly test extension devices, semantics
+/// can differ based on the implementation
+#define ARROW_DEVICE_EXT_DEV = 12
+/// \brief CUDA managed/unified memory allocated by cudaMallocManaged
+#define ARROW_DEVICE_CUDA_MANAGED = 13
+/// \brief unified shared memory allocated on a oneAPI
+/// non-partitioned device.
+///
+/// A call to the oneAPI runtime is required to determine the device
+/// type, the USM allocation type, and the sycl context it is bound to.
+#define ARROW_DEVICE_ONEAPI = 14
+/// \brief GPU support for next-gen WebGPU standard
+#define ARROW_DEVICE_WEBGPU = 15
+/// \brief Qualcomm Hexagon DSP
+#define ARROW_DEVICE_HEXAGON = 16
+
+/// @}
+
+/// \brief Struct for passing an Arrow Array alongside
+/// device memory information.
+struct ArrowDeviceArray {
+  /// \brief the Allocated Array
+  ///
+  /// the buffers in the array (along with the buffers of any
+  /// children) are what is allocated on the device.
+  ///
+  /// the private_data and release callback of the arrow array
+  /// should contain any necessary information and structures
+  /// related to freeing the array according to the device it
+  /// is allocated on, rather than having a separate release
+  /// callback embedded here.
+  struct ArrowArray array;
+  /// \brief The device id to identify a specific device
+  /// if multiple of this type are on the system.
+  ///
+  /// the semantics of the id will be hardware dependant.
+  int64_t device_id;
+  /// \brief The type of device which can access this memory.
+  ArrowDeviceType device_type;
+  /// \brief An event-like object to synchronize on if needed.
+  ///
+  /// Many devices, like GPUs, are primarily asynchronous with
+  /// respect to CPU processing. As such in order to safely access
+  /// memory, it is often necessary to have an object to synchronize
+  /// processing on. Since different devices will use different types
+  /// to specify this we use a void* that can be coerced into
+  /// whatever the device appropriate type is (e.g. cudaEvent_t for
+  /// CUDA and hipEvent_t for HIP).
+  ///
+  /// If synchronization is not needed this can be null. If this is
+  /// non-null, then it should be used to call the appropriate sync
+  /// method for the device (e.g. cudaStreamWaitEvent / hipStreamWaitEvent).
+  void* sync_event;

Review Comment:
   Would you please help me understand the `void* sync_event` pointer a bit better in the context of CUDA/C++? Is this a function pointer that is [expected to be called](https://docs.nvidia.com/cuda/cuda-runtime-api/group__CUDART__STREAM.html) with a `cudaStream_t` parameter provided by the application? Would there be a benefit from storing a `cudaStream_t` in `ArrowDeviceArray` (in the `reserved` bytes or elsewhere)?



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [arrow] zeroshade commented on a diff in pull request #34972: GH-34971: [Format] Enhance C-Data API to support non-cpu cases

["zeroshade (via GitHub)" <gi...@apache.org>]

zeroshade commented on code in PR #34972:
URL: https://github.com/apache/arrow/pull/34972#discussion_r1190198185


##########
cpp/src/arrow/c/abi.h:
##########
@@ -106,6 +212,98 @@ struct ArrowArrayStream {
 
 #endif  // ARROW_C_STREAM_INTERFACE
 
+#ifndef ARROW_C_DEVICE_STREAM_INTERFACE
+#define ARROW_C_DEVICE_STREAM_INTERFACE
+
+/// \brief Equivalent to ArrowArrayStream, but for ArrowDeviceArrays.
+///
+/// This stream is intended to provide a stream of data on a single
+/// device, if a producer wants data to be produced on multiple devices
+/// then multiple streams should be provided. One per device.
+struct ArrowDeviceArrayStream {
+  /// \brief The device that this stream produces data on.
+  ///
+  /// All ArrowDeviceArrays that are produced by this
+  /// stream should have the same device_type as set
+  /// here. The device_type needs to be provided here
+  /// so that consumers can provide the correct type
+  /// of queue_ptr when calling get_next.
+  ArrowDeviceType device_type;
+
+  /// \brief Callback to get the stream schema
+  /// (will be the same for all arrays in the stream).
+  ///
+  /// If successful, the ArrowSchema must be released independantly from the stream.
+  /// The schema should be accessible via CPU memory.
+  ///
+  /// \param[in] self The ArrowDeviceArrayStream object itself
+  /// \param[out] out C struct to export the schema to
+  /// \return 0 if successful, an `errno`-compatible error code otherwise.
+  int (*get_schema)(struct ArrowDeviceArrayStream* self, struct ArrowSchema* out);
+
+  /// \brief Callback to get the device id for the next array.
+  ///
+  /// This is necessary so that the proper/correct stream pointer can be provided
+  /// to get_next.
+  ///
+  /// The next call to `get_next` should provide an ArrowDeviceArray whose
+  /// device_id matches what is provided here, and whose device_type is the
+  /// same as the device_type member of this stream.
+  ///
+  /// \param[in] self The ArrowDeviceArrayStream object itself
+  /// \param[out] out_device_id Pointer to be populated with the device id, must not be
+  /// null \return 0 if successful, an `errno`-compatible error code otherwise.
+  int (*get_next_device_id)(struct ArrowDeviceArrayStream* self, int64_t* out_device_id);
+
+  /// \brief Callback to get the next array
+  ///
+  /// If there is no error and the returned array has been released, the stream
+  /// has ended. If successful, the ArrowArray must be released independently
+  /// from the stream.
+  ///
+  /// Because different frameworks use different types to represent this, we
+  /// accept a void* which should then be reinterpreted into whatever the
+  /// appropriate type is (e.g. cudaStream_t) for use by the producer.
+  ///
+  /// \param[in] self The ArrowDeviceArrayStream object itself
+  /// \param[in] queue_ptr The appropriate queue, stream, or
+  /// equivalent object for the device that the data is allocated on
+  /// to indicate where the consumer wants the data to be accessible.
+  /// If queue_ptr is NULL then the default stream (e.g. CUDA stream 0)
+  /// should be used to ensure that the memory is accessible from any stream.

Review Comment:
   Since it seems we're going to take the Producer providing an event path, there isn't really a need for the `get_next_device_id` callback anymore, correct? Or am I missing something?



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [arrow] JohnZed commented on a diff in pull request #34972: GH-34971: [Format] Enhance C-Data API to support non-cpu cases

["JohnZed (via GitHub)" <gi...@apache.org>]

JohnZed commented on code in PR #34972:
URL: https://github.com/apache/arrow/pull/34972#discussion_r1192518281


##########
cpp/src/arrow/c/abi.h:
##########
@@ -65,6 +85,110 @@ struct ArrowArray {
 
 #endif  // ARROW_C_DATA_INTERFACE
 
+#ifndef ARROW_C_DEVICE_DATA_INTERFACE
+#define ARROW_C_DEVICE_DATA_INTERFACE
+
+/// \defgroup arrow-device-types Device Types
+/// These macros are compatible with the dlpack DLDeviceType values,
+/// using the same value for each enum as the equivalent kDL<type>
+/// from dlpack.h. This list should continue to be kept in sync with
+/// the equivalent dlpack.h enum values over time to ensure
+/// compatibility, rather than potentially diverging.
+///
+/// To ensure predictability with the ABI we use macros instead of
+/// an enum so the storage type is not compiler dependent.
+///
+/// @{
+
+/// \brief DeviceType for the allocated memory
+typedef int32_t ArrowDeviceType;
+
+/// \brief CPU device, same as using ArrowArray directly
+#define ARROW_DEVICE_CPU = 1
+/// \brief CUDA GPU Device
+#define ARROW_DEVICE_CUDA = 2
+/// \brief Pinned CUDA CPU memory by cudaMallocHost
+#define ARROW_DEVICE_CUDA_HOST = 3
+/// \brief OpenCL Device
+#define ARROW_DEVICE_OPENCL = 4
+/// \brief Vulkan buffer for next-gen graphics
+#define ARROW_DEVICE_VULKAN = 7
+/// \brief Metal for Apple GPU
+#define ARROW_DEVICE_METAL = 8
+/// \brief Verilog simulator buffer
+#define ARROW_DEVICE_VPI = 9
+/// \brief ROCm GPUs for AMD GPUs
+#define ARROW_DEVICE_ROCM = 10
+/// \brief Pinned ROCm CPU memory allocated by hipMallocHost
+#define ARROW_DEVICE_ROCMHOST = 11

Review Comment:
   ```suggestion
   #define ARROW_DEVICE_ROCM_HOST = 11
   ```
   underscore to be consistent with ARROW_DEVICE_CUDA_HOST



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [arrow] zeroshade commented on a diff in pull request #34972: GH-34971: [Format] Enhance C-Data API to support non-cpu cases

["zeroshade (via GitHub)" <gi...@apache.org>]

zeroshade commented on code in PR #34972:
URL: https://github.com/apache/arrow/pull/34972#discussion_r1194007969


##########
cpp/src/arrow/c/abi.h:
##########
@@ -65,6 +85,110 @@ struct ArrowArray {
 
 #endif  // ARROW_C_DATA_INTERFACE
 
+#ifndef ARROW_C_DEVICE_DATA_INTERFACE
+#define ARROW_C_DEVICE_DATA_INTERFACE
+
+/// \defgroup arrow-device-types Device Types
+/// These macros are compatible with the dlpack DLDeviceType values,
+/// using the same value for each enum as the equivalent kDL<type>
+/// from dlpack.h. This list should continue to be kept in sync with
+/// the equivalent dlpack.h enum values over time to ensure
+/// compatibility, rather than potentially diverging.
+///
+/// To ensure predictability with the ABI we use macros instead of
+/// an enum so the storage type is not compiler dependent.
+///
+/// @{
+
+/// \brief DeviceType for the allocated memory
+typedef int32_t ArrowDeviceType;
+
+/// \brief CPU device, same as using ArrowArray directly
+#define ARROW_DEVICE_CPU = 1
+/// \brief CUDA GPU Device
+#define ARROW_DEVICE_CUDA = 2
+/// \brief Pinned CUDA CPU memory by cudaMallocHost
+#define ARROW_DEVICE_CUDA_HOST = 3
+/// \brief OpenCL Device
+#define ARROW_DEVICE_OPENCL = 4
+/// \brief Vulkan buffer for next-gen graphics
+#define ARROW_DEVICE_VULKAN = 7
+/// \brief Metal for Apple GPU
+#define ARROW_DEVICE_METAL = 8
+/// \brief Verilog simulator buffer
+#define ARROW_DEVICE_VPI = 9
+/// \brief ROCm GPUs for AMD GPUs
+#define ARROW_DEVICE_ROCM = 10
+/// \brief Pinned ROCm CPU memory allocated by hipMallocHost
+#define ARROW_DEVICE_ROCMHOST = 11
+/// \brief Reserved for extension
+///
+/// used to quickly test extension devices, semantics
+/// can differ based on the implementation
+#define ARROW_DEVICE_EXT_DEV = 12
+/// \brief CUDA managed/unified memory allocated by cudaMallocManaged
+#define ARROW_DEVICE_CUDA_MANAGED = 13
+/// \brief unified shared memory allocated on a oneAPI
+/// non-partitioned device.
+///
+/// A call to the oneAPI runtime is required to determine the device
+/// type, the USM allocation type, and the sycl context it is bound to.
+#define ARROW_DEVICE_ONEAPI = 14
+/// \brief GPU support for next-gen WebGPU standard
+#define ARROW_DEVICE_WEBGPU = 15
+/// \brief Qualcomm Hexagon DSP
+#define ARROW_DEVICE_HEXAGON = 16
+
+/// @}
+
+/// \brief Struct for passing an Arrow Array alongside
+/// device memory information.
+struct ArrowDeviceArray {
+  /// \brief the Allocated Array
+  ///
+  /// the buffers in the array (along with the buffers of any
+  /// children) are what is allocated on the device.
+  ///
+  /// the private_data and release callback of the arrow array
+  /// should contain any necessary information and structures
+  /// related to freeing the array according to the device it
+  /// is allocated on, rather than having a separate release
+  /// callback embedded here.
+  struct ArrowArray array;
+  /// \brief The device id to identify a specific device
+  /// if multiple of this type are on the system.
+  ///
+  /// the semantics of the id will be hardware dependant.
+  int64_t device_id;
+  /// \brief The type of device which can access this memory.
+  ArrowDeviceType device_type;
+  /// \brief An event-like object to synchronize on if needed.
+  ///
+  /// Many devices, like GPUs, are primarily asynchronous with
+  /// respect to CPU processing. As such in order to safely access
+  /// memory, it is often necessary to have an object to synchronize
+  /// processing on. Since different devices will use different types
+  /// to specify this we use a void* that can be coerced into
+  /// whatever the device appropriate type is (e.g. cudaEvent_t for
+  /// CUDA and hipEvent_t for HIP).
+  ///
+  /// If synchronization is not needed this can be null. If this is
+  /// non-null, then it should be used to call the appropriate sync
+  /// method for the device (e.g. cudaStreamWaitEvent / hipStreamWaitEvent).
+  void* sync_event;

Review Comment:
   Currently, in the context of CUDA/C++ the `void* sync_event` would be expected to be a `cudaEvent_t*` that the producer created and will trigger based on when the data is available.
   
   A consumer would then do something like:
   
   ```c++
   auto event = reinterpret_cast<cudaEvent_t*>(device_array->sync_event);
   cudaStreamWaitEvent(stream, event);
   // add processing for device_array data to stream to process *after* waiting on the event.
   ```
   
   As per the previous discussions, most frameworks for cuda don't actually make their internal streams easily externally available so we aren't expecting a `cudaStream_t` to get passed. In the future if usage deems it necessary, we could absolutely leverage the reserved bytes to add a stream/queue pointer or something. But the initial pass here is intended to pass an event via the `void* sync_event` that a queue can wait on from the producer and then just operate on the data from there.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [arrow] pitrou commented on a diff in pull request #34972: GH-34971: [Format] Enhance C-Data API to support non-cpu cases

["pitrou (via GitHub)" <gi...@apache.org>]

pitrou commented on code in PR #34972:
URL: https://github.com/apache/arrow/pull/34972#discussion_r1161080258


##########
cpp/src/arrow/c/abi.h:
##########
@@ -106,6 +169,77 @@ struct ArrowArrayStream {
 
 #endif  // ARROW_C_STREAM_INTERFACE
 
+#ifndef ARROW_C_DEVICE_STREAM_INTERFACE
+#define ARROW_C_DEVICE_STREAM_INTERFACE
+
+struct ArrowDeviceArrayStream {
+  // The device that this stream produces data on.
+  // All ArrowDeviceArrays that are produced by this
+  // stream should have the same device_type as set
+  // here. The device_type needs to be provided here
+  // so that consumers can provide the correct type
+  // of stream_ptr when calling get_next.
+  ArrowDeviceType device_type;
+
+  // Callback to get the stream schema
+  // (will be the same for all arrays in the stream).
+  //
+  // Return value: 0 if successful, an `errno`-compatible error code otherwise.
+  //
+  // If successful, the ArrowSchema must be released independently from the stream.
+  int (*get_schema)(struct ArrowDeviceArrayStream*, struct ArrowSchema* out);
+
+  // Callback to get the device id for the next array.
+  // This is necessary so that the proper/correct stream pointer can be provided
+  // to get_next. The parameter provided must not be null.
+  //
+  // Return value: 0 if successful, an `errno`-compatible error code otherwise.
+  //
+  // The next call to `get_next` should provide an ArrowDeviceArray whose
+  // device_id matches what is provided here, and whose device_type is the
+  // same as the device_type member of this stream.
+  int (*get_next_device_id)(struct ArrowDeviceArrayStream*, int* out_device_id);
+
+  // Callback to get the next array
+  // (if no error and the array is released, the stream has ended)
+  //
+  // the provided stream_ptr should be the appropriate stream, or
+  // equivalent object, for the device that the data is allocated on
+  // to indicate where the consumer wants the data to be accessible.
+  // if stream_ptr is NULL then the default stream (e.g. CUDA stream 0)
+  // should be used to ensure that the memory is accessible from any stream.

Review Comment:
   Does this make sense for all backends, or are we trying to encode device-specific behaviour in a generic API?



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [arrow] zeroshade commented on a diff in pull request #34972: GH-34971: [Format] Enhance C-Data API to support non-cpu cases

["zeroshade (via GitHub)" <gi...@apache.org>]

zeroshade commented on code in PR #34972:
URL: https://github.com/apache/arrow/pull/34972#discussion_r1161978769


##########
cpp/src/arrow/c/abi.h:
##########
@@ -106,6 +212,98 @@ struct ArrowArrayStream {
 
 #endif  // ARROW_C_STREAM_INTERFACE
 
+#ifndef ARROW_C_DEVICE_STREAM_INTERFACE
+#define ARROW_C_DEVICE_STREAM_INTERFACE
+
+/// \brief Equivalent to ArrowArrayStream, but for ArrowDeviceArrays.
+///
+/// This stream is intended to provide a stream of data on a single
+/// device, if a producer wants data to be produced on multiple devices
+/// then multiple streams should be provided. One per device.
+struct ArrowDeviceArrayStream {
+  /// \brief The device that this stream produces data on.
+  ///
+  /// All ArrowDeviceArrays that are produced by this
+  /// stream should have the same device_type as set
+  /// here. The device_type needs to be provided here
+  /// so that consumers can provide the correct type
+  /// of queue_ptr when calling get_next.
+  ArrowDeviceType device_type;
+
+  /// \brief Callback to get the stream schema
+  /// (will be the same for all arrays in the stream).
+  ///
+  /// If successful, the ArrowSchema must be released independantly from the stream.
+  /// The schema should be accessible via CPU memory.
+  ///
+  /// \param[in] self The ArrowDeviceArrayStream object itself
+  /// \param[out] out C struct to export the schema to
+  /// \return 0 if successful, an `errno`-compatible error code otherwise.
+  int (*get_schema)(struct ArrowDeviceArrayStream* self, struct ArrowSchema* out);
+
+  /// \brief Callback to get the device id for the next array.
+  ///
+  /// This is necessary so that the proper/correct stream pointer can be provided
+  /// to get_next.
+  ///
+  /// The next call to `get_next` should provide an ArrowDeviceArray whose
+  /// device_id matches what is provided here, and whose device_type is the
+  /// same as the device_type member of this stream.

Review Comment:
   no, I was referring to the `device_id` member and `device_type` member that get populated in the `ArrowDeviceArray` that is returned from `get_next`



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [arrow] kkraus14 commented on a diff in pull request #34972: GH-34971: [Format] Enhance C-Data API to support non-cpu cases

["kkraus14 (via GitHub)" <gi...@apache.org>]

kkraus14 commented on code in PR #34972:
URL: https://github.com/apache/arrow/pull/34972#discussion_r1162969100


##########
cpp/src/arrow/c/abi.h:
##########
@@ -106,6 +212,98 @@ struct ArrowArrayStream {
 
 #endif  // ARROW_C_STREAM_INTERFACE
 
+#ifndef ARROW_C_DEVICE_STREAM_INTERFACE
+#define ARROW_C_DEVICE_STREAM_INTERFACE
+
+/// \brief Equivalent to ArrowArrayStream, but for ArrowDeviceArrays.
+///
+/// This stream is intended to provide a stream of data on a single
+/// device, if a producer wants data to be produced on multiple devices
+/// then multiple streams should be provided. One per device.
+struct ArrowDeviceArrayStream {
+  /// \brief The device that this stream produces data on.
+  ///
+  /// All ArrowDeviceArrays that are produced by this
+  /// stream should have the same device_type as set
+  /// here. The device_type needs to be provided here
+  /// so that consumers can provide the correct type
+  /// of queue_ptr when calling get_next.
+  ArrowDeviceType device_type;
+
+  /// \brief Callback to get the stream schema
+  /// (will be the same for all arrays in the stream).
+  ///
+  /// If successful, the ArrowSchema must be released independantly from the stream.
+  /// The schema should be accessible via CPU memory.
+  ///
+  /// \param[in] self The ArrowDeviceArrayStream object itself
+  /// \param[out] out C struct to export the schema to
+  /// \return 0 if successful, an `errno`-compatible error code otherwise.
+  int (*get_schema)(struct ArrowDeviceArrayStream* self, struct ArrowSchema* out);
+
+  /// \brief Callback to get the device id for the next array.
+  ///
+  /// This is necessary so that the proper/correct stream pointer can be provided
+  /// to get_next.
+  ///
+  /// The next call to `get_next` should provide an ArrowDeviceArray whose
+  /// device_id matches what is provided here, and whose device_type is the
+  /// same as the device_type member of this stream.
+  ///
+  /// \param[in] self The ArrowDeviceArrayStream object itself
+  /// \param[out] out_device_id Pointer to be populated with the device id, must not be
+  /// null \return 0 if successful, an `errno`-compatible error code otherwise.
+  int (*get_next_device_id)(struct ArrowDeviceArrayStream* self, int64_t* out_device_id);
+
+  /// \brief Callback to get the next array
+  ///
+  /// If there is no error and the returned array has been released, the stream
+  /// has ended. If successful, the ArrowArray must be released independently
+  /// from the stream.
+  ///
+  /// Because different frameworks use different types to represent this, we
+  /// accept a void* which should then be reinterpreted into whatever the
+  /// appropriate type is (e.g. cudaStream_t) for use by the producer.
+  ///
+  /// \param[in] self The ArrowDeviceArrayStream object itself
+  /// \param[in] queue_ptr The appropriate queue, stream, or
+  /// equivalent object for the device that the data is allocated on
+  /// to indicate where the consumer wants the data to be accessible.
+  /// If queue_ptr is NULL then the default stream (e.g. CUDA stream 0)
+  /// should be used to ensure that the memory is accessible from any stream.

Review Comment:
   The idea of a consumer providing a stream is that it only needs to manage the lifetime of its provided stream. A flow would look something like:
   
   - Producer creates stream 3
   - Producer runs some kernel that produces data safe to consume on stream 3
   - Consumer creates stream 4
   - Consumer calls Producer API asking for C Data Interface and provides it stream 4
   - Producer creates an Event (see cudaEventCreate or hipEventCreate) and records it on stream 3
   - Producer waits the passed in stream 4 on the event (see cudaStreamWaitEvent or hipStreamWaitEvent)
   - Producer returns the C Data Interface struct to the consumer
   
   With this, the producer's stream 3 can be destroyed at any point safely after the C Data Interface struct is returned so you don't need to worry about two different libraries / frameworks needing to be able to control the lifetimes of the others' streams.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [arrow] pitrou commented on a diff in pull request #34972: GH-34971: [Format] Enhance C-Data API to support non-cpu cases

["pitrou (via GitHub)" <gi...@apache.org>]

pitrou commented on code in PR #34972:
URL: https://github.com/apache/arrow/pull/34972#discussion_r1161991117


##########
cpp/src/arrow/c/abi.h:
##########
@@ -106,6 +212,98 @@ struct ArrowArrayStream {
 
 #endif  // ARROW_C_STREAM_INTERFACE
 
+#ifndef ARROW_C_DEVICE_STREAM_INTERFACE
+#define ARROW_C_DEVICE_STREAM_INTERFACE
+
+/// \brief Equivalent to ArrowArrayStream, but for ArrowDeviceArrays.
+///
+/// This stream is intended to provide a stream of data on a single
+/// device, if a producer wants data to be produced on multiple devices
+/// then multiple streams should be provided. One per device.
+struct ArrowDeviceArrayStream {
+  /// \brief The device that this stream produces data on.
+  ///
+  /// All ArrowDeviceArrays that are produced by this
+  /// stream should have the same device_type as set
+  /// here. The device_type needs to be provided here
+  /// so that consumers can provide the correct type
+  /// of queue_ptr when calling get_next.
+  ArrowDeviceType device_type;
+
+  /// \brief Callback to get the stream schema
+  /// (will be the same for all arrays in the stream).
+  ///
+  /// If successful, the ArrowSchema must be released independantly from the stream.
+  /// The schema should be accessible via CPU memory.
+  ///
+  /// \param[in] self The ArrowDeviceArrayStream object itself
+  /// \param[out] out C struct to export the schema to
+  /// \return 0 if successful, an `errno`-compatible error code otherwise.
+  int (*get_schema)(struct ArrowDeviceArrayStream* self, struct ArrowSchema* out);
+
+  /// \brief Callback to get the device id for the next array.
+  ///
+  /// This is necessary so that the proper/correct stream pointer can be provided
+  /// to get_next.
+  ///
+  /// The next call to `get_next` should provide an ArrowDeviceArray whose
+  /// device_id matches what is provided here, and whose device_type is the
+  /// same as the device_type member of this stream.
+  ///
+  /// \param[in] self The ArrowDeviceArrayStream object itself
+  /// \param[out] out_device_id Pointer to be populated with the device id, must not be
+  /// null \return 0 if successful, an `errno`-compatible error code otherwise.
+  int (*get_next_device_id)(struct ArrowDeviceArrayStream* self, int64_t* out_device_id);
+
+  /// \brief Callback to get the next array
+  ///
+  /// If there is no error and the returned array has been released, the stream
+  /// has ended. If successful, the ArrowArray must be released independently
+  /// from the stream.
+  ///
+  /// Because different frameworks use different types to represent this, we
+  /// accept a void* which should then be reinterpreted into whatever the
+  /// appropriate type is (e.g. cudaStream_t) for use by the producer.
+  ///
+  /// \param[in] self The ArrowDeviceArrayStream object itself
+  /// \param[in] queue_ptr The appropriate queue, stream, or
+  /// equivalent object for the device that the data is allocated on
+  /// to indicate where the consumer wants the data to be accessible.
+  /// If queue_ptr is NULL then the default stream (e.g. CUDA stream 0)
+  /// should be used to ensure that the memory is accessible from any stream.

Review Comment:
   The standard "mechanism to share a queue and control its lifetime over a boundary like this" in the C Data Interface would be the release callback.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [arrow] pitrou commented on a diff in pull request #34972: GH-34971: [Format] Enhance C-Data API to support non-cpu cases

["pitrou (via GitHub)" <gi...@apache.org>]

pitrou commented on code in PR #34972:
URL: https://github.com/apache/arrow/pull/34972#discussion_r1161305258


##########
cpp/src/arrow/c/abi.h:
##########
@@ -65,6 +65,69 @@ struct ArrowArray {
 
 #endif  // ARROW_C_DATA_INTERFACE
 
+#ifndef ARROW_C_DEVICE_DATA_INTERFACE
+#define ARROW_C_DEVICE_DATA_INTERFACE
+
+// ArrowDeviceType is compatible with dlpack DLDeviceType for portability
+// it uses the same values for each enum as the equivalent kDL<type> from dlpack.h
+#ifdef __cplusplus
+typedef enum : int32_t {
+#else
+typedef enum {
+#endif

Review Comment:
   @paleolimbot is right, I think, that this threatens to make the ABI incompatible between C and C++. We could either keep this an untyped enum, or switch to macros as suggested.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [arrow] pitrou commented on a diff in pull request #34972: GH-34971: [Format] Enhance C-Data API to support non-cpu cases

["pitrou (via GitHub)" <gi...@apache.org>]

pitrou commented on code in PR #34972:
URL: https://github.com/apache/arrow/pull/34972#discussion_r1163075318


##########
cpp/src/arrow/c/abi.h:
##########
@@ -106,6 +212,98 @@ struct ArrowArrayStream {
 
 #endif  // ARROW_C_STREAM_INTERFACE
 
+#ifndef ARROW_C_DEVICE_STREAM_INTERFACE
+#define ARROW_C_DEVICE_STREAM_INTERFACE
+
+/// \brief Equivalent to ArrowArrayStream, but for ArrowDeviceArrays.
+///
+/// This stream is intended to provide a stream of data on a single
+/// device, if a producer wants data to be produced on multiple devices
+/// then multiple streams should be provided. One per device.
+struct ArrowDeviceArrayStream {
+  /// \brief The device that this stream produces data on.
+  ///
+  /// All ArrowDeviceArrays that are produced by this
+  /// stream should have the same device_type as set
+  /// here. The device_type needs to be provided here
+  /// so that consumers can provide the correct type
+  /// of queue_ptr when calling get_next.
+  ArrowDeviceType device_type;
+
+  /// \brief Callback to get the stream schema
+  /// (will be the same for all arrays in the stream).
+  ///
+  /// If successful, the ArrowSchema must be released independantly from the stream.
+  /// The schema should be accessible via CPU memory.
+  ///
+  /// \param[in] self The ArrowDeviceArrayStream object itself
+  /// \param[out] out C struct to export the schema to
+  /// \return 0 if successful, an `errno`-compatible error code otherwise.
+  int (*get_schema)(struct ArrowDeviceArrayStream* self, struct ArrowSchema* out);
+
+  /// \brief Callback to get the device id for the next array.
+  ///
+  /// This is necessary so that the proper/correct stream pointer can be provided
+  /// to get_next.
+  ///
+  /// The next call to `get_next` should provide an ArrowDeviceArray whose
+  /// device_id matches what is provided here, and whose device_type is the
+  /// same as the device_type member of this stream.

Review Comment:
   >     * [Specify synchronization semantics dmlc/dlpack#57](https://github.com/dmlc/dlpack/issues/57)
   
   I tried to read through this and did not find a convincing argument that we shouldn't let the producer expose the original stream value. The entire discussion seemed to revolve around API convenience, not correctness. Also, people there seem to agree that the CUDA Array Interface from Numba is sane.
   
   Did I miss something?
   



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [arrow] pitrou commented on pull request #34972: GH-34971: [Format] Enhance C-Data API to support non-cpu cases

["pitrou (via GitHub)" <gi...@apache.org>]

pitrou commented on PR #34972:
URL: https://github.com/apache/arrow/pull/34972#issuecomment-1505254070

   (also, to make things clear, I am not saying this proposal is wrong; I just want to make sure we evaluate the issues carefully and accurately - hence the questions)


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [arrow] pitrou commented on pull request #34972: GH-34971: [Format] Enhance C-Data API to support non-cpu cases

["pitrou (via GitHub)" <gi...@apache.org>]

pitrou commented on PR #34972:
URL: https://github.com/apache/arrow/pull/34972#issuecomment-1520360143

   > @pitrou Do we need an equivalent for `Event` or just use a `void*` and let other backends use whatever they would need appropriately?
   
   Probably a `int64_t`, so that the event can fit in the field regardless of the backend (some backends may use a pointer, but some may use some kind of opaque handle).
   
   However, we should probably define what the field _means_ for each backend. Otherwise we'll have a half-spec that nobody can implement rigorously.
   
   Also, I think this means the device array must have its own release callback, to properly handle the lifetime of said event.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [arrow] kurapov-peter commented on a diff in pull request #34972: GH-34971: [Format] Enhance C-Data API to support non-cpu cases

["kurapov-peter (via GitHub)" <gi...@apache.org>]

kurapov-peter commented on code in PR #34972:
URL: https://github.com/apache/arrow/pull/34972#discussion_r1164469074


##########
cpp/src/arrow/c/abi.h:
##########
@@ -65,6 +65,69 @@ struct ArrowArray {
 
 #endif  // ARROW_C_DATA_INTERFACE
 
+#ifndef ARROW_C_DEVICE_DATA_INTERFACE
+#define ARROW_C_DEVICE_DATA_INTERFACE
+
+// ArrowDeviceType is compatible with dlpack DLDeviceType for portability
+// it uses the same values for each enum as the equivalent kDL<type> from dlpack.h
+#ifdef __cplusplus
+typedef enum : int32_t {
+#else
+typedef enum {
+#endif
+  // CPU device, same as using ArrowArray directly
+  kArrowCPU = 1,
+  // CUDA GPU Device
+  kArrowCUDA = 2,
+  // Pinned CUDA CPU memory by cudaMallocHost
+  kArrowCUDAHost = 3,
+  // OpenCL Device
+  kArrowOpenCL = 4,
+  // Vulkan buffer for next-gen graphics
+  kArrowVulkan = 7,
+  // Metal for Apple GPU
+  kArrowMetal = 8,
+  // Verilog simulator buffer
+  kArrowVPI = 9,
+  // ROCm GPUs for AMD GPUs
+  kArrowROCM = 10,
+  // Pinned ROCm CPU memory allocated by hipMallocHost
+  kArrowROCMHost = 11,
+  // Reserved for extension
+  // used to quickly test extension devices,
+  // semantics can differ based on the implementation
+  kArrowExtDev = 12,
+  // CUDA managed/unified memory allocated by cudaMallocManaged
+  kArrowCUDAManaged = 13,
+  // unified shared memory allocated on a oneAPI non-partitioned
+  // device. call to oneAPI runtime is required to determine the
+  // device type, the USM allocation type and the sycl context it
+  // is bound to
+  kArrowOneAPI = 14,
+  // GPU support for next-gen WebGPU standard
+  kArrowWebGPU = 15,
+  // Qualcomm Hexagon DSP
+  kArrowHexagon = 16,
+} ArrowDeviceType;
+
+struct ArrowDeviceArray {
+  // the private_date and release callback of the arrow array
+  // should contain any necessary information and structures
+  // related to freeing the array according to the device it
+  // is allocated on, rather than having a separate release
+  // callback embedded here.
+  struct ArrowArray* array;
+  int device_id;

Review Comment:
   > CUDA (NVIDIA GPUs) and ROCm (AMD GPUs) both use `int` to specify a device ID in their APIs.
   > 
   > SYCL uses a `cl_device_id` which is an opaque type where I'm not sure if an `int32_t` or `int64_t` is sufficient to capture it. Maybe @aregm could have someone help guide here?
   > 
   > dlpack has historically used an `int` / `int32_t` for 6+ years.
   
   DPC++ uses level zero (https://github.com/oneapi-src/level-zero/tree/master) as the primary C API. As to the device ID it is stored as `uint32_t` currently (see https://spec.oneapi.io/level-zero/latest/core/api.html#_CPPv422ze_device_properties_t) and it is not expected to exceed `int64_t`, so +1 as well.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [arrow] pitrou commented on a diff in pull request #34972: GH-34971: [Format] Enhance C-Data API to support non-cpu cases

["pitrou (via GitHub)" <gi...@apache.org>]

pitrou commented on code in PR #34972:
URL: https://github.com/apache/arrow/pull/34972#discussion_r1162448210


##########
cpp/src/arrow/c/abi.h:
##########
@@ -106,6 +212,98 @@ struct ArrowArrayStream {
 
 #endif  // ARROW_C_STREAM_INTERFACE
 
+#ifndef ARROW_C_DEVICE_STREAM_INTERFACE
+#define ARROW_C_DEVICE_STREAM_INTERFACE
+
+/// \brief Equivalent to ArrowArrayStream, but for ArrowDeviceArrays.
+///
+/// This stream is intended to provide a stream of data on a single
+/// device, if a producer wants data to be produced on multiple devices
+/// then multiple streams should be provided. One per device.
+struct ArrowDeviceArrayStream {
+  /// \brief The device that this stream produces data on.
+  ///
+  /// All ArrowDeviceArrays that are produced by this
+  /// stream should have the same device_type as set
+  /// here. The device_type needs to be provided here
+  /// so that consumers can provide the correct type
+  /// of queue_ptr when calling get_next.
+  ArrowDeviceType device_type;
+
+  /// \brief Callback to get the stream schema
+  /// (will be the same for all arrays in the stream).
+  ///
+  /// If successful, the ArrowSchema must be released independantly from the stream.
+  /// The schema should be accessible via CPU memory.
+  ///
+  /// \param[in] self The ArrowDeviceArrayStream object itself
+  /// \param[out] out C struct to export the schema to
+  /// \return 0 if successful, an `errno`-compatible error code otherwise.
+  int (*get_schema)(struct ArrowDeviceArrayStream* self, struct ArrowSchema* out);
+
+  /// \brief Callback to get the device id for the next array.
+  ///
+  /// This is necessary so that the proper/correct stream pointer can be provided
+  /// to get_next.
+  ///
+  /// The next call to `get_next` should provide an ArrowDeviceArray whose
+  /// device_id matches what is provided here, and whose device_type is the
+  /// same as the device_type member of this stream.

Review Comment:
   > the existing frameworks and APIs don't provide any good way to manage the stream's lifetime easily
   
   What do you mean by that? Would you care to give a more concrete example? For example CUDA allows you to destroy a stream:
   https://docs.nvidia.com/cuda/cuda-runtime-api/group__CUDART__STREAM.html#group__CUDART__STREAM_1gfda584f1788ca983cb21c5f4d2033a62
   



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [arrow] lidavidm commented on a diff in pull request #34972: GH-34971: [Format] Enhance C-Data API to support non-cpu cases

["lidavidm (via GitHub)" <gi...@apache.org>]

lidavidm commented on code in PR #34972:
URL: https://github.com/apache/arrow/pull/34972#discussion_r1161362955


##########
cpp/src/arrow/c/abi.h:
##########
@@ -106,6 +169,77 @@ struct ArrowArrayStream {
 
 #endif  // ARROW_C_STREAM_INTERFACE
 
+#ifndef ARROW_C_DEVICE_STREAM_INTERFACE
+#define ARROW_C_DEVICE_STREAM_INTERFACE
+
+struct ArrowDeviceArrayStream {
+  // The device that this stream produces data on.
+  // All ArrowDeviceArrays that are produced by this
+  // stream should have the same device_type as set
+  // here. The device_type needs to be provided here
+  // so that consumers can provide the correct type
+  // of stream_ptr when calling get_next.
+  ArrowDeviceType device_type;
+
+  // Callback to get the stream schema
+  // (will be the same for all arrays in the stream).
+  //
+  // Return value: 0 if successful, an `errno`-compatible error code otherwise.
+  //
+  // If successful, the ArrowSchema must be released independently from the stream.
+  int (*get_schema)(struct ArrowDeviceArrayStream*, struct ArrowSchema* out);
+
+  // Callback to get the device id for the next array.
+  // This is necessary so that the proper/correct stream pointer can be provided
+  // to get_next. The parameter provided must not be null.
+  //
+  // Return value: 0 if successful, an `errno`-compatible error code otherwise.
+  //
+  // The next call to `get_next` should provide an ArrowDeviceArray whose
+  // device_id matches what is provided here, and whose device_type is the
+  // same as the device_type member of this stream.
+  int (*get_next_device_id)(struct ArrowDeviceArrayStream*, int* out_device_id);
+
+  // Callback to get the next array
+  // (if no error and the array is released, the stream has ended)
+  //
+  // the provided stream_ptr should be the appropriate stream, or

Review Comment:
   I agree with Antoine's comment above that we need a different term for this in this context. But my general point is that it would help to have a set of definitions up front. As it stands this seems to assume the reader is already very familiar with CUDA specifically. That may not be great if the idea is to support non-CPU memory more generally.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [arrow] kkraus14 commented on a diff in pull request #34972: GH-34971: [Format] Enhance C-Data API to support non-cpu cases

["kkraus14 (via GitHub)" <gi...@apache.org>]

kkraus14 commented on code in PR #34972:
URL: https://github.com/apache/arrow/pull/34972#discussion_r1161875449


##########
cpp/src/arrow/c/abi.h:
##########
@@ -65,6 +65,69 @@ struct ArrowArray {
 
 #endif  // ARROW_C_DATA_INTERFACE
 
+#ifndef ARROW_C_DEVICE_DATA_INTERFACE
+#define ARROW_C_DEVICE_DATA_INTERFACE
+
+// ArrowDeviceType is compatible with dlpack DLDeviceType for portability
+// it uses the same values for each enum as the equivalent kDL<type> from dlpack.h
+#ifdef __cplusplus
+typedef enum : int32_t {
+#else
+typedef enum {
+#endif
+  // CPU device, same as using ArrowArray directly
+  kArrowCPU = 1,
+  // CUDA GPU Device
+  kArrowCUDA = 2,
+  // Pinned CUDA CPU memory by cudaMallocHost
+  kArrowCUDAHost = 3,
+  // OpenCL Device
+  kArrowOpenCL = 4,
+  // Vulkan buffer for next-gen graphics
+  kArrowVulkan = 7,
+  // Metal for Apple GPU
+  kArrowMetal = 8,
+  // Verilog simulator buffer
+  kArrowVPI = 9,
+  // ROCm GPUs for AMD GPUs
+  kArrowROCM = 10,
+  // Pinned ROCm CPU memory allocated by hipMallocHost
+  kArrowROCMHost = 11,
+  // Reserved for extension
+  // used to quickly test extension devices,
+  // semantics can differ based on the implementation
+  kArrowExtDev = 12,
+  // CUDA managed/unified memory allocated by cudaMallocManaged
+  kArrowCUDAManaged = 13,
+  // unified shared memory allocated on a oneAPI non-partitioned
+  // device. call to oneAPI runtime is required to determine the
+  // device type, the USM allocation type and the sycl context it
+  // is bound to
+  kArrowOneAPI = 14,
+  // GPU support for next-gen WebGPU standard
+  kArrowWebGPU = 15,
+  // Qualcomm Hexagon DSP
+  kArrowHexagon = 16,
+} ArrowDeviceType;
+
+struct ArrowDeviceArray {
+  // the private_date and release callback of the arrow array
+  // should contain any necessary information and structures
+  // related to freeing the array according to the device it
+  // is allocated on, rather than having a separate release
+  // callback embedded here.
+  struct ArrowArray* array;
+  int device_id;

Review Comment:
   CUDA (NVIDIA GPUs) and ROCm (AMD GPUs) both use `int` to specify a device ID in their APIs.
   
   SYCL uses a `cl_device_id` which is an opaque type where I'm not sure if an `int32_t` or `int64_t` is sufficient to capture it. Maybe @aregm could have someone help guide here? 
   
   dlpack has historically used an `int` / `int32_t` for 6+ years.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [arrow] pitrou commented on pull request #34972: GH-34971: [Format] Enhance C-Data API to support non-cpu cases

["pitrou (via GitHub)" <gi...@apache.org>]

pitrou commented on PR #34972:
URL: https://github.com/apache/arrow/pull/34972#issuecomment-1504976683

   Ok, who is supposed to be the consumer of the C Device Data Interface? I would expect it to be an Arrow implementation, or an Arrow-compatible library. Realistically they probably already deal with CUDA streams if they support CUDA?
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [arrow] zeroshade commented on a diff in pull request #34972: GH-34971: [Format] Enhance C-Data API to support non-cpu cases

["zeroshade (via GitHub)" <gi...@apache.org>]

zeroshade commented on code in PR #34972:
URL: https://github.com/apache/arrow/pull/34972#discussion_r1162002035


##########
cpp/src/arrow/c/abi.h:
##########
@@ -106,6 +169,77 @@ struct ArrowArrayStream {
 
 #endif  // ARROW_C_STREAM_INTERFACE
 
+#ifndef ARROW_C_DEVICE_STREAM_INTERFACE
+#define ARROW_C_DEVICE_STREAM_INTERFACE
+
+struct ArrowDeviceArrayStream {
+  // The device that this stream produces data on.
+  // All ArrowDeviceArrays that are produced by this
+  // stream should have the same device_type as set
+  // here. The device_type needs to be provided here
+  // so that consumers can provide the correct type
+  // of stream_ptr when calling get_next.
+  ArrowDeviceType device_type;
+
+  // Callback to get the stream schema
+  // (will be the same for all arrays in the stream).
+  //
+  // Return value: 0 if successful, an `errno`-compatible error code otherwise.
+  //
+  // If successful, the ArrowSchema must be released independently from the stream.
+  int (*get_schema)(struct ArrowDeviceArrayStream*, struct ArrowSchema* out);
+
+  // Callback to get the device id for the next array.
+  // This is necessary so that the proper/correct stream pointer can be provided
+  // to get_next. The parameter provided must not be null.
+  //
+  // Return value: 0 if successful, an `errno`-compatible error code otherwise.
+  //
+  // The next call to `get_next` should provide an ArrowDeviceArray whose
+  // device_id matches what is provided here, and whose device_type is the
+  // same as the device_type member of this stream.
+  int (*get_next_device_id)(struct ArrowDeviceArrayStream*, int* out_device_id);
+
+  // Callback to get the next array
+  // (if no error and the array is released, the stream has ended)
+  //
+  // the provided stream_ptr should be the appropriate stream, or
+  // equivalent object, for the device that the data is allocated on
+  // to indicate where the consumer wants the data to be accessible.

Review Comment:
   Correct. This is why there isn't a `void*` in the `ArrowDeviceArray` struct itself.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [arrow] pitrou commented on a diff in pull request #34972: GH-34971: [Format] Enhance C-Data API to support non-cpu cases

["pitrou (via GitHub)" <gi...@apache.org>]

pitrou commented on code in PR #34972:
URL: https://github.com/apache/arrow/pull/34972#discussion_r1161304981


##########
cpp/src/arrow/c/abi.h:
##########
@@ -106,6 +169,77 @@ struct ArrowArrayStream {
 
 #endif  // ARROW_C_STREAM_INTERFACE
 
+#ifndef ARROW_C_DEVICE_STREAM_INTERFACE
+#define ARROW_C_DEVICE_STREAM_INTERFACE
+
+struct ArrowDeviceArrayStream {
+  // The device that this stream produces data on.
+  // All ArrowDeviceArrays that are produced by this
+  // stream should have the same device_type as set
+  // here. The device_type needs to be provided here
+  // so that consumers can provide the correct type
+  // of stream_ptr when calling get_next.
+  ArrowDeviceType device_type;
+
+  // Callback to get the stream schema
+  // (will be the same for all arrays in the stream).
+  //
+  // Return value: 0 if successful, an `errno`-compatible error code otherwise.
+  //
+  // If successful, the ArrowSchema must be released independently from the stream.
+  int (*get_schema)(struct ArrowDeviceArrayStream*, struct ArrowSchema* out);
+
+  // Callback to get the device id for the next array.
+  // This is necessary so that the proper/correct stream pointer can be provided
+  // to get_next. The parameter provided must not be null.
+  //
+  // Return value: 0 if successful, an `errno`-compatible error code otherwise.
+  //
+  // The next call to `get_next` should provide an ArrowDeviceArray whose
+  // device_id matches what is provided here, and whose device_type is the
+  // same as the device_type member of this stream.
+  int (*get_next_device_id)(struct ArrowDeviceArrayStream*, int* out_device_id);
+
+  // Callback to get the next array
+  // (if no error and the array is released, the stream has ended)
+  //
+  // the provided stream_ptr should be the appropriate stream, or
+  // equivalent object, for the device that the data is allocated on
+  // to indicate where the consumer wants the data to be accessible.

Review Comment:
   This seems a bit gratuitously inconsistent with the handling of the lifetime of memory buffers (which goes through the release callback, telling the producer when it can safely release them).
   
   It is also gratuitously inconsistent with Numba's CUDA array interface, and I would trust the Numba developers' experience here.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [arrow] kkraus14 commented on a diff in pull request #34972: GH-34971: [Format] Enhance C-Data API to support non-cpu cases

["kkraus14 (via GitHub)" <gi...@apache.org>]

kkraus14 commented on code in PR #34972:
URL: https://github.com/apache/arrow/pull/34972#discussion_r1161837328


##########
cpp/src/arrow/c/abi.h:
##########
@@ -106,6 +169,77 @@ struct ArrowArrayStream {
 
 #endif  // ARROW_C_STREAM_INTERFACE
 
+#ifndef ARROW_C_DEVICE_STREAM_INTERFACE
+#define ARROW_C_DEVICE_STREAM_INTERFACE
+
+struct ArrowDeviceArrayStream {
+  // The device that this stream produces data on.
+  // All ArrowDeviceArrays that are produced by this
+  // stream should have the same device_type as set
+  // here. The device_type needs to be provided here
+  // so that consumers can provide the correct type
+  // of stream_ptr when calling get_next.
+  ArrowDeviceType device_type;
+
+  // Callback to get the stream schema
+  // (will be the same for all arrays in the stream).
+  //
+  // Return value: 0 if successful, an `errno`-compatible error code otherwise.
+  //
+  // If successful, the ArrowSchema must be released independently from the stream.
+  int (*get_schema)(struct ArrowDeviceArrayStream*, struct ArrowSchema* out);
+
+  // Callback to get the device id for the next array.
+  // This is necessary so that the proper/correct stream pointer can be provided
+  // to get_next. The parameter provided must not be null.
+  //
+  // Return value: 0 if successful, an `errno`-compatible error code otherwise.
+  //
+  // The next call to `get_next` should provide an ArrowDeviceArray whose
+  // device_id matches what is provided here, and whose device_type is the
+  // same as the device_type member of this stream.
+  int (*get_next_device_id)(struct ArrowDeviceArrayStream*, int* out_device_id);
+
+  // Callback to get the next array
+  // (if no error and the array is released, the stream has ended)
+  //
+  // the provided stream_ptr should be the appropriate stream, or
+  // equivalent object, for the device that the data is allocated on
+  // to indicate where the consumer wants the data to be accessible.

Review Comment:
   I drove a lot of the development on the `__cuda_array_interface__` with @seibert and @sklam and the stream was added later and I haven't seen it used. The challenge is that most frameworks internally manage their streams and don't have the ability to share a stream and control its lifetime nicely.
   
   The idea of a consumer of the interface passing a stream to the producer of the interface is that the producer can guarantee that the data is safe to consume on the passed in stream. A bunch of array library maintainers including folks from Tensorflow, PyTorch, JAX, CuPy, etc. all agreed on these semantics: https://data-apis.org/array-api/latest/API_specification/generated/array_api.array.__dlpack__.html



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [arrow] oleksandr-pavlyk commented on pull request #34972: GH-34971: [Format] Enhance C-Data API to support non-cpu cases

["oleksandr-pavlyk (via GitHub)" <gi...@apache.org>]

oleksandr-pavlyk commented on PR #34972:
URL: https://github.com/apache/arrow/pull/34972#issuecomment-1522131776

   I was referring to the comment as I understood it:
   
   > Just to clarify: are we considering that we'd add the Event to the ArrowDeviceArray struct and remove the stream argument from the get_next ?
   
   In SYCL, the stream corresponds to in-order `sycl::queue` which provides a way to retrieve the `sycl::context` associated with USM allocation. Now that I reread your comment, I see that I _misunderstood_ the comment. It is not stating that stream is going to be removed from the `ArrowDeviceArray` structure, but rather only from arguments of `get_next`.
   
   Focusing on the synchronization question part only from now on. Event-based synchronization is _cheaper_ than stream-based one (since it requires submitting queue barriers), at least in the SYCL world. Additionally, stream-based synchronization is not even possible with current SYCL 2020 without leveraging `oneapi_queue_barrier` extension. This may be a weaker argument since the entire enterprise of sharing USM allocations without explicitly sharing underlying context requires an extensions too.
   
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [arrow] kkraus14 commented on a diff in pull request #34972: GH-34971: [Format] Enhance C-Data API to support non-cpu cases

["kkraus14 (via GitHub)" <gi...@apache.org>]

kkraus14 commented on code in PR #34972:
URL: https://github.com/apache/arrow/pull/34972#discussion_r1192657775


##########
cpp/src/arrow/c/abi.h:
##########
@@ -65,6 +85,110 @@ struct ArrowArray {
 
 #endif  // ARROW_C_DATA_INTERFACE
 
+#ifndef ARROW_C_DEVICE_DATA_INTERFACE
+#define ARROW_C_DEVICE_DATA_INTERFACE
+
+/// \defgroup arrow-device-types Device Types
+/// These macros are compatible with the dlpack DLDeviceType values,
+/// using the same value for each enum as the equivalent kDL<type>
+/// from dlpack.h. This list should continue to be kept in sync with
+/// the equivalent dlpack.h enum values over time to ensure
+/// compatibility, rather than potentially diverging.
+///
+/// To ensure predictability with the ABI we use macros instead of
+/// an enum so the storage type is not compiler dependent.
+///
+/// @{
+
+/// \brief DeviceType for the allocated memory
+typedef int32_t ArrowDeviceType;
+
+/// \brief CPU device, same as using ArrowArray directly
+#define ARROW_DEVICE_CPU = 1
+/// \brief CUDA GPU Device
+#define ARROW_DEVICE_CUDA = 2
+/// \brief Pinned CUDA CPU memory by cudaMallocHost
+#define ARROW_DEVICE_CUDA_HOST = 3
+/// \brief OpenCL Device
+#define ARROW_DEVICE_OPENCL = 4
+/// \brief Vulkan buffer for next-gen graphics
+#define ARROW_DEVICE_VULKAN = 7
+/// \brief Metal for Apple GPU
+#define ARROW_DEVICE_METAL = 8
+/// \brief Verilog simulator buffer
+#define ARROW_DEVICE_VPI = 9
+/// \brief ROCm GPUs for AMD GPUs
+#define ARROW_DEVICE_ROCM = 10
+/// \brief Pinned ROCm CPU memory allocated by hipMallocHost
+#define ARROW_DEVICE_ROCMHOST = 11
+/// \brief Reserved for extension
+///
+/// used to quickly test extension devices, semantics
+/// can differ based on the implementation
+#define ARROW_DEVICE_EXT_DEV = 12
+/// \brief CUDA managed/unified memory allocated by cudaMallocManaged
+#define ARROW_DEVICE_CUDA_MANAGED = 13
+/// \brief unified shared memory allocated on a oneAPI
+/// non-partitioned device.
+///
+/// A call to the oneAPI runtime is required to determine the device
+/// type, the USM allocation type, and the sycl context it is bound to.
+#define ARROW_DEVICE_ONEAPI = 14
+/// \brief GPU support for next-gen WebGPU standard
+#define ARROW_DEVICE_WEBGPU = 15
+/// \brief Qualcomm Hexagon DSP
+#define ARROW_DEVICE_HEXAGON = 16
+
+/// @}
+
+/// \brief Struct for passing an Arrow Array alongside
+/// device memory information.
+struct ArrowDeviceArray {
+  /// \brief the Allocated Array
+  ///
+  /// the buffers in the array (along with the buffers of any
+  /// children) are what is allocated on the device.
+  ///
+  /// the private_data and release callback of the arrow array
+  /// should contain any necessary information and structures
+  /// related to freeing the array according to the device it
+  /// is allocated on, rather than having a separate release
+  /// callback embedded here.
+  struct ArrowArray array;
+  /// \brief The device id to identify a specific device
+  /// if multiple of this type are on the system.
+  ///
+  /// the semantics of the id will be hardware dependant.
+  int64_t device_id;
+  /// \brief The type of device which can access this memory.
+  ArrowDeviceType device_type;
+  /// \brief An event-like object to synchronize on if needed.
+  ///
+  /// Many devices, like GPUs, are primarily asynchronous with
+  /// respect to CPU processing. As such in order to safely access
+  /// memory, it is often necessary to have an object to synchronize
+  /// processing on. Since different devices will use different types
+  /// to specify this we use a void* that can be coerced into
+  /// whatever the device appropriate type is (e.g. cudaEvent_t for
+  /// CUDA and hipEvent_t for HIP).
+  ///
+  /// If synchronization is not needed this can be null. If this is
+  /// non-null, then it should be used to call the appropriate sync
+  /// method for the device (e.g. cudaStreamWaitEvent / hipStreamWaitEvent).
+  void* sync_event;

Review Comment:
   In the case of CUDA, this would be a pointer to a `cudaEvent_t`. It would be the producer's responsibility to create the event and call `cudaEventRecord`  to ensure the relevant work in the stream(s) are captured in the event. The consumer can then call `cudaStreamWaitEvent` which is typically a device-side more efficient stream synchronization mechanism than `cudaStreamSynchronize` (if they need to wait for host code they can still use `cudaEventSynchronize` instead).
   
   If both sides are on the same stream, then the `cudaStreamWaitEvent` call should have negligible overhead.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [arrow] leofang commented on pull request #34972: GH-34971: [Format] Enhance C-Data API to support non-cpu cases

["leofang (via GitHub)" <gi...@apache.org>]

leofang commented on PR #34972:
URL: https://github.com/apache/arrow/pull/34972#issuecomment-1545564082

   cc: @shwina for vis


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [arrow] zeroshade commented on a diff in pull request #34972: GH-34971: [Format] Enhance C-Data API to support non-cpu cases

["zeroshade (via GitHub)" <gi...@apache.org>]

zeroshade commented on code in PR #34972:
URL: https://github.com/apache/arrow/pull/34972#discussion_r1204334126


##########
docs/source/format/CDeviceDataInterface.rst:
##########
@@ -0,0 +1,613 @@
+.. Licensed to the Apache Software Foundation (ASF) under one
+.. or more contributor license agreements.  See the NOTICE file
+.. distributed with this work for additional information
+.. regarding copyright ownership.  The ASF licenses this file
+.. to you under the Apache License, Version 2.0 (the
+.. "License"); you may not use this file except in compliance
+.. with the License.  You may obtain a copy of the License at
+
+..   http://www.apache.org/licenses/LICENSE-2.0
+
+.. Unless required by applicable law or agreed to in writing,
+.. software distributed under the License is distributed on an
+.. "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+.. KIND, either express or implied.  See the License for the
+.. specific language governing permissions and limitations
+.. under the License.
+
+.. highlight:: c
+
+.. _c-device-data-interface:
+
+=================================
+The Arrow C Device data interface
+=================================
+
+.. note:: The Arrow C Device Data Interface should be considered experimental
+
+Rationale
+=========
+
+The current :ref:`C Data Interface <c-data-interface>`, and most
+implementations of it, make the assumption that all data buffers provided
+are CPU buffers. Since Apache Arrow is designed to be a universal in-memory
+format for representing tabular ("columnar") data, there will be the desire
+to leverage this data on Non-CPU hardware such as GPUs. One example of such
+a case is the `RAPIDS cuDF library` which uses the Arrow memory format with
+CUDA for NVIDIA GPUs. Since copying data from host to device and back is
+expensive, the ideal would be to be able to leave the data on the device
+for as long as possible, even when passing it between runtimes and
+libraries.
+
+The Arrow C Device data interface builds on the existing C data interface
+by adding a very small, stable set of C definitions to it. For non-C/C++
+languages and runtimes, translating the C definitions to corresponding
+C FFI declarations should be just as simple as with the current C data
+interface.
+
+Applications and libraries can then use Arrow schemas and Arrow formatted
+memory on non-cpu devices to exchange data just as easily as they do
+now with CPU data. This will enable leaving data on those devices longer
+and avoiding costly copies back and forth between the host and device
+just to leverage new libraries and runtimes.
+
+Goals
+-----
+
+* Expose an ABI-stable interface built on the existing C data interface.
+* Make it easy for third-party projects to implement support with little
+  initial investment.
+* Allow zero-copy sharing of Arrow formatted device memory between
+  independant runtimes and components running in the same process.
+* Avoid the need for one-to-one adaptation layers such as the
+  `CUDA Array Interface` for Python processes to pass CUDA data.
+* Enable integration without explicit dependencies (either at compile-time
+  or runtime) on the Arrow software project itself.
+
+The intent is for the Arrow C Device data interface to expand the reach
+of the current C data interface, allowing it to also become the standard
+low-level building block for columnar processing on devices like GPUs or
+FPGAs.
+
+Structure definitions
+=====================
+
+Because this is built on the C data interface, the C Device data interface
+uses the ``ArrowSchema`` and ``ArrowArray`` structures as defined in the
+:ref:`C data interface spec <c-data-interface-struct-defs>`. It then adds the
+following free-standing definitions. Like the rest of the Arrow project,
+they are available under the Apache License 2.0.
+
+.. code-block:: c
+
+    #ifndef ARROW_C_DEVICE_DATA_INTERFACE
+    #define ARROW_C_DEVICE_DATA_INTERFACE
+
+    // Device type for the allocated memory
+    typedef int32_t ArrowDeviceType;
+
+    // CPU device, same as using ArrowArray directly
+    #define ARROW_DEVICE_CPU 1
+    // CUDA GPU Device
+    #define ARROW_DEVICE_CUDA 2
+    // Pinned CUDA CPU memory by cudaMallocHost
+    #define ARROW_DEVICE_CUDA_HOST 3
+    // OpenCL Device
+    #define ARROW_DEVICE_OPENCL 4
+    // Vulkan buffer for next-gen graphics
+    #define ARROW_DEVICE_VULKAN 7
+    // Metal for Apple GPU
+    #define ARROW_DEVICE_METAL 8
+    // Verilog simulator buffer
+    #define ARROW_DEVICE_VPI 9
+    // ROCm GPUs for AMD GPUs
+    #define ARROW_DEVICE_ROCM 10
+    // Pinned ROCm CPU memory allocated by hipMallocHost
+    #define ARROW_DEVICE_ROCM_HOST 11
+    // Reserved for extension
+    //
+    // used to quickly test extension devices, semantics
+    // can differ based on implementation
+    #define ARROW_DEVICE_EXT_DEV 12
+    // CUDA managed/unified memory allocated by cudaMallocManaged
+    #define ARROW_DEVICE_CUDA_MANAGED 13
+    // Unified shared memory allocated on a oneAPI
+    // non-partitioned device.
+    //
+    // A call to the oneAPI runtime is required to determine the
+    // device type, the USM allocation type and the sycl context
+    // that it is bound to.
+    #define ARROW_DEVICE_ONEAPI 14
+    // GPU support for next-gen WebGPU standard
+    #define ARROW_DEVICE_WEBGPU 15
+    // Qualcomm Hexagon DSP
+    #define ARROW_DEVICE_HEXAGON 16
+
+    struct ArrowDeviceArray {
+      struct ArrowArray array;
+      int64_t device_id;
+      ArrowDeviceType device_type;
+      void* sync_event;
+
+      // reserved bytes for future expansion
+      int64_t reserved[3];
+    };
+
+    #endif  // ARROW_C_DEVICE_DATA_INTERFACE
+
+.. note::
+   The canonical guard ``ARROW_C_DEVICE_DATA_INTERFACE`` is meant to avoid
+   duplicate definitions if two projects copy the definitions in their own
+   headers, and a third-party project includes from these two projects. It
+   is therefore important that this guard is kept exactly as-is when these
+   definitions are copied.
+
+ArrowDeviceType
+---------------
+
+The ``ArrowDeviceType`` typedef is used to indicate what type of device the
+provided memory buffers were allocated on. This, in conjunction with the
+``device_id``, should be sufficient to reference the correct data buffers.
+
+We then use macros to define values for different device types. The provided
+macro values are compatible with the widely used `dlpack` ``DLDeviceType``
+definition values, using the same value for each as the equivalent
+``kDL<type>`` enum from dlpack.h. The list will be kept in sync with those
+equivalent enum values over time to ensure compatibility, rather than
+potentially diverging. To avoid the Arrow project having to be in the
+position of vetting new hardware devices, new additions should first be
+added to dlpack before we add a corresponding macro here.
+
+To ensure predictability with the ABI, we use macros instead of an ``enum``
+so the storage type is not compiler dependent.
+
+.. c:macro:: ARROW_DEVICE_CPU
+
+    CPU Device, equivalent to just using ``ArrowArray`` directly instead of
+    using ``ArrowDeviceArray``.
+
+.. c:macro:: ARROW_DEVICE_CUDA
+
+    A `CUDA` GPU Device. This could represent data allocated either with the
+    runtime library (``cudaMalloc``) or the device driver (``cuMemAlloc``).
+
+.. c:macro:: ARROW_DEVICE_CUDA_HOST
+
+    CPU memory that was pinned and page-locked by CUDA by using
+    ``cudaMallocHost`` or ``cuMemAllocHost``.
+
+.. c:macro:: ARROW_DEVICE_OPENCL
+
+    Data allocated on the device by using the `OpenCL (Open Computing Language)`
+    framework.
+
+.. c:macro:: ARROW_DEVICE_VULKAN
+
+    Data allocated by the `Vulkan` framework and libraries.
+
+.. c:macro:: ARROW_DEVICE_METAL
+
+    Data on Apple GPU devices using the `Metal` framework and libraries.
+
+.. c:macro:: ARROW_DEVICE_VPI
+
+    Indicates usage of a Verilog simulator buffer.
+
+.. c:macro:: ARROW_DEVICE_ROCM
+
+    An AMD device using the `ROCm` stack.
+
+.. c:macro:: ARROW_DEVICE_ROCM_HOST
+
+    CPU memory pinned and page-locked allocated ``hipMallocHost``.
+
+.. c:macro:: ARROW_DEVICE_EXT_DEV
+
+    This value is an escape-hatch for devices to extend which aren't
+    currently represented otherwise. Producers would need to provide
+    additional information/context specific to the device if using
+    this device type. This is used to quickly test extension devices
+    and semantics can differ based on the implementation.
+
+.. c:macro:: ARROW_DEVICE_CUDA_MANAGED
+
+    CUDA managed/unified memory which is allocated by ``cudaMallocManaged``.
+
+.. c:macro:: ARROW_DEVICE_ONEAPI
+
+    Unified shared memory allocated on an Intel `oneAPI` non-partitioned
+    device. A call to the ``oneAPI`` runtime is required to determine
+    the specific device type, the USM allocation type and the sycl context
+    that it is bound to.
+
+.. c:macro:: ARROW_DEVICE_WEBGPU
+
+    GPU support for next-gen WebGPU standards
+
+.. c:macro:: ARROW_DEVICE_HEXAGON
+
+    Data allocated on a Qualcomm Hexagon DSP device.
+
+The ArrowDeviceArray structure
+------------------------------
+
+The ``ArrowDeviceArray`` structure embeds the C data ``ArrowArray`` structure
+and adds additional information necessary for consumers to use the data. It
+has the following fields:
+
+.. c:member:: struct ArrowArray ArrowDeviceArray.array
+
+    The allocated array data. The values in the ``void**`` buffers (along
+    with the buffers of any children) are what is allocated on the device.
+    The buffer values should be device pointers. The rest of the structure
+    should be accessible to the CPU.
+
+    The ``private_data`` and ``release`` callback of this structure should
+    contain any necessary information and structures related to freeing
+    the array according to the device it is allocated on, rather than
+    having a separate release callback and ``private_data`` pointer here.
+
+.. c:member:: int64_t ArrowDeviceArray.device_id
+
+    The device id to identify a specific device if multiple devices of this
+    type are on the system. The semantics of the id will be hardware dependent,
+    but we use an ``int64_t`` to future-proof the id as devices change over time.
+
+.. c:member:: ArrowDeviceType ArrowDeviceArray.device_type
+
+    The type of the device which can access the buffers in the array.
+
+.. c:member:: void* ArrowDeviceArray.sync_event
+
+    Optional. An event-like object to synchronize on if needed.
+
+    Many devices, like GPUs, are primarily asynchronous with respect to
+    CPU processing. As such in order to safely access memory, it is often
+    necessary to have an object to synchronize processing with. Since
+    different devices will use different types to specify this, we use a
+    void* which can be coerced into a pointer to whatever the device
+    appropriate type is.
+
+    If synchronization is not needed, this can be null. If this is non-null
+    then it MUST be used to call the appropriate sync method for the device
+    (e.g. ``cudaStreamWaitEvent`` or ``hipStreamWaitEvent``) before attempting
+    to access the memory in the buffers.
+
+    Expected types to coerce this ``void*`` to depending on the reported
+    device type:
+
+    * CUDA: ``cudaEvent_t*``
+    * ROCm: ``hipEvent_t*``
+    * OpenCL: ``cl_event*``
+    * Vulkan: ``VkEvent*``
+    * Metal: ``MTLEvent*``
+    * OneAPI: ``sycl::event*``
+
+    If an event is provided, then the producer MUST ensure that the event
+    is triggered/recorded at the end of the processing stream once the data
+    is considered available for use.
+
+
+.. c:member:: int64_t ArrowDeviceArray.reserved[3]
+
+    As non-CPU development expands, there may be a need to expand this
+    structure. In order to do so without potentially breaking ABI changes,
+    we reserve 24 bytes at the end of the object. This also has the added
+    benefit of bringing the total size of this structure to exactly 128
+    bytes (a power of 2) on 64-bit systems. These bytes should be zero'd
+    out after allocation in order to ensure safe evolution of the ABI in
+    the future.
+
+.. note::
+    Rather than store the shape / types of the data alongside the
+    ``ArrowDeviceArray``, users should utilize the existing ``ArrowSchema``
+    structure to pass any data type and shape information.
+
+Semantics
+=========
+
+Memory management
+-----------------
+
+The ``ArrowDeviceArray`` structure contains an ``ArrowArray`` object which
+itself has :ref:`specific semantics <c-data-interface-semantics>` for releasing
+memory. The term *"base structure"* below refers to the ``ArrowDeviceArray``
+object that is passed directly between the producer and consumer -- not any
+child structure thereof.
+
+It is intended for the base structure to be stack- or heap-allocated by the
+*consumer*. In this case, the producer API should take a pointer to the
+consumer-allocated structure.
+
+However, any data pointed to by the struct MUST be allocated and maintained
+by the producer. This includes the ``sync_event`` member if it is not null,
+along with any pointers in the ``ArrowArray`` object as usual. Data lifetime
+is managed through the ``release`` callback of the ``ArrowArray`` member.
+
+For an ``ArrowDeviceArray``, the semantics of a released structure and the
+callback semantics are identical to those for
+:ref:`ArrowArray itself <c-data-interface-released>`. Any producer specific context
+information necessary for releasing the device data buffers, in addition to
+any allocated event, should be stored in the ``private_data`` member of
+the ``ArrowArray`` and managed by the ``release`` callback.
+
+Moving an array
+'''''''''''''''
+
+The consumer can *move* the ``ArrowDeviceArray`` structure by bitwise copying
+or shallow member-wise copying. Then it MUST mark the source structure released
+by setting the ``release`` member of the embedded ``ArrowArray`` structure to
+``NULL``, but *without* calling that release callback. This ensures that only
+one live copy of the struct is active at any given time and that lifetime is
+correctly communicated to the producer.
+
+As usual, the release callback will be called on the destination structure
+when it is not needed anymore.
+
+Record batches
+--------------
+As with the C data interface itself, a record batch can be trivially considered
+as an equivalent struct array. In this case the metadata of the top-level
+``ArrowSchema`` can be used for schema-level metadata of the record batch.
+
+Mutability
+----------
+
+Both the producer and the consumer SHOULD consider the exported data (that
+is, the data reachable on the device through the ``buffers`` member of
+the embedded ``ArrowArray``) to be immutable, as either party could otherwise
+see inconsistent data while the other is mutating it.
+
+Likewise, if the ``sync_event`` member is non-NULL, the consumer should not
+attempt to access or read the data until they have synchronized on that event.
+
+C producer example
+====================
+
+Exporting a simple ``int32`` device array
+-----------------------------------------
+
+Export a non-nullable ``int32`` type with empty metadata. An example of this
+can be seen in the :ref:`C data interface docs directly <c-data-interface-export-int32-schema>`.
+
+To export the data itself, we transfer ownership to the consumer through
+the release callback. This example will use CUDA, but the equivalent calls
+could be used for any device:
+
+.. code-block:: c
+
+    static void release_int32_device_array(struct ArrowArray* array) {
+        assert(array->n_buffers == 2);
+        // destroy the event
+        cudaEvent_t* ev_ptr = reinterpret_cast<cudaEvent_t*>(array->private_data);
+        cudaError_t status = cudaEventDestroy(*ev_ptr);
+        assert(status == cudaSuccess);
+        free(ev_ptr);
+
+        // free the buffers and the buffers array
+        status = cudaFree(array->buffers[1]);
+        assert(status == cudaSuccess);
+        free(array->buffers);
+
+        // mark released
+        array->release = NULL;
+    }
+
+    __host__ void export_int32_device_array(void* cudaAllocdPtr,
+                                            cudaStream_t stream,
+                                            int64_t length,
+                                            struct ArrowDeviceArray* array) {
+        // get device id
+        int device;
+        cudaError_t status;
+        status = cudaGetDevice(&device);
+        assert(status == cudaSuccess);
+
+        cudaEvent_t* ev_ptr = (cudaEvent_t*)malloc(sizeof(cudaEvent_t));
+        assert(ev_ptr != NULL);
+        status = cudaEventCreate(ev_ptr);
+        assert(status == cudaSuccess);
+
+        // record event on the stream, assuming that the passed in
+        // stream is where the work to produce the data will be processing.        
+        status = cudaEventRecord(*ev_ptr, stream);
+        assert(status == cudaSuccess);
+
+        // initialize fields
+        *array = (struct ArrowDeviceArray) {
+            .array = (struct ArrowArray) {
+                .length = length,
+                .null_count = 0,
+                .offset = 0,
+                .n_buffers = 2,
+                .n_children = 0,
+                .children = NULL,
+                .dictionary = NULL,
+                // bookeeping
+                .release = &release_int32_device_array,
+                .private_data = reinterpret_cast<void*>(ev_ptr),
+            },
+            .device_id = static_cast<int64_t>(device),
+            .device_type = ARROW_DEVICE_CUDA,
+            // pass the event pointer to the consumer
+            .sync_event = reinterpret_cast<void*>(ev_ptr),
+        };
+
+        // allocate list of buffers
+        array->array.buffers = (const void**)malloc(sizeof(void*) * array->array.n_buffers);
+        assert(array->array.buffers != NULL);
+        array->array.buffers[0] = NULL;
+        array->array.buffers[1] = cudaAllocdPtr;
+    }
+
+================
+Device Stream Interface
+================
+
+Like the :ref:`C stream interface <c-stream-interface>`, the C Device data 
+interface also specifies a higher-level structure for easing communication
+of streaming data within a single process. Defining an ``ArrowDeviceArrayStream``
+structure.
+
+Semantics
+=========
+
+An Arrow C device stream exposes a streaming source of data chunks, each with
+the same schema. Chunks are obtained by calling a blocking pull-style iteration
+function. It is expected that all chunks should be providing data on the same
+device type (but not necessarily the same device id). If it is necessary
+to provide a stream of data on multiple device types, a producer should 
+provide a separate stream object for each device type.
+
+Structure definition
+====================
+
+The C device stream interface is defined by a single ``struct`` definition:
+
+.. code-block:: c
+
+    #ifndef ARROW_C_DEVICE_STREAM_INTERFACE
+    #define ARROW_C_DEVICE_STREAM_INTERFACE
+
+    struct ArrowDeviceArrayStream {
+        // device type that all arrays will be accessible from
+        ArrowDeviceType device_type;
+        // callbacks
+        int (*get_schema)(struct ArrowDeviceArrayStream*, struct ArrowSchema*);
+        int (*get_next)(struct ArrowDeviceArrayStream*, struct ArrowDeviceArray*);
+        const char* (*get_last_error)(struct ArrowDeviceArrayStream*);
+
+        // release callback
+        void (*release)(struct ArrowDeviceArrayStream*);
+
+        // opaque producer-specific data
+        void* private_data;
+    };
+
+    #endif  // ARROW_C_DEVICE_STREAM_INTERFACE
+
+.. note::
+    The canonical guard ``ARROW_C_DEVICE_STREAM_INTERFACE`` is meant to avoid
+    duplicate definitions if two projects copy the C device stream interface
+    definitions into their own headers, and a third-party project includes
+    from these two projects. It is therefore important that this guard is
+    kept exactly as-is when these definitions are copied.
+
+The ArrowDeviceArrayStream structure
+------------------------------------
+
+The ``ArrowDeviceArrayStream`` provides a device type that can access the
+resulting data along with the required callbacks to interact with a
+streaming source of Arrow arrays. It has the following fields:
+
+.. c:member:: ArrowDeviceType device_type
+
+    The device type that this stream produces data on. All 
+    ``ArrowDeviceArray``s that are produced by this stream should have the
+    same device type as is set here. This is a convenience for the consumer
+    to not have to check every array that is retrieved and instead allows
+    higher-level coding constructs for streams.
+
+.. c:member:: int (*ArrowDeviceArrayStream.get_schema)(struct ArrowDeviceArrayStream*, struct ArrowSchema* out)
+
+    *Mandatory.* This callback allows the consumer to query the schema of
+    the chunks of data in the stream. The schema is the same for all data
+    chunks.
+
+    This callback must NOT be called on a released ``ArrowDeviceArrayStream``.
+
+    *Return value:* 0 on success, a non-zero
+    :ref:`error code <c-stream-interface-error-codes>` otherwise.
+
+.. c:member:: int (*ArrowDeviceArrayStream.get_next)(struct ArrowDeviceArrayStream*, struct ArrowDeviceArray* out)
+
+    *Mandatory.* This callback allows the consumer to get the next chunk of
+    data in the stream.
+
+    This callback must NOT be called on a released ``ArrowDeviceArrayStream``.
+
+    The next chunk of data MUST be accessible from a device type matching the
+    :c:member:`ArrowDeviceArrayStream.device_type`.
+
+    *Return value:* 0 on success, a non-zero
+    :ref:`error code <c-stream-interface-error-codes>` otherwise.
+
+    On success, the consumer must check whether the ``ArrowDeviceArray``'s
+    embedded ``ArrowArray`` is marked :ref:`released <c-data-interface-released>`.
+    If the embedded ``ArrowDeviceArray.array`` is released, then the end of the 
+    stream has been reached. Otherwise, the ``ArrowDeviceArray`` contains a
+    valid data chunk.
+
+.. c:member:: const char* (*ArrowDeviceArrayStream.get_last_error)(struct ArrowDeviceArrayStream*)
+
+    *Mandatory.* This callback allows the consumer to get a textual description
+    of the last error.
+
+    This callback must ONLY be called if the last operation on the 
+    ``ArrowDeviceArrayStream`` returned an error. It must NOT be called on a
+    released ``ArrowDeviceArrayStream``.
+
+    *Return value:* a pointer to a NULL-terminated character string 
+    (UTF8-encoded). NULL can also be returned if no detailed description is
+    available.
+
+    The returned pointer is only guaranteed to be valid until the next call
+    of one of the stream's callbacks. The character string it points to should
+    be copied to consumer-managed storage if it is intended to survive longer.
+
+.. c:member:: void (*ArrowDeviceArrayStream.release)(struct ArrowDeviceArrayStream*)
+
+    *Mandatory.* A pointer to a producer-provided release callback.
+
+.. c:member:: void* ArrowDeviceArrayStream.private_data
+
+    *Optional.* An opaque pointer to producer-provided private data.
+
+    Consumers MUST NOT process this member. Lifetime of this member is
+    handled by the producer, and especially by the release callback.
+
+Result lifetimes
+----------------
+
+The data returned by the ``get_schema`` and ``get_next`` callbacks must be
+released independantly. Their lifetimes are not tied to that of 
+``ArrowDeviceArrayStream``.
+
+Stream lifetime
+---------------
+
+Lifetime of the C stream is managed using a release callback with similar 
+usage as in :ref:`C data interface <c-data-interface-released>`.
+
+Thread safety
+-------------
+
+The stream source is not assumed to be thread-safe. Consumers wanting to
+call ``get_next`` from several threads should ensure those calls are 
+serialized.
+
+Updating this specification
+===========================
+
+Once this specification is supported in an official Arrow release, the C ABI
+is frozen. This means that the ``ArrowDeviceArray`` structure definition 

Review Comment:
   Hmm, I guess it's because of the reserved bytes? But that's a good point. I should clarify this a bit. 
   
   In my mind, I guess the experimental portion is because we don't know what we don't know so it's possible there will end up needing to be some changes to this despite our best efforts. I'll clarify this language a bit.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [arrow] kkraus14 commented on pull request #34972: GH-34971: [Format] Enhance C-Data API to support non-cpu cases

["kkraus14 (via GitHub)" <gi...@apache.org>]

kkraus14 commented on PR #34972:
URL: https://github.com/apache/arrow/pull/34972#issuecomment-1512135930

   cc @aregm @oleksandr-pavlyk
   
   Any chance you could provide feedback from the SYCL perspective or ping someone who could?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [arrow] lidavidm commented on a diff in pull request #34972: GH-34971: [Format] Enhance C-Data API to support non-cpu cases

["lidavidm (via GitHub)" <gi...@apache.org>]

lidavidm commented on code in PR #34972:
URL: https://github.com/apache/arrow/pull/34972#discussion_r1191366934


##########
cpp/src/arrow/c/abi.h:
##########
@@ -65,6 +85,111 @@ struct ArrowArray {
 
 #endif  // ARROW_C_DATA_INTERFACE
 
+#ifndef ARROW_C_DEVICE_DATA_INTERFACE
+#define ARROW_C_DEVICE_DATA_INTERFACE
+
+/// \defgroup arrow-device-types Device Types
+/// These macros are compatible with the dlpack DLDeviceType values,
+/// using the same value for each enum as the equivalent kDL<type>
+/// from dlpack.h. This list should continue to be kept in sync with
+/// the equivalent dlpack.h enum values over time to ensure 
+/// compatibility, rather than potentially diverging.
+///
+/// To ensure predictability with the ABI we use macros instead of
+/// an enum so the storage type is not compiler dependent.
+///
+/// @{
+
+/// \brief DeviceType for the allocated memory
+typedef int32_t ArrowDeviceType;
+
+/// \brief CPU device, same as using ArrowArray directly
+#define ARROW_DEVICE_CPU = 1
+/// \brief CUDA GPU Device
+#define ARROW_DEVICE_CUDA = 2
+/// \brief Pinned CUDA CPU memory by cudaMallocHost
+#define ARROW_DEVICE_CUDA_HOST = 3
+/// \brief OpenCL Device
+#define ARROW_DEVICE_OPENCL = 4
+/// \brief Vulkan buffer for next-gen graphics
+#define ARROW_DEVICE_VULKAN = 7
+/// \brief Metal for Apple GPU
+#define ARROW_DEVICE_METAL = 8
+/// \brief Verilog simulator buffer
+#define ARROW_DEVICE_VPI = 9
+/// \brief ROCm GPUs for AMD GPUs
+#define ARROW_DEVICE_ROCM = 10
+/// \brief Pinned ROCm CPU memory allocated by hipMallocHost
+#define ARROW_DEVICE_ROCMHOST = 11
+/// \brief Reserved for extension
+///
+/// used to quickly test extension devices, semantics
+/// can differ based on the implementation
+#define ARROW_DEVICE_EXT_DEV = 12
+/// \brief CUDA managed/unified memory allocated by cudaMallocManaged
+#define ARROW_DEVICE_CUDA_MANAGED = 13
+/// \brief unified shared memory allocated on a oneAPI
+/// non-partitioned device.
+///
+/// A call to the oneAPI runtime is required to determine the device
+/// type, the USM allocation type, and the sycl context it is bound to.
+#define ARROW_DEVICE_ONEAPI = 14
+/// \brief GPU support for next-gen WebGPU standard
+#define ARROW_DEVICE_WEBGPU = 15
+/// \brief Qualcomm Hexagon DSP
+#define ARROW_DEVICE_HEXAGON = 16
+
+/// @}
+
+/// \brief Struct for passing an Arrow Array alongside
+/// device memory information.
+struct ArrowDeviceArray {
+  /// \brief the Allocated Array
+  ///
+  /// the buffers in the array (along with the buffers of any
+  /// children) are what is allocated on the device.
+  ///
+  /// the private_data and release callback of the arrow array
+  /// should contain any necessary information and structures
+  /// related to freeing the array according to the device it
+  /// is allocated on, rather than having a separate release
+  /// callback embedded here.
+  struct ArrowArray array;
+  /// \brief The device id to identify a specific device
+  /// if multiple of this type are on the system.
+  ///
+  /// the semantics of the id will be hardware dependant.
+  int64_t device_id;
+  /// \brief The type of device which can access this memory.
+  ArrowDeviceType device_type;
+  /// \brief An event-like object to synchronize on if needed.
+  ///
+  /// Many devices, like GPUs, are primarily asynchronous with
+  /// respect to CPU processing. As such in order to safely access
+  /// memory, it is often necessary to have an object to synchronize
+  /// processing on. Since different devices will use different types

Review Comment:
   I suppose, but I think that decreases the value of this if a producer can't know what type to use based on the device type alone - that means they have to write producer-specific code.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [arrow] oleksandr-pavlyk commented on a diff in pull request #34972: GH-34971: [Format] Enhance C-Data API to support non-cpu cases

["oleksandr-pavlyk (via GitHub)" <gi...@apache.org>]

oleksandr-pavlyk commented on code in PR #34972:
URL: https://github.com/apache/arrow/pull/34972#discussion_r1191667582


##########
cpp/src/arrow/c/abi.h:
##########
@@ -65,6 +85,111 @@ struct ArrowArray {
 
 #endif  // ARROW_C_DATA_INTERFACE
 
+#ifndef ARROW_C_DEVICE_DATA_INTERFACE
+#define ARROW_C_DEVICE_DATA_INTERFACE
+
+/// \defgroup arrow-device-types Device Types
+/// These macros are compatible with the dlpack DLDeviceType values,
+/// using the same value for each enum as the equivalent kDL<type>
+/// from dlpack.h. This list should continue to be kept in sync with
+/// the equivalent dlpack.h enum values over time to ensure 
+/// compatibility, rather than potentially diverging.
+///
+/// To ensure predictability with the ABI we use macros instead of
+/// an enum so the storage type is not compiler dependent.
+///
+/// @{
+
+/// \brief DeviceType for the allocated memory
+typedef int32_t ArrowDeviceType;
+
+/// \brief CPU device, same as using ArrowArray directly
+#define ARROW_DEVICE_CPU = 1
+/// \brief CUDA GPU Device
+#define ARROW_DEVICE_CUDA = 2
+/// \brief Pinned CUDA CPU memory by cudaMallocHost
+#define ARROW_DEVICE_CUDA_HOST = 3
+/// \brief OpenCL Device
+#define ARROW_DEVICE_OPENCL = 4
+/// \brief Vulkan buffer for next-gen graphics
+#define ARROW_DEVICE_VULKAN = 7
+/// \brief Metal for Apple GPU
+#define ARROW_DEVICE_METAL = 8
+/// \brief Verilog simulator buffer
+#define ARROW_DEVICE_VPI = 9
+/// \brief ROCm GPUs for AMD GPUs
+#define ARROW_DEVICE_ROCM = 10
+/// \brief Pinned ROCm CPU memory allocated by hipMallocHost
+#define ARROW_DEVICE_ROCMHOST = 11
+/// \brief Reserved for extension
+///
+/// used to quickly test extension devices, semantics
+/// can differ based on the implementation
+#define ARROW_DEVICE_EXT_DEV = 12
+/// \brief CUDA managed/unified memory allocated by cudaMallocManaged
+#define ARROW_DEVICE_CUDA_MANAGED = 13
+/// \brief unified shared memory allocated on a oneAPI
+/// non-partitioned device.
+///
+/// A call to the oneAPI runtime is required to determine the device
+/// type, the USM allocation type, and the sycl context it is bound to.
+#define ARROW_DEVICE_ONEAPI = 14
+/// \brief GPU support for next-gen WebGPU standard
+#define ARROW_DEVICE_WEBGPU = 15
+/// \brief Qualcomm Hexagon DSP
+#define ARROW_DEVICE_HEXAGON = 16
+
+/// @}
+
+/// \brief Struct for passing an Arrow Array alongside
+/// device memory information.
+struct ArrowDeviceArray {
+  /// \brief the Allocated Array
+  ///
+  /// the buffers in the array (along with the buffers of any
+  /// children) are what is allocated on the device.
+  ///
+  /// the private_data and release callback of the arrow array
+  /// should contain any necessary information and structures
+  /// related to freeing the array according to the device it
+  /// is allocated on, rather than having a separate release
+  /// callback embedded here.
+  struct ArrowArray array;
+  /// \brief The device id to identify a specific device
+  /// if multiple of this type are on the system.
+  ///
+  /// the semantics of the id will be hardware dependant.
+  int64_t device_id;
+  /// \brief The type of device which can access this memory.
+  ArrowDeviceType device_type;
+  /// \brief An event-like object to synchronize on if needed.
+  ///
+  /// Many devices, like GPUs, are primarily asynchronous with
+  /// respect to CPU processing. As such in order to safely access
+  /// memory, it is often necessary to have an object to synchronize
+  /// processing on. Since different devices will use different types

Review Comment:
   `sycl::event` is also supported in oneAPI DPC++, as it implements SYCL 2020 standard: https://registry.khronos.org/SYCL/specs/sycl-2020/html/sycl-2020.html#sec:interface.event



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [arrow] zeroshade commented on a diff in pull request #34972: GH-34971: [Format] Enhance C-Data API to support non-cpu cases

["zeroshade (via GitHub)" <gi...@apache.org>]

zeroshade commented on code in PR #34972:
URL: https://github.com/apache/arrow/pull/34972#discussion_r1191668287


##########
cpp/src/arrow/c/abi.h:
##########
@@ -65,6 +85,111 @@ struct ArrowArray {
 
 #endif  // ARROW_C_DATA_INTERFACE
 
+#ifndef ARROW_C_DEVICE_DATA_INTERFACE
+#define ARROW_C_DEVICE_DATA_INTERFACE
+
+/// \defgroup arrow-device-types Device Types
+/// These macros are compatible with the dlpack DLDeviceType values,
+/// using the same value for each enum as the equivalent kDL<type>
+/// from dlpack.h. This list should continue to be kept in sync with
+/// the equivalent dlpack.h enum values over time to ensure 
+/// compatibility, rather than potentially diverging.
+///
+/// To ensure predictability with the ABI we use macros instead of
+/// an enum so the storage type is not compiler dependent.
+///
+/// @{
+
+/// \brief DeviceType for the allocated memory
+typedef int32_t ArrowDeviceType;
+
+/// \brief CPU device, same as using ArrowArray directly
+#define ARROW_DEVICE_CPU = 1
+/// \brief CUDA GPU Device
+#define ARROW_DEVICE_CUDA = 2
+/// \brief Pinned CUDA CPU memory by cudaMallocHost
+#define ARROW_DEVICE_CUDA_HOST = 3
+/// \brief OpenCL Device
+#define ARROW_DEVICE_OPENCL = 4
+/// \brief Vulkan buffer for next-gen graphics
+#define ARROW_DEVICE_VULKAN = 7
+/// \brief Metal for Apple GPU
+#define ARROW_DEVICE_METAL = 8
+/// \brief Verilog simulator buffer
+#define ARROW_DEVICE_VPI = 9
+/// \brief ROCm GPUs for AMD GPUs
+#define ARROW_DEVICE_ROCM = 10
+/// \brief Pinned ROCm CPU memory allocated by hipMallocHost
+#define ARROW_DEVICE_ROCMHOST = 11
+/// \brief Reserved for extension
+///
+/// used to quickly test extension devices, semantics
+/// can differ based on the implementation
+#define ARROW_DEVICE_EXT_DEV = 12
+/// \brief CUDA managed/unified memory allocated by cudaMallocManaged
+#define ARROW_DEVICE_CUDA_MANAGED = 13
+/// \brief unified shared memory allocated on a oneAPI
+/// non-partitioned device.
+///
+/// A call to the oneAPI runtime is required to determine the device
+/// type, the USM allocation type, and the sycl context it is bound to.
+#define ARROW_DEVICE_ONEAPI = 14
+/// \brief GPU support for next-gen WebGPU standard
+#define ARROW_DEVICE_WEBGPU = 15
+/// \brief Qualcomm Hexagon DSP
+#define ARROW_DEVICE_HEXAGON = 16
+
+/// @}
+
+/// \brief Struct for passing an Arrow Array alongside
+/// device memory information.
+struct ArrowDeviceArray {
+  /// \brief the Allocated Array
+  ///
+  /// the buffers in the array (along with the buffers of any
+  /// children) are what is allocated on the device.
+  ///
+  /// the private_data and release callback of the arrow array
+  /// should contain any necessary information and structures
+  /// related to freeing the array according to the device it
+  /// is allocated on, rather than having a separate release
+  /// callback embedded here.
+  struct ArrowArray array;
+  /// \brief The device id to identify a specific device
+  /// if multiple of this type are on the system.
+  ///
+  /// the semantics of the id will be hardware dependant.
+  int64_t device_id;
+  /// \brief The type of device which can access this memory.
+  ArrowDeviceType device_type;
+  /// \brief An event-like object to synchronize on if needed.
+  ///
+  /// Many devices, like GPUs, are primarily asynchronous with
+  /// respect to CPU processing. As such in order to safely access
+  /// memory, it is often necessary to have an object to synchronize
+  /// processing on. Since different devices will use different types

Review Comment:
   sounds good, i'll update the comments with this list of types :)



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [arrow] zeroshade commented on a diff in pull request #34972: GH-34971: [Format] Add non-CPU version of C Data Interface

["zeroshade (via GitHub)" <gi...@apache.org>]

zeroshade commented on code in PR #34972:
URL: https://github.com/apache/arrow/pull/34972#discussion_r1213608271


##########
docs/source/format/CDeviceDataInterface.rst:
##########
@@ -0,0 +1,628 @@
+.. Licensed to the Apache Software Foundation (ASF) under one
+.. or more contributor license agreements.  See the NOTICE file
+.. distributed with this work for additional information
+.. regarding copyright ownership.  The ASF licenses this file
+.. to you under the Apache License, Version 2.0 (the
+.. "License"); you may not use this file except in compliance
+.. with the License.  You may obtain a copy of the License at
+
+..   http://www.apache.org/licenses/LICENSE-2.0
+
+.. Unless required by applicable law or agreed to in writing,
+.. software distributed under the License is distributed on an
+.. "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+.. KIND, either express or implied.  See the License for the
+.. specific language governing permissions and limitations
+.. under the License.
+
+.. highlight:: c
+
+.. _c-device-data-interface:
+
+=================================
+The Arrow C Device data interface
+=================================
+
+.. note:: The Arrow C Device Data Interface should be considered experimental
+
+Rationale
+=========
+
+The current :ref:`C Data Interface <c-data-interface>`, and most
+implementations of it, make the assumption that all data buffers provided
+are CPU buffers. Since Apache Arrow is designed to be a universal in-memory
+format for representing tabular ("columnar") data, there will be the desire
+to leverage this data on non-CPU hardware such as GPUs. One example of such
+a case is the `RAPIDS cuDF library`_ which uses the Arrow memory format with
+CUDA for NVIDIA GPUs. Since copying data from host to device and back is
+expensive, the ideal would be to be able to leave the data on the device
+for as long as possible, even when passing it between runtimes and
+libraries.
+
+The Arrow C Device data interface builds on the existing C data interface
+by adding a very small, stable set of C definitions to it. These definitions
+are equivalents to the ``ArrowArray`` and ``ArrowArrayStream`` structures
+from the C Data Interface which add members to allow specifying the device
+type and pass necessary information to synchronize with the producer.
+For non-C/C++ languages and runtimes, translating the C definitions to
+corresponding C FFI declarations should be just as simple as with the
+current C data interface.
+
+Applications and libraries can then use Arrow schemas and Arrow formatted
+memory on non-CPU devices to exchange data just as easily as they do
+now with CPU data. This will enable leaving data on those devices longer
+and avoiding costly copies back and forth between the host and device
+just to leverage new libraries and runtimes.
+
+Goals
+-----
+
+* Expose an ABI-stable interface built on the existing C data interface.
+* Make it easy for third-party projects to implement support with little
+  initial investment.
+* Allow zero-copy sharing of Arrow formatted device memory between
+  independant runtimes and components running in the same process.
+* Avoid the need for one-to-one adaptation layers such as the
+  `CUDA Array Interface`_ for Python processes to pass CUDA data.
+* Enable integration without explicit dependencies (either at compile-time
+  or runtime) on the Arrow software project itself.
+
+The intent is for the Arrow C Device data interface to expand the reach
+of the current C data interface, allowing it to also become the standard
+low-level building block for columnar processing on devices like GPUs or
+FPGAs.
+
+Structure definitions
+=====================
+
+Because this is built on the C data interface, the C Device data interface
+uses the ``ArrowSchema`` and ``ArrowArray`` structures as defined in the
+:ref:`C data interface spec <c-data-interface-struct-defs>`. It then adds the
+following free-standing definitions. Like the rest of the Arrow project,
+they are available under the Apache License 2.0.
+
+.. code-block:: c
+
+    #ifndef ARROW_C_DEVICE_DATA_INTERFACE
+    #define ARROW_C_DEVICE_DATA_INTERFACE
+
+    // Device type for the allocated memory
+    typedef int32_t ArrowDeviceType;
+
+    // CPU device, same as using ArrowArray directly
+    #define ARROW_DEVICE_CPU 1
+    // CUDA GPU Device
+    #define ARROW_DEVICE_CUDA 2
+    // Pinned CUDA CPU memory by cudaMallocHost
+    #define ARROW_DEVICE_CUDA_HOST 3
+    // OpenCL Device
+    #define ARROW_DEVICE_OPENCL 4
+    // Vulkan buffer for next-gen graphics
+    #define ARROW_DEVICE_VULKAN 7
+    // Metal for Apple GPU
+    #define ARROW_DEVICE_METAL 8
+    // Verilog simulator buffer
+    #define ARROW_DEVICE_VPI 9
+    // ROCm GPUs for AMD GPUs
+    #define ARROW_DEVICE_ROCM 10
+    // Pinned ROCm CPU memory allocated by hipMallocHost
+    #define ARROW_DEVICE_ROCM_HOST 11
+    // Reserved for extension
+    //
+    // used to quickly test extension devices, semantics
+    // can differ based on implementation
+    #define ARROW_DEVICE_EXT_DEV 12
+    // CUDA managed/unified memory allocated by cudaMallocManaged
+    #define ARROW_DEVICE_CUDA_MANAGED 13
+    // Unified shared memory allocated on a oneAPI
+    // non-partitioned device.
+    //
+    // A call to the oneAPI runtime is required to determine the
+    // device type, the USM allocation type and the sycl context
+    // that it is bound to.
+    #define ARROW_DEVICE_ONEAPI 14
+    // GPU support for next-gen WebGPU standard
+    #define ARROW_DEVICE_WEBGPU 15
+    // Qualcomm Hexagon DSP
+    #define ARROW_DEVICE_HEXAGON 16
+
+    struct ArrowDeviceArray {
+      struct ArrowArray array;
+      int64_t device_id;
+      ArrowDeviceType device_type;
+      void* sync_event;
+
+      // reserved bytes for future expansion
+      int64_t reserved[3];
+    };
+
+    #endif  // ARROW_C_DEVICE_DATA_INTERFACE
+
+.. note::
+   The canonical guard ``ARROW_C_DEVICE_DATA_INTERFACE`` is meant to avoid
+   duplicate definitions if two projects copy the definitions in their own
+   headers, and a third-party project includes from these two projects. It
+   is therefore important that this guard is kept exactly as-is when these
+   definitions are copied.
+
+ArrowDeviceType
+---------------
+
+The ``ArrowDeviceType`` typedef is used to indicate what type of device the
+provided memory buffers were allocated on. This, in conjunction with the
+``device_id``, should be sufficient to reference the correct data buffers.
+
+We then use macros to define values for different device types. The provided
+macro values are compatible with the widely used `dlpack`_ ``DLDeviceType``
+definition values, using the same value for each as the equivalent
+``kDL<type>`` enum from dlpack.h. The list will be kept in sync with those
+equivalent enum values over time to ensure compatibility, rather than
+potentially diverging. To avoid the Arrow project having to be in the
+position of vetting new hardware devices, new additions should first be
+added to dlpack before we add a corresponding macro here.
+
+To ensure predictability with the ABI, we use macros instead of an ``enum``
+so the storage type is not compiler dependent.
+
+.. c:macro:: ARROW_DEVICE_CPU
+
+    CPU Device, equivalent to just using ``ArrowArray`` directly instead of
+    using ``ArrowDeviceArray``.
+
+.. c:macro:: ARROW_DEVICE_CUDA
+
+    A `CUDA`_ GPU Device. This could represent data allocated either with the
+    runtime library (``cudaMalloc``) or the device driver (``cuMemAlloc``).
+
+.. c:macro:: ARROW_DEVICE_CUDA_HOST
+
+    CPU memory that was pinned and page-locked by CUDA by using
+    ``cudaMallocHost`` or ``cuMemAllocHost``.
+
+.. c:macro:: ARROW_DEVICE_OPENCL
+
+    Data allocated on the device by using the `OpenCL (Open Computing Language)`_
+    framework.
+
+.. c:macro:: ARROW_DEVICE_VULKAN
+
+    Data allocated by the `Vulkan`_ framework and libraries.
+
+.. c:macro:: ARROW_DEVICE_METAL
+
+    Data on Apple GPU devices using the `Metal`_ framework and libraries.
+
+.. c:macro:: ARROW_DEVICE_VPI
+
+    Indicates usage of a Verilog simulator buffer.
+
+.. c:macro:: ARROW_DEVICE_ROCM
+
+    An AMD device using the `ROCm`_ stack.
+
+.. c:macro:: ARROW_DEVICE_ROCM_HOST
+
+    CPU memory pinned and page-locked allocated ``hipMallocHost``.
+
+.. c:macro:: ARROW_DEVICE_EXT_DEV
+
+    This value is an escape-hatch for devices to extend which aren't
+    currently represented otherwise. Producers would need to provide
+    additional information/context specific to the device if using
+    this device type. This is used to quickly test extension devices
+    and semantics can differ based on the implementation.
+
+.. c:macro:: ARROW_DEVICE_CUDA_MANAGED
+
+    CUDA managed/unified memory which is allocated by ``cudaMallocManaged``.
+
+.. c:macro:: ARROW_DEVICE_ONEAPI
+
+    Unified shared memory allocated on an Intel `oneAPI`_ non-partitioned
+    device. A call to the ``oneAPI`` runtime is required to determine
+    the specific device type, the USM allocation type and the sycl context
+    that it is bound to.
+
+.. c:macro:: ARROW_DEVICE_WEBGPU
+
+    GPU support for next-gen WebGPU standards
+
+.. c:macro:: ARROW_DEVICE_HEXAGON
+
+    Data allocated on a Qualcomm Hexagon DSP device.
+
+The ArrowDeviceArray structure
+------------------------------
+
+The ``ArrowDeviceArray`` structure embeds the C data ``ArrowArray`` structure
+and adds additional information necessary for consumers to use the data. It
+has the following fields:
+
+.. c:member:: struct ArrowArray ArrowDeviceArray.array
+
+    The allocated array data. The values in the ``void**`` buffers (along
+    with the buffers of any children) are what is allocated on the device.
+    The buffer values should be device pointers. The rest of the structure
+    should be accessible to the CPU.
+
+    The ``private_data`` and ``release`` callback of this structure should
+    contain any necessary information and structures related to freeing
+    the array according to the device it is allocated on, rather than
+    having a separate release callback and ``private_data`` pointer here.
+
+.. c:member:: int64_t ArrowDeviceArray.device_id
+
+    The device id to identify a specific device if multiple devices of this
+    type are on the system. The semantics of the id will be hardware dependent,
+    but we use an ``int64_t`` to future-proof the id as devices change over time.
+
+.. c:member:: ArrowDeviceType ArrowDeviceArray.device_type
+
+    The type of the device which can access the buffers in the array.
+
+.. c:member:: void* ArrowDeviceArray.sync_event
+
+    Optional. An event-like object to synchronize on if needed.
+
+    Many devices, like GPUs, are primarily asynchronous with respect to
+    CPU processing. As such in order to safely access memory, it is often
+    necessary to have an object to synchronize processing with. Since
+    different devices will use different types to specify this, we use a
+    void* which can be coerced into a pointer to whatever the device
+    appropriate type is.
+
+    If synchronization is not needed, this can be null. If this is non-null
+    then it MUST be used to call the appropriate sync method for the device
+    (e.g. ``cudaStreamWaitEvent`` or ``hipStreamWaitEvent``) before attempting
+    to access the memory in the buffers.
+
+    Expected types to coerce this ``void*`` to depending on the reported
+    device type:
+
+    * CUDA: ``cudaEvent_t*``
+    * ROCm: ``hipEvent_t*``
+    * OpenCL: ``cl_event*``
+    * Vulkan: ``VkEvent*``
+    * Metal: ``MTLEvent*``
+    * OneAPI: ``sycl::event*``
+
+    If an event is provided, then the producer MUST ensure that the event
+    is triggered/recorded at the end of the processing stream once the data
+    is considered available for use.
+
+
+.. c:member:: int64_t ArrowDeviceArray.reserved[3]
+
+    As non-CPU development expands, there may be a need to expand this
+    structure. In order to do so without potentially breaking ABI changes,
+    we reserve 24 bytes at the end of the object. This also has the added
+    benefit of bringing the total size of this structure to exactly 128
+    bytes (a power of 2) on 64-bit systems. These bytes should be zero'd
+    out after allocation in order to ensure safe evolution of the ABI in
+    the future.
+
+.. note::
+    Rather than store the shape / types of the data alongside the
+    ``ArrowDeviceArray``, users should utilize the existing ``ArrowSchema``
+    structure to pass any data type and shape information.
+
+Semantics
+=========
+
+Memory management
+-----------------
+
+The ``ArrowDeviceArray`` structure contains an ``ArrowArray`` object which
+itself has :ref:`specific semantics <c-data-interface-semantics>` for releasing
+memory. The term *"base structure"* below refers to the ``ArrowDeviceArray``
+object that is passed directly between the producer and consumer -- not any
+child structure thereof.
+
+It is intended for the base structure to be stack- or heap-allocated by the
+*consumer*. In this case, the producer API should take a pointer to the
+consumer-allocated structure.
+
+However, any data pointed to by the struct MUST be allocated and maintained
+by the producer. This includes the ``sync_event`` member if it is not null,
+along with any pointers in the ``ArrowArray`` object as usual. Data lifetime
+is managed through the ``release`` callback of the ``ArrowArray`` member.
+
+For an ``ArrowDeviceArray``, the semantics of a released structure and the
+callback semantics are identical to those for
+:ref:`ArrowArray itself <c-data-interface-released>`. Any producer specific context
+information necessary for releasing the device data buffers, in addition to
+any allocated event, should be stored in the ``private_data`` member of
+the ``ArrowArray`` and managed by the ``release`` callback.
+
+Moving an array
+'''''''''''''''
+
+The consumer can *move* the ``ArrowDeviceArray`` structure by bitwise copying
+or shallow member-wise copying. Then it MUST mark the source structure released
+by setting the ``release`` member of the embedded ``ArrowArray`` structure to
+``NULL``, but *without* calling that release callback. This ensures that only
+one live copy of the struct is active at any given time and that lifetime is
+correctly communicated to the producer.
+
+As usual, the release callback will be called on the destination structure
+when it is not needed anymore.
+
+Record batches
+--------------
+As with the C data interface itself, a record batch can be trivially considered
+as an equivalent struct array. In this case the metadata of the top-level
+``ArrowSchema`` can be used for schema-level metadata of the record batch.
+
+Mutability
+----------
+
+Both the producer and the consumer SHOULD consider the exported data (that
+is, the data reachable on the device through the ``buffers`` member of
+the embedded ``ArrowArray``) to be immutable, as either party could otherwise
+see inconsistent data while the other is mutating it.
+
+Likewise, if the ``sync_event`` member is non-NULL, the consumer should not
+attempt to access or read the data until they have synchronized on that event.
+
+C producer example
+====================
+
+Exporting a simple ``int32`` device array
+-----------------------------------------
+
+Export a non-nullable ``int32`` type with empty metadata. An example of this
+can be seen in the :ref:`C data interface docs directly <c-data-interface-export-int32-schema>`.
+
+To export the data itself, we transfer ownership to the consumer through
+the release callback. This example will use CUDA, but the equivalent calls
+could be used for any device:
+
+.. code-block:: c
+
+    static void release_int32_device_array(struct ArrowArray* array) {
+        assert(array->n_buffers == 2);
+        // destroy the event
+        cudaEvent_t* ev_ptr = reinterpret_cast<cudaEvent_t*>(array->private_data);
+        cudaError_t status = cudaEventDestroy(*ev_ptr);
+        assert(status == cudaSuccess);
+        free(ev_ptr);
+
+        // free the buffers and the buffers array
+        status = cudaFree(array->buffers[1]);
+        assert(status == cudaSuccess);
+        free(array->buffers);
+
+        // mark released
+        array->release = NULL;
+    }
+
+    __host__ void export_int32_device_array(void* cudaAllocdPtr,
+                                            cudaStream_t stream,
+                                            int64_t length,
+                                            struct ArrowDeviceArray* array) {
+        // get device id
+        int device;
+        cudaError_t status;
+        status = cudaGetDevice(&device);
+        assert(status == cudaSuccess);
+
+        cudaEvent_t* ev_ptr = (cudaEvent_t*)malloc(sizeof(cudaEvent_t));
+        assert(ev_ptr != NULL);
+        status = cudaEventCreate(ev_ptr);
+        assert(status == cudaSuccess);
+
+        // record event on the stream, assuming that the passed in
+        // stream is where the work to produce the data will be processing.
+        status = cudaEventRecord(*ev_ptr, stream);
+        assert(status == cudaSuccess);
+
+        // initialize fields
+        *array = (struct ArrowDeviceArray) {
+            .array = (struct ArrowArray) {
+                .length = length,
+                .null_count = 0,
+                .offset = 0,
+                .n_buffers = 2,
+                .n_children = 0,
+                .children = NULL,
+                .dictionary = NULL,
+                // bookeeping
+                .release = &release_int32_device_array,
+                .private_data = reinterpret_cast<void*>(ev_ptr),
+            },
+            .device_id = static_cast<int64_t>(device),
+            .device_type = ARROW_DEVICE_CUDA,
+            // pass the event pointer to the consumer
+            .sync_event = reinterpret_cast<void*>(ev_ptr),
+        };
+
+        // allocate list of buffers
+        array->array.buffers = (const void**)malloc(sizeof(void*) * array->array.n_buffers);
+        assert(array->array.buffers != NULL);
+        array->array.buffers[0] = NULL;
+        array->array.buffers[1] = cudaAllocdPtr;
+    }
+
+================
+Device Stream Interface
+================
+
+Like the :ref:`C stream interface <c-stream-interface>`, the C Device data
+interface also specifies a higher-level structure for easing communication
+of streaming data within a single process. Defining an ``ArrowDeviceArrayStream``
+structure.
+
+Semantics
+=========
+
+An Arrow C device stream exposes a streaming source of data chunks, each with
+the same schema. Chunks are obtained by calling a blocking pull-style iteration
+function. It is expected that all chunks should be providing data on the same
+device type (but not necessarily the same device id). If it is necessary
+to provide a stream of data on multiple device types, a producer should
+provide a separate stream object for each device type.
+
+Structure definition
+====================
+
+The C device stream interface is defined by a single ``struct`` definition:
+
+.. code-block:: c
+
+    #ifndef ARROW_C_DEVICE_STREAM_INTERFACE
+    #define ARROW_C_DEVICE_STREAM_INTERFACE
+
+    struct ArrowDeviceArrayStream {
+        // device type that all arrays will be accessible from
+        ArrowDeviceType device_type;
+        // callbacks
+        int (*get_schema)(struct ArrowDeviceArrayStream*, struct ArrowSchema*);
+        int (*get_next)(struct ArrowDeviceArrayStream*, struct ArrowDeviceArray*);
+        const char* (*get_last_error)(struct ArrowDeviceArrayStream*);
+
+        // release callback
+        void (*release)(struct ArrowDeviceArrayStream*);
+
+        // opaque producer-specific data
+        void* private_data;
+    };
+
+    #endif  // ARROW_C_DEVICE_STREAM_INTERFACE
+
+.. note::
+    The canonical guard ``ARROW_C_DEVICE_STREAM_INTERFACE`` is meant to avoid
+    duplicate definitions if two projects copy the C device stream interface
+    definitions into their own headers, and a third-party project includes
+    from these two projects. It is therefore important that this guard is
+    kept exactly as-is when these definitions are copied.
+
+The ArrowDeviceArrayStream structure
+------------------------------------
+
+The ``ArrowDeviceArrayStream`` provides a device type that can access the
+resulting data along with the required callbacks to interact with a
+streaming source of Arrow arrays. It has the following fields:
+
+.. c:member:: ArrowDeviceType device_type
+
+    The device type that this stream produces data on. All
+    ``ArrowDeviceArray``s that are produced by this stream should have the
+    same device type as is set here. This is a convenience for the consumer
+    to not have to check every array that is retrieved and instead allows
+    higher-level coding constructs for streams.
+
+.. c:member:: int (*ArrowDeviceArrayStream.get_schema)(struct ArrowDeviceArrayStream*, struct ArrowSchema* out)
+
+    *Mandatory.* This callback allows the consumer to query the schema of
+    the chunks of data in the stream. The schema is the same for all data
+    chunks.
+
+    This callback must NOT be called on a released ``ArrowDeviceArrayStream``.
+
+    *Return value:* 0 on success, a non-zero
+    :ref:`error code <c-stream-interface-error-codes>` otherwise.
+
+.. c:member:: int (*ArrowDeviceArrayStream.get_next)(struct ArrowDeviceArrayStream*, struct ArrowDeviceArray* out)
+
+    *Mandatory.* This callback allows the consumer to get the next chunk of
+    data in the stream.
+
+    This callback must NOT be called on a released ``ArrowDeviceArrayStream``.
+
+    The next chunk of data MUST be accessible from a device type matching the
+    :c:member:`ArrowDeviceArrayStream.device_type`.
+
+    *Return value:* 0 on success, a non-zero
+    :ref:`error code <c-stream-interface-error-codes>` otherwise.
+
+    On success, the consumer must check whether the ``ArrowDeviceArray``'s
+    embedded ``ArrowArray`` is marked :ref:`released <c-data-interface-released>`.
+    If the embedded ``ArrowDeviceArray.array`` is released, then the end of the
+    stream has been reached. Otherwise, the ``ArrowDeviceArray`` contains a
+    valid data chunk.
+
+.. c:member:: const char* (*ArrowDeviceArrayStream.get_last_error)(struct ArrowDeviceArrayStream*)
+
+    *Mandatory.* This callback allows the consumer to get a textual description
+    of the last error.
+
+    This callback must ONLY be called if the last operation on the
+    ``ArrowDeviceArrayStream`` returned an error. It must NOT be called on a
+    released ``ArrowDeviceArrayStream``.
+
+    *Return value:* a pointer to a NULL-terminated character string
+    (UTF8-encoded). NULL can also be returned if no detailed description is
+    available.
+
+    The returned pointer is only guaranteed to be valid until the next call
+    of one of the stream's callbacks. The character string it points to should
+    be copied to consumer-managed storage if it is intended to survive longer.
+
+.. c:member:: void (*ArrowDeviceArrayStream.release)(struct ArrowDeviceArrayStream*)
+
+    *Mandatory.* A pointer to a producer-provided release callback.
+
+.. c:member:: void* ArrowDeviceArrayStream.private_data
+
+    *Optional.* An opaque pointer to producer-provided private data.
+
+    Consumers MUST NOT process this member. Lifetime of this member is
+    handled by the producer, and especially by the release callback.
+
+Result lifetimes
+----------------
+
+The data returned by the ``get_schema`` and ``get_next`` callbacks must be
+released independantly. Their lifetimes are not tied to that of
+``ArrowDeviceArrayStream``.
+
+Stream lifetime
+---------------
+
+Lifetime of the C stream is managed using a release callback with similar
+usage as in :ref:`C data interface <c-data-interface-released>`.
+
+Thread safety
+-------------
+
+The stream source is not assumed to be thread-safe. Consumers wanting to
+call ``get_next`` from several threads should ensure those calls are
+serialized.
+
+Updating this specification
+===========================
+
+.. note::
+    Since this specification is still considered experimental, there is the
+    (still very low) possibility it might change slightly. Once it is
+    supported in an official Arrow release and the "experimental" tag is
+    removed from it, this section will apply and the ABI will be frozen.
+    
+    The reason for the "experimental" tag is because we don't know what we
+    don't know. While it was attempted to ensure this is generic enough to
+    work with a multitude of different frameworks, it's also possible that
+    something was missed. Once there is some usage of this and we are
+    confident there isn't any necessary modifications, the "experimental"
+    tag will be removed and the ABI frozen.

Review Comment:
   Is the new version better?



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [arrow] kkraus14 commented on pull request #34972: GH-34971: [Format] Enhance C-Data API to support non-cpu cases

["kkraus14 (via GitHub)" <gi...@apache.org>]

kkraus14 commented on PR #34972:
URL: https://github.com/apache/arrow/pull/34972#issuecomment-1511646211

   > so, @gmarkall @pitrou @kkraus14 do we have a consensus on the best way forward here? You three definitely have a lot more insight into this than I do, so I'll move forward with whichever approach you all can agree on 😄
   
   I'm comfortable with either option assuming we use an `Event` pointer as opposed to a `Stream` pointer in the producer-supplied case, with a slight preference to the consumer-supplied case.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [arrow] leofang commented on a diff in pull request #34972: GH-34971: [Format] Enhance C-Data API to support non-cpu cases

["leofang (via GitHub)" <gi...@apache.org>]

leofang commented on code in PR #34972:
URL: https://github.com/apache/arrow/pull/34972#discussion_r1169068200


##########
cpp/src/arrow/c/abi.h:
##########
@@ -106,6 +212,98 @@ struct ArrowArrayStream {
 
 #endif  // ARROW_C_STREAM_INTERFACE
 
+#ifndef ARROW_C_DEVICE_STREAM_INTERFACE
+#define ARROW_C_DEVICE_STREAM_INTERFACE
+
+/// \brief Equivalent to ArrowArrayStream, but for ArrowDeviceArrays.
+///
+/// This stream is intended to provide a stream of data on a single
+/// device, if a producer wants data to be produced on multiple devices
+/// then multiple streams should be provided. One per device.
+struct ArrowDeviceArrayStream {
+  /// \brief The device that this stream produces data on.
+  ///
+  /// All ArrowDeviceArrays that are produced by this
+  /// stream should have the same device_type as set
+  /// here. The device_type needs to be provided here
+  /// so that consumers can provide the correct type
+  /// of queue_ptr when calling get_next.
+  ArrowDeviceType device_type;
+
+  /// \brief Callback to get the stream schema
+  /// (will be the same for all arrays in the stream).
+  ///
+  /// If successful, the ArrowSchema must be released independantly from the stream.
+  /// The schema should be accessible via CPU memory.
+  ///
+  /// \param[in] self The ArrowDeviceArrayStream object itself
+  /// \param[out] out C struct to export the schema to
+  /// \return 0 if successful, an `errno`-compatible error code otherwise.
+  int (*get_schema)(struct ArrowDeviceArrayStream* self, struct ArrowSchema* out);
+
+  /// \brief Callback to get the device id for the next array.
+  ///
+  /// This is necessary so that the proper/correct stream pointer can be provided
+  /// to get_next.
+  ///
+  /// The next call to `get_next` should provide an ArrowDeviceArray whose
+  /// device_id matches what is provided here, and whose device_type is the
+  /// same as the device_type member of this stream.

Review Comment:
   > do you happen to know if there was any other discussion captured that could be linked here regarding the decision to have a consumer hand a stream to the producer
   
   Sorry I wasn't able to respond promptly. Is the question still open?
   
   In the case of CAI, [it is required](https://numba.readthedocs.io/en/stable/cuda/cuda_array_interface.html#streams) that someone handles the exporting stream's lifetime properly:
   > Like data, CUDA streams also have a finite lifetime. It is therefore required that a Producer exporting data on the interface with an associated stream ensures that the exported stream’s lifetime is equal to or surpasses the lifetime of the object from which the interface was exported.
   
   and this was considered a burden when discussing the DLPack support. A few libraries like Numba, for example, had to [hold the reference to the underlying stream](https://numba.readthedocs.io/en/stable/cuda/cuda_array_interface.html#producing-arrays). I believe this was the main concern for DLPack to place the requirement on the consumer instead of the producer.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [arrow] zeroshade commented on a diff in pull request #34972: GH-34971: [Format] Enhance C-Data API to support non-cpu cases

["zeroshade (via GitHub)" <gi...@apache.org>]

zeroshade commented on code in PR #34972:
URL: https://github.com/apache/arrow/pull/34972#discussion_r1161312010


##########
cpp/src/arrow/c/abi.h:
##########
@@ -65,6 +65,69 @@ struct ArrowArray {
 
 #endif  // ARROW_C_DATA_INTERFACE
 
+#ifndef ARROW_C_DEVICE_DATA_INTERFACE
+#define ARROW_C_DEVICE_DATA_INTERFACE
+
+// ArrowDeviceType is compatible with dlpack DLDeviceType for portability
+// it uses the same values for each enum as the equivalent kDL<type> from dlpack.h
+#ifdef __cplusplus
+typedef enum : int32_t {
+#else
+typedef enum {
+#endif

Review Comment:
   I got the idea for the ifdef to have it be concretely `int32_t` in C++ from the dlpack C API, but I'm fine with switching to an untyped enum or macros if preferred. I personally would prefer an untyped enum, but if everyone thinks the consistency with the ADBC header is more important I'll switch it to macros.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [arrow] kkraus14 commented on a diff in pull request #34972: GH-34971: [Format] Enhance C-Data API to support non-cpu cases

["kkraus14 (via GitHub)" <gi...@apache.org>]

kkraus14 commented on code in PR #34972:
URL: https://github.com/apache/arrow/pull/34972#discussion_r1163350151


##########
cpp/src/arrow/c/abi.h:
##########
@@ -106,6 +212,98 @@ struct ArrowArrayStream {
 
 #endif  // ARROW_C_STREAM_INTERFACE
 
+#ifndef ARROW_C_DEVICE_STREAM_INTERFACE
+#define ARROW_C_DEVICE_STREAM_INTERFACE
+
+/// \brief Equivalent to ArrowArrayStream, but for ArrowDeviceArrays.
+///
+/// This stream is intended to provide a stream of data on a single
+/// device, if a producer wants data to be produced on multiple devices
+/// then multiple streams should be provided. One per device.
+struct ArrowDeviceArrayStream {
+  /// \brief The device that this stream produces data on.
+  ///
+  /// All ArrowDeviceArrays that are produced by this
+  /// stream should have the same device_type as set
+  /// here. The device_type needs to be provided here
+  /// so that consumers can provide the correct type
+  /// of queue_ptr when calling get_next.
+  ArrowDeviceType device_type;
+
+  /// \brief Callback to get the stream schema
+  /// (will be the same for all arrays in the stream).
+  ///
+  /// If successful, the ArrowSchema must be released independantly from the stream.
+  /// The schema should be accessible via CPU memory.
+  ///
+  /// \param[in] self The ArrowDeviceArrayStream object itself
+  /// \param[out] out C struct to export the schema to
+  /// \return 0 if successful, an `errno`-compatible error code otherwise.
+  int (*get_schema)(struct ArrowDeviceArrayStream* self, struct ArrowSchema* out);
+
+  /// \brief Callback to get the device id for the next array.
+  ///
+  /// This is necessary so that the proper/correct stream pointer can be provided
+  /// to get_next.
+  ///
+  /// The next call to `get_next` should provide an ArrowDeviceArray whose
+  /// device_id matches what is provided here, and whose device_type is the
+  /// same as the device_type member of this stream.

Review Comment:
   cc @rgommers @leofang @tqchen (please feel free to include anyone else) do you happen to know if there was any other discussion captured that could be linked here regarding the decision to have a consumer hand a stream to the producer?



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [arrow] kkraus14 commented on a diff in pull request #34972: GH-34971: [Format] Enhance C-Data API to support non-cpu cases

["kkraus14 (via GitHub)" <gi...@apache.org>]

kkraus14 commented on code in PR #34972:
URL: https://github.com/apache/arrow/pull/34972#discussion_r1161842471


##########
cpp/src/arrow/c/abi.h:
##########
@@ -106,6 +169,77 @@ struct ArrowArrayStream {
 
 #endif  // ARROW_C_STREAM_INTERFACE
 
+#ifndef ARROW_C_DEVICE_STREAM_INTERFACE
+#define ARROW_C_DEVICE_STREAM_INTERFACE
+
+struct ArrowDeviceArrayStream {
+  // The device that this stream produces data on.
+  // All ArrowDeviceArrays that are produced by this
+  // stream should have the same device_type as set
+  // here. The device_type needs to be provided here
+  // so that consumers can provide the correct type
+  // of stream_ptr when calling get_next.
+  ArrowDeviceType device_type;
+
+  // Callback to get the stream schema
+  // (will be the same for all arrays in the stream).
+  //
+  // Return value: 0 if successful, an `errno`-compatible error code otherwise.
+  //
+  // If successful, the ArrowSchema must be released independently from the stream.
+  int (*get_schema)(struct ArrowDeviceArrayStream*, struct ArrowSchema* out);
+
+  // Callback to get the device id for the next array.
+  // This is necessary so that the proper/correct stream pointer can be provided
+  // to get_next. The parameter provided must not be null.
+  //
+  // Return value: 0 if successful, an `errno`-compatible error code otherwise.
+  //
+  // The next call to `get_next` should provide an ArrowDeviceArray whose
+  // device_id matches what is provided here, and whose device_type is the
+  // same as the device_type member of this stream.
+  int (*get_next_device_id)(struct ArrowDeviceArrayStream*, int* out_device_id);
+
+  // Callback to get the next array
+  // (if no error and the array is released, the stream has ended)
+  //
+  // the provided stream_ptr should be the appropriate stream, or
+  // equivalent object, for the device that the data is allocated on
+  // to indicate where the consumer wants the data to be accessible.
+  // if stream_ptr is NULL then the default stream (e.g. CUDA stream 0)
+  // should be used to ensure that the memory is accessible from any stream.

Review Comment:
   CUDA (NVIDIA GPUs) and ROCm (AMD GPUs) both use the term "stream". The SYCL equivalent is an in-order "queue". I'm not familiar enough with other device programming models to know what their terminology is or if a single entity is enough here.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [arrow] kkraus14 commented on a diff in pull request #34972: GH-34971: [Format] Enhance C-Data API to support non-cpu cases

["kkraus14 (via GitHub)" <gi...@apache.org>]

kkraus14 commented on code in PR #34972:
URL: https://github.com/apache/arrow/pull/34972#discussion_r1162959045


##########
cpp/src/arrow/c/abi.h:
##########
@@ -106,6 +212,98 @@ struct ArrowArrayStream {
 
 #endif  // ARROW_C_STREAM_INTERFACE
 
+#ifndef ARROW_C_DEVICE_STREAM_INTERFACE
+#define ARROW_C_DEVICE_STREAM_INTERFACE
+
+/// \brief Equivalent to ArrowArrayStream, but for ArrowDeviceArrays.
+///
+/// This stream is intended to provide a stream of data on a single
+/// device, if a producer wants data to be produced on multiple devices
+/// then multiple streams should be provided. One per device.
+struct ArrowDeviceArrayStream {
+  /// \brief The device that this stream produces data on.
+  ///
+  /// All ArrowDeviceArrays that are produced by this
+  /// stream should have the same device_type as set
+  /// here. The device_type needs to be provided here
+  /// so that consumers can provide the correct type
+  /// of queue_ptr when calling get_next.
+  ArrowDeviceType device_type;
+
+  /// \brief Callback to get the stream schema
+  /// (will be the same for all arrays in the stream).
+  ///
+  /// If successful, the ArrowSchema must be released independantly from the stream.
+  /// The schema should be accessible via CPU memory.
+  ///
+  /// \param[in] self The ArrowDeviceArrayStream object itself
+  /// \param[out] out C struct to export the schema to
+  /// \return 0 if successful, an `errno`-compatible error code otherwise.
+  int (*get_schema)(struct ArrowDeviceArrayStream* self, struct ArrowSchema* out);
+
+  /// \brief Callback to get the device id for the next array.
+  ///
+  /// This is necessary so that the proper/correct stream pointer can be provided
+  /// to get_next.
+  ///
+  /// The next call to `get_next` should provide an ArrowDeviceArray whose
+  /// device_id matches what is provided here, and whose device_type is the
+  /// same as the device_type member of this stream.

Review Comment:
   Many frameworks manage their streams internally and don't expose those streams in their APIs, so there isn't a clean way to share or release ownership of a stream in a way that a stream can be guaranteed to not be destroyed until the release callback is called.
   
   See the conversations here:
   - https://github.com/dmlc/dlpack/issues/57
   - https://github.com/data-apis/consortium-feedback/issues/1
   - https://github.com/data-apis/array-api/pull/106



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [arrow] pitrou commented on a diff in pull request #34972: GH-34971: [Format] Enhance C-Data API to support non-cpu cases

["pitrou (via GitHub)" <gi...@apache.org>]

pitrou commented on code in PR #34972:
URL: https://github.com/apache/arrow/pull/34972#discussion_r1163354415


##########
cpp/src/arrow/c/abi.h:
##########
@@ -106,6 +212,98 @@ struct ArrowArrayStream {
 
 #endif  // ARROW_C_STREAM_INTERFACE
 
+#ifndef ARROW_C_DEVICE_STREAM_INTERFACE
+#define ARROW_C_DEVICE_STREAM_INTERFACE
+
+/// \brief Equivalent to ArrowArrayStream, but for ArrowDeviceArrays.
+///
+/// This stream is intended to provide a stream of data on a single
+/// device, if a producer wants data to be produced on multiple devices
+/// then multiple streams should be provided. One per device.
+struct ArrowDeviceArrayStream {
+  /// \brief The device that this stream produces data on.
+  ///
+  /// All ArrowDeviceArrays that are produced by this
+  /// stream should have the same device_type as set
+  /// here. The device_type needs to be provided here
+  /// so that consumers can provide the correct type
+  /// of queue_ptr when calling get_next.
+  ArrowDeviceType device_type;
+
+  /// \brief Callback to get the stream schema
+  /// (will be the same for all arrays in the stream).
+  ///
+  /// If successful, the ArrowSchema must be released independantly from the stream.
+  /// The schema should be accessible via CPU memory.
+  ///
+  /// \param[in] self The ArrowDeviceArrayStream object itself
+  /// \param[out] out C struct to export the schema to
+  /// \return 0 if successful, an `errno`-compatible error code otherwise.
+  int (*get_schema)(struct ArrowDeviceArrayStream* self, struct ArrowSchema* out);
+
+  /// \brief Callback to get the device id for the next array.
+  ///
+  /// This is necessary so that the proper/correct stream pointer can be provided
+  /// to get_next.
+  ///
+  /// The next call to `get_next` should provide an ArrowDeviceArray whose
+  /// device_id matches what is provided here, and whose device_type is the
+  /// same as the device_type member of this stream.

Review Comment:
   Thanks the pointers.
   
   > I think this comment does a good job of summarizing the options that were considered: [dmlc/dlpack#57 (comment)](https://github.com/dmlc/dlpack/issues/57#issuecomment-753220425)
   
   Yes, I read this. It looks like solution S1, which is also the one I'm proposing, is considered the most flexible (I don't understand the "harder for compilers" comment, though).
   
   > And then this comment summarizes discussion of those options: [dmlc/dlpack#57 (comment)](https://github.com/dmlc/dlpack/issues/57#issuecomment-774482973)
   
   I read this too, but it doesn't actually mention S1, for reasons I wasn't able to understand.
   
   >  In most cases libraries don't associate a stream with the object since it's valid to use multiple streams with a single object.
   
   But you have to actually synchronize on the right stream before being able to use the object, right? How does the user know which stream to synchronize on, if they didn't produce the data themselves?
   



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [arrow] westonpace commented on a diff in pull request #34972: GH-34971: [Format] Enhance C-Data API to support non-cpu cases

["westonpace (via GitHub)" <gi...@apache.org>]

westonpace commented on code in PR #34972:
URL: https://github.com/apache/arrow/pull/34972#discussion_r1161955047


##########
cpp/src/arrow/c/abi.h:
##########
@@ -106,6 +212,98 @@ struct ArrowArrayStream {
 
 #endif  // ARROW_C_STREAM_INTERFACE
 
+#ifndef ARROW_C_DEVICE_STREAM_INTERFACE
+#define ARROW_C_DEVICE_STREAM_INTERFACE
+
+/// \brief Equivalent to ArrowArrayStream, but for ArrowDeviceArrays.
+///
+/// This stream is intended to provide a stream of data on a single
+/// device, if a producer wants data to be produced on multiple devices
+/// then multiple streams should be provided. One per device.
+struct ArrowDeviceArrayStream {
+  /// \brief The device that this stream produces data on.
+  ///
+  /// All ArrowDeviceArrays that are produced by this
+  /// stream should have the same device_type as set
+  /// here. The device_type needs to be provided here
+  /// so that consumers can provide the correct type
+  /// of queue_ptr when calling get_next.
+  ArrowDeviceType device_type;
+
+  /// \brief Callback to get the stream schema
+  /// (will be the same for all arrays in the stream).
+  ///
+  /// If successful, the ArrowSchema must be released independantly from the stream.
+  /// The schema should be accessible via CPU memory.
+  ///
+  /// \param[in] self The ArrowDeviceArrayStream object itself
+  /// \param[out] out C struct to export the schema to
+  /// \return 0 if successful, an `errno`-compatible error code otherwise.
+  int (*get_schema)(struct ArrowDeviceArrayStream* self, struct ArrowSchema* out);
+
+  /// \brief Callback to get the device id for the next array.
+  ///
+  /// This is necessary so that the proper/correct stream pointer can be provided
+  /// to get_next.
+  ///
+  /// The next call to `get_next` should provide an ArrowDeviceArray whose
+  /// device_id matches what is provided here, and whose device_type is the
+  /// same as the device_type member of this stream.

Review Comment:
   I'm not certain I follow.  Isn't the `ArrowDeviceArray` passed to `get_next` an "out" parameter?  Are you saying that the `ArrowDeviceArray` struct itself (not the buffers) needs to be allocated on the device?



##########
cpp/src/arrow/c/abi.h:
##########
@@ -106,6 +212,98 @@ struct ArrowArrayStream {
 
 #endif  // ARROW_C_STREAM_INTERFACE
 
+#ifndef ARROW_C_DEVICE_STREAM_INTERFACE
+#define ARROW_C_DEVICE_STREAM_INTERFACE
+
+/// \brief Equivalent to ArrowArrayStream, but for ArrowDeviceArrays.
+///
+/// This stream is intended to provide a stream of data on a single
+/// device, if a producer wants data to be produced on multiple devices
+/// then multiple streams should be provided. One per device.
+struct ArrowDeviceArrayStream {
+  /// \brief The device that this stream produces data on.
+  ///
+  /// All ArrowDeviceArrays that are produced by this
+  /// stream should have the same device_type as set
+  /// here. The device_type needs to be provided here
+  /// so that consumers can provide the correct type
+  /// of queue_ptr when calling get_next.
+  ArrowDeviceType device_type;
+
+  /// \brief Callback to get the stream schema
+  /// (will be the same for all arrays in the stream).
+  ///
+  /// If successful, the ArrowSchema must be released independantly from the stream.
+  /// The schema should be accessible via CPU memory.
+  ///
+  /// \param[in] self The ArrowDeviceArrayStream object itself
+  /// \param[out] out C struct to export the schema to
+  /// \return 0 if successful, an `errno`-compatible error code otherwise.
+  int (*get_schema)(struct ArrowDeviceArrayStream* self, struct ArrowSchema* out);
+
+  /// \brief Callback to get the device id for the next array.
+  ///
+  /// This is necessary so that the proper/correct stream pointer can be provided
+  /// to get_next.
+  ///
+  /// The next call to `get_next` should provide an ArrowDeviceArray whose
+  /// device_id matches what is provided here, and whose device_type is the
+  /// same as the device_type member of this stream.
+  ///
+  /// \param[in] self The ArrowDeviceArrayStream object itself
+  /// \param[out] out_device_id Pointer to be populated with the device id, must not be
+  /// null \return 0 if successful, an `errno`-compatible error code otherwise.
+  int (*get_next_device_id)(struct ArrowDeviceArrayStream* self, int64_t* out_device_id);
+
+  /// \brief Callback to get the next array
+  ///
+  /// If there is no error and the returned array has been released, the stream
+  /// has ended. If successful, the ArrowArray must be released independently
+  /// from the stream.
+  ///
+  /// Because different frameworks use different types to represent this, we
+  /// accept a void* which should then be reinterpreted into whatever the
+  /// appropriate type is (e.g. cudaStream_t) for use by the producer.
+  ///
+  /// \param[in] self The ArrowDeviceArrayStream object itself
+  /// \param[in] queue_ptr The appropriate queue, stream, or
+  /// equivalent object for the device that the data is allocated on
+  /// to indicate where the consumer wants the data to be accessible.
+  /// If queue_ptr is NULL then the default stream (e.g. CUDA stream 0)
+  /// should be used to ensure that the memory is accessible from any stream.

Review Comment:
   I'm a little confused here.  It sounds like I need to call `get_next_device_id` to determine which queue to use and then I need to pass that queue on to the call to `get_next`.  But why?  Why isn't the producer managing the queues?
   
   If the producer controls which device id gets used (`get_next_device_id` seems to suggest this) then why does the consumer need to give it the queue?  For example, if I were merging streams from two different devices it seems like I would do something like (apologies for the butchered pseudo-code)...
   
   ```
   // Dummy class merging two infinite streams in an inefficient round-robin fashion
   class MergedStream {
   
     int get_next(ArrowDeviceArray* out) {
       if (merged_arrays_.empty()) {
         ArrowDeviceArray arr;
         left_.get_next(&arr);
         merged_arrays_.push(arr);
         right_.get_next(&arr);
         merged_arrays_.push(arr);
       }
       *out = merged_arrays_.pop();
     }
   };
   ```



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [arrow] zeroshade commented on a diff in pull request #34972: GH-34971: [Format] Enhance C-Data API to support non-cpu cases

["zeroshade (via GitHub)" <gi...@apache.org>]

zeroshade commented on code in PR #34972:
URL: https://github.com/apache/arrow/pull/34972#discussion_r1163060504


##########
cpp/src/arrow/c/abi.h:
##########
@@ -65,6 +65,69 @@ struct ArrowArray {
 
 #endif  // ARROW_C_DATA_INTERFACE
 
+#ifndef ARROW_C_DEVICE_DATA_INTERFACE
+#define ARROW_C_DEVICE_DATA_INTERFACE
+
+// ArrowDeviceType is compatible with dlpack DLDeviceType for portability
+// it uses the same values for each enum as the equivalent kDL<type> from dlpack.h
+#ifdef __cplusplus
+typedef enum : int32_t {
+#else
+typedef enum {
+#endif
+  // CPU device, same as using ArrowArray directly
+  kArrowCPU = 1,
+  // CUDA GPU Device
+  kArrowCUDA = 2,
+  // Pinned CUDA CPU memory by cudaMallocHost
+  kArrowCUDAHost = 3,
+  // OpenCL Device
+  kArrowOpenCL = 4,
+  // Vulkan buffer for next-gen graphics
+  kArrowVulkan = 7,
+  // Metal for Apple GPU
+  kArrowMetal = 8,
+  // Verilog simulator buffer
+  kArrowVPI = 9,
+  // ROCm GPUs for AMD GPUs
+  kArrowROCM = 10,
+  // Pinned ROCm CPU memory allocated by hipMallocHost
+  kArrowROCMHost = 11,
+  // Reserved for extension
+  // used to quickly test extension devices,
+  // semantics can differ based on the implementation
+  kArrowExtDev = 12,
+  // CUDA managed/unified memory allocated by cudaMallocManaged
+  kArrowCUDAManaged = 13,
+  // unified shared memory allocated on a oneAPI non-partitioned
+  // device. call to oneAPI runtime is required to determine the
+  // device type, the USM allocation type and the sycl context it
+  // is bound to
+  kArrowOneAPI = 14,
+  // GPU support for next-gen WebGPU standard
+  kArrowWebGPU = 15,
+  // Qualcomm Hexagon DSP
+  kArrowHexagon = 16,
+} ArrowDeviceType;
+
+struct ArrowDeviceArray {
+  // the private_date and release callback of the arrow array
+  // should contain any necessary information and structures
+  // related to freeing the array according to the device it
+  // is allocated on, rather than having a separate release
+  // callback embedded here.
+  struct ArrowArray* array;
+  int device_id;

Review Comment:
   anyone have objections to using `int64_t` for future-proof reasons? Should I just use `int32_t` instead?



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [arrow] zeroshade commented on a diff in pull request #34972: GH-34971: [Format] Enhance C-Data API to support non-cpu cases

["zeroshade (via GitHub)" <gi...@apache.org>]

zeroshade commented on code in PR #34972:
URL: https://github.com/apache/arrow/pull/34972#discussion_r1161875233


##########
cpp/src/arrow/c/abi.h:
##########
@@ -65,6 +65,69 @@ struct ArrowArray {
 
 #endif  // ARROW_C_DATA_INTERFACE
 
+#ifndef ARROW_C_DEVICE_DATA_INTERFACE
+#define ARROW_C_DEVICE_DATA_INTERFACE
+
+// ArrowDeviceType is compatible with dlpack DLDeviceType for portability
+// it uses the same values for each enum as the equivalent kDL<type> from dlpack.h
+#ifdef __cplusplus
+typedef enum : int32_t {
+#else
+typedef enum {
+#endif
+  // CPU device, same as using ArrowArray directly
+  kArrowCPU = 1,
+  // CUDA GPU Device
+  kArrowCUDA = 2,
+  // Pinned CUDA CPU memory by cudaMallocHost
+  kArrowCUDAHost = 3,
+  // OpenCL Device
+  kArrowOpenCL = 4,
+  // Vulkan buffer for next-gen graphics
+  kArrowVulkan = 7,
+  // Metal for Apple GPU
+  kArrowMetal = 8,
+  // Verilog simulator buffer
+  kArrowVPI = 9,
+  // ROCm GPUs for AMD GPUs
+  kArrowROCM = 10,
+  // Pinned ROCm CPU memory allocated by hipMallocHost
+  kArrowROCMHost = 11,
+  // Reserved for extension
+  // used to quickly test extension devices,
+  // semantics can differ based on the implementation
+  kArrowExtDev = 12,
+  // CUDA managed/unified memory allocated by cudaMallocManaged
+  kArrowCUDAManaged = 13,
+  // unified shared memory allocated on a oneAPI non-partitioned
+  // device. call to oneAPI runtime is required to determine the
+  // device type, the USM allocation type and the sycl context it
+  // is bound to
+  kArrowOneAPI = 14,
+  // GPU support for next-gen WebGPU standard
+  kArrowWebGPU = 15,
+  // Qualcomm Hexagon DSP
+  kArrowHexagon = 16,
+} ArrowDeviceType;
+
+struct ArrowDeviceArray {
+  // the private_date and release callback of the arrow array
+  // should contain any necessary information and structures
+  // related to freeing the array according to the device it
+  // is allocated on, rather than having a separate release
+  // callback embedded here.
+  struct ArrowArray* array;

Review Comment:
   The `ArrowArray` struct itself can't change because that would break the ABI, but we might potentially create an `ArrowArrayV2` that would be good to easily be able to plug in here. 
   
   That said, "who owns the memory" is a good question, and technically if we did evolve this to point to an `ArrowArrayV2` in theory that would still be breaking because while the ABI would still work just fine (all the sizes are the same) attempting to access the members would fail because the struct itself would be different. So my entire idea of using a pointer here is flawed in general. I'll update this to just embed the struct.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [arrow] zeroshade commented on a diff in pull request #34972: GH-34971: [Format] Enhance C-Data API to support non-cpu cases

["zeroshade (via GitHub)" <gi...@apache.org>]

zeroshade commented on code in PR #34972:
URL: https://github.com/apache/arrow/pull/34972#discussion_r1161876798


##########
cpp/src/arrow/c/abi.h:
##########
@@ -106,6 +169,77 @@ struct ArrowArrayStream {
 
 #endif  // ARROW_C_STREAM_INTERFACE
 
+#ifndef ARROW_C_DEVICE_STREAM_INTERFACE
+#define ARROW_C_DEVICE_STREAM_INTERFACE
+
+struct ArrowDeviceArrayStream {
+  // The device that this stream produces data on.
+  // All ArrowDeviceArrays that are produced by this
+  // stream should have the same device_type as set
+  // here. The device_type needs to be provided here
+  // so that consumers can provide the correct type
+  // of stream_ptr when calling get_next.
+  ArrowDeviceType device_type;
+
+  // Callback to get the stream schema
+  // (will be the same for all arrays in the stream).
+  //
+  // Return value: 0 if successful, an `errno`-compatible error code otherwise.
+  //
+  // If successful, the ArrowSchema must be released independently from the stream.
+  int (*get_schema)(struct ArrowDeviceArrayStream*, struct ArrowSchema* out);
+
+  // Callback to get the device id for the next array.
+  // This is necessary so that the proper/correct stream pointer can be provided
+  // to get_next. The parameter provided must not be null.
+  //
+  // Return value: 0 if successful, an `errno`-compatible error code otherwise.
+  //
+  // The next call to `get_next` should provide an ArrowDeviceArray whose
+  // device_id matches what is provided here, and whose device_type is the
+  // same as the device_type member of this stream.
+  int (*get_next_device_id)(struct ArrowDeviceArrayStream*, int* out_device_id);
+
+  // Callback to get the next array
+  // (if no error and the array is released, the stream has ended)
+  //
+  // the provided stream_ptr should be the appropriate stream, or
+  // equivalent object, for the device that the data is allocated on
+  // to indicate where the consumer wants the data to be accessible.
+  // if stream_ptr is NULL then the default stream (e.g. CUDA stream 0)
+  // should be used to ensure that the memory is accessible from any stream.

Review Comment:
   what do we all think about using `queue` here then instead of `stream`? Is that enough to avoid the ambiguity?



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [arrow] zeroshade commented on pull request #34972: GH-34971: [Format] Enhance C-Data API to support non-cpu cases

["zeroshade (via GitHub)" <gi...@apache.org>]

zeroshade commented on PR #34972:
URL: https://github.com/apache/arrow/pull/34972#issuecomment-1520345125

   @pitrou Do we need an equivalent for `Event` or just use a `void*` and let other backends use whatever they would need appropriately?
   
   @aregm @oleksandr-pavlyk just poking again to see if either of you can respond to @kkraus14's comment? Thanks!
   
   Just to clarify: are we considering that we'd add the `Event` to the `ArrowDeviceArray` struct and remove the stream argument from the `get_next` ?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [arrow] zeroshade commented on a diff in pull request #34972: GH-34971: [Format] Enhance C-Data API to support non-cpu cases

["zeroshade (via GitHub)" <gi...@apache.org>]

zeroshade commented on code in PR #34972:
URL: https://github.com/apache/arrow/pull/34972#discussion_r1191316190


##########
cpp/src/arrow/c/abi.h:
##########
@@ -65,6 +85,111 @@ struct ArrowArray {
 
 #endif  // ARROW_C_DATA_INTERFACE
 
+#ifndef ARROW_C_DEVICE_DATA_INTERFACE
+#define ARROW_C_DEVICE_DATA_INTERFACE
+
+/// \defgroup arrow-device-types Device Types
+/// These macros are compatible with the dlpack DLDeviceType values,
+/// using the same value for each enum as the equivalent kDL<type>
+/// from dlpack.h. This list should continue to be kept in sync with
+/// the equivalent dlpack.h enum values over time to ensure 
+/// compatibility, rather than potentially diverging.
+///
+/// To ensure predictability with the ABI we use macros instead of
+/// an enum so the storage type is not compiler dependent.
+///
+/// @{
+
+/// \brief DeviceType for the allocated memory
+typedef int32_t ArrowDeviceType;
+
+/// \brief CPU device, same as using ArrowArray directly
+#define ARROW_DEVICE_CPU = 1
+/// \brief CUDA GPU Device
+#define ARROW_DEVICE_CUDA = 2
+/// \brief Pinned CUDA CPU memory by cudaMallocHost
+#define ARROW_DEVICE_CUDA_HOST = 3
+/// \brief OpenCL Device
+#define ARROW_DEVICE_OPENCL = 4
+/// \brief Vulkan buffer for next-gen graphics
+#define ARROW_DEVICE_VULKAN = 7
+/// \brief Metal for Apple GPU
+#define ARROW_DEVICE_METAL = 8
+/// \brief Verilog simulator buffer
+#define ARROW_DEVICE_VPI = 9
+/// \brief ROCm GPUs for AMD GPUs
+#define ARROW_DEVICE_ROCM = 10
+/// \brief Pinned ROCm CPU memory allocated by hipMallocHost
+#define ARROW_DEVICE_ROCMHOST = 11
+/// \brief Reserved for extension
+///
+/// used to quickly test extension devices, semantics
+/// can differ based on the implementation
+#define ARROW_DEVICE_EXT_DEV = 12
+/// \brief CUDA managed/unified memory allocated by cudaMallocManaged
+#define ARROW_DEVICE_CUDA_MANAGED = 13
+/// \brief unified shared memory allocated on a oneAPI
+/// non-partitioned device.
+///
+/// A call to the oneAPI runtime is required to determine the device
+/// type, the USM allocation type, and the sycl context it is bound to.
+#define ARROW_DEVICE_ONEAPI = 14
+/// \brief GPU support for next-gen WebGPU standard
+#define ARROW_DEVICE_WEBGPU = 15
+/// \brief Qualcomm Hexagon DSP
+#define ARROW_DEVICE_HEXAGON = 16
+
+/// @}
+
+/// \brief Struct for passing an Arrow Array alongside
+/// device memory information.
+struct ArrowDeviceArray {
+  /// \brief the Allocated Array
+  ///
+  /// the buffers in the array (along with the buffers of any
+  /// children) are what is allocated on the device.
+  ///
+  /// the private_data and release callback of the arrow array
+  /// should contain any necessary information and structures
+  /// related to freeing the array according to the device it
+  /// is allocated on, rather than having a separate release
+  /// callback embedded here.
+  struct ArrowArray array;
+  /// \brief The device id to identify a specific device
+  /// if multiple of this type are on the system.
+  ///
+  /// the semantics of the id will be hardware dependant.
+  int64_t device_id;
+  /// \brief The type of device which can access this memory.
+  ArrowDeviceType device_type;
+  /// \brief An event-like object to synchronize on if needed.
+  ///
+  /// Many devices, like GPUs, are primarily asynchronous with
+  /// respect to CPU processing. As such in order to safely access
+  /// memory, it is often necessary to have an object to synchronize
+  /// processing on. Since different devices will use different types
+  /// to specify this we use a void* that can be coerced into
+  /// whatever the device appropriate type is (e.g. cudaEvent_t for
+  /// CUDA and hipEvent_t for HIP). 
+  ///
+  /// If synchronization is not needed this can be null. If this is
+  /// non-null, then it should be used to call the appropriate sync
+  /// method for the device (e.g. cudaStreamWaitEvent / hipStreamWaitEvent).
+  void* sync_event;
+  /// \brief Reserved bytes for future expansion.
+  ///
+  /// As non-CPU development expands we can update this struct
+  /// without ABI breaking changes. This also rounds out the
+  /// total size of this struct to be 128 bytes (power of 2)
+  /// on 64-bit systems. These bytes should be zero'd out after 
+  /// allocation in order to ensure safe evolution of the ABI in 
+  /// the future.
+  int64_t reserved[3];
+  int32_t reserved_addl;

Review Comment:
   Nice catch on the compiler padding I forgot about that, I confirmed locally that if i remove the additional `int32_t reserved_addl;` then the static assert is valid (though it wouldn't be valid on a 32-bit machine so I'm not going to actually add that assert to the code).



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [arrow] lidavidm commented on a diff in pull request #34972: GH-34971: [Format] Enhance C-Data API to support non-cpu cases

["lidavidm (via GitHub)" <gi...@apache.org>]

lidavidm commented on code in PR #34972:
URL: https://github.com/apache/arrow/pull/34972#discussion_r1191133057


##########
cpp/src/arrow/c/abi.h:
##########
@@ -65,6 +85,111 @@ struct ArrowArray {
 
 #endif  // ARROW_C_DATA_INTERFACE
 
+#ifndef ARROW_C_DEVICE_DATA_INTERFACE
+#define ARROW_C_DEVICE_DATA_INTERFACE
+
+/// \defgroup arrow-device-types Device Types
+/// These macros are compatible with the dlpack DLDeviceType values,
+/// using the same value for each enum as the equivalent kDL<type>
+/// from dlpack.h. This list should continue to be kept in sync with
+/// the equivalent dlpack.h enum values over time to ensure 
+/// compatibility, rather than potentially diverging.
+///
+/// To ensure predictability with the ABI we use macros instead of
+/// an enum so the storage type is not compiler dependent.
+///
+/// @{
+
+/// \brief DeviceType for the allocated memory
+typedef int32_t ArrowDeviceType;
+
+/// \brief CPU device, same as using ArrowArray directly
+#define ARROW_DEVICE_CPU = 1
+/// \brief CUDA GPU Device
+#define ARROW_DEVICE_CUDA = 2
+/// \brief Pinned CUDA CPU memory by cudaMallocHost
+#define ARROW_DEVICE_CUDA_HOST = 3
+/// \brief OpenCL Device
+#define ARROW_DEVICE_OPENCL = 4
+/// \brief Vulkan buffer for next-gen graphics
+#define ARROW_DEVICE_VULKAN = 7
+/// \brief Metal for Apple GPU
+#define ARROW_DEVICE_METAL = 8
+/// \brief Verilog simulator buffer
+#define ARROW_DEVICE_VPI = 9
+/// \brief ROCm GPUs for AMD GPUs
+#define ARROW_DEVICE_ROCM = 10
+/// \brief Pinned ROCm CPU memory allocated by hipMallocHost
+#define ARROW_DEVICE_ROCMHOST = 11
+/// \brief Reserved for extension
+///
+/// used to quickly test extension devices, semantics
+/// can differ based on the implementation
+#define ARROW_DEVICE_EXT_DEV = 12
+/// \brief CUDA managed/unified memory allocated by cudaMallocManaged
+#define ARROW_DEVICE_CUDA_MANAGED = 13
+/// \brief unified shared memory allocated on a oneAPI
+/// non-partitioned device.
+///
+/// A call to the oneAPI runtime is required to determine the device
+/// type, the USM allocation type, and the sycl context it is bound to.
+#define ARROW_DEVICE_ONEAPI = 14
+/// \brief GPU support for next-gen WebGPU standard
+#define ARROW_DEVICE_WEBGPU = 15
+/// \brief Qualcomm Hexagon DSP
+#define ARROW_DEVICE_HEXAGON = 16
+
+/// @}
+
+/// \brief Struct for passing an Arrow Array alongside
+/// device memory information.
+struct ArrowDeviceArray {
+  /// \brief the Allocated Array
+  ///
+  /// the buffers in the array (along with the buffers of any
+  /// children) are what is allocated on the device.
+  ///
+  /// the private_data and release callback of the arrow array
+  /// should contain any necessary information and structures
+  /// related to freeing the array according to the device it
+  /// is allocated on, rather than having a separate release
+  /// callback embedded here.
+  struct ArrowArray array;
+  /// \brief The device id to identify a specific device
+  /// if multiple of this type are on the system.
+  ///
+  /// the semantics of the id will be hardware dependant.
+  int64_t device_id;
+  /// \brief The type of device which can access this memory.
+  ArrowDeviceType device_type;
+  /// \brief An event-like object to synchronize on if needed.
+  ///
+  /// Many devices, like GPUs, are primarily asynchronous with
+  /// respect to CPU processing. As such in order to safely access
+  /// memory, it is often necessary to have an object to synchronize
+  /// processing on. Since different devices will use different types

Review Comment:
   Should we spell out the expected type for each device?



##########
cpp/src/arrow/c/abi.h:
##########
@@ -106,6 +231,78 @@ struct ArrowArrayStream {
 
 #endif  // ARROW_C_STREAM_INTERFACE
 
+#ifndef ARROW_C_DEVICE_STREAM_INTERFACE
+#define ARROW_C_DEVICE_STREAM_INTERFACE
+
+/// \brief Equivalent to ArrowArrayStream, but for ArrowDeviceArrays.
+///
+/// This stream is intended to provide a stream of data on a single
+/// device, if a producer wants data to be produced on multiple devices
+/// then multiple streams should be provided. One per device.
+struct ArrowDeviceArrayStream {
+  /// \brief The device that this stream produces data on.
+  ///
+  /// All ArrowDeviceArrays that are produced by this
+  /// stream should have the same device_type as set
+  /// here. The device_type needs to be provided here
+  /// so that consumers can provide the correct type
+  /// of queue_ptr when calling get_next.
+  ArrowDeviceType device_type;
+
+  /// \brief Callback to get the stream schema
+  /// (will be the same for all arrays in the stream).
+  ///
+  /// If successful, the ArrowSchema must be released independantly from the stream.
+  /// The schema should be accessible via CPU memory.
+  ///
+  /// \param[in] self The ArrowDeviceArrayStream object itself
+  /// \param[out] out C struct to export the schema to
+  /// \return 0 if successful, an `errno`-compatible error code otherwise.
+  int (*get_schema)(struct ArrowDeviceArrayStream* self, struct ArrowSchema* out);
+
+  /// \brief Callback to get the next array
+  ///
+  /// If there is no error and the returned array has been released, the stream
+  /// has ended. If successful, the ArrowArray must be released independently
+  /// from the stream.
+  ///
+  /// Because different frameworks use different types to represent this, we
+  /// accept a void* which should then be reinterpreted into whatever the
+  /// appropriate type is (e.g. cudaStream_t) for use by the producer.

Review Comment:
   Outdated?



##########
cpp/src/arrow/c/abi.h:
##########
@@ -65,6 +85,111 @@ struct ArrowArray {
 
 #endif  // ARROW_C_DATA_INTERFACE
 
+#ifndef ARROW_C_DEVICE_DATA_INTERFACE
+#define ARROW_C_DEVICE_DATA_INTERFACE
+
+/// \defgroup arrow-device-types Device Types
+/// These macros are compatible with the dlpack DLDeviceType values,
+/// using the same value for each enum as the equivalent kDL<type>
+/// from dlpack.h. This list should continue to be kept in sync with
+/// the equivalent dlpack.h enum values over time to ensure 
+/// compatibility, rather than potentially diverging.
+///
+/// To ensure predictability with the ABI we use macros instead of
+/// an enum so the storage type is not compiler dependent.
+///
+/// @{
+
+/// \brief DeviceType for the allocated memory
+typedef int32_t ArrowDeviceType;
+
+/// \brief CPU device, same as using ArrowArray directly
+#define ARROW_DEVICE_CPU = 1
+/// \brief CUDA GPU Device
+#define ARROW_DEVICE_CUDA = 2
+/// \brief Pinned CUDA CPU memory by cudaMallocHost
+#define ARROW_DEVICE_CUDA_HOST = 3
+/// \brief OpenCL Device
+#define ARROW_DEVICE_OPENCL = 4
+/// \brief Vulkan buffer for next-gen graphics
+#define ARROW_DEVICE_VULKAN = 7
+/// \brief Metal for Apple GPU
+#define ARROW_DEVICE_METAL = 8
+/// \brief Verilog simulator buffer
+#define ARROW_DEVICE_VPI = 9
+/// \brief ROCm GPUs for AMD GPUs
+#define ARROW_DEVICE_ROCM = 10
+/// \brief Pinned ROCm CPU memory allocated by hipMallocHost
+#define ARROW_DEVICE_ROCMHOST = 11
+/// \brief Reserved for extension
+///
+/// used to quickly test extension devices, semantics
+/// can differ based on the implementation
+#define ARROW_DEVICE_EXT_DEV = 12
+/// \brief CUDA managed/unified memory allocated by cudaMallocManaged
+#define ARROW_DEVICE_CUDA_MANAGED = 13
+/// \brief unified shared memory allocated on a oneAPI
+/// non-partitioned device.
+///
+/// A call to the oneAPI runtime is required to determine the device
+/// type, the USM allocation type, and the sycl context it is bound to.
+#define ARROW_DEVICE_ONEAPI = 14
+/// \brief GPU support for next-gen WebGPU standard
+#define ARROW_DEVICE_WEBGPU = 15
+/// \brief Qualcomm Hexagon DSP
+#define ARROW_DEVICE_HEXAGON = 16
+
+/// @}
+
+/// \brief Struct for passing an Arrow Array alongside
+/// device memory information.
+struct ArrowDeviceArray {
+  /// \brief the Allocated Array
+  ///
+  /// the buffers in the array (along with the buffers of any
+  /// children) are what is allocated on the device.
+  ///
+  /// the private_data and release callback of the arrow array
+  /// should contain any necessary information and structures
+  /// related to freeing the array according to the device it
+  /// is allocated on, rather than having a separate release
+  /// callback embedded here.
+  struct ArrowArray array;
+  /// \brief The device id to identify a specific device
+  /// if multiple of this type are on the system.
+  ///
+  /// the semantics of the id will be hardware dependant.
+  int64_t device_id;
+  /// \brief The type of device which can access this memory.
+  ArrowDeviceType device_type;
+  /// \brief An event-like object to synchronize on if needed.
+  ///
+  /// Many devices, like GPUs, are primarily asynchronous with
+  /// respect to CPU processing. As such in order to safely access
+  /// memory, it is often necessary to have an object to synchronize
+  /// processing on. Since different devices will use different types
+  /// to specify this we use a void* that can be coerced into
+  /// whatever the device appropriate type is (e.g. cudaEvent_t for
+  /// CUDA and hipEvent_t for HIP). 
+  ///
+  /// If synchronization is not needed this can be null. If this is
+  /// non-null, then it should be used to call the appropriate sync
+  /// method for the device (e.g. cudaStreamWaitEvent / hipStreamWaitEvent).
+  void* sync_event;
+  /// \brief Reserved bytes for future expansion.
+  ///
+  /// As non-CPU development expands we can update this struct
+  /// without ABI breaking changes. This also rounds out the
+  /// total size of this struct to be 128 bytes (power of 2)
+  /// on 64-bit systems. These bytes should be zero'd out after 
+  /// allocation in order to ensure safe evolution of the ABI in 
+  /// the future.
+  int64_t reserved[3];
+  int32_t reserved_addl;

Review Comment:
   Is this padding needed? (ArrowDeviceType is 32 bits, but there's 4 bytes of padding after it so that the 64 bit field after it is aligned.)
   
   Also:
   
   ```
   > g++ abi.h 
   abi.h:308:40: error: static assertion failed
     308 | static_assert(sizeof(ArrowDeviceArray) == 128, "");
         |               ~~~~~~~~~~~~~~~~~~~~~~~~~^~~~~~
   ```



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [arrow] pitrou commented on a diff in pull request #34972: GH-34971: [Format] Enhance C-Data API to support non-cpu cases

["pitrou (via GitHub)" <gi...@apache.org>]

pitrou commented on code in PR #34972:
URL: https://github.com/apache/arrow/pull/34972#discussion_r1163062198


##########
cpp/src/arrow/c/abi.h:
##########
@@ -106,6 +212,98 @@ struct ArrowArrayStream {
 
 #endif  // ARROW_C_STREAM_INTERFACE
 
+#ifndef ARROW_C_DEVICE_STREAM_INTERFACE
+#define ARROW_C_DEVICE_STREAM_INTERFACE
+
+/// \brief Equivalent to ArrowArrayStream, but for ArrowDeviceArrays.
+///
+/// This stream is intended to provide a stream of data on a single
+/// device, if a producer wants data to be produced on multiple devices
+/// then multiple streams should be provided. One per device.
+struct ArrowDeviceArrayStream {
+  /// \brief The device that this stream produces data on.
+  ///
+  /// All ArrowDeviceArrays that are produced by this
+  /// stream should have the same device_type as set
+  /// here. The device_type needs to be provided here
+  /// so that consumers can provide the correct type
+  /// of queue_ptr when calling get_next.
+  ArrowDeviceType device_type;
+
+  /// \brief Callback to get the stream schema
+  /// (will be the same for all arrays in the stream).
+  ///
+  /// If successful, the ArrowSchema must be released independantly from the stream.
+  /// The schema should be accessible via CPU memory.
+  ///
+  /// \param[in] self The ArrowDeviceArrayStream object itself
+  /// \param[out] out C struct to export the schema to
+  /// \return 0 if successful, an `errno`-compatible error code otherwise.
+  int (*get_schema)(struct ArrowDeviceArrayStream* self, struct ArrowSchema* out);
+
+  /// \brief Callback to get the device id for the next array.
+  ///
+  /// This is necessary so that the proper/correct stream pointer can be provided
+  /// to get_next.
+  ///
+  /// The next call to `get_next` should provide an ArrowDeviceArray whose
+  /// device_id matches what is provided here, and whose device_type is the
+  /// same as the device_type member of this stream.
+  ///
+  /// \param[in] self The ArrowDeviceArrayStream object itself
+  /// \param[out] out_device_id Pointer to be populated with the device id, must not be
+  /// null \return 0 if successful, an `errno`-compatible error code otherwise.
+  int (*get_next_device_id)(struct ArrowDeviceArrayStream* self, int64_t* out_device_id);
+
+  /// \brief Callback to get the next array
+  ///
+  /// If there is no error and the returned array has been released, the stream
+  /// has ended. If successful, the ArrowArray must be released independently
+  /// from the stream.
+  ///
+  /// Because different frameworks use different types to represent this, we
+  /// accept a void* which should then be reinterpreted into whatever the
+  /// appropriate type is (e.g. cudaStream_t) for use by the producer.
+  ///
+  /// \param[in] self The ArrowDeviceArrayStream object itself
+  /// \param[in] queue_ptr The appropriate queue, stream, or
+  /// equivalent object for the device that the data is allocated on
+  /// to indicate where the consumer wants the data to be accessible.
+  /// If queue_ptr is NULL then the default stream (e.g. CUDA stream 0)
+  /// should be used to ensure that the memory is accessible from any stream.

Review Comment:
   Ok, what's wrong with the following:
   * Producer creates CUDA stream 3
   * Producer runs some kernel that produces data safe to consume on CUDA stream 3
   * Consumer gets a C Data Interface export of the previously produced data; the `ArrowArray`'s release callback is responsible for handling the lifetime of stream 3, in addition to the lifetime of the allocated data
   * Consumer either syncs on stream 3 directly, or chooses to sync stream 3 to its own stream 4 using the event APIs
   * When consumer is done, it calls the release callback, letting the producer dealing with CUDA stream 3 lifetime
   



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [arrow] kkraus14 commented on pull request #34972: GH-34971: [Format] Enhance C-Data API to support non-cpu cases

["kkraus14 (via GitHub)" <gi...@apache.org>]

kkraus14 commented on PR #34972:
URL: https://github.com/apache/arrow/pull/34972#issuecomment-1504195071

   @pitrou I think the other thing that hasn't been discussed in the above threads with regards to producer vs consumer provided stream is the development burden that will have to be incurred by the consumer in the different situations.
   
   In producer provided stream / event:
   - The default behavior would be for the producer to return a stream or event in the struct if appropriate based on the producer
   - It forces the consumer to understand and handle device streams / events, which could be a non-trivial ask
   - There's no way for the consumer to ask for the data to be synchronized and safe to access from any stream and for the stream or event to be nullptr
   
   In consumer provided stream:
   - The default behavior would be the consumer not providing a stream which would inform the producer to guarantee the data to be synchronized and safe to access from any stream
   - Someone could use consumer code that doesn't understand or utilize device streams or events in any way without issue


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [arrow] kkraus14 commented on a diff in pull request #34972: GH-34971: [Format] Enhance C-Data API to support non-cpu cases

["kkraus14 (via GitHub)" <gi...@apache.org>]

kkraus14 commented on code in PR #34972:
URL: https://github.com/apache/arrow/pull/34972#discussion_r1163354517


##########
cpp/src/arrow/c/abi.h:
##########
@@ -106,6 +212,98 @@ struct ArrowArrayStream {
 
 #endif  // ARROW_C_STREAM_INTERFACE
 
+#ifndef ARROW_C_DEVICE_STREAM_INTERFACE
+#define ARROW_C_DEVICE_STREAM_INTERFACE
+
+/// \brief Equivalent to ArrowArrayStream, but for ArrowDeviceArrays.
+///
+/// This stream is intended to provide a stream of data on a single
+/// device, if a producer wants data to be produced on multiple devices
+/// then multiple streams should be provided. One per device.
+struct ArrowDeviceArrayStream {
+  /// \brief The device that this stream produces data on.
+  ///
+  /// All ArrowDeviceArrays that are produced by this
+  /// stream should have the same device_type as set
+  /// here. The device_type needs to be provided here
+  /// so that consumers can provide the correct type
+  /// of queue_ptr when calling get_next.
+  ArrowDeviceType device_type;
+
+  /// \brief Callback to get the stream schema
+  /// (will be the same for all arrays in the stream).
+  ///
+  /// If successful, the ArrowSchema must be released independantly from the stream.
+  /// The schema should be accessible via CPU memory.
+  ///
+  /// \param[in] self The ArrowDeviceArrayStream object itself
+  /// \param[out] out C struct to export the schema to
+  /// \return 0 if successful, an `errno`-compatible error code otherwise.
+  int (*get_schema)(struct ArrowDeviceArrayStream* self, struct ArrowSchema* out);
+
+  /// \brief Callback to get the device id for the next array.
+  ///
+  /// This is necessary so that the proper/correct stream pointer can be provided
+  /// to get_next.
+  ///
+  /// The next call to `get_next` should provide an ArrowDeviceArray whose
+  /// device_id matches what is provided here, and whose device_type is the
+  /// same as the device_type member of this stream.
+  ///
+  /// \param[in] self The ArrowDeviceArrayStream object itself
+  /// \param[out] out_device_id Pointer to be populated with the device id, must not be
+  /// null \return 0 if successful, an `errno`-compatible error code otherwise.
+  int (*get_next_device_id)(struct ArrowDeviceArrayStream* self, int64_t* out_device_id);
+
+  /// \brief Callback to get the next array
+  ///
+  /// If there is no error and the returned array has been released, the stream
+  /// has ended. If successful, the ArrowArray must be released independently
+  /// from the stream.
+  ///
+  /// Because different frameworks use different types to represent this, we
+  /// accept a void* which should then be reinterpreted into whatever the
+  /// appropriate type is (e.g. cudaStream_t) for use by the producer.
+  ///
+  /// \param[in] self The ArrowDeviceArrayStream object itself
+  /// \param[in] queue_ptr The appropriate queue, stream, or
+  /// equivalent object for the device that the data is allocated on
+  /// to indicate where the consumer wants the data to be accessible.
+  /// If queue_ptr is NULL then the default stream (e.g. CUDA stream 0)
+  /// should be used to ensure that the memory is accessible from any stream.

Review Comment:
   Tensorflow for example. It uses CUDA streams internally but doesn't expose them in any real way.
   
   In many places CUDA streams get treated like CPU Threads, where you need to avoid data races on them and want to schedule work onto the same thread / stream where possible, but many consider them to be an internal implementation detail and don't build out their use of them to have their lifetime controlled externally.
   
   I.E. Arrow ThreadPool where you schedule work onto the thread pool, but there's no separate Thread object that controls the lifetime of a specific thread.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [arrow] kkraus14 commented on pull request #34972: GH-34971: [Format] Enhance C-Data API to support non-cpu cases

["kkraus14 (via GitHub)" <gi...@apache.org>]

kkraus14 commented on PR #34972:
URL: https://github.com/apache/arrow/pull/34972#issuecomment-1507322834

   > Am I missing something in this bullet list?
   
   In the producer-supplied case it might make more sense to expose an Event as opposed to a Stream. That would allow the producer to continue doing work on whatever stream they're using without any risk of the consumer and producer stepping on each others toes. If they happen to be on the same stream (i.e. default stream) then the `cudaStreamWaitEvent` / `hipStreamWaitEvent` should have minimal overhead.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [arrow] zeroshade commented on a diff in pull request #34972: GH-34971: [Format] Enhance C-Data API to support non-cpu cases

["zeroshade (via GitHub)" <gi...@apache.org>]

zeroshade commented on code in PR #34972:
URL: https://github.com/apache/arrow/pull/34972#discussion_r1161907250


##########
cpp/src/arrow/c/abi.h:
##########
@@ -65,6 +65,69 @@ struct ArrowArray {
 
 #endif  // ARROW_C_DATA_INTERFACE
 
+#ifndef ARROW_C_DEVICE_DATA_INTERFACE
+#define ARROW_C_DEVICE_DATA_INTERFACE
+
+// ArrowDeviceType is compatible with dlpack DLDeviceType for portability
+// it uses the same values for each enum as the equivalent kDL<type> from dlpack.h
+#ifdef __cplusplus
+typedef enum : int32_t {
+#else
+typedef enum {
+#endif
+  // CPU device, same as using ArrowArray directly
+  kArrowCPU = 1,
+  // CUDA GPU Device
+  kArrowCUDA = 2,
+  // Pinned CUDA CPU memory by cudaMallocHost
+  kArrowCUDAHost = 3,
+  // OpenCL Device
+  kArrowOpenCL = 4,
+  // Vulkan buffer for next-gen graphics
+  kArrowVulkan = 7,
+  // Metal for Apple GPU
+  kArrowMetal = 8,
+  // Verilog simulator buffer
+  kArrowVPI = 9,
+  // ROCm GPUs for AMD GPUs
+  kArrowROCM = 10,
+  // Pinned ROCm CPU memory allocated by hipMallocHost
+  kArrowROCMHost = 11,
+  // Reserved for extension
+  // used to quickly test extension devices,
+  // semantics can differ based on the implementation
+  kArrowExtDev = 12,
+  // CUDA managed/unified memory allocated by cudaMallocManaged
+  kArrowCUDAManaged = 13,
+  // unified shared memory allocated on a oneAPI non-partitioned
+  // device. call to oneAPI runtime is required to determine the
+  // device type, the USM allocation type and the sycl context it
+  // is bound to
+  kArrowOneAPI = 14,
+  // GPU support for next-gen WebGPU standard
+  kArrowWebGPU = 15,
+  // Qualcomm Hexagon DSP
+  kArrowHexagon = 16,
+} ArrowDeviceType;
+
+struct ArrowDeviceArray {
+  // the private_date and release callback of the arrow array
+  // should contain any necessary information and structures
+  // related to freeing the array according to the device it
+  // is allocated on, rather than having a separate release
+  // callback embedded here.
+  struct ArrowArray* array;
+  int device_id;
+  ArrowDeviceType device_type;
+  // reserve 128 bytes for future expansion
+  // of this struct as non-CPU development expands
+  // so that we can update without ABI breaking
+  // changes.

Review Comment:
   comment updated.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [arrow] zeroshade commented on a diff in pull request #34972: GH-34971: [Format] Enhance C-Data API to support non-cpu cases

["zeroshade (via GitHub)" <gi...@apache.org>]

zeroshade commented on code in PR #34972:
URL: https://github.com/apache/arrow/pull/34972#discussion_r1161910213


##########
cpp/src/arrow/c/abi.h:
##########
@@ -106,6 +169,77 @@ struct ArrowArrayStream {
 
 #endif  // ARROW_C_STREAM_INTERFACE
 
+#ifndef ARROW_C_DEVICE_STREAM_INTERFACE
+#define ARROW_C_DEVICE_STREAM_INTERFACE
+
+struct ArrowDeviceArrayStream {
+  // The device that this stream produces data on.
+  // All ArrowDeviceArrays that are produced by this
+  // stream should have the same device_type as set
+  // here. The device_type needs to be provided here
+  // so that consumers can provide the correct type
+  // of stream_ptr when calling get_next.
+  ArrowDeviceType device_type;
+
+  // Callback to get the stream schema
+  // (will be the same for all arrays in the stream).
+  //
+  // Return value: 0 if successful, an `errno`-compatible error code otherwise.
+  //
+  // If successful, the ArrowSchema must be released independently from the stream.
+  int (*get_schema)(struct ArrowDeviceArrayStream*, struct ArrowSchema* out);
+
+  // Callback to get the device id for the next array.
+  // This is necessary so that the proper/correct stream pointer can be provided
+  // to get_next. The parameter provided must not be null.
+  //
+  // Return value: 0 if successful, an `errno`-compatible error code otherwise.
+  //
+  // The next call to `get_next` should provide an ArrowDeviceArray whose
+  // device_id matches what is provided here, and whose device_type is the
+  // same as the device_type member of this stream.
+  int (*get_next_device_id)(struct ArrowDeviceArrayStream*, int* out_device_id);
+
+  // Callback to get the next array
+  // (if no error and the array is released, the stream has ended)
+  //
+  // the provided stream_ptr should be the appropriate stream, or

Review Comment:
   resolving this in favor of the other conversation on the same topic: https://github.com/apache/arrow/pull/34972/files#r1161080258
   
   Please unresolve and comment if this should be continued separately here for some reason.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [arrow] pitrou commented on a diff in pull request #34972: GH-34971: [Format] Enhance C-Data API to support non-cpu cases

["pitrou (via GitHub)" <gi...@apache.org>]

pitrou commented on code in PR #34972:
URL: https://github.com/apache/arrow/pull/34972#discussion_r1163890065


##########
cpp/src/arrow/c/abi.h:
##########
@@ -106,6 +212,98 @@ struct ArrowArrayStream {
 
 #endif  // ARROW_C_STREAM_INTERFACE
 
+#ifndef ARROW_C_DEVICE_STREAM_INTERFACE
+#define ARROW_C_DEVICE_STREAM_INTERFACE
+
+/// \brief Equivalent to ArrowArrayStream, but for ArrowDeviceArrays.
+///
+/// This stream is intended to provide a stream of data on a single
+/// device, if a producer wants data to be produced on multiple devices
+/// then multiple streams should be provided. One per device.
+struct ArrowDeviceArrayStream {
+  /// \brief The device that this stream produces data on.
+  ///
+  /// All ArrowDeviceArrays that are produced by this
+  /// stream should have the same device_type as set
+  /// here. The device_type needs to be provided here
+  /// so that consumers can provide the correct type
+  /// of queue_ptr when calling get_next.
+  ArrowDeviceType device_type;
+
+  /// \brief Callback to get the stream schema
+  /// (will be the same for all arrays in the stream).
+  ///
+  /// If successful, the ArrowSchema must be released independantly from the stream.
+  /// The schema should be accessible via CPU memory.
+  ///
+  /// \param[in] self The ArrowDeviceArrayStream object itself
+  /// \param[out] out C struct to export the schema to
+  /// \return 0 if successful, an `errno`-compatible error code otherwise.
+  int (*get_schema)(struct ArrowDeviceArrayStream* self, struct ArrowSchema* out);
+
+  /// \brief Callback to get the device id for the next array.
+  ///
+  /// This is necessary so that the proper/correct stream pointer can be provided
+  /// to get_next.
+  ///
+  /// The next call to `get_next` should provide an ArrowDeviceArray whose
+  /// device_id matches what is provided here, and whose device_type is the
+  /// same as the device_type member of this stream.
+  ///
+  /// \param[in] self The ArrowDeviceArrayStream object itself
+  /// \param[out] out_device_id Pointer to be populated with the device id, must not be
+  /// null \return 0 if successful, an `errno`-compatible error code otherwise.
+  int (*get_next_device_id)(struct ArrowDeviceArrayStream* self, int64_t* out_device_id);
+
+  /// \brief Callback to get the next array
+  ///
+  /// If there is no error and the returned array has been released, the stream
+  /// has ended. If successful, the ArrowArray must be released independently
+  /// from the stream.
+  ///
+  /// Because different frameworks use different types to represent this, we
+  /// accept a void* which should then be reinterpreted into whatever the
+  /// appropriate type is (e.g. cudaStream_t) for use by the producer.
+  ///
+  /// \param[in] self The ArrowDeviceArrayStream object itself
+  /// \param[in] queue_ptr The appropriate queue, stream, or
+  /// equivalent object for the device that the data is allocated on
+  /// to indicate where the consumer wants the data to be accessible.
+  /// If queue_ptr is NULL then the default stream (e.g. CUDA stream 0)
+  /// should be used to ensure that the memory is accessible from any stream.

Review Comment:
   Ok, so Tensorflow doesn't expose its CUDA streams to the user... but can the user request that Tensorflow data be tied to a specific CUDA stream? Otherwise, how can we implement the "consumer-provided CUDA stream" semantics?



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [arrow] zeroshade commented on a diff in pull request #34972: GH-34971: [Format] Add non-CPU version of C Data Interface

["zeroshade (via GitHub)" <gi...@apache.org>]

zeroshade commented on code in PR #34972:
URL: https://github.com/apache/arrow/pull/34972#discussion_r1213607755


##########
docs/source/format/CDeviceDataInterface.rst:
##########
@@ -0,0 +1,628 @@
+.. Licensed to the Apache Software Foundation (ASF) under one
+.. or more contributor license agreements.  See the NOTICE file
+.. distributed with this work for additional information
+.. regarding copyright ownership.  The ASF licenses this file
+.. to you under the Apache License, Version 2.0 (the
+.. "License"); you may not use this file except in compliance
+.. with the License.  You may obtain a copy of the License at
+
+..   http://www.apache.org/licenses/LICENSE-2.0
+
+.. Unless required by applicable law or agreed to in writing,
+.. software distributed under the License is distributed on an
+.. "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+.. KIND, either express or implied.  See the License for the
+.. specific language governing permissions and limitations
+.. under the License.
+
+.. highlight:: c
+
+.. _c-device-data-interface:
+
+=================================
+The Arrow C Device data interface
+=================================
+
+.. note:: The Arrow C Device Data Interface should be considered experimental
+
+Rationale
+=========
+
+The current :ref:`C Data Interface <c-data-interface>`, and most
+implementations of it, make the assumption that all data buffers provided
+are CPU buffers. Since Apache Arrow is designed to be a universal in-memory
+format for representing tabular ("columnar") data, there will be the desire
+to leverage this data on non-CPU hardware such as GPUs. One example of such
+a case is the `RAPIDS cuDF library`_ which uses the Arrow memory format with
+CUDA for NVIDIA GPUs. Since copying data from host to device and back is
+expensive, the ideal would be to be able to leave the data on the device
+for as long as possible, even when passing it between runtimes and
+libraries.
+
+The Arrow C Device data interface builds on the existing C data interface
+by adding a very small, stable set of C definitions to it. These definitions
+are equivalents to the ``ArrowArray`` and ``ArrowArrayStream`` structures
+from the C Data Interface which add members to allow specifying the device
+type and pass necessary information to synchronize with the producer.
+For non-C/C++ languages and runtimes, translating the C definitions to
+corresponding C FFI declarations should be just as simple as with the
+current C data interface.
+
+Applications and libraries can then use Arrow schemas and Arrow formatted
+memory on non-CPU devices to exchange data just as easily as they do
+now with CPU data. This will enable leaving data on those devices longer
+and avoiding costly copies back and forth between the host and device
+just to leverage new libraries and runtimes.
+
+Goals
+-----
+
+* Expose an ABI-stable interface built on the existing C data interface.
+* Make it easy for third-party projects to implement support with little
+  initial investment.
+* Allow zero-copy sharing of Arrow formatted device memory between
+  independant runtimes and components running in the same process.
+* Avoid the need for one-to-one adaptation layers such as the
+  `CUDA Array Interface`_ for Python processes to pass CUDA data.
+* Enable integration without explicit dependencies (either at compile-time
+  or runtime) on the Arrow software project itself.
+
+The intent is for the Arrow C Device data interface to expand the reach
+of the current C data interface, allowing it to also become the standard
+low-level building block for columnar processing on devices like GPUs or
+FPGAs.
+
+Structure definitions
+=====================
+
+Because this is built on the C data interface, the C Device data interface
+uses the ``ArrowSchema`` and ``ArrowArray`` structures as defined in the
+:ref:`C data interface spec <c-data-interface-struct-defs>`. It then adds the
+following free-standing definitions. Like the rest of the Arrow project,
+they are available under the Apache License 2.0.
+
+.. code-block:: c
+
+    #ifndef ARROW_C_DEVICE_DATA_INTERFACE
+    #define ARROW_C_DEVICE_DATA_INTERFACE
+
+    // Device type for the allocated memory
+    typedef int32_t ArrowDeviceType;
+
+    // CPU device, same as using ArrowArray directly
+    #define ARROW_DEVICE_CPU 1
+    // CUDA GPU Device
+    #define ARROW_DEVICE_CUDA 2
+    // Pinned CUDA CPU memory by cudaMallocHost
+    #define ARROW_DEVICE_CUDA_HOST 3
+    // OpenCL Device
+    #define ARROW_DEVICE_OPENCL 4
+    // Vulkan buffer for next-gen graphics
+    #define ARROW_DEVICE_VULKAN 7
+    // Metal for Apple GPU
+    #define ARROW_DEVICE_METAL 8
+    // Verilog simulator buffer
+    #define ARROW_DEVICE_VPI 9
+    // ROCm GPUs for AMD GPUs
+    #define ARROW_DEVICE_ROCM 10
+    // Pinned ROCm CPU memory allocated by hipMallocHost
+    #define ARROW_DEVICE_ROCM_HOST 11
+    // Reserved for extension
+    //
+    // used to quickly test extension devices, semantics
+    // can differ based on implementation
+    #define ARROW_DEVICE_EXT_DEV 12
+    // CUDA managed/unified memory allocated by cudaMallocManaged
+    #define ARROW_DEVICE_CUDA_MANAGED 13
+    // Unified shared memory allocated on a oneAPI
+    // non-partitioned device.
+    //
+    // A call to the oneAPI runtime is required to determine the
+    // device type, the USM allocation type and the sycl context
+    // that it is bound to.
+    #define ARROW_DEVICE_ONEAPI 14
+    // GPU support for next-gen WebGPU standard
+    #define ARROW_DEVICE_WEBGPU 15
+    // Qualcomm Hexagon DSP
+    #define ARROW_DEVICE_HEXAGON 16
+
+    struct ArrowDeviceArray {
+      struct ArrowArray array;
+      int64_t device_id;
+      ArrowDeviceType device_type;
+      void* sync_event;
+
+      // reserved bytes for future expansion
+      int64_t reserved[3];
+    };
+
+    #endif  // ARROW_C_DEVICE_DATA_INTERFACE
+
+.. note::
+   The canonical guard ``ARROW_C_DEVICE_DATA_INTERFACE`` is meant to avoid
+   duplicate definitions if two projects copy the definitions in their own
+   headers, and a third-party project includes from these two projects. It
+   is therefore important that this guard is kept exactly as-is when these
+   definitions are copied.
+
+ArrowDeviceType
+---------------
+
+The ``ArrowDeviceType`` typedef is used to indicate what type of device the
+provided memory buffers were allocated on. This, in conjunction with the
+``device_id``, should be sufficient to reference the correct data buffers.
+
+We then use macros to define values for different device types. The provided
+macro values are compatible with the widely used `dlpack`_ ``DLDeviceType``
+definition values, using the same value for each as the equivalent
+``kDL<type>`` enum from dlpack.h. The list will be kept in sync with those
+equivalent enum values over time to ensure compatibility, rather than
+potentially diverging. To avoid the Arrow project having to be in the
+position of vetting new hardware devices, new additions should first be
+added to dlpack before we add a corresponding macro here.
+
+To ensure predictability with the ABI, we use macros instead of an ``enum``
+so the storage type is not compiler dependent.
+
+.. c:macro:: ARROW_DEVICE_CPU
+
+    CPU Device, equivalent to just using ``ArrowArray`` directly instead of
+    using ``ArrowDeviceArray``.
+
+.. c:macro:: ARROW_DEVICE_CUDA
+
+    A `CUDA`_ GPU Device. This could represent data allocated either with the
+    runtime library (``cudaMalloc``) or the device driver (``cuMemAlloc``).
+
+.. c:macro:: ARROW_DEVICE_CUDA_HOST
+
+    CPU memory that was pinned and page-locked by CUDA by using
+    ``cudaMallocHost`` or ``cuMemAllocHost``.
+
+.. c:macro:: ARROW_DEVICE_OPENCL
+
+    Data allocated on the device by using the `OpenCL (Open Computing Language)`_
+    framework.
+
+.. c:macro:: ARROW_DEVICE_VULKAN
+
+    Data allocated by the `Vulkan`_ framework and libraries.
+
+.. c:macro:: ARROW_DEVICE_METAL
+
+    Data on Apple GPU devices using the `Metal`_ framework and libraries.
+
+.. c:macro:: ARROW_DEVICE_VPI
+
+    Indicates usage of a Verilog simulator buffer.
+
+.. c:macro:: ARROW_DEVICE_ROCM
+
+    An AMD device using the `ROCm`_ stack.
+
+.. c:macro:: ARROW_DEVICE_ROCM_HOST
+
+    CPU memory pinned and page-locked allocated ``hipMallocHost``.
+
+.. c:macro:: ARROW_DEVICE_EXT_DEV
+
+    This value is an escape-hatch for devices to extend which aren't
+    currently represented otherwise. Producers would need to provide
+    additional information/context specific to the device if using
+    this device type. This is used to quickly test extension devices
+    and semantics can differ based on the implementation.
+
+.. c:macro:: ARROW_DEVICE_CUDA_MANAGED
+
+    CUDA managed/unified memory which is allocated by ``cudaMallocManaged``.
+
+.. c:macro:: ARROW_DEVICE_ONEAPI
+
+    Unified shared memory allocated on an Intel `oneAPI`_ non-partitioned
+    device. A call to the ``oneAPI`` runtime is required to determine
+    the specific device type, the USM allocation type and the sycl context
+    that it is bound to.
+
+.. c:macro:: ARROW_DEVICE_WEBGPU
+
+    GPU support for next-gen WebGPU standards
+
+.. c:macro:: ARROW_DEVICE_HEXAGON
+
+    Data allocated on a Qualcomm Hexagon DSP device.
+
+The ArrowDeviceArray structure
+------------------------------
+
+The ``ArrowDeviceArray`` structure embeds the C data ``ArrowArray`` structure
+and adds additional information necessary for consumers to use the data. It
+has the following fields:
+
+.. c:member:: struct ArrowArray ArrowDeviceArray.array
+
+    The allocated array data. The values in the ``void**`` buffers (along
+    with the buffers of any children) are what is allocated on the device.
+    The buffer values should be device pointers. The rest of the structure
+    should be accessible to the CPU.
+
+    The ``private_data`` and ``release`` callback of this structure should
+    contain any necessary information and structures related to freeing
+    the array according to the device it is allocated on, rather than
+    having a separate release callback and ``private_data`` pointer here.
+
+.. c:member:: int64_t ArrowDeviceArray.device_id
+
+    The device id to identify a specific device if multiple devices of this
+    type are on the system. The semantics of the id will be hardware dependent,
+    but we use an ``int64_t`` to future-proof the id as devices change over time.
+
+.. c:member:: ArrowDeviceType ArrowDeviceArray.device_type
+
+    The type of the device which can access the buffers in the array.
+
+.. c:member:: void* ArrowDeviceArray.sync_event
+
+    Optional. An event-like object to synchronize on if needed.
+
+    Many devices, like GPUs, are primarily asynchronous with respect to
+    CPU processing. As such in order to safely access memory, it is often
+    necessary to have an object to synchronize processing with. Since
+    different devices will use different types to specify this, we use a
+    void* which can be coerced into a pointer to whatever the device
+    appropriate type is.
+
+    If synchronization is not needed, this can be null. If this is non-null
+    then it MUST be used to call the appropriate sync method for the device
+    (e.g. ``cudaStreamWaitEvent`` or ``hipStreamWaitEvent``) before attempting
+    to access the memory in the buffers.
+
+    Expected types to coerce this ``void*`` to depending on the reported

Review Comment:
   Good idea, i've updated with the table and it's own section.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [arrow] zeroshade commented on a diff in pull request #34972: GH-34971: [Format] Add non-CPU version of C Data Interface

["zeroshade (via GitHub)" <gi...@apache.org>]

zeroshade commented on code in PR #34972:
URL: https://github.com/apache/arrow/pull/34972#discussion_r1219874770


##########
docs/source/format/CDeviceDataInterface.rst:
##########
@@ -0,0 +1,628 @@
+.. Licensed to the Apache Software Foundation (ASF) under one
+.. or more contributor license agreements.  See the NOTICE file
+.. distributed with this work for additional information
+.. regarding copyright ownership.  The ASF licenses this file
+.. to you under the Apache License, Version 2.0 (the
+.. "License"); you may not use this file except in compliance
+.. with the License.  You may obtain a copy of the License at
+
+..   http://www.apache.org/licenses/LICENSE-2.0
+
+.. Unless required by applicable law or agreed to in writing,
+.. software distributed under the License is distributed on an
+.. "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+.. KIND, either express or implied.  See the License for the
+.. specific language governing permissions and limitations
+.. under the License.
+
+.. highlight:: c
+
+.. _c-device-data-interface:
+
+=================================
+The Arrow C Device data interface
+=================================
+
+.. note:: The Arrow C Device Data Interface should be considered experimental
+
+Rationale
+=========
+
+The current :ref:`C Data Interface <c-data-interface>`, and most
+implementations of it, make the assumption that all data buffers provided
+are CPU buffers. Since Apache Arrow is designed to be a universal in-memory
+format for representing tabular ("columnar") data, there will be the desire
+to leverage this data on non-CPU hardware such as GPUs. One example of such
+a case is the `RAPIDS cuDF library`_ which uses the Arrow memory format with
+CUDA for NVIDIA GPUs. Since copying data from host to device and back is
+expensive, the ideal would be to be able to leave the data on the device
+for as long as possible, even when passing it between runtimes and
+libraries.
+
+The Arrow C Device data interface builds on the existing C data interface
+by adding a very small, stable set of C definitions to it. These definitions
+are equivalents to the ``ArrowArray`` and ``ArrowArrayStream`` structures
+from the C Data Interface which add members to allow specifying the device
+type and pass necessary information to synchronize with the producer.
+For non-C/C++ languages and runtimes, translating the C definitions to
+corresponding C FFI declarations should be just as simple as with the
+current C data interface.
+
+Applications and libraries can then use Arrow schemas and Arrow formatted
+memory on non-CPU devices to exchange data just as easily as they do
+now with CPU data. This will enable leaving data on those devices longer
+and avoiding costly copies back and forth between the host and device
+just to leverage new libraries and runtimes.
+
+Goals
+-----
+
+* Expose an ABI-stable interface built on the existing C data interface.
+* Make it easy for third-party projects to implement support with little
+  initial investment.
+* Allow zero-copy sharing of Arrow formatted device memory between
+  independant runtimes and components running in the same process.
+* Avoid the need for one-to-one adaptation layers such as the
+  `CUDA Array Interface`_ for Python processes to pass CUDA data.
+* Enable integration without explicit dependencies (either at compile-time
+  or runtime) on the Arrow software project itself.
+
+The intent is for the Arrow C Device data interface to expand the reach
+of the current C data interface, allowing it to also become the standard
+low-level building block for columnar processing on devices like GPUs or
+FPGAs.
+
+Structure definitions
+=====================
+
+Because this is built on the C data interface, the C Device data interface
+uses the ``ArrowSchema`` and ``ArrowArray`` structures as defined in the
+:ref:`C data interface spec <c-data-interface-struct-defs>`. It then adds the
+following free-standing definitions. Like the rest of the Arrow project,
+they are available under the Apache License 2.0.
+
+.. code-block:: c
+
+    #ifndef ARROW_C_DEVICE_DATA_INTERFACE
+    #define ARROW_C_DEVICE_DATA_INTERFACE
+
+    // Device type for the allocated memory
+    typedef int32_t ArrowDeviceType;
+
+    // CPU device, same as using ArrowArray directly
+    #define ARROW_DEVICE_CPU 1
+    // CUDA GPU Device
+    #define ARROW_DEVICE_CUDA 2
+    // Pinned CUDA CPU memory by cudaMallocHost
+    #define ARROW_DEVICE_CUDA_HOST 3
+    // OpenCL Device
+    #define ARROW_DEVICE_OPENCL 4
+    // Vulkan buffer for next-gen graphics
+    #define ARROW_DEVICE_VULKAN 7
+    // Metal for Apple GPU
+    #define ARROW_DEVICE_METAL 8
+    // Verilog simulator buffer
+    #define ARROW_DEVICE_VPI 9
+    // ROCm GPUs for AMD GPUs
+    #define ARROW_DEVICE_ROCM 10
+    // Pinned ROCm CPU memory allocated by hipMallocHost
+    #define ARROW_DEVICE_ROCM_HOST 11
+    // Reserved for extension
+    //
+    // used to quickly test extension devices, semantics
+    // can differ based on implementation
+    #define ARROW_DEVICE_EXT_DEV 12
+    // CUDA managed/unified memory allocated by cudaMallocManaged
+    #define ARROW_DEVICE_CUDA_MANAGED 13
+    // Unified shared memory allocated on a oneAPI
+    // non-partitioned device.
+    //
+    // A call to the oneAPI runtime is required to determine the
+    // device type, the USM allocation type and the sycl context
+    // that it is bound to.
+    #define ARROW_DEVICE_ONEAPI 14
+    // GPU support for next-gen WebGPU standard
+    #define ARROW_DEVICE_WEBGPU 15
+    // Qualcomm Hexagon DSP
+    #define ARROW_DEVICE_HEXAGON 16
+
+    struct ArrowDeviceArray {
+      struct ArrowArray array;
+      int64_t device_id;
+      ArrowDeviceType device_type;
+      void* sync_event;
+
+      // reserved bytes for future expansion
+      int64_t reserved[3];
+    };
+
+    #endif  // ARROW_C_DEVICE_DATA_INTERFACE
+
+.. note::
+   The canonical guard ``ARROW_C_DEVICE_DATA_INTERFACE`` is meant to avoid
+   duplicate definitions if two projects copy the definitions in their own
+   headers, and a third-party project includes from these two projects. It
+   is therefore important that this guard is kept exactly as-is when these
+   definitions are copied.
+
+ArrowDeviceType
+---------------
+
+The ``ArrowDeviceType`` typedef is used to indicate what type of device the
+provided memory buffers were allocated on. This, in conjunction with the
+``device_id``, should be sufficient to reference the correct data buffers.
+
+We then use macros to define values for different device types. The provided
+macro values are compatible with the widely used `dlpack`_ ``DLDeviceType``
+definition values, using the same value for each as the equivalent
+``kDL<type>`` enum from dlpack.h. The list will be kept in sync with those
+equivalent enum values over time to ensure compatibility, rather than
+potentially diverging. To avoid the Arrow project having to be in the
+position of vetting new hardware devices, new additions should first be
+added to dlpack before we add a corresponding macro here.
+
+To ensure predictability with the ABI, we use macros instead of an ``enum``
+so the storage type is not compiler dependent.
+
+.. c:macro:: ARROW_DEVICE_CPU
+
+    CPU Device, equivalent to just using ``ArrowArray`` directly instead of
+    using ``ArrowDeviceArray``.
+
+.. c:macro:: ARROW_DEVICE_CUDA
+
+    A `CUDA`_ GPU Device. This could represent data allocated either with the
+    runtime library (``cudaMalloc``) or the device driver (``cuMemAlloc``).
+
+.. c:macro:: ARROW_DEVICE_CUDA_HOST
+
+    CPU memory that was pinned and page-locked by CUDA by using
+    ``cudaMallocHost`` or ``cuMemAllocHost``.
+
+.. c:macro:: ARROW_DEVICE_OPENCL
+
+    Data allocated on the device by using the `OpenCL (Open Computing Language)`_
+    framework.
+
+.. c:macro:: ARROW_DEVICE_VULKAN
+
+    Data allocated by the `Vulkan`_ framework and libraries.
+
+.. c:macro:: ARROW_DEVICE_METAL
+
+    Data on Apple GPU devices using the `Metal`_ framework and libraries.
+
+.. c:macro:: ARROW_DEVICE_VPI
+
+    Indicates usage of a Verilog simulator buffer.
+
+.. c:macro:: ARROW_DEVICE_ROCM
+
+    An AMD device using the `ROCm`_ stack.
+
+.. c:macro:: ARROW_DEVICE_ROCM_HOST
+
+    CPU memory pinned and page-locked allocated ``hipMallocHost``.
+
+.. c:macro:: ARROW_DEVICE_EXT_DEV
+
+    This value is an escape-hatch for devices to extend which aren't
+    currently represented otherwise. Producers would need to provide
+    additional information/context specific to the device if using
+    this device type. This is used to quickly test extension devices
+    and semantics can differ based on the implementation.
+
+.. c:macro:: ARROW_DEVICE_CUDA_MANAGED
+
+    CUDA managed/unified memory which is allocated by ``cudaMallocManaged``.
+
+.. c:macro:: ARROW_DEVICE_ONEAPI
+
+    Unified shared memory allocated on an Intel `oneAPI`_ non-partitioned
+    device. A call to the ``oneAPI`` runtime is required to determine
+    the specific device type, the USM allocation type and the sycl context
+    that it is bound to.
+
+.. c:macro:: ARROW_DEVICE_WEBGPU
+
+    GPU support for next-gen WebGPU standards
+
+.. c:macro:: ARROW_DEVICE_HEXAGON
+
+    Data allocated on a Qualcomm Hexagon DSP device.
+
+The ArrowDeviceArray structure
+------------------------------
+
+The ``ArrowDeviceArray`` structure embeds the C data ``ArrowArray`` structure
+and adds additional information necessary for consumers to use the data. It
+has the following fields:
+
+.. c:member:: struct ArrowArray ArrowDeviceArray.array
+
+    The allocated array data. The values in the ``void**`` buffers (along
+    with the buffers of any children) are what is allocated on the device.
+    The buffer values should be device pointers. The rest of the structure
+    should be accessible to the CPU.
+
+    The ``private_data`` and ``release`` callback of this structure should
+    contain any necessary information and structures related to freeing
+    the array according to the device it is allocated on, rather than
+    having a separate release callback and ``private_data`` pointer here.
+
+.. c:member:: int64_t ArrowDeviceArray.device_id
+
+    The device id to identify a specific device if multiple devices of this
+    type are on the system. The semantics of the id will be hardware dependent,
+    but we use an ``int64_t`` to future-proof the id as devices change over time.
+
+.. c:member:: ArrowDeviceType ArrowDeviceArray.device_type
+
+    The type of the device which can access the buffers in the array.
+
+.. c:member:: void* ArrowDeviceArray.sync_event
+
+    Optional. An event-like object to synchronize on if needed.
+
+    Many devices, like GPUs, are primarily asynchronous with respect to
+    CPU processing. As such in order to safely access memory, it is often
+    necessary to have an object to synchronize processing with. Since
+    different devices will use different types to specify this, we use a
+    void* which can be coerced into a pointer to whatever the device
+    appropriate type is.
+
+    If synchronization is not needed, this can be null. If this is non-null
+    then it MUST be used to call the appropriate sync method for the device
+    (e.g. ``cudaStreamWaitEvent`` or ``hipStreamWaitEvent``) before attempting
+    to access the memory in the buffers.
+
+    Expected types to coerce this ``void*`` to depending on the reported
+    device type:
+
+    * CUDA: ``cudaEvent_t*``
+    * ROCm: ``hipEvent_t*``
+    * OpenCL: ``cl_event*``
+    * Vulkan: ``VkEvent*``
+    * Metal: ``MTLEvent*``
+    * OneAPI: ``sycl::event*``
+
+    If an event is provided, then the producer MUST ensure that the event
+    is triggered/recorded at the end of the processing stream once the data
+    is considered available for use.
+
+
+.. c:member:: int64_t ArrowDeviceArray.reserved[3]
+
+    As non-CPU development expands, there may be a need to expand this
+    structure. In order to do so without potentially breaking ABI changes,
+    we reserve 24 bytes at the end of the object. This also has the added
+    benefit of bringing the total size of this structure to exactly 128
+    bytes (a power of 2) on 64-bit systems. These bytes should be zero'd
+    out after allocation in order to ensure safe evolution of the ABI in
+    the future.
+
+.. note::
+    Rather than store the shape / types of the data alongside the

Review Comment:
   added the new section



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [arrow] GregoryKimball commented on a diff in pull request #34972: GH-34971: [Format] Enhance C-Data API to support non-cpu cases

["GregoryKimball (via GitHub)" <gi...@apache.org>]

GregoryKimball commented on code in PR #34972:
URL: https://github.com/apache/arrow/pull/34972#discussion_r1192642045


##########
cpp/src/arrow/c/abi.h:
##########
@@ -65,6 +85,110 @@ struct ArrowArray {
 
 #endif  // ARROW_C_DATA_INTERFACE
 
+#ifndef ARROW_C_DEVICE_DATA_INTERFACE
+#define ARROW_C_DEVICE_DATA_INTERFACE
+
+/// \defgroup arrow-device-types Device Types
+/// These macros are compatible with the dlpack DLDeviceType values,
+/// using the same value for each enum as the equivalent kDL<type>
+/// from dlpack.h. This list should continue to be kept in sync with
+/// the equivalent dlpack.h enum values over time to ensure
+/// compatibility, rather than potentially diverging.
+///
+/// To ensure predictability with the ABI we use macros instead of
+/// an enum so the storage type is not compiler dependent.
+///
+/// @{
+
+/// \brief DeviceType for the allocated memory
+typedef int32_t ArrowDeviceType;
+
+/// \brief CPU device, same as using ArrowArray directly
+#define ARROW_DEVICE_CPU = 1
+/// \brief CUDA GPU Device
+#define ARROW_DEVICE_CUDA = 2
+/// \brief Pinned CUDA CPU memory by cudaMallocHost
+#define ARROW_DEVICE_CUDA_HOST = 3
+/// \brief OpenCL Device
+#define ARROW_DEVICE_OPENCL = 4
+/// \brief Vulkan buffer for next-gen graphics
+#define ARROW_DEVICE_VULKAN = 7
+/// \brief Metal for Apple GPU
+#define ARROW_DEVICE_METAL = 8
+/// \brief Verilog simulator buffer
+#define ARROW_DEVICE_VPI = 9
+/// \brief ROCm GPUs for AMD GPUs
+#define ARROW_DEVICE_ROCM = 10
+/// \brief Pinned ROCm CPU memory allocated by hipMallocHost
+#define ARROW_DEVICE_ROCMHOST = 11
+/// \brief Reserved for extension
+///
+/// used to quickly test extension devices, semantics
+/// can differ based on the implementation
+#define ARROW_DEVICE_EXT_DEV = 12
+/// \brief CUDA managed/unified memory allocated by cudaMallocManaged
+#define ARROW_DEVICE_CUDA_MANAGED = 13
+/// \brief unified shared memory allocated on a oneAPI
+/// non-partitioned device.
+///
+/// A call to the oneAPI runtime is required to determine the device
+/// type, the USM allocation type, and the sycl context it is bound to.
+#define ARROW_DEVICE_ONEAPI = 14
+/// \brief GPU support for next-gen WebGPU standard
+#define ARROW_DEVICE_WEBGPU = 15
+/// \brief Qualcomm Hexagon DSP
+#define ARROW_DEVICE_HEXAGON = 16
+
+/// @}
+
+/// \brief Struct for passing an Arrow Array alongside
+/// device memory information.
+struct ArrowDeviceArray {
+  /// \brief the Allocated Array
+  ///
+  /// the buffers in the array (along with the buffers of any
+  /// children) are what is allocated on the device.
+  ///
+  /// the private_data and release callback of the arrow array
+  /// should contain any necessary information and structures
+  /// related to freeing the array according to the device it
+  /// is allocated on, rather than having a separate release
+  /// callback embedded here.
+  struct ArrowArray array;
+  /// \brief The device id to identify a specific device
+  /// if multiple of this type are on the system.
+  ///
+  /// the semantics of the id will be hardware dependant.
+  int64_t device_id;
+  /// \brief The type of device which can access this memory.
+  ArrowDeviceType device_type;
+  /// \brief An event-like object to synchronize on if needed.
+  ///
+  /// Many devices, like GPUs, are primarily asynchronous with
+  /// respect to CPU processing. As such in order to safely access
+  /// memory, it is often necessary to have an object to synchronize
+  /// processing on. Since different devices will use different types
+  /// to specify this we use a void* that can be coerced into
+  /// whatever the device appropriate type is (e.g. cudaEvent_t for
+  /// CUDA and hipEvent_t for HIP).
+  ///
+  /// If synchronization is not needed this can be null. If this is
+  /// non-null, then it should be used to call the appropriate sync
+  /// method for the device (e.g. cudaStreamWaitEvent / hipStreamWaitEvent).
+  void* sync_event;

Review Comment:
   Would you please help me understand the `void* sync_event` pointer a bit better in the context of CUDA/C++? Is this a function pointer that is expected to be called with a `cudaStream_t` parameter provided by the application? Would there be a benefit from storing a `cudaStream_t` in `ArrowDeviceArray` (in the `reserved` bytes or elsewhere)?



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [arrow] lidavidm commented on a diff in pull request #34972: GH-34971: [Format] Enhance C-Data API to support non-cpu cases

["lidavidm (via GitHub)" <gi...@apache.org>]

lidavidm commented on code in PR #34972:
URL: https://github.com/apache/arrow/pull/34972#discussion_r1191514014


##########
cpp/src/arrow/c/abi.h:
##########
@@ -65,6 +85,111 @@ struct ArrowArray {
 
 #endif  // ARROW_C_DATA_INTERFACE
 
+#ifndef ARROW_C_DEVICE_DATA_INTERFACE
+#define ARROW_C_DEVICE_DATA_INTERFACE
+
+/// \defgroup arrow-device-types Device Types
+/// These macros are compatible with the dlpack DLDeviceType values,
+/// using the same value for each enum as the equivalent kDL<type>
+/// from dlpack.h. This list should continue to be kept in sync with
+/// the equivalent dlpack.h enum values over time to ensure 
+/// compatibility, rather than potentially diverging.
+///
+/// To ensure predictability with the ABI we use macros instead of
+/// an enum so the storage type is not compiler dependent.
+///
+/// @{
+
+/// \brief DeviceType for the allocated memory
+typedef int32_t ArrowDeviceType;
+
+/// \brief CPU device, same as using ArrowArray directly
+#define ARROW_DEVICE_CPU = 1
+/// \brief CUDA GPU Device
+#define ARROW_DEVICE_CUDA = 2
+/// \brief Pinned CUDA CPU memory by cudaMallocHost
+#define ARROW_DEVICE_CUDA_HOST = 3
+/// \brief OpenCL Device
+#define ARROW_DEVICE_OPENCL = 4
+/// \brief Vulkan buffer for next-gen graphics
+#define ARROW_DEVICE_VULKAN = 7
+/// \brief Metal for Apple GPU
+#define ARROW_DEVICE_METAL = 8
+/// \brief Verilog simulator buffer
+#define ARROW_DEVICE_VPI = 9
+/// \brief ROCm GPUs for AMD GPUs
+#define ARROW_DEVICE_ROCM = 10
+/// \brief Pinned ROCm CPU memory allocated by hipMallocHost
+#define ARROW_DEVICE_ROCMHOST = 11
+/// \brief Reserved for extension
+///
+/// used to quickly test extension devices, semantics
+/// can differ based on the implementation
+#define ARROW_DEVICE_EXT_DEV = 12
+/// \brief CUDA managed/unified memory allocated by cudaMallocManaged
+#define ARROW_DEVICE_CUDA_MANAGED = 13
+/// \brief unified shared memory allocated on a oneAPI
+/// non-partitioned device.
+///
+/// A call to the oneAPI runtime is required to determine the device
+/// type, the USM allocation type, and the sycl context it is bound to.
+#define ARROW_DEVICE_ONEAPI = 14
+/// \brief GPU support for next-gen WebGPU standard
+#define ARROW_DEVICE_WEBGPU = 15
+/// \brief Qualcomm Hexagon DSP
+#define ARROW_DEVICE_HEXAGON = 16
+
+/// @}
+
+/// \brief Struct for passing an Arrow Array alongside
+/// device memory information.
+struct ArrowDeviceArray {
+  /// \brief the Allocated Array
+  ///
+  /// the buffers in the array (along with the buffers of any
+  /// children) are what is allocated on the device.
+  ///
+  /// the private_data and release callback of the arrow array
+  /// should contain any necessary information and structures
+  /// related to freeing the array according to the device it
+  /// is allocated on, rather than having a separate release
+  /// callback embedded here.
+  struct ArrowArray array;
+  /// \brief The device id to identify a specific device
+  /// if multiple of this type are on the system.
+  ///
+  /// the semantics of the id will be hardware dependant.
+  int64_t device_id;
+  /// \brief The type of device which can access this memory.
+  ArrowDeviceType device_type;
+  /// \brief An event-like object to synchronize on if needed.
+  ///
+  /// Many devices, like GPUs, are primarily asynchronous with
+  /// respect to CPU processing. As such in order to safely access
+  /// memory, it is often necessary to have an object to synchronize
+  /// processing on. Since different devices will use different types

Review Comment:
   We can leave it then if it's unclear (maybe just specify it for CUDA?)



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [arrow] zeroshade commented on a diff in pull request #34972: GH-34971: [Format] Enhance C-Data API to support non-cpu cases

["zeroshade (via GitHub)" <gi...@apache.org>]

zeroshade commented on code in PR #34972:
URL: https://github.com/apache/arrow/pull/34972#discussion_r1204368645


##########
docs/source/format/CDeviceDataInterface.rst:
##########
@@ -0,0 +1,613 @@
+.. Licensed to the Apache Software Foundation (ASF) under one
+.. or more contributor license agreements.  See the NOTICE file
+.. distributed with this work for additional information
+.. regarding copyright ownership.  The ASF licenses this file
+.. to you under the Apache License, Version 2.0 (the
+.. "License"); you may not use this file except in compliance
+.. with the License.  You may obtain a copy of the License at
+
+..   http://www.apache.org/licenses/LICENSE-2.0
+
+.. Unless required by applicable law or agreed to in writing,
+.. software distributed under the License is distributed on an
+.. "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+.. KIND, either express or implied.  See the License for the
+.. specific language governing permissions and limitations
+.. under the License.
+
+.. highlight:: c
+
+.. _c-device-data-interface:
+
+=================================
+The Arrow C Device data interface
+=================================
+
+.. note:: The Arrow C Device Data Interface should be considered experimental
+
+Rationale
+=========
+
+The current :ref:`C Data Interface <c-data-interface>`, and most
+implementations of it, make the assumption that all data buffers provided
+are CPU buffers. Since Apache Arrow is designed to be a universal in-memory
+format for representing tabular ("columnar") data, there will be the desire
+to leverage this data on Non-CPU hardware such as GPUs. One example of such
+a case is the `RAPIDS cuDF library` which uses the Arrow memory format with
+CUDA for NVIDIA GPUs. Since copying data from host to device and back is
+expensive, the ideal would be to be able to leave the data on the device
+for as long as possible, even when passing it between runtimes and
+libraries.
+
+The Arrow C Device data interface builds on the existing C data interface
+by adding a very small, stable set of C definitions to it. For non-C/C++
+languages and runtimes, translating the C definitions to corresponding
+C FFI declarations should be just as simple as with the current C data
+interface.

Review Comment:
   added a sentence here to explain. Let me know what you think



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [arrow] pitrou commented on a diff in pull request #34972: GH-34971: [Format] Enhance C-Data API to support non-cpu cases

["pitrou (via GitHub)" <gi...@apache.org>]

pitrou commented on code in PR #34972:
URL: https://github.com/apache/arrow/pull/34972#discussion_r1202435196


##########
cpp/src/arrow/c/abi.h:
##########
@@ -65,6 +85,119 @@ struct ArrowArray {
 
 #endif  // ARROW_C_DATA_INTERFACE
 
+#ifndef ARROW_C_DEVICE_DATA_INTERFACE
+#define ARROW_C_DEVICE_DATA_INTERFACE
+
+/// \defgroup arrow-device-types Device Types
+/// These macros are compatible with the dlpack DLDeviceType values,
+/// using the same value for each enum as the equivalent kDL<type>
+/// from dlpack.h. This list should continue to be kept in sync with
+/// the equivalent dlpack.h enum values over time to ensure
+/// compatibility, rather than potentially diverging.
+///
+/// To ensure predictability with the ABI we use macros instead of
+/// an enum so the storage type is not compiler dependent.
+///
+/// @{
+
+/// \brief DeviceType for the allocated memory
+typedef int32_t ArrowDeviceType;
+
+/// \brief CPU device, same as using ArrowArray directly
+#define ARROW_DEVICE_CPU = 1
+/// \brief CUDA GPU Device
+#define ARROW_DEVICE_CUDA = 2
+/// \brief Pinned CUDA CPU memory by cudaMallocHost
+#define ARROW_DEVICE_CUDA_HOST = 3
+/// \brief OpenCL Device
+#define ARROW_DEVICE_OPENCL = 4
+/// \brief Vulkan buffer for next-gen graphics
+#define ARROW_DEVICE_VULKAN = 7
+/// \brief Metal for Apple GPU
+#define ARROW_DEVICE_METAL = 8
+/// \brief Verilog simulator buffer
+#define ARROW_DEVICE_VPI = 9
+/// \brief ROCm GPUs for AMD GPUs
+#define ARROW_DEVICE_ROCM = 10
+/// \brief Pinned ROCm CPU memory allocated by hipMallocHost
+#define ARROW_DEVICE_ROCM_HOST = 11
+/// \brief Reserved for extension
+///
+/// used to quickly test extension devices, semantics
+/// can differ based on the implementation
+#define ARROW_DEVICE_EXT_DEV = 12
+/// \brief CUDA managed/unified memory allocated by cudaMallocManaged
+#define ARROW_DEVICE_CUDA_MANAGED = 13
+/// \brief unified shared memory allocated on a oneAPI
+/// non-partitioned device.
+///
+/// A call to the oneAPI runtime is required to determine the device
+/// type, the USM allocation type, and the sycl context it is bound to.
+#define ARROW_DEVICE_ONEAPI = 14
+/// \brief GPU support for next-gen WebGPU standard
+#define ARROW_DEVICE_WEBGPU = 15
+/// \brief Qualcomm Hexagon DSP
+#define ARROW_DEVICE_HEXAGON = 16
+
+/// @}
+
+/// \brief Struct for passing an Arrow Array alongside
+/// device memory information.
+struct ArrowDeviceArray {
+  /// \brief the Allocated Array

Review Comment:
   As for `ArrowArray` I would describe the struct and its members in the spec itself, rather than in this `.h` file.



##########
cpp/src/arrow/c/abi.h:
##########
@@ -106,6 +239,74 @@ struct ArrowArrayStream {
 
 #endif  // ARROW_C_STREAM_INTERFACE
 
+#ifndef ARROW_C_DEVICE_STREAM_INTERFACE
+#define ARROW_C_DEVICE_STREAM_INTERFACE
+
+/// \brief Equivalent to ArrowArrayStream, but for ArrowDeviceArrays.

Review Comment:
   Same here: describe the struct and its members in the spec, not in the `.h`.



##########
cpp/src/arrow/c/abi.h:
##########
@@ -106,6 +239,74 @@ struct ArrowArrayStream {
 
 #endif  // ARROW_C_STREAM_INTERFACE
 
+#ifndef ARROW_C_DEVICE_STREAM_INTERFACE
+#define ARROW_C_DEVICE_STREAM_INTERFACE
+
+/// \brief Equivalent to ArrowArrayStream, but for ArrowDeviceArrays.
+///
+/// This stream is intended to provide a stream of data on a single
+/// device, if a producer wants data to be produced on multiple devices
+/// then multiple streams should be provided. One per device.
+struct ArrowDeviceArrayStream {
+  /// \brief The device that this stream produces data on.
+  ///
+  /// All ArrowDeviceArrays that are produced by this
+  /// stream should have the same device_type as set
+  /// here. The device_type needs to be provided here
+  /// so that consumers can provide the correct type
+  /// of queue_ptr when calling get_next.

Review Comment:
   This seems obsolete as consumers do not need to provide a queue ptr anymore?



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [arrow] github-actions[bot] commented on pull request #34972: GH-34971: [Format] Enhance C-Data API to support non-cpu cases

["github-actions[bot] (via GitHub)" <gi...@apache.org>]

github-actions[bot] commented on PR #34972:
URL: https://github.com/apache/arrow/pull/34972#issuecomment-1500457519

   :warning: GitHub issue #34971 **has been automatically assigned in GitHub** to PR creator.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [arrow] pitrou commented on a diff in pull request #34972: GH-34971: [Format] Enhance C-Data API to support non-cpu cases

["pitrou (via GitHub)" <gi...@apache.org>]

pitrou commented on code in PR #34972:
URL: https://github.com/apache/arrow/pull/34972#discussion_r1161080057


##########
cpp/src/arrow/c/abi.h:
##########
@@ -65,6 +65,69 @@ struct ArrowArray {
 
 #endif  // ARROW_C_DATA_INTERFACE
 
+#ifndef ARROW_C_DEVICE_DATA_INTERFACE
+#define ARROW_C_DEVICE_DATA_INTERFACE
+
+// ArrowDeviceType is compatible with dlpack DLDeviceType for portability
+// it uses the same values for each enum as the equivalent kDL<type> from dlpack.h
+#ifdef __cplusplus
+typedef enum : int32_t {
+#else
+typedef enum {
+#endif
+  // CPU device, same as using ArrowArray directly
+  kArrowCPU = 1,
+  // CUDA GPU Device
+  kArrowCUDA = 2,
+  // Pinned CUDA CPU memory by cudaMallocHost
+  kArrowCUDAHost = 3,
+  // OpenCL Device
+  kArrowOpenCL = 4,
+  // Vulkan buffer for next-gen graphics
+  kArrowVulkan = 7,
+  // Metal for Apple GPU
+  kArrowMetal = 8,
+  // Verilog simulator buffer
+  kArrowVPI = 9,
+  // ROCm GPUs for AMD GPUs
+  kArrowROCM = 10,
+  // Pinned ROCm CPU memory allocated by hipMallocHost
+  kArrowROCMHost = 11,
+  // Reserved for extension
+  // used to quickly test extension devices,
+  // semantics can differ based on the implementation
+  kArrowExtDev = 12,
+  // CUDA managed/unified memory allocated by cudaMallocManaged
+  kArrowCUDAManaged = 13,
+  // unified shared memory allocated on a oneAPI non-partitioned
+  // device. call to oneAPI runtime is required to determine the
+  // device type, the USM allocation type and the sycl context it
+  // is bound to
+  kArrowOneAPI = 14,
+  // GPU support for next-gen WebGPU standard
+  kArrowWebGPU = 15,
+  // Qualcomm Hexagon DSP
+  kArrowHexagon = 16,
+} ArrowDeviceType;
+
+struct ArrowDeviceArray {
+  // the private_date and release callback of the arrow array
+  // should contain any necessary information and structures
+  // related to freeing the array according to the device it
+  // is allocated on, rather than having a separate release
+  // callback embedded here.
+  struct ArrowArray* array;
+  int device_id;
+  ArrowDeviceType device_type;
+  // reserve 128 bytes for future expansion
+  // of this struct as non-CPU development expands
+  // so that we can update without ABI breaking
+  // changes.

Review Comment:
   That's assuming those reserved bytes are currently forced to zero, right? This should be specified somewhere.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [arrow] lidavidm commented on a diff in pull request #34972: GH-34971: [Format] Enhance C-Data API to support non-cpu cases

["lidavidm (via GitHub)" <gi...@apache.org>]

lidavidm commented on code in PR #34972:
URL: https://github.com/apache/arrow/pull/34972#discussion_r1161260923


##########
cpp/src/arrow/c/abi.h:
##########
@@ -65,6 +65,69 @@ struct ArrowArray {
 
 #endif  // ARROW_C_DATA_INTERFACE
 
+#ifndef ARROW_C_DEVICE_DATA_INTERFACE
+#define ARROW_C_DEVICE_DATA_INTERFACE
+
+// ArrowDeviceType is compatible with dlpack DLDeviceType for portability

Review Comment:
   It would be great to have docstrings for enums, new structs, etc.



##########
cpp/src/arrow/c/abi.h:
##########
@@ -65,6 +65,69 @@ struct ArrowArray {
 
 #endif  // ARROW_C_DATA_INTERFACE
 
+#ifndef ARROW_C_DEVICE_DATA_INTERFACE
+#define ARROW_C_DEVICE_DATA_INTERFACE
+
+// ArrowDeviceType is compatible with dlpack DLDeviceType for portability
+// it uses the same values for each enum as the equivalent kDL<type> from dlpack.h
+#ifdef __cplusplus
+typedef enum : int32_t {
+#else
+typedef enum {
+#endif
+  // CPU device, same as using ArrowArray directly
+  kArrowCPU = 1,

Review Comment:
   Should we explicitly have `0` as the 'unset' value then?



##########
cpp/src/arrow/c/abi.h:
##########
@@ -106,6 +169,77 @@ struct ArrowArrayStream {
 
 #endif  // ARROW_C_STREAM_INTERFACE
 
+#ifndef ARROW_C_DEVICE_STREAM_INTERFACE
+#define ARROW_C_DEVICE_STREAM_INTERFACE
+
+struct ArrowDeviceArrayStream {
+  // The device that this stream produces data on.
+  // All ArrowDeviceArrays that are produced by this
+  // stream should have the same device_type as set
+  // here. The device_type needs to be provided here
+  // so that consumers can provide the correct type
+  // of stream_ptr when calling get_next.
+  ArrowDeviceType device_type;
+
+  // Callback to get the stream schema
+  // (will be the same for all arrays in the stream).
+  //
+  // Return value: 0 if successful, an `errno`-compatible error code otherwise.
+  //
+  // If successful, the ArrowSchema must be released independently from the stream.
+  int (*get_schema)(struct ArrowDeviceArrayStream*, struct ArrowSchema* out);
+
+  // Callback to get the device id for the next array.
+  // This is necessary so that the proper/correct stream pointer can be provided
+  // to get_next. The parameter provided must not be null.
+  //
+  // Return value: 0 if successful, an `errno`-compatible error code otherwise.
+  //
+  // The next call to `get_next` should provide an ArrowDeviceArray whose
+  // device_id matches what is provided here, and whose device_type is the
+  // same as the device_type member of this stream.
+  int (*get_next_device_id)(struct ArrowDeviceArrayStream*, int* out_device_id);
+
+  // Callback to get the next array
+  // (if no error and the array is released, the stream has ended)
+  //
+  // the provided stream_ptr should be the appropriate stream, or

Review Comment:
   to be clear, this is "CUDA stream"?
   
   while this is a draft, the language here is a little imprecise/feels like it assumes too much knowledge of a particular programming model. It would be great to define any terminology needed so that things can reasonably stand alone.



##########
cpp/src/arrow/c/abi.h:
##########
@@ -65,6 +65,69 @@ struct ArrowArray {
 
 #endif  // ARROW_C_DATA_INTERFACE
 
+#ifndef ARROW_C_DEVICE_DATA_INTERFACE
+#define ARROW_C_DEVICE_DATA_INTERFACE
+
+// ArrowDeviceType is compatible with dlpack DLDeviceType for portability
+// it uses the same values for each enum as the equivalent kDL<type> from dlpack.h
+#ifdef __cplusplus
+typedef enum : int32_t {
+#else
+typedef enum {
+#endif

Review Comment:
   Explicit `int32_t` should be safe AFAIK, but I agree it's a little goofy + is inconsistent with existing use of `#define`s



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [arrow] zeroshade commented on a diff in pull request #34972: GH-34971: [Format] Enhance C-Data API to support non-cpu cases

["zeroshade (via GitHub)" <gi...@apache.org>]

zeroshade commented on code in PR #34972:
URL: https://github.com/apache/arrow/pull/34972#discussion_r1161302363


##########
cpp/src/arrow/c/abi.h:
##########
@@ -65,6 +65,69 @@ struct ArrowArray {
 
 #endif  // ARROW_C_DATA_INTERFACE
 
+#ifndef ARROW_C_DEVICE_DATA_INTERFACE
+#define ARROW_C_DEVICE_DATA_INTERFACE
+
+// ArrowDeviceType is compatible with dlpack DLDeviceType for portability
+// it uses the same values for each enum as the equivalent kDL<type> from dlpack.h
+#ifdef __cplusplus
+typedef enum : int32_t {
+#else
+typedef enum {
+#endif
+  // CPU device, same as using ArrowArray directly
+  kArrowCPU = 1,
+  // CUDA GPU Device
+  kArrowCUDA = 2,
+  // Pinned CUDA CPU memory by cudaMallocHost
+  kArrowCUDAHost = 3,
+  // OpenCL Device
+  kArrowOpenCL = 4,
+  // Vulkan buffer for next-gen graphics
+  kArrowVulkan = 7,
+  // Metal for Apple GPU
+  kArrowMetal = 8,
+  // Verilog simulator buffer
+  kArrowVPI = 9,
+  // ROCm GPUs for AMD GPUs
+  kArrowROCM = 10,
+  // Pinned ROCm CPU memory allocated by hipMallocHost
+  kArrowROCMHost = 11,
+  // Reserved for extension
+  // used to quickly test extension devices,
+  // semantics can differ based on the implementation
+  kArrowExtDev = 12,
+  // CUDA managed/unified memory allocated by cudaMallocManaged
+  kArrowCUDAManaged = 13,
+  // unified shared memory allocated on a oneAPI non-partitioned
+  // device. call to oneAPI runtime is required to determine the
+  // device type, the USM allocation type and the sycl context it
+  // is bound to
+  kArrowOneAPI = 14,
+  // GPU support for next-gen WebGPU standard
+  kArrowWebGPU = 15,
+  // Qualcomm Hexagon DSP
+  kArrowHexagon = 16,
+} ArrowDeviceType;
+
+struct ArrowDeviceArray {
+  // the private_date and release callback of the arrow array
+  // should contain any necessary information and structures
+  // related to freeing the array according to the device it
+  // is allocated on, rather than having a separate release
+  // callback embedded here.
+  struct ArrowArray* array;
+  int device_id;

Review Comment:
   I'll future proof and set it to an `int64_t` just in case.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [arrow] zeroshade commented on a diff in pull request #34972: GH-34971: [Format] Add non-CPU version of C Data Interface

["zeroshade (via GitHub)" <gi...@apache.org>]

zeroshade commented on code in PR #34972:
URL: https://github.com/apache/arrow/pull/34972#discussion_r1219872402


##########
docs/source/format/CDeviceDataInterface.rst:
##########
@@ -0,0 +1,688 @@
+.. Licensed to the Apache Software Foundation (ASF) under one
+.. or more contributor license agreements.  See the NOTICE file
+.. distributed with this work for additional information
+.. regarding copyright ownership.  The ASF licenses this file
+.. to you under the Apache License, Version 2.0 (the
+.. "License"); you may not use this file except in compliance
+.. with the License.  You may obtain a copy of the License at
+
+..   http://www.apache.org/licenses/LICENSE-2.0
+
+.. Unless required by applicable law or agreed to in writing,
+.. software distributed under the License is distributed on an
+.. "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+.. KIND, either express or implied.  See the License for the
+.. specific language governing permissions and limitations
+.. under the License.
+
+.. highlight:: c
+
+.. _c-device-data-interface:
+
+=================================
+The Arrow C Device data interface
+=================================
+
+.. warning:: The Arrow C Device Data Interface should be considered experimental
+
+Rationale
+=========
+
+The current :ref:`C Data Interface <c-data-interface>`, and most
+implementations of it, make the assumption that all data buffers provided
+are CPU buffers. Since Apache Arrow is designed to be a universal in-memory
+format for representing tabular ("columnar") data, there will be the desire
+to leverage this data on non-CPU hardware such as GPUs. One example of such
+a case is the `RAPIDS cuDF library`_ which uses the Arrow memory format with
+CUDA for NVIDIA GPUs. Since copying data from host to device and back is
+expensive, the ideal would be to be able to leave the data on the device
+for as long as possible, even when passing it between runtimes and
+libraries.
+
+The Arrow C Device data interface builds on the existing C data interface
+by adding a very small, stable set of C definitions to it. These definitions
+are equivalents to the ``ArrowArray`` and ``ArrowArrayStream`` structures
+from the C Data Interface which add members to allow specifying the device
+type and pass necessary information to synchronize with the producer.
+For non-C/C++ languages and runtimes, translating the C definitions to
+corresponding C FFI declarations should be just as simple as with the
+current C data interface.
+
+Applications and libraries can then use Arrow schemas and Arrow formatted
+memory on non-CPU devices to exchange data just as easily as they do
+now with CPU data. This will enable leaving data on those devices longer
+and avoiding costly copies back and forth between the host and device
+just to leverage new libraries and runtimes.
+
+Goals
+-----
+
+* Expose an ABI-stable interface built on the existing C data interface.
+* Make it easy for third-party projects to implement support with little
+  initial investment.
+* Allow zero-copy sharing of Arrow formatted device memory between
+  independant runtimes and components running in the same process.
+* Avoid the need for one-to-one adaptation layers such as the
+  `CUDA Array Interface`_ for Python processes to pass CUDA data.
+* Enable integration without explicit dependencies (either at compile-time
+  or runtime) on the Arrow software project itself.
+
+The intent is for the Arrow C Device data interface to expand the reach
+of the current C data interface, allowing it to also become the standard
+low-level building block for columnar processing on devices like GPUs or
+FPGAs.
+
+Structure definitions
+=====================
+
+Because this is built on the C data interface, the C Device data interface
+uses the ``ArrowSchema`` and ``ArrowArray`` structures as defined in the
+:ref:`C data interface spec <c-data-interface-struct-defs>`. It then adds the
+following free-standing definitions. Like the rest of the Arrow project,
+they are available under the Apache License 2.0.
+
+.. code-block:: c
+
+    #ifndef ARROW_C_DEVICE_DATA_INTERFACE
+    #define ARROW_C_DEVICE_DATA_INTERFACE
+
+    // Device type for the allocated memory
+    typedef int32_t ArrowDeviceType;
+
+    // CPU device, same as using ArrowArray directly
+    #define ARROW_DEVICE_CPU 1
+    // CUDA GPU Device
+    #define ARROW_DEVICE_CUDA 2
+    // Pinned CUDA CPU memory by cudaMallocHost
+    #define ARROW_DEVICE_CUDA_HOST 3
+    // OpenCL Device
+    #define ARROW_DEVICE_OPENCL 4
+    // Vulkan buffer for next-gen graphics
+    #define ARROW_DEVICE_VULKAN 7
+    // Metal for Apple GPU
+    #define ARROW_DEVICE_METAL 8
+    // Verilog simulator buffer
+    #define ARROW_DEVICE_VPI 9
+    // ROCm GPUs for AMD GPUs
+    #define ARROW_DEVICE_ROCM 10
+    // Pinned ROCm CPU memory allocated by hipMallocHost
+    #define ARROW_DEVICE_ROCM_HOST 11
+    // Reserved for extension
+    //
+    // used to quickly test extension devices, semantics
+    // can differ based on implementation
+    #define ARROW_DEVICE_EXT_DEV 12
+    // CUDA managed/unified memory allocated by cudaMallocManaged
+    #define ARROW_DEVICE_CUDA_MANAGED 13
+    // Unified shared memory allocated on a oneAPI
+    // non-partitioned device.
+    //
+    // A call to the oneAPI runtime is required to determine the
+    // device type, the USM allocation type and the sycl context
+    // that it is bound to.
+    #define ARROW_DEVICE_ONEAPI 14
+    // GPU support for next-gen WebGPU standard
+    #define ARROW_DEVICE_WEBGPU 15
+    // Qualcomm Hexagon DSP
+    #define ARROW_DEVICE_HEXAGON 16
+
+    struct ArrowDeviceArray {
+      struct ArrowArray array;
+      int64_t device_id;
+      ArrowDeviceType device_type;
+      void* sync_event;
+
+      // reserved bytes for future expansion
+      int64_t reserved[3];
+    };
+
+    #endif  // ARROW_C_DEVICE_DATA_INTERFACE
+
+.. note::
+   The canonical guard ``ARROW_C_DEVICE_DATA_INTERFACE`` is meant to avoid
+   duplicate definitions if two projects copy the definitions in their own
+   headers, and a third-party project includes from these two projects. It
+   is therefore important that this guard is kept exactly as-is when these
+   definitions are copied.
+
+ArrowDeviceType
+---------------
+
+The ``ArrowDeviceType`` typedef is used to indicate what type of device the
+provided memory buffers were allocated on. This, in conjunction with the
+``device_id``, should be sufficient to reference the correct data buffers.
+
+We then use macros to define values for different device types. The provided
+macro values are compatible with the widely used `dlpack`_ ``DLDeviceType``
+definition values, using the same value for each as the equivalent
+``kDL<type>`` enum from dlpack.h. The list will be kept in sync with those
+equivalent enum values over time to ensure compatibility, rather than
+potentially diverging. To avoid the Arrow project having to be in the
+position of vetting new hardware devices, new additions should first be
+added to dlpack before we add a corresponding macro here.
+
+To ensure predictability with the ABI, we use macros instead of an ``enum``
+so the storage type is not compiler dependent.
+
+.. c:macro:: ARROW_DEVICE_CPU
+
+    CPU Device, equivalent to just using ``ArrowArray`` directly instead of
+    using ``ArrowDeviceArray``.
+
+.. c:macro:: ARROW_DEVICE_CUDA
+
+    A `CUDA`_ GPU Device. This could represent data allocated either with the
+    runtime library (``cudaMalloc``) or the device driver (``cuMemAlloc``).
+
+.. c:macro:: ARROW_DEVICE_CUDA_HOST
+
+    CPU memory that was pinned and page-locked by CUDA by using
+    ``cudaMallocHost`` or ``cuMemAllocHost``.
+
+.. c:macro:: ARROW_DEVICE_OPENCL
+
+    Data allocated on the device by using the `OpenCL (Open Computing Language)`_
+    framework.
+
+.. c:macro:: ARROW_DEVICE_VULKAN
+
+    Data allocated by the `Vulkan`_ framework and libraries.
+
+.. c:macro:: ARROW_DEVICE_METAL
+
+    Data on Apple GPU devices using the `Metal`_ framework and libraries.
+
+.. c:macro:: ARROW_DEVICE_VPI
+
+    Indicates usage of a Verilog simulator buffer.
+
+.. c:macro:: ARROW_DEVICE_ROCM
+
+    An AMD device using the `ROCm`_ stack.
+
+.. c:macro:: ARROW_DEVICE_ROCM_HOST
+
+    CPU memory pinned and page-locked allocated ``hipMallocHost``.
+
+.. c:macro:: ARROW_DEVICE_EXT_DEV
+
+    This value is an escape-hatch for devices to extend which aren't
+    currently represented otherwise. Producers would need to provide
+    additional information/context specific to the device if using
+    this device type. This is used to quickly test extension devices
+    and semantics can differ based on the implementation.
+
+.. c:macro:: ARROW_DEVICE_CUDA_MANAGED
+
+    CUDA managed/unified memory which is allocated by ``cudaMallocManaged``.
+
+.. c:macro:: ARROW_DEVICE_ONEAPI
+
+    Unified shared memory allocated on an Intel `oneAPI`_ non-partitioned
+    device. A call to the ``oneAPI`` runtime is required to determine
+    the specific device type, the USM allocation type and the sycl context
+    that it is bound to.
+
+.. c:macro:: ARROW_DEVICE_WEBGPU
+
+    GPU support for next-gen WebGPU standards
+
+.. c:macro:: ARROW_DEVICE_HEXAGON
+
+    Data allocated on a Qualcomm Hexagon DSP device.
+
+The ArrowDeviceArray structure
+------------------------------
+
+The ``ArrowDeviceArray`` structure embeds the C data ``ArrowArray`` structure
+and adds additional information necessary for consumers to use the data. It
+has the following fields:
+
+.. c:member:: struct ArrowArray ArrowDeviceArray.array
+
+    The allocated array data. The values in the ``void**`` buffers (along
+    with the buffers of any children) are what is allocated on the device.
+    The buffer values should be device pointers. The rest of the structure
+    should be accessible to the CPU.
+
+    The ``private_data`` and ``release`` callback of this structure should
+    contain any necessary information and structures related to freeing
+    the array according to the device it is allocated on, rather than
+    having a separate release callback and ``private_data`` pointer here.
+
+.. c:member:: int64_t ArrowDeviceArray.device_id
+
+    The device id to identify a specific device if multiple devices of this
+    type are on the system. The semantics of the id will be hardware dependent,
+    but we use an ``int64_t`` to future-proof the id as devices change over time.
+
+.. c:member:: ArrowDeviceType ArrowDeviceArray.device_type
+
+    The type of the device which can access the buffers in the array.
+
+.. c:member:: void* ArrowDeviceArray.sync_event
+
+    Optional. An event-like object to synchronize on if needed.
+
+    Many devices, like GPUs, are primarily asynchronous with respect to
+    CPU processing. As such in order to safely access memory, it is often
+    necessary to have an object to synchronize processing with. Since
+    different devices will use different types to specify this, we use a
+    void* which can be coerced into a pointer to whatever the device
+    appropriate type is.
+
+    If synchronization is not needed, this can be null. If this is non-null
+    then it MUST be used to call the appropriate sync method for the device
+    (e.g. ``cudaStreamWaitEvent`` or ``hipStreamWaitEvent``) before attempting
+    to access the memory in the buffers.
+
+    Expected types to coerce this ``void*`` to depending on the reported
+    device type:
+
+    * CUDA: ``cudaEvent_t*``
+    * ROCm: ``hipEvent_t*``
+    * OpenCL: ``cl_event*``
+    * Vulkan: ``VkEvent*``
+    * Metal: ``MTLEvent*``
+    * OneAPI: ``sycl::event*``
+
+    If an event is provided, then the producer MUST ensure that the exported
+    data is available on the device before the event is triggered. The 
+    consumer SHOULD wait on the event before trying to access the exported
+    data.
+
+.. c:member:: int64_t ArrowDeviceArray.reserved[3]
+
+    As non-CPU development expands, there may be a need to expand this
+    structure. In order to do so without potentially breaking ABI changes,
+    we reserve 24 bytes at the end of the object. This also has the added
+    benefit of bringing the total size of this structure to exactly 128
+    bytes (a power of 2) on 64-bit systems. These bytes MUST be zero'd
+    out after initialization by the producer in order to ensure safe 
+    evolution of the ABI in the future.
+
+.. note::
+    Rather than store the shape / types of the data alongside the
+    ``ArrowDeviceArray``, users should utilize the existing ``ArrowSchema``
+    structure to pass any data type and shape information.
+
+Synchronization event types
+---------------------------
+
+The table below lists the expected event types for each device type.
+If no event type is supported ("N/A"), then the ``sync_event`` member
+should always be null. 
+
+Remember that the event *CAN* be null if synchronization is not needed
+to access the data.
+
++---------------------------+--------------------+---------+
+| Device Type               | Actual Event Type  | Notes   |
++===========================+====================+=========+
+| ARROW_DEVICE_CPU          | N/A                |         |
++---------------------------+--------------------+---------+
+| ARROW_DEVICE_CUDA         | ``cudaEvent_t*``   |         |
++---------------------------+--------------------+---------+
+| ARROW_DEVICE_CUDA_HOST    | ``cudaEvent_t*``   |         |
++---------------------------+--------------------+---------+
+| ARROW_DEVICE_OPENCL       | ``cl_event*``      |         |
++---------------------------+--------------------+---------+
+| ARROW_DEVICE_VULKAN       | ``VkEvent*``       |         |
++---------------------------+--------------------+---------+
+| ARROW_DEVICE_METAL        | ``MTLEvent*``      |         |
++---------------------------+--------------------+---------+
+| ARROW_DEVICE_VPI          | N/A                | (1)     |
++---------------------------+--------------------+---------+
+| ARROW_DEVICE_ROCM         | ``hipEvent_t*``    |         |
++---------------------------+--------------------+---------+
+| ARROW_DEVICE_ROCM_HOST    | ``hipEvent_t*``    |         |
++---------------------------+--------------------+---------+
+| ARROW_DEVICE_EXT_DEV      |                    | (2)     |
++---------------------------+--------------------+---------+
+| ARROW_DEVICE_CUDA_MANAGED | ``cudaEvent_t*``   |         |
++---------------------------+--------------------+---------+
+| ARROW_DEVICE_ONEAPI       | ``sycl::event*``   |         |
++---------------------------+--------------------+---------+
+| ARROW_DEVICE_WEBGPU       | N/A                | (1)     |
++---------------------------+--------------------+---------+
+| ARROW_DEVICE_HEXAGON      | N/A                | (1)     |
++---------------------------+--------------------+---------+
+
+Notes:
+
+* \(1) Currently unknown if framework has an event type to support.
+* \(2) Extension Device has producer defined semantics and thus if
+       synchronization is needed for an extension device, the producer
+       should document the type.
+
+
+Semantics
+=========

Review Comment:
   added at the top of the `Memory Management` section.



##########
docs/source/format/CDeviceDataInterface.rst:
##########
@@ -0,0 +1,688 @@
+.. Licensed to the Apache Software Foundation (ASF) under one
+.. or more contributor license agreements.  See the NOTICE file
+.. distributed with this work for additional information
+.. regarding copyright ownership.  The ASF licenses this file
+.. to you under the Apache License, Version 2.0 (the
+.. "License"); you may not use this file except in compliance
+.. with the License.  You may obtain a copy of the License at
+
+..   http://www.apache.org/licenses/LICENSE-2.0
+
+.. Unless required by applicable law or agreed to in writing,
+.. software distributed under the License is distributed on an
+.. "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+.. KIND, either express or implied.  See the License for the
+.. specific language governing permissions and limitations
+.. under the License.
+
+.. highlight:: c
+
+.. _c-device-data-interface:
+
+=================================
+The Arrow C Device data interface
+=================================
+
+.. warning:: The Arrow C Device Data Interface should be considered experimental
+
+Rationale
+=========
+
+The current :ref:`C Data Interface <c-data-interface>`, and most
+implementations of it, make the assumption that all data buffers provided
+are CPU buffers. Since Apache Arrow is designed to be a universal in-memory
+format for representing tabular ("columnar") data, there will be the desire
+to leverage this data on non-CPU hardware such as GPUs. One example of such
+a case is the `RAPIDS cuDF library`_ which uses the Arrow memory format with
+CUDA for NVIDIA GPUs. Since copying data from host to device and back is
+expensive, the ideal would be to be able to leave the data on the device
+for as long as possible, even when passing it between runtimes and
+libraries.
+
+The Arrow C Device data interface builds on the existing C data interface
+by adding a very small, stable set of C definitions to it. These definitions
+are equivalents to the ``ArrowArray`` and ``ArrowArrayStream`` structures
+from the C Data Interface which add members to allow specifying the device
+type and pass necessary information to synchronize with the producer.
+For non-C/C++ languages and runtimes, translating the C definitions to
+corresponding C FFI declarations should be just as simple as with the
+current C data interface.
+
+Applications and libraries can then use Arrow schemas and Arrow formatted
+memory on non-CPU devices to exchange data just as easily as they do
+now with CPU data. This will enable leaving data on those devices longer
+and avoiding costly copies back and forth between the host and device
+just to leverage new libraries and runtimes.
+
+Goals
+-----
+
+* Expose an ABI-stable interface built on the existing C data interface.
+* Make it easy for third-party projects to implement support with little
+  initial investment.
+* Allow zero-copy sharing of Arrow formatted device memory between
+  independant runtimes and components running in the same process.
+* Avoid the need for one-to-one adaptation layers such as the
+  `CUDA Array Interface`_ for Python processes to pass CUDA data.
+* Enable integration without explicit dependencies (either at compile-time
+  or runtime) on the Arrow software project itself.
+
+The intent is for the Arrow C Device data interface to expand the reach
+of the current C data interface, allowing it to also become the standard
+low-level building block for columnar processing on devices like GPUs or
+FPGAs.
+
+Structure definitions
+=====================
+
+Because this is built on the C data interface, the C Device data interface
+uses the ``ArrowSchema`` and ``ArrowArray`` structures as defined in the
+:ref:`C data interface spec <c-data-interface-struct-defs>`. It then adds the
+following free-standing definitions. Like the rest of the Arrow project,
+they are available under the Apache License 2.0.
+
+.. code-block:: c
+
+    #ifndef ARROW_C_DEVICE_DATA_INTERFACE
+    #define ARROW_C_DEVICE_DATA_INTERFACE
+
+    // Device type for the allocated memory
+    typedef int32_t ArrowDeviceType;
+
+    // CPU device, same as using ArrowArray directly
+    #define ARROW_DEVICE_CPU 1
+    // CUDA GPU Device
+    #define ARROW_DEVICE_CUDA 2
+    // Pinned CUDA CPU memory by cudaMallocHost
+    #define ARROW_DEVICE_CUDA_HOST 3
+    // OpenCL Device
+    #define ARROW_DEVICE_OPENCL 4
+    // Vulkan buffer for next-gen graphics
+    #define ARROW_DEVICE_VULKAN 7
+    // Metal for Apple GPU
+    #define ARROW_DEVICE_METAL 8
+    // Verilog simulator buffer
+    #define ARROW_DEVICE_VPI 9
+    // ROCm GPUs for AMD GPUs
+    #define ARROW_DEVICE_ROCM 10
+    // Pinned ROCm CPU memory allocated by hipMallocHost
+    #define ARROW_DEVICE_ROCM_HOST 11
+    // Reserved for extension
+    //
+    // used to quickly test extension devices, semantics
+    // can differ based on implementation
+    #define ARROW_DEVICE_EXT_DEV 12
+    // CUDA managed/unified memory allocated by cudaMallocManaged
+    #define ARROW_DEVICE_CUDA_MANAGED 13
+    // Unified shared memory allocated on a oneAPI
+    // non-partitioned device.
+    //
+    // A call to the oneAPI runtime is required to determine the
+    // device type, the USM allocation type and the sycl context
+    // that it is bound to.
+    #define ARROW_DEVICE_ONEAPI 14
+    // GPU support for next-gen WebGPU standard
+    #define ARROW_DEVICE_WEBGPU 15
+    // Qualcomm Hexagon DSP
+    #define ARROW_DEVICE_HEXAGON 16
+
+    struct ArrowDeviceArray {
+      struct ArrowArray array;
+      int64_t device_id;
+      ArrowDeviceType device_type;
+      void* sync_event;
+
+      // reserved bytes for future expansion
+      int64_t reserved[3];
+    };
+
+    #endif  // ARROW_C_DEVICE_DATA_INTERFACE
+
+.. note::
+   The canonical guard ``ARROW_C_DEVICE_DATA_INTERFACE`` is meant to avoid
+   duplicate definitions if two projects copy the definitions in their own
+   headers, and a third-party project includes from these two projects. It
+   is therefore important that this guard is kept exactly as-is when these
+   definitions are copied.
+
+ArrowDeviceType
+---------------
+
+The ``ArrowDeviceType`` typedef is used to indicate what type of device the
+provided memory buffers were allocated on. This, in conjunction with the
+``device_id``, should be sufficient to reference the correct data buffers.
+
+We then use macros to define values for different device types. The provided
+macro values are compatible with the widely used `dlpack`_ ``DLDeviceType``
+definition values, using the same value for each as the equivalent
+``kDL<type>`` enum from dlpack.h. The list will be kept in sync with those
+equivalent enum values over time to ensure compatibility, rather than
+potentially diverging. To avoid the Arrow project having to be in the
+position of vetting new hardware devices, new additions should first be
+added to dlpack before we add a corresponding macro here.
+
+To ensure predictability with the ABI, we use macros instead of an ``enum``
+so the storage type is not compiler dependent.
+
+.. c:macro:: ARROW_DEVICE_CPU
+
+    CPU Device, equivalent to just using ``ArrowArray`` directly instead of
+    using ``ArrowDeviceArray``.
+
+.. c:macro:: ARROW_DEVICE_CUDA
+
+    A `CUDA`_ GPU Device. This could represent data allocated either with the
+    runtime library (``cudaMalloc``) or the device driver (``cuMemAlloc``).
+
+.. c:macro:: ARROW_DEVICE_CUDA_HOST
+
+    CPU memory that was pinned and page-locked by CUDA by using
+    ``cudaMallocHost`` or ``cuMemAllocHost``.
+
+.. c:macro:: ARROW_DEVICE_OPENCL
+
+    Data allocated on the device by using the `OpenCL (Open Computing Language)`_
+    framework.
+
+.. c:macro:: ARROW_DEVICE_VULKAN
+
+    Data allocated by the `Vulkan`_ framework and libraries.
+
+.. c:macro:: ARROW_DEVICE_METAL
+
+    Data on Apple GPU devices using the `Metal`_ framework and libraries.
+
+.. c:macro:: ARROW_DEVICE_VPI
+
+    Indicates usage of a Verilog simulator buffer.
+
+.. c:macro:: ARROW_DEVICE_ROCM
+
+    An AMD device using the `ROCm`_ stack.
+
+.. c:macro:: ARROW_DEVICE_ROCM_HOST
+
+    CPU memory pinned and page-locked allocated ``hipMallocHost``.
+
+.. c:macro:: ARROW_DEVICE_EXT_DEV
+
+    This value is an escape-hatch for devices to extend which aren't
+    currently represented otherwise. Producers would need to provide
+    additional information/context specific to the device if using
+    this device type. This is used to quickly test extension devices
+    and semantics can differ based on the implementation.
+
+.. c:macro:: ARROW_DEVICE_CUDA_MANAGED
+
+    CUDA managed/unified memory which is allocated by ``cudaMallocManaged``.
+
+.. c:macro:: ARROW_DEVICE_ONEAPI
+
+    Unified shared memory allocated on an Intel `oneAPI`_ non-partitioned
+    device. A call to the ``oneAPI`` runtime is required to determine
+    the specific device type, the USM allocation type and the sycl context
+    that it is bound to.
+
+.. c:macro:: ARROW_DEVICE_WEBGPU
+
+    GPU support for next-gen WebGPU standards
+
+.. c:macro:: ARROW_DEVICE_HEXAGON
+
+    Data allocated on a Qualcomm Hexagon DSP device.
+
+The ArrowDeviceArray structure
+------------------------------
+
+The ``ArrowDeviceArray`` structure embeds the C data ``ArrowArray`` structure
+and adds additional information necessary for consumers to use the data. It
+has the following fields:
+
+.. c:member:: struct ArrowArray ArrowDeviceArray.array
+
+    The allocated array data. The values in the ``void**`` buffers (along
+    with the buffers of any children) are what is allocated on the device.
+    The buffer values should be device pointers. The rest of the structure
+    should be accessible to the CPU.
+
+    The ``private_data`` and ``release`` callback of this structure should
+    contain any necessary information and structures related to freeing
+    the array according to the device it is allocated on, rather than
+    having a separate release callback and ``private_data`` pointer here.
+
+.. c:member:: int64_t ArrowDeviceArray.device_id
+
+    The device id to identify a specific device if multiple devices of this
+    type are on the system. The semantics of the id will be hardware dependent,
+    but we use an ``int64_t`` to future-proof the id as devices change over time.
+
+.. c:member:: ArrowDeviceType ArrowDeviceArray.device_type
+
+    The type of the device which can access the buffers in the array.
+
+.. c:member:: void* ArrowDeviceArray.sync_event
+
+    Optional. An event-like object to synchronize on if needed.
+
+    Many devices, like GPUs, are primarily asynchronous with respect to
+    CPU processing. As such in order to safely access memory, it is often
+    necessary to have an object to synchronize processing with. Since
+    different devices will use different types to specify this, we use a
+    void* which can be coerced into a pointer to whatever the device
+    appropriate type is.
+
+    If synchronization is not needed, this can be null. If this is non-null
+    then it MUST be used to call the appropriate sync method for the device
+    (e.g. ``cudaStreamWaitEvent`` or ``hipStreamWaitEvent``) before attempting
+    to access the memory in the buffers.
+
+    Expected types to coerce this ``void*`` to depending on the reported
+    device type:
+
+    * CUDA: ``cudaEvent_t*``
+    * ROCm: ``hipEvent_t*``
+    * OpenCL: ``cl_event*``
+    * Vulkan: ``VkEvent*``
+    * Metal: ``MTLEvent*``
+    * OneAPI: ``sycl::event*``
+
+    If an event is provided, then the producer MUST ensure that the exported
+    data is available on the device before the event is triggered. The 
+    consumer SHOULD wait on the event before trying to access the exported
+    data.
+
+.. c:member:: int64_t ArrowDeviceArray.reserved[3]
+
+    As non-CPU development expands, there may be a need to expand this
+    structure. In order to do so without potentially breaking ABI changes,
+    we reserve 24 bytes at the end of the object. This also has the added
+    benefit of bringing the total size of this structure to exactly 128
+    bytes (a power of 2) on 64-bit systems. These bytes MUST be zero'd
+    out after initialization by the producer in order to ensure safe 
+    evolution of the ABI in the future.
+
+.. note::
+    Rather than store the shape / types of the data alongside the
+    ``ArrowDeviceArray``, users should utilize the existing ``ArrowSchema``
+    structure to pass any data type and shape information.
+
+Synchronization event types
+---------------------------
+
+The table below lists the expected event types for each device type.
+If no event type is supported ("N/A"), then the ``sync_event`` member
+should always be null. 
+
+Remember that the event *CAN* be null if synchronization is not needed
+to access the data.
+
++---------------------------+--------------------+---------+
+| Device Type               | Actual Event Type  | Notes   |
++===========================+====================+=========+
+| ARROW_DEVICE_CPU          | N/A                |         |
++---------------------------+--------------------+---------+
+| ARROW_DEVICE_CUDA         | ``cudaEvent_t*``   |         |
++---------------------------+--------------------+---------+
+| ARROW_DEVICE_CUDA_HOST    | ``cudaEvent_t*``   |         |
++---------------------------+--------------------+---------+
+| ARROW_DEVICE_OPENCL       | ``cl_event*``      |         |
++---------------------------+--------------------+---------+
+| ARROW_DEVICE_VULKAN       | ``VkEvent*``       |         |
++---------------------------+--------------------+---------+
+| ARROW_DEVICE_METAL        | ``MTLEvent*``      |         |
++---------------------------+--------------------+---------+
+| ARROW_DEVICE_VPI          | N/A                | (1)     |
++---------------------------+--------------------+---------+
+| ARROW_DEVICE_ROCM         | ``hipEvent_t*``    |         |
++---------------------------+--------------------+---------+
+| ARROW_DEVICE_ROCM_HOST    | ``hipEvent_t*``    |         |
++---------------------------+--------------------+---------+
+| ARROW_DEVICE_EXT_DEV      |                    | (2)     |
++---------------------------+--------------------+---------+
+| ARROW_DEVICE_CUDA_MANAGED | ``cudaEvent_t*``   |         |
++---------------------------+--------------------+---------+
+| ARROW_DEVICE_ONEAPI       | ``sycl::event*``   |         |
++---------------------------+--------------------+---------+
+| ARROW_DEVICE_WEBGPU       | N/A                | (1)     |
++---------------------------+--------------------+---------+
+| ARROW_DEVICE_HEXAGON      | N/A                | (1)     |
++---------------------------+--------------------+---------+
+
+Notes:
+
+* \(1) Currently unknown if framework has an event type to support.
+* \(2) Extension Device has producer defined semantics and thus if
+       synchronization is needed for an extension device, the producer
+       should document the type.
+
+
+Semantics
+=========
+
+Memory management
+-----------------
+
+The ``ArrowDeviceArray`` structure contains an ``ArrowArray`` object which
+itself has :ref:`specific semantics <c-data-interface-semantics>` for releasing
+memory. The term *"base structure"* below refers to the ``ArrowDeviceArray``
+object that is passed directly between the producer and consumer -- not any
+child structure thereof.
+
+It is intended for the base structure to be stack- or heap-allocated by the
+*consumer*. In this case, the producer API should take a pointer to the
+consumer-allocated structure.
+
+However, any data pointed to by the struct MUST be allocated and maintained
+by the producer. This includes the ``sync_event`` member if it is not null,
+along with any pointers in the ``ArrowArray`` object as usual. Data lifetime
+is managed through the ``release`` callback of the ``ArrowArray`` member.
+
+For an ``ArrowDeviceArray``, the semantics of a released structure and the
+callback semantics are identical to those for
+:ref:`ArrowArray itself <c-data-interface-released>`. Any producer specific context
+information necessary for releasing the device data buffers, in addition to
+any allocated event, should be stored in the ``private_data`` member of
+the ``ArrowArray`` and managed by the ``release`` callback.
+
+Moving an array
+'''''''''''''''
+
+The consumer can *move* the ``ArrowDeviceArray`` structure by bitwise copying
+or shallow member-wise copying. Then it MUST mark the source structure released
+by setting the ``release`` member of the embedded ``ArrowArray`` structure to
+``NULL``, but *without* calling that release callback. This ensures that only
+one live copy of the struct is active at any given time and that lifetime is
+correctly communicated to the producer.
+
+As usual, the release callback will be called on the destination structure
+when it is not needed anymore.
+
+Record batches
+--------------
+As with the C data interface itself, a record batch can be trivially considered
+as an equivalent struct array. In this case the metadata of the top-level
+``ArrowSchema`` can be used for schema-level metadata of the record batch.
+
+Mutability
+----------
+
+Both the producer and the consumer SHOULD consider the exported data (that
+is, the data reachable on the device through the ``buffers`` member of
+the embedded ``ArrowArray``) to be immutable, as either party could otherwise
+see inconsistent data while the other is mutating it.
+
+Synchronization
+---------------
+
+If the ``sync_event`` member is non-NULL, the consumer should not attempt 
+to access or read the data until they have synchronized on that event. If
+the ``sync_event`` member is NULL, then it MUST be safe to access the data
+without any synchronization necessary on the part of the consumer.
+
+C producer example
+====================
+
+Exporting a simple ``int32`` device array
+-----------------------------------------
+
+Export a non-nullable ``int32`` type with empty metadata. An example of this
+can be seen in the :ref:`C data interface docs directly <c-data-interface-export-int32-schema>`.
+
+To export the data itself, we transfer ownership to the consumer through
+the release callback. This example will use CUDA, but the equivalent calls
+could be used for any device:
+
+.. code-block:: c
+
+    static void release_int32_device_array(struct ArrowArray* array) {
+        assert(array->n_buffers == 2);
+        // destroy the event
+        cudaEvent_t* ev_ptr = (cudaEvent_t*)(array->private_data);
+        cudaError_t status = cudaEventDestroy(*ev_ptr);
+        assert(status == cudaSuccess);
+        free(ev_ptr);
+
+        // free the buffers and the buffers array
+        status = cudaFree(array->buffers[1]);
+        assert(status == cudaSuccess);
+        free(array->buffers);
+
+        // mark released
+        array->release = NULL;
+    }
+
+    void export_int32_device_array(void* cudaAllocdPtr,
+                                   cudaStream_t stream,
+                                   int64_t length,
+                                   struct ArrowDeviceArray* array) {
+        // get device id
+        int device;
+        cudaError_t status;
+        status = cudaGetDevice(&device);
+        assert(status == cudaSuccess);
+
+        cudaEvent_t* ev_ptr = (cudaEvent_t*)malloc(sizeof(cudaEvent_t));
+        assert(ev_ptr != NULL);
+        status = cudaEventCreate(ev_ptr);
+        assert(status == cudaSuccess);
+
+        // record event on the stream, assuming that the passed in
+        // stream is where the work to produce the data will be processing.
+        status = cudaEventRecord(*ev_ptr, stream);
+        assert(status == cudaSuccess);
+
+        // initialize fields
+        *array = (struct ArrowDeviceArray) {

Review Comment:
   added



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [arrow] zeroshade commented on a diff in pull request #34972: GH-34971: [Format] Add non-CPU version of C Data Interface

["zeroshade (via GitHub)" <gi...@apache.org>]

zeroshade commented on code in PR #34972:
URL: https://github.com/apache/arrow/pull/34972#discussion_r1219872783


##########
docs/source/format/CDeviceDataInterface.rst:
##########
@@ -0,0 +1,688 @@
+.. Licensed to the Apache Software Foundation (ASF) under one
+.. or more contributor license agreements.  See the NOTICE file
+.. distributed with this work for additional information
+.. regarding copyright ownership.  The ASF licenses this file
+.. to you under the Apache License, Version 2.0 (the
+.. "License"); you may not use this file except in compliance
+.. with the License.  You may obtain a copy of the License at
+
+..   http://www.apache.org/licenses/LICENSE-2.0
+
+.. Unless required by applicable law or agreed to in writing,
+.. software distributed under the License is distributed on an
+.. "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+.. KIND, either express or implied.  See the License for the
+.. specific language governing permissions and limitations
+.. under the License.
+
+.. highlight:: c
+
+.. _c-device-data-interface:
+
+=================================
+The Arrow C Device data interface
+=================================
+
+.. warning:: The Arrow C Device Data Interface should be considered experimental
+
+Rationale
+=========
+
+The current :ref:`C Data Interface <c-data-interface>`, and most
+implementations of it, make the assumption that all data buffers provided
+are CPU buffers. Since Apache Arrow is designed to be a universal in-memory
+format for representing tabular ("columnar") data, there will be the desire
+to leverage this data on non-CPU hardware such as GPUs. One example of such
+a case is the `RAPIDS cuDF library`_ which uses the Arrow memory format with
+CUDA for NVIDIA GPUs. Since copying data from host to device and back is
+expensive, the ideal would be to be able to leave the data on the device
+for as long as possible, even when passing it between runtimes and
+libraries.
+
+The Arrow C Device data interface builds on the existing C data interface
+by adding a very small, stable set of C definitions to it. These definitions
+are equivalents to the ``ArrowArray`` and ``ArrowArrayStream`` structures
+from the C Data Interface which add members to allow specifying the device
+type and pass necessary information to synchronize with the producer.
+For non-C/C++ languages and runtimes, translating the C definitions to
+corresponding C FFI declarations should be just as simple as with the
+current C data interface.
+
+Applications and libraries can then use Arrow schemas and Arrow formatted
+memory on non-CPU devices to exchange data just as easily as they do
+now with CPU data. This will enable leaving data on those devices longer
+and avoiding costly copies back and forth between the host and device
+just to leverage new libraries and runtimes.
+
+Goals
+-----
+
+* Expose an ABI-stable interface built on the existing C data interface.
+* Make it easy for third-party projects to implement support with little
+  initial investment.
+* Allow zero-copy sharing of Arrow formatted device memory between
+  independant runtimes and components running in the same process.
+* Avoid the need for one-to-one adaptation layers such as the
+  `CUDA Array Interface`_ for Python processes to pass CUDA data.
+* Enable integration without explicit dependencies (either at compile-time
+  or runtime) on the Arrow software project itself.
+
+The intent is for the Arrow C Device data interface to expand the reach
+of the current C data interface, allowing it to also become the standard
+low-level building block for columnar processing on devices like GPUs or
+FPGAs.
+
+Structure definitions
+=====================
+
+Because this is built on the C data interface, the C Device data interface
+uses the ``ArrowSchema`` and ``ArrowArray`` structures as defined in the
+:ref:`C data interface spec <c-data-interface-struct-defs>`. It then adds the
+following free-standing definitions. Like the rest of the Arrow project,
+they are available under the Apache License 2.0.
+
+.. code-block:: c
+
+    #ifndef ARROW_C_DEVICE_DATA_INTERFACE
+    #define ARROW_C_DEVICE_DATA_INTERFACE
+
+    // Device type for the allocated memory
+    typedef int32_t ArrowDeviceType;
+
+    // CPU device, same as using ArrowArray directly
+    #define ARROW_DEVICE_CPU 1
+    // CUDA GPU Device
+    #define ARROW_DEVICE_CUDA 2
+    // Pinned CUDA CPU memory by cudaMallocHost
+    #define ARROW_DEVICE_CUDA_HOST 3
+    // OpenCL Device
+    #define ARROW_DEVICE_OPENCL 4
+    // Vulkan buffer for next-gen graphics
+    #define ARROW_DEVICE_VULKAN 7
+    // Metal for Apple GPU
+    #define ARROW_DEVICE_METAL 8
+    // Verilog simulator buffer
+    #define ARROW_DEVICE_VPI 9
+    // ROCm GPUs for AMD GPUs
+    #define ARROW_DEVICE_ROCM 10
+    // Pinned ROCm CPU memory allocated by hipMallocHost
+    #define ARROW_DEVICE_ROCM_HOST 11
+    // Reserved for extension
+    //
+    // used to quickly test extension devices, semantics
+    // can differ based on implementation
+    #define ARROW_DEVICE_EXT_DEV 12
+    // CUDA managed/unified memory allocated by cudaMallocManaged
+    #define ARROW_DEVICE_CUDA_MANAGED 13
+    // Unified shared memory allocated on a oneAPI
+    // non-partitioned device.
+    //
+    // A call to the oneAPI runtime is required to determine the
+    // device type, the USM allocation type and the sycl context
+    // that it is bound to.
+    #define ARROW_DEVICE_ONEAPI 14
+    // GPU support for next-gen WebGPU standard
+    #define ARROW_DEVICE_WEBGPU 15
+    // Qualcomm Hexagon DSP
+    #define ARROW_DEVICE_HEXAGON 16
+
+    struct ArrowDeviceArray {
+      struct ArrowArray array;
+      int64_t device_id;
+      ArrowDeviceType device_type;
+      void* sync_event;
+
+      // reserved bytes for future expansion
+      int64_t reserved[3];
+    };
+
+    #endif  // ARROW_C_DEVICE_DATA_INTERFACE
+
+.. note::
+   The canonical guard ``ARROW_C_DEVICE_DATA_INTERFACE`` is meant to avoid
+   duplicate definitions if two projects copy the definitions in their own
+   headers, and a third-party project includes from these two projects. It
+   is therefore important that this guard is kept exactly as-is when these
+   definitions are copied.
+
+ArrowDeviceType
+---------------
+
+The ``ArrowDeviceType`` typedef is used to indicate what type of device the
+provided memory buffers were allocated on. This, in conjunction with the
+``device_id``, should be sufficient to reference the correct data buffers.
+
+We then use macros to define values for different device types. The provided
+macro values are compatible with the widely used `dlpack`_ ``DLDeviceType``
+definition values, using the same value for each as the equivalent
+``kDL<type>`` enum from dlpack.h. The list will be kept in sync with those
+equivalent enum values over time to ensure compatibility, rather than
+potentially diverging. To avoid the Arrow project having to be in the
+position of vetting new hardware devices, new additions should first be
+added to dlpack before we add a corresponding macro here.
+
+To ensure predictability with the ABI, we use macros instead of an ``enum``
+so the storage type is not compiler dependent.
+
+.. c:macro:: ARROW_DEVICE_CPU
+
+    CPU Device, equivalent to just using ``ArrowArray`` directly instead of
+    using ``ArrowDeviceArray``.
+
+.. c:macro:: ARROW_DEVICE_CUDA
+
+    A `CUDA`_ GPU Device. This could represent data allocated either with the
+    runtime library (``cudaMalloc``) or the device driver (``cuMemAlloc``).
+
+.. c:macro:: ARROW_DEVICE_CUDA_HOST
+
+    CPU memory that was pinned and page-locked by CUDA by using
+    ``cudaMallocHost`` or ``cuMemAllocHost``.
+
+.. c:macro:: ARROW_DEVICE_OPENCL
+
+    Data allocated on the device by using the `OpenCL (Open Computing Language)`_
+    framework.
+
+.. c:macro:: ARROW_DEVICE_VULKAN
+
+    Data allocated by the `Vulkan`_ framework and libraries.
+
+.. c:macro:: ARROW_DEVICE_METAL
+
+    Data on Apple GPU devices using the `Metal`_ framework and libraries.
+
+.. c:macro:: ARROW_DEVICE_VPI
+
+    Indicates usage of a Verilog simulator buffer.
+
+.. c:macro:: ARROW_DEVICE_ROCM
+
+    An AMD device using the `ROCm`_ stack.
+
+.. c:macro:: ARROW_DEVICE_ROCM_HOST
+
+    CPU memory pinned and page-locked allocated ``hipMallocHost``.
+
+.. c:macro:: ARROW_DEVICE_EXT_DEV
+
+    This value is an escape-hatch for devices to extend which aren't
+    currently represented otherwise. Producers would need to provide
+    additional information/context specific to the device if using
+    this device type. This is used to quickly test extension devices
+    and semantics can differ based on the implementation.
+
+.. c:macro:: ARROW_DEVICE_CUDA_MANAGED
+
+    CUDA managed/unified memory which is allocated by ``cudaMallocManaged``.
+
+.. c:macro:: ARROW_DEVICE_ONEAPI
+
+    Unified shared memory allocated on an Intel `oneAPI`_ non-partitioned
+    device. A call to the ``oneAPI`` runtime is required to determine
+    the specific device type, the USM allocation type and the sycl context
+    that it is bound to.
+
+.. c:macro:: ARROW_DEVICE_WEBGPU
+
+    GPU support for next-gen WebGPU standards
+
+.. c:macro:: ARROW_DEVICE_HEXAGON
+
+    Data allocated on a Qualcomm Hexagon DSP device.
+
+The ArrowDeviceArray structure
+------------------------------
+
+The ``ArrowDeviceArray`` structure embeds the C data ``ArrowArray`` structure
+and adds additional information necessary for consumers to use the data. It
+has the following fields:
+
+.. c:member:: struct ArrowArray ArrowDeviceArray.array
+
+    The allocated array data. The values in the ``void**`` buffers (along
+    with the buffers of any children) are what is allocated on the device.
+    The buffer values should be device pointers. The rest of the structure
+    should be accessible to the CPU.
+
+    The ``private_data`` and ``release`` callback of this structure should
+    contain any necessary information and structures related to freeing
+    the array according to the device it is allocated on, rather than
+    having a separate release callback and ``private_data`` pointer here.
+
+.. c:member:: int64_t ArrowDeviceArray.device_id
+
+    The device id to identify a specific device if multiple devices of this
+    type are on the system. The semantics of the id will be hardware dependent,
+    but we use an ``int64_t`` to future-proof the id as devices change over time.
+
+.. c:member:: ArrowDeviceType ArrowDeviceArray.device_type
+
+    The type of the device which can access the buffers in the array.
+
+.. c:member:: void* ArrowDeviceArray.sync_event
+
+    Optional. An event-like object to synchronize on if needed.
+
+    Many devices, like GPUs, are primarily asynchronous with respect to
+    CPU processing. As such in order to safely access memory, it is often
+    necessary to have an object to synchronize processing with. Since
+    different devices will use different types to specify this, we use a
+    void* which can be coerced into a pointer to whatever the device
+    appropriate type is.
+
+    If synchronization is not needed, this can be null. If this is non-null
+    then it MUST be used to call the appropriate sync method for the device
+    (e.g. ``cudaStreamWaitEvent`` or ``hipStreamWaitEvent``) before attempting
+    to access the memory in the buffers.
+
+    Expected types to coerce this ``void*`` to depending on the reported
+    device type:
+
+    * CUDA: ``cudaEvent_t*``
+    * ROCm: ``hipEvent_t*``
+    * OpenCL: ``cl_event*``
+    * Vulkan: ``VkEvent*``
+    * Metal: ``MTLEvent*``
+    * OneAPI: ``sycl::event*``
+
+    If an event is provided, then the producer MUST ensure that the exported
+    data is available on the device before the event is triggered. The 
+    consumer SHOULD wait on the event before trying to access the exported
+    data.
+
+.. c:member:: int64_t ArrowDeviceArray.reserved[3]
+
+    As non-CPU development expands, there may be a need to expand this
+    structure. In order to do so without potentially breaking ABI changes,
+    we reserve 24 bytes at the end of the object. This also has the added
+    benefit of bringing the total size of this structure to exactly 128
+    bytes (a power of 2) on 64-bit systems. These bytes MUST be zero'd
+    out after initialization by the producer in order to ensure safe 
+    evolution of the ABI in the future.
+
+.. note::
+    Rather than store the shape / types of the data alongside the
+    ``ArrowDeviceArray``, users should utilize the existing ``ArrowSchema``
+    structure to pass any data type and shape information.
+
+Synchronization event types
+---------------------------
+
+The table below lists the expected event types for each device type.
+If no event type is supported ("N/A"), then the ``sync_event`` member
+should always be null. 
+
+Remember that the event *CAN* be null if synchronization is not needed
+to access the data.
+
++---------------------------+--------------------+---------+
+| Device Type               | Actual Event Type  | Notes   |
++===========================+====================+=========+
+| ARROW_DEVICE_CPU          | N/A                |         |
++---------------------------+--------------------+---------+
+| ARROW_DEVICE_CUDA         | ``cudaEvent_t*``   |         |
++---------------------------+--------------------+---------+
+| ARROW_DEVICE_CUDA_HOST    | ``cudaEvent_t*``   |         |
++---------------------------+--------------------+---------+
+| ARROW_DEVICE_OPENCL       | ``cl_event*``      |         |
++---------------------------+--------------------+---------+
+| ARROW_DEVICE_VULKAN       | ``VkEvent*``       |         |
++---------------------------+--------------------+---------+
+| ARROW_DEVICE_METAL        | ``MTLEvent*``      |         |
++---------------------------+--------------------+---------+
+| ARROW_DEVICE_VPI          | N/A                | (1)     |
++---------------------------+--------------------+---------+
+| ARROW_DEVICE_ROCM         | ``hipEvent_t*``    |         |
++---------------------------+--------------------+---------+
+| ARROW_DEVICE_ROCM_HOST    | ``hipEvent_t*``    |         |
++---------------------------+--------------------+---------+
+| ARROW_DEVICE_EXT_DEV      |                    | (2)     |
++---------------------------+--------------------+---------+
+| ARROW_DEVICE_CUDA_MANAGED | ``cudaEvent_t*``   |         |
++---------------------------+--------------------+---------+
+| ARROW_DEVICE_ONEAPI       | ``sycl::event*``   |         |
++---------------------------+--------------------+---------+
+| ARROW_DEVICE_WEBGPU       | N/A                | (1)     |
++---------------------------+--------------------+---------+
+| ARROW_DEVICE_HEXAGON      | N/A                | (1)     |
++---------------------------+--------------------+---------+
+
+Notes:
+
+* \(1) Currently unknown if framework has an event type to support.
+* \(2) Extension Device has producer defined semantics and thus if
+       synchronization is needed for an extension device, the producer
+       should document the type.
+
+
+Semantics
+=========
+
+Memory management
+-----------------
+
+The ``ArrowDeviceArray`` structure contains an ``ArrowArray`` object which
+itself has :ref:`specific semantics <c-data-interface-semantics>` for releasing
+memory. The term *"base structure"* below refers to the ``ArrowDeviceArray``
+object that is passed directly between the producer and consumer -- not any
+child structure thereof.
+
+It is intended for the base structure to be stack- or heap-allocated by the
+*consumer*. In this case, the producer API should take a pointer to the
+consumer-allocated structure.
+
+However, any data pointed to by the struct MUST be allocated and maintained
+by the producer. This includes the ``sync_event`` member if it is not null,
+along with any pointers in the ``ArrowArray`` object as usual. Data lifetime
+is managed through the ``release`` callback of the ``ArrowArray`` member.
+
+For an ``ArrowDeviceArray``, the semantics of a released structure and the
+callback semantics are identical to those for
+:ref:`ArrowArray itself <c-data-interface-released>`. Any producer specific context
+information necessary for releasing the device data buffers, in addition to
+any allocated event, should be stored in the ``private_data`` member of
+the ``ArrowArray`` and managed by the ``release`` callback.
+
+Moving an array
+'''''''''''''''
+
+The consumer can *move* the ``ArrowDeviceArray`` structure by bitwise copying
+or shallow member-wise copying. Then it MUST mark the source structure released
+by setting the ``release`` member of the embedded ``ArrowArray`` structure to
+``NULL``, but *without* calling that release callback. This ensures that only
+one live copy of the struct is active at any given time and that lifetime is
+correctly communicated to the producer.
+
+As usual, the release callback will be called on the destination structure
+when it is not needed anymore.
+
+Record batches
+--------------
+As with the C data interface itself, a record batch can be trivially considered
+as an equivalent struct array. In this case the metadata of the top-level
+``ArrowSchema`` can be used for schema-level metadata of the record batch.
+
+Mutability
+----------
+
+Both the producer and the consumer SHOULD consider the exported data (that
+is, the data reachable on the device through the ``buffers`` member of
+the embedded ``ArrowArray``) to be immutable, as either party could otherwise
+see inconsistent data while the other is mutating it.
+
+Synchronization
+---------------
+
+If the ``sync_event`` member is non-NULL, the consumer should not attempt 
+to access or read the data until they have synchronized on that event. If
+the ``sync_event`` member is NULL, then it MUST be safe to access the data
+without any synchronization necessary on the part of the consumer.
+
+C producer example
+====================
+
+Exporting a simple ``int32`` device array
+-----------------------------------------
+
+Export a non-nullable ``int32`` type with empty metadata. An example of this
+can be seen in the :ref:`C data interface docs directly <c-data-interface-export-int32-schema>`.
+
+To export the data itself, we transfer ownership to the consumer through
+the release callback. This example will use CUDA, but the equivalent calls
+could be used for any device:
+
+.. code-block:: c
+
+    static void release_int32_device_array(struct ArrowArray* array) {
+        assert(array->n_buffers == 2);
+        // destroy the event
+        cudaEvent_t* ev_ptr = (cudaEvent_t*)(array->private_data);
+        cudaError_t status = cudaEventDestroy(*ev_ptr);
+        assert(status == cudaSuccess);
+        free(ev_ptr);
+
+        // free the buffers and the buffers array
+        status = cudaFree(array->buffers[1]);
+        assert(status == cudaSuccess);
+        free(array->buffers);
+
+        // mark released
+        array->release = NULL;
+    }
+
+    void export_int32_device_array(void* cudaAllocdPtr,
+                                   cudaStream_t stream,
+                                   int64_t length,
+                                   struct ArrowDeviceArray* array) {
+        // get device id
+        int device;
+        cudaError_t status;
+        status = cudaGetDevice(&device);
+        assert(status == cudaSuccess);
+
+        cudaEvent_t* ev_ptr = (cudaEvent_t*)malloc(sizeof(cudaEvent_t));
+        assert(ev_ptr != NULL);
+        status = cudaEventCreate(ev_ptr);
+        assert(status == cudaSuccess);
+
+        // record event on the stream, assuming that the passed in
+        // stream is where the work to produce the data will be processing.
+        status = cudaEventRecord(*ev_ptr, stream);
+        assert(status == cudaSuccess);
+
+        // initialize fields
+        *array = (struct ArrowDeviceArray) {
+            .array = (struct ArrowArray) {
+                .length = length,
+                .null_count = 0,
+                .offset = 0,
+                .n_buffers = 2,
+                .n_children = 0,
+                .children = NULL,
+                .dictionary = NULL,
+                // bookeeping
+                .release = &release_int32_device_array,
+                // store the event pointer as private data in the array
+                // so that we can access it in the release callback.
+                .private_data = (void*)(ev_ptr),
+            },
+            .device_id = (int64_t)(device),
+            .device_type = ARROW_DEVICE_CUDA,
+            // pass the event pointer to the consumer
+            .sync_event = (void*)(ev_ptr),
+        };
+
+        // allocate list of buffers
+        array->array.buffers = (const void**)malloc(sizeof(void*) * array->array.n_buffers);
+        assert(array->array.buffers != NULL);
+        array->array.buffers[0] = NULL;
+        array->array.buffers[1] = cudaAllocdPtr;
+    }
+
+    // calling the release callback should be done using the array member
+    // of the device array.
+    static void release_device_array_helper(struct ArrowDeviceArray* arr) {
+        arr->array.release(&arr->array);
+    }
+
+=======================
+Device Stream Interface
+=======================
+
+Like the :ref:`C stream interface <c-stream-interface>`, the C Device data
+interface also specifies a higher-level structure for easing communication
+of streaming data within a single process. Defining an ``ArrowDeviceArrayStream``
+structure.
+
+Semantics
+=========
+
+An Arrow C device stream exposes a streaming source of data chunks, each with
+the same schema. Chunks are obtained by calling a blocking pull-style iteration
+function. It is expected that all chunks should be providing data on the same
+device type (but not necessarily the same device id). If it is necessary
+to provide a stream of data on multiple device types, a producer should
+provide a separate stream object for each device type.
+
+Structure definition
+====================
+
+The C device stream interface is defined by a single ``struct`` definition:
+
+.. code-block:: c
+
+    #ifndef ARROW_C_DEVICE_STREAM_INTERFACE
+    #define ARROW_C_DEVICE_STREAM_INTERFACE
+
+    struct ArrowDeviceArrayStream {
+        // device type that all arrays will be accessible from
+        ArrowDeviceType device_type;
+        // callbacks
+        int (*get_schema)(struct ArrowDeviceArrayStream*, struct ArrowSchema*);
+        int (*get_next)(struct ArrowDeviceArrayStream*, struct ArrowDeviceArray*);
+        const char* (*get_last_error)(struct ArrowDeviceArrayStream*);
+
+        // release callback
+        void (*release)(struct ArrowDeviceArrayStream*);
+
+        // opaque producer-specific data
+        void* private_data;
+    };
+
+    #endif  // ARROW_C_DEVICE_STREAM_INTERFACE
+
+.. note::
+    The canonical guard ``ARROW_C_DEVICE_STREAM_INTERFACE`` is meant to avoid
+    duplicate definitions if two projects copy the C device stream interface
+    definitions into their own headers, and a third-party project includes
+    from these two projects. It is therefore important that this guard is
+    kept exactly as-is when these definitions are copied.
+
+The ArrowDeviceArrayStream structure
+------------------------------------
+
+The ``ArrowDeviceArrayStream`` provides a device type that can access the
+resulting data along with the required callbacks to interact with a
+streaming source of Arrow arrays. It has the following fields:
+
+.. c:member:: ArrowDeviceType device_type
+
+    The device type that this stream produces data on. All

Review Comment:
   added



##########
docs/source/format/CDeviceDataInterface.rst:
##########
@@ -0,0 +1,688 @@
+.. Licensed to the Apache Software Foundation (ASF) under one
+.. or more contributor license agreements.  See the NOTICE file
+.. distributed with this work for additional information
+.. regarding copyright ownership.  The ASF licenses this file
+.. to you under the Apache License, Version 2.0 (the
+.. "License"); you may not use this file except in compliance
+.. with the License.  You may obtain a copy of the License at
+
+..   http://www.apache.org/licenses/LICENSE-2.0
+
+.. Unless required by applicable law or agreed to in writing,
+.. software distributed under the License is distributed on an
+.. "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+.. KIND, either express or implied.  See the License for the
+.. specific language governing permissions and limitations
+.. under the License.
+
+.. highlight:: c
+
+.. _c-device-data-interface:
+
+=================================
+The Arrow C Device data interface
+=================================
+
+.. warning:: The Arrow C Device Data Interface should be considered experimental
+
+Rationale
+=========
+
+The current :ref:`C Data Interface <c-data-interface>`, and most
+implementations of it, make the assumption that all data buffers provided
+are CPU buffers. Since Apache Arrow is designed to be a universal in-memory
+format for representing tabular ("columnar") data, there will be the desire
+to leverage this data on non-CPU hardware such as GPUs. One example of such
+a case is the `RAPIDS cuDF library`_ which uses the Arrow memory format with
+CUDA for NVIDIA GPUs. Since copying data from host to device and back is
+expensive, the ideal would be to be able to leave the data on the device
+for as long as possible, even when passing it between runtimes and
+libraries.
+
+The Arrow C Device data interface builds on the existing C data interface
+by adding a very small, stable set of C definitions to it. These definitions
+are equivalents to the ``ArrowArray`` and ``ArrowArrayStream`` structures
+from the C Data Interface which add members to allow specifying the device
+type and pass necessary information to synchronize with the producer.
+For non-C/C++ languages and runtimes, translating the C definitions to
+corresponding C FFI declarations should be just as simple as with the
+current C data interface.
+
+Applications and libraries can then use Arrow schemas and Arrow formatted
+memory on non-CPU devices to exchange data just as easily as they do
+now with CPU data. This will enable leaving data on those devices longer
+and avoiding costly copies back and forth between the host and device
+just to leverage new libraries and runtimes.
+
+Goals
+-----
+
+* Expose an ABI-stable interface built on the existing C data interface.
+* Make it easy for third-party projects to implement support with little
+  initial investment.
+* Allow zero-copy sharing of Arrow formatted device memory between
+  independant runtimes and components running in the same process.
+* Avoid the need for one-to-one adaptation layers such as the
+  `CUDA Array Interface`_ for Python processes to pass CUDA data.
+* Enable integration without explicit dependencies (either at compile-time
+  or runtime) on the Arrow software project itself.
+
+The intent is for the Arrow C Device data interface to expand the reach
+of the current C data interface, allowing it to also become the standard
+low-level building block for columnar processing on devices like GPUs or
+FPGAs.
+
+Structure definitions
+=====================
+
+Because this is built on the C data interface, the C Device data interface
+uses the ``ArrowSchema`` and ``ArrowArray`` structures as defined in the
+:ref:`C data interface spec <c-data-interface-struct-defs>`. It then adds the
+following free-standing definitions. Like the rest of the Arrow project,
+they are available under the Apache License 2.0.
+
+.. code-block:: c
+
+    #ifndef ARROW_C_DEVICE_DATA_INTERFACE
+    #define ARROW_C_DEVICE_DATA_INTERFACE
+
+    // Device type for the allocated memory
+    typedef int32_t ArrowDeviceType;
+
+    // CPU device, same as using ArrowArray directly
+    #define ARROW_DEVICE_CPU 1
+    // CUDA GPU Device
+    #define ARROW_DEVICE_CUDA 2
+    // Pinned CUDA CPU memory by cudaMallocHost
+    #define ARROW_DEVICE_CUDA_HOST 3
+    // OpenCL Device
+    #define ARROW_DEVICE_OPENCL 4
+    // Vulkan buffer for next-gen graphics
+    #define ARROW_DEVICE_VULKAN 7
+    // Metal for Apple GPU
+    #define ARROW_DEVICE_METAL 8
+    // Verilog simulator buffer
+    #define ARROW_DEVICE_VPI 9
+    // ROCm GPUs for AMD GPUs
+    #define ARROW_DEVICE_ROCM 10
+    // Pinned ROCm CPU memory allocated by hipMallocHost
+    #define ARROW_DEVICE_ROCM_HOST 11
+    // Reserved for extension
+    //
+    // used to quickly test extension devices, semantics
+    // can differ based on implementation
+    #define ARROW_DEVICE_EXT_DEV 12
+    // CUDA managed/unified memory allocated by cudaMallocManaged
+    #define ARROW_DEVICE_CUDA_MANAGED 13
+    // Unified shared memory allocated on a oneAPI
+    // non-partitioned device.
+    //
+    // A call to the oneAPI runtime is required to determine the
+    // device type, the USM allocation type and the sycl context
+    // that it is bound to.
+    #define ARROW_DEVICE_ONEAPI 14
+    // GPU support for next-gen WebGPU standard
+    #define ARROW_DEVICE_WEBGPU 15
+    // Qualcomm Hexagon DSP
+    #define ARROW_DEVICE_HEXAGON 16
+
+    struct ArrowDeviceArray {
+      struct ArrowArray array;
+      int64_t device_id;
+      ArrowDeviceType device_type;
+      void* sync_event;
+
+      // reserved bytes for future expansion
+      int64_t reserved[3];
+    };
+
+    #endif  // ARROW_C_DEVICE_DATA_INTERFACE
+
+.. note::
+   The canonical guard ``ARROW_C_DEVICE_DATA_INTERFACE`` is meant to avoid
+   duplicate definitions if two projects copy the definitions in their own
+   headers, and a third-party project includes from these two projects. It
+   is therefore important that this guard is kept exactly as-is when these
+   definitions are copied.
+
+ArrowDeviceType
+---------------
+
+The ``ArrowDeviceType`` typedef is used to indicate what type of device the
+provided memory buffers were allocated on. This, in conjunction with the
+``device_id``, should be sufficient to reference the correct data buffers.
+
+We then use macros to define values for different device types. The provided
+macro values are compatible with the widely used `dlpack`_ ``DLDeviceType``
+definition values, using the same value for each as the equivalent
+``kDL<type>`` enum from dlpack.h. The list will be kept in sync with those
+equivalent enum values over time to ensure compatibility, rather than
+potentially diverging. To avoid the Arrow project having to be in the
+position of vetting new hardware devices, new additions should first be
+added to dlpack before we add a corresponding macro here.
+
+To ensure predictability with the ABI, we use macros instead of an ``enum``
+so the storage type is not compiler dependent.
+
+.. c:macro:: ARROW_DEVICE_CPU
+
+    CPU Device, equivalent to just using ``ArrowArray`` directly instead of
+    using ``ArrowDeviceArray``.
+
+.. c:macro:: ARROW_DEVICE_CUDA
+
+    A `CUDA`_ GPU Device. This could represent data allocated either with the
+    runtime library (``cudaMalloc``) or the device driver (``cuMemAlloc``).
+
+.. c:macro:: ARROW_DEVICE_CUDA_HOST
+
+    CPU memory that was pinned and page-locked by CUDA by using
+    ``cudaMallocHost`` or ``cuMemAllocHost``.
+
+.. c:macro:: ARROW_DEVICE_OPENCL
+
+    Data allocated on the device by using the `OpenCL (Open Computing Language)`_
+    framework.
+
+.. c:macro:: ARROW_DEVICE_VULKAN
+
+    Data allocated by the `Vulkan`_ framework and libraries.
+
+.. c:macro:: ARROW_DEVICE_METAL
+
+    Data on Apple GPU devices using the `Metal`_ framework and libraries.
+
+.. c:macro:: ARROW_DEVICE_VPI
+
+    Indicates usage of a Verilog simulator buffer.
+
+.. c:macro:: ARROW_DEVICE_ROCM
+
+    An AMD device using the `ROCm`_ stack.
+
+.. c:macro:: ARROW_DEVICE_ROCM_HOST
+
+    CPU memory pinned and page-locked allocated ``hipMallocHost``.
+
+.. c:macro:: ARROW_DEVICE_EXT_DEV
+
+    This value is an escape-hatch for devices to extend which aren't
+    currently represented otherwise. Producers would need to provide
+    additional information/context specific to the device if using
+    this device type. This is used to quickly test extension devices
+    and semantics can differ based on the implementation.
+
+.. c:macro:: ARROW_DEVICE_CUDA_MANAGED
+
+    CUDA managed/unified memory which is allocated by ``cudaMallocManaged``.
+
+.. c:macro:: ARROW_DEVICE_ONEAPI
+
+    Unified shared memory allocated on an Intel `oneAPI`_ non-partitioned
+    device. A call to the ``oneAPI`` runtime is required to determine
+    the specific device type, the USM allocation type and the sycl context
+    that it is bound to.
+
+.. c:macro:: ARROW_DEVICE_WEBGPU
+
+    GPU support for next-gen WebGPU standards
+
+.. c:macro:: ARROW_DEVICE_HEXAGON
+
+    Data allocated on a Qualcomm Hexagon DSP device.
+
+The ArrowDeviceArray structure
+------------------------------
+
+The ``ArrowDeviceArray`` structure embeds the C data ``ArrowArray`` structure
+and adds additional information necessary for consumers to use the data. It
+has the following fields:
+
+.. c:member:: struct ArrowArray ArrowDeviceArray.array

Review Comment:
   done



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [arrow] zeroshade commented on a diff in pull request #34972: GH-34971: [Format] Add non-CPU version of C Data Interface

["zeroshade (via GitHub)" <gi...@apache.org>]

zeroshade commented on code in PR #34972:
URL: https://github.com/apache/arrow/pull/34972#discussion_r1213607454


##########
docs/source/format/CDeviceDataInterface.rst:
##########
@@ -0,0 +1,628 @@
+.. Licensed to the Apache Software Foundation (ASF) under one
+.. or more contributor license agreements.  See the NOTICE file
+.. distributed with this work for additional information
+.. regarding copyright ownership.  The ASF licenses this file
+.. to you under the Apache License, Version 2.0 (the
+.. "License"); you may not use this file except in compliance
+.. with the License.  You may obtain a copy of the License at
+
+..   http://www.apache.org/licenses/LICENSE-2.0
+
+.. Unless required by applicable law or agreed to in writing,
+.. software distributed under the License is distributed on an
+.. "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+.. KIND, either express or implied.  See the License for the
+.. specific language governing permissions and limitations
+.. under the License.
+
+.. highlight:: c
+
+.. _c-device-data-interface:
+
+=================================
+The Arrow C Device data interface
+=================================
+
+.. note:: The Arrow C Device Data Interface should be considered experimental
+
+Rationale
+=========
+
+The current :ref:`C Data Interface <c-data-interface>`, and most
+implementations of it, make the assumption that all data buffers provided
+are CPU buffers. Since Apache Arrow is designed to be a universal in-memory
+format for representing tabular ("columnar") data, there will be the desire
+to leverage this data on non-CPU hardware such as GPUs. One example of such
+a case is the `RAPIDS cuDF library`_ which uses the Arrow memory format with
+CUDA for NVIDIA GPUs. Since copying data from host to device and back is
+expensive, the ideal would be to be able to leave the data on the device
+for as long as possible, even when passing it between runtimes and
+libraries.
+
+The Arrow C Device data interface builds on the existing C data interface
+by adding a very small, stable set of C definitions to it. These definitions
+are equivalents to the ``ArrowArray`` and ``ArrowArrayStream`` structures
+from the C Data Interface which add members to allow specifying the device
+type and pass necessary information to synchronize with the producer.
+For non-C/C++ languages and runtimes, translating the C definitions to
+corresponding C FFI declarations should be just as simple as with the
+current C data interface.
+
+Applications and libraries can then use Arrow schemas and Arrow formatted
+memory on non-CPU devices to exchange data just as easily as they do
+now with CPU data. This will enable leaving data on those devices longer
+and avoiding costly copies back and forth between the host and device
+just to leverage new libraries and runtimes.
+
+Goals
+-----
+
+* Expose an ABI-stable interface built on the existing C data interface.
+* Make it easy for third-party projects to implement support with little
+  initial investment.
+* Allow zero-copy sharing of Arrow formatted device memory between
+  independant runtimes and components running in the same process.
+* Avoid the need for one-to-one adaptation layers such as the
+  `CUDA Array Interface`_ for Python processes to pass CUDA data.
+* Enable integration without explicit dependencies (either at compile-time
+  or runtime) on the Arrow software project itself.
+
+The intent is for the Arrow C Device data interface to expand the reach
+of the current C data interface, allowing it to also become the standard
+low-level building block for columnar processing on devices like GPUs or
+FPGAs.
+
+Structure definitions
+=====================
+
+Because this is built on the C data interface, the C Device data interface
+uses the ``ArrowSchema`` and ``ArrowArray`` structures as defined in the
+:ref:`C data interface spec <c-data-interface-struct-defs>`. It then adds the
+following free-standing definitions. Like the rest of the Arrow project,
+they are available under the Apache License 2.0.
+
+.. code-block:: c
+
+    #ifndef ARROW_C_DEVICE_DATA_INTERFACE
+    #define ARROW_C_DEVICE_DATA_INTERFACE
+
+    // Device type for the allocated memory
+    typedef int32_t ArrowDeviceType;
+
+    // CPU device, same as using ArrowArray directly
+    #define ARROW_DEVICE_CPU 1
+    // CUDA GPU Device
+    #define ARROW_DEVICE_CUDA 2
+    // Pinned CUDA CPU memory by cudaMallocHost
+    #define ARROW_DEVICE_CUDA_HOST 3
+    // OpenCL Device
+    #define ARROW_DEVICE_OPENCL 4
+    // Vulkan buffer for next-gen graphics
+    #define ARROW_DEVICE_VULKAN 7
+    // Metal for Apple GPU
+    #define ARROW_DEVICE_METAL 8
+    // Verilog simulator buffer
+    #define ARROW_DEVICE_VPI 9
+    // ROCm GPUs for AMD GPUs
+    #define ARROW_DEVICE_ROCM 10
+    // Pinned ROCm CPU memory allocated by hipMallocHost
+    #define ARROW_DEVICE_ROCM_HOST 11
+    // Reserved for extension
+    //
+    // used to quickly test extension devices, semantics
+    // can differ based on implementation
+    #define ARROW_DEVICE_EXT_DEV 12
+    // CUDA managed/unified memory allocated by cudaMallocManaged
+    #define ARROW_DEVICE_CUDA_MANAGED 13
+    // Unified shared memory allocated on a oneAPI
+    // non-partitioned device.
+    //
+    // A call to the oneAPI runtime is required to determine the
+    // device type, the USM allocation type and the sycl context
+    // that it is bound to.
+    #define ARROW_DEVICE_ONEAPI 14
+    // GPU support for next-gen WebGPU standard
+    #define ARROW_DEVICE_WEBGPU 15
+    // Qualcomm Hexagon DSP
+    #define ARROW_DEVICE_HEXAGON 16
+
+    struct ArrowDeviceArray {
+      struct ArrowArray array;
+      int64_t device_id;
+      ArrowDeviceType device_type;
+      void* sync_event;
+
+      // reserved bytes for future expansion
+      int64_t reserved[3];
+    };
+
+    #endif  // ARROW_C_DEVICE_DATA_INTERFACE
+
+.. note::
+   The canonical guard ``ARROW_C_DEVICE_DATA_INTERFACE`` is meant to avoid
+   duplicate definitions if two projects copy the definitions in their own
+   headers, and a third-party project includes from these two projects. It
+   is therefore important that this guard is kept exactly as-is when these
+   definitions are copied.
+
+ArrowDeviceType
+---------------
+
+The ``ArrowDeviceType`` typedef is used to indicate what type of device the
+provided memory buffers were allocated on. This, in conjunction with the
+``device_id``, should be sufficient to reference the correct data buffers.
+
+We then use macros to define values for different device types. The provided
+macro values are compatible with the widely used `dlpack`_ ``DLDeviceType``
+definition values, using the same value for each as the equivalent
+``kDL<type>`` enum from dlpack.h. The list will be kept in sync with those
+equivalent enum values over time to ensure compatibility, rather than
+potentially diverging. To avoid the Arrow project having to be in the
+position of vetting new hardware devices, new additions should first be
+added to dlpack before we add a corresponding macro here.
+
+To ensure predictability with the ABI, we use macros instead of an ``enum``
+so the storage type is not compiler dependent.
+
+.. c:macro:: ARROW_DEVICE_CPU
+
+    CPU Device, equivalent to just using ``ArrowArray`` directly instead of
+    using ``ArrowDeviceArray``.
+
+.. c:macro:: ARROW_DEVICE_CUDA
+
+    A `CUDA`_ GPU Device. This could represent data allocated either with the
+    runtime library (``cudaMalloc``) or the device driver (``cuMemAlloc``).
+
+.. c:macro:: ARROW_DEVICE_CUDA_HOST
+
+    CPU memory that was pinned and page-locked by CUDA by using
+    ``cudaMallocHost`` or ``cuMemAllocHost``.
+
+.. c:macro:: ARROW_DEVICE_OPENCL
+
+    Data allocated on the device by using the `OpenCL (Open Computing Language)`_
+    framework.
+
+.. c:macro:: ARROW_DEVICE_VULKAN
+
+    Data allocated by the `Vulkan`_ framework and libraries.
+
+.. c:macro:: ARROW_DEVICE_METAL
+
+    Data on Apple GPU devices using the `Metal`_ framework and libraries.
+
+.. c:macro:: ARROW_DEVICE_VPI
+
+    Indicates usage of a Verilog simulator buffer.
+
+.. c:macro:: ARROW_DEVICE_ROCM
+
+    An AMD device using the `ROCm`_ stack.
+
+.. c:macro:: ARROW_DEVICE_ROCM_HOST
+
+    CPU memory pinned and page-locked allocated ``hipMallocHost``.
+
+.. c:macro:: ARROW_DEVICE_EXT_DEV
+
+    This value is an escape-hatch for devices to extend which aren't
+    currently represented otherwise. Producers would need to provide
+    additional information/context specific to the device if using
+    this device type. This is used to quickly test extension devices
+    and semantics can differ based on the implementation.
+
+.. c:macro:: ARROW_DEVICE_CUDA_MANAGED
+
+    CUDA managed/unified memory which is allocated by ``cudaMallocManaged``.
+
+.. c:macro:: ARROW_DEVICE_ONEAPI
+
+    Unified shared memory allocated on an Intel `oneAPI`_ non-partitioned
+    device. A call to the ``oneAPI`` runtime is required to determine
+    the specific device type, the USM allocation type and the sycl context
+    that it is bound to.
+
+.. c:macro:: ARROW_DEVICE_WEBGPU
+
+    GPU support for next-gen WebGPU standards
+
+.. c:macro:: ARROW_DEVICE_HEXAGON
+
+    Data allocated on a Qualcomm Hexagon DSP device.
+
+The ArrowDeviceArray structure
+------------------------------
+
+The ``ArrowDeviceArray`` structure embeds the C data ``ArrowArray`` structure
+and adds additional information necessary for consumers to use the data. It
+has the following fields:
+
+.. c:member:: struct ArrowArray ArrowDeviceArray.array
+
+    The allocated array data. The values in the ``void**`` buffers (along
+    with the buffers of any children) are what is allocated on the device.
+    The buffer values should be device pointers. The rest of the structure
+    should be accessible to the CPU.
+
+    The ``private_data`` and ``release`` callback of this structure should
+    contain any necessary information and structures related to freeing
+    the array according to the device it is allocated on, rather than
+    having a separate release callback and ``private_data`` pointer here.
+
+.. c:member:: int64_t ArrowDeviceArray.device_id
+
+    The device id to identify a specific device if multiple devices of this
+    type are on the system. The semantics of the id will be hardware dependent,
+    but we use an ``int64_t`` to future-proof the id as devices change over time.
+
+.. c:member:: ArrowDeviceType ArrowDeviceArray.device_type
+
+    The type of the device which can access the buffers in the array.
+
+.. c:member:: void* ArrowDeviceArray.sync_event
+
+    Optional. An event-like object to synchronize on if needed.
+
+    Many devices, like GPUs, are primarily asynchronous with respect to
+    CPU processing. As such in order to safely access memory, it is often
+    necessary to have an object to synchronize processing with. Since
+    different devices will use different types to specify this, we use a
+    void* which can be coerced into a pointer to whatever the device
+    appropriate type is.
+
+    If synchronization is not needed, this can be null. If this is non-null
+    then it MUST be used to call the appropriate sync method for the device
+    (e.g. ``cudaStreamWaitEvent`` or ``hipStreamWaitEvent``) before attempting
+    to access the memory in the buffers.
+
+    Expected types to coerce this ``void*`` to depending on the reported
+    device type:
+
+    * CUDA: ``cudaEvent_t*``
+    * ROCm: ``hipEvent_t*``
+    * OpenCL: ``cl_event*``
+    * Vulkan: ``VkEvent*``
+    * Metal: ``MTLEvent*``
+    * OneAPI: ``sycl::event*``
+
+    If an event is provided, then the producer MUST ensure that the event
+    is triggered/recorded at the end of the processing stream once the data
+    is considered available for use.

Review Comment:
   I've simplified and updated this, let me know if the version is good or if you think it should be further simplified.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [arrow] zeroshade commented on pull request #34972: GH-34971: [Format] Enhance C-Data API to support non-cpu cases

["zeroshade (via GitHub)" <gi...@apache.org>]

zeroshade commented on PR #34972:
URL: https://github.com/apache/arrow/pull/34972#issuecomment-1522011162

   @oleksandr-pavlyk Thanks for the context and information there! Given the proposals so far here and what you mentioned in your comment. Do you have a preference between the concept of a producer-supplied event vs a consumer provided stream (or `syscl::queue`)?
   
   Also, you said:
   
   > I would therefore not recommend deleting information to reconstruct context from `ArrowDeviceArray`....
   
   What information were you referring to as deleting?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [arrow] zeroshade commented on pull request #34972: GH-34971: [Format] Enhance C-Data API to support non-cpu cases

["zeroshade (via GitHub)" <gi...@apache.org>]

zeroshade commented on PR #34972:
URL: https://github.com/apache/arrow/pull/34972#issuecomment-1522352955

   @oleksandr-pavlyk currently there isn't a stream that is a member of the `ArrowDeviceArray` struct. I appreciate the information and context, but can we boil it down to a conclusion/recommendation?
   
   There are two questions that I can think of based on the information you've provided that I'd like to get a concrete opinion/conclusion from  you on if at all possible:
   
   1. Should we include a event/stream/queue/some combination of these in the `ArrowDeviceArray` struct directly? If so, should it be some combination of these objects or should it just be one?
   2. Would it be preferable to have the Producer provide an event inside the `ArrowDeviceArray` struct when it produces the array (consumer has to perform synchronization), or should we have the consumer provide a queue/stream to the producer that it would like the memory accessible from (producer performs synchronization)?
   
   If there's something I'm not mentioning please let me know. I just want to try to move this proposal forward in a way that makes the most sense and is future proof-ed, so it would help to clarify precisely what your conclusions are in regards to those two questions. Thanks much!


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [arrow] gmarkall commented on pull request #34972: GH-34971: [Format] Enhance C-Data API to support non-cpu cases

["gmarkall (via GitHub)" <gi...@apache.org>]

gmarkall commented on PR #34972:
URL: https://github.com/apache/arrow/pull/34972#issuecomment-1545548452

   > @pitrou @lidavidm @paleolimbot @oleksandr-pavlyk @westonpace @leofang @gmarkall
   > 
   > I've updated the header and comments based on the discussions here for the past few weeks. If possible I'd like to reach a consensus here before the end of the week at which point I'll update the arrow dev mailing list for more discussion and hopefully a vote.
   
   Many thanks for the ping - I've no further comments.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [arrow] zeroshade commented on a diff in pull request #34972: GH-34971: [Format] Enhance C-Data API to support non-cpu cases

["zeroshade (via GitHub)" <gi...@apache.org>]

zeroshade commented on code in PR #34972:
URL: https://github.com/apache/arrow/pull/34972#discussion_r1161302114


##########
cpp/src/arrow/c/abi.h:
##########
@@ -106,6 +169,77 @@ struct ArrowArrayStream {
 
 #endif  // ARROW_C_STREAM_INTERFACE
 
+#ifndef ARROW_C_DEVICE_STREAM_INTERFACE
+#define ARROW_C_DEVICE_STREAM_INTERFACE
+
+struct ArrowDeviceArrayStream {
+  // The device that this stream produces data on.
+  // All ArrowDeviceArrays that are produced by this
+  // stream should have the same device_type as set
+  // here. The device_type needs to be provided here
+  // so that consumers can provide the correct type
+  // of stream_ptr when calling get_next.
+  ArrowDeviceType device_type;
+
+  // Callback to get the stream schema
+  // (will be the same for all arrays in the stream).
+  //
+  // Return value: 0 if successful, an `errno`-compatible error code otherwise.
+  //
+  // If successful, the ArrowSchema must be released independently from the stream.
+  int (*get_schema)(struct ArrowDeviceArrayStream*, struct ArrowSchema* out);
+
+  // Callback to get the device id for the next array.
+  // This is necessary so that the proper/correct stream pointer can be provided
+  // to get_next. The parameter provided must not be null.
+  //
+  // Return value: 0 if successful, an `errno`-compatible error code otherwise.
+  //
+  // The next call to `get_next` should provide an ArrowDeviceArray whose
+  // device_id matches what is provided here, and whose device_type is the
+  // same as the device_type member of this stream.
+  int (*get_next_device_id)(struct ArrowDeviceArrayStream*, int* out_device_id);
+
+  // Callback to get the next array
+  // (if no error and the array is released, the stream has ended)
+  //
+  // the provided stream_ptr should be the appropriate stream, or
+  // equivalent object, for the device that the data is allocated on
+  // to indicate where the consumer wants the data to be accessible.

Review Comment:
   Stream lifetime is a tough thing to handle for GPUs because everything is async. So after discussing with @kkraus14 we came to the conclusion that the safest API would be for a consumer to specify which stream they wanted to be able to access the data from and the producer would manage any event and stream synchronizing necessary to make it happen. This puts the stream lifetime management into the consumer (where it's easier to manage) rather than the producer.
   
   @kkraus14 you can probably explain this better than I just did.  



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [arrow] zeroshade commented on a diff in pull request #34972: GH-34971: [Format] Enhance C-Data API to support non-cpu cases

["zeroshade (via GitHub)" <gi...@apache.org>]

zeroshade commented on code in PR #34972:
URL: https://github.com/apache/arrow/pull/34972#discussion_r1161312173


##########
cpp/src/arrow/c/abi.h:
##########
@@ -65,6 +65,69 @@ struct ArrowArray {
 
 #endif  // ARROW_C_DATA_INTERFACE
 
+#ifndef ARROW_C_DEVICE_DATA_INTERFACE
+#define ARROW_C_DEVICE_DATA_INTERFACE
+
+// ArrowDeviceType is compatible with dlpack DLDeviceType for portability

Review Comment:
   I'll fix the comment format to be docstrings. Good point 



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [arrow] kkraus14 commented on a diff in pull request #34972: GH-34971: [Format] Enhance C-Data API to support non-cpu cases

["kkraus14 (via GitHub)" <gi...@apache.org>]

kkraus14 commented on code in PR #34972:
URL: https://github.com/apache/arrow/pull/34972#discussion_r1161863963


##########
cpp/src/arrow/c/abi.h:
##########
@@ -65,6 +65,69 @@ struct ArrowArray {
 
 #endif  // ARROW_C_DATA_INTERFACE
 
+#ifndef ARROW_C_DEVICE_DATA_INTERFACE
+#define ARROW_C_DEVICE_DATA_INTERFACE
+
+// ArrowDeviceType is compatible with dlpack DLDeviceType for portability
+// it uses the same values for each enum as the equivalent kDL<type> from dlpack.h
+#ifdef __cplusplus
+typedef enum : int32_t {
+#else
+typedef enum {
+#endif
+  // CPU device, same as using ArrowArray directly
+  kArrowCPU = 1,
+  // CUDA GPU Device
+  kArrowCUDA = 2,
+  // Pinned CUDA CPU memory by cudaMallocHost
+  kArrowCUDAHost = 3,
+  // OpenCL Device
+  kArrowOpenCL = 4,
+  // Vulkan buffer for next-gen graphics
+  kArrowVulkan = 7,
+  // Metal for Apple GPU
+  kArrowMetal = 8,
+  // Verilog simulator buffer
+  kArrowVPI = 9,
+  // ROCm GPUs for AMD GPUs
+  kArrowROCM = 10,
+  // Pinned ROCm CPU memory allocated by hipMallocHost
+  kArrowROCMHost = 11,
+  // Reserved for extension
+  // used to quickly test extension devices,
+  // semantics can differ based on the implementation
+  kArrowExtDev = 12,
+  // CUDA managed/unified memory allocated by cudaMallocManaged
+  kArrowCUDAManaged = 13,
+  // unified shared memory allocated on a oneAPI non-partitioned
+  // device. call to oneAPI runtime is required to determine the
+  // device type, the USM allocation type and the sycl context it
+  // is bound to
+  kArrowOneAPI = 14,
+  // GPU support for next-gen WebGPU standard
+  kArrowWebGPU = 15,
+  // Qualcomm Hexagon DSP
+  kArrowHexagon = 16,
+} ArrowDeviceType;
+
+struct ArrowDeviceArray {
+  // the private_date and release callback of the arrow array
+  // should contain any necessary information and structures
+  // related to freeing the array according to the device it
+  // is allocated on, rather than having a separate release
+  // callback embedded here.
+  struct ArrowArray* array;

Review Comment:
   One of the nice things in this being a pointer is that `ArrowArray` can evolve without also breaking the `ArrowDeviceArray` abi.
   
   If we're going to inline this maybe we should put it at the end of the `ArrowDeviceArray` struct? I'm not sure if there's magic alignment that compilers do that could lead to UB either way if the `ArrowArray` struct changes.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [arrow] kkraus14 commented on a diff in pull request #34972: GH-34971: [Format] Enhance C-Data API to support non-cpu cases

["kkraus14 (via GitHub)" <gi...@apache.org>]

kkraus14 commented on code in PR #34972:
URL: https://github.com/apache/arrow/pull/34972#discussion_r1163346998


##########
cpp/src/arrow/c/abi.h:
##########
@@ -106,6 +212,98 @@ struct ArrowArrayStream {
 
 #endif  // ARROW_C_STREAM_INTERFACE
 
+#ifndef ARROW_C_DEVICE_STREAM_INTERFACE
+#define ARROW_C_DEVICE_STREAM_INTERFACE
+
+/// \brief Equivalent to ArrowArrayStream, but for ArrowDeviceArrays.
+///
+/// This stream is intended to provide a stream of data on a single
+/// device, if a producer wants data to be produced on multiple devices
+/// then multiple streams should be provided. One per device.
+struct ArrowDeviceArrayStream {
+  /// \brief The device that this stream produces data on.
+  ///
+  /// All ArrowDeviceArrays that are produced by this
+  /// stream should have the same device_type as set
+  /// here. The device_type needs to be provided here
+  /// so that consumers can provide the correct type
+  /// of queue_ptr when calling get_next.
+  ArrowDeviceType device_type;
+
+  /// \brief Callback to get the stream schema
+  /// (will be the same for all arrays in the stream).
+  ///
+  /// If successful, the ArrowSchema must be released independantly from the stream.
+  /// The schema should be accessible via CPU memory.
+  ///
+  /// \param[in] self The ArrowDeviceArrayStream object itself
+  /// \param[out] out C struct to export the schema to
+  /// \return 0 if successful, an `errno`-compatible error code otherwise.
+  int (*get_schema)(struct ArrowDeviceArrayStream* self, struct ArrowSchema* out);
+
+  /// \brief Callback to get the device id for the next array.
+  ///
+  /// This is necessary so that the proper/correct stream pointer can be provided
+  /// to get_next.
+  ///
+  /// The next call to `get_next` should provide an ArrowDeviceArray whose
+  /// device_id matches what is provided here, and whose device_type is the
+  /// same as the device_type member of this stream.

Review Comment:
   There's a lot of discussion in that issue related to internal stream handling under contexts and schedulers and similar terms. It all boils down to the same discussion of many producers being unable to release or share ownership of their streams.
   
   I think this comment does a good job of summarizing the options that were considered: https://github.com/dmlc/dlpack/issues/57#issuecomment-753220425
   
   And then this comment summarizes discussion of those options: https://github.com/dmlc/dlpack/issues/57#issuecomment-774482973
   
   
   The lifetime management of streams as defined by the Numba documentation for `__cuda_array_interface__` (https://numba.readthedocs.io/en/stable/cuda/cuda_array_interface.html#streams) requires that keeping the object (typically array) that produces the `__cuda_array_interface__` alive also keeps the stream alive. In most cases libraries don't associate a stream with the object since it's valid to use multiple streams with a single object.
   
   Here's the current state of things across a handful of projects:
   - Numba
     - Handles stream lifetime properly for `__cuda_array_interface__` since they store a stream object as part of their array: https://github.com/numba/numba/blob/008077553b558bd183668ecd581d4d0bc54bd32c/numba/cuda/cudadrv/devicearray.py#L119-L139
     - Doesn't support `__dlpack__`
   - CuPy
     - Doesn't implement stream lifetime properly for `__cuda_array_interface__` as far as I can tell, where they just get a stream ptr as an integer and if it's not the default stream someone could change the current stream ptr and end up having it be destroyed out from underneath it: https://github.com/cupy/cupy/blob/c92d5bc16293300297b843b4ebb364125697c131/cupy/_core/core.pyx#L258-L262 (cc @leofang)
     - Handles `__dlpack__` properly: https://github.com/cupy/cupy/blob/c92d5bc16293300297b843b4ebb364125697c131/cupy/_core/core.pyx#L285-L327
   - PyTorch
     - Doesn't support passing the stream in `__cuda_array_interface__` currently: https://github.com/pytorch/pytorch/blob/def50d253401540cfdc6c0fffa444d0ee643cc11/torch/_tensor.py#L1005-L1066
     - Handles `__dlpack__` properly: https://github.com/pytorch/pytorch/blob/def50d253401540cfdc6c0fffa444d0ee643cc11/torch/_tensor.py#L1345
   - Tensorflow and JAX
     - Doesn't support passing the stream in `__cuda_array_interface__` currently: https://github.com/tensorflow/tensorflow/blob/ea6a0f282d2b7ce20891dfc24ec8fe107eeaf22d/tensorflow/compiler/xla/python/py_buffer.cc#L254-L300
     - Doesn't support `__dlpack__`, but has explicit `to` and `from` functions for using dlpack pycapsules but does not handle streams: https://github.com/tensorflow/tensorflow/blob/6a050b6c15ed2a545693bc171f5e95dacbe05839/tensorflow/compiler/xla/python/dlpack.cc#L285-L363
   - cuDF
     - Doesn't support passing the stream in `__cuda_array_interface__` currently: https://github.com/rapidsai/cudf/blob/50718e673ff53b18706cf66c6e02cda8e30681fe/python/cudf/cudf/core/column/numerical.py#L169-L191
     - Doesn't support `__dlpack__` but has explicit `to` and `from` functions for using dlpack pycapsules but does not handle streams: https://github.com/rapidsai/cudf/blob/50718e673ff53b18706cf66c6e02cda8e30681fe/cpp/src/interop/dlpack.cpp#L218-L294



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [arrow] zeroshade commented on a diff in pull request #34972: GH-34971: [Format] Enhance C-Data API to support non-cpu cases

["zeroshade (via GitHub)" <gi...@apache.org>]

zeroshade commented on code in PR #34972:
URL: https://github.com/apache/arrow/pull/34972#discussion_r1202570573


##########
cpp/src/arrow/c/abi.h:
##########
@@ -106,6 +239,74 @@ struct ArrowArrayStream {
 
 #endif  // ARROW_C_STREAM_INTERFACE
 
+#ifndef ARROW_C_DEVICE_STREAM_INTERFACE
+#define ARROW_C_DEVICE_STREAM_INTERFACE
+
+/// \brief Equivalent to ArrowArrayStream, but for ArrowDeviceArrays.
+///
+/// This stream is intended to provide a stream of data on a single
+/// device, if a producer wants data to be produced on multiple devices
+/// then multiple streams should be provided. One per device.
+struct ArrowDeviceArrayStream {
+  /// \brief The device that this stream produces data on.
+  ///
+  /// All ArrowDeviceArrays that are produced by this
+  /// stream should have the same device_type as set
+  /// here. The device_type needs to be provided here
+  /// so that consumers can provide the correct type
+  /// of queue_ptr when calling get_next.

Review Comment:
   good point, will update.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [arrow] zeroshade commented on a diff in pull request #34972: GH-34971: [Format] Enhance C-Data API to support non-cpu cases

["zeroshade (via GitHub)" <gi...@apache.org>]

zeroshade commented on code in PR #34972:
URL: https://github.com/apache/arrow/pull/34972#discussion_r1204330140


##########
docs/source/format/CDeviceDataInterface.rst:
##########
@@ -0,0 +1,613 @@
+.. Licensed to the Apache Software Foundation (ASF) under one
+.. or more contributor license agreements.  See the NOTICE file
+.. distributed with this work for additional information
+.. regarding copyright ownership.  The ASF licenses this file
+.. to you under the Apache License, Version 2.0 (the
+.. "License"); you may not use this file except in compliance
+.. with the License.  You may obtain a copy of the License at
+
+..   http://www.apache.org/licenses/LICENSE-2.0
+
+.. Unless required by applicable law or agreed to in writing,
+.. software distributed under the License is distributed on an
+.. "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+.. KIND, either express or implied.  See the License for the
+.. specific language governing permissions and limitations
+.. under the License.
+
+.. highlight:: c
+
+.. _c-device-data-interface:
+
+=================================
+The Arrow C Device data interface
+=================================
+
+.. note:: The Arrow C Device Data Interface should be considered experimental
+
+Rationale
+=========
+
+The current :ref:`C Data Interface <c-data-interface>`, and most
+implementations of it, make the assumption that all data buffers provided
+are CPU buffers. Since Apache Arrow is designed to be a universal in-memory
+format for representing tabular ("columnar") data, there will be the desire
+to leverage this data on Non-CPU hardware such as GPUs. One example of such
+a case is the `RAPIDS cuDF library` which uses the Arrow memory format with
+CUDA for NVIDIA GPUs. Since copying data from host to device and back is
+expensive, the ideal would be to be able to leave the data on the device
+for as long as possible, even when passing it between runtimes and
+libraries.
+
+The Arrow C Device data interface builds on the existing C data interface
+by adding a very small, stable set of C definitions to it. For non-C/C++
+languages and runtimes, translating the C definitions to corresponding
+C FFI declarations should be just as simple as with the current C data
+interface.

Review Comment:
   I do that, just later in the prose rather than right here. But I can add that extra bit here if you think it's necessary.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [arrow] zeroshade commented on a diff in pull request #34972: GH-34971: [Format] Enhance C-Data API to support non-cpu cases

["zeroshade (via GitHub)" <gi...@apache.org>]

zeroshade commented on code in PR #34972:
URL: https://github.com/apache/arrow/pull/34972#discussion_r1204342449


##########
docs/source/format/CDeviceDataInterface.rst:
##########
@@ -0,0 +1,613 @@
+.. Licensed to the Apache Software Foundation (ASF) under one
+.. or more contributor license agreements.  See the NOTICE file
+.. distributed with this work for additional information
+.. regarding copyright ownership.  The ASF licenses this file
+.. to you under the Apache License, Version 2.0 (the
+.. "License"); you may not use this file except in compliance
+.. with the License.  You may obtain a copy of the License at
+
+..   http://www.apache.org/licenses/LICENSE-2.0
+
+.. Unless required by applicable law or agreed to in writing,
+.. software distributed under the License is distributed on an
+.. "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+.. KIND, either express or implied.  See the License for the
+.. specific language governing permissions and limitations
+.. under the License.
+
+.. highlight:: c
+
+.. _c-device-data-interface:
+
+=================================
+The Arrow C Device data interface
+=================================
+
+.. note:: The Arrow C Device Data Interface should be considered experimental
+
+Rationale
+=========
+
+The current :ref:`C Data Interface <c-data-interface>`, and most
+implementations of it, make the assumption that all data buffers provided
+are CPU buffers. Since Apache Arrow is designed to be a universal in-memory
+format for representing tabular ("columnar") data, there will be the desire
+to leverage this data on Non-CPU hardware such as GPUs. One example of such
+a case is the `RAPIDS cuDF library` which uses the Arrow memory format with

Review Comment:
   It does link to rapids, i'll update it to use the `:ref:` syntax but the link is at the bottom of the file.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [arrow] zeroshade commented on a diff in pull request #34972: GH-34971: [Format] Enhance C-Data API to support non-cpu cases

["zeroshade (via GitHub)" <gi...@apache.org>]

zeroshade commented on code in PR #34972:
URL: https://github.com/apache/arrow/pull/34972#discussion_r1161908085


##########
cpp/src/arrow/c/abi.h:
##########
@@ -65,6 +65,69 @@ struct ArrowArray {
 
 #endif  // ARROW_C_DATA_INTERFACE
 
+#ifndef ARROW_C_DEVICE_DATA_INTERFACE
+#define ARROW_C_DEVICE_DATA_INTERFACE
+
+// ArrowDeviceType is compatible with dlpack DLDeviceType for portability

Review Comment:
   comment formats updated to be docstrings in the same format as found in `bridge.h`



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [arrow] pitrou commented on a diff in pull request #34972: GH-34971: [Format] Enhance C-Data API to support non-cpu cases

["pitrou (via GitHub)" <gi...@apache.org>]

pitrou commented on code in PR #34972:
URL: https://github.com/apache/arrow/pull/34972#discussion_r1161990656


##########
cpp/src/arrow/c/abi.h:
##########
@@ -106,6 +212,98 @@ struct ArrowArrayStream {
 
 #endif  // ARROW_C_STREAM_INTERFACE
 
+#ifndef ARROW_C_DEVICE_STREAM_INTERFACE
+#define ARROW_C_DEVICE_STREAM_INTERFACE
+
+/// \brief Equivalent to ArrowArrayStream, but for ArrowDeviceArrays.
+///
+/// This stream is intended to provide a stream of data on a single
+/// device, if a producer wants data to be produced on multiple devices
+/// then multiple streams should be provided. One per device.
+struct ArrowDeviceArrayStream {
+  /// \brief The device that this stream produces data on.
+  ///
+  /// All ArrowDeviceArrays that are produced by this
+  /// stream should have the same device_type as set
+  /// here. The device_type needs to be provided here
+  /// so that consumers can provide the correct type
+  /// of queue_ptr when calling get_next.
+  ArrowDeviceType device_type;
+
+  /// \brief Callback to get the stream schema
+  /// (will be the same for all arrays in the stream).
+  ///
+  /// If successful, the ArrowSchema must be released independantly from the stream.
+  /// The schema should be accessible via CPU memory.
+  ///
+  /// \param[in] self The ArrowDeviceArrayStream object itself
+  /// \param[out] out C struct to export the schema to
+  /// \return 0 if successful, an `errno`-compatible error code otherwise.
+  int (*get_schema)(struct ArrowDeviceArrayStream* self, struct ArrowSchema* out);
+
+  /// \brief Callback to get the device id for the next array.
+  ///
+  /// This is necessary so that the proper/correct stream pointer can be provided
+  /// to get_next.
+  ///
+  /// The next call to `get_next` should provide an ArrowDeviceArray whose
+  /// device_id matches what is provided here, and whose device_type is the
+  /// same as the device_type member of this stream.

Review Comment:
   This is a weird API choice, all because you want the consumer to pass its CUDA stream of choice...



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [arrow] kkraus14 commented on a diff in pull request #34972: GH-34971: [Format] Enhance C-Data API to support non-cpu cases

["kkraus14 (via GitHub)" <gi...@apache.org>]

kkraus14 commented on code in PR #34972:
URL: https://github.com/apache/arrow/pull/34972#discussion_r1161977597


##########
cpp/src/arrow/c/abi.h:
##########
@@ -106,6 +212,98 @@ struct ArrowArrayStream {
 
 #endif  // ARROW_C_STREAM_INTERFACE
 
+#ifndef ARROW_C_DEVICE_STREAM_INTERFACE
+#define ARROW_C_DEVICE_STREAM_INTERFACE
+
+/// \brief Equivalent to ArrowArrayStream, but for ArrowDeviceArrays.
+///
+/// This stream is intended to provide a stream of data on a single
+/// device, if a producer wants data to be produced on multiple devices
+/// then multiple streams should be provided. One per device.
+struct ArrowDeviceArrayStream {
+  /// \brief The device that this stream produces data on.
+  ///
+  /// All ArrowDeviceArrays that are produced by this
+  /// stream should have the same device_type as set
+  /// here. The device_type needs to be provided here
+  /// so that consumers can provide the correct type
+  /// of queue_ptr when calling get_next.
+  ArrowDeviceType device_type;
+
+  /// \brief Callback to get the stream schema
+  /// (will be the same for all arrays in the stream).
+  ///
+  /// If successful, the ArrowSchema must be released independantly from the stream.
+  /// The schema should be accessible via CPU memory.
+  ///
+  /// \param[in] self The ArrowDeviceArrayStream object itself
+  /// \param[out] out C struct to export the schema to
+  /// \return 0 if successful, an `errno`-compatible error code otherwise.
+  int (*get_schema)(struct ArrowDeviceArrayStream* self, struct ArrowSchema* out);
+
+  /// \brief Callback to get the device id for the next array.
+  ///
+  /// This is necessary so that the proper/correct stream pointer can be provided
+  /// to get_next.
+  ///
+  /// The next call to `get_next` should provide an ArrowDeviceArray whose
+  /// device_id matches what is provided here, and whose device_type is the
+  /// same as the device_type member of this stream.
+  ///
+  /// \param[in] self The ArrowDeviceArrayStream object itself
+  /// \param[out] out_device_id Pointer to be populated with the device id, must not be
+  /// null \return 0 if successful, an `errno`-compatible error code otherwise.
+  int (*get_next_device_id)(struct ArrowDeviceArrayStream* self, int64_t* out_device_id);
+
+  /// \brief Callback to get the next array
+  ///
+  /// If there is no error and the returned array has been released, the stream
+  /// has ended. If successful, the ArrowArray must be released independently
+  /// from the stream.
+  ///
+  /// Because different frameworks use different types to represent this, we
+  /// accept a void* which should then be reinterpreted into whatever the
+  /// appropriate type is (e.g. cudaStream_t) for use by the producer.
+  ///
+  /// \param[in] self The ArrowDeviceArrayStream object itself
+  /// \param[in] queue_ptr The appropriate queue, stream, or
+  /// equivalent object for the device that the data is allocated on
+  /// to indicate where the consumer wants the data to be accessible.
+  /// If queue_ptr is NULL then the default stream (e.g. CUDA stream 0)
+  /// should be used to ensure that the memory is accessible from any stream.

Review Comment:
   @westonpace The idea here is that the consumer of the interface provides a queue to the producer and the producer is responsible for ensuring that the data is safe to consume on the provided queue.
   
   The reason for doing this instead of the producer returning a pointer to a queue that the data is safe to consume on is that frameworks generally manage these queues internally and don't have a mechanism to share a queue and control its lifetime over a boundary like this.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [arrow] pitrou commented on a diff in pull request #34972: GH-34971: [Format] Enhance C-Data API to support non-cpu cases

["pitrou (via GitHub)" <gi...@apache.org>]

pitrou commented on code in PR #34972:
URL: https://github.com/apache/arrow/pull/34972#discussion_r1162446351


##########
cpp/src/arrow/c/abi.h:
##########
@@ -106,6 +212,98 @@ struct ArrowArrayStream {
 
 #endif  // ARROW_C_STREAM_INTERFACE
 
+#ifndef ARROW_C_DEVICE_STREAM_INTERFACE
+#define ARROW_C_DEVICE_STREAM_INTERFACE
+
+/// \brief Equivalent to ArrowArrayStream, but for ArrowDeviceArrays.
+///
+/// This stream is intended to provide a stream of data on a single
+/// device, if a producer wants data to be produced on multiple devices
+/// then multiple streams should be provided. One per device.
+struct ArrowDeviceArrayStream {
+  /// \brief The device that this stream produces data on.
+  ///
+  /// All ArrowDeviceArrays that are produced by this
+  /// stream should have the same device_type as set
+  /// here. The device_type needs to be provided here
+  /// so that consumers can provide the correct type
+  /// of queue_ptr when calling get_next.
+  ArrowDeviceType device_type;
+
+  /// \brief Callback to get the stream schema
+  /// (will be the same for all arrays in the stream).
+  ///
+  /// If successful, the ArrowSchema must be released independantly from the stream.
+  /// The schema should be accessible via CPU memory.
+  ///
+  /// \param[in] self The ArrowDeviceArrayStream object itself
+  /// \param[out] out C struct to export the schema to
+  /// \return 0 if successful, an `errno`-compatible error code otherwise.
+  int (*get_schema)(struct ArrowDeviceArrayStream* self, struct ArrowSchema* out);
+
+  /// \brief Callback to get the device id for the next array.
+  ///
+  /// This is necessary so that the proper/correct stream pointer can be provided
+  /// to get_next.
+  ///
+  /// The next call to `get_next` should provide an ArrowDeviceArray whose
+  /// device_id matches what is provided here, and whose device_type is the
+  /// same as the device_type member of this stream.
+  ///
+  /// \param[in] self The ArrowDeviceArrayStream object itself
+  /// \param[out] out_device_id Pointer to be populated with the device id, must not be
+  /// null \return 0 if successful, an `errno`-compatible error code otherwise.
+  int (*get_next_device_id)(struct ArrowDeviceArrayStream* self, int64_t* out_device_id);
+
+  /// \brief Callback to get the next array
+  ///
+  /// If there is no error and the returned array has been released, the stream
+  /// has ended. If successful, the ArrowArray must be released independently
+  /// from the stream.
+  ///
+  /// Because different frameworks use different types to represent this, we
+  /// accept a void* which should then be reinterpreted into whatever the
+  /// appropriate type is (e.g. cudaStream_t) for use by the producer.
+  ///
+  /// \param[in] self The ArrowDeviceArrayStream object itself
+  /// \param[in] queue_ptr The appropriate queue, stream, or
+  /// equivalent object for the device that the data is allocated on
+  /// to indicate where the consumer wants the data to be accessible.
+  /// If queue_ptr is NULL then the default stream (e.g. CUDA stream 0)
+  /// should be used to ensure that the memory is accessible from any stream.

Review Comment:
   That doesn't make sense, does it? How is the consumer supposed to manage the stream's lifetime if "there isn't a mechanism that you could call in the release callback to cleanly control the lifetime"?



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [arrow] kkraus14 commented on pull request #34972: GH-34971: [Format] Enhance C-Data API to support non-cpu cases

["kkraus14 (via GitHub)" <gi...@apache.org>]

kkraus14 commented on PR #34972:
URL: https://github.com/apache/arrow/pull/34972#issuecomment-1505405339

   > Ok, who is supposed to be the consumer of the C Device Data Interface? I would expect it to be an Arrow implementation, or an Arrow-compatible library.
   
   I agree it would be primarily other libraries who want to exchange Arrow data. Users would likely initiate the exchange between the two libraries and the C Device Data Interface would be an implementation detail of how that exchange would work.
   
   I do think there would be some users who want to directly interact with the C Device Data Interface structs though as well, as I had seen similar with `__cuda_array_interface__` and dlpack.
   
   > Realistically they probably already deal with CUDA streams if they support CUDA?
   
   Not necessarily. There's a bunch of CUDA code out in the wild that just uses the default stream implicitly. For example, most numba and cupy cuda code I've seen doesn't use streams. Numba and CuPy internally both support using CUDA Streams but I don't think it would make sense for either of them to directly support the Arrow C Device Data Interface since it doesn't match their memory layout. So that would become a library author or user's responsibility to handle it and CUDA Streams / Events may be a new concept to them.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [arrow] pitrou commented on a diff in pull request #34972: GH-34971: [Format] Enhance C-Data API to support non-cpu cases

["pitrou (via GitHub)" <gi...@apache.org>]

pitrou commented on code in PR #34972:
URL: https://github.com/apache/arrow/pull/34972#discussion_r1164262107


##########
cpp/src/arrow/c/abi.h:
##########
@@ -106,6 +212,98 @@ struct ArrowArrayStream {
 
 #endif  // ARROW_C_STREAM_INTERFACE
 
+#ifndef ARROW_C_DEVICE_STREAM_INTERFACE
+#define ARROW_C_DEVICE_STREAM_INTERFACE
+
+/// \brief Equivalent to ArrowArrayStream, but for ArrowDeviceArrays.
+///
+/// This stream is intended to provide a stream of data on a single
+/// device, if a producer wants data to be produced on multiple devices
+/// then multiple streams should be provided. One per device.
+struct ArrowDeviceArrayStream {
+  /// \brief The device that this stream produces data on.
+  ///
+  /// All ArrowDeviceArrays that are produced by this
+  /// stream should have the same device_type as set
+  /// here. The device_type needs to be provided here
+  /// so that consumers can provide the correct type
+  /// of queue_ptr when calling get_next.
+  ArrowDeviceType device_type;
+
+  /// \brief Callback to get the stream schema
+  /// (will be the same for all arrays in the stream).
+  ///
+  /// If successful, the ArrowSchema must be released independantly from the stream.
+  /// The schema should be accessible via CPU memory.
+  ///
+  /// \param[in] self The ArrowDeviceArrayStream object itself
+  /// \param[out] out C struct to export the schema to
+  /// \return 0 if successful, an `errno`-compatible error code otherwise.
+  int (*get_schema)(struct ArrowDeviceArrayStream* self, struct ArrowSchema* out);
+
+  /// \brief Callback to get the device id for the next array.
+  ///
+  /// This is necessary so that the proper/correct stream pointer can be provided
+  /// to get_next.
+  ///
+  /// The next call to `get_next` should provide an ArrowDeviceArray whose
+  /// device_id matches what is provided here, and whose device_type is the
+  /// same as the device_type member of this stream.
+  ///
+  /// \param[in] self The ArrowDeviceArrayStream object itself
+  /// \param[out] out_device_id Pointer to be populated with the device id, must not be
+  /// null \return 0 if successful, an `errno`-compatible error code otherwise.
+  int (*get_next_device_id)(struct ArrowDeviceArrayStream* self, int64_t* out_device_id);
+
+  /// \brief Callback to get the next array
+  ///
+  /// If there is no error and the returned array has been released, the stream
+  /// has ended. If successful, the ArrowArray must be released independently
+  /// from the stream.
+  ///
+  /// Because different frameworks use different types to represent this, we
+  /// accept a void* which should then be reinterpreted into whatever the
+  /// appropriate type is (e.g. cudaStream_t) for use by the producer.
+  ///
+  /// \param[in] self The ArrowDeviceArrayStream object itself
+  /// \param[in] queue_ptr The appropriate queue, stream, or
+  /// equivalent object for the device that the data is allocated on
+  /// to indicate where the consumer wants the data to be accessible.
+  /// If queue_ptr is NULL then the default stream (e.g. CUDA stream 0)
+  /// should be used to ensure that the memory is accessible from any stream.

Review Comment:
   > Tensorflow can handle the consumer provided stream internally as part of it's implementation in producing dlpack or Arrow C Data Interface structs.
   
   Hmm, wait :-) If it's Tensorflow implementing the C Data Interface (rather than a third-party using Tensorflow as a backend), they can also implement the "producer-supplied CUDA stream" semantics, since they can access their own internals.
   



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [arrow] aregm commented on a diff in pull request #34972: GH-34971: [Format] Enhance C-Data API to support non-cpu cases

["aregm (via GitHub)" <gi...@apache.org>]

aregm commented on code in PR #34972:
URL: https://github.com/apache/arrow/pull/34972#discussion_r1164473963


##########
cpp/src/arrow/c/abi.h:
##########
@@ -65,6 +65,69 @@ struct ArrowArray {
 
 #endif  // ARROW_C_DATA_INTERFACE
 
+#ifndef ARROW_C_DEVICE_DATA_INTERFACE
+#define ARROW_C_DEVICE_DATA_INTERFACE
+
+// ArrowDeviceType is compatible with dlpack DLDeviceType for portability
+// it uses the same values for each enum as the equivalent kDL<type> from dlpack.h
+#ifdef __cplusplus
+typedef enum : int32_t {
+#else
+typedef enum {
+#endif
+  // CPU device, same as using ArrowArray directly
+  kArrowCPU = 1,
+  // CUDA GPU Device
+  kArrowCUDA = 2,
+  // Pinned CUDA CPU memory by cudaMallocHost
+  kArrowCUDAHost = 3,
+  // OpenCL Device
+  kArrowOpenCL = 4,
+  // Vulkan buffer for next-gen graphics
+  kArrowVulkan = 7,
+  // Metal for Apple GPU
+  kArrowMetal = 8,
+  // Verilog simulator buffer
+  kArrowVPI = 9,
+  // ROCm GPUs for AMD GPUs
+  kArrowROCM = 10,
+  // Pinned ROCm CPU memory allocated by hipMallocHost
+  kArrowROCMHost = 11,
+  // Reserved for extension
+  // used to quickly test extension devices,
+  // semantics can differ based on the implementation
+  kArrowExtDev = 12,
+  // CUDA managed/unified memory allocated by cudaMallocManaged
+  kArrowCUDAManaged = 13,
+  // unified shared memory allocated on a oneAPI non-partitioned
+  // device. call to oneAPI runtime is required to determine the
+  // device type, the USM allocation type and the sycl context it
+  // is bound to
+  kArrowOneAPI = 14,
+  // GPU support for next-gen WebGPU standard
+  kArrowWebGPU = 15,
+  // Qualcomm Hexagon DSP
+  kArrowHexagon = 16,
+} ArrowDeviceType;
+
+struct ArrowDeviceArray {
+  // the private_date and release callback of the arrow array
+  // should contain any necessary information and structures
+  // related to freeing the array according to the device it
+  // is allocated on, rather than having a separate release
+  // callback embedded here.
+  struct ArrowArray* array;
+  int device_id;

Review Comment:
   One remark, though - the assumption is that Arrow will not go beyond a single host, anyone who decided to go low-level knows how to map [UUID](https://spec.oneapi.io/level-zero/latest/core/api.html#ze-uuid-t) to host/device. But with the upcoming CXL capability, this can be a problem. Let's leave it int64 and address the issue when it comes. So +1.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org