Build PyArrow with GPU support

[Wenlei Xie <we...@gmail.com>]

Hello Apache Arrow team,

I am trying to use Arrow on GPU device on PyArrow (6.0 installed via pip).
But it errored with

> ModuleNotFoundError: No module named 'pyarrow._cuda'

I assume the pip package doesn't enable CUDA feature at build time. I am
wondering how to build pyarrow with GPU support from source? -- I only
found Building Arrow C++ guide:
https://arrow.apache.org/docs/developers/cpp/building.html

Also wondering if Arrow Compute functions are supported on GPU-buffered
Array?

Thanks!

Re: Build PyArrow with GPU support

[Wes McKinney <we...@gmail.com>]

hi Wenlei,

We haven't made any releases with the CUDA capabilities enabled, but
we have tested it periodically so turning it on should be a matter of
setting the flags when building the C++ and Python libraries. It would
be nice to be able to release this functionality in a separate wheel
but there would be some engineering to do to make that possible.

The compute functions only work with CPU buffers for now.

Thanks,
Wes

On Fri, Nov 5, 2021 at 11:03 AM Wenlei Xie <we...@gmail.com> wrote:
>
> Hello Apache Arrow team,
>
> I am trying to use Arrow on GPU device on PyArrow (6.0 installed via pip). But it errored with
>
> > ModuleNotFoundError: No module named 'pyarrow._cuda'
>
> I assume the pip package doesn't enable CUDA feature at build time. I am wondering how to build pyarrow with GPU support from source? -- I only found Building Arrow C++ guide: https://arrow.apache.org/docs/developers/cpp/building.html
>
> Also wondering if Arrow Compute functions are supported on GPU-buffered Array?
>
> Thanks!
>
>