CUDA context management questions

[Pearu Peterson <pe...@quansight.com>]

Hi,

While researching CUDA context management code in arrow,

https://github.com/apache/arrow/blob/master/cpp/src/arrow/gpu/cuda_context.cc
,
I noticed the following:
CudaContextImpl has a member own_context_ that seems not being initialized
anywhere. The member own_context_ determines whether the context will be
destroyed or not (see CudaContextImpl Close method).
Is this a bug or am I missing something?

The reason why I was looking at this, was establishing a interoperability
between numba.cuda and pyarrow.cuda. The idea is to be able to use
numba.cuda defined context within pyarrow.cuda somehow and the own_context_
member seems to be relevant for that. However, there seems to be no API to
use it.
Is this work-in-progress?

Finally, what do you think, is it realistic to obtain interoperability
between numba.cuda and pyarrow.cuda? I am able to feed pyarrow.cuda buffer
data to numba jitted functions and it works only up to some point where the
context managers seem to interfere with each other that leads to CUDA
errors...

Best regards,
Pearu

Re: CUDA context management questions

[Wes McKinney <we...@gmail.com>]

hi Pearu,


On Fri, Oct 5, 2018 at 12:57 PM Pearu Peterson
<pe...@quansight.com> wrote:
>
> Hi,
>
> While researching CUDA context management code in arrow,
>
> https://github.com/apache/arrow/blob/master/cpp/src/arrow/gpu/cuda_context.cc
> ,
> I noticed the following:
> CudaContextImpl has a member own_context_ that seems not being initialized
> anywhere. The member own_context_ determines whether the context will be
> destroyed or not (see CudaContextImpl Close method).
> Is this a bug or am I missing something?
>
> The reason why I was looking at this, was establishing a interoperability
> between numba.cuda and pyarrow.cuda. The idea is to be able to use
> numba.cuda defined context within pyarrow.cuda somehow and the own_context_
> member seems to be relevant for that. However, there seems to be no API to
> use it.
> Is this work-in-progress?

This was a placeholder for
https://issues.apache.org/jira/browse/ARROW-1423. The objective
exactly was to be able to bridge into contexts created by others, like
Numba. So I've just been waiting for someone with the need to take up
this task

>
> Finally, what do you think, is it realistic to obtain interoperability
> between numba.cuda and pyarrow.cuda? I am able to feed pyarrow.cuda buffer
> data to numba jitted functions and it works only up to some point where the
> context managers seem to interfere with each other that leads to CUDA
> errors...

Yes, it's realistic. Let me know if you need some help

Thanks,
Wes

>
> Best regards,
> Pearu