[Julia] C Data Interface for Julia

[Samay Kapadia <sa...@gmail.com>]

Hi!

I would like to implement the C Data Interface for Julia but I'm getting a
bit stuck.

My goal is to implement the py_to_jl function for any Arrow Table, but I'm
at a loss for where I can access the C Data Interface from an Arrow table
in Python..

For example, I would run this in python

import pyarrow.parquet as pq
data = pq.read_table("path/to/file.snappy.parquet")

And from this point I would like to access the variable "data" through the
C Data Interface but I can't find anything on the official docs.

The Julia code looks very similar to reach this point:

using PyCall
pq = pyimport("pyarrow.parquet")
data = pq.read_table("path/to/file.snappy.parquet")

Any guidance would be much appreciated!

Cheers

Re: [Julia] C Data Interface for Julia

[Jorge Cardoso Leitão <jo...@gmail.com>]

Hi Samay,

That is great to know!

The interfaces are not documented, but they are `_import_from_c` and
`_export_to_c`. Both expect two pointers, one to the array and the other to
the schema (that naturally must have a C representation).

Tests within c++ are available here
<https://github.com/apache/arrow/blob/97879eb970bac52d93d2247200b9ca7acf6f3f93/python/pyarrow/tests/test_cffi.py>
Tests for Rust are available here
<https://github.com/apache/arrow/blob/4327e4b3591ad233bb97b9269a0c8c2a852b7bb5/rust/arrow-pyarrow-integration-testing/src/lib.rs>

Good luck and feel free to ping if you need assistance.

Best,
Jorge




On Mon, Feb 15, 2021 at 12:47 AM Samay Kapadia <sa...@gmail.com>
wrote:

> Hi!
>
> I would like to implement the C Data Interface for Julia but I'm getting a
> bit stuck.
>
> My goal is to implement the py_to_jl function for any Arrow Table, but I'm
> at a loss for where I can access the C Data Interface from an Arrow table
> in Python..
>
> For example, I would run this in python
>
> import pyarrow.parquet as pq
> data = pq.read_table("path/to/file.snappy.parquet")
>
> And from this point I would like to access the variable "data" through the
> C Data Interface but I can't find anything on the official docs.
>
> The Julia code looks very similar to reach this point:
>
> using PyCall
> pq = pyimport("pyarrow.parquet")
> data = pq.read_table("path/to/file.snappy.parquet")
>
> Any guidance would be much appreciated!
>
> Cheers
>