You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@arrow.apache.org by Arvin <ar...@gmail.com> on 2019/01/08 05:39:22 UTC

Arrow connector for computing engine

Hey all,
       I want to let apache flink to ingest arrow format data, it seems
that flink doesn't support arrow currently. Is there arrow connector for
apache flink, so that I can pass arrow vector data to flink through it? And
I see that spark's usage of arrow is restricted to pyspark. Is there any
computing engine support arrow natively? And is there any plan to integrate
arrow with flink, spark or presto?

Arvin

Re: Arrow connector for computing engine

Posted by Wes McKinney <we...@gmail.com>.
hi Arvin,

It is up to developers who work with these other systems to develop
integration with Arrow. It seems inevitable that streaming systems
(Spark, Flink, Kafka) and database systems (like Presto) will offer
some level of Arrow support, as it will accelerate many use cases. It
is hard to say when the work will get done though. I would suggest
describing your use cases to those communities to see if there are any
contributors who wish to get involved.

best
Wes

On Mon, Jan 7, 2019 at 11:39 PM Arvin <ar...@gmail.com> wrote:
>
> Hey all,
>        I want to let apache flink to ingest arrow format data, it seems that flink doesn't support arrow currently. Is there arrow connector for apache flink, so that I can pass arrow vector data to flink through it? And I see that spark's usage of arrow is restricted to pyspark. Is there any computing engine support arrow natively? And is there any plan to integrate arrow with flink, spark or presto?
>
> Arvin