You are viewing a plain text version of this content. The canonical link for it is here.

Posted to user@arrow.apache.org by Hokam Singh Chauhan <ho...@gmail.com> on 2022/10/31 06:40:36 UTC

How to share spark dataset from one spark session to another spark session via apache arrow?

Hi Team,

We are working on one project where we need to share the incremental data
(which is coming from streaming source say Kafka) to a process, that is
running as separate spark app(pyspark) with spark context(session).

How this can be achieved via Apache arrow ?



Thanks,

Hokam