You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@spark.apache.org by Amit Sharma <re...@gmail.com> on 2020/07/28 11:23:49 UTC
how to copy from one cassandra cluster to another
Hi, I have table A in the cassandra cluster cluster -1 in one data
center. I have table B in cluster -2 in another data center. I want to copy
the data from one cluster to another using spark. I faced the problem that
I can not create two spark sessions as we need spark sessions per cluster.
Please let me know if there is any way to use spark batch job to copy data
among two cassandra clusters.
Thanks
Amit
Re: how to copy from one cassandra cluster to another
Posted by Russell Spitzer <ru...@gmail.com>.
You do not need one spark session per cluster.
Spark SQL with Datasource v1
http://www.russellspitzer.com/2016/02/16/Multiple-Clusters-SparkSql-Cassandra/
DatasourceV2
Would require making two catalog references then copying between them
https://github.com/datastax/spark-cassandra-connector/blob/master/doc/1_connecting.md#example-of-reading-from-one-cluster-and-writing-to-another
On Tue, Jul 28, 2020, 6:26 AM Amit Sharma <re...@gmail.com> wrote:
> Hi, I have table A in the cassandra cluster cluster -1 in one data
> center. I have table B in cluster -2 in another data center. I want to copy
> the data from one cluster to another using spark. I faced the problem that
> I can not create two spark sessions as we need spark sessions per cluster.
> Please let me know if there is any way to use spark batch job to copy data
> among two cassandra clusters.
>
>
> Thanks
> Amit
>