You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@spark.apache.org by Amit Sharma <re...@gmail.com> on 2020/07/28 11:23:49 UTC

how to copy from one cassandra cluster to another

Hi, I have table A in the cassandra cluster  cluster -1  in one data
center. I have table B in cluster -2 in another data center. I want to copy
the data from one cluster to another using spark. I faced the problem that
I can not create two spark sessions as we need spark sessions per cluster.
Please let me know if there is any way to use spark batch job to copy data
among two cassandra clusters.


Thanks
Amit

Re: how to copy from one cassandra cluster to another

Posted by Russell Spitzer <ru...@gmail.com>.
You do not need one spark session per cluster.

Spark SQL with Datasource v1
http://www.russellspitzer.com/2016/02/16/Multiple-Clusters-SparkSql-Cassandra/

DatasourceV2

Would require making two catalog references then copying between them


https://github.com/datastax/spark-cassandra-connector/blob/master/doc/1_connecting.md#example-of-reading-from-one-cluster-and-writing-to-another



On Tue, Jul 28, 2020, 6:26 AM Amit Sharma <re...@gmail.com> wrote:

> Hi, I have table A in the cassandra cluster  cluster -1  in one data
> center. I have table B in cluster -2 in another data center. I want to copy
> the data from one cluster to another using spark. I faced the problem that
> I can not create two spark sessions as we need spark sessions per cluster.
> Please let me know if there is any way to use spark batch job to copy data
> among two cassandra clusters.
>
>
> Thanks
> Amit
>