Spark on cassandra

[Prem Yadav <ip...@gmail.com>]

Hi,
Is it better to use Spark APIs to do join on cassandra tables or should we
use SPARK-SQL.
We have been struggling with SPARK-SQL as we need to do multiple large
table joins and there is always failure.

I tried to do joins using the API like this:
val join1 =
sc.cassandraTable("Keyspace1","table1").joinWithCassandraTable("keyspace1","table2").on(SomeColumns("column1"))

however, we need to join multiple table from multiple keyspaces. How can we
do that?


Thanks,
Prem

Re: Spark on cassandra

[Ravi <ra...@gmail.com>]

I did join on single big table and it's working fine using code you showed
below.

Can we do table join on non partition key? Or not a primary key column ?



Thanks,
Ravi
On Thu, Nov 12, 2015 at 5:41 AM DuyHai Doan <do...@gmail.com> wrote:

> Hello Prem
>
> I believe it's better to ask your question on the ML of the Spark
> Cassandra connector:
> http://groups.google.com/a/lists.datastax.com/forum/#!forum/spark-connector-user
>
> Second "we need to join multiple table from multiple keyspaces. How can
> we do that?", the response is given in your example, you pass the keyspace
> name as parameter for each operation with Cassandra.
>
> On Thu, Nov 12, 2015 at 2:12 PM, Prem Yadav <ip...@gmail.com> wrote:
>
>> Hi,
>> Is it better to use Spark APIs to do join on cassandra tables or should
>> we use SPARK-SQL.
>> We have been struggling with SPARK-SQL as we need to do multiple large
>> table joins and there is always failure.
>>
>> I tried to do joins using the API like this:
>> val join1 =
>> sc.cassandraTable("Keyspace1","table1").joinWithCassandraTable("keyspace1","table2").on(SomeColumns("column1"))
>>
>> however, we need to join multiple table from multiple keyspaces. How can
>> we do that?
>>
>>
>> Thanks,
>> Prem
>>
>
>

Re: Spark on cassandra

[DuyHai Doan <do...@gmail.com>]

Hello Prem

I believe it's better to ask your question on the ML of the Spark Cassandra
connector:
http://groups.google.com/a/lists.datastax.com/forum/#!forum/spark-connector-user

Second "we need to join multiple table from multiple keyspaces. How can we
do that?", the response is given in your example, you pass the keyspace
name as parameter for each operation with Cassandra.

On Thu, Nov 12, 2015 at 2:12 PM, Prem Yadav <ip...@gmail.com> wrote:

> Hi,
> Is it better to use Spark APIs to do join on cassandra tables or should we
> use SPARK-SQL.
> We have been struggling with SPARK-SQL as we need to do multiple large
> table joins and there is always failure.
>
> I tried to do joins using the API like this:
> val join1 =
> sc.cassandraTable("Keyspace1","table1").joinWithCassandraTable("keyspace1","table2").on(SomeColumns("column1"))
>
> however, we need to join multiple table from multiple keyspaces. How can
> we do that?
>
>
> Thanks,
> Prem
>