spark sql in-clause problem

[onmstester onmstester <on...@zoho.com>]

I'm reading from this table in cassandra:

Table mytable (
Integer Key,

Integer X,

Interger Y



Using:

sparkSqlContext.sql(select * from mytable where key = 1 and (X,Y) in ((1,2),(3,4)))



Encountered error: 



StructType(StructField((X,IntegerType,true),StructField((Y,IntegerType,true)) != StructType(StructField((X,IntegerType,false),StructField((Y,IntegerType,false))





Sent using Zoho Mail

RE: spark sql in-clause problem

[Shiva Prashanth Vallabhaneni <sh...@tavant.com>]

Assuming the list of values in the “IN” clause is small, you could try using

sparkSqlContext.sql(select * from mytable where key = 1 and ( (X,Y) = (1,2) OR (X,Y) = (3,4) )

Another solution could be to load the possible values for X & Y into a table and then using this table in the sub-query;

Table coordinates (
Integer X,
Integer Y
)

sparkSqlContext.sql(select * from mytable where key = 1 and (X,Y) IN (select X, Y from coordinates))

From: onmstester onmstester <on...@zoho.com>
Sent: Wednesday, May 23, 2018 10:33 AM
To: user <us...@spark.apache.org>
Subject: spark sql in-clause problem

I'm reading from this table in cassandra:
Table mytable (
Integer Key,
Integer X,
Interger Y

Using:
sparkSqlContext.sql(select * from mytable where key = 1 and (X,Y) in ((1,2),(3,4)))

Encountered error:

StructType(StructField((X,IntegerType,true),StructField((Y,IntegerType,true)) != StructType(StructField((X,IntegerType,false),StructField((Y,IntegerType,false))



Sent using Zoho Mail<https://apac01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.zoho.com%2Fmail%2F&data=01%7C01%7Cshiva.vallabhaneni%40tavant.com%7C265fec50a27a40bac0fd08d5c06a75be%7Cc6c1e9da5d0c4f8f9a023c67206efbd6%7C0&sdata=J1X8iT4xh31pLX0VHPUmFsJTQCDXXskxI9oACy7IhnY%3D&reserved=0>


________________________________
Any comments or statements made in this email are not necessarily those of Tavant Technologies. The information transmitted is intended only for the person or entity to which it is addressed and may contain confidential and/or privileged material. If you have received this in error, please contact the sender and delete the material from any computer. All emails sent from or to Tavant Technologies may be subject to our monitoring procedures.