You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@spark.apache.org by lmk <la...@gmail.com> on 2014/09/11 11:37:59 UTC

SchemaRDD saveToCassandra

Hi,
My requirement is to extract certain fields from json files, run queries on
them and save the result to cassandra.
I was able to parse json , filter the result and save the rdd(regular) to
cassandra.

Now, when I try to read the json file through sqlContext , execute some
queries on the same and then save the SchemaRDD to cassandra using
saveToCassandra function, I am getting the following error:

java.lang.NoSuchMethodException: Cannot resolve any suitable constructor for
class org.apache.spark.sql.catalyst.expressions.Row

Pls let me know if a spark SchemaRDD can be directly saved to cassandra just
like the regular rdd?

If that is not possible, is there any way to convert the schema RDD to a
regular RDD ?

Please advise.

Regards,
lmk



--
View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/SchemaRDD-saveToCassandra-tp13951.html
Sent from the Apache Spark User List mailing list archive at Nabble.com.

---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@spark.apache.org
For additional commands, e-mail: user-help@spark.apache.org

Re: SchemaRDD saveToCassandra

Posted by lmk <la...@gmail.com>.
Hi Michael,
Please correct me if I am wrong. The error seems to originate from spark
only. Please have a look at the stack trace of the error which is as
follows:

[error] (run-main-0) java.lang.NoSuchMethodException: Cannot resolve any
suitable constructor for class org.apache.spark.sql.catalyst.expressions.Row
java.lang.NoSuchMethodException: Cannot resolve any suitable constructor for
class org.apache.spark.sql.catalyst.expressions.Row
        at
com.datastax.spark.connector.rdd.reader.AnyObjectFactory$$anonfun$resolveConstructor$2.apply(AnyObjectFactory.scala:134)
        at
com.datastax.spark.connector.rdd.reader.AnyObjectFactory$$anonfun$resolveConstructor$2.apply(AnyObjectFactory.scala:134)
        at scala.util.Try.getOrElse(Try.scala:77)
        at
com.datastax.spark.connector.rdd.reader.AnyObjectFactory$.resolveConstructor(AnyObjectFactory.scala:133)
        at
com.datastax.spark.connector.mapper.ReflectionColumnMapper.columnMap(ReflectionColumnMapper.scala:46)
        at
com.datastax.spark.connector.writer.DefaultRowWriter.<init>(DefaultRowWriter.scala:20)
        at
com.datastax.spark.connector.writer.DefaultRowWriter$$anon$2.rowWriter(DefaultRowWriter.scala:109)
        at
com.datastax.spark.connector.writer.DefaultRowWriter$$anon$2.rowWriter(DefaultRowWriter.scala:107)
        at
com.datastax.spark.connector.writer.TableWriter$.apply(TableWriter.scala:171)
        at
com.datastax.spark.connector.RDDFunctions.tableWriter(RDDFunctions.scala:76)
        at
com.datastax.spark.connector.RDDFunctions.saveToCassandra(RDDFunctions.scala:56)
        at ScalaSparkCassandra$.main(ScalaSparkCassandra.scala:66)
        at ScalaSparkCassandra.main(ScalaSparkCassandra.scala)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
        at java.lang.reflect.Method.invoke(Unknown Source)

Please clarify.



--
View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/SchemaRDD-saveToCassandra-tp13951p14340.html
Sent from the Apache Spark User List mailing list archive at Nabble.com.

---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@spark.apache.org
For additional commands, e-mail: user-help@spark.apache.org

Re: SchemaRDD saveToCassandra

Posted by Michael Armbrust <mi...@databricks.com>.
This might be a better question to ask on the cassandra mailing list as I
believe that is where the exception is coming from.

On Thu, Sep 11, 2014 at 2:37 AM, lmk <la...@gmail.com>
wrote:

> Hi,
> My requirement is to extract certain fields from json files, run queries on
> them and save the result to cassandra.
> I was able to parse json , filter the result and save the rdd(regular) to
> cassandra.
>
> Now, when I try to read the json file through sqlContext , execute some
> queries on the same and then save the SchemaRDD to cassandra using
> saveToCassandra function, I am getting the following error:
>
> java.lang.NoSuchMethodException: Cannot resolve any suitable constructor
> for
> class org.apache.spark.sql.catalyst.expressions.Row
>
> Pls let me know if a spark SchemaRDD can be directly saved to cassandra
> just
> like the regular rdd?
>
> If that is not possible, is there any way to convert the schema RDD to a
> regular RDD ?
>
> Please advise.
>
> Regards,
> lmk
>
>
>
> --
> View this message in context:
> http://apache-spark-user-list.1001560.n3.nabble.com/SchemaRDD-saveToCassandra-tp13951.html
> Sent from the Apache Spark User List mailing list archive at Nabble.com.
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: user-unsubscribe@spark.apache.org
> For additional commands, e-mail: user-help@spark.apache.org
>
>