You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@spark.apache.org by Conconscious <co...@gmail.com> on 2017/07/01 10:54:00 UTC
json in Cassandra to RDDs
Hi list,
I'm using Cassandra with only 2 fields (id, json).
I'm using Spark to query the json. Until now I can use a json file and
query that file, but Cassandra and RDDs of the json field not yet.
sc = spark.sparkContext
path = "/home/me/red50k.json"
redirectsDF = spark.read.json(path)
redirectsDF.createOrReplaceTempView("red")
result = spark.sql("select idv from red where idv = '9'")
result.show()
val conf = new SparkConf(true)
.set("spark.cassandra.connection.host", "192.168.1.74")
.set("spark.cassandra.auth.username", "cassandra")
.set("spark.cassandra.auth.password", "cassandra")
val sc = new SparkContext("spark://192.168.1.74:7077", "test", conf)
val table = sc.cassandraTable("test", "ttable")
println(ttable.count)
Some help please to join both things. Scala or Python code for me it's ok.
Thanks in advance.
Cheers.
---------------------------------------------------------------------
To unsubscribe e-mail: user-unsubscribe@spark.apache.org
Re: json in Cassandra to RDDs
Posted by ayan guha <gu...@gmail.com>.
Hi
If you are asking how to parse the json column from Cassandra, I would
suggest you to look into from_json function. It would help you to parse a
json field, given you know the schema upfront.
On Sat, Jul 1, 2017 at 8:54 PM, Conconscious <co...@gmail.com> wrote:
> Hi list,
>
> I'm using Cassandra with only 2 fields (id, json).
> I'm using Spark to query the json. Until now I can use a json file and
> query that file, but Cassandra and RDDs of the json field not yet.
>
> sc = spark.sparkContext
> path = "/home/me/red50k.json"
> redirectsDF = spark.read.json(path)
> redirectsDF.createOrReplaceTempView("red")
> result = spark.sql("select idv from red where idv = '9'")
> result.show()
>
> val conf = new SparkConf(true)
> .set("spark.cassandra.connection.host", "192.168.1.74")
> .set("spark.cassandra.auth.username", "cassandra")
> .set("spark.cassandra.auth.password", "cassandra")
> val sc = new SparkContext("spark://192.168.1.74:7077", "test", conf)
> val table = sc.cassandraTable("test", "ttable")
> println(ttable.count)
>
> Some help please to join both things. Scala or Python code for me it's ok.
> Thanks in advance.
> Cheers.
>
>
> ---------------------------------------------------------------------
> To unsubscribe e-mail: user-unsubscribe@spark.apache.org
>
>
--
Best Regards,
Ayan Guha