SqlContext registerTempTable issue

[Nihal Bhagchandani <ni...@yahoo.com>]

Hi Gurus,
I am trying to access some json data using sqlContext.jsonFile in zeppelin, following code execute without any error:
import sys.process._

val sqlCon = new org.apache.spark.sql.SQLContext(sc)
val jfile = sqlCon.jsonFile(s"file:///usr/local/src/knoldus/projects/scaladay_data/scalaDays2015Amsterdam_tweets.json")

import sqlContext.implicits._
jfile.registerTempTable("jTable01")

output :
import sys.process._sqlCon: org.apache.spark.sql.SQLContext = org.apache.spark.sql.SQLContext@7034473jfile: org.apache.spark.sql.DataFrame = [_id: struct<$oid:string>, content: string, hashtags: array<string>, score: struct<negative:bigint,neutral:bigint,positive:bigint>, session: string, timestamp: bigint, tweetId: bigint, username: string]import sqlContext.implicits._
next I verify the table name which I just registeredsqlCon.tableNames().foreach(println)
output :
jTable01 

but when I try to run following I get an error:
%sql
select * from jTable01

output :
no such table jTable01; line 1 pos 14
at the same time when I run the tutorial example for "bank" it works....the only difference I could make out is that in bank tutorial we are using sc.textFile but in my case I wanted to use sqlContext.jsonFile.
could you please provide any guidance as how I can fix this issue?
RegardsNihal

Re: SqlContext registerTempTable issue

[moon soo Lee <mo...@apache.org>]

Hi,

I think if you remove following line

val sqlCon = new org.apache.spark.sql.SQLContext(sc)

and use sqlContext (which is automatically created by Zeppelin)
then i think i'll work.

Hope this helps.
moon


On Sat, Jun 13, 2015 at 6:00 AM Nihal Bhagchandani <
nihal_bhagchandani@yahoo.com> wrote:

> Hi Gurus,
>
> I am trying to access some json data using sqlContext.jsonFile in
> zeppelin, following code execute without any error:
>
> import sys.process._
>
> val sqlCon = new org.apache.spark.sql.SQLContext(sc)
> val jfile =
> sqlCon.jsonFile(s"file:///usr/local/src/knoldus/projects/scaladay_data/scalaDays2015Amsterdam_tweets.json")
>
> import sqlContext.implicits._
> jfile.registerTempTable("jTable01")
>
> output :
> import sys.process._ sqlCon: org.apache.spark.sql.SQLContext =
> org.apache.spark.sql.SQLContext@7034473 jfile:
> org.apache.spark.sql.DataFrame = [_id: struct<$oid:string>, content:
> string, hashtags: array<string>, score:
> struct<negative:bigint,neutral:bigint,positive:bigint>, session: string,
> timestamp: bigint, tweetId: bigint, username: string] import
> sqlContext.implicits._
>
> next I verify the table name which I just registered
> sqlCon.tableNames().foreach(println)
>
> output :
> jTable01
>
> but when I try to run following I get an error:
>
> %sql
> select * from jTable01
>
> output :
> no such table jTable01; line 1 pos 14
>
> at the same time when I run the tutorial example for "bank" it works....
> the only difference I could make out is that in bank tutorial we are using
> sc.textFile but in my case I wanted to use sqlContext.jsonFile.
>
> could you please provide any guidance as how I can fix this issue?
>
> Regards
> Nihal
>
>