You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@spark.apache.org by IT CTO <go...@gmail.com> on 2015/03/23 13:25:42 UTC
registerTempTable is not a member of RDD on spark 1.2?
Hi,
I am running spark when I use sc.version I get 1.2 but when I call
registerTempTable("MyTable") I get error saying registedTempTable is not a
member of RDD
Why?
--
Eran | CTO
Re: registerTempTable is not a member of RDD on spark 1.2?
Posted by IT CTO <go...@gmail.com>.
Yes!
Any reason this happen in my environment and not in any sample code I found?
Should I fix something in the path or env?
Eran
On Mon, Mar 23, 2015 at 3:50 PM, Ted Yu <yu...@gmail.com> wrote:
> Have you tried adding the following ?
>
> import org.apache.spark.sql.SQLContext
>
> Cheers
>
> On Mon, Mar 23, 2015 at 6:45 AM, IT CTO <go...@gmail.com> wrote:
>
>> Thanks.
>> I am new to the environment and running cloudera CDH5.3 with spark in it.
>>
>> apparently when running in spark-shell this command val sqlContext = new
>> SQLContext(sc)
>> I am failing with the not found type SQLContext
>>
>> Any idea why?
>>
>> On Mon, Mar 23, 2015 at 3:05 PM, Dean Wampler <de...@gmail.com>
>> wrote:
>>
>>> In 1.2 it's a member of SchemaRDD and it becomes available on RDD
>>> (through the "type class" mechanism) when you add a SQLContext, like so.
>>>
>>> val sqlContext = new SQLContext(sc)import sqlContext._
>>>
>>>
>>> In 1.3, the method has moved to the new DataFrame type.
>>>
>>> Dean Wampler, Ph.D.
>>> Author: Programming Scala, 2nd Edition
>>> <http://shop.oreilly.com/product/0636920033073.do> (O'Reilly)
>>> Typesafe <http://typesafe.com>
>>> @deanwampler <http://twitter.com/deanwampler>
>>> http://polyglotprogramming.com
>>>
>>> On Mon, Mar 23, 2015 at 5:25 AM, IT CTO <go...@gmail.com> wrote:
>>>
>>>> Hi,
>>>>
>>>> I am running spark when I use sc.version I get 1.2 but when I call
>>>> registerTempTable("MyTable") I get error saying registedTempTable is not a
>>>> member of RDD
>>>>
>>>> Why?
>>>>
>>>> --
>>>> Eran | CTO
>>>>
>>>
>>>
>>
>>
>> --
>> Eran | CTO
>>
>
>
--
Eran | CTO
Re: registerTempTable is not a member of RDD on spark 1.2?
Posted by Ted Yu <yu...@gmail.com>.
Have you tried adding the following ?
import org.apache.spark.sql.SQLContext
Cheers
On Mon, Mar 23, 2015 at 6:45 AM, IT CTO <go...@gmail.com> wrote:
> Thanks.
> I am new to the environment and running cloudera CDH5.3 with spark in it.
>
> apparently when running in spark-shell this command val sqlContext = new
> SQLContext(sc)
> I am failing with the not found type SQLContext
>
> Any idea why?
>
> On Mon, Mar 23, 2015 at 3:05 PM, Dean Wampler <de...@gmail.com>
> wrote:
>
>> In 1.2 it's a member of SchemaRDD and it becomes available on RDD
>> (through the "type class" mechanism) when you add a SQLContext, like so.
>>
>> val sqlContext = new SQLContext(sc)import sqlContext._
>>
>>
>> In 1.3, the method has moved to the new DataFrame type.
>>
>> Dean Wampler, Ph.D.
>> Author: Programming Scala, 2nd Edition
>> <http://shop.oreilly.com/product/0636920033073.do> (O'Reilly)
>> Typesafe <http://typesafe.com>
>> @deanwampler <http://twitter.com/deanwampler>
>> http://polyglotprogramming.com
>>
>> On Mon, Mar 23, 2015 at 5:25 AM, IT CTO <go...@gmail.com> wrote:
>>
>>> Hi,
>>>
>>> I am running spark when I use sc.version I get 1.2 but when I call
>>> registerTempTable("MyTable") I get error saying registedTempTable is not a
>>> member of RDD
>>>
>>> Why?
>>>
>>> --
>>> Eran | CTO
>>>
>>
>>
>
>
> --
> Eran | CTO
>
Re: registerTempTable is not a member of RDD on spark 1.2?
Posted by IT CTO <go...@gmail.com>.
Thanks.
I am new to the environment and running cloudera CDH5.3 with spark in it.
apparently when running in spark-shell this command val sqlContext = new
SQLContext(sc)
I am failing with the not found type SQLContext
Any idea why?
On Mon, Mar 23, 2015 at 3:05 PM, Dean Wampler <de...@gmail.com> wrote:
> In 1.2 it's a member of SchemaRDD and it becomes available on RDD (through
> the "type class" mechanism) when you add a SQLContext, like so.
>
> val sqlContext = new SQLContext(sc)import sqlContext._
>
>
> In 1.3, the method has moved to the new DataFrame type.
>
> Dean Wampler, Ph.D.
> Author: Programming Scala, 2nd Edition
> <http://shop.oreilly.com/product/0636920033073.do> (O'Reilly)
> Typesafe <http://typesafe.com>
> @deanwampler <http://twitter.com/deanwampler>
> http://polyglotprogramming.com
>
> On Mon, Mar 23, 2015 at 5:25 AM, IT CTO <go...@gmail.com> wrote:
>
>> Hi,
>>
>> I am running spark when I use sc.version I get 1.2 but when I call
>> registerTempTable("MyTable") I get error saying registedTempTable is not a
>> member of RDD
>>
>> Why?
>>
>> --
>> Eran | CTO
>>
>
>
--
Eran | CTO
Re: registerTempTable is not a member of RDD on spark 1.2?
Posted by Dean Wampler <de...@gmail.com>.
In 1.2 it's a member of SchemaRDD and it becomes available on RDD (through
the "type class" mechanism) when you add a SQLContext, like so.
val sqlContext = new SQLContext(sc)import sqlContext._
In 1.3, the method has moved to the new DataFrame type.
Dean Wampler, Ph.D.
Author: Programming Scala, 2nd Edition
<http://shop.oreilly.com/product/0636920033073.do> (O'Reilly)
Typesafe <http://typesafe.com>
@deanwampler <http://twitter.com/deanwampler>
http://polyglotprogramming.com
On Mon, Mar 23, 2015 at 5:25 AM, IT CTO <go...@gmail.com> wrote:
> Hi,
>
> I am running spark when I use sc.version I get 1.2 but when I call
> registerTempTable("MyTable") I get error saying registedTempTable is not a
> member of RDD
>
> Why?
>
> --
> Eran | CTO
>