You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@spark.apache.org by Stephen Boesch <ja...@gmail.com> on 2014/10/28 10:09:06 UTC
How to import mllib.rdd.RDDFunctions into the spark-shell
I seem to recall there were some specific requirements on how to import the
implicits.
Here is the issue:
scala> import org.apache.spark.mllib.rdd.RDDFunctions._
<console>:10: error: object RDDFunctions in package rdd cannot be accessed
in package org.apache.spark.mllib.rdd
import org.apache.spark.mllib.rdd.RDDFunctions._
Re: How to import mllib.rdd.RDDFunctions into the spark-shell
Posted by Xiangrui Meng <me...@gmail.com>.
FYI, there is a PR to make mllib.rdd.RDDFunctions public:
https://github.com/apache/spark/pull/2907 -Xiangrui
On Tue, Oct 28, 2014 at 5:18 AM, Yanbo Liang <ya...@gmail.com> wrote:
> Yes, it can import org.apache.spark.mllib.rdd.RDDFunctions but you can not
> use any method in this class or even new an object of this class. So I infer
> that if you import org.apache.spark.mllib.rdd.RDDFunctions._, it may call
> some method of that object.
>
> 2014-10-28 17:29 GMT+08:00 Stephen Boesch <ja...@gmail.com>:
>>
>> HI Yanbo,
>> That is not the issue: notice that importing the object is fine:
>>
>> scala> import org.apache.spark.mllib.rdd.RDDFunctions
>> import org.apache.spark.mllib.rdd.RDDFunctions
>>
>> scala> import org.apache.spark.mllib.rdd.RDDFunctions._
>> <console>:11: error: object RDDFunctions in package rdd cannot be accessed
>> in package org.apache.spark.mllib.rdd
>> import org.apache.spark.mllib.rdd.RDDFunctions._
>>
>>
>> It has to do with the implicits.
>>
>>
>>
>> 2014-10-28 2:25 GMT-07:00 Yanbo Liang <ya...@gmail.com>:
>>
>>> Because that org.apache.spark.mllib.rdd.RDDFunctions._ is mllib private
>>> class, it can only be called by function in mllib.
>>>
>>> 2014-10-28 17:09 GMT+08:00 Stephen Boesch <ja...@gmail.com>:
>>>>
>>>> I seem to recall there were some specific requirements on how to import
>>>> the implicits.
>>>>
>>>> Here is the issue:
>>>>
>>>> scala> import org.apache.spark.mllib.rdd.RDDFunctions._
>>>> <console>:10: error: object RDDFunctions in package rdd cannot be
>>>> accessed in package org.apache.spark.mllib.rdd
>>>> import org.apache.spark.mllib.rdd.RDDFunctions._
>>>
>>>
>>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@spark.apache.org
For additional commands, e-mail: user-help@spark.apache.org
Re: How to import mllib.rdd.RDDFunctions into the spark-shell
Posted by Yanbo Liang <ya...@gmail.com>.
Yes, it can import org.apache.spark.mllib.rdd.RDDFunctions but you can not
use any method in this class or even new an object of this class. So I
infer that if you import org.apache.spark.mllib.rdd.RDDFunctions._, it may
call some method of that object.
2014-10-28 17:29 GMT+08:00 Stephen Boesch <ja...@gmail.com>:
> HI Yanbo,
> That is not the issue: notice that importing the object is fine:
>
> scala> import org.apache.spark.mllib.rdd.RDDFunctions
> import org.apache.spark.mllib.rdd.RDDFunctions
>
> scala> import org.apache.spark.mllib.rdd.RDDFunctions._
> <console>:11: error: object RDDFunctions in package rdd cannot be accessed
> in package org.apache.spark.mllib.rdd
> import org.apache.spark.mllib.rdd.RDDFunctions._
>
>
> It has to do with the implicits.
>
>
>
> 2014-10-28 2:25 GMT-07:00 Yanbo Liang <ya...@gmail.com>:
>
> Because that org.apache.spark.mllib.rdd.RDDFunctions._ is mllib private
>> class, it can only be called by function in mllib.
>>
>> 2014-10-28 17:09 GMT+08:00 Stephen Boesch <ja...@gmail.com>:
>>
>>> I seem to recall there were some specific requirements on how to import
>>> the implicits.
>>>
>>> Here is the issue:
>>>
>>> scala> import org.apache.spark.mllib.rdd.RDDFunctions._
>>> <console>:10: error: object RDDFunctions in package rdd cannot be
>>> accessed in package org.apache.spark.mllib.rdd
>>> import org.apache.spark.mllib.rdd.RDDFunctions._
>>>
>>
>>
>
Re: How to import mllib.rdd.RDDFunctions into the spark-shell
Posted by Stephen Boesch <ja...@gmail.com>.
HI Yanbo,
That is not the issue: notice that importing the object is fine:
scala> import org.apache.spark.mllib.rdd.RDDFunctions
import org.apache.spark.mllib.rdd.RDDFunctions
scala> import org.apache.spark.mllib.rdd.RDDFunctions._
<console>:11: error: object RDDFunctions in package rdd cannot be accessed
in package org.apache.spark.mllib.rdd
import org.apache.spark.mllib.rdd.RDDFunctions._
It has to do with the implicits.
2014-10-28 2:25 GMT-07:00 Yanbo Liang <ya...@gmail.com>:
> Because that org.apache.spark.mllib.rdd.RDDFunctions._ is mllib private
> class, it can only be called by function in mllib.
>
> 2014-10-28 17:09 GMT+08:00 Stephen Boesch <ja...@gmail.com>:
>
>> I seem to recall there were some specific requirements on how to import
>> the implicits.
>>
>> Here is the issue:
>>
>> scala> import org.apache.spark.mllib.rdd.RDDFunctions._
>> <console>:10: error: object RDDFunctions in package rdd cannot be
>> accessed in package org.apache.spark.mllib.rdd
>> import org.apache.spark.mllib.rdd.RDDFunctions._
>>
>
>
Re: How to import mllib.rdd.RDDFunctions into the spark-shell
Posted by Yanbo Liang <ya...@gmail.com>.
Because that org.apache.spark.mllib.rdd.RDDFunctions._ is mllib private
class, it can only be called by function in mllib.
2014-10-28 17:09 GMT+08:00 Stephen Boesch <ja...@gmail.com>:
> I seem to recall there were some specific requirements on how to import
> the implicits.
>
> Here is the issue:
>
> scala> import org.apache.spark.mllib.rdd.RDDFunctions._
> <console>:10: error: object RDDFunctions in package rdd cannot be accessed
> in package org.apache.spark.mllib.rdd
> import org.apache.spark.mllib.rdd.RDDFunctions._
>