You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@spark.apache.org by janardhan shetty <ja...@gmail.com> on 2016/11/19 17:03:02 UTC
Usage of mllib api in ml
Hi,
I am trying to use the evaluation metrics offered by mllib
multiclassmetrics in ml dataframe setting.
Is there any examples how to use it?
Re: Usage of mllib api in ml
Posted by Marco Mistroni <mm...@gmail.com>.
Hi
If it is an rdd based can't u use data frame.rdd (though I don't know if u
will have an rdd of vectors....u might need to convert each row to a vector
yourself......
Hth
On 20 Nov 2016 4:29 pm, "janardhan shetty" <ja...@gmail.com> wrote:
> Hi Marco and Yanbo,
>
> It is not the usage of MulticlassClassificationEvaluator. Probably I was
> not clear. Let me explain:
>
> I am trying to use confusionMatrix which is not present in
> MulticlassClassificationEvaluator ml version where as it is present in
> MulticlassMetrics of mllib.
> How to leverage RDD version using ml dataframes ?
>
> *mllib*: MulticlassMetrics
> *ml*: MulticlassClassificationEvaluator
>
> On Sun, Nov 20, 2016 at 4:52 AM, Marco Mistroni <mm...@gmail.com>
> wrote:
>
>> Hi
>> you can also have a look at this example,
>>
>> https://github.com/sryza/aas/blob/master/ch04-rdf/src/main/s
>> cala/com/cloudera/datascience/rdf/RunRDF.scala#L220
>>
>> kr
>> marco
>>
>> On Sun, Nov 20, 2016 at 9:09 AM, Yanbo Liang <yb...@gmail.com> wrote:
>>
>>> You can refer this example(http://spark.apache.or
>>> g/docs/latest/ml-tuning.html#example-model-selection-via-cro
>>> ss-validation) which use BinaryClassificationEvaluator, and it should
>>> be very straightforward to switch to MulticlassClassificationEvaluator.
>>>
>>> Thanks
>>> Yanbo
>>>
>>> On Sat, Nov 19, 2016 at 9:03 AM, janardhan shetty <
>>> janardhanp22@gmail.com> wrote:
>>>
>>>> Hi,
>>>>
>>>> I am trying to use the evaluation metrics offered by mllib
>>>> multiclassmetrics in ml dataframe setting.
>>>> Is there any examples how to use it?
>>>>
>>>
>>>
>>
>
Re: Usage of mllib api in ml
Posted by janardhan shetty <ja...@gmail.com>.
Hi Marco and Yanbo,
It is not the usage of MulticlassClassificationEvaluator. Probably I was
not clear. Let me explain:
I am trying to use confusionMatrix which is not present in
MulticlassClassificationEvaluator ml version where as it is present in
MulticlassMetrics of mllib.
How to leverage RDD version using ml dataframes ?
*mllib*: MulticlassMetrics
*ml*: MulticlassClassificationEvaluator
On Sun, Nov 20, 2016 at 4:52 AM, Marco Mistroni <mm...@gmail.com> wrote:
> Hi
> you can also have a look at this example,
>
> https://github.com/sryza/aas/blob/master/ch04-rdf/src/main/
> scala/com/cloudera/datascience/rdf/RunRDF.scala#L220
>
> kr
> marco
>
> On Sun, Nov 20, 2016 at 9:09 AM, Yanbo Liang <yb...@gmail.com> wrote:
>
>> You can refer this example(http://spark.apache.or
>> g/docs/latest/ml-tuning.html#example-model-selection-via-cross-validation)
>> which use BinaryClassificationEvaluator, and it should be very
>> straightforward to switch to MulticlassClassificationEvaluator.
>>
>> Thanks
>> Yanbo
>>
>> On Sat, Nov 19, 2016 at 9:03 AM, janardhan shetty <janardhanp22@gmail.com
>> > wrote:
>>
>>> Hi,
>>>
>>> I am trying to use the evaluation metrics offered by mllib
>>> multiclassmetrics in ml dataframe setting.
>>> Is there any examples how to use it?
>>>
>>
>>
>
Re: Usage of mllib api in ml
Posted by Marco Mistroni <mm...@gmail.com>.
Hi
you can also have a look at this example,
https://github.com/sryza/aas/blob/master/ch04-rdf/src/main/scala/com/cloudera/datascience/rdf/RunRDF.scala#L220
kr
marco
On Sun, Nov 20, 2016 at 9:09 AM, Yanbo Liang <yb...@gmail.com> wrote:
> You can refer this example(http://spark.apache.org/docs/latest/ml-tuning.
> html#example-model-selection-via-cross-validation) which use
> BinaryClassificationEvaluator, and it should be very straightforward to
> switch to MulticlassClassificationEvaluator.
>
> Thanks
> Yanbo
>
> On Sat, Nov 19, 2016 at 9:03 AM, janardhan shetty <ja...@gmail.com>
> wrote:
>
>> Hi,
>>
>> I am trying to use the evaluation metrics offered by mllib
>> multiclassmetrics in ml dataframe setting.
>> Is there any examples how to use it?
>>
>
>
Re: Usage of mllib api in ml
Posted by Yanbo Liang <yb...@gmail.com>.
You can refer this example(
http://spark.apache.org/docs/latest/ml-tuning.html#example-model-selection-via-cross-validation)
which use BinaryClassificationEvaluator, and it should be very
straightforward to switch to MulticlassClassificationEvaluator.
Thanks
Yanbo
On Sat, Nov 19, 2016 at 9:03 AM, janardhan shetty <ja...@gmail.com>
wrote:
> Hi,
>
> I am trying to use the evaluation metrics offered by mllib
> multiclassmetrics in ml dataframe setting.
> Is there any examples how to use it?
>