You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@spark.apache.org by michelleyang <mi...@gmail.com> on 2018/01/30 14:55:01 UTC
ML:One vs Rest with crossValidator for multinomial in logistic
regression
I tried to use One vs Rest in spark ml with pipeline and crossValidator for
multimultinomial in logistic regression.
It came out with empty coefficients. I figured out it was the setting of
ParamGridBuilder. Can anyone help me understand how does the parameter
setting affect the crossValidator process?
the orginal code: //output empty coefficients.
val logist=new LogisticRegression
val ova = new OneVsRest().setClassifier(logist)
val paramMaps = new ParamGridBuilder().addGrid(ova.classifier,
Array(logist.getRegParam))
New code://output multi classes coefficients
val logist=new LogisticRegression
val ova = new OneVsRest().setClassifier(logist)
val classifier1 = new LogisticRegression().setRegParam(2.0)
val classifier2 = new LogisticRegression().setRegParam(3.0)
val paramMaps = new ParamGridBuilder() .addGrid(ova.classifier,
Array(classifier1, classifier2))
Please help!!!! Thanks.
--
Sent from: http://apache-spark-user-list.1001560.n3.nabble.com/
---------------------------------------------------------------------
To unsubscribe e-mail: user-unsubscribe@spark.apache.org
Re: ML:One vs Rest with crossValidator for multinomial in logistic
regression
Posted by Nicolas Paris <ni...@gmail.com>.
Brian
This is absolutely this problem.
Good to hear it will be fix in 2.3 release
Le 09 févr. 2018 à 02:17, Bryan Cutler écrivait :
> Nicolas, are you referring to printing the model params in that example with
> "print(model1.extractParamMap())"? There was a problem with pyspark models not
> having params after being fit, which causes this example to show nothing for
> model paramMaps. This was fixed in https://issues.apache.org/jira/browse/
> SPARK-10931 and the example now shows all model params. The fix will be in the
> Spark 2.3 release.
>
> Bryan
>
> On Wed, Jan 31, 2018 at 10:20 PM, Nicolas Paris <ni...@gmail.com> wrote:
>
> Hey
>
> I am also interested in how to get those parameters.
> For example, the demo code spark-2.2.1-bin-hadoop2.7/examples/src/main/
> python/ml/estimator_transformer_param_example.py
> return empty parameters when printing "lr.extractParamMap()"
>
> That's weird
>
> Thanks
>
> Le 30 janv. 2018 à 23:10, Bryan Cutler écrivait :
> > Hi Michelle,
> >
> > Your original usage of ParamGridBuilder was not quite right, `addGrid`
> expects
> > (some parameter, array of values for that parameter). If you want to do
> a grid
> > search with different regularization values, you would do the following:
> >
> > val paramMaps = new ParamGridBuilder().addGrid(logist.regParam, Array
> (0.1,
> > 0.3)).build()
> >
> > * don't forget to build the grid after adding values
> >
> > On Tue, Jan 30, 2018 at 6:55 AM, michelleyang <
> michelle1026shuyu@gmail.com>
> > wrote:
> >
> > I tried to use One vs Rest in spark ml with pipeline and
> crossValidator for
> > multimultinomial in logistic regression.
> >
> > It came out with empty coefficients. I figured out it was the setting
> of
> > ParamGridBuilder. Can anyone help me understand how does the
> parameter
> > setting affect the crossValidator process?
> >
> > the orginal code: //output empty coefficients.
> >
> > val logist=new LogisticRegression
> >
> > val ova = new OneVsRest().setClassifier(logist)
> >
> > val paramMaps = new ParamGridBuilder().addGrid(ova.classifier,
> > Array(logist.getRegParam))
> >
> > New code://output multi classes coefficients
> >
> > val logist=new LogisticRegression
> >
> > val ova = new OneVsRest().setClassifier(logist)
> >
> > val classifier1 = new LogisticRegression().setRegParam(2.0)
> >
> > val classifier2 = new LogisticRegression().setRegParam(3.0)
> >
> > val paramMaps = new ParamGridBuilder() .addGrid(ova.classifier,
> > Array(classifier1, classifier2))
> >
> > Please help!!!! Thanks.
> >
> >
> >
> > --
> > Sent from: http://apache-spark-user-list.1001560.n3.nabble.com/
> >
> > ---------------------------------------------------------------------
> > To unsubscribe e-mail: user-unsubscribe@spark.apache.org
> >
> >
> >
>
>
---------------------------------------------------------------------
To unsubscribe e-mail: user-unsubscribe@spark.apache.org
Re: ML:One vs Rest with crossValidator for multinomial in logistic regression
Posted by Bryan Cutler <cu...@gmail.com>.
Nicolas, are you referring to printing the model params in that example
with "print(model1.extractParamMap())"? There was a problem with pyspark
models not having params after being fit, which causes this example to show
nothing for model paramMaps. This was fixed in
https://issues.apache.org/jira/browse/SPARK-10931 and the example now shows
all model params. The fix will be in the Spark 2.3 release.
Bryan
On Wed, Jan 31, 2018 at 10:20 PM, Nicolas Paris <ni...@gmail.com> wrote:
> Hey
>
> I am also interested in how to get those parameters.
> For example, the demo code spark-2.2.1-bin-hadoop2.7/
> examples/src/main/python/ml/estimator_transformer_param_example.py
> return empty parameters when printing "lr.extractParamMap()"
>
> That's weird
>
> Thanks
>
> Le 30 janv. 2018 à 23:10, Bryan Cutler écrivait :
> > Hi Michelle,
> >
> > Your original usage of ParamGridBuilder was not quite right, `addGrid`
> expects
> > (some parameter, array of values for that parameter). If you want to do
> a grid
> > search with different regularization values, you would do the following:
> >
> > val paramMaps = new ParamGridBuilder().addGrid(logist.regParam,
> Array(0.1,
> > 0.3)).build()
> >
> > * don't forget to build the grid after adding values
> >
> > On Tue, Jan 30, 2018 at 6:55 AM, michelleyang <
> michelle1026shuyu@gmail.com>
> > wrote:
> >
> > I tried to use One vs Rest in spark ml with pipeline and
> crossValidator for
> > multimultinomial in logistic regression.
> >
> > It came out with empty coefficients. I figured out it was the
> setting of
> > ParamGridBuilder. Can anyone help me understand how does the
> parameter
> > setting affect the crossValidator process?
> >
> > the orginal code: //output empty coefficients.
> >
> > val logist=new LogisticRegression
> >
> > val ova = new OneVsRest().setClassifier(logist)
> >
> > val paramMaps = new ParamGridBuilder().addGrid(ova.classifier,
> > Array(logist.getRegParam))
> >
> > New code://output multi classes coefficients
> >
> > val logist=new LogisticRegression
> >
> > val ova = new OneVsRest().setClassifier(logist)
> >
> > val classifier1 = new LogisticRegression().setRegParam(2.0)
> >
> > val classifier2 = new LogisticRegression().setRegParam(3.0)
> >
> > val paramMaps = new ParamGridBuilder() .addGrid(ova.classifier,
> > Array(classifier1, classifier2))
> >
> > Please help!!!! Thanks.
> >
> >
> >
> > --
> > Sent from: http://apache-spark-user-list.1001560.n3.nabble.com/
> >
> > ------------------------------------------------------------
> ---------
> > To unsubscribe e-mail: user-unsubscribe@spark.apache.org
> >
> >
> >
>
Re: ML:One vs Rest with crossValidator for multinomial in logistic
regression
Posted by Nicolas Paris <ni...@gmail.com>.
Hey
I am also interested in how to get those parameters.
For example, the demo code spark-2.2.1-bin-hadoop2.7/examples/src/main/python/ml/estimator_transformer_param_example.py
return empty parameters when printing "lr.extractParamMap()"
That's weird
Thanks
Le 30 janv. 2018 à 23:10, Bryan Cutler écrivait :
> Hi Michelle,
>
> Your original usage of ParamGridBuilder was not quite right, `addGrid` expects
> (some parameter, array of values for that parameter). If you want to do a grid
> search with different regularization values, you would do the following:
>
> val paramMaps = new ParamGridBuilder().addGrid(logist.regParam, Array(0.1,
> 0.3)).build()
>
> * don't forget to build the grid after adding values
>
> On Tue, Jan 30, 2018 at 6:55 AM, michelleyang <mi...@gmail.com>
> wrote:
>
> I tried to use One vs Rest in spark ml with pipeline and crossValidator for
> multimultinomial in logistic regression.
>
> It came out with empty coefficients. I figured out it was the setting of
> ParamGridBuilder. Can anyone help me understand how does the parameter
> setting affect the crossValidator process?
>
> the orginal code: //output empty coefficients.
>
> val logist=new LogisticRegression
>
> val ova = new OneVsRest().setClassifier(logist)
>
> val paramMaps = new ParamGridBuilder().addGrid(ova.classifier,
> Array(logist.getRegParam))
>
> New code://output multi classes coefficients
>
> val logist=new LogisticRegression
>
> val ova = new OneVsRest().setClassifier(logist)
>
> val classifier1 = new LogisticRegression().setRegParam(2.0)
>
> val classifier2 = new LogisticRegression().setRegParam(3.0)
>
> val paramMaps = new ParamGridBuilder() .addGrid(ova.classifier,
> Array(classifier1, classifier2))
>
> Please help!!!! Thanks.
>
>
>
> --
> Sent from: http://apache-spark-user-list.1001560.n3.nabble.com/
>
> ---------------------------------------------------------------------
> To unsubscribe e-mail: user-unsubscribe@spark.apache.org
>
>
>
---------------------------------------------------------------------
To unsubscribe e-mail: user-unsubscribe@spark.apache.org
Re: ML:One vs Rest with crossValidator for multinomial in logistic regression
Posted by Bryan Cutler <cu...@gmail.com>.
Hi Michelle,
Your original usage of ParamGridBuilder was not quite right, `addGrid`
expects (some parameter, array of values for that parameter). If you want
to do a grid search with different regularization values, you would do the
following:
val paramMaps = new ParamGridBuilder().addGrid(logist.regParam, Array(0.1,
0.3)).build()
* don't forget to build the grid after adding values
On Tue, Jan 30, 2018 at 6:55 AM, michelleyang <mi...@gmail.com>
wrote:
> I tried to use One vs Rest in spark ml with pipeline and crossValidator for
> multimultinomial in logistic regression.
>
> It came out with empty coefficients. I figured out it was the setting of
> ParamGridBuilder. Can anyone help me understand how does the parameter
> setting affect the crossValidator process?
>
> the orginal code: //output empty coefficients.
>
> val logist=new LogisticRegression
>
> val ova = new OneVsRest().setClassifier(logist)
>
> val paramMaps = new ParamGridBuilder().addGrid(ova.classifier,
> Array(logist.getRegParam))
>
> New code://output multi classes coefficients
>
> val logist=new LogisticRegression
>
> val ova = new OneVsRest().setClassifier(logist)
>
> val classifier1 = new LogisticRegression().setRegParam(2.0)
>
> val classifier2 = new LogisticRegression().setRegParam(3.0)
>
> val paramMaps = new ParamGridBuilder() .addGrid(ova.classifier,
> Array(classifier1, classifier2))
>
> Please help!!!! Thanks.
>
>
>
> --
> Sent from: http://apache-spark-user-list.1001560.n3.nabble.com/
>
> ---------------------------------------------------------------------
> To unsubscribe e-mail: user-unsubscribe@spark.apache.org
>
>