You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@spark.apache.org by fjeg <fr...@gmail.com> on 2014/07/15 20:12:47 UTC
MLLib - Regularized logistic regression in python
Hi All,
I am trying to perform regularized logistic regression with mllib in python.
I have seen that this is possible in the following scala example:
https://github.com/apache/spark/blob/master/examples/src/main/scala/org/apache/spark/examples/mllib/BinaryClassification.scala
But I do not see any way to set the regType and regParam when training
logistic regression through python.
Additionally, I would like to output the activations -- i.e. P(Y=1 | X).
Currently, LogisticRegressionModel.predict() just thresholds at 0.5 and does
not return the actual probability. Do I just have to do this by hand by
grabbing the weights from the trained model, or is there a built in way to
do this?
Best,
Francisco Gimenez
--
View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/MLLib-Regularized-logistic-regression-in-python-tp9780.html
Sent from the Apache Spark User List mailing list archive at Nabble.com.
Re: MLLib - Regularized logistic regression in python
Posted by fjeg <fr...@gmail.com>.
Thanks for all your helpful replies.
Best,
Francisco
--
View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/MLLib-Regularized-logistic-regression-in-python-tp9780p10184.html
Sent from the Apache Spark User List mailing list archive at Nabble.com.
Re: MLLib - Regularized logistic regression in python
Posted by Xiangrui Meng <me...@gmail.com>.
1) This is a miss, unfortunately ... We will add support for
regularization and intercept in the coming v1.1. (JIRA:
https://issues.apache.org/jira/browse/SPARK-2550)
2) It has overflow problems in Python but not in Scala. We can
stabilize the computation by ensuring exp only takes a negative value:
1 / ( 1 + e^ x) = 1 - 1 / ( 1 + e^{-x} ) . (JIRA:
https://issues.apache.org/jira/browse/SPARK-2552)
-Xiangrui
On Wed, Jul 16, 2014 at 7:58 PM, Yanbo Liang <ya...@gmail.com> wrote:
> AFAIK for question 2, there is no built-in method to account for that
> problem.
> At right now, we can only perform one type of regularization.
> However, the elastic net implementation is just underway.
> You can refer this topic for further discussion.
> https://issues.apache.org/jira/browse/SPARK-1543
>
>
> 2014-07-17 2:08 GMT+08:00 fjeg <fr...@gmail.com>:
>
>> 1) Okay, to clarify, there is *no* way to regularize logistic regression
>> in
>> python (sorry if I'm repeating your answer).
>>
>> 2) This method you described will have overflow errors when abs(margin) >
>> 750. Is there a built-in method to account for this? Otherwise, I will
>> probably have to implement something like this:
>>
>> http://lingpipe-blog.com/2012/02/16/howprevent-overflow-underflow-logistic-regression
>>
>> Also, another question about the Scala implementation:
>> Can we only do one type of regularization? Is there any way to perform
>> elastic net which is a combination of L1 and L2?
>>
>>
>>
>>
>>
>> --
>> View this message in context:
>> http://apache-spark-user-list.1001560.n3.nabble.com/MLLib-Regularized-logistic-regression-in-python-tp9780p9963.html
>> Sent from the Apache Spark User List mailing list archive at Nabble.com.
>
>
Re: MLLib - Regularized logistic regression in python
Posted by Yanbo Liang <ya...@gmail.com>.
AFAIK for question 2, there is no built-in method to account for that
problem.
At right now, we can only perform one type of regularization.
However, the elastic net implementation is just underway.
You can refer this topic for further discussion.
https://issues.apache.org/jira/browse/SPARK-1543
2014-07-17 2:08 GMT+08:00 fjeg <fr...@gmail.com>:
> 1) Okay, to clarify, there is *no* way to regularize logistic regression in
> python (sorry if I'm repeating your answer).
>
> 2) This method you described will have overflow errors when abs(margin) >
> 750. Is there a built-in method to account for this? Otherwise, I will
> probably have to implement something like this:
>
> http://lingpipe-blog.com/2012/02/16/howprevent-overflow-underflow-logistic-regression
>
> Also, another question about the Scala implementation:
> Can we only do one type of regularization? Is there any way to perform
> elastic net which is a combination of L1 and L2?
>
>
>
>
>
> --
> View this message in context:
> http://apache-spark-user-list.1001560.n3.nabble.com/MLLib-Regularized-logistic-regression-in-python-tp9780p9963.html
> Sent from the Apache Spark User List mailing list archive at Nabble.com.
>
Re: MLLib - Regularized logistic regression in python
Posted by fjeg <fr...@gmail.com>.
1) Okay, to clarify, there is *no* way to regularize logistic regression in
python (sorry if I'm repeating your answer).
2) This method you described will have overflow errors when abs(margin) >
750. Is there a built-in method to account for this? Otherwise, I will
probably have to implement something like this:
http://lingpipe-blog.com/2012/02/16/howprevent-overflow-underflow-logistic-regression
Also, another question about the Scala implementation:
Can we only do one type of regularization? Is there any way to perform
elastic net which is a combination of L1 and L2?
--
View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/MLLib-Regularized-logistic-regression-in-python-tp9780p9963.html
Sent from the Apache Spark User List mailing list archive at Nabble.com.
Re: MLLib - Regularized logistic regression in python
Posted by Yanbo Liang <ya...@gmail.com>.
1) AFAIK Spark Python API does not supply interface to set regType and
regParam.
If you want to personalize your own LR model with proper regularized
parameters, strong recommend to user scala API.
You can reference the following code at
spark-1.0.0/python/pyspark/mllib/classification.py.
class LogisticRegressionWithSGD(object):
@classmethod
def train(cls, data, iterations=100, step=1.0, miniBatchFraction=1.0,
initialWeights=None):
"""Train a logistic regression model on the given data."""
sc = data.context
train_func = lambda d, i:
sc._jvm.PythonMLLibAPI().trainLogisticRegressionModelWithSGD(
d._jrdd, iterations, step, miniBatchFraction, i)
return _regression_train_wrapper(sc, train_func,
LogisticRegressionModel, data,
initialWeights)
2) The actual probability has been computed but without output.
If you want to output the score, just customize the following function at class
LogisticRegressionModel.
override protected def predictPoint(dataMatrix: Vector, weightMatrix:
Vector,
intercept: Double) = {
val margin = weightMatrix.toBreeze.dot(dataMatrix.toBreeze) + intercept
val score = 1.0/ (1.0 + math.exp(-margin))
threshold match {
case Some(t) => if (score < t) 0.0 else 1.0
case None => score
}
}
2014-07-16 2:12 GMT+08:00 fjeg <fr...@gmail.com>:
> Hi All,
>
> I am trying to perform regularized logistic regression with mllib in
> python.
> I have seen that this is possible in the following scala example:
>
> https://github.com/apache/spark/blob/master/examples/src/main/scala/org/apache/spark/examples/mllib/BinaryClassification.scala
>
> But I do not see any way to set the regType and regParam when training
> logistic regression through python.
>
> Additionally, I would like to output the activations -- i.e. P(Y=1 | X).
> Currently, LogisticRegressionModel.predict() just thresholds at 0.5 and
> does
> not return the actual probability. Do I just have to do this by hand by
> grabbing the weights from the trained model, or is there a built in way to
> do this?
>
> Best,
> Francisco Gimenez
>
>
>
> --
> View this message in context:
> http://apache-spark-user-list.1001560.n3.nabble.com/MLLib-Regularized-logistic-regression-in-python-tp9780.html
> Sent from the Apache Spark User List mailing list archive at Nabble.com.
>