You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@spark.apache.org by aditya1702 <ad...@gmail.com> on 2016/10/18 17:09:34 UTC
Making more features in Logistic Regression
Hello,
I am trying to solve a problem of Logistic Regression using Spark. I am
still a newbie to machine learning. I wanted to ask that if I have 2
features for logistic regression and if the features are non-linear
(regularized logistic regression) do we have to make more features by
considering the higher powers of the features?
--
View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/Making-more-features-in-Logistic-Regression-tp27915.html
Sent from the Apache Spark User List mailing list archive at Nabble.com.
---------------------------------------------------------------------
To unsubscribe e-mail: user-unsubscribe@spark.apache.org
Re: Making more features in Logistic Regression
Posted by aditya1702 <ad...@gmail.com>.
<http://apache-spark-user-list.1001560.n3.nabble.com/file/n27917/Screen_Shot_2016-10-19_at_12.png>
<http://apache-spark-user-list.1001560.n3.nabble.com/file/n27917/Screen_Shot_2016-10-19_at_12.png>
Here is the graph and the features with their corresponding data
--
View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/Making-more-features-in-Logistic-Regression-tp27915p27917.html
Sent from the Apache Spark User List mailing list archive at Nabble.com.
---------------------------------------------------------------------
To unsubscribe e-mail: user-unsubscribe@spark.apache.org
Re: Making more features in Logistic Regression
Posted by eliasah <ab...@gmail.com>.
Your question isn't clear. Would you care elaborate ?
--
View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/Making-more-features-in-Logistic-Regression-tp27915p27960.html
Sent from the Apache Spark User List mailing list archive at Nabble.com.
---------------------------------------------------------------------
To unsubscribe e-mail: user-unsubscribe@spark.apache.org
Re: Making more features in Logistic Regression
Posted by Nick Pentreath <ni...@gmail.com>.
You can use the PolynomialExpansion in Spark ML (
http://spark.apache.org/docs/latest/api/scala/index.html#org.apache.spark.ml.feature.PolynomialExpansion
)
On Tue, 18 Oct 2016 at 21:47 miro <mi...@gmail.com> wrote:
> Yes, I was thinking going down this road:
>
>
> http://scikit-learn.org/stable/modules/linear_model.html#polynomial-regression-extending-linear-models-with-basis-functions
>
> http://stats.stackexchange.com/questions/58739/polynomial-regression-using-scikit-learn
>
>
> But I’m not sure if spark actually has polynomial regression implemented
> (I couldn’t find it) - maybe SparkML gurus can help here?
>
> You could take a look also at scikit integration package with Spark (
> https://github.com/databricks/spark-sklearn).
>
> Hope it helped :)
>
> All the best,
> m.
>
>
>
> On 18 Oct 2016, at 20:36, aditya1702 <ad...@gmail.com> wrote:
>
> <http://apache-spark-user-list.1001560.n3.nabble.com/file/n27918/pic1.png>
>
> <http://apache-spark-user-list.1001560.n3.nabble.com/file/n27918/pic2.png>
>
>
>
>
> --
> View this message in context:
> http://apache-spark-user-list.1001560.n3.nabble.com/Making-more-features-in-Logistic-Regression-tp27915p27918.html
> Sent from the Apache Spark User List mailing list archive at Nabble.com.
>
> ---------------------------------------------------------------------
> To unsubscribe e-mail: user-unsubscribe@spark.apache.org
>
>
>
Re: Making more features in Logistic Regression
Posted by miro <mi...@gmail.com>.
Yes, I was thinking going down this road:
http://scikit-learn.org/stable/modules/linear_model.html#polynomial-regression-extending-linear-models-with-basis-functions <http://scikit-learn.org/stable/modules/linear_model.html#polynomial-regression-extending-linear-models-with-basis-functions>
http://stats.stackexchange.com/questions/58739/polynomial-regression-using-scikit-learn <http://stats.stackexchange.com/questions/58739/polynomial-regression-using-scikit-learn>
But I’m not sure if spark actually has polynomial regression implemented (I couldn’t find it) - maybe SparkML gurus can help here?
You could take a look also at scikit integration package with Spark (https://github.com/databricks/spark-sklearn <https://github.com/databricks/spark-sklearn>).
Hope it helped :)
All the best,
m.
> On 18 Oct 2016, at 20:36, aditya1702 <ad...@gmail.com> wrote:
>
> <http://apache-spark-user-list.1001560.n3.nabble.com/file/n27918/pic1.png>
> <http://apache-spark-user-list.1001560.n3.nabble.com/file/n27918/pic2.png>
>
>
>
> --
> View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/Making-more-features-in-Logistic-Regression-tp27915p27918.html
> Sent from the Apache Spark User List mailing list archive at Nabble.com.
>
> ---------------------------------------------------------------------
> To unsubscribe e-mail: user-unsubscribe@spark.apache.org
>
Re: Making more features in Logistic Regression
Posted by aditya1702 <ad...@gmail.com>.
<http://apache-spark-user-list.1001560.n3.nabble.com/file/n27918/pic1.png>
<http://apache-spark-user-list.1001560.n3.nabble.com/file/n27918/pic2.png>
--
View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/Making-more-features-in-Logistic-Regression-tp27915p27918.html
Sent from the Apache Spark User List mailing list archive at Nabble.com.
---------------------------------------------------------------------
To unsubscribe e-mail: user-unsubscribe@spark.apache.org
Re: Making more features in Logistic Regression
Posted by miro <mi...@gmail.com>.
Hi,
I think it depends on how non-linear data you have. You could add polynomial to your model,..but everything depends on your data. If you could share more details maybe a scatter plot, would help to investigate the problem further.
All the best,
Miro
> On 18 Oct 2016, at 19:09, aditya1702 <ad...@gmail.com> wrote:
>
> Hello,
> I am trying to solve a problem of Logistic Regression using Spark. I am
> still a newbie to machine learning. I wanted to ask that if I have 2
> features for logistic regression and if the features are non-linear
> (regularized logistic regression) do we have to make more features by
> considering the higher powers of the features?
>
>
>
> --
> View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/Making-more-features-in-Logistic-Regression-tp27915.html
> Sent from the Apache Spark User List mailing list archive at Nabble.com.
>
> ---------------------------------------------------------------------
> To unsubscribe e-mail: user-unsubscribe@spark.apache.org
>
---------------------------------------------------------------------
To unsubscribe e-mail: user-unsubscribe@spark.apache.org