You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@spark.apache.org by jamborta <ja...@gmail.com> on 2014/06/16 17:13:06 UTC
pyspark regression results way off
Hi all,
I am testing the regression methods (SGD) using pyspark. Tried to tune the
parameters, but they are far off from the results obtained using R. Is there
some way to set these parameters more efficiently?
thanks,
--
View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/pyspark-regression-results-way-off-tp7672.html
Sent from the Apache Spark User List mailing list archive at Nabble.com.
Re: pyspark regression results way off
Posted by frol <fr...@gmail.com>.
Here is my conversation about the same issue with regression methods:
https://issues.apache.org/jira/browse/SPARK-1859
--
View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/pyspark-regression-results-way-off-tp7672p8139.html
Sent from the Apache Spark User List mailing list archive at Nabble.com.
Re: pyspark regression results way off
Posted by jamborta <ja...@gmail.com>.
Thanks, will try normalising it.
--
View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/pyspark-regression-results-way-off-tp7672p7720.html
Sent from the Apache Spark User List mailing list archive at Nabble.com.
Re: pyspark regression results way off
Posted by DB Tsai <db...@dbtsai.com>.
There is no python binding for LBFGS. Feel free to submit a PR.
Sincerely,
DB Tsai
-------------------------------------------------------
My Blog: https://www.dbtsai.com
LinkedIn: https://www.linkedin.com/in/dbtsai
On Wed, Jun 25, 2014 at 1:41 PM, Mohit Jaggi <mo...@gmail.com> wrote:
> Is a python binding for LBFGS in the works? My co-worker has written one and
> can contribute back if it helps.
>
>
> On Mon, Jun 16, 2014 at 11:00 AM, DB Tsai <db...@stanford.edu> wrote:
>>
>> Is your data normalized? Sometimes, GD doesn't work well if the data
>> has wide range. If you are willing to write scala code, you can try
>> LBFGS optimizer which converges better than GD.
>>
>> Sincerely,
>>
>> DB Tsai
>> -------------------------------------------------------
>> My Blog: https://www.dbtsai.com
>> LinkedIn: https://www.linkedin.com/in/dbtsai
>>
>>
>> On Mon, Jun 16, 2014 at 8:14 AM, jamborta <ja...@gmail.com> wrote:
>> > forgot to mention that I'm running spark 1.0
>> >
>> >
>> >
>> > --
>> > View this message in context:
>> > http://apache-spark-user-list.1001560.n3.nabble.com/pyspark-regression-results-way-off-tp7672p7673.html
>> > Sent from the Apache Spark User List mailing list archive at Nabble.com.
>
>
Re: pyspark regression results way off
Posted by Mohit Jaggi <mo...@gmail.com>.
Is a python binding for LBFGS in the works? My co-worker has written one
and can contribute back if it helps.
On Mon, Jun 16, 2014 at 11:00 AM, DB Tsai <db...@stanford.edu> wrote:
> Is your data normalized? Sometimes, GD doesn't work well if the data
> has wide range. If you are willing to write scala code, you can try
> LBFGS optimizer which converges better than GD.
>
> Sincerely,
>
> DB Tsai
> -------------------------------------------------------
> My Blog: https://www.dbtsai.com
> LinkedIn: https://www.linkedin.com/in/dbtsai
>
>
> On Mon, Jun 16, 2014 at 8:14 AM, jamborta <ja...@gmail.com> wrote:
> > forgot to mention that I'm running spark 1.0
> >
> >
> >
> > --
> > View this message in context:
> http://apache-spark-user-list.1001560.n3.nabble.com/pyspark-regression-results-way-off-tp7672p7673.html
> > Sent from the Apache Spark User List mailing list archive at Nabble.com.
>
Re: pyspark regression results way off
Posted by DB Tsai <db...@stanford.edu>.
Is your data normalized? Sometimes, GD doesn't work well if the data
has wide range. If you are willing to write scala code, you can try
LBFGS optimizer which converges better than GD.
Sincerely,
DB Tsai
-------------------------------------------------------
My Blog: https://www.dbtsai.com
LinkedIn: https://www.linkedin.com/in/dbtsai
On Mon, Jun 16, 2014 at 8:14 AM, jamborta <ja...@gmail.com> wrote:
> forgot to mention that I'm running spark 1.0
>
>
>
> --
> View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/pyspark-regression-results-way-off-tp7672p7673.html
> Sent from the Apache Spark User List mailing list archive at Nabble.com.
Re: pyspark regression results way off
Posted by jamborta <ja...@gmail.com>.
forgot to mention that I'm running spark 1.0
--
View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/pyspark-regression-results-way-off-tp7672p7673.html
Sent from the Apache Spark User List mailing list archive at Nabble.com.