You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@commons.apache.org by marios michaelidis <mi...@hotmail.com> on 2012/09/07 12:22:02 UTC
[math] Logistic, Probit regerssion and Tolerance checks
Hi,
My name is Marios and I have very good
academic background as well as I have worked as modeling analyst in big
projects thus I have experience with prediction and optimization algorithms.
Recently (before 5 months) , I started
learning JAVA and I have made my life much more simple by using Java and Common
math rather than depending on the common packages (SAS SPSS etc). Obviously, I
owe common math a lot.
I have noticed that the site does not
have logistic regression and probit regression, very commonly used in
classification problems. Additionally, The math package does not provide a way
to assess Tolerance (or VIF), very commonly used to avoid multi-colinearity
issues and singular matrices in optimization algorithms, prior to running them.
I am willing to provide complete
Logistic and Probit regression algorithms, optimizable by newton Raphson
optimization maximum-likelihood method , in a very programmatically easy way
(e.g regression(double matrix [][], double Target[], String
Constant, double precision, double tolerance) , with academic references and
very quick (3 secs for 60k set), with getter methods for all the common
statistics such as null Deviance, Deviance, AIC, BIC, Chi-square f the model,
betas, Wald statistics and p values, Cox_snell R square, Nagelkerke’s R-Square,
Pseudo_r2, residuals, probabilities, classification matrix.
I have also included steps for checking
tolerance so that we avoid cases that fail to converge. Generally the algorithm
is not very expensive for the RAM (because I have approximated the Hessian
Matrix) and the only external jar that I use is common math for multiplications
of matrices.
Regards
Re: [math] Logistic, Probit regerssion and Tolerance checks
Posted by Phil Steitz <ph...@gmail.com>.
On 9/7/12 9:22 AM, marios michaelidis wrote:
> Hi Giles,
> I will start exproling the links you gave me.
> I would suggest Logistic/probit regression to go under the regerssion package.
Yes, these should go in regression. Thanks in advance for your
contribution!
Phil
> Not that clustering is really any different, but it makes sense to find logistic "regerssion" in the a package named as such.
> Regards
> Marios
>
>> Date: Fri, 7 Sep 2012 17:48:12 +0200
>> From: gilles@harfang.homelinux.org
>> To: dev@commons.apache.org
>> Subject: Re: [math] Logistic, Probit regerssion and Tolerance checks
>>
>> Hi.
>>
>>> My name is Marios and I have very good
>>> academic background as well as I have worked as modeling analyst in big
>>> projects thus I have experience with prediction and optimization algorithms.
>>>
>> Welcome to Commons Math's forum.
>>
>>> Recently (before 5 months) , I started
>>> learning JAVA and I have made my life much more simple by using Java and Common
>>> math rather than depending on the common packages (SAS SPSS etc). Obviously, I
>>> owe common math a lot.
>> That's good to read.
>>
>>> I have noticed that the site does not
>>> have logistic regression and probit regression, very commonly used in
>>> classification problems. Additionally, The math package does not provide a way
>>> to assess Tolerance (or VIF), very commonly used to avoid multi-colinearity
>>> issues and singular matrices in optimization algorithms, prior to running them.
>>>
>>>
>>>
>>> I am willing to provide complete
>>> Logistic and Probit regression algorithms, optimizable by newton Raphson
>>> optimization maximum-likelihood method , in a very programmatically easy way
>>> (e.g regression(double matrix [][], double Target[], String
>>> Constant, double precision, double tolerance) , with academic references and
>>> very quick (3 secs for 60k set), with getter methods for all the common
>>> statistics such as null Deviance, Deviance, AIC, BIC, Chi-square f the model,
>>> betas, Wald statistics and p values, Cox_snell R square, Nagelkerke’s R-Square,
>>> Pseudo_r2, residuals, probabilities, classification matrix.
>> Such contributions would certainly be most welcome.
>>
>> But care must be taken in how to fit those features into Commons Math. I mean
>> that the new implementations should be integrated in the API of similar
>> functionalities, if they currently exist.
>>
>> IIUC, the proposal could be related to code currently in package
>> org.apache.commons.math3.stat.clustering
>> and/or to the pending improvements suggested in this report:
>> https://issues.apache.org/jira/browse/MATH-748
>>
>> [By the way, I wonder whether "clustering" should really be under "stat",
>> rather than, say, "optimization" or a package of its own, one level up.]
>>
>> In any case, it might be worth discussing here some design issues, before you
>> start adapting your code. At the same time, you should open tickets on the
>> bug tracking system:
>> https://issues.apache.org/jira/browse/MATH
>> Preferably, there should be a general request for "New feature"; then
>> several "sub-issues" could be linked to that one, each referring to a
>> specific task (typically a class, with its unit tests).
>>
>>> I have also included steps for checking
>>> tolerance so that we avoid cases that fail to converge. Generally the algorithm
>>> is not very expensive for the RAM (because I have approximated the Hessian
>>> Matrix) and the only external jar that I use is common math for multiplications
>>> of matrices.
>> Although the performance issue is certainly important, it is an
>> "implementation detail" that should not preempt a clear API (i.e. one that
>> reflects the mathematical concepts) and the reuse of existing classes (those
>> can be improved at the same time, if your proposal reveals that something is
>> lacking).
>>
>>
>> Thanks for your interest,
>> Gilles
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
>> For additional commands, e-mail: dev-help@commons.apache.org
>>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
For additional commands, e-mail: dev-help@commons.apache.org
RE: [math] Logistic, Probit regerssion and Tolerance checks
Posted by marios michaelidis <mi...@hotmail.com>.
Hi Giles,
I will start exproling the links you gave me.
I would suggest Logistic/probit regression to go under the regerssion package. Not that clustering is really any different, but it makes sense to find logistic "regerssion" in the a package named as such.
Regards
Marios
> Date: Fri, 7 Sep 2012 17:48:12 +0200
> From: gilles@harfang.homelinux.org
> To: dev@commons.apache.org
> Subject: Re: [math] Logistic, Probit regerssion and Tolerance checks
>
> Hi.
>
> >
> > My name is Marios and I have very good
> > academic background as well as I have worked as modeling analyst in big
> > projects thus I have experience with prediction and optimization algorithms.
> >
>
> Welcome to Commons Math's forum.
>
> >
> > Recently (before 5 months) , I started
> > learning JAVA and I have made my life much more simple by using Java and Common
> > math rather than depending on the common packages (SAS SPSS etc). Obviously, I
> > owe common math a lot.
>
> That's good to read.
>
> >
> > I have noticed that the site does not
> > have logistic regression and probit regression, very commonly used in
> > classification problems. Additionally, The math package does not provide a way
> > to assess Tolerance (or VIF), very commonly used to avoid multi-colinearity
> > issues and singular matrices in optimization algorithms, prior to running them.
> >
> >
> >
> > I am willing to provide complete
> > Logistic and Probit regression algorithms, optimizable by newton Raphson
> > optimization maximum-likelihood method , in a very programmatically easy way
> > (e.g regression(double matrix [][], double Target[], String
> > Constant, double precision, double tolerance) , with academic references and
> > very quick (3 secs for 60k set), with getter methods for all the common
> > statistics such as null Deviance, Deviance, AIC, BIC, Chi-square f the model,
> > betas, Wald statistics and p values, Cox_snell R square, Nagelkerke’s R-Square,
> > Pseudo_r2, residuals, probabilities, classification matrix.
>
> Such contributions would certainly be most welcome.
>
> But care must be taken in how to fit those features into Commons Math. I mean
> that the new implementations should be integrated in the API of similar
> functionalities, if they currently exist.
>
> IIUC, the proposal could be related to code currently in package
> org.apache.commons.math3.stat.clustering
> and/or to the pending improvements suggested in this report:
> https://issues.apache.org/jira/browse/MATH-748
>
> [By the way, I wonder whether "clustering" should really be under "stat",
> rather than, say, "optimization" or a package of its own, one level up.]
>
> In any case, it might be worth discussing here some design issues, before you
> start adapting your code. At the same time, you should open tickets on the
> bug tracking system:
> https://issues.apache.org/jira/browse/MATH
> Preferably, there should be a general request for "New feature"; then
> several "sub-issues" could be linked to that one, each referring to a
> specific task (typically a class, with its unit tests).
>
> > I have also included steps for checking
> > tolerance so that we avoid cases that fail to converge. Generally the algorithm
> > is not very expensive for the RAM (because I have approximated the Hessian
> > Matrix) and the only external jar that I use is common math for multiplications
> > of matrices.
>
> Although the performance issue is certainly important, it is an
> "implementation detail" that should not preempt a clear API (i.e. one that
> reflects the mathematical concepts) and the reuse of existing classes (those
> can be improved at the same time, if your proposal reveals that something is
> lacking).
>
>
> Thanks for your interest,
> Gilles
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
> For additional commands, e-mail: dev-help@commons.apache.org
>
Re: [math] Logistic, Probit regerssion and Tolerance checks
Posted by Gilles Sadowski <gi...@harfang.homelinux.org>.
Hi.
>
> My name is Marios and I have very good
> academic background as well as I have worked as modeling analyst in big
> projects thus I have experience with prediction and optimization algorithms.
>
Welcome to Commons Math's forum.
>
> Recently (before 5 months) , I started
> learning JAVA and I have made my life much more simple by using Java and Common
> math rather than depending on the common packages (SAS SPSS etc). Obviously, I
> owe common math a lot.
That's good to read.
>
> I have noticed that the site does not
> have logistic regression and probit regression, very commonly used in
> classification problems. Additionally, The math package does not provide a way
> to assess Tolerance (or VIF), very commonly used to avoid multi-colinearity
> issues and singular matrices in optimization algorithms, prior to running them.
>
>
>
> I am willing to provide complete
> Logistic and Probit regression algorithms, optimizable by newton Raphson
> optimization maximum-likelihood method , in a very programmatically easy way
> (e.g regression(double matrix [][], double Target[], String
> Constant, double precision, double tolerance) , with academic references and
> very quick (3 secs for 60k set), with getter methods for all the common
> statistics such as null Deviance, Deviance, AIC, BIC, Chi-square f the model,
> betas, Wald statistics and p values, Cox_snell R square, Nagelkerke’s R-Square,
> Pseudo_r2, residuals, probabilities, classification matrix.
Such contributions would certainly be most welcome.
But care must be taken in how to fit those features into Commons Math. I mean
that the new implementations should be integrated in the API of similar
functionalities, if they currently exist.
IIUC, the proposal could be related to code currently in package
org.apache.commons.math3.stat.clustering
and/or to the pending improvements suggested in this report:
https://issues.apache.org/jira/browse/MATH-748
[By the way, I wonder whether "clustering" should really be under "stat",
rather than, say, "optimization" or a package of its own, one level up.]
In any case, it might be worth discussing here some design issues, before you
start adapting your code. At the same time, you should open tickets on the
bug tracking system:
https://issues.apache.org/jira/browse/MATH
Preferably, there should be a general request for "New feature"; then
several "sub-issues" could be linked to that one, each referring to a
specific task (typically a class, with its unit tests).
> I have also included steps for checking
> tolerance so that we avoid cases that fail to converge. Generally the algorithm
> is not very expensive for the RAM (because I have approximated the Hessian
> Matrix) and the only external jar that I use is common math for multiplications
> of matrices.
Although the performance issue is certainly important, it is an
"implementation detail" that should not preempt a clear API (i.e. one that
reflects the mathematical concepts) and the reuse of existing classes (those
can be improved at the same time, if your proposal reveals that something is
lacking).
Thanks for your interest,
Gilles
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
For additional commands, e-mail: dev-help@commons.apache.org
Re: [math] Logistic, Probit regerssion and Tolerance checks
Posted by Phil Steitz <ph...@gmail.com>.
On 9/8/12 8:43 AM, marios michaelidis wrote:
> Hi Giles,
> Per your questions, everything is done, so I just uploaded the class in a txt format. You may run it too and tell me how it performs.
> The only thing I am not certain is in which package it should go. My proposal is to go with correlations as it checks multi-colinearity.
> If I got everything in the format you want and generally everything is right (which I rarely manage with the first try!), I will start uploading other works.
> Regards
> Marios
As noted on the JIRA, I think this should go in regression, as it is
really a regression diagnostic. At first, I thought about just
adding it to the OLS multiple regression class, but given that it
may be used by multiple regression classes, I agreee it is better to
keep it separate. Per Gilles' comments below, in order to get this
committed, we need unit tests and clean checkstyle results. Feel
free to ask here or privately if you need help getting your code
integrated into the [math] build or getting the static analysis
checks to run (checkstyle, findbugs).
Phil
>
>> Date: Sat, 8 Sep 2012 11:59:10 +0200
>> From: gilles@harfang.homelinux.org
>> To: dev@commons.apache.org
>> Subject: Re: [math] Logistic, Probit regerssion and Tolerance checks
>>
>> On Sat, Sep 08, 2012 at 10:50:29AM +0300, marios michaelidis wrote:
>>> Hi everyone,
>>> I will start with the tolerance and Vif checks, because it is smaller and I want to get familiar with the system. I have created a relevant ticket labelled as "
>>> I would like to include a VIF and TOLERANCE check for a 2 dimensional double array, vey commonly used by major packages to determine variables that cause multi-colinearity issues and should be excluded from the models"
>>> Shall I just upload the class?
>> If you think that all design issues are clear, and you are ready to adapt
>> your code for inclusion to Commons Math (or have done so already), then yes,
>> you could just attach your proposed code to the issue on the bug-tracking
>> system.
>>
>> If you want to discuss design issues (such as how to best adapt your code),
>> it is better to ask specific questions on this ML.
>>
>>> What else do I need to do?
>> It depends on where you are in the process...
>>
>> Did you already check out the CM source code?
>> Did you include your contribution in your working copy of CM?
>> Does it follow the style used in CM (API and formatting)?
>> Does it compile?
>> Did you write unit tests that cover all functionalities?
>> Do they pass?
>>
>>
>> Best regards,
>> Gilles
>>
>>> Regards
>>> Marios
>>>> From: ted.dunning@gmail.com
>>>> Date: Fri, 7 Sep 2012 16:09:34 -0700
>>>> Subject: Re: [math] Logistic, Probit regerssion and Tolerance checks
>>>> To: dev@commons.apache.org
>>>>
>>>> On Fri, Sep 7, 2012 at 4:06 PM, Phil Steitz <ph...@gmail.com> wrote:
>>>>
>>>>> On 9/7/12 3:28 PM, Ted Dunning wrote:
>>>>>>> Patches welcome!
>>>>>>>
>>>>>> Would normally love to, but I can't do more than kibitz based on other
>>>>>> implementations I have done.
>>>>>>
>>>>> OK, lets start with what Marios has and see where we can take it.
>>>>>
>>>>>
>>>> Cool.
>>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
>> For additional commands, e-mail: dev-help@commons.apache.org
>>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
For additional commands, e-mail: dev-help@commons.apache.org
RE: [math] Logistic, Probit regerssion and Tolerance checks
Posted by marios michaelidis <mi...@hotmail.com>.
Hi Giles,
Per your questions, everything is done, so I just uploaded the class in a txt format. You may run it too and tell me how it performs.
The only thing I am not certain is in which package it should go. My proposal is to go with correlations as it checks multi-colinearity.
If I got everything in the format you want and generally everything is right (which I rarely manage with the first try!), I will start uploading other works.
Regards
Marios
> Date: Sat, 8 Sep 2012 11:59:10 +0200
> From: gilles@harfang.homelinux.org
> To: dev@commons.apache.org
> Subject: Re: [math] Logistic, Probit regerssion and Tolerance checks
>
> On Sat, Sep 08, 2012 at 10:50:29AM +0300, marios michaelidis wrote:
> >
> > Hi everyone,
> > I will start with the tolerance and Vif checks, because it is smaller and I want to get familiar with the system. I have created a relevant ticket labelled as "
> > I would like to include a VIF and TOLERANCE check for a 2 dimensional double array, vey commonly used by major packages to determine variables that cause multi-colinearity issues and should be excluded from the models"
> > Shall I just upload the class?
>
> If you think that all design issues are clear, and you are ready to adapt
> your code for inclusion to Commons Math (or have done so already), then yes,
> you could just attach your proposed code to the issue on the bug-tracking
> system.
>
> If you want to discuss design issues (such as how to best adapt your code),
> it is better to ask specific questions on this ML.
>
> > What else do I need to do?
>
> It depends on where you are in the process...
>
> Did you already check out the CM source code?
> Did you include your contribution in your working copy of CM?
> Does it follow the style used in CM (API and formatting)?
> Does it compile?
> Did you write unit tests that cover all functionalities?
> Do they pass?
>
>
> Best regards,
> Gilles
>
> > Regards
> > Marios
> > > From: ted.dunning@gmail.com
> > > Date: Fri, 7 Sep 2012 16:09:34 -0700
> > > Subject: Re: [math] Logistic, Probit regerssion and Tolerance checks
> > > To: dev@commons.apache.org
> > >
> > > On Fri, Sep 7, 2012 at 4:06 PM, Phil Steitz <ph...@gmail.com> wrote:
> > >
> > > > On 9/7/12 3:28 PM, Ted Dunning wrote:
> > > > >> Patches welcome!
> > > > >>
> > > > > Would normally love to, but I can't do more than kibitz based on other
> > > > > implementations I have done.
> > > > >
> > > > OK, lets start with what Marios has and see where we can take it.
> > > >
> > > >
> > > Cool.
> >
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
> For additional commands, e-mail: dev-help@commons.apache.org
>
Re: [math] Logistic, Probit regerssion and Tolerance checks
Posted by Gilles Sadowski <gi...@harfang.homelinux.org>.
On Sat, Sep 08, 2012 at 10:50:29AM +0300, marios michaelidis wrote:
>
> Hi everyone,
> I will start with the tolerance and Vif checks, because it is smaller and I want to get familiar with the system. I have created a relevant ticket labelled as "
> I would like to include a VIF and TOLERANCE check for a 2 dimensional double array, vey commonly used by major packages to determine variables that cause multi-colinearity issues and should be excluded from the models"
> Shall I just upload the class?
If you think that all design issues are clear, and you are ready to adapt
your code for inclusion to Commons Math (or have done so already), then yes,
you could just attach your proposed code to the issue on the bug-tracking
system.
If you want to discuss design issues (such as how to best adapt your code),
it is better to ask specific questions on this ML.
> What else do I need to do?
It depends on where you are in the process...
Did you already check out the CM source code?
Did you include your contribution in your working copy of CM?
Does it follow the style used in CM (API and formatting)?
Does it compile?
Did you write unit tests that cover all functionalities?
Do they pass?
Best regards,
Gilles
> Regards
> Marios
> > From: ted.dunning@gmail.com
> > Date: Fri, 7 Sep 2012 16:09:34 -0700
> > Subject: Re: [math] Logistic, Probit regerssion and Tolerance checks
> > To: dev@commons.apache.org
> >
> > On Fri, Sep 7, 2012 at 4:06 PM, Phil Steitz <ph...@gmail.com> wrote:
> >
> > > On 9/7/12 3:28 PM, Ted Dunning wrote:
> > > >> Patches welcome!
> > > >>
> > > > Would normally love to, but I can't do more than kibitz based on other
> > > > implementations I have done.
> > > >
> > > OK, lets start with what Marios has and see where we can take it.
> > >
> > >
> > Cool.
>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
For additional commands, e-mail: dev-help@commons.apache.org
RE: [math] Logistic, Probit regerssion and Tolerance checks
Posted by marios michaelidis <mi...@hotmail.com>.
Hi everyone,
I will start with the tolerance and Vif checks, because it is smaller and I want to get familiar with the system. I have created a relevant ticket labelled as "
I would like to include a VIF and TOLERANCE check for a 2 dimensional double array, vey commonly used by major packages to determine variables that cause multi-colinearity issues and should be excluded from the models"
Shall I just upload the class?
What else do I need to do?
Regards
Marios
> From: ted.dunning@gmail.com
> Date: Fri, 7 Sep 2012 16:09:34 -0700
> Subject: Re: [math] Logistic, Probit regerssion and Tolerance checks
> To: dev@commons.apache.org
>
> On Fri, Sep 7, 2012 at 4:06 PM, Phil Steitz <ph...@gmail.com> wrote:
>
> > On 9/7/12 3:28 PM, Ted Dunning wrote:
> > >> Patches welcome!
> > >>
> > > Would normally love to, but I can't do more than kibitz based on other
> > > implementations I have done.
> > >
> > OK, lets start with what Marios has and see where we can take it.
> >
> >
> Cool.
Re: [math] Logistic, Probit regerssion and Tolerance checks
Posted by Ted Dunning <te...@gmail.com>.
On Fri, Sep 7, 2012 at 4:06 PM, Phil Steitz <ph...@gmail.com> wrote:
> On 9/7/12 3:28 PM, Ted Dunning wrote:
> >> Patches welcome!
> >>
> > Would normally love to, but I can't do more than kibitz based on other
> > implementations I have done.
> >
> OK, lets start with what Marios has and see where we can take it.
>
>
Cool.
Re: [math] Logistic, Probit regerssion and Tolerance checks
Posted by Phil Steitz <ph...@gmail.com>.
On 9/7/12 3:28 PM, Ted Dunning wrote:
> On Fri, Sep 7, 2012 at 3:24 PM, Phil Steitz <ph...@gmail.com> wrote:
>
>>> It might be interesting to think for a moment how easy it would be to
>>> support generalized linear regression in this same package. Small
>> changes
>>> to the loss function in the optimization should allow you to have not
>> just
>>> logistic and probit regression, but also to get Poisson regression and
>> SVM
>>> in the same framework.
>> +1
>> Patches welcome!
>>
> Would normally love to, but I can't do more than kibitz based on other
> implementations I have done.
>
OK, lets start with what Marios has and see where we can take it.
Phil
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
For additional commands, e-mail: dev-help@commons.apache.org
Re: [math] Logistic, Probit regerssion and Tolerance checks
Posted by Ted Dunning <te...@gmail.com>.
On Fri, Sep 7, 2012 at 3:24 PM, Phil Steitz <ph...@gmail.com> wrote:
> > It might be interesting to think for a moment how easy it would be to
> > support generalized linear regression in this same package. Small
> changes
> > to the loss function in the optimization should allow you to have not
> just
> > logistic and probit regression, but also to get Poisson regression and
> SVM
> > in the same framework.
>
> +1
> Patches welcome!
>
Would normally love to, but I can't do more than kibitz based on other
implementations I have done.
Re: [math] Logistic, Probit regerssion and Tolerance checks
Posted by Phil Steitz <ph...@gmail.com>.
On 9/7/12 2:22 PM, Ted Dunning wrote:
> This is great.
>
> A very useful feature would be to allow basic L_1 and L_2 regularization.
>
> This makes it much easier to avoid problems with separable problems.
>
> It might be interesting to think for a moment how easy it would be to
> support generalized linear regression in this same package. Small changes
> to the loss function in the optimization should allow you to have not just
> logistic and probit regression, but also to get Poisson regression and SVM
> in the same framework.
+1
Patches welcome!
Phil
>
> On Fri, Sep 7, 2012 at 3:22 AM, marios michaelidis <mi...@hotmail.com>wrote:
>
>> I am willing to provide complete
>> Logistic and Probit regression algorithms, optimizable by newton Raphson
>> optimization maximum-likelihood method , in a very programmatically easy
>> way
>> (e.g regression(double matrix [][], double Target[], String
>> Constant, double precision, double tolerance) , with academic references
>> and
>> very quick (3 secs for 60k set), with getter methods for all the common
>> statistics such as null Deviance, Deviance, AIC, BIC, Chi-square f the
>> model,
>> betas, Wald statistics and p values, Cox_snell R square, Nagelkerke’s
>> R-Square,
>> Pseudo_r2, residuals, probabilities, classification matrix.
>>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
For additional commands, e-mail: dev-help@commons.apache.org
Re: [math] Logistic, Probit regerssion and Tolerance checks
Posted by Ted Dunning <te...@gmail.com>.
This is great.
A very useful feature would be to allow basic L_1 and L_2 regularization.
This makes it much easier to avoid problems with separable problems.
It might be interesting to think for a moment how easy it would be to
support generalized linear regression in this same package. Small changes
to the loss function in the optimization should allow you to have not just
logistic and probit regression, but also to get Poisson regression and SVM
in the same framework.
On Fri, Sep 7, 2012 at 3:22 AM, marios michaelidis <mi...@hotmail.com>wrote:
> I am willing to provide complete
> Logistic and Probit regression algorithms, optimizable by newton Raphson
> optimization maximum-likelihood method , in a very programmatically easy
> way
> (e.g regression(double matrix [][], double Target[], String
> Constant, double precision, double tolerance) , with academic references
> and
> very quick (3 secs for 60k set), with getter methods for all the common
> statistics such as null Deviance, Deviance, AIC, BIC, Chi-square f the
> model,
> betas, Wald statistics and p values, Cox_snell R square, Nagelkerke’s
> R-Square,
> Pseudo_r2, residuals, probabilities, classification matrix.
>