You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@commons.apache.org by Phil Steitz <ph...@gmail.com> on 2011/07/13 21:35:23 UTC
[math] Refactoring multiple regression classes
To effectively integrate Greg's contributions on updating
regression, we really should refactor the multiple regression
hierarchy so that we have consistency in results reporting and data
definition. Here are some initial ideas. Better ideas and/or
patches welcome.
0. Deprecate MultipleLinearRegression (basically being replaced).
1. Create a LinearRegression interface with what is now in
UpdatingMultipleLinearRegression, minus the addObservation methods.
2. Define UpdatingLinearRegression extending LinearRegression
(adding back addObs*).
3. Define FixedLinearRegression extending LinearRegression, adding
the new*Data data definition methods now in
AbstractMultipleLinearRegression.
4. Deprecate AbstractMultipleLinearRegression.
5. Add AbstractFixedLinearRegression implementing
FixedLinearRegression and providing default impls for the new*Data
methods (taken from AbstractMultipleLinearRegression)
6. Modify OLSMultipleLinearRegression and
GLSMultipleLinearRegression to extend AbstractFixedLinearRegression,
possibly preserving MultipleLinearRegression implementation until 4.0.
7. If there is value to an abstract parent for the updating impls,
add this implementing UpdatingLinearRegression; otherwise just
implement directly.
We can talk about the names in 1-3. We can also talk about getting
rid of the interfaces entirely and just have LinearRegression and
its immediate children abstract classes with all abstract methods.
Or maybe others have entirely different and better ideas. Thanks
in advance for ideas on this. The one thing we really need is to
have the fixed matrix models report results using the new
RegressionResults object. Without that, we will not be able to
refactor the tests to enable cross-validation (and it will obviously
be confusing for users to have the classes implementing the same
functionality exposing completely different APIs).
Phil
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
For additional commands, e-mail: dev-help@commons.apache.org
Re: [math] Refactoring multiple regression classes
Posted by Greg Sterijevski <gs...@gmail.com>.
Sorry, sent the previous email without finishing (aparently gmail is not
idiot proof...)
So,
public class ConstrainedRegressionResults. extends RegressionResults{
private double[] lagrangian;
private double[] varcov;
public ConstrainedRegressionResults( double[] lagrangian, double[]
varcov, ...){
super( ... );
//copy over the data
}
}
The real growth and layering would occur in the results. Almost everything
in the linear regression world revolves around the sum of squares matrix or
its decomposition. The real action is what you do with it.
-Greg
On Wed, Jul 13, 2011 at 9:14 PM, Greg Sterijevski <gs...@gmail.com>wrote:
> Phil,
>
>
> "How exactly do interfaces make the hierarchy flatter in this case?
> I agree we should aim for as simple a structure as possible. The
> question is, what is that structure?"
>
> They may or may not make the structure different. Any design we come up
> with today is likely to be outmoded in 6 months. (In war throw your battle
> plans out the window after the first five minutes.) What I propose is an
> interface which is the most minimal set of functionality (identifiable now)
> that comprise regression. Over time, as we define more and more
> implementations of regression we might see further functionality which is
> common across regressions. These methods will migrate to the interface. The
> interface will grow organically. More importantly any dependency which is
> not too picky can use the interface reference, instead of referencing the
> concrete class. Dependencies which care, will and should have intimate
> knowledge of the class. Most pieces of code which depend on regression will
> not. The interface will not preclude abstract classes.
>
> The way I see it, you would have a core interface:
>
> public interface RegressionIface{
> boolean hasIntercept();
> long getN();
> void addObservation(double[] x, double y);
> void addObservation(double[] xy);
> RegressionResults regress()
> RegressionResults regress(int[] vars)
> }
>
> You would then have a subinterface
> public interface UpdatingRegression{
> void clear();
> void addObservations( double[][] x, double[] y);
> }
>
> Why should code which is running a regression need to know more than this?
> If for example, the QR regression and the SVD based regression share common
> functionality for manipulating the data incore, then they can inherit from
> an abstract base class which implements RegressionIface. The user in most
> cases will not care. He/she may care whether the data is incore or not, but
> thats about it.
>
> The real action, in my opinion, is in the RegressionResults class. Here you
> might need a bushy, thick tree. All regressions must generation an immutable
> RegressionResults. However, that is the minimum info that would be
> generated. We might, for example, have ConstrainedRegressionResults.
>
> public class ConstrainedRegressionResults. extends RegressionResults{
> private double[] lagrangian;
>
>
> }
>
>
Re: [math] Refactoring multiple regression classes
Posted by Greg Sterijevski <gs...@gmail.com>.
Does it make sense to define a class like RegressionGlobalFitStatistics? It
would look like:
public class RegressionGlobalFitStatistics{
... private constructors
public double getRsquared(){
return _rsq;
}
public double getAdjRsquared(){
return _adjrsq;
}
... and so forth
}
The RegressionResults class would use static factory methods in
RegressionGlobalFitStatistics to properly instantiate the different fit
measures. One could even keep flags if a variable is set or not, throwing an
exception is someone is attempting to access data that has not been set. For
example,
public double getAdjRsquared(){
if( _adjrsqIsSet ){
return _adjrsq;
}
throw new MathException("Illegal Request");
}
I think you need to disambiguate between not setting a variable and a NaN
returned because of bad data or some other issue.
The inclusion of GlobalFitStatistics class inside of RegressionResults would
minimize changes to RegressionResults, since GlobalFitStatistics is most
likely to be the class that grows and changes over time.
Thoughts?
-Greg
On Wed, Jul 13, 2011 at 11:53 PM, Greg Sterijevski
<gs...@gmail.com>wrote:
> I was not aware of the versioning document. Thank you. A few more things
> make sense now. It is still a bit bittersweet. In my opinion the interface
> is a programmer's most terrific tool. It keeps him honest as well. I
> understand your point though.
>
> -Greg
>
>
> On Wed, Jul 13, 2011 at 11:42 PM, Phil Steitz <ph...@gmail.com>wrote:
>
>> On 7/13/11 9:05 PM, Greg Sterijevski wrote:
>> > Phil,
>> >
>> > "Fortunately for users, maybe less fortunately for developers, we
>> > can't really "evolve" our API rapidly .... This is why we
>> > favor abstract classes over interfaces."
>> >
>> > Just so that I am sure I understand, let me restate the objection to an
>> > interface. An interface approach is more likely to be used broadly, so
>> any
>> > changes which might be made would have a disproportionately large
>> impact. Is
>> > this correct?
>>
>> No, our rule is we make incompatible changes only in major
>> releases. See [1]. Changes to interfaces are incompatible changes,
>> so whatever interfaces we define we have to be prepared to live with
>> until the next major release. We have no way of telling who is
>> using what or how, so our rules are expressed simply in terms of
>> source and binary compatibility. Abstract classes can add
>> (non-abstract) methods without breaking compatibility. That is why
>> we favor them where possible over interfaces.
>> >
>> > Isn't this a bit of a punt? On the surface the user gets a guarantee,
>> but
>> > the guarantee is a deadend. If we decide a different approach is
>> warranted,
>> > the object is deprecated and ditched at some point. More importantly, if
>> it
>> > is a successful interface isn't that a good thing?
>> > If it makes its way into
>> > a lot of other code, then I doubt we will need to make many changes.
>> Also, I
>> > foresee a two track evolution of features. The interface defines the
>> basic,
>> > most limited set of functionality. Everything else is throw into
>> abstract
>> > and concrete implementations. When a functionality seems to exist in
>> most
>> > derived classes, then the debate should commence about its inclusion in
>> the
>> > regression interface. The evolution would most likely be new features
>> in, as
>> > opposed to current features cut out. There would be a lag of probably a
>> year
>> > in moving a new feature up the inheritance structure to the interface.
>> Maybe
>> > your experience is different. I could be wrong about user demands...
>>
>> What it comes down to is being certain that the core material that
>> we put into the interface is not going to change. Our experience in
>> Commons has been that that is often very hard to do and in most
>> cases abstract classes will serve the same purpose without imposing
>> the same kind of constraints. The evolution you describe above is
>> natural and bound to happen. Trying to avoid interfaces gives us
>> more flexibility in timing, enabling innovative change between major
>> releases.
>>
>> Phil
>>
>> [1] http://commons.apache.org/releases/versioning.html
>> >
>> > Thanks,
>> >
>> > -Greg
>> >
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
>> For additional commands, e-mail: dev-help@commons.apache.org
>>
>>
>
Re: [math] Refactoring multiple regression classes
Posted by Greg Sterijevski <gs...@gmail.com>.
I was not aware of the versioning document. Thank you. A few more things
make sense now. It is still a bit bittersweet. In my opinion the interface
is a programmer's most terrific tool. It keeps him honest as well. I
understand your point though.
-Greg
On Wed, Jul 13, 2011 at 11:42 PM, Phil Steitz <ph...@gmail.com> wrote:
> On 7/13/11 9:05 PM, Greg Sterijevski wrote:
> > Phil,
> >
> > "Fortunately for users, maybe less fortunately for developers, we
> > can't really "evolve" our API rapidly .... This is why we
> > favor abstract classes over interfaces."
> >
> > Just so that I am sure I understand, let me restate the objection to an
> > interface. An interface approach is more likely to be used broadly, so
> any
> > changes which might be made would have a disproportionately large impact.
> Is
> > this correct?
>
> No, our rule is we make incompatible changes only in major
> releases. See [1]. Changes to interfaces are incompatible changes,
> so whatever interfaces we define we have to be prepared to live with
> until the next major release. We have no way of telling who is
> using what or how, so our rules are expressed simply in terms of
> source and binary compatibility. Abstract classes can add
> (non-abstract) methods without breaking compatibility. That is why
> we favor them where possible over interfaces.
> >
> > Isn't this a bit of a punt? On the surface the user gets a guarantee, but
> > the guarantee is a deadend. If we decide a different approach is
> warranted,
> > the object is deprecated and ditched at some point. More importantly, if
> it
> > is a successful interface isn't that a good thing?
> > If it makes its way into
> > a lot of other code, then I doubt we will need to make many changes.
> Also, I
> > foresee a two track evolution of features. The interface defines the
> basic,
> > most limited set of functionality. Everything else is throw into abstract
> > and concrete implementations. When a functionality seems to exist in most
> > derived classes, then the debate should commence about its inclusion in
> the
> > regression interface. The evolution would most likely be new features in,
> as
> > opposed to current features cut out. There would be a lag of probably a
> year
> > in moving a new feature up the inheritance structure to the interface.
> Maybe
> > your experience is different. I could be wrong about user demands...
>
> What it comes down to is being certain that the core material that
> we put into the interface is not going to change. Our experience in
> Commons has been that that is often very hard to do and in most
> cases abstract classes will serve the same purpose without imposing
> the same kind of constraints. The evolution you describe above is
> natural and bound to happen. Trying to avoid interfaces gives us
> more flexibility in timing, enabling innovative change between major
> releases.
>
> Phil
>
> [1] http://commons.apache.org/releases/versioning.html
> >
> > Thanks,
> >
> > -Greg
> >
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
> For additional commands, e-mail: dev-help@commons.apache.org
>
>
Re: [math] Refactoring multiple regression classes
Posted by Phil Steitz <ph...@gmail.com>.
On 7/13/11 9:05 PM, Greg Sterijevski wrote:
> Phil,
>
> "Fortunately for users, maybe less fortunately for developers, we
> can't really "evolve" our API rapidly .... This is why we
> favor abstract classes over interfaces."
>
> Just so that I am sure I understand, let me restate the objection to an
> interface. An interface approach is more likely to be used broadly, so any
> changes which might be made would have a disproportionately large impact. Is
> this correct?
No, our rule is we make incompatible changes only in major
releases. See [1]. Changes to interfaces are incompatible changes,
so whatever interfaces we define we have to be prepared to live with
until the next major release. We have no way of telling who is
using what or how, so our rules are expressed simply in terms of
source and binary compatibility. Abstract classes can add
(non-abstract) methods without breaking compatibility. That is why
we favor them where possible over interfaces.
>
> Isn't this a bit of a punt? On the surface the user gets a guarantee, but
> the guarantee is a deadend. If we decide a different approach is warranted,
> the object is deprecated and ditched at some point. More importantly, if it
> is a successful interface isn't that a good thing?
> If it makes its way into
> a lot of other code, then I doubt we will need to make many changes. Also, I
> foresee a two track evolution of features. The interface defines the basic,
> most limited set of functionality. Everything else is throw into abstract
> and concrete implementations. When a functionality seems to exist in most
> derived classes, then the debate should commence about its inclusion in the
> regression interface. The evolution would most likely be new features in, as
> opposed to current features cut out. There would be a lag of probably a year
> in moving a new feature up the inheritance structure to the interface. Maybe
> your experience is different. I could be wrong about user demands...
What it comes down to is being certain that the core material that
we put into the interface is not going to change. Our experience in
Commons has been that that is often very hard to do and in most
cases abstract classes will serve the same purpose without imposing
the same kind of constraints. The evolution you describe above is
natural and bound to happen. Trying to avoid interfaces gives us
more flexibility in timing, enabling innovative change between major
releases.
Phil
[1] http://commons.apache.org/releases/versioning.html
>
> Thanks,
>
> -Greg
>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
For additional commands, e-mail: dev-help@commons.apache.org
Re: [math] Refactoring multiple regression classes
Posted by Greg Sterijevski <gs...@gmail.com>.
Phil,
"Fortunately for users, maybe less fortunately for developers, we
can't really "evolve" our API rapidly .... This is why we
favor abstract classes over interfaces."
Just so that I am sure I understand, let me restate the objection to an
interface. An interface approach is more likely to be used broadly, so any
changes which might be made would have a disproportionately large impact. Is
this correct?
Isn't this a bit of a punt? On the surface the user gets a guarantee, but
the guarantee is a deadend. If we decide a different approach is warranted,
the object is deprecated and ditched at some point. More importantly, if it
is a successful interface isn't that a good thing? If it makes its way into
a lot of other code, then I doubt we will need to make many changes. Also, I
foresee a two track evolution of features. The interface defines the basic,
most limited set of functionality. Everything else is throw into abstract
and concrete implementations. When a functionality seems to exist in most
derived classes, then the debate should commence about its inclusion in the
regression interface. The evolution would most likely be new features in, as
opposed to current features cut out. There would be a lag of probably a year
in moving a new feature up the inheritance structure to the interface. Maybe
your experience is different. I could be wrong about user demands...
Thanks,
-Greg
Re: [math] Refactoring multiple regression classes
Posted by Phil Steitz <ph...@gmail.com>.
On 7/13/11 7:14 PM, Greg Sterijevski wrote:
> Phil,
>
> "How exactly do interfaces make the hierarchy flatter in this case?
> I agree we should aim for as simple a structure as possible. The
> question is, what is that structure?"
>
> They may or may not make the structure different. Any design we come up with
> today is likely to be outmoded in 6 months. (In war throw your battle plans
> out the window after the first five minutes.) What I propose is an interface
> which is the most minimal set of functionality (identifiable now) that
> comprise regression. Over time, as we define more and more implementations
> of regression we might see further functionality which is common across
> regressions. These methods will migrate to the interface. The interface will
> grow organically. More importantly any dependency which is not too picky can
> use the interface reference, instead of referencing the concrete class.
> Dependencies which care, will and should have intimate knowledge of the
> class. Most pieces of code which depend on regression will not. The
> interface will not preclude abstract classes.
Fortunately for users, maybe less fortunately for developers, we
can't really "evolve" our API rapidly and incrementally, unless that
evolution avoids backward-incompatible change. The reason for this
is that we combine bug fixes and API changes in point releases and
users need to be able to upgrade to point releases without having to
make code changes. We make incompatible changes in major releases
only. The good news is that we are in the runup right now to a
major release of [math], so we have once-every-few-years opportunity
to make incompatible changes. The maybe less wonderful news is that
what we design for 3.0 we will need to live with for a couple of
years, so we need to be careful not to lock ourselves in to design
constraints that will be hard to innovate within. This is why we
favor abstract classes over interfaces.
>
> The way I see it, you would have a core interface:
>
> public interface RegressionIface{
> boolean hasIntercept();
> long getN();
> void addObservation(double[] x, double y);
> void addObservation(double[] xy);
> RegressionResults regress()
> RegressionResults regress(int[] vars)
> }
>
> You would then have a subinterface
> public interface UpdatingRegression{
> void clear();
> void addObservations( double[][] x, double[] y);
> }
I thought about that model; but the "fixed model" versions may not
need to or want to support the "addAll" semantics - just setData. I
was thinking that addObservations above would be included in the
base, since it could always be implemented serially.
> Why should code which is running a regression need to know more than this?
> If for example, the QR regression and the SVD based regression share common
> functionality for manipulating the data incore, then they can inherit from
> an abstract base class which implements RegressionIface. The user in most
> cases will not care. He/she may care whether the data is incore or not, but
> thats about it.
Exactly, which is why I like your design at the top level.
>
> The real action, in my opinion, is in the RegressionResults class. Here you
> might need a bushy, thick tree. All regressions must generation an immutable
> RegressionResults. However, that is the minimum info that would be
> generated. We might, for example, have ConstrainedRegressionResults.
>
> public class ConstrainedRegressionResults. extends RegressionResults{
> private double[] lagrangian;
>
>
> }
Agree here again. RegressionResults should include only the basic
stuff that every model will include and subclasses will extend it.
>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
For additional commands, e-mail: dev-help@commons.apache.org
Re: [math] Refactoring multiple regression classes
Posted by Greg Sterijevski <gs...@gmail.com>.
Phil,
"How exactly do interfaces make the hierarchy flatter in this case?
I agree we should aim for as simple a structure as possible. The
question is, what is that structure?"
They may or may not make the structure different. Any design we come up with
today is likely to be outmoded in 6 months. (In war throw your battle plans
out the window after the first five minutes.) What I propose is an interface
which is the most minimal set of functionality (identifiable now) that
comprise regression. Over time, as we define more and more implementations
of regression we might see further functionality which is common across
regressions. These methods will migrate to the interface. The interface will
grow organically. More importantly any dependency which is not too picky can
use the interface reference, instead of referencing the concrete class.
Dependencies which care, will and should have intimate knowledge of the
class. Most pieces of code which depend on regression will not. The
interface will not preclude abstract classes.
The way I see it, you would have a core interface:
public interface RegressionIface{
boolean hasIntercept();
long getN();
void addObservation(double[] x, double y);
void addObservation(double[] xy);
RegressionResults regress()
RegressionResults regress(int[] vars)
}
You would then have a subinterface
public interface UpdatingRegression{
void clear();
void addObservations( double[][] x, double[] y);
}
Why should code which is running a regression need to know more than this?
If for example, the QR regression and the SVD based regression share common
functionality for manipulating the data incore, then they can inherit from
an abstract base class which implements RegressionIface. The user in most
cases will not care. He/she may care whether the data is incore or not, but
thats about it.
The real action, in my opinion, is in the RegressionResults class. Here you
might need a bushy, thick tree. All regressions must generation an immutable
RegressionResults. However, that is the minimum info that would be
generated. We might, for example, have ConstrainedRegressionResults.
public class ConstrainedRegressionResults. extends RegressionResults{
private double[] lagrangian;
}
Re: [math] Refactoring multiple regression classes
Posted by Phil Steitz <ph...@gmail.com>.
On 7/13/11 5:15 PM, Greg Sterijevski wrote:
> Hello All,
>
> Sorry for being a bit slow on the uptake... I am still in the wilds of
> numerical imprecision with the longley data. I am getting close to figuring
> out where the error is being accumulated.
>
> I agree that interfaces impose rigidity in the design. However, there are
> broad similarities in linear regression. Whether we are doing OLS, panel
> regression, ridge regression, robust regression, etc., the model is linear:
> Y = XB + e. The estimation technique may be complicated (or non-linear) ,
> but that is an implementation detail. I like interfaces because it forces a
> discipline on the code. It forces you to specify a minimum contract which
> oftentimes makes larger problems more tractable. The end user should be able
> to swap out one technique for another with minimal recoding. It also makes
> it easy to write RMI stubs and make distributed calls.
>
> I like abstract classes in general, but we might end up with an abstract
> class with no concrete methods, or even worse. We might have a very very
> deep inheritance tree ( AbstractRegression>>
> AbstractUpdatingLinearRegression>>
> PanelRegression>>
> OneWayFixedEffects. ).
How exactly do interfaces make the hierarchy flatter in this case?
I agree we should aim for as simple a structure as possible. The
question is, what is that structure?
>
> That being said, Phil and Ted, you guys are definitely the experts on the
> design. I thought I would add my opinion to the mix.
There are no "experts" here - or maybe we are *all* experts :) In
any case, your opinions are much appreciated. Apart from the
interface/abstract class question at the top of the hierarchy, do
you have other suggestions on how to do this? Can you see a simpler
model using interfaces instead of abstract classes at other levels?
>
> On the features front, as we deliberate over the design issues its important
> that we have an eye to what is missing.
This is probably the best argument against interfaces - they make it
basically impossible to add "what is missing" once you cut a release
including them. The ideas below are great and illustrate how we
need to keep things as flexible as possible. We should think about
how the refactored structure we are defining will accommodate them.
Phil
> Here are some features which I
> believe should be in the regression package:
>
> 1. A SVD based OLS regression. (because sometimes messing with eigenvalues
> is a must)
> 2. A functionality to impose arbitrary linear equality restrictions:
> Perhaps a regress method with the following signature,
> public RegressionResult regress( RealMatrix coeff, RealVector const);
> 3. Related to (2) linear hypothesis testing
> 4. Related to (2) estimates of the LaGrangian and its variance covariance
> matrix
> 5. Robust variance covariance estimators
> 6. Perhaps panel regression.
> This one is a bit larger than regression. We would need to track the
> other dimensions of an observation (if we are studying income for a group
> of people, we might track the individual, the year, race and so forth as
> other dimensions which impose level shifts in the hyperplane). The
> regression might need a pull mechanism to make two passes through the data.
> First pass builds things like means or augments the design matrix with dummy
> variables. The second pass would actually run the regression on the
> transformed data.
> 7. Some sort of redundancy indicator. This is especially important when you
> allow for parameter restrictions since you want to know which parameter is
> not being used.
> 8. Some meta structure to allow for stepwise regression.
>
> -Greg
>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
For additional commands, e-mail: dev-help@commons.apache.org
Re: [math] Refactoring multiple regression classes
Posted by Greg Sterijevski <gs...@gmail.com>.
Hello All,
Sorry for being a bit slow on the uptake... I am still in the wilds of
numerical imprecision with the longley data. I am getting close to figuring
out where the error is being accumulated.
I agree that interfaces impose rigidity in the design. However, there are
broad similarities in linear regression. Whether we are doing OLS, panel
regression, ridge regression, robust regression, etc., the model is linear:
Y = XB + e. The estimation technique may be complicated (or non-linear) ,
but that is an implementation detail. I like interfaces because it forces a
discipline on the code. It forces you to specify a minimum contract which
oftentimes makes larger problems more tractable. The end user should be able
to swap out one technique for another with minimal recoding. It also makes
it easy to write RMI stubs and make distributed calls.
I like abstract classes in general, but we might end up with an abstract
class with no concrete methods, or even worse. We might have a very very
deep inheritance tree ( AbstractRegression>>
AbstractUpdatingLinearRegression>>
PanelRegression>>
OneWayFixedEffects. ).
That being said, Phil and Ted, you guys are definitely the experts on the
design. I thought I would add my opinion to the mix.
On the features front, as we deliberate over the design issues its important
that we have an eye to what is missing. Here are some features which I
believe should be in the regression package:
1. A SVD based OLS regression. (because sometimes messing with eigenvalues
is a must)
2. A functionality to impose arbitrary linear equality restrictions:
Perhaps a regress method with the following signature,
public RegressionResult regress( RealMatrix coeff, RealVector const);
3. Related to (2) linear hypothesis testing
4. Related to (2) estimates of the LaGrangian and its variance covariance
matrix
5. Robust variance covariance estimators
6. Perhaps panel regression.
This one is a bit larger than regression. We would need to track the
other dimensions of an observation (if we are studying income for a group
of people, we might track the individual, the year, race and so forth as
other dimensions which impose level shifts in the hyperplane). The
regression might need a pull mechanism to make two passes through the data.
First pass builds things like means or augments the design matrix with dummy
variables. The second pass would actually run the regression on the
transformed data.
7. Some sort of redundancy indicator. This is especially important when you
allow for parameter restrictions since you want to know which parameter is
not being used.
8. Some meta structure to allow for stepwise regression.
-Greg
Re: [math] Refactoring multiple regression classes
Posted by Phil Steitz <ph...@gmail.com>.
On 7/13/11 12:45 PM, Ted Dunning wrote:
> +1 to the overall thrust here. This is a good thing.
>
> One item that I would like to add is that we should agree on the general
> direction before getting bogged down in debates on the details.
> Establishing some momentum in the direction of agreement might help shorten
> the ensuing spelling discussions.
Thanks, yeah that's why I laid out the straw man below. I agree
with your comment on interfaces as well, so my personal opinion is
that LinearRegression below should be an abstract class.
Phil
>
> On Wed, Jul 13, 2011 at 12:35 PM, Phil Steitz <ph...@gmail.com> wrote:
>
>> To effectively integrate Greg's contributions on updating
>> regression, we really should refactor the multiple regression
>> hierarchy so that we have consistency in results reporting and data
>> definition. Here are some initial ideas. Better ideas and/or
>> patches welcome.
>>
>> 0. Deprecate MultipleLinearRegression (basically being replaced).
>>
>> 1. Create a LinearRegression interface with what is now in
>> UpdatingMultipleLinearRegression, minus the addObservation methods.
>>
>> 2. Define UpdatingLinearRegression extending LinearRegression
>> (adding back addObs*).
>>
>> 3. Define FixedLinearRegression extending LinearRegression, adding
>> the new*Data data definition methods now in
>> AbstractMultipleLinearRegression.
>>
>> 4. Deprecate AbstractMultipleLinearRegression.
>>
>> 5. Add AbstractFixedLinearRegression implementing
>> FixedLinearRegression and providing default impls for the new*Data
>> methods (taken from AbstractMultipleLinearRegression)
>>
>> 6. Modify OLSMultipleLinearRegression and
>> GLSMultipleLinearRegression to extend AbstractFixedLinearRegression,
>> possibly preserving MultipleLinearRegression implementation until 4.0.
>>
>> 7. If there is value to an abstract parent for the updating impls,
>> add this implementing UpdatingLinearRegression; otherwise just
>> implement directly.
>>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
For additional commands, e-mail: dev-help@commons.apache.org
Re: [math] Refactoring multiple regression classes
Posted by Ted Dunning <te...@gmail.com>.
+1 to the overall thrust here. This is a good thing.
One item that I would like to add is that we should agree on the general
direction before getting bogged down in debates on the details.
Establishing some momentum in the direction of agreement might help shorten
the ensuing spelling discussions.
On Wed, Jul 13, 2011 at 12:35 PM, Phil Steitz <ph...@gmail.com> wrote:
> To effectively integrate Greg's contributions on updating
> regression, we really should refactor the multiple regression
> hierarchy so that we have consistency in results reporting and data
> definition. Here are some initial ideas. Better ideas and/or
> patches welcome.
>
> 0. Deprecate MultipleLinearRegression (basically being replaced).
>
> 1. Create a LinearRegression interface with what is now in
> UpdatingMultipleLinearRegression, minus the addObservation methods.
>
> 2. Define UpdatingLinearRegression extending LinearRegression
> (adding back addObs*).
>
> 3. Define FixedLinearRegression extending LinearRegression, adding
> the new*Data data definition methods now in
> AbstractMultipleLinearRegression.
>
> 4. Deprecate AbstractMultipleLinearRegression.
>
> 5. Add AbstractFixedLinearRegression implementing
> FixedLinearRegression and providing default impls for the new*Data
> methods (taken from AbstractMultipleLinearRegression)
>
> 6. Modify OLSMultipleLinearRegression and
> GLSMultipleLinearRegression to extend AbstractFixedLinearRegression,
> possibly preserving MultipleLinearRegression implementation until 4.0.
>
> 7. If there is value to an abstract parent for the updating impls,
> add this implementing UpdatingLinearRegression; otherwise just
> implement directly.
>
Re: [math] Refactoring multiple regression classes
Posted by Ted Dunning <te...@gmail.com>.
Interfaces are not a great idea in a changing API.
On Wed, Jul 13, 2011 at 12:35 PM, Phil Steitz <ph...@gmail.com> wrote:
> We can also talk about getting
> rid of the interfaces entirely and just have LinearRegression and
> its immediate children abstract classes with all abstract methods.
> Or maybe others have entirely different and better ideas.
>