You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@commons.apache.org by "Mark R. Diggory" <md...@latte.harvard.edu> on 2003/09/08 18:42:23 UTC
[math] Project Maturity?
I think we should discuss what really needs to get completed for the 1.0
release of math. I think there are three points of interest.
A.) If it isn't a major refactoring, it probably stands that we can
provide a feature in a later minor release, for example, "adding
confidence intervals", extending distributions with discrete capabilities.
B.) Many tasks are "ongoing" and are never "really" finished, ie
accuracy testing, checkstyle and javadoc.
C.) Many aspects of the project already out-perform even that which is
in commons proper. Documentation and JUnit testing is fairly thorough.
So I wonder, would it be logical to request a vote from the community
proper to consider that maturity of the math project and if its ready to
release a version? There will always be discovered issues. But we do
need to get to the point where we have actual users to discover those
issues.
-Mark Diggory
Here's a list of the tasks from the current task list:
1.) Develop user's guide following the package structure.
Provide any comments on this task here.
2.) Performance and accuracy testing.
If anyone is interested in helping out here, what we could really
use is a wider selection of test cases for the core numerical functions
and validation against either other packages (e.g. R for the statistical
stuff), verified datasets, or experiments comparing implement ions using
floats to doubles.
3.) Test Coverage.
Clover tests show gaps in test path coverage. Get all tests to 100%
coverage. Also improve test data and boundary conditions coverage.
4.) Code review.
Code review is a continuous process that all Contributors and
Developers should practice while working on the code base.
* Javadoc generation is still throwing warnings. Bring the
Javadoc into compliance (i.e. reach zero warnings).
* Verify that the code matches the documentation and identify
obvious inefficiencies or numerical problems. All feedback/suggestions
for improvement/patches are welcome.
* CheckStyle with modified properties still shows many errors.
Try to clean these up.
Algorithm Development Tasks
5.) Add confidence intervals to Univariate implementations.
Provide any comments on this task here.
Distributions.
6.) Extend distribution framework to support discrete distributions and
implement binomial and hypergeometric distributions.
Analysis.
* Framework and implementation strategie(s) for finding roots
or real-valued functions of one (real) variable.
* Cubic spline interpolation.
* CheckStyle with modified properties still shows many errors.
7.) Finalize the contents of MathUtils and StatUtils. Suggest any
additions -- ideally with patches -- to these utility classes.
Re: [math] Project Maturity?
Posted by __matthewHawthorne <ma...@phreaker.net>.
>>> 3.) Test Coverage.
>>
>>
>> The coverage report doesn't seem to be available from the jakarta
>> site. Can I have a look at it somewhere else?
>>
>
> Is this something we don't have setup properly in maven?
To my knowledge, the clover report isn't generated automatically by
maven. I think you need to add it to the <reports> tag in the project.xml.
Double check the maven docs, or the mailing list to be sure.
Re: [math] Project Maturity?
Posted by "Mark R. Diggory" <md...@latte.harvard.edu>.
Mark R. Diggory wrote:
> J.Pietschmann wrote:
>
>> Mark R. Diggory wrote:
>>> 4.) Code review.
>>
>>
>> I just discovered a bunch of printStackTrace() in BeanTransformer.
>> Any ideas how this should be handled in a better way? Throwing
>> a MathException?
>>
>
> Yes, that would probibly be better. I would have the transformer method
> in NumberTransformer actually throw a MathException and construct the
> MathExceptions in BeanTransformer to include the exception as the cause
>
> throw new MathException(e.getMessage(),e);
> -Mark
>
I noticed however, that the ListUnivariateImpl generally ignores failed
transform cases though, I'm not sure if this is a logical expected
behavior too. What do others think.
If a ListUnivariate can't transform one of its elements to a Number,
should it throw an exception, or should it silently moveonto the the
next element?
At least it should be logged so that it can be seen when debugging.
-Mark.
Re: [math] Project Maturity?
Posted by "Mark R. Diggory" <md...@latte.harvard.edu>.
J.Pietschmann wrote:
> Mark R. Diggory wrote:
>
>> 3.) Test Coverage.
>
> The coverage report doesn't seem to be available from the jakarta
> site. Can I have a look at it somewhere else?
>
Is this something we don't have setup properly in maven?
>> 4.) Code review.
>
> I just discovered a bunch of printStackTrace() in BeanTransformer.
> Any ideas how this should be handled in a better way? Throwing
> a MathException?
>
Yes, that would probibly be better. I would have the transformer method
in NumberTransformer actually throw a MathException and construct the
MathExceptions in BeanTransformer to include the exception as the cause
throw new MathException(e.getMessage(),e);
>> Algorithm Development Tasks
>> * Framework and implementation strategie(s) for finding roots
>> or real-valued functions of one (real) variable.
>> * Cubic spline interpolation.
>
> Should be already there?
Thats what needs to get reviewed, I know somethings were completed
without the task list getting updated, we need to make sure we are in sync.
>
> Additions:
> - Find a way to make [math] compile and run under JDK 1.3 again.
> This mainly consists of deriving the exceptions from [lang]
> NestableException and adding a GUMP/Maven dependency on [lang].
why is this required to run under 1.3? (stupid me I wasn't paying
attention to that discussion...) The dependency on lang and
NestableException makes sense to me in and of itself.
> - Clean up and unify exceptions. The ConvergenceException thrown
> in ContinuedFraction is a runtime exception not derived from
> MathException. The issues here:
> + Root finding should throw a ConvergenceException in case of
> convergence problems (falling out of the loop without
> decreasing the interval enough). However, because the
> exception is not derived from MathException, it must be
> handled separately by callers if this is done so.
> + Deriving ConvergenceException from the checked MathException
> would add a lot of throws clauses. This seems to be justified
> for the relative complex ContinuedFraction, but unfortunately
> this is used in the gamma function, where throwing a checked
> exception may throw off people (but then, java.io throws the
> checked IOException even for close()).
> This has been discussed in general terms, but without a conclusion
> for our specific case.
> - Make the default root finder run time configurable again. Well,
> if necessary.
> - I'd like to refactor the linear equation solving a bit, adding
> a new class for the matrix decomposition so that solvers can
> be written which can take advantage of special matrix forms
> (e.g. symmetric or tridiagonal). As usual, time is lacking.
Writing the framework would probibly be easier then implementing the
solvers, if the framework can be put in place before release 1.0, then
more solvers could be added in minor releases.
> - The "internal data exposure issue" in RealMatrix and in the
> interpolator has to be discussed and resolved. See the comment
> in the interpolator class for details.
Ok,
> - Possibly get rid of RootFinding in favor of moving the
> functionality to UnivariateRealSolverUtil? Dunno.
> - Rename UnivariateRealSolverUtil to UnivariateRealSolverUtils?
> - Rename UnivariateRealSolverImpl to UnivariateRealSolverSPI?
> - It has been already discussed but can't we load off DoubleArray
> and related classes to [collections] or the new [pcollections]?
>
I've thought about this, I'm concerned with the uncertainty of where
these primitive collections are going to land. I suspect we should wait
until that stablizes.
There's always going to be room to improve on this stuff, I want to see
a more producitve release schedule once we get out of the sandbox, don't
expect that just because something doesn't go into 1.0 that it'll be a
year or so before it shows up in the next release. I'd like to see
incremental releases on a stable schedule.
-Mark
RE: [math] Project Maturity?
Posted by Brent Worden <br...@worden.org>.
> -----Original Message-----
> From: J.Pietschmann [mailto:j3322ptm@yahoo.de]
> Sent: Monday, September 08, 2003 1:39 PM
> To: Jakarta Commons Developers List
> Subject: Re: [math] Project Maturity?
>
> I just discovered a bunch of printStackTrace() in BeanTransformer.
> Any ideas how this should be handled in a better way? Throwing
> a MathException?
One of my patches replaced these with logging calls. I know this doesn't
answer your questions; I just wanted to remove those statements without
changing the current behavior. I'm more than open to a debate.
> Additions:
> - Find a way to make [math] compile and run under JDK 1.3 again.
> This mainly consists of deriving the exceptions from [lang]
> NestableException and adding a GUMP/Maven dependency on [lang].
This is in one of my patches.
> - Make the default root finder run time configurable again. Well,
> if necessary.
This is in one of my patches. In now uses commons-discovery.
DiscoveryClass uses, among other things, a system property to determine the
real solver factory.
> - Possibly get rid of RootFinding in favor of moving the
> functionality to UnivariateRealSolverUtil? Dunno.
This is in one of my patches. RootFinding is now no more.
> - Rename UnivariateRealSolverUtil to UnivariateRealSolverUtils?
This is in one of my patches.
> - Rename UnivariateRealSolverImpl to UnivariateRealSolverSPI?
Since this in an abstract class. I would prefer
AbstractUnivariateRealSolver.
Brent Worden
http://www.brent.worden.org
RE: [math] Project Maturity?
Posted by Brent Worden <br...@worden.org>.
> -----Original Message-----
> From: J.Pietschmann [mailto:j3322ptm@yahoo.de]
> Sent: Monday, September 08, 2003 1:39 PM
> To: Jakarta Commons Developers List
> Subject: Re: [math] Project Maturity?
>
> I just discovered a bunch of printStackTrace() in BeanTransformer.
> Any ideas how this should be handled in a better way? Throwing
> a MathException?
One of my patches replaced these with logging calls. I know this doesn't
answer your questions; I just wanted to remove those statements without
changing the current behavior. I'm more than open to a debate.
> Additions:
> - Find a way to make [math] compile and run under JDK 1.3 again.
> This mainly consists of deriving the exceptions from [lang]
> NestableException and adding a GUMP/Maven dependency on [lang].
This is in one of my patches.
> - Make the default root finder run time configurable again. Well,
> if necessary.
This is in one of my patches. In now uses commons-discovery.
DiscoveryClass uses, among other things, a system property to determine the
real solver factory.
> - Possibly get rid of RootFinding in favor of moving the
> functionality to UnivariateRealSolverUtil? Dunno.
This is in one of my patches. RootFinding is now no more.
> - Rename UnivariateRealSolverUtil to UnivariateRealSolverUtils?
This is in one of my patches.
> - Rename UnivariateRealSolverImpl to UnivariateRealSolverSPI?
Since this in an abstract class. I would prefer
AbstractUnivariateRealSolver.
Brent Worden
http://www.brent.worden.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commons-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: commons-dev-help@jakarta.apache.org
Re: [math] Project Maturity?
Posted by "J.Pietschmann" <j3...@yahoo.de>.
Mark R. Diggory wrote:
> 3.) Test Coverage.
The coverage report doesn't seem to be available from the jakarta
site. Can I have a look at it somewhere else?
> 4.) Code review.
I just discovered a bunch of printStackTrace() in BeanTransformer.
Any ideas how this should be handled in a better way? Throwing
a MathException?
> Algorithm Development Tasks
> * Framework and implementation strategie(s) for finding roots or
> real-valued functions of one (real) variable.
> * Cubic spline interpolation.
Should be already there?
Additions:
- Find a way to make [math] compile and run under JDK 1.3 again.
This mainly consists of deriving the exceptions from [lang]
NestableException and adding a GUMP/Maven dependency on [lang].
- Clean up and unify exceptions. The ConvergenceException thrown
in ContinuedFraction is a runtime exception not derived from
MathException. The issues here:
+ Root finding should throw a ConvergenceException in case of
convergence problems (falling out of the loop without
decreasing the interval enough). However, because the
exception is not derived from MathException, it must be
handled separately by callers if this is done so.
+ Deriving ConvergenceException from the checked MathException
would add a lot of throws clauses. This seems to be justified
for the relative complex ContinuedFraction, but unfortunately
this is used in the gamma function, where throwing a checked
exception may throw off people (but then, java.io throws the
checked IOException even for close()).
This has been discussed in general terms, but without a conclusion
for our specific case.
- Make the default root finder run time configurable again. Well,
if necessary.
- I'd like to refactor the linear equation solving a bit, adding
a new class for the matrix decomposition so that solvers can
be written which can take advantage of special matrix forms
(e.g. symmetric or tridiagonal). As usual, time is lacking.
- The "internal data exposure issue" in RealMatrix and in the
interpolator has to be discussed and resolved. See the comment
in the interpolator class for details.
- Possibly get rid of RootFinding in favor of moving the
functionality to UnivariateRealSolverUtil? Dunno.
- Rename UnivariateRealSolverUtil to UnivariateRealSolverUtils?
- Rename UnivariateRealSolverImpl to UnivariateRealSolverSPI?
- It has been already discussed but can't we load off DoubleArray
and related classes to [collections] or the new [pcollections]?
J.Pietschmann
RE: [math] Project Maturity?
Posted by Brent Worden <br...@worden.org>.
> -----Original Message-----
> From: Mark R. Diggory [mailto:mdiggory@latte.harvard.edu]
> Sent: Monday, September 08, 2003 11:42 AM
> To: Jakarta Commons Developers List
> Subject: [math] Project Maturity?
>
>
> I think we should discuss what really needs to get completed for the 1.0
> release of math. I think there are three points of interest.
>
> A.) If it isn't a major refactoring, it probably stands that we can
> provide a feature in a later minor release, for example, "adding
> confidence intervals", extending distributions with discrete capabilities.
Agreed. With the hypergeometric patch, all the distributions promised for
the first release have been delivered.
>
> B.) Many tasks are "ongoing" and are never "really" finished, ie
> accuracy testing, checkstyle and javadoc.
Again I agree. With my latest javadoc patch, checkstyle violations were
down to under 20 and javadoc warnings under 50. This I believe is
acceptable since other commons projects have checkstyle errors numbering in
the thousands.
>
> C.) Many aspects of the project already out-perform even that which is
> in commons proper. Documentation and JUnit testing is fairly thorough.
The only thing I would like to see (but it doesn't have to happen before a
release) is the expanding of the user guide.
>
> So I wonder, would it be logical to request a vote from the community
> proper to consider that maturity of the math project and if its ready to
> release a version? There will always be discovered issues. But we do
> need to get to the point where we have actual users to discover those
> issues.
>
> -Mark Diggory
>
I have no problem with having a vote.
>
>
> Here's a list of the tasks from the current task list:
>
> 1.) Develop user's guide following the package structure.
> Provide any comments on this task here.
This would be the only thing, at this time, I think we should work on prior
to a vote. However, since the javadoc is quit complete, not having this
done should not impact any vote outcome by much.
>
> 2.) Performance and accuracy testing.
> If anyone is interested in helping out here, what we could really
> use is a wider selection of test cases for the core numerical functions
> and validation against either other packages (e.g. R for the statistical
> stuff), verified datasets, or experiments comparing implement ions using
> floats to doubles.
>
> 3.) Test Coverage.
> Clover tests show gaps in test path coverage. Get all tests to 100%
> coverage. Also improve test data and boundary conditions coverage.
FYI, 100% coverage is impossible with our design decisions. There are
private, default constructors that never get called and are not intended to
get called. As such, the methods can never be covered during test
execution.
With my test-coverage patch, I got just over 90% coverage. There were a
couple of classes with individual coverage of <50% which, if test were
written, could boost the coverage closer to 95%.
In short, I think our unit test are more than adequate and we should change
the 100% notion on the site to something realistic like 90% or 95%.
Brent Worden
http://www.brent.worden.org
RE: [math] Project Maturity?
Posted by Brent Worden <br...@worden.org>.
> -----Original Message-----
> From: Mark R. Diggory [mailto:mdiggory@latte.harvard.edu]
> Sent: Monday, September 08, 2003 11:42 AM
> To: Jakarta Commons Developers List
> Subject: [math] Project Maturity?
>
>
> I think we should discuss what really needs to get completed for the 1.0
> release of math. I think there are three points of interest.
>
> A.) If it isn't a major refactoring, it probably stands that we can
> provide a feature in a later minor release, for example, "adding
> confidence intervals", extending distributions with discrete capabilities.
Agreed. With the hypergeometric patch, all the distributions promised for
the first release have been delivered.
>
> B.) Many tasks are "ongoing" and are never "really" finished, ie
> accuracy testing, checkstyle and javadoc.
Again I agree. With my latest javadoc patch, checkstyle violations were
down to under 20 and javadoc warnings under 50. This I believe is
acceptable since other commons projects have checkstyle errors numbering in
the thousands.
>
> C.) Many aspects of the project already out-perform even that which is
> in commons proper. Documentation and JUnit testing is fairly thorough.
The only thing I would like to see (but it doesn't have to happen before a
release) is the expanding of the user guide.
>
> So I wonder, would it be logical to request a vote from the community
> proper to consider that maturity of the math project and if its ready to
> release a version? There will always be discovered issues. But we do
> need to get to the point where we have actual users to discover those
> issues.
>
> -Mark Diggory
>
I have no problem with having a vote.
>
>
> Here's a list of the tasks from the current task list:
>
> 1.) Develop user's guide following the package structure.
> Provide any comments on this task here.
This would be the only thing, at this time, I think we should work on prior
to a vote. However, since the javadoc is quit complete, not having this
done should not impact any vote outcome by much.
>
> 2.) Performance and accuracy testing.
> If anyone is interested in helping out here, what we could really
> use is a wider selection of test cases for the core numerical functions
> and validation against either other packages (e.g. R for the statistical
> stuff), verified datasets, or experiments comparing implement ions using
> floats to doubles.
>
> 3.) Test Coverage.
> Clover tests show gaps in test path coverage. Get all tests to 100%
> coverage. Also improve test data and boundary conditions coverage.
FYI, 100% coverage is impossible with our design decisions. There are
private, default constructors that never get called and are not intended to
get called. As such, the methods can never be covered during test
execution.
With my test-coverage patch, I got just over 90% coverage. There were a
couple of classes with individual coverage of <50% which, if test were
written, could boost the coverage closer to 95%.
In short, I think our unit test are more than adequate and we should change
the 100% notion on the site to something realistic like 90% or 95%.
Brent Worden
http://www.brent.worden.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commons-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: commons-dev-help@jakarta.apache.org