You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@commons.apache.org by Kasper Nielsen <ne...@kav.dk> on 2003/11/05 14:54:58 UTC
[Math] common-math and bloated 3rd party libraries
Hi gang,
I love commons-math, one problem though!
lets take a look at the dependecies
common-lang: 189 kb
commons-beanutils: 116 kb
commons-collections-SNAPSHOT.jar 463
commons-discovery 70 kb
commons-logging-1.0.3.jar 31 kb kb
Thats 850 kb!!! of 3rd party libraries that are only used in a few places.
So to calculate a simple mean I need to include around 6 jars (including
commons-math)
So lets get the list down a bit.
* Commons-lang
Getting rid of Commons-lang is pretty easy since it is only used in one
place: MathException
Solution : Let MathException extend Exception instead of
NestableException. There aren't really anywhere we use the ability to
nest Exceptions inside other Exceptions in commons-math.
* Commons-collections
Getting rid of commons-collections is also pretty easy
Solution: Getting a copy of HashBag (and the Testcase) and put into
math.util (no need to copy the interface)
now we got rid of ~ 650 kb in around 2 minutes, 3 jars left, this is a
fun game!!
* Commons-Beanutils
Okay the transformers are nice but come on how many people are going to
use them?
Solution: put them into a new small library: commons-math-transformers.jar
** Commons-Discovery
KISS keep it simple stupid, who on earth is going to provide there own
UnivariateRealSolverFactory??
and for those few people that need it... I think they are smart enough
to do figure it out themself.
Solution: remove it (or do something like we do for commons-logging)
** Commons-logging
Lastly commons-logging...
I would think returning NaN is enough, but okay if people insist we can
do something like (pseudo code)
public class logutil
static Method logMethod;
static {
try
{
Class clazz = Class.forName("commons.LogFactory");
logMethod = clazz.getMethod("error");
}
catch (ClassNotFoundException e) {}
}
public static logError(String msg, Throwable t)
{
if (logMethod!=null)
{
logMethod.invoke(msg + t);
}
}
}
and whoops we have now gotten rid of all the libraries, and we have easy
embeddable little commons math jar.
regards
Kasper
--------
Kasper Nielsen
kaspern at apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commons-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: commons-dev-help@jakarta.apache.org
Re: [Math] common-math and bloated 3rd party libraries
Posted by "Mark R. Diggory" <md...@latte.harvard.edu>.
Brian McCallister wrote:
>
> On Wednesday, November 5, 2003, at 09:26 AM, __matthewHawthorne wrote:
> Why does a math module depend on a logging module for deployment? Have
> the unit tests log, not the math functions.
>
> =)
>
> -Brian
Bright Idea...I hadn't really thought about that. :-)
--
Mark Diggory
Software Developer
Harvard MIT Data Center
http://www.hmdc.harvard.edu
---------------------------------------------------------------------
To unsubscribe, e-mail: commons-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: commons-dev-help@jakarta.apache.org
Re: [Math] common-math and bloated 3rd party libraries
Posted by Brian McCallister <mc...@forthillcompany.com>.
On Wednesday, November 5, 2003, at 09:26 AM, __matthewHawthorne wrote:
> I, for one, like the idea of commons projects depending on each other
> when necessary. There is always a lot of controversy with regards to
> "including another jar" that I don't quite understand. I agree that
> if there are only 1 or 2 references, it may be reasonable to include
> the dependencies as package private classes, or make more of an effort
> to avoid them in the first place. But this avoidance of code reuse
> sometimes disturbs me.
>
> Are you short on disk space or something? To me, 850 kb isn't really
> that much.
Reuse where the price of reuse is lower than the price of a better
design is fine, but here the better design cost is negligible. Don't
add dependencies unless you really need them, please.
Why does a math module depend on a logging module for deployment? Have
the unit tests log, not the math functions.
=)
-Brian
Re: [Math] common-math and bloated 3rd party libraries
Posted by __matthewHawthorne <ma...@phreaker.net>.
I, for one, like the idea of commons projects depending on each other
when necessary. There is always a lot of controversy with regards to
"including another jar" that I don't quite understand. I agree that if
there are only 1 or 2 references, it may be reasonable to include the
dependencies as package private classes, or make more of an effort to
avoid them in the first place. But this avoidance of code reuse
sometimes disturbs me.
Are you short on disk space or something? To me, 850 kb isn't really
that much.
Kasper Nielsen wrote:
> Hi gang,
>
> I love commons-math, one problem though!
>
> lets take a look at the dependecies
>
> common-lang: 189 kb
> commons-beanutils: 116 kb
> commons-collections-SNAPSHOT.jar 463
> commons-discovery 70 kb
> commons-logging-1.0.3.jar 31 kb kb
>
> Thats 850 kb!!! of 3rd party libraries that are only used in a few places.
> So to calculate a simple mean I need to include around 6 jars (including
> commons-math)
>
> So lets get the list down a bit.
>
> * Commons-lang
> Getting rid of Commons-lang is pretty easy since it is only used in one
> place: MathException
> Solution : Let MathException extend Exception instead of
> NestableException. There aren't really anywhere we use the ability to
> nest Exceptions inside other Exceptions in commons-math.
>
> * Commons-collections
> Getting rid of commons-collections is also pretty easy
> Solution: Getting a copy of HashBag (and the Testcase) and put into
> math.util (no need to copy the interface)
>
> now we got rid of ~ 650 kb in around 2 minutes, 3 jars left, this is a
> fun game!!
>
> * Commons-Beanutils
> Okay the transformers are nice but come on how many people are going to
> use them?
> Solution: put them into a new small library: commons-math-transformers.jar
>
> ** Commons-Discovery
> KISS keep it simple stupid, who on earth is going to provide there own
> UnivariateRealSolverFactory??
> and for those few people that need it... I think they are smart enough
> to do figure it out themself.
> Solution: remove it (or do something like we do for commons-logging)
>
> ** Commons-logging
> Lastly commons-logging...
> I would think returning NaN is enough, but okay if people insist we can
> do something like (pseudo code)
>
> public class logutil
> static Method logMethod;
> static {
> try
> {
> Class clazz = Class.forName("commons.LogFactory");
> logMethod = clazz.getMethod("error");
> }
> catch (ClassNotFoundException e) {}
> }
> public static logError(String msg, Throwable t)
> {
> if (logMethod!=null)
> {
> logMethod.invoke(msg + t);
> }
> }
> }
>
> and whoops we have now gotten rid of all the libraries, and we have easy
> embeddable little commons math jar.
>
> regards
> Kasper
>
> --------
> Kasper Nielsen
> kaspern at apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commons-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: commons-dev-help@jakarta.apache.org
Re: [Math] common-math and bloated 3rd party libraries
Posted by "Mark R. Diggory" <md...@latte.harvard.edu>.
Hehehe, thats a novel idea. Ok to be devils advocate...
Your coming at this from "one jar" perspective. Which leads me to
wonder why having math be "one jar" is important to you? Can you please
elaborate on this?
And to the rest of the community I postulate: Is this a critical usage case?
The commons libraries are planned/designed to be used together. So we
make great gains in removing replicated effort by using them when the
proper situation provides itself. Key word here "proper". Because in the
long run, the circular dependency issues and usage of our math component
also becomes critical to any group who doesn't wish to drag along a
bunch of dependencies, and this may very well occur, even with another
commons project using our library. So I'll go through and try to make
some appropriate comments below.
Kasper Nielsen wrote:
> Hi gang,
>
> I love commons-math, one problem though!
>
> lets take a look at the dependecies
>
> common-lang: 189 kb
> commons-beanutils: 116 kb
> commons-collections-SNAPSHOT.jar 463
> commons-discovery 70 kb
> commons-logging-1.0.3.jar 31 kb kb
>
> Thats 850 kb!!! of 3rd party libraries that are only used in a few places.
> So to calculate a simple mean I need to include around 6 jars (including
> commons-math)
>
> So lets get the list down a bit.
>
> * Commons-lang
> Getting rid of Commons-lang is pretty easy since it is only used in one
> place: MathException
> Solution : Let MathException extend Exception instead of
> NestableException. There aren't really anywhere we use the ability to
> nest Exceptions inside other Exceptions in commons-math.
>
Really, I'm not so sure I really like NestableException now that I look
at it. It just adds a whole bunch of complexity to the simple and
integral idea that something throws an exception and you catch it. I'm
not sure, but I think the idea of throwing the same exception object
multiple times is just horrid. I wouldn't mind seeing it go away from
our Exceptions. Others may think otherwise...
> * Commons-collections
> Getting rid of commons-collections is also pretty easy
> Solution: Getting a copy of HashBag (and the Testcase) and put into
> math.util (no need to copy the interface)
>
Not so sure I'm into moving classes out of other commons projects and
adding them to ours. HashBag is a fairly generic and useful class,
chances are that if your using commons-math you'll probably also want to
be using commons-collections.
> now we got rid of ~ 650 kb in around 2 minutes, 3 jars left, this is a
> fun game!!
>
> * Commons-Beanutils
> Okay the transformers are nice but come on how many people are going to
> use them?
> Solution: put them into a new small library: commons-math-transformers.jar
>
I'm thinking it would be just as easy to directly use the reflection api
in our case.
> ** Commons-Discovery
> KISS keep it simple stupid, who on earth is going to provide there own
> UnivariateRealSolverFactory??
> and for those few people that need it... I think they are smart enough
> to do figure it out themself.
> Solution: remove it (or do something like we do for commons-logging)
>
Its also used to discover new distribution factories under
o.a.m.stat.distributions Its interesting, I guess I don't understand
what "discovery" really adds, as a feature, on top of Java's own Service
Provider discovery? Anyways, is a pretty small library, I'm less
inclined to chuck it because of the "bloat" issue.
> ** Commons-logging
> Lastly commons-logging...
> I would think returning NaN is enough, but okay if people insist we can
> do something like (pseudo code)
>
> public class logutil
> static Method logMethod;
> static {
> try
> {
> Class clazz = Class.forName("commons.LogFactory");
> logMethod = clazz.getMethod("error");
> }
> catch (ClassNotFoundException e) {}
> }
> public static logError(String msg, Throwable t)
> {
> if (logMethod!=null)
> {
> logMethod.invoke(msg + t);
> }
> }
> }
Logging is a damn good thing to do in a standardized way, commons
logging really provides a solid transitional api between commons tools
and other logging api's. Its tiny, I'm not sure I think it's a great
idea to replace it with custom code.
The above code scares me because reflection always adds a "layer" of
complexity to stack traces and this type of approach makes it harder to
interpret the logging and errors.
>
> and whoops we have now gotten rid of all the libraries, and we have easy
> embeddable little commons math jar.
>
> regards
> Kasper
>
> --------
> Kasper Nielsen
> kaspern at apache.org
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: commons-dev-unsubscribe@jakarta.apache.org
> For additional commands, e-mail: commons-dev-help@jakarta.apache.org
>
--
Mark Diggory
Software Developer
Harvard MIT Data Center
http://www.hmdc.harvard.edu
---------------------------------------------------------------------
To unsubscribe, e-mail: commons-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: commons-dev-help@jakarta.apache.org
RE: [Math] common-math and bloated 3rd party libraries
Posted by "Todd V. Jonker" <To...@ConsciousCode.com>.
On Wed, 5 Nov 2003 06:54:14 -0800 (PST), "David Graham"
<gr...@yahoo.com> said:
> Agreed, especially because Jakarta's mission is to create *server*
> side libraries.
[...]
> The need to support 1.3 is diminishing over time. Java 1.4 is
> available and runs well on all the major platforms I can think of.
While I tend to be in the "go with 1.4" camp, this is MUCH easier tent to
pitch in NON-server environments. Commercial J2EE containers tend to be
dinosaurs with respect to JDK versions, and commercial users of said
containers tend to be dinosaurs too. So I typically see sites that are a
couple versions behind in upgrading their container, so their developers
are stuck back in 1.3-land.
I also want to gripe mildly about the server-side mission, since it feels
myopic to me. The lines between client and server blur more every day.
With JNLP it is easy and beneficial to shove increasing functionality to
the client side, which means downloading jars to the desktop.
Cheers,
.T.
---------------------------------------------------------------------
To unsubscribe, e-mail: commons-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: commons-dev-help@jakarta.apache.org
Re: [Math] common-math and bloated 3rd party libraries
Posted by "Mark R. Diggory" <md...@latte.harvard.edu>.
David Graham wrote:
>
> In this case, it looks like commons-lang and commons-logging are only
> needed because math doesn't use Java 1.4 as the base level. Moving to
> Java 1.4 has the advantage of providing exception chaining and logging in
> the Java core and eliminates 2 jars. Obviously, the disadvantage is that
> 1.3 users can't use commons-math.
>
> The need to support 1.3 is diminishing over time. Java 1.4 is available
> and runs well on all the major platforms I can think of.
>
> David
And really, in the long run, if your developing and your stuck back on
1.3.1, you have alot greater issues to be focusing on then adding a new
library to your tool. If your not focusing on making the transition off
an older obsolete JVM , your not going to be able to run on current
platforms. Specifically, why should we tailor our packages for your
obsolete code?
-Mark
--
Mark Diggory
Software Developer
Harvard MIT Data Center
http://www.hmdc.harvard.edu
---------------------------------------------------------------------
To unsubscribe, e-mail: commons-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: commons-dev-help@jakarta.apache.org
RE: [Math] common-math and bloated 3rd party libraries
Posted by David Graham <gr...@yahoo.com>.
--- Eric Pugh <ep...@upstate.com> wrote:
> This backlash against multiple commons jars is happening in a lot of
> places.
> However, I think it is a bit shortsighted. If you are in a non server
> environment, I understand the problem, but in a server environment with
> lots
> of harddrive space, I don't.
Agreed, especially because Jakarta's mission is to create *server* side
libraries.
> Additionally, since in a server app you
> are
> likely to need all thoses dependencies any way. I think almost every
> app I
> work on has commons-lang, commons-loggin, and commons-collections
> included.
> And then depending on what else, commons-discovery and commons-beanutils
> show up all the time!
>
> By removing the dependency on commons-lang etc you also remove the
> ability
> to leverage their code when something better comes out. You end up
> copying
> and pasting more and more out of all these projects in math. And don't
> get
> the benefit of the testing, bugfixing etc they go through..
In this case, it looks like commons-lang and commons-logging are only
needed because math doesn't use Java 1.4 as the base level. Moving to
Java 1.4 has the advantage of providing exception chaining and logging in
the Java core and eliminates 2 jars. Obviously, the disadvantage is that
1.3 users can't use commons-math.
The need to support 1.3 is diminishing over time. Java 1.4 is available
and runs well on all the major platforms I can think of.
David
>
> Maybe a better approach is to try and figure out why things like
> commons-collections are so big, and should they be shrunk down or
> partioned?
>
> Eric
>
> > -----Original Message-----
> > From: Kasper Nielsen [mailto:news@kav.dk]
> > Sent: Wednesday, November 05, 2003 2:55 PM
> > To: commons-dev@jakarta.apache.org
> > Subject: [Math] common-math and bloated 3rd party libraries
> >
> >
> > Hi gang,
> >
> > I love commons-math, one problem though!
> >
> > lets take a look at the dependecies
> >
> > common-lang: 189 kb
> > commons-beanutils: 116 kb
> > commons-collections-SNAPSHOT.jar 463
> > commons-discovery 70 kb
> > commons-logging-1.0.3.jar 31 kb kb
> >
> > Thats 850 kb!!! of 3rd party libraries that are only used in
> > a few places.
> > So to calculate a simple mean I need to include around 6 jars
> > (including
> > commons-math)
> >
> > So lets get the list down a bit.
> >
> > * Commons-lang
> > Getting rid of Commons-lang is pretty easy since it is only
> > used in one
> > place: MathException
> > Solution : Let MathException extend Exception instead of
> > NestableException. There aren't really anywhere we use the ability to
> > nest Exceptions inside other Exceptions in commons-math.
> >
> > * Commons-collections
> > Getting rid of commons-collections is also pretty easy
> > Solution: Getting a copy of HashBag (and the Testcase) and put into
> > math.util (no need to copy the interface)
> >
> > now we got rid of ~ 650 kb in around 2 minutes, 3 jars left,
> > this is a
> > fun game!!
> >
> > * Commons-Beanutils
> > Okay the transformers are nice but come on how many people
> > are going to
> > use them?
> > Solution: put them into a new small library:
> > commons-math-transformers.jar
> >
> > ** Commons-Discovery
> > KISS keep it simple stupid, who on earth is going to provide
> > there own
> > UnivariateRealSolverFactory??
> > and for those few people that need it... I think they are
> > smart enough
> > to do figure it out themself.
> > Solution: remove it (or do something like we do for commons-logging)
> >
> > ** Commons-logging
> > Lastly commons-logging...
> > I would think returning NaN is enough, but okay if people
> > insist we can
> > do something like (pseudo code)
> >
> > public class logutil
> > static Method logMethod;
> > static {
> > try
> > {
> > Class clazz = Class.forName("commons.LogFactory");
> > logMethod = clazz.getMethod("error");
> > }
> > catch (ClassNotFoundException e) {}
> > }
> > public static logError(String msg, Throwable t)
> > {
> > if (logMethod!=null)
> > {
> > logMethod.invoke(msg + t);
> > }
> > }
> > }
> >
> > and whoops we have now gotten rid of all the libraries, and
> > we have easy
> > embeddable little commons math jar.
> >
> > regards
> > Kasper
> >
> > --------
> > Kasper Nielsen
> > kaspern at apache.org
> >
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: commons-dev-unsubscribe@jakarta.apache.org
> > For additional commands, e-mail: commons-dev-help@jakarta.apache.org
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: commons-dev-unsubscribe@jakarta.apache.org
> For additional commands, e-mail: commons-dev-help@jakarta.apache.org
>
__________________________________
Do you Yahoo!?
Protect your identity with Yahoo! Mail AddressGuard
http://antispam.yahoo.com/whatsnewfree
---------------------------------------------------------------------
To unsubscribe, e-mail: commons-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: commons-dev-help@jakarta.apache.org
RE: [Math] common-math and bloated 3rd party libraries
Posted by Eric Pugh <ep...@upstate.com>.
This backlash against multiple commons jars is happening in a lot of places.
However, I think it is a bit shortsighted. If you are in a non server
environment, I understand the problem, but in a server environment with lots
of harddrive space, I don't. Additionally, since in a server app you are
likely to need all thoses dependencies any way. I think almost every app I
work on has commons-lang, commons-loggin, and commons-collections included.
And then depending on what else, commons-discovery and commons-beanutils
show up all the time!
By removing the dependency on commons-lang etc you also remove the ability
to leverage their code when something better comes out. You end up copying
and pasting more and more out of all these projects in math. And don't get
the benefit of the testing, bugfixing etc they go through..
Maybe a better approach is to try and figure out why things like
commons-collections are so big, and should they be shrunk down or partioned?
Eric
> -----Original Message-----
> From: Kasper Nielsen [mailto:news@kav.dk]
> Sent: Wednesday, November 05, 2003 2:55 PM
> To: commons-dev@jakarta.apache.org
> Subject: [Math] common-math and bloated 3rd party libraries
>
>
> Hi gang,
>
> I love commons-math, one problem though!
>
> lets take a look at the dependecies
>
> common-lang: 189 kb
> commons-beanutils: 116 kb
> commons-collections-SNAPSHOT.jar 463
> commons-discovery 70 kb
> commons-logging-1.0.3.jar 31 kb kb
>
> Thats 850 kb!!! of 3rd party libraries that are only used in
> a few places.
> So to calculate a simple mean I need to include around 6 jars
> (including
> commons-math)
>
> So lets get the list down a bit.
>
> * Commons-lang
> Getting rid of Commons-lang is pretty easy since it is only
> used in one
> place: MathException
> Solution : Let MathException extend Exception instead of
> NestableException. There aren't really anywhere we use the ability to
> nest Exceptions inside other Exceptions in commons-math.
>
> * Commons-collections
> Getting rid of commons-collections is also pretty easy
> Solution: Getting a copy of HashBag (and the Testcase) and put into
> math.util (no need to copy the interface)
>
> now we got rid of ~ 650 kb in around 2 minutes, 3 jars left,
> this is a
> fun game!!
>
> * Commons-Beanutils
> Okay the transformers are nice but come on how many people
> are going to
> use them?
> Solution: put them into a new small library:
> commons-math-transformers.jar
>
> ** Commons-Discovery
> KISS keep it simple stupid, who on earth is going to provide
> there own
> UnivariateRealSolverFactory??
> and for those few people that need it... I think they are
> smart enough
> to do figure it out themself.
> Solution: remove it (or do something like we do for commons-logging)
>
> ** Commons-logging
> Lastly commons-logging...
> I would think returning NaN is enough, but okay if people
> insist we can
> do something like (pseudo code)
>
> public class logutil
> static Method logMethod;
> static {
> try
> {
> Class clazz = Class.forName("commons.LogFactory");
> logMethod = clazz.getMethod("error");
> }
> catch (ClassNotFoundException e) {}
> }
> public static logError(String msg, Throwable t)
> {
> if (logMethod!=null)
> {
> logMethod.invoke(msg + t);
> }
> }
> }
>
> and whoops we have now gotten rid of all the libraries, and
> we have easy
> embeddable little commons math jar.
>
> regards
> Kasper
>
> --------
> Kasper Nielsen
> kaspern at apache.org
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: commons-dev-unsubscribe@jakarta.apache.org
> For additional commands, e-mail: commons-dev-help@jakarta.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commons-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: commons-dev-help@jakarta.apache.org