You are viewing a plain text version of this content. The canonical link for it is here.
Posted to general@jakarta.apache.org by Conor MacNeill <co...@cognet.com.au> on 2001/02/08 13:48:38 UTC
What is reuse
Continuing the "What is ..." theme.
I don't have a stake in the current DB Connection pooling argument. I don't
use either pool in my day to day work or within Ant. However, I have found
the discussion interesting, especially as I have just read Richard
Gabriel's "Patterns of Software". He deals with reuse a little and I think
he makes some interesting points:
"First you need a central repository of code. It doesn't help if developers
have to go around to other developers to locate code you might be able to
use"
"Second, there has to be a means of locating the right piece of code, which
usually requires a good classification scheme. It does no good to have the
right piece of code if no one can find it."
"Third, there must be good documentation of what each piece of code in the
repository does."
"For a lot of pieces of code it is just plain simpler to write it yourself
than to go through the process of finding and understanding reusable code".
I think this last point is particularly strong in open source projects
where a lot of people want to implement things themselves.
Anyway, what all this says to me is that if we want strong reuse across
Jakarta projects, we need to make reuse easier. We need a way to find code
that can be reused. If we are to have a utility project, then it must
produce not only the code, but a code catalog which facilitates reuse. I
think such a group could range over all the Jakarta projects, looking for
code that can be packaged for reuse. I would prefer to see modules with
little coupling rather than a framework approach. Building such components
can be hard. If we don't want a centralized project to own the code,
another idea could be to have a central catalog maintained across the
Jakarta projects. The Turbine folks, Struts folks, etc could "advertise"
all the modules they provide. There would be implications for how projects
package their software.
So now we have connection pooling code in two places. Duplication isn't
great but it happens, and sometimes it even happens in the same project. It
is a refactoring opportunity. Now that we have recognized that, surely we
can find the best solution, find somewhere for it to live (which may be
where it already lives) and increase our reuse going forward. The bigger
issue is to improve our communications, cataloging, etc so reuse is easier,
going forward.
This seems to me to be exactly the sort of cross-project issue the PMC is
designed to handle. Perhaps the PMC should consider whether to adopt a
policy of reuse across Jakarta projects, what that policy might be, whether
the policy should be promoted through a library subproject, etc.
I can recommend Gabriel's book. It is a fascinating and very different book
about software.
That is my 2 cents worth. Back to releasing Ant 1.3 ...
Conor
Re: What is reuse
Posted by Federico Barbieri <sc...@betaversion.org>.
Conor MacNeill wrote:
>
> Continuing the "What is ..." theme.
>
> I don't have a stake in the current DB Connection pooling argument. I don't
> use either pool in my day to day work or within Ant. However, I have found
> the discussion interesting, especially as I have just read Richard
> Gabriel's "Patterns of Software". He deals with reuse a little and I think
> he makes some interesting points:
>
> "First you need a central repository of code. It doesn't help if developers
> have to go around to other developers to locate code you might be able to
> use"
>
can't agree more on this!
> "Second, there has to be a means of locating the right piece of code, which
> usually requires a good classification scheme. It does no good to have the
> right piece of code if no one can find it."
>
> "Third, there must be good documentation of what each piece of code in the
> repository does."
>
Good documentation in a repository can solve the search problem quite
well.
> "For a lot of pieces of code it is just plain simpler to write it yourself
> than to go through the process of finding and understanding reusable code".
>
> I think this last point is particularly strong in open source projects
> where a lot of people want to implement things themselves.
>
This makes sense if I want to rewrite a piece of code I don't like. Most
times I'll rewite code 'cause I don't know there is already an
implementation.
One of the biggest mistake (IMHO) is to say "it takes me 2 days to write
the code. Why should I spend a week just to learn how to use other's
people code?". Well usually because in two days you can't do what others
have done in a year. Good architecture, debugging, features you may not
need now but can relif a lot of work later etc. As I said reuse is the
only way to create and enforce standards from the bottom up (not from
some high authority). And standards are the only way to scale projects.
One place to search is critical but it's not the only reason I think one
CVS is better... one repository guarantee all this packages order and
coherence. This for example means it takes less time to learn how a
pakage works from the second time on.
Can we have a kind of votation on a new jakarta-util project? (details
about can be finalized later once we agree on goals)
Federico
fede@apache.org
Re: What is reuse
Posted by David Weinrich <dw...@home.com>.
Ceki:
I agree, CPAN is a great model to look at in terms of a good way of
collecting and distributing useful stuff...but wait, there's more! If you
take CPAN a step further and invoke the CPAN shell on just about any unixish
system ( chances are you know about the CPAN shell but just in case...):
perl -MCPAN -e shell
or on some systems just
cpan
you will see the true coolness of CPAN. While the cpan shell isn't perfect,
it is incredibly useful. Being able to find the right module to fit your
needs without even having to open up a browser is pretty cool. Being able to
install it and the modules it might depend on without leaving that shell is
*extremely* good stuff.
This kinda brings me to one point that I've been waiting to inject into
this thread. To me we have at least two seperate issues:
* A set of well focused utility classes built by members of the
jakarta community to address common areas of need/duplication
for people building server based applications. ( What Ted has
proposed as zarf ).
* As many people have mentioned, each individual component
could pretty much stand alone.
* In cases where it makes sense, CPAN style Bundles could
package related components that have reached a point
where the API is stable ( does this ever happen? ).
* An emphasis on differentiating 'stable' and 'devel' versions
could help with some of the 'moving target' issues many
people have mentioned.
* I agree with many people who have pointed out that a set of
well focused utilities does not really prevent the creation
of a few different tools that do the same job, in slightly
different ways.
* The initial focus could be a small set of tools that have
already been identified. In the short term these projects
could help identify how this project could/should/may work
best.
* A CPAN style project that would provide:
* Developers a place to make well focused utilities available.
* A distribution system that helps promote sane packaging,
documentation, and versioning.
* The biggest distinction between this and the jakarta
utilities would be the scale/scope of the project.
* This particular project would, as Ceki pointed out,
take large amounts of effort and time just to get
the ball rolling.
* A large number of mirror sites would seem to be implied
by the nature of this project.
* Not knowing the finer points of the apache and jakarta
missions, I would assume it may very well be out of the
intended scope of either. I do think that the tools used
to create this system would most likely come from within
the projects already under way in jakarta ( ant,
alexandria, many others I am sure... )
Well this email is long enough already...but on a final note I would love to
contribute any way I can ( my programming skills are somewhat suspect, but I
am sure I could find something useful to do with myself ). Oh and Ted, how
about 'Jakarta Sledge' as an alternative to zarf. For some reason this is
stuck in my mind, I keep having vague memories of a saying that goes
something like "If all you have is a hammer, everything starts to look like
a nail". Funny enough, I think I read that somewhere on perl.com...
David Weinrich
Re: What is reuse
Posted by Ceki Gülcü <cg...@qos.ch>.
CPAN was mentioned a number of times in this thread. I would like to relate my very recent experience with CPAN. See http://www.cpan.org/modules/index.html and in particular http://www.cpan.org/modules/00modlist.long.html.
I wanted to install a Perl-based tool called dtdnsclient on my linux box. This tool required the package Term::ReadKey. I had no idea where to get this package from. (I have not programmed anything substantial in Perl in many years.) So I went to my good old RH 6.2 CD and tried to find the package there. I did not find it on the CD tough it must be said I did not look very hard.
Then CPAN came to my mind since I had heard of it here. I typed www.cpan.org browsed the site for about 22.4 seconds and landed on the modlist page. I just clicked on Term::ReadKey and a few moments later TermReadKey-2.14.tgz was on my disk. After reading the INSTALL, I did:
To install, unpack somewhere, type "perl Makefile.PL", and then "make test".
If the compilation and the tests are successful, then change to root and run
"make install".
I tried to run dtdnsclient but now LWP:UserAgent was missing. I repeated downloaded LWP:UserAgent but it required the URI, MIME-Base64, HTML-Tagset packages. I downloaded these and did "perl Makefile.PL", and then "make test." I realized that additional packages were missing, I installed these, and tried to run my script and still some other package was missing. I installed that, typed "perl Makefile.PL", and then "make test".
And behold, it worked! After about 15 minutes, decent dose of panic, a completely clueless and impatient user could install all these packages. I was and still am deeply impressed.
Let me also add that I could have probably written a limited version of dtdnsclient myself in less than a day without using any additional tools other then Perl. But I had better things to do...
To sum it up, a Java version of CPAN would be awesome but it will involve *massive* amounts of work and it will probably not look like anything useful before at least year. Just my 2 centimes, Ceki
ps: If CPAN is not a temple glorifying reuse I don't know what is.
At 01:09 09.02.2001 +1100, you wrote:
>At 11:48 8/2/01 +1100, Conor MacNeill wrote:
>>I think this last point is particularly strong in open source projects
>>where a lot of people want to implement things themselves.
>
>I think this is a more significant issue than any other. People like
>hacking at code and unless the candidate code is significantly better or
>there is someother incentive to work with it then it aint going to happen.
>When the code is part of another largish framework then it is almost
>invariably unwise to adopt it unless there is collaborators on both sides
>who want it to happen.
>
>>Anyway, what all this says to me is that if we want strong reuse across
>>Jakarta projects, we need to make reuse easier. We need a way to find code
>>that can be reused. If we are to have a utility project, then it must
>>produce not only the code, but a code catalog which facilitates reuse. I
>>think such a group could range over all the Jakarta projects, looking for
>>code that can be packaged for reuse. I would prefer to see modules with
>>little coupling rather than a framework approach. Building such components
>>can be hard. If we don't want a centralized project to own the code,
>>another idea could be to have a central catalog maintained across the
>>Jakarta projects. The Turbine folks, Struts folks, etc could "advertise"
>>all the modules they provide. There would be implications for how projects
>>package their software.
>
>There is a problem with this view point I suspect. There has already been
>one project with such an aim. The charter of the Avalon project was to
>provide support for things of exactly this nature.
>
>http://java.apache.org/framework/need-for-avalon.html
>http://java.apache.org/framework/call-to-vote.html
>
>While it has accumulated some code - there was only support from 2 other
>projects (namely JAMES and cocoon) - it hasn't become anywhere near what it
>was originally envisioned to be. Instead what you have is rapid duplication
>throughout all the projects ;)
>
>Even worse - if you are watching commits you will have noticed that turbine
>just increased it's charter to include general purpose server utilities (ie
>it now encompases Avalons charter). Ironic that Jons complaining about the
>way Struts aren't sticking to their charter, no ? ;) I don't see this as a
>bad thing - just natural tendency. People want to see their work used by
>greatest audience generally but want the "other" to adopt their framework
>aswell.
>
>Creating a new project to act as a repository for util code could work.
>However things like DB pools that work best when integrated into a
>framework will unlikely to see a lot of reuse (except the copy-paste kind)
>if the framework is not adopted.
>
>The only way to "fix" this is to make each component self-contianed (and
>very bean-esque). This will of course entail a lot of work and will
>actually also increase duplicity. Each bean will duplicate functionality
>present in frameworks because it can not rely on the framework. Hence you
>have not stopped duplication - you just changed it from per-framework to
>per-component.
>
>>The bigger
>>issue is to improve our communications, cataloging, etc so reuse is easier,
>>going forward.
>
>I think there needs to be more infrastructure in place. Sams tinderbox
>stuff with automatic notification and a constant integration build would
>solve this. Combine this with the ability to build finer grain .jars and
>distribute them ala CJAN and it may work but I have my doubts ;)
>
>>This seems to me to be exactly the sort of cross-project issue the PMC is
>>designed to handle. Perhaps the PMC should consider whether to adopt a
>>policy of reuse across Jakarta projects, what that policy might be, whether
>>the policy should be promoted through a library subproject, etc.
>
>It's up to individual projects IMHO. However if a project spurns reuse due
>to the NIH syndrome then they probably shouldn't have been accepted into
>apache in the first place ...
>
>Cheers,
>
>Pete
>
>*-----------------------------------------------------*
>| "Faced with the choice between changing one's mind, |
>| and proving that there is no need to do so - almost |
>| everyone gets busy on the proof." |
>| - John Kenneth Galbraith |
>*-----------------------------------------------------*
>
>
>---------------------------------------------------------------------
>To unsubscribe, e-mail: general-unsubscribe@jakarta.apache.org
>For additional commands, e-mail: general-help@jakarta.apache.org
----
Ceki Gülcü e-mail: cgu@qos.ch (preferred)
av. de Rumine 5 ceki_gulcu@yahoo.com
CH-1005 Lausanne
Switzerland Tel: ++41 21 351 23 15
Re: What is reuse
Posted by cm...@yahoo.com.
> Creating a new project to act as a repository for util code could work.
> However things like DB pools that work best when integrated into a
> framework will unlikely to see a lot of reuse (except the copy-paste kind)
> if the framework is not adopted.
>
> The only way to "fix" this is to make each component self-contianed (and
> very bean-esque). This will of course entail a lot of work and will
> actually also increase duplicity. Each bean will duplicate functionality
> present in frameworks because it can not rely on the framework. Hence you
> have not stopped duplication - you just changed it from per-framework to
> per-component.
Again - CPAN.
I can't see why it is possible there and it's not possible here.
The whole issue is about the design of the component - if it is
"integrated" with the framework ( spaghetti-style ? ) or it's just using
other components and APIs ( Log ? ).
It is very easy to create "adapters" between different APIs, but it's very
hard to extract components from the spaghetti and reuse them.
--
Costin
Re: What is reuse
Posted by Peter Donald <do...@apache.org>.
At 11:48 8/2/01 +1100, Conor MacNeill wrote:
>I think this last point is particularly strong in open source projects
>where a lot of people want to implement things themselves.
I think this is a more significant issue than any other. People like
hacking at code and unless the candidate code is significantly better or
there is someother incentive to work with it then it aint going to happen.
When the code is part of another largish framework then it is almost
invariably unwise to adopt it unless there is collaborators on both sides
who want it to happen.
>Anyway, what all this says to me is that if we want strong reuse across
>Jakarta projects, we need to make reuse easier. We need a way to find code
>that can be reused. If we are to have a utility project, then it must
>produce not only the code, but a code catalog which facilitates reuse. I
>think such a group could range over all the Jakarta projects, looking for
>code that can be packaged for reuse. I would prefer to see modules with
>little coupling rather than a framework approach. Building such components
>can be hard. If we don't want a centralized project to own the code,
>another idea could be to have a central catalog maintained across the
>Jakarta projects. The Turbine folks, Struts folks, etc could "advertise"
>all the modules they provide. There would be implications for how projects
>package their software.
There is a problem with this view point I suspect. There has already been
one project with such an aim. The charter of the Avalon project was to
provide support for things of exactly this nature.
http://java.apache.org/framework/need-for-avalon.html
http://java.apache.org/framework/call-to-vote.html
While it has accumulated some code - there was only support from 2 other
projects (namely JAMES and cocoon) - it hasn't become anywhere near what it
was originally envisioned to be. Instead what you have is rapid duplication
throughout all the projects ;)
Even worse - if you are watching commits you will have noticed that turbine
just increased it's charter to include general purpose server utilities (ie
it now encompases Avalons charter). Ironic that Jons complaining about the
way Struts aren't sticking to their charter, no ? ;) I don't see this as a
bad thing - just natural tendency. People want to see their work used by
greatest audience generally but want the "other" to adopt their framework
aswell.
Creating a new project to act as a repository for util code could work.
However things like DB pools that work best when integrated into a
framework will unlikely to see a lot of reuse (except the copy-paste kind)
if the framework is not adopted.
The only way to "fix" this is to make each component self-contianed (and
very bean-esque). This will of course entail a lot of work and will
actually also increase duplicity. Each bean will duplicate functionality
present in frameworks because it can not rely on the framework. Hence you
have not stopped duplication - you just changed it from per-framework to
per-component.
>The bigger
>issue is to improve our communications, cataloging, etc so reuse is easier,
>going forward.
I think there needs to be more infrastructure in place. Sams tinderbox
stuff with automatic notification and a constant integration build would
solve this. Combine this with the ability to build finer grain .jars and
distribute them ala CJAN and it may work but I have my doubts ;)
>This seems to me to be exactly the sort of cross-project issue the PMC is
>designed to handle. Perhaps the PMC should consider whether to adopt a
>policy of reuse across Jakarta projects, what that policy might be, whether
>the policy should be promoted through a library subproject, etc.
It's up to individual projects IMHO. However if a project spurns reuse due
to the NIH syndrome then they probably shouldn't have been accepted into
apache in the first place ...
Cheers,
Pete
*-----------------------------------------------------*
| "Faced with the choice between changing one's mind, |
| and proving that there is no need to do so - almost |
| everyone gets busy on the proof." |
| - John Kenneth Galbraith |
*-----------------------------------------------------*
Re: What is reuse
Posted by cm...@yahoo.com.
Few things seem clear ( for me ):
- without a common repository, we'll go nowhere.
- the CPAN model works - i.e. individual modules doing specialized jobs.
- each project has big chunks of code that can be clasified as "utils" and
may be reused, but without a common repository there is little incentive
on making it "independent".
- it is not easy to make this code "independent" and package it as a CPAN
module - and without that the code is hard to reuse without taking the
whole.
- code that depends on a "framework" or project internals is less
reusable than "independent" code ( or code that depends on few other CPAN
modules ).
- you can't force someone to use your util, and a project that is going to
create utils for other has little chance. The reverse ( joining utils from
different projects and organizing them in a consistent way ) may work and
will probably lead to reuse and mergings.
What I'm trying to say is that if we want reuse, we need to do something -
and deciding which project has the "right" db pool is not going to help.
The PMC can easily create a jakarta-util and add all commiters from all
projects as commiters to the new repository.
It's not hard to organize code in packages - org.apache.util.XXX, with
each commiter beeing free to create new packages. The only requirement is
that util code shouldn't have deps on anything outside org.apache.util,
and any dependency to another org.apache.util.xxx module should be
documented ( a simple DEPENDS file ).
BTW, the rule should be "no rule" - it's better to have more modules
doing the same thing in different ways than deciding what is the
"right" way to do something. It's absolutely normal to have multiple
solutions to a problem ( like connection pooling ).
So - we either go ahead and create ( ask the PMC to create ) a CJAN
repository, or we stop wasting the time in mail flood.
Costin
PS.
I would be very interested in ANT moving the ProjectHelper and the XML
reading stuff in jakarta-util. It'll fit perfectly in tomcat3.x config
mechanism.
I spent a lot of time refactoring tc3.3, and I did looked around in other
projects for code and models - and there are quite a few pieces that would
be great to be able to reuse, but the overhead is too big.
BTW, this would be a good way to merge tc3.x and tc4.x ( if we add a
jakarta-modules repository with various modules for various projects that
are not core and can be "adapted" in multiple places :-).