You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@directory.apache.org by Alex Karasulu <ao...@bellsouth.net> on 2004/09/15 02:20:47 UTC
[seda] API & IMPL project/jar consolidation
Hi,
Steve had made some excellent points a while back about potential class
loader conflicts that can result in IoC containers when service
interfaces and implementations are contained within the same deployment
jar. Steve I guess I'm bringing this us in case we need re-edumacation
again :-).
>>From my understanding most of the problems have to do with managing the
dependencies of the implementation classes and conflicts these
dependencies can introduce. It occurred to me that the api and impl
projects for the SEDA core can be merged together and deployed together
if the number of dependencies are drastically reduced. The POJO
implementations themselves are very insignificant and get wrapped
anyway. Without very many dependencies these two projects can be merged
together I what I'm thinking. This will lead to a more managable SEDA
project with a single deliverable (artifact).
Right now I have reduced the number of build time dependencies in the
SEDA impl down to the following:
codec-stateful
commons-logging
The following deps exist for testing:
junit
commons-threadpool
And this last dep, seda-api, should go away when we merge the two
projects. The codec-stateful package is needed by both projects so it
will always have to stick around as the same version for both. The
logging is specific to the implementation classes but its an adapter
package and not an actual logging implementation so we're good I think.
So with these I ask if reuniting these two projects is the right path.
I'm leaning towards that but I wanted to get Steve's take on this with
these new details.
Alex
Re: [seda] API & IMPL project/jar consolidation
Posted by Trustin Lee <tr...@gmail.com>.
> Right now I have reduced the number of build time dependencies in the
> SEDA impl down to the following:
>
> codec-stateful
> commons-logging
In the long term, is forking the classes of other commons libraries
good? I was thinking there is no need to forki the classes of mature
libraries.
> So with these I ask if reuniting these two projects is the right path.
> I'm leaning towards that but I wanted to get Steve's take on this with
> these new details.
I love your idea, and I think it is an absolutely right path. For
example, Hibernate2 bundles everything in one jar and describes which
library is required to use the specific feature. I believe only the
runtime dependency is important. Who cares the compile time
dependency?
Trustin
--
what we call human nature in actually is human habit
--
http://gleamynode.net/
Re: [seda] API & IMPL project/jar consolidation
Posted by Alex Karasulu <ao...@bellsouth.net>.
On Sun, 2004-09-19 at 22:06, Niclas Hedhman wrote:
> On Monday 20 September 2004 08:24, Noel J. Bergman wrote:
> > Stephen McConnell wrote:
> > > If I understand correctly you are including a reference to a
> > > version of commons-logging in classes that would not be
> > > isolated behind an api.
> >
> > Commons Logging is intended to be an API. The implementations are supposed
> > to be decoupled. If you have issues with how Commons is structured, you
> > should probably raise them (there).
>
> The problems associated with c-loggings are subtle, but by now well-known.
> http://radio.weblogs.com/0122027/2003/08/15.html
> http://www.qos.ch/logging/thinkAgain.html
Just finished reading these. Wow I'm so shocked yet again by how much
of a problem this can be.
> You are right that this is not the forum for discussing whether c-logging has
> the right structure or not, and how it can be improved. BUT it is the right
> place to discuss whether c-loggings should be used at all.
>
> Alex has previously proudly declared that the NoLoggingStrategy has been
> employed, and I shortly wonder why this is not the case any more or for the
> particular case of SEDA.
Hold up Niclas I' still doing the NoLoggingStrategy thingy! Yeah I just
happen to have some monitor implementations that use commons logging.
It does not need to be used as the default. In fact it is not used as
the default - adapters for monitors are used even for the POJO
implementations.
I just got tired of having so many people complain about too many
projects. Vince, Alan, Jeff, Trustin have all wanted things to be in a
single jar. The NoLogging strategy still works even though we have
commons-logging dependent classes here. These classes are optional
implementations of monitor interfaces for the various components.
Take for example the service interface BufferPool. It has a monitor
interface called BufferPoolMonitor. There is a do nothing adapter
BufferPoolMonitorAdapter and there is the optional LoggingPoolMonitor
implementation which depends on commons-logging (CL). When BufferPool
implementations are created the adapter without deps is used. The
monitor must be replaced with the CL dependent monitor to impose the CL
dependency.
If the LoggingPoolMonitor is not used, the commons-logging dependency
jar does NOT need to be accessible in any classloader ancestor. So
we're still cool. It's a very subtle thing here so its easy to loose
track if you're not in the code the day the conversation comes up :-).
> Furthermore, I urge us to maintain a API, (SPI) & Impl separation for the SEDA
> package itself, in good IoC manner, which will enable powerful solutions in
> the future, without any runtime performance hit.
You know I'm getting torn thinking about this over and over again.
Steve's pretty damn right about his CL strategies. However people don't
want a gazzillion jars even if he's right. Noel actually had
recommended making it one project with two possible distros and I
thought about that too.
Then I just cleared my mind. I stopped thinking about everyone's
recommendations and just tried to think this through. Here are some of
the conclusions that I came to:
1). The SEDA framework as a framework is intended to be reused. Meaning
the plumbing will be shared by all network protocol servers. The
framework is designed like UNIX xinetd to run multiple services from the
same daemon. The analogy does not go any further but we reuse the same
frontend plumbing for any protocol that pops up. So we're going to need
a nice ClassLoader strategy for ourselves for protocol provider
implementations. Bottom line we don't expect to create a new frontend
instance for every protocol server running in the vm.
2). Component reuse is not as much of an integral issue nor is component
reuse in other frameworks. We go the distance to be able to integrate
via wrappers into other frameworks like Avalon but past this we're not
trying to write generic components for other people to use. We want to
write a SEDA based protocol server framework.
3). Our goal is to make it as easy as possible for someone to implement
and run their network protocol using our framework. If that means less
projects and a single distribution jar then we need to rethink breaking
things up.
4). The SEDA framework is not an implementation. It's an API. Yes it
contains POJO based implementations of various service interfaces
however these POJO do not need to be used. Perhaps most importantly
they have no dependencies on anything else - its the optional stuff that
comes packaged that has the deps on CL for example. I personally
removed deps on commons-lang and commons-codec for this reason and even
forked a handful of useless classes that will disappear by the time we
move to jdk1.5. Our POJO's have ZERO runtime dependencies. If you opt
to use the LoggingXxxxxMonitors well then you're introducing the runtime
dep on CL. That's the user's call not ours. We just make life easy if
that's what they want to do by putting the optional stuff in the distro
jar.
5). The true implementation jar with respect to a target framework such
as Avalon or Spring, or an app server like Geronimo will be the jar
containing the wrappers around these default POJO implementations.
These wrappers as you know adapter the POJO to the lifecycles of the
framework and implement the service interface. It's up to the wrapper
to implement its own monitor or use the one with the CL dependency that
comes out of the box. We don't force anything upon the implementor.
As part (b) of 5 consider the wrapper for the BufferPool service in
Avalon Merlin. We would have a MerlinBufferPool that implements
BufferPool and wraps a DefaultBufferPool member. Basically all
interface calls on MerlinBufferPool just call the DefaultBufferPool
member. Simple enough right? No there is an AvalonBufferPoolMonitor
that comes with the wrapper jar for Merlin. This BufferPoolMonitor
implementation uses the Avalon Logger to log monitored events. It
depends on avalon-framework and logkit etcetera. There is no CL
dependency during runtime since the LoggingBufferPoolMonitor that is
based on CL is never used. During the life-cycle stages, specially the
LogEnabled stage, the AvalonBufferPoolMonitor is created and set as the
monitor in the DefaultBufferPool in the initialize stage after the
wrapped DefaultBufferPool member has been created.
Does this make sense? I know it was lengthy but I think I have finally
found the middle ground between so many correct viewpoints.
> Bad organization should not be the result of inadequate and hard to maintain
> build system tools.
After reading my 5 reasons for the consolidation, do you still feel a
single API jar is dangerous for the SEDA framework in terms of the
ClassLoader issues discussion?
I'm hoping I mentioned a few things that others might not have
considered while making their points clear on this trail. Most of the
points made were right on and applicable under most situations. These
subtle 5 points however make this case more than just the standard
situation. That is why I have chosen this path: one seda.jar as the API
artifact.
Alex
RE: [seda] API & IMPL project/jar consolidation
Posted by "Noel J. Bergman" <no...@devtech.com>.
Niclas Hedhman wrote:
> You are right that this is not the forum for discussing
> whether c-logging has the right structure or not, and how
> it can be improved. BUT it is the right place to discuss
> whether c-loggings should be used at all.
As you say, that is a different issue. Although if all of our projects
adopt a monitor strategy in lieu of a common logging API, it could moot the
Commons Logging issue.
> I urge us to maintain a API, (SPI) & Impl separation for
> the SEDA package itself
Another reasonable suggestion, albeit one that can be carried too far in
terms of granularity.
--- Noel
Re: [seda] API & IMPL project/jar consolidation
Posted by Niclas Hedhman <ni...@hedhman.org>.
On Monday 20 September 2004 08:24, Noel J. Bergman wrote:
> Stephen McConnell wrote:
> > If I understand correctly you are including a reference to a
> > version of commons-logging in classes that would not be
> > isolated behind an api.
>
> Commons Logging is intended to be an API. The implementations are supposed
> to be decoupled. If you have issues with how Commons is structured, you
> should probably raise them (there).
The problems associated with c-loggings are subtle, but by now well-known.
http://radio.weblogs.com/0122027/2003/08/15.html
http://www.qos.ch/logging/thinkAgain.html
You are right that this is not the forum for discussing whether c-logging has
the right structure or not, and how it can be improved. BUT it is the right
place to discuss whether c-loggings should be used at all.
Alex has previously proudly declared that the NoLoggingStrategy has been
employed, and I shortly wonder why this is not the case any more or for the
particular case of SEDA.
Furthermore, I urge us to maintain a API, (SPI) & Impl separation for the SEDA
package itself, in good IoC manner, which will enable powerful solutions in
the future, without any runtime performance hit.
Bad organization should not be the result of inadequate and hard to maintain
build system tools.
Cheers
Niclas
--
+------//-------------------+
/ http://www.bali.ac /
/ http://niclas.hedhman.org /
+------//-------------------+
RE: [seda] API & IMPL project/jar consolidation
Posted by Alex Karasulu <ao...@bellsouth.net>.
Hey that actually sounds fair to me. What do others think?
Alex
On Mon, 2004-09-20 at 12:31, Stephen McConnell wrote:
> > -----Original Message-----
> > From: Alex Karasulu [mailto:aok123@bellsouth.net]
> > Sent: 20 September 2004 17:56
> > To: Apache Directory Developers List
> > Subject: RE: [seda] API & IMPL project/jar consolidation
> >
> > <snip/>
> >
> > > (just
> > > throw in a package protected console logger and you will eliminate
> the
> > > api pollution and close a security hole at the same time).
> >
> > So you mean make the monitors with deps on commons-logging package
> > protected?
>
> No.
>
> Take commons-logging and make it disappear. Ok, so now you have done
> this - you still have this pesky demo class that runs without a
> container but you need to log stuff. So just throw together a console
> logger of your own design - i.e. writes something dirty to System.out
> inside a class called ConsoleMonitor or whatever.
>
> You end up delivering the demonstrable solution (that could be used in a
> real scenario) but you are not opening up a security hole and will not
> be introducing another "dirty api" (there are more than enough of them
> out there already).
>
> Cheers, Steve.
>
>
>
>
RE: [seda] API & IMPL project/jar consolidation
Posted by Stephen McConnell <mc...@apache.org>.
> -----Original Message-----
> From: Alex Karasulu [mailto:aok123@bellsouth.net]
> Sent: 20 September 2004 17:56
> To: Apache Directory Developers List
> Subject: RE: [seda] API & IMPL project/jar consolidation
>
> <snip/>
>
> > (just
> > throw in a package protected console logger and you will eliminate
the
> > api pollution and close a security hole at the same time).
>
> So you mean make the monitors with deps on commons-logging package
> protected?
No.
Take commons-logging and make it disappear. Ok, so now you have done
this - you still have this pesky demo class that runs without a
container but you need to log stuff. So just throw together a console
logger of your own design - i.e. writes something dirty to System.out
inside a class called ConsoleMonitor or whatever.
You end up delivering the demonstrable solution (that could be used in a
real scenario) but you are not opening up a security hole and will not
be introducing another "dirty api" (there are more than enough of them
out there already).
Cheers, Steve.
RE: [seda] API & IMPL project/jar consolidation
Posted by Alex Karasulu <ao...@bellsouth.net>.
<snip/>
> (just
> throw in a package protected console logger and you will eliminate the
> api pollution and close a security hole at the same time).
So you mean make the monitors with deps on commons-logging package
protected?
Alex
RE: [seda] API & IMPL project/jar consolidation
Posted by Stephen McConnell <mc...@apache.org>.
> -----Original Message-----
> From: Alex Karasulu [mailto:aok123@bellsouth.net]
> Sent: 20 September 2004 17:07
> To: Apache Directory Developers List
> Subject: RE: [seda] API & IMPL project/jar consolidation
>
> Hi,
>
> Most of my comments are in line. However before going further I just
> wanted to say that I asked Stephen to show, really show where I was
> messing up. I asked for his expert opinion because of his expert
> experience with ClassLoader issues in building IoC containers. I also
> asked because I wanted to make absolutely sure I was not doing
something
> that was going to come back to bite me in the arse. I'm greatful that
> he is willing to give the time to make sure the right outcome is
> reached.
>
> Ok with that said I still respectfully disagree with him for numerous
> reasons outlined below. Thanks to Stephen I think we can be sure that
> there will be no CL issues in container environments unless someone
opts
> to use the packaged convenience Logging monitor classes within a
> container environment.
You seem to have missed my point.
I'm saying that you *are* making a mistake and that in the process you
are opening a security hole in your implementation. Putting in a demo
class with logging does not require exposure of commons-logging (just
throw in a package protected console logger and you will eliminate the
api pollution and close a security hole at the same time).
As to POJOs in APIs, that's another subject altogether.
Cheers, Steve,
RE: [seda] API & IMPL project/jar consolidation
Posted by Alex Karasulu <ao...@bellsouth.net>.
Hi,
Most of my comments are in line. However before going further I just
wanted to say that I asked Stephen to show, really show where I was
messing up. I asked for his expert opinion because of his expert
experience with ClassLoader issues in building IoC containers. I also
asked because I wanted to make absolutely sure I was not doing something
that was going to come back to bite me in the arse. I'm greatful that
he is willing to give the time to make sure the right outcome is
reached.
Ok with that said I still respectfully disagree with him for numerous
reasons outlined below. Thanks to Stephen I think we can be sure that
there will be no CL issues in container environments unless someone opts
to use the packaged convenience Logging monitor classes within a
container environment.
On Mon, 2004-09-20 at 05:45, Stephen McConnell wrote:
> > -----Original Message-----
> > From: Noel J. Bergman [mailto:noel@devtech.com]
> > Sent: 20 September 2004 02:24
> > To: Apache Directory Developers List
> > Subject: RE: [seda] API & IMPL project/jar consolidation
> >
> > Stephen McConnell wrote:
> > > If I understand correctly you are including a reference to a
> > > version of commons-logging in classes that would not be
> > > isolated behind an api.
> >
> > Commons Logging is intended to be an API. The implementations are
> > supposed
> > to be decoupled. If you have issues with how Commons is structured,
> you
> > should probably raise them (there).
>
> My point was not specific to commons logging - it was specific to
> consequences related to non-separation of services from implementation
> concerns at the packaging level and addressing the example from Alex
Packaging level separation analogies to interface+imlp code sparation
can only be taken so far. To facilitate reusable components in a
framework I agree this is the best approach to deal with the various
ClassLoader issues you have shown to arrise. You're totally right when
we're talking about people developing a common set of shared, reusable
components.
> linked to the inclusion of common-logging under the SEDA API. Logging
> in general is not part of a service contract (any logging) - its part of
> the implementation of the service.
All POJO classes within the seda-x.y.jar will not depend on
commons-logging (CL). If users of the API opt to replace the default
monitor adapters for POJO monitors with Logging monitors packaged for
convenience then they impose a RT depenency on CL. All we can say is
Caveat emptor (let the buyer beware). Use these Logging monitors at
your own risk - do not to use them in container environments where
ClassLoader conflicts may result.
> Keep in mind that separation of API and implementation at the packaging
> level is similar to the principals of separation of interface and class.
Again there is a limit to where this analogy can go. We've approached
those limits here.
> One defines a contract while the other is the implementation of the
> contract. With separation at package level a container can isolate
> implementation classes away from the main api classloader chain. This
Right but this all depends on what you interpret as implementation and
as API. An API can have classes in it. API's are not just for
interfaces and exception classes. As far as I am concerned the default
POJO implementations are part of the API. They have ZERO deps like
their service interfaces. They can be used or not but using them will
NOT impose a dependency on CL thanks to the NoLogging approach used
here:
http://wiki.apache.org/avalon/AvalonNoLogging
> means that "implementation" dependencies are placed in separate isolated
> classloaders derived from the api chain - thereby largely minimizing the
> potential conflicts that can occur.
Absolutely right if we were building reusable components for something
like Avalon Planet. Still we do not have this problem for the SEDA API.
> Achieving good isolation requires that the entire api chain is composed
> of clean apis. An api exposing common-logging (or any other
> implementation artifact) is not a clean api. The result is the
> effective infection of the api chain by a badly structure api. While
> infection may not be intentional, the existence of implementations
> classes opens up potential for conflict and potential security risks.
We package these logging monitors as optional monitor classes. If they
are not used CL is never needed in any ClassLoader in the lineage path
of ClassLoaders. There is no infection. My Merlin wrappers never touch
these logging monitors because I use Avalon specific logging classes in
these wrapper implementations. As far as I am concerned wrapper jars
will be separated as true implementation classes not being packaged as
part of the seda-x.y.jar.
If the logging monitors dependening on CL are used and the SEDA POJO's
live inside a container environment then we have the potential
'infection' you refer to true.
These monitors are here for a special standalone harness application
with a main() so it is not only used for unit testing. It's used for
beta testing protocols our users develop inside the framework without
needing a container like Merlin, or Geronimo. Or its used when they do
not want to run this thing with Merlin or Geronimo but want to keep
things really thin for there protocol server app. The harness hardwires
all the POJOs so the frontend can come up without a microkernel. For
this little application/tool I would like some logging to happen and I
use the CL monitors to achieve that.
I could have packaged this harness app/tool in a separate JAR but I
don't want more entropy. Having this code sit in the same JAR when run
inside say Avalon containers does not impose the CL dependency unless I
start using these logging monitors in my wrappers.
> In this case .. the combining of the real SEDA api with implementation
> cases (or test classes exposing implementation classes mixed in with an
> api) will create a bad api in that it is the source of api chain
> infection.
I'm willing to accept a comprimise. If this move bites me in the arse I
can always come back and correct things by splitting up the JAR.
Nothing is set in stone. Look in the end you have some very good points
as a purist in the IoC world. I'm willing to accept that you're right -
I just don't know if your conclusions are statistically significant
across use cases. I'm willing to take a risk here.
I would rather have less entropy, less confused users, happy users with
one jar hence a single artifact, and all the tools needed for a protocol
developer in one place. This all in exchange for a small risk where
users of the API may contaminate a ClassLoader lineage path in a
container. In the end 99% of the users will never create the problem u
mention. Keeping them happy is my goal so long as this API spreads like
wild fire ;). A single jar will help make that happen.
Thanks,
Alex
RE: [seda] API & IMPL project/jar consolidation
Posted by Stephen McConnell <mc...@apache.org>.
> -----Original Message-----
> From: Noel J. Bergman [mailto:noel@devtech.com]
> Sent: 20 September 2004 02:24
> To: Apache Directory Developers List
> Subject: RE: [seda] API & IMPL project/jar consolidation
>
> Stephen McConnell wrote:
> > If I understand correctly you are including a reference to a
> > version of commons-logging in classes that would not be
> > isolated behind an api.
>
> Commons Logging is intended to be an API. The implementations are
> supposed
> to be decoupled. If you have issues with how Commons is structured,
you
> should probably raise them (there).
My point was not specific to commons logging - it was specific to
consequences related to non-separation of services from implementation
concerns at the packaging level and addressing the example from Alex
linked to the inclusion of common-logging under the SEDA API. Logging
in general is not part of a service contract (any logging) - its part of
the implementation of the service.
Keep in mind that separation of API and implementation at the packaging
level is similar to the principals of separation of interface and class.
One defines a contract while the other is the implementation of the
contract. With separation at package level a container can isolate
implementation classes away from the main api classloader chain. This
means that "implementation" dependencies are placed in separate isolated
classloaders derived from the api chain - thereby largely minimizing the
potential conflicts that can occur.
Achieving good isolation requires that the entire api chain is composed
of clean apis. An api exposing common-logging (or any other
implementation artifact) is not a clean api. The result is the
effective infection of the api chain by a badly structure api. While
infection may not be intentional, the existence of implementations
classes opens up potential for conflict and potential security risks.
In this case .. the combining of the real SEDA api with implementation
cases (or test classes exposing implementation classes mixed in with an
api) will create a bad api in that it is the source of api chain
infection.
Stephen.
RE: [seda] API & IMPL project/jar consolidation
Posted by "Noel J. Bergman" <no...@devtech.com>.
Stephen McConnell wrote:
> If I understand correctly you are including a reference to a
> version of commons-logging in classes that would not be
> isolated behind an api.
Commons Logging is intended to be an API. The implementations are supposed
to be decoupled. If you have issues with how Commons is structured, you
should probably raise them (there).
> if the version of common logging that you depend on may not be
> the same as the version of commons logging in the classloader
> chain above you.
That would be true for any API that changes.
--- Noel
RE: [seda] API & IMPL project/jar consolidation
Posted by Stephen McConnell <mc...@apache.org>.
> -----Original Message-----
> From: Alex Karasulu [mailto:aok123@bellsouth.net]
> Sent: 19 September 2004 08:30
> To: Apache Directory Developers List
> Subject: RE: [seda] API & IMPL project/jar consolidation
>
> On Sun, 2004-09-19 at 02:20, Stephen McConnell wrote:
> > > Right now I have reduced the number of build time dependencies in
the
> > > SEDA impl down to the following:
> > >
> > > codec-stateful
> > > commons-logging
> >
> > I don't know anything about what is exposed by codec-stateful - but
you
> > should definitely *not* expose commons-logging - that's an
> > implementation package.
>
> Can you elaborate Steve? Right now we just have it as a dependency.
If I understand correctly you are including a reference to a version of
commons-logging in classes that would not be isolated behind an api.
This means that any class running in a classloader below your classes
will be locked into that version of commons-logging. Equally, if the
version of common logging that you depend on may not be the same as the
version of commons logging in the classloader chain above you.
On the otherhand - if you package you stuff with a clean api and a
separate impl, none of the above potential problems can occur.
Cheers, Steve.
>
> > Cheers, Steve.
> >
> >
> >
RE: [seda] API & IMPL project/jar consolidation
Posted by Alex Karasulu <ao...@bellsouth.net>.
On Sun, 2004-09-19 at 02:20, Stephen McConnell wrote:
> > Right now I have reduced the number of build time dependencies in the
> > SEDA impl down to the following:
> >
> > codec-stateful
> > commons-logging
>
> I don't know anything about what is exposed by codec-stateful - but you
> should definitely *not* expose commons-logging - that's an
> implementation package.
Can you elaborate Steve? Right now we just have it as a dependency.
> Cheers, Steve.
>
>
>
RE: [seda] API & IMPL project/jar consolidation
Posted by Stephen McConnell <mc...@apache.org>.
> Right now I have reduced the number of build time dependencies in the
> SEDA impl down to the following:
>
> codec-stateful
> commons-logging
I don't know anything about what is exposed by codec-stateful - but you
should definitely *not* expose commons-logging - that's an
implementation package.
Cheers, Steve.