Re: [primitives][collections] primitive collections and PROPOSAL

[Stephen Colebourne <sc...@btopenworld.com>]

Inline, with PROPOSAL at bottom.

From: "Rodney Waldhoff" <rw...@apache.org>
> * I'm -1 to moving collections.primitives, or more generally, any release
> ready code, to the sandbox, as this moves it further away from a release
> rather than closer.
I'm balancing this worry with the desire to not release something new that
is in the wrong place. The basic issue is that primitive collections are
quite disconnected (ie. totally) from the rest of the package. This argues
for a separate project, as you have done in the past over parts of lang
(which has happened).

> a) The two code bases are not truly independent, they share a unit test
> suite, so if we split the two we'll probably want to extract the shared
> unit testing framework.
I would see this as a compile-time dependency on [collections].

> b) It is quite likely that we'll eventually want primitive implementations
> of Bag and other commons-collections-only extensions, which would
> introduce direct collections/primitive-collections dependencies, at least
> at the adapter level.  This would also imply that collections and
> primitive-collections packages are likely to change together, since
changes
> to commons-collections extensions would imply changes to the primitive
> collections.
Primitive collections may depend on the Bag interface, but probably little
else. This could be solved by adding a [collections] dependency, or by
simply including the Bag interface in the primitive-collections jar. (I know
that sounds scary, but an isolated interface, thus non-changing, should be
fine in two jars)

> * I'm +0 to having the commons-collections component release multiple
> JARs, for example, commons-collections.jar (everything minus
> *.primitives), commons-collections-primitives.jar (*.primitives.*), and
> commons-collections-all.jar (everything).
I'm -0.5 to this approach. I think it confuses users and leads to classpath
issues with the same class in two jars.

> * I'm unsure about the notion of "primitives" as opposed to "primitive
> collections".  Precisely what non-collection stuff do we believe to be in
> scope for "primitives"?
Well, comparators aren't actually collections. Plus Mutable priimtive
wrapper classes are also possibilities.


PROPOSAL:
We copy the existing [collections] primitive code to a new sandbox project
[pcollections] unaltered except for the package name. (I've already tried
this locally and it works). (Thus I volunteer for this)

[pcollections] and [primitives] can then cooexist, and Darwin will then
decide. If you want to push for [pcollections] promotion and release, that
will be your call. Alternatively you could wait a month or so to see how
[primitives] turns out :-)

This actually works well, as I want to try implementing [primitives] as
direct implementations of List, Collection, Map et al. - no separate
adaptors. So, there is a good case for the separation.

How does this sound???

Stephen

Re: [primitives][collections] primitive collections and PROPOSAL

[__matthewHawthorne <mh...@alumni.pitt.edu>]

I wasn't around for any voting or coding on primitive collections, but here's my 2 cents:

Other than adapters, I don't see [collections] and [primitives]/[pcollections] as projects that have much to share at all.   They seems to have the potential of being 2 separate code bases, and with the size that [collections] has grown to, I think it's a good choice and opportunity to split them apart.

>I've already got a significant (and for the record, largely open
>source) body of code dependent on the current implementation and the
>current packaging, as well as a substantial installed user base.
>Repackaging this code to move or remove the word "collections" seems to
>create a lot of trouble with little if any benefit.

Creating dependencies on unreleased code seems like an action that is undertaken "at your own risk".  In my opinion, that fact that the code is being used in its current form by you, or anyone else, isn't a good enough argument to keep it that way.

Moving unreleased commons proper code to a sandbox component doesn't appear to decrease the stability at all.  Both areas are volatile and subject to change without an official deprecation stage.

>The code is well within the scope of commons-collections, so demoting
>it for the sole purpose of removing it from commons-collections seems
>questionable at best.

I agree that the code falls within the scope of [collections], however I also think that it falls more directly in the scope of a project like [primitives].  I think that keeping the code together brings the potential of [collections] growing too large.  The amount of classes needed for [pcollections] is tremendous, enough to warrant a redefinition of the scope of [collections] to Object-based versions only. 

Is that legal?  Am I making sense?




Rodney Waldhoff wrote:

>On Wed, 3 Sep 2003, Stephen Colebourne wrote:
>
>sc> From: "Rodney Waldhoff" <rw...@apache.org>
>
>rw> > * I'm -1 to moving collections.primitives, or more generally,
>rw> > any release ready code, to the sandbox, as this moves it
>rw> > further away from a release rather than closer.
>
>sc> I'm balancing this worry with the desire to not release something new
>sc> that is in the wrong place. The basic issue is that primitive
>sc> collections are quite disconnected (ie. totally) from the rest of
>sc> the package. This argues
>sc> for a separate project, as you have done in the
>sc> past over parts of lang (which has happened).
>
>What I have argued in the past, and I'll argue now, is that cohesive
>components are constructed out of classes that are (a) interdependent, (b)
>commonly used together, and/or (c) commonly change together.  Adapters and
>test suites aside, currently collections.* and collections.primitives.*
>fail the first test but pass the second two.  Of course, many of the
>collections.* classes have this profile, for just one example, LRUMap and
>ArrayIterator.
>
>rw> > a) The two code bases are not truly independent, they share
>rw> > a unit test suite, so if we split the two we'll probably want
>rw> > to extract the shared unit testing framework.
>
>sc> I would see this as a compile-time dependency on [collections].
>
>Except that the unit testing framework isn't currently distributed in any
>form.  This is a compile-time (or test-time) dependency on code that's
>currently internal to the commons-collections code base.
>
>rw> > b) It is quite likely that we'll eventually want primitive
>rw> > implementations of Bag and other commons-collections-only
>rw> > extensions, which would introduce direct
>rw> > collections/primitive-collections dependencies, at
>rw> > least at the adapter level.  This would also imply that
>rw> > collections  and primitive-collections packages are likely to change
>rw> > together, since changes to commons-collections extensions would
>rw> > imply changes to the primitive collections.
>
>sc> Primitive collections may depend on the Bag interface, but
>sc> probably little else.
>
>>From a quick glance, Bag, BoundedCollection, Buffer, MultiMap, OrderedMap,
>OrderedSet, PriorityQueue, ResetableIterator, ResetableListIterator and
>SortedBag all seem like interfaces that we'd quite likely want to support
>within the primitive collections sooner or later.  Many interfaces that
>aren't commonly implemented via decorators would likely fall into this
>category as well.  Any change to the object based implementations will
>likely necessitate a change in the primitive implementations and/or their
>adapters.
>
>sc> This could be solved by adding a [collections] dependency, or by
>sc> simply including the Bag interface in the primitive-collections jar.
>sc> (I know that sounds scary, but an isolated interface, thus
>sc> non-changing, should be fine in two jars)
>
>This is indeed one approach, but what problem are we trying to solve?
>
>rw> > * I'm +0 to having the commons-collections component release
>rw> > multiple JARs, for example, commons-collections.jar (everything
>rw> > minus *.primitives), commons-collections-primitives.jar
>rw> > (*.primitives.*),  and
>rw> > commons-collections-all.jar (everything).
>
>sc> I'm -0.5 to this approach. I think it confuses users
>
>"Primitive collections are there, object collections are here", what's
>confusing about that?  How's [pcollections] and [collections] less
>confusing, or for that matter, different?
>
>sc> and leads to classpath issues with the same class in two jars.
>
>Fair enough, so drop commons-collections-all.jar.
>
>rw> > * I'm unsure about the notion of "primitives" as opposed to
>rw> > "primitive collections".  Precisely what non-collection stuff
>rw> > do we believe to be in scope for "primitives"?
>
>sc> Well, comparators aren't actually collections. Plus Mutable priimtive
>sc> wrapper classes are also possibilities.
>
>Primitive comparators are pretty much defined by the built-in operators
>(either '-' or '<'), and/or by auto-boxing.  (And the most common use of
>comparators is handled by java.util.System methods anyway).
>
>Currently object-level comparators are in collections, so I don't think
>this is making the problem any worse. (Not that there are primitive
>comparator implementations in the code base currently anyway).  Arguably
>there is overlap with lang, math, collections and functor on this point.
>
>sc> PROPOSAL:
>sc> We copy the existing [collections] primitive code to a new
>sc> sandbox project [pcollections] unaltered except for the package name.
>sc> (I've already tried this locally and it works). (Thus I volunteer for
>sc> this)
>sc>
>sc> [pcollections] and [primitives] can then cooexist, and Darwin will
>sc> then decide. If you want to push for [pcollections] promotion and
>sc> release,  that will be your call. Alternatively you could wait a month
>sc> or so to see how [primitives] turns out :-)
>
>Independent of moving the component or jar distributions around, I'm -1 to
>repackaging code from org.apache.commons.collections.primitives at this
>time.  I've already got a significant (and for the record, largely open
>source) body of code dependent on the current implementation and the
>current packaging, as well as a substantial installed user base.
>Repackaging this code to move or remove the word "collections" seems to
>create a lot of trouble with little if any benefit.  I can't imagine what
>else would go into that namespace anyway, so if we're going to change it,
>it should be done via a slow deprecation rather than by fiat.
>
>As before, I'm -1 to moving code from commons.collections.primitives into
>the sandbox, since:
>
>(a) The code is well within the scope of commons-collections, so demoting
>it for the sole purpose of removing it from commons-collections seems
>questionable at best.
>
>(b) The code is well over a year old, stable, 100% tested, and well used
>in production environments (including two generations of commercial,
>off-the-shelf products), and other open source projects (including some
>here in commons) use or have expressed an interest in using these classes.
>Moving this stuff further away from a formal release is a disservice to
>those users.
>
>(c) Adding the primitive collections to the commons-collections code base
>is an old decision that was openly discussed on this list.  Indeed, you
>yourself expressed approval for this action
>(http://nagoya.apache.org/eyebrowse/ReadMsg?listName=commons-dev@jakarta.apache.org&msgNo=7694),
>as did a total of 4 committers (with no dissenters).
>
>(d) When viewed through a common-use or common-cause-for-change metric,
>the primitive collections and the collections packages have very high
>cohesion.  Indeed, they have higher cohesion on the latter point than most
>arbitrary pairs of types in commons-collections.
>
>sc> This actually works well, as I want to try implementing [primitives]
>sc> as direct implementations of List, Collection, Map et al. - no
>sc> separate adaptors. So, there is a good case for the separation.
>
>As you are well aware, we've played with this approach in the past and
>found it wanting.  Among other drawbacks, you lose the ability to use the
>primitive collections independent of the java.util Collections (for
>example on many micro and J2ME platforms).  Perhaps more importantly, the
>pollution of the namespace becomes significant, a point that you and
>others have noted in the past:
><http://nagoya.apache.org/eyebrowse/ReadMsg?listName=commons-dev@jakarta.apache.org&msgId=504141>
>
>(On this note, your preferences on the "design decisions" thread seem
>inconsistent on this point.  You state a preference for extending the
>Collections interfaces (e.g., "IntCollection extends Collection"), but you
>also state a preference for not "following the Sun Framework closely".)
>
>Having authored and extensively worked with the code both ways, personally
>I strongly prefer the adapter approach utilizing small, focused,
>composable pieces.
>
>sc> How does this sound???
>
>As before, I'm open to (a) leaving the primitive collections where they
>are, (b) splitting the distribution of commons-collections into primitive
>and object based implementations or (c) proposing a new commons proper
>component, dependent upon commons-collections, containing the primitive
>collections and possibly more.  Note that to do (c) will require
>re-implementing, copying, refactoring or releasing the commons-collections
>test framework.
>
>Specifically, I'm not strictly opposed to introducing a [primitives] or
>[pcollections] component that contains what's currently
>org.apache.commons.collections.primitives.* and whatever else seems to fit
>well.  (Although I'm not confident that's a good idea either.)  I am
>opposed to moving code from commons proper to the sandbox for that
>purpose.
>
> - Rod <http://radio.weblogs.com/0122027/>
>
>sc> Stephen
>
>
>---------------------------------------------------------------------
>To unsubscribe, e-mail: commons-dev-unsubscribe@jakarta.apache.org
>For additional commands, e-mail: commons-dev-help@jakarta.apache.org
>
>
>  
>

Re: [primitives][collections] primitive collections and PROPOSAL

[Phil Steitz <ph...@steitz.com>]

Anton Tagunov wrote:
> Hello Stephen!
> 
> SC>  commons-collections.jar - all Object collections (no primitives)
> SC>  commons-collections-primitive.jar - all primitive collections (no Objects)
> SC> Thus, no package changes, just an additional jar giving users the choice as
> SC> to what they want.
> 
> SC> If you are happy with this approach, then I will make it so in the release.
> SC> Ant shouldn't be a problem, although I'd love to know how Maven copes with
> SC> cases like this ;-)
> 
> Have never done this myself but have read megabytes of talks about
> this on all kinds of lists. The verdict is that you need to have two
> projects in maven sence. Two project.xml (or maven.xml?) files.

You can have a single project with subprojects, using one maven.xml to 
get the multi-project build to work.  I did this recently in the sandbox 
commons-naming component.  I followed these examples provided in an 
earlier post by Eric Pugh:

http://xingu.sf.net   (maven build-all)
http://jakarta.apache.org/turbine/fulcrum/  (maven build)

To do it without using Jelly script to execute things manually, the only 
way that I could see to make it work was to break the source tree into 
branches corresponding to the subprojects. So, for example, if you want 
to build org.apache.commons.collection.* (minus primitives) and 
org.apache.commons.collections.primitives.* separately, you would need 
to move from collections/src/... to collections/core/src and 
collections/primitives/src and move the primitives code to the 
primitives tree.  Assuming that core does not depend on primitives, but 
primitives does depend on core, you could add the core jar as a 
dependency in the primitives project.xml and the build should work. 
This is more or less exactly what I did when splitting the resource 
factories out from the core naming stuff in [naming].  I am no maven 
expert, so there is a good chance that there is an easier way to do 
this.  As long as you can split things up as above, however, the 
maven.xml and project.xml setup is very simple.

I would be interested to know if anyone knows a simpler, less disruptive 
approach to make this kind of thing work in maven.

Phil
> 
> No other way.
> 
> Anton
> 
> 
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: commons-dev-unsubscribe@jakarta.apache.org
> For additional commands, e-mail: commons-dev-help@jakarta.apache.org
> 

Re[2]: [primitives][collections] primitive collections and PROPOSAL

[Anton Tagunov <at...@mail.cnt.ru>]

Hello Stephen!

SC>  commons-collections.jar - all Object collections (no primitives)
SC>  commons-collections-primitive.jar - all primitive collections (no Objects)
SC> Thus, no package changes, just an additional jar giving users the choice as
SC> to what they want.

SC> If you are happy with this approach, then I will make it so in the release.
SC> Ant shouldn't be a problem, although I'd love to know how Maven copes with
SC> cases like this ;-)

Have never done this myself but have read megabytes of talks about
this on all kinds of lists. The verdict is that you need to have two
projects in maven sence. Two project.xml (or maven.xml?) files.

No other way.

Anton

Re: [primitives][collections] primitive collections and PROPOSAL

[Stephen Colebourne <sc...@btopenworld.com>]

This is pretty much what I was suggesting in the proposal, however I would
still like to release [collections] as two jar files. One for Objects and
one for primitives.

(In an ideal world I would have a separate project for the primitives, but
I'm not going to block things for this)

So,
 commons-collections.jar - all Object collections (no primitives)
 commons-collections-primitive.jar - all primitive collections (no Objects)
Thus, no package changes, just an additional jar giving users the choice as
to what they want.

If you are happy with this approach, then I will make it so in the release.
Ant shouldn't be a problem, although I'd love to know how Maven copes with
cases like this ;-)

Stephen


----- Original Message -----
From: "Rodney Waldhoff" <rw...@apache.org>
To: "Jakarta Commons Developers List" <co...@jakarta.apache.org>
Sent: Friday, September 05, 2003 3:59 PM
Subject: Re: [primitives][collections] primitive collections and PROPOSAL


> It seems that you are interested in experimenting with a new scope and
> possibly a new implementation for the collections.primitives stuff.  My
> suggestion then is this:
>
> A) Go ahead with a commons-collections release including
> collections.primtives.*
>
> B) Continue experimenting with the new primitives component in the
> sandbox, perhaps seeding it with a repackaged version of
> collections.primitives.
>
> C) If at some future point we decide that collections.primitives.* is a
> better fit elsewhere, we deprecate collections.primitives.* in one
> release, say Collections 4.0 (roughly coincident with a release of the
> alternative packaging and/or implementation of that functionality), and
> remove it in a following release, say Collections 5.0.
>
> Holding back on stable, release-ready, unit- and field-tested code so that
> we can play around with alternative implementations or groupings seems
> unnecessary when there are backward and forward compatible options
> available.
>
> - Rod <http://radio.weblogs.com/0122027/>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: commons-dev-unsubscribe@jakarta.apache.org
> For additional commands, e-mail: commons-dev-help@jakarta.apache.org
>

Re: [primitives][collections] primitive collections and PROPOSAL

[Rodney Waldhoff <rw...@apache.org>]

It seems that you are interested in experimenting with a new scope and
possibly a new implementation for the collections.primitives stuff.  My
suggestion then is this:

A) Go ahead with a commons-collections release including
collections.primtives.*

B) Continue experimenting with the new primitives component in the
sandbox, perhaps seeding it with a repackaged version of
collections.primitives.

C) If at some future point we decide that collections.primitives.* is a
better fit elsewhere, we deprecate collections.primitives.* in one
release, say Collections 4.0 (roughly coincident with a release of the
alternative packaging and/or implementation of that functionality), and
remove it in a following release, say Collections 5.0.

Holding back on stable, release-ready, unit- and field-tested code so that
we can play around with alternative implementations or groupings seems
unnecessary when there are backward and forward compatible options
available.

- Rod <http://radio.weblogs.com/0122027/>

Re: [primitives][collections] primitive collections and PROPOSAL

[Stephen Colebourne <sc...@btopenworld.com>]

Inline, snipped as much as I sensibly could.
From: "Rodney Waldhoff" <rw...@apache.org>
> rw> > a) The two code bases are not truly independent, they share
> rw> > a unit test suite, so if we split the two we'll probably want
> rw> > to extract the shared unit testing framework.
> sc> I would see this as a compile-time dependency on [collections].
> rw> Except that the unit testing framework isn't currently distributed in
any
> rw> form.  This is a compile-time (or test-time) dependency on code that's
> rw> currently internal to the commons-collections code base.
The unit testing framework is distributed in the collections source zip
file, so is available. I think that this could be communicated to users.

> rw> > b) It is quite likely that we'll eventually want primitive
> rw> > implementations of Bag and other commons-collections-only
> rw> > extensions,...
> sc> Primitive collections may depend on the Bag interface, but
> sc> probably little else.
> rw>From a quick glance, Bag, BoundedCollection, Buffer, MultiMap,
OrderedMap,
> rw> OrderedSet, PriorityQueue, ResetableIterator, ResetableListIterator
and
> rw> SortedBag all seem like interfaces that we'd quite likely want to
support
> rw> within the primitive collections sooner or later.  Many interfaces
that
> rw> aren't commonly implemented via decorators would likely fall into this
> rw> category as well.  Any change to the object based implementations will
> rw> likely necessitate a change in the primitive implementations and/or
their
> rw> adapters.
There is definitely a case for a dependency here if more interfaces are to
be implemented. Of course at present, none are. This could be solved either
by hiding the dependency within collections, or exposing it by adding a
separate project.

> rw> > multiple JARs, for example, commons-collections.jar (everything
> rw> > minus *.primitives), commons-collections-primitives.jar
> rw> > (*.primitives.*)
> rw> "Primitive collections are there, object collections are here", what's
> rw> confusing about that?  How's [pcollections] and [collections] less
> rw> confusing, or for that matter, different?
I prefer one project to release one jar with one package name structure.

> rw> > * I'm unsure about the notion of "primitives" as opposed to
> rw> > "primitive collections".  Precisely what non-collection stuff
> rw> > do we believe to be in scope for "primitives"?
A better answer to this question is I don't know yet :-) I see [primitives]
as being sandbox-like at present, exploring options and seeing if a
community develops.

> Independent of moving the component or jar distributions around, I'm -1 to
> repackaging code from org.apache.commons.collections.primitives at this
> time.  I've already got a significant (and for the record, largely open
> source) body of code dependent on the current implementation and the
> current packaging, as well as a substantial installed user base.
> Repackaging this code to move or remove the word "collections" seems to
> create a lot of trouble with little if any benefit.  I can't imagine what
> else would go into that namespace anyway, so if we're going to change it,
> it should be done via a slow deprecation rather than by fiat.
IMHO unreleased code is liable to change, whether in the sandbox or not.

To explain further, my preferred approach to managing the commons projects
I'm involved in is to let code be added to CVS, then to radically refactor
just before a release. This is because it is only when you have all the code
that you can see what should be moved/removed/renamed.

For [lang] this worked well. Once release was getting close, the functor and
reflect code was reviewed and removed and the util package disbanded. IMHO
this late refactoring approach works well.


> (a) The code is well within the scope of commons-collections, so demoting
> it for the sole purpose of removing it from commons-collections seems
> questionable at best.
> (b) The code is well over a year old, stable, 100% tested, and well used
> in production environments (including two generations of commercial,
> off-the-shelf products), and other open source projects (including some
> here in commons) use or have expressed an interest in using these classes.
> Moving this stuff further away from a formal release is a disservice to
> those users.
I am happy for the code to be placed in a new commons proper component
instead of the sandbox. I would volunteer to create the new component but
not release it.

> (c) Adding the primitive collections to the commons-collections code base
> is an old decision that was openly discussed on this list.  Indeed, you
> yourself expressed approval for this action
>
(http://nagoya.apache.org/eyebrowse/ReadMsg?listName=commons-dev@jakarta.apa
che.org&msgNo=7694),
> as did a total of 4 committers (with no dissenters).
Both Object and Primitive collections have grown since then. I'm trying to
avoid serious class overload if Maps and Sets get added too.

> sc> How does this sound???
>
> As before, I'm open to (a) leaving the primitive collections where they
> are, (b) splitting the distribution of commons-collections into primitive
> and object based implementations or (c) proposing a new commons proper
> component, dependent upon commons-collections, containing the primitive
> collections and possibly more.  Note that to do (c) will require
> re-implementing, copying, refactoring or releasing the commons-collections
> test framework.
I prefer (c) and I don't believe it requires a release of the testing
framework beyond that of the collections source zip.

If I am forced, I will accept (b) although it doesn't feel right to me.

In addition, I wish to continue working on [primitives] in the sandbox with
the no adaptors design. It may be the case that both designs have their
place.

Stephen

Re: [primitives][collections] primitive collections and PROPOSAL

[Rodney Waldhoff <rw...@apache.org>]

On Wed, 3 Sep 2003, Stephen Colebourne wrote:

sc> From: "Rodney Waldhoff" <rw...@apache.org>

rw> > * I'm -1 to moving collections.primitives, or more generally,
rw> > any release ready code, to the sandbox, as this moves it
rw> > further away from a release rather than closer.

sc> I'm balancing this worry with the desire to not release something new
sc> that is in the wrong place. The basic issue is that primitive
sc> collections are quite disconnected (ie. totally) from the rest of
sc> the package. This argues
sc> for a separate project, as you have done in the
sc> past over parts of lang (which has happened).

What I have argued in the past, and I'll argue now, is that cohesive
components are constructed out of classes that are (a) interdependent, (b)
commonly used together, and/or (c) commonly change together.  Adapters and
test suites aside, currently collections.* and collections.primitives.*
fail the first test but pass the second two.  Of course, many of the
collections.* classes have this profile, for just one example, LRUMap and
ArrayIterator.

rw> > a) The two code bases are not truly independent, they share
rw> > a unit test suite, so if we split the two we'll probably want
rw> > to extract the shared unit testing framework.

sc> I would see this as a compile-time dependency on [collections].

Except that the unit testing framework isn't currently distributed in any
form.  This is a compile-time (or test-time) dependency on code that's
currently internal to the commons-collections code base.

rw> > b) It is quite likely that we'll eventually want primitive
rw> > implementations of Bag and other commons-collections-only
rw> > extensions, which would introduce direct
rw> > collections/primitive-collections dependencies, at
rw> > least at the adapter level.  This would also imply that
rw> > collections  and primitive-collections packages are likely to change
rw> > together, since changes to commons-collections extensions would
rw> > imply changes to the primitive collections.

sc> Primitive collections may depend on the Bag interface, but
sc> probably little else.