Re: [proposal] cornerstone roadmap

[Stephen McConnell <mc...@apache.org>]

Hi Leo:

I have committed maven based builds for four seperated cornerstone 
components.  I have seperated out the sources into distinct builds, each 
in their own project.  The project seperates the api and impl for each 
components and builds the respective consolidated jar file.

  1. cornerstone/threads
  2. cornerstone/sockets
  3. cornerstone/connection
  4. cornerstone/scheduler

All of the above build cleanly using Maven 9 (just hist "maven" and 
everything is done). I should have the following two blocks in-place 
later this evening.

  5. cornerstone/datasources
  6. cornerstne/masterstore

Something important to note is that I'm validating each of the 
components against James as I move forward.  A local version of James is 
running nicely with the threads, sockets, connection, and scheduler 
components.

Some additional comments in-line.

Steve.


Leo Simons wrote:

> Hi gang,
>
> with fortress & friends released, next up is getting cornerstone into 
> shape. Since it is currently still somewhat unreleased (and messy), it 
> makes sense to make all the tough decisions now.
>
>
> Summary
> -------
> Proposal: create cvs repository avalon-components, put in place maven 
> build, put most valuable cornerstone packages in there (list is 
> below), change package and directory structure but keep the 
> cornerstone name around, release after that. 


-1

Package names are in use and should be maintained.  Current package 
names are fine - no need to change them.

>
>
> I am volunteering to do everything excluding releases (not saying I 
> don't want to do releases, but not volunteering right now :D) in 
> roughly the same way I've done for fortress & framework.


For the moment I'm focussing only on the James dependencies.

>
> the rest of this e-mail is ramblings about alternatives and what made 
> me choose the above.
>
>
>
>
> Repository reorganisation
> -------------------------
> We can do this in a few ways. One thing I want to do is split up into 
> multiple packages (ie seperate source dirs for seperate components). 


Already done - see cornerstone subdirectories /threads, /sockets, 
/connection and /scheduler.

> Another thing I want to do is enforce api/impl 


+1
(already in place in the packages I've updated)

> seperation in directory structure (ie src/api/ and src/impl/). Yet 
> another thing I want to do is convert the build to maven.


The structure I've put in place (and recommend) is to place the impl and 
apis into seperate subprojects of a common group.  I.e. threads contains 
sup-project api and subproject impl (as per discussion on list a while 
back).  I not keen on changing that again because the current maven 
builds in cornerstone are working really well (and reflect the earlier 
discussion on the subject).

>
> I think everyone is okay with the above, some people more than willing 
> to help, everyone also okay with me just doing all that :D


I would like to see a little stabalization of the new Avalon build.  
Based on an attempt to build using Maven early yesterday I had a bunch 
of messages telling me I had to do a bunch of things (and the message 
was way too long).  This should not be necessary.  Using Maven the 
entire process should be automated.  I would prefer to see a clean 
solution (i.e. get maven, cd to your project, type "maven" and your 
done).  Following this we could integrate the framework split into the 
dependencies of the respective cornerstone projects.

>
>
> Repository consolidation
> ------------------------
> We have 2 component repositories: avalon-excalibur and 
> avalon-cornerstone. The division makes sense because one contains 
> components that run inside phoenix and the other contains components 
> that run inside ecm. 


I disagree with this distinction.  For me, Excalibur is about low-level 
utilities.  Cornerstone components are larger grain units.  Typically a 
cornerstone component is built using one or more Excalibur utility 
components.  Secondly, Cornerstone has been Phoenix independent for ages 
now.  In fact all of the builds I've put in place are 100% container 
independent.

> Except that it doesn't make sense to have components that run in one 
> container and not in the other when it is relatively easy to make them 
> quite portable, we now have many more containers than before, so the 
> division is purely historical. We have a few options:
>
> 1) new cvs repository avalon-components
>     (so you get avalon-components/cornerstone-datasource)
> 2) new location avalon/cornerstone
>     (so you get avalon/cornerstone/datasource)
> 3) new location avalon/excalibur
>     (so you get avalon/excalibur/cornerstone-datasource)
> 4) new location avalon-excalibur
>     (so you get avalon-excalibur/cornerstone-datasource)
> 5) same location
>     (so you get avalon-cornerstone/datasource) 


My preference - (5) - less work - lets us focus on getting the 
cornerstone components in-place.  Restructuring is way down on the list 
of priorities as far as I can see and can always be done once the 
cornerstone releases are completed.

>
>
> Now, I think merging everything into the single avalon/ repository is 
> not good as we've decided that we want to give out access to cocoon 
> people (and james people, potentially, and other people) to maintain 
> components but not mess with "core" stuff, and commit priviledge 
> partitioning is only possible on a per-repository basis. That rules 
> out #2 and #3.
>
> I also think that we should have a single repository (with single 
> build structure, single website, etc etc) for all our components. So 
> I'd like to rule out #5 too. Leaving me with #1 and #4. 


This is only valid if you ignore the different granularity of the 
object/components provided under excalibur versus cornerstone.  There is 
a very real difference from my point of view.

> The advantage of #1 is that it "feels" all new, we have no existing 
> repo history so very little cruft, and the name is more obvious to 
> casual visitors. The disadvantage is that, unless we refactor package 
> names to remove the name "cornerstone" alltogether, it is going to 
> totally stop making sense to those visitors that anything is named 
> "cornerstone". Furthermore when/if we refactor excalibur as well, that 
> can happen in the new repo so work interruption would generally be less.
>
>
>
> Package name refactoring
> ------------------------
> Going even further, we could actually refactor code as well, giving us
>
> sockets/src/api/org/apache/excalibur/sockets
> sockets/src/impl/org/apache/excalibur/sockets
>
>     or at least
>
> cornerstone-sockets/src/api/org/apache/avalon/cornerstone/sockets
> cornerstone-sockets/src/impl/org/apache/avalon/cornerstone/sockets
>
>     instead of
>
> cornerstone/src/java/org/apache/avalon/cornerstone/services/sockets
> cornerstone/src/java/org/apache/avalon/cornerstone/blocks/sockets


-1
No needed to change package names.
Packages are in use and we have committed to release at least the core 
suite.

>
>
> I've really grown to dislike the cornerstone package structure. Of 
> course, this might hit some users hard.
>
>
> Releases
> --------
> Conclusions from the "cornerstone cleanup" thread:
>
> These are in use by James and other projects and will be readied for 
> release ASAP:
>
>        cornerstone-sockets
>        cornerstone-store
>        cornerstone-connection
>        cornerstone-threads
>        cornerstone-datasource(s)
>        cornerstone-scheduler                (1) 


Process is underway (without package change).

>
>
> These should move into avalon-sandbox, into a location consistent with 
> whatever we decide for repo structure, until they are more mature:
>
>        cornerstone-dom
>        cornerstone-event
>        cornerstone-rmification
>        cornerstone-sax 


+1

>
>
> These are unused and unmaintained so they should be removed from cvs:
>
>        cornerstone-channels
>        cornerstone-packet
>        cornerstone-soapification 


+1

>
>
>
> IMNSHO
> ------ 


:-)

>
>
> 1) we should create a brand-new repository avalon-components, with a 
> maven build structure and a directory structure similar to what I have 
> put in place for the avalon repository
>
> 2) we should move the packages in list (1) there 


we should discuss this after a release of the core suite

>
>
> 3) we should keep the "cornerstone" name and package name around 
> because it has been around so long 


+1

>
>
> 4) we should "refactor mercilessly" and remove the (unclean, dated) 
> "services"/"blocks" package split in favor of split source structures 
> and jar files 


Can I presume that what you are thinking is 100% consitent with the 
stuff I've committed yesterday and earlier today?

>
> 5) once this is done, we should do an immediate beta1 release, 
> followed by a beta2 about 2 weeks later, followed by a beta3 yet 
> another 2 weeks later, followed by a final release (which will be 
> voted upon) yet another 2 weeks later (though there may be some summer 
> holidays in between ;)


Sounds reasonable.   If we defer the subject of avalon-components 
repository for the moment - we can actually focus on getting the release 
done. This also enables an important discussion that should take place 
concerning the distinction in the levels of abstractions - IMHO - 
excalibur is a very different level of abstraction to cornerstone - and 
these abstractions should not be merged together.

Cheers, Steve.

-- 

Stephen J. McConnell
mailto:mcconnell@apache.org
http://www.osm.net

Sent via James running under Merlin as an NT service.
http://avalon.apache.org/sandbox/merlin




---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@avalon.apache.org
For additional commands, e-mail: dev-help@avalon.apache.org

Re: [proposal] cornerstone roadmap

[Leo Simons <le...@apache.org>]

Berin Loritsch wrote:
>>>> seperation in directory structure (ie src/api/ and src/impl/).
>>>
>>> The structure I've put in place (and recommend) is to place the impl
>>> and apis into seperate subprojects of a common group.
 >>
 >> How important is it we consistently pick the same way?
> 
> Very important.

in that case we need a decision...I think seperate subprojects is an 
ugly setup to work around maven limitations regarding source directory 
structure (a limitation that is planned to go away). I think a 
workaround with a little more scripting to allow the conceptually 
cleaner setup like in the avalon/ repo is nicer.

I am, however, not feeling like doing the work to change things that 
have just been set up yet again :/

---

>> releases built and deployed. To keep docs from appearing, just change
>> buildsystem/maven.xml to have "dist" as the default goal (or something
>> like that) instead of "avalon:info". But that is a bad idea.
> 
> More common than "dist" would to build and run tests.  I would have
> "test" as the default target, and distributions get built later.

you can also have 'jar:jar' as the default target, which builds the jar 
and runs the tests. Unfortunately, it doesn't generate test reports; for 
that you need the 'maven-junitreport-plugin:report' goal IIRC.

- LSD



---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@avalon.apache.org
For additional commands, e-mail: dev-help@avalon.apache.org

Re: [proposal] cornerstone roadmap

[Berin Loritsch <bl...@apache.org>]

Leo Simons wrote:

> Stephen McConnell wrote: a lot. I reorganised some stuff to extract 
> bullet points:
> 
>     * reuse is bliss
>     * rtfm
>     * it doesn't matter if the standard sucks?
>     * restructure, then release
>     * leaky abstractions?
>     * who needs consensus anyway?
> 

> 
> It doesn't matter if the standard sucks?
> ----------------------------------------
> 
>>> seperation in directory structure (ie src/api/ and src/impl/). Yet
>>>  another thing I want to do is convert the build to maven.
>>
>>
>> The structure I've put in place (and recommend) is to place the impl
>> and apis into seperate subprojects of a common group.  I.e. threads
>> contains sup-project api and subproject impl (as per discussion on
>> list a while back).
> 
> 
> link / keyword? There's a 100 ways to do it. How important is it we 
> consistently pick the same way?

Very important.



> RTFM
> ----
> 
>> I would like to see a little stabalization of the new Avalon build.
>>  Based on an attempt to build using Maven early yesterday I had a
>> bunch of messages telling me I had to do a bunch of things (and the
>> message was way too long).  This should not be necessary.  Using
>> Maven the entire process should be automated.  I would prefer to see
>> a clean solution (i.e. get maven, cd to your project, type "maven"
>> and your done).
> 
> 
> MPSRTFM (More People Should Read The F$#@% Manual) :D
> 
>     cd avalon/framework
>     maven dist -Doverride.version=4.2
>     maven avalon:deploy -Doverride.version=4.2
> 
>     cd ../fortress/container
>     maven dist -Doverride.version=1.1
>     maven avalon:deploy -Doverride.version=1.1
> 
>     cd ../tools
>     maven dist -Doverride.version=1.1
>     maven avalon:deploy -Doverride.version=1.1
> 
> releases built and deployed. To keep docs from appearing, just change
> buildsystem/maven.xml to have "dist" as the default goal (or something
> like that) instead of "avalon:info". But that is a bad idea.

More common than "dist" would to build and run tests.  I would have
"test" as the default target, and distributions get built later.

> Restructure, then release
> -------------------------
> 
>>> 1) we should create a brand-new repository avalon-components, with
>>> a maven build structure and a directory structure similar to what I
>>> have put in place for the avalon repository
>>>
>>> 2) we should move the packages in list (1) there
>>
>>
>> we should discuss this after a release of the core suite
> 
> 
>> Restructuring is way down on the list of priorities as far as I can
>> see and can always be done once the cornerstone releases are
>> completed.
> 
> 
> hmm. This is exactly my point: we should do restructure now, get
> something stable in place, *then* release. Otherwise we'll end up with
> incompatible releases or big changes further down the road which will
> just annoy your users.
> 
> putting it another way: I'm not going to do any of this /after/ a
> release because at that point the incentive to do it is gone. I am
> willing to invest the time /once/ to refactor builds and automate
> everything.

How about this: we *rename* "Cornerstone" to "avalon-components".
After we are done with the restructuring, we can move/merge the
Excalibur components (DataSource, Monitor, SourceResolve, XMLUtil,
etc.) into that repo.

Whatever is left we can look at later.  It will provide a clean
separation of components and reusable container utility code.


> Leaky abstractions?
> -------------------
> 
>> I disagree with this distinction.  For me, Excalibur is about
>> low-level utilities.
> 
> 
> well, we disagree :D. I think many of the components in excalibur would 
> be sm.lookup()ped together with many of the components in cornerstone 
> and it would make sense. Oh well, we're not going to agree so lets drop 
> that for now...
> 
>> [a single repository (with single build structure, single website,
>> etc etc) for all our components] is only valid if you ignore the
>> different granularity of the object/components provided under
>> excalibur versus cornerstone.
> 
> 
> why?

I agree, the distinction is artificial.  Some of the Excalibur
components are actually higher level than the Cornerstone components.
All *components* should be in one repository--but we can do that
incrementally.

> Who needs consensus anyway?
> ---------------------------
>  > Can I presume that what you are thinking is 100% consitent with the
>  > stuff I've committed yesterday and earlier today?
> 
> well, nope :D. But that's okay.
> 
> you go ahead and whip cornerstone into shape, I'll keep out of the way. 
> Consider my proposal dropped in favor of Just Getting Things Done (By 
> You, Not Me).
> 
> cheers!


Ok.


---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@avalon.apache.org
For additional commands, e-mail: dev-help@avalon.apache.org

Re: [proposal] cornerstone roadmap

[Leo Simons <le...@apache.org>]

Stephen McConnell wrote: a lot. I reorganised some stuff to extract 
bullet points:

	* reuse is bliss
	* rtfm
	* it doesn't matter if the standard sucks?
	* restructure, then release
	* leaky abstractions?
	* who needs consensus anyway?

Reuse Is Bliss
--------------
> I have committed maven based builds for four seperated cornerstone 
> components.

cool!

while that setup is quite simple, it has the disadvantage that there's a
lot of files to maintain:

project.xml
${component}/maven.xml
${component}/project.xml
${component}/api/maven.xml
${component}/api/project.xml
${component}/impl/maven.xml
${component}/impl/project.xml

of which there's actually more than a few files one would need to edit
to change aspects of the build. I'd like that to just be

${component}/project.xml

with everything that is common actually shared.


It doesn't matter if the standard sucks?
----------------------------------------
>> seperation in directory structure (ie src/api/ and src/impl/). Yet
>>  another thing I want to do is convert the build to maven.
> 
> The structure I've put in place (and recommend) is to place the impl
> and apis into seperate subprojects of a common group.  I.e. threads
> contains sup-project api and subproject impl (as per discussion on
> list a while back).

link / keyword? There's a 100 ways to do it. How important is it we 
consistently pick the same way?


RTFM
----
> I would like to see a little stabalization of the new Avalon build.
>  Based on an attempt to build using Maven early yesterday I had a
> bunch of messages telling me I had to do a bunch of things (and the
> message was way too long).  This should not be necessary.  Using
> Maven the entire process should be automated.  I would prefer to see
> a clean solution (i.e. get maven, cd to your project, type "maven"
> and your done).

MPSRTFM (More People Should Read The F$#@% Manual) :D

	cd avalon/framework
	maven dist -Doverride.version=4.2
	maven avalon:deploy -Doverride.version=4.2

	cd ../fortress/container
	maven dist -Doverride.version=1.1
	maven avalon:deploy -Doverride.version=1.1

	cd ../tools
	maven dist -Doverride.version=1.1
	maven avalon:deploy -Doverride.version=1.1

releases built and deployed. To keep docs from appearing, just change
buildsystem/maven.xml to have "dist" as the default goal (or something
like that) instead of "avalon:info". But that is a bad idea.


Restructure, then release
-------------------------
>> 1) we should create a brand-new repository avalon-components, with
>> a maven build structure and a directory structure similar to what I
>> have put in place for the avalon repository
>> 
>> 2) we should move the packages in list (1) there
> 
> we should discuss this after a release of the core suite

> Restructuring is way down on the list of priorities as far as I can
> see and can always be done once the cornerstone releases are
> completed.

hmm. This is exactly my point: we should do restructure now, get
something stable in place, *then* release. Otherwise we'll end up with
incompatible releases or big changes further down the road which will
just annoy your users.

putting it another way: I'm not going to do any of this /after/ a
release because at that point the incentive to do it is gone. I am
willing to invest the time /once/ to refactor builds and automate
everything.


Leaky abstractions?
-------------------
> I disagree with this distinction.  For me, Excalibur is about
> low-level utilities.

well, we disagree :D. I think many of the components in excalibur would 
be sm.lookup()ped together with many of the components in cornerstone 
and it would make sense. Oh well, we're not going to agree so lets drop 
that for now...

> [a single repository (with single build structure, single website,
> etc etc) for all our components] is only valid if you ignore the
> different granularity of the object/components provided under
> excalibur versus cornerstone.

why?

> This also enables an important discussion that should take place 
> concerning the distinction in the levels of abstractions - IMHO - 
> excalibur is a very different level of abstraction to cornerstone -
> and these abstractions should not be merged together.

I think the differences are not as clear-cut as you paint them, nor do I
think that matters. I really don't see why the fact that the 
cornerstone-sockets interface is a little more abstract than the 
excalibur-sourceresolve would make it inconvenient to have these live 
side by side. We don't have the energy nor the desire to maintain all 
these different repos and build structures.


Who needs consensus anyway?
---------------------------
 > Can I presume that what you are thinking is 100% consitent with the
 > stuff I've committed yesterday and earlier today?

well, nope :D. But that's okay.

you go ahead and whip cornerstone into shape, I'll keep out of the way. 
Consider my proposal dropped in favor of Just Getting Things Done (By 
You, Not Me).

cheers!

- LSD, late for work (tm)



---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@avalon.apache.org
For additional commands, e-mail: dev-help@avalon.apache.org