You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@tuscany.apache.org by Jeremy Boynes <jb...@apache.org> on 2006/12/09 21:34:50 UTC
SCA Modularity
One of the challenges we faced with M2 was coordinating the release
of so many components. In the end, it took a very long time to get
the release out because we were waiting for Axis2 1.1 to be released
- a dependency that only affected some modules but for which we had
to hold up everything.
Also, Rick pinged me yesterday as he was having problems building
trunk. I tried and ran into (different) problems. The fix was simple
(thanks Raymond) but it illustrates the problem that trunk has a
whole has become too large for people to easily work with. There's so
much to build and it's fragile enough that people are just building
sub-sections and that is causing another set of problems.
I think an approach to this is to break the project down into a set
of modules along the lines where people typically work and to have a
much looser relationship between those modules. Each module becomes
responsible for its own code, testing, documentation, samples,
releases etc. but organizationally remains part of a single project
(one mailing list, one PMC, one SVN ACL). The boundary between
modules is technical - interfaces and a commitment to maintain them -
rather than social.
We started down this route in M2 with the modularization of Spec,
SDO, DAS and SCA; going forward I think we need to expand this down
into SCA. I'd propose the following:
* SCA Kernel (the apis, spis and default core impl)
* SCA Runtimes (hosts for the kernel that support users' runtime
scenarios)
Standalone
Webapp
Maven I-Test
Eclipse
etc.
* Extensions (components that add function to the bare runtime)
Axis2 web services (axiom, axis2 ws binding, ...)
Scripting and Web2.0 (script container, json databinding, ...)
J2EE integration (ejb container, rmi/iiop binding, ...)
Spring integration
OSGi integration
and many many more
* Distributions (things we'd expect a user to download and install)
* Integration test suites
* Sample applications (compound applications that integrate multiple
technologies)
Key to this is we should not operate as if everyone is always
building everything all the time - let's face it, many people are not
doing that now. Developers should be able to build, commit and
release any of these modules as needed. For that to work, developers
of modules that other people rely on need to make sure that the
changes they make cause as little disruption as possible - basically,
define a interface to the module, strive not to make incompatible
changes, and if you do let people know. To avoid the need to build
everything locally, modules can publish snapshots to the maven repo
that their downstream dependencies can use.
For example, if we publish a snapshot of the kernel module, all
extensions etc. can rely on those jars from the repo rather than
needing to build them ever time. If we keep the SPIs compatible, we
can make changes to the core impl without disrupting any of the
extensions; also we can add features to an extension, or even add a
new extension, without changing the kernel or impacting other
extensions. This does mean more work on both sides - we have to find
ways to add function without making incompatible changes to the SPI -
but it means that more people can get things done as each module can
move at its own pace.
Technically I think we're fairly close to having this already. This
is more of a social change in the project - changing our view away
from "build it all" and being willing to do the work to keep the
looser modules working together. I think though that we need to do
this if we're going to maintain progress.
Thanks
--
Jeremy
---------------------------------------------------------------------
To unsubscribe, e-mail: tuscany-dev-unsubscribe@ws.apache.org
For additional commands, e-mail: tuscany-dev-help@ws.apache.org
Re: SCA Modularity
Posted by Jim Marino <jm...@myromatours.com>.
On Dec 11, 2006, at 6:46 AM, Rick wrote:
> I like the fist break out that Jeremy proposed as I see there may
> be issues with keeping all this organized and lets give this a
> shot and see how well we manage it. After a while if we think
> things are going smooth start breaking out extensions on their own.
We need to break the extensions out upfront and not just reshuffle
the tree, right? The tight build and distribution coupling between
extensions and the kernel was what caused us problems and, IMO, we
need to address that.
> I do hope we can organize it to still allow building the latest in
> SVN TRUNK of all components so if we want to test how that all
> still hangs together it's possible without being too painful. The
> danger of going to long without catching incompatibilities I think
> is real too.
> Moving samples related to extensions to the extensions makes some
> sense, but I still think in the future we may want to offer some
> distribution artifact where they are bundled together and allow for
> a user to do a one shot download to play with them.
I agree with the "one shot" download if it is possible (probably
organized by runtime type). By bunding samples for a specific
extension with the latter, it would allow the extension to be
released on its own, giving more flexibility. Samples that used
multiple extensions would still be bundled under the separate samples
directory.
Jim
---------------------------------------------------------------------
To unsubscribe, e-mail: tuscany-dev-unsubscribe@ws.apache.org
For additional commands, e-mail: tuscany-dev-help@ws.apache.org
Re: SCA Modularity
Posted by Rick <cr...@gmail.com>.
I like the fist break out that Jeremy proposed as I see there may be issues with
keeping all this organized and lets give this a shot and see how well we
manage it. After a while if we think things are going smooth start breaking out
extensions on their own.
I do hope we can organize it to still allow building the latest in SVN TRUNK of
all components so if we want to test how that all still hangs together it's
possible without being too painful. The danger of going to long without
catching incompatibilities I think is real too.
Moving samples related to extensions to the extensions makes some sense, but I
still think in the future we may want to offer some distribution artifact where
they are bundled together and allow for a user to do a one shot download to play
with them.
Jim Marino wrote:
>
> On Dec 10, 2006, at 7:03 AM, Jeremy Boynes wrote:
>
>> On Dec 9, 2006, at 2:52 PM, Jim Marino wrote:
>>>> * Extensions (components that add function to the bare runtime)
>>>> Axis2 web services (axiom, axis2 ws binding, ...)
>>>> Scripting and Web2.0 (script container, json databinding, ...)
>>> A minor nit here - I would separate Scripting and Web 2.0 since I
>>> don't think they are inherently tied together, e.g. JSON could be a
>>> binding used by back-end components. Also, I'm not sure if we want to
>>> lump a bunch of languages under scripting, for example, Ruby or
>>> Groovy. Maybe we categorize more by functional role such as binding
>>> or component impl type and how people are likely to evolve them? I
>>> don't have a really strong preference on this.
>>
>> These suggestions were just meant to be illustrative and I also don't
>> have a strong feelings on the groupings. I wanted to capture that
>> there are likely to be groupings at a level above an individual module
>> but I think we'll figure out what they are as we go.
>>
>>>> J2EE integration (ejb container, rmi/iiop binding, ...)
>>>> Spring integration
>>>> OSGi integration
>>>> and many many more
>>> I like the idea of having a hierarchy of extension modules. I don't
>>> feel strongly about this either, but it may be useful to group by
>>> subtype like we have it today, e.g. binding extensions, component
>>> impl type extensions, etc. It may make things a little more
>>> manageable moving forward as the number of extensions increases.
>>
>> I've been wondering if this is the right breakdown. IMO the grouping
>> of a module should be driven by what we would like to release together
>> and that may span different types of modules. For example, "Axis2 Web
>> Services" would include the AXIOM databinding, the Axis2 binding, and
>> perhaps a (hypothetical) JAX-WS container with the main reason for
>> grouping them together being that they all need to use the same
>> version of Axis2. On the converse side, I don't see why we would
>> release a diverse bag of scripting containers together so breaking
>> "Scripting" apart as you suggested above is a good idea.
>>
> I think this approach makes sense. This has come up as well in the
> situation where modules depend on versions of other modules such as the
> WSDL2Java tool. We should perhaps group things by how they are
> released/evolved together and also provide some higher-level
> organization. For example, the Axis2 and related modules would go under
> bindings/axis2.
>
> Jim
>
>> --
>> Jeremy
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: tuscany-dev-unsubscribe@ws.apache.org
>> For additional commands, e-mail: tuscany-dev-help@ws.apache.org
>>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: tuscany-dev-unsubscribe@ws.apache.org
> For additional commands, e-mail: tuscany-dev-help@ws.apache.org
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: tuscany-dev-unsubscribe@ws.apache.org
For additional commands, e-mail: tuscany-dev-help@ws.apache.org
Re: SCA Modularity
Posted by Jim Marino <jm...@myromatours.com>.
On Dec 10, 2006, at 7:03 AM, Jeremy Boynes wrote:
> On Dec 9, 2006, at 2:52 PM, Jim Marino wrote:
>>> * Extensions (components that add function to the bare runtime)
>>> Axis2 web services (axiom, axis2 ws binding, ...)
>>> Scripting and Web2.0 (script container, json databinding, ...)
>> A minor nit here - I would separate Scripting and Web 2.0 since I
>> don't think they are inherently tied together, e.g. JSON could be
>> a binding used by back-end components. Also, I'm not sure if we
>> want to lump a bunch of languages under scripting, for example,
>> Ruby or Groovy. Maybe we categorize more by functional role such
>> as binding or component impl type and how people are likely to
>> evolve them? I don't have a really strong preference on this.
>
> These suggestions were just meant to be illustrative and I also
> don't have a strong feelings on the groupings. I wanted to capture
> that there are likely to be groupings at a level above an
> individual module but I think we'll figure out what they are as we go.
>
>>> J2EE integration (ejb container, rmi/iiop binding, ...)
>>> Spring integration
>>> OSGi integration
>>> and many many more
>> I like the idea of having a hierarchy of extension modules. I
>> don't feel strongly about this either, but it may be useful to
>> group by subtype like we have it today, e.g. binding extensions,
>> component impl type extensions, etc. It may make things a little
>> more manageable moving forward as the number of extensions increases.
>
> I've been wondering if this is the right breakdown. IMO the
> grouping of a module should be driven by what we would like to
> release together and that may span different types of modules. For
> example, "Axis2 Web Services" would include the AXIOM databinding,
> the Axis2 binding, and perhaps a (hypothetical) JAX-WS container
> with the main reason for grouping them together being that they all
> need to use the same version of Axis2. On the converse side, I
> don't see why we would release a diverse bag of scripting
> containers together so breaking "Scripting" apart as you suggested
> above is a good idea.
>
I think this approach makes sense. This has come up as well in the
situation where modules depend on versions of other modules such as
the WSDL2Java tool. We should perhaps group things by how they are
released/evolved together and also provide some higher-level
organization. For example, the Axis2 and related modules would go
under bindings/axis2.
Jim
> --
> Jeremy
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: tuscany-dev-unsubscribe@ws.apache.org
> For additional commands, e-mail: tuscany-dev-help@ws.apache.org
>
---------------------------------------------------------------------
To unsubscribe, e-mail: tuscany-dev-unsubscribe@ws.apache.org
For additional commands, e-mail: tuscany-dev-help@ws.apache.org
Re: SCA Modularity
Posted by Jeremy Boynes <jb...@apache.org>.
On Dec 9, 2006, at 2:52 PM, Jim Marino wrote:
>> * Extensions (components that add function to the bare runtime)
>> Axis2 web services (axiom, axis2 ws binding, ...)
>> Scripting and Web2.0 (script container, json databinding, ...)
> A minor nit here - I would separate Scripting and Web 2.0 since I
> don't think they are inherently tied together, e.g. JSON could be a
> binding used by back-end components. Also, I'm not sure if we want
> to lump a bunch of languages under scripting, for example, Ruby or
> Groovy. Maybe we categorize more by functional role such as binding
> or component impl type and how people are likely to evolve them? I
> don't have a really strong preference on this.
These suggestions were just meant to be illustrative and I also don't
have a strong feelings on the groupings. I wanted to capture that
there are likely to be groupings at a level above an individual
module but I think we'll figure out what they are as we go.
>> J2EE integration (ejb container, rmi/iiop binding, ...)
>> Spring integration
>> OSGi integration
>> and many many more
> I like the idea of having a hierarchy of extension modules. I don't
> feel strongly about this either, but it may be useful to group by
> subtype like we have it today, e.g. binding extensions, component
> impl type extensions, etc. It may make things a little more
> manageable moving forward as the number of extensions increases.
I've been wondering if this is the right breakdown. IMO the grouping
of a module should be driven by what we would like to release
together and that may span different types of modules. For example,
"Axis2 Web Services" would include the AXIOM databinding, the Axis2
binding, and perhaps a (hypothetical) JAX-WS container with the main
reason for grouping them together being that they all need to use the
same version of Axis2. On the converse side, I don't see why we would
release a diverse bag of scripting containers together so breaking
"Scripting" apart as you suggested above is a good idea.
--
Jeremy
---------------------------------------------------------------------
To unsubscribe, e-mail: tuscany-dev-unsubscribe@ws.apache.org
For additional commands, e-mail: tuscany-dev-help@ws.apache.org
Re: SCA Modularity
Posted by Jim Marino <jm...@myromatours.com>.
On Dec 9, 2006, at 12:34 PM, Jeremy Boynes wrote:
> One of the challenges we faced with M2 was coordinating the release
> of so many components. In the end, it took a very long time to get
> the release out because we were waiting for Axis2 1.1 to be
> released - a dependency that only affected some modules but for
> which we had to hold up everything.
>
> Also, Rick pinged me yesterday as he was having problems building
> trunk. I tried and ran into (different) problems. The fix was
> simple (thanks Raymond) but it illustrates the problem that trunk
> has a whole has become too large for people to easily work with.
> There's so much to build and it's fragile enough that people are
> just building sub-sections and that is causing another set of
> problems.
>
> I think an approach to this is to break the project down into a set
> of modules along the lines where people typically work and to have
> a much looser relationship between those modules. Each module
> becomes responsible for its own code, testing, documentation,
> samples, releases etc. but organizationally remains part of a
> single project (one mailing list, one PMC, one SVN ACL). The
> boundary between modules is technical - interfaces and a commitment
> to maintain them - rather than social.
>
Overall, I like this. I have included some specific comments below.
> We started down this route in M2 with the modularization of Spec,
> SDO, DAS and SCA; going forward I think we need to expand this down
> into SCA. I'd propose the following:
> * SCA Kernel (the apis, spis and default core impl)
> * SCA Runtimes (hosts for the kernel that support users' runtime
> scenarios)
> Standalone
> Webapp
> Maven I-Test
> Eclipse
You didn't mention IDEA, does this mean you have switched to
Eclipse? ;-)
> etc.
> * Extensions (components that add function to the bare runtime)
> Axis2 web services (axiom, axis2 ws binding, ...)
> Scripting and Web2.0 (script container, json databinding, ...)
A minor nit here - I would separate Scripting and Web 2.0 since I
don't think they are inherently tied together, e.g. JSON could be a
binding used by back-end components. Also, I'm not sure if we want to
lump a bunch of languages under scripting, for example, Ruby or
Groovy. Maybe we categorize more by functional role such as binding
or component impl type and how people are likely to evolve them? I
don't have a really strong preference on this.
> J2EE integration (ejb container, rmi/iiop binding, ...)
> Spring integration
> OSGi integration
> and many many more
I like the idea of having a hierarchy of extension modules. I don't
feel strongly about this either, but it may be useful to group by
subtype like we have it today, e.g. binding extensions, component
impl type extensions, etc. It may make things a little more
manageable moving forward as the number of extensions increases.
> * Distributions (things we'd expect a user to download and install)
> * Integration test suites
We will need to think how we want to structure this as well.
> * Sample applications (compound applications that integrate
> multiple technologies)
>
I assume samples that just deal with one particular extension would
be located with the extension
> Key to this is we should not operate as if everyone is always
> building everything all the time - let's face it, many people are
> not doing that now. Developers should be able to build, commit and
> release any of these modules as needed. For that to work,
> developers of modules that other people rely on need to make sure
> that the changes they make cause as little disruption as possible -
> basically, define a interface to the module, strive not to make
> incompatible changes, and if you do let people know. To avoid the
> need to build everything locally, modules can publish snapshots to
> the maven repo that their downstream dependencies can use.
>
> For example, if we publish a snapshot of the kernel module, all
> extensions etc. can rely on those jars from the repo rather than
> needing to build them ever time. If we keep the SPIs compatible, we
> can make changes to the core impl without disrupting any of the
> extensions; also we can add features to an extension, or even add a
> new extension, without changing the kernel or impacting other
> extensions. This does mean more work on both sides - we have to
> find ways to add function without making incompatible changes to
> the SPI - but it means that more people can get things done as each
> module can move at its own pace.
>
> Technically I think we're fairly close to having this already. This
> is more of a social change in the project - changing our view away
> from "build it all" and being willing to do the work to keep the
> looser modules working together. I think though that we need to do
> this if we're going to maintain progress.
I would agree, progress has really slowed in the last several months.
Jim
>
> Thanks
> --
> Jeremy
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: tuscany-dev-unsubscribe@ws.apache.org
> For additional commands, e-mail: tuscany-dev-help@ws.apache.org
>
---------------------------------------------------------------------
To unsubscribe, e-mail: tuscany-dev-unsubscribe@ws.apache.org
For additional commands, e-mail: tuscany-dev-help@ws.apache.org