You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@tuscany.apache.org by Jean-Sebastien Delfino <js...@apache.org> on 2008/05/01 01:40:11 UTC

Re: Improving support for running in OSGi

Graham Charters wrote:
> Hi Raymond, thanks for your comments.  I've added some more below.
> 
> Regards, Graham.
> 
> 2008/4/30 Raymond Feng <en...@gmail.com>:
>> More comments inline.
>>
>>  Thanks,
>>  Raymond
>>  --------------------------------------------------
>>  From: "Graham Charters" <gc...@googlemail.com>
>>  Sent: Wednesday, April 30, 2008 9:43 AM
>>
>>  To: <tu...@ws.apache.org>
>>  Subject: Re: Improving support for running in OSGi
>>
>>
>>> 2008/4/30 Raymond Feng <en...@gmail.com>:
>>>
>>>> "Enforcing" the modularity via OSGi is a good way to validate our
>>>> modularity/extensibility story in Tuscany. I think we already have a
>> fairly
>>>> well organized module structure in Tuscany (from the maven dependency
>>>> perspective). To be consistent, should we consider directly mapping our
>>>> module structure into OSGi bundles (one bundle per existing Tuscany
>> module)?
>>>>
>>> I wonder if that might be too fine-grained and unmanageable.
>>> Personally, I'd prefer an approach which took into consideration how
>>> we see folks extending or sub-setting the runtime (e.g. the things
>>> Yang referred to - implementation types, etc.).
>>>
>>>
>>  IMHO, big bundles lead to defeat modularity and increase coupling. It also
>> become difficult to embed subset of the Tuscany runtime. But anyway, we can
>> adjust the scheme over time as the picture becomes clearer.
>>
>>
> 
> To a certain extent I agree with you, but there's also a school of
> thought that says when adopting OSGi it's best to start with big
> modules and then decompose as understanding grows.  I think the
> thought processes we go through will ensure we end up with the right
> modules that match the requirements, such as runtime subsetting.
> 
>>>>  There may be different levels for the OSGi enablement.
>>>>
>>>>  1) Add OSGi entries to the META-INF/MANIFEST.MF so that Tuscany jars
>> can be
>>>> consumed as OSGi bundles
>>>>  2) Run Tuscany bundles with an OSGi runtime (using OSGi as the
>>>> modularization mechanism for the Tuscany runtime, system level)
>>>>
>>> I think we need this step to occur regularly and frequently to stop
>>> the support decaying.  I've fixed these kinds of issues twice in the
>>> last week, which occur because new modules are added to Tuscany, but
>>> not the OSGi support (not surprising since the OSGi support is outside
>>> the main build).
>>>
>>>
>>  Is there a checklist we can use to make sure new modules or changes won't
>> break the OSGi support? If not, can we produce one and post it to the
>> Tuscany wiki?
>>
>>
> 
> I think we could produce a checklist to help folks.  It would also be
> nice to make it easier for people to get things right without it.  The
> idea of updating the module structure to reflect the bundles we create
> is interesting.  I guess if the number of bundles is changing rapidly
> then this might be a bit painful.  We could also have the bundles
> build on continuum so we find problems earlier.  Having everyone build
> the bundles might not be acceptable.
> 
>>>>  3) Support OSGi bundles as SCA contributions (application level, how
>> does
>>>> the OSGi import/export relate to the sca-contributions.xml?).
>>>>
>>>>  The other interesting issue is how we deal with the 3rd party jars,
>> most of
>>>> which are not OSGi bundles. To support the case that different Tuscany
>>>> extensions may require different versions of the same 3rd party jar, we
>> need
>>>> to figure out a good way to "turn" 3rd party jars into OSGi bundles.
>>>>
>>>>
>>> This is a knotty problem.  It's not a new problem and is discussed
>>> here:  https://mail.osgi.org/pipermail/jsr-291-eg/2006-March/000023.html
>>>
>>> Licensing and signing can prevent us from adding information to the
>>> jars, which leaves us with three options, I think:
>>>
>>> 1.  Create wrapper bundles which contain the jars, which raises
>>> concerns about disk space.
>>> 2.  Use a non-OSGi mechanism to refer to jars outside a bundle
>>> (Equinox allows this).
>>> 3.  Devise a new mechanism to create 'virtual bundles' on the fly (see
>>> https://www.osgi.org/members/bugzilla/show_bug.cgi?id=277).
>>>
>>>
>>  Unfortunately, the URL requires a log-in. Is it possible to post the
>> content here?
>>
> 
> My apologies, I hadn't realized this was a member bug and therefore
> required access (I must have already been logged in when I tested it).
>  I can't paste in the details, but I can summarize.  This bug
> discusses the requirement raised in the first link.  It discusses the
> approach to allow the bundle classpath to reference jars outside the
> bundle (the equinox approach), and also considers a 'virtual bundle'
> concept (which seems preferred).  There aren't many details about the
> latter.
> 
>>
>>
>>> 3 might be the best solution longer term, but is the most work.
>>>
>>>
>>>>  Thanks,
>>>>  Raymond
>>>>
>>>>  --------------------------------------------------
>>>>  From: "Yang Lei" <ge...@gmail.com>
>>>>  Sent: Tuesday, April 29, 2008 9:09 PM
>>>>  To: <tu...@ws.apache.org>
>>>>  Subject: Re: Improving support for running in OSGi
>>>>
>>>>
>>>>
>>>>
>>>>> I think enabling OSGI can help modularity with a clear definition of
>>>>> package visibility, so we can have a much cleaner "module"
>>>>> dependencies through osgi bundle import/export on package.   I think
>>>>> it will help Tuscany adopters a lot in the scenarios such as: when
>>>>> implementing new implementation type, binding type, or data binding
>>>>> types, there is clear indication which set of packages can be used
>>>>> (exported API/SPI ). So upgrading to new Tuscany level can be as
>>>>> simple as plug and play if there is no API/SPI changes, and  version
>>>>> control (multiple version co-existence) can also be made available
>>>>> through OSGI capabilities.
>>>>>
>>>>> Regards,
>>>>>
>>>>> Yang
>>>>>
>>>>> On Tue, Apr 29, 2008 at 1:49 PM, Konradi, Philipp (CT)
>>>>> <ph...@siemens.com> wrote:
>>>>>
>>>>>> Hi,
>>>>>>
>>>>>>> I'd like to get people's thoughts on whether the idea of > > >
>> 'promoting'
>>>>>>> OSGi is a good one,
>>>>>> IMHO support of OSGi is very important and I glad to see increasing
>>>> interest of the community here.
>>>>>>> and get ideas on how best to proceed.
>>>>>> I personally have currently not a very deep insight into > >
>> implementation
>>>> details yet, but we are currently prototyping and have there also OSGi
>>>> services.
>>>>>> What I could offer today is only to feed our findings about > >
>> limitations
>>>> and rooms for improvement back.
>>>>>> Another important thing which I see on the horizon, is the ongoing
>>>> standardization of Distributed OSGi (RFC119) and the benefit to support
>> that
>>>> standard in Tuscany's OSGi bits. So from mid-term perspective I suggest
>> to
>>>> keep an eye on that as well.
>>>>>> Regards,
>>>>>> Philipp
>>>>>>
>>>>>> -----Ursprüngliche Nachricht-----
>>>>>> Von: Graham Charters [mailto:gcharters@googlemail.com]
>>>>>> Gesendet: Montag, 28. April 2008 09:48
>>>>>> An: tuscany-dev@ws.apache.org
>>>>>> Betreff: Improving support for running in OSGi
>>>>>>
>>>>>>
>>>>>> Hi All,
>>>>>>
>>>>>> I'd like to get more involved in the OSGi support in Tuscany (both >
>>> the
>>>>>> modularity work (itest/osgi-tuscany) and the implementation.osgi).
>> I
>>>>>> recently started looking at the work to run Tuscany in OSGi,
>> embodied
>>>>>> in itest/osgi-tuscany and described in the thread entitled
>>>>>> "Classloading in Tuscany".  I've noticed a couple of others on the
>>>>>> list also interested in Tuscany running in OSGi and wondered if it
>>>>>> might be worth considering making this a "first-class" option.  At >
>>> the
>>>>>> moment the five bundles are only built by folks who want the OSGi
>>>>>> support and go into the itest/osgi-tuscany directory to create it.
>>>>>> This can result in any problems being discovered late, but also
>> could
>>>>>> create the impression that OSGi is considered a second-class
>>>>>> environment (which I don't believe is the case).
>>>>>>
>>>>>> Aside from the obvious benefits to OSGi users in doing this, I think
>>>>>> there's a potential for Tuscany to use the OSGi build as a test-bed
>>>>>> for highlighting and working through modularity issues, which would
>>>>>> also benefit Tuscany in general, not just in an OSGi runtime.
>>>>>>
>>>>>> I'd like to get people's thoughts on whether the idea of 'promoting'
>>>>>> OSGi is a good one, and get ideas on how best to proceed.  We could
>>>>>> then start discussing what some of the issues might be (e.g. size of
>>>>>> builds, time to build, etc...).
>>>>>>
>>>>>> Regards,
>>>>>>
>>>>>> Graham.
>>>>>>
>>>>>>
>>>>

My 2c:

+1 to promote OSGi to a first class Tuscany runtime environment

+1 for an OSGi continuum build (thinking about a build profile that'll 
run the Tuscany itest suite in an OSGi environment, similar to the 
profiles we have for Web containers for example)

Here's what I imagined we'd do:
1. add OSGi entries to each of our JAR manifests
2. have developers maintain them and pay attention to imports/exports
3. use the OSGi build to detect API and SPI import/export violations
4. find the best way to OSGi-enable 3rd party dependency JARs

I realize that my suggestion [1] is not very popular and most people on 
this list would prefer to come up with bigger bundles grouping several 
of our JARs/modules. I don't think that the 'bigger aggregate bundle' 
approach will work, but I'll be happy to watch people try it :) if they 
  want to.

With respect to [4] I find rather funny to see many projects out there 
claim OSGi enablement without having OSGified their 3rd party 
dependencies. I wonder how that works, can an OSGi-enabled project 
really leverage the OSGi classloader isolation and versioning 
capabilities when 99% of the JARs it requires are not OSGi bundles? I 
must be missing something... and I hope we can do better in Tuscany with 
a real end-to-end OSGi enablement story :)
-- 
Jean-Sebastien

Re: Improving support for running in OSGi

Posted by Jean-Sebastien Delfino <js...@apache.org>.
ant elder wrote:
> On Thu, May 1, 2008 at 12:40 AM, Jean-Sebastien Delfino <
> jsdelfino@apache.org> wrote:
> 
> <snip>
> 
> 
>> Here's what I imagined we'd do:
>> 1. add OSGi entries to each of our JAR manifests
>> 2. have developers maintain them and pay attention to imports/exports
>> 3. use the OSGi build to detect API and SPI import/export violations
>> 4. find the best way to OSGi-enable 3rd party dependency JARs
>>
>> I realize that my suggestion [1] is not very popular and most people on
>> this list would prefer to come up with bigger bundles grouping several of
>> our JARs/modules. I don't think that the 'bigger aggregate bundle' approach
>> will work, but I'll be happy to watch people try it :) if they  want to.
>>
> 
> Perfectly ok but would you say a bit more about why you don't think the
> bigger aggregate bundle approach will work? I like the sound of less bundles
> as it seems like it could be easier to use but if you know of issues it
> would be nice to hear them now.
> 
>    ...ant
> 

A few issues:

- We're already running into problems where modules are not fine grain 
enough, some of our implementations/bindings can't be used to 
introspect/represent SCA components and bindings without dragging the 
whole runtime. Aggregating more will make that worse.

- Shared 3rd party or internal dependencies will make it difficult to 
create these big bundles, unless you copy them into each bundle, or 
create a bundle for these shared dependencies. Once you've gone through 
that exercise you'll get really close to 1 bundle <-> 1 module.

- Aggregate bundles will make it more difficult to use different levels 
of dependencies. For example if our BPEL and XQuery implementation 
extensions need different levels of Saxon, you can't put them in the 
same bundle.

and I'm sure you'll find more issues if you actually try.
-- 
Jean-Sebastien

Re: Improving support for running in OSGi

Posted by ant elder <an...@gmail.com>.
On Thu, May 1, 2008 at 12:40 AM, Jean-Sebastien Delfino <
jsdelfino@apache.org> wrote:

<snip>


> Here's what I imagined we'd do:
> 1. add OSGi entries to each of our JAR manifests
> 2. have developers maintain them and pay attention to imports/exports
> 3. use the OSGi build to detect API and SPI import/export violations
> 4. find the best way to OSGi-enable 3rd party dependency JARs
>
> I realize that my suggestion [1] is not very popular and most people on
> this list would prefer to come up with bigger bundles grouping several of
> our JARs/modules. I don't think that the 'bigger aggregate bundle' approach
> will work, but I'll be happy to watch people try it :) if they  want to.
>

Perfectly ok but would you say a bit more about why you don't think the
bigger aggregate bundle approach will work? I like the sound of less bundles
as it seems like it could be easier to use but if you know of issues it
would be nice to hear them now.

   ...ant

Re: Improving support for running in OSGi

Posted by Jean-Sebastien Delfino <js...@apache.org>.
Simon Laws wrote:
> On Thu, May 1, 2008 at 1:00 PM, Graham Charters <gc...@googlemail.com>
> wrote:
> 
>> It would seem that the fine-grained/coarse-grained thoughts have
>> people divided.  Rajini's note (aside from the fact she has a tonne of
>> experience having done most, if not all, of the OSGi work in Tuscany)
>> paints a picture where the two are not mutually exclusive.  I don't
>> typically like doing two options because that seems like indecision,
>> but in this case they do appear to complement each other.
>>
>> Based on what I've seen in this thread, I'm hoping this briefly
>> summarizes the collection of thoughts on how we might proceed:
>>
>> Tuscany Running in OSGi
>>
>> 1.  Add bundle manifests to all the Tuscany modules (using maven
>> bundle plugin).  This will ensure the most fine-grained decomposition
>> works and therefore coarser-grained distributions will also work.
>> 2.  Add a distribution which creates bundles around manageable
>> collections of Tuscany modules aligned with how we see the runtime
>> being extended/subsetted/replaced.  Have this build and test on
>> Continuum.
>> 3.  Create 'virtual bundles' for third-party libraries to avoid
>> licensing and disk space issues (based on Rajini's suggestions, which
>> I need to better understand).
>> 4.  Provide guidance on the wiki on how to avoid breaking OSGi.
>>
>> There's also the suggestion that implementation.osgi relate to
>> Distributed OSGi (RFC 119), which shouldn't be lost.
>>
>> Have I missed anything?
>>
>> It sounds like a staging of 1 & 3 first, followed by 2 would work (and
>> 4 if things keep breaking :-( ).  I'd be concerned if we didn't get to
>> 2, and as part of 1&3, we need to make sure these are regularly tested
>> under osgi.
>>
>>
>> 2008/5/1 Mike Edwards <mi...@gmail.com>:
>>> Simon Laws wrote:
>>>
>>>> On Thu, May 1, 2008 at 10:03 AM, Rajini Sivaram <
>>>> rajinisivaram@googlemail.com> wrote:
>>>>
>>>>
>>>>> On 5/1/08, Jean-Sebastien Delfino <js...@apache.org> wrote:
>>>>>
>>>>>> My 2c:
>>>>>>
>>>>>> +1 to promote OSGi to a first class Tuscany runtime environment
>>>>>>
>>>>>> +1 for an OSGi continuum build (thinking about a build profile
>> that'll
>>>>> run
>>>>>
>>>>>> the Tuscany itest suite in an OSGi environment, similar to the
>>> profiles
>>>>> we
>>>>>
>>>>>> have for Web containers for example)
>>>>>>
>>>>>> Here's what I imagined we'd do:
>>>>>> 1. add OSGi entries to each of our JAR manifests
>>>>>> 2. have developers maintain them and pay attention to
>> imports/exports
>>>>>> 3. use the OSGi build to detect API and SPI import/export
>> violations
>>>>>> 4. find the best way to OSGi-enable 3rd party dependency JARs
>>>>>>
>>>>>> I realize that my suggestion [1] is not very popular and most
>> people
>>> on
>>>>>> this list would prefer to come up with bigger bundles grouping
>> several
>>>>> of
>>>>>
>>>>>> our JARs/modules. I don't think that the 'bigger aggregate bundle'
>>>>>>
>>>>> approach
>>>>>
>>>>>> will work, but I'll be happy to watch people try it :) if they
>>  want
>>> to.
>>>>>> With respect to [4] I find rather funny to see many projects out
>> there
>>>>>> claim OSGi enablement without having OSGified their 3rd party
>>>>>>
>>>>> dependencies.
>>>>>
>>>>>> I wonder how that works, can an OSGi-enabled project really
>> leverage
>>> the
>>>>>> OSGi classloader isolation and versioning capabilities when 99% of
>> the
>>>>> JARs
>>>>>
>>>>>> it requires are not OSGi bundles? I must be missing something...
>> and I
>>>>> hope
>>>>>
>>>>>> we can do better in Tuscany with a real end-to-end OSGi enablement
>>> story
>>>>> :)
>>>>>
>>>>>> --
>>>>>> Jean-Sebastien
>>>>>>
>>>>>>
>>>>> I agree with Sebastien's suggestions1-4. But I would like to suggest
>> a
>>>>> slight variation to the first suggestion.
>>>>>
>>>>>
>>>>>  1. Use maven-bundle-plugin to introduce OSGi manifest entries into
>> all
>>>>>  the Tuscany modules, with auto-generated import/export statements.
>>>>> Modify
>>>>>  itest/osgi-tuscany to run these modules under OSGi, with all 3rd
>> party
>>>>> jars
>>>>>  installed into OSGi using virtual bundles created on the fly.
>>>>>
>>>>> This step will provide a version of osgi-tuscany tests that is less
>>> prone
>>>>> to
>>>>> breakage than the one we have today. It will also help fix any
>> remaining
>>>>> classloading issues that we have left in Tuscany (and hopefully help
>>>>> in maintaining the classloader isolation). This is not a big piece
>> of
>>> work
>>>>> since this is just bringing together the different pieces that we
>>> already
>>>>> have. I will be happy to contribute the code towards this first
>> step, so
>>>>> others can concentrate on what we really want to achieve in terms of
>>>>> modularity, distribution etc. We can also use this step to explore
>>>>> versioning - in particular about having multiple extensions
>> referring to
>>>>> different versions of 3rd party libraries. This will be very useful
>> for
>>> 4.
>>>>> Suggestions 2-3 are not requirements for OSGi, but these are clearly
>>> cases
>>>>> where OSGi technology can help Tuscany improve modularity. If we
>> want to
>>>>> have explicitly hard-coded import/export statements in the modules
>> to
>>>>> enforce modularity, that can be introduced in step 2.
>>>>>
>>>>> And I would expect that there will be more work in terms of building
>> the
>>>>> distributions suitable for OSGi as well as non-OSGi after 1-4.
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> Thank you...
>>>>>
>>>>> Regards,
>>>>>
>>>>> Rajini
>>>>>
>>>>>
>>>> Re. modularity, it strikes me that there is lack of consensus about
>>> whether
>>>> big bundles or small bundles, relative to our maven modules,  are
>>> preferable
>>>> but that there is consensus about using OSGi bundles to help us test
>> and
>>>> refine out modularity story.
>>>>
>>>> So maybe we should agree to disregard the question about how big OSGi
>>>> bundles should be relative to our maven modules and concentrate on the
>>>> question about whether our maven modules are correctly factored. Using
>>> OSGi
>>>> as a tool to help us with this of course.
>>>>
>>>> Simon
>>>>
>>>>
>>>  I support both Rajini's and Simon's points.  We can get the mechanics
>> going
>>> first with the bundles following the existing grain of the modules - and
>> we
>>> can discuss and tinker with the modules as we please later to find the
>> best
>>> arrangements.
>>>
>>>  I suspect that the differing opinions about module size reflect
>> different
>>> usecases.  Lots of small modules can promote great flexibility in the
>>> construction of runtimes, but at the expense of making it much harder
>> for
>>> users to make sense of the options.  Larger modules cut down the options
>> but
>>> make things easier for end users.
>>>
>>>
>>>  Yours,  Mike.
>>>
> 
> I would further propose that we take a subset of the output of 1 and 3, for
> example, enough to run the calculator sample, as an initial test case
> separate from the full range of itest/osgi-tuscany tests. This would give us
> something to shoot re. inclusion in the regular build.
> 
> Simon
> 

+1 Very good idea

-- 
Jean-Sebastien

Re: Improving support for running in OSGi

Posted by Simon Laws <si...@googlemail.com>.
On Thu, May 1, 2008 at 1:00 PM, Graham Charters <gc...@googlemail.com>
wrote:

> It would seem that the fine-grained/coarse-grained thoughts have
> people divided.  Rajini's note (aside from the fact she has a tonne of
> experience having done most, if not all, of the OSGi work in Tuscany)
> paints a picture where the two are not mutually exclusive.  I don't
> typically like doing two options because that seems like indecision,
> but in this case they do appear to complement each other.
>
> Based on what I've seen in this thread, I'm hoping this briefly
> summarizes the collection of thoughts on how we might proceed:
>
> Tuscany Running in OSGi
>
> 1.  Add bundle manifests to all the Tuscany modules (using maven
> bundle plugin).  This will ensure the most fine-grained decomposition
> works and therefore coarser-grained distributions will also work.
> 2.  Add a distribution which creates bundles around manageable
> collections of Tuscany modules aligned with how we see the runtime
> being extended/subsetted/replaced.  Have this build and test on
> Continuum.
> 3.  Create 'virtual bundles' for third-party libraries to avoid
> licensing and disk space issues (based on Rajini's suggestions, which
> I need to better understand).
> 4.  Provide guidance on the wiki on how to avoid breaking OSGi.
>
> There's also the suggestion that implementation.osgi relate to
> Distributed OSGi (RFC 119), which shouldn't be lost.
>
> Have I missed anything?
>
> It sounds like a staging of 1 & 3 first, followed by 2 would work (and
> 4 if things keep breaking :-( ).  I'd be concerned if we didn't get to
> 2, and as part of 1&3, we need to make sure these are regularly tested
> under osgi.
>
>
> 2008/5/1 Mike Edwards <mi...@gmail.com>:
> >
> > Simon Laws wrote:
> >
> > > On Thu, May 1, 2008 at 10:03 AM, Rajini Sivaram <
> > > rajinisivaram@googlemail.com> wrote:
> > >
> > >
> > > > On 5/1/08, Jean-Sebastien Delfino <js...@apache.org> wrote:
> > > >
> > > > > My 2c:
> > > > >
> > > > > +1 to promote OSGi to a first class Tuscany runtime environment
> > > > >
> > > > > +1 for an OSGi continuum build (thinking about a build profile
> that'll
> > > > >
> > > > run
> > > >
> > > > > the Tuscany itest suite in an OSGi environment, similar to the
> > profiles
> > > > >
> > > > we
> > > >
> > > > > have for Web containers for example)
> > > > >
> > > > > Here's what I imagined we'd do:
> > > > > 1. add OSGi entries to each of our JAR manifests
> > > > > 2. have developers maintain them and pay attention to
> imports/exports
> > > > > 3. use the OSGi build to detect API and SPI import/export
> violations
> > > > > 4. find the best way to OSGi-enable 3rd party dependency JARs
> > > > >
> > > > > I realize that my suggestion [1] is not very popular and most
> people
> > on
> > > > > this list would prefer to come up with bigger bundles grouping
> several
> > > > >
> > > > of
> > > >
> > > > > our JARs/modules. I don't think that the 'bigger aggregate bundle'
> > > > >
> > > > approach
> > > >
> > > > > will work, but I'll be happy to watch people try it :) if they
>  want
> > to.
> > > > >
> > > > > With respect to [4] I find rather funny to see many projects out
> there
> > > > > claim OSGi enablement without having OSGified their 3rd party
> > > > >
> > > > dependencies.
> > > >
> > > > > I wonder how that works, can an OSGi-enabled project really
> leverage
> > the
> > > > > OSGi classloader isolation and versioning capabilities when 99% of
> the
> > > > >
> > > > JARs
> > > >
> > > > > it requires are not OSGi bundles? I must be missing something...
> and I
> > > > >
> > > > hope
> > > >
> > > > > we can do better in Tuscany with a real end-to-end OSGi enablement
> > story
> > > > >
> > > > :)
> > > >
> > > > > --
> > > > > Jean-Sebastien
> > > > >
> > > > >
> > > >
> > > > I agree with Sebastien's suggestions1-4. But I would like to suggest
> a
> > > > slight variation to the first suggestion.
> > > >
> > > >
> > > >  1. Use maven-bundle-plugin to introduce OSGi manifest entries into
> all
> > > >  the Tuscany modules, with auto-generated import/export statements.
> > > > Modify
> > > >  itest/osgi-tuscany to run these modules under OSGi, with all 3rd
> party
> > > > jars
> > > >  installed into OSGi using virtual bundles created on the fly.
> > > >
> > > > This step will provide a version of osgi-tuscany tests that is less
> > prone
> > > > to
> > > > breakage than the one we have today. It will also help fix any
> remaining
> > > > classloading issues that we have left in Tuscany (and hopefully help
> > > > in maintaining the classloader isolation). This is not a big piece
> of
> > work
> > > > since this is just bringing together the different pieces that we
> > already
> > > > have. I will be happy to contribute the code towards this first
> step, so
> > > > others can concentrate on what we really want to achieve in terms of
> > > > modularity, distribution etc. We can also use this step to explore
> > > > versioning - in particular about having multiple extensions
> referring to
> > > > different versions of 3rd party libraries. This will be very useful
> for
> > 4.
> > > >
> > > > Suggestions 2-3 are not requirements for OSGi, but these are clearly
> > cases
> > > > where OSGi technology can help Tuscany improve modularity. If we
> want to
> > > > have explicitly hard-coded import/export statements in the modules
> to
> > > > enforce modularity, that can be introduced in step 2.
> > > >
> > > > And I would expect that there will be more work in terms of building
> the
> > > > distributions suitable for OSGi as well as non-OSGi after 1-4.
> > > >
> > > >
> > > >
> > > >
> > > > Thank you...
> > > >
> > > > Regards,
> > > >
> > > > Rajini
> > > >
> > > >
> > >
> > > Re. modularity, it strikes me that there is lack of consensus about
> > whether
> > > big bundles or small bundles, relative to our maven modules,  are
> > preferable
> > > but that there is consensus about using OSGi bundles to help us test
> and
> > > refine out modularity story.
> > >
> > > So maybe we should agree to disregard the question about how big OSGi
> > > bundles should be relative to our maven modules and concentrate on the
> > > question about whether our maven modules are correctly factored. Using
> > OSGi
> > > as a tool to help us with this of course.
> > >
> > > Simon
> > >
> > >
> >  I support both Rajini's and Simon's points.  We can get the mechanics
> going
> > first with the bundles following the existing grain of the modules - and
> we
> > can discuss and tinker with the modules as we please later to find the
> best
> > arrangements.
> >
> >  I suspect that the differing opinions about module size reflect
> different
> > usecases.  Lots of small modules can promote great flexibility in the
> > construction of runtimes, but at the expense of making it much harder
> for
> > users to make sense of the options.  Larger modules cut down the options
> but
> > make things easier for end users.
> >
> >
> >  Yours,  Mike.
> >
>

I would further propose that we take a subset of the output of 1 and 3, for
example, enough to run the calculator sample, as an initial test case
separate from the full range of itest/osgi-tuscany tests. This would give us
something to shoot re. inclusion in the regular build.

Simon

Re: Improving support for running in OSGi

Posted by Rajini Sivaram <ra...@googlemail.com>.
On 5/13/08, Jean-Sebastien Delfino <js...@apache.org> wrote:
>
> Rajini Sivaram wrote:
>
> >
> > Can't it just be much simpler than that?
> > > - 1 bundle per dependency JAR
> > > - containing the OSGi metadata describing that JAR and what it
> > > actually
> > > imports/exports?
> > >
> >
> >
> > Yes, that is the goal. But unfortunately this is not "simpler" - it
> > requires
> > some more work with the build. The code itself works with individual
> > meta-data or a collective one. The build at the moment is naive and
> > creates
> > a single manifest file.
> >
> >
> Maybe I can help, can you say what 'more work with the build' this will
> require?


I have modified the code to generate manifest entries for virtual bundles
and install 3rd party jars as individual bundles. These bundles export
everything in the jar, and import any required package dynamically.
itest/osgi-tuscany now installs 200 bundles into OSGi - the performance
impact on classloading is quite severe - explicit imports in 3rd party
bundles will help improve this, but I am not sure to what extent.

The work that still needs to be done is the build-time generation of
manifest entries for 3rd party bundles. Based on your comment below, should
we start a discussion on whether we can convert 3rd party jars into bundles,
rather than generate separate manifests? That would give us a cleaner
distribution (and make the build easier). We have been working on the
assumption that we cannot modify 3rd party jars, and hence the manifest
entries had to be generated and stored separately - hence the virtual
bundles.

 This is the approach that SpringSource for example seems to have chosen
> > > for they application platform OSGi repository. IMHO they are on the
> > > right
> > > path with this.
> > >
> >
> >
> > Well, this is not quite the approach SpringSource have taken.
> > SpringSource
> > repositories contain actual OSGi bundles (jars with OSGi manifest
> > entries
> > including export/import statements).
> >
>
> It is the approach that SpringSource has taken, or maybe I've not been
> clear... I meant 1 bundle per dependency JAR, i.e. that bundle is the JAR
> itself.
>
> From what I have seen and heard so far,
>
> > Tuscany seems very reluctant to take that step.
> >
>
> That's too bad, as it's the right approach IMO.
>
> --
> Jean-Sebastien
>



-- 
Thank you...

Regards,

Rajini

Re: Improving support for running in OSGi

Posted by Jean-Sebastien Delfino <js...@apache.org>.
Rajini Sivaram wrote:
> 
>> Can't it just be much simpler than that?
>> - 1 bundle per dependency JAR
>> - containing the OSGi metadata describing that JAR and what it actually
>> imports/exports?
> 
> 
> Yes, that is the goal. But unfortunately this is not "simpler" - it requires
> some more work with the build. The code itself works with individual
> meta-data or a collective one. The build at the moment is naive and creates
> a single manifest file.
> 

Maybe I can help, can you say what 'more work with the build' this will 
require?

> 
>> This is the approach that SpringSource for example seems to have chosen
>> for they application platform OSGi repository. IMHO they are on the right
>> path with this.
> 
> 
> Well, this is not quite the approach SpringSource have taken. SpringSource
> repositories contain actual OSGi bundles (jars with OSGi manifest entries
> including export/import statements).

It is the approach that SpringSource has taken, or maybe I've not been 
clear... I meant 1 bundle per dependency JAR, i.e. that bundle is the 
JAR itself.

 From what I have seen and heard so far,
> Tuscany seems very reluctant to take that step. 

That's too bad, as it's the right approach IMO.

-- 
Jean-Sebastien

Re: Improving support for running in OSGi

Posted by Rajini Sivaram <ra...@googlemail.com>.
On 5/12/08, Jean-Sebastien Delfino <js...@apache.org> wrote:
>
> Rajini Sivaram wrote:
>
> > At the moment, itest/osgi-tuscany generates a manifest jar file called
> > tuscany-sca-manifest.jar  using a copy of the pom in distribution. I was
> > hoping that we could use a single jar for both OSGi and non-OSGi. The
> > list
> > of virtual 3rd party bundles to be installed and the location of their
> > plain
> > jar files is based on the Class-Path entry in this
> > tuscany-sca-manifest.jar.
> > I do understand that this jar doesn't work in Eclipse, but I am not sure
> > what we gain by having an additional jar for OSGi rather than reuse the
> > jar
> > which is already in distribution. Are we planning to get rid of
> > tuscany-sca-manifest.jar in distribution?
> >
> >
> Here's what really puzzles me:
>
> a) We want to use OSGi as it allows us to cut Tuscany in self contained
> and isolated bundle JARs, enables us to pick the right subsets of JARs for a
> particular environment, can even enable us to combine different levels of
> dependency JARs when required by our extensions.
>
> b) But now all the info for all the dependency JARs is mashed in a single
> tuscany-sca-manifest.jar, and frozen in that 'manifest' JAR when we build
> the Tuscany distribution.
>
> Sorry, but I'm having a hard time understanding how to reconcile (a) and
> (b).


We are learning to walk (or in this case crawl), before starting to run.
(b) is a temporary step - it is the way it is purely because I work on
Tuscany in my (rather non-existent) free time. And it was meant to provide
something for Graham to get started on.



> Can't it just be much simpler than that?
> - 1 bundle per dependency JAR
> - containing the OSGi metadata describing that JAR and what it actually
> imports/exports?


Yes, that is the goal. But unfortunately this is not "simpler" - it requires
some more work with the build. The code itself works with individual
meta-data or a collective one. The build at the moment is naive and creates
a single manifest file.


> This is the approach that SpringSource for example seems to have chosen
> for they application platform OSGi repository. IMHO they are on the right
> path with this.


Well, this is not quite the approach SpringSource have taken. SpringSource
repositories contain actual OSGi bundles (jars with OSGi manifest entries
including export/import statements). From what I have seen and heard so far,
Tuscany seems very reluctant to take that step. We still seem to take the
view that OSGi is an add-on, something that we would like to use (maybe, but
not quite so sure yet). And that is part of the problem. OSGi tools are
geared for building OSGi bundles - maven-pax-plugin for instance would be
easy to use for converting our third party jars to bundles. It is slightly
harder (messier, but not impossible) to just generate the meta-data we need
for using virtual bundles.

And we still haven't addressed the issue of bundle dependencies - how do we
decide which bundles to install? Do we go for something like OBR in Felix,
or invent something else? SpringSource have their own implementation of
repositories. Having 100 separate 3rd party bundles (virtual or otherwise)
doesn't make sense until we also have a mechanism for automagically
determining dependencies across bundles and installing the required bundles.
We need additional meta-data apart from the manifest entries to enable this.
I do agree that all this needs to be done to get the full benefit of OSGi -
I just feel that it requires a lot more thought.



> --
> Jean-Sebastien
>



-- 
Thank you...

Regards,

Rajini

Re: Improving support for running in OSGi

Posted by Jean-Sebastien Delfino <js...@apache.org>.
Rajini Sivaram wrote:
> At the moment, itest/osgi-tuscany generates a manifest jar file called
> tuscany-sca-manifest.jar  using a copy of the pom in distribution. I was
> hoping that we could use a single jar for both OSGi and non-OSGi. The list
> of virtual 3rd party bundles to be installed and the location of their plain
> jar files is based on the Class-Path entry in this tuscany-sca-manifest.jar.
> I do understand that this jar doesn't work in Eclipse, but I am not sure
> what we gain by having an additional jar for OSGi rather than reuse the jar
> which is already in distribution. Are we planning to get rid of
> tuscany-sca-manifest.jar in distribution?
> 

Here's what really puzzles me:

a) We want to use OSGi as it allows us to cut Tuscany in self contained 
and isolated bundle JARs, enables us to pick the right subsets of JARs 
for a particular environment, can even enable us to combine different 
levels of dependency JARs when required by our extensions.

b) But now all the info for all the dependency JARs is mashed in a 
single tuscany-sca-manifest.jar, and frozen in that 'manifest' JAR when 
we build the Tuscany distribution.

Sorry, but I'm having a hard time understanding how to reconcile (a) and 
(b).

Can't it just be much simpler than that?
- 1 bundle per dependency JAR
- containing the OSGi metadata describing that JAR and what it actually 
imports/exports?

This is the approach that SpringSource for example seems to have chosen 
for they application platform OSGi repository. IMHO they are on the 
right path with this.
-- 
Jean-Sebastien

Re: Improving support for running in OSGi

Posted by Rajini Sivaram <ra...@googlemail.com>.
On 5/10/08, Jean-Sebastien Delfino <js...@apache.org> wrote:

> Rajini Sivaram wrote:
>
> > On 5/5/08, Jean-Sebastien Delfino <js...@apache.org> wrote:
> >
> > Rajini Sivaram wrote:
> > >
> > > At the moment, I am creating a single virtual 3rd party bundle. For
> > > > the
> > > > longer term, if there are use-cases where different Tuscany
> > > > extensions
> > > > require different versions of 3rd party libs, we may want to split
> > > > this
> > > > into
> > > > multiple virtual bundles.
> > > >
> > > >
> > > > How about having one virtual bundle per 3rd party JAR?
> > >
> > > Isn't that what we'll need to really leverage the OSGi classloader
> > > isolation and versioning capabilities?
> > >
> > > Isn't that also what these 3rd parties will do once they OSGi-enable
> > > their
> > > own JARs?
> > >
> >
> >
> >
> > Yes, you are absolutely right.
> >
> > I started with the assumption that to leverage versioning of 3rd party
> > libraries, we will need explicit versioned import/export statements in
> > all
> > 3rd party bundles. I preferred to generate these offline during the
> > build process, and the generated manifest file is added to
> > tuscany-sca-manifest.jar (not as a manifest, but as a plain resource).
> >
> > The code which generates virtual bundles at the moment looks for .mf
> > files
> > corresponding to the 3rd party jars. If a file (eg. stax-api-1.0-2.mf)
> > is
> > found in tuscany-sca-manifest.jar, a separate virtual bundle is
> > generated
> > for it (ie, stax-api-1.0-2 becomes a separate bundle). All the remaining
> > jars without specific .mf files and bundled together into a single
> > virtual
> > bundle.
> >
>
> OK, the idea of .mf looks good to me, but I can't really use
> tuscany-sca-manifest.jar as it doesn't work in Eclipse, and out of Eclipse
> it drags all the Tuscany JARs which I don't always need.
>
> Could you put the .mf files in another JAR that I can use in an OSGi
> environment?


At the moment, itest/osgi-tuscany generates a manifest jar file called
tuscany-sca-manifest.jar  using a copy of the pom in distribution. I was
hoping that we could use a single jar for both OSGi and non-OSGi. The list
of virtual 3rd party bundles to be installed and the location of their plain
jar files is based on the Class-Path entry in this tuscany-sca-manifest.jar.
I do understand that this jar doesn't work in Eclipse, but I am not sure
what we gain by having an additional jar for OSGi rather than reuse the jar
which is already in distribution. Are we planning to get rid of
tuscany-sca-manifest.jar in distribution?



> At the moment, the build generates only a single combined manifest entry,
> > and hence a single virtual bundle is used.
> >
>
> I'm confused, can you point me to that single manifest entry and the code
> or build that generates it?


The code is in itest/osgi-tuscany. It is not part of the main build at the
moment. It still contains the older 5-bundle version, so the directory
structure may be a bit confusing. It is not a particularly efficient build -
I have just done enough to get Tuscany running under OSGi with bundle-ized
modules and virtual 3rd party libs. The build itself could be tidied up a
lot.


   - tuscany-3rdparty-manifest - The 3rd party manifest entries for OSGi
   (generates org/apache/tuscany/manifest/MANIFEST.MF)
   - tuscany-manifest  - Generates  tuscany-sca-manifest.jar (similar to
   the manifest jar in distribution, but in addition to its standard manifest,
   this jar contains a bundle activator that installs 3rd party libs as virtual
   bundles, and the manifest file above).
   - test-bundles - Generates two bundle-ized tests (one simple
   implementation.java sample, and a second webservice sample) for sniff
   testing Tuscany under OSGi
   - osgi-tuscany-test - Testcases for OSGi-based Tuscany (runs the two
   sniff tests from test-bundles, and a bunch of samples directly from the
   samples build)


The following correspond to the old 5-bundle version of Tuscany that are not
used anymore.

   - sca-api
   - tuscany-spi
   - tuscany-runtime
   - tuscany-extensions
   - tuscany-3rdparty




> Thanks!
> --
> Jean-Sebastien
>



-- 
Thank you...

Regards,

Rajini

Re: Improving support for running in OSGi

Posted by Jean-Sebastien Delfino <js...@apache.org>.
Rajini Sivaram wrote:
> On 5/5/08, Jean-Sebastien Delfino <js...@apache.org> wrote:
> 
>> Rajini Sivaram wrote:
>>
>>> At the moment, I am creating a single virtual 3rd party bundle. For the
>>> longer term, if there are use-cases where different Tuscany extensions
>>> require different versions of 3rd party libs, we may want to split this
>>> into
>>> multiple virtual bundles.
>>>
>>>
>> How about having one virtual bundle per 3rd party JAR?
>>
>> Isn't that what we'll need to really leverage the OSGi classloader
>> isolation and versioning capabilities?
>>
>> Isn't that also what these 3rd parties will do once they OSGi-enable their
>> own JARs?
> 
> 
> 
> Yes, you are absolutely right.
> 
> I started with the assumption that to leverage versioning of 3rd party
> libraries, we will need explicit versioned import/export statements in all
> 3rd party bundles. I preferred to generate these offline during the
> build process, and the generated manifest file is added to
> tuscany-sca-manifest.jar (not as a manifest, but as a plain resource).
> 
> The code which generates virtual bundles at the moment looks for .mf files
> corresponding to the 3rd party jars. If a file (eg. stax-api-1.0-2.mf) is
> found in tuscany-sca-manifest.jar, a separate virtual bundle is generated
> for it (ie, stax-api-1.0-2 becomes a separate bundle). All the remaining
> jars without specific .mf files and bundled together into a single virtual
> bundle.

OK, the idea of .mf looks good to me, but I can't really use 
tuscany-sca-manifest.jar as it doesn't work in Eclipse, and out of 
Eclipse it drags all the Tuscany JARs which I don't always need.

Could you put the .mf files in another JAR that I can use in an OSGi 
environment?


> At the moment, the build generates only a single combined manifest entry,
> and hence a single virtual bundle is used.

I'm confused, can you point me to that single manifest entry and the 
code or build that generates it?

Thanks!
-- 
Jean-Sebastien

Re: Improving support for running in OSGi

Posted by Rajini Sivaram <ra...@googlemail.com>.
On 5/5/08, Jean-Sebastien Delfino <js...@apache.org> wrote:

> Rajini Sivaram wrote:
>
> >
> > At the moment, I am creating a single virtual 3rd party bundle. For the
> > longer term, if there are use-cases where different Tuscany extensions
> > require different versions of 3rd party libs, we may want to split this
> > into
> > multiple virtual bundles.
> >
> >
> How about having one virtual bundle per 3rd party JAR?
>
> Isn't that what we'll need to really leverage the OSGi classloader
> isolation and versioning capabilities?
>
> Isn't that also what these 3rd parties will do once they OSGi-enable their
> own JARs?



Yes, you are absolutely right.

I started with the assumption that to leverage versioning of 3rd party
libraries, we will need explicit versioned import/export statements in all
3rd party bundles. I preferred to generate these offline during the
build process, and the generated manifest file is added to
tuscany-sca-manifest.jar (not as a manifest, but as a plain resource).

The code which generates virtual bundles at the moment looks for .mf files
corresponding to the 3rd party jars. If a file (eg. stax-api-1.0-2.mf) is
found in tuscany-sca-manifest.jar, a separate virtual bundle is generated
for it (ie, stax-api-1.0-2 becomes a separate bundle). All the remaining
jars without specific .mf files and bundled together into a single virtual
bundle.

At the moment, the build generates only a single combined manifest entry,
and hence a single virtual bundle is used. This is mainly because I couldn't
figure out an easy way to generate separate manifest entries using a single
pom and maven-bundle-plugin (but then my knowledge of maven is very very
limited).

In the short term, just sorting out the build to generate individual
manifest entries for 3rd party jars will give us something to explore
versioning of 3rd party libs, and side-by-side execution of these in
Tuscany.

For the longer term, we need to look at the best way to generate manifest
files for 3rd party libs. Should we for instance do this at runtime, rather
than build-time? Could we generate simpler export/dynamic-import statements
at runtime without using maven-bundle-plugin, and still support versioning
of third party libs? What is the performance impact of individual virtual
bundles on classloading? At the moment, we haven't done any analysis of who
requires what on TCCL. So all 200 bundles will be on TCCL (making TCCL based
classloading rather inefficient).



> --
> Jean-Sebastien
>



-- 
Thank you...

Regards,

Rajini

Re: Improving support for running in OSGi

Posted by Jean-Sebastien Delfino <js...@apache.org>.
Rajini Sivaram wrote:
> 
> At the moment, I am creating a single virtual 3rd party bundle. For the
> longer term, if there are use-cases where different Tuscany extensions
> require different versions of 3rd party libs, we may want to split this into
> multiple virtual bundles.
> 

How about having one virtual bundle per 3rd party JAR?

Isn't that what we'll need to really leverage the OSGi classloader 
isolation and versioning capabilities?

Isn't that also what these 3rd parties will do once they OSGi-enable 
their own JARs?

-- 
Jean-Sebastien

Re: Improving support for running in OSGi

Posted by Rajini Sivaram <ra...@googlemail.com>.
 I have a new working itest/osgi-tuscany with the following:

   1. modules/ are built with OSGi manifest headers using
   maven-bundle-plugin
   2. For 3rd party jars, a manifest jar is created, similar to
   tuscany-sca-manifest.jar from the distribution. It contains the standard
   manifest file (with Jar classpath), a separate OSGi manifest file with
   import/exports for 3rd party libs created using maven-bundle-plugin, and a
   bundle activator. The bundle activator creates a virtual bundle for the
   third party jars using the Jar classpath and installs it into OSGi.
   3. itest/osgi-tuscany loads Tuscany modules from the modules build.
   (it no longer creates the 5 combined bundles). The manifest jar is generated
   by itest/osgi-tuscany, which also copies all 3rd party libs (so it looks
   identical to a regular Tuscany distribution). It is not particularly
   efficient, but it does the job.

By adding a bundle activator to tuscany-sca-manifest.jar which can install
Tuscany into OSGi, when the manifest bundle is installed into OSGi, we can
have a Tuscany distribution which installs itself into OSGi straight
out-of-the-box, without doubling the size of the distribution.  Users who
don't like to have virtual 3rd party bundles can generate a concrete OSGi
bundle for the third party jars with a simple jar command using the 3rd
party manifest file stored in tuscany-sca-manifest.jar.

If Tuscany distribution is split into multiple smaller distributions, I
assume we will continue to have a manifest.jar for each part, and the same
bundle activator can be inserted into each manifest.jar to cause that part
of the distribution to be self-installing for OSGi.

At the moment, I am creating a single virtual 3rd party bundle. For the
longer term, if there are use-cases where different Tuscany extensions
require different versions of 3rd party libs, we may want to split this into
multiple virtual bundles.

There are a couple of issues that I raised JIRAs for, and once I get
responses to those, I can check the code into SVN if there are are no
objections. With these changes, you should have enough to look at modularity
in Tuscany and decide how to move that forward.



On 5/2/08, Graham Charters <gc...@googlemail.com> wrote:

> Hi Rajini,
>
> FWIW, I agree with starting with 1.  I missed your earlier note saying
> you'd be happy to contribute code to do this.  Please let me know what
> I can do to help.  I'm particularly interested in the 'virtual bundle'
> idea.  I took a look at the way the samples are run in
> itest/osgi-tuscany and now I have a better understanding, I agree,
> it's not as much work as I first thought (fear of the unknown ;-) ).
> I think the challenge will be in finding the right design to enable it
> for third-party libraries so it's usable and flexible enough.  Perhaps
> you already have thoughts on this?
>
> I was a little surprised you didn't suggest 3 as a stepping stone
> towards 4 (I guess the '"half-hearted" comment was a bit of a
> give-away ;-) ).  I agree it's not ideal from a modularity perspective
> but it does have the advantage that I think it could be introduced a
> lot sooner than 4 and would therefore accelerate sensitizing folks to
> the issues.
>
> Regards, Graham.
>
> 2008/5/2 Rajini Sivaram <ra...@googlemail.com>:
> > On 5/1/08, Jean-Sebastien Delfino <js...@apache.org> wrote:
> >
> >
> >
> > > Graham Charters wrote:
> >  >
> >  > > It would seem that the fine-grained/coarse-grained thoughts have
> >  > > people divided.  Rajini's note (aside from the fact she has a tonne
> of
> >  > > experience having done most, if not all, of the OSGi work in
> Tuscany)
> >  > > paints a picture where the two are not mutually exclusive.  I don't
> >  > > typically like doing two options because that seems like
> indecision,
> >  > > but in this case they do appear to complement each other.
> >  > >
> >  > > Based on what I've seen in this thread, I'm hoping this briefly
> >  > > summarizes the collection of thoughts on how we might proceed:
> >  > >
> >  > > Tuscany Running in OSGi
> >  > >
> >  > > 1.  Add bundle manifests to all the Tuscany modules (using maven
> >  > > bundle plugin).  This will ensure the most fine-grained
> decomposition
> >  > > works and therefore coarser-grained distributions will also work.
> >  > > 2.  Add a distribution which creates bundles around manageable
> >  > > collections of Tuscany modules aligned with how we see the runtime
> >  > > being extended/subsetted/replaced.  Have this build and test on
> >  > > Continuum.
> >  > > 3.  Create 'virtual bundles' for third-party libraries to avoid
> >  > > licensing and disk space issues (based on Rajini's suggestions,
> which
> >  > > I need to better understand).
> >  > > 4.  Provide guidance on the wiki on how to avoid breaking OSGi.
> >  > >
> >  > > There's also the suggestion that implementation.osgi relate to
> >  > > Distributed OSGi (RFC 119), which shouldn't be lost.
> >  > >
> >  > > Have I missed anything?
> >  > >
> >  > > It sounds like a staging of 1 & 3 first, followed by 2 would work
> (and
> >  > > 4 if things keep breaking :-( ).  I'd be concerned if we didn't get
> to
> >  > > 2, and as part of 1&3, we need to make sure these are regularly
> tested
> >  > > under osgi.
> >  > >
> >  > >
> >  > Sounds good to me, with the following comments:
> >  >
> >  > When you have (1) + (3) you can already run a continuum OSGi build,
> before
> >  > attacking (2). I'd actually suggest to start building on continuum as
> soon
> >  > as we have (1).
> >  >
> >  > - I'm not clear on what you meant by 'use the Maven bundle plugin'.
> Will
> >  > we write something like this for example:
> >  > <project>
> >  >  ...
> >  >  <packaging>bundle</packaging>
> >  >  ...
> >  >  <plugin>
> >  >    <groupId>org.apache.felix</groupId>
> >  >    <artifactId>maven-bundle-plugin</artifactId>
> >  >    <extensions>true</extensions>
> >  >    <configuration>
> >  >      <instructions>
> >  >        <Export-Package>org.apache.tuscany.sca.foo</Export-Package>
> >  >        <Import-Package>org.apache.tuscany.sca.bar</Import-Package>
>
> >  >        <Private-Package>org.apache.tuscany.sca.impl.*</Private-Package>
> >  >      </instructions>
> >  >    </configuration>
> >  >  </plugin>
> >  > ...
> >
> >
> >
> >  I think we have four options on how we start bundle-izing Tuscany
> modules,
> >  using maven-bundle-plugin.
> >
> >
> >  1. Use the manifest goal to generate the manifest entry in all module
> jars,
> >  with <Export-Package>*</Export-Package> and
> >  <Import-Package>*</Import-Package>, which export all the packages from
> the
> >  module and import everything used by the module. The Export-Package and
> >  Import-Package statements in the generated manifest will use explicit
> >  package import/exports like:
> >
> >  Import-Package:
>
> >  javax.xml.namespace,javax.xml.parsers,org.apache.tuscany.sca.contribution.resolver,...
> >  Export-Package:
>
> >  org.apache.tuscany.sca.implementation.osgi;version="2.0";uses:="o.a.t.s.assembly",/...
> >
> >
> >  The analysis of Import/Export will be done by the maven-bundle-plugin
> in
> >  this case. The actual generation of the module jar will remain the same
> as
> >  now, the only addition will be a new generated manifest file.
> >
> >  We might not use <Export-Package>*</Export-Package> , but instead it
> would
> >  be something more like
>
> >  <Export-Package>org.apache.tuscany.sca.implementation.osgi.*</Export-Package>,
> >  where the packages are wildcarded, but no assumptions about modularity
> are
> >  made.
> >
> >  2. Use bundle goal with Import/Export as above (generated by
> >  maven-bundle-plugin rather than explicitly specified). In this case the
> jar
> >  itself will be generated using maven-bundle-plugin. The output jar
> should be
> >  identical to 1.
> >
> >  3. Use manifest goal with explicitly listed <Export-Package/>,
> >  <Import-Package/> as in Sebastien's example. All exported and (maybe)
> >  imported packages  are explicitly specified in the pom. Everytime a
> >  new package is added, the pom should be updated.
> >
> >  4.  Use bundle goal with explicitly listed <Export-Package/>,
> >  <Import-Package/> and <Private-Package/> as in Sebastien's example.
> This
> >  option corresponds to Sebastien's example. In this case both the
> contents of
> >  the jar, as well as the manifest are based on the explicitly listed
> >  packages.
> >
> >
> >  1. is the easiest to implement. It makes no assumptions about
> modularity,
> >  and will not have any impact on non-OSGi applications since only the
> >  manifest entry is affected. From an OSGi point of view, this gives all
> we
> >  need for osgi-tuscany.
> >
> >  2. does not add any value over 1.
> >
> >  3. This is a half-hearted attempt to enforce modularity. osgi-tuscany
> will
> >  break everytime someone introduces a new package  and doesn't update
> the
> >  pom. But Tuscany will continue to operate without OSGi since only the
> >  manifest is broken. 3. is not a requirement for OSGi-based Tuscany, but
> >  merely the use of OSGi technology to improve modularity. osgi-tuscany
> will
> >  take on the role on policing modularity(apart from testing that Tuscany
> can
> >  be run in an OSGi runtime).
> >
> >  4. This is OSGi best practice. If a new package is introduced and the
> pom is
> >  not updated, the jar itself will be unusable. Basically this requires
> all
> >  developers to be OSGi-aware. The value-add of this option over 1. is
> the
> >  enforcement of modularity, and like 3, this is not a requirement for
> >  OSGi-based Tuscany. I am not sure we are ready for this yet. I think
> there
> >  is a real risk that frequent breakages will create the perception that
> OSGi
> >  integration is impossible to maintain, even though this is more to do
> with
> >  enforcing modularity rather than OSGi.
> >
> >
> >  I would like to suggest that we start with 1., use the generated
> manifest
> >  files to understand and improve modularity and jump to 4. to enforce
> >  modularity when we are ready for it.
> >
> >
> >
> >
> >
> >
> >  > or did you mean something else?
> >  >
> >  > --
> >  > Jean-Sebastien
> >  >
> >
> >
> >
> >  --
> >  Thank you...
> >
> >  Regards,
> >
> >  Rajini
> >
>



-- 
Thank you...

Regards,

Rajini

Re: Improving support for running in OSGi

Posted by Rajini Sivaram <ra...@googlemail.com>.
On 5/2/08, Graham Charters <gc...@googlemail.com> wrote:
>
> Hi Rajini,
>
> FWIW, I agree with starting with 1.  I missed your earlier note saying
> you'd be happy to contribute code to do this.  Please let me know what
> I can do to help.  I'm particularly interested in the 'virtual bundle'
> idea.  I took a look at the way the samples are run in
> itest/osgi-tuscany and now I have a better understanding, I agree,
> it's not as much work as I first thought (fear of the unknown ;-) ).
> I think the challenge will be in finding the right design to enable it
> for third-party libraries so it's usable and flexible enough.  Perhaps
> you already have thoughts on this?



I thought it would be very unfair of me to suggest that this is not a very
big task, when you were the one who had to go and implement it. But now that
you agree that it is not too much work, can I change my mind? Seriously
though, I could help with getting something up and running quickly, and you
could take over from there (I will continue to help if you run into
problems).

Let me take a look this weekend and see if I can get a quick build with OSGi
manifest entries for the modules. There are some modules which require
additional options like Require-Bundle for the split packages. Since I have
done this once before (even though that was with different plugins) for
running Tuscany using OBR, it may be quicker for me to add these (and you
can refine them later). I will also try to update osgi-tuscany to use these
with some basic support for virtual 3rd party bundles copied over from the
existing code and see if they run into any problems. You could then take
over and drive it forward, initially fine tuning the OSGi support
(finalizing the design for 3rd party libs as well as reducing disk space for
tests, getting them to run on Continuum etc) followed by improving
modularity. I suspect modularity requires a lot more discussion since it
affects everyone. You might want to start a new thread on the mailing list
without including "OSGi" in
the subject :-).

I was a little surprised you didn't suggest 3 as a stepping stone
> towards 4 (I guess the '"half-hearted" comment was a bit of a
> give-away ;-) ).  I agree it's not ideal from a modularity perspective
> but it does have the advantage that I think it could be introduced a
> lot sooner than 4 and would therefore accelerate sensitizing folks to
> the issues.



My problem with option 3. is purely psychological. It seems rather strange
to have centralized control of modularity :-). And paranoia - who is going
to maintain a continually breaking itest/osgi-tuscany? Probably you,
still...

I do completely agree that 3. will help us get there faster. But it could
reduce the motivation for 4, and throw away the opportunity to involve the
entire Tuscany community to drive the improvements in modularity.

There was a purpose to writing itest/osgi-tuscany - and that was to ensure
that Tuscany could run inside an OSGi runtime. It already has the
responsibility to test classloader usage in Tuscany, other issues like URL
handling, which used to break under OSGi, and OSGi-specific problems in
Tuscany. It might collapse under pressure if also burdened with defining and
maintaining Tuscany modularity.

But honestly, I think you should choose whichever path makes it easiest for
you to drive the modularity work.


Regards, Graham.



2008/5/2 Rajini Sivaram <ra...@googlemail.com>:
> > On 5/1/08, Jean-Sebastien Delfino <js...@apache.org> wrote:
> >
> >
> >
> > > Graham Charters wrote:
> >  >
> >  > > It would seem that the fine-grained/coarse-grained thoughts have
> >  > > people divided.  Rajini's note (aside from the fact she has a tonne
> of
> >  > > experience having done most, if not all, of the OSGi work in
> Tuscany)
> >  > > paints a picture where the two are not mutually exclusive.  I don't
> >  > > typically like doing two options because that seems like
> indecision,
> >  > > but in this case they do appear to complement each other.
> >  > >
> >  > > Based on what I've seen in this thread, I'm hoping this briefly
> >  > > summarizes the collection of thoughts on how we might proceed:
> >  > >
> >  > > Tuscany Running in OSGi
> >  > >
> >  > > 1.  Add bundle manifests to all the Tuscany modules (using maven
> >  > > bundle plugin).  This will ensure the most fine-grained
> decomposition
> >  > > works and therefore coarser-grained distributions will also work.
> >  > > 2.  Add a distribution which creates bundles around manageable
> >  > > collections of Tuscany modules aligned with how we see the runtime
> >  > > being extended/subsetted/replaced.  Have this build and test on
> >  > > Continuum.
> >  > > 3.  Create 'virtual bundles' for third-party libraries to avoid
> >  > > licensing and disk space issues (based on Rajini's suggestions,
> which
> >  > > I need to better understand).
> >  > > 4.  Provide guidance on the wiki on how to avoid breaking OSGi.
> >  > >
> >  > > There's also the suggestion that implementation.osgi relate to
> >  > > Distributed OSGi (RFC 119), which shouldn't be lost.
> >  > >
> >  > > Have I missed anything?
> >  > >
> >  > > It sounds like a staging of 1 & 3 first, followed by 2 would work
> (and
> >  > > 4 if things keep breaking :-( ).  I'd be concerned if we didn't get
> to
> >  > > 2, and as part of 1&3, we need to make sure these are regularly
> tested
> >  > > under osgi.
> >  > >
> >  > >
> >  > Sounds good to me, with the following comments:
> >  >
> >  > When you have (1) + (3) you can already run a continuum OSGi build,
> before
> >  > attacking (2). I'd actually suggest to start building on continuum as
> soon
> >  > as we have (1).
> >  >
> >  > - I'm not clear on what you meant by 'use the Maven bundle plugin'.
> Will
> >  > we write something like this for example:
> >  > <project>
> >  >  ...
> >  >  <packaging>bundle</packaging>
> >  >  ...
> >  >  <plugin>
> >  >    <groupId>org.apache.felix</groupId>
> >  >    <artifactId>maven-bundle-plugin</artifactId>
> >  >    <extensions>true</extensions>
> >  >    <configuration>
> >  >      <instructions>
> >  >        <Export-Package>org.apache.tuscany.sca.foo</Export-Package>
> >  >        <Import-Package>org.apache.tuscany.sca.bar</Import-Package>
>
> >  >        <Private-Package>org.apache.tuscany.sca.impl.*</Private-Package>
> >  >      </instructions>
> >  >    </configuration>
> >  >  </plugin>
> >  > ...
> >
> >
> >
> >  I think we have four options on how we start bundle-izing Tuscany
> modules,
> >  using maven-bundle-plugin.
> >
> >
> >  1. Use the manifest goal to generate the manifest entry in all module
> jars,
> >  with <Export-Package>*</Export-Package> and
> >  <Import-Package>*</Import-Package>, which export all the packages from
> the
> >  module and import everything used by the module. The Export-Package and
> >  Import-Package statements in the generated manifest will use explicit
> >  package import/exports like:
> >
> >  Import-Package:
>
> >  javax.xml.namespace,javax.xml.parsers,org.apache.tuscany.sca.contribution.resolver,...
> >  Export-Package:
>
> >  org.apache.tuscany.sca.implementation.osgi;version="2.0";uses:="o.a.t.s.assembly",/...
> >
> >
> >  The analysis of Import/Export will be done by the maven-bundle-plugin
> in
> >  this case. The actual generation of the module jar will remain the same
> as
> >  now, the only addition will be a new generated manifest file.
> >
> >  We might not use <Export-Package>*</Export-Package> , but instead it
> would
> >  be something more like
>
> >  <Export-Package>org.apache.tuscany.sca.implementation.osgi.*</Export-Package>,
> >  where the packages are wildcarded, but no assumptions about modularity
> are
> >  made.
> >
> >  2. Use bundle goal with Import/Export as above (generated by
> >  maven-bundle-plugin rather than explicitly specified). In this case the
> jar
> >  itself will be generated using maven-bundle-plugin. The output jar
> should be
> >  identical to 1.
> >
> >  3. Use manifest goal with explicitly listed <Export-Package/>,
> >  <Import-Package/> as in Sebastien's example. All exported and (maybe)
> >  imported packages  are explicitly specified in the pom. Everytime a
> >  new package is added, the pom should be updated.
> >
> >  4.  Use bundle goal with explicitly listed <Export-Package/>,
> >  <Import-Package/> and <Private-Package/> as in Sebastien's example.
> This
> >  option corresponds to Sebastien's example. In this case both the
> contents of
> >  the jar, as well as the manifest are based on the explicitly listed
> >  packages.
> >
> >
> >  1. is the easiest to implement. It makes no assumptions about
> modularity,
> >  and will not have any impact on non-OSGi applications since only the
> >  manifest entry is affected. From an OSGi point of view, this gives all
> we
> >  need for osgi-tuscany.
> >
> >  2. does not add any value over 1.
> >
> >  3. This is a half-hearted attempt to enforce modularity. osgi-tuscany
> will
> >  break everytime someone introduces a new package  and doesn't update
> the
> >  pom. But Tuscany will continue to operate without OSGi since only the
> >  manifest is broken. 3. is not a requirement for OSGi-based Tuscany, but
> >  merely the use of OSGi technology to improve modularity. osgi-tuscany
> will
> >  take on the role on policing modularity(apart from testing that Tuscany
> can
> >  be run in an OSGi runtime).
> >
> >  4. This is OSGi best practice. If a new package is introduced and the
> pom is
> >  not updated, the jar itself will be unusable. Basically this requires
> all
> >  developers to be OSGi-aware. The value-add of this option over 1. is
> the
> >  enforcement of modularity, and like 3, this is not a requirement for
> >  OSGi-based Tuscany. I am not sure we are ready for this yet. I think
> there
> >  is a real risk that frequent breakages will create the perception that
> OSGi
> >  integration is impossible to maintain, even though this is more to do
> with
> >  enforcing modularity rather than OSGi.
> >
> >
> >  I would like to suggest that we start with 1., use the generated
> manifest
> >  files to understand and improve modularity and jump to 4. to enforce
> >  modularity when we are ready for it.
> >
> >
> >
> >
> >
> >
> >  > or did you mean something else?
> >  >
> >  > --
> >  > Jean-Sebastien
> >  >
> >
> >
> >
> >  --
> >  Thank you...
> >
> >  Regards,
> >
> >  Rajini
> >
>



-- 
Thank you...

Regards,

Rajini

Re: Improving support for running in OSGi

Posted by Graham Charters <gc...@googlemail.com>.
Hi Rajini,

FWIW, I agree with starting with 1.  I missed your earlier note saying
you'd be happy to contribute code to do this.  Please let me know what
I can do to help.  I'm particularly interested in the 'virtual bundle'
idea.  I took a look at the way the samples are run in
itest/osgi-tuscany and now I have a better understanding, I agree,
it's not as much work as I first thought (fear of the unknown ;-) ).
I think the challenge will be in finding the right design to enable it
for third-party libraries so it's usable and flexible enough.  Perhaps
you already have thoughts on this?

I was a little surprised you didn't suggest 3 as a stepping stone
towards 4 (I guess the '"half-hearted" comment was a bit of a
give-away ;-) ).  I agree it's not ideal from a modularity perspective
but it does have the advantage that I think it could be introduced a
lot sooner than 4 and would therefore accelerate sensitizing folks to
the issues.

Regards, Graham.

2008/5/2 Rajini Sivaram <ra...@googlemail.com>:
> On 5/1/08, Jean-Sebastien Delfino <js...@apache.org> wrote:
>
>
>
> > Graham Charters wrote:
>  >
>  > > It would seem that the fine-grained/coarse-grained thoughts have
>  > > people divided.  Rajini's note (aside from the fact she has a tonne of
>  > > experience having done most, if not all, of the OSGi work in Tuscany)
>  > > paints a picture where the two are not mutually exclusive.  I don't
>  > > typically like doing two options because that seems like indecision,
>  > > but in this case they do appear to complement each other.
>  > >
>  > > Based on what I've seen in this thread, I'm hoping this briefly
>  > > summarizes the collection of thoughts on how we might proceed:
>  > >
>  > > Tuscany Running in OSGi
>  > >
>  > > 1.  Add bundle manifests to all the Tuscany modules (using maven
>  > > bundle plugin).  This will ensure the most fine-grained decomposition
>  > > works and therefore coarser-grained distributions will also work.
>  > > 2.  Add a distribution which creates bundles around manageable
>  > > collections of Tuscany modules aligned with how we see the runtime
>  > > being extended/subsetted/replaced.  Have this build and test on
>  > > Continuum.
>  > > 3.  Create 'virtual bundles' for third-party libraries to avoid
>  > > licensing and disk space issues (based on Rajini's suggestions, which
>  > > I need to better understand).
>  > > 4.  Provide guidance on the wiki on how to avoid breaking OSGi.
>  > >
>  > > There's also the suggestion that implementation.osgi relate to
>  > > Distributed OSGi (RFC 119), which shouldn't be lost.
>  > >
>  > > Have I missed anything?
>  > >
>  > > It sounds like a staging of 1 & 3 first, followed by 2 would work (and
>  > > 4 if things keep breaking :-( ).  I'd be concerned if we didn't get to
>  > > 2, and as part of 1&3, we need to make sure these are regularly tested
>  > > under osgi.
>  > >
>  > >
>  > Sounds good to me, with the following comments:
>  >
>  > When you have (1) + (3) you can already run a continuum OSGi build, before
>  > attacking (2). I'd actually suggest to start building on continuum as soon
>  > as we have (1).
>  >
>  > - I'm not clear on what you meant by 'use the Maven bundle plugin'. Will
>  > we write something like this for example:
>  > <project>
>  >  ...
>  >  <packaging>bundle</packaging>
>  >  ...
>  >  <plugin>
>  >    <groupId>org.apache.felix</groupId>
>  >    <artifactId>maven-bundle-plugin</artifactId>
>  >    <extensions>true</extensions>
>  >    <configuration>
>  >      <instructions>
>  >        <Export-Package>org.apache.tuscany.sca.foo</Export-Package>
>  >        <Import-Package>org.apache.tuscany.sca.bar</Import-Package>
>  >        <Private-Package>org.apache.tuscany.sca.impl.*</Private-Package>
>  >      </instructions>
>  >    </configuration>
>  >  </plugin>
>  > ...
>
>
>
>  I think we have four options on how we start bundle-izing Tuscany modules,
>  using maven-bundle-plugin.
>
>
>  1. Use the manifest goal to generate the manifest entry in all module jars,
>  with <Export-Package>*</Export-Package> and
>  <Import-Package>*</Import-Package>, which export all the packages from the
>  module and import everything used by the module. The Export-Package and
>  Import-Package statements in the generated manifest will use explicit
>  package import/exports like:
>
>  Import-Package:
>  javax.xml.namespace,javax.xml.parsers,org.apache.tuscany.sca.contribution.resolver,...
>  Export-Package:
>  org.apache.tuscany.sca.implementation.osgi;version="2.0";uses:="o.a.t.s.assembly",/...
>
>
>  The analysis of Import/Export will be done by the maven-bundle-plugin in
>  this case. The actual generation of the module jar will remain the same as
>  now, the only addition will be a new generated manifest file.
>
>  We might not use <Export-Package>*</Export-Package> , but instead it would
>  be something more like
>  <Export-Package>org.apache.tuscany.sca.implementation.osgi.*</Export-Package>,
>  where the packages are wildcarded, but no assumptions about modularity are
>  made.
>
>  2. Use bundle goal with Import/Export as above (generated by
>  maven-bundle-plugin rather than explicitly specified). In this case the jar
>  itself will be generated using maven-bundle-plugin. The output jar should be
>  identical to 1.
>
>  3. Use manifest goal with explicitly listed <Export-Package/>,
>  <Import-Package/> as in Sebastien's example. All exported and (maybe)
>  imported packages  are explicitly specified in the pom. Everytime a
>  new package is added, the pom should be updated.
>
>  4.  Use bundle goal with explicitly listed <Export-Package/>,
>  <Import-Package/> and <Private-Package/> as in Sebastien's example. This
>  option corresponds to Sebastien's example. In this case both the contents of
>  the jar, as well as the manifest are based on the explicitly listed
>  packages.
>
>
>  1. is the easiest to implement. It makes no assumptions about modularity,
>  and will not have any impact on non-OSGi applications since only the
>  manifest entry is affected. From an OSGi point of view, this gives all we
>  need for osgi-tuscany.
>
>  2. does not add any value over 1.
>
>  3. This is a half-hearted attempt to enforce modularity. osgi-tuscany will
>  break everytime someone introduces a new package  and doesn't update the
>  pom. But Tuscany will continue to operate without OSGi since only the
>  manifest is broken. 3. is not a requirement for OSGi-based Tuscany, but
>  merely the use of OSGi technology to improve modularity. osgi-tuscany will
>  take on the role on policing modularity(apart from testing that Tuscany can
>  be run in an OSGi runtime).
>
>  4. This is OSGi best practice. If a new package is introduced and the pom is
>  not updated, the jar itself will be unusable. Basically this requires all
>  developers to be OSGi-aware. The value-add of this option over 1. is the
>  enforcement of modularity, and like 3, this is not a requirement for
>  OSGi-based Tuscany. I am not sure we are ready for this yet. I think there
>  is a real risk that frequent breakages will create the perception that OSGi
>  integration is impossible to maintain, even though this is more to do with
>  enforcing modularity rather than OSGi.
>
>
>  I would like to suggest that we start with 1., use the generated manifest
>  files to understand and improve modularity and jump to 4. to enforce
>  modularity when we are ready for it.
>
>
>
>
>
>
>  > or did you mean something else?
>  >
>  > --
>  > Jean-Sebastien
>  >
>
>
>
>  --
>  Thank you...
>
>  Regards,
>
>  Rajini
>

Re: Improving support for running in OSGi

Posted by Rajini Sivaram <ra...@googlemail.com>.
On 5/1/08, Jean-Sebastien Delfino <js...@apache.org> wrote:

> Graham Charters wrote:
>
> > It would seem that the fine-grained/coarse-grained thoughts have
> > people divided.  Rajini's note (aside from the fact she has a tonne of
> > experience having done most, if not all, of the OSGi work in Tuscany)
> > paints a picture where the two are not mutually exclusive.  I don't
> > typically like doing two options because that seems like indecision,
> > but in this case they do appear to complement each other.
> >
> > Based on what I've seen in this thread, I'm hoping this briefly
> > summarizes the collection of thoughts on how we might proceed:
> >
> > Tuscany Running in OSGi
> >
> > 1.  Add bundle manifests to all the Tuscany modules (using maven
> > bundle plugin).  This will ensure the most fine-grained decomposition
> > works and therefore coarser-grained distributions will also work.
> > 2.  Add a distribution which creates bundles around manageable
> > collections of Tuscany modules aligned with how we see the runtime
> > being extended/subsetted/replaced.  Have this build and test on
> > Continuum.
> > 3.  Create 'virtual bundles' for third-party libraries to avoid
> > licensing and disk space issues (based on Rajini's suggestions, which
> > I need to better understand).
> > 4.  Provide guidance on the wiki on how to avoid breaking OSGi.
> >
> > There's also the suggestion that implementation.osgi relate to
> > Distributed OSGi (RFC 119), which shouldn't be lost.
> >
> > Have I missed anything?
> >
> > It sounds like a staging of 1 & 3 first, followed by 2 would work (and
> > 4 if things keep breaking :-( ).  I'd be concerned if we didn't get to
> > 2, and as part of 1&3, we need to make sure these are regularly tested
> > under osgi.
> >
> >
> Sounds good to me, with the following comments:
>
> When you have (1) + (3) you can already run a continuum OSGi build, before
> attacking (2). I'd actually suggest to start building on continuum as soon
> as we have (1).
>
> - I'm not clear on what you meant by 'use the Maven bundle plugin'. Will
> we write something like this for example:
> <project>
>  ...
>  <packaging>bundle</packaging>
>  ...
>  <plugin>
>    <groupId>org.apache.felix</groupId>
>    <artifactId>maven-bundle-plugin</artifactId>
>    <extensions>true</extensions>
>    <configuration>
>      <instructions>
>        <Export-Package>org.apache.tuscany.sca.foo</Export-Package>
>        <Import-Package>org.apache.tuscany.sca.bar</Import-Package>
>        <Private-Package>org.apache.tuscany.sca.impl.*</Private-Package>
>      </instructions>
>    </configuration>
>  </plugin>
> ...



I think we have four options on how we start bundle-izing Tuscany modules,
using maven-bundle-plugin.


1. Use the manifest goal to generate the manifest entry in all module jars,
with <Export-Package>*</Export-Package> and
<Import-Package>*</Import-Package>, which export all the packages from the
module and import everything used by the module. The Export-Package and
Import-Package statements in the generated manifest will use explicit
package import/exports like:

Import-Package:
javax.xml.namespace,javax.xml.parsers,org.apache.tuscany.sca.contribution.resolver,...
Export-Package:
org.apache.tuscany.sca.implementation.osgi;version="2.0";uses:="o.a.t.s.assembly",/...


The analysis of Import/Export will be done by the maven-bundle-plugin in
this case. The actual generation of the module jar will remain the same as
now, the only addition will be a new generated manifest file.

We might not use <Export-Package>*</Export-Package> , but instead it would
be something more like
<Export-Package>org.apache.tuscany.sca.implementation.osgi.*</Export-Package>,
where the packages are wildcarded, but no assumptions about modularity are
made.

2. Use bundle goal with Import/Export as above (generated by
maven-bundle-plugin rather than explicitly specified). In this case the jar
itself will be generated using maven-bundle-plugin. The output jar should be
identical to 1.

3. Use manifest goal with explicitly listed <Export-Package/>,
<Import-Package/> as in Sebastien's example. All exported and (maybe)
imported packages  are explicitly specified in the pom. Everytime a
new package is added, the pom should be updated.

4.  Use bundle goal with explicitly listed <Export-Package/>,
<Import-Package/> and <Private-Package/> as in Sebastien's example. This
option corresponds to Sebastien's example. In this case both the contents of
the jar, as well as the manifest are based on the explicitly listed
packages.


1. is the easiest to implement. It makes no assumptions about modularity,
and will not have any impact on non-OSGi applications since only the
manifest entry is affected. From an OSGi point of view, this gives all we
need for osgi-tuscany.

2. does not add any value over 1.

3. This is a half-hearted attempt to enforce modularity. osgi-tuscany will
break everytime someone introduces a new package  and doesn't update the
pom. But Tuscany will continue to operate without OSGi since only the
manifest is broken. 3. is not a requirement for OSGi-based Tuscany, but
merely the use of OSGi technology to improve modularity. osgi-tuscany will
take on the role on policing modularity(apart from testing that Tuscany can
be run in an OSGi runtime).

4. This is OSGi best practice. If a new package is introduced and the pom is
not updated, the jar itself will be unusable. Basically this requires all
developers to be OSGi-aware. The value-add of this option over 1. is the
enforcement of modularity, and like 3, this is not a requirement for
OSGi-based Tuscany. I am not sure we are ready for this yet. I think there
is a real risk that frequent breakages will create the perception that OSGi
integration is impossible to maintain, even though this is more to do with
enforcing modularity rather than OSGi.


I would like to suggest that we start with 1., use the generated manifest
files to understand and improve modularity and jump to 4. to enforce
modularity when we are ready for it.





> or did you mean something else?
>
> --
> Jean-Sebastien
>



-- 
Thank you...

Regards,

Rajini

Re: Improving support for running in OSGi

Posted by Jean-Sebastien Delfino <js...@apache.org>.
Graham Charters wrote:
> It would seem that the fine-grained/coarse-grained thoughts have
> people divided.  Rajini's note (aside from the fact she has a tonne of
> experience having done most, if not all, of the OSGi work in Tuscany)
> paints a picture where the two are not mutually exclusive.  I don't
> typically like doing two options because that seems like indecision,
> but in this case they do appear to complement each other.
> 
> Based on what I've seen in this thread, I'm hoping this briefly
> summarizes the collection of thoughts on how we might proceed:
> 
> Tuscany Running in OSGi
> 
> 1.  Add bundle manifests to all the Tuscany modules (using maven
> bundle plugin).  This will ensure the most fine-grained decomposition
> works and therefore coarser-grained distributions will also work.
> 2.  Add a distribution which creates bundles around manageable
> collections of Tuscany modules aligned with how we see the runtime
> being extended/subsetted/replaced.  Have this build and test on
> Continuum.
> 3.  Create 'virtual bundles' for third-party libraries to avoid
> licensing and disk space issues (based on Rajini's suggestions, which
> I need to better understand).
> 4.  Provide guidance on the wiki on how to avoid breaking OSGi.
> 
> There's also the suggestion that implementation.osgi relate to
> Distributed OSGi (RFC 119), which shouldn't be lost.
> 
> Have I missed anything?
> 
> It sounds like a staging of 1 & 3 first, followed by 2 would work (and
> 4 if things keep breaking :-( ).  I'd be concerned if we didn't get to
> 2, and as part of 1&3, we need to make sure these are regularly tested
> under osgi.
> 

Sounds good to me, with the following comments:

When you have (1) + (3) you can already run a continuum OSGi build, 
before attacking (2). I'd actually suggest to start building on 
continuum as soon as we have (1).

- I'm not clear on what you meant by 'use the Maven bundle plugin'. Will 
we write something like this for example:
<project>
   ...
   <packaging>bundle</packaging>
   ...
   <plugin>
     <groupId>org.apache.felix</groupId>
     <artifactId>maven-bundle-plugin</artifactId>
     <extensions>true</extensions>
     <configuration>
       <instructions>
         <Export-Package>org.apache.tuscany.sca.foo</Export-Package>
         <Import-Package>org.apache.tuscany.sca.bar</Import-Package>
         <Private-Package>org.apache.tuscany.sca.impl.*</Private-Package>
       </instructions>
     </configuration>
   </plugin>
...

or did you mean something else?

-- 
Jean-Sebastien

Re: Improving support for running in OSGi

Posted by Graham Charters <gc...@googlemail.com>.
It would seem that the fine-grained/coarse-grained thoughts have
people divided.  Rajini's note (aside from the fact she has a tonne of
experience having done most, if not all, of the OSGi work in Tuscany)
paints a picture where the two are not mutually exclusive.  I don't
typically like doing two options because that seems like indecision,
but in this case they do appear to complement each other.

Based on what I've seen in this thread, I'm hoping this briefly
summarizes the collection of thoughts on how we might proceed:

Tuscany Running in OSGi

1.  Add bundle manifests to all the Tuscany modules (using maven
bundle plugin).  This will ensure the most fine-grained decomposition
works and therefore coarser-grained distributions will also work.
2.  Add a distribution which creates bundles around manageable
collections of Tuscany modules aligned with how we see the runtime
being extended/subsetted/replaced.  Have this build and test on
Continuum.
3.  Create 'virtual bundles' for third-party libraries to avoid
licensing and disk space issues (based on Rajini's suggestions, which
I need to better understand).
4.  Provide guidance on the wiki on how to avoid breaking OSGi.

There's also the suggestion that implementation.osgi relate to
Distributed OSGi (RFC 119), which shouldn't be lost.

Have I missed anything?

It sounds like a staging of 1 & 3 first, followed by 2 would work (and
4 if things keep breaking :-( ).  I'd be concerned if we didn't get to
2, and as part of 1&3, we need to make sure these are regularly tested
under osgi.


2008/5/1 Mike Edwards <mi...@gmail.com>:
>
> Simon Laws wrote:
>
> > On Thu, May 1, 2008 at 10:03 AM, Rajini Sivaram <
> > rajinisivaram@googlemail.com> wrote:
> >
> >
> > > On 5/1/08, Jean-Sebastien Delfino <js...@apache.org> wrote:
> > >
> > > > My 2c:
> > > >
> > > > +1 to promote OSGi to a first class Tuscany runtime environment
> > > >
> > > > +1 for an OSGi continuum build (thinking about a build profile that'll
> > > >
> > > run
> > >
> > > > the Tuscany itest suite in an OSGi environment, similar to the
> profiles
> > > >
> > > we
> > >
> > > > have for Web containers for example)
> > > >
> > > > Here's what I imagined we'd do:
> > > > 1. add OSGi entries to each of our JAR manifests
> > > > 2. have developers maintain them and pay attention to imports/exports
> > > > 3. use the OSGi build to detect API and SPI import/export violations
> > > > 4. find the best way to OSGi-enable 3rd party dependency JARs
> > > >
> > > > I realize that my suggestion [1] is not very popular and most people
> on
> > > > this list would prefer to come up with bigger bundles grouping several
> > > >
> > > of
> > >
> > > > our JARs/modules. I don't think that the 'bigger aggregate bundle'
> > > >
> > > approach
> > >
> > > > will work, but I'll be happy to watch people try it :) if they  want
> to.
> > > >
> > > > With respect to [4] I find rather funny to see many projects out there
> > > > claim OSGi enablement without having OSGified their 3rd party
> > > >
> > > dependencies.
> > >
> > > > I wonder how that works, can an OSGi-enabled project really leverage
> the
> > > > OSGi classloader isolation and versioning capabilities when 99% of the
> > > >
> > > JARs
> > >
> > > > it requires are not OSGi bundles? I must be missing something... and I
> > > >
> > > hope
> > >
> > > > we can do better in Tuscany with a real end-to-end OSGi enablement
> story
> > > >
> > > :)
> > >
> > > > --
> > > > Jean-Sebastien
> > > >
> > > >
> > >
> > > I agree with Sebastien's suggestions1-4. But I would like to suggest a
> > > slight variation to the first suggestion.
> > >
> > >
> > >  1. Use maven-bundle-plugin to introduce OSGi manifest entries into all
> > >  the Tuscany modules, with auto-generated import/export statements.
> > > Modify
> > >  itest/osgi-tuscany to run these modules under OSGi, with all 3rd party
> > > jars
> > >  installed into OSGi using virtual bundles created on the fly.
> > >
> > > This step will provide a version of osgi-tuscany tests that is less
> prone
> > > to
> > > breakage than the one we have today. It will also help fix any remaining
> > > classloading issues that we have left in Tuscany (and hopefully help
> > > in maintaining the classloader isolation). This is not a big piece of
> work
> > > since this is just bringing together the different pieces that we
> already
> > > have. I will be happy to contribute the code towards this first step, so
> > > others can concentrate on what we really want to achieve in terms of
> > > modularity, distribution etc. We can also use this step to explore
> > > versioning - in particular about having multiple extensions referring to
> > > different versions of 3rd party libraries. This will be very useful for
> 4.
> > >
> > > Suggestions 2-3 are not requirements for OSGi, but these are clearly
> cases
> > > where OSGi technology can help Tuscany improve modularity. If we want to
> > > have explicitly hard-coded import/export statements in the modules to
> > > enforce modularity, that can be introduced in step 2.
> > >
> > > And I would expect that there will be more work in terms of building the
> > > distributions suitable for OSGi as well as non-OSGi after 1-4.
> > >
> > >
> > >
> > >
> > > Thank you...
> > >
> > > Regards,
> > >
> > > Rajini
> > >
> > >
> >
> > Re. modularity, it strikes me that there is lack of consensus about
> whether
> > big bundles or small bundles, relative to our maven modules,  are
> preferable
> > but that there is consensus about using OSGi bundles to help us test and
> > refine out modularity story.
> >
> > So maybe we should agree to disregard the question about how big OSGi
> > bundles should be relative to our maven modules and concentrate on the
> > question about whether our maven modules are correctly factored. Using
> OSGi
> > as a tool to help us with this of course.
> >
> > Simon
> >
> >
>  I support both Rajini's and Simon's points.  We can get the mechanics going
> first with the bundles following the existing grain of the modules - and we
> can discuss and tinker with the modules as we please later to find the best
> arrangements.
>
>  I suspect that the differing opinions about module size reflect different
> usecases.  Lots of small modules can promote great flexibility in the
> construction of runtimes, but at the expense of making it much harder for
> users to make sense of the options.  Larger modules cut down the options but
> make things easier for end users.
>
>
>  Yours,  Mike.
>

Re: Improving support for running in OSGi

Posted by Mike Edwards <mi...@gmail.com>.
Simon Laws wrote:
> On Thu, May 1, 2008 at 10:03 AM, Rajini Sivaram <
> rajinisivaram@googlemail.com> wrote:
> 
>> On 5/1/08, Jean-Sebastien Delfino <js...@apache.org> wrote:
>>> My 2c:
>>>
>>> +1 to promote OSGi to a first class Tuscany runtime environment
>>>
>>> +1 for an OSGi continuum build (thinking about a build profile that'll
>> run
>>> the Tuscany itest suite in an OSGi environment, similar to the profiles
>> we
>>> have for Web containers for example)
>>>
>>> Here's what I imagined we'd do:
>>> 1. add OSGi entries to each of our JAR manifests
>>> 2. have developers maintain them and pay attention to imports/exports
>>> 3. use the OSGi build to detect API and SPI import/export violations
>>> 4. find the best way to OSGi-enable 3rd party dependency JARs
>>>
>>> I realize that my suggestion [1] is not very popular and most people on
>>> this list would prefer to come up with bigger bundles grouping several
>> of
>>> our JARs/modules. I don't think that the 'bigger aggregate bundle'
>> approach
>>> will work, but I'll be happy to watch people try it :) if they  want to.
>>>
>>> With respect to [4] I find rather funny to see many projects out there
>>> claim OSGi enablement without having OSGified their 3rd party
>> dependencies.
>>> I wonder how that works, can an OSGi-enabled project really leverage the
>>> OSGi classloader isolation and versioning capabilities when 99% of the
>> JARs
>>> it requires are not OSGi bundles? I must be missing something... and I
>> hope
>>> we can do better in Tuscany with a real end-to-end OSGi enablement story
>> :)
>>> --
>>> Jean-Sebastien
>>>
>>
>> I agree with Sebastien's suggestions1-4. But I would like to suggest a
>> slight variation to the first suggestion.
>>
>>
>>   1. Use maven-bundle-plugin to introduce OSGi manifest entries into all
>>   the Tuscany modules, with auto-generated import/export statements.
>> Modify
>>   itest/osgi-tuscany to run these modules under OSGi, with all 3rd party
>> jars
>>   installed into OSGi using virtual bundles created on the fly.
>>
>> This step will provide a version of osgi-tuscany tests that is less prone
>> to
>> breakage than the one we have today. It will also help fix any remaining
>> classloading issues that we have left in Tuscany (and hopefully help
>> in maintaining the classloader isolation). This is not a big piece of work
>> since this is just bringing together the different pieces that we already
>> have. I will be happy to contribute the code towards this first step, so
>> others can concentrate on what we really want to achieve in terms of
>> modularity, distribution etc. We can also use this step to explore
>> versioning - in particular about having multiple extensions referring to
>> different versions of 3rd party libraries. This will be very useful for 4.
>>
>> Suggestions 2-3 are not requirements for OSGi, but these are clearly cases
>> where OSGi technology can help Tuscany improve modularity. If we want to
>> have explicitly hard-coded import/export statements in the modules to
>> enforce modularity, that can be introduced in step 2.
>>
>> And I would expect that there will be more work in terms of building the
>> distributions suitable for OSGi as well as non-OSGi after 1-4.
>>
>>
>>
>>
>> Thank you...
>>
>> Regards,
>>
>> Rajini
>>
> 
> Re. modularity, it strikes me that there is lack of consensus about whether
> big bundles or small bundles, relative to our maven modules,  are preferable
> but that there is consensus about using OSGi bundles to help us test and
> refine out modularity story.
> 
> So maybe we should agree to disregard the question about how big OSGi
> bundles should be relative to our maven modules and concentrate on the
> question about whether our maven modules are correctly factored. Using OSGi
> as a tool to help us with this of course.
> 
> Simon
> 
I support both Rajini's and Simon's points.  We can get the mechanics 
going first with the bundles following the existing grain of the modules 
- and we can discuss and tinker with the modules as we please later to 
find the best arrangements.

I suspect that the differing opinions about module size reflect 
different usecases.  Lots of small modules can promote great flexibility 
in the construction of runtimes, but at the expense of making it much 
harder for users to make sense of the options.  Larger modules cut down 
the options but make things easier for end users.


Yours,  Mike.

Re: Improving support for running in OSGi

Posted by Simon Laws <si...@googlemail.com>.
On Thu, May 1, 2008 at 10:03 AM, Rajini Sivaram <
rajinisivaram@googlemail.com> wrote:

> On 5/1/08, Jean-Sebastien Delfino <js...@apache.org> wrote:
> >
> > My 2c:
> >
> > +1 to promote OSGi to a first class Tuscany runtime environment
> >
> > +1 for an OSGi continuum build (thinking about a build profile that'll
> run
> > the Tuscany itest suite in an OSGi environment, similar to the profiles
> we
> > have for Web containers for example)
> >
> > Here's what I imagined we'd do:
> > 1. add OSGi entries to each of our JAR manifests
> > 2. have developers maintain them and pay attention to imports/exports
> > 3. use the OSGi build to detect API and SPI import/export violations
> > 4. find the best way to OSGi-enable 3rd party dependency JARs
> >
> > I realize that my suggestion [1] is not very popular and most people on
> > this list would prefer to come up with bigger bundles grouping several
> of
> > our JARs/modules. I don't think that the 'bigger aggregate bundle'
> approach
> > will work, but I'll be happy to watch people try it :) if they  want to.
> >
> > With respect to [4] I find rather funny to see many projects out there
> > claim OSGi enablement without having OSGified their 3rd party
> dependencies.
> > I wonder how that works, can an OSGi-enabled project really leverage the
> > OSGi classloader isolation and versioning capabilities when 99% of the
> JARs
> > it requires are not OSGi bundles? I must be missing something... and I
> hope
> > we can do better in Tuscany with a real end-to-end OSGi enablement story
> :)
> > --
> > Jean-Sebastien
> >
>
>
> I agree with Sebastien's suggestions1-4. But I would like to suggest a
> slight variation to the first suggestion.
>
>
>   1. Use maven-bundle-plugin to introduce OSGi manifest entries into all
>   the Tuscany modules, with auto-generated import/export statements.
> Modify
>   itest/osgi-tuscany to run these modules under OSGi, with all 3rd party
> jars
>   installed into OSGi using virtual bundles created on the fly.
>
> This step will provide a version of osgi-tuscany tests that is less prone
> to
> breakage than the one we have today. It will also help fix any remaining
> classloading issues that we have left in Tuscany (and hopefully help
> in maintaining the classloader isolation). This is not a big piece of work
> since this is just bringing together the different pieces that we already
> have. I will be happy to contribute the code towards this first step, so
> others can concentrate on what we really want to achieve in terms of
> modularity, distribution etc. We can also use this step to explore
> versioning - in particular about having multiple extensions referring to
> different versions of 3rd party libraries. This will be very useful for 4.
>
> Suggestions 2-3 are not requirements for OSGi, but these are clearly cases
> where OSGi technology can help Tuscany improve modularity. If we want to
> have explicitly hard-coded import/export statements in the modules to
> enforce modularity, that can be introduced in step 2.
>
> And I would expect that there will be more work in terms of building the
> distributions suitable for OSGi as well as non-OSGi after 1-4.
>
>
>
>
> Thank you...
>
> Regards,
>
> Rajini
>

Re. modularity, it strikes me that there is lack of consensus about whether
big bundles or small bundles, relative to our maven modules,  are preferable
but that there is consensus about using OSGi bundles to help us test and
refine out modularity story.

So maybe we should agree to disregard the question about how big OSGi
bundles should be relative to our maven modules and concentrate on the
question about whether our maven modules are correctly factored. Using OSGi
as a tool to help us with this of course.

Simon

Re: Improving support for running in OSGi

Posted by Rajini Sivaram <ra...@googlemail.com>.
On 5/1/08, Jean-Sebastien Delfino <js...@apache.org> wrote:
>
> My 2c:
>
> +1 to promote OSGi to a first class Tuscany runtime environment
>
> +1 for an OSGi continuum build (thinking about a build profile that'll run
> the Tuscany itest suite in an OSGi environment, similar to the profiles we
> have for Web containers for example)
>
> Here's what I imagined we'd do:
> 1. add OSGi entries to each of our JAR manifests
> 2. have developers maintain them and pay attention to imports/exports
> 3. use the OSGi build to detect API and SPI import/export violations
> 4. find the best way to OSGi-enable 3rd party dependency JARs
>
> I realize that my suggestion [1] is not very popular and most people on
> this list would prefer to come up with bigger bundles grouping several of
> our JARs/modules. I don't think that the 'bigger aggregate bundle' approach
> will work, but I'll be happy to watch people try it :) if they  want to.
>
> With respect to [4] I find rather funny to see many projects out there
> claim OSGi enablement without having OSGified their 3rd party dependencies.
> I wonder how that works, can an OSGi-enabled project really leverage the
> OSGi classloader isolation and versioning capabilities when 99% of the JARs
> it requires are not OSGi bundles? I must be missing something... and I hope
> we can do better in Tuscany with a real end-to-end OSGi enablement story :)
> --
> Jean-Sebastien
>


I agree with Sebastien's suggestions1-4. But I would like to suggest a
slight variation to the first suggestion.


   1. Use maven-bundle-plugin to introduce OSGi manifest entries into all
   the Tuscany modules, with auto-generated import/export statements. Modify
   itest/osgi-tuscany to run these modules under OSGi, with all 3rd party jars
   installed into OSGi using virtual bundles created on the fly.

This step will provide a version of osgi-tuscany tests that is less prone to
breakage than the one we have today. It will also help fix any remaining
classloading issues that we have left in Tuscany (and hopefully help
in maintaining the classloader isolation). This is not a big piece of work
since this is just bringing together the different pieces that we already
have. I will be happy to contribute the code towards this first step, so
others can concentrate on what we really want to achieve in terms of
modularity, distribution etc. We can also use this step to explore
versioning - in particular about having multiple extensions referring to
different versions of 3rd party libraries. This will be very useful for 4.

Suggestions 2-3 are not requirements for OSGi, but these are clearly cases
where OSGi technology can help Tuscany improve modularity. If we want to
have explicitly hard-coded import/export statements in the modules to
enforce modularity, that can be introduced in step 2.

And I would expect that there will be more work in terms of building the
distributions suitable for OSGi as well as non-OSGi after 1-4.




Thank you...

Regards,

Rajini