You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@servicemix.apache.org by Guillaume Nodet <gn...@gmail.com> on 2011/11/17 14:15:28 UTC
[DISCUSS] Enhance specs to work better with JRE
Deploying java specs jars and providers is a bit tricky in osgi, even when
using the servicemix specs.
I think we face two problems:
* when deploying a spec bundle, we need to hide the packages from the jre
and by doing so we forbid the use of the implementation provided by the jre
* when deploying an implementation, it won't be available until the
bundle is started but this constraint isn't captured
For the first problem, I was thinking about deploying the specs jars in the
lib/endorsed dir so that they would be used.by the jre and enhance them so
that they would be able to discover the implementations in bundles but also
in the jre. This would work as they would use the same packages. This may
require some modifications in karaf, though i'm not sure yet.
For the second point, i'm still hoping that the Generic Capabilities
introduced in osgi 4.3 can be leveraged fo that. I think that's aso
related to obr.
Anyway, i'd like to find a solution to those problems, so any idea is of
course welcomed.
--
------------------------
Guillaume Nodet
------------------------
Blog: http://gnodet.blogspot.com/
------------------------
Open Source SOA
http://fusesource.com
Re: [DISCUSS] Enhance specs to work better with JRE
Posted by Guillaume Nodet <gn...@gmail.com>.
I hear you, but you're talking about the implementations, not the specs,
right ?
Last week, we discussed a problem resulting in hiding the javax.xml.stream
package. When hiding that package, the javax.xml.transform.stax isn't
usable anymore and that one isn't really provided by any spec, so we'd have
to find a way to ship JAXP again.
The solution I'm experimenting would really be to hack (by using endorsed
jars) the specs, but provide the implementation as bundles (that would be
shipped by default by ServiceMix when it makes sense), and I do agree that
Servicemix should provide a good out of the box experience on that.
On Fri, Nov 18, 2011 at 16:16, Daniel Kulp <dk...@apache.org> wrote:
>
> Well, I guess I have two questions/thoughts on this:
>
> 1) CAN we get them to support the in JDK versions?
> 2) SHOULD we, specifically for ServiceMix, etc... Particularly by default.
>
> You've obviously been digging into the code to find the answer to #1. I'll
> leave that to you.
>
> However, for #2, I STRONGLY STRONGLY feel that the default SMX installs
> need
> to provide a high quality and consistent experience for the users. To me,
> that means making sure we have the in-jdk versions blocked and provide
> bundles
> to provide that functionality. With the exception of XSLT/XPath, the
> versions in the JDK are buggy and way behind the versions that are
> available
> (and that we test with for the most part). What's worse, the versions in
> the
> JDK are very inconsistent. Depending on what version of the JDK the user
> uses, they can get COMPLETELY different results. Also, IBM JDK is very
> different than Sun's which is different than the Oracle JRocket which is
> different than the Mac, etc..... Couple of examples:
>
> StAX - the version in the JDK is very slow, buggy (has issues with some
> namespace handling, large text contents, etc..), and the factories are not
> thread safe. If the libraries/users are used to Woodstox, they might not
> even realize that last point and you can get all kinds of issues.
>
> JAXB - the version in the JDK up to _26 causes thread local leaks and jar
> locking of applications preventing undeploys, causes PermGen contention,
> and
> has other issues. _27 updated to a version that fixes that, but the later
> versions provide fixes for 3 other things that CXF users have reported. I
> don't think the Mac JDK has been updated to even that level yet. Not
> sure on
> the IBM JDK.
>
> Unfortunately, the XSLT stuff is the big exception. Xalan hasn't done a
> release in a long time and the version in the JDK is quite a bit ahead of
> the
> last xalan release. (supports StAXSource for example)
>
> In any case, I strongly feel we need to make sure that SMX "out of the box"
> provides the best experience for the users and, more importantly, the most
> consistent and supportable experience for us just to reduce the support
> load.
> Not knowing what versions of various technologies the users are using will
> just cause us a lot more headaches.
>
>
> Dan
>
>
>
>
> On Thursday, November 17, 2011 2:15:28 PM Guillaume Nodet wrote:
> > Deploying java specs jars and providers is a bit tricky in osgi, even
> when
> > using the servicemix specs.
> > I think we face two problems:
> > * when deploying a spec bundle, we need to hide the packages from the
> jre
> > and by doing so we forbid the use of the implementation provided by the
> jre
> > * when deploying an implementation, it won't be available until the
> > bundle is started but this constraint isn't captured
> >
> > For the first problem, I was thinking about deploying the specs jars in
> the
> > lib/endorsed dir so that they would be used.by the jre and enhance them
> so
> > that they would be able to discover the implementations in bundles but
> also
> > in the jre. This would work as they would use the same packages. This
> may
> > require some modifications in karaf, though i'm not sure yet.
> >
> > For the second point, i'm still hoping that the Generic Capabilities
> > introduced in osgi 4.3 can be leveraged fo that. I think that's aso
> > related to obr.
> >
> > Anyway, i'd like to find a solution to those problems, so any idea is of
> > course welcomed.
> --
> Daniel Kulp
> dkulp@apache.org - http://dankulp.com/blog
> Talend Community Coder - http://coders.talend.com
>
--
------------------------
Guillaume Nodet
------------------------
Blog: http://gnodet.blogspot.com/
------------------------
Open Source SOA
http://fusesource.com
Re: [DISCUSS] Enhance specs to work better with JRE
Posted by Daniel Kulp <dk...@apache.org>.
Well, I guess I have two questions/thoughts on this:
1) CAN we get them to support the in JDK versions?
2) SHOULD we, specifically for ServiceMix, etc... Particularly by default.
You've obviously been digging into the code to find the answer to #1. I'll
leave that to you.
However, for #2, I STRONGLY STRONGLY feel that the default SMX installs need
to provide a high quality and consistent experience for the users. To me,
that means making sure we have the in-jdk versions blocked and provide bundles
to provide that functionality. With the exception of XSLT/XPath, the
versions in the JDK are buggy and way behind the versions that are available
(and that we test with for the most part). What's worse, the versions in the
JDK are very inconsistent. Depending on what version of the JDK the user
uses, they can get COMPLETELY different results. Also, IBM JDK is very
different than Sun's which is different than the Oracle JRocket which is
different than the Mac, etc..... Couple of examples:
StAX - the version in the JDK is very slow, buggy (has issues with some
namespace handling, large text contents, etc..), and the factories are not
thread safe. If the libraries/users are used to Woodstox, they might not
even realize that last point and you can get all kinds of issues.
JAXB - the version in the JDK up to _26 causes thread local leaks and jar
locking of applications preventing undeploys, causes PermGen contention, and
has other issues. _27 updated to a version that fixes that, but the later
versions provide fixes for 3 other things that CXF users have reported. I
don't think the Mac JDK has been updated to even that level yet. Not sure on
the IBM JDK.
Unfortunately, the XSLT stuff is the big exception. Xalan hasn't done a
release in a long time and the version in the JDK is quite a bit ahead of the
last xalan release. (supports StAXSource for example)
In any case, I strongly feel we need to make sure that SMX "out of the box"
provides the best experience for the users and, more importantly, the most
consistent and supportable experience for us just to reduce the support load.
Not knowing what versions of various technologies the users are using will
just cause us a lot more headaches.
Dan
On Thursday, November 17, 2011 2:15:28 PM Guillaume Nodet wrote:
> Deploying java specs jars and providers is a bit tricky in osgi, even when
> using the servicemix specs.
> I think we face two problems:
> * when deploying a spec bundle, we need to hide the packages from the jre
> and by doing so we forbid the use of the implementation provided by the jre
> * when deploying an implementation, it won't be available until the
> bundle is started but this constraint isn't captured
>
> For the first problem, I was thinking about deploying the specs jars in the
> lib/endorsed dir so that they would be used.by the jre and enhance them so
> that they would be able to discover the implementations in bundles but also
> in the jre. This would work as they would use the same packages. This may
> require some modifications in karaf, though i'm not sure yet.
>
> For the second point, i'm still hoping that the Generic Capabilities
> introduced in osgi 4.3 can be leveraged fo that. I think that's aso
> related to obr.
>
> Anyway, i'd like to find a solution to those problems, so any idea is of
> course welcomed.
--
Daniel Kulp
dkulp@apache.org - http://dankulp.com/blog
Talend Community Coder - http://coders.talend.com
Re: [DISCUSS] Enhance specs to work better with JRE
Posted by Guillaume Nodet <gn...@gmail.com>.
I'm not sure it can be done. The reason is that the provider default
class name is usually embedded in a static method of the factory which is
part of the api and not in the FactoryFinder.
For example I have the following code coming from the stax XMLInputFactory
class on my mac jre:
public static XMLInputFactory newInstance()
throws FactoryConfigurationError
{
return (XMLInputFactory) FactoryFinder.find(
"javax.xml.stream.XMLInputFactory",
"com.sun.xml.internal.stream.XMLInputFactoryImpl");
}
Loading this class is not really easy if we deploy our own specs as
endorsed and I suppose we'd have ClassCastException if we really use that
one.
Also, there's always the possibility to configure the system properties
which would be used after any implementation provided by an osgi bundle,
but before the default factory.
If we look at how the packages are loaded by the karaf main class using
${jre-${java.specification.version}}, we might be able to do the same for
those properties using ${java.specification.vendor} .
On Fri, Nov 18, 2011 at 16:45, Daniel Kulp <dk...@apache.org> wrote:
> On Friday, November 18, 2011 4:39:57 PM Guillaume Nodet wrote:
> > On Fri, Nov 18, 2011 at 16:25, Guillaume Nodet <gn...@gmail.com> wrote:
> > > On Fri, Nov 18, 2011 at 16:19, Daniel Kulp <dk...@apache.org> wrote:
> > >> Can I have more details on that last point? Keep in mind, the class
> > >> names
> > >> for the defaults are very different depending on the JDK in use.
> > >> Thus,
> > >> you
> > >> cannot just change the default classname in spec jar as that may not
> > >> work on
> > >> the JDK. You may need to find a way to actually get it to dig into
> > >> the Provider factory thing in the JDK to get the right class name.
> > >
> > > Yes, that's really one point I still have to investigate as I'd like to
> > > be able to use the JRE provided implementations when possible.
> > > I don't really see any other solution than to hardcode the names as the
> > > implementation class names aren't available easily (I really don't want
> > > to start hacking into byte code to find the value of parameter calls).
> > > I suppose that would mean supporting a limited set of JDK that we would
> > > have tested our default values with.
>
> Is there a way to get the bytecode for the original FactoryFinder out of
> the
> rt.jar (or wherever on Mac), create class out of it and just call it?
>
> Guess that still involves a little bytecode manipulation. :-(
>
>
> --
> Daniel Kulp
> dkulp@apache.org - http://dankulp.com/blog
> Talend Community Coder - http://coders.talend.com
>
--
------------------------
Guillaume Nodet
------------------------
Blog: http://gnodet.blogspot.com/
------------------------
Open Source SOA
http://fusesource.com
Re: [DISCUSS] Enhance specs to work better with JRE
Posted by Daniel Kulp <dk...@apache.org>.
On Friday, November 18, 2011 4:39:57 PM Guillaume Nodet wrote:
> On Fri, Nov 18, 2011 at 16:25, Guillaume Nodet <gn...@gmail.com> wrote:
> > On Fri, Nov 18, 2011 at 16:19, Daniel Kulp <dk...@apache.org> wrote:
> >> Can I have more details on that last point? Keep in mind, the class
> >> names
> >> for the defaults are very different depending on the JDK in use.
> >> Thus,
> >> you
> >> cannot just change the default classname in spec jar as that may not
> >> work on
> >> the JDK. You may need to find a way to actually get it to dig into
> >> the Provider factory thing in the JDK to get the right class name.
> >
> > Yes, that's really one point I still have to investigate as I'd like to
> > be able to use the JRE provided implementations when possible.
> > I don't really see any other solution than to hardcode the names as the
> > implementation class names aren't available easily (I really don't want
> > to start hacking into byte code to find the value of parameter calls).
> > I suppose that would mean supporting a limited set of JDK that we would
> > have tested our default values with.
Is there a way to get the bytecode for the original FactoryFinder out of the
rt.jar (or wherever on Mac), create class out of it and just call it?
Guess that still involves a little bytecode manipulation. :-(
--
Daniel Kulp
dkulp@apache.org - http://dankulp.com/blog
Talend Community Coder - http://coders.talend.com
Re: [DISCUSS] Enhance specs to work better with JRE
Posted by Guillaume Nodet <gn...@gmail.com>.
Missed dev@ as a recipient.
On Fri, Nov 18, 2011 at 16:25, Guillaume Nodet <gn...@gmail.com> wrote:
>
>
> On Fri, Nov 18, 2011 at 16:19, Daniel Kulp <dk...@apache.org> wrote:
>
>> On Friday, November 18, 2011 8:59:48 AM Guillaume Nodet wrote:
>> > I've been experimenting a bit with the first part. and it seems I have
>> > something working. It consists of the following:
>> > * a modified karaf to support karaf activators (some kind of bundle
>> > activators but loaded from jars in the lib/ folder instead of bundles)
>> > * the spec jars dropped into lib/endorsed
>> > * a karaf activator (same as the one in the spec) but in a separate
>> jar
>> > in the lib folder
>> > With that (and provided the spec defaults to the implementation in the
>> JRE
>> > which require a few modifications for some),
>>
>> Can I have more details on that last point? Keep in mind, the class
>> names
>> for the defaults are very different depending on the JDK in use. Thus,
>> you
>> cannot just change the default classname in spec jar as that may not work
>> on
>> the JDK. You may need to find a way to actually get it to dig into the
>> Provider factory thing in the JDK to get the right class name.
>>
>
> Yes, that's really one point I still have to investigate as I'd like to be
> able to use the JRE provided implementations when possible.
> I don't really see any other solution than to hardcode the names as the
> implementation class names aren't available easily (I really don't want to
> start hacking into byte code to find the value of parameter calls).
> I suppose that would mean supporting a limited set of JDK that we would
> have tested our default values with.
>
>
>>
>> Dan
>>
>>
>> > it seems I can safely install
>> > a bundle containing an implementation of any spec and have it used
>> instead
>> > of the default jre implementation, without having to hide packages.
>> > The drawback is that such installation can't be done from inside karaf
>> > itself as it require rebooting the jre, but in the case of servicemix,
>> it
>> > should not be a problem.
>> > Another benefit is that instead of having each spec jar scanning
>> bundles,
>> > we only have a single activator for all specs deployed that way, which
>> is
>> > slightly more efficient.
>> >
>> > I'll commit the needed changes in karaf and specs so that we can further
>> > experiment and try to look at the second problem.
>> >
>> > On Thu, Nov 17, 2011 at 14:15, Guillaume Nodet <gn...@gmail.com>
>> wrote:
>> > > Deploying java specs jars and providers is a bit tricky in osgi, even
>> > > when using the servicemix specs.
>> > >
>> > > I think we face two problems:
>> > > * when deploying a spec bundle, we need to hide the packages from
>> > > the
>> > >
>> > > jre and by doing so we forbid the use of the implementation provided
>> by
>> > > the jre
>> > >
>> > > * when deploying an implementation, it won't be available until the
>> > >
>> > > bundle is started but this constraint isn't captured
>> > >
>> > > For the first problem, I was thinking about deploying the specs jars
>> in
>> > > the lib/endorsed dir so that they would be used.by the jre and
>> enhance
>> > > them so that they would be able to discover the implementations in
>> > > bundles but also in the jre. This would work as they would use the
>> > > same packages.>
>> > > This may require some modifications in karaf, though i'm not sure
>> yet.
>> > >
>> > > For the second point, i'm still hoping that the Generic Capabilities
>> > > introduced in osgi 4.3 can be leveraged fo that. I think that's aso
>> > > related to obr.
>> > >
>> > > Anyway, i'd like to find a solution to those problems, so any idea is
>> of
>> > > course welcomed.
>> > >
>> > > --
>> > > ------------------------
>> > > Guillaume Nodet
>> > > ------------------------
>> > > Blog: http://gnodet.blogspot.com/
>> > > ------------------------
>> > > Open Source SOA
>> > > http://fusesource.com
>> --
>> Daniel Kulp
>> dkulp@apache.org - http://dankulp.com/blog
>> Talend Community Coder - http://coders.talend.com
>>
>
>
>
> --
> ------------------------
> Guillaume Nodet
> ------------------------
> Blog: http://gnodet.blogspot.com/
> ------------------------
> Open Source SOA
> http://fusesource.com
>
>
--
------------------------
Guillaume Nodet
------------------------
Blog: http://gnodet.blogspot.com/
------------------------
Open Source SOA
http://fusesource.com
Re: [DISCUSS] Enhance specs to work better with JRE
Posted by Daniel Kulp <dk...@apache.org>.
On Friday, November 18, 2011 8:59:48 AM Guillaume Nodet wrote:
> I've been experimenting a bit with the first part. and it seems I have
> something working. It consists of the following:
> * a modified karaf to support karaf activators (some kind of bundle
> activators but loaded from jars in the lib/ folder instead of bundles)
> * the spec jars dropped into lib/endorsed
> * a karaf activator (same as the one in the spec) but in a separate jar
> in the lib folder
> With that (and provided the spec defaults to the implementation in the JRE
> which require a few modifications for some),
Can I have more details on that last point? Keep in mind, the class names
for the defaults are very different depending on the JDK in use. Thus, you
cannot just change the default classname in spec jar as that may not work on
the JDK. You may need to find a way to actually get it to dig into the
Provider factory thing in the JDK to get the right class name.
Dan
> it seems I can safely install
> a bundle containing an implementation of any spec and have it used instead
> of the default jre implementation, without having to hide packages.
> The drawback is that such installation can't be done from inside karaf
> itself as it require rebooting the jre, but in the case of servicemix, it
> should not be a problem.
> Another benefit is that instead of having each spec jar scanning bundles,
> we only have a single activator for all specs deployed that way, which is
> slightly more efficient.
>
> I'll commit the needed changes in karaf and specs so that we can further
> experiment and try to look at the second problem.
>
> On Thu, Nov 17, 2011 at 14:15, Guillaume Nodet <gn...@gmail.com> wrote:
> > Deploying java specs jars and providers is a bit tricky in osgi, even
> > when using the servicemix specs.
> >
> > I think we face two problems:
> > * when deploying a spec bundle, we need to hide the packages from
> > the
> >
> > jre and by doing so we forbid the use of the implementation provided by
> > the jre
> >
> > * when deploying an implementation, it won't be available until the
> >
> > bundle is started but this constraint isn't captured
> >
> > For the first problem, I was thinking about deploying the specs jars in
> > the lib/endorsed dir so that they would be used.by the jre and enhance
> > them so that they would be able to discover the implementations in
> > bundles but also in the jre. This would work as they would use the
> > same packages.>
> > This may require some modifications in karaf, though i'm not sure yet.
> >
> > For the second point, i'm still hoping that the Generic Capabilities
> > introduced in osgi 4.3 can be leveraged fo that. I think that's aso
> > related to obr.
> >
> > Anyway, i'd like to find a solution to those problems, so any idea is of
> > course welcomed.
> >
> > --
> > ------------------------
> > Guillaume Nodet
> > ------------------------
> > Blog: http://gnodet.blogspot.com/
> > ------------------------
> > Open Source SOA
> > http://fusesource.com
--
Daniel Kulp
dkulp@apache.org - http://dankulp.com/blog
Talend Community Coder - http://coders.talend.com
Re: [DISCUSS] Enhance specs to work better with JRE
Posted by Guillaume Nodet <gn...@gmail.com>.
I've been experimenting a bit with the first part. and it seems I have
something working. It consists of the following:
* a modified karaf to support karaf activators (some kind of bundle
activators but loaded from jars in the lib/ folder instead of bundles)
* the spec jars dropped into lib/endorsed
* a karaf activator (same as the one in the spec) but in a separate jar
in the lib folder
With that (and provided the spec defaults to the implementation in the JRE
which require a few modifications for some), it seems I can safely install
a bundle containing an implementation of any spec and have it used instead
of the default jre implementation, without having to hide packages.
The drawback is that such installation can't be done from inside karaf
itself as it require rebooting the jre, but in the case of servicemix, it
should not be a problem.
Another benefit is that instead of having each spec jar scanning bundles,
we only have a single activator for all specs deployed that way, which is
slightly more efficient.
I'll commit the needed changes in karaf and specs so that we can further
experiment and try to look at the second problem.
On Thu, Nov 17, 2011 at 14:15, Guillaume Nodet <gn...@gmail.com> wrote:
> Deploying java specs jars and providers is a bit tricky in osgi, even when
> using the servicemix specs.
> I think we face two problems:
> * when deploying a spec bundle, we need to hide the packages from the
> jre and by doing so we forbid the use of the implementation provided by the
> jre
> * when deploying an implementation, it won't be available until the
> bundle is started but this constraint isn't captured
>
> For the first problem, I was thinking about deploying the specs jars in
> the lib/endorsed dir so that they would be used.by the jre and enhance
> them so that they would be able to discover the implementations in bundles
> but also in the jre. This would work as they would use the same packages.
> This may require some modifications in karaf, though i'm not sure yet.
>
> For the second point, i'm still hoping that the Generic Capabilities
> introduced in osgi 4.3 can be leveraged fo that. I think that's aso
> related to obr.
>
> Anyway, i'd like to find a solution to those problems, so any idea is of
> course welcomed.
>
> --
> ------------------------
> Guillaume Nodet
> ------------------------
> Blog: http://gnodet.blogspot.com/
> ------------------------
> Open Source SOA
> http://fusesource.com
>
>
--
------------------------
Guillaume Nodet
------------------------
Blog: http://gnodet.blogspot.com/
------------------------
Open Source SOA
http://fusesource.com
Re: [DISCUSS] Enhance specs to work better with JRE
Posted by Jean-Baptiste Onofré <jb...@nanthrax.net>.
Hi Guillaume,
sorry for my late feedback.
My comments inline:
> * when deploying a spec bundle, we need to hide the packages from the jre
> and by doing so we forbid the use of the implementation provided by the jre
Good idea. As we discussed last week, this could be handle by the Karaf
profiles (using patch/diff/merge mechanism). We can also enhance the
karaf-maven-plugin to provision the spec bundle in the lib/endorsed.
At least, we have always the "custom distribution" way using an assembly.
> * when deploying an implementation, it won't be available until the
> bundle is started but this constraint isn't captured
The first read about the new OBR spec looks interesting for this point.
Having some kind of events, with callback handlers, will allow us to react.
Waiting for the new OBR, maybe we can implement a specific behavior for
spec. We can imagine to add an specific attribute in the MANIFEST
(SpecBundle or something like that), and handle a specific life cycle
and events for this kind of bundles (monitoring the directory for
instance).
Another way is to have a Spec OSGi service specific to handle the Spec
bundle live cycle. We can imagine that the Spec bundle expose a given
service interface with properties describing the services. In that case
we can have a service tracker to know when the service is started and ready.
I'm not sure it's good ideas, but it's the only ones that I have for now ;)
Regards
JB
--
Jean-Baptiste Onofré
jbonofre@apache.org
http://blog.nanthrax.net
Talend - http://www.talend.com