You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@jena.apache.org by Stian Soiland-Reyes <st...@apache.org> on 2015/09/10 23:50:08 UTC
Jena OSGi (was: [] 201508 Release of 23 Clerezza modules)
On 10 September 2015 at 18:13, ajs6f@virginia.edu <aj...@virginia.edu> wrote:
> If this is a matter of "just a couple of lines in the manifest file" cannot a patch be created to do that in Jena itself? Are there inter-module dependency issues that make that difficult?
In theory just setting
<packaging>bundle</packaging>
and using the maven-bundle-plugin
is enough to auto-generate the correct META-INF metadata for OSGi.
This can be customized (as we do for the apache-jena-osgi/jena-osgi
module).
One complication is if the external dependencies are OSGi or not -
httpclient is one tricky one as it has done the same as Jena and
provided a separate wrapper httpclient-osgi (and httpcore-osgi) -
however the way they did this with Maven means that just using it as a
dependency would still pull in a dependency on the regular httpclient
library. So if you are a non-OSGi user you would then see the
httpclient classes in two JARs - which with Maven version resolution
could easily become in mismatched versions.
Ironically httpclient-osgi does not depend on httpcore-osgi - so the
one dependency that it truly need isn't stated in its pom.
In jena-osgi I therefore excluded all those deeper dependencies:
See https://github.com/apache/jena/blob/master/apache-jena-osgi/jena-osgi/pom.xml#L165
To avoid Jena-Maven-OSGi users the same issue, I similarly here
slightly misused the <scope>provided</scope> for the dependencies that
are not to be dependencies of the final jena-osgi JAR, but which are
shadowed inside.
https://github.com/apache/jena/blob/master/apache-jena-osgi/jena-osgi/pom.xml#L107
If we move to <packaging>bundle</packaging> then we should get
httpclient folks to sort out their poms upstream so we can rely on
them in a cleaner fashion across Jena. (or put this exclusion rule
into <dependencyManagement> of jena-parent) - I don't think
copy-pasting that big <exclusion> block around anything that directly
or indirectly requires things like httpclient is good.
Ideally they should also move to <packaging>bundle</packaging> and
avoid *-osgi, which would of course simplify things.
There are other potential issues as Class.forName() which the current
jena-osgi is narrowly cheating around by effectively making a single
class loader for all of Jena (including as Reto pointed out, TDB etc)
--
Stian Soiland-Reyes
Apache Taverna (incubating), Apache Commons RDF (incubating)
http://orcid.org/0000-0001-9842-9718
Re: Jena OSGi (was: [] 201508 Release of 23 Clerezza modules)
Posted by Andy Seaborne <an...@apache.org>.
On 13/09/15 05:44, Stian Soiland-Reyes wrote:
> Agree that for the assembler subsystem you would need something like
> ServiceLoader and a Class.forName() fallback. It is mainly the
> parser/writer APIs I am concerned about.
>
> The classloaders here are already an issue if trying to add a custom
> plugin to a Fuseki WAR install (this was a users@ question earlier in
> the year I think).
That is a "feature" of WAR files surely?
Andy
>
>
>
>
>
> On 11 September 2015 at 12:44, ajs6f@virginia.edu <aj...@virginia.edu> wrote:
>> I've used OSGi enough to understand why Class.forName() is problematic. Some of these uses, however, seem like pretty legitimate dynamic code, for example the assembler subsystem. An OSGi solution to that need might be the OSGi service registry, but that's obviously not useful here.
>>
>> Some of the other uses could be replaced with the use of a plain Java ServiceLoader<T>. I'm not sure what you mean by "Such registrations should instead by done with the java.lang.Class parameters - which can then be used directly." but I think your message was cut off?
>>
>> ---
>> A. Soroka
>> The University of Virginia Library
>>
>> On Sep 10, 2015, at 5:50 PM, Stian Soiland-Reyes <st...@apache.org> wrote:
>>
>>> Last time I looked at interdependencies there were several
>>> Class.forName() calls around in jena-core and jena-arq
>>> - see https://paste.apache.org/5y0W
>>>
>>> Class.forName() depends on the ClassLoader of the caller (by
>>> introspecting the call stack) - but in OSGi there are multiple
>>> ClassLoaders, think of it as one per JAR - and they can only access
>>> packages that are declared as Imports in their META-INF/MANIFEST.MF
>>>
>>>
>>> This would falls apart if the class to be called is not explicitly
>>> included in the OSGi imports. Some of these were for instance with
>>> jena-arq parsers and writers registering themselves with classname in
>>> jena-core - but jena-core can't access jena-arq classes in OSGi
>>> (Although circular imports are technically allowed in OSGi it's not
>>> usually a good idea).
>>>
>>>
>>> Now we have Jena 3, but we still have the duplication between
>>> RDFReaderFImpl in jena-core and IO_JenaReaders in jena-arq - so this
>>> is very much a real problem, because using riot would autoregister its
>>> classnames in RDFReaderFImp. Third-party callers could also be
>>> registering - although RDFReaderFImp is screaming "imp imp" all over
>>> the place, so we should be free to change that.
>>>
>>>
>>> Such registrations should instead by done with the java.lang.Class
>>> parameters - which can then be used directly. The
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>> On 10 September 2015 at 22:50, Stian Soiland-Reyes <st...@apache.org> wrote:
>>>> On 10 September 2015 at 18:13, ajs6f@virginia.edu <aj...@virginia.edu> wrote:
>>>>> If this is a matter of "just a couple of lines in the manifest file" cannot a patch be created to do that in Jena itself? Are there inter-module dependency issues that make that difficult?
>>>>
>>>>
>>>> In theory just setting
>>>>
>>>> <packaging>bundle</packaging>
>>>>
>>>> and using the maven-bundle-plugin
>>>> is enough to auto-generate the correct META-INF metadata for OSGi.
>>>> This can be customized (as we do for the apache-jena-osgi/jena-osgi
>>>> module).
>>>>
>>>> One complication is if the external dependencies are OSGi or not -
>>>> httpclient is one tricky one as it has done the same as Jena and
>>>> provided a separate wrapper httpclient-osgi (and httpcore-osgi) -
>>>> however the way they did this with Maven means that just using it as a
>>>> dependency would still pull in a dependency on the regular httpclient
>>>> library. So if you are a non-OSGi user you would then see the
>>>> httpclient classes in two JARs - which with Maven version resolution
>>>> could easily become in mismatched versions.
>>>>
>>>> Ironically httpclient-osgi does not depend on httpcore-osgi - so the
>>>> one dependency that it truly need isn't stated in its pom.
>>>>
>>>>
>>>> In jena-osgi I therefore excluded all those deeper dependencies:
>>>>
>>>> See https://github.com/apache/jena/blob/master/apache-jena-osgi/jena-osgi/pom.xml#L165
>>>>
>>>> To avoid Jena-Maven-OSGi users the same issue, I similarly here
>>>> slightly misused the <scope>provided</scope> for the dependencies that
>>>> are not to be dependencies of the final jena-osgi JAR, but which are
>>>> shadowed inside.
>>>> https://github.com/apache/jena/blob/master/apache-jena-osgi/jena-osgi/pom.xml#L107
>>>>
>>>>
>>>> If we move to <packaging>bundle</packaging> then we should get
>>>> httpclient folks to sort out their poms upstream so we can rely on
>>>> them in a cleaner fashion across Jena. (or put this exclusion rule
>>>> into <dependencyManagement> of jena-parent) - I don't think
>>>> copy-pasting that big <exclusion> block around anything that directly
>>>> or indirectly requires things like httpclient is good.
>>>>
>>>> Ideally they should also move to <packaging>bundle</packaging> and
>>>> avoid *-osgi, which would of course simplify things.
>>>>
>>>>
>>>> There are other potential issues as Class.forName() which the current
>>>> jena-osgi is narrowly cheating around by effectively making a single
>>>> class loader for all of Jena (including as Reto pointed out, TDB etc)
>>>>
>>>>
>>>>
>>>>
>>>>
>>>> --
>>>> Stian Soiland-Reyes
>>>> Apache Taverna (incubating), Apache Commons RDF (incubating)
>>>> http://orcid.org/0000-0001-9842-9718
>>>
>>>
>>>
>>> --
>>> Stian Soiland-Reyes
>>> Apache Taverna (incubating), Apache Commons RDF (incubating)
>>> http://orcid.org/0000-0001-9842-9718
>>
>
>
>
Re: Jena OSGi (was: [] 201508 Release of 23 Clerezza modules)
Posted by Stian Soiland-Reyes <st...@apache.org>.
Agree that for the assembler subsystem you would need something like
ServiceLoader and a Class.forName() fallback. It is mainly the
parser/writer APIs I am concerned about.
The classloaders here are already an issue if trying to add a custom
plugin to a Fuseki WAR install (this was a users@ question earlier in
the year I think).
On 11 September 2015 at 12:44, ajs6f@virginia.edu <aj...@virginia.edu> wrote:
> I've used OSGi enough to understand why Class.forName() is problematic. Some of these uses, however, seem like pretty legitimate dynamic code, for example the assembler subsystem. An OSGi solution to that need might be the OSGi service registry, but that's obviously not useful here.
>
> Some of the other uses could be replaced with the use of a plain Java ServiceLoader<T>. I'm not sure what you mean by "Such registrations should instead by done with the java.lang.Class parameters - which can then be used directly." but I think your message was cut off?
>
> ---
> A. Soroka
> The University of Virginia Library
>
> On Sep 10, 2015, at 5:50 PM, Stian Soiland-Reyes <st...@apache.org> wrote:
>
>> Last time I looked at interdependencies there were several
>> Class.forName() calls around in jena-core and jena-arq
>> - see https://paste.apache.org/5y0W
>>
>> Class.forName() depends on the ClassLoader of the caller (by
>> introspecting the call stack) - but in OSGi there are multiple
>> ClassLoaders, think of it as one per JAR - and they can only access
>> packages that are declared as Imports in their META-INF/MANIFEST.MF
>>
>>
>> This would falls apart if the class to be called is not explicitly
>> included in the OSGi imports. Some of these were for instance with
>> jena-arq parsers and writers registering themselves with classname in
>> jena-core - but jena-core can't access jena-arq classes in OSGi
>> (Although circular imports are technically allowed in OSGi it's not
>> usually a good idea).
>>
>>
>> Now we have Jena 3, but we still have the duplication between
>> RDFReaderFImpl in jena-core and IO_JenaReaders in jena-arq - so this
>> is very much a real problem, because using riot would autoregister its
>> classnames in RDFReaderFImp. Third-party callers could also be
>> registering - although RDFReaderFImp is screaming "imp imp" all over
>> the place, so we should be free to change that.
>>
>>
>> Such registrations should instead by done with the java.lang.Class
>> parameters - which can then be used directly. The
>>
>>
>>
>>
>>
>>
>>
>> On 10 September 2015 at 22:50, Stian Soiland-Reyes <st...@apache.org> wrote:
>>> On 10 September 2015 at 18:13, ajs6f@virginia.edu <aj...@virginia.edu> wrote:
>>>> If this is a matter of "just a couple of lines in the manifest file" cannot a patch be created to do that in Jena itself? Are there inter-module dependency issues that make that difficult?
>>>
>>>
>>> In theory just setting
>>>
>>> <packaging>bundle</packaging>
>>>
>>> and using the maven-bundle-plugin
>>> is enough to auto-generate the correct META-INF metadata for OSGi.
>>> This can be customized (as we do for the apache-jena-osgi/jena-osgi
>>> module).
>>>
>>> One complication is if the external dependencies are OSGi or not -
>>> httpclient is one tricky one as it has done the same as Jena and
>>> provided a separate wrapper httpclient-osgi (and httpcore-osgi) -
>>> however the way they did this with Maven means that just using it as a
>>> dependency would still pull in a dependency on the regular httpclient
>>> library. So if you are a non-OSGi user you would then see the
>>> httpclient classes in two JARs - which with Maven version resolution
>>> could easily become in mismatched versions.
>>>
>>> Ironically httpclient-osgi does not depend on httpcore-osgi - so the
>>> one dependency that it truly need isn't stated in its pom.
>>>
>>>
>>> In jena-osgi I therefore excluded all those deeper dependencies:
>>>
>>> See https://github.com/apache/jena/blob/master/apache-jena-osgi/jena-osgi/pom.xml#L165
>>>
>>> To avoid Jena-Maven-OSGi users the same issue, I similarly here
>>> slightly misused the <scope>provided</scope> for the dependencies that
>>> are not to be dependencies of the final jena-osgi JAR, but which are
>>> shadowed inside.
>>> https://github.com/apache/jena/blob/master/apache-jena-osgi/jena-osgi/pom.xml#L107
>>>
>>>
>>> If we move to <packaging>bundle</packaging> then we should get
>>> httpclient folks to sort out their poms upstream so we can rely on
>>> them in a cleaner fashion across Jena. (or put this exclusion rule
>>> into <dependencyManagement> of jena-parent) - I don't think
>>> copy-pasting that big <exclusion> block around anything that directly
>>> or indirectly requires things like httpclient is good.
>>>
>>> Ideally they should also move to <packaging>bundle</packaging> and
>>> avoid *-osgi, which would of course simplify things.
>>>
>>>
>>> There are other potential issues as Class.forName() which the current
>>> jena-osgi is narrowly cheating around by effectively making a single
>>> class loader for all of Jena (including as Reto pointed out, TDB etc)
>>>
>>>
>>>
>>>
>>>
>>> --
>>> Stian Soiland-Reyes
>>> Apache Taverna (incubating), Apache Commons RDF (incubating)
>>> http://orcid.org/0000-0001-9842-9718
>>
>>
>>
>> --
>> Stian Soiland-Reyes
>> Apache Taverna (incubating), Apache Commons RDF (incubating)
>> http://orcid.org/0000-0001-9842-9718
>
--
Stian Soiland-Reyes
Apache Taverna (incubating), Apache Commons RDF (incubating)
http://orcid.org/0000-0001-9842-9718
Re: Jena OSGi (was: [] 201508 Release of 23 Clerezza modules)
Posted by "ajs6f@virginia.edu" <aj...@virginia.edu>.
I've used OSGi enough to understand why Class.forName() is problematic. Some of these uses, however, seem like pretty legitimate dynamic code, for example the assembler subsystem. An OSGi solution to that need might be the OSGi service registry, but that's obviously not useful here.
Some of the other uses could be replaced with the use of a plain Java ServiceLoader<T>. I'm not sure what you mean by "Such registrations should instead by done with the java.lang.Class parameters - which can then be used directly." but I think your message was cut off?
---
A. Soroka
The University of Virginia Library
On Sep 10, 2015, at 5:50 PM, Stian Soiland-Reyes <st...@apache.org> wrote:
> Last time I looked at interdependencies there were several
> Class.forName() calls around in jena-core and jena-arq
> - see https://paste.apache.org/5y0W
>
> Class.forName() depends on the ClassLoader of the caller (by
> introspecting the call stack) - but in OSGi there are multiple
> ClassLoaders, think of it as one per JAR - and they can only access
> packages that are declared as Imports in their META-INF/MANIFEST.MF
>
>
> This would falls apart if the class to be called is not explicitly
> included in the OSGi imports. Some of these were for instance with
> jena-arq parsers and writers registering themselves with classname in
> jena-core - but jena-core can't access jena-arq classes in OSGi
> (Although circular imports are technically allowed in OSGi it's not
> usually a good idea).
>
>
> Now we have Jena 3, but we still have the duplication between
> RDFReaderFImpl in jena-core and IO_JenaReaders in jena-arq - so this
> is very much a real problem, because using riot would autoregister its
> classnames in RDFReaderFImp. Third-party callers could also be
> registering - although RDFReaderFImp is screaming "imp imp" all over
> the place, so we should be free to change that.
>
>
> Such registrations should instead by done with the java.lang.Class
> parameters - which can then be used directly. The
>
>
>
>
>
>
>
> On 10 September 2015 at 22:50, Stian Soiland-Reyes <st...@apache.org> wrote:
>> On 10 September 2015 at 18:13, ajs6f@virginia.edu <aj...@virginia.edu> wrote:
>>> If this is a matter of "just a couple of lines in the manifest file" cannot a patch be created to do that in Jena itself? Are there inter-module dependency issues that make that difficult?
>>
>>
>> In theory just setting
>>
>> <packaging>bundle</packaging>
>>
>> and using the maven-bundle-plugin
>> is enough to auto-generate the correct META-INF metadata for OSGi.
>> This can be customized (as we do for the apache-jena-osgi/jena-osgi
>> module).
>>
>> One complication is if the external dependencies are OSGi or not -
>> httpclient is one tricky one as it has done the same as Jena and
>> provided a separate wrapper httpclient-osgi (and httpcore-osgi) -
>> however the way they did this with Maven means that just using it as a
>> dependency would still pull in a dependency on the regular httpclient
>> library. So if you are a non-OSGi user you would then see the
>> httpclient classes in two JARs - which with Maven version resolution
>> could easily become in mismatched versions.
>>
>> Ironically httpclient-osgi does not depend on httpcore-osgi - so the
>> one dependency that it truly need isn't stated in its pom.
>>
>>
>> In jena-osgi I therefore excluded all those deeper dependencies:
>>
>> See https://github.com/apache/jena/blob/master/apache-jena-osgi/jena-osgi/pom.xml#L165
>>
>> To avoid Jena-Maven-OSGi users the same issue, I similarly here
>> slightly misused the <scope>provided</scope> for the dependencies that
>> are not to be dependencies of the final jena-osgi JAR, but which are
>> shadowed inside.
>> https://github.com/apache/jena/blob/master/apache-jena-osgi/jena-osgi/pom.xml#L107
>>
>>
>> If we move to <packaging>bundle</packaging> then we should get
>> httpclient folks to sort out their poms upstream so we can rely on
>> them in a cleaner fashion across Jena. (or put this exclusion rule
>> into <dependencyManagement> of jena-parent) - I don't think
>> copy-pasting that big <exclusion> block around anything that directly
>> or indirectly requires things like httpclient is good.
>>
>> Ideally they should also move to <packaging>bundle</packaging> and
>> avoid *-osgi, which would of course simplify things.
>>
>>
>> There are other potential issues as Class.forName() which the current
>> jena-osgi is narrowly cheating around by effectively making a single
>> class loader for all of Jena (including as Reto pointed out, TDB etc)
>>
>>
>>
>>
>>
>> --
>> Stian Soiland-Reyes
>> Apache Taverna (incubating), Apache Commons RDF (incubating)
>> http://orcid.org/0000-0001-9842-9718
>
>
>
> --
> Stian Soiland-Reyes
> Apache Taverna (incubating), Apache Commons RDF (incubating)
> http://orcid.org/0000-0001-9842-9718
Re: Jena OSGi (was: [] 201508 Release of 23 Clerezza modules)
Posted by Stian Soiland-Reyes <st...@apache.org>.
Last time I looked at interdependencies there were several
Class.forName() calls around in jena-core and jena-arq
- see https://paste.apache.org/5y0W
Class.forName() depends on the ClassLoader of the caller (by
introspecting the call stack) - but in OSGi there are multiple
ClassLoaders, think of it as one per JAR - and they can only access
packages that are declared as Imports in their META-INF/MANIFEST.MF
This would falls apart if the class to be called is not explicitly
included in the OSGi imports. Some of these were for instance with
jena-arq parsers and writers registering themselves with classname in
jena-core - but jena-core can't access jena-arq classes in OSGi
(Although circular imports are technically allowed in OSGi it's not
usually a good idea).
Now we have Jena 3, but we still have the duplication between
RDFReaderFImpl in jena-core and IO_JenaReaders in jena-arq - so this
is very much a real problem, because using riot would autoregister its
classnames in RDFReaderFImp. Third-party callers could also be
registering - although RDFReaderFImp is screaming "imp imp" all over
the place, so we should be free to change that.
Such registrations should instead by done with the java.lang.Class
parameters - which can then be used directly. The
On 10 September 2015 at 22:50, Stian Soiland-Reyes <st...@apache.org> wrote:
> On 10 September 2015 at 18:13, ajs6f@virginia.edu <aj...@virginia.edu> wrote:
>> If this is a matter of "just a couple of lines in the manifest file" cannot a patch be created to do that in Jena itself? Are there inter-module dependency issues that make that difficult?
>
>
> In theory just setting
>
> <packaging>bundle</packaging>
>
> and using the maven-bundle-plugin
> is enough to auto-generate the correct META-INF metadata for OSGi.
> This can be customized (as we do for the apache-jena-osgi/jena-osgi
> module).
>
> One complication is if the external dependencies are OSGi or not -
> httpclient is one tricky one as it has done the same as Jena and
> provided a separate wrapper httpclient-osgi (and httpcore-osgi) -
> however the way they did this with Maven means that just using it as a
> dependency would still pull in a dependency on the regular httpclient
> library. So if you are a non-OSGi user you would then see the
> httpclient classes in two JARs - which with Maven version resolution
> could easily become in mismatched versions.
>
> Ironically httpclient-osgi does not depend on httpcore-osgi - so the
> one dependency that it truly need isn't stated in its pom.
>
>
> In jena-osgi I therefore excluded all those deeper dependencies:
>
> See https://github.com/apache/jena/blob/master/apache-jena-osgi/jena-osgi/pom.xml#L165
>
> To avoid Jena-Maven-OSGi users the same issue, I similarly here
> slightly misused the <scope>provided</scope> for the dependencies that
> are not to be dependencies of the final jena-osgi JAR, but which are
> shadowed inside.
> https://github.com/apache/jena/blob/master/apache-jena-osgi/jena-osgi/pom.xml#L107
>
>
> If we move to <packaging>bundle</packaging> then we should get
> httpclient folks to sort out their poms upstream so we can rely on
> them in a cleaner fashion across Jena. (or put this exclusion rule
> into <dependencyManagement> of jena-parent) - I don't think
> copy-pasting that big <exclusion> block around anything that directly
> or indirectly requires things like httpclient is good.
>
> Ideally they should also move to <packaging>bundle</packaging> and
> avoid *-osgi, which would of course simplify things.
>
>
> There are other potential issues as Class.forName() which the current
> jena-osgi is narrowly cheating around by effectively making a single
> class loader for all of Jena (including as Reto pointed out, TDB etc)
>
>
>
>
>
> --
> Stian Soiland-Reyes
> Apache Taverna (incubating), Apache Commons RDF (incubating)
> http://orcid.org/0000-0001-9842-9718
--
Stian Soiland-Reyes
Apache Taverna (incubating), Apache Commons RDF (incubating)
http://orcid.org/0000-0001-9842-9718