You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@daffodil.apache.org by Mike Beckerle <mb...@apache.org> on 2022/03/31 16:57:09 UTC
Re: OGSi was: Re: "org.apache.daffodil.api" split package conflict
Taking this thread over to our dev list.
Please sanity check this notion on how to readily fix the conflict from
packages that are split across jars.
Consider a module such as daffodil-lib. It contains code in package
org.apache.daffodil.api today.
Some of this code is written by hand, other code is generated by a code
generator.
We simply insert a package in the chain so that all this code is now in
package org.apache.daffodil.lib.api
Other modules similarly grow a module-named package tier. E.g.,
daffodil-runtime1 all packages get put under org.apache.daffodil.runtime1.
Modules would then no longer be sharing packages. All the package nests
become 1 step deeper.
I know this will break quite a few things that are depending on internal
APIs right now, but hey, those are depending on internal APIs.
Is the needed change as simple as this?
On Thu, Mar 31, 2022 at 11:01 AM Martin Lichtin <li...@yahoo.com> wrote:
> Thanks for adding me to the thread.
>
> Regarding your last comment, let's ignore services, this is a higher-level
> feature that we do not need to discuss in the context of the Daffodil
> library. As you say, Daffodil is just a library (a very useful one, btw)
> and that's ok, no need to change anything in this regard. It's also fine to
> split the code into several JARs, no problem there.
>
> The only issue I'm trying to address is the split-package that I'm seeing
> for package "org.apache.daffodil.api" (and perhaps others as you mention).
> Classes of this package reside in more than one JAR and that's a problem
> with regards to modularity (see also Java 9 modules where this is not
> allowed either, for example
> https://docs.microsoft.com/en-us/java/openjdk/transition-from-java-8-to-java-11#noclassdeffounderror-caused-by-split-packages
> ).
>
> Think of a package as being _exported_ by one (and only one) JAR and a
> class loader can therefore remember this upfront and doesn't need to search
> the complete class path to find a class at run-time.
>
> I haven't yet looked at org.apache.daffodil.processors, but if you using
> sub-packages for each converter, such as org.apache.daffodil.processors.a,
> org.apache.daffodil.processors.b, org.apache.daffodil.processors.c,
> org.apache.daffodil.processors.test then "a", "b", "c", "test" can
> certainly reside in separate JARs, that again is not a problem. It's only a
> problem if the split is on the same package level, e.g. if class
> org.apache.daffodil.processors.A.class and
> org.apache.daffodil.processors.B.class would not be packaged in the same
> JAR.
>
> Sorry for the lengthy mail, it all may seem a bit strange, but the concept
> is around for years and it really helps with modular programming. Just as
> an exercise, check the MANIFEST.MF of some of your favorite open-source
> JARs, most in fact are built as bundles, and have Import- and
> Export-Package declarations to support modular container runtimes.
>
> - Martin
>
>
> On 31/03/2022 16:17, Mike Beckerle wrote:
>
> (resend making sure M Lichtin is on this thread)
>
> It seems that an OGSi module is supposed to provide a "service" which can
> be activated.
>
> Our daffodil jars are nothing like organized as separate service
> providers. They're more ad-hoc libraries of code with shared concerns.
>
> A sensible "service" from daffodil would involve running code from several
> of the jars.
>
> Can you comment on this - when a sensible OGSI service requires code that
> is distributed across many jars?
>
>
>> On Thu, Mar 31, 2022 at 10:03 AM Mike Beckerle <mb...@apache.org>
>> wrote:
>>
>>> Created https://issues.apache.org/jira/browse/DAFFODIL-2683 to track
>>> this issue.
>>>
>>>
>>>
>>> On Thu, Mar 31, 2022 at 9:55 AM Mike Beckerle <mb...@apache.org>
>>> wrote:
>>>
>>>> So, it seems there is, in general, an assumption in OGSi that a package
>>>> does not get contributions from multiple jar files?
>>>>
>>>> I expect that the org.apache.daffodil.api conflict is only the first of
>>>> many such conflicts you would hit. Several of our packages are split across
>>>> the jars. Participation in a package is kind of orthogonal to presence in a
>>>> jar in Daffodil right now.
>>>>
>>>> org.apache.daffodil.processors is split across 5 modules. 6 if you
>>>> count the test code.
>>>>
>>>> This can of course be fixed but this is our first experience with this
>>>> requirement.
>>>>
>>>> I will create a JIRA ticket for this.
>>>>
>>>> In the mean time, I'm not sure what to suggest as a workaround. Perhaps
>>>> you have to unjar everything, put all the files in a common directory tree,
>>>> and re-jar it all?
>>>>
>>>>
>>>> On Thu, Mar 31, 2022 at 1:42 AM Martin Lichtin <li...@yahoo.com>
>>>> wrote:
>>>>
>>>>> Hi
>>>>>
>>>>> Trying to run Daffodil inside Apache Karaf (OSGi container) I noticed
>>>>> that the Daffodil JARs are not built as bundles. That's not a problem,
>>>>> they can be bundle'ized on the fly.
>>>>>
>>>>> However, what's an issue is that two JARs contain the same package.
>>>>> Both
>>>>> daffodil-runtime1_2.12 and daffodil-lib_2.12 expose
>>>>> "org.apache.daffodil.api" and therefore this package is split and
>>>>> causes
>>>>> a conflict.
>>>>>
>>>>> Could perhaps this package moved into an "api" JAR, or one of the two
>>>>> JARs renames it such that there's no longer a split-package situation.
>>>>>
>>>>> - Martin
>>>>>
>>>>>
>>>>>
Re: OGSi was: Re: "org.apache.daffodil.api" split package conflict
Posted by Martin Lichtin <li...@yahoo.com.INVALID>.
Yes such related classes can be split in separate jars (assuming
base/interface class is not residing in the same package).
On 31/03/2022 20:05, Mike Beckerle wrote:
> A specific question: what about class to base-class relationships? Can
> a base-class or java interface be in a separate jar/module?
>
> On Thu, Mar 31, 2022 at 12:57 PM Mike Beckerle <mb...@apache.org>
> wrote:
>
> Taking this thread over to our dev list.
>
> Please sanity check this notion on how to readily fix the conflict
> from packages that are split across jars.
>
> Consider a module such as daffodil-lib. It contains code in
> package org.apache.daffodil.api today.
>
> Some of this code is written by hand, other code is generated by a
> code generator.
>
> We simply insert a package in the chain so that all this code is
> now in package org.apache.daffodil.lib.api
>
> Other modules similarly grow a module-named package tier. E.g.,
> daffodil-runtime1 all packages get put under
> org.apache.daffodil.runtime1.
>
> Modules would then no longer be sharing packages. All the package
> nests become 1 step deeper.
>
> I know this will break quite a few things that are depending on
> internal APIs right now, but hey, those are depending on internal
> APIs.
>
> Is the needed change as simple as this?
>
>
>
>
>
> On Thu, Mar 31, 2022 at 11:01 AM Martin Lichtin
> <li...@yahoo.com> wrote:
>
> Thanks for adding me to the thread.
>
> Regarding your last comment, let's ignore services, this is a
> higher-level feature that we do not need to discuss in the
> context of the Daffodil library. As you say, Daffodil is just
> a library (a very useful one, btw) and that's ok, no need to
> change anything in this regard. It's also fine to split the
> code into several JARs, no problem there.
>
> The only issue I'm trying to address is the split-package that
> I'm seeing for package "org.apache.daffodil.api" (and perhaps
> others as you mention). Classes of this package reside in more
> than one JAR and that's a problem with regards to modularity
> (see also Java 9 modules where this is not allowed either, for
> example
> https://docs.microsoft.com/en-us/java/openjdk/transition-from-java-8-to-java-11#noclassdeffounderror-caused-by-split-packages).
>
> Think of a package as being _exported_ by one (and only one)
> JAR and a class loader can therefore remember this upfront and
> doesn't need to search the complete class path to find a class
> at run-time.
>
> I haven't yet looked at org.apache.daffodil.processors, but if
> you using sub-packages for each converter, such as
> org.apache.daffodil.processors.a,
> org.apache.daffodil.processors.b,
> org.apache.daffodil.processors.c,
> org.apache.daffodil.processors.test then "a", "b", "c", "test"
> can certainly reside in separate JARs, that again is not a
> problem. It's only a problem if the split is on the same
> package level, e.g. if class
> org.apache.daffodil.processors.A.class and
> org.apache.daffodil.processors.B.class would not be packaged
> in the same JAR.
>
> Sorry for the lengthy mail, it all may seem a bit strange, but
> the concept is around for years and it really helps with
> modular programming. Just as an exercise, check the
> MANIFEST.MF of some of your favorite open-source JARs, most in
> fact are built as bundles, and have Import- and Export-Package
> declarations to support modular container runtimes.
>
> - Martin
>
>
> On 31/03/2022 16:17, Mike Beckerle wrote:
>> (resend making sure M Lichtin is on this thread)
>>
>> It seems that an OGSi module is supposed to provide a
>> "service" which can be activated.
>>
>> Our daffodil jars are nothing like organized as separate
>> service providers. They're more ad-hoc libraries of code with
>> shared concerns.
>>
>> A sensible "service" from daffodil would involve running code
>> from several of the jars.
>>
>> Can you comment on this - when a sensible OGSI service
>> requires code that is distributed across many jars?
>>
>>
>> On Thu, Mar 31, 2022 at 10:03 AM Mike Beckerle
>> <mb...@apache.org> wrote:
>>
>> Created
>> https://issues.apache.org/jira/browse/DAFFODIL-2683
>> to track this issue.
>>
>>
>>
>> On Thu, Mar 31, 2022 at 9:55 AM Mike Beckerle
>> <mb...@apache.org> wrote:
>>
>> So, it seems there is, in general, an assumption
>> in OGSi that a package does not get contributions
>> from multiple jar files?
>>
>> I expect that the org.apache.daffodil.api
>> conflict is only the first of many such conflicts
>> you would hit. Several of our packages are split
>> across the jars. Participation in a package is
>> kind of orthogonal to presence in a jar in
>> Daffodil right now.
>>
>> org.apache.daffodil.processors is split across 5
>> modules. 6 if you count the test code.
>>
>> This can of course be fixed but this is our first
>> experience with this requirement.
>>
>> I will create a JIRA ticket for this.
>>
>> In the mean time, I'm not sure what to suggest as
>> a workaround. Perhaps you have to unjar
>> everything, put all the files in a common
>> directory tree, and re-jar it all?
>>
>>
>> On Thu, Mar 31, 2022 at 1:42 AM Martin Lichtin
>> <li...@yahoo.com> wrote:
>>
>> Hi
>>
>> Trying to run Daffodil inside Apache Karaf
>> (OSGi container) I noticed
>> that the Daffodil JARs are not built as
>> bundles. That's not a problem,
>> they can be bundle'ized on the fly.
>>
>> However, what's an issue is that two JARs
>> contain the same package. Both
>> daffodil-runtime1_2.12 and daffodil-lib_2.12
>> expose
>> "org.apache.daffodil.api" and therefore this
>> package is split and causes
>> a conflict.
>>
>> Could perhaps this package moved into an
>> "api" JAR, or one of the two
>> JARs renames it such that there's no longer a
>> split-package situation.
>>
>> - Martin
>>
>>
Re: OGSi was: Re: "org.apache.daffodil.api" split package conflict
Posted by Mike Beckerle <mb...@apache.org>.
A specific question: what about class to base-class relationships? Can a
base-class or java interface be in a separate jar/module?
On Thu, Mar 31, 2022 at 12:57 PM Mike Beckerle <mb...@apache.org> wrote:
> Taking this thread over to our dev list.
>
> Please sanity check this notion on how to readily fix the conflict from
> packages that are split across jars.
>
> Consider a module such as daffodil-lib. It contains code in package
> org.apache.daffodil.api today.
>
> Some of this code is written by hand, other code is generated by a code
> generator.
>
> We simply insert a package in the chain so that all this code is now in
> package org.apache.daffodil.lib.api
>
> Other modules similarly grow a module-named package tier. E.g.,
> daffodil-runtime1 all packages get put under org.apache.daffodil.runtime1.
>
> Modules would then no longer be sharing packages. All the package nests
> become 1 step deeper.
>
> I know this will break quite a few things that are depending on internal
> APIs right now, but hey, those are depending on internal APIs.
>
> Is the needed change as simple as this?
>
>
>
>
>
> On Thu, Mar 31, 2022 at 11:01 AM Martin Lichtin <li...@yahoo.com> wrote:
>
>> Thanks for adding me to the thread.
>>
>> Regarding your last comment, let's ignore services, this is a
>> higher-level feature that we do not need to discuss in the context of the
>> Daffodil library. As you say, Daffodil is just a library (a very useful
>> one, btw) and that's ok, no need to change anything in this regard. It's
>> also fine to split the code into several JARs, no problem there.
>>
>> The only issue I'm trying to address is the split-package that I'm seeing
>> for package "org.apache.daffodil.api" (and perhaps others as you mention).
>> Classes of this package reside in more than one JAR and that's a problem
>> with regards to modularity (see also Java 9 modules where this is not
>> allowed either, for example
>> https://docs.microsoft.com/en-us/java/openjdk/transition-from-java-8-to-java-11#noclassdeffounderror-caused-by-split-packages
>> ).
>>
>> Think of a package as being _exported_ by one (and only one) JAR and a
>> class loader can therefore remember this upfront and doesn't need to search
>> the complete class path to find a class at run-time.
>>
>> I haven't yet looked at org.apache.daffodil.processors, but if you using
>> sub-packages for each converter, such as org.apache.daffodil.processors.a,
>> org.apache.daffodil.processors.b, org.apache.daffodil.processors.c,
>> org.apache.daffodil.processors.test then "a", "b", "c", "test" can
>> certainly reside in separate JARs, that again is not a problem. It's only a
>> problem if the split is on the same package level, e.g. if class
>> org.apache.daffodil.processors.A.class and
>> org.apache.daffodil.processors.B.class would not be packaged in the same
>> JAR.
>>
>> Sorry for the lengthy mail, it all may seem a bit strange, but the
>> concept is around for years and it really helps with modular programming.
>> Just as an exercise, check the MANIFEST.MF of some of your favorite
>> open-source JARs, most in fact are built as bundles, and have Import- and
>> Export-Package declarations to support modular container runtimes.
>>
>> - Martin
>>
>>
>> On 31/03/2022 16:17, Mike Beckerle wrote:
>>
>> (resend making sure M Lichtin is on this thread)
>>
>> It seems that an OGSi module is supposed to provide a "service" which can
>> be activated.
>>
>> Our daffodil jars are nothing like organized as separate service
>> providers. They're more ad-hoc libraries of code with shared concerns.
>>
>> A sensible "service" from daffodil would involve running code from
>> several of the jars.
>>
>> Can you comment on this - when a sensible OGSI service requires code that
>> is distributed across many jars?
>>
>>
>>> On Thu, Mar 31, 2022 at 10:03 AM Mike Beckerle <mb...@apache.org>
>>> wrote:
>>>
>>>> Created https://issues.apache.org/jira/browse/DAFFODIL-2683 to track
>>>> this issue.
>>>>
>>>>
>>>>
>>>> On Thu, Mar 31, 2022 at 9:55 AM Mike Beckerle <mb...@apache.org>
>>>> wrote:
>>>>
>>>>> So, it seems there is, in general, an assumption in OGSi that a
>>>>> package does not get contributions from multiple jar files?
>>>>>
>>>>> I expect that the org.apache.daffodil.api conflict is only the first
>>>>> of many such conflicts you would hit. Several of our packages are split
>>>>> across the jars. Participation in a package is kind of orthogonal to
>>>>> presence in a jar in Daffodil right now.
>>>>>
>>>>> org.apache.daffodil.processors is split across 5 modules. 6 if you
>>>>> count the test code.
>>>>>
>>>>> This can of course be fixed but this is our first experience with this
>>>>> requirement.
>>>>>
>>>>> I will create a JIRA ticket for this.
>>>>>
>>>>> In the mean time, I'm not sure what to suggest as a workaround.
>>>>> Perhaps you have to unjar everything, put all the files in a common
>>>>> directory tree, and re-jar it all?
>>>>>
>>>>>
>>>>> On Thu, Mar 31, 2022 at 1:42 AM Martin Lichtin <li...@yahoo.com>
>>>>> wrote:
>>>>>
>>>>>> Hi
>>>>>>
>>>>>> Trying to run Daffodil inside Apache Karaf (OSGi container) I noticed
>>>>>> that the Daffodil JARs are not built as bundles. That's not a
>>>>>> problem,
>>>>>> they can be bundle'ized on the fly.
>>>>>>
>>>>>> However, what's an issue is that two JARs contain the same package.
>>>>>> Both
>>>>>> daffodil-runtime1_2.12 and daffodil-lib_2.12 expose
>>>>>> "org.apache.daffodil.api" and therefore this package is split and
>>>>>> causes
>>>>>> a conflict.
>>>>>>
>>>>>> Could perhaps this package moved into an "api" JAR, or one of the two
>>>>>> JARs renames it such that there's no longer a split-package situation.
>>>>>>
>>>>>> - Martin
>>>>>>
>>>>>>
>>>>>>