You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@felix.apache.org by "A. Rothman" <am...@amichais.net> on 2013/06/24 11:35:24 UTC
Bundleall replacement
Hi,
Following the Maven Bundle Plugin's deprecation/removal of the bundleall
goal I was pointed to the option of creating an uber-bundle for my
project's dependencies (see
https://issues.apache.org/jira/browse/FELIX-4145). This took
considerable effort compared to using the bundleall goal, and I'd like
to raise some thoughts about the process - I don't know if these are
bugs, features, or my own misunderstanding, but I hope they can be used
to improve the migration experience for anyone else who currently uses
the bundleall/wrap goals and will shortly be required to migrate away
from them.
What I did in a nutshell: I created a project for the uber-bundle, with
the dependencies being the packages it will need to include (all with
explicit versions inherited from the parent pom's DependencyManagement).
The new project's version is currently 0.1-SNAPSHOT. I'm using the
Embed-Dependency configuration along with Embed-Transitive set to true
to hand-pick the dependencies and transitive dependencies that I need to
add to the bundle. The project contains nothing other than its pom (no
real sources or resources).
1. In the generated bundle, the generated Export-Package header in the
manifest specifies the version of the exported packages as version
0.1.SNAPSHOT. When using the bundleall goal it would correctly take the
artifact version and use that in the exports version, but now this
information seems lost, and I have to manually go over all the exports
and specify their respective versions in the _exportcontents
configuration. Is this a bug? Why not use maven's existing version
information for these dependencies and save all the tedious work,
duplicate/error-prone versioning info? (btw one of the dependencies
already had the OSGi headers - in this case the dependency's export
version seems to have been copied over correctly to the uber-bundle's
exports).
2. The uber-bundle's Export-Package header seems to include non-package
resources such as the dependency jar names themselves, a License.txt
file it probably picked up from one of the bundles, etc. The maven build
actually gives a warning, e.g. "Invalid package name: 'guava-14.0.jar'".
I ended up adding !*.jar,!*.xml,!*.txt to the _exportcontents
configuration to prevent the warnings and incorrect exports header. Is
this a bug or a feature?
3. Similarly, when I try to set Embed-StripGroup to false, the jars are
placed ain group-named directories, but then the directory names
themselves are added to the exports header (without the jar name). One
of the group names happened to contain a hyphen, which resulted in an
"Invalid package name" header as well. I added an additional !*-* to the
exportcontents configuration to prevent this as well. Again, a lot of
unnecessary and tedious troubleshooting.
4. One of the embedded dependencies already had OSGi headers, and while
the export info (e.g. version, see #1) seemed to be copied over
correctly, the imports, and specifically the "resolution:=optional"
parameter, was not - this meanse I need to manually go over all imports
of the bundles and explicit copy over this parameter into the
uber-bundle's Import-Package configuration for all optional imports of
all dependency bundles. Very tedious and unnecessary. A bug?
5. By default the embedded dependencies are not exported by the
uber-budnle, until Exported-Package is specified. It is natural in many
cases (particularly when migrating from bundleall) to want to specify *
and export everything as before. However, as I eventually found in the
documentation, the default behavior is to then embed everything twice in
the bundle - one in the jars, and once inlined, and Export-Package needs
to be replaced with _exportcontents in order to get the desired result.
I don't know what the use case is where one would want to include
everything twice, but perhaps this should at least not be made the
default behavior?
In summary, what I had to do to get the desired uber-bundle is create
the new project, add its dependencies, set Embed-Transitive to true,
specify the actual jars to embed in Embed-Dependency (up to here this
makes sense), generate the uber-jar, figure out why I'm getting warning
about jar names appearing as invalid package names, add hacky globs to
Export-Package to prevent them, (I took the detour of trying
Embed-StripGroup as well and doubling the invalid package name effort),
generate the uber-jar again, manually go over the exports in its
manifest looking for all those with the wrong version, find the correct
version for each and specify the package+version manually in
Export-Package, go over all imports in all OSGi-ready embedded
dependencies and specify the package+resolution:=optional parameter
manually in Import-Package for all those with an optional resolution,
move all of Export-Package into _exportcontents once I figured it needs
to be done, and finally arrive in a working bundle (I hope :-) ). That's
a lot of work that seems unnecessary, error-prone and difficult to
maintain when something changes in the future, and took even more time
to try googling, asking, reading documentation and experimenting with
until I figured out what needs to be done.
All that being said, I hope this feedback can be used to improve the
experience for the next guy or gal that'll need to migrate away from the
bundleall goal and save them a lot of time and effort. At the very
least, there should be configurable options to do these things
automatically, or even a "bundleall" configuration option that does what
one would expect (embed jars, retain embedded osgi headers and assign
proper version numbers, remove duplication and unnecessary included
resources, etc.). Anyone wishing to further tweak or override these can
of course already do that.
I'll only add another little suggestion:
6. It would be nice if the helpful original response I got in
FELIX-4145, and possibly some more examples specific to bundleall
migration and overcoming some of the above issues, would be documented
and linked from the wrap/bundleall documentation pages, right near the
deprecation notice - that's the first place many people would look when
trying to figure out how they can move past the deprecation and figure
out what the new recommended solution is.
I realize some of this hard work may have been due to my own
misunderstandings, but that can happen to the next guy as well, so any
way to save him the trouble would be a blessing.
Thanks, and I hope this has been useful to someone :-)
Amichai
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@felix.apache.org
For additional commands, e-mail: users-help@felix.apache.org
Re: Bundleall replacement
Posted by Stuart McCulloch <mc...@gmail.com>.
Hi Amichai,
On 1 Aug 2013, at 10:21, A. Rothman wrote:
> Should I open a JIRA ticket for this? Or multiple tickets? Are these confirmed bugs or is something by design?
First note that the maven-bundle-plugin is effectively a wrapper around the bnd library (https://github.com/bndtools/bnd) which itself has no implicit knowledge of Maven.
The plugin tries to automate as much as possible by translating (standard) Maven configuration to bnd instructions, but there will be some inconsistencies / mismatch along the way.
> On 06/24/2013 12:35 PM, A. Rothman wrote:
>> Hi,
>>
>> Following the Maven Bundle Plugin's deprecation/removal of the bundleall goal I was pointed to the option of creating an uber-bundle for my project's dependencies (see https://issues.apache.org/jira/browse/FELIX-4145). This took considerable effort compared to using the bundleall goal, and I'd like to raise some thoughts about the process - I don't know if these are bugs, features, or my own misunderstanding, but I hope they can be used to improve the migration experience for anyone else who currently uses the bundleall/wrap goals and will shortly be required to migrate away from them.
>>
>> What I did in a nutshell: I created a project for the uber-bundle, with the dependencies being the packages it will need to include (all with explicit versions inherited from the parent pom's DependencyManagement). The new project's version is currently 0.1-SNAPSHOT. I'm using the Embed-Dependency configuration along with Embed-Transitive set to true to hand-pick the dependencies and transitive dependencies that I need to add to the bundle. The project contains nothing other than its pom (no real sources or resources).
>>
>> 1. In the generated bundle, the generated Export-Package header in the manifest specifies the version of the exported packages as version 0.1.SNAPSHOT. When using the bundleall goal it would correctly take the artifact version and use that in the exports version, but now this information seems lost, and I have to manually go over all the exports and specify their respective versions in the _exportcontents configuration. Is this a bug?
For the bundle goal the default package version (when no version information is available) is taken from the project version - some people think the default should be empty, others prefer the current default. The bnd library will look for additional OSGi package information on the classpath, but it has no visibility of Maven metadata and the bundleplugin doesn't pass this information along at the moment.
>> Why not use maven's existing version information for these dependencies and save all the tedious work, duplicate/error-prone versioning info?
Basically because a lot of projects are not semantically versioned; some increment the major version (1.0->2.0) while maintaining compatibility while others break compatibility while keeping the major version the same. Since bundle authors are recommended to review their package versions they are generally in the best place to map the Maven dependency version to OSGi (which may involve adding/removing version segments).
However I'd be happy to look at patches that pass Maven dependency versions onto bnd when embedding.
>> (btw one of the dependencies already had the OSGi headers - in this case the dependency's export version seems to have been copied over correctly to the uber-bundle's exports).
That would be the bnd library detecting the OSGi headers.
>> 2. The uber-bundle's Export-Package header seems to include non-package resources such as the dependency jar names themselves, a License.txt file it probably picked up from one of the bundles, etc. The maven build actually gives a warning, e.g. "Invalid package name: 'guava-14.0.jar'". I ended up adding !*.jar,!*.xml,!*.txt to the _exportcontents configuration to prevent the warnings and incorrect exports header. Is this a bug or a feature?
Sounds like a bug, could be in the plugin or even in bnd (when expanding the _exportcontents) - if you raise an issue please attach a test project that recreates the problem. If the issue is with the expansion of _exportcontents then as a workaround I would avoid using a global wildcard like * and instead use something like org.*,com.* since most packages fall under a limited set of top-level names.
>> 3. Similarly, when I try to set Embed-StripGroup to false, the jars are placed ain group-named directories, but then the directory names themselves are added to the exports header (without the jar name). One of the group names happened to contain a hyphen, which resulted in an "Invalid package name" header as well. I added an additional !*-* to the exportcontents configuration to prevent this as well. Again, a lot of unnecessary and tedious troubleshooting.
This also sounds like a bug; the embedding code will generate the Include-Resource and Bundle-ClassPath instructions passed to bnd, but shouldn't alter the exports (at least the initial instructions passed to bnd)
Note if you turn on debug (-X) then the plugin will show both the instructions and properties passed to bnd as well as the generated manifest, which can help determine where any mis-translation might be occurring.
>> 4. One of the embedded dependencies already had OSGi headers, and while the export info (e.g. version, see #1) seemed to be copied over correctly, the imports, and specifically the "resolution:=optional" parameter, was not - this meanse I need to manually go over all imports of the bundles and explicit copy over this parameter into the uber-bundle's Import-Package configuration for all optional imports of all dependency bundles. Very tedious and unnecessary. A bug?
This sounds like an issue with bnd, not sure if the omission is intentional (because the classes are being repackaged) or not.
>> 5. By default the embedded dependencies are not exported by the uber-budnle, until Exported-Package is specified. It is natural in many cases (particularly when migrating from bundleall) to want to specify * and export everything as before. However, as I eventually found in the documentation, the default behavior is to then embed everything twice in the bundle - one in the jars, and once inlined, and Export-Package needs to be replaced with _exportcontents in order to get the desired result. I don't know what the use case is where one would want to include everything twice, but perhaps this should at least not be made the default behavior?
This is standard bnd behaviour: http://felix.apache.org/site/apache-felix-bundle-plugin-faq.html#ApacheFelixBundlePluginFAQ-WhenIembedadependencywhydoIseeduplicatedcontent%253F
>> In summary, what I had to do to get the desired uber-bundle is create the new project, add its dependencies, set Embed-Transitive to true, specify the actual jars to embed in Embed-Dependency (up to here this makes sense), generate the uber-jar, figure out why I'm getting warning about jar names appearing as invalid package names, add hacky globs to Export-Package to prevent them, (I took the detour of trying Embed-StripGroup as well and doubling the invalid package name effort), generate the uber-jar again, manually go over the exports in its manifest looking for all those with the wrong version,
I'd always recommend reviewing both the bundle content and the packages in the manifest (you can use bnd's print command to get a clear view of the package details: http://www.aqute.biz/Bnd/CommandLine)
While this can be tedious you do get a good idea of what is in the bundle and how to break it down further. I agree it's annoying when you have to iterate, so suggestions on minimising this are welcome - but I don't think you can completely remove the need to check the final metadata.
>> find the correct version for each and specify the package+version manually in Export-Package, go over all imports in all OSGi-ready embedded dependencies and specify the package+resolution:=optional parameter manually in Import-Package for all those with an optional resolution, move all of Export-Package into _exportcontents once I figured it needs to be done, and finally arrive in a working bundle (I hope :-) ). That's a lot of work that seems unnecessary, error-prone and difficult to maintain when something changes in the future, and took even more time to try googling, asking, reading documentation and experimenting with until I figured out what needs to be done.
The difference between _exportcontents and Export-Package and how it affects embedding is in the documentation (http://felix.apache.org/documentation/subprojects/apache-felix-maven-bundle-plugin-bnd.html#embed-dependency-and-export-package) and is the first entry in the FAQ - was there another place where this could have been called out earlier? Another option could be to try and detect when someone wants to embed (vs. inline) and translate any Export-Package instructions to _exportcontents - but this could end up being too much magic and just make things harder to debug.
>> All that being said, I hope this feedback can be used to improve the experience for the next guy or gal that'll need to migrate away from the bundleall goal and save them a lot of time and effort. At the very least, there should be configurable options to do these things automatically, or even a "bundleall" configuration option that does what one would expect (embed jars, retain embedded osgi headers and assign proper version numbers, remove duplication and unnecessary included resources, etc.). Anyone wishing to further tweak or override these can of course already do that.
>>
>> I'll only add another little suggestion:
>>
>> 6. It would be nice if the helpful original response I got in FELIX-4145, and possibly some more examples specific to bundleall migration and overcoming some of the above issues, would be documented and linked from the wrap/bundleall documentation pages, right near the deprecation notice - that's the first place many people would look when trying to figure out how they can move past the deprecation and figure out what the new recommended solution is.
>> I realize some of this hard work may have been due to my own misunderstandings, but that can happen to the next guy as well, so any way to save him the trouble would be a blessing.
>>
>> Thanks, and I hope this has been useful to someone :-)
Feedback is always welcome - as are patches and extra/updated documentation (can be attached to a JIRA issue or just sent to this list)
--
Cheers, Stuart
>> Amichai
>>
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@felix.apache.org
For additional commands, e-mail: users-help@felix.apache.org
Re: Bundleall replacement
Posted by "A. Rothman" <am...@amichais.net>.
Should I open a JIRA ticket for this? Or multiple tickets? Are these
confirmed bugs or is something by design?
On 06/24/2013 12:35 PM, A. Rothman wrote:
> Hi,
>
> Following the Maven Bundle Plugin's deprecation/removal of the
> bundleall goal I was pointed to the option of creating an uber-bundle
> for my project's dependencies (see
> https://issues.apache.org/jira/browse/FELIX-4145). This took
> considerable effort compared to using the bundleall goal, and I'd like
> to raise some thoughts about the process - I don't know if these are
> bugs, features, or my own misunderstanding, but I hope they can be
> used to improve the migration experience for anyone else who currently
> uses the bundleall/wrap goals and will shortly be required to migrate
> away from them.
>
> What I did in a nutshell: I created a project for the uber-bundle,
> with the dependencies being the packages it will need to include (all
> with explicit versions inherited from the parent pom's
> DependencyManagement). The new project's version is currently
> 0.1-SNAPSHOT. I'm using the Embed-Dependency configuration along with
> Embed-Transitive set to true to hand-pick the dependencies and
> transitive dependencies that I need to add to the bundle. The project
> contains nothing other than its pom (no real sources or resources).
>
> 1. In the generated bundle, the generated Export-Package header in the
> manifest specifies the version of the exported packages as version
> 0.1.SNAPSHOT. When using the bundleall goal it would correctly take
> the artifact version and use that in the exports version, but now this
> information seems lost, and I have to manually go over all the exports
> and specify their respective versions in the _exportcontents
> configuration. Is this a bug? Why not use maven's existing version
> information for these dependencies and save all the tedious work,
> duplicate/error-prone versioning info? (btw one of the dependencies
> already had the OSGi headers - in this case the dependency's export
> version seems to have been copied over correctly to the uber-bundle's
> exports).
>
> 2. The uber-bundle's Export-Package header seems to include
> non-package resources such as the dependency jar names themselves, a
> License.txt file it probably picked up from one of the bundles, etc.
> The maven build actually gives a warning, e.g. "Invalid package name:
> 'guava-14.0.jar'". I ended up adding !*.jar,!*.xml,!*.txt to the
> _exportcontents configuration to prevent the warnings and incorrect
> exports header. Is this a bug or a feature?
>
> 3. Similarly, when I try to set Embed-StripGroup to false, the jars
> are placed ain group-named directories, but then the directory names
> themselves are added to the exports header (without the jar name). One
> of the group names happened to contain a hyphen, which resulted in an
> "Invalid package name" header as well. I added an additional !*-* to
> the exportcontents configuration to prevent this as well. Again, a lot
> of unnecessary and tedious troubleshooting.
>
> 4. One of the embedded dependencies already had OSGi headers, and
> while the export info (e.g. version, see #1) seemed to be copied over
> correctly, the imports, and specifically the "resolution:=optional"
> parameter, was not - this meanse I need to manually go over all
> imports of the bundles and explicit copy over this parameter into the
> uber-bundle's Import-Package configuration for all optional imports of
> all dependency bundles. Very tedious and unnecessary. A bug?
>
> 5. By default the embedded dependencies are not exported by the
> uber-budnle, until Exported-Package is specified. It is natural in
> many cases (particularly when migrating from bundleall) to want to
> specify * and export everything as before. However, as I eventually
> found in the documentation, the default behavior is to then embed
> everything twice in the bundle - one in the jars, and once inlined,
> and Export-Package needs to be replaced with _exportcontents in order
> to get the desired result. I don't know what the use case is where one
> would want to include everything twice, but perhaps this should at
> least not be made the default behavior?
>
> In summary, what I had to do to get the desired uber-bundle is create
> the new project, add its dependencies, set Embed-Transitive to true,
> specify the actual jars to embed in Embed-Dependency (up to here this
> makes sense), generate the uber-jar, figure out why I'm getting
> warning about jar names appearing as invalid package names, add hacky
> globs to Export-Package to prevent them, (I took the detour of trying
> Embed-StripGroup as well and doubling the invalid package name
> effort), generate the uber-jar again, manually go over the exports in
> its manifest looking for all those with the wrong version, find the
> correct version for each and specify the package+version manually in
> Export-Package, go over all imports in all OSGi-ready embedded
> dependencies and specify the package+resolution:=optional parameter
> manually in Import-Package for all those with an optional resolution,
> move all of Export-Package into _exportcontents once I figured it
> needs to be done, and finally arrive in a working bundle (I hope :-)
> ). That's a lot of work that seems unnecessary, error-prone and
> difficult to maintain when something changes in the future, and took
> even more time to try googling, asking, reading documentation and
> experimenting with until I figured out what needs to be done.
>
> All that being said, I hope this feedback can be used to improve the
> experience for the next guy or gal that'll need to migrate away from
> the bundleall goal and save them a lot of time and effort. At the very
> least, there should be configurable options to do these things
> automatically, or even a "bundleall" configuration option that does
> what one would expect (embed jars, retain embedded osgi headers and
> assign proper version numbers, remove duplication and unnecessary
> included resources, etc.). Anyone wishing to further tweak or override
> these can of course already do that.
>
> I'll only add another little suggestion:
>
> 6. It would be nice if the helpful original response I got in
> FELIX-4145, and possibly some more examples specific to bundleall
> migration and overcoming some of the above issues, would be documented
> and linked from the wrap/bundleall documentation pages, right near the
> deprecation notice - that's the first place many people would look
> when trying to figure out how they can move past the deprecation and
> figure out what the new recommended solution is.
>
> I realize some of this hard work may have been due to my own
> misunderstandings, but that can happen to the next guy as well, so any
> way to save him the trouble would be a blessing.
>
> Thanks, and I hope this has been useful to someone :-)
>
> Amichai
>
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@felix.apache.org
For additional commands, e-mail: users-help@felix.apache.org