You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@brooklyn.apache.org by Paul Campbell <pa...@cloudsoftcorp.com> on 2018/10/10 09:24:41 UTC
Re: Rebind historic persisted state: referencing wrap:mvn bundles
Hi all,
I've opened a JIRA ticket for Aled's issue:
https://issues.apache.org/jira/browse/BROOKLYN-605
I'm also investigation his proposed solution of adding support to
class-renames here:
https://github.com/kemitix/brooklyn-server/tree/BROOKLYN-605-rebind-fails-for-historic-state
--
Paul Campbell
On Thu, 20 Sep 2018 at 14:53, Aled Sage <al...@gmail.com> wrote:
> Hi all,
>
> TL;DR: I've hit a problem with rebinding to historic persisted state,
> when wrap:mvn has been used for OSGi bundles - the symbolic name
> changed, so classloading didn't work on rebind. Which of the solutions
> should we go for?
>
>
> _*PROBLEM*_
> The persisted state refers to a class in aws-java-sdk 1.11.245, but in
> my newer brooklyn I've upgraded this bundle to 1.11.411 (the old bundle
> is not there). Because this jar was added using wrap-mvn, the two
> versions of the bundle have different symbolic names! Brooklyn therefore
> doesn't know to look in the newer aws-java-sdk bundle.
>
> The bundle was added via a feature.xml containing:
>
>
> <bundle>wrap:mvn:com.amazonaws/aws-java-sdk-bundle/${aws.java.sdk.version}</bundle>
>
> When built locally, this produced a bundle with the very strange
> symbolic name:
>
>
> wrap_file__Users_aledsage_amp_cloudsoft-amp-karaf-5.2.0_system_com_amazonaws_aws-java-sdk-bundle_1.11.245_aws-java-sdk-bundle-1.11.245.jar
>
> _*EXISTING FUNCTIONALITY*_
> Brooklyn currently supports a couple of related features:
>
> 1. The persisted state will by default not include bundle versions. We
> are therefore willing to use newer versions of a given bundle.
> 2. When adding your own bundle, you can include metadata in it's
> MANIFST.mf to say what bundle/version it replaces.
> See brooklyn-docs guide/ops/upgrades/_blueprints.md
>
> However, I don't want to use (2) because that would involve fiddling
> with the wrap:mvn to add more metadata.
>
>
> _*POSSIBLE SOLUTIONS*_
> There are no doubt many ways to solve this problem. I describe a few of
> them below.
>
> I think I favour the class-renames approach because of its simplicity.
>
> *_Add support to class-renames_*
> Our deserializingClassRenames.properties allows rebind to handle class
> renames, or a specific class moving from one bundle to another. This is
> useful for historic persisted state.
>
> We could add support for bundle wildcards, to say that all classes in
> one bundle can be loaded from another bundle.
>
> For example:
>
> wrap_blah_aws-java-sdk-bundle-1.11.245.jar\:* :
> wrap_blah_aws-java-sdk-bundle-1.11.411.jar\:*
>
> *_Support a bundle-upgrade configuration file_*
> We could add support for a config file that gives explicit named bundle
> replacements. This would augment the existing functionality (2 above),
> but instead of the replacement bundle being defined in the new bundle's
> metadata, it could also be defined in this configuration file.
>
> For example, $BROOKLYN_HOME/etc/org.apache.brooklyn.bundleupgrade.cfg
> could contain something like:
>
> wrap_blah_aws-java-sdk-bundle-1.11.411.jar:
> Brooklyn-Catalog-Upgrade-For-Bundles:
> "wrap_blah_aws-java-sdk-bundle-1.11.245.jar: *"
>
> (would need to figure out the cfg versus yaml format here, obviously!)
>
> *_Recognise 'wrap' bundles, and allow newer versions_*
> When loading the class, we could recognise the "wrap_" prefix. We could
> figure out from the symbolic name what it was built from, and be willing
> to use bundles that are newer versions.
>
> However, playing with wrap:mvn, the bundle naming is not as obvious as
> one would expect. The symbolic name below is a very different structure
> from that above:
>
>
> wrap_mvn_com.example.brooklyn.test.resources.osgi_brooklyn-test-osgi-com-example-plainoldjar_1.0.0
>
> This example was created in karaf client by running:
>
> bundle:install
>
> wrap:mvn:com.example.brooklyn.test.resources.osgi/brooklyn-test-osgi-com-example-plainoldjar/1.0.0
>
> See brooklyn-server's
> org.apache.brooklyn.util.core.ClassLoaderUtils.tryLoadFromBundle.
>
> *_Require users to set the symbolic name, when using wrap:mvn_*
> We could require users to *not* use the simple "wrap:mvn", and instead
> force them to ensure a more predictable symbolic name + version is used.
>
> However, that sounds harder for users. It also doesn't solve the problem
> for anyone with such historic persisted state.
>
>
> _*LONG TERM / DOCS
> *_We should warn people about this in our docs, including a description
> of how to work around it.
>
> We should discourage the use of complex types in config keys and
> sensors, where we (or the user) don't explicitly control the versioning
> and schema of those types.
>
> **_**_
>
--
Paul Campbell
Software Engineer
*Cloudsoft <https://cloudsoft.io/> *| Bringing Business to the Cloud
E: paul.campbell@cloudsoft.io
M: 07476981644 <+447476981644>
T: kemitixcode <https://twitter.com/kemitixcode>
L: https://www.linkedin.com/in/paulkcampbell/
Need a hand with AWS? Get a Free Consultation.
<https://go.cloudsoft.io/healthcheck/>
Re: Rebind historic persisted state: referencing wrap:mvn bundles
Posted by Aled Sage <al...@gmail.com>.
Hi all,
I've just been speaking to Alex about this.
Turns out it's more complicated...
The name generated by wrap:mvn will vary depending on the user's runtime
environment. For example, when Alex took a brooklyn build and looked at
the symbolic name of a `wrap:mvn` bundle, it included the runtime path
of his brooklyn install!
---
First, I think we should strongly recommend that people using `wrap:mvn`
specify the symbolic name and version explicitly [1], such as:
wrap:mvn:com.amazonaws/aws-java-sdk-bundle/1.11.411$Bundle-SymbolicName=com.amazonaws.aws-java-sdk-bundle&Bundle-Version=1.11.411
---
If we went down the deserializingClassRenames.properties approach, we'd
need to support wildcards in the bundle name:
wrap_.*_aws-java-sdk-bundle-1.11.245.jar\:* :
com.amazonaws.aws-java-sdk-bundle\:*
That doesn't feel good - evaluating the regex on every class-load will
be inefficient.
---
Another way I'm looking at, and have discussed with Alex, is to include
in the newer version of the bundle the header:
Brooklyn-Catalog-Upgrade-For-Bundles =
wrap:com_amazonaws_aws-java-sdk-bundle:0
(we can add that header to the `wrap:mvn` command, for when we generate
the new version of aws-java-sdk-bundle:1.11.411.)
We already support this header for upgrading explicit bundle names, but
not for auto-generated weird bundle names!
This would tell Brooklyn that our new version of the bundle will upgrade
auto-wrapped bundle of com_amazonaws_aws-java-sdk-bundle.
I'm looking at how easy it would be to add this to
org.apache.brooklyn.core.typereg.BundleUpgradeParser, and whether we'd
avoid the regex evaluation of every class-load.
Aled
[1] https://ops4j1.jira.com/wiki/spaces/paxurl/pages/3833898/Wrap+Protocol
On 10/10/2018 02:24, Paul Campbell wrote:
> Hi all,
>
> I've opened a JIRA ticket for Aled's issue:
> https://issues.apache.org/jira/browse/BROOKLYN-605
>
> I'm also investigation his proposed solution of adding support to
> class-renames here:
> https://github.com/kemitix/brooklyn-server/tree/BROOKLYN-605-rebind-fails-for-historic-state
>
> --
> Paul Campbell
>
>
> On Thu, 20 Sep 2018 at 14:53, Aled Sage <al...@gmail.com> wrote:
>
>> Hi all,
>>
>> TL;DR: I've hit a problem with rebinding to historic persisted state,
>> when wrap:mvn has been used for OSGi bundles - the symbolic name
>> changed, so classloading didn't work on rebind. Which of the solutions
>> should we go for?
>>
>>
>> _*PROBLEM*_
>> The persisted state refers to a class in aws-java-sdk 1.11.245, but in
>> my newer brooklyn I've upgraded this bundle to 1.11.411 (the old bundle
>> is not there). Because this jar was added using wrap-mvn, the two
>> versions of the bundle have different symbolic names! Brooklyn therefore
>> doesn't know to look in the newer aws-java-sdk bundle.
>>
>> The bundle was added via a feature.xml containing:
>>
>>
>> <bundle>wrap:mvn:com.amazonaws/aws-java-sdk-bundle/${aws.java.sdk.version}</bundle>
>>
>> When built locally, this produced a bundle with the very strange
>> symbolic name:
>>
>>
>> wrap_file__Users_aledsage_amp_cloudsoft-amp-karaf-5.2.0_system_com_amazonaws_aws-java-sdk-bundle_1.11.245_aws-java-sdk-bundle-1.11.245.jar
>>
>> _*EXISTING FUNCTIONALITY*_
>> Brooklyn currently supports a couple of related features:
>>
>> 1. The persisted state will by default not include bundle versions. We
>> are therefore willing to use newer versions of a given bundle.
>> 2. When adding your own bundle, you can include metadata in it's
>> MANIFST.mf to say what bundle/version it replaces.
>> See brooklyn-docs guide/ops/upgrades/_blueprints.md
>>
>> However, I don't want to use (2) because that would involve fiddling
>> with the wrap:mvn to add more metadata.
>>
>>
>> _*POSSIBLE SOLUTIONS*_
>> There are no doubt many ways to solve this problem. I describe a few of
>> them below.
>>
>> I think I favour the class-renames approach because of its simplicity.
>>
>> *_Add support to class-renames_*
>> Our deserializingClassRenames.properties allows rebind to handle class
>> renames, or a specific class moving from one bundle to another. This is
>> useful for historic persisted state.
>>
>> We could add support for bundle wildcards, to say that all classes in
>> one bundle can be loaded from another bundle.
>>
>> For example:
>>
>> wrap_blah_aws-java-sdk-bundle-1.11.245.jar\:* :
>> wrap_blah_aws-java-sdk-bundle-1.11.411.jar\:*
>>
>> *_Support a bundle-upgrade configuration file_*
>> We could add support for a config file that gives explicit named bundle
>> replacements. This would augment the existing functionality (2 above),
>> but instead of the replacement bundle being defined in the new bundle's
>> metadata, it could also be defined in this configuration file.
>>
>> For example, $BROOKLYN_HOME/etc/org.apache.brooklyn.bundleupgrade.cfg
>> could contain something like:
>>
>> wrap_blah_aws-java-sdk-bundle-1.11.411.jar:
>> Brooklyn-Catalog-Upgrade-For-Bundles:
>> "wrap_blah_aws-java-sdk-bundle-1.11.245.jar: *"
>>
>> (would need to figure out the cfg versus yaml format here, obviously!)
>>
>> *_Recognise 'wrap' bundles, and allow newer versions_*
>> When loading the class, we could recognise the "wrap_" prefix. We could
>> figure out from the symbolic name what it was built from, and be willing
>> to use bundles that are newer versions.
>>
>> However, playing with wrap:mvn, the bundle naming is not as obvious as
>> one would expect. The symbolic name below is a very different structure
>> from that above:
>>
>>
>> wrap_mvn_com.example.brooklyn.test.resources.osgi_brooklyn-test-osgi-com-example-plainoldjar_1.0.0
>>
>> This example was created in karaf client by running:
>>
>> bundle:install
>>
>> wrap:mvn:com.example.brooklyn.test.resources.osgi/brooklyn-test-osgi-com-example-plainoldjar/1.0.0
>>
>> See brooklyn-server's
>> org.apache.brooklyn.util.core.ClassLoaderUtils.tryLoadFromBundle.
>>
>> *_Require users to set the symbolic name, when using wrap:mvn_*
>> We could require users to *not* use the simple "wrap:mvn", and instead
>> force them to ensure a more predictable symbolic name + version is used.
>>
>> However, that sounds harder for users. It also doesn't solve the problem
>> for anyone with such historic persisted state.
>>
>>
>> _*LONG TERM / DOCS
>> *_We should warn people about this in our docs, including a description
>> of how to work around it.
>>
>> We should discourage the use of complex types in config keys and
>> sensors, where we (or the user) don't explicitly control the versioning
>> and schema of those types.
>>
>> **_**_
>>
>