You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@karaf.apache.org by Stephen Kitt <sk...@redhat.com> on 2017/06/13 08:35:37 UTC
Understanding features and dependencies
Hi,
As we continue to try to migrate OpenDaylight to Karaf 4, it’s become
very clear that we need to understand exactly what feature dependencies
entail. (And yes, I’ll submit documentation patches once we’ve done
so.)
OpenDaylight (ODL) defines lots and lots of features, some of which
contain bundles which survive refreshes, many of which don’t. In
real-world deployments, the Karaf distributions which are built tend to
use boot features to start the features they’re interested in. We make
extensive use of feature dependencies of course, e.g.
<feature name='odl-ttp-model-rest' version='${project.version}'>
<feature version='${project.version}'>odl-ttp-model</feature>
<feature version='${restconf.version}'>odl-mdsal-apidocs</feature>
<feature version='${restconf.version}'>odl-restconf</feature>
<feature version='${controller.version}'>odl-mdsal-broker</feature>
</feature>
In Karaf 3 all this works fine.
In Karaf 4 we’re having lots of issues related to bundle refreshes, so
we’ve been trying to figure out how to avoid them. Of course,
“feature:install --no-auto-refresh” works, but that’s not ideal (at
least, I don’t think it is). We thought perhaps using the dependency or
prerequisite attributes might help.
As we understand things (and this is the main point I would like to
clarify), in Karaf 4, based on the feature declaration above:
* (re)starting odl-ttp-model-rest would restart odl-ttp-model,
odl-mdsal-apidocs etc. (which is a problem because e.g.
odl-mdsal-broker is very likely to already be running when we install
another feature)
* adding “dependency="true"” would disable this
* adding “prerequisite="true"” would also disable this, and ensure that
the corresponding feature is started before odl-ttp-model-rest
We hoped the second option would solve our problems, but in our
experimentation it seems that declaring features in this way means they
aren’t necessarily installed. (I’m hoping I can still dig up examples.)
As in,
<feature name="a">
<feature name="b" dependency="true"/>
</feature>
results in a distribution where
feature:install a
doesn’t necessarily install b. Is that supposed to be the case?
We also thought that declaring prerequisites would work too, especially
since our feature dependencies are acyclic and so it is actually
possible to initialise all our features in a given order. But when we
tried this we ran into stack overflows. (I’ll do it again and file
bugs.)
Does this make any sense at all?
Regards,
Stephen
Re: Understanding features and dependencies
Posted by Achim Nierbeck <bc...@googlemail.com>.
Regarding the capabilities,
both.
It's determined by A) the bundles manifest entry. But in certain cases
those bundles don't explicit describe their capabilities fully.
In those cases it helps if the feature itself does describe that
capability.
regards, Achim
2017-06-13 13:55 GMT+02:00 Stephen Kitt <sk...@redhat.com>:
> Hi Jean-Baptiste,
>
> On Tue, 13 Jun 2017 10:41:26 +0200
> Jean-Baptiste Onofré <jb...@nanthrax.net> wrote:
> > I think there's two questions:
> >
> > 1. Why does a refresh happen ?
> > A refresh can happen for instance when an optional import is resolved
> > (when installing a feature or a bundle) or when a new package version
> > is installed matching a version range.
> > So, especially in the case of optional import, you should check the
> > import of your bundle.
>
> OK, thanks.
>
> > 2. Why does your bundle not able to refresh ?
> > That's probably the most important. Maybe, you keep the bundle
> > context as member of a class and so it becomes invalid after the
> > refresh.
>
> That’s possible in some cases. Our main issue is that some of our
> bundles maintain state and don’t really like being stopped and started
> (in particular, our persistence bundles and our clustering bundles).
>
> > Before dealing with dependency or prerequisite, I would start to
> > address those points.
> >
> > The dependency flag is not a "complete" inner feature, so it's normal
> > that the feature is not installed if another one already provides the
> > corresponding capabilities.
>
> OK. Are capabilities here determined by explicit <capability/>
> declarations in the feature XML, or are they based on bundle manifest
> declarations?
>
> Regards,
>
> Stephen
>
--
Apache Member
Apache Karaf <http://karaf.apache.org/> Committer & PMC
OPS4J Pax Web <http://wiki.ops4j.org/display/paxweb/Pax+Web/> Committer &
Project Lead
blog <http://notizblog.nierbeck.de/>
Co-Author of Apache Karaf Cookbook <http://bit.ly/1ps9rkS>
Software Architect / Project Manager / Scrum Master
Re: Understanding features and dependencies
Posted by Stephen Kitt <sk...@redhat.com>.
Hi Jean-Baptiste,
On Tue, 13 Jun 2017 10:41:26 +0200
Jean-Baptiste Onofré <jb...@nanthrax.net> wrote:
> I think there's two questions:
>
> 1. Why does a refresh happen ?
> A refresh can happen for instance when an optional import is resolved
> (when installing a feature or a bundle) or when a new package version
> is installed matching a version range.
> So, especially in the case of optional import, you should check the
> import of your bundle.
OK, thanks.
> 2. Why does your bundle not able to refresh ?
> That's probably the most important. Maybe, you keep the bundle
> context as member of a class and so it becomes invalid after the
> refresh.
That’s possible in some cases. Our main issue is that some of our
bundles maintain state and don’t really like being stopped and started
(in particular, our persistence bundles and our clustering bundles).
> Before dealing with dependency or prerequisite, I would start to
> address those points.
>
> The dependency flag is not a "complete" inner feature, so it's normal
> that the feature is not installed if another one already provides the
> corresponding capabilities.
OK. Are capabilities here determined by explicit <capability/>
declarations in the feature XML, or are they based on bundle manifest
declarations?
Regards,
Stephen
Re: Understanding features and dependencies
Posted by Jean-Baptiste Onofré <jb...@nanthrax.net>.
Hi Stephen,
I think there's two questions:
1. Why does a refresh happen ?
A refresh can happen for instance when an optional import is resolved (when
installing a feature or a bundle) or when a new package version is installed
matching a version range.
So, especially in the case of optional import, you should check the import of
your bundle.
2. Why does your bundle not able to refresh ?
That's probably the most important. Maybe, you keep the bundle context as member
of a class and so it becomes invalid after the refresh.
Before dealing with dependency or prerequisite, I would start to address those
points.
The dependency flag is not a "complete" inner feature, so it's normal that the
feature is not installed if another one already provides the corresponding
capabilities.
Regards
JB
On 06/13/2017 10:35 AM, Stephen Kitt wrote:
> Hi,
>
> As we continue to try to migrate OpenDaylight to Karaf 4, it’s become
> very clear that we need to understand exactly what feature dependencies
> entail. (And yes, I’ll submit documentation patches once we’ve done
> so.)
>
> OpenDaylight (ODL) defines lots and lots of features, some of which
> contain bundles which survive refreshes, many of which don’t. In
> real-world deployments, the Karaf distributions which are built tend to
> use boot features to start the features they’re interested in. We make
> extensive use of feature dependencies of course, e.g.
>
> <feature name='odl-ttp-model-rest' version='${project.version}'>
> <feature version='${project.version}'>odl-ttp-model</feature>
> <feature version='${restconf.version}'>odl-mdsal-apidocs</feature>
> <feature version='${restconf.version}'>odl-restconf</feature>
> <feature version='${controller.version}'>odl-mdsal-broker</feature>
> </feature>
>
> In Karaf 3 all this works fine.
>
> In Karaf 4 we’re having lots of issues related to bundle refreshes, so
> we’ve been trying to figure out how to avoid them. Of course,
> “feature:install --no-auto-refresh” works, but that’s not ideal (at
> least, I don’t think it is). We thought perhaps using the dependency or
> prerequisite attributes might help.
>
> As we understand things (and this is the main point I would like to
> clarify), in Karaf 4, based on the feature declaration above:
> * (re)starting odl-ttp-model-rest would restart odl-ttp-model,
> odl-mdsal-apidocs etc. (which is a problem because e.g.
> odl-mdsal-broker is very likely to already be running when we install
> another feature)
> * adding “dependency="true"” would disable this
> * adding “prerequisite="true"” would also disable this, and ensure that
> the corresponding feature is started before odl-ttp-model-rest
>
> We hoped the second option would solve our problems, but in our
> experimentation it seems that declaring features in this way means they
> aren’t necessarily installed. (I’m hoping I can still dig up examples.)
> As in,
>
> <feature name="a">
> <feature name="b" dependency="true"/>
> </feature>
>
> results in a distribution where
>
> feature:install a
>
> doesn’t necessarily install b. Is that supposed to be the case?
>
> We also thought that declaring prerequisites would work too, especially
> since our feature dependencies are acyclic and so it is actually
> possible to initialise all our features in a given order. But when we
> tried this we ran into stack overflows. (I’ll do it again and file
> bugs.)
>
> Does this make any sense at all?
>
> Regards,
>
> Stephen
>
--
Jean-Baptiste Onofré
jbonofre@apache.org
http://blog.nanthrax.net
Talend - http://www.talend.com
Re: Understanding features and dependencies
Posted by Stephen Kitt <sk...@redhat.com>.
On Tue, 13 Jun 2017 13:27:24 +0200
Guillaume Nodet <gn...@apache.org> wrote:
> 2017-06-13 10:35 GMT+02:00 Stephen Kitt <sk...@redhat.com>:
> > As we understand things (and this is the main point I would like to
> > clarify), in Karaf 4, based on the feature declaration above:
> > * (re)starting odl-ttp-model-rest would restart odl-ttp-model,
> > odl-mdsal-apidocs etc. (which is a problem because e.g.
> > odl-mdsal-broker is very likely to already be running when we
> > install another feature)
> > * adding “dependency="true"” would disable this
> > * adding “prerequisite="true"” would also disable this, and ensure
> > that the corresponding feature is started before odl-ttp-model-rest
>
> That's wrong. There's no relationship between how a feature is
> defined and the bundle being refreshed when the feature is installed.
> A bundle will be refreshed if:
> * it has been updated
> * it has been uninstalled
> * one of its dependencies is refreshed
> * a new optional dependency has been installed
OK, thanks, so reasoning about refreshes only makes sense at the bundle
level and features should only be considered as defining groups of
bundles, with no impact on refresh “propagation”. Is that right?
What exactly do “dependency="true"” and “prerequisite="true"” do? The
documentation isn’t very clear on this, and trying to extrapolate rules
based on behaviour obviously isn’t productive ;-).
> When installing feature, you can use the --verbose flag which will
> print why bundles are being refreshed.
Yes, I’ve been using that quite a bit.
This leads me to a follow-on: given the above refresh rules, why would
installing the *same* feature twice in a row cause some of its bundles
to be refreshed? Can wrapped bundles wreak havoc here? I’m seeing stuff
like
Refreshing bundles:
com.fasterxml.jackson.datatype.jackson-datatype-json-org/2.3.2
(Should be wired to:
org.opendaylight.netconf.sal-rest-docgen/1.6.0.SNAPSHOT (through
[com.fasterxml.jackson.datatype.jackson-datatype-json-org/2.3.2]
osgi.wiring.package; filter:="(osgi.wiring.package=org.json)"))
and further down
org.opendaylight.netconf.sal-rest-docgen/1.6.0.SNAPSHOT (Wired to
com.fasterxml.jackson.datatype.jackson-datatype-json-org/2.3.2
which is being refreshed)
which suggests there’s a wiring loop — could it be related to wrapping
org.json?
Regards,
Stephen
Re: Understanding features and dependencies
Posted by Guillaume Nodet <gn...@apache.org>.
2017-06-13 10:35 GMT+02:00 Stephen Kitt <sk...@redhat.com>:
> Hi,
>
> As we continue to try to migrate OpenDaylight to Karaf 4, it’s become
> very clear that we need to understand exactly what feature dependencies
> entail. (And yes, I’ll submit documentation patches once we’ve done
> so.)
>
> OpenDaylight (ODL) defines lots and lots of features, some of which
> contain bundles which survive refreshes, many of which don’t. In
> real-world deployments, the Karaf distributions which are built tend to
> use boot features to start the features they’re interested in. We make
> extensive use of feature dependencies of course, e.g.
>
> <feature name='odl-ttp-model-rest' version='${project.version}'>
> <feature version='${project.version}'>odl-ttp-model</feature>
> <feature version='${restconf.version}'>odl-mdsal-apidocs</feature>
> <feature version='${restconf.version}'>odl-restconf</feature>
> <feature version='${controller.version}
> '>odl-mdsal-broker</feature>
> </feature>
>
> In Karaf 3 all this works fine.
>
> In Karaf 4 we’re having lots of issues related to bundle refreshes, so
> we’ve been trying to figure out how to avoid them. Of course,
> “feature:install --no-auto-refresh” works, but that’s not ideal (at
> least, I don’t think it is). We thought perhaps using the dependency or
> prerequisite attributes might help.
>
> As we understand things (and this is the main point I would like to
> clarify), in Karaf 4, based on the feature declaration above:
> * (re)starting odl-ttp-model-rest would restart odl-ttp-model,
> odl-mdsal-apidocs etc. (which is a problem because e.g.
> odl-mdsal-broker is very likely to already be running when we install
> another feature)
> * adding “dependency="true"” would disable this
> * adding “prerequisite="true"” would also disable this, and ensure that
> the corresponding feature is started before odl-ttp-model-rest
>
That's wrong. There's no relationship between how a feature is defined and
the bundle being refreshed when the feature is installed.
A bundle will be refreshed if:
* it has been updated
* it has been uninstalled
* one of its dependencies is refreshed
* a new optional dependency has been installed
When installing feature, you can use the --verbose flag which will print
why bundles are being refreshed.
>
> We hoped the second option would solve our problems, but in our
> experimentation it seems that declaring features in this way means they
> aren’t necessarily installed. (I’m hoping I can still dig up examples.)
> As in,
>
> <feature name="a">
> <feature name="b" dependency="true"/>
> </feature>
>
> results in a distribution where
>
> feature:install a
>
> doesn’t necessarily install b. Is that supposed to be the case?
>
> We also thought that declaring prerequisites would work too, especially
> since our feature dependencies are acyclic and so it is actually
> possible to initialise all our features in a given order. But when we
> tried this we ran into stack overflows. (I’ll do it again and file
> bugs.)
>
> Does this make any sense at all?
>
> Regards,
>
> Stephen
>
--
------------------------
Guillaume Nodet