Re: [Discuss] Improvements in karaf feature handling

[Guillaume Nodet <gn...@apache.org>]

I think this general problem already has a clean solution.

I'd like to keep features transitively closed, so that they can easily be
installed without any additional requirements.
To open them up, it's just a matter of specifying the dependency="true"
flag on the required features.
The resolver will then only use them as one possible way to solve the
requirements expressed by other bundles.

2015-08-20 15:15 GMT+02:00 Jean-Baptiste Onofré <jb...@nanthrax.net>:

> By the way, about Cave, it's exactly the way that you describe: Cave is an
> OSGi Repository implementation (basically OBR), that the features resolver
> can use.
>
> I created a Jira to document this and I have a blog post in progress to
> explain how it works.
>
> Regards
> JB
>
>
> On 08/20/2015 03:13 PM, Jean-Baptiste Onofré wrote:
>
>> Hi Christian,
>>
>> and what about if the persistence-api feature provides a capability
>> (more generic) and transaction/jpa just have a requirement ?
>>
>> It's the way that we improved the pax-web feature to be able to deal
>> with jetty or tomcat without coupling to jetty or tomcat features.
>>
>> About the version, I agree that it would be better to externalise the
>> features repositories and not embed them (and use repo-add to add the
>> feature that we need). It was the purpose of Cave: provide a Karaf
>> Feature Respository manager: instead of embedding the features, Karaf
>> can be just a client to Cave providing all features (with different
>> version). It's also the step forward that we started as Cave can be a
>> feature resource repository (that you can add in
>> etc/org.apache.karaf.features.cfg using the resourcesRepository property).
>>
>> Regards
>> JB
>>
>> On 08/20/2015 03:08 PM, Christian Schneider wrote:
>>
>>> Currently karaf features are done in a way that each feature is
>>> consistent in itself.
>>> While this makes the features easy to install I think the big problems
>>> is that this creates a coupling between features that should not be
>>> directly
>>> related.
>>>
>>> As an example consider the transaction and jpa features.
>>>
>>> transaction -> persistence-api [1.1,2)
>>> jpa -> persistence-api [1.1,2)
>>>
>>> So both of them depend on the persistence-api feature. All three
>>> features currently are in the enterprise feature shipped with karaf.
>>> The problem is that this creates a tight coupling between the karaf
>>> version and the aries jpa and aries transaction versions. The problem is
>>> similar with other
>>> features like hibernate, openjpa, ...
>>>
>>> Currently we provide several feature versions in the same feature file
>>> to give the user a bigger choice in what to install. The big problem
>>> there is though that we can not
>>> include all the possible versions and of course we can not include
>>> versions that are not yet released. So when a customer hits a bug in any
>>> of the bundles he can not easily upgrade to a new version of say aries
>>> jpa.
>>>
>>> So what I would like to do is create features in the place where the
>>> software is released and have as loose coupling between the modules as
>>> possible.
>>> So for example I would like to create a the aries transaction feature
>>> file in the aries transaction module and the aries jpa feature file in
>>> the aries jpa module. So whenever a release of these is done there is
>>> also a  new feature file that makes the newest release available to
>>> users. So they do not need to wait until a new karaf version picks up
>>> the new aries jpa release.
>>>
>>> So in the aris jpa feature file I would put the jpa feature but I do not
>>> want to have a dependency on the aries jpa feature file.
>>> At the moment this is not possible as I need to define a reference to
>>> the persistence-api feature to make the aries jpa featur consistent.
>>>
>>> So I wonder if we could allow features to kind of have open
>>> requirements. Like the aries jpa feature would have an open requirement
>>> on the package javax.transaction [1.1,2) possibly also a requirement for
>>> a TransactionManager service. These requirements would not even have to
>>> be defined as the bundles contain all the needed information.
>>>
>>> Then when I try to install the jpa feature in karaf the feature service
>>> could tell me that there are the unresolved requirements above. It could
>>> then even go through all other features and propose the ones that offer
>>> the respective capabilities. So it could work like this:
>>>  > feature:install jpa
>>> Unresolved requirements:
>>> Package: javax.persistence [1.1,2)
>>> Service: javax.transaction.TransactionManager
>>> The requirements can be resolved when also installing
>>> 1. transaction transaction-api/1.1
>>> 2. transaction transaction-api/1.2
>>>
>>> In an interactive mode the user could then select the features to also
>>> install from the list. In a non interactive mode there would just be an
>>> error and the user would have to add the features to the command.
>>>
>>> In the same way it would work for a whole set of features:
>>>  > feature:install jpa/2.1.1 hibernate/4.3.6.Final
>>> Unresolved requirements:
>>> Package: javax.persistence [1.2,2)
>>> Service: javax.transaction.TransactionManager
>>> The requirements can be resolved when also installing
>>> 1. transaction transaction-api/1.2
>>>
>>> In this case as there is only one solution the feature service could
>>> even auto install the necessary features without them being mentioned in
>>> any of the features to be installed. So it would act a bit like and OBR
>>> on feature level.
>>>
>>> I think it would make much more sense to handle features like this than
>>> how we do it right now. It would allow to use the new requirements and
>>> capabilties and the new resolver much better. At the same time the
>>> features themselves would become much more loosely coupled and easier to
>>> maintain.
>>>
>>> WDYT?
>>>
>>> Christian
>>>
>>>
>>
> --
> Jean-Baptiste Onofré
> jbonofre@apache.org
> http://blog.nanthrax.net
> Talend - http://www.talend.com
>

Re: [Discuss] Improvements in karaf feature handling

[Christian Schneider <ch...@die-schneider.net>]

The problem is that we create an artificial dependency on a feature 
name/version and btw a dependency that is completely unnecesary.
You say that a feature should be transitively closed to be easy 
installable. I think we can have easy installation without that.

If karaf knows that one feature can provide the needed requirements then 
karaf can be set to automatically install it.
So we can keep the feature clean of the strict dependency on a feature 
and still have the ease of use.

On the other hand I like the validation aspect at build time. Again I 
think it can be done without the feature being transitively closed.
We can just provide the needed feature in the pom so our new feature can 
be validated without having the hard dependency on it in the feature itself.

Btw. I had this problem for the cxf http transport.
it requires the servlet API but I did not want to create a feature 
dependency on the http feature as it marks an implementation. There was 
also no feature for just the servlet api. So in the end I embedded the 
servlet api into the bundle. Unfortunately this creates rewirings on 
karaf 4 as pax web of course also provides the servlet api.

If we would allow open features we could simply leave the cxf http 
feature open in regard to the servlet api import and karaf provides it 
at runtime by installing the http feature.

Similarly for the aries jpa feature. It needs the jta api but it should 
not have a dependency on an aribtrarily feature named persistence-api. 
Instead it simply has the well known package import javax.transaction. 
This is much better than the feature depdendency as it only relies on a 
package defined in a standard.

Christian



On 20.08.2015 15:57, Guillaume Nodet wrote:
> I think this general problem already has a clean solution.
>
> I'd like to keep features transitively closed, so that they can easily be
> installed without any additional requirements.
> To open them up, it's just a matter of specifying the dependency="true"
> flag on the required features.
> The resolver will then only use them as one possible way to solve the
> requirements expressed by other bundles.
>
> 2015-08-20 15:15 GMT+02:00 Jean-Baptiste Onofré <jb...@nanthrax.net>:
>
>> By the way, about Cave, it's exactly the way that you describe: Cave is an
>> OSGi Repository implementation (basically OBR), that the features resolver
>> can use.
>>
>> I created a Jira to document this and I have a blog post in progress to
>> explain how it works.
>>
>> Regards
>> JB
>>
>>
>> On 08/20/2015 03:13 PM, Jean-Baptiste Onofré wrote:
>>
>>> Hi Christian,
>>>
>>> and what about if the persistence-api feature provides a capability
>>> (more generic) and transaction/jpa just have a requirement ?
>>>
>>> It's the way that we improved the pax-web feature to be able to deal
>>> with jetty or tomcat without coupling to jetty or tomcat features.
>>>
>>> About the version, I agree that it would be better to externalise the
>>> features repositories and not embed them (and use repo-add to add the
>>> feature that we need). It was the purpose of Cave: provide a Karaf
>>> Feature Respository manager: instead of embedding the features, Karaf
>>> can be just a client to Cave providing all features (with different
>>> version). It's also the step forward that we started as Cave can be a
>>> feature resource repository (that you can add in
>>> etc/org.apache.karaf.features.cfg using the resourcesRepository property).
>>>
>>> Regards
>>> JB
>>>
>>> On 08/20/2015 03:08 PM, Christian Schneider wrote:
>>>
>>>> Currently karaf features are done in a way that each feature is
>>>> consistent in itself.
>>>> While this makes the features easy to install I think the big problems
>>>> is that this creates a coupling between features that should not be
>>>> directly
>>>> related.
>>>>
>>>> As an example consider the transaction and jpa features.
>>>>
>>>> transaction -> persistence-api [1.1,2)
>>>> jpa -> persistence-api [1.1,2)
>>>>
>>>> So both of them depend on the persistence-api feature. All three
>>>> features currently are in the enterprise feature shipped with karaf.
>>>> The problem is that this creates a tight coupling between the karaf
>>>> version and the aries jpa and aries transaction versions. The problem is
>>>> similar with other
>>>> features like hibernate, openjpa, ...
>>>>
>>>> Currently we provide several feature versions in the same feature file
>>>> to give the user a bigger choice in what to install. The big problem
>>>> there is though that we can not
>>>> include all the possible versions and of course we can not include
>>>> versions that are not yet released. So when a customer hits a bug in any
>>>> of the bundles he can not easily upgrade to a new version of say aries
>>>> jpa.
>>>>
>>>> So what I would like to do is create features in the place where the
>>>> software is released and have as loose coupling between the modules as
>>>> possible.
>>>> So for example I would like to create a the aries transaction feature
>>>> file in the aries transaction module and the aries jpa feature file in
>>>> the aries jpa module. So whenever a release of these is done there is
>>>> also a  new feature file that makes the newest release available to
>>>> users. So they do not need to wait until a new karaf version picks up
>>>> the new aries jpa release.
>>>>
>>>> So in the aris jpa feature file I would put the jpa feature but I do not
>>>> want to have a dependency on the aries jpa feature file.
>>>> At the moment this is not possible as I need to define a reference to
>>>> the persistence-api feature to make the aries jpa featur consistent.
>>>>
>>>> So I wonder if we could allow features to kind of have open
>>>> requirements. Like the aries jpa feature would have an open requirement
>>>> on the package javax.transaction [1.1,2) possibly also a requirement for
>>>> a TransactionManager service. These requirements would not even have to
>>>> be defined as the bundles contain all the needed information.
>>>>
>>>> Then when I try to install the jpa feature in karaf the feature service
>>>> could tell me that there are the unresolved requirements above. It could
>>>> then even go through all other features and propose the ones that offer
>>>> the respective capabilities. So it could work like this:
>>>>   > feature:install jpa
>>>> Unresolved requirements:
>>>> Package: javax.persistence [1.1,2)
>>>> Service: javax.transaction.TransactionManager
>>>> The requirements can be resolved when also installing
>>>> 1. transaction transaction-api/1.1
>>>> 2. transaction transaction-api/1.2
>>>>
>>>> In an interactive mode the user could then select the features to also
>>>> install from the list. In a non interactive mode there would just be an
>>>> error and the user would have to add the features to the command.
>>>>
>>>> In the same way it would work for a whole set of features:
>>>>   > feature:install jpa/2.1.1 hibernate/4.3.6.Final
>>>> Unresolved requirements:
>>>> Package: javax.persistence [1.2,2)
>>>> Service: javax.transaction.TransactionManager
>>>> The requirements can be resolved when also installing
>>>> 1. transaction transaction-api/1.2
>>>>
>>>> In this case as there is only one solution the feature service could
>>>> even auto install the necessary features without them being mentioned in
>>>> any of the features to be installed. So it would act a bit like and OBR
>>>> on feature level.
>>>>
>>>> I think it would make much more sense to handle features like this than
>>>> how we do it right now. It would allow to use the new requirements and
>>>> capabilties and the new resolver much better. At the same time the
>>>> features themselves would become much more loosely coupled and easier to
>>>> maintain.
>>>>
>>>> WDYT?
>>>>
>>>> Christian
>>>>
>>>>
>> --
>> Jean-Baptiste Onofré
>> jbonofre@apache.org
>> http://blog.nanthrax.net
>> Talend - http://www.talend.com
>>


-- 
Christian Schneider
http://www.liquid-reality.de

Open Source Architect
http://www.talend.com

Re: [Discuss] Improvements in karaf feature handling

[Jean-Baptiste Onofré <jb...@nanthrax.net>]

Yes, it's what I'm explaining in the blog and show how the features 
resolver can use Cave to resolve the "requirements".

Thanks Guillaume !

Regards
JB

On 08/20/2015 03:57 PM, Guillaume Nodet wrote:
> I think this general problem already has a clean solution.
>
> I'd like to keep features transitively closed, so that they can easily be
> installed without any additional requirements.
> To open them up, it's just a matter of specifying the dependency="true"
> flag on the required features.
> The resolver will then only use them as one possible way to solve the
> requirements expressed by other bundles.
>
> 2015-08-20 15:15 GMT+02:00 Jean-Baptiste Onofré <jb...@nanthrax.net>:
>
>> By the way, about Cave, it's exactly the way that you describe: Cave is an
>> OSGi Repository implementation (basically OBR), that the features resolver
>> can use.
>>
>> I created a Jira to document this and I have a blog post in progress to
>> explain how it works.
>>
>> Regards
>> JB
>>
>>
>> On 08/20/2015 03:13 PM, Jean-Baptiste Onofré wrote:
>>
>>> Hi Christian,
>>>
>>> and what about if the persistence-api feature provides a capability
>>> (more generic) and transaction/jpa just have a requirement ?
>>>
>>> It's the way that we improved the pax-web feature to be able to deal
>>> with jetty or tomcat without coupling to jetty or tomcat features.
>>>
>>> About the version, I agree that it would be better to externalise the
>>> features repositories and not embed them (and use repo-add to add the
>>> feature that we need). It was the purpose of Cave: provide a Karaf
>>> Feature Respository manager: instead of embedding the features, Karaf
>>> can be just a client to Cave providing all features (with different
>>> version). It's also the step forward that we started as Cave can be a
>>> feature resource repository (that you can add in
>>> etc/org.apache.karaf.features.cfg using the resourcesRepository property).
>>>
>>> Regards
>>> JB
>>>
>>> On 08/20/2015 03:08 PM, Christian Schneider wrote:
>>>
>>>> Currently karaf features are done in a way that each feature is
>>>> consistent in itself.
>>>> While this makes the features easy to install I think the big problems
>>>> is that this creates a coupling between features that should not be
>>>> directly
>>>> related.
>>>>
>>>> As an example consider the transaction and jpa features.
>>>>
>>>> transaction -> persistence-api [1.1,2)
>>>> jpa -> persistence-api [1.1,2)
>>>>
>>>> So both of them depend on the persistence-api feature. All three
>>>> features currently are in the enterprise feature shipped with karaf.
>>>> The problem is that this creates a tight coupling between the karaf
>>>> version and the aries jpa and aries transaction versions. The problem is
>>>> similar with other
>>>> features like hibernate, openjpa, ...
>>>>
>>>> Currently we provide several feature versions in the same feature file
>>>> to give the user a bigger choice in what to install. The big problem
>>>> there is though that we can not
>>>> include all the possible versions and of course we can not include
>>>> versions that are not yet released. So when a customer hits a bug in any
>>>> of the bundles he can not easily upgrade to a new version of say aries
>>>> jpa.
>>>>
>>>> So what I would like to do is create features in the place where the
>>>> software is released and have as loose coupling between the modules as
>>>> possible.
>>>> So for example I would like to create a the aries transaction feature
>>>> file in the aries transaction module and the aries jpa feature file in
>>>> the aries jpa module. So whenever a release of these is done there is
>>>> also a  new feature file that makes the newest release available to
>>>> users. So they do not need to wait until a new karaf version picks up
>>>> the new aries jpa release.
>>>>
>>>> So in the aris jpa feature file I would put the jpa feature but I do not
>>>> want to have a dependency on the aries jpa feature file.
>>>> At the moment this is not possible as I need to define a reference to
>>>> the persistence-api feature to make the aries jpa featur consistent.
>>>>
>>>> So I wonder if we could allow features to kind of have open
>>>> requirements. Like the aries jpa feature would have an open requirement
>>>> on the package javax.transaction [1.1,2) possibly also a requirement for
>>>> a TransactionManager service. These requirements would not even have to
>>>> be defined as the bundles contain all the needed information.
>>>>
>>>> Then when I try to install the jpa feature in karaf the feature service
>>>> could tell me that there are the unresolved requirements above. It could
>>>> then even go through all other features and propose the ones that offer
>>>> the respective capabilities. So it could work like this:
>>>>   > feature:install jpa
>>>> Unresolved requirements:
>>>> Package: javax.persistence [1.1,2)
>>>> Service: javax.transaction.TransactionManager
>>>> The requirements can be resolved when also installing
>>>> 1. transaction transaction-api/1.1
>>>> 2. transaction transaction-api/1.2
>>>>
>>>> In an interactive mode the user could then select the features to also
>>>> install from the list. In a non interactive mode there would just be an
>>>> error and the user would have to add the features to the command.
>>>>
>>>> In the same way it would work for a whole set of features:
>>>>   > feature:install jpa/2.1.1 hibernate/4.3.6.Final
>>>> Unresolved requirements:
>>>> Package: javax.persistence [1.2,2)
>>>> Service: javax.transaction.TransactionManager
>>>> The requirements can be resolved when also installing
>>>> 1. transaction transaction-api/1.2
>>>>
>>>> In this case as there is only one solution the feature service could
>>>> even auto install the necessary features without them being mentioned in
>>>> any of the features to be installed. So it would act a bit like and OBR
>>>> on feature level.
>>>>
>>>> I think it would make much more sense to handle features like this than
>>>> how we do it right now. It would allow to use the new requirements and
>>>> capabilties and the new resolver much better. At the same time the
>>>> features themselves would become much more loosely coupled and easier to
>>>> maintain.
>>>>
>>>> WDYT?
>>>>
>>>> Christian
>>>>
>>>>
>>>
>> --
>> Jean-Baptiste Onofré
>> jbonofre@apache.org
>> http://blog.nanthrax.net
>> Talend - http://www.talend.com
>>
>

-- 
Jean-Baptiste Onofré
jbonofre@apache.org
http://blog.nanthrax.net
Talend - http://www.talend.com