You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@ace.apache.org by Premek Brada <br...@kiv.zcu.cz> on 2010/09/16 14:07:08 UTC
proposal: a versioning repository
Hello all,
I am a researcher and software engineering lecturer at University of West
Bohemia, Czech Republic. In my team we have been researching component
compatibility methods and their applications to industrial technologies,
in particular OSGi.
Some time ago I came to know the Apache ACE project, through Marcel
Offermans.
During our discussions, the idea came up that an automated mechanism of
assigning version numbers to bundles and their exported packages, which we
came up with, could be useful for ACE. To cut a long story short, we
are now able
to reliably generate semantic versions (as per the recent OSGi Technical
Whitepaper)
when a bundle is newly stored in ACE repository. The versions are
stored/updated in
the appropriate bundle manifest headers.
This effectively ensures version-implementation consistency across all
bundles managed
by the given ACE repo instance, in other words, no nasty runtime errors
/ exceptions due
to type mismatches in wired bundles obtained from such a repository.
Also, it means
the bundle programmer can be lazy and leave versioning chores to the
repository ;-)
Now then, I have a couple of questions to the community here:
- First, is the explanation clear enough?
- Does this sound interesting? (To me, it certainly does :-) Why yes/no?
- [if yes] How would you imagine/suggest such a thing should be integrated
into ACE?
We of course have an implementation, it extends the plain
org.apache.ace.obr.storage.BundleStore interface to include versioning --
see the attached pieces of code to get an idea how it works (there is
about 0.5MB of libraries hidden behind this small interface). However,
this is our first cut and by no means an ideal solution.
I would be therefore very much interested in any thoughts and ideas
you may have about this versioning ACE repository.
Best,
Premek
PS: A sort of advertisement: we have a bundle versioning service at
http://osgi.kiv.zcu.cz/obvs/ <http://osgi.kiv.zcu.cz/obvs/index.html> -
you are invited to give it a try, I would be interested in any feedback.
--
Premek Brada (Ing et MSc, PhD)
Lecturer in Software enginering, Erasmus coordinator, Webmaster
Department of Computer Science and Engineering
University of West Bohemia, Pilsen, CZ
<< brada at kiv.zcu.cz | www.kiv.zcu.cz/~brada/ | +420-377-63-2435>>
Re: proposal: a versioning repository
Posted by Premek Brada <br...@kiv.zcu.cz>.
Hello all,
the promised versioning OBR proposal is a wrapper over ApacheFelix's
BundleFileStore implementation + a separate servlet which provides a
REST API. The first part is a VersionedBundleStore interface which
(currently) extends the o.a.ace.obr.storage.BundleStore (maybe it should
actually be a separate interface alongside, or better instead-of, the
plain BundleStore), see attached source which has the necessary javadoc.
The REST API looks as follows, working mostly with bundle data streams
and some HTTP headers.
GET /obr/bundle-symbolic-name/version
=> get(symbolic-name, version)
GET /obr/bundle-symbolic-name
=> getLatest(symbolic-name), plus X-BundleVersion header set
GET /obr/bundle-symbolic-name?version-range
=> getLatest(symbolic-name, VersionRange), plus X-BundleVersion header set
PUT, POST /obr
=> put(), returns the resulting bundle metadata in X-BundleFileName,
X-BundleVersion headers
DELETE /obr/symbolic-name/version
=> remove()
OPTIONS /obr
=> listBundles(), returns the list of bundle symbolic names delimited by
newline in response body
OPTIONS /obr/bundle-symbolic-name
=> listVersions(), returns the list of bundle versions delimited by
newline in response body
We opted for a bundle-aware, bundle-only interface instead of for just
putting the versioning functionality underneath the OBR BundleStore --
we think this makes sense because the other resources which can be
stored in OBR don't lend themselves easily to OSGi versioning policy &
automated versioning; secondly, we think bundles as core elements
deserve this special treatment, and last, it makes the versioning bundle
repo much easier to work with.
Underneath the versioned file bundle store there is a bundle versioning
service based on our type-based bundle compatibility comparator, see the
second attached interface. The complete implementation we have is split
into several bundles, runs on Felix (I can provide it if anyone wants,
currently it is in an internal subversion repo).
I am pretty curious what you think, so any feedback is welcome.
Premek
On 11.2.2011 16:13, Premek Brada wrote:
> On 11.2.2011 15:33, Jeremy Hughes wrote:
>> Hi, I was wondering if anything came of this. dev@aries.a.o we're
>> discussing the need for this kind of thing.
>
> Hi,
>
> we have a prototype implementation working and will be posting a
> proposal to the ace-dev shortly (next week). So stay tuned please :)
>
> Premek
>
>
>> Thanks,
>> Jeremy
>>
>> On 23 September 2010 14:30, Premek Brada<br...@kiv.zcu.cz> wrote:
>>> Hello Marcel,
>>>
>>> back to these emails again:
>>>
>>> On 17.9.2010 23:47, Marcel Offermans wrote:
>>>>>>> - [if yes] How would you imagine/suggest such a thing should be
>>>>>>> integrated
>>>>>>> into ACE?
>>>>>> As you know, currently ACE supports a REST like interface for
>>>>>> OBR, so it
>>>>>> would be nice if we could integrate it at that level. Eventually
>>>>>> we need to
>>>>>> support adding bundles to the OBR from the web UI so integrating
>>>>>> it there
>>>>>> would also be nice.
>>>>> OK, that's a plan already; I would be interested in putting
>>>>> together the
>>>>> signatures for the REST interface.
>>> I suggest that we have a look at this in London, time permitting.
>>>
>>>>> Any suggestions? I haven't yet tried the web UI properly, will
>>>>> explore
>>>>> that and try to come up with ideas.
>>>> The web UI as it is right now consists of 4 columns (artifacts,
>>>> features,
>>>> distributions and targets). Dragging a local file into the artifact
>>>> column
>>>> could trigger an upload of that file to the OBR, and go through your
>>>> versioning tooling.
>>> That looks like a good idea, I have added it to the feature
>>> suggestions to
>>> consider. However, we probably do not have enough people currently
>>> to work
>>> on this; so I would concentrate on the REST interface for the time
>>> being.
>>>
>>>>>>> PS: A sort of advertisement: we have a bundle versioning service at
>>>>>>> http://osgi.kiv.zcu.cz/obvs/<http://osgi.kiv.zcu.cz/obvs/index.html>
>>>>>>> -
>>>>>>> you are invited to give it a try, I would be interested in any
>>>>>>> feedback.
>>>>>> From the service on this website, I would like to get a bit more
>>>>>> feedback about the actual analysis. As a test I submitted
>>>>>> different versions
>>>>>> of the compendium bundle, and that went well (in essence, the
>>>>>> tool concluded
>>>>>> that the versions were correct).
>>>>> What kind of feedback would you like to get?
>>>> I would like to see some kind of side by side comparison of the
>>>> older and
>>>> newer version and any changes that were made to the newer version:
>>> <snip>
>>>
>>> Sensible and interesting indeed, thanks for the idea. I am putting
>>> it to the
>>> task list for the next version of the service.
>>>
>>> Best ,
>>> Premek
>>>
>>>
>>> --
>>> Premek Brada (Ing et MSc, PhD)
>>> Lecturer in Software enginering, Erasmus coordinator, Webmaster
>>> Department of Computer Science and Engineering
>>> University of West Bohemia, Pilsen, CZ
>>> << brada at kiv.zcu.cz | www.kiv.zcu.cz/~brada/ | +420-377-63-2435>>
>>>
>>>
>
>
--
Premek Brada (Ing et MSc, PhD)
Lecturer in Software enginering, Webmaster
Department of Computer Science and Engineering
University of West Bohemia, Pilsen, CZ
<< brada at kiv.zcu.cz | www.kiv.zcu.cz/~brada/ | +420-377-63-2435>>
Re: proposal: a versioning repository
Posted by Premek Brada <br...@kiv.zcu.cz>.
On 11.2.2011 15:33, Jeremy Hughes wrote:
> Hi, I was wondering if anything came of this. dev@aries.a.o we're
> discussing the need for this kind of thing.
Hi,
we have a prototype implementation working and will be posting a
proposal to the ace-dev shortly (next week). So stay tuned please :)
Premek
> Thanks,
> Jeremy
>
> On 23 September 2010 14:30, Premek Brada<br...@kiv.zcu.cz> wrote:
>> Hello Marcel,
>>
>> back to these emails again:
>>
>> On 17.9.2010 23:47, Marcel Offermans wrote:
>>>>>> - [if yes] How would you imagine/suggest such a thing should be
>>>>>> integrated
>>>>>> into ACE?
>>>>> As you know, currently ACE supports a REST like interface for OBR, so it
>>>>> would be nice if we could integrate it at that level. Eventually we need to
>>>>> support adding bundles to the OBR from the web UI so integrating it there
>>>>> would also be nice.
>>>> OK, that's a plan already; I would be interested in putting together the
>>>> signatures for the REST interface.
>> I suggest that we have a look at this in London, time permitting.
>>
>>>> Any suggestions? I haven't yet tried the web UI properly, will explore
>>>> that and try to come up with ideas.
>>> The web UI as it is right now consists of 4 columns (artifacts, features,
>>> distributions and targets). Dragging a local file into the artifact column
>>> could trigger an upload of that file to the OBR, and go through your
>>> versioning tooling.
>> That looks like a good idea, I have added it to the feature suggestions to
>> consider. However, we probably do not have enough people currently to work
>> on this; so I would concentrate on the REST interface for the time being.
>>
>>>>>> PS: A sort of advertisement: we have a bundle versioning service at
>>>>>> http://osgi.kiv.zcu.cz/obvs/<http://osgi.kiv.zcu.cz/obvs/index.html> -
>>>>>> you are invited to give it a try, I would be interested in any feedback.
>>>>> From the service on this website, I would like to get a bit more
>>>>> feedback about the actual analysis. As a test I submitted different versions
>>>>> of the compendium bundle, and that went well (in essence, the tool concluded
>>>>> that the versions were correct).
>>>> What kind of feedback would you like to get?
>>> I would like to see some kind of side by side comparison of the older and
>>> newer version and any changes that were made to the newer version:
>> <snip>
>>
>> Sensible and interesting indeed, thanks for the idea. I am putting it to the
>> task list for the next version of the service.
>>
>> Best ,
>> Premek
>>
>>
>> --
>> Premek Brada (Ing et MSc, PhD)
>> Lecturer in Software enginering, Erasmus coordinator, Webmaster
>> Department of Computer Science and Engineering
>> University of West Bohemia, Pilsen, CZ
>> << brada at kiv.zcu.cz | www.kiv.zcu.cz/~brada/ | +420-377-63-2435>>
>>
>>
--
Premek Brada (Ing et MSc, PhD)
Lecturer in Software enginering, Webmaster
Department of Computer Science and Engineering
University of West Bohemia, Pilsen, CZ
<< brada at kiv.zcu.cz | www.kiv.zcu.cz/~brada/ | +420-377-63-2435>>
Re: proposal: a versioning repository
Posted by Premek Brada <br...@kiv.zcu.cz>.
Hello Marcel,
back to these emails again:
On 17.9.2010 23:47, Marcel Offermans wrote:
>
>>>> - [if yes] How would you imagine/suggest such a thing should be
>>>> integrated
>>>> into ACE?
>>>
>>> As you know, currently ACE supports a REST like interface for OBR,
>>> so it would be nice if we could integrate it at that level.
>>> Eventually we need to support adding bundles to the OBR from the web
>>> UI so integrating it there would also be nice.
>>
>> OK, that's a plan already; I would be interested in putting together
>> the signatures for the REST interface.
I suggest that we have a look at this in London, time permitting.
>> Any suggestions? I haven't yet tried the web UI properly, will
>> explore that and try to come up with ideas.
>
> The web UI as it is right now consists of 4 columns (artifacts,
> features, distributions and targets). Dragging a local file into the
> artifact column could trigger an upload of that file to the OBR, and
> go through your versioning tooling.
That looks like a good idea, I have added it to the feature suggestions
to consider. However, we probably do not have enough people currently to
work on this; so I would concentrate on the REST interface for the time
being.
>
>>>> PS: A sort of advertisement: we have a bundle versioning service at
>>>> http://osgi.kiv.zcu.cz/obvs/
>>>> <http://osgi.kiv.zcu.cz/obvs/index.html> - you are invited to give
>>>> it a try, I would be interested in any feedback.
>>>
>>> From the service on this website, I would like to get a bit more
>>> feedback about the actual analysis. As a test I submitted different
>>> versions of the compendium bundle, and that went well (in essence,
>>> the tool concluded that the versions were correct).
>>
>> What kind of feedback would you like to get?
>
> I would like to see some kind of side by side comparison of the older
> and newer version and any changes that were made to the newer version:
<snip>
Sensible and interesting indeed, thanks for the idea. I am putting it to
the task list for the next version of the service.
Best ,
Premek
--
Premek Brada (Ing et MSc, PhD)
Lecturer in Software enginering, Erasmus coordinator, Webmaster
Department of Computer Science and Engineering
University of West Bohemia, Pilsen, CZ
<< brada at kiv.zcu.cz | www.kiv.zcu.cz/~brada/ | +420-377-63-2435>>
Re: proposal: a versioning repository
Posted by Marcel Offermans <ma...@luminis.nl>.
Hello Premek,
On 17 Sep 2010, at 15:21 , Premek Brada wrote:
> On 17.9.2010 10:24, Marcel Offermans wrote:
>>
>>> This effectively ensures version-implementation consistency across all bundles managed
>>> by the given ACE repo instance, in other words, no nasty runtime errors / exceptions due
>>> to type mismatches in wired bundles obtained from such a repository. Also, it means
>>> the bundle programmer can be lazy and leave versioning chores to the repository ;-)
>>
>> You rightfully put a smiley there, as not all semantic changes can be automatically detected by tooling of course. However, I do like the idea a lot.
>
> That's correct, the versioning algorithm -> API -> tool handles syntactic changes on public class elements only; OTOH, that is in line with the semantic versioning whitepaper which deals mainly with the interface/type/syntactic changes and reserves the micro and qualifier parts of the version identifier for semantic and internal changes. (Leaving aside the issue that detecting, classifying and versioning semantic changes properly is a lot more difficult task.)
Agreed.
>>> - [if yes] How would you imagine/suggest such a thing should be integrated
>>> into ACE?
>>
>> As you know, currently ACE supports a REST like interface for OBR, so it would be nice if we could integrate it at that level. Eventually we need to support adding bundles to the OBR from the web UI so integrating it there would also be nice.
>
> OK, that's a plan already; I would be interested in putting together the signatures for the REST interface. Any suggestions? I haven't yet tried the web UI properly, will explore that and try to come up with ideas.
The web UI as it is right now consists of 4 columns (artifacts, features, distributions and targets). Dragging a local file into the artifact column could trigger an upload of that file to the OBR, and go through your versioning tooling.
>>> We of course have an implementation, it extends the plain
>>> org.apache.ace.obr.storage.BundleStore interface to include versioning --
>>> see the attached pieces of code to get an idea how it works (there is
>>> about 0.5MB of libraries hidden behind this small interface). However,
>>> this is our first cut and by no means an ideal solution.
>>
>> Would you consider donating and supporting this?
>
> In principle, yes. We however first need to sort out intellectual property stuff and get rid of some of the bugs we know about.
Fair enough.
>>> PS: A sort of advertisement: we have a bundle versioning service at
>>> http://osgi.kiv.zcu.cz/obvs/ - you are invited to give it a try, I would be interested in any feedback.
>>
>> From the service on this website, I would like to get a bit more feedback about the actual analysis. As a test I submitted different versions of the compendium bundle, and that went well (in essence, the tool concluded that the versions were correct).
>
> What kind of feedback would you like to get?
I would like to see some kind of side by side comparison of the older and newer version and any changes that were made to the newer version:
bundle-1.0.0 <-> bundle-2.0.0 [was 1.2.0]
From there it would be nice if you could "drill down" from the bundle to its packages:
bundle-1.0.0 <-> bundle-2.0.0 [was 1.0.1]
Exported:
package.a 1.0.0 <-> package.a 2.0.0 [was 1.0.0]
package.b 1.2.0 <-> package.a 1.3.0
Private:
package.c <-> package.c (changed)
From this level of detail, one could see the user forgot about the change in package.a which lead to an incompatible API change, so a major version bump. You also see package.b was versioned correctly, and that the private package.c has changed (but since those are not versioned, you cannot say much more than that).
You could even drill down to class/method level.
Does this make sense?
Greetings, Marcel
Re: proposal: a versioning repository
Posted by Premek Brada <br...@kiv.zcu.cz>.
Hello Marcel,
On 17.9.2010 10:24, Marcel Offermans wrote:
>
>
>> This effectively ensures version-implementation consistency across
>> all bundles managed
>> by the given ACE repo instance, in other words, no nasty runtime
>> errors / exceptions due
>> to type mismatches in wired bundles obtained from such a repository.
>> Also, it means
>> the bundle programmer can be lazy and leave versioning chores to the
>> repository ;-)
>
> You rightfully put a smiley there, as not all semantic changes can be
> automatically detected by tooling of course. However, I do like the
> idea a lot.
That's correct, the versioning algorithm -> API -> tool handles
syntactic changes on public class elements only; OTOH, that is in line
with the semantic versioning whitepaper which deals mainly with the
interface/type/syntactic changes and reserves the micro and qualifier
parts of the version identifier for semantic and internal changes.
(Leaving aside the issue that detecting, classifying and versioning
semantic changes properly is a lot more difficult task.)
>
>> - [if yes] How would you imagine/suggest such a thing should be
>> integrated
>> into ACE?
>
> As you know, currently ACE supports a REST like interface for OBR, so
> it would be nice if we could integrate it at that level. Eventually we
> need to support adding bundles to the OBR from the web UI so
> integrating it there would also be nice.
OK, that's a plan already; I would be interested in putting together the
signatures for the REST interface. Any suggestions? I haven't yet tried
the web UI properly, will explore that and try to come up with ideas.
>
>> We of course have an implementation, it extends the plain
>> org.apache.ace.obr.storage.BundleStore interface to include versioning --
>> see the attached pieces of code to get an idea how it works (there is
>> about 0.5MB of libraries hidden behind this small interface). However,
>> this is our first cut and by no means an ideal solution.
>
> Would you consider donating and supporting this?
In principle, yes. We however first need to sort out intellectual
property stuff and get rid of some of the bugs we know about.
>
>> PS: A sort of advertisement: we have a bundle versioning service at
>> http://osgi.kiv.zcu.cz/obvs/ <http://osgi.kiv.zcu.cz/obvs/index.html>
>> - you are invited to give it a try, I would be interested in any
>> feedback.
>
> From the service on this website, I would like to get a bit more
> feedback about the actual analysis. As a test I submitted different
> versions of the compendium bundle, and that went well (in essence, the
> tool concluded that the versions were correct).
What kind of feedback would you like to get?
Best,
Premek
--
Premek Brada (Ing et MSc, PhD)
Lecturer in Software enginering, Erasmus coordinator, Webmaster
Department of Computer Science and Engineering
University of West Bohemia, Pilsen, CZ
<< brada at kiv.zcu.cz | www.kiv.zcu.cz/~brada/ | +420-377-63-2435>>
Re: proposal: a versioning repository
Posted by Marcel Offermans <ma...@luminis.nl>.
Hello Premek,
On 16 Sep 2010, at 14:07 , Premek Brada wrote:
> I am a researcher and software engineering lecturer at University of West
> Bohemia, Czech Republic. In my team we have been researching component
> compatibility methods and their applications to industrial technologies,
> in particular OSGi.
>
> Some time ago I came to know the Apache ACE project, through Marcel Offermans.
> During our discussions, the idea came up that an automated mechanism of
> assigning version numbers to bundles and their exported packages, which we
> came up with, could be useful for ACE. To cut a long story short, we are now able
> to reliably generate semantic versions (as per the recent OSGi Technical Whitepaper)
> when a bundle is newly stored in ACE repository. The versions are stored/updated in
> the appropriate bundle manifest headers.
Great!
> This effectively ensures version-implementation consistency across all bundles managed
> by the given ACE repo instance, in other words, no nasty runtime errors / exceptions due
> to type mismatches in wired bundles obtained from such a repository. Also, it means
> the bundle programmer can be lazy and leave versioning chores to the repository ;-)
You rightfully put a smiley there, as not all semantic changes can be automatically detected by tooling of course. However, I do like the idea a lot.
> Now then, I have a couple of questions to the community here:
> - First, is the explanation clear enough?
For me, yes. Of course it would be interesting to learn more about how this is implemented, and what actual version policy is being used, but as a general overview I understand it.
> - Does this sound interesting? (To me, it certainly does :-) Why yes/no?
Yes.
> - [if yes] How would you imagine/suggest such a thing should be integrated
> into ACE?
As you know, currently ACE supports a REST like interface for OBR, so it would be nice if we could integrate it at that level. Eventually we need to support adding bundles to the OBR from the web UI so integrating it there would also be nice.
> We of course have an implementation, it extends the plain
> org.apache.ace.obr.storage.BundleStore interface to include versioning --
> see the attached pieces of code to get an idea how it works (there is
> about 0.5MB of libraries hidden behind this small interface). However,
> this is our first cut and by no means an ideal solution.
Would you consider donating and supporting this?
> I would be therefore very much interested in any thoughts and ideas
> you may have about this versioning ACE repository.
> PS: A sort of advertisement: we have a bundle versioning service at
> http://osgi.kiv.zcu.cz/obvs/ - you are invited to give it a try, I would be interested in any feedback.
From the service on this website, I would like to get a bit more feedback about the actual analysis. As a test I submitted different versions of the compendium bundle, and that went well (in essence, the tool concluded that the versions were correct).
Greetings, Marcel