You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@jmeter.apache.org by Jens Dietrich <je...@gmail.com> on 2014/10/20 03:34:46 UTC
API stability and versioning
Hi all,
My name is Jens Dietrich, I am an academic from Massey Uni in NZ.
We recently did a study on (binary and src) compatibility in Java
programs using the Qualitas Corpus dataset of about 100 real-world
Java programs. JMeter is part of this data set (see
http://qualitascorpus.com/docs/catalogue/20130901/corpus_catalogue-jmeter.html
for some extracted stats), and is unique as it was one of only two
programs (the other one being FreeCol) that are "API stable", i.e., we
did not detect API-breaking changes (like changing the signatures or
API methods) between versions.
I wonder whether somebody could offer some insights why this is the
case. Are there any particular project-specific policies to ensure API
stability, and if so, how are they enforced? Also, how are the
decisions about version numbers being made ?
Any help is highly appreciated !
Cheers, Jens
Re: API stability and versioning
Posted by Jens Dietrich <je...@gmail.com>.
Hi Philippe,
Thanks for the info, this is helpful. The link I sent pointed to the
project meta data in the corpus dataset, not to the actual study. A
preprint of the study is available here:
https://sites.google.com/site/jensdietrich/publications/preprints (its the
first document, "Broken Promises .."). There are also some slides that
summarise this project and a developer survey on the topic (with surprising
results):
http://www.slideshare.net/JensDietrich/what-java-developers-dont-know-about-api-compatibility-35628432
.
It would be interesting to get more input on the version policy used. Have
you used tools like clirr <http://clirr.sourceforge.net/> in the project,
and are you aware of semantic versioning initiatives like semver.org ?
Cheers, Jens
On Sat, Nov 1, 2014 at 3:59 AM, Philippe Mouawad <philippe.mouawad@gmail.com
> wrote:
> Hello Jens,
>
> Thank you for your interest in JMeter.
> Interesting study.
> 1 question on it:
> - Clicking on the provided link, how can we see this analysis of "stable
> API" ?
>
>
> Some answsers, only my 2 cents:
> - We take care in the project not to break existing behaviour unless
> clearly documented in changes, but I don't see a direct link with API.
> - We also tend to conserve backward compatibiliy regarding properties (and
> related behaviour) and the "public" APIs , among them I see:
>
> - SampleResult
> - JMeterVariables
> - JMeterContext
> - Sampler
> - BeanInfoSupport
>
> - Regarding "internal APIs" more related to plugin development, we nearly
> have an Abstract base class for all of them in order to ease changes
> without breaking subclasses, but this can happen, I remember it happened in
> version 2.10 I think
> - We have a number of Unit Test and Overall Tests (that run JMeter on
> Sample Test plans) that we maintain accross versions
> - Regarding version numbers, our policy is to increase the second number
> for major versions and the last one for hotfixes. The first one is in fact
> very rarely updated and maybe sebb can answer on why a switch from 1.9.1 to
> 2.0.1 occured in the past
>
>
> Regards
> Philippe M.
> @philmdot
>
> On Mon, Oct 20, 2014 at 3:34 AM, Jens Dietrich <je...@gmail.com>
> wrote:
>
> > Hi all,
> >
> > My name is Jens Dietrich, I am an academic from Massey Uni in NZ.
> >
> > We recently did a study on (binary and src) compatibility in Java
> > programs using the Qualitas Corpus dataset of about 100 real-world
> > Java programs. JMeter is part of this data set (see
> >
> >
> http://qualitascorpus.com/docs/catalogue/20130901/corpus_catalogue-jmeter.html
> > for some extracted stats), and is unique as it was one of only two
> > programs (the other one being FreeCol) that are "API stable", i.e., we
> > did not detect API-breaking changes (like changing the signatures or
> > API methods) between versions.
> >
> > I wonder whether somebody could offer some insights why this is the
> > case. Are there any particular project-specific policies to ensure API
> > stability, and if so, how are they enforced? Also, how are the
> > decisions about version numbers being made ?
> >
> > Any help is highly appreciated !
> >
> > Cheers, Jens
> >
>
>
>
> --
> Cordialement.
> Philippe Mouawad.
>
Re: API stability and versioning
Posted by Philippe Mouawad <ph...@gmail.com>.
Hello Jens,
Thank you for your interest in JMeter.
Interesting study.
1 question on it:
- Clicking on the provided link, how can we see this analysis of "stable
API" ?
Some answsers, only my 2 cents:
- We take care in the project not to break existing behaviour unless
clearly documented in changes, but I don't see a direct link with API.
- We also tend to conserve backward compatibiliy regarding properties (and
related behaviour) and the "public" APIs , among them I see:
- SampleResult
- JMeterVariables
- JMeterContext
- Sampler
- BeanInfoSupport
- Regarding "internal APIs" more related to plugin development, we nearly
have an Abstract base class for all of them in order to ease changes
without breaking subclasses, but this can happen, I remember it happened in
version 2.10 I think
- We have a number of Unit Test and Overall Tests (that run JMeter on
Sample Test plans) that we maintain accross versions
- Regarding version numbers, our policy is to increase the second number
for major versions and the last one for hotfixes. The first one is in fact
very rarely updated and maybe sebb can answer on why a switch from 1.9.1 to
2.0.1 occured in the past
Regards
Philippe M.
@philmdot
On Mon, Oct 20, 2014 at 3:34 AM, Jens Dietrich <je...@gmail.com>
wrote:
> Hi all,
>
> My name is Jens Dietrich, I am an academic from Massey Uni in NZ.
>
> We recently did a study on (binary and src) compatibility in Java
> programs using the Qualitas Corpus dataset of about 100 real-world
> Java programs. JMeter is part of this data set (see
>
> http://qualitascorpus.com/docs/catalogue/20130901/corpus_catalogue-jmeter.html
> for some extracted stats), and is unique as it was one of only two
> programs (the other one being FreeCol) that are "API stable", i.e., we
> did not detect API-breaking changes (like changing the signatures or
> API methods) between versions.
>
> I wonder whether somebody could offer some insights why this is the
> case. Are there any particular project-specific policies to ensure API
> stability, and if so, how are they enforced? Also, how are the
> decisions about version numbers being made ?
>
> Any help is highly appreciated !
>
> Cheers, Jens
>
--
Cordialement.
Philippe Mouawad.