Automated tests, at least under /bundles

[Bertrand Delacretaz <bd...@apache.org>]

Hi,

We just had a little ironic exchange with Carsten in SLING-2813...am I
the only one who thinks that all our core code should be covered by
automated tests?

I don't mean requiring stupid 100% test coverage everywhere, as that
doesn't always make sense, but adding code to our core (i.e. under
/bundles) with no tests at all feels very wrong to me.

Can we agree on requiring at least "decent" automated tests coverage
for everything that's under /bundles?

-Bertrand

Re: Automated tests, at least under /bundles

[Felix Meschberger <fm...@adobe.com>]

Who could say "no" to this question ? ;-)

Regards
Felix

Am 05.04.2013 um 10:53 schrieb Bertrand Delacretaz:

> Hi,
> 
> We just had a little ironic exchange with Carsten in SLING-2813...am I
> the only one who thinks that all our core code should be covered by
> automated tests?
> 
> I don't mean requiring stupid 100% test coverage everywhere, as that
> doesn't always make sense, but adding code to our core (i.e. under
> /bundles) with no tests at all feels very wrong to me.
> 
> Can we agree on requiring at least "decent" automated tests coverage
> for everything that's under /bundles?
> 
> -Bertrand


--
Felix Meschberger | Principal Scientist | Adobe

Re: Automated tests, at least under /bundles

[Ian Boston <ie...@tfd.co.uk>]

On Saturday, April 6, 2013, Bertrand Delacretaz wrote:

> On Fri, Apr 5, 2013 at 2:22 PM, Justin Edelson <justin@justinedelson.com<javascript:;>>
> wrote:
> > ...To be clear, are you proposing that no commit can be made to Sling
> without
> > automated tests?...
>
> For core stuff I think we should clearly aim for that.



Aim, but please don't make it a condition of commit. We are humans and we
need to be able to be pragmatic if we are to achieve perfection.



>
> Another metric that might be more flexible is to look at test coverage
> before releasing any module, and going back to adding tests if we're
> not happy with the result.


+1, it's one of a number of metrics that should inform readiness to
release. Quality of coverage is more important than absolute coverage.
Documentation and readability are also important, softer, metrics.

For me, code coverage is about being able to sleep at night, knowing, that
somewhere in the world 10s of 1000s of people's daily lives and future
depend on something I committed to.

Ian



>
> -Bertrand
>

Re: Automated tests, at least under /bundles

[Bertrand Delacretaz <bd...@apache.org>]

On Fri, Apr 5, 2013 at 2:22 PM, Justin Edelson <ju...@justinedelson.com> wrote:
> ...To be clear, are you proposing that no commit can be made to Sling without
> automated tests?...

For core stuff I think we should clearly aim for that.

Another metric that might be more flexible is to look at test coverage
before releasing any module, and going back to adding tests if we're
not happy with the result.

-Bertrand

Re: Automated tests, at least under /bundles

[Justin Edelson <ju...@justinedelson.com>]

Hi Bertrand,
To be clear, are you proposing that no commit can be made to Sling without
automated tests?

While I'm certainly in favor of automated tests (and better documentation),
this is a significantly higher bar than we have today and I just want us to
be crystal clear about what is being proposed.

Regards,
Justin




On Fri, Apr 5, 2013 at 4:53 AM, Bertrand Delacretaz
<bd...@apache.org>wrote:

> Hi,
>
> We just had a little ironic exchange with Carsten in SLING-2813...am I
> the only one who thinks that all our core code should be covered by
> automated tests?
>
> I don't mean requiring stupid 100% test coverage everywhere, as that
> doesn't always make sense, but adding code to our core (i.e. under
> /bundles) with no tests at all feels very wrong to me.
>
> Can we agree on requiring at least "decent" automated tests coverage
> for everything that's under /bundles?
>
> -Bertrand
>

RE: Automated tests, at least under /bundles

[Mike Müller <mi...@mysign.ch>]

+1

To achieve that, it would be good, if there would be some
documentation available how testing stuff (especially integration
tests) should be done in the Sling project.

best regards
Mike

> -----Original Message-----
> From: Bertrand Delacretaz [mailto:bdelacretaz@apache.org]
> Sent: Friday, April 05, 2013 11:11 AM
> To: dev@sling.apache.org
> Subject: Re: Automated tests, at least under /bundles
> 
> On Fri, Apr 5, 2013 at 10:56 AM, Carsten Ziegeler <cz...@apache.org> wrote:
> > ...basically +1 - but I don't consider this a blocker
> >
> > We have the same situation with documentation...
> 
> The impact of incomplete documentation is way less than having
> untested code to our core.
> 
> Either we care about Sling being a high-quality framework, and we
> supply tests to demonstrate it and ensure it stays that way - or it's
> just a pile of code that's supposed to work.
> 
> I'm in favor of the former ;-)
> 
> -Bertrand

Re: Automated tests, at least under /bundles

[Carsten Ziegeler <cz...@apache.org>]

+1


2013/4/5 Bertrand Delacretaz <bd...@apache.org>

> On Fri, Apr 5, 2013 at 10:56 AM, Carsten Ziegeler <cz...@apache.org>
> wrote:
> > ...basically +1 - but I don't consider this a blocker
> >
> > We have the same situation with documentation...
>
> The impact of incomplete documentation is way less than having
> untested code to our core.
>
> Either we care about Sling being a high-quality framework, and we
> supply tests to demonstrate it and ensure it stays that way - or it's
> just a pile of code that's supposed to work.
>
> I'm in favor of the former ;-)
>
> -Bertrand
>



-- 
Carsten Ziegeler
cziegeler@apache.org

RE: Automated tests, at least under /bundles

[Dan Klco <da...@sixdimensions.com>]

+1 to documentation and unit tests.  Especially documentation in the form of examples. 

-----Original Message-----
From: Oliver Lietz [mailto:apache@oliverlietz.de] 
Sent: Friday, April 05, 2013 6:37 AM
To: dev@sling.apache.org
Subject: Re: Automated tests, at least under /bundles

Am Freitag, 5. April 2013 schrieb Bertrand Delacretaz:
> On Fri, Apr 5, 2013 at 10:56 AM, Carsten Ziegeler 
> <cz...@apache.org>
wrote:

hello,

> > ...basically +1 - but I don't consider this a blocker
> > 
> > We have the same situation with documentation...
> 
> The impact of incomplete documentation is way less than having 
> untested code to our core.

incomplete documentation has a huge impact on 3rd parties...

> Either we care about Sling being a high-quality framework, and we 
> supply tests to demonstrate it and ensure it stays that way - or it's 
> just a pile of code that's supposed to work.

and without good documentation it IS just a pile of code.

> I'm in favor of the former ;-)
> 
> -Bertrand

O.


-----
No virus found in this message.
Checked by AVG - www.avg.com
Version: 2013.0.3272 / Virus Database: 3162/6224 - Release Date: 04/04/13

Re: Automated tests, at least under /bundles

[Oliver Lietz <ap...@oliverlietz.de>]

Am Freitag, 5. April 2013 schrieb Bertrand Delacretaz:
> On Fri, Apr 5, 2013 at 10:56 AM, Carsten Ziegeler <cz...@apache.org> 
wrote:

hello,

> > ...basically +1 - but I don't consider this a blocker
> > 
> > We have the same situation with documentation...
> 
> The impact of incomplete documentation is way less than having
> untested code to our core.

incomplete documentation has a huge impact on 3rd parties...

> Either we care about Sling being a high-quality framework, and we
> supply tests to demonstrate it and ensure it stays that way - or it's
> just a pile of code that's supposed to work.

and without good documentation it IS just a pile of code.

> I'm in favor of the former ;-)
> 
> -Bertrand

O.

Re: Automated tests, at least under /bundles

[Bertrand Delacretaz <bd...@apache.org>]

On Fri, Apr 5, 2013 at 10:56 AM, Carsten Ziegeler <cz...@apache.org> wrote:
> ...basically +1 - but I don't consider this a blocker
>
> We have the same situation with documentation...

The impact of incomplete documentation is way less than having
untested code to our core.

Either we care about Sling being a high-quality framework, and we
supply tests to demonstrate it and ensure it stays that way - or it's
just a pile of code that's supposed to work.

I'm in favor of the former ;-)

-Bertrand

Re: Automated tests, at least under /bundles

[Carsten Ziegeler <cz...@apache.org>]

While I totally agree that we should try this, there is no way to enforce
it - and it always depends on personal opinion about whether something is
worth testing or is too simple to be tested - yes, I know better being safe
than sorry.

So, basically +1 - but I don't consider this a blocker

We have the same situation with documentation...

Regards
Carsten


2013/4/5 Bertrand Delacretaz <bd...@apache.org>

> Hi,
>
> We just had a little ironic exchange with Carsten in SLING-2813...am I
> the only one who thinks that all our core code should be covered by
> automated tests?
>
> I don't mean requiring stupid 100% test coverage everywhere, as that
> doesn't always make sense, but adding code to our core (i.e. under
> /bundles) with no tests at all feels very wrong to me.
>
> Can we agree on requiring at least "decent" automated tests coverage
> for everything that's under /bundles?
>
> -Bertrand
>



-- 
Carsten Ziegeler
cziegeler@apache.org