Re: contract tests

[Andy Seaborne <an...@apache.org>]

On 20/05/15 18:48, Claude Warren wrote:
> parallel for now, but I expect we would move a fair number of tests to it
> as a replacement.

I'm not so sure replacement is the right thing to do and I'd welcome 
veryone's thoughts.  It would introduce a signifcant dependency on 
org.xenei:junit-contracts.   With all due respect, having longterm 
(years!) jena testing depend on that is not ideal.

	Andy

>
>
> On Wed, May 20, 2015 at 10:25 AM, Andy Seaborne <an...@apache.org> wrote:
>
>> On 19/05/15 20:25, Claude Warren wrote:
>>
>>> There is a set of contract tests (and test helpers) on the
>>> "add-contract-tests" branch.  That branch works and has minimal change
>>> from
>>> the current tests.  Those changes are adding the junit-contract runner and
>>> plugins.
>>>
>>> It makes no change to the execution.
>>>
>>> The problem that I am having is keeping it up to date with the current
>>> change rate of the Jena packages.  Granted the contract tests are only
>>> implemented for the jena-core module, we have been keeing the entire suite
>>> up to date.
>>>
>>> Is there anyone that has any objection to moving the contract tests to the
>>> main code branch?
>>>
>>
>> Seems like a good idea - would this be in parallel to the existing tests,
>> or a partial replacement?
>>
>>          Andy
>>
>>
>>> Claude
>>>
>>>
>>
>
>

Re: contract tests

[Claude Warren <cl...@xenei.com>]

GraphMem_CS is the contract suite.  As you have figured out it detects the
tests that need to be run.  You will probably want to look at
GraphContractTests.java as well for the other half of the code.

There are 2 ways to find out what is discovered.

the command line client (wait for new packaging later today) and the maven
plugin.  Which brings us to the role of the plugin.

The plugin produces 3 text files.  They should be in
jena-core/target/contract_tests (or some thing similar) but I don't see it
on the jenkins server so it must have been disabled.

All of the reports have the ability to filter out specific classes, keep
that in mind while reading the descriptions below.

interfaces.txt
contains a list of all the discovered interfaces and the contract tests for
them as well as a list of all classes with flags to indicate if they are
annotated with @Contract, @ContractImpl, or @NoContract and a list of all
annotations on the class.  This data drives the following 2 reports and is
really here so someone can perform a sanity check on the data.

untestedInterfaces.txt
contains a list of all interfaces that do not have contract tests.

unimplemented.txt
contains a list of all classes that implement an interface that has a
contract test but for which there is no contract suite.

It looks like I need to create another report showing the contract tests
that are executed for each suite.

As to the logging messages, it may be that some of them should be moved
from info to debug.  I'll look into that.  If you have specific suggestions
I am open to them.

Claude







On Fri, Jun 12, 2015 at 5:33 PM, Andy Seaborne <an...@apache.org> wrote:

> Claude,
>
> I trying to understand the contract testing.  I have been looking at
> GraphMem_CS and DeltaTest mainly.  Are they good examples?
>
> What's the way to find out what classes are discovered and tested?
>
> contract-test-maven-plugin -- what's the role of this plugin? because the
> surefire setup has:   <include>**/*_CS.java</include>
>
> There is various logging messages now as well but some seem to come from
> include the Contract engine.  Should they be suppressed?
>
> see for example:
>
>
> https://builds.apache.org/user/andy/my-views/view/Jena/job/Jena_Development_Test/lastSuccessfulBuild/consoleFull
>
>         Andy
>
>


-- 
I like: Like Like - The likeliest place on the web
<http://like-like.xenei.com>
LinkedIn: http://www.linkedin.com/in/claudewarren

Re: contract tests

[Andy Seaborne <an...@apache.org>]

Claude,

I trying to understand the contract testing.  I have been looking at 
GraphMem_CS and DeltaTest mainly.  Are they good examples?

What's the way to find out what classes are discovered and tested?

contract-test-maven-plugin -- what's the role of this plugin? because 
the surefire setup has: 	<include>**/*_CS.java</include>

There is various logging messages now as well but some seem to come from 
include the Contract engine.  Should they be suppressed?

see for example:

https://builds.apache.org/user/andy/my-views/view/Jena/job/Jena_Development_Test/lastSuccessfulBuild/consoleFull

	Andy