You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@jena.apache.org by Claude Warren <cl...@xenei.com> on 2015/05/19 21:25:32 UTC
contract tests
There is a set of contract tests (and test helpers) on the
"add-contract-tests" branch. That branch works and has minimal change from
the current tests. Those changes are adding the junit-contract runner and
plugins.
It makes no change to the execution.
The problem that I am having is keeping it up to date with the current
change rate of the Jena packages. Granted the contract tests are only
implemented for the jena-core module, we have been keeing the entire suite
up to date.
Is there anyone that has any objection to moving the contract tests to the
main code branch?
Claude
--
I like: Like Like - The likeliest place on the web
<http://like-like.xenei.com>
LinkedIn: http://www.linkedin.com/in/claudewarren
Re: contract tests
Posted by Claude Warren <cl...@xenei.com>.
GraphMem_CS is the contract suite. As you have figured out it detects the
tests that need to be run. You will probably want to look at
GraphContractTests.java as well for the other half of the code.
There are 2 ways to find out what is discovered.
the command line client (wait for new packaging later today) and the maven
plugin. Which brings us to the role of the plugin.
The plugin produces 3 text files. They should be in
jena-core/target/contract_tests (or some thing similar) but I don't see it
on the jenkins server so it must have been disabled.
All of the reports have the ability to filter out specific classes, keep
that in mind while reading the descriptions below.
interfaces.txt
contains a list of all the discovered interfaces and the contract tests for
them as well as a list of all classes with flags to indicate if they are
annotated with @Contract, @ContractImpl, or @NoContract and a list of all
annotations on the class. This data drives the following 2 reports and is
really here so someone can perform a sanity check on the data.
untestedInterfaces.txt
contains a list of all interfaces that do not have contract tests.
unimplemented.txt
contains a list of all classes that implement an interface that has a
contract test but for which there is no contract suite.
It looks like I need to create another report showing the contract tests
that are executed for each suite.
As to the logging messages, it may be that some of them should be moved
from info to debug. I'll look into that. If you have specific suggestions
I am open to them.
Claude
On Fri, Jun 12, 2015 at 5:33 PM, Andy Seaborne <an...@apache.org> wrote:
> Claude,
>
> I trying to understand the contract testing. I have been looking at
> GraphMem_CS and DeltaTest mainly. Are they good examples?
>
> What's the way to find out what classes are discovered and tested?
>
> contract-test-maven-plugin -- what's the role of this plugin? because the
> surefire setup has: <include>**/*_CS.java</include>
>
> There is various logging messages now as well but some seem to come from
> include the Contract engine. Should they be suppressed?
>
> see for example:
>
>
> https://builds.apache.org/user/andy/my-views/view/Jena/job/Jena_Development_Test/lastSuccessfulBuild/consoleFull
>
> Andy
>
>
--
I like: Like Like - The likeliest place on the web
<http://like-like.xenei.com>
LinkedIn: http://www.linkedin.com/in/claudewarren
Re: contract tests
Posted by Andy Seaborne <an...@apache.org>.
Claude,
I trying to understand the contract testing. I have been looking at
GraphMem_CS and DeltaTest mainly. Are they good examples?
What's the way to find out what classes are discovered and tested?
contract-test-maven-plugin -- what's the role of this plugin? because
the surefire setup has: <include>**/*_CS.java</include>
There is various logging messages now as well but some seem to come from
include the Contract engine. Should they be suppressed?
see for example:
https://builds.apache.org/user/andy/my-views/view/Jena/job/Jena_Development_Test/lastSuccessfulBuild/consoleFull
Andy
Re: contract tests
Posted by Andy Seaborne <an...@apache.org>.
On 20/05/15 18:48, Claude Warren wrote:
> parallel for now, but I expect we would move a fair number of tests to it
> as a replacement.
I'm not so sure replacement is the right thing to do and I'd welcome
veryone's thoughts. It would introduce a signifcant dependency on
org.xenei:junit-contracts. With all due respect, having longterm
(years!) jena testing depend on that is not ideal.
Andy
>
>
> On Wed, May 20, 2015 at 10:25 AM, Andy Seaborne <an...@apache.org> wrote:
>
>> On 19/05/15 20:25, Claude Warren wrote:
>>
>>> There is a set of contract tests (and test helpers) on the
>>> "add-contract-tests" branch. That branch works and has minimal change
>>> from
>>> the current tests. Those changes are adding the junit-contract runner and
>>> plugins.
>>>
>>> It makes no change to the execution.
>>>
>>> The problem that I am having is keeping it up to date with the current
>>> change rate of the Jena packages. Granted the contract tests are only
>>> implemented for the jena-core module, we have been keeing the entire suite
>>> up to date.
>>>
>>> Is there anyone that has any objection to moving the contract tests to the
>>> main code branch?
>>>
>>
>> Seems like a good idea - would this be in parallel to the existing tests,
>> or a partial replacement?
>>
>> Andy
>>
>>
>>> Claude
>>>
>>>
>>
>
>
Re: contract tests
Posted by Claude Warren <cl...@xenei.com>.
parallel for now, but I expect we would move a fair number of tests to it
as a replacement.
On Wed, May 20, 2015 at 10:25 AM, Andy Seaborne <an...@apache.org> wrote:
> On 19/05/15 20:25, Claude Warren wrote:
>
>> There is a set of contract tests (and test helpers) on the
>> "add-contract-tests" branch. That branch works and has minimal change
>> from
>> the current tests. Those changes are adding the junit-contract runner and
>> plugins.
>>
>> It makes no change to the execution.
>>
>> The problem that I am having is keeping it up to date with the current
>> change rate of the Jena packages. Granted the contract tests are only
>> implemented for the jena-core module, we have been keeing the entire suite
>> up to date.
>>
>> Is there anyone that has any objection to moving the contract tests to the
>> main code branch?
>>
>
> Seems like a good idea - would this be in parallel to the existing tests,
> or a partial replacement?
>
> Andy
>
>
>> Claude
>>
>>
>
--
I like: Like Like - The likeliest place on the web
<http://like-like.xenei.com>
LinkedIn: http://www.linkedin.com/in/claudewarren
Re: contract tests
Posted by Andy Seaborne <an...@apache.org>.
On 19/05/15 20:25, Claude Warren wrote:
> There is a set of contract tests (and test helpers) on the
> "add-contract-tests" branch. That branch works and has minimal change from
> the current tests. Those changes are adding the junit-contract runner and
> plugins.
>
> It makes no change to the execution.
>
> The problem that I am having is keeping it up to date with the current
> change rate of the Jena packages. Granted the contract tests are only
> implemented for the jena-core module, we have been keeing the entire suite
> up to date.
>
> Is there anyone that has any objection to moving the contract tests to the
> main code branch?
Seems like a good idea - would this be in parallel to the existing
tests, or a partial replacement?
Andy
>
> Claude
>
Re: contract tests
Posted by Claude Warren <cl...@xenei.com>.
I got the cli extracted from the main junit-contract code and managed to
get it released. I will update the jena pom this evening to use the new
version. That should remove the CLI dependency.
On Thu, Jun 11, 2015 at 4:10 PM, Andy Seaborne <an...@apache.org> wrote:
> On 11/06/15 14:35, Claude Warren wrote:
>
>> I'll have to go back and review the dependency tree.
>>
>> The contract tests should only introduce:
>> 1) a junit runner and associated annotations.
>> 2) a maven mojo to report on the testing structure.
>>
>> I think the client requirement comes in becaue there is a command line
>> version of the mojo. I'll see what I can do about extracting that into a
>> command line module separate from the main code -- shouldn't be hard.
>>
>> As for the non-apache licensed pieces.... is there a way to generate a
>> list
>> of them? All the xenei code is apache, the only other bits should be
>> junit
>> and maven mojo depdendencies.
>>
>> Claude
>>
>>
> Hi Claude,
>
> I looked at the output of "mvn dependency:tree" and for org.xenei,
> org.mockito, here is an extract of output:
>
> +- org.xenei:junit-contracts:jar:0.1.2:test
> | +- commons-cli:commons-cli:jar:1.2:test
> | \- commons-io:commons-io:jar:2.4:test
> +- org.xenei:contract-test-maven-plugin:jar:0.1.2:test
> | +- org.apache.maven:maven-plugin-api:jar:3.2.5:test
> | | +- org.apache.maven:maven-model:jar:3.2.5:test
> | | +- org.apache.maven:maven-artifact:jar:3.2.5:test
> | | \- org.eclipse.sisu:org.eclipse.sisu.plexus:jar:0.3.0.M1:test
> | | +- javax.enterprise:cdi-api:jar:1.0:test
> | | | \- javax.annotation:jsr250-api:jar:1.0:test
> | | \- org.eclipse.sisu:org.eclipse.sisu.inject:jar:0.3.0.M1:test
> | +- org.apache.maven:maven-core:jar:3.2.5:test
> | | +- org.apache.maven:maven-settings:jar:3.2.5:test
> | | +- org.apache.maven:maven-settings-builder:jar:3.2.5:test
> | | +- org.apache.maven:maven-repository-metadata:jar:3.2.5:test
> | | +- org.apache.maven:maven-model-builder:jar:3.2.5:test
> | | +- org.apache.maven:maven-aether-provider:jar:3.2.5:test
> | | | \- org.eclipse.aether:aether-spi:jar:1.0.0.v20140518:test
> | | +- org.eclipse.aether:aether-impl:jar:1.0.0.v20140518:test
> | | +- org.eclipse.aether:aether-api:jar:1.0.0.v20140518:test
> | | +- org.eclipse.aether:aether-util:jar:1.0.0.v20140518:test
> | | +- org.sonatype.sisu:sisu-guice:jar:no_aop:3.2.3:test
> | | | +- javax.inject:javax.inject:jar:1:test
> | | | \- aopalliance:aopalliance:jar:1.0:test
> | | +- org.codehaus.plexus:plexus-interpolation:jar:1.21:test
> | | +- org.codehaus.plexus:plexus-utils:jar:3.0.20:test
> | | +- org.codehaus.plexus:plexus-classworlds:jar:2.5.2:test
> | | +- org.codehaus.plexus:plexus-component-annotations:jar:1.5.5:test
> | | \- org.sonatype.plexus:plexus-sec-dispatcher:jar:1.3:test
> | | \- org.sonatype.plexus:plexus-cipher:jar:1.4:test
> | \- org.apache.maven:maven-compat:jar:3.2.5:test
> | \- org.apache.maven.wagon:wagon-provider-api:jar:2.8:test
> +- org.mockito:mockito-all:jar:1.9.5:test
>
>
> On Thu, Jun 11, 2015 at 12:36 PM, Andy Seaborne <an...@apache.org> wrote:
>>
>> On 19/05/15 20:25, Claude Warren wrote:
>>>
>>> There is a set of contract tests (and test helpers) on the
>>>> "add-contract-tests" branch. That branch works and has minimal change
>>>> from
>>>> the current tests. Those changes are adding the junit-contract runner
>>>> and
>>>> plugins.
>>>>
>>>> It makes no change to the execution.
>>>>
>>>> The problem that I am having is keeping it up to date with the current
>>>> change rate of the Jena packages. Granted the contract tests are only
>>>> implemented for the jena-core module, we have been keeing the entire
>>>> suite
>>>> up to date.
>>>>
>>>> Is there anyone that has any objection to moving the contract tests to
>>>> the
>>>> main code branch?
>>>>
>>>> Claude
>>>>
>>>>
>>>> Claude,
>>>
>>> There are some rough edges:
>>>
>>> 1/ The POM has been converted from spaces to tabs around commit 608c2b4
>>>
>>> 2/ There are new dependencies, and recursive some are not Apache
>>> Licensed. Is there anything that should go in LICENSE or NOTICE, even if
>>> that comes via org.apache.maven?
>>>
>>> 3/ Why does testing depend on commons-cli?
>>>
>>> PR#76 proposes use of commons:common-cli (different version)
>>> which caused me to see that.
>>>
>>> 4/ Can we have version mgt in jena-parent please? All other version mgt
>>> or non-jena dependencies is done there (jena-* ones have to be done
>>> explicitly for the release plug-in)
>>>
>>> Andy
>>>
>>>
>>
>>
>>
>
--
I like: Like Like - The likeliest place on the web
<http://like-like.xenei.com>
LinkedIn: http://www.linkedin.com/in/claudewarren
Re: contract tests
Posted by Andy Seaborne <an...@apache.org>.
On 11/06/15 14:35, Claude Warren wrote:
> I'll have to go back and review the dependency tree.
>
> The contract tests should only introduce:
> 1) a junit runner and associated annotations.
> 2) a maven mojo to report on the testing structure.
>
> I think the client requirement comes in becaue there is a command line
> version of the mojo. I'll see what I can do about extracting that into a
> command line module separate from the main code -- shouldn't be hard.
>
> As for the non-apache licensed pieces.... is there a way to generate a list
> of them? All the xenei code is apache, the only other bits should be junit
> and maven mojo depdendencies.
>
> Claude
>
Hi Claude,
I looked at the output of "mvn dependency:tree" and for org.xenei,
org.mockito, here is an extract of output:
+- org.xenei:junit-contracts:jar:0.1.2:test
| +- commons-cli:commons-cli:jar:1.2:test
| \- commons-io:commons-io:jar:2.4:test
+- org.xenei:contract-test-maven-plugin:jar:0.1.2:test
| +- org.apache.maven:maven-plugin-api:jar:3.2.5:test
| | +- org.apache.maven:maven-model:jar:3.2.5:test
| | +- org.apache.maven:maven-artifact:jar:3.2.5:test
| | \- org.eclipse.sisu:org.eclipse.sisu.plexus:jar:0.3.0.M1:test
| | +- javax.enterprise:cdi-api:jar:1.0:test
| | | \- javax.annotation:jsr250-api:jar:1.0:test
| | \- org.eclipse.sisu:org.eclipse.sisu.inject:jar:0.3.0.M1:test
| +- org.apache.maven:maven-core:jar:3.2.5:test
| | +- org.apache.maven:maven-settings:jar:3.2.5:test
| | +- org.apache.maven:maven-settings-builder:jar:3.2.5:test
| | +- org.apache.maven:maven-repository-metadata:jar:3.2.5:test
| | +- org.apache.maven:maven-model-builder:jar:3.2.5:test
| | +- org.apache.maven:maven-aether-provider:jar:3.2.5:test
| | | \- org.eclipse.aether:aether-spi:jar:1.0.0.v20140518:test
| | +- org.eclipse.aether:aether-impl:jar:1.0.0.v20140518:test
| | +- org.eclipse.aether:aether-api:jar:1.0.0.v20140518:test
| | +- org.eclipse.aether:aether-util:jar:1.0.0.v20140518:test
| | +- org.sonatype.sisu:sisu-guice:jar:no_aop:3.2.3:test
| | | +- javax.inject:javax.inject:jar:1:test
| | | \- aopalliance:aopalliance:jar:1.0:test
| | +- org.codehaus.plexus:plexus-interpolation:jar:1.21:test
| | +- org.codehaus.plexus:plexus-utils:jar:3.0.20:test
| | +- org.codehaus.plexus:plexus-classworlds:jar:2.5.2:test
| | +- org.codehaus.plexus:plexus-component-annotations:jar:1.5.5:test
| | \- org.sonatype.plexus:plexus-sec-dispatcher:jar:1.3:test
| | \- org.sonatype.plexus:plexus-cipher:jar:1.4:test
| \- org.apache.maven:maven-compat:jar:3.2.5:test
| \- org.apache.maven.wagon:wagon-provider-api:jar:2.8:test
+- org.mockito:mockito-all:jar:1.9.5:test
> On Thu, Jun 11, 2015 at 12:36 PM, Andy Seaborne <an...@apache.org> wrote:
>
>> On 19/05/15 20:25, Claude Warren wrote:
>>
>>> There is a set of contract tests (and test helpers) on the
>>> "add-contract-tests" branch. That branch works and has minimal change
>>> from
>>> the current tests. Those changes are adding the junit-contract runner and
>>> plugins.
>>>
>>> It makes no change to the execution.
>>>
>>> The problem that I am having is keeping it up to date with the current
>>> change rate of the Jena packages. Granted the contract tests are only
>>> implemented for the jena-core module, we have been keeing the entire suite
>>> up to date.
>>>
>>> Is there anyone that has any objection to moving the contract tests to the
>>> main code branch?
>>>
>>> Claude
>>>
>>>
>> Claude,
>>
>> There are some rough edges:
>>
>> 1/ The POM has been converted from spaces to tabs around commit 608c2b4
>>
>> 2/ There are new dependencies, and recursive some are not Apache
>> Licensed. Is there anything that should go in LICENSE or NOTICE, even if
>> that comes via org.apache.maven?
>>
>> 3/ Why does testing depend on commons-cli?
>>
>> PR#76 proposes use of commons:common-cli (different version)
>> which caused me to see that.
>>
>> 4/ Can we have version mgt in jena-parent please? All other version mgt
>> or non-jena dependencies is done there (jena-* ones have to be done
>> explicitly for the release plug-in)
>>
>> Andy
>>
>
>
>
Re: contract tests
Posted by Claude Warren <cl...@xenei.com>.
I'll have to go back and review the dependency tree.
The contract tests should only introduce:
1) a junit runner and associated annotations.
2) a maven mojo to report on the testing structure.
I think the client requirement comes in becaue there is a command line
version of the mojo. I'll see what I can do about extracting that into a
command line module separate from the main code -- shouldn't be hard.
As for the non-apache licensed pieces.... is there a way to generate a list
of them? All the xenei code is apache, the only other bits should be junit
and maven mojo depdendencies.
Claude
On Thu, Jun 11, 2015 at 12:36 PM, Andy Seaborne <an...@apache.org> wrote:
> On 19/05/15 20:25, Claude Warren wrote:
>
>> There is a set of contract tests (and test helpers) on the
>> "add-contract-tests" branch. That branch works and has minimal change
>> from
>> the current tests. Those changes are adding the junit-contract runner and
>> plugins.
>>
>> It makes no change to the execution.
>>
>> The problem that I am having is keeping it up to date with the current
>> change rate of the Jena packages. Granted the contract tests are only
>> implemented for the jena-core module, we have been keeing the entire suite
>> up to date.
>>
>> Is there anyone that has any objection to moving the contract tests to the
>> main code branch?
>>
>> Claude
>>
>>
> Claude,
>
> There are some rough edges:
>
> 1/ The POM has been converted from spaces to tabs around commit 608c2b4
>
> 2/ There are new dependencies, and recursive some are not Apache
> Licensed. Is there anything that should go in LICENSE or NOTICE, even if
> that comes via org.apache.maven?
>
> 3/ Why does testing depend on commons-cli?
>
> PR#76 proposes use of commons:common-cli (different version)
> which caused me to see that.
>
> 4/ Can we have version mgt in jena-parent please? All other version mgt
> or non-jena dependencies is done there (jena-* ones have to be done
> explicitly for the release plug-in)
>
> Andy
>
--
I like: Like Like - The likeliest place on the web
<http://like-like.xenei.com>
LinkedIn: http://www.linkedin.com/in/claudewarren
Re: contract tests
Posted by Andy Seaborne <an...@apache.org>.
On 19/05/15 20:25, Claude Warren wrote:
> There is a set of contract tests (and test helpers) on the
> "add-contract-tests" branch. That branch works and has minimal change from
> the current tests. Those changes are adding the junit-contract runner and
> plugins.
>
> It makes no change to the execution.
>
> The problem that I am having is keeping it up to date with the current
> change rate of the Jena packages. Granted the contract tests are only
> implemented for the jena-core module, we have been keeing the entire suite
> up to date.
>
> Is there anyone that has any objection to moving the contract tests to the
> main code branch?
>
> Claude
>
Claude,
There are some rough edges:
1/ The POM has been converted from spaces to tabs around commit 608c2b4
2/ There are new dependencies, and recursive some are not Apache
Licensed. Is there anything that should go in LICENSE or NOTICE, even
if that comes via org.apache.maven?
3/ Why does testing depend on commons-cli?
PR#76 proposes use of commons:common-cli (different version)
which caused me to see that.
4/ Can we have version mgt in jena-parent please? All other version mgt
or non-jena dependencies is done there (jena-* ones have to be done
explicitly for the release plug-in)
Andy