You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@karaf.apache.org by Tim Jones <ti...@mccarthy.co.nz> on 2012/08/21 03:08:58 UTC
Guidance on integrated testing solutions
Hello,
firstly apologies upfront if any of my terminology is incorrect, I am newbie
to OSGI and struggling with the concepts.
I am working on a project that uses Karaf with Felix to provide OSGI based
services that service a Swing GUI client.
The services have been developed over the past 3 years or so but don't have
any automated integration tests.
My goal is to be able to run integration tests within a continuous
integration tool such as Jenkins.
I am looking for guidance on possible integration test solutions. I have
tried to implement a test using Pax Exam and
had a brief look at SpringDM but I am having difficulty provisioning the
required bundles.
The project consists of about a dozen maven modules which build OSGI
bundles. There are some custom configuration properties files
with properties such as org.osgi.framework.system.packages.extra and
org.osgi.framework.system.packages, also an XML
file specifying the features.
I am not sure that I fully understand but it seems to me that in order to
get Pax Exam to test these bundles I must
necessarily specify all of bundles that the bundle in question to test
requires e.g. given a bundle called 'broker' using maven
coordinates I would have do something like the following
.
.
.
mavenBundle().groupId( "org.springframework" ).artifactId( "spring-beans" )
mavenBundle().groupId( "org.springframework" ).artifactId( "spring-tx" )
mavenBundle().groupId( "uniworks" ).artifactId( "broker" )
I have seen cases with someone has written a method to try and read files
from a directory to try and reduce the
manual work but these solutions seem a poor fit for our requirements as the
project depends on 100+ external libraries
most of which are OSGIsed, and some not, which are specified in the
org.osgi.framework.system.packages.extra property.
The analogy I would like to describe is integration testing with the Spring
framework (AbstractTransactionalSpringContextTests or similar)
where often the configuration application context files used in production
can be reused verbatim (with a few changes, say different database)
when running the tests. This to me seems to provide a huge advantage over
having to restate and constantly maintain the 'configuration'
needed to test the classes.
I started to look at
http://felix.apache.org/site/apache-felix-ipojo-junit4osgi-maven.html which
if I understand correctly should allow
tests to be run against the bundles as they are currently configured. So
provided the OSGI bundles can be started before the maven integration-test
phase, say
in the pre-integration-test phase, then the tests should be able to be
implemented without having to specifically specify all the required bundles
and config files
needed to start the dozen or so bundles to perform the integration test.
Note that I am not interested (at least at the moment) in testing a bundle
in isolation so the 'running' system as whole (connects to database(s),
legacy COBOL server(s))
is required to complete a test. Please, I do not want to get into a
philosophical debate on testing in general i.e. whether you have a opinion
on testing systems
in their entirety or more finely grained.
Am I on the right track with iPOJO? Are there better/other alternatives?
Thanks,
Tim
--
View this message in context: http://karaf.922171.n3.nabble.com/Guidance-on-integrated-testing-solutions-tp4025742.html
Sent from the Karaf - User mailing list archive at Nabble.com.
Re: Guidance on integrated testing solutions
Posted by David Jencks <da...@yahoo.com>.
Can't you often run itests in the same maven project as the code by using the failsafe plugin which runs tests after packaging is complete?
thanks
david jencks
On Aug 23, 2012, at 4:41 AM, Brian Topping wrote:
> Tim,
>
> I'm coming in late on this and haven't read the entire thread, but we've started some publicly visible integration tests with pax-exam-karaf as well, you can find them at http://source.marketcetera.org/root/branches/3.0.0/itests/. I point it out because it is so bare right now that it's a great place to start from. (Note this tree is a *very* fluid work-in-progress though!)
>
> Depending on your experience level with Maven, you may ignore this pointer, but note that you must set up a separate project for your tests in this manner, with dependencies on the projects that you want to test as POM dependencies. This is because surefire runs before the project artifact for a specific build is pushed to the repository, and of course without the artifact in place, one can't test it. This is desirable though, as it forces developers to not treat the integration tests as unit tests. The dependencies, simply ensure that Maven reactor runs the itests after the modules under test have successfully built.
>
> The only issue that I've found was noted the other day, that the Karaf artifacts in the maven repository are sometimes missing. In that case, the Karaf binary archive may be missing, and you'll have to build Karaf from source in order for your repository to be properly populated before running the tests.
>
> Hope that helps,
>
> Brian
>
>
> On Aug 23, 2012, at 1:00 PM, Ioannis Canellos <io...@gmail.com> wrote:
>
>> Hi Tim,
>>
>> what pax-exam-karaf provides in addition to plain pax-exam testing, is that the test is run inside a real karaf container and you don't need to describe how you need the container itself to be provisioned.
>>
>> On top of that, it allows you to make use of all Karaf services from inside your test, including the features service itself which make provisioning of your application inside the integration test even more easy.
>>
>> In your test you can inject any OSGi service. You don't need to use karaf commands at all if you don't wish. But seeing how the executCommand method is implemented might help you understand how you can reuse OSGi services.
>>
>> I have some additional links to provide with automated itests using pax-exam-karaf:
>>
>> https://github.com/jclouds/jclouds-karaf/tree/master/itests
>> https://svn.apache.org/repos/asf/whirr/trunk/platforms/karaf/itests/
>> https://github.com/fusesource/fuse/tree/master/fabric/fabric-itests/fabric-pax-exam
>>
>> Again, in those tests you'll see using the commands a lot, but that's more because I was trying to emulate use cases that reproduce bugs etc. Its not mandatory to do so.
>>
>> --
>> Ioannis Canellos
>> FuseSource
>>
>> Blog: http://iocanel.blogspot.com
>> Twitter: iocanel
>>
>>
>
Re: Guidance on integrated testing solutions
Posted by Brian Topping <to...@codehaus.org>.
Tim,
I'm coming in late on this and haven't read the entire thread, but we've started some publicly visible integration tests with pax-exam-karaf as well, you can find them at http://source.marketcetera.org/root/branches/3.0.0/itests/. I point it out because it is so bare right now that it's a great place to start from. (Note this tree is a *very* fluid work-in-progress though!)
Depending on your experience level with Maven, you may ignore this pointer, but note that you must set up a separate project for your tests in this manner, with dependencies on the projects that you want to test as POM dependencies. This is because surefire runs before the project artifact for a specific build is pushed to the repository, and of course without the artifact in place, one can't test it. This is desirable though, as it forces developers to not treat the integration tests as unit tests. The dependencies, simply ensure that Maven reactor runs the itests after the modules under test have successfully built.
The only issue that I've found was noted the other day, that the Karaf artifacts in the maven repository are sometimes missing. In that case, the Karaf binary archive may be missing, and you'll have to build Karaf from source in order for your repository to be properly populated before running the tests.
Hope that helps,
Brian
On Aug 23, 2012, at 1:00 PM, Ioannis Canellos <io...@gmail.com> wrote:
> Hi Tim,
>
> what pax-exam-karaf provides in addition to plain pax-exam testing, is that the test is run inside a real karaf container and you don't need to describe how you need the container itself to be provisioned.
>
> On top of that, it allows you to make use of all Karaf services from inside your test, including the features service itself which make provisioning of your application inside the integration test even more easy.
>
> In your test you can inject any OSGi service. You don't need to use karaf commands at all if you don't wish. But seeing how the executCommand method is implemented might help you understand how you can reuse OSGi services.
>
> I have some additional links to provide with automated itests using pax-exam-karaf:
>
> https://github.com/jclouds/jclouds-karaf/tree/master/itests
> https://svn.apache.org/repos/asf/whirr/trunk/platforms/karaf/itests/
> https://github.com/fusesource/fuse/tree/master/fabric/fabric-itests/fabric-pax-exam
>
> Again, in those tests you'll see using the commands a lot, but that's more because I was trying to emulate use cases that reproduce bugs etc. Its not mandatory to do so.
>
> --
> Ioannis Canellos
> FuseSource
>
> Blog: http://iocanel.blogspot.com
> Twitter: iocanel
>
>
Re: Guidance on integrated testing solutions
Posted by Tim Jones <ti...@mccarthy.co.nz>.
Thanks all for the replies.
I have looked through examples provided in the URLs and have implemented the
example PaxExam Karaf integration test in
https://github.com/openengsb/labs-paxexam-karaf/wiki.
I am still a bit confused. Something I should have said earlier is that in
this case Karaf is shipped in the application and Karaf is started with the
application using org.apache.karaf.main.Main
Given that the maven build/installation process creates the target directory
structure with the necessary files (e.g. property files, features, bundles).
Just before invoking Main.main() our application code sets some system
properties e.g. 'karaf.home' and 'karaf.base'. Looking at the source code in
Main these properties then form the basis from which the code can launch the
container and customise (e.g. add user defined features) into the running
container.
I think the bit I am struggling with is how to 'bootstrap' our system using
paxexam-karaf, I can't understand from the URLs you sent me how this can be
achieved.
I was hoping it could be as simple as setting system properties in the
config() method of the test as below. The container would then be started
using similar logic as in Main i.e. using the system properties to find a
directory structure that would ultimately dictate how to provision the
container.
@Configuration
public Option[] config() {
return new Option[]{
karafDistributionConfiguration()
.frameworkUrl(maven().groupId("org.apache.karaf").artifactId("apache-karaf").type("zip").versionAsInProject())
.karafVersion("2.2.5")
.name("Apache Karaf"),
systemProperty("karaf.home").value("/j2ee/uniworks/server/target/classes/osgi"),
systemProperty("karaf.base").value("/j2ee/uniworks/server/target/classes/osgi")
};
}
I am obviously missing some important concepts here as I can see from the
log the KarafTestContainer starting but not starting the application's
bundles so I am guessing the idea above is quite wrong.
Thanks,
Tim
--
View this message in context: http://karaf.922171.n3.nabble.com/Guidance-on-integrated-testing-solutions-tp4025742p4025803.html
Sent from the Karaf - User mailing list archive at Nabble.com.
Re: Guidance on integrated testing solutions
Posted by Ioannis Canellos <io...@gmail.com>.
Hi Tim,
what pax-exam-karaf provides in addition to plain pax-exam testing, is that
the test is run inside a real karaf container and you don't need to
describe how you need the container itself to be provisioned.
On top of that, it allows you to make use of all Karaf services from inside
your test, including the features service itself which make provisioning of
your application inside the integration test even more easy.
In your test you can inject any OSGi service. You don't need to use karaf
commands at all if you don't wish. But seeing how the executCommand method
is implemented might help you understand how you can reuse OSGi services.
I have some additional links to provide with automated itests using
pax-exam-karaf:
https://github.com/jclouds/jclouds-karaf/tree/master/itests
https://svn.apache.org/repos/asf/whirr/trunk/platforms/karaf/itests/
https://github.com/fusesource/fuse/tree/master/fabric/fabric-itests/fabric-pax-exam
Again, in those tests you'll see using the commands a lot, but that's more
because I was trying to emulate use cases that reproduce bugs etc. Its not
mandatory to do so.
--
*Ioannis Canellos*
*
FuseSource <http://fusesource.com>
**
Blog: http://iocanel.blogspot.com
**
Twitter: iocanel
*
Re: Guidance on integrated testing solutions
Posted by Jean-Baptiste Onofré <jb...@nanthrax.net>.
Hi Tim,
my answers inline:
> I tried to download the Cellar source for 2.2.4 as per
> http://karaf.apache.org/index/subprojects/cellar/download.html but none of
> the download sites I accessed had 2.2.4, only 2.2.1
You can find the Cellar artifacts here:
http://repo.maven.apache.org/maven2/org/apache/karaf/cellar/apache-karaf-cellar/2.2.4/
However, you are right about the link, it seems that the download
mirrors are have not been updated. I'm going to fix that.
>
> Without being able to run the tests myself, from what I can make out
>
> 1) The CellarSampleDosgiGreeterTest test loads the 'dosgi-greeter'
> features which include the client and service Greeter Services.
Correct
>
> 2) Invoking String greetOutput = executeCommand("dosgi-greeter:greet Hi
> 10"); will execute the 'greet' method on the client? and/or service?
Not it's a client, that will contact the "remote" service.
>
> It does not seem possible to get access to the GreetResponse object itself
> returned from the Greeter service as you have to exercise the methods via
> executing commands only, is this correct?
If you use command yes, but you can also directly access to the service
layer from your tests (and so make a service reference, etc).
Regards
JB
>
>
> Thanks,
>
> Tim
>
>
>
>
>
>
>
>
> --
> View this message in context: http://karaf.922171.n3.nabble.com/Guidance-on-integrated-testing-solutions-tp4025742p4025785.html
> Sent from the Karaf - User mailing list archive at Nabble.com.
>
--
Jean-Baptiste Onofré
jbonofre@apache.org
http://blog.nanthrax.net
Talend - http://www.talend.com
Re: Guidance on integrated testing solutions
Posted by Tim Jones <ti...@mccarthy.co.nz>.
Thanks for the quick response Jean-Baptiste. I have looked through the Cellar
integration tests but not sure I understand everything that is going on.
I tried to download the Cellar source for 2.2.4 as per
http://karaf.apache.org/index/subprojects/cellar/download.html but none of
the download sites I accessed had 2.2.4, only 2.2.1
Without being able to run the tests myself, from what I can make out
1) The CellarSampleDosgiGreeterTest test loads the 'dosgi-greeter'
features which include the client and service Greeter Services.
2) Invoking String greetOutput = executeCommand("dosgi-greeter:greet Hi
10"); will execute the 'greet' method on the client? and/or service?
It does not seem possible to get access to the GreetResponse object itself
returned from the Greeter service as you have to exercise the methods via
executing commands only, is this correct?
Thanks,
Tim
--
View this message in context: http://karaf.922171.n3.nabble.com/Guidance-on-integrated-testing-solutions-tp4025742p4025785.html
Sent from the Karaf - User mailing list archive at Nabble.com.
Re: Guidance on integrated testing solutions
Posted by Jean-Baptiste Onofré <jb...@nanthrax.net>.
Hi Tim,
I invite you to take a look on the Cellar integration tests:
http://svn.apache.org/repos/asf/karaf/cellar/branches/cellar-2.2.x/itests/
It uses karaf-paxexam, allowing to easily bootstrap a (or more) Karaf
instances, execute commands in the instance, and make some assertion.
Regards
JB
On 08/21/2012 03:08 AM, Tim Jones wrote:
> Hello,
>
> firstly apologies upfront if any of my terminology is incorrect, I am newbie
> to OSGI and struggling with the concepts.
>
> I am working on a project that uses Karaf with Felix to provide OSGI based
> services that service a Swing GUI client.
>
> The services have been developed over the past 3 years or so but don't have
> any automated integration tests.
>
> My goal is to be able to run integration tests within a continuous
> integration tool such as Jenkins.
>
> I am looking for guidance on possible integration test solutions. I have
> tried to implement a test using Pax Exam and
> had a brief look at SpringDM but I am having difficulty provisioning the
> required bundles.
>
> The project consists of about a dozen maven modules which build OSGI
> bundles. There are some custom configuration properties files
> with properties such as org.osgi.framework.system.packages.extra and
> org.osgi.framework.system.packages, also an XML
> file specifying the features.
>
> I am not sure that I fully understand but it seems to me that in order to
> get Pax Exam to test these bundles I must
> necessarily specify all of bundles that the bundle in question to test
> requires e.g. given a bundle called 'broker' using maven
> coordinates I would have do something like the following
>
> .
> .
> .
> mavenBundle().groupId( "org.springframework" ).artifactId( "spring-beans" )
> mavenBundle().groupId( "org.springframework" ).artifactId( "spring-tx" )
> mavenBundle().groupId( "uniworks" ).artifactId( "broker" )
>
> I have seen cases with someone has written a method to try and read files
> from a directory to try and reduce the
> manual work but these solutions seem a poor fit for our requirements as the
> project depends on 100+ external libraries
> most of which are OSGIsed, and some not, which are specified in the
> org.osgi.framework.system.packages.extra property.
>
> The analogy I would like to describe is integration testing with the Spring
> framework (AbstractTransactionalSpringContextTests or similar)
> where often the configuration application context files used in production
> can be reused verbatim (with a few changes, say different database)
> when running the tests. This to me seems to provide a huge advantage over
> having to restate and constantly maintain the 'configuration'
> needed to test the classes.
>
> I started to look at
> http://felix.apache.org/site/apache-felix-ipojo-junit4osgi-maven.html which
> if I understand correctly should allow
> tests to be run against the bundles as they are currently configured. So
> provided the OSGI bundles can be started before the maven integration-test
> phase, say
> in the pre-integration-test phase, then the tests should be able to be
> implemented without having to specifically specify all the required bundles
> and config files
> needed to start the dozen or so bundles to perform the integration test.
>
> Note that I am not interested (at least at the moment) in testing a bundle
> in isolation so the 'running' system as whole (connects to database(s),
> legacy COBOL server(s))
> is required to complete a test. Please, I do not want to get into a
> philosophical debate on testing in general i.e. whether you have a opinion
> on testing systems
> in their entirety or more finely grained.
>
> Am I on the right track with iPOJO? Are there better/other alternatives?
>
>
> Thanks,
>
> Tim
>
>
>
> --
> View this message in context: http://karaf.922171.n3.nabble.com/Guidance-on-integrated-testing-solutions-tp4025742.html
> Sent from the Karaf - User mailing list archive at Nabble.com.
>
--
Jean-Baptiste Onofré
jbonofre@apache.org
http://blog.nanthrax.net
Talend - http://www.talend.com