You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@sling.apache.org by Petr Shypila <ik...@gmail.com> on 2015/06/22 11:26:30 UTC
contentloader module testing
Hi Bertrand,
Following our discussion in SLING-4541 I completely agree that unit testing
of BundleContentProvider as pretty hard and difficult to understand. So use
of PaxExam looks a very good idea.
Few days ago I already tried to write some PaxExam tests for
DefaultContentCreator and got a ClassNotFoundException
on DefaultContentCreator class explicit initialization.
I loaded a bundle for test but it doesn't help me. I tried it to initialize
with @Inject annotation but it didn't help me as well. Probably I do
something wrong? Does injection works only for Services?
Best regards,
Petr
Re: contentloader module testing
Posted by Bertrand Delacretaz <bd...@apache.org>.
Hi Petr,
On Fri, Jun 26, 2015 at 9:16 AM, Petr Shypila <ik...@gmail.com> wrote:
> ...The only problem I have now is JaCoCo report....
The parent pom 23 is now released, here's what I get for the
bundles/commons/contentdetection module for example:
$ mvn clean install -P jacoco-report
...
$ ls target/site/jacoco-*/index.html
target/site/jacoco-merged/index.html target/site/jacoco-unit/index.html
The jacoco-it report is missing, I have created SLING-4835 for this,
will have a look today. Feel free to comment there if you have ideas
or if you can confirm my hypothesis.
-Bertrand
Re: contentloader module testing
Posted by Petr Shypila <ik...@gmail.com>.
Hi Bertrand,
I finally solved all problems which I had with this integration tests and
for now everything is clear for me.
Yesterday I was stuck for a while(because of my poor OSGi knowledge), but
as I wrote, now everything is fine.
So integration tests for this module will be presented.
The only problem I have now is JaCoCo report. It looks like it doesn't
creates reports for integration tests(Few weeks ago everything was fine). I
updated contentloader's parent to 24-SNAPSHOT(I tried 23-SNAPSHOT as well)
but it didn't help me.
<parent>
<groupId>org.apache.sling</groupId>
<artifactId>sling</artifactId>
<version>24-SNAPSHOT</version>
<relativePath/>
</parent>
<artifactId>org.apache.sling.jcr.contentloader</artifactId>
<version>2.1.11-SNAPSHOT</version>
<packaging>bundle</packaging>
I build module with this command: mvn clean install -Pjacoco-report.
2015-06-25 12:04 GMT+03:00 Bertrand Delacretaz <bd...@apache.org>:
> Hi Petr,
>
> On Thu, Jun 25, 2015 at 9:40 AM, Petr Shypila <ik...@gmail.com> wrote:
> > ...I also got a look on other modules under ./bundles and it looks like
> engine
> > is a very good one, because it's coverage is only 20%(e.g.
> resourceresolver
> > coverage is 70%)....
>
> Both engine and resourceresolver are good candidates, as they are
> critical core bundles.
>
> I suspect that our launchpad integration tests also cover quite a bit
> of those modules, it might be nice to have a look at the aggregate
> coverage of (for example) the bundles/engine module based on both its
> own tests and on the launchpad/testing tests.
>
> I did experiment with generating that aggregate launchpad + local
> coverage in SLING-1803 (the integration-deploys-jacoco-exec.patch) in
> addition to SLING-4760. Once you're mostly done with the contentloader
> tests it might be good to look at this - if we can easily get that
> aggregate coverage that might help focus your efforts better. That
> patch is probably stale but we could reuse the general idea - having
> launchpad/testing deploy its coverage data to Maven so that other
> modules can merge it with their own coverage.
>
> > ...I see that most tests are written with
> > JMock. Could be there some other useful information for me how to test
> this
> > module in better way?...
>
> I've been moving to Mockito lately, the code is more readable than
> JMock IMO. If a module has few tests so far you might switch it to
> Mockito for consistency. Maybe JMock has also improved in the
> meantime.
>
> -Bertrand
>
Re: contentloader module testing
Posted by Bertrand Delacretaz <bd...@apache.org>.
Hi Petr,
On Thu, Jun 25, 2015 at 9:40 AM, Petr Shypila <ik...@gmail.com> wrote:
> ...I also got a look on other modules under ./bundles and it looks like engine
> is a very good one, because it's coverage is only 20%(e.g. resourceresolver
> coverage is 70%)....
Both engine and resourceresolver are good candidates, as they are
critical core bundles.
I suspect that our launchpad integration tests also cover quite a bit
of those modules, it might be nice to have a look at the aggregate
coverage of (for example) the bundles/engine module based on both its
own tests and on the launchpad/testing tests.
I did experiment with generating that aggregate launchpad + local
coverage in SLING-1803 (the integration-deploys-jacoco-exec.patch) in
addition to SLING-4760. Once you're mostly done with the contentloader
tests it might be good to look at this - if we can easily get that
aggregate coverage that might help focus your efforts better. That
patch is probably stale but we could reuse the general idea - having
launchpad/testing deploy its coverage data to Maven so that other
modules can merge it with their own coverage.
> ...I see that most tests are written with
> JMock. Could be there some other useful information for me how to test this
> module in better way?...
I've been moving to Mockito lately, the code is more readable than
JMock IMO. If a module has few tests so far you might switch it to
Mockito for consistency. Maybe JMock has also improved in the
meantime.
-Bertrand
Re: contentloader module testing
Posted by Petr Shypila <ik...@gmail.com>.
Hi Bertrand,
Thank you so much that you found time to write this example during a
conference. Very appreciate.
If you encounter issues and are blocked, don't hesitate to work on
> other, simpler unit tests in the meantime.
>
That's exactly what I did. Right now my github/trunk branch contains only
unit tests. I'm doing PaxExam tests with all it's dependencies under
another branch.
I also got a look on other modules under ./bundles and it looks like engine
is a very good one, because it's coverage is only 20%(e.g. resourceresolver
coverage is 70%). What do you think? I see that most tests are written with
JMock. Could be there some other useful information for me how to test this
module in better way?
Best regards,
Petr
Re: contentloader module testing
Posted by Bertrand Delacretaz <bd...@apache.org>.
Hi Petr,
On Tue, Jun 23, 2015 at 11:33 PM, Petr Shypila <ik...@gmail.com> wrote:
> ...I have created a test bundle using ./installer/it/pom.xml with required
> Sling-Initial-Content header in manifest file. Bundle installs fine, I can
> get it's id, and get it back by this id. However I don't see any imported
> content in a repository....
That might have been because your bundle was not resolved or not started.
I have now added a BasicInitialContentIT that can use as an example to
create similar tests for the various content loading scenarios and
formats, details in SLING-4757.
This was rather tricky to setup but it should now be easy to reuse the
pattern for similar tests.
If you encounter issues and are blocked, don't hesitate to work on
other, simpler unit tests in the meantime.
-Bertrand
Re: contentloader module testing
Posted by Petr Shypila <ik...@gmail.com>.
Hello Bertrand,
Thank you very much for this email. It helped me a lot. My problem is that
didn't work with OSGi good enough and there is a big gap for me. But I also
agree that I tackle this I'll be well equipped to write a large range of
Sling tests.
I have created a test bundle using ./installer/it/pom.xml with required
Sling-Initial-Content header in manifest file. Bundle installs fine, I can
get it's id, and get it back by this id. However I don't see any imported
content in a repository.
I did everything same as it described on Content Loading webpage[1], but it
still doesn't work.
My bundle[2] contains META-INF/MANIFEST.MF file with Sling-Initial-Content
header and SLING-INF/content/home.xml file as well. However in JCR root
node I see only jcr:system and rep:policy nodes. You can watch my last
changes in my GitHub repo[3] or I have also created a Gist on GitHub[4].
[1]
https://sling.apache.org/documentation/bundles/content-loading-jcr-contentloader.html
[2]
https://github.com/PetrShypila/sling-builder/blob/trunk/bundles/jcr/contentloader/src/test/resources/org.apache.sling.installer.it-3.6.7-SNAPSHOT-testbundle-1.0.jar
[3]
https://github.com/PetrShypila/sling-builder/tree/trunk/bundles/jcr/contentloader
[4] https://gist.github.com/PetrShypila/3a81b26be7d2bd5f995e
Could me someone probably say, what I'm doing wrong?
Thank you so much,
Petr
2015-06-23 18:30 GMT+03:00 Bertrand Delacretaz <bd...@apache.org>:
> Hi Petr,
>
> On Mon, Jun 22, 2015 at 7:41 PM, Petr Shypila <ik...@gmail.com> wrote:
> > ...So the only service which injects fine is
> > o.a.s.jcr.contentloader.ContentReader...
>
> That's correct.
>
> If you look at the details of the org.apache.sling.jcr.contentloader
> bundle at http://localhost:8080/system/console/bundles on a standard
> Sling instance (for example java -jar
> target/org.apache.sling.launchpad-8-SNAPSHOT.jar ad launchpad/builder)
> you see that this bundle only supplies ContentReader services along
> with a ContentReaderWhiteboard service that's an internal class, not
> exported AFAICS. If you look the the details of the ContentReader
> services you can see that they all have an "extensions" property which
> allows the bundle's internal code to select them. This would be
> visible from the source code of course but it's easier to find out in
> that way initially.
>
> You could call those ContentReader services directly for testing them
> but IMO it's more useful to just install new bundles that supply
> initial content and verify that the content is correctly installed as
> per
> https://sling.apache.org/documentation/bundles/content-loading-jcr-contentloader.html
> . And maybe also update and remove those bundles to verify that the
> initial content is correctly handled. You'll probably need the
> RetryRule of the bundles/commons/testing module in your tests as the
> content won't appear immediately after a test bundle is installed.
>
> To create such test bundles you might be able to use
> https://ops4j1.jira.com/wiki/display/ops4j/Tinybundles but I'm not
> sure how to include json or other files in such bundles created on the
> fly - it looks like you just pull stuff from your current classpath so
> that might work. Otherwise you can use the technique of
> ./installer/it/pom.xml to generate a few test bundles at build time
> (look for "Create several simple test bundles" in there).
>
> This contentloader bundle is a bit unusual as it uses the so-called
> "extender pattern" to listen for new bundles that appear with a bundle
> header that's interesting in terms of content loading. The bundle's
> main role is not to provided services, it's to process such bundles,
> so IMO it's best to test that use case directly.
>
> Those tests are a bit tricky to setup though, so feel free to ask if
> you have more questions! Once you tackle this you'll be well equipped
> to write a large range of Sling tests.
>
> -Bertrand
>
Re: contentloader module testing
Posted by Bertrand Delacretaz <bd...@apache.org>.
Hi Petr,
On Mon, Jun 22, 2015 at 7:41 PM, Petr Shypila <ik...@gmail.com> wrote:
> ...So the only service which injects fine is
> o.a.s.jcr.contentloader.ContentReader...
That's correct.
If you look at the details of the org.apache.sling.jcr.contentloader
bundle at http://localhost:8080/system/console/bundles on a standard
Sling instance (for example java -jar
target/org.apache.sling.launchpad-8-SNAPSHOT.jar ad launchpad/builder)
you see that this bundle only supplies ContentReader services along
with a ContentReaderWhiteboard service that's an internal class, not
exported AFAICS. If you look the the details of the ContentReader
services you can see that they all have an "extensions" property which
allows the bundle's internal code to select them. This would be
visible from the source code of course but it's easier to find out in
that way initially.
You could call those ContentReader services directly for testing them
but IMO it's more useful to just install new bundles that supply
initial content and verify that the content is correctly installed as
per https://sling.apache.org/documentation/bundles/content-loading-jcr-contentloader.html
. And maybe also update and remove those bundles to verify that the
initial content is correctly handled. You'll probably need the
RetryRule of the bundles/commons/testing module in your tests as the
content won't appear immediately after a test bundle is installed.
To create such test bundles you might be able to use
https://ops4j1.jira.com/wiki/display/ops4j/Tinybundles but I'm not
sure how to include json or other files in such bundles created on the
fly - it looks like you just pull stuff from your current classpath so
that might work. Otherwise you can use the technique of
./installer/it/pom.xml to generate a few test bundles at build time
(look for "Create several simple test bundles" in there).
This contentloader bundle is a bit unusual as it uses the so-called
"extender pattern" to listen for new bundles that appear with a bundle
header that's interesting in terms of content loading. The bundle's
main role is not to provided services, it's to process such bundles,
so IMO it's best to test that use case directly.
Those tests are a bit tricky to setup though, so feel free to ask if
you have more questions! Once you tackle this you'll be well equipped
to write a large range of Sling tests.
-Bertrand
Re: contentloader module testing
Posted by Petr Shypila <ik...@gmail.com>.
Thank you very much for your help.
Unfortunately I'm still have some problems with services injection.
As I understand I can inject services from o.a.s. jcr.contentloader package.
1. When I try to inject o.a.s. jcr.contentloader.ContentCreator I'm getting
exception: ServiceLookupException: gave up waiting for service
org.apache.sling.jcr.contentloader.ContentCreator. I tried to add
@Filter(timeout=10000L) annotation. But it didn't help me. Tried on Java 6
and 7.
2. When I'm trying to inject o.a.s. jcr.contentloader.ContentCreator I'm
getting exception: java.lang.IllegalArgumentException: Can not set
org.apache.sling.jcr.contentloader.ContentImporter field
org.apache.sling.jcr.contentloader.internal.it.BundleContentLoaderIT.whiteboard
to org.apache.sling.jcr.contentloader.internal.DefaultContentImporter
So the only service which injects fine is
o.a.s.jcr.contentloader.ContentReader.
In logs the only exceptions I found are:
1. org.apache.sling.servlets.post
[org.apache.sling.servlets.post.impl.SlingPostServlet(66)] The
unbindContentImporter method has thrown an exception
(java.lang.IllegalArgumentException: argument type mismatch)
2. org.apache.sling.servlets.post
[org.apache.sling.servlets.post.impl.SlingPostServlet(66)] The
bindContentImporter method has thrown an exception
(java.lang.IllegalArgumentException: argument type mismatch)
Have some of you ever faced with same problem?
Thank you so much,
Petr
2015-06-22 18:34 GMT+03:00 Robert Munteanu <ro...@apache.org>:
> On Mon, 2015-06-22 at 16:02 +0300, Petr Shypila wrote:
> > Thanks for everyone.
> >
> > So as I understand for now Sling Mocks is the preferable and most
> > modern
> > way to write unit tests. Is that correct?
> > What about JCR data store? Does it save data between tests and should
> > take
> > care of that?
>
> It depends on each provider implementation. The JCR_JACKRABBIT resource
> provider type saves the data between runs, while the JCR_OAK one does
> not.
>
> Robert
>
Re: contentloader module testing
Posted by Robert Munteanu <ro...@apache.org>.
On Mon, 2015-06-22 at 16:02 +0300, Petr Shypila wrote:
> Thanks for everyone.
>
> So as I understand for now Sling Mocks is the preferable and most
> modern
> way to write unit tests. Is that correct?
> What about JCR data store? Does it save data between tests and should
> take
> care of that?
It depends on each provider implementation. The JCR_JACKRABBIT resource
provider type saves the data between runs, while the JCR_OAK one does
not.
Robert
Re: contentloader module testing
Posted by Julian Sedding <js...@gmail.com>.
Hi Petr
I'm mostly guessing, so YMMV.
Your test imports "org.apache.sling.jcr.contentloader.internal.*". By
convention the maven-bundle-plugin treats "impl" and "internal"
packages as private, i.e. they are not exported.
Now you're building a PaxExam test, where OSGi semantics are
respected. Therefore the classes are not accessible (as designed).
If you can write your tests in Sling Mocks you should not suffer the
same issue, as they use a conventional java classpath.
I'm wondering what you are trying to test? Does it make sense to test
internals on an OSGi level? Or could you test the internals using
standard unit tests?
Regards
Julian
On Mon, Jun 22, 2015 at 3:02 PM, Petr Shypila <ik...@gmail.com> wrote:
> Thanks for everyone.
>
> So as I understand for now Sling Mocks is the preferable and most modern
> way to write unit tests. Is that correct?
> What about JCR data store? Does it save data between tests and should take
> care of that?
>
> A question about PaxExam. Right now I have problems with
> BundleContentLoader initialization. Every time I getting
> ClassNotFoundException.
> I have added a patch with an example. Could you please take a look and
> write me what I'm doing wrong?
>
> Thank you very much,
> Petr
>
> https://www.dropbox.com/s/8k7v7houbynom6k/PaxExam-ClassNotFoundException.patch?dl=0
>
> 2015-06-22 13:03 GMT+03:00 Robert Munteanu <ro...@apache.org>:
>
>> On Mon, Jun 22, 2015 at 12:48 PM, Bertrand Delacretaz
>> <bd...@apache.org> wrote:
>> > On Mon, Jun 22, 2015 at 11:33 AM, Robert Munteanu <ro...@lmn.ro> wrote:
>> >> ...It would be interesting to see
>> >> what testing scenario you had in mind and how to better address it -
>> >> whether it's Sling mocks or Pax-Exam or something different....
>> >
>> > Yes, I agree that discussing this on a very concrete case would help,
>> > and Petr's intensive work on testing is a good opportunity to come up
>> > with best practices around that.
>> >
>> > My comments on SLING-4541 where based on testing the
>> > (bundles/jcr/contentloader) ContentLoaderService's handling of
>> > incoming and updated bundles: it needs to register incoming bundles
>> > which supply initial content, and correctly handle updates to those
>> > bundles.
>> >
>> > Testing this is quite natural in a real OSGi environment, though I
>> > agree that that runs slows that unit or mocked tests of course.
>> >
>> > Robert, do you think this can be tested in a valid and understandable
>> > way using mocks?
>>
>> Looking at ContentLoaderServiceTest.testBundleResolvedBundleChanged it
>> looks like a perfect example of using Pax-Exam - it relies too much on
>> the OSGi framework properly creating bundle objects and issuing events
>> and too little on the Sling / JCR infrastructure.
>>
>> IMO if a test uses only OSGi ( and not the JCR or Sling APIs ) and
>> does so beyond simple API usage that can be easily served by the OSGi
>> mocks then it should use Pax-Exam.
>>
>> The value for the Sling Mocks comes in when either using a large part
>> of our stack - it's convenient for OSGi + JCR tests for instance.
>>
>> > Setting up pax exam is quite easy using our testing/sling-pax-util
>> > module for example (but we should convert that module to use the Sling
>> > provisioning model now).
>>
>> Speed of setting up an OSGi container is not a major concern for me
>> when writing Pax-Exam tests, but rather all the scaffolding needed to
>> get the container up and running, and then keeping it in sync with the
>> 'latest' bundles. sling-pax-util looks like it goes a long way in
>> simplifying that setup , which is great.
>>
>> What I also see as a blurry line is when to write a Pax-Exam test
>> which uses the repository and when to write a full-fledged integration
>> test. That's also something to discuss as we have different approaches
>> throughout the codebase, but perhaps for later :-)
>>
>> Cheers,
>>
>> Robert
>>
>> > -Bertrand
>>
Re: contentloader module testing
Posted by Bertrand Delacretaz <bd...@apache.org>.
Hi Petr,
On Mon, Jun 22, 2015 at 3:02 PM, Petr Shypila <ik...@gmail.com> wrote:
> ...So as I understand for now Sling Mocks is the preferable and most modern
> way to write unit tests. Is that correct?...
Preferable in certain cases...I'd say in order of setup complexity /
performance you have
1. unit tests
2. generic mocks (Mockito etc) or Sling Mocks
3. pax exam
4. Sling integration tests under launchpad/testing
The general rule is to use the first of those options that works for
what you need to test.
But for option 2. now that we have Sling Mocks they are generally
preferable to generic mocks, as they fit the Sling environment better.
> ...What about JCR data store? Does it save data between tests and should take
> care of that?...
Not sure which one you mean, if you use a RepositoryProvider from
sling.commons.testing then yes, the data persists through all tests.
It makes them faster as the repository is setup only once, but you
need to implement isolation in your tests, using unique paths etc.
I don't know how that works with the Sling Mocks variant, I haven't
used that so far.
>
> A question about PaxExam. Right now I have problems with
> BundleContentLoader initialization. Every time I getting
> ClassNotFoundException...
As soon as you move up to pax exam you're in an OSGi environment,
where only java packages that are exported by the bundle under test
are available.
The package that contains BundleContentLoader is not exported by the
bundle under test (you could see that at /system/console/bundles in a
Sling instance, or look at the bundles META-INF/MANIFEST.MF, or look
for package-info.java source files in the source code to find out), so
you don't see it from a pax exam test. Generally you'll only inject
service APIs in those tests, you can look around the code base for
tests that contain @Inject for examples.
So pax exam is good for black-box testing at the OSGi level, which is
a good way to test services IMO.
-Bertrand
Re: contentloader module testing
Posted by Petr Shypila <ik...@gmail.com>.
Thanks for everyone.
So as I understand for now Sling Mocks is the preferable and most modern
way to write unit tests. Is that correct?
What about JCR data store? Does it save data between tests and should take
care of that?
A question about PaxExam. Right now I have problems with
BundleContentLoader initialization. Every time I getting
ClassNotFoundException.
I have added a patch with an example. Could you please take a look and
write me what I'm doing wrong?
Thank you very much,
Petr
https://www.dropbox.com/s/8k7v7houbynom6k/PaxExam-ClassNotFoundException.patch?dl=0
2015-06-22 13:03 GMT+03:00 Robert Munteanu <ro...@apache.org>:
> On Mon, Jun 22, 2015 at 12:48 PM, Bertrand Delacretaz
> <bd...@apache.org> wrote:
> > On Mon, Jun 22, 2015 at 11:33 AM, Robert Munteanu <ro...@lmn.ro> wrote:
> >> ...It would be interesting to see
> >> what testing scenario you had in mind and how to better address it -
> >> whether it's Sling mocks or Pax-Exam or something different....
> >
> > Yes, I agree that discussing this on a very concrete case would help,
> > and Petr's intensive work on testing is a good opportunity to come up
> > with best practices around that.
> >
> > My comments on SLING-4541 where based on testing the
> > (bundles/jcr/contentloader) ContentLoaderService's handling of
> > incoming and updated bundles: it needs to register incoming bundles
> > which supply initial content, and correctly handle updates to those
> > bundles.
> >
> > Testing this is quite natural in a real OSGi environment, though I
> > agree that that runs slows that unit or mocked tests of course.
> >
> > Robert, do you think this can be tested in a valid and understandable
> > way using mocks?
>
> Looking at ContentLoaderServiceTest.testBundleResolvedBundleChanged it
> looks like a perfect example of using Pax-Exam - it relies too much on
> the OSGi framework properly creating bundle objects and issuing events
> and too little on the Sling / JCR infrastructure.
>
> IMO if a test uses only OSGi ( and not the JCR or Sling APIs ) and
> does so beyond simple API usage that can be easily served by the OSGi
> mocks then it should use Pax-Exam.
>
> The value for the Sling Mocks comes in when either using a large part
> of our stack - it's convenient for OSGi + JCR tests for instance.
>
> > Setting up pax exam is quite easy using our testing/sling-pax-util
> > module for example (but we should convert that module to use the Sling
> > provisioning model now).
>
> Speed of setting up an OSGi container is not a major concern for me
> when writing Pax-Exam tests, but rather all the scaffolding needed to
> get the container up and running, and then keeping it in sync with the
> 'latest' bundles. sling-pax-util looks like it goes a long way in
> simplifying that setup , which is great.
>
> What I also see as a blurry line is when to write a Pax-Exam test
> which uses the repository and when to write a full-fledged integration
> test. That's also something to discuss as we have different approaches
> throughout the codebase, but perhaps for later :-)
>
> Cheers,
>
> Robert
>
> > -Bertrand
>
Re: contentloader module testing
Posted by Robert Munteanu <ro...@apache.org>.
On Mon, 2015-06-22 at 14:58 +0200, Bertrand Delacretaz wrote:
> Hi Robert,
>
> On Mon, Jun 22, 2015 at 12:03 PM, Robert Munteanu <rombert@apache.org
> > wrote:
> > ...IMO if a test uses only OSGi ( and not the JCR or Sling APIs )
> > and
> > does so beyond simple API usage that can be easily served by the
> > OSGi
> > mocks then it should use Pax-Exam....
>
> Ok, sounds good to me as a guideline - we might refine
> http://sling.apache.org/documentation/tutorials-how-tos/testing-sling
> -based-applications.html
> with such guidelines as Petr's project progresses.
>
> > ...Speed of setting up an OSGi container is not a major concern for
> > me
> > when writing Pax-Exam tests, but rather all the scaffolding needed
> > to
> > get the container up and running, and then keeping it in sync with
> > the
> > 'latest' bundles. sling-pax-util looks like it goes a long way in
> > simplifying that setup , which is great...
>
> Yes, the only drag is that if it refers to a snapshot of the bundle
> list that it uses (usually launchpad) which is suboptimal. We might
> move the launchpad's provisioning model to its own module to be able
> to release it more often, so that tests can refer to those releases.
By the provisioning model do you mean
launchpad/builder/src/main/provisioning [2]?
I have been thinking about how we can reuse that data so we can have
more stable building blocks for tests and other applications ; out of
the current features
- boot and standalone are probably quite stable and can be used moved
into separate artifacts
- jackrabbit and oak are in the middle area, updating every now and
then based on upstream releases
- sling is however quite fast-moving and I am not sure we can slow down
the release pace with its current contents.
Perhaps we can split the sling feature further? We have a section for
JCR support [3], one for the web console [4] which look relatively self
-contained.
>
> >
> > ...What I also see as a blurry line is when to write a Pax-Exam
> > test
> > which uses the repository and when to write a full-fledged
> > integration
> > test. That's also something to discuss as we have different
> > approaches
> > throughout the codebase, but perhaps for later :-)...
>
> As for full-fledged do you mean something under launchpad/integration
> tests? My current thinking is to move integration tests closer to
> their modules, as much as we can.
Yes, full-fledged ITs are the ones living under launchpad/integration .
And I definitely agree that ITs should live close to the modules they
test.
>
> For example I recently added ContentAwareMimeTypeServiceImplIT. [1]
> which runs with pax exam, has very little boilerplate code and does
> not mess up the pom too much either, it just requires the pax
> dependencies and some simple maven-failsafe-plugin configs there. I
> think that's a nice way of adding integration tests right in their
> modules, and with the SLING-4760 Jacoco setup you can easily get the
> aggregate coverage of unit + integration tests. If we do that role of
> the the launchpad/integration-tests is really only to check that the
> assembly works, it doesn't have to go into detail anymore.
+1 . At that point, with a fast-starting launchpad, we can even move
the tests into the launchpad/builder project itself and rely on a `mvn
clean verify` to assert that the bundles function correctly together.
Robert
[2]: https://github.com/apache/sling/tree/555c5204b1486c666d612cb9dee72
9b3f2c878fd/launchpad/builder/src/main/provisioning
[3]: https://github.com/apache/sling/blob/555c5204b1486c666d612cb9dee72
9b3f2c878fd/launchpad/builder/src/main/provisioning/sling.txt#L105-L120
[4]: https://github.com/apache/sling/blob/555c5204b1486c666d612cb9dee72
9b3f2c878fd/launchpad/builder/src/main/provisioning/sling.txt#L77-L97
>
> (BTW Petr [1] might be a good example if you want to move to pax exam
> for some of your SLING-4541 tests - you can also create a prototype
> that we can discuss if you prefer)
> -Bertrand
>
> [1] https://svn.apache.org/repos/asf/sling/trunk/bundles/commons/cont
> entdetection/src/test/java/org/apache/sling/commons/contentdetection/
> internal/it/ContentAwareMimeTypeServiceImplIT.java
Re: contentloader module testing
Posted by Bertrand Delacretaz <bd...@apache.org>.
Hi Robert,
On Mon, Jun 22, 2015 at 12:03 PM, Robert Munteanu <ro...@apache.org> wrote:
> ...IMO if a test uses only OSGi ( and not the JCR or Sling APIs ) and
> does so beyond simple API usage that can be easily served by the OSGi
> mocks then it should use Pax-Exam....
Ok, sounds good to me as a guideline - we might refine
http://sling.apache.org/documentation/tutorials-how-tos/testing-sling-based-applications.html
with such guidelines as Petr's project progresses.
> ...Speed of setting up an OSGi container is not a major concern for me
> when writing Pax-Exam tests, but rather all the scaffolding needed to
> get the container up and running, and then keeping it in sync with the
> 'latest' bundles. sling-pax-util looks like it goes a long way in
> simplifying that setup , which is great...
Yes, the only drag is that if it refers to a snapshot of the bundle
list that it uses (usually launchpad) which is suboptimal. We might
move the launchpad's provisioning model to its own module to be able
to release it more often, so that tests can refer to those releases.
>
> ...What I also see as a blurry line is when to write a Pax-Exam test
> which uses the repository and when to write a full-fledged integration
> test. That's also something to discuss as we have different approaches
> throughout the codebase, but perhaps for later :-)...
As for full-fledged do you mean something under launchpad/integration
tests? My current thinking is to move integration tests closer to
their modules, as much as we can.
For example I recently added ContentAwareMimeTypeServiceImplIT. [1]
which runs with pax exam, has very little boilerplate code and does
not mess up the pom too much either, it just requires the pax
dependencies and some simple maven-failsafe-plugin configs there. I
think that's a nice way of adding integration tests right in their
modules, and with the SLING-4760 Jacoco setup you can easily get the
aggregate coverage of unit + integration tests. If we do that role of
the the launchpad/integration-tests is really only to check that the
assembly works, it doesn't have to go into detail anymore.
(BTW Petr [1] might be a good example if you want to move to pax exam
for some of your SLING-4541 tests - you can also create a prototype
that we can discuss if you prefer)
-Bertrand
[1] https://svn.apache.org/repos/asf/sling/trunk/bundles/commons/contentdetection/src/test/java/org/apache/sling/commons/contentdetection/internal/it/ContentAwareMimeTypeServiceImplIT.java
Re: contentloader module testing
Posted by Robert Munteanu <ro...@apache.org>.
On Mon, Jun 22, 2015 at 12:48 PM, Bertrand Delacretaz
<bd...@apache.org> wrote:
> On Mon, Jun 22, 2015 at 11:33 AM, Robert Munteanu <ro...@lmn.ro> wrote:
>> ...It would be interesting to see
>> what testing scenario you had in mind and how to better address it -
>> whether it's Sling mocks or Pax-Exam or something different....
>
> Yes, I agree that discussing this on a very concrete case would help,
> and Petr's intensive work on testing is a good opportunity to come up
> with best practices around that.
>
> My comments on SLING-4541 where based on testing the
> (bundles/jcr/contentloader) ContentLoaderService's handling of
> incoming and updated bundles: it needs to register incoming bundles
> which supply initial content, and correctly handle updates to those
> bundles.
>
> Testing this is quite natural in a real OSGi environment, though I
> agree that that runs slows that unit or mocked tests of course.
>
> Robert, do you think this can be tested in a valid and understandable
> way using mocks?
Looking at ContentLoaderServiceTest.testBundleResolvedBundleChanged it
looks like a perfect example of using Pax-Exam - it relies too much on
the OSGi framework properly creating bundle objects and issuing events
and too little on the Sling / JCR infrastructure.
IMO if a test uses only OSGi ( and not the JCR or Sling APIs ) and
does so beyond simple API usage that can be easily served by the OSGi
mocks then it should use Pax-Exam.
The value for the Sling Mocks comes in when either using a large part
of our stack - it's convenient for OSGi + JCR tests for instance.
> Setting up pax exam is quite easy using our testing/sling-pax-util
> module for example (but we should convert that module to use the Sling
> provisioning model now).
Speed of setting up an OSGi container is not a major concern for me
when writing Pax-Exam tests, but rather all the scaffolding needed to
get the container up and running, and then keeping it in sync with the
'latest' bundles. sling-pax-util looks like it goes a long way in
simplifying that setup , which is great.
What I also see as a blurry line is when to write a Pax-Exam test
which uses the repository and when to write a full-fledged integration
test. That's also something to discuss as we have different approaches
throughout the codebase, but perhaps for later :-)
Cheers,
Robert
> -Bertrand
Re: contentloader module testing
Posted by Bertrand Delacretaz <bd...@apache.org>.
On Mon, Jun 22, 2015 at 11:33 AM, Robert Munteanu <ro...@lmn.ro> wrote:
> ...It would be interesting to see
> what testing scenario you had in mind and how to better address it -
> whether it's Sling mocks or Pax-Exam or something different....
Yes, I agree that discussing this on a very concrete case would help,
and Petr's intensive work on testing is a good opportunity to come up
with best practices around that.
My comments on SLING-4541 where based on testing the
(bundles/jcr/contentloader) ContentLoaderService's handling of
incoming and updated bundles: it needs to register incoming bundles
which supply initial content, and correctly handle updates to those
bundles.
Testing this is quite natural in a real OSGi environment, though I
agree that that runs slows that unit or mocked tests of course.
Robert, do you think this can be tested in a valid and understandable
way using mocks?
Setting up pax exam is quite easy using our testing/sling-pax-util
module for example (but we should convert that module to use the Sling
provisioning model now).
-Bertrand
Re: contentloader module testing
Posted by Robert Munteanu <ro...@lmn.ro>.
Hi Petr,
On Mon, Jun 22, 2015 at 12:26 PM, Petr Shypila <ik...@gmail.com> wrote:
> Hi Bertrand,
>
> Following our discussion in SLING-4541 I completely agree that unit testing
> of BundleContentProvider as pretty hard and difficult to understand.
I'm unable to find a reference to such a class, where does it live?
> So use
> of PaxExam looks a very good idea.
> Few days ago I already tried to write some PaxExam tests for
> DefaultContentCreator and got a ClassNotFoundException
> on DefaultContentCreator class explicit initialization.
> I loaded a bundle for test but it doesn't help me. I tried it to initialize
> with @Inject annotation but it didn't help me as well. Probably I do
> something wrong? Does injection works only for Services?
I think we've been (slowly) moving from Pax-Exam tests to Sling mocks
[1] based tests.
The major difference is that Pax-Exam expects you to build a container
out of a list of OSGi bundles while Sling mocks attempts to wire
everything up from jars in your classpath, so there is no need to set
up an OSGi framework. So while Pax-Exam is closer to integration
testing, Sling Mocks are closer to unit testing based on ( drumroll
... ) mocks.
I personally find testing with Sling mocks simpler and easier to
maintain in the long term and also better suited for Sling, but of
course that's just a personal opinion. It would be interesting to see
what testing scenario you had in mind and how to better address it -
whether it's Sling mocks or Pax-Exam or something different.
Cheers,
Robert
[1]: https://sling.apache.org/documentation/development/sling-mock.html