You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@uima.apache.org by Marshall Schor <ms...@schor.com> on 2013/06/11 21:49:34 UTC
jcasgen-maven-plugin integration test timing
The regular tests for this take about 3 seconds on my laptop.
The integration tests took a very long time (the 2nd one took about 8 minutes !)
[INFO] --- maven-invoker-plugin:1.7:run (integration-test) @
jcasgen-maven-plugin ---
[INFO] Building: classpath\pom.xml
[INFO] ..SUCCESS (23.5 s)
[INFO] Building: m2e\pom.xml
[INFO] ..SUCCESS (529.5 s)
[INFO] Building: multimodule\pom.xml
[INFO] ..SUCCESS (13.4 s)
[INFO] Building: simple\pom.xml
[INFO] ..SUCCESS (9.7 s)
[INFO] -------------------------------------------------
[INFO] Build Summary:
[INFO] Passed: 4, Failed: 0, Errors: 0, Skipped: 0
I ran this again (just in case some initial startup thing was slowing it down)
and got similar results: 19.0 s, 576.1 s, 13.6 s, and 8.5 s.
Anyone have any idea what is going on, why some of these are running so slowly?
It would be good if someone could post a walk through of how the integration tests
are setup and how they work.
-Marshall
Re: jcasgen-maven-plugin integration test timing
Posted by Richard Eckart de Castilho <ri...@gmail.com>.
Add a profile which activates on an IBM vendor URL, e.g. "http://www.ibm.com"? I don't have an IBM JVM here, but you can probably start it and System.out the value of the "java.vendor.url" system property.
-- Richard
Am 11.06.2013 um 22:21 schrieb Marshall Schor <ms...@schor.com>:
> I notice the integration test for this has it's own special pom with a section
> that looks like:
>
> <profiles>
> <profile>
> <id>sun-jvm</id>
> <activation>
> <property>
> <name>java.vendor.url</name>
> <value>http://java.sun.com/</value>
> ...
> </profile>
> <profile>
> <id>osx</id>
> <activation>
> <property>
> <name>java.vendor.url</name>
> <value>http://www.apple.com/</value>
> </property>
> ,,,
> </profile>
> </profiles>
>
> I often am running on an IBM JVM, which is neither of the above.
>
> The profiles set some Java -X and -XX style properties and for the mac, adds a
> dependency for org.eclipse.jdt.launching.macosx to a tycho-surefire-plugin.
>
> Any ideas how to set these for the IBM java?
>
> -Marshall
Re: jcasgen-maven-plugin integration test timing
Posted by Marshall Schor <ms...@schor.com>.
I notice the integration test for this has it's own special pom with a section
that looks like:
<profiles>
<profile>
<id>sun-jvm</id>
<activation>
<property>
<name>java.vendor.url</name>
<value>http://java.sun.com/</value>
...
</profile>
<profile>
<id>osx</id>
<activation>
<property>
<name>java.vendor.url</name>
<value>http://www.apple.com/</value>
</property>
,,,
</profile>
</profiles>
I often am running on an IBM JVM, which is neither of the above.
The profiles set some Java -X and -XX style properties and for the mac, adds a
dependency for org.eclipse.jdt.launching.macosx to a tycho-surefire-plugin.
Any ideas how to set these for the IBM java?
-Marshall
On 6/11/2013 3:49 PM, Marshall Schor wrote:
> The regular tests for this take about 3 seconds on my laptop.
>
> The integration tests took a very long time (the 2nd one took about 8 minutes !)
>
> [INFO] --- maven-invoker-plugin:1.7:run (integration-test) @
> jcasgen-maven-plugin ---
> [INFO] Building: classpath\pom.xml
> [INFO] ..SUCCESS (23.5 s)
> [INFO] Building: m2e\pom.xml
> [INFO] ..SUCCESS (529.5 s)
> [INFO] Building: multimodule\pom.xml
> [INFO] ..SUCCESS (13.4 s)
> [INFO] Building: simple\pom.xml
> [INFO] ..SUCCESS (9.7 s)
> [INFO] -------------------------------------------------
> [INFO] Build Summary:
> [INFO] Passed: 4, Failed: 0, Errors: 0, Skipped: 0
>
> I ran this again (just in case some initial startup thing was slowing it down)
> and got similar results: 19.0 s, 576.1 s, 13.6 s, and 8.5 s.
>
> Anyone have any idea what is going on, why some of these are running so slowly?
>
> It would be good if someone could post a walk through of how the integration tests
> are setup and how they work.
>
> -Marshall
>
Re: jcasgen-maven-plugin integration test timing
Posted by Richard Eckart de Castilho <ri...@gmail.com>.
Btw. getting the m2e integration test running was really the most annoying part of integrating this contribution… at the time I'd have really hated wasting considerable time getting this to work as an integration test (was a full separate Maven module before) and then disabling it.
If you think it's too heavy, I'd agree. The question is, whether it should be fully disabled or whether some two-phase build should be used, e.g. "regular" build first on Jenkins, then, if that worked, run a "slow" build afterwards.
Cheers,
-- Richard
Am 11.06.2013 um 21:57 schrieb Richard Eckart de Castilho <ri...@gmail.com>:
> The m2e integration test takes *very* long because it materializes a full OSGI runtime environment on the machine. The downloads take forever.
>
> Detailed information about the execution of the integration tests can be found in target/it/<test>/build.log (or something like that).
>
> I had considered disabling the m2e integration test, but went for leaving it on for the time being.
>
> -- Richard
>
> Am 11.06.2013 um 21:49 schrieb Marshall Schor <ms...@schor.com>:
>
>> The regular tests for this take about 3 seconds on my laptop.
>>
>> The integration tests took a very long time (the 2nd one took about 8 minutes !)
>>
>> [INFO] --- maven-invoker-plugin:1.7:run (integration-test) @
>> jcasgen-maven-plugin ---
>> [INFO] Building: classpath\pom.xml
>> [INFO] ..SUCCESS (23.5 s)
>> [INFO] Building: m2e\pom.xml
>> [INFO] ..SUCCESS (529.5 s)
>> [INFO] Building: multimodule\pom.xml
>> [INFO] ..SUCCESS (13.4 s)
>> [INFO] Building: simple\pom.xml
>> [INFO] ..SUCCESS (9.7 s)
>> [INFO] -------------------------------------------------
>> [INFO] Build Summary:
>> [INFO] Passed: 4, Failed: 0, Errors: 0, Skipped: 0
>>
>> I ran this again (just in case some initial startup thing was slowing it down)
>> and got similar results: 19.0 s, 576.1 s, 13.6 s, and 8.5 s.
>>
>> Anyone have any idea what is going on, why some of these are running so slowly?
>>
>> It would be good if someone could post a walk through of how the integration tests
>> are setup and how they work.
>>
>> -Marshall
>
Re: jcasgen-maven-plugin integration test timing
Posted by Marshall Schor <ms...@schor.com>.
On 6/11/2013 3:57 PM, Richard Eckart de Castilho wrote:
> The m2e integration test takes *very* long because it materializes a full OSGI runtime environment on the machine. The downloads take forever.
>
> Detailed information about the execution of the integration tests can be found in target/it/<test>/build.log (or something like that).
>
> I had considered disabling the m2e integration test, but went for leaving it on for the time being.
I think that we should run this "occasionally", since it takes so long.
For instance, I expect to do a bunch of running of the full builds for uimaj as
part of getting the release candidate into good shape, and a full build seems to
take on my little laptop 23 minutes, of which approx. 15 1/2 minutes is the
jcasgen maven plugin...
We can leave the running of this configured, but add the -Dinvoker.skip to the
command line.
-Marshall
> -- Richard
>
> Am 11.06.2013 um 21:49 schrieb Marshall Schor <ms...@schor.com>:
>
>> The regular tests for this take about 3 seconds on my laptop.
>>
>> The integration tests took a very long time (the 2nd one took about 8 minutes !)
>>
>> [INFO] --- maven-invoker-plugin:1.7:run (integration-test) @
>> jcasgen-maven-plugin ---
>> [INFO] Building: classpath\pom.xml
>> [INFO] ..SUCCESS (23.5 s)
>> [INFO] Building: m2e\pom.xml
>> [INFO] ..SUCCESS (529.5 s)
>> [INFO] Building: multimodule\pom.xml
>> [INFO] ..SUCCESS (13.4 s)
>> [INFO] Building: simple\pom.xml
>> [INFO] ..SUCCESS (9.7 s)
>> [INFO] -------------------------------------------------
>> [INFO] Build Summary:
>> [INFO] Passed: 4, Failed: 0, Errors: 0, Skipped: 0
>>
>> I ran this again (just in case some initial startup thing was slowing it down)
>> and got similar results: 19.0 s, 576.1 s, 13.6 s, and 8.5 s.
>>
>> Anyone have any idea what is going on, why some of these are running so slowly?
>>
>> It would be good if someone could post a walk through of how the integration tests
>> are setup and how they work.
>>
>> -Marshall
>
Re: jcasgen-maven-plugin integration test timing
Posted by Marshall Schor <ms...@schor.com>.
I tried it, it works but you have to also specify the tests to run. I'll check
in a change.
-Marshall
On 6/12/2013 10:47 AM, Richard Eckart de Castilho wrote:
> Am 12.06.2013 um 16:41 schrieb Marshall Schor <ms...@schor.com>:
>
>> Given that the integration test for m2e involves very large network traffic as well,
>> I'm disabling this one test (leaving the other 3 integration tests, though).
>>
>> It's disabled in a not too good way - I commented out parts of its POM.
>> Improvements welcome :-).
> It should be possibly to exclude specific tests using the invokerTest configuration option:
>
> http://maven.apache.org/plugins/maven-invoker-plugin/integration-test-mojo.html#invokerTest
>
> <configuration>
> <invokerTest>!m2e</invokerTest>
> …
> </configuration
>
> If nobody else does, I'll try later.
>
> -- Richard
Re: jcasgen-maven-plugin integration test timing
Posted by Richard Eckart de Castilho <ri...@gmail.com>.
Am 12.06.2013 um 16:41 schrieb Marshall Schor <ms...@schor.com>:
> Given that the integration test for m2e involves very large network traffic as well,
> I'm disabling this one test (leaving the other 3 integration tests, though).
>
> It's disabled in a not too good way - I commented out parts of its POM.
> Improvements welcome :-).
It should be possibly to exclude specific tests using the invokerTest configuration option:
http://maven.apache.org/plugins/maven-invoker-plugin/integration-test-mojo.html#invokerTest
<configuration>
<invokerTest>!m2e</invokerTest>
…
</configuration
If nobody else does, I'll try later.
-- Richard
Re: jcasgen-maven-plugin integration test timing
Posted by Marshall Schor <ms...@schor.com>.
On 6/12/2013 3:55 AM, Richard Eckart de Castilho wrote:
> As far as I understood, the motivation for the m2e build was to test
> if the right output is produced in the build and if the plugin triggers
> and endless build loop in m2e. Steven would need to say more about that,
> I didn't write the tests. The integration tests do not even have asserts,
> they just check if the builds terminate without errors.
>
> If we only care to make the build faster for the usual developer, then
> we should disable it by default and keep it running on Jenkins. If we
> care about load on Jenkins as well, we should completely disable it
> and only turn it on manually when doing major changes.
>
> I'm fine with both ways.
Given that the integration test for m2e involves very large network traffic as well,
I'm disabling this one test (leaving the other 3 integration tests, though).
It's disabled in a not too good way - I commented out parts of its POM.
Improvements welcome :-).
-Marshall
> -- Richard
>
> Am 12.06.2013 um 00:24 schrieb Marshall Schor <ms...@schor.com>:
>
>> On 6/11/2013 3:57 PM, Richard Eckart de Castilho wrote:
>>> The m2e integration test takes *very* long because it materializes a full OSGI runtime environment on the machine. The downloads take forever.
>>>
>>> Detailed information about the execution of the integration tests can be found in target/it/<test>/build.log (or something like that).
>>>
>>> I had considered disabling the m2e integration test, but went for leaving it on for the time being.
>> After taking another look at the tests, I would support disabling this one test,
>> but leaving the others. (We can enable it temporarily when the jcasgen plugin
>> project is updated, just to verify this build environment).
>>
>> In testing the jcasgen plugin, it seems to me that basic tests are to run this
>> plugin in various maven build scenarios to see if it produces the right things
>> (or at least doesn't crash).
>> The m2e integration test simulates running a special maven build that Eclipse
>> developers use, when developing the Eclipse platform, as I understand it (please
>> correct as needed). Since the vast majority of our users are not in the world
>> of Eclipse developers, perhaps that's not so important.
>>
>> And it also seems unlikely that that one form of test would fail while the
>> others, which also run the jcasgen plugin in some different scenarios, would
>> succeed.
>>
>> WDYT? -Marshall
>
Re: jcasgen-maven-plugin integration test timing
Posted by Richard Eckart de Castilho <ri...@gmail.com>.
As far as I understood, the motivation for the m2e build was to test
if the right output is produced in the build and if the plugin triggers
and endless build loop in m2e. Steven would need to say more about that,
I didn't write the tests. The integration tests do not even have asserts,
they just check if the builds terminate without errors.
If we only care to make the build faster for the usual developer, then
we should disable it by default and keep it running on Jenkins. If we
care about load on Jenkins as well, we should completely disable it
and only turn it on manually when doing major changes.
I'm fine with both ways.
-- Richard
Am 12.06.2013 um 00:24 schrieb Marshall Schor <ms...@schor.com>:
> On 6/11/2013 3:57 PM, Richard Eckart de Castilho wrote:
>> The m2e integration test takes *very* long because it materializes a full OSGI runtime environment on the machine. The downloads take forever.
>>
>> Detailed information about the execution of the integration tests can be found in target/it/<test>/build.log (or something like that).
>>
>> I had considered disabling the m2e integration test, but went for leaving it on for the time being.
> After taking another look at the tests, I would support disabling this one test,
> but leaving the others. (We can enable it temporarily when the jcasgen plugin
> project is updated, just to verify this build environment).
>
> In testing the jcasgen plugin, it seems to me that basic tests are to run this
> plugin in various maven build scenarios to see if it produces the right things
> (or at least doesn't crash).
> The m2e integration test simulates running a special maven build that Eclipse
> developers use, when developing the Eclipse platform, as I understand it (please
> correct as needed). Since the vast majority of our users are not in the world
> of Eclipse developers, perhaps that's not so important.
>
> And it also seems unlikely that that one form of test would fail while the
> others, which also run the jcasgen plugin in some different scenarios, would
> succeed.
>
> WDYT? -Marshall
Re: jcasgen-maven-plugin integration test timing
Posted by Marshall Schor <ms...@schor.com>.
On 6/11/2013 3:57 PM, Richard Eckart de Castilho wrote:
> The m2e integration test takes *very* long because it materializes a full OSGI runtime environment on the machine. The downloads take forever.
>
> Detailed information about the execution of the integration tests can be found in target/it/<test>/build.log (or something like that).
>
> I had considered disabling the m2e integration test, but went for leaving it on for the time being.
After taking another look at the tests, I would support disabling this one test,
but leaving the others. (We can enable it temporarily when the jcasgen plugin
project is updated, just to verify this build environment).
In testing the jcasgen plugin, it seems to me that basic tests are to run this
plugin in various maven build scenarios to see if it produces the right things
(or at least doesn't crash).
The m2e integration test simulates running a special maven build that Eclipse
developers use, when developing the Eclipse platform, as I understand it (please
correct as needed). Since the vast majority of our users are not in the world
of Eclipse developers, perhaps that's not so important.
And it also seems unlikely that that one form of test would fail while the
others, which also run the jcasgen plugin in some different scenarios, would
succeed.
WDYT? -Marshall
Re: jcasgen-maven-plugin integration test timing
Posted by Marshall Schor <ms...@schor.com>.
On 6/11/2013 6:53 PM, Marshall Schor wrote:
> On 6/11/2013 5:31 PM, Richard Eckart de Castilho wrote:
>> Am 11.06.2013 um 23:15 schrieb Marshall Schor <ms...@schor.com>:
>>
>>> On 6/11/2013 5:04 PM, Richard Eckart de Castilho wrote:
>>>> Am 11.06.2013 um 22:55 schrieb Marshall Schor <ms...@schor.com>:
>>>>
>>>>> I think the scenario that is driving the logic to install the maven plugin under
>>>>> test to a special isolated repository may not apply to this one.
>>>>>
>>>>> This is because this plugin is not used to build any other things (other than
>>>>> user projects); that is, it's not used in building any of the UIMA projects.
>>>>>
>>>>> It seems to me that we could operate by having this plugin install itself to the
>>>>> developer's local .m2 repository, and then run the integration test.
>>>> Consider the integration tests of this plugin fail. Then we already have a broken
>>>> plugin installed in the .m2/repository of Jenkins of of the developers machine
>>> True
>>>> Consider further that uimaFIT (or maybe at some point cTakes or whatever) is using
>>>> this plugins in its builds. Their jobs in Jenkins may break just because we let
>>>> a broken plugin escape into the wild.
>>> Well, it would "escape" into the developer's machine, not any further :-) The
>>> fix for this would be "simple" - just delete the plugin in the .m2 repo. The
>>> next invocation would then download a (presumably) good version from Maven
>>> Central :-)
>> On Jenkins it would affect other builds. This could be avoided by activating the "use private maven repository" option for the UIMA SDK builds. It wouldn't exactly safe disk space on the poor Jenkins build slaves of the ASF though.
> I didn't know that on Jenkins, every build (for other projects) shared the same
> .m2 repository. Is that really true? I would have thought that each defined
> "build" had its own workspace, but I don't actually know...
Well, I went to builds.apache.org and took a look at the uimaj-sdk build
configuration, and found, under the "advanced" tab in the build section, some
tic boxes, one of which was to use a private Maven repo. The help for this says
that all jobs on one particular build machine do normally share a .m2 repo.
Learn something new every day:-)
-Marshall
>
> -Marshall
>
Re: jcasgen-maven-plugin integration test timing
Posted by Marshall Schor <ms...@schor.com>.
On 6/11/2013 5:31 PM, Richard Eckart de Castilho wrote:
> Am 11.06.2013 um 23:15 schrieb Marshall Schor <ms...@schor.com>:
>
>> On 6/11/2013 5:04 PM, Richard Eckart de Castilho wrote:
>>> Am 11.06.2013 um 22:55 schrieb Marshall Schor <ms...@schor.com>:
>>>
>>>> I think the scenario that is driving the logic to install the maven plugin under
>>>> test to a special isolated repository may not apply to this one.
>>>>
>>>> This is because this plugin is not used to build any other things (other than
>>>> user projects); that is, it's not used in building any of the UIMA projects.
>>>>
>>>> It seems to me that we could operate by having this plugin install itself to the
>>>> developer's local .m2 repository, and then run the integration test.
>>> Consider the integration tests of this plugin fail. Then we already have a broken
>>> plugin installed in the .m2/repository of Jenkins of of the developers machine
>> True
>>> Consider further that uimaFIT (or maybe at some point cTakes or whatever) is using
>>> this plugins in its builds. Their jobs in Jenkins may break just because we let
>>> a broken plugin escape into the wild.
>> Well, it would "escape" into the developer's machine, not any further :-) The
>> fix for this would be "simple" - just delete the plugin in the .m2 repo. The
>> next invocation would then download a (presumably) good version from Maven
>> Central :-)
> On Jenkins it would affect other builds. This could be avoided by activating the "use private maven repository" option for the UIMA SDK builds. It wouldn't exactly safe disk space on the poor Jenkins build slaves of the ASF though.
I didn't know that on Jenkins, every build (for other projects) shared the same
.m2 repository. Is that really true? I would have thought that each defined
"build" had its own workspace, but I don't actually know...
-Marshall
Re: jcasgen-maven-plugin integration test timing
Posted by Richard Eckart de Castilho <ri...@gmail.com>.
Am 11.06.2013 um 23:15 schrieb Marshall Schor <ms...@schor.com>:
> On 6/11/2013 5:04 PM, Richard Eckart de Castilho wrote:
>> Am 11.06.2013 um 22:55 schrieb Marshall Schor <ms...@schor.com>:
>>
>>> I think the scenario that is driving the logic to install the maven plugin under
>>> test to a special isolated repository may not apply to this one.
>>>
>>> This is because this plugin is not used to build any other things (other than
>>> user projects); that is, it's not used in building any of the UIMA projects.
>>>
>>> It seems to me that we could operate by having this plugin install itself to the
>>> developer's local .m2 repository, and then run the integration test.
>> Consider the integration tests of this plugin fail. Then we already have a broken
>> plugin installed in the .m2/repository of Jenkins of of the developers machine
> True
>>
>> Consider further that uimaFIT (or maybe at some point cTakes or whatever) is using
>> this plugins in its builds. Their jobs in Jenkins may break just because we let
>> a broken plugin escape into the wild.
> Well, it would "escape" into the developer's machine, not any further :-) The
> fix for this would be "simple" - just delete the plugin in the .m2 repo. The
> next invocation would then download a (presumably) good version from Maven
> Central :-)
On Jenkins it would affect other builds. This could be avoided by activating the "use private maven repository" option for the UIMA SDK builds. It wouldn't exactly safe disk space on the poor Jenkins build slaves of the ASF though.
> For me, this kind of thing would be a good trade-off if we could get the
> integration test for the plugin to work in a few seconds instead of many minutes.
I don't know… to increase the turn-around on a developer machine, how abound using
"-DskipInvocation=true" to skip the integration tests and only run them occasionally?
If good measures are taken to contain broken artifacts, I don't find a good argument for a -1, but I wouldn't +1 either.
That disk space appears to be a scarce resource on the ASF Jenkins may be slightly in favor of an alternative to activating the private repository option, like skipping integration tests unless one wants to run them… or skipping them by default but activating them on Jenkins.
-- Richard
Re: jcasgen-maven-plugin integration test timing
Posted by Marshall Schor <ms...@schor.com>.
On 6/11/2013 5:04 PM, Richard Eckart de Castilho wrote:
> Am 11.06.2013 um 22:55 schrieb Marshall Schor <ms...@schor.com>:
>
>> I think the scenario that is driving the logic to install the maven plugin under
>> test to a special isolated repository may not apply to this one.
>>
>> This is because this plugin is not used to build any other things (other than
>> user projects); that is, it's not used in building any of the UIMA projects.
>>
>> It seems to me that we could operate by having this plugin install itself to the
>> developer's local .m2 repository, and then run the integration test.
> Consider the integration tests of this plugin fail. Then we already have a broken
> plugin installed in the .m2/repository of Jenkins of of the developers machine
True
>
> Consider further that uimaFIT (or maybe at some point cTakes or whatever) is using
> this plugins in its builds. Their jobs in Jenkins may break just because we let
> a broken plugin escape into the wild.
Well, it would "escape" into the developer's machine, not any further :-) The
fix for this would be "simple" - just delete the plugin in the .m2 repo. The
next invocation would then download a (presumably) good version from Maven
Central :-)
For me, this kind of thing would be a good trade-off if we could get the
integration test for the plugin to work in a few seconds instead of many minutes.
-Marshall
>
> The Maven life cycle defines the intergration-test phase before the install phase.
> The installation of the plugin under test is afaik a special feature of the
> maven-invoker-plugin. If we want to rely on the "normal" install of the plugin, then
> we'd have to create a separate Maven module for the integration tests - which I tried
> to avoid.
>
> -- Richard
Re: jcasgen-maven-plugin integration test timing
Posted by Richard Eckart de Castilho <ri...@gmail.com>.
I tried to avoid having tests as a separate module, to avoid exactly this situation that
broken artifacts escape from the build. I think that's in fact what the integration-test
phase and concept are good for.
-- Richard
Am 11.06.2013 um 23:04 schrieb Richard Eckart de Castilho <ri...@gmail.com>:
> Am 11.06.2013 um 22:55 schrieb Marshall Schor <ms...@schor.com>:
>
>> I think the scenario that is driving the logic to install the maven plugin under
>> test to a special isolated repository may not apply to this one.
>>
>> This is because this plugin is not used to build any other things (other than
>> user projects); that is, it's not used in building any of the UIMA projects.
>>
>> It seems to me that we could operate by having this plugin install itself to the
>> developer's local .m2 repository, and then run the integration test.
>
> Consider the integration tests of this plugin fail. Then we already have a broken
> plugin installed in the .m2/repository of Jenkins of of the developers machine
>
> Consider further that uimaFIT (or maybe at some point cTakes or whatever) is using
> this plugins in its builds. Their jobs in Jenkins may break just because we let
> a broken plugin escape into the wild.
>
> The Maven life cycle defines the intergration-test phase before the install phase.
> The installation of the plugin under test is afaik a special feature of the
> maven-invoker-plugin. If we want to rely on the "normal" install of the plugin, then
> we'd have to create a separate Maven module for the integration tests - which I tried
> to avoid.
>
> -- Richard
Re: jcasgen-maven-plugin integration test timing
Posted by Richard Eckart de Castilho <ri...@gmail.com>.
Am 11.06.2013 um 22:55 schrieb Marshall Schor <ms...@schor.com>:
> I think the scenario that is driving the logic to install the maven plugin under
> test to a special isolated repository may not apply to this one.
>
> This is because this plugin is not used to build any other things (other than
> user projects); that is, it's not used in building any of the UIMA projects.
>
> It seems to me that we could operate by having this plugin install itself to the
> developer's local .m2 repository, and then run the integration test.
Consider the integration tests of this plugin fail. Then we already have a broken
plugin installed in the .m2/repository of Jenkins of of the developers machine
Consider further that uimaFIT (or maybe at some point cTakes or whatever) is using
this plugins in its builds. Their jobs in Jenkins may break just because we let
a broken plugin escape into the wild.
The Maven life cycle defines the intergration-test phase before the install phase.
The installation of the plugin under test is afaik a special feature of the
maven-invoker-plugin. If we want to rely on the "normal" install of the plugin, then
we'd have to create a separate Maven module for the integration tests - which I tried
to avoid.
-- Richard
Re: jcasgen-maven-plugin integration test timing
Posted by Marshall Schor <ms...@schor.com>.
I think the scenario that is driving the logic to install the maven plugin under
test to a special isolated repository may not apply to this one.
This is because this plugin is not used to build any other things (other than
user projects); that is, it's not used in building any of the UIMA projects.
It seems to me that we could operate by having this plugin install itself to the
developer's local .m2 repository, and then run the integration test.
This would require that any parent pom be installed also to the local
repository. That's needed, anyways, for other parts of the build.
So - if we did that, I wonder if the rest of the build could be made to work
without a custom repository?
-Marshall
On 6/11/2013 4:26 PM, Richard Eckart de Castilho wrote:
> Am 11.06.2013 um 22:17 schrieb Marshall Schor <ms...@schor.com>:
>
>> On 6/11/2013 3:57 PM, Richard Eckart de Castilho wrote:
>>> The m2e integration test takes *very* long because it materializes a full OSGI runtime environment on the machine. The downloads take forever.
>> This seems to download to a local maven repository which is temporarily built in
>> the target/local-repo area.
>>
>> I wonder if this could be changed to use the build-machine's .m2 repo, and thus
>> cache or otherwise avoid downloading things?
> The problem appears to be that the integration test must be able to install the plugin under test into a repository, so Maven can use it from there.
>
> A quite detailed description is given here: http://maven.apache.org/plugin-testing/maven-plugin-testing-tools/
>
> <quote>
> In addition to plugin-version inconsistencies, integration-testing of Maven plugins must cope with the need to cleanup after themselves. Remember, Maven only loads plugins from its own local repository (possibly after resolving them from a remote repository). Because of this, the only practical way to use the current plugin within an integration-test build is to install it into some sort of local repository, then direct Maven to use that local repository for the build. To avoid side effects in other builds (remember, we're building Maven plugins here... and they're used to build other projects), it's critical to avoid using the main local repository used for normal Maven builds. Therefore, plugin integration tests require a custom local repository, to which the integration-testing builds can be directed. Further complicating this testing scenario is the fact that the plugin's parent POM or one of its ancestor POMs could be a snapshot which has been installed in the main local repository, but which is not reachable using the plugin-POM's <relativePath/> parent-element.+
>
> + All of this seems to indicate that the best approach for ensuring the presence of all required POMs and plugins is to use the normal local repository to construct a new local repository for testing purposes. Work on this is ongoing.
> </quote>
>
> The integration tests run with a special "settings.xml" file which declares the target/local-repo as custom repository and which include the actual build system repo as "upstream" repository. Since the OSGI artifacts are not available in the "upstream" repository, they are downloaded. If there were available upstream, I suppose they's be copied.
>
> So far my understanding.
>
> -- Richard
Re: jcasgen-maven-plugin integration test timing
Posted by Richard Eckart de Castilho <ri...@gmail.com>.
Am 11.06.2013 um 22:17 schrieb Marshall Schor <ms...@schor.com>:
> On 6/11/2013 3:57 PM, Richard Eckart de Castilho wrote:
>> The m2e integration test takes *very* long because it materializes a full OSGI runtime environment on the machine. The downloads take forever.
> This seems to download to a local maven repository which is temporarily built in
> the target/local-repo area.
>
> I wonder if this could be changed to use the build-machine's .m2 repo, and thus
> cache or otherwise avoid downloading things?
The problem appears to be that the integration test must be able to install the plugin under test into a repository, so Maven can use it from there.
A quite detailed description is given here: http://maven.apache.org/plugin-testing/maven-plugin-testing-tools/
<quote>
In addition to plugin-version inconsistencies, integration-testing of Maven plugins must cope with the need to cleanup after themselves. Remember, Maven only loads plugins from its own local repository (possibly after resolving them from a remote repository). Because of this, the only practical way to use the current plugin within an integration-test build is to install it into some sort of local repository, then direct Maven to use that local repository for the build. To avoid side effects in other builds (remember, we're building Maven plugins here... and they're used to build other projects), it's critical to avoid using the main local repository used for normal Maven builds. Therefore, plugin integration tests require a custom local repository, to which the integration-testing builds can be directed. Further complicating this testing scenario is the fact that the plugin's parent POM or one of its ancestor POMs could be a snapshot which has been installed in the main local repository, but which is not reachable using the plugin-POM's <relativePath/> parent-element.+
+ All of this seems to indicate that the best approach for ensuring the presence of all required POMs and plugins is to use the normal local repository to construct a new local repository for testing purposes. Work on this is ongoing.
</quote>
The integration tests run with a special "settings.xml" file which declares the target/local-repo as custom repository and which include the actual build system repo as "upstream" repository. Since the OSGI artifacts are not available in the "upstream" repository, they are downloaded. If there were available upstream, I suppose they's be copied.
So far my understanding.
-- Richard
Re: jcasgen-maven-plugin integration test timing
Posted by Marshall Schor <ms...@schor.com>.
On 6/11/2013 3:57 PM, Richard Eckart de Castilho wrote:
> The m2e integration test takes *very* long because it materializes a full OSGI runtime environment on the machine. The downloads take forever.
This seems to download to a local maven repository which is temporarily built in
the target/local-repo area.
I wonder if this could be changed to use the build-machine's .m2 repo, and thus
cache or otherwise avoid downloading things?
-Marshall
> Detailed information about the execution of the integration tests can be found in target/it/<test>/build.log (or something like that).
>
> I had considered disabling the m2e integration test, but went for leaving it on for the time being.
>
> -- Richard
>
> Am 11.06.2013 um 21:49 schrieb Marshall Schor <ms...@schor.com>:
>
>> The regular tests for this take about 3 seconds on my laptop.
>>
>> The integration tests took a very long time (the 2nd one took about 8 minutes !)
>>
>> [INFO] --- maven-invoker-plugin:1.7:run (integration-test) @
>> jcasgen-maven-plugin ---
>> [INFO] Building: classpath\pom.xml
>> [INFO] ..SUCCESS (23.5 s)
>> [INFO] Building: m2e\pom.xml
>> [INFO] ..SUCCESS (529.5 s)
>> [INFO] Building: multimodule\pom.xml
>> [INFO] ..SUCCESS (13.4 s)
>> [INFO] Building: simple\pom.xml
>> [INFO] ..SUCCESS (9.7 s)
>> [INFO] -------------------------------------------------
>> [INFO] Build Summary:
>> [INFO] Passed: 4, Failed: 0, Errors: 0, Skipped: 0
>>
>> I ran this again (just in case some initial startup thing was slowing it down)
>> and got similar results: 19.0 s, 576.1 s, 13.6 s, and 8.5 s.
>>
>> Anyone have any idea what is going on, why some of these are running so slowly?
>>
>> It would be good if someone could post a walk through of how the integration tests
>> are setup and how they work.
>>
>> -Marshall
>
Re: jcasgen-maven-plugin integration test timing
Posted by Richard Eckart de Castilho <ri...@gmail.com>.
The m2e integration test takes *very* long because it materializes a full OSGI runtime environment on the machine. The downloads take forever.
Detailed information about the execution of the integration tests can be found in target/it/<test>/build.log (or something like that).
I had considered disabling the m2e integration test, but went for leaving it on for the time being.
-- Richard
Am 11.06.2013 um 21:49 schrieb Marshall Schor <ms...@schor.com>:
> The regular tests for this take about 3 seconds on my laptop.
>
> The integration tests took a very long time (the 2nd one took about 8 minutes !)
>
> [INFO] --- maven-invoker-plugin:1.7:run (integration-test) @
> jcasgen-maven-plugin ---
> [INFO] Building: classpath\pom.xml
> [INFO] ..SUCCESS (23.5 s)
> [INFO] Building: m2e\pom.xml
> [INFO] ..SUCCESS (529.5 s)
> [INFO] Building: multimodule\pom.xml
> [INFO] ..SUCCESS (13.4 s)
> [INFO] Building: simple\pom.xml
> [INFO] ..SUCCESS (9.7 s)
> [INFO] -------------------------------------------------
> [INFO] Build Summary:
> [INFO] Passed: 4, Failed: 0, Errors: 0, Skipped: 0
>
> I ran this again (just in case some initial startup thing was slowing it down)
> and got similar results: 19.0 s, 576.1 s, 13.6 s, and 8.5 s.
>
> Anyone have any idea what is going on, why some of these are running so slowly?
>
> It would be good if someone could post a walk through of how the integration tests
> are setup and how they work.
>
> -Marshall