You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@ant.apache.org by Antoine Levy Lambert <an...@gmx.de> on 2014/03/23 17:28:02 UTC
Re: svn commit: r1580520 - in /ant/core/trunk: ./ manual/Tasks/ src/main/org/apache/tools/ant/taskdefs/optional/junit/
Hi,
Thanks to John Elion for this contribution.
I have tried it on the Ant test cases. This makes the execution of the test cases shorter by 3 minutes with 2 threads [ not sure what is the total time because I also run the antunit tests ].
Some of our test cases do not support parallelism because they are creating and dropping temporary directories and files which have the same names.
this makes errors looking like this one :
[junit] Testcase: testNoAddNewLine(org.apache.tools.ant.filters.NoNewLineTest): Caused an ERROR
[junit] Warning: Could not find file /Users/antoine/dev/asf/ant-core/src/etc/testcases/filters/result/nonl to copy.
[junit] /Users/antoine/dev/asf/ant-core/src/etc/testcases/filters/build.xml:111: Warning: Could not find file /Users/antoine/dev/asf/ant-core/src/etc/testcases/filters/result/nonl to copy.
[junit] at org.apache.tools.ant.taskdefs.Copy.copySingleFile(Copy.java:639)
Does someone have a suggestion about how to adapt the test cases to support parallelism.
I was thinking of adding somewhere in the Ant codebase a utility class or method to get the PID of the currently running process
and add the PID somewhere in the name of temporary folders created for tests.
Any thoughts on that ?
see http://stackoverflow.com/questions/35842/how-can-a-java-program-get-its-own-process-id
Regards,
Antoine
On Mar 23, 2014, at 12:18 PM, antoine@apache.org wrote:
> Author: antoine
> Date: Sun Mar 23 16:18:29 2014
> New Revision: 1580520
>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@ant.apache.org
For additional commands, e-mail: dev-help@ant.apache.org
Re: svn commit: r1580520 - in /ant/core/trunk: ./ manual/Tasks/ src/main/org/apache/tools/ant/taskdefs/optional/junit/
Posted by Matt Sicker <bo...@gmail.com>.
@Michael: I'll fork your repo then to continue work on that.
I actually got interested in Ant finally due to build troubles at my job.
We've got two separate build systems on one of our largest projects: a
terribly written ant build, and a terribly slow gradle build. I've taken
over rewriting the ant build, and in the process, I ended up rewriting an
antlib we were using which got me further into Ant internals. I couldn't
get into gradle or maven, but ant was actually understandable!
On 31 March 2014 20:46, Antoine Levy Lambert <an...@gmx.de> wrote:
> Hello Michael,
>
> yes please go back and continue this, this is interesting.
>
> @Matt we are always happy to have new volunteers to help us maintain Ant
> :-)
>
> Ant’s minimum Java version is 1.5 so we are good on that side.
>
> Regards,
>
> Antoine
> On Mar 31, 2014, at 5:28 PM, Michael Clarke <mi...@gmail.com>
> wrote:
>
> > I had started this a few months back (
> > https://github.com/mc1arke/ant/tree/JUnit4Conversion) but got
> side-tracked
> > due to a job change. I'd be happy to go back and continue/finish the work
> > if there's a general demand for it.
> >
> >
> > On 31 March 2014 01:14, Matt Sicker <bo...@gmail.com> wrote:
> >
> >> I'd be a willing volunteer to help port the unit tests to JUnit 4. There
> >> are various methods to using JUnit 4 with JUnit 3 test cases, suites,
> etc.,
> >> that allow for easier migration as well. I do know that JUnit 4 has a
> >> minimum requirement of Java 1.5 at least due to annotations.
> >>
> >>
> >> On 30 March 2014 18:53, Antoine Levy Lambert <an...@gmx.de> wrote:
> >>
> >>> Hello Matt,
> >>>
> >>> thanks for this suggestion.
> >>>
> >>> I have not used the JUnit TemporaryFolder rule because it is introduced
> >> in
> >>> JUnit 4 and the Ant test cases are extending
> >>> a class of JUnit 3.
> >>>
> >>> The policy of the Ant project is usually to keep everything binary
> >>> compatible ...
> >>>
> >>> If there is interest and willing volunteers and a consensus we could
> >>> change that, at least in the case of BuildFileTest and JUnit 3/4 and
> base
> >>> "BuildFileTest" on JUnit 4.
> >>>
> >>> Regards,
> >>>
> >>> Antoine
> >>> On Mar 23, 2014, at 1:50 PM, Matt Sicker <bo...@gmail.com> wrote:
> >>>
> >>>> Could you use the JUnit TemporaryFolder rule? That appears to be
> rather
> >>>> threadsafe.
> >>>>
> >>>>
> >>>> On 23 March 2014 11:28, Antoine Levy Lambert <an...@gmx.de> wrote:
> >>>>
> >>>>> Hi,
> >>>>>
> >>>>> Thanks to John Elion for this contribution.
> >>>>>
> >>>>> I have tried it on the Ant test cases. This makes the execution of
> the
> >>>>> test cases shorter by 3 minutes with 2 threads [ not sure what is the
> >>> total
> >>>>> time because I also run the antunit tests ].
> >>>>>
> >>>>> Some of our test cases do not support parallelism because they are
> >>>>> creating and dropping temporary directories and files which have the
> >>> same
> >>>>> names.
> >>>>>
> >>>
> >>>
> >>> ---------------------------------------------------------------------
> >>> To unsubscribe, e-mail: dev-unsubscribe@ant.apache.org
> >>> For additional commands, e-mail: dev-help@ant.apache.org
> >>>
> >>>
> >>
> >>
> >> --
> >> Matt Sicker <bo...@gmail.com>
> >>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@ant.apache.org
> For additional commands, e-mail: dev-help@ant.apache.org
>
>
--
Matt Sicker <bo...@gmail.com>
Re: svn commit: r1580520 - in /ant/core/trunk: ./ manual/Tasks/ src/main/org/apache/tools/ant/taskdefs/optional/junit/
Posted by Antoine Levy Lambert <an...@gmx.de>.
Hello Michael,
yes please go back and continue this, this is interesting.
@Matt we are always happy to have new volunteers to help us maintain Ant :-)
Ant’s minimum Java version is 1.5 so we are good on that side.
Regards,
Antoine
On Mar 31, 2014, at 5:28 PM, Michael Clarke <mi...@gmail.com> wrote:
> I had started this a few months back (
> https://github.com/mc1arke/ant/tree/JUnit4Conversion) but got side-tracked
> due to a job change. I'd be happy to go back and continue/finish the work
> if there's a general demand for it.
>
>
> On 31 March 2014 01:14, Matt Sicker <bo...@gmail.com> wrote:
>
>> I'd be a willing volunteer to help port the unit tests to JUnit 4. There
>> are various methods to using JUnit 4 with JUnit 3 test cases, suites, etc.,
>> that allow for easier migration as well. I do know that JUnit 4 has a
>> minimum requirement of Java 1.5 at least due to annotations.
>>
>>
>> On 30 March 2014 18:53, Antoine Levy Lambert <an...@gmx.de> wrote:
>>
>>> Hello Matt,
>>>
>>> thanks for this suggestion.
>>>
>>> I have not used the JUnit TemporaryFolder rule because it is introduced
>> in
>>> JUnit 4 and the Ant test cases are extending
>>> a class of JUnit 3.
>>>
>>> The policy of the Ant project is usually to keep everything binary
>>> compatible ...
>>>
>>> If there is interest and willing volunteers and a consensus we could
>>> change that, at least in the case of BuildFileTest and JUnit 3/4 and base
>>> "BuildFileTest" on JUnit 4.
>>>
>>> Regards,
>>>
>>> Antoine
>>> On Mar 23, 2014, at 1:50 PM, Matt Sicker <bo...@gmail.com> wrote:
>>>
>>>> Could you use the JUnit TemporaryFolder rule? That appears to be rather
>>>> threadsafe.
>>>>
>>>>
>>>> On 23 March 2014 11:28, Antoine Levy Lambert <an...@gmx.de> wrote:
>>>>
>>>>> Hi,
>>>>>
>>>>> Thanks to John Elion for this contribution.
>>>>>
>>>>> I have tried it on the Ant test cases. This makes the execution of the
>>>>> test cases shorter by 3 minutes with 2 threads [ not sure what is the
>>> total
>>>>> time because I also run the antunit tests ].
>>>>>
>>>>> Some of our test cases do not support parallelism because they are
>>>>> creating and dropping temporary directories and files which have the
>>> same
>>>>> names.
>>>>>
>>>
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: dev-unsubscribe@ant.apache.org
>>> For additional commands, e-mail: dev-help@ant.apache.org
>>>
>>>
>>
>>
>> --
>> Matt Sicker <bo...@gmail.com>
>>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@ant.apache.org
For additional commands, e-mail: dev-help@ant.apache.org
Re: svn commit: r1580520 - in /ant/core/trunk: ./ manual/Tasks/ src/main/org/apache/tools/ant/taskdefs/optional/junit/
Posted by Michael Clarke <mi...@gmail.com>.
I had started this a few months back (
https://github.com/mc1arke/ant/tree/JUnit4Conversion) but got side-tracked
due to a job change. I'd be happy to go back and continue/finish the work
if there's a general demand for it.
On 31 March 2014 01:14, Matt Sicker <bo...@gmail.com> wrote:
> I'd be a willing volunteer to help port the unit tests to JUnit 4. There
> are various methods to using JUnit 4 with JUnit 3 test cases, suites, etc.,
> that allow for easier migration as well. I do know that JUnit 4 has a
> minimum requirement of Java 1.5 at least due to annotations.
>
>
> On 30 March 2014 18:53, Antoine Levy Lambert <an...@gmx.de> wrote:
>
> > Hello Matt,
> >
> > thanks for this suggestion.
> >
> > I have not used the JUnit TemporaryFolder rule because it is introduced
> in
> > JUnit 4 and the Ant test cases are extending
> > a class of JUnit 3.
> >
> > The policy of the Ant project is usually to keep everything binary
> > compatible ...
> >
> > If there is interest and willing volunteers and a consensus we could
> > change that, at least in the case of BuildFileTest and JUnit 3/4 and base
> > "BuildFileTest" on JUnit 4.
> >
> > Regards,
> >
> > Antoine
> > On Mar 23, 2014, at 1:50 PM, Matt Sicker <bo...@gmail.com> wrote:
> >
> > > Could you use the JUnit TemporaryFolder rule? That appears to be rather
> > > threadsafe.
> > >
> > >
> > > On 23 March 2014 11:28, Antoine Levy Lambert <an...@gmx.de> wrote:
> > >
> > >> Hi,
> > >>
> > >> Thanks to John Elion for this contribution.
> > >>
> > >> I have tried it on the Ant test cases. This makes the execution of the
> > >> test cases shorter by 3 minutes with 2 threads [ not sure what is the
> > total
> > >> time because I also run the antunit tests ].
> > >>
> > >> Some of our test cases do not support parallelism because they are
> > >> creating and dropping temporary directories and files which have the
> > same
> > >> names.
> > >>
> >
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: dev-unsubscribe@ant.apache.org
> > For additional commands, e-mail: dev-help@ant.apache.org
> >
> >
>
>
> --
> Matt Sicker <bo...@gmail.com>
>
Re: svn commit: r1580520 - in /ant/core/trunk: ./ manual/Tasks/ src/main/org/apache/tools/ant/taskdefs/optional/junit/
Posted by Matt Sicker <bo...@gmail.com>.
I'd be a willing volunteer to help port the unit tests to JUnit 4. There
are various methods to using JUnit 4 with JUnit 3 test cases, suites, etc.,
that allow for easier migration as well. I do know that JUnit 4 has a
minimum requirement of Java 1.5 at least due to annotations.
On 30 March 2014 18:53, Antoine Levy Lambert <an...@gmx.de> wrote:
> Hello Matt,
>
> thanks for this suggestion.
>
> I have not used the JUnit TemporaryFolder rule because it is introduced in
> JUnit 4 and the Ant test cases are extending
> a class of JUnit 3.
>
> The policy of the Ant project is usually to keep everything binary
> compatible …
>
> If there is interest and willing volunteers and a consensus we could
> change that, at least in the case of BuildFileTest and JUnit 3/4 and base
> “BuildFileTest” on JUnit 4.
>
> Regards,
>
> Antoine
> On Mar 23, 2014, at 1:50 PM, Matt Sicker <bo...@gmail.com> wrote:
>
> > Could you use the JUnit TemporaryFolder rule? That appears to be rather
> > threadsafe.
> >
> >
> > On 23 March 2014 11:28, Antoine Levy Lambert <an...@gmx.de> wrote:
> >
> >> Hi,
> >>
> >> Thanks to John Elion for this contribution.
> >>
> >> I have tried it on the Ant test cases. This makes the execution of the
> >> test cases shorter by 3 minutes with 2 threads [ not sure what is the
> total
> >> time because I also run the antunit tests ].
> >>
> >> Some of our test cases do not support parallelism because they are
> >> creating and dropping temporary directories and files which have the
> same
> >> names.
> >>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@ant.apache.org
> For additional commands, e-mail: dev-help@ant.apache.org
>
>
--
Matt Sicker <bo...@gmail.com>
Re: svn commit: r1580520 - in /ant/core/trunk: ./ manual/Tasks/ src/main/org/apache/tools/ant/taskdefs/optional/junit/
Posted by Antoine Levy Lambert <an...@gmx.de>.
Hello Matt,
thanks for this suggestion.
I have not used the JUnit TemporaryFolder rule because it is introduced in JUnit 4 and the Ant test cases are extending
a class of JUnit 3.
The policy of the Ant project is usually to keep everything binary compatible …
If there is interest and willing volunteers and a consensus we could change that, at least in the case of BuildFileTest and JUnit 3/4 and base “BuildFileTest” on JUnit 4.
Regards,
Antoine
On Mar 23, 2014, at 1:50 PM, Matt Sicker <bo...@gmail.com> wrote:
> Could you use the JUnit TemporaryFolder rule? That appears to be rather
> threadsafe.
>
>
> On 23 March 2014 11:28, Antoine Levy Lambert <an...@gmx.de> wrote:
>
>> Hi,
>>
>> Thanks to John Elion for this contribution.
>>
>> I have tried it on the Ant test cases. This makes the execution of the
>> test cases shorter by 3 minutes with 2 threads [ not sure what is the total
>> time because I also run the antunit tests ].
>>
>> Some of our test cases do not support parallelism because they are
>> creating and dropping temporary directories and files which have the same
>> names.
>>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@ant.apache.org
For additional commands, e-mail: dev-help@ant.apache.org
Re: svn commit: r1580520 - in /ant/core/trunk: ./ manual/Tasks/ src/main/org/apache/tools/ant/taskdefs/optional/junit/
Posted by Matt Sicker <bo...@gmail.com>.
Could you use the JUnit TemporaryFolder rule? That appears to be rather
threadsafe.
On 23 March 2014 11:28, Antoine Levy Lambert <an...@gmx.de> wrote:
> Hi,
>
> Thanks to John Elion for this contribution.
>
> I have tried it on the Ant test cases. This makes the execution of the
> test cases shorter by 3 minutes with 2 threads [ not sure what is the total
> time because I also run the antunit tests ].
>
> Some of our test cases do not support parallelism because they are
> creating and dropping temporary directories and files which have the same
> names.
>
> this makes errors looking like this one :
> [junit] Testcase:
> testNoAddNewLine(org.apache.tools.ant.filters.NoNewLineTest): Caused an
> ERROR
> [junit] Warning: Could not find file
> /Users/antoine/dev/asf/ant-core/src/etc/testcases/filters/result/nonl to
> copy.
> [junit]
> /Users/antoine/dev/asf/ant-core/src/etc/testcases/filters/build.xml:111:
> Warning: Could not find file
> /Users/antoine/dev/asf/ant-core/src/etc/testcases/filters/result/nonl to
> copy.
> [junit] at
> org.apache.tools.ant.taskdefs.Copy.copySingleFile(Copy.java:639)
>
> Does someone have a suggestion about how to adapt the test cases to
> support parallelism.
>
> I was thinking of adding somewhere in the Ant codebase a utility class or
> method to get the PID of the currently running process
> and add the PID somewhere in the name of temporary folders created for
> tests.
>
> Any thoughts on that ?
>
> see
> http://stackoverflow.com/questions/35842/how-can-a-java-program-get-its-own-process-id
>
>
> Regards,
>
> Antoine
> On Mar 23, 2014, at 12:18 PM, antoine@apache.org wrote:
>
> > Author: antoine
> > Date: Sun Mar 23 16:18:29 2014
> > New Revision: 1580520
> >
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@ant.apache.org
> For additional commands, e-mail: dev-help@ant.apache.org
>
>
--
Matt Sicker <bo...@gmail.com>
Re: svn commit: r1580520 - in /ant/core/trunk: ./ manual/Tasks/ src/main/org/apache/tools/ant/taskdefs/optional/junit/
Posted by Antoine Levy Lambert <an...@gmx.de>.
Hi,
I have had a first shot at creating temporary folders with names including the current PID and the current Thread Name in order
to allow to run tests in parallel. I did not have the time today to do the work for all the Ant test cases but it seems achievable to finish that work.
On Mar 24, 2014, at 1:08 AM, Stefan Bodewig <bo...@apache.org> wrote:
> On 2014-03-23, Antoine Levy Lambert wrote:
>
>> Some of our test cases do not support parallelism because they are creating and dropping temporary directories and files which have the same names.
>
> On first glance there also may be a race condition for the test that
> expects certain build events. See the Jenkins test error on JDK8 - and
> the nightly, I suspect, but Jenkins is currently down.
this error happened once, I do not know exactly what caused it, the next build was green.
>
>> I was thinking of adding somewhere in the Ant codebase a utility class
>> or method to get the PID of the currently running process and add the
>> PID somewhere in the name of temporary folders created for tests.
>
> Yes, unique temporary directory names is the best you can do. I haven't
> looked at the code, yet. From the name of the attribute I would have
> expected multiple threads inside the same VM rather than multiple
> processes. In the case of the same VM you'd certainly use the Thread-Id
> or name instead.
This JUnit multi-thread implementation is currently working if you choose fork=true and forkMode=perTest.
>
> You may want to move test directory creation/deletion into BuildFileTest
> similar to what we do with antunit-base.
I have added the setting of properties ant.pid and ant.threadname in BuildFileTest, these properties are then used
to give a name to a temporary folder ${output} in a new base build file similar to what exists for the AntUnit tests.
>
> Stefan
>
Thanks,
Antoine
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@ant.apache.org
For additional commands, e-mail: dev-help@ant.apache.org
Re: svn commit: r1580520 - in /ant/core/trunk: ./ manual/Tasks/ src/main/org/apache/tools/ant/taskdefs/optional/junit/
Posted by Stefan Bodewig <bo...@apache.org>.
On 2014-03-23, Antoine Levy Lambert wrote:
> Some of our test cases do not support parallelism because they are creating and dropping temporary directories and files which have the same names.
On first glance there also may be a race condition for the test that
expects certain build events. See the Jenkins test error on JDK8 - and
the nightly, I suspect, but Jenkins is currently down.
> I was thinking of adding somewhere in the Ant codebase a utility class
> or method to get the PID of the currently running process and add the
> PID somewhere in the name of temporary folders created for tests.
Yes, unique temporary directory names is the best you can do. I haven't
looked at the code, yet. From the name of the attribute I would have
expected multiple threads inside the same VM rather than multiple
processes. In the case of the same VM you'd certainly use the Thread-Id
or name instead.
You may want to move test directory creation/deletion into BuildFileTest
similar to what we do with antunit-base.
Stefan
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@ant.apache.org
For additional commands, e-mail: dev-help@ant.apache.org