You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucene.apache.org by "Uwe Schindler (JIRA)" <ji...@apache.org> on 2014/08/10 00:22:12 UTC
[jira] [Commented] (LUCENE-5881) Add test beaster
[ https://issues.apache.org/jira/browse/LUCENE-5881?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14091926#comment-14091926 ]
Uwe Schindler commented on LUCENE-5881:
---------------------------------------
>From mailing list:
{quote}
Hi,
attached you will find the beaster:
- Only modifies common-build.xml, so no inherit down (makes no sense otherwise, as you would never run "ant beast-test" from top-level. So you have to go to correct submodule and run "ant beast-test -Dbeast.iters=n -Dtestcase=..." from there
- Uses "antcall" in a loop, invoking the internal dependency-less "-test" target. My first impl used the test-macro directly, but this did not work, because test-macro sets non-local properties, which are then available on second round, causing errors or use always same seed. Antcall creates a new project each time and runs tests.
I can open an issue or just commit this :-)
Uwe
-----
Uwe Schindler
H.-H.-Meier-Allee 63, D-28213 Bremen
http://www.thetaphi.de
eMail: uwe@thetaphi.de
> -----Original Message-----
> From: Uwe Schindler [mailto:uwe@thetaphi.de]
> Sent: Friday, August 08, 2014 8:13 PM
> To: dev@lucene.apache.org
> Subject: RE: Test iterations
>
> Hi,
>
> I will look into that as a Groovy Skript: The main problem is: You
> cannot simply use <antcall/> in a loop, because this would also
> execute the dependencies on each run.
>
> My idea is to do the following:
> - maybe subclass antcall Task with Groovy (not sure if this is needed)
> - instantiate it with current project
> - execute dependent targets
> - execute the inner target multiple times: store the project
> properties first and restore them after execution. This is done,
> because ANT properties can only be set *once*. If you don't give a
> fixed test seed, each run would pick a new one (because the project
> properties are reset, so the seed from the previous execution is gone).
>
> Uwe
>
> -----
> Uwe Schindler
> H.-H.-Meier-Allee 63, D-28213 Bremen
> http://www.thetaphi.de
> eMail: uwe@thetaphi.de
>
>
> > -----Original Message-----
> > From: Ryan Ernst [mailto:ryan@iernst.net]
> > Sent: Friday, August 08, 2014 5:08 PM
> > To: dev@lucene.apache.org
> > Subject: Re: Test iterations
> >
> > Thanks for the extremely thorough answer, Dawid! Entertaining as
> > always. :)
> >
> > > Should we provide this "beaster" in common-build?
> >
> > I would use it! It sounds like there is a lot of work involved in
> > making tests.iters work better with LuceneTestCase. In the mean
> > time, this sounds like a quick solution that might not be as
> > efficient (multiple JVMs), but still better than having to come up with a bash script?
> >
> > On Fri, Aug 8, 2014 at 7:28 AM, Michael McCandless
> > <lu...@mikemccandless.com> wrote:
> > > +1, this sounds awesome?
> > >
> > > Mike McCandless
> > >
> > > http://blog.mikemccandless.com
> > >
> > >
> > > On Fri, Aug 8, 2014 at 10:06 AM, Uwe Schindler <uw...@thetaphi.de>
> wrote:
> > >> Hi,
> > >>
> > >> We could emulate the same thing (the repeating beaster) with pure
> Ant:
> > >>
> > >> Just repeat the "test" target, which can be done using ant-contrib's "for"
> > task or (much simplier) a groovy script using antcall on the test target.
> > >> Should we provide this "beaster" in common-build?
> > >>
> > >> "ant beast-tests -Dbeast.iter=100 -Dtestcase=..."
> > >>
> > >> Very easy to implement and makes it easier to use for the python
> > >> haters -
> > and comes embedded...
> > >>
> > >> Uwe
> > >>
> > >> -----
> > >> Uwe Schindler
> > >> H.-H.-Meier-Allee 63, D-28213 Bremen http://www.thetaphi.de
> > >> eMail: uwe@thetaphi.de
> > >>
> > >>
> > >>> -----Original Message-----
> > >>> From: Michael McCandless [mailto:lucene@mikemccandless.com]
> > >>> Sent: Friday, August 08, 2014 3:48 PM
> > >>> To: Lucene/Solr dev
> > >>> Subject: Re: Test iterations
> > >>>
> > >>> On Fri, Aug 8, 2014 at 9:35 AM, Uwe Schindler <uw...@thetaphi.de>
> > wrote:
> > >>> > Hi Dawid,
> > >>> >
> > >>> > Thanks for the very good explanation! Indeed the main problem
> > >>> > with
> > >>> tests.iters is the static initializers. Maybe put that
> > >>> explanation into the Wiki! I sometimes also need to remember it,
> > >>> so it should be
> > documented.
> > >>> >
> > >>> > One (only theoretical) way to solve the whole thing could be:
> > >>> > Load the class(es) in a separate classloader for every
> > >>> > repeated execution,... but of course this will very fast blow
> > >>> > up your permgen (java 6, 7) or anything else we don't know about (java 8).
> > >>> > In fact the separate classloader approach is not different
> > >>> > from Mike's scripts, just that Mike's script creates a new
> > >>> > classloader by forking a new JVM. In fact I don't think the
> > >>> > separate classloader approach would be much faster, because
> > >>> > the class clones will all have separate compilation paths in
> > >>> > Hotspot, so Hotspot cannot share the same assembler code. So
> > >>> > except the JVM startup time, you gain nothing. Just permgen
> > >>> > issues :-)
> > >>>
> > >>> The big thing the python beasting scripts avoids is all the ant
> > >>> overhead to just get to the point where it actually spawns the
> > >>> JVM to run the test. Really, that's all the beasting script does:
> > >>> directly spawn the JVM on the test runner (after running "ant
> > >>> test-compile" up
> > >>> front) and then parse its output/events.
> > >>>
> > >>> The distributed test runner, which uses rsync/ssh to run tests
> > >>> on N machines, is very different from the beasting script: it
> > >>> runs all Lucene's tests (instead of a single test over and over)
> > >>> across N JVMs on M machines. It "cheats" by taking the union of
> > >>> all
> CLASSPATHs
> > ...
> > >>> but this is a huge win because it means all testing is fully
> > >>> concurrent, not just concurrent within one module. This script
> > >>> can also repeat, which means once all lucene tests finish,
> > >>> re-en-queue all
> of
> > them again.
{quote}
> Add test beaster
> ----------------
>
> Key: LUCENE-5881
> URL: https://issues.apache.org/jira/browse/LUCENE-5881
> Project: Lucene - Core
> Issue Type: Task
> Components: general/build, general/test
> Reporter: Uwe Schindler
> Assignee: Uwe Schindler
> Attachments: LUCENE-5881.patch
>
>
> On dev@lao we discussed about integrating a test beaster directly into google.
> This extra target in common-build.xml does the same like Mike's Python script using Groovy.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org