You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucene.apache.org by Yonik Seeley <yo...@lucidimagination.com> on 2011/12/02 15:07:27 UTC

getting incremental output from our tests

Right now, all test output is buffered.  If a test dies or hangs, you
get no output.
If your test is too long or goes into an infinite loop, it causes an
OOM from all the buffering.

In the past, I could just change the formatter from "xml" to "plain"
to get rid of the buffering, but this no longer works.
I think this is because Lucene now does it's own buffering to keep
output from multiple tests straight?

So how can one enable incremental non-buffered-till-end-of-test output
these days?

-Yonik
http://www.lucidimagination.com

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org

Re: getting incremental output from our tests

Posted by Yonik Seeley <yo...@lucidimagination.com>.
On Fri, Dec 2, 2011 at 10:23 AM, Robert Muir <rc...@gmail.com> wrote:
> On Fri, Dec 2, 2011 at 10:14 AM, Yonik Seeley
> <yo...@lucidimagination.com> wrote:
>> On Fri, Dec 2, 2011 at 9:57 AM, Robert Muir <rc...@gmail.com> wrote:
>>> On Fri, Dec 2, 2011 at 9:45 AM, Yonik Seeley <yo...@lucidimagination.com> wrote:
>>>>
>>>> Should we add a tests.console or tests.buffer property to control this
>>>> from the command line?
>>>>
>>>
>>> Can't we just turn it on if you use -Dtestcase? this seems to work well for me:
>>
>> That was my first thought, but then I thought that someone might still
>> want a nice output file to attach to issues, etc.
>> If just redirecting the ant output to a file is acceptable, then this
>> would be a good solution.
>>
>
> Also, what about the output is not nice?

It's just different.  I didn't know if anyone would care (I don't myself)

Anyway, +1 for doing it by default w/ -Dtestcase

-Yonik
http://www.lucidimagination.com

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org

Re: getting incremental output from our tests

Posted by Robert Muir <rc...@gmail.com>.
On Fri, Dec 2, 2011 at 10:14 AM, Yonik Seeley
<yo...@lucidimagination.com> wrote:
> On Fri, Dec 2, 2011 at 9:57 AM, Robert Muir <rc...@gmail.com> wrote:
>> On Fri, Dec 2, 2011 at 9:45 AM, Yonik Seeley <yo...@lucidimagination.com> wrote:
>>>
>>> Should we add a tests.console or tests.buffer property to control this
>>> from the command line?
>>>
>>
>> Can't we just turn it on if you use -Dtestcase? this seems to work well for me:
>
> That was my first thought, but then I thought that someone might still
> want a nice output file to attach to issues, etc.
> If just redirecting the ant output to a file is acceptable, then this
> would be a good solution.
>

Also, what about the output is not nice? I tested this locally and I
think the output looks fine, its a good solution since it bypasses the
formatter.

I don't understand the theoretical? 'use case' of nice output file to
attach to issues, the output looks pretty much the same to me, only
when you run a single test you don't have to wait...

-- 
lucidimagination.com

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org

Re: getting incremental output from our tests

Posted by Robert Muir <rc...@gmail.com>.
On Fri, Dec 2, 2011 at 10:14 AM, Yonik Seeley
<yo...@lucidimagination.com> wrote:
> On Fri, Dec 2, 2011 at 9:57 AM, Robert Muir <rc...@gmail.com> wrote:
>> On Fri, Dec 2, 2011 at 9:45 AM, Yonik Seeley <yo...@lucidimagination.com> wrote:
>>>
>>> Should we add a tests.console or tests.buffer property to control this
>>> from the command line?
>>>
>>
>> Can't we just turn it on if you use -Dtestcase? this seems to work well for me:
>
> That was my first thought, but then I thought that someone might still
> want a nice output file to attach to issues, etc.
> If just redirecting the ant output to a file is acceptable, then this
> would be a good solution.
>

Well i think we should turn it on for -Dtestcase in any event!

But by using variable names like that, someone can always also
explicitly override -Djunit.showoutput etc, too.

-- 
lucidimagination.com

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org

Re: getting incremental output from our tests

Posted by Yonik Seeley <yo...@lucidimagination.com>.
On Fri, Dec 2, 2011 at 9:57 AM, Robert Muir <rc...@gmail.com> wrote:
> On Fri, Dec 2, 2011 at 9:45 AM, Yonik Seeley <yo...@lucidimagination.com> wrote:
>>
>> Should we add a tests.console or tests.buffer property to control this
>> from the command line?
>>
>
> Can't we just turn it on if you use -Dtestcase? this seems to work well for me:

That was my first thought, but then I thought that someone might still
want a nice output file to attach to issues, etc.
If just redirecting the ant output to a file is acceptable, then this
would be a good solution.

-Yonik
http://www.lucidimagination.com

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org

Re: getting incremental output from our tests

Posted by Robert Muir <rc...@gmail.com>.
On Fri, Dec 2, 2011 at 9:45 AM, Yonik Seeley <yo...@lucidimagination.com> wrote:
>
> Should we add a tests.console or tests.buffer property to control this
> from the command line?
>

Can't we just turn it on if you use -Dtestcase? this seems to work well for me:

Index: common-build.xml
===================================================================
--- common-build.xml	(revision 1209506)
+++ common-build.xml	(working copy)
@@ -58,6 +58,16 @@
     <pathelement location="${junit-location.jar}"/>
   </path>

+  <condition property="junit.showoutput">
+    <isset property="testcase"/>
+  </condition>
+
+  <condition property="junit.outputtoformatters">
+    <not>
+      <isset property="testcase"/>
+    </not>
+  </condition>
+
   <path id="ant-path">
     <fileset dir="${common.dir}/lib" includes="ant-*.jar"/>
   </path>
@@ -540,7 +550,8 @@
 	    -->
     	<touch file="@{tempDir}/@{threadNum}/quiet.ant" verbose="false"
mkdirs="true"/>
 	    <junit printsummary="off" haltonfailure="no" maxmemory="512M"
tempdir="@{tempDir}/@{threadNum}"
-	      errorProperty="tests.failed" failureProperty="tests.failed"
forkmode="perBatch" dir="@{tempDir}/@{threadNum}">
+	      errorProperty="tests.failed" failureProperty="tests.failed"
forkmode="perBatch" dir="@{tempDir}/@{threadNum}"
+              showoutput="${junit.showoutput}"
outputtoformatters="${junit.outputtoformatters}">
 	      <classpath refid="@{junit.classpath}"/>
 	      <assertions>
 	        <enable package="org.apache.lucene"/>

-- 
lucidimagination.com

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org

RE: getting incremental output from our tests

Posted by Steven A Rowe <sa...@syr.edu>.
+1

I would prefer a more descriptive name, though tests.unbuffered.console.output seems a bit much...

> -----Original Message-----
> From: yseeley@gmail.com [mailto:yseeley@gmail.com] On Behalf Of Yonik
> Seeley
> Sent: Friday, December 02, 2011 9:46 AM
> To: dev@lucene.apache.org
> Subject: Re: getting incremental output from our tests
> 
> On Fri, Dec 2, 2011 at 9:43 AM, Yonik Seeley <yo...@lucidimagination.com>
> wrote:
> > OK, so if you add outputtoformatters="false" and showoutput="true",
> > you will get incremental output (not to a file, but at least to the
> > console that you ran ant from).
> 
> Should we add a tests.console or tests.buffer property to control this
> from the command line?
> 
> -Yonik
> http://www.lucidimagination.com
> 
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
> For additional commands, e-mail: dev-help@lucene.apache.org

Re: getting incremental output from our tests

Posted by Yonik Seeley <yo...@lucidimagination.com>.
On Fri, Dec 2, 2011 at 9:43 AM, Yonik Seeley <yo...@lucidimagination.com> wrote:
> OK, so if you add outputtoformatters="false" and showoutput="true",
> you will get incremental output (not to a file, but at least to the
> console that you ran ant from).

Should we add a tests.console or tests.buffer property to control this
from the command line?

-Yonik
http://www.lucidimagination.com

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org

Re: getting incremental output from our tests

Posted by Yonik Seeley <yo...@lucidimagination.com>.
On Fri, Dec 2, 2011 at 9:30 AM, Yonik Seeley <yo...@lucidimagination.com> wrote:
> On Fri, Dec 2, 2011 at 9:23 AM, Robert Muir <rc...@gmail.com> wrote:
>> no, its because ant does this buffering in its formatters. its not
>> anything we do.
>
> Maybe it's a change in JUnit at some point then.  In the past (the far
> past), "plain" (or something like that) didn't buffer.  Maybe there
> was a way to not even use a formatter or something.  All I know is
> that I got it to work (again, in the far past).

OK, so if you add outputtoformatters="false" and showoutput="true",
you will get incremental output (not to a file, but at least to the
console that you ran ant from).

-Yonik
http://www.lucidimagination.com

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org

Re: getting incremental output from our tests

Posted by Yonik Seeley <yo...@lucidimagination.com>.
On Fri, Dec 2, 2011 at 9:23 AM, Robert Muir <rc...@gmail.com> wrote:
> no, its because ant does this buffering in its formatters. its not
> anything we do.

Maybe it's a change in JUnit at some point then.  In the past (the far
past), "plain" (or something like that) didn't buffer.  Maybe there
was a way to not even use a formatter or something.  All I know is
that I got it to work (again, in the far past).

-Yonik
http://www.lucidimagination.com

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org

Re: getting incremental output from our tests

Posted by Robert Muir <rc...@gmail.com>.
On Fri, Dec 2, 2011 at 9:07 AM, Yonik Seeley <yo...@lucidimagination.com> wrote:
> Right now, all test output is buffered.  If a test dies or hangs, you
> get no output.
> If your test is too long or goes into an infinite loop, it causes an
> OOM from all the buffering.
>
> In the past, I could just change the formatter from "xml" to "plain"
> to get rid of the buffering, but this no longer works.
> I think this is because Lucene now does it's own buffering to keep
> output from multiple tests straight?

no, its because ant does this buffering in its formatters. its not
anything we do.

>
> So how can one enable incremental non-buffered-till-end-of-test output
> these days?

you never could: see the interface of JUnitResultFormatter:
http://svn.apache.org/repos/asf/ant/core/trunk/src/main/org/apache/tools/ant/taskdefs/optional/junit/JUnitResultFormatter.java

    /**
     * This is what the test has written to System.out
     * @param out the string to write.
     */
    void setSystemOutput(String out);

    /**
     * This is what the test has written to System.err
     * @param err the string to write.
     */
    void setSystemError(String err);

-- 
lucidimagination.com

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org