You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucene.apache.org by Terry Smith <sh...@gmail.com> on 2014/03/06 23:09:33 UTC
Stalled unit tests
I'm sure that I'm just missing something obvious but I'm having trouble
getting the unit tests to run to completion on my laptop and was hoping
that someone would be kind enough to point me in the right direction.
I've cloned the repository from GitHub (
http://git.apache.org/lucene-solr.git) and checked out the latest commit on
branch_4x.
commit 6e06247cec1410f32592bfd307c1020b814def06
Author: Robert Muir <rm...@apache.org>
Date: Thu Mar 6 19:54:07 2014 +0000
disable slow solr tests in smoketester
git-svn-id:
https://svn.apache.org/repos/asf/lucene/dev/branches/branch_4x@157502513f79535-47bb-0310-9956-ffa450edef68
Executing "ant clean test" from the top level directory of the project
shows the tests running but they seems to get stuck in loop with some
stalled heartbeat messages. If I run the tests directly from lucene/ then
they complete successfully after about 10 minutes.
I'm using Java 6 under OS X (10.9.2).
$ java -version
java version "1.6.0_65"
Java(TM) SE Runtime Environment (build 1.6.0_65-b14-462-11M4609)
Java HotSpot(TM) 64-Bit Server VM (build 20.65-b04-462, mixed mode)
My terminal lists repeating stalled heartbeat messages like so:
HEARTBEAT J2 PID(20104@onyx.local): 2014-03-06T16:53:35, stalled for 2111s
at: HdfsLockFactoryTest.testBasic
HEARTBEAT J0 PID(20106@onyx.local): 2014-03-06T16:53:47, stalled for 2108s
at: TestSurroundQueryParser.testQueryParser
HEARTBEAT J1 PID(20103@onyx.local): 2014-03-06T16:54:11, stalled for 2167s
at: TestRecoveryHdfs.testBuffering
HEARTBEAT J3 PID(20105@onyx.local): 2014-03-06T16:54:23, stalled for 2165s
at: HdfsDirectoryTest.testEOF
My machine does have 3 java processes chewing CPU, see attached jstack
dumps for more information.
Should I expect the tests to complete on my platform? Do I need to specify
any special flags to give them more memory or to avoid any bad apples?
Thanks in advance,
--Terry
RE: Stalled unit tests
Posted by Uwe Schindler <uw...@thetaphi.de>.
I looked at this code yesterday and was confused, too. Common-build.xml currently reads the build properties only from the "lucene/" directory (which is common.dir) or from the working directory of the actually processed build.xml file (base.dir). We might change this (I was about to do this), because I added support for the smoketester sitting in the root folder (which ignored the properties file). We should fix this in a separate folder, so common-build.xml reads from home dir or *only* the root folder of lucene/solr.
The most simple thing is to place a file "lucene.build.properties" in your $HOME (~) directory. This is how I have it set up. Build.properties would also work, but this is most sepcfic and would work for every Lucene checkout you are working on.
Uwe
-----
Uwe Schindler
H.-H.-Meier-Allee 63, D-28213 Bremen
http://www.thetaphi.de
eMail: uwe@thetaphi.de
> -----Original Message-----
> From: dawid.weiss@gmail.com [mailto:dawid.weiss@gmail.com] On Behalf
> Of Dawid Weiss
> Sent: Thursday, March 13, 2014 4:14 PM
> To: dev@lucene.apache.org
> Subject: Re: Stalled unit tests
>
> The "root" of the project meaning Solr and Lucene codebase? Yeah... I don't
> think it'll work. Lucene and Solr still try to separate their codebases, even if
> there is a single SVN root.
>
> Dawid
>
> On Thu, Mar 13, 2014 at 3:59 PM, Terry Smith <sh...@gmail.com> wrote:
> > Dawid,
> >
> > Thanks, I didn't even know about it until Mike mentioned it earlier in
> > this thread. I've had it work from ~/lucene.build.properties and
> > ~/build.properties but didn't have any luck putting it in the root of
> > project (I'm probably just miss reading the ant file).
> >
> > --Terry
> >
> >
> >
> > On Thu, Mar 13, 2014 at 9:35 AM, Dawid Weiss
> > <da...@cs.put.poznan.pl>
> > wrote:
> >>
> >> Terry,
> >>
> >> The build.properties file holds the current user's config, as opposed
> >> to the defaults stored in the repository. In fact, there are more
> >> locations where you can put such defaults (see common-build.xml's
> >> header):
> >>
> >> <!-- Give user a chance to override without editing this file
> >> (and without typing -D each time it compiles it -->
> >> <property file="${user.home}/lucene.build.properties"/>
> >> <property file="${user.home}/build.properties"/>
> >> <property file="${basedir}/build.properties"/>
> >> <property file="${common.dir}/build.properties"/>
> >>
> >>
> >> Dawid
> >>
> >> On Thu, Mar 13, 2014 at 2:29 PM, Terry Smith <sh...@gmail.com>
> wrote:
> >> > It seems that you need to run the tests with
> >> > `-Dtests.disableHdfs=true` for them to succeed. Is there any
> >> > interested in making this the default behavior?
> >> >
> >> > If not, I'll happily start a new email thread to get wiki
> >> > permissions so that the contribution pages linked from the main
> >> > README.txt will both mention this important flag.
> >> >
> >> > http://wiki.apache.org/lucene-java/HowToContribute
> >> > http://wiki.apache.org/solr/HowToContribute
> >> >
> >> >
> >> > Right now they both state that you can run `ant clean test`,
> >> > unfortunately that command will fail if you run the tests from
> >> > either the top level of the project or the solr subdirectory unless
> >> > you instead run `ant -Dtests.disableHdfs=true clean test` or create
> >> > a build.properties file.
> >> >
> >> > I also couldn't find any references to build.properties on the
> >> > wiki, here are the searches I tried:
> >> >
> >> >
> >> > http://wiki.apache.org/general/FrontPage?action=fullsearch&context=
> >> > 180&value=build.properties&fullsearch=Text
> >> >
> >> >
> http://www.google.com/?q=%22build.properties%22+site:wiki.apache.or
> >> > g%2Flucene
> >> >
> >> >
> http://www.google.com/?q=%22build.properties%22+site:wiki.apache.or
> >> > g%2Fsolr
> >> >
> >> >
> >> > Is this documented somewhere else? I'd be happy to back some out
> >> > from the ant files, collate documentation from other sources and
> >> > make it easier to find.
> >> >
> >> > --Terry
> >> >
> >> >
> >> > On Mon, Mar 10, 2014 at 2:55 PM, Dawid Weiss
> >> > <da...@cs.put.poznan.pl>
> >> > wrote:
> >> >>
> >> >> > Dawid: Boy, those are some large timeouts!
> >> >>
> >> >> I know... I wasn't the one to bump them; my default was, I think,
> >> >> about 3 minutes per class...
> >> >>
> >> >> Dawid
> >> >>
> >> >> ------------------------------------------------------------------
> >> >> --- To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org For
> >> >> additional commands, e-mail: dev-help@lucene.apache.org
> >> >>
> >> >
> >>
> >> ---------------------------------------------------------------------
> >> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org For
> >> additional commands, e-mail: dev-help@lucene.apache.org
> >>
> >
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org For additional
> commands, e-mail: dev-help@lucene.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
Re: Stalled unit tests
Posted by Dawid Weiss <da...@cs.put.poznan.pl>.
The "root" of the project meaning Solr and Lucene codebase? Yeah... I
don't think it'll work. Lucene and Solr still try to separate their
codebases, even if there is a single SVN root.
Dawid
On Thu, Mar 13, 2014 at 3:59 PM, Terry Smith <sh...@gmail.com> wrote:
> Dawid,
>
> Thanks, I didn't even know about it until Mike mentioned it earlier in this
> thread. I've had it work from ~/lucene.build.properties and
> ~/build.properties but didn't have any luck putting it in the root of
> project (I'm probably just miss reading the ant file).
>
> --Terry
>
>
>
> On Thu, Mar 13, 2014 at 9:35 AM, Dawid Weiss <da...@cs.put.poznan.pl>
> wrote:
>>
>> Terry,
>>
>> The build.properties file holds the current user's config, as opposed
>> to the defaults stored in the repository. In fact, there are more
>> locations where you can put such defaults (see common-build.xml's
>> header):
>>
>> <!-- Give user a chance to override without editing this file
>> (and without typing -D each time it compiles it -->
>> <property file="${user.home}/lucene.build.properties"/>
>> <property file="${user.home}/build.properties"/>
>> <property file="${basedir}/build.properties"/>
>> <property file="${common.dir}/build.properties"/>
>>
>>
>> Dawid
>>
>> On Thu, Mar 13, 2014 at 2:29 PM, Terry Smith <sh...@gmail.com> wrote:
>> > It seems that you need to run the tests with `-Dtests.disableHdfs=true`
>> > for
>> > them to succeed. Is there any interested in making this the default
>> > behavior?
>> >
>> > If not, I'll happily start a new email thread to get wiki permissions so
>> > that the contribution pages linked from the main README.txt will both
>> > mention this important flag.
>> >
>> > http://wiki.apache.org/lucene-java/HowToContribute
>> > http://wiki.apache.org/solr/HowToContribute
>> >
>> >
>> > Right now they both state that you can run `ant clean test`,
>> > unfortunately
>> > that command will fail if you run the tests from either the top level of
>> > the
>> > project or the solr subdirectory unless you instead run `ant
>> > -Dtests.disableHdfs=true clean test` or create a build.properties file.
>> >
>> > I also couldn't find any references to build.properties on the wiki,
>> > here
>> > are the searches I tried:
>> >
>> >
>> > http://wiki.apache.org/general/FrontPage?action=fullsearch&context=180&value=build.properties&fullsearch=Text
>> >
>> > http://www.google.com/?q=%22build.properties%22+site:wiki.apache.org%2Flucene
>> >
>> > http://www.google.com/?q=%22build.properties%22+site:wiki.apache.org%2Fsolr
>> >
>> >
>> > Is this documented somewhere else? I'd be happy to back some out from
>> > the
>> > ant files, collate documentation from other sources and make it easier
>> > to
>> > find.
>> >
>> > --Terry
>> >
>> >
>> > On Mon, Mar 10, 2014 at 2:55 PM, Dawid Weiss
>> > <da...@cs.put.poznan.pl>
>> > wrote:
>> >>
>> >> > Dawid: Boy, those are some large timeouts!
>> >>
>> >> I know... I wasn't the one to bump them; my default was, I think,
>> >> about 3 minutes per class...
>> >>
>> >> Dawid
>> >>
>> >> ---------------------------------------------------------------------
>> >> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
>> >> For additional commands, e-mail: dev-help@lucene.apache.org
>> >>
>> >
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
>> For additional commands, e-mail: dev-help@lucene.apache.org
>>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
Re: Stalled unit tests
Posted by Terry Smith <sh...@gmail.com>.
Dawid,
Thanks, I didn't even know about it until Mike mentioned it earlier in this
thread. I've had it work from ~/lucene.build.properties and
~/build.properties but didn't have any luck putting it in the root of
project (I'm probably just miss reading the ant file).
--Terry
On Thu, Mar 13, 2014 at 9:35 AM, Dawid Weiss
<da...@cs.put.poznan.pl>wrote:
> Terry,
>
> The build.properties file holds the current user's config, as opposed
> to the defaults stored in the repository. In fact, there are more
> locations where you can put such defaults (see common-build.xml's
> header):
>
> <!-- Give user a chance to override without editing this file
> (and without typing -D each time it compiles it -->
> <property file="${user.home}/lucene.build.properties"/>
> <property file="${user.home}/build.properties"/>
> <property file="${basedir}/build.properties"/>
> <property file="${common.dir}/build.properties"/>
>
>
> Dawid
>
> On Thu, Mar 13, 2014 at 2:29 PM, Terry Smith <sh...@gmail.com> wrote:
> > It seems that you need to run the tests with `-Dtests.disableHdfs=true`
> for
> > them to succeed. Is there any interested in making this the default
> > behavior?
> >
> > If not, I'll happily start a new email thread to get wiki permissions so
> > that the contribution pages linked from the main README.txt will both
> > mention this important flag.
> >
> > http://wiki.apache.org/lucene-java/HowToContribute
> > http://wiki.apache.org/solr/HowToContribute
> >
> >
> > Right now they both state that you can run `ant clean test`,
> unfortunately
> > that command will fail if you run the tests from either the top level of
> the
> > project or the solr subdirectory unless you instead run `ant
> > -Dtests.disableHdfs=true clean test` or create a build.properties file.
> >
> > I also couldn't find any references to build.properties on the wiki, here
> > are the searches I tried:
> >
> >
> http://wiki.apache.org/general/FrontPage?action=fullsearch&context=180&value=build.properties&fullsearch=Text
> >
> http://www.google.com/?q=%22build.properties%22+site:wiki.apache.org%2Flucene
> >
> http://www.google.com/?q=%22build.properties%22+site:wiki.apache.org%2Fsolr
> >
> >
> > Is this documented somewhere else? I'd be happy to back some out from the
> > ant files, collate documentation from other sources and make it easier to
> > find.
> >
> > --Terry
> >
> >
> > On Mon, Mar 10, 2014 at 2:55 PM, Dawid Weiss <
> dawid.weiss@cs.put.poznan.pl>
> > wrote:
> >>
> >> > Dawid: Boy, those are some large timeouts!
> >>
> >> I know... I wasn't the one to bump them; my default was, I think,
> >> about 3 minutes per class...
> >>
> >> Dawid
> >>
> >> ---------------------------------------------------------------------
> >> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
> >> For additional commands, e-mail: dev-help@lucene.apache.org
> >>
> >
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
> For additional commands, e-mail: dev-help@lucene.apache.org
>
>
Re: Stalled unit tests
Posted by Dawid Weiss <da...@cs.put.poznan.pl>.
Terry,
The build.properties file holds the current user's config, as opposed
to the defaults stored in the repository. In fact, there are more
locations where you can put such defaults (see common-build.xml's
header):
<!-- Give user a chance to override without editing this file
(and without typing -D each time it compiles it -->
<property file="${user.home}/lucene.build.properties"/>
<property file="${user.home}/build.properties"/>
<property file="${basedir}/build.properties"/>
<property file="${common.dir}/build.properties"/>
Dawid
On Thu, Mar 13, 2014 at 2:29 PM, Terry Smith <sh...@gmail.com> wrote:
> It seems that you need to run the tests with `-Dtests.disableHdfs=true` for
> them to succeed. Is there any interested in making this the default
> behavior?
>
> If not, I'll happily start a new email thread to get wiki permissions so
> that the contribution pages linked from the main README.txt will both
> mention this important flag.
>
> http://wiki.apache.org/lucene-java/HowToContribute
> http://wiki.apache.org/solr/HowToContribute
>
>
> Right now they both state that you can run `ant clean test`, unfortunately
> that command will fail if you run the tests from either the top level of the
> project or the solr subdirectory unless you instead run `ant
> -Dtests.disableHdfs=true clean test` or create a build.properties file.
>
> I also couldn't find any references to build.properties on the wiki, here
> are the searches I tried:
>
> http://wiki.apache.org/general/FrontPage?action=fullsearch&context=180&value=build.properties&fullsearch=Text
> http://www.google.com/?q=%22build.properties%22+site:wiki.apache.org%2Flucene
> http://www.google.com/?q=%22build.properties%22+site:wiki.apache.org%2Fsolr
>
>
> Is this documented somewhere else? I'd be happy to back some out from the
> ant files, collate documentation from other sources and make it easier to
> find.
>
> --Terry
>
>
> On Mon, Mar 10, 2014 at 2:55 PM, Dawid Weiss <da...@cs.put.poznan.pl>
> wrote:
>>
>> > Dawid: Boy, those are some large timeouts!
>>
>> I know... I wasn't the one to bump them; my default was, I think,
>> about 3 minutes per class...
>>
>> Dawid
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
>> For additional commands, e-mail: dev-help@lucene.apache.org
>>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
Re: Stalled unit tests
Posted by Terry Smith <sh...@gmail.com>.
It seems that you need to run the tests with `-Dtests.disableHdfs=true` for
them to succeed. Is there any interested in making this the default
behavior?
If not, I'll happily start a new email thread to get wiki permissions so
that the contribution pages linked from the main
README.txt<https://github.com/apache/lucene-solr/blob/trunk/README.txt>
will
both mention this important flag.
- http://wiki.apache.org/lucene-java/HowToContribute
- http://wiki.apache.org/solr/HowToContribute
Right now they both state that you can run `ant clean test`, unfortunately
that command will fail if you run the tests from either the top level of
the project or the solr subdirectory unless you instead run `ant
-Dtests.disableHdfs=true clean test` or create a build.properties file.
I also couldn't find any references to build.properties on the wiki, here
are the searches I tried:
-
http://wiki.apache.org/general/FrontPage?action=fullsearch&context=180&value=build.properties&fullsearch=Text
-
http://www.google.com/?q=%22build.properties%22+site:wiki.apache.org%2Flucene
-
http://www.google.com/?q=%22build.properties%22+site:wiki.apache.org%2Fsolr
Is this documented somewhere else? I'd be happy to back some out from the
ant files, collate documentation from other sources and make it easier to
find.
--Terry
On Mon, Mar 10, 2014 at 2:55 PM, Dawid Weiss
<da...@cs.put.poznan.pl>wrote:
> > Dawid: Boy, those are some large timeouts!
>
> I know... I wasn't the one to bump them; my default was, I think,
> about 3 minutes per class...
>
> Dawid
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
> For additional commands, e-mail: dev-help@lucene.apache.org
>
>
Re: Stalled unit tests
Posted by Dawid Weiss <da...@cs.put.poznan.pl>.
> Dawid: Boy, those are some large timeouts!
I know... I wasn't the one to bump them; my default was, I think,
about 3 minutes per class...
Dawid
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
Re: Stalled unit tests
Posted by Terry Smith <sh...@gmail.com>.
Shalin: That makes sense. Both the machines I used for testing have SSDs.
On Mon, Mar 10, 2014 at 9:35 AM, Shalin Shekhar Mangar <
shalinmangar@gmail.com> wrote:
> In my experience, the test suite is much faster on an SSD. Around 18
> minutes on my mac book pro and 12 minutes on my PC for just the Solr
> tests with -Dtests.slow=true (both have SSDs)
>
> On Mon, Mar 10, 2014 at 7:02 PM, Terry Smith <sh...@gmail.com> wrote:
> > Oops, the second set of timings on the Mid 2012 MacBook Pro were for JUST
> > the solr tests.
> >
> >
> >
> > On Mon, Mar 10, 2014 at 9:31 AM, Terry Smith <sh...@gmail.com> wrote:
> >>
> >> Dawid: Boy, those are some large timeouts!
> >>
> >> Mike: The build.properties suggestion resolved my issue. I can now run
> the
> >> test to completion.
> >>
> >> On a Mid 2009 MacBook Pro running Mavericks and using Java 6 executing
> ant
> >> from the top level of the lucene-solr project I get the following
> timings:
> >>
> >> ant clean compile -- 3 minutes
> >> ant clean test (tests.disableHdfs=true, tests.slow=false) -- 55 minutes
> >> ant clean test (tests.disableHdfs=true) -- 88 minutes
> >>
> >> On a Mid 2012 MacBook Pro with the same software stack:
> >>
> >> ant clean compile -- 1 minute
> >> ant clean test (tests.disableHdfs=true, tests.slow=false) -- 8 minutes
> >>
> >> All running from the same git commit mentioned at the top of this
> thread.
> >>
> >> The tests make great use of multiple CPU/cores so a faster machine
> makes a
> >> huge difference to the total runtime.
> >>
> >> Do the HDFS tests fail due to test bugs or implementation issues?
> >>
> >> How do you feel about changing the default value of tests.disableHdfs to
> >> true versus updating the wiki documentation to let knew contributors
> know
> >> how to work around this?
> >>
> >> --Terry
> >>
> >>
> >>
> >>
> >> On Fri, Mar 7, 2014 at 12:46 PM, Michael McCandless
> >> <lu...@mikemccandless.com> wrote:
> >>>
> >>> I just ran "ant test" under Solr; it took 4 minutes 25 seconds.
> >>>
> >>> But, in my ~/build.properties I have:
> >>>
> >>> tests.disableHdfs=true
> >>> tests.slow=false
> >>>
> >>> Which makes things substantially faster, and also [seems to] sidestep
> >>> the Solr tests that false fail.
> >>>
> >>> Mike McCandless
> >>>
> >>> http://blog.mikemccandless.com
> >>>
> >>>
> >>> On Fri, Mar 7, 2014 at 9:04 AM, Terry Smith <sh...@gmail.com> wrote:
> >>> > Mike,
> >>> >
> >>> > Fair enough. I'll let them run for more than 30 minutes and see what
> >>> > happens.
> >>> >
> >>> > How long does it take on your machine? I'm happy to signup for the
> wiki
> >>> > and
> >>> > add some extra information to
> >>> > http://wiki.apache.org/lucene-java/HowToContribute for folks
> wanting to
> >>> > tinker with Lucene.
> >>> >
> >>> > Do the Lucene developers typically run a subset of the test suite to
> >>> > make
> >>> > committing cheaper?
> >>> >
> >>> > Thanks,
> >>> >
> >>> > --Terry
> >>> >
> >>> >
> >>> >
> >>> > On Fri, Mar 7, 2014 at 5:52 AM, Michael McCandless
> >>> > <lu...@mikemccandless.com> wrote:
> >>> >>
> >>> >> Unfortunately, some tests take a very long time, and the test infra
> >>> >> will print these HEARTBEAT messages notifying you that they are
> still
> >>> >> running. They should eventually finish?
> >>> >>
> >>> >> Mike McCandless
> >>> >>
> >>> >> http://blog.mikemccandless.com
> >>> >>
> >>> >>
> >>> >> On Thu, Mar 6, 2014 at 5:09 PM, Terry Smith <sh...@gmail.com>
> wrote:
> >>> >> > I'm sure that I'm just missing something obvious but I'm having
> >>> >> > trouble
> >>> >> > getting the unit tests to run to completion on my laptop and was
> >>> >> > hoping
> >>> >> > that
> >>> >> > someone would be kind enough to point me in the right direction.
> >>> >> >
> >>> >> > I've cloned the repository from GitHub
> >>> >> > (http://git.apache.org/lucene-solr.git) and checked out the
> latest
> >>> >> > commit on
> >>> >> > branch_4x.
> >>> >> >
> >>> >> > commit 6e06247cec1410f32592bfd307c1020b814def06
> >>> >> >
> >>> >> > Author: Robert Muir <rm...@apache.org>
> >>> >> >
> >>> >> > Date: Thu Mar 6 19:54:07 2014 +0000
> >>> >> >
> >>> >> >
> >>> >> > disable slow solr tests in smoketester
> >>> >> >
> >>> >> >
> >>> >> >
> >>> >> > git-svn-id:
> >>> >> >
> >>> >> >
> https://svn.apache.org/repos/asf/lucene/dev/branches/branch_4x@1575025
> >>> >> > 13f79535-47bb-0310-9956-ffa450edef68
> >>> >> >
> >>> >> >
> >>> >> > Executing "ant clean test" from the top level directory of the
> >>> >> > project
> >>> >> > shows
> >>> >> > the tests running but they seems to get stuck in loop with some
> >>> >> > stalled
> >>> >> > heartbeat messages. If I run the tests directly from lucene/ then
> >>> >> > they
> >>> >> > complete successfully after about 10 minutes.
> >>> >> >
> >>> >> > I'm using Java 6 under OS X (10.9.2).
> >>> >> >
> >>> >> > $ java -version
> >>> >> >
> >>> >> > java version "1.6.0_65"
> >>> >> >
> >>> >> > Java(TM) SE Runtime Environment (build 1.6.0_65-b14-462-11M4609)
> >>> >> >
> >>> >> > Java HotSpot(TM) 64-Bit Server VM (build 20.65-b04-462, mixed
> mode)
> >>> >> >
> >>> >> >
> >>> >> > My terminal lists repeating stalled heartbeat messages like so:
> >>> >> >
> >>> >> > HEARTBEAT J2 PID(20104@onyx.local): 2014-03-06T16:53:35, stalled
> for
> >>> >> > 2111s
> >>> >> > at: HdfsLockFactoryTest.testBasic
> >>> >> >
> >>> >> > HEARTBEAT J0 PID(20106@onyx.local): 2014-03-06T16:53:47, stalled
> for
> >>> >> > 2108s
> >>> >> > at: TestSurroundQueryParser.testQueryParser
> >>> >> >
> >>> >> > HEARTBEAT J1 PID(20103@onyx.local): 2014-03-06T16:54:11, stalled
> for
> >>> >> > 2167s
> >>> >> > at: TestRecoveryHdfs.testBuffering
> >>> >> >
> >>> >> > HEARTBEAT J3 PID(20105@onyx.local): 2014-03-06T16:54:23, stalled
> for
> >>> >> > 2165s
> >>> >> > at: HdfsDirectoryTest.testEOF
> >>> >> >
> >>> >> >
> >>> >> > My machine does have 3 java processes chewing CPU, see attached
> >>> >> > jstack
> >>> >> > dumps
> >>> >> > for more information.
> >>> >> >
> >>> >> > Should I expect the tests to complete on my platform? Do I need to
> >>> >> > specify
> >>> >> > any special flags to give them more memory or to avoid any bad
> >>> >> > apples?
> >>> >> >
> >>> >> > Thanks in advance,
> >>> >> >
> >>> >> > --Terry
> >>> >> >
> >>> >> >
> >>> >> >
> >>> >> >
> >>> >> >
> >>> >> >
> ---------------------------------------------------------------------
> >>> >> > To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
> >>> >> > For additional commands, e-mail: dev-help@lucene.apache.org
> >>> >>
> >>> >>
> ---------------------------------------------------------------------
> >>> >> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
> >>> >> For additional commands, e-mail: dev-help@lucene.apache.org
> >>> >>
> >>> >
> >>>
> >>> ---------------------------------------------------------------------
> >>> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
> >>> For additional commands, e-mail: dev-help@lucene.apache.org
> >>>
> >>
> >
>
>
>
> --
> Regards,
> Shalin Shekhar Mangar.
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
> For additional commands, e-mail: dev-help@lucene.apache.org
>
>
Re: Stalled unit tests
Posted by Shalin Shekhar Mangar <sh...@gmail.com>.
In my experience, the test suite is much faster on an SSD. Around 18
minutes on my mac book pro and 12 minutes on my PC for just the Solr
tests with -Dtests.slow=true (both have SSDs)
On Mon, Mar 10, 2014 at 7:02 PM, Terry Smith <sh...@gmail.com> wrote:
> Oops, the second set of timings on the Mid 2012 MacBook Pro were for JUST
> the solr tests.
>
>
>
> On Mon, Mar 10, 2014 at 9:31 AM, Terry Smith <sh...@gmail.com> wrote:
>>
>> Dawid: Boy, those are some large timeouts!
>>
>> Mike: The build.properties suggestion resolved my issue. I can now run the
>> test to completion.
>>
>> On a Mid 2009 MacBook Pro running Mavericks and using Java 6 executing ant
>> from the top level of the lucene-solr project I get the following timings:
>>
>> ant clean compile -- 3 minutes
>> ant clean test (tests.disableHdfs=true, tests.slow=false) -- 55 minutes
>> ant clean test (tests.disableHdfs=true) -- 88 minutes
>>
>> On a Mid 2012 MacBook Pro with the same software stack:
>>
>> ant clean compile -- 1 minute
>> ant clean test (tests.disableHdfs=true, tests.slow=false) -- 8 minutes
>>
>> All running from the same git commit mentioned at the top of this thread.
>>
>> The tests make great use of multiple CPU/cores so a faster machine makes a
>> huge difference to the total runtime.
>>
>> Do the HDFS tests fail due to test bugs or implementation issues?
>>
>> How do you feel about changing the default value of tests.disableHdfs to
>> true versus updating the wiki documentation to let knew contributors know
>> how to work around this?
>>
>> --Terry
>>
>>
>>
>>
>> On Fri, Mar 7, 2014 at 12:46 PM, Michael McCandless
>> <lu...@mikemccandless.com> wrote:
>>>
>>> I just ran "ant test" under Solr; it took 4 minutes 25 seconds.
>>>
>>> But, in my ~/build.properties I have:
>>>
>>> tests.disableHdfs=true
>>> tests.slow=false
>>>
>>> Which makes things substantially faster, and also [seems to] sidestep
>>> the Solr tests that false fail.
>>>
>>> Mike McCandless
>>>
>>> http://blog.mikemccandless.com
>>>
>>>
>>> On Fri, Mar 7, 2014 at 9:04 AM, Terry Smith <sh...@gmail.com> wrote:
>>> > Mike,
>>> >
>>> > Fair enough. I'll let them run for more than 30 minutes and see what
>>> > happens.
>>> >
>>> > How long does it take on your machine? I'm happy to signup for the wiki
>>> > and
>>> > add some extra information to
>>> > http://wiki.apache.org/lucene-java/HowToContribute for folks wanting to
>>> > tinker with Lucene.
>>> >
>>> > Do the Lucene developers typically run a subset of the test suite to
>>> > make
>>> > committing cheaper?
>>> >
>>> > Thanks,
>>> >
>>> > --Terry
>>> >
>>> >
>>> >
>>> > On Fri, Mar 7, 2014 at 5:52 AM, Michael McCandless
>>> > <lu...@mikemccandless.com> wrote:
>>> >>
>>> >> Unfortunately, some tests take a very long time, and the test infra
>>> >> will print these HEARTBEAT messages notifying you that they are still
>>> >> running. They should eventually finish?
>>> >>
>>> >> Mike McCandless
>>> >>
>>> >> http://blog.mikemccandless.com
>>> >>
>>> >>
>>> >> On Thu, Mar 6, 2014 at 5:09 PM, Terry Smith <sh...@gmail.com> wrote:
>>> >> > I'm sure that I'm just missing something obvious but I'm having
>>> >> > trouble
>>> >> > getting the unit tests to run to completion on my laptop and was
>>> >> > hoping
>>> >> > that
>>> >> > someone would be kind enough to point me in the right direction.
>>> >> >
>>> >> > I've cloned the repository from GitHub
>>> >> > (http://git.apache.org/lucene-solr.git) and checked out the latest
>>> >> > commit on
>>> >> > branch_4x.
>>> >> >
>>> >> > commit 6e06247cec1410f32592bfd307c1020b814def06
>>> >> >
>>> >> > Author: Robert Muir <rm...@apache.org>
>>> >> >
>>> >> > Date: Thu Mar 6 19:54:07 2014 +0000
>>> >> >
>>> >> >
>>> >> > disable slow solr tests in smoketester
>>> >> >
>>> >> >
>>> >> >
>>> >> > git-svn-id:
>>> >> >
>>> >> > https://svn.apache.org/repos/asf/lucene/dev/branches/branch_4x@1575025
>>> >> > 13f79535-47bb-0310-9956-ffa450edef68
>>> >> >
>>> >> >
>>> >> > Executing "ant clean test" from the top level directory of the
>>> >> > project
>>> >> > shows
>>> >> > the tests running but they seems to get stuck in loop with some
>>> >> > stalled
>>> >> > heartbeat messages. If I run the tests directly from lucene/ then
>>> >> > they
>>> >> > complete successfully after about 10 minutes.
>>> >> >
>>> >> > I'm using Java 6 under OS X (10.9.2).
>>> >> >
>>> >> > $ java -version
>>> >> >
>>> >> > java version "1.6.0_65"
>>> >> >
>>> >> > Java(TM) SE Runtime Environment (build 1.6.0_65-b14-462-11M4609)
>>> >> >
>>> >> > Java HotSpot(TM) 64-Bit Server VM (build 20.65-b04-462, mixed mode)
>>> >> >
>>> >> >
>>> >> > My terminal lists repeating stalled heartbeat messages like so:
>>> >> >
>>> >> > HEARTBEAT J2 PID(20104@onyx.local): 2014-03-06T16:53:35, stalled for
>>> >> > 2111s
>>> >> > at: HdfsLockFactoryTest.testBasic
>>> >> >
>>> >> > HEARTBEAT J0 PID(20106@onyx.local): 2014-03-06T16:53:47, stalled for
>>> >> > 2108s
>>> >> > at: TestSurroundQueryParser.testQueryParser
>>> >> >
>>> >> > HEARTBEAT J1 PID(20103@onyx.local): 2014-03-06T16:54:11, stalled for
>>> >> > 2167s
>>> >> > at: TestRecoveryHdfs.testBuffering
>>> >> >
>>> >> > HEARTBEAT J3 PID(20105@onyx.local): 2014-03-06T16:54:23, stalled for
>>> >> > 2165s
>>> >> > at: HdfsDirectoryTest.testEOF
>>> >> >
>>> >> >
>>> >> > My machine does have 3 java processes chewing CPU, see attached
>>> >> > jstack
>>> >> > dumps
>>> >> > for more information.
>>> >> >
>>> >> > Should I expect the tests to complete on my platform? Do I need to
>>> >> > specify
>>> >> > any special flags to give them more memory or to avoid any bad
>>> >> > apples?
>>> >> >
>>> >> > Thanks in advance,
>>> >> >
>>> >> > --Terry
>>> >> >
>>> >> >
>>> >> >
>>> >> >
>>> >> >
>>> >> > ---------------------------------------------------------------------
>>> >> > To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
>>> >> > For additional commands, e-mail: dev-help@lucene.apache.org
>>> >>
>>> >> ---------------------------------------------------------------------
>>> >> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
>>> >> For additional commands, e-mail: dev-help@lucene.apache.org
>>> >>
>>> >
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
>>> For additional commands, e-mail: dev-help@lucene.apache.org
>>>
>>
>
--
Regards,
Shalin Shekhar Mangar.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
Re: Stalled unit tests
Posted by Terry Smith <sh...@gmail.com>.
Oops, the second set of timings on the Mid 2012 MacBook Pro were for JUST
the solr tests.
On Mon, Mar 10, 2014 at 9:31 AM, Terry Smith <sh...@gmail.com> wrote:
> Dawid: Boy, those are some large timeouts!
>
> Mike: The build.properties suggestion resolved my issue. I can now run the
> test to completion.
>
> On a Mid 2009 MacBook Pro running Mavericks and using Java 6 executing ant
> from the top level of the lucene-solr project I get the following timings:
>
> ant clean compile -- 3 minutes
> ant clean test (tests.disableHdfs=true, tests.slow=false) -- 55 minutes
> ant clean test (tests.disableHdfs=true) -- 88 minutes
>
> On a Mid 2012 MacBook Pro with the same software stack:
>
> ant clean compile -- 1 minute
> ant clean test (tests.disableHdfs=true, tests.slow=false) -- 8 minutes
>
> All running from the same git commit mentioned at the top of this thread.
>
> The tests make great use of multiple CPU/cores so a faster machine makes a
> huge difference to the total runtime.
>
> Do the HDFS tests fail due to test bugs or implementation issues?
>
> How do you feel about changing the default value of tests.disableHdfs to
> true versus updating the wiki documentation to let knew contributors know
> how to work around this?
>
> --Terry
>
>
>
>
> On Fri, Mar 7, 2014 at 12:46 PM, Michael McCandless <
> lucene@mikemccandless.com> wrote:
>
>> I just ran "ant test" under Solr; it took 4 minutes 25 seconds.
>>
>> But, in my ~/build.properties I have:
>>
>> tests.disableHdfs=true
>> tests.slow=false
>>
>> Which makes things substantially faster, and also [seems to] sidestep
>> the Solr tests that false fail.
>>
>> Mike McCandless
>>
>> http://blog.mikemccandless.com
>>
>>
>> On Fri, Mar 7, 2014 at 9:04 AM, Terry Smith <sh...@gmail.com> wrote:
>> > Mike,
>> >
>> > Fair enough. I'll let them run for more than 30 minutes and see what
>> > happens.
>> >
>> > How long does it take on your machine? I'm happy to signup for the wiki
>> and
>> > add some extra information to
>> > http://wiki.apache.org/lucene-java/HowToContribute for folks wanting to
>> > tinker with Lucene.
>> >
>> > Do the Lucene developers typically run a subset of the test suite to
>> make
>> > committing cheaper?
>> >
>> > Thanks,
>> >
>> > --Terry
>> >
>> >
>> >
>> > On Fri, Mar 7, 2014 at 5:52 AM, Michael McCandless
>> > <lu...@mikemccandless.com> wrote:
>> >>
>> >> Unfortunately, some tests take a very long time, and the test infra
>> >> will print these HEARTBEAT messages notifying you that they are still
>> >> running. They should eventually finish?
>> >>
>> >> Mike McCandless
>> >>
>> >> http://blog.mikemccandless.com
>> >>
>> >>
>> >> On Thu, Mar 6, 2014 at 5:09 PM, Terry Smith <sh...@gmail.com> wrote:
>> >> > I'm sure that I'm just missing something obvious but I'm having
>> trouble
>> >> > getting the unit tests to run to completion on my laptop and was
>> hoping
>> >> > that
>> >> > someone would be kind enough to point me in the right direction.
>> >> >
>> >> > I've cloned the repository from GitHub
>> >> > (http://git.apache.org/lucene-solr.git) and checked out the latest
>> >> > commit on
>> >> > branch_4x.
>> >> >
>> >> > commit 6e06247cec1410f32592bfd307c1020b814def06
>> >> >
>> >> > Author: Robert Muir <rm...@apache.org>
>> >> >
>> >> > Date: Thu Mar 6 19:54:07 2014 +0000
>> >> >
>> >> >
>> >> > disable slow solr tests in smoketester
>> >> >
>> >> >
>> >> >
>> >> > git-svn-id:
>> >> >
>> https://svn.apache.org/repos/asf/lucene/dev/branches/branch_4x@1575025
>> >> > 13f79535-47bb-0310-9956-ffa450edef68
>> >> >
>> >> >
>> >> > Executing "ant clean test" from the top level directory of the
>> project
>> >> > shows
>> >> > the tests running but they seems to get stuck in loop with some
>> stalled
>> >> > heartbeat messages. If I run the tests directly from lucene/ then
>> they
>> >> > complete successfully after about 10 minutes.
>> >> >
>> >> > I'm using Java 6 under OS X (10.9.2).
>> >> >
>> >> > $ java -version
>> >> >
>> >> > java version "1.6.0_65"
>> >> >
>> >> > Java(TM) SE Runtime Environment (build 1.6.0_65-b14-462-11M4609)
>> >> >
>> >> > Java HotSpot(TM) 64-Bit Server VM (build 20.65-b04-462, mixed mode)
>> >> >
>> >> >
>> >> > My terminal lists repeating stalled heartbeat messages like so:
>> >> >
>> >> > HEARTBEAT J2 PID(20104@onyx.local): 2014-03-06T16:53:35, stalled for
>> >> > 2111s
>> >> > at: HdfsLockFactoryTest.testBasic
>> >> >
>> >> > HEARTBEAT J0 PID(20106@onyx.local): 2014-03-06T16:53:47, stalled for
>> >> > 2108s
>> >> > at: TestSurroundQueryParser.testQueryParser
>> >> >
>> >> > HEARTBEAT J1 PID(20103@onyx.local): 2014-03-06T16:54:11, stalled for
>> >> > 2167s
>> >> > at: TestRecoveryHdfs.testBuffering
>> >> >
>> >> > HEARTBEAT J3 PID(20105@onyx.local): 2014-03-06T16:54:23, stalled for
>> >> > 2165s
>> >> > at: HdfsDirectoryTest.testEOF
>> >> >
>> >> >
>> >> > My machine does have 3 java processes chewing CPU, see attached
>> jstack
>> >> > dumps
>> >> > for more information.
>> >> >
>> >> > Should I expect the tests to complete on my platform? Do I need to
>> >> > specify
>> >> > any special flags to give them more memory or to avoid any bad
>> apples?
>> >> >
>> >> > Thanks in advance,
>> >> >
>> >> > --Terry
>> >> >
>> >> >
>> >> >
>> >> >
>> >> > ---------------------------------------------------------------------
>> >> > To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
>> >> > For additional commands, e-mail: dev-help@lucene.apache.org
>> >>
>> >> ---------------------------------------------------------------------
>> >> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
>> >> For additional commands, e-mail: dev-help@lucene.apache.org
>> >>
>> >
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
>> For additional commands, e-mail: dev-help@lucene.apache.org
>>
>>
>
Re: Stalled unit tests
Posted by Terry Smith <sh...@gmail.com>.
Dawid: Boy, those are some large timeouts!
Mike: The build.properties suggestion resolved my issue. I can now run the
test to completion.
On a Mid 2009 MacBook Pro running Mavericks and using Java 6 executing ant
from the top level of the lucene-solr project I get the following timings:
ant clean compile -- 3 minutes
ant clean test (tests.disableHdfs=true, tests.slow=false) -- 55 minutes
ant clean test (tests.disableHdfs=true) -- 88 minutes
On a Mid 2012 MacBook Pro with the same software stack:
ant clean compile -- 1 minute
ant clean test (tests.disableHdfs=true, tests.slow=false) -- 8 minutes
All running from the same git commit mentioned at the top of this thread.
The tests make great use of multiple CPU/cores so a faster machine makes a
huge difference to the total runtime.
Do the HDFS tests fail due to test bugs or implementation issues?
How do you feel about changing the default value of tests.disableHdfs to
true versus updating the wiki documentation to let knew contributors know
how to work around this?
--Terry
On Fri, Mar 7, 2014 at 12:46 PM, Michael McCandless <
lucene@mikemccandless.com> wrote:
> I just ran "ant test" under Solr; it took 4 minutes 25 seconds.
>
> But, in my ~/build.properties I have:
>
> tests.disableHdfs=true
> tests.slow=false
>
> Which makes things substantially faster, and also [seems to] sidestep
> the Solr tests that false fail.
>
> Mike McCandless
>
> http://blog.mikemccandless.com
>
>
> On Fri, Mar 7, 2014 at 9:04 AM, Terry Smith <sh...@gmail.com> wrote:
> > Mike,
> >
> > Fair enough. I'll let them run for more than 30 minutes and see what
> > happens.
> >
> > How long does it take on your machine? I'm happy to signup for the wiki
> and
> > add some extra information to
> > http://wiki.apache.org/lucene-java/HowToContribute for folks wanting to
> > tinker with Lucene.
> >
> > Do the Lucene developers typically run a subset of the test suite to make
> > committing cheaper?
> >
> > Thanks,
> >
> > --Terry
> >
> >
> >
> > On Fri, Mar 7, 2014 at 5:52 AM, Michael McCandless
> > <lu...@mikemccandless.com> wrote:
> >>
> >> Unfortunately, some tests take a very long time, and the test infra
> >> will print these HEARTBEAT messages notifying you that they are still
> >> running. They should eventually finish?
> >>
> >> Mike McCandless
> >>
> >> http://blog.mikemccandless.com
> >>
> >>
> >> On Thu, Mar 6, 2014 at 5:09 PM, Terry Smith <sh...@gmail.com> wrote:
> >> > I'm sure that I'm just missing something obvious but I'm having
> trouble
> >> > getting the unit tests to run to completion on my laptop and was
> hoping
> >> > that
> >> > someone would be kind enough to point me in the right direction.
> >> >
> >> > I've cloned the repository from GitHub
> >> > (http://git.apache.org/lucene-solr.git) and checked out the latest
> >> > commit on
> >> > branch_4x.
> >> >
> >> > commit 6e06247cec1410f32592bfd307c1020b814def06
> >> >
> >> > Author: Robert Muir <rm...@apache.org>
> >> >
> >> > Date: Thu Mar 6 19:54:07 2014 +0000
> >> >
> >> >
> >> > disable slow solr tests in smoketester
> >> >
> >> >
> >> >
> >> > git-svn-id:
> >> >
> https://svn.apache.org/repos/asf/lucene/dev/branches/branch_4x@1575025
> >> > 13f79535-47bb-0310-9956-ffa450edef68
> >> >
> >> >
> >> > Executing "ant clean test" from the top level directory of the project
> >> > shows
> >> > the tests running but they seems to get stuck in loop with some
> stalled
> >> > heartbeat messages. If I run the tests directly from lucene/ then they
> >> > complete successfully after about 10 minutes.
> >> >
> >> > I'm using Java 6 under OS X (10.9.2).
> >> >
> >> > $ java -version
> >> >
> >> > java version "1.6.0_65"
> >> >
> >> > Java(TM) SE Runtime Environment (build 1.6.0_65-b14-462-11M4609)
> >> >
> >> > Java HotSpot(TM) 64-Bit Server VM (build 20.65-b04-462, mixed mode)
> >> >
> >> >
> >> > My terminal lists repeating stalled heartbeat messages like so:
> >> >
> >> > HEARTBEAT J2 PID(20104@onyx.local): 2014-03-06T16:53:35, stalled for
> >> > 2111s
> >> > at: HdfsLockFactoryTest.testBasic
> >> >
> >> > HEARTBEAT J0 PID(20106@onyx.local): 2014-03-06T16:53:47, stalled for
> >> > 2108s
> >> > at: TestSurroundQueryParser.testQueryParser
> >> >
> >> > HEARTBEAT J1 PID(20103@onyx.local): 2014-03-06T16:54:11, stalled for
> >> > 2167s
> >> > at: TestRecoveryHdfs.testBuffering
> >> >
> >> > HEARTBEAT J3 PID(20105@onyx.local): 2014-03-06T16:54:23, stalled for
> >> > 2165s
> >> > at: HdfsDirectoryTest.testEOF
> >> >
> >> >
> >> > My machine does have 3 java processes chewing CPU, see attached jstack
> >> > dumps
> >> > for more information.
> >> >
> >> > Should I expect the tests to complete on my platform? Do I need to
> >> > specify
> >> > any special flags to give them more memory or to avoid any bad apples?
> >> >
> >> > Thanks in advance,
> >> >
> >> > --Terry
> >> >
> >> >
> >> >
> >> >
> >> > ---------------------------------------------------------------------
> >> > To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
> >> > For additional commands, e-mail: dev-help@lucene.apache.org
> >>
> >> ---------------------------------------------------------------------
> >> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
> >> For additional commands, e-mail: dev-help@lucene.apache.org
> >>
> >
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
> For additional commands, e-mail: dev-help@lucene.apache.org
>
>
Re: Stalled unit tests
Posted by Michael McCandless <lu...@mikemccandless.com>.
I just ran "ant test" under Solr; it took 4 minutes 25 seconds.
But, in my ~/build.properties I have:
tests.disableHdfs=true
tests.slow=false
Which makes things substantially faster, and also [seems to] sidestep
the Solr tests that false fail.
Mike McCandless
http://blog.mikemccandless.com
On Fri, Mar 7, 2014 at 9:04 AM, Terry Smith <sh...@gmail.com> wrote:
> Mike,
>
> Fair enough. I'll let them run for more than 30 minutes and see what
> happens.
>
> How long does it take on your machine? I'm happy to signup for the wiki and
> add some extra information to
> http://wiki.apache.org/lucene-java/HowToContribute for folks wanting to
> tinker with Lucene.
>
> Do the Lucene developers typically run a subset of the test suite to make
> committing cheaper?
>
> Thanks,
>
> --Terry
>
>
>
> On Fri, Mar 7, 2014 at 5:52 AM, Michael McCandless
> <lu...@mikemccandless.com> wrote:
>>
>> Unfortunately, some tests take a very long time, and the test infra
>> will print these HEARTBEAT messages notifying you that they are still
>> running. They should eventually finish?
>>
>> Mike McCandless
>>
>> http://blog.mikemccandless.com
>>
>>
>> On Thu, Mar 6, 2014 at 5:09 PM, Terry Smith <sh...@gmail.com> wrote:
>> > I'm sure that I'm just missing something obvious but I'm having trouble
>> > getting the unit tests to run to completion on my laptop and was hoping
>> > that
>> > someone would be kind enough to point me in the right direction.
>> >
>> > I've cloned the repository from GitHub
>> > (http://git.apache.org/lucene-solr.git) and checked out the latest
>> > commit on
>> > branch_4x.
>> >
>> > commit 6e06247cec1410f32592bfd307c1020b814def06
>> >
>> > Author: Robert Muir <rm...@apache.org>
>> >
>> > Date: Thu Mar 6 19:54:07 2014 +0000
>> >
>> >
>> > disable slow solr tests in smoketester
>> >
>> >
>> >
>> > git-svn-id:
>> > https://svn.apache.org/repos/asf/lucene/dev/branches/branch_4x@1575025
>> > 13f79535-47bb-0310-9956-ffa450edef68
>> >
>> >
>> > Executing "ant clean test" from the top level directory of the project
>> > shows
>> > the tests running but they seems to get stuck in loop with some stalled
>> > heartbeat messages. If I run the tests directly from lucene/ then they
>> > complete successfully after about 10 minutes.
>> >
>> > I'm using Java 6 under OS X (10.9.2).
>> >
>> > $ java -version
>> >
>> > java version "1.6.0_65"
>> >
>> > Java(TM) SE Runtime Environment (build 1.6.0_65-b14-462-11M4609)
>> >
>> > Java HotSpot(TM) 64-Bit Server VM (build 20.65-b04-462, mixed mode)
>> >
>> >
>> > My terminal lists repeating stalled heartbeat messages like so:
>> >
>> > HEARTBEAT J2 PID(20104@onyx.local): 2014-03-06T16:53:35, stalled for
>> > 2111s
>> > at: HdfsLockFactoryTest.testBasic
>> >
>> > HEARTBEAT J0 PID(20106@onyx.local): 2014-03-06T16:53:47, stalled for
>> > 2108s
>> > at: TestSurroundQueryParser.testQueryParser
>> >
>> > HEARTBEAT J1 PID(20103@onyx.local): 2014-03-06T16:54:11, stalled for
>> > 2167s
>> > at: TestRecoveryHdfs.testBuffering
>> >
>> > HEARTBEAT J3 PID(20105@onyx.local): 2014-03-06T16:54:23, stalled for
>> > 2165s
>> > at: HdfsDirectoryTest.testEOF
>> >
>> >
>> > My machine does have 3 java processes chewing CPU, see attached jstack
>> > dumps
>> > for more information.
>> >
>> > Should I expect the tests to complete on my platform? Do I need to
>> > specify
>> > any special flags to give them more memory or to avoid any bad apples?
>> >
>> > Thanks in advance,
>> >
>> > --Terry
>> >
>> >
>> >
>> >
>> > ---------------------------------------------------------------------
>> > To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
>> > For additional commands, e-mail: dev-help@lucene.apache.org
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
>> For additional commands, e-mail: dev-help@lucene.apache.org
>>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
Re: Stalled unit tests
Posted by Dawid Weiss <da...@cs.put.poznan.pl>.
> How long does it take on your machine?
It really depends... check out the limit on some heavy nightly tests
like this one:
@TimeoutSuite(millis = 80 * TimeUnits.HOUR)
@Ignore("takes ~ 45 minutes")
(Somebody should really inspect this inconsistency :).
Or this one:
@Ignore("Requires tons of heap to run (420G works)")
@TimeoutSuite(millis = 100 * TimeUnits.HOUR)
Wait... how many Gs? :)
And seriously the top parent class of all tests declares:
@TimeoutSuite(millis = 2 * TimeUnits.HOUR)
And this unfortunately means that a test class will timeout after 2
hours of inactivity. To me, it's absurdly high but in the past tests
ran on very slow virtualized machines and were actually hitting these
limits.
Dawid
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
Re: Stalled unit tests
Posted by Terry Smith <sh...@gmail.com>.
Mike,
Fair enough. I'll let them run for more than 30 minutes and see what
happens.
How long does it take on your machine? I'm happy to signup for the wiki and
add some extra information to
http://wiki.apache.org/lucene-java/HowToContribute for folks wanting to
tinker with Lucene.
Do the Lucene developers typically run a subset of the test suite to make
committing cheaper?
Thanks,
--Terry
On Fri, Mar 7, 2014 at 5:52 AM, Michael McCandless <
lucene@mikemccandless.com> wrote:
> Unfortunately, some tests take a very long time, and the test infra
> will print these HEARTBEAT messages notifying you that they are still
> running. They should eventually finish?
>
> Mike McCandless
>
> http://blog.mikemccandless.com
>
>
> On Thu, Mar 6, 2014 at 5:09 PM, Terry Smith <sh...@gmail.com> wrote:
> > I'm sure that I'm just missing something obvious but I'm having trouble
> > getting the unit tests to run to completion on my laptop and was hoping
> that
> > someone would be kind enough to point me in the right direction.
> >
> > I've cloned the repository from GitHub
> > (http://git.apache.org/lucene-solr.git) and checked out the latest
> commit on
> > branch_4x.
> >
> > commit 6e06247cec1410f32592bfd307c1020b814def06
> >
> > Author: Robert Muir <rm...@apache.org>
> >
> > Date: Thu Mar 6 19:54:07 2014 +0000
> >
> >
> > disable slow solr tests in smoketester
> >
> >
> >
> > git-svn-id:
> > https://svn.apache.org/repos/asf/lucene/dev/branches/branch_4x@1575025
> > 13f79535-47bb-0310-9956-ffa450edef68
> >
> >
> > Executing "ant clean test" from the top level directory of the project
> shows
> > the tests running but they seems to get stuck in loop with some stalled
> > heartbeat messages. If I run the tests directly from lucene/ then they
> > complete successfully after about 10 minutes.
> >
> > I'm using Java 6 under OS X (10.9.2).
> >
> > $ java -version
> >
> > java version "1.6.0_65"
> >
> > Java(TM) SE Runtime Environment (build 1.6.0_65-b14-462-11M4609)
> >
> > Java HotSpot(TM) 64-Bit Server VM (build 20.65-b04-462, mixed mode)
> >
> >
> > My terminal lists repeating stalled heartbeat messages like so:
> >
> > HEARTBEAT J2 PID(20104@onyx.local): 2014-03-06T16:53:35, stalled for
> 2111s
> > at: HdfsLockFactoryTest.testBasic
> >
> > HEARTBEAT J0 PID(20106@onyx.local): 2014-03-06T16:53:47, stalled for
> 2108s
> > at: TestSurroundQueryParser.testQueryParser
> >
> > HEARTBEAT J1 PID(20103@onyx.local): 2014-03-06T16:54:11, stalled for
> 2167s
> > at: TestRecoveryHdfs.testBuffering
> >
> > HEARTBEAT J3 PID(20105@onyx.local): 2014-03-06T16:54:23, stalled for
> 2165s
> > at: HdfsDirectoryTest.testEOF
> >
> >
> > My machine does have 3 java processes chewing CPU, see attached jstack
> dumps
> > for more information.
> >
> > Should I expect the tests to complete on my platform? Do I need to
> specify
> > any special flags to give them more memory or to avoid any bad apples?
> >
> > Thanks in advance,
> >
> > --Terry
> >
> >
> >
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
> > For additional commands, e-mail: dev-help@lucene.apache.org
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
> For additional commands, e-mail: dev-help@lucene.apache.org
>
>
Re: Stalled unit tests
Posted by Michael McCandless <lu...@mikemccandless.com>.
Unfortunately, some tests take a very long time, and the test infra
will print these HEARTBEAT messages notifying you that they are still
running. They should eventually finish?
Mike McCandless
http://blog.mikemccandless.com
On Thu, Mar 6, 2014 at 5:09 PM, Terry Smith <sh...@gmail.com> wrote:
> I'm sure that I'm just missing something obvious but I'm having trouble
> getting the unit tests to run to completion on my laptop and was hoping that
> someone would be kind enough to point me in the right direction.
>
> I've cloned the repository from GitHub
> (http://git.apache.org/lucene-solr.git) and checked out the latest commit on
> branch_4x.
>
> commit 6e06247cec1410f32592bfd307c1020b814def06
>
> Author: Robert Muir <rm...@apache.org>
>
> Date: Thu Mar 6 19:54:07 2014 +0000
>
>
> disable slow solr tests in smoketester
>
>
>
> git-svn-id:
> https://svn.apache.org/repos/asf/lucene/dev/branches/branch_4x@1575025
> 13f79535-47bb-0310-9956-ffa450edef68
>
>
> Executing "ant clean test" from the top level directory of the project shows
> the tests running but they seems to get stuck in loop with some stalled
> heartbeat messages. If I run the tests directly from lucene/ then they
> complete successfully after about 10 minutes.
>
> I'm using Java 6 under OS X (10.9.2).
>
> $ java -version
>
> java version "1.6.0_65"
>
> Java(TM) SE Runtime Environment (build 1.6.0_65-b14-462-11M4609)
>
> Java HotSpot(TM) 64-Bit Server VM (build 20.65-b04-462, mixed mode)
>
>
> My terminal lists repeating stalled heartbeat messages like so:
>
> HEARTBEAT J2 PID(20104@onyx.local): 2014-03-06T16:53:35, stalled for 2111s
> at: HdfsLockFactoryTest.testBasic
>
> HEARTBEAT J0 PID(20106@onyx.local): 2014-03-06T16:53:47, stalled for 2108s
> at: TestSurroundQueryParser.testQueryParser
>
> HEARTBEAT J1 PID(20103@onyx.local): 2014-03-06T16:54:11, stalled for 2167s
> at: TestRecoveryHdfs.testBuffering
>
> HEARTBEAT J3 PID(20105@onyx.local): 2014-03-06T16:54:23, stalled for 2165s
> at: HdfsDirectoryTest.testEOF
>
>
> My machine does have 3 java processes chewing CPU, see attached jstack dumps
> for more information.
>
> Should I expect the tests to complete on my platform? Do I need to specify
> any special flags to give them more memory or to avoid any bad apples?
>
> Thanks in advance,
>
> --Terry
>
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
> For additional commands, e-mail: dev-help@lucene.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org