You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucene.apache.org by "Robert Muir (JIRA)" <ji...@apache.org> on 2010/12/18 22:24:01 UTC
[jira] Created: (LUCENE-2822) TimeLimitingCollector starts thread
in static {} with no way to stop them
TimeLimitingCollector starts thread in static {} with no way to stop them
-------------------------------------------------------------------------
Key: LUCENE-2822
URL: https://issues.apache.org/jira/browse/LUCENE-2822
Project: Lucene - Java
Issue Type: Bug
Reporter: Robert Muir
See the comment in LuceneTestCase.
If you even do Class.forName("TimeLimitingCollector") it starts up a thread in a static method, and there isn't a way to kill it.
This is broken.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] [Updated] (LUCENE-2822) TimeLimitingCollector starts thread
in static {} with no way to stop them
Posted by "Simon Willnauer (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-2822?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Simon Willnauer updated LUCENE-2822:
------------------------------------
Attachment: LUCENE-2822.patch
I added a changes entry and opened up setting a baseline for the collector manually. Now by default we set the baseline once the first reader is set instead of during construction time. I think we are close here?!
> TimeLimitingCollector starts thread in static {} with no way to stop them
> -------------------------------------------------------------------------
>
> Key: LUCENE-2822
> URL: https://issues.apache.org/jira/browse/LUCENE-2822
> Project: Lucene - Java
> Issue Type: Bug
> Reporter: Robert Muir
> Fix For: 3.5, 4.0
>
> Attachments: LUCENE-2822.patch, LUCENE-2822.patch, LUCENE-2822.patch
>
>
> See the comment in LuceneTestCase.
> If you even do Class.forName("TimeLimitingCollector") it starts up a thread in a static method, and there isn't a way to kill it.
> This is broken.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] Commented: (LUCENE-2822) TimeLimitingCollector starts thread
in static {} with no way to stop them
Posted by "Mark Harwood (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-2822?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13002141#comment-13002141 ]
Mark Harwood commented on LUCENE-2822:
--------------------------------------
bq. does not use a System.nanoTime()/System.currentTimeMillis(), so what is your comment about?
There's already a solution that was designed to avoid any overhead related to making either of these calls.
> TimeLimitingCollector starts thread in static {} with no way to stop them
> -------------------------------------------------------------------------
>
> Key: LUCENE-2822
> URL: https://issues.apache.org/jira/browse/LUCENE-2822
> Project: Lucene - Java
> Issue Type: Bug
> Reporter: Robert Muir
>
> See the comment in LuceneTestCase.
> If you even do Class.forName("TimeLimitingCollector") it starts up a thread in a static method, and there isn't a way to kill it.
> This is broken.
--
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] Commented: (LUCENE-2822) TimeLimitingCollector starts thread
in static {} with no way to stop them
Posted by "Robert Muir (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-2822?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13002002#comment-13002002 ]
Robert Muir commented on LUCENE-2822:
-------------------------------------
bq. I think we should stick with our private timer thread (and we should definitely make it stop-able).
And no private thread should start in the static initializer... its fine for all instances to share a single private timer thread but this should be lazy-loaded.
> TimeLimitingCollector starts thread in static {} with no way to stop them
> -------------------------------------------------------------------------
>
> Key: LUCENE-2822
> URL: https://issues.apache.org/jira/browse/LUCENE-2822
> Project: Lucene - Java
> Issue Type: Bug
> Reporter: Robert Muir
>
> See the comment in LuceneTestCase.
> If you even do Class.forName("TimeLimitingCollector") it starts up a thread in a static method, and there isn't a way to kill it.
> This is broken.
--
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] [Commented] (LUCENE-2822) TimeLimitingCollector starts
thread in static {} with no way to stop them
Posted by "Torsten Krah (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-2822?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13135749#comment-13135749 ]
Torsten Krah commented on LUCENE-2822:
--------------------------------------
Some workaround - until this one is resolved - to stop the thread (e.g. in a container shutdown callback) may be this one using reflection:
Thread t = (Thread) FieldUtils.readDeclaredStaticField(TimeLimitingCollector.class, "TIMER_THREAD", true);
if (t != null && t.isAlive()) {
t.interrupt();
t.join(1000);
}
Little bit ugly but theres no other way yet.
> TimeLimitingCollector starts thread in static {} with no way to stop them
> -------------------------------------------------------------------------
>
> Key: LUCENE-2822
> URL: https://issues.apache.org/jira/browse/LUCENE-2822
> Project: Lucene - Java
> Issue Type: Bug
> Reporter: Robert Muir
> Fix For: 3.5, 4.0
>
>
> See the comment in LuceneTestCase.
> If you even do Class.forName("TimeLimitingCollector") it starts up a thread in a static method, and there isn't a way to kill it.
> This is broken.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] Commented: (LUCENE-2822) TimeLimitingCollector starts thread
in static {} with no way to stop them
Posted by "Uwe Schindler (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-2822?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13001889#comment-13001889 ]
Uwe Schindler commented on LUCENE-2822:
---------------------------------------
{quote}
It's 2011.
We can use Java 1.5 now in core Lucene.
so why don't we just rip out the TimerThread and use System.nanoTime() ?
{quote}
One of our large customers presenters used System.currentTimeMillies() in their own TLC collector implementation, which led to a immense slowdown as then on every hit you get a quite expensive system call to the kernel. This would not be more efficient with nanoTime, as its still a system call (at least on lot's of JVMs like on Windows).
So the thread is much more effective (its only reading a volatile field), so we need some shutdown. Another idea is to change this collector to not call nanoTime or currentTimeMillies on every hit (as hit collection should be fast), to maybe do this only every 1000 hit. The granularity could be a ctor param. This saves the thread and its still controlable how exact the measurement should be. A default of 1000 or maybe 10000 should be fine. Of course on the first collect hit (modulo 0) it should already check the timeout (because lots of queries do most work before the collection of hits).
> TimeLimitingCollector starts thread in static {} with no way to stop them
> -------------------------------------------------------------------------
>
> Key: LUCENE-2822
> URL: https://issues.apache.org/jira/browse/LUCENE-2822
> Project: Lucene - Java
> Issue Type: Bug
> Reporter: Robert Muir
>
> See the comment in LuceneTestCase.
> If you even do Class.forName("TimeLimitingCollector") it starts up a thread in a static method, and there isn't a way to kill it.
> This is broken.
--
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] Commented: (LUCENE-2822) TimeLimitingCollector starts thread
in static {} with no way to stop them
Posted by "Hoss Man (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-2822?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13001883#comment-13001883 ]
Hoss Man commented on LUCENE-2822:
----------------------------------
Having read through all of the comments in LUCENE-997 I fail to see why any Threads are needed in TimeLimitedCollector at all -- repeatedly folks mention that the use of a Timer thread is purely because System.currentTimeMillis isn't reliable enough and/or not efficient enough, but if we could use Java 1.5, System.nanoTime would be exactly what we need.
It's 2011.
We can use Java 1.5 now in core Lucene.
so why don't we just rip out the TimerThread and use System.nanoTime() ?
> TimeLimitingCollector starts thread in static {} with no way to stop them
> -------------------------------------------------------------------------
>
> Key: LUCENE-2822
> URL: https://issues.apache.org/jira/browse/LUCENE-2822
> Project: Lucene - Java
> Issue Type: Bug
> Reporter: Robert Muir
>
> See the comment in LuceneTestCase.
> If you even do Class.forName("TimeLimitingCollector") it starts up a thread in a static method, and there isn't a way to kill it.
> This is broken.
--
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] [Commented] (LUCENE-2822) TimeLimitingCollector starts
thread in static {} with no way to stop them
Posted by "Simon Willnauer (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-2822?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13148680#comment-13148680 ]
Simon Willnauer commented on LUCENE-2822:
-----------------------------------------
If nobody objects I am going to commit this later today and backport to 3.x - this will break bw compat for time limiting collector but I think we should do that in this particular case
> TimeLimitingCollector starts thread in static {} with no way to stop them
> -------------------------------------------------------------------------
>
> Key: LUCENE-2822
> URL: https://issues.apache.org/jira/browse/LUCENE-2822
> Project: Lucene - Java
> Issue Type: Bug
> Reporter: Robert Muir
> Fix For: 3.5, 4.0
>
> Attachments: LUCENE-2822.patch, LUCENE-2822.patch, LUCENE-2822.patch
>
>
> See the comment in LuceneTestCase.
> If you even do Class.forName("TimeLimitingCollector") it starts up a thread in a static method, and there isn't a way to kill it.
> This is broken.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] Commented: (LUCENE-2822) TimeLimitingCollector starts thread
in static {} with no way to stop them
Posted by "Uwe Schindler (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-2822?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13002071#comment-13002071 ]
Uwe Schindler commented on LUCENE-2822:
---------------------------------------
bq. I think we should stick with our private timer thread (and we should definitely make it stop-able).
I think this is still the best reason, as both System.nanoTime() and currentTimeMillies use system calls that are really expensive. But nanoTime() has no wallclock problems, thats true, but is still a no-go for every collected hit!
> TimeLimitingCollector starts thread in static {} with no way to stop them
> -------------------------------------------------------------------------
>
> Key: LUCENE-2822
> URL: https://issues.apache.org/jira/browse/LUCENE-2822
> Project: Lucene - Java
> Issue Type: Bug
> Reporter: Robert Muir
>
> See the comment in LuceneTestCase.
> If you even do Class.forName("TimeLimitingCollector") it starts up a thread in a static method, and there isn't a way to kill it.
> This is broken.
--
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] Commented: (LUCENE-2822) TimeLimitingCollector starts thread
in static {} with no way to stop them
Posted by "Uwe Schindler (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-2822?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13002129#comment-13002129 ]
Uwe Schindler commented on LUCENE-2822:
---------------------------------------
Mark: But LUCENE-1720 does not use a System.nanoTime()/System.currentTimeMillis(), so what is your comment about?
> TimeLimitingCollector starts thread in static {} with no way to stop them
> -------------------------------------------------------------------------
>
> Key: LUCENE-2822
> URL: https://issues.apache.org/jira/browse/LUCENE-2822
> Project: Lucene - Java
> Issue Type: Bug
> Reporter: Robert Muir
>
> See the comment in LuceneTestCase.
> If you even do Class.forName("TimeLimitingCollector") it starts up a thread in a static method, and there isn't a way to kill it.
> This is broken.
--
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] [Commented] (LUCENE-2822) TimeLimitingCollector starts
thread in static {} with no way to stop them
Posted by "Simon Willnauer (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-2822?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13147956#comment-13147956 ]
Simon Willnauer commented on LUCENE-2822:
-----------------------------------------
catching up here... I wonder if we can make this TLC simply use a o.a.l.utils.Counter so people can implement this on to of TLC. there could be a ThreadedCounter in TLC people can pull and use their own static variable?
> TimeLimitingCollector starts thread in static {} with no way to stop them
> -------------------------------------------------------------------------
>
> Key: LUCENE-2822
> URL: https://issues.apache.org/jira/browse/LUCENE-2822
> Project: Lucene - Java
> Issue Type: Bug
> Reporter: Robert Muir
> Fix For: 3.5, 4.0
>
>
> See the comment in LuceneTestCase.
> If you even do Class.forName("TimeLimitingCollector") it starts up a thread in a static method, and there isn't a way to kill it.
> This is broken.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] Commented: (LUCENE-2822) TimeLimitingCollector starts thread
in static {} with no way to stop them
Posted by "Robert Muir (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-2822?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13002260#comment-13002260 ]
Robert Muir commented on LUCENE-2822:
-------------------------------------
bq. I think this is still the best variant, as both System.nanoTime() and currentTimeMillies use system calls that are really expensive.
Sorry its too funny, playing with LUCENE-2948 I saw a big slowdown on windows that mike didn't see on linux... finally tracked it down to an uncommented nanoTime :)
> TimeLimitingCollector starts thread in static {} with no way to stop them
> -------------------------------------------------------------------------
>
> Key: LUCENE-2822
> URL: https://issues.apache.org/jira/browse/LUCENE-2822
> Project: Lucene - Java
> Issue Type: Bug
> Reporter: Robert Muir
>
> See the comment in LuceneTestCase.
> If you even do Class.forName("TimeLimitingCollector") it starts up a thread in a static method, and there isn't a way to kill it.
> This is broken.
--
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] Commented: (LUCENE-2822) TimeLimitingCollector starts thread
in static {} with no way to stop them
Posted by "Shai Erera (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-2822?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12988089#action_12988089 ]
Shai Erera commented on LUCENE-2822:
------------------------------------
For reference, TimeLimitedCollector was added in LUCENE-997, where TimerThread's functionality is discussed.
When we resolve this one, I think SOLR-1735 can be resolved as well.
>From LUCENE-997:
{noformat}
TimerThread provides a pseudo-clock service to all searching threads, so that they can count elapsed time with less overhead than repeatedly calling System.currentTimeMillis. A single thread should be created to be used for all searches.
{noformat}
The opinions about the effectiveness of TimerThread went both ways. For example, in LUCENE-1720, the timeout mechanism works differently, and I wonder if we can apply the same logic here, or something very similar.
Instead of having a static TimerThread, we can have start()/finish() API on TimeLimitingCollector(). start() will start a Thread that will sleep in nanosecs granularity until timeout exceeds. When it exceeds, it raises a flag. In collect() (and in setNextReader/Scorer too, to increase accuracy) all we do is check the flag and if it's on, we fail, just like collect() is impled today. App can call finish() that will signal the thread to stop.
I think it's simple and shouldn't reduce accuracy of current impl (maybe only improve it). Assuming apps don't set timeout to 20 msec, Thread.sleep w/ nanosec granularity should be very accurate. Also, today timeout is counted from the moment you create TLC, but the app may do few more things before IndexSearcher.search() is actually called. start/finish will allow the app to time the search() method (or whatever it wants).
Also, TimerThread was intended to serve all search requests, hence why it is static, but TLC itself cannot be shared across searches at all ...
This will solve this issue, as well as simplify TLC IMO.
> TimeLimitingCollector starts thread in static {} with no way to stop them
> -------------------------------------------------------------------------
>
> Key: LUCENE-2822
> URL: https://issues.apache.org/jira/browse/LUCENE-2822
> Project: Lucene - Java
> Issue Type: Bug
> Reporter: Robert Muir
>
> See the comment in LuceneTestCase.
> If you even do Class.forName("TimeLimitingCollector") it starts up a thread in a static method, and there isn't a way to kill it.
> This is broken.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] Commented: (LUCENE-2822) TimeLimitingCollector starts thread
in static {} with no way to stop them
Posted by "Shai Erera (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-2822?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13001896#comment-13001896 ]
Shai Erera commented on LUCENE-2822:
------------------------------------
I don't think we should check for time every N hits -- finding the next hit can take some time, and currently TimeLimitingCollector is not accurate b/c of that. I.e., it guarantees that when the time threshold has elapsed, it will stop the query processing, but it doesn't guarantee how accurate it is.
The TimerThread was indeed included because a system call on every hit was too expensive. I think that my proposal above, introducing start/finish API on TLC will solve the thread kept alive issue, won't affect performance, and keep TLC's accuracy as it is today (not perfect, but better than 'check every N hits').
> TimeLimitingCollector starts thread in static {} with no way to stop them
> -------------------------------------------------------------------------
>
> Key: LUCENE-2822
> URL: https://issues.apache.org/jira/browse/LUCENE-2822
> Project: Lucene - Java
> Issue Type: Bug
> Reporter: Robert Muir
>
> See the comment in LuceneTestCase.
> If you even do Class.forName("TimeLimitingCollector") it starts up a thread in a static method, and there isn't a way to kill it.
> This is broken.
--
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] Issue Comment Edited: (LUCENE-2822) TimeLimitingCollector
starts thread in static {} with no way to stop them
Posted by "Uwe Schindler (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-2822?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13001889#comment-13001889 ]
Uwe Schindler edited comment on LUCENE-2822 at 3/3/11 6:35 AM:
---------------------------------------------------------------
{quote}
It's 2011.
We can use Java 1.5 now in core Lucene.
so why don't we just rip out the TimerThread and use System.nanoTime() ?
{quote}
One of our large customers used System.currentTimeMillies() in their own TLC collector implementation, which led to a immense slowdown as then on every hit you get a quite expensive system call to the kernel. This would not be more efficient with nanoTime, as its still a system call (at least on lot's of JVMs like on Windows).
So the thread is much more effective (its only reading a volatile field), so we need some shutdown. Another idea is to change this collector to not call nanoTime or currentTimeMillies on every hit (as hit collection should be fast), to maybe do this only every 1000 hit. The granularity could be a ctor param. This saves the thread and its still controlable how exact the measurement should be. A default of 1000 or maybe 10000 should be fine. Of course on the first collect hit (modulo 0) it should already check the timeout (because lots of queries do most work before the collection of hits).
was (Author: thetaphi):
{quote}
It's 2011.
We can use Java 1.5 now in core Lucene.
so why don't we just rip out the TimerThread and use System.nanoTime() ?
{quote}
One of our large customers presenters used System.currentTimeMillies() in their own TLC collector implementation, which led to a immense slowdown as then on every hit you get a quite expensive system call to the kernel. This would not be more efficient with nanoTime, as its still a system call (at least on lot's of JVMs like on Windows).
So the thread is much more effective (its only reading a volatile field), so we need some shutdown. Another idea is to change this collector to not call nanoTime or currentTimeMillies on every hit (as hit collection should be fast), to maybe do this only every 1000 hit. The granularity could be a ctor param. This saves the thread and its still controlable how exact the measurement should be. A default of 1000 or maybe 10000 should be fine. Of course on the first collect hit (modulo 0) it should already check the timeout (because lots of queries do most work before the collection of hits).
> TimeLimitingCollector starts thread in static {} with no way to stop them
> -------------------------------------------------------------------------
>
> Key: LUCENE-2822
> URL: https://issues.apache.org/jira/browse/LUCENE-2822
> Project: Lucene - Java
> Issue Type: Bug
> Reporter: Robert Muir
>
> See the comment in LuceneTestCase.
> If you even do Class.forName("TimeLimitingCollector") it starts up a thread in a static method, and there isn't a way to kill it.
> This is broken.
--
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] Commented: (LUCENE-2822) TimeLimitingCollector starts thread
in static {} with no way to stop them
Posted by "Mark Harwood (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-2822?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13002115#comment-13002115 ]
Mark Harwood commented on LUCENE-2822:
--------------------------------------
FYI - I visited a site today using Lucene 1720 live on a large index (>2 billion docs, sharded with 5 minute update intervals). They haven't noticed any significant degrading of search performance as a result of using this approach.
> TimeLimitingCollector starts thread in static {} with no way to stop them
> -------------------------------------------------------------------------
>
> Key: LUCENE-2822
> URL: https://issues.apache.org/jira/browse/LUCENE-2822
> Project: Lucene - Java
> Issue Type: Bug
> Reporter: Robert Muir
>
> See the comment in LuceneTestCase.
> If you even do Class.forName("TimeLimitingCollector") it starts up a thread in a static method, and there isn't a way to kill it.
> This is broken.
--
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] Commented: (LUCENE-2822) TimeLimitingCollector starts thread
in static {} with no way to stop them
Posted by "Michael McCandless (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-2822?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13001967#comment-13001967 ]
Michael McCandless commented on LUCENE-2822:
--------------------------------------------
I think we should stick with our private timer thread (and we should definitely make it stop-able).
I've seen too many problems associated with relying on the system's time for "important" things like timing out queries, eg when daylight savings time strikes, or the clock is being "aggressively corrected", and suddenly a bunch of queries are truncated. In theory System.nanoTime should be immune to this (it's the system's timer and not any notion of "wall clock time"), but in practice, I don't think we should risk it.
> TimeLimitingCollector starts thread in static {} with no way to stop them
> -------------------------------------------------------------------------
>
> Key: LUCENE-2822
> URL: https://issues.apache.org/jira/browse/LUCENE-2822
> Project: Lucene - Java
> Issue Type: Bug
> Reporter: Robert Muir
>
> See the comment in LuceneTestCase.
> If you even do Class.forName("TimeLimitingCollector") it starts up a thread in a static method, and there isn't a way to kill it.
> This is broken.
--
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] [Updated] (LUCENE-2822) TimeLimitingCollector starts thread
in static {} with no way to stop them
Posted by "Simon Willnauer (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-2822?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Simon Willnauer updated LUCENE-2822:
------------------------------------
Attachment: LUCENE-2822.patch
next iteration
> TimeLimitingCollector starts thread in static {} with no way to stop them
> -------------------------------------------------------------------------
>
> Key: LUCENE-2822
> URL: https://issues.apache.org/jira/browse/LUCENE-2822
> Project: Lucene - Java
> Issue Type: Bug
> Reporter: Robert Muir
> Fix For: 3.5, 4.0
>
> Attachments: LUCENE-2822.patch, LUCENE-2822.patch
>
>
> See the comment in LuceneTestCase.
> If you even do Class.forName("TimeLimitingCollector") it starts up a thread in a static method, and there isn't a way to kill it.
> This is broken.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] Issue Comment Edited: (LUCENE-2822) TimeLimitingCollector
starts thread in static {} with no way to stop them
Posted by "Uwe Schindler (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-2822?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13002071#comment-13002071 ]
Uwe Schindler edited comment on LUCENE-2822 at 3/3/11 5:01 PM:
---------------------------------------------------------------
bq. I think we should stick with our private timer thread (and we should definitely make it stop-able).
I think this is still the best variant, as both System.nanoTime() and currentTimeMillies use system calls that are really expensive. nanoTime() has no wallclock problems, thats true, but is still a no-go for every collected hit!
was (Author: thetaphi):
bq. I think we should stick with our private timer thread (and we should definitely make it stop-able).
I think this is still the best reason, as both System.nanoTime() and currentTimeMillies use system calls that are really expensive. But nanoTime() has no wallclock problems, thats true, but is still a no-go for every collected hit!
> TimeLimitingCollector starts thread in static {} with no way to stop them
> -------------------------------------------------------------------------
>
> Key: LUCENE-2822
> URL: https://issues.apache.org/jira/browse/LUCENE-2822
> Project: Lucene - Java
> Issue Type: Bug
> Reporter: Robert Muir
>
> See the comment in LuceneTestCase.
> If you even do Class.forName("TimeLimitingCollector") it starts up a thread in a static method, and there isn't a way to kill it.
> This is broken.
--
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] [Assigned] (LUCENE-2822) TimeLimitingCollector starts thread
in static {} with no way to stop them
Posted by "Simon Willnauer (Assigned) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-2822?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Simon Willnauer reassigned LUCENE-2822:
---------------------------------------
Assignee: Simon Willnauer
> TimeLimitingCollector starts thread in static {} with no way to stop them
> -------------------------------------------------------------------------
>
> Key: LUCENE-2822
> URL: https://issues.apache.org/jira/browse/LUCENE-2822
> Project: Lucene - Java
> Issue Type: Bug
> Reporter: Robert Muir
> Assignee: Simon Willnauer
> Fix For: 3.5, 4.0
>
> Attachments: LUCENE-2822.patch, LUCENE-2822.patch, LUCENE-2822.patch, LUCENE-2822.patch
>
>
> See the comment in LuceneTestCase.
> If you even do Class.forName("TimeLimitingCollector") it starts up a thread in a static method, and there isn't a way to kill it.
> This is broken.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] [Updated] (LUCENE-2822) TimeLimitingCollector starts thread
in static {} with no way to stop them
Posted by "Simon Willnauer (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-2822?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Simon Willnauer updated LUCENE-2822:
------------------------------------
Attachment: LUCENE-2822.patch
here is a first version of what I have in mind. I'd like completely divorce the thread from the collector eventually and let the application create the thread. Solr should take care of this stuff itself. For now I used a little workaround for this.
> TimeLimitingCollector starts thread in static {} with no way to stop them
> -------------------------------------------------------------------------
>
> Key: LUCENE-2822
> URL: https://issues.apache.org/jira/browse/LUCENE-2822
> Project: Lucene - Java
> Issue Type: Bug
> Reporter: Robert Muir
> Fix For: 3.5, 4.0
>
> Attachments: LUCENE-2822.patch
>
>
> See the comment in LuceneTestCase.
> If you even do Class.forName("TimeLimitingCollector") it starts up a thread in a static method, and there isn't a way to kill it.
> This is broken.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] [Updated] (LUCENE-2822) TimeLimitingCollector starts thread
in static {} with no way to stop them
Posted by "Michael McCandless (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-2822?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Michael McCandless updated LUCENE-2822:
---------------------------------------
Fix Version/s: (was: 3.4)
3.5
> TimeLimitingCollector starts thread in static {} with no way to stop them
> -------------------------------------------------------------------------
>
> Key: LUCENE-2822
> URL: https://issues.apache.org/jira/browse/LUCENE-2822
> Project: Lucene - Java
> Issue Type: Bug
> Reporter: Robert Muir
> Fix For: 3.5, 4.0
>
>
> See the comment in LuceneTestCase.
> If you even do Class.forName("TimeLimitingCollector") it starts up a thread in a static method, and there isn't a way to kill it.
> This is broken.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] [Resolved] (LUCENE-2822) TimeLimitingCollector starts thread
in static {} with no way to stop them
Posted by "Simon Willnauer (Resolved) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-2822?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Simon Willnauer resolved LUCENE-2822.
-------------------------------------
Resolution: Fixed
Lucene Fields: New,Patch Available (was: New)
committed to trunk & backported to 3.x
> TimeLimitingCollector starts thread in static {} with no way to stop them
> -------------------------------------------------------------------------
>
> Key: LUCENE-2822
> URL: https://issues.apache.org/jira/browse/LUCENE-2822
> Project: Lucene - Java
> Issue Type: Bug
> Reporter: Robert Muir
> Assignee: Simon Willnauer
> Fix For: 3.5, 4.0
>
> Attachments: LUCENE-2822.patch, LUCENE-2822.patch, LUCENE-2822.patch, LUCENE-2822.patch
>
>
> See the comment in LuceneTestCase.
> If you even do Class.forName("TimeLimitingCollector") it starts up a thread in a static method, and there isn't a way to kill it.
> This is broken.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] [Updated] (LUCENE-2822) TimeLimitingCollector starts thread
in static {} with no way to stop them
Posted by "Simon Willnauer (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-2822?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Simon Willnauer updated LUCENE-2822:
------------------------------------
Attachment: LUCENE-2822.patch
update patch to trunk
> TimeLimitingCollector starts thread in static {} with no way to stop them
> -------------------------------------------------------------------------
>
> Key: LUCENE-2822
> URL: https://issues.apache.org/jira/browse/LUCENE-2822
> Project: Lucene - Java
> Issue Type: Bug
> Reporter: Robert Muir
> Fix For: 3.5, 4.0
>
> Attachments: LUCENE-2822.patch, LUCENE-2822.patch, LUCENE-2822.patch, LUCENE-2822.patch
>
>
> See the comment in LuceneTestCase.
> If you even do Class.forName("TimeLimitingCollector") it starts up a thread in a static method, and there isn't a way to kill it.
> This is broken.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] Updated: (LUCENE-2822) TimeLimitingCollector starts thread
in static {} with no way to stop them
Posted by "Michael McCandless (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-2822?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Michael McCandless updated LUCENE-2822:
---------------------------------------
Fix Version/s: 4.0
3.2
> TimeLimitingCollector starts thread in static {} with no way to stop them
> -------------------------------------------------------------------------
>
> Key: LUCENE-2822
> URL: https://issues.apache.org/jira/browse/LUCENE-2822
> Project: Lucene - Java
> Issue Type: Bug
> Reporter: Robert Muir
> Fix For: 3.2, 4.0
>
>
> See the comment in LuceneTestCase.
> If you even do Class.forName("TimeLimitingCollector") it starts up a thread in a static method, and there isn't a way to kill it.
> This is broken.
--
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org