You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucene.apache.org by "Joey Surls (JIRA)" <ji...@apache.org> on 2009/11/06 23:08:32 UTC
[jira] Created: (LUCENE-2041) Complete parallelizaton of
ParallelMultiSearcher
Complete parallelizaton of ParallelMultiSearcher
------------------------------------------------
Key: LUCENE-2041
URL: https://issues.apache.org/jira/browse/LUCENE-2041
Project: Lucene - Java
Issue Type: Improvement
Components: Search
Affects Versions: 2.9
Reporter: Joey Surls
Priority: Minor
Fix For: 3.0
ParallelMultiSearcher is parallel only for the method signatures of 'search'.
Part of a query process calls the method docFreq(). There was a TODO comment to parallelize this. Parallelizing this method actually increases the performance of a query on multiple indexes, especially remotely.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Updated: (LUCENE-2041) Complete parallelizaton of
ParallelMultiSearcher
Posted by "Simon Willnauer (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-2041?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Simon Willnauer updated LUCENE-2041:
------------------------------------
Attachment: LUCENE_2041.patch
I drove this a bit further and refactored the whole code to use the java 1.5 concurrent utils. I seem to be somewhat cleaner with executors / callable and a little refactoring.
simon
> Complete parallelizaton of ParallelMultiSearcher
> ------------------------------------------------
>
> Key: LUCENE-2041
> URL: https://issues.apache.org/jira/browse/LUCENE-2041
> Project: Lucene - Java
> Issue Type: Improvement
> Components: Search
> Affects Versions: 2.9
> Reporter: Joey Surls
> Assignee: Simon Willnauer
> Priority: Minor
> Fix For: 3.1
>
> Attachments: LUCENE-2041.patch, LUCENE_2041.patch
>
> Original Estimate: 1h
> Remaining Estimate: 1h
>
> ParallelMultiSearcher is parallel only for the method signatures of 'search'.
> Part of a query process calls the method docFreq(). There was a TODO comment to parallelize this. Parallelizing this method actually increases the performance of a query on multiple indexes, especially remotely.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Updated: (LUCENE-2041) Complete parallelizaton of
ParallelMultiSearcher
Posted by "Simon Willnauer (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-2041?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Simon Willnauer updated LUCENE-2041:
------------------------------------
Fix Version/s: (was: 3.0)
3.1
Assignee: Simon Willnauer
moving to 3.1 for now. we will see how this turns out
> Complete parallelizaton of ParallelMultiSearcher
> ------------------------------------------------
>
> Key: LUCENE-2041
> URL: https://issues.apache.org/jira/browse/LUCENE-2041
> Project: Lucene - Java
> Issue Type: Improvement
> Components: Search
> Affects Versions: 2.9
> Reporter: Joey Surls
> Assignee: Simon Willnauer
> Priority: Minor
> Fix For: 3.1
>
> Attachments: LUCENE-2041.patch
>
> Original Estimate: 1h
> Remaining Estimate: 1h
>
> ParallelMultiSearcher is parallel only for the method signatures of 'search'.
> Part of a query process calls the method docFreq(). There was a TODO comment to parallelize this. Parallelizing this method actually increases the performance of a query on multiple indexes, especially remotely.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Commented: (LUCENE-2041) Complete parallelizaton of
ParallelMultiSearcher
Posted by "Simon Willnauer (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-2041?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12774974#action_12774974 ]
Simon Willnauer commented on LUCENE-2041:
-----------------------------------------
thanks uwe!
> Complete parallelizaton of ParallelMultiSearcher
> ------------------------------------------------
>
> Key: LUCENE-2041
> URL: https://issues.apache.org/jira/browse/LUCENE-2041
> Project: Lucene - Java
> Issue Type: Improvement
> Components: Search
> Affects Versions: 2.9
> Reporter: Joey Surls
> Assignee: Uwe Schindler
> Priority: Minor
> Fix For: 3.1
>
> Attachments: LUCENE-2041.patch, LUCENE_2041.patch
>
> Original Estimate: 1h
> Remaining Estimate: 1h
>
> ParallelMultiSearcher is parallel only for the method signatures of 'search'.
> Part of a query process calls the method docFreq(). There was a TODO comment to parallelize this. Parallelizing this method actually increases the performance of a query on multiple indexes, especially remotely.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Commented: (LUCENE-2041) Complete parallelizaton of
ParallelMultiSearcher
Posted by "Uwe Schindler (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-2041?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12774983#action_12774983 ]
Uwe Schindler commented on LUCENE-2041:
---------------------------------------
I also changed some formatting, we require whitespace after control statements like for, while, catch,... and before {
> Complete parallelizaton of ParallelMultiSearcher
> ------------------------------------------------
>
> Key: LUCENE-2041
> URL: https://issues.apache.org/jira/browse/LUCENE-2041
> Project: Lucene - Java
> Issue Type: Improvement
> Components: Search
> Affects Versions: 2.9
> Reporter: Joey Surls
> Assignee: Uwe Schindler
> Priority: Minor
> Fix For: 3.1
>
> Attachments: LUCENE-2041.patch, LUCENE-2041.patch, LUCENE-2041.patch, LUCENE_2041.patch
>
> Original Estimate: 1h
> Remaining Estimate: 1h
>
> ParallelMultiSearcher is parallel only for the method signatures of 'search'.
> Part of a query process calls the method docFreq(). There was a TODO comment to parallelize this. Parallelizing this method actually increases the performance of a query on multiple indexes, especially remotely.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Commented: (LUCENE-2041) Complete parallelizaton of
ParallelMultiSearcher
Posted by "Simon Willnauer (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-2041?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12774492#action_12774492 ]
Simon Willnauer commented on LUCENE-2041:
-----------------------------------------
Hey Joey,
good to see you coming up with patches, nice that you catch up on that TODO. You added support for parallel DocFreq which is good! Did you think about using an interface from the new concurrent utils instead of subclassing thread? I would like to see people using the new features which make code slightly more readable too. You would be able to simply throw the exception in the Callable#call() method and retrieve it once it has terminated.
you could also use a Pooled Executor to run those threads so you don't have to spawn them each time you call docFreq on the multi searcher.
one more thing, your patch is hard to read as you formated the source code. please try to do not format it - that would make it way easier to read and see what has been added / changed.
> Complete parallelizaton of ParallelMultiSearcher
> ------------------------------------------------
>
> Key: LUCENE-2041
> URL: https://issues.apache.org/jira/browse/LUCENE-2041
> Project: Lucene - Java
> Issue Type: Improvement
> Components: Search
> Affects Versions: 2.9
> Reporter: Joey Surls
> Priority: Minor
> Fix For: 3.0
>
> Attachments: LUCENE-2041.patch
>
> Original Estimate: 1h
> Remaining Estimate: 1h
>
> ParallelMultiSearcher is parallel only for the method signatures of 'search'.
> Part of a query process calls the method docFreq(). There was a TODO comment to parallelize this. Parallelizing this method actually increases the performance of a query on multiple indexes, especially remotely.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Commented: (LUCENE-2041) Complete parallelizaton of
ParallelMultiSearcher
Posted by "Joey Surls (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-2041?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12774498#action_12774498 ]
Joey Surls commented on LUCENE-2041:
------------------------------------
I agree with you on the Pooled Executor and concurrent utils, I was actually trying to copy the semantics of the search methods in place so someone could quickly grasp the code.
Ya, I thought that's why the patch was full was from the formatted code. Sorry about that. I'll look for that in the future. I noticed the link in the wiki for the formatting under Helpful Resources doesn't work anymore.
http://wiki.apache.org/lucene-java/HowToContribute
> Complete parallelizaton of ParallelMultiSearcher
> ------------------------------------------------
>
> Key: LUCENE-2041
> URL: https://issues.apache.org/jira/browse/LUCENE-2041
> Project: Lucene - Java
> Issue Type: Improvement
> Components: Search
> Affects Versions: 2.9
> Reporter: Joey Surls
> Assignee: Simon Willnauer
> Priority: Minor
> Fix For: 3.1
>
> Attachments: LUCENE-2041.patch
>
> Original Estimate: 1h
> Remaining Estimate: 1h
>
> ParallelMultiSearcher is parallel only for the method signatures of 'search'.
> Part of a query process calls the method docFreq(). There was a TODO comment to parallelize this. Parallelizing this method actually increases the performance of a query on multiple indexes, especially remotely.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Updated: (LUCENE-2041) Complete parallelizaton of
ParallelMultiSearcher
Posted by "Joey Surls (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-2041?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Joey Surls updated LUCENE-2041:
-------------------------------
Attachment: LUCENE-2041.patch
> Complete parallelizaton of ParallelMultiSearcher
> ------------------------------------------------
>
> Key: LUCENE-2041
> URL: https://issues.apache.org/jira/browse/LUCENE-2041
> Project: Lucene - Java
> Issue Type: Improvement
> Components: Search
> Affects Versions: 2.9
> Reporter: Joey Surls
> Priority: Minor
> Fix For: 3.0
>
> Attachments: LUCENE-2041.patch
>
> Original Estimate: 1h
> Remaining Estimate: 1h
>
> ParallelMultiSearcher is parallel only for the method signatures of 'search'.
> Part of a query process calls the method docFreq(). There was a TODO comment to parallelize this. Parallelizing this method actually increases the performance of a query on multiple indexes, especially remotely.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Updated: (LUCENE-2041) Complete parallelizaton of
ParallelMultiSearcher
Posted by "Uwe Schindler (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-2041?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Uwe Schindler updated LUCENE-2041:
----------------------------------
Attachment: LUCENE-2041.patch
Advanced patch:
- Removed @SuppressWarnings by duplicating some code, but it is type-safe now
- Added support for maxScore in sorted mode
- Refactoring: made all internal classes static inner classes
- Fix thread safety: hq.setFields() synchronized
> Complete parallelizaton of ParallelMultiSearcher
> ------------------------------------------------
>
> Key: LUCENE-2041
> URL: https://issues.apache.org/jira/browse/LUCENE-2041
> Project: Lucene - Java
> Issue Type: Improvement
> Components: Search
> Affects Versions: 2.9
> Reporter: Joey Surls
> Assignee: Uwe Schindler
> Priority: Minor
> Fix For: 3.1
>
> Attachments: LUCENE-2041.patch, LUCENE-2041.patch, LUCENE-2041.patch, LUCENE-2041.patch, LUCENE_2041.patch
>
> Original Estimate: 1h
> Remaining Estimate: 1h
>
> ParallelMultiSearcher is parallel only for the method signatures of 'search'.
> Part of a query process calls the method docFreq(). There was a TODO comment to parallelize this. Parallelizing this method actually increases the performance of a query on multiple indexes, especially remotely.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Updated: (LUCENE-2041) Complete parallelizaton of
ParallelMultiSearcher
Posted by "Uwe Schindler (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-2041?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Uwe Schindler updated LUCENE-2041:
----------------------------------
Fix Version/s: (was: 3.1)
3.0
> Complete parallelizaton of ParallelMultiSearcher
> ------------------------------------------------
>
> Key: LUCENE-2041
> URL: https://issues.apache.org/jira/browse/LUCENE-2041
> Project: Lucene - Java
> Issue Type: Improvement
> Components: Search
> Affects Versions: 2.9
> Reporter: Joey Surls
> Assignee: Uwe Schindler
> Priority: Minor
> Fix For: 3.0
>
> Attachments: LUCENE-2041-refactor.patch, LUCENE-2041-refactor.patch, LUCENE-2041.patch, LUCENE-2041.patch, LUCENE-2041.patch, LUCENE-2041.patch, LUCENE-2041.patch, LUCENE_2041.patch
>
> Original Estimate: 1h
> Remaining Estimate: 1h
>
> ParallelMultiSearcher is parallel only for the method signatures of 'search'.
> Part of a query process calls the method docFreq(). There was a TODO comment to parallelize this. Parallelizing this method actually increases the performance of a query on multiple indexes, especially remotely.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Commented: (LUCENE-2041) Complete parallelizaton of
ParallelMultiSearcher
Posted by "Simon Willnauer (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-2041?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12775945#action_12775945 ]
Simon Willnauer commented on LUCENE-2041:
-----------------------------------------
+1 go for it - seems to be quite pretty compared to the current trunk version.
:)
> Complete parallelizaton of ParallelMultiSearcher
> ------------------------------------------------
>
> Key: LUCENE-2041
> URL: https://issues.apache.org/jira/browse/LUCENE-2041
> Project: Lucene - Java
> Issue Type: Improvement
> Components: Search
> Affects Versions: 2.9
> Reporter: Joey Surls
> Assignee: Uwe Schindler
> Priority: Minor
> Fix For: 3.0
>
> Attachments: LUCENE-2041-final.patch, LUCENE-2041-refactor.patch, LUCENE-2041-refactor.patch, LUCENE-2041.patch, LUCENE-2041.patch, LUCENE-2041.patch, LUCENE-2041.patch, LUCENE-2041.patch, LUCENE_2041.patch
>
> Original Estimate: 1h
> Remaining Estimate: 1h
>
> ParallelMultiSearcher is parallel only for the method signatures of 'search'.
> Part of a query process calls the method docFreq(). There was a TODO comment to parallelize this. Parallelizing this method actually increases the performance of a query on multiple indexes, especially remotely.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Updated: (LUCENE-2041) Complete parallelizaton of
ParallelMultiSearcher
Posted by "Uwe Schindler (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-2041?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Uwe Schindler updated LUCENE-2041:
----------------------------------
Attachment: LUCENE-2041.patch
Updated patch.
> Complete parallelizaton of ParallelMultiSearcher
> ------------------------------------------------
>
> Key: LUCENE-2041
> URL: https://issues.apache.org/jira/browse/LUCENE-2041
> Project: Lucene - Java
> Issue Type: Improvement
> Components: Search
> Affects Versions: 2.9
> Reporter: Joey Surls
> Assignee: Uwe Schindler
> Priority: Minor
> Fix For: 3.1
>
> Attachments: LUCENE-2041.patch, LUCENE-2041.patch, LUCENE_2041.patch
>
> Original Estimate: 1h
> Remaining Estimate: 1h
>
> ParallelMultiSearcher is parallel only for the method signatures of 'search'.
> Part of a query process calls the method docFreq(). There was a TODO comment to parallelize this. Parallelizing this method actually increases the performance of a query on multiple indexes, especially remotely.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Updated: (LUCENE-2041) Complete parallelizaton of
ParallelMultiSearcher
Posted by "Uwe Schindler (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-2041?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Uwe Schindler updated LUCENE-2041:
----------------------------------
Attachment: LUCENE-2041-refactor.patch
Here another possibility, removing more duplicated code: MultiSearcher contains now the merge Callables as static inner classes. These Callables are used directly without Executor in MultiSearcher, but with Executor in ParallelMultiSearcher.
> Complete parallelizaton of ParallelMultiSearcher
> ------------------------------------------------
>
> Key: LUCENE-2041
> URL: https://issues.apache.org/jira/browse/LUCENE-2041
> Project: Lucene - Java
> Issue Type: Improvement
> Components: Search
> Affects Versions: 2.9
> Reporter: Joey Surls
> Assignee: Uwe Schindler
> Priority: Minor
> Fix For: 3.1
>
> Attachments: LUCENE-2041-refactor.patch, LUCENE-2041.patch, LUCENE-2041.patch, LUCENE-2041.patch, LUCENE-2041.patch, LUCENE_2041.patch
>
> Original Estimate: 1h
> Remaining Estimate: 1h
>
> ParallelMultiSearcher is parallel only for the method signatures of 'search'.
> Part of a query process calls the method docFreq(). There was a TODO comment to parallelize this. Parallelizing this method actually increases the performance of a query on multiple indexes, especially remotely.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Commented: (LUCENE-2041) Complete parallelizaton of
ParallelMultiSearcher
Posted by "Uwe Schindler (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-2041?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12775927#action_12775927 ]
Uwe Schindler commented on LUCENE-2041:
---------------------------------------
Oh I didn't see your patch. But its not so different, I also did lots of code cleanup.
I also used ReentrantLocks. I added a new DummyLock to utils, that is a replacement for ReentrantLock, but does simply nothing. By this, the synchronized around the HitQueue in MultiSearcher can be ommitted.
> Complete parallelizaton of ParallelMultiSearcher
> ------------------------------------------------
>
> Key: LUCENE-2041
> URL: https://issues.apache.org/jira/browse/LUCENE-2041
> Project: Lucene - Java
> Issue Type: Improvement
> Components: Search
> Affects Versions: 2.9
> Reporter: Joey Surls
> Assignee: Uwe Schindler
> Priority: Minor
> Fix For: 3.0
>
> Attachments: LUCENE-2041-refactor.patch, LUCENE-2041-refactor.patch, LUCENE-2041.patch, LUCENE-2041.patch, LUCENE-2041.patch, LUCENE-2041.patch, LUCENE-2041.patch, LUCENE_2041.patch
>
> Original Estimate: 1h
> Remaining Estimate: 1h
>
> ParallelMultiSearcher is parallel only for the method signatures of 'search'.
> Part of a query process calls the method docFreq(). There was a TODO comment to parallelize this. Parallelizing this method actually increases the performance of a query on multiple indexes, especially remotely.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Commented: (LUCENE-2041) Complete parallelizaton of
ParallelMultiSearcher
Posted by "Uwe Schindler (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-2041?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12774501#action_12774501 ]
Uwe Schindler commented on LUCENE-2041:
---------------------------------------
bq. Ya, I thought that's why the patch was full was from the formatted code. Sorry about that. I'll look for that in the future. I noticed the link in the wiki for the formatting under Helpful Resources doesn't work anymore.
Fixed that and uploaded a new Eclipse Galieo style also containing our way for Java 5 Generics.
> Complete parallelizaton of ParallelMultiSearcher
> ------------------------------------------------
>
> Key: LUCENE-2041
> URL: https://issues.apache.org/jira/browse/LUCENE-2041
> Project: Lucene - Java
> Issue Type: Improvement
> Components: Search
> Affects Versions: 2.9
> Reporter: Joey Surls
> Assignee: Simon Willnauer
> Priority: Minor
> Fix For: 3.1
>
> Attachments: LUCENE-2041.patch
>
> Original Estimate: 1h
> Remaining Estimate: 1h
>
> ParallelMultiSearcher is parallel only for the method signatures of 'search'.
> Part of a query process calls the method docFreq(). There was a TODO comment to parallelize this. Parallelizing this method actually increases the performance of a query on multiple indexes, especially remotely.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Updated: (LUCENE-2041) Complete parallelizaton of
ParallelMultiSearcher
Posted by "Uwe Schindler (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-2041?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Uwe Schindler updated LUCENE-2041:
----------------------------------
Attachment: LUCENE-2041-final.patch
Merged our patches somehow, but left HitQueue generic param (I like this more, because it conforms to the sort inner class). I think this is final now.
Ready to commit, all tests are running.
> Complete parallelizaton of ParallelMultiSearcher
> ------------------------------------------------
>
> Key: LUCENE-2041
> URL: https://issues.apache.org/jira/browse/LUCENE-2041
> Project: Lucene - Java
> Issue Type: Improvement
> Components: Search
> Affects Versions: 2.9
> Reporter: Joey Surls
> Assignee: Uwe Schindler
> Priority: Minor
> Fix For: 3.0
>
> Attachments: LUCENE-2041-final.patch, LUCENE-2041-refactor.patch, LUCENE-2041-refactor.patch, LUCENE-2041.patch, LUCENE-2041.patch, LUCENE-2041.patch, LUCENE-2041.patch, LUCENE-2041.patch, LUCENE_2041.patch
>
> Original Estimate: 1h
> Remaining Estimate: 1h
>
> ParallelMultiSearcher is parallel only for the method signatures of 'search'.
> Part of a query process calls the method docFreq(). There was a TODO comment to parallelize this. Parallelizing this method actually increases the performance of a query on multiple indexes, especially remotely.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Updated: (LUCENE-2041) Complete parallelizaton of
ParallelMultiSearcher
Posted by "Simon Willnauer (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-2041?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Simon Willnauer updated LUCENE-2041:
------------------------------------
Attachment: LUCENE-2041.patch
Added some more final keywords and generalized HitQueue to PriorityQueue<ScoreDoc> in search (no sort)
I guess we are set with this patch, good team work Uwe! Lets get this out for heavy committing! :)
I get the impression that we should think about moving stuff like the Function interface out to utils. This could be useful in many situations though. Thoughts?
> Complete parallelizaton of ParallelMultiSearcher
> ------------------------------------------------
>
> Key: LUCENE-2041
> URL: https://issues.apache.org/jira/browse/LUCENE-2041
> Project: Lucene - Java
> Issue Type: Improvement
> Components: Search
> Affects Versions: 2.9
> Reporter: Joey Surls
> Assignee: Uwe Schindler
> Priority: Minor
> Fix For: 3.1
>
> Attachments: LUCENE-2041-refactor.patch, LUCENE-2041.patch, LUCENE-2041.patch, LUCENE-2041.patch, LUCENE-2041.patch, LUCENE-2041.patch, LUCENE_2041.patch
>
> Original Estimate: 1h
> Remaining Estimate: 1h
>
> ParallelMultiSearcher is parallel only for the method signatures of 'search'.
> Part of a query process calls the method docFreq(). There was a TODO comment to parallelize this. Parallelizing this method actually increases the performance of a query on multiple indexes, especially remotely.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Commented: (LUCENE-2041) Complete parallelizaton of
ParallelMultiSearcher
Posted by "Israel Tsadok (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-2041?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12786932#action_12786932 ]
Israel Tsadok commented on LUCENE-2041:
---------------------------------------
This has turned into a complete refactoring of the class, but I'd like to comment that the original intent of this patch was a bit misguided: docFreq() is not called during normal use of ParallelMultiSearcher. What does get called is docFreqs() on the sub-searchers. This is done in MultiSearcher.createWeight(), and is done sequentially.
I created issue LUCENE-2128 to suggest a solution to this.
> Complete parallelizaton of ParallelMultiSearcher
> ------------------------------------------------
>
> Key: LUCENE-2041
> URL: https://issues.apache.org/jira/browse/LUCENE-2041
> Project: Lucene - Java
> Issue Type: Improvement
> Components: Search
> Affects Versions: 2.9
> Reporter: Joey Surls
> Assignee: Uwe Schindler
> Priority: Minor
> Fix For: 3.0
>
> Attachments: LUCENE-2041-final.patch, LUCENE-2041-refactor.patch, LUCENE-2041-refactor.patch, LUCENE-2041.patch, LUCENE-2041.patch, LUCENE-2041.patch, LUCENE-2041.patch, LUCENE-2041.patch, LUCENE_2041.patch
>
> Original Estimate: 1h
> Remaining Estimate: 1h
>
> ParallelMultiSearcher is parallel only for the method signatures of 'search'.
> Part of a query process calls the method docFreq(). There was a TODO comment to parallelize this. Parallelizing this method actually increases the performance of a query on multiple indexes, especially remotely.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Assigned: (LUCENE-2041) Complete parallelizaton of
ParallelMultiSearcher
Posted by "Uwe Schindler (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-2041?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Uwe Schindler reassigned LUCENE-2041:
-------------------------------------
Assignee: Uwe Schindler (was: Simon Willnauer)
I take it as communicated.
> Complete parallelizaton of ParallelMultiSearcher
> ------------------------------------------------
>
> Key: LUCENE-2041
> URL: https://issues.apache.org/jira/browse/LUCENE-2041
> Project: Lucene - Java
> Issue Type: Improvement
> Components: Search
> Affects Versions: 2.9
> Reporter: Joey Surls
> Assignee: Uwe Schindler
> Priority: Minor
> Fix For: 3.1
>
> Attachments: LUCENE-2041.patch, LUCENE_2041.patch
>
> Original Estimate: 1h
> Remaining Estimate: 1h
>
> ParallelMultiSearcher is parallel only for the method signatures of 'search'.
> Part of a query process calls the method docFreq(). There was a TODO comment to parallelize this. Parallelizing this method actually increases the performance of a query on multiple indexes, especially remotely.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Resolved: (LUCENE-2041) Complete parallelizaton of
ParallelMultiSearcher
Posted by "Uwe Schindler (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-2041?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Uwe Schindler resolved LUCENE-2041.
-----------------------------------
Resolution: Fixed
(Heavy) committed revision: 834550
Thanks Joey & Simon
> Complete parallelizaton of ParallelMultiSearcher
> ------------------------------------------------
>
> Key: LUCENE-2041
> URL: https://issues.apache.org/jira/browse/LUCENE-2041
> Project: Lucene - Java
> Issue Type: Improvement
> Components: Search
> Affects Versions: 2.9
> Reporter: Joey Surls
> Assignee: Uwe Schindler
> Priority: Minor
> Fix For: 3.0
>
> Attachments: LUCENE-2041-final.patch, LUCENE-2041-refactor.patch, LUCENE-2041-refactor.patch, LUCENE-2041.patch, LUCENE-2041.patch, LUCENE-2041.patch, LUCENE-2041.patch, LUCENE-2041.patch, LUCENE_2041.patch
>
> Original Estimate: 1h
> Remaining Estimate: 1h
>
> ParallelMultiSearcher is parallel only for the method signatures of 'search'.
> Part of a query process calls the method docFreq(). There was a TODO comment to parallelize this. Parallelizing this method actually increases the performance of a query on multiple indexes, especially remotely.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Updated: (LUCENE-2041) Complete parallelizaton of
ParallelMultiSearcher
Posted by "Uwe Schindler (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-2041?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Uwe Schindler updated LUCENE-2041:
----------------------------------
Attachment: LUCENE-2041-refactor.patch
Here my latest patch, will commit soon.
Uwe
> Complete parallelizaton of ParallelMultiSearcher
> ------------------------------------------------
>
> Key: LUCENE-2041
> URL: https://issues.apache.org/jira/browse/LUCENE-2041
> Project: Lucene - Java
> Issue Type: Improvement
> Components: Search
> Affects Versions: 2.9
> Reporter: Joey Surls
> Assignee: Uwe Schindler
> Priority: Minor
> Fix For: 3.1
>
> Attachments: LUCENE-2041-refactor.patch, LUCENE-2041-refactor.patch, LUCENE-2041.patch, LUCENE-2041.patch, LUCENE-2041.patch, LUCENE-2041.patch, LUCENE-2041.patch, LUCENE_2041.patch
>
> Original Estimate: 1h
> Remaining Estimate: 1h
>
> ParallelMultiSearcher is parallel only for the method signatures of 'search'.
> Part of a query process calls the method docFreq(). There was a TODO comment to parallelize this. Parallelizing this method actually increases the performance of a query on multiple indexes, especially remotely.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Commented: (LUCENE-2041) Complete parallelizaton of
ParallelMultiSearcher
Posted by "Simon Willnauer (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-2041?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12774982#action_12774982 ]
Simon Willnauer commented on LUCENE-2041:
-----------------------------------------
hehe - my patch was quicker but you comment appears first :)
> Complete parallelizaton of ParallelMultiSearcher
> ------------------------------------------------
>
> Key: LUCENE-2041
> URL: https://issues.apache.org/jira/browse/LUCENE-2041
> Project: Lucene - Java
> Issue Type: Improvement
> Components: Search
> Affects Versions: 2.9
> Reporter: Joey Surls
> Assignee: Uwe Schindler
> Priority: Minor
> Fix For: 3.1
>
> Attachments: LUCENE-2041.patch, LUCENE-2041.patch, LUCENE-2041.patch, LUCENE_2041.patch
>
> Original Estimate: 1h
> Remaining Estimate: 1h
>
> ParallelMultiSearcher is parallel only for the method signatures of 'search'.
> Part of a query process calls the method docFreq(). There was a TODO comment to parallelize this. Parallelizing this method actually increases the performance of a query on multiple indexes, especially remotely.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Updated: (LUCENE-2041) Complete parallelizaton of
ParallelMultiSearcher
Posted by "Simon Willnauer (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-2041?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Simon Willnauer updated LUCENE-2041:
------------------------------------
Attachment: LUCENE-2041.patch
Fixed the Java5 issue.
Damned I should use java 5 on my notebook.
simon
> Complete parallelizaton of ParallelMultiSearcher
> ------------------------------------------------
>
> Key: LUCENE-2041
> URL: https://issues.apache.org/jira/browse/LUCENE-2041
> Project: Lucene - Java
> Issue Type: Improvement
> Components: Search
> Affects Versions: 2.9
> Reporter: Joey Surls
> Assignee: Uwe Schindler
> Priority: Minor
> Fix For: 3.1
>
> Attachments: LUCENE-2041.patch, LUCENE-2041.patch, LUCENE-2041.patch, LUCENE_2041.patch
>
> Original Estimate: 1h
> Remaining Estimate: 1h
>
> ParallelMultiSearcher is parallel only for the method signatures of 'search'.
> Part of a query process calls the method docFreq(). There was a TODO comment to parallelize this. Parallelizing this method actually increases the performance of a query on multiple indexes, especially remotely.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Commented: (LUCENE-2041) Complete parallelizaton of
ParallelMultiSearcher
Posted by "Uwe Schindler (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-2041?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12774976#action_12774976 ]
Uwe Schindler commented on LUCENE-2041:
---------------------------------------
String.isEmpty() is Java 6 only, changed this to length() == 0.
> Complete parallelizaton of ParallelMultiSearcher
> ------------------------------------------------
>
> Key: LUCENE-2041
> URL: https://issues.apache.org/jira/browse/LUCENE-2041
> Project: Lucene - Java
> Issue Type: Improvement
> Components: Search
> Affects Versions: 2.9
> Reporter: Joey Surls
> Assignee: Uwe Schindler
> Priority: Minor
> Fix For: 3.1
>
> Attachments: LUCENE-2041.patch, LUCENE_2041.patch
>
> Original Estimate: 1h
> Remaining Estimate: 1h
>
> ParallelMultiSearcher is parallel only for the method signatures of 'search'.
> Part of a query process calls the method docFreq(). There was a TODO comment to parallelize this. Parallelizing this method actually increases the performance of a query on multiple indexes, especially remotely.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Commented: (LUCENE-2041) Complete parallelizaton of
ParallelMultiSearcher
Posted by "Uwe Schindler (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-2041?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12774987#action_12774987 ]
Uwe Schindler commented on LUCENE-2041:
---------------------------------------
I will also readd support for MaxScore in the fielded case (it is still supported by Searchable.search(SortField))!!!
> Complete parallelizaton of ParallelMultiSearcher
> ------------------------------------------------
>
> Key: LUCENE-2041
> URL: https://issues.apache.org/jira/browse/LUCENE-2041
> Project: Lucene - Java
> Issue Type: Improvement
> Components: Search
> Affects Versions: 2.9
> Reporter: Joey Surls
> Assignee: Uwe Schindler
> Priority: Minor
> Fix For: 3.1
>
> Attachments: LUCENE-2041.patch, LUCENE-2041.patch, LUCENE-2041.patch, LUCENE_2041.patch
>
> Original Estimate: 1h
> Remaining Estimate: 1h
>
> ParallelMultiSearcher is parallel only for the method signatures of 'search'.
> Part of a query process calls the method docFreq(). There was a TODO comment to parallelize this. Parallelizing this method actually increases the performance of a query on multiple indexes, especially remotely.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org