You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@lucene.apache.org by "benwtrent (via GitHub)" <gi...@apache.org> on 2023/05/12 14:43:32 UTC
[GitHub] [lucene] benwtrent opened a new pull request, #12288: Backport: Concurrent rewrite for KnnVectorQuery (#12160)
benwtrent opened a new pull request, #12288:
URL: https://github.com/apache/lucene/pull/12288
Backport of #12160
---------------------------------------------------
### Description
Issue #11862
### Solution
`AbstractKnnVectorQuery` currently performs HNSW searches (one per-segment) iteratively
Since this is done in a single thread, we can make it concurrent by spinning off per-segment searches to different threads (and make use of available processors)
The actual search is performed in `Query#rewrite`, and support to allow concurrency there was added recently (#11838) by passing an `IndexSearcher` (which wraps an `IndexReader` and `Executor`)
Proposing to achieve this by `CompletableFuture`:
- If the `Executor` is not set, we can perform a blocking call to `CompletableFuture#completedFuture`
- Else we submit the task of per-segment search to the `Executor` using `CompletableFuture#supplyAsync`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org
[GitHub] [lucene] benwtrent merged pull request #12288: Backport: Concurrent rewrite for KnnVectorQuery (#12160)
Posted by "benwtrent (via GitHub)" <gi...@apache.org>.
benwtrent merged PR #12288:
URL: https://github.com/apache/lucene/pull/12288
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org
[GitHub] [lucene] benwtrent commented on pull request #12288: Backport: Concurrent rewrite for KnnVectorQuery (#12160)
Posted by "benwtrent (via GitHub)" <gi...@apache.org>.
benwtrent commented on PR #12288:
URL: https://github.com/apache/lucene/pull/12288#issuecomment-1545889373
//cc @zhaih && @kaivalnp
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org
[GitHub] [lucene] kaivalnp commented on pull request #12288: Backport: Concurrent rewrite for KnnVectorQuery (#12160)
Posted by "kaivalnp (via GitHub)" <gi...@apache.org>.
kaivalnp commented on PR #12288:
URL: https://github.com/apache/lucene/pull/12288#issuecomment-1546547247
Thank you for following up on this @benwtrent! Much appreciated!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org