You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucene.apache.org by GitBox <gi...@apache.org> on 2019/07/05 17:09:50 UTC

[GitHub] [lucene-solr] tokee commented on issue #754: LUCENE-8875: Introduce Optimized Collector For Large Number Of Hits

tokee commented on issue #754: LUCENE-8875: Introduce Optimized Collector For Large Number Of Hits
URL: https://github.com/apache/lucene-solr/pull/754#issuecomment-508818049
 
 
   I don't know if I have much to add to https://sbdevel.wordpress.com/2015/10/05/speeding-up-core-search/ that Atri linked to on the JIRA: Yes, it's definitely possible to do tricks with large result sets.  Especially with the simple "just sort on score"-case where the really large wins in my book is less GC pressure by using a single `long[]` to hold the structure instead of a gazillion small objects.
   
   Here's the but: I haven't pursued it further as we (Royal Danish Library) have little use for it. Being able to handle large result sets in a single shards does not help much with multi-shard setups, where the merging node is likely to blow up. An iterative approach, such as `cursorMark` or Solr's `export`, is less prone to surprises and scales indefinitely. That being said, I won't stand in the way of building a fine foot blowing gun - for some use cases it would be a great win.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org