[jira] [Reopened] (SOLR-5783) Can we stop opening a new searcher when the index hasn't changed?

["Hoss Man (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/SOLR-5783?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Hoss Man reopened SOLR-5783:
----------------------------


* I'm not really understanding Yonik's alternative suggestion, but i don't have the code in front of me -- if there is a better way to accomplish the same thing, then great.
* I"m also not really understanding what problems Yonik & MArk are saying exist / may-exist with what got committed as part of this issue -- but it should have just been a optimization, if it's causing problems we should definitely roll back.
* I'm not really in a position to commit anything over the next few days, and then i'm going to be completely offline for over a week -- so if one of you two ([~yonik@apache.org], [~markrmiller@gmail.com]) who understands why it's problematic could please revert this ASAP i'd really appreciate it.
* If you guys could attach patches with tests cases (or pseudo code descriptions showing how to create test cases) demonstrating the problems you see with the current code that would be really helpful when i finally get a chance to revisit this in a few weeks.



> Can we stop opening a new searcher when the index hasn't changed?
> -----------------------------------------------------------------
>
>                 Key: SOLR-5783
>                 URL: https://issues.apache.org/jira/browse/SOLR-5783
>             Project: Solr
>          Issue Type: Improvement
>            Reporter: Hoss Man
>             Fix For: 4.8, 5.0
>
>         Attachments: SOLR-5783.patch, SOLR-5783.patch, SOLR-5783.patch, SOLR-5783.patch
>
>
> I've been thinking recently about how/when we re-open searchers -- and what the overhead of that is in terms of caches and what not -- even if the underlying index hasn't changed.  
> The particular real world case that got me thinking about this recently is when a deleteByQuery gets forwarded to all shards in a collection, and then the subsequent (soft)Commit (either auto or explicit) opens a new searcher -- even if that shard was completley uneffected by the delete.
> It got me wondering: why don't re-use the same searcher when the index is unchanged?
> From what I can tell, we're basically 99% of the way there (in {{<nrtMode/>}})...
> * IndexWriter.commit is already smart enough to short circut if there's nothing to commit
> * SolrCore.openNewSearcher already uses DirectoryReader.openIfChanged to see if the reader can be re-used.
> * for "realtime" purposes, SolrCore.openNewSearcher will return the existing searcher if it exists and the DirectoryReader hasn't changed
> ...The only reason I could think of for not _always_ re-using the same searcher when the underlying DirectoryReader is identical (ie: that last bullet above) is in the situation where the "live" schema has changed -- but that seems pretty trivial to account for.
> Is there any other reason why this wouldn't be a good idea for improving performance?



--
This message was sent by Atlassian JIRA
(v6.2#6252)

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org