You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucene.apache.org by "Mark Miller (JIRA)" <ji...@apache.org> on 2011/08/10 20:01:28 UTC
[jira] [Created] (SOLR-2705) On reload, IndexWriterProvider holds
onto the initial SolrCore it was created with
On reload, IndexWriterProvider holds onto the initial SolrCore it was created with
----------------------------------------------------------------------------------
Key: SOLR-2705
URL: https://issues.apache.org/jira/browse/SOLR-2705
Project: Solr
Issue Type: Bug
Reporter: Mark Miller
Assignee: Mark Miller
Fix For: 4.0
dug up by Yury Kays in SOLR-2326
The IWP needs to be updated to ref the new core on reload - else if some settings have changed (eg newIndexDir change caused by replication), the IndexWriter will be incorrectly reopened.
This should really only be an issue when replication and core reloads are involved for the most part.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] [Commented] (SOLR-2705) On reload, IndexWriterProvider holds
onto the initial SolrCore it was created with
Posted by "Yury Kats (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-2705?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13082589#comment-13082589 ]
Yury Kats commented on SOLR-2705:
---------------------------------
Tested some more, all is well. Thanks a lot, Mark. This is outstanding support and turn around time from a committer. Hats off!
I saw in the patch that you've added getIndexversion and getCommits, but haven't used them. Would be a shame not to put them to work. It would also ensure that replication works right after a RELOAD, which can be a case in my app, so I'm in favor of testing that angle as well. :)
> On reload, IndexWriterProvider holds onto the initial SolrCore it was created with
> ----------------------------------------------------------------------------------
>
> Key: SOLR-2705
> URL: https://issues.apache.org/jira/browse/SOLR-2705
> Project: Solr
> Issue Type: Bug
> Reporter: Mark Miller
> Assignee: Mark Miller
> Fix For: 4.0
>
> Attachments: SOLR-2705.patch
>
>
> dug up by Yury Kats in SOLR-2326
> The IWP needs to be updated to ref the new core on reload - else if some settings have changed (eg newIndexDir change caused by replication), the IndexWriter will be incorrectly reopened.
> This should really only be an issue when replication and core reloads are involved for the most part.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] [Commented] (SOLR-2705) On reload, IndexWriterProvider holds
onto the initial SolrCore it was created with
Posted by "Mark Miller (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-2705?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13082860#comment-13082860 ]
Mark Miller commented on SOLR-2705:
-----------------------------------
bq. I saw in the patch that you've added getIndexversion and getCommits, but haven't used them.
Right - they where good for debugging and I figured I would use them to investigate SOLR-2326.
bq. Can Junit check for indexversion/commits to be same right before and right after coreReload?
I've committed this check.
> On reload, IndexWriterProvider holds onto the initial SolrCore it was created with
> ----------------------------------------------------------------------------------
>
> Key: SOLR-2705
> URL: https://issues.apache.org/jira/browse/SOLR-2705
> Project: Solr
> Issue Type: Bug
> Reporter: Mark Miller
> Assignee: Mark Miller
> Fix For: 4.0
>
> Attachments: SOLR-2705.patch, SOLR-2705.patch
>
>
> dug up by Yury Kats in SOLR-2326
> The IWP needs to be updated to ref the new core on reload - else if some settings have changed (eg newIndexDir change caused by replication), the IndexWriter will be incorrectly reopened.
> This should really only be an issue when replication and core reloads are involved for the most part.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] [Commented] (SOLR-2705) On reload, IndexWriterProvider holds
onto the initial SolrCore it was created with
Posted by "Mark Miller (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-2705?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13082578#comment-13082578 ]
Mark Miller commented on SOLR-2705:
-----------------------------------
Sure, likely easy enough to add.
The current test fails without the fix because the SolrCore is changed on the reload, and replication will sometimes kick off the creation of a new index writer. If it does this using the old SolrCore's settings/fields, as was happening, it might not even be writing to the right index! And it's DeletionPolicy will be the incorrect instance from the old core, and... When the new writer is made, it needs it's info to come from the new SolrCore of course. This bug was nasty - unfortunately we didn't have a test that could tickle it.
So in this test, after the reload, if you add any further documents, they won't show up in any searches, even after commit.
So the test adds 10 docs, ensures they are in the index, then adds 2 more docs, and ensures 12 docs are in the index - without the fix it would find 10 rather than 12 - until you restart the server.
> On reload, IndexWriterProvider holds onto the initial SolrCore it was created with
> ----------------------------------------------------------------------------------
>
> Key: SOLR-2705
> URL: https://issues.apache.org/jira/browse/SOLR-2705
> Project: Solr
> Issue Type: Bug
> Reporter: Mark Miller
> Assignee: Mark Miller
> Fix For: 4.0
>
> Attachments: SOLR-2705.patch
>
>
> dug up by Yury Kats in SOLR-2326
> The IWP needs to be updated to ref the new core on reload - else if some settings have changed (eg newIndexDir change caused by replication), the IndexWriter will be incorrectly reopened.
> This should really only be an issue when replication and core reloads are involved for the most part.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] [Commented] (SOLR-2705) On reload, IndexWriterProvider holds
onto the initial SolrCore it was created with
Posted by "Yury Kats (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-2705?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13082549#comment-13082549 ]
Yury Kats commented on SOLR-2705:
---------------------------------
Thanks, I will try.
Looking at the patch, it's not clear to me whether the reloaded core would return proper Commits right after it's been reloaded, or only after the subsequent commit.
Can Junit check for indexversion/commits to be same right before and right after coreReload?
> On reload, IndexWriterProvider holds onto the initial SolrCore it was created with
> ----------------------------------------------------------------------------------
>
> Key: SOLR-2705
> URL: https://issues.apache.org/jira/browse/SOLR-2705
> Project: Solr
> Issue Type: Bug
> Reporter: Mark Miller
> Assignee: Mark Miller
> Fix For: 4.0
>
> Attachments: SOLR-2705.patch
>
>
> dug up by Yury Kats in SOLR-2326
> The IWP needs to be updated to ref the new core on reload - else if some settings have changed (eg newIndexDir change caused by replication), the IndexWriter will be incorrectly reopened.
> This should really only be an issue when replication and core reloads are involved for the most part.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] [Commented] (SOLR-2705) On reload, IndexWriterProvider holds
onto the initial SolrCore it was created with
Posted by "Mark Miller (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-2705?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13082547#comment-13082547 ]
Mark Miller commented on SOLR-2705:
-----------------------------------
Committed - Yury, can you confirm this works for you use case now?
> On reload, IndexWriterProvider holds onto the initial SolrCore it was created with
> ----------------------------------------------------------------------------------
>
> Key: SOLR-2705
> URL: https://issues.apache.org/jira/browse/SOLR-2705
> Project: Solr
> Issue Type: Bug
> Reporter: Mark Miller
> Assignee: Mark Miller
> Fix For: 4.0
>
> Attachments: SOLR-2705.patch
>
>
> dug up by Yury Kats in SOLR-2326
> The IWP needs to be updated to ref the new core on reload - else if some settings have changed (eg newIndexDir change caused by replication), the IndexWriter will be incorrectly reopened.
> This should really only be an issue when replication and core reloads are involved for the most part.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] [Commented] (SOLR-2705) On reload, IndexWriterProvider holds
onto the initial SolrCore it was created with
Posted by "Yury Kats (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-2705?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13082538#comment-13082538 ]
Yury Kats commented on SOLR-2705:
---------------------------------
The problem is reproducible on the stock multicore example
* configure multicore's ReplicationHandler to replicate after startup and commit
* launch multicore example
* index some example docs
* ask for commits: http://localhost:8983/solr/core0/replication?command=commits
* reload core0: http://localhost:8983/solr/admin/cores?action=RELOAD&core=core0
* ask for commits again: http://localhost:8983/solr/core0/replication?command=commits
* observe no commits being returned after reload.
> On reload, IndexWriterProvider holds onto the initial SolrCore it was created with
> ----------------------------------------------------------------------------------
>
> Key: SOLR-2705
> URL: https://issues.apache.org/jira/browse/SOLR-2705
> Project: Solr
> Issue Type: Bug
> Reporter: Mark Miller
> Assignee: Mark Miller
> Fix For: 4.0
>
> Attachments: SOLR-2705.patch
>
>
> dug up by Yury Kats in SOLR-2326
> The IWP needs to be updated to ref the new core on reload - else if some settings have changed (eg newIndexDir change caused by replication), the IndexWriter will be incorrectly reopened.
> This should really only be an issue when replication and core reloads are involved for the most part.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] [Resolved] (SOLR-2705) On reload, IndexWriterProvider holds
onto the initial SolrCore it was created with
Posted by "Mark Miller (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-2705?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Mark Miller resolved SOLR-2705.
-------------------------------
Resolution: Fixed
Thanks Yury!
> On reload, IndexWriterProvider holds onto the initial SolrCore it was created with
> ----------------------------------------------------------------------------------
>
> Key: SOLR-2705
> URL: https://issues.apache.org/jira/browse/SOLR-2705
> Project: Solr
> Issue Type: Bug
> Reporter: Mark Miller
> Assignee: Mark Miller
> Fix For: 4.0
>
> Attachments: SOLR-2705.patch, SOLR-2705.patch
>
>
> dug up by Yury Kats in SOLR-2326
> The IWP needs to be updated to ref the new core on reload - else if some settings have changed (eg newIndexDir change caused by replication), the IndexWriter will be incorrectly reopened.
> This should really only be an issue when replication and core reloads are involved for the most part.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] [Updated] (SOLR-2705) On reload, IndexWriterProvider holds
onto the initial SolrCore it was created with
Posted by "Mark Miller (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-2705?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Mark Miller updated SOLR-2705:
------------------------------
Attachment: SOLR-2705.patch
Hmm...so that improved things, but it's not quite the right fix. If multiple SolrCores are going to share the IndexWriterProvider for any length of time, it cannot hold a ref to a SolrCore. It's rare to have multiple reloaded SolrCores up at the same time with a Writer (I know because it wasn't even possible a short time ago without running into IndexLock exceptions), but this should now be fine to do. So a better solution is to pass the SolrCore to the methods of IndexWriterProver.
> On reload, IndexWriterProvider holds onto the initial SolrCore it was created with
> ----------------------------------------------------------------------------------
>
> Key: SOLR-2705
> URL: https://issues.apache.org/jira/browse/SOLR-2705
> Project: Solr
> Issue Type: Bug
> Reporter: Mark Miller
> Assignee: Mark Miller
> Fix For: 4.0
>
> Attachments: SOLR-2705.patch, SOLR-2705.patch
>
>
> dug up by Yury Kats in SOLR-2326
> The IWP needs to be updated to ref the new core on reload - else if some settings have changed (eg newIndexDir change caused by replication), the IndexWriter will be incorrectly reopened.
> This should really only be an issue when replication and core reloads are involved for the most part.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] [Updated] (SOLR-2705) On reload, IndexWriterProvider holds
onto the initial SolrCore it was created with
Posted by "Mark Miller (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-2705?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Mark Miller updated SOLR-2705:
------------------------------
Description:
dug up by Yury Kats in SOLR-2326
The IWP needs to be updated to ref the new core on reload - else if some settings have changed (eg newIndexDir change caused by replication), the IndexWriter will be incorrectly reopened.
This should really only be an issue when replication and core reloads are involved for the most part.
was:
dug up by Yury Kays in SOLR-2326
The IWP needs to be updated to ref the new core on reload - else if some settings have changed (eg newIndexDir change caused by replication), the IndexWriter will be incorrectly reopened.
This should really only be an issue when replication and core reloads are involved for the most part.
> On reload, IndexWriterProvider holds onto the initial SolrCore it was created with
> ----------------------------------------------------------------------------------
>
> Key: SOLR-2705
> URL: https://issues.apache.org/jira/browse/SOLR-2705
> Project: Solr
> Issue Type: Bug
> Reporter: Mark Miller
> Assignee: Mark Miller
> Fix For: 4.0
>
>
> dug up by Yury Kats in SOLR-2326
> The IWP needs to be updated to ref the new core on reload - else if some settings have changed (eg newIndexDir change caused by replication), the IndexWriter will be incorrectly reopened.
> This should really only be an issue when replication and core reloads are involved for the most part.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] [Commented] (SOLR-2705) On reload, IndexWriterProvider holds
onto the initial SolrCore it was created with
Posted by "Mark Miller (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-2705?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13082545#comment-13082545 ]
Mark Miller commented on SOLR-2705:
-----------------------------------
I'm going to pop this one in now. Ugly little bug.
> On reload, IndexWriterProvider holds onto the initial SolrCore it was created with
> ----------------------------------------------------------------------------------
>
> Key: SOLR-2705
> URL: https://issues.apache.org/jira/browse/SOLR-2705
> Project: Solr
> Issue Type: Bug
> Reporter: Mark Miller
> Assignee: Mark Miller
> Fix For: 4.0
>
> Attachments: SOLR-2705.patch
>
>
> dug up by Yury Kats in SOLR-2326
> The IWP needs to be updated to ref the new core on reload - else if some settings have changed (eg newIndexDir change caused by replication), the IndexWriter will be incorrectly reopened.
> This should really only be an issue when replication and core reloads are involved for the most part.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] [Commented] (SOLR-2705) On reload, IndexWriterProvider holds
onto the initial SolrCore it was created with
Posted by "Yury Kats (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-2705?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13082567#comment-13082567 ]
Yury Kats commented on SOLR-2705:
---------------------------------
First quick test looks good!
> On reload, IndexWriterProvider holds onto the initial SolrCore it was created with
> ----------------------------------------------------------------------------------
>
> Key: SOLR-2705
> URL: https://issues.apache.org/jira/browse/SOLR-2705
> Project: Solr
> Issue Type: Bug
> Reporter: Mark Miller
> Assignee: Mark Miller
> Fix For: 4.0
>
> Attachments: SOLR-2705.patch
>
>
> dug up by Yury Kats in SOLR-2326
> The IWP needs to be updated to ref the new core on reload - else if some settings have changed (eg newIndexDir change caused by replication), the IndexWriter will be incorrectly reopened.
> This should really only be an issue when replication and core reloads are involved for the most part.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] [Updated] (SOLR-2705) On reload, IndexWriterProvider holds
onto the initial SolrCore it was created with
Posted by "Mark Miller (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-2705?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Mark Miller updated SOLR-2705:
------------------------------
Attachment: SOLR-2705.patch
Test case + fix
> On reload, IndexWriterProvider holds onto the initial SolrCore it was created with
> ----------------------------------------------------------------------------------
>
> Key: SOLR-2705
> URL: https://issues.apache.org/jira/browse/SOLR-2705
> Project: Solr
> Issue Type: Bug
> Reporter: Mark Miller
> Assignee: Mark Miller
> Fix For: 4.0
>
> Attachments: SOLR-2705.patch
>
>
> dug up by Yury Kats in SOLR-2326
> The IWP needs to be updated to ref the new core on reload - else if some settings have changed (eg newIndexDir change caused by replication), the IndexWriter will be incorrectly reopened.
> This should really only be an issue when replication and core reloads are involved for the most part.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org