You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucene.apache.org by "Simon Willnauer (JIRA)" <ji...@apache.org> on 2011/04/14 12:31:05 UTC
[jira] [Created] (LUCENE-3028) IW.getReader() returns inconsistent
reader on RT Branch
IW.getReader() returns inconsistent reader on RT Branch
-------------------------------------------------------
Key: LUCENE-3028
URL: https://issues.apache.org/jira/browse/LUCENE-3028
Project: Lucene - Java
Issue Type: Bug
Components: Index
Affects Versions: Realtime Branch
Reporter: Simon Willnauer
Assignee: Simon Willnauer
Fix For: Realtime Branch
I extended the testcase TestRollingUpdates#testUpdateSameDoc to pull a NRT reader after each update and asserted that is always sees only one document. Yet, this fails with current branch since there is a problem in how we flush in the getReader() case. What happens here is that we flush all threads and then release the lock (letting other flushes which came in after we entered the flushAllThread context, continue) so that we could concurrently get a new segment that transports global deletes without the corresponding add. They sneak in while we continue to open the NRT reader which in turn sees inconsistent results.
I will upload a patch soon
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] [Updated] (LUCENE-3028) IW.getReader() returns inconsistent
reader on RT Branch
Posted by "selckin (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-3028?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
selckin updated LUCENE-3028:
----------------------------
Attachment: realtime-1.txt
> IW.getReader() returns inconsistent reader on RT Branch
> -------------------------------------------------------
>
> Key: LUCENE-3028
> URL: https://issues.apache.org/jira/browse/LUCENE-3028
> Project: Lucene - Java
> Issue Type: Bug
> Components: Index
> Affects Versions: Realtime Branch
> Reporter: Simon Willnauer
> Assignee: Simon Willnauer
> Fix For: Realtime Branch
>
> Attachments: LUCENE-3028.patch, LUCENE-3028.patch, realtime-1.txt
>
>
> I extended the testcase TestRollingUpdates#testUpdateSameDoc to pull a NRT reader after each update and asserted that is always sees only one document. Yet, this fails with current branch since there is a problem in how we flush in the getReader() case. What happens here is that we flush all threads and then release the lock (letting other flushes which came in after we entered the flushAllThread context, continue) so that we could concurrently get a new segment that transports global deletes without the corresponding add. They sneak in while we continue to open the NRT reader which in turn sees inconsistent results.
> I will upload a patch soon
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] [Updated] (LUCENE-3028) IW.getReader() returns inconsistent
reader on RT Branch
Posted by "Simon Willnauer (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-3028?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Simon Willnauer updated LUCENE-3028:
------------------------------------
Attachment: LUCENE-3028.patch
next iteration, edited some asserts in DW
> IW.getReader() returns inconsistent reader on RT Branch
> -------------------------------------------------------
>
> Key: LUCENE-3028
> URL: https://issues.apache.org/jira/browse/LUCENE-3028
> Project: Lucene - Java
> Issue Type: Bug
> Components: Index
> Affects Versions: Realtime Branch
> Reporter: Simon Willnauer
> Assignee: Simon Willnauer
> Fix For: Realtime Branch
>
> Attachments: LUCENE-3028.patch, LUCENE-3028.patch
>
>
> I extended the testcase TestRollingUpdates#testUpdateSameDoc to pull a NRT reader after each update and asserted that is always sees only one document. Yet, this fails with current branch since there is a problem in how we flush in the getReader() case. What happens here is that we flush all threads and then release the lock (letting other flushes which came in after we entered the flushAllThread context, continue) so that we could concurrently get a new segment that transports global deletes without the corresponding add. They sneak in while we continue to open the NRT reader which in turn sees inconsistent results.
> I will upload a patch soon
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] [Updated] (LUCENE-3028) IW.getReader() returns inconsistent
reader on RT Branch
Posted by "Simon Willnauer (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-3028?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Simon Willnauer updated LUCENE-3028:
------------------------------------
Attachment: LUCENE-3028.patch
here is a first patch
> IW.getReader() returns inconsistent reader on RT Branch
> -------------------------------------------------------
>
> Key: LUCENE-3028
> URL: https://issues.apache.org/jira/browse/LUCENE-3028
> Project: Lucene - Java
> Issue Type: Bug
> Components: Index
> Affects Versions: Realtime Branch
> Reporter: Simon Willnauer
> Assignee: Simon Willnauer
> Fix For: Realtime Branch
>
> Attachments: LUCENE-3028.patch
>
>
> I extended the testcase TestRollingUpdates#testUpdateSameDoc to pull a NRT reader after each update and asserted that is always sees only one document. Yet, this fails with current branch since there is a problem in how we flush in the getReader() case. What happens here is that we flush all threads and then release the lock (letting other flushes which came in after we entered the flushAllThread context, continue) so that we could concurrently get a new segment that transports global deletes without the corresponding add. They sneak in while we continue to open the NRT reader which in turn sees inconsistent results.
> I will upload a patch soon
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] [Resolved] (LUCENE-3028) IW.getReader() returns inconsistent
reader on RT Branch
Posted by "Simon Willnauer (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-3028?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Simon Willnauer resolved LUCENE-3028.
-------------------------------------
Resolution: Fixed
fixed in RT
> IW.getReader() returns inconsistent reader on RT Branch
> -------------------------------------------------------
>
> Key: LUCENE-3028
> URL: https://issues.apache.org/jira/browse/LUCENE-3028
> Project: Lucene - Java
> Issue Type: Bug
> Components: Index
> Affects Versions: Realtime Branch
> Reporter: Simon Willnauer
> Assignee: Simon Willnauer
> Fix For: Realtime Branch
>
> Attachments: LUCENE-3028.patch, LUCENE-3028.patch, realtime-1.txt
>
>
> I extended the testcase TestRollingUpdates#testUpdateSameDoc to pull a NRT reader after each update and asserted that is always sees only one document. Yet, this fails with current branch since there is a problem in how we flush in the getReader() case. What happens here is that we flush all threads and then release the lock (letting other flushes which came in after we entered the flushAllThread context, continue) so that we could concurrently get a new segment that transports global deletes without the corresponding add. They sneak in while we continue to open the NRT reader which in turn sees inconsistent results.
> I will upload a patch soon
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] [Commented] (LUCENE-3028) IW.getReader() returns
inconsistent reader on RT Branch
Posted by "Simon Willnauer (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-3028?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13019806#comment-13019806 ]
Simon Willnauer commented on LUCENE-3028:
-----------------------------------------
I will commit this latest patch to the branch we can still iterates but since we have jenkins running builds I want to let that sink a bit too....
simon
> IW.getReader() returns inconsistent reader on RT Branch
> -------------------------------------------------------
>
> Key: LUCENE-3028
> URL: https://issues.apache.org/jira/browse/LUCENE-3028
> Project: Lucene - Java
> Issue Type: Bug
> Components: Index
> Affects Versions: Realtime Branch
> Reporter: Simon Willnauer
> Assignee: Simon Willnauer
> Fix For: Realtime Branch
>
> Attachments: LUCENE-3028.patch, LUCENE-3028.patch
>
>
> I extended the testcase TestRollingUpdates#testUpdateSameDoc to pull a NRT reader after each update and asserted that is always sees only one document. Yet, this fails with current branch since there is a problem in how we flush in the getReader() case. What happens here is that we flush all threads and then release the lock (letting other flushes which came in after we entered the flushAllThread context, continue) so that we could concurrently get a new segment that transports global deletes without the corresponding add. They sneak in while we continue to open the NRT reader which in turn sees inconsistent results.
> I will upload a patch soon
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] [Updated] (LUCENE-3028) IW.getReader() returns inconsistent
reader on RT Branch
Posted by "Simon Willnauer (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-3028?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Simon Willnauer updated LUCENE-3028:
------------------------------------
Attachment: (was: LUCENE-3028.patch)
> IW.getReader() returns inconsistent reader on RT Branch
> -------------------------------------------------------
>
> Key: LUCENE-3028
> URL: https://issues.apache.org/jira/browse/LUCENE-3028
> Project: Lucene - Java
> Issue Type: Bug
> Components: Index
> Affects Versions: Realtime Branch
> Reporter: Simon Willnauer
> Assignee: Simon Willnauer
> Fix For: Realtime Branch
>
> Attachments: LUCENE-3028.patch
>
>
> I extended the testcase TestRollingUpdates#testUpdateSameDoc to pull a NRT reader after each update and asserted that is always sees only one document. Yet, this fails with current branch since there is a problem in how we flush in the getReader() case. What happens here is that we flush all threads and then release the lock (letting other flushes which came in after we entered the flushAllThread context, continue) so that we could concurrently get a new segment that transports global deletes without the corresponding add. They sneak in while we continue to open the NRT reader which in turn sees inconsistent results.
> I will upload a patch soon
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] [Commented] (LUCENE-3028) IW.getReader() returns
inconsistent reader on RT Branch
Posted by "selckin (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-3028?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13020222#comment-13020222 ]
selckin commented on LUCENE-3028:
---------------------------------
hasn't failed since above fix
> IW.getReader() returns inconsistent reader on RT Branch
> -------------------------------------------------------
>
> Key: LUCENE-3028
> URL: https://issues.apache.org/jira/browse/LUCENE-3028
> Project: Lucene - Java
> Issue Type: Bug
> Components: Index
> Affects Versions: Realtime Branch
> Reporter: Simon Willnauer
> Assignee: Simon Willnauer
> Fix For: Realtime Branch
>
> Attachments: LUCENE-3028.patch, LUCENE-3028.patch, realtime-1.txt
>
>
> I extended the testcase TestRollingUpdates#testUpdateSameDoc to pull a NRT reader after each update and asserted that is always sees only one document. Yet, this fails with current branch since there is a problem in how we flush in the getReader() case. What happens here is that we flush all threads and then release the lock (letting other flushes which came in after we entered the flushAllThread context, continue) so that we could concurrently get a new segment that transports global deletes without the corresponding add. They sneak in while we continue to open the NRT reader which in turn sees inconsistent results.
> I will upload a patch soon
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] [Commented] (LUCENE-3028) IW.getReader() returns
inconsistent reader on RT Branch
Posted by "selckin (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-3028?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13019996#comment-13019996 ]
selckin commented on LUCENE-3028:
---------------------------------
Seems to fail once every 6-8 runs quite consistently (at least i think this is the issue)
brachnes/realtime_search r1092329
{{
[junit] Testsuite: org.apache.lucene.index.TestRollingUpdates
[junit] Testcase: testUpdateSameDoc(org.apache.lucene.index.TestRollingUpdates): Caused an ERROR
[junit] MockDirectoryWrapper: cannot close: there are still open files: {_ho.fdt=1, _ho.prx=1, _ho.fdx=1, _ho.nrm=1, _j0.fdt=1, _ho.tis=1, _j0.fdx=1, _j0.tis=1, _j0.prx=1, _ho.frq=1, _ho.tvx=1, _ho.tvd=1, _j0.nrm=1, _ho.tvf=1, _j0.frq=1, _j0.tvf=1, _j0.tvd=1, _j0.tvx=1}
[junit] java.lang.RuntimeException: MockDirectoryWrapper: cannot close: there are still open files: {_ho.fdt=1, _ho.prx=1, _ho.fdx=1, _ho.nrm=1, _j0.fdt=1, _ho.tis=1, _j0.fdx=1, _j0.tis=1, _j0.prx=1, _ho.frq=1, _ho.tvx=1, _ho.tvd=1, _j0.nrm=1, _ho.tvf=1, _j0.frq=1, _j0.tvf=1
, _j0.tvd=1, _j0.tvx=1}
[junit] at org.apache.lucene.store.MockDirectoryWrapper.close(MockDirectoryWrapper.java:414)
[junit] at org.apache.lucene.index.TestRollingUpdates.testUpdateSameDoc(TestRollingUpdates.java:104)
[junit] at org.apache.lucene.util.LuceneTestCase$LuceneTestCaseRunner.runChild(LuceneTestCase.java:1226)
[junit] at org.apache.lucene.util.LuceneTestCase$LuceneTestCaseRunner.runChild(LuceneTestCase.java:1154)
[junit] Caused by: java.lang.RuntimeException: unclosed IndexInput
[junit] at org.apache.lucene.store.MockDirectoryWrapper.openInput(MockDirectoryWrapper.java:369)
[junit] at org.apache.lucene.store.Directory.openInput(Directory.java:122)
[junit] at org.apache.lucene.index.TermVectorsReader.<init>(TermVectorsReader.java:86)
[junit] at org.apache.lucene.index.SegmentReader$CoreReaders.openDocStores(SegmentReader.java:236)
[junit] at org.apache.lucene.index.SegmentReader.get(SegmentReader.java:495)
[junit] at org.apache.lucene.index.IndexWriter$ReaderPool.get(IndexWriter.java:629)
[junit] at org.apache.lucene.index.IndexWriter$ReaderPool.getReadOnlyClone(IndexWriter.java:587)
[junit] at org.apache.lucene.index.DirectoryReader.<init>(DirectoryReader.java:172)
[junit] at org.apache.lucene.index.IndexWriter.getReader(IndexWriter.java:377)
[junit] at org.apache.lucene.index.DirectoryReader.doReopenFromWriter(DirectoryReader.java:419)
[junit] at org.apache.lucene.index.DirectoryReader.doReopen(DirectoryReader.java:432)
[junit] at org.apache.lucene.index.DirectoryReader.reopen(DirectoryReader.java:392)
[junit] at org.apache.lucene.index.TestRollingUpdates$IndexingThread.run(TestRollingUpdates.java:129)
[junit]
[junit]
[junit] Testcase: testUpdateSameDoc(org.apache.lucene.index.TestRollingUpdates): FAILED
[junit] Some threads threw uncaught exceptions!
[junit] junit.framework.AssertionFailedError: Some threads threw uncaught exceptions!
[junit] at org.apache.lucene.util.LuceneTestCase.tearDown(LuceneTestCase.java:521)
[junit] at org.apache.lucene.util.LuceneTestCase$LuceneTestCaseRunner.runChild(LuceneTestCase.java:1226)
[junit] at org.apache.lucene.util.LuceneTestCase$LuceneTestCaseRunner.runChild(LuceneTestCase.java:1154)
[junit]
[junit]
[junit] Tests run: 2, Failures: 1, Errors: 1, Time elapsed: 6.649 sec
[junit]
[junit] ------------- Standard Error -----------------
[junit] NOTE: reproduce with: ant test -Dtestcase=TestRollingUpdates -Dtestmethod=testUpdateSameDoc -Dtests.seed=-4094951767438954769:-1203905293622856057
[junit] NOTE: reproduce with: ant test -Dtestcase=TestRollingUpdates -Dtestmethod=testUpdateSameDoc -Dtests.seed=-4094951767438954769:-1203905293622856057
[junit] The following exceptions were thrown by threads:
[junit] *** Thread: Thread-103 ***
[junit] java.lang.AssertionError: expected: org.apache.lucene.index.DocumentsWriterDeleteQueue@18635827but was: org.apache.lucene.index.DocumentsWriterDeleteQueue@223074f3 false
[junit] at org.apache.lucene.index.DocumentsWriter.doFlush(DocumentsWriter.java:359)
[junit] at org.apache.lucene.index.DocumentsWriter.updateDocument(DocumentsWriter.java:346)
[junit] at org.apache.lucene.index.IndexWriter.updateDocument(IndexWriter.java:1367)
[junit] at org.apache.lucene.index.IndexWriter.updateDocument(IndexWriter.java:1339)
[junit] at org.apache.lucene.index.TestRollingUpdates$IndexingThread.run(TestRollingUpdates.java:125)
[junit] *** Thread: Thread-106 ***
[junit] java.lang.AssertionError: expected: org.apache.lucene.index.DocumentsWriterDeleteQueue@18635827but was: org.apache.lucene.index.DocumentsWriterDeleteQueue@223074f3 false
[junit] at org.apache.lucene.index.DocumentsWriter.doFlush(DocumentsWriter.java:359)
[junit] at org.apache.lucene.index.DocumentsWriter.updateDocument(DocumentsWriter.java:346)
[junit] at org.apache.lucene.index.IndexWriter.updateDocument(IndexWriter.java:1367)
[junit] at org.apache.lucene.index.IndexWriter.updateDocument(IndexWriter.java:1339)
[junit] at org.apache.lucene.index.TestRollingUpdates$IndexingThread.run(TestRollingUpdates.java:125)
[junit] NOTE: test params are: codec=PreFlex, locale=th, timezone=America/St_Kitts
[junit] NOTE: all tests run in this JVM:
[junit] [TestSearch, TestCharTermAttributeImpl, TestCheckIndex, TestConsistentFieldNumbers, TestCrash, TestDeletionPolicy, TestDocumentWriter, TestIndexReaderCloneNorms, TestLongPostings, TestPayloads, TestPerFieldCodecSupport, TestRollingUpdates]
[junit] NOTE: Linux 2.6.37-gentoo amd64/Sun Microsystems Inc. 1.6.0_24 (64-bit)/cpus=8,threads=1,free=45768760,total=246743040
[junit] ------------- ---------------- ---------------
[junit] TEST org.apache.lucene.index.TestRollingUpdates FAILED
}}
> IW.getReader() returns inconsistent reader on RT Branch
> -------------------------------------------------------
>
> Key: LUCENE-3028
> URL: https://issues.apache.org/jira/browse/LUCENE-3028
> Project: Lucene - Java
> Issue Type: Bug
> Components: Index
> Affects Versions: Realtime Branch
> Reporter: Simon Willnauer
> Assignee: Simon Willnauer
> Fix For: Realtime Branch
>
> Attachments: LUCENE-3028.patch, LUCENE-3028.patch
>
>
> I extended the testcase TestRollingUpdates#testUpdateSameDoc to pull a NRT reader after each update and asserted that is always sees only one document. Yet, this fails with current branch since there is a problem in how we flush in the getReader() case. What happens here is that we flush all threads and then release the lock (letting other flushes which came in after we entered the flushAllThread context, continue) so that we could concurrently get a new segment that transports global deletes without the corresponding add. They sneak in while we continue to open the NRT reader which in turn sees inconsistent results.
> I will upload a patch soon
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] [Commented] (LUCENE-3028) IW.getReader() returns
inconsistent reader on RT Branch
Posted by "Simon Willnauer (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-3028?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13020002#comment-13020002 ]
Simon Willnauer commented on LUCENE-3028:
-----------------------------------------
hmm I can't even after 1k runs :(
> IW.getReader() returns inconsistent reader on RT Branch
> -------------------------------------------------------
>
> Key: LUCENE-3028
> URL: https://issues.apache.org/jira/browse/LUCENE-3028
> Project: Lucene - Java
> Issue Type: Bug
> Components: Index
> Affects Versions: Realtime Branch
> Reporter: Simon Willnauer
> Assignee: Simon Willnauer
> Fix For: Realtime Branch
>
> Attachments: LUCENE-3028.patch, LUCENE-3028.patch
>
>
> I extended the testcase TestRollingUpdates#testUpdateSameDoc to pull a NRT reader after each update and asserted that is always sees only one document. Yet, this fails with current branch since there is a problem in how we flush in the getReader() case. What happens here is that we flush all threads and then release the lock (letting other flushes which came in after we entered the flushAllThread context, continue) so that we could concurrently get a new segment that transports global deletes without the corresponding add. They sneak in while we continue to open the NRT reader which in turn sees inconsistent results.
> I will upload a patch soon
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] [Updated] (LUCENE-3028) IW.getReader() returns inconsistent
reader on RT Branch
Posted by "Simon Willnauer (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-3028?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Simon Willnauer updated LUCENE-3028:
------------------------------------
Comment: was deleted
(was: here is a first patch)
> IW.getReader() returns inconsistent reader on RT Branch
> -------------------------------------------------------
>
> Key: LUCENE-3028
> URL: https://issues.apache.org/jira/browse/LUCENE-3028
> Project: Lucene - Java
> Issue Type: Bug
> Components: Index
> Affects Versions: Realtime Branch
> Reporter: Simon Willnauer
> Assignee: Simon Willnauer
> Fix For: Realtime Branch
>
> Attachments: LUCENE-3028.patch
>
>
> I extended the testcase TestRollingUpdates#testUpdateSameDoc to pull a NRT reader after each update and asserted that is always sees only one document. Yet, this fails with current branch since there is a problem in how we flush in the getReader() case. What happens here is that we flush all threads and then release the lock (letting other flushes which came in after we entered the flushAllThread context, continue) so that we could concurrently get a new segment that transports global deletes without the corresponding add. They sneak in while we continue to open the NRT reader which in turn sees inconsistent results.
> I will upload a patch soon
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] [Commented] (LUCENE-3028) IW.getReader() returns
inconsistent reader on RT Branch
Posted by "Simon Willnauer (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-3028?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13020005#comment-13020005 ]
Simon Willnauer commented on LUCENE-3028:
-----------------------------------------
I just committed a fix for this - seems like the assert which resets the current flushing queue was at the wrong position.
> IW.getReader() returns inconsistent reader on RT Branch
> -------------------------------------------------------
>
> Key: LUCENE-3028
> URL: https://issues.apache.org/jira/browse/LUCENE-3028
> Project: Lucene - Java
> Issue Type: Bug
> Components: Index
> Affects Versions: Realtime Branch
> Reporter: Simon Willnauer
> Assignee: Simon Willnauer
> Fix For: Realtime Branch
>
> Attachments: LUCENE-3028.patch, LUCENE-3028.patch, realtime-1.txt
>
>
> I extended the testcase TestRollingUpdates#testUpdateSameDoc to pull a NRT reader after each update and asserted that is always sees only one document. Yet, this fails with current branch since there is a problem in how we flush in the getReader() case. What happens here is that we flush all threads and then release the lock (letting other flushes which came in after we entered the flushAllThread context, continue) so that we could concurrently get a new segment that transports global deletes without the corresponding add. They sneak in while we continue to open the NRT reader which in turn sees inconsistent results.
> I will upload a patch soon
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] [Commented] (LUCENE-3028) IW.getReader() returns
inconsistent reader on RT Branch
Posted by "Simon Willnauer (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-3028?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13020224#comment-13020224 ]
Simon Willnauer commented on LUCENE-3028:
-----------------------------------------
bq. hasn't failed since above fix
thanks for reporting back, the failure you reported was due to a reset call at the wrong position. I was allowing blocked flushed to continue before I reset the the var that ensures that the blocked flushes continue before a full flush finished.
> IW.getReader() returns inconsistent reader on RT Branch
> -------------------------------------------------------
>
> Key: LUCENE-3028
> URL: https://issues.apache.org/jira/browse/LUCENE-3028
> Project: Lucene - Java
> Issue Type: Bug
> Components: Index
> Affects Versions: Realtime Branch
> Reporter: Simon Willnauer
> Assignee: Simon Willnauer
> Fix For: Realtime Branch
>
> Attachments: LUCENE-3028.patch, LUCENE-3028.patch, realtime-1.txt
>
>
> I extended the testcase TestRollingUpdates#testUpdateSameDoc to pull a NRT reader after each update and asserted that is always sees only one document. Yet, this fails with current branch since there is a problem in how we flush in the getReader() case. What happens here is that we flush all threads and then release the lock (letting other flushes which came in after we entered the flushAllThread context, continue) so that we could concurrently get a new segment that transports global deletes without the corresponding add. They sneak in while we continue to open the NRT reader which in turn sees inconsistent results.
> I will upload a patch soon
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org