You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucene.apache.org by "Simon Willnauer (JIRA)" <ji...@apache.org> on 2018/04/25 14:59:00 UTC
[jira] [Resolved] (LUCENE-8275) Push up #checkPendingDeletes to
Directory
[ https://issues.apache.org/jira/browse/LUCENE-8275?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Simon Willnauer resolved LUCENE-8275.
-------------------------------------
Resolution: Fixed
> Push up #checkPendingDeletes to Directory
> -----------------------------------------
>
> Key: LUCENE-8275
> URL: https://issues.apache.org/jira/browse/LUCENE-8275
> Project: Lucene - Core
> Issue Type: Bug
> Affects Versions: 7.4, master (8.0)
> Reporter: Simon Willnauer
> Priority: Major
> Fix For: 7.4, master (8.0)
>
> Attachments: LUCENE-8275.patch, LUCENE-8275.patch
>
>
> IndexWriter checks in it's ctor if the incoming directory is an
> FSDirectory. If that is the case it ensures that the directory retries
> deleting it's pending deletes and if there are pending deletes it will
> fail creating the writer. Yet, this check didn't unwrap filter directories
> or subclasses like FileSwitchDirectory such that in the case of MDW we
> never checked for pending deletes.
>
> There are also two places in FSDirectory that first removed the file
> that was supposed to be created / renamed to from the pending deletes set
> and then tried to clean up pending deletes which excluded the file. These
> places now remove the file from the set after the pending deletes are checked.
>
> This caused some test failures lately unfortunately very timing dependent:
>
> {noformat}
> FAILED: junit.framework.TestSuite.org.apache.lucene.search.TestSearcherManager
> Error Message:
> Captured an uncaught exception in thread: Thread[id=1567, name=Thread-1363, state=RUNNABLE, group=TGRP-TestSearcherManager]
> Stack Trace:
> com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=1567, name=Thread-1363, state=RUNNABLE, group=TGRP-TestSearcherManager]
> Caused by: java.lang.RuntimeException: java.nio.file.FileAlreadyExistsException: /home/jenkins/workspace/Lucene-Solr-master-Linux/lucene/build/core/test/J1/temp/lucene.search.TestSearcherManager_BA998C838D219DA9-001/tempDir-001/_0.fdt
> at __randomizedtesting.SeedInfo.seed([BA998C838D219DA9]:0)
> at org.apache.lucene.search.TestSearcherManager$8.run(TestSearcherManager.java:590)
> Caused by: java.nio.file.FileAlreadyExistsException: /home/jenkins/workspace/Lucene-Solr-master-Linux/lucene/build/core/test/J1/temp/lucene.search.TestSearcherManager_BA998C838D219DA9-001/tempDir-001/_0.fdt
> at java.base/sun.nio.fs.UnixException.translateToIOException(UnixException.java:94)
> at java.base/sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:111)
> at java.base/sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:116)
> at java.base/sun.nio.fs.UnixFileSystemProvider.newByteChannel(UnixFileSystemProvider.java:215)
> at java.base/java.nio.file.spi.FileSystemProvider.newOutputStream(FileSystemProvider.java:434)
> at org.apache.lucene.mockfile.FilterFileSystemProvider.newOutputStream(FilterFileSystemProvider.java:197)
> at org.apache.lucene.mockfile.FilterFileSystemProvider.newOutputStream(FilterFileSystemProvider.java:197)
> at org.apache.lucene.mockfile.HandleTrackingFS.newOutputStream(HandleTrackingFS.java:129)
> at org.apache.lucene.mockfile.HandleTrackingFS.newOutputStream(HandleTrackingFS.java:129)
> at org.apache.lucene.mockfile.HandleTrackingFS.newOutputStream(HandleTrackingFS.java:129)
> at org.apache.lucene.mockfile.FilterFileSystemProvider.newOutputStream(FilterFileSystemProvider.java:197)
> at java.base/java.nio.file.Files.newOutputStream(Files.java:218)
> at org.apache.lucene.store.FSDirectory$FSIndexOutput.<init>(FSDirectory.java:413)
> at org.apache.lucene.store.FSDirectory$FSIndexOutput.<init>(FSDirectory.java:409)
> at org.apache.lucene.store.FSDirectory.createOutput(FSDirectory.java:253)
> at org.apache.lucene.store.MockDirectoryWrapper.createOutput(MockDirectoryWrapper.java:665)
> at org.apache.lucene.store.LockValidatingDirectoryWrapper.createOutput(LockValidatingDirectoryWrapper.java:44)
> at org.apache.lucene.store.TrackingDirectoryWrapper.createOutput(TrackingDirectoryWrapper.java:43)
> at org.apache.lucene.codecs.compressing.CompressingStoredFieldsWriter.<init>(CompressingStoredFieldsWriter.java:116)
> at org.apache.lucene.codecs.compressing.CompressingStoredFieldsFormat.fieldsWriter(CompressingStoredFieldsFormat.java:128)
> at org.apache.lucene.codecs.lucene50.Lucene50StoredFieldsFormat.fieldsWriter(Lucene50StoredFieldsFormat.java:183)
> at org.apache.lucene.codecs.asserting.AssertingStoredFieldsFormat.fieldsWriter(AssertingStoredFieldsFormat.java:48)
> at org.apache.lucene.index.StoredFieldsConsumer.initStoredFieldsWriter(StoredFieldsConsumer.java:39)
> at org.apache.lucene.index.StoredFieldsConsumer.startDocument(StoredFieldsConsumer.java:46)
> at org.apache.lucene.index.DefaultIndexingChain.startStoredFields(DefaultIndexingChain.java:363)
> at org.apache.lucene.index.DefaultIndexingChain.processDocument(DefaultIndexingChain.java:399)
> at org.apache.lucene.index.DocumentsWriterPerThread.updateDocument(DocumentsWriterPerThread.java:251)
> at org.apache.lucene.index.DocumentsWriter.updateDocument(DocumentsWriter.java:490)
> at org.apache.lucene.index.IndexWriter.updateDocument(IndexWriter.java:1518)
> at org.apache.lucene.index.IndexWriter.addDocument(IndexWriter.java:1210)
> at org.apache.lucene.search.TestSearcherManager$8.run(TestSearcherManager.java:574)
> FAILED: org.apache.lucene.search.TestSearcherManager.testConcurrentIndexCloseSearchAndRefresh
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org