You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucene.apache.org by "Michael McCandless (JIRA)" <ji...@apache.org> on 2015/09/23 12:32:04 UTC
[jira] [Updated] (LUCENE-6813) OfflineSorter.sort isn't thread-safe
[ https://issues.apache.org/jira/browse/LUCENE-6813?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Michael McCandless updated LUCENE-6813:
---------------------------------------
Attachment: LUCENE-6813.patch
Patch, but I still need to add a thread test to see if it provokes the original issue I hit.
I think the only reason why OfflineSorter.sort wasn't thread safe was because it removed the output file up front, instead of replacing it later with the atomic move ... I just removed that Files.deleteIfExists and then added StandardCopyOption.REPLACE_EXISTING later when we do the Files.move.
> OfflineSorter.sort isn't thread-safe
> ------------------------------------
>
> Key: LUCENE-6813
> URL: https://issues.apache.org/jira/browse/LUCENE-6813
> Project: Lucene - Core
> Issue Type: Bug
> Reporter: Michael McCandless
> Assignee: Michael McCandless
> Fix For: Trunk, 5.4
>
> Attachments: LUCENE-6813.patch
>
>
> The new BKD tree classes, and NumericRangeTree (just a 1D BKD tree),
> make heavy use of OfflineSorter to build their data structures at
> indexing time when the number of indexed documents is biggish.
> But when I was first building them (LUCENE-6477), I hit a thread
> safety issue in OfflineSorter, and at that time I just worked around
> it by creating my own private temp directory each time I need to write
> a BKD tree.
> This workaround is sort of messy, and it causes problems with "pending
> delete" files on Windows when we try to remove that temp directory,
> causing test failures like http://jenkins.thetaphi.de/job/Lucene-Solr-5.x-Windows/5149/
> I think instead we should fix the root cause ... i.e. make
> OfflineSorter thread safe. It looks like it's simple...
> Separately I'd like to somehow fix these BKD tests to catch any leaked
> file handles ... I'm not sure they are today.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org