You are viewing a plain text version of this content. The canonical link for it is here.
Posted to java-user@lucene.apache.org by Parth Tyagi <pt...@egain.com> on 2025/12/16 15:49:12 UTC

Old Lucene Index Directories Not Being Deleted

Hello,
On Rebuilding Lucene Indexes, a new directory is created and connections to old directories are closed. But, when logic to delete old directories is called -> it fails with Exception:
Caused by: java.nio.file.AccessDeniedException: C:\example-1\example-2\index\example-3-1\example-4\index_folder__202512161121\_0.fdt
      at java.base/sun.nio.fs.WindowsException.translateToIOException(WindowsException.java:89)
      at java.base/sun.nio.fs.WindowsException.rethrowAsIOException(WindowsException.java:103)
      at java.base/sun.nio.fs.WindowsException.rethrowAsIOException(WindowsException.java:108)
      at java.base/sun.nio.fs.WindowsFileSystemProvider.implDelete(WindowsFileSystemProvider.java:273)
      at java.base/sun.nio.fs.AbstractFileSystemProvider.deleteIfExists(AbstractFileSystemProvider.java:109)
      at java.base/java.nio.file.Files.deleteIfExists(Files.java:1191)
      at org.apache.commons.io.file.DeletingPathVisitor.visitFile(DeletingPathVisitor.java:158)
      at org.apache.commons.io.file.DeletingPathVisitor.visitFile(DeletingPathVisitor.java:37)
      at java.base/java.nio.file.Files.walkFileTree(Files.java:2786)
      at java.base/java.nio.file.Files.walkFileTree(Files.java:2857)
      at org.apache.commons.io.file.PathUtils.visitFileTree(PathUtils.java:1845)
      at org.apache.commons.io.file.PathUtils.deleteDirectory(PathUtils.java:597)
      at org.apache.commons.io.file.PathUtils.delete(PathUtils.java:556)
      at org.apache.commons.io.FileUtils.forceDelete(FileUtils.java:1421)

This is what I have tried so far :

Tried following things to delete old directory after rebuilding:

  1.
Observed reference of searcher manager in heap dump, so made changes to remove searcher in {package}.{class}#setIndexDirectoryAndWriterTuple, but after changes directories were not deleted. Observed heap dump after removing searcher manager reference, in heap dump reference was removed but directory was not deleted.
  2.
Tried refresh and release searcher.
  3.
Added retries to delete directories in com.{package}.FileUtils#deleteFolder. It never works consistently or on a consistent time frame. The period can vary from 20-50 minutes when it works or never working even after 2-3 hours.
  4.
Closed index writer just before deleting directory in {package}.LuceneDirectoryDeleter#deleteDirectory(org.apache.lucene.store.FSDirectory).(Directories for 1-2 indexes were deleted but not for others).
  5.
Tried deleting old directory with unit test and directory was deleted successfully and issue was not reproducible through unit test.
  6.
Tried obtainLock() method of Lock class in lucene store to identify if any lock is already obtained on the directory but lock is successfully obtained without any exception.

Can you help with this ?

Thanks,
Parth.


Get Outlook for Mac <https://aka.ms/GetOutlookForMac>

Re: Old Lucene Index Directories Not Being Deleted

Posted by Dawid Weiss <da...@gmail.com>.
Your application very likely doesn't close opened index readers/
directories properly and open file handles keep those files un-removable in
(older) Windows versions. If you clean up properly, all file handles should
be closed. Also - don't delete those files manually, they will get deleted
automatically when new index commits are made and when nothing holds those
files.

Dawid

On Wed, Dec 17, 2025 at 7:02 PM Parth Tyagi <pt...@egain.com> wrote:

> Hello,
> On Rebuilding Lucene Indexes, a new directory is created and connections
> to old directories are closed. But, when logic to delete old directories is
> called -> it fails with Exception:
> Caused by: java.nio.file.AccessDeniedException:
> C:\example-1\example-2\index\example-3-1\example-4\index_folder__202512161121\_0.fdt
> at
> java.base/sun.nio.fs.WindowsException.translateToIOException(WindowsException.java:89)
> at
> java.base/sun.nio.fs.WindowsException.rethrowAsIOException(WindowsException.java:103)
> at
> java.base/sun.nio.fs.WindowsException.rethrowAsIOException(WindowsException.java:108)
> at
> java.base/sun.nio.fs.WindowsFileSystemProvider.implDelete(WindowsFileSystemProvider.java:273)
> at
> java.base/sun.nio.fs.AbstractFileSystemProvider.deleteIfExists(AbstractFileSystemProvider.java:109)
> at java.base/java.nio.file.Files.deleteIfExists(Files.java:1191)
> at
> org.apache.commons.io.file.DeletingPathVisitor.visitFile(DeletingPathVisitor.java:158)
> at
> org.apache.commons.io.file.DeletingPathVisitor.visitFile(DeletingPathVisitor.java:37)
> at java.base/java.nio.file.Files.walkFileTree(Files.java:2786)
> at java.base/java.nio.file.Files.walkFileTree(Files.java:2857)
> at org.apache.commons.io.file.PathUtils.visitFileTree(PathUtils.java:1845)
> at org.apache.commons.io.file.PathUtils.deleteDirectory(PathUtils.java:597)
> at org.apache.commons.io.file.PathUtils.delete(PathUtils.java:556)
> at org.apache.commons.io.FileUtils.forceDelete(FileUtils.java:1421)
>
> This is what I have tried so far :
>
> Tried following things to delete old directory after rebuilding:
>
>   1.
> Observed reference of searcher manager in heap dump, so made changes to
> remove searcher in {package}.{class}#setIndexDirectoryAndWriterTuple, but
> after changes directories were not deleted. Observed heap dump after
> removing searcher manager reference, in heap dump reference was removed but
> directory was not deleted.
>   2.
> Tried refresh and release searcher.
>   3.
> Added retries to delete directories in
> com.{package}.FileUtils#deleteFolder. It never works consistently or on a
> consistent time frame. The period can vary from 20-50 minutes when it works
> or never working even after 2-3 hours.
>   4.
> Closed index writer just before deleting directory in
> {package}.LuceneDirectoryDeleter#deleteDirectory(org.apache.lucene.store.FSDirectory).(Directories
> for 1-2 indexes were deleted but not for others).
>   5.
> Tried deleting old directory with unit test and directory was deleted
> successfully and issue was not reproducible through unit test.
>   6.
> Tried obtainLock() method of Lock class in lucene store to identify if any
> lock is already obtained on the directory but lock is successfully obtained
> without any exception.
>
> Can you help with this ?
>
> Thanks,
> Parth.
>
>
> Get Outlook for Mac <https://aka.ms/GetOutlookForMac>
>