You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@commons.apache.org by "Gary D. Gregory (Jira)" <ji...@apache.org> on 2022/06/20 12:56:00 UTC
[jira] [Updated] (IO-755) Using FileUtils.listFiles with background changes fails on Linux
[ https://issues.apache.org/jira/browse/IO-755?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Gary D. Gregory updated IO-755:
-------------------------------
Assignee: Gary D. Gregory
> Using FileUtils.listFiles with background changes fails on Linux
> ----------------------------------------------------------------
>
> Key: IO-755
> URL: https://issues.apache.org/jira/browse/IO-755
> Project: Commons IO
> Issue Type: Bug
> Components: Utilities
> Affects Versions: 2.9.0, 2.10.0, 2.11.0
> Reporter: Alexis Suter
> Assignee: Gary D. Gregory
> Priority: Major
> Labels: listFiles
>
> When calling {{FileUtils.listFiles}} and a file is being removed from this or a subdirectory during the execution the following Error occurs on linux.
> h3. Stacktrace
> {code:trace}
> Caused by: java.nio.file.NoSuchFileException: /**
> at java.base/sun.nio.fs.UnixException.translateToIOException(UnixException.java:92)
> at java.base/sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:111)
> at java.base/sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:116)
> at java.base/sun.nio.fs.UnixFileAttributeViews$Basic.readAttributes(UnixFileAttributeViews.java:55)
> at java.base/sun.nio.fs.UnixFileSystemProvider.readAttributes(UnixFileSystemProvider.java:149)
> at java.base/sun.nio.fs.LinuxFileSystemProvider.readAttributes(LinuxFileSystemProvider.java:99)
> at java.base/java.nio.file.Files.readAttributes(Files.java:1764)
> at java.base/java.nio.file.FileTreeWalker.getAttributes(FileTreeWalker.java:219)
> at java.base/java.nio.file.FileTreeWalker.visit(FileTreeWalker.java:276)
> at java.base/java.nio.file.FileTreeWalker.next(FileTreeWalker.java:373)
> at java.base/java.nio.file.Files.walkFileTree(Files.java:2761)
> at org.apache.commons.io.FileUtils.listAccumulate(FileUtils.java:2076)
> at org.apache.commons.io.FileUtils.listFiles(FileUtils.java:2132)
> {code}
> h3. Upgrade of apache commons-io
> The version of the dependency was upgraded from 2.8 to 2.11. In 2.9 the library started using java.nio instead of java.io which might behave differently on unix, when a file is not existent during processing of directories.
> h3. Causing functionality in our implementation
> In some part we have multithreaded access to the filesystem. A folder is accessed from multiple threads which access the folder with {{org.apache.commons.io.FileUtils#listFiles(java.io.File, org.apache.commons.io.filefilter.IOFileFilter, org.apache.commons.io.filefilter.IOFileFilter)}}. These threads also *move*, *delete* and *rename* files from this directory. While this seems to be working on a windows machine, it is not the case on a Linux machine. In 2.8 apparently there was no issue with this.
> *In short*: While a directory is being listed with {{FileUtils.listFiles()}}, a file is being deleted. When trying to read its file-attributes it stops because of a {{NoSuchFileException}}.
> h3. Other commons-io functionalities
> We tried using {{PathUtils.walk(directory filters, 5, false).map(Path::toFile).collect(Collectors.toList())}}. This method was introduced with the same change.
> This produced a similar stacktrace:
> {code:trace}
> Caused by: java.io.UncheckedIOException: java.nio.file.NoSuchFileException: /*
> at java.base/java.nio.file.FileTreeIterator.fetchNextIfNeeded(FileTreeIterator.java:87)
> at java.base/java.nio.file.FileTreeIterator.hasNext(FileTreeIterator.java:103)
> at java.base/java.util.Iterator.forEachRemaining(Iterator.java:132)
> at java.base/java.util.Spliterators$IteratorSpliterator.forEachRemaining(Spliterators.java:1801)
> at java.base/java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:484)
> at java.base/java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:474)
> at java.base/java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:913)
> at java.base/java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
> at java.base/java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:578)
> ... 47 more
> Caused by: java.nio.file.NoSuchFileException: /*
> at java.base/sun.nio.fs.UnixException.translateToIOException(UnixException.java:92)
> at java.base/sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:111)
> at java.base/sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:116)
> at java.base/sun.nio.fs.UnixFileAttributeViews$Basic.readAttributes(UnixFileAttributeViews.java:55)
> at java.base/sun.nio.fs.UnixFileSystemProvider.readAttributes(UnixFileSystemProvider.java:149)
> at java.base/sun.nio.fs.LinuxFileSystemProvider.readAttributes(LinuxFileSystemProvider.java:99)
> at java.base/java.nio.file.Files.readAttributes(Files.java:1764)
> at java.base/java.nio.file.FileTreeWalker.getAttributes(FileTreeWalker.java:219)
> at java.base/java.nio.file.FileTreeWalker.visit(FileTreeWalker.java:276)
> at java.base/java.nio.file.FileTreeWalker.next(FileTreeWalker.java:373)
> at java.base/java.nio.file.FileTreeIterator.fetchNextIfNeeded(FileTreeIterator.java:83)
> ... 59 more{code}
> h3. git commit
> https://github.com/apache/commons-io/commit/4a514d3306b55b3667d1449ebd4cbe5f19dd7af0
> h3. Other Questions
> * What was the reason to wrap all {{IOException}} into {{UncheckedIOException}} in those methods?
--
This message was sent by Atlassian Jira
(v8.20.7#820007)