[jira] [Commented] (NIFI-5426) Use NIO.2 API for ListFile to avoid multiple disk reads

["Mark Payne (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/NIFI-5426?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16543212#comment-16543212 ] 

Mark Payne commented on NIFI-5426:
----------------------------------

To clarify, the benefit here of NIO.2 is that when a listing is obtained, the Path objects are able (at least in many circumstances) to cache the basic file attributes. However, when using the File API, a listing contains only the list of filenames, and we then have to ask the File System again about the basic attributes such as "is this a directory?" "last mod time". While this is still fast in most scenarios, for a network-mounted drive, this can sometimes lead to many minutes to list just a couple thousand files.

> Use NIO.2 API for ListFile to avoid multiple disk reads
> -------------------------------------------------------
>
>                 Key: NIFI-5426
>                 URL: https://issues.apache.org/jira/browse/NIFI-5426
>             Project: Apache NiFi
>          Issue Type: Improvement
>          Components: Extensions
>            Reporter: Mark Payne
>            Priority: Major
>
> NIFI-4631 attempted to do something similar but had issues when running against CIFS mounts, reportedly. However, I think the issue was not the use necessarily of NIO.2 but the way in which it was used. It attempted to use walkTree and did did not appear to follow symlinks. As a result, this caused failures in some scenarios. We should look at using NIO.2 and probably obtain a Stream<Path>.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)