[GitHub] [nifi] tpalfy commented on a change in pull request #4195: NIFI-7292 Preventing file listing from fail because of insufficient privileges

[GitBox <gi...@apache.org>]

tpalfy commented on a change in pull request #4195: NIFI-7292 Preventing file listing from fail because of insufficient privileges
URL: https://github.com/apache/nifi/pull/4195#discussion_r408066379
 
 

 ##########
 File path: nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/ListFile.java
 ##########
 @@ -547,31 +549,67 @@ public boolean test(final Path path, final BasicFileAttributes attributes) {
             }
         };
 
-        final Stream<Path> inputStream = getPathStream(basePath, maxDepth, matcher);
+        try {
+            final long start = System.currentTimeMillis();
+            final List<FileInfo> result = new LinkedList<>();
 
-        final Stream<FileInfo> listing = inputStream.map(p -> {
-            File file = p.toFile();
-            BasicFileAttributes attributes = lastModifiedMap.get(p);
+            Files.walkFileTree(basePath, Collections.singleton(FileVisitOption.FOLLOW_LINKS), maxDepth, new FileVisitor<Path>() {
+                @Override
+                public FileVisitResult preVisitDirectory(final Path dir, final BasicFileAttributes attributes) throws IOException {
+                    if (Files.isReadable(dir)) {
+                        return FileVisitResult.CONTINUE;
+                    } else {
+                        getLogger().debug("The following directory is not readable: {}", new Object[] {dir.toString()});
+                        return FileVisitResult.SKIP_SUBTREE;
+                    }
+                }
 
-            final FileInfo fileInfo = new FileInfo.Builder()
-                .directory(false)
-                .filename(file.getName())
-                .fullPathFileName(file.getAbsolutePath())
-                .lastModifiedTime(attributes.lastModifiedTime().toMillis())
-                .size(attributes.size())
-                .build();
+                @Override
+                public FileVisitResult visitFile(final Path path, final BasicFileAttributes attributes) throws IOException {
+                    if (matcher.test(path, attributes)) {
+                        final File file = path.toFile();
+                        final BasicFileAttributes fileAttributes = lastModifiedMap.get(path);
+                        final FileInfo fileInfo = new FileInfo.Builder()
+                                .directory(false)
+                                .filename(file.getName())
+                                .fullPathFileName(file.getAbsolutePath())
+                                .lastModifiedTime(fileAttributes.lastModifiedTime().toMillis())
+                                .size(fileAttributes.size())
+                                .build();
+
+                        result.add(fileInfo);
+                    }
 
-            return fileInfo;
-        });
+                    return FileVisitResult.CONTINUE;
+                }
+
+                @Override
+                public FileVisitResult visitFileFailed(final Path path, final IOException e) throws IOException {
+                    if (e instanceof AccessDeniedException) {
+                        getLogger().debug("The following file is not readable: {}", new Object[] {path.toString()});
+                        return FileVisitResult.SKIP_SUBTREE;
+                    } else {
+                        getLogger().error("Error during visiting file {}: {}", new Object[] {path.toString(), e.getMessage()}, e);
+                        e.printStackTrace();
+                        return FileVisitResult.TERMINATE;
+                    }
+                }
+
+                @Override
+                public FileVisitResult postVisitDirectory(final Path dir, final IOException e) throws IOException {
+                    if (e != null) {
+                        getLogger().error("Error during visiting directory {}: {}", new Object[] {dir.toString(), e.getMessage()}, e);
+                        e.printStackTrace();
 
 Review comment:
   Can't we get rid of this `e.printStackTrace()`?

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services