[GitHub] [pinot] Jackie-Jiang commented on a diff in pull request #9466: collect file info like mtime, length while listing files for free

[GitBox <gi...@apache.org>]

Jackie-Jiang commented on code in PR #9466:
URL: https://github.com/apache/pinot/pull/9466#discussion_r1028532956


##########
pinot-plugins/pinot-file-system/pinot-s3/src/main/java/org/apache/pinot/plugin/filesystem/S3PinotFS.java:
##########
@@ -435,8 +437,48 @@ public long length(URI fileUri)
   @Override
   public String[] listFiles(URI fileUri, boolean recursive)
       throws IOException {
+    ImmutableList.Builder<String> builder = ImmutableList.builder();
+    visitFiles(fileUri, recursive, s3Object -> {
+      // TODO: Looks like S3PinotFS filters out directories, inconsistent with the other implementations.
+      // Only add files and not directories

Review Comment:
   Can you elaborate more on how it will break?
   In S3, there is no folder concept under a bucket (flat structure), and that's probably the reason why folder is not returned



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org