[GitHub] [arrow-datafusion] qrilka commented on a diff in pull request #6526: Add support for appending data to external tables - CSV

["qrilka (via GitHub)" <gi...@apache.org>]

qrilka commented on code in PR #6526:
URL: https://github.com/apache/arrow-datafusion/pull/6526#discussion_r1224460068


##########
datafusion/core/src/physical_plan/file_format/mod.rs:
##########
@@ -287,41 +311,53 @@ struct FileGroupsDisplay<'a>(&'a [Vec<PartitionedFile>]);
 
 impl<'a> Display for FileGroupsDisplay<'a> {
     fn fmt(&self, f: &mut Formatter) -> FmtResult {
-        let mut first_group = true;
-        let groups = if self.0.len() == 1 { "group" } else { "groups" };
-        write!(f, "{{{} {}: [", self.0.len(), groups)?;
+        let n_group = self.0.len();
+        let groups = if n_group == 1 { "group" } else { "groups" };
+        write!(f, "{{{n_group} {groups}: [")?;
         // To avoid showing too many partitions
         let max_groups = 5;
-        for group in self.0.iter().take(max_groups) {
-            if !first_group {
+        for (idx, group) in self.0.iter().take(max_groups).enumerate() {
+            if idx > 0 {
                 write!(f, ", ")?;
             }
-            first_group = false;
-            write!(f, "[")?;
-
-            let mut first_file = true;
-            for pf in group {
-                if !first_file {
-                    write!(f, ", ")?;
-                }
-                first_file = false;
-
-                write!(f, "{}", pf.object_meta.location.as_ref())?;
-
-                if let Some(range) = pf.range.as_ref() {
-                    write!(f, ":{}..{}", range.start, range.end)?;
-                }
-            }
-            write!(f, "]")?;
+            write!(f, "{}", FileGroupDisplay(group))?;
         }
-        if self.0.len() > max_groups {
+        // Remaining elements are showed as `...` (to indicate there is more)
+        if n_group > max_groups {
             write!(f, ", ...")?;
         }
         write!(f, "]}}")?;
         Ok(())
     }
 }
 
+/// A wrapper to customize partitioned file display
+///
+/// Prints in the format:
+/// ```text
+/// [file1, file2,...]

Review Comment:
   @mustafasrepo could you comment why this doesn't match the implementation? I don't see the trailing "..." there. `FileGroupsDisplay` uses 5 as the largest number or groups shown. Probably we should also limit this list to 5? Maybe having an explicit constant could be useful? 
   I found this conflicting with my attempt to implement https://github.com/apache/arrow-datafusion/issues/6383



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org