Re: [PR] feat: Make `FanoutWriter` writer configurable [iceberg-rust]

["liurenjie1024 (via GitHub)" <gi...@apache.org>]

liurenjie1024 commented on code in PR #1872:
URL: https://github.com/apache/iceberg-rust/pull/1872#discussion_r2588001736


##########
crates/iceberg/src/spec/table_properties.rs:
##########
@@ -49,6 +49,8 @@ pub struct TableProperties {
     pub write_format_default: String,
     /// The target file size for files.
     pub write_target_file_size_bytes: usize,
+    /// Whether to use `FanoutWriter` for partitioned tables.
+    pub write_fanout_enabled: bool,

Review Comment:
   ```suggestion
       pub write_datafusion_fanout_enabled: bool,
   ```



##########
crates/integrations/datafusion/src/physical_plan/write.rs:
##########
@@ -266,8 +266,26 @@ impl ExecutionPlan for IcebergWriteExec {
         let data_file_writer_builder = DataFileWriterBuilder::new(rolling_writer_builder);
 
         // Create TaskWriter
-        // TODO: Make fanout_enabled configurable via table properties
-        let fanout_enabled = true;
+        let fanout_enabled = self
+            .table
+            .metadata()
+            .properties()
+            .get(TableProperties::PROPERTY_DATAFUSION_WRITE_FANOUT_ENABLED)
+            .map(|value| {
+                value
+                    .parse::<bool>()
+                    .map_err(|e| {
+                        Error::new(
+                            ErrorKind::DataInvalid,
+                            "Invalid value for write.fanout.enabled, expected 'true' or 'false'",

Review Comment:
   ```suggestion
                               "Invalid value for {}, expected 'true' or 'false'", WRITE_DATAFUSION_FANOUT_ENABLED
   ```



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org