Re: [PR] feat: add compression level configuration for JSON/CSV writers [datafusion]

["Smotrov (via GitHub)" <gi...@apache.org>]

Smotrov commented on code in PR #18954:
URL: https://github.com/apache/datafusion/pull/18954#discussion_r2586273603


##########
datafusion/common/src/file_options/csv_writer.rs:
##########
@@ -41,6 +43,20 @@ impl CsvWriterOptions {
         Self {
             writer_options,
             compression,
+            compression_level: None,
+        }
+    }
+
+    /// Create a new `CsvWriterOptions` with the specified compression level.
+    pub fn new_with_level(
+        writer_options: WriterBuilder,
+        compression: CompressionTypeVariant,
+        compression_level: Option<u32>,

Review Comment:
   I'm not an LLM :-)) at least so far. 
   Actually I was refereeing datafusion/datafusion/common/src/file_options/csv_writer.rs which returns `CsvWriterOptions` with optional compression level. 
   But you are correct. It constructs directly not with `new_with()` or 'new()'. Looked at the wrong place, sorry for misleading comment. 
   
   But coming back to the original question: do you prefer `compression_level: u32` or `Option<u32>`? 
   Perhaps if someone is calling `new_with_level()` there is not so much sense to pass `None` as a compression level. 
   
   How do you think? 



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org