[GitHub] [iceberg] hililiwei commented on a change in pull request #3810: ORC: Add ORC support for write properties

[GitBox <gi...@apache.org>]

hililiwei commented on a change in pull request #3810:
URL: https://github.com/apache/iceberg/pull/3810#discussion_r817529575



##########
File path: orc/src/main/java/org/apache/iceberg/orc/ORC.java
##########
@@ -143,12 +152,73 @@ public WriteBuilder metricsConfig(MetricsConfig newMetricsConfig) {
       return this;
     }
 
+    // supposed to always be a private method used strictly by data and delete write builders
+    private WriteBuilder createContextFunc(Function<Configuration, Context> newCreateContextFunc) {
+      this.createContextFunc = newCreateContextFunc;
+      return this;
+    }
+
     public <D> FileAppender<D> build() {
       Preconditions.checkNotNull(schema, "Schema is required");
+
+      // Map Iceberg properties to pass down to the ORC writer
+      Context context = createContextFunc.apply(conf);
+      long stripeSize = context.stripeSize();
+      long blockSize = context.blockSize();
+
+      conf.setLong(OrcConf.STRIPE_SIZE.getAttribute(), stripeSize);
+      conf.setLong(OrcConf.BLOCK_SIZE.getAttribute(), blockSize);
+
       return new OrcFileAppender<>(schema,
           this.file, createWriterFunc, conf, metadata,
           conf.getInt(VECTOR_ROW_BATCH_SIZE, VectorizedRowBatch.DEFAULT_SIZE), metricsConfig);
     }
+
+    private static class Context {
+      private final long stripeSize;
+      private final long blockSize;
+
+      public long stripeSize() {
+        return stripeSize;
+      }
+
+      public long blockSize() {
+        return blockSize;
+      }
+
+      private Context(long stripeSize, long blockSize) {
+        this.stripeSize = stripeSize;
+        this.blockSize = blockSize;
+      }
+
+      static Context dataContext(Configuration conf) {
+        String stripeSizeStr =
+            conf.get(TableProperties.ORC_STRIPE_SIZE_BYTES, conf.get(OrcConf.STRIPE_SIZE.getAttribute()));
+        long stripeSize = stripeSizeStr != null ? Long.parseLong(stripeSizeStr) :
+            TableProperties.ORC_STRIPE_SIZE_BYTES_DEFAULT;

Review comment:
       If the value of `TableProperties.ORC_STRIPE_SIZE_BYTES` is null, first attempts to obtain the value of native `OrcConf.STRIPE_SIZE`. `TableProperties.ORC_STRIPE_SIZE_BYTES_DEFAULT` is used only when neither `TableProperties.ORC_STRIPE_SIZE_BYTES` nor `OrcConf.STRIPE_SIZE` is configured.




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org