Re: [PR] parquet: don't parse a full Hadoop config file for each Parquet write if not needed [iceberg]

["snicholasbarton (via GitHub)" <gi...@apache.org>]

snicholasbarton commented on code in PR #18398:
URL: https://github.com/apache/iceberg/pull/18398#discussion_r4208707504


##########
parquet/src/main/java/org/apache/iceberg/parquet/ParquetCodecFactory.java:
##########
@@ -21,18 +21,26 @@
 import org.apache.hadoop.conf.Configuration;
 import org.apache.hadoop.io.compress.CompressionCodec;
 import org.apache.hadoop.util.ReflectionUtils;
+import org.apache.parquet.conf.HadoopParquetConfiguration;
+import org.apache.parquet.conf.ParquetConfiguration;
 import org.apache.parquet.hadoop.BadConfigurationException;
 import org.apache.parquet.hadoop.CodecFactory;
 import org.apache.parquet.hadoop.metadata.CompressionCodecName;
+import org.apache.parquet.hadoop.util.ConfigurationUtil;
 
 /**
- * This class implements a codec factory that is used when reading from Parquet. It adds a
- * workaround to cache codecs by name and level, not just by name. This can be removed when this
- * change is made to Parquet.
+ * This class implements a codec factory that is used when writing Parquet. It adds a workaround to
+ * cache codecs by name and level, not just by name, and also honors the legacy
+ * "io.compression.codec.zstd.level" property. This can be removed when this change is made to
+ * Parquet.
  */
 public class ParquetCodecFactory extends CodecFactory {
 
   public ParquetCodecFactory(Configuration configuration, int pageSize) {

Review Comment:
   Yep; was mostly just being conservative given this is library code, but I did another look over and seems it's for internal-use only so have marked as deprecated + made the new constructor package-private



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org