[GitHub] [iceberg] aokolnychyi commented on a change in pull request #2048: Add validation for metrics config during parquet write

[GitBox <gi...@apache.org>]

aokolnychyi commented on a change in pull request #2048:
URL: https://github.com/apache/iceberg/pull/2048#discussion_r554206175



##########
File path: parquet/src/main/java/org/apache/iceberg/parquet/Parquet.java
##########
@@ -226,6 +226,9 @@ private CompressionCodecName codec() {
       set("parquet.avro.write-old-list-structure", "false");
       MessageType type = ParquetSchemaUtil.convert(schema, name);
 
+      // Check that our metrics make sense
+      metricsConfig.validateProperties(schema);

Review comment:
       I'd also cover table creation. I feel that we should fail table creation if the metrics config refers to non-existing columns. I have seen so many cases when people clone an example but forget to update the metrics config and they realize the problem only after ingestion is done.




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org