[GitHub] [iceberg] nastra commented on a diff in pull request #6385: Fix Spark JMH Benchmarks

[GitBox <gi...@apache.org>]

nastra commented on code in PR #6385:
URL: https://github.com/apache/iceberg/pull/6385#discussion_r1043244017


##########
spark/v3.3/spark/src/jmh/java/org/apache/iceberg/spark/data/parquet/SparkParquetReadersFlatDataBenchmark.java:
##########
@@ -155,6 +155,7 @@ public void readUsingSparkReader(Blackhole blackhole) throws IOException {
             .set("spark.sql.parquet.binaryAsString", "false")
             .set("spark.sql.parquet.int96AsTimestamp", "false")
             .set("spark.sql.caseSensitive", "false")
+            .set("spark.sql.parquet.fieldId.write.enabled", "false")

Review Comment:
   this flag was introduced with Spark 3.3.0 and `SparkToParquetSchemaConverter` requires this flag to be set



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org