[GitHub] [spark] cxzl25 commented on a diff in pull request #36769: [SPARK-39381][SQL] Make vectorized orc columar writer batch size configurable

[GitBox <gi...@apache.org>]

cxzl25 commented on code in PR #36769:
URL: https://github.com/apache/spark/pull/36769#discussion_r895125836


##########
sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/orc/OrcQuerySuite.scala:
##########
@@ -844,6 +844,21 @@ abstract class OrcQuerySuite extends OrcQueryTest with SharedSparkSession {
       df.write.format("orc").save(path)
     }
   }
+
+  test("SPARK-39387: Make vectorized orc columar writer batch size configurable") {
+    Seq(10, 100).foreach(batchSize => {
+      withSQLConf(SQLConf.ORC_VECTORIZED_WRITER_BATCH_SIZE.key -> batchSize.toString) {

Review Comment:
   If the default value of 1024 is used here, although intercet Throwable, OOM causes the spark process to exit abnormally, and the test will not succeed.
   
   



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org