[jira] [Created] (SPARK-33119) Only return the first two rows to avoid Driver OOM

["Yuming Wang (Jira)" <ji...@apache.org>]

Yuming Wang created SPARK-33119:
-----------------------------------

             Summary: Only return the first two rows to avoid Driver OOM
                 Key: SPARK-33119
                 URL: https://issues.apache.org/jira/browse/SPARK-33119
             Project: Spark
          Issue Type: Improvement
          Components: SQL
    Affects Versions: 3.1.0
            Reporter: Yuming Wang


{noformat}
Exception in thread "subquery-2871" java.lang.OutOfMemoryError: Requested array size exceeds VM limit
 at scala.collection.mutable.ResizableArray$class.ensureSize(ResizableArray.scala:103)
 at scala.collection.mutable.ArrayBuffer.ensureSize(ArrayBuffer.scala:48)
 at scala.collection.mutable.ArrayBuffer.$plus$eq(ArrayBuffer.scala:84)
 at org.apache.spark.sql.execution.SparkPlan$$anonfun$executeCollect$1$$anonfun$apply$2.apply(SparkPlan.scala:352)
 at org.apache.spark.sql.execution.SparkPlan$$anonfun$executeCollect$1$$anonfun$apply$2.apply(SparkPlan.scala:352)
 at scala.collection.Iterator$class.foreach(Iterator.scala:893)
 at org.apache.spark.sql.execution.SparkPlan$$anon$1.foreach(SparkPlan.scala:330)
 at org.apache.spark.sql.execution.SparkPlan$$anonfun$executeCollect$1.apply(SparkPlan.scala:352)
 at org.apache.spark.sql.execution.SparkPlan$$anonfun$executeCollect$1.apply(SparkPlan.scala:351)
 at scala.collection.IndexedSeqOptimized$class.foreach(IndexedSeqOptimized.scala:33)
 at scala.collection.mutable.ArrayOps$ofRef.foreach(ArrayOps.scala:186)
 at org.apache.spark.sql.execution.SparkPlan.executeCollect(SparkPlan.scala:351)
 at org.apache.spark.sql.execution.adaptive.AdaptiveSparkPlanExec.executeCollect(AdaptiveSparkPlanExec.scala:274)
 at org.apache.spark.sql.execution.SubqueryExec$$anonfun$relationFuture$1$$anonfun$apply$3.apply(basicPhysicalOperators.scala:830)
 at org.apache.spark.sql.execution.SubqueryExec$$anonfun$relationFuture$1$$anonfun$apply$3.apply(basicPhysicalOperators.scala:827)
 at org.apache.spark.sql.execution.SQLExecution$$anonfun$withExecutionId$1.apply(SQLExecution.scala:132)
 at org.apache.spark.sql.execution.SQLExecution$.withSQLConfPropagated(SQLExecution.scala:156)
 at org.apache.spark.sql.execution.SQLExecution$.withExecutionId(SQLExecution.scala:129)
 at org.apache.spark.sql.execution.SubqueryExec$$anonfun$relationFuture$1.apply(basicPhysicalOperators.scala:827)
 at org.apache.spark.sql.execution.SubqueryExec$$anonfun$relationFuture$1.apply(basicPhysicalOperators.scala:827)
 at scala.
{noformat}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org