[GitHub] [incubator-kyuubi] turboFei commented on a change in pull request #1198: [FEATURE] Support incremental collection

[GitBox <gi...@apache.org>]

turboFei commented on a change in pull request #1198:
URL: https://github.com/apache/incubator-kyuubi/pull/1198#discussion_r725490920



##########
File path: externals/kyuubi-spark-sql-engine/src/main/scala/org/apache/kyuubi/engine/spark/operation/ExecuteStatement.scala
##########
@@ -88,11 +92,17 @@ class ExecuteStatement(
       // TODO: Make it configurable
       spark.sparkContext.addSparkListener(operationListener)
       result = spark.sql(statement)
-      // TODO( #921): COMPILED need consider eagerly executed commands
+      // TODO #921: COMPILED need consider eagerly executed commands
       statementEvent.queryExecution = result.queryExecution.toString()
       setState(OperationState.COMPILED)
       debug(result.queryExecution)
-      iter = new ArrayFetchIterator(result.collect())
+      iter = if (incrementCollect) {
+        new IterableFetchIterator[Row](new Iterable[Row] {
+          override def iterator: Iterator[Row] = result.toLocalIterator.asScala
+        })

Review comment:
       I wonder is it different with below?
   ```
           new IterableFetchIterator[Row](result.toLocalIterator().asScala.toIterable)
   ```




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@kyuubi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org