[GitHub] [incubator-kyuubi] hddong commented on a diff in pull request #2678: [KYUUBI #2285] trino's result fetching method is changed to a streaming iterator mode to avoid hold data at server side

[GitBox <gi...@apache.org>]

hddong commented on code in PR #2678:
URL: https://github.com/apache/incubator-kyuubi/pull/2678#discussion_r875391455


##########
externals/kyuubi-trino-engine/src/main/scala/org/apache/kyuubi/engine/trino/TrinoStatement.scala:
##########
@@ -80,63 +74,34 @@ class TrinoStatement(
     }
   }
 
-  /**
-   * Execute sql and return ResultSet.
-   */
-  def execute(): Iterable[List[Any]] = {
-    val rowQueue = new ArrayBlockingQueue[List[Any]](MAX_QUEUED_ROWS)
-
-    val dataProcessing = Future[Unit] {
-      while (trino.isRunning) {
-        val data = trino.currentData().getData()
-        if (data != null) {
-          data.asScala.map(_.asScala.toList)
-            .foreach(e => putOrThrow(rowQueue, e))
+  def execute(): Iterator[List[Any]] = {
+    Iterator.continually {
+      @tailrec
+      def getData(): (Boolean, List[List[Any]]) = {
+        if (trino.isRunning) {
+          val data = trino.currentData().getData()
+          trino.advance()

Review Comment:
   Can we move it to line 87? When data is null, we try to get next batch.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For additional commands, e-mail: notifications-help@kyuubi.apache.org