[GitHub] [spark] bogao007 commented on a diff in pull request #41752: [SPARK-44201][CONNECT][SS]Add support for Streaming Listener in Scala for Spark Connect

["bogao007 (via GitHub)" <gi...@apache.org>]

bogao007 commented on code in PR #41752:
URL: https://github.com/apache/spark/pull/41752#discussion_r1244154732


##########
connector/connect/client/jvm/src/main/scala/org/apache/spark/sql/streaming/StreamingQueryManager.scala:
##########
@@ -125,6 +129,45 @@ class StreamingQueryManager private[sql] (sparkSession: SparkSession) {
     executeManagerCmd(_.setResetTerminated(true))
   }
 
+  /**
+   * Register a [[StreamingQueryListener]] to receive up-calls for life cycle events of
+   * [[StreamingQuery]].
+   *
+   * @since 3.5.0
+   */
+  def addListener(listener: StreamingQueryListener): Unit = {
+    executeManagerCmd(
+      _.getAddListenerBuilder
+        .setListenerPayload(ByteString.copyFrom(Utils
+          .serialize(StreamingListenerPacket(listener)))))
+  }
+
+  /**
+   * Deregister a [[StreamingQueryListener]].
+   *
+   * @since 3.5.0
+   */
+  def removeListener(listener: StreamingQueryListener): Unit = {
+    executeManagerCmd(
+      _.getRemoveListenerBuilder
+        .setListenerPayload(ByteString.copyFrom(Utils
+          .serialize(StreamingListenerPacket(listener)))))
+  }
+
+  /**
+   * List all [[StreamingQueryListener]]s attached to this [[StreamingQueryManager]].
+   *
+   * @since 3.5.0
+   */
+  def listListeners(): Array[StreamingQueryListener] = {
+    executeManagerCmd(_.setListListeners(true)).getListListeners.getListenersList.asScala.map {
+      listener =>
+        Utils.deserialize[StreamingListenerPacket](listener.getListenerPayload.toByteArray,

Review Comment:
   It seems that directly deserializing with `StreamingQueryListener` would lose local variables and other methods defined in the custom listener that extends `StreamingQueryListener`. Wrapping it with a `StreamingListenerPacket` seems not able to solve the problem.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org