You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by "vicennial (via GitHub)" <gi...@apache.org> on 2023/07/12 10:01:10 UTC

[GitHub] [spark] vicennial commented on a diff in pull request #41752: [SPARK-44201][CONNECT][SS]Add support for Streaming Listener in Scala for Spark Connect

vicennial commented on code in PR #41752:
URL: https://github.com/apache/spark/pull/41752#discussion_r1260934037


##########
connector/connect/client/jvm/src/main/scala/org/apache/spark/sql/streaming/StreamingQueryManager.scala:
##########
@@ -36,6 +41,13 @@ import org.apache.spark.sql.SparkSession
 @Evolving
 class StreamingQueryManager private[sql] (sparkSession: SparkSession) extends Logging {
 
+  // Mapping from StreamingQueryListener to id. There's another mapping from id to
+  // StreamingQueryListener on server side. This is used by removeListener() to find the id
+  // of previously added StreamingQueryListener and pass it to server side to find the
+  // corresponding listener on server side.
+  private lazy val listenerCache: ConcurrentMap[StreamingQueryListener, String] =

Review Comment:
   Do we have a test case here that check's the behaviour when two instances of a `StreamingQueryListener` whose constructor values were the same are inserted into this Map? 
   
   Looking at the how the default hashes of an object are created [here](https://docs.oracle.com/javase/7/docs/api/java/lang/Object.html#hashCode%28%29), different instances should have different hashes but they also specify the following:
   
   > This is typically implemented by converting the internal address of the object into an integer, but this implementation technique is not required by the JavaTM programming language.
   
   Let's make sure that there aren't hash collisions in this case



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org