Re: [PR] [SPARK-54423][SS] Introducing OffsetMap to enable Named Streaming Sources [spark]

["anishshri-db (via GitHub)" <gi...@apache.org>]

anishshri-db commented on code in PR #53123:
URL: https://github.com/apache/spark/pull/53123#discussion_r2579262674


##########
sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/runtime/MicroBatchExecution.scala:
##########
@@ -100,6 +100,14 @@ class MicroBatchExecution(
 
   @volatile protected var sources: Seq[SparkDataStream] = Seq.empty
 
+  // Source ID mapping for OffsetMap support
+  // Using index as sourceId initially, can be extended to support user-provided names
+  // This is initialized in the same path as the sources Seq (defined above) and is used
+  // in the same way, when OffsetLog v2 is used.
+  @volatile protected var sourceIdMap: Map[String, SparkDataStream] = Map.empty

Review Comment:
   Why do we mark this as `volatile` ?



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org