You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by GitBox <gi...@apache.org> on 2023/01/19 09:42:24 UTC

[GitHub] [spark] LuciferYang commented on a diff in pull request #39642: [WIP][SPARK-41677][CORE][SQL][SS][UI] Add Protobuf serializer for `StreamingQueryProgressWrapper`

LuciferYang commented on code in PR #39642:
URL: https://github.com/apache/spark/pull/39642#discussion_r1081018834


##########
sql/core/src/main/scala/org/apache/spark/sql/streaming/progress.scala:
##########
@@ -125,14 +127,16 @@ class StateOperatorProgress private[sql](
  * @since 2.1.0
  */
 @Evolving
-class StreamingQueryProgress private[sql](
+class StreamingQueryProgress private[spark](
   val id: UUID,
   val runId: UUID,
   val name: String,
   val timestamp: String,
   val batchId: Long,
   val batchDuration: Long,
   val durationMs: ju.Map[String, JLong],
+  // If `eventTime` is no longer read-only, need change
+  // `StreamingQueryProgressSerializer#deserialize` to return mutable Java Map
   val eventTime: ju.Map[String, String],

Review Comment:
   There are 6 `ju.Map`,  `durationMs` and `observedMetrics` already use mutable Java Map, the remaining 4 Java Map should be read-only at present, but for safety, we can also directly return mutable Java Map in the deserialize method
   
    



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org