You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by "HeartSaVioR (via GitHub)" <gi...@apache.org> on 2023/05/01 01:19:45 UTC
[GitHub] [spark] HeartSaVioR opened a new pull request, #41001: [SPARK-43328][SS] Add latest timestamp on no-execution trigger for Idle event in streaming query listener
HeartSaVioR opened a new pull request, #41001:
URL: https://github.com/apache/spark/pull/41001
### What changes were proposed in this pull request?
This PR proposes to add the latest timestamp on no-execution trigger for QueryIdleEvent.
### Why are the changes needed?
This adds a "human readable" timestamp which is useful for user side verification as well as be consistent with other events.
### Does this PR introduce _any_ user-facing change?
Yes, but QueryIdleEvent is not released yet.
### How was this patch tested?
Existing tests.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] viirya commented on a diff in pull request #41001: [SPARK-43328][SS] Add latest timestamp on no-execution trigger for Idle event in streaming query listener
Posted by "viirya (via GitHub)" <gi...@apache.org>.
viirya commented on code in PR #41001:
URL: https://github.com/apache/spark/pull/41001#discussion_r1181989173
##########
sql/core/src/main/scala/org/apache/spark/sql/streaming/StreamingQueryListener.scala:
##########
@@ -140,7 +140,8 @@ object StreamingQueryListener {
@Evolving
class QueryIdleEvent private[sql](
val id: UUID,
- val runId: UUID) extends Event
+ val runId: UUID,
+ val timestamp: String) extends Event
Review Comment:
It seems better to document what the timestamp is.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] HeartSaVioR commented on pull request #41001: [SPARK-43328][SS] Add latest timestamp on no-execution trigger for Idle event in streaming query listener
Posted by "HeartSaVioR (via GitHub)" <gi...@apache.org>.
HeartSaVioR commented on PR #41001:
URL: https://github.com/apache/spark/pull/41001#issuecomment-1530918000
Thanks, merging to master!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] HeartSaVioR closed pull request #41001: [SPARK-43328][SS] Add latest timestamp on no-execution trigger for Idle event in streaming query listener
Posted by "HeartSaVioR (via GitHub)" <gi...@apache.org>.
HeartSaVioR closed pull request #41001: [SPARK-43328][SS] Add latest timestamp on no-execution trigger for Idle event in streaming query listener
URL: https://github.com/apache/spark/pull/41001
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] HeartSaVioR commented on pull request #41001: [SPARK-43328][SS] Add latest timestamp on no-execution trigger for Idle event in streaming query listener
Posted by "HeartSaVioR (via GitHub)" <gi...@apache.org>.
HeartSaVioR commented on PR #41001:
URL: https://github.com/apache/spark/pull/41001#issuecomment-1529666648
cc. @cloud-fan @HyukjinKwon @zsxwing @viirya
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] viirya commented on a diff in pull request #41001: [SPARK-43328][SS] Add latest timestamp on no-execution trigger for Idle event in streaming query listener
Posted by "viirya (via GitHub)" <gi...@apache.org>.
viirya commented on code in PR #41001:
URL: https://github.com/apache/spark/pull/41001#discussion_r1181989326
##########
sql/core/src/main/scala/org/apache/spark/sql/streaming/StreamingQueryListener.scala:
##########
@@ -140,7 +140,8 @@ object StreamingQueryListener {
@Evolving
class QueryIdleEvent private[sql](
val id: UUID,
- val runId: UUID) extends Event
+ val runId: UUID,
+ val timestamp: String) extends Event
Review Comment:
For example, the python one looks good.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] HeartSaVioR commented on a diff in pull request #41001: [SPARK-43328][SS] Add latest timestamp on no-execution trigger for Idle event in streaming query listener
Posted by "HeartSaVioR (via GitHub)" <gi...@apache.org>.
HeartSaVioR commented on code in PR #41001:
URL: https://github.com/apache/spark/pull/41001#discussion_r1181993402
##########
sql/core/src/main/scala/org/apache/spark/sql/streaming/StreamingQueryListener.scala:
##########
@@ -140,7 +140,8 @@ object StreamingQueryListener {
@Evolving
class QueryIdleEvent private[sql](
val id: UUID,
- val runId: UUID) extends Event
+ val runId: UUID,
+ val timestamp: String) extends Event
Review Comment:
I'll bring the parameters in the classdoc for the same of python side. Thanks for the input!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org