You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by GitBox <gi...@apache.org> on 2022/03/11 08:06:23 UTC
[GitHub] [spark] HeartSaVioR opened a new pull request #35816: [SPARK-38522][SS] Enrich the method contract of iterator in StateStore to not expect strong consistency on certain condition
HeartSaVioR opened a new pull request #35816:
URL: https://github.com/apache/spark/pull/35816
### What changes were proposed in this pull request?
This PR proposes to enrich the method contract of iterator in StateStore, that the returned iterator is not guaranteed to reflect all updates being performed "after" it has been initialized.
### Why are the changes needed?
The lack of information on the method contract could mislead callers to expect some guarantees although not described. It would be nice to add to the contract to make clear. It could also help to implementations that they are not required to guarantee such thing.
### Does this PR introduce _any_ user-facing change?
No
### How was this patch tested?
N/A
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] HeartSaVioR commented on a change in pull request #35816: [SPARK-38522][SS] Enrich the method contract of iterator in StateStore to not expect strong consistency on certain condition
Posted by GitBox <gi...@apache.org>.
HeartSaVioR commented on a change in pull request #35816:
URL: https://github.com/apache/spark/pull/35816#discussion_r832745981
##########
File path: sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/state/StateStore.scala
##########
@@ -126,7 +126,10 @@ trait StateStore extends ReadStateStore {
/**
* Return an iterator containing all the key-value pairs in the StateStore. Implementations must
- * ensure that updates (puts, removes) can be made while iterating over this iterator.
+ * ensure that updates (puts, removes) can be made while iterating over this iterator, but it is
Review comment:
Thanks for the suggestion! I'll reflect the change, and go merging since this is a nit comment.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] HeartSaVioR commented on pull request #35816: [SPARK-38522][SS] Enrich the method contract of iterator in StateStore to not expect strong consistency on certain condition
Posted by GitBox <gi...@apache.org>.
HeartSaVioR commented on pull request #35816:
URL: https://github.com/apache/spark/pull/35816#issuecomment-1071973640
friendly reminder cc. @viirya @xuanyuanking @alex-balikov
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] HeartSaVioR commented on pull request #35816: [SPARK-38522][SS] Enrich the method contract of iterator in StateStore to not expect strong consistency on certain condition
Posted by GitBox <gi...@apache.org>.
HeartSaVioR commented on pull request #35816:
URL: https://github.com/apache/spark/pull/35816#issuecomment-1075804970
This is just a comment change and I verified GA passed before the latest commit [50aa38d](https://github.com/apache/spark/pull/35816/commits/50aa38db2f566fe9f482af1cfe7bccc6b23ecbc0) and manually ran scalastyle with the latest commit and it passed.
Thanks all for the reviewing! Merging to master.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] alex-balikov commented on a change in pull request #35816: [SPARK-38522][SS] Enrich the method contract of iterator in StateStore to not expect strong consistency on certain condition
Posted by GitBox <gi...@apache.org>.
alex-balikov commented on a change in pull request #35816:
URL: https://github.com/apache/spark/pull/35816#discussion_r832657989
##########
File path: sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/state/StateStore.scala
##########
@@ -126,7 +126,10 @@ trait StateStore extends ReadStateStore {
/**
* Return an iterator containing all the key-value pairs in the StateStore. Implementations must
- * ensure that updates (puts, removes) can be made while iterating over this iterator.
+ * ensure that updates (puts, removes) can be made while iterating over this iterator, but it is
Review comment:
Split the ', but... ' part as a separate sentence.
/**
* Return an iterator containing all the key-value pairs in the StateStore. Implementations must
* ensure that updates (puts, removes) can be made while iterating over this iterator.
* It is not required for implementations ...
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] HeartSaVioR commented on pull request #35816: [SPARK-38522][SS] Enrich the method contract of iterator in StateStore to not expect strong consistency on certain condition
Posted by GitBox <gi...@apache.org>.
HeartSaVioR commented on pull request #35816:
URL: https://github.com/apache/spark/pull/35816#issuecomment-1071973640
friendly reminder cc. @viirya @xuanyuanking @alex-balikov
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] HeartSaVioR commented on pull request #35816: [SPARK-38522][SS] Enrich the method contract of iterator in StateStore to not expect strong consistency on certain condition
Posted by GitBox <gi...@apache.org>.
HeartSaVioR commented on pull request #35816:
URL: https://github.com/apache/spark/pull/35816#issuecomment-1064871222
cc. @viirya @alex-balikov
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] HeartSaVioR closed pull request #35816: [SPARK-38522][SS] Enrich the method contract of iterator in StateStore to not expect strong consistency on certain condition
Posted by GitBox <gi...@apache.org>.
HeartSaVioR closed pull request #35816:
URL: https://github.com/apache/spark/pull/35816
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org