You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by GitBox <gi...@apache.org> on 2021/08/09 06:06:46 UTC
[GitHub] [spark] xuanyuanking opened a new pull request #33683: [SPARK-36041][SS][DOCS] Introduce the RocksDBStateStoreProvider in the programming guide
xuanyuanking opened a new pull request #33683:
URL: https://github.com/apache/spark/pull/33683
### What changes were proposed in this pull request?
Add the document for the new RocksDBStateStoreProvider.
### Why are the changes needed?
User guide for the new feature.
### Does this PR introduce _any_ user-facing change?
No, doc only.
### How was this patch tested?
Doc only.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] gengliangwang commented on pull request #33683: [SPARK-36041][SS][DOCS] Introduce the RocksDBStateStoreProvider in the programming guide
Posted by GitBox <gi...@apache.org>.
gengliangwang commented on pull request #33683:
URL: https://github.com/apache/spark/pull/33683#issuecomment-899208164
@xuanyuanking The section "State Store and task locality" introduces State Store as well. Shall we combine the new content with it? Or at least move the new content above the section "State Store and task locality"?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #33683: [SPARK-36041][SS][DOCS] Introduce the RocksDBStateStoreProvider in the programming guide
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #33683:
URL: https://github.com/apache/spark/pull/33683#issuecomment-899162779
Kubernetes integration test status success
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/46983/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #33683: [SPARK-36041][SS][DOCS] Introduce the RocksDBStateStoreProvider in the programming guide
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #33683:
URL: https://github.com/apache/spark/pull/33683#issuecomment-899138062
**[Test build #142473 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/142473/testReport)** for PR 33683 at commit [`dec0bae`](https://github.com/apache/spark/commit/dec0baee2f9c9981e60945716cbc1ba91545c031).
* This patch passes all tests.
* This patch merges cleanly.
* This patch adds no public classes.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #33683: [SPARK-36041][SS][DOCS] Introduce the RocksDBStateStoreProvider in the programming guide
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #33683:
URL: https://github.com/apache/spark/pull/33683#issuecomment-899168846
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/46983/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #33683: [SPARK-36041][SS][DOCS] Introduce the RocksDBStateStoreProvider in the programming guide
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #33683:
URL: https://github.com/apache/spark/pull/33683#issuecomment-899253895
Kubernetes integration test unable to build dist.
exiting with code: 1
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/46987/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #33683: [SPARK-36041][SS][DOCS] Introduce the RocksDBStateStoreProvider in the programming guide
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #33683:
URL: https://github.com/apache/spark/pull/33683#issuecomment-899233222
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/142488/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] viirya closed pull request #33683: [SPARK-36041][SS][DOCS] Introduce the RocksDBStateStoreProvider in the programming guide
Posted by GitBox <gi...@apache.org>.
viirya closed pull request #33683:
URL: https://github.com/apache/spark/pull/33683
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #33683: [SPARK-36041][SS][DOCS] Introduce the RocksDBStateStoreProvider in the programming guide
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #33683:
URL: https://github.com/apache/spark/pull/33683#issuecomment-899150258
**[Test build #142475 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/142475/testReport)** for PR 33683 at commit [`d026ea0`](https://github.com/apache/spark/commit/d026ea05c0ff02a8eb3b35f7a7d65feff91bbdf9).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #33683: [SPARK-36041][SS][DOCS] Introduce the RocksDBStateStoreProvider in the programming guide
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #33683:
URL: https://github.com/apache/spark/pull/33683#issuecomment-899138151
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/142473/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #33683: [SPARK-36041][SS][DOCS] Introduce the RocksDBStateStoreProvider in the programming guide
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #33683:
URL: https://github.com/apache/spark/pull/33683#issuecomment-899254331
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/46987/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] viirya commented on a change in pull request #33683: [SPARK-36041][SS][DOCS] Introduce the RocksDBStateStoreProvider in the programming guide
Posted by GitBox <gi...@apache.org>.
viirya commented on a change in pull request #33683:
URL: https://github.com/apache/spark/pull/33683#discussion_r689247394
##########
File path: docs/structured-streaming-programming-guide.md
##########
@@ -1792,7 +1792,85 @@ hence the number is not same as the number of original input rows. You'd like to
There's a known workaround: split your streaming query into multiple queries per stateful operator, and ensure
end-to-end exactly once per query. Ensuring end-to-end exactly once for the last query is optional.
-### State Store and task locality
+### State Store
+
+State store is a versioned key-value store which provides both read and write operations. In
+structured streaming, we use the state store provider to handle the stateful operations across
Review comment:
You use `Structured Streaming` below. Either `Structured Streaming` or `structured streaming` are okay, just better to keep consistent.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] xuanyuanking commented on a change in pull request #33683: [SPARK-36041][SS][DOCS] Introduce the RocksDBStateStoreProvider in the programming guide
Posted by GitBox <gi...@apache.org>.
xuanyuanking commented on a change in pull request #33683:
URL: https://github.com/apache/spark/pull/33683#discussion_r689705878
##########
File path: docs/structured-streaming-programming-guide.md
##########
@@ -1792,7 +1792,85 @@ hence the number is not same as the number of original input rows. You'd like to
There's a known workaround: split your streaming query into multiple queries per stateful operator, and ensure
end-to-end exactly once per query. Ensuring end-to-end exactly once for the last query is optional.
-### State Store and task locality
+### State Store
+
+State store is a versioned key-value store which provides both read and write operations. In
+structured streaming, we use the state store provider to handle the stateful operations across
+batches. There are two built-in state store provider implementations. End users can also implement
+their own state store provider by extending StateStoreProvider interface.
+
+#### HDFS state store provider
+
+The HDFS backend state store provider is the default implementation of [[StateStoreProvider]] and
+[[StateStore]] in which all the data is stated in memory map in the first stage, and then backed
+by files in an HDFS-compatible file system. All updates to the store have to be done in sets
+transactionally, and each set of updates increments the store's version. These versions can be
+used to re-execute the updates (by retries in RDD operations) on the correct version of the store,
+and regenerate the store version.
+
+#### RocksDB state store implementation
+
+As of Spark 3.2, we add a new built-in state store implementation, RocksDB state store provider.
+
+If you have stateful operations in your streaming query (for example, streaming aggregation,
+streaming dropDuplicates, stream-stream joins, mapGroupsWithState, or flatMapGroupsWithState)
+and you want to maintain millions of keys in the state, then you may face issues related to large
+JVM garbage collection (GC) pauses causing high variations in the micro-batch processing times.
+This occurs because, by the implementation of HDFSBackedStateStore, the state data is maintained
+in the JVM memory of the executors and large number of state objects puts memory pressure on the
+JVM causing high GC pauses.
+
+In such cases, you can choose to use a more optimized state management solution based on
+[RocksDB](https://rocksdb.org/). Rather than keeping the state in the JVM memory, this solution
+uses RocksDB to efficiently manage the state in the native memory and the local disk. Furthermore,
+any changes to this state are automatically saved by Structured Streaming to the checkpoint
+location you have provided, thus providing full fault-tolerance guarantees (the same as default
+state management).
+
+To enable the new build-in state store implementation, set `spark.sql.streaming.stateStore.providerClass`
+to `org.apache.spark.sql.execution.streaming.state.RocksDBStateStoreProvider`.
+
+Here are the configs regarding to RocksDB instance of the state store provider:
+
+<table class="table">
+ <tr>
+ <th>Config Name</th>
+ <th>Description</th>
+ <th>Default Value</th>
+ </tr>
+ <tr>
+ <td>spark.sql.streaming.stateStore.rocksdb.compactOnCommit</td>
+ <td>Whether we perform a range compaction of RocksDB instance for commit operation</td>
+ <td>False</td>
+ </tr>
+ <tr>
+ <td>spark.sql.streaming.stateStore.rocksdb.blockSizeKB</td>
+ <td>Approximate size in KB of user data packed per block for a RocksDB BlockBasedTable, which is a RocksDB's default SST file format.</td>
+ <td>4</td>
+ </tr>
+ <tr>
+ <td>spark.sql.streaming.stateStore.rocksdb.blockCacheSizeMB</td>
+ <td>The size capacity in MB for a cache of blocks.</td>
+ <td>8</td>
+ </tr>
+ <tr>
+ <td>spark.sql.streaming.stateStore.rocksdb.blockCacheSizeMB</td>
+ <td>The size capacity in MB for a cache of blocks.</td>
+ <td>8</td>
Review comment:
Ah yea, thanks! done in 2de0e24
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #33683: [SPARK-36041][SS][DOCS] Introduce the RocksDBStateStoreProvider in the programming guide
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #33683:
URL: https://github.com/apache/spark/pull/33683#issuecomment-899224072
**[Test build #142488 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/142488/testReport)** for PR 33683 at commit [`7de4b21`](https://github.com/apache/spark/commit/7de4b21d40f3bff2235b1deb9af4a3becb1581f6).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] xuanyuanking edited a comment on pull request #33683: [SPARK-36041][SS][DOCS] Introduce the RocksDBStateStoreProvider in the programming guide
Posted by GitBox <gi...@apache.org>.
xuanyuanking edited a comment on pull request #33683:
URL: https://github.com/apache/spark/pull/33683#issuecomment-899084300
Ah yes, I forgot to push. Sorry for the late, I’ll update this soon!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #33683: [SPARK-36041][SS][DOCS] Introduce the RocksDBStateStoreProvider in the programming guide
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #33683:
URL: https://github.com/apache/spark/pull/33683#issuecomment-899685287
**[Test build #142517 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/142517/testReport)** for PR 33683 at commit [`2de0e24`](https://github.com/apache/spark/commit/2de0e249f215340c8166256c7dfe6dcee71b7087).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #33683: [SPARK-36041][SS][DOCS] Introduce the RocksDBStateStoreProvider in the programming guide
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #33683:
URL: https://github.com/apache/spark/pull/33683#issuecomment-899254331
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/46987/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #33683: [SPARK-36041][SS][DOCS] Introduce the RocksDBStateStoreProvider in the programming guide
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #33683:
URL: https://github.com/apache/spark/pull/33683#issuecomment-899694143
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #33683: [SPARK-36041][SS][DOCS] Introduce the RocksDBStateStoreProvider in the programming guide
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #33683:
URL: https://github.com/apache/spark/pull/33683#issuecomment-894972160
**[Test build #142209 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/142209/testReport)** for PR 33683 at commit [`09f6aeb`](https://github.com/apache/spark/commit/09f6aeb529ee390b2e6c61c9e780fe41b89fc41c).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #33683: [SPARK-36041][SS][DOCS] Introduce the RocksDBStateStoreProvider in the programming guide
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #33683:
URL: https://github.com/apache/spark/pull/33683#issuecomment-899685287
**[Test build #142517 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/142517/testReport)** for PR 33683 at commit [`2de0e24`](https://github.com/apache/spark/commit/2de0e249f215340c8166256c7dfe6dcee71b7087).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] xuanyuanking commented on a change in pull request #33683: [SPARK-36041][SS][DOCS] Introduce the RocksDBStateStoreProvider in the programming guide
Posted by GitBox <gi...@apache.org>.
xuanyuanking commented on a change in pull request #33683:
URL: https://github.com/apache/spark/pull/33683#discussion_r689705742
##########
File path: docs/structured-streaming-programming-guide.md
##########
@@ -1792,7 +1792,85 @@ hence the number is not same as the number of original input rows. You'd like to
There's a known workaround: split your streaming query into multiple queries per stateful operator, and ensure
end-to-end exactly once per query. Ensuring end-to-end exactly once for the last query is optional.
-### State Store and task locality
+### State Store
+
+State store is a versioned key-value store which provides both read and write operations. In
+structured streaming, we use the state store provider to handle the stateful operations across
+batches. There are two built-in state store provider implementations. End users can also implement
+their own state store provider by extending StateStoreProvider interface.
+
+#### HDFS state store provider
+
+The HDFS backend state store provider is the default implementation of [[StateStoreProvider]] and
+[[StateStore]] in which all the data is stated in memory map in the first stage, and then backed
Review comment:
Thanks, done in 2de0e24
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #33683: [SPARK-36041][SS][DOCS] Introduce the RocksDBStateStoreProvider in the programming guide
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #33683:
URL: https://github.com/apache/spark/pull/33683#issuecomment-899155028
**[Test build #142475 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/142475/testReport)** for PR 33683 at commit [`d026ea0`](https://github.com/apache/spark/commit/d026ea05c0ff02a8eb3b35f7a7d65feff91bbdf9).
* This patch passes all tests.
* This patch merges cleanly.
* This patch adds no public classes.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] viirya commented on a change in pull request #33683: [SPARK-36041][SS][DOCS] Introduce the RocksDBStateStoreProvider in the programming guide
Posted by GitBox <gi...@apache.org>.
viirya commented on a change in pull request #33683:
URL: https://github.com/apache/spark/pull/33683#discussion_r689246669
##########
File path: docs/structured-streaming-programming-guide.md
##########
@@ -1792,7 +1792,85 @@ hence the number is not same as the number of original input rows. You'd like to
There's a known workaround: split your streaming query into multiple queries per stateful operator, and ensure
end-to-end exactly once per query. Ensuring end-to-end exactly once for the last query is optional.
-### State Store and task locality
+### State Store
+
+State store is a versioned key-value store which provides both read and write operations. In
+structured streaming, we use the state store provider to handle the stateful operations across
+batches. There are two built-in state store provider implementations. End users can also implement
+their own state store provider by extending StateStoreProvider interface.
+
+#### HDFS state store provider
+
+The HDFS backend state store provider is the default implementation of [[StateStoreProvider]] and
+[[StateStore]] in which all the data is stated in memory map in the first stage, and then backed
+by files in an HDFS-compatible file system. All updates to the store have to be done in sets
+transactionally, and each set of updates increments the store's version. These versions can be
+used to re-execute the updates (by retries in RDD operations) on the correct version of the store,
+and regenerate the store version.
+
+#### RocksDB state store implementation
+
+As of Spark 3.2, we add a new built-in state store implementation, RocksDB state store provider.
+
+If you have stateful operations in your streaming query (for example, streaming aggregation,
+streaming dropDuplicates, stream-stream joins, mapGroupsWithState, or flatMapGroupsWithState)
+and you want to maintain millions of keys in the state, then you may face issues related to large
+JVM garbage collection (GC) pauses causing high variations in the micro-batch processing times.
+This occurs because, by the implementation of HDFSBackedStateStore, the state data is maintained
+in the JVM memory of the executors and large number of state objects puts memory pressure on the
+JVM causing high GC pauses.
+
+In such cases, you can choose to use a more optimized state management solution based on
+[RocksDB](https://rocksdb.org/). Rather than keeping the state in the JVM memory, this solution
+uses RocksDB to efficiently manage the state in the native memory and the local disk. Furthermore,
+any changes to this state are automatically saved by Structured Streaming to the checkpoint
+location you have provided, thus providing full fault-tolerance guarantees (the same as default
+state management).
+
+To enable the new build-in state store implementation, set `spark.sql.streaming.stateStore.providerClass`
+to `org.apache.spark.sql.execution.streaming.state.RocksDBStateStoreProvider`.
+
+Here are the configs regarding to RocksDB instance of the state store provider:
+
+<table class="table">
+ <tr>
+ <th>Config Name</th>
+ <th>Description</th>
+ <th>Default Value</th>
+ </tr>
+ <tr>
+ <td>spark.sql.streaming.stateStore.rocksdb.compactOnCommit</td>
+ <td>Whether we perform a range compaction of RocksDB instance for commit operation</td>
+ <td>False</td>
+ </tr>
+ <tr>
+ <td>spark.sql.streaming.stateStore.rocksdb.blockSizeKB</td>
+ <td>Approximate size in KB of user data packed per block for a RocksDB BlockBasedTable, which is a RocksDB's default SST file format.</td>
+ <td>4</td>
+ </tr>
+ <tr>
+ <td>spark.sql.streaming.stateStore.rocksdb.blockCacheSizeMB</td>
+ <td>The size capacity in MB for a cache of blocks.</td>
+ <td>8</td>
+ </tr>
+ <tr>
+ <td>spark.sql.streaming.stateStore.rocksdb.blockCacheSizeMB</td>
+ <td>The size capacity in MB for a cache of blocks.</td>
+ <td>8</td>
Review comment:
Is `spark.sql.streaming.stateStore.rocksdb.blockCacheSizeMB` duplicate?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] xuanyuanking commented on a change in pull request #33683: [SPARK-36041][SS][DOCS] Introduce the RocksDBStateStoreProvider in the programming guide
Posted by GitBox <gi...@apache.org>.
xuanyuanking commented on a change in pull request #33683:
URL: https://github.com/apache/spark/pull/33683#discussion_r689169372
##########
File path: docs/structured-streaming-programming-guide.md
##########
@@ -1814,6 +1814,23 @@ Specifically for built-in HDFS state store provider, users can check the state s
it is best if cache missing count is minimized that means Spark won't waste too much time on loading checkpointed state.
User can increase Spark locality waiting configurations to avoid loading state store providers in different executors across batches.
+### RocksDB state store implementation
+
+As of Spark 3.2, we add a new build-in state store implementation, RocksDB state store provider.
+
+The current build-in HDFS state store provider has two major drawbacks:
+
+* The amount of state that can be maintained is limited by the heap size of the executors
+* State expiration by watermark and/or timeouts require full scans over all the data
+
+The RocksDB-based State Store implementation can address these drawbacks:
+
+* RocksDB can serve data from the disk with a configurable amount of non-JVM memory.
+* Sorting keys using the appropriate column should avoid full scans to find the to-be-dropped keys.
+
+To enable the new build-in state store implementation, set `spark.sql.streaming.stateStore.providerClass`
+to `org.apache.spark.sql.execution.streaming.state.RocksDBStateStoreProvider`.
Review comment:
Thanks, done in d026ea0
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] xuanyuanking commented on a change in pull request #33683: [SPARK-36041][SS][DOCS] Introduce the RocksDBStateStoreProvider in the programming guide
Posted by GitBox <gi...@apache.org>.
xuanyuanking commented on a change in pull request #33683:
URL: https://github.com/apache/spark/pull/33683#discussion_r689241527
##########
File path: docs/structured-streaming-programming-guide.md
##########
@@ -1814,6 +1814,82 @@ Specifically for built-in HDFS state store provider, users can check the state s
it is best if cache missing count is minimized that means Spark won't waste too much time on loading checkpointed state.
User can increase Spark locality waiting configurations to avoid loading state store providers in different executors across batches.
+### State Store
+
+State store is a versioned key-value store which provides both read and write operations. In
+structured streaming, we use the state store provider to handle the state store operations crossing
Review comment:
Both were done in 7de4b21. Thanks!
##########
File path: docs/structured-streaming-programming-guide.md
##########
@@ -1814,6 +1814,82 @@ Specifically for built-in HDFS state store provider, users can check the state s
it is best if cache missing count is minimized that means Spark won't waste too much time on loading checkpointed state.
User can increase Spark locality waiting configurations to avoid loading state store providers in different executors across batches.
+### State Store
+
+State store is a versioned key-value store which provides both read and write operations. In
+structured streaming, we use the state store provider to handle the state store operations crossing
+batches. There are two build-in state store provider implementations. End users can also implement
+their own state store provider by extending StateStoreProvider interface.
+
+#### HDFS state store provider
+
+The HDFS backend state store provider is the default implementation of [[StateStoreProvider]] and
+[[StateStore]] in which all the data is backed by files in an HDFS-compatible file system. All
+updates to the store has to be done in sets transactionally, and each set of updates increments
+the store's version. These versions can be used to re-execute the updates (by retries in RDD
+operations) on the correct version of the store, and regenerate the store version.
+
+### RocksDB state store implementation
+
+As of Spark 3.2, we add a new build-in state store implementation, RocksDB state store provider.
Review comment:
Copy, done in 7de4b21
##########
File path: docs/structured-streaming-programming-guide.md
##########
@@ -1814,6 +1814,82 @@ Specifically for built-in HDFS state store provider, users can check the state s
it is best if cache missing count is minimized that means Spark won't waste too much time on loading checkpointed state.
User can increase Spark locality waiting configurations to avoid loading state store providers in different executors across batches.
+### State Store
+
+State store is a versioned key-value store which provides both read and write operations. In
+structured streaming, we use the state store provider to handle the state store operations crossing
+batches. There are two build-in state store provider implementations. End users can also implement
+their own state store provider by extending StateStoreProvider interface.
+
+#### HDFS state store provider
+
+The HDFS backend state store provider is the default implementation of [[StateStoreProvider]] and
+[[StateStore]] in which all the data is backed by files in an HDFS-compatible file system. All
+updates to the store has to be done in sets transactionally, and each set of updates increments
+the store's version. These versions can be used to re-execute the updates (by retries in RDD
+operations) on the correct version of the store, and regenerate the store version.
+
+### RocksDB state store implementation
+
+As of Spark 3.2, we add a new build-in state store implementation, RocksDB state store provider.
+
+If you have stateful operations in your streaming query (for example, streaming aggregation,
+streaming dropDuplicates, stream-stream joins, mapGroupsWithState, or flatMapGroupsWithState)
+and you want to maintain millions of keys in the state, then you may face issues related to large
+JVM garbage collection (GC) pauses causing high variations in the micro-batch processing times.
+This occurs because, by default, the state data is maintained in the JVM memory of the executors
Review comment:
Yep, done in 7de4b21
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #33683: [SPARK-36041][SS][DOCS] Introduce the RocksDBStateStoreProvider in the programming guide
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #33683:
URL: https://github.com/apache/spark/pull/33683#issuecomment-899157737
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/46981/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #33683: [SPARK-36041][SS][DOCS] Introduce the RocksDBStateStoreProvider in the programming guide
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #33683:
URL: https://github.com/apache/spark/pull/33683#issuecomment-899135775
**[Test build #142473 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/142473/testReport)** for PR 33683 at commit [`dec0bae`](https://github.com/apache/spark/commit/dec0baee2f9c9981e60945716cbc1ba91545c031).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #33683: [SPARK-36041][SS][DOCS] Introduce the RocksDBStateStoreProvider in the programming guide
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #33683:
URL: https://github.com/apache/spark/pull/33683#issuecomment-899712154
Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/47018/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] xuanyuanking commented on a change in pull request #33683: [SPARK-36041][SS][DOCS] Introduce the RocksDBStateStoreProvider in the programming guide
Posted by GitBox <gi...@apache.org>.
xuanyuanking commented on a change in pull request #33683:
URL: https://github.com/apache/spark/pull/33683#discussion_r688823506
##########
File path: docs/structured-streaming-programming-guide.md
##########
@@ -1814,6 +1814,23 @@ Specifically for built-in HDFS state store provider, users can check the state s
it is best if cache missing count is minimized that means Spark won't waste too much time on loading checkpointed state.
User can increase Spark locality waiting configurations to avoid loading state store providers in different executors across batches.
+### RocksDB state store implementation
+
+As of Spark 3.2, we add a new build-in state store implementation, RocksDB state store provider.
+
+The current build-in HDFS state store provider has two major drawbacks:
+
+* The amount of state that can be maintained is limited by the heap size of the executors
+* State expiration by watermark and/or timeouts require full scans over all the data
+
+The RocksDB-based State Store implementation can address these drawbacks:
+
+* RocksDB can serve data from the disk with a configurable amount of non-JVM memory.
+* Sorting keys using the appropriate column should avoid full scans to find the to-be-dropped keys.
Review comment:
Make sense. I rewrite this part. Please take a look.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #33683: [SPARK-36041][SS][DOCS] Introduce the RocksDBStateStoreProvider in the programming guide
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #33683:
URL: https://github.com/apache/spark/pull/33683#issuecomment-899155166
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/142475/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] viirya commented on pull request #33683: [SPARK-36041][SS][DOCS] Introduce the RocksDBStateStoreProvider in the programming guide
Posted by GitBox <gi...@apache.org>.
viirya commented on pull request #33683:
URL: https://github.com/apache/spark/pull/33683#issuecomment-899677700
I'll merge this after CI passes.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #33683: [SPARK-36041][SS][DOCS] Introduce the RocksDBStateStoreProvider in the programming guide
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #33683:
URL: https://github.com/apache/spark/pull/33683#issuecomment-899747120
Kubernetes integration test status failure
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/47018/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] xuanyuanking commented on a change in pull request #33683: [SPARK-36041][SS][DOCS] Introduce the RocksDBStateStoreProvider in the programming guide
Posted by GitBox <gi...@apache.org>.
xuanyuanking commented on a change in pull request #33683:
URL: https://github.com/apache/spark/pull/33683#discussion_r689241802
##########
File path: docs/structured-streaming-programming-guide.md
##########
@@ -1814,6 +1814,82 @@ Specifically for built-in HDFS state store provider, users can check the state s
it is best if cache missing count is minimized that means Spark won't waste too much time on loading checkpointed state.
User can increase Spark locality waiting configurations to avoid loading state store providers in different executors across batches.
+### State Store
+
+State store is a versioned key-value store which provides both read and write operations. In
+structured streaming, we use the state store provider to handle the state store operations crossing
+batches. There are two build-in state store provider implementations. End users can also implement
+their own state store provider by extending StateStoreProvider interface.
+
+#### HDFS state store provider
+
+The HDFS backend state store provider is the default implementation of [[StateStoreProvider]] and
+[[StateStore]] in which all the data is backed by files in an HDFS-compatible file system. All
+updates to the store has to be done in sets transactionally, and each set of updates increments
Review comment:
Thanks, done in 7de4b21
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] gengliangwang commented on a change in pull request #33683: [SPARK-36041][SS][DOCS] Introduce the RocksDBStateStoreProvider in the programming guide
Posted by GitBox <gi...@apache.org>.
gengliangwang commented on a change in pull request #33683:
URL: https://github.com/apache/spark/pull/33683#discussion_r689232945
##########
File path: docs/structured-streaming-programming-guide.md
##########
@@ -1814,6 +1814,82 @@ Specifically for built-in HDFS state store provider, users can check the state s
it is best if cache missing count is minimized that means Spark won't waste too much time on loading checkpointed state.
User can increase Spark locality waiting configurations to avoid loading state store providers in different executors across batches.
+### State Store
+
+State store is a versioned key-value store which provides both read and write operations. In
+structured streaming, we use the state store provider to handle the state store operations crossing
+batches. There are two build-in state store provider implementations. End users can also implement
+their own state store provider by extending StateStoreProvider interface.
+
+#### HDFS state store provider
+
+The HDFS backend state store provider is the default implementation of [[StateStoreProvider]] and
+[[StateStore]] in which all the data is backed by files in an HDFS-compatible file system. All
+updates to the store has to be done in sets transactionally, and each set of updates increments
+the store's version. These versions can be used to re-execute the updates (by retries in RDD
+operations) on the correct version of the store, and regenerate the store version.
+
+### RocksDB state store implementation
Review comment:
Should it be '####' here?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #33683: [SPARK-36041][SS][DOCS] Introduce the RocksDBStateStoreProvider in the programming guide
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #33683:
URL: https://github.com/apache/spark/pull/33683#issuecomment-899232440
**[Test build #142488 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/142488/testReport)** for PR 33683 at commit [`7de4b21`](https://github.com/apache/spark/commit/7de4b21d40f3bff2235b1deb9af4a3becb1581f6).
* This patch passes all tests.
* This patch merges cleanly.
* This patch adds no public classes.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] viirya commented on a change in pull request #33683: [SPARK-36041][SS][DOCS] Introduce the RocksDBStateStoreProvider in the programming guide
Posted by GitBox <gi...@apache.org>.
viirya commented on a change in pull request #33683:
URL: https://github.com/apache/spark/pull/33683#discussion_r689118183
##########
File path: docs/structured-streaming-programming-guide.md
##########
@@ -1814,6 +1814,23 @@ Specifically for built-in HDFS state store provider, users can check the state s
it is best if cache missing count is minimized that means Spark won't waste too much time on loading checkpointed state.
User can increase Spark locality waiting configurations to avoid loading state store providers in different executors across batches.
+### RocksDB state store implementation
Review comment:
Actually we don't mention state store clearly in this doc before. The doc only mentions `spark.sql.streaming.stateStore.providerClass` could be used to set up state store implementation.
How about introducing both state stores? E.g.,
``` markdown
### State Store
#### HDFS state store provider
.....
#### RocksDB state store provider
.....
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #33683: [SPARK-36041][SS][DOCS] Introduce the RocksDBStateStoreProvider in the programming guide
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #33683:
URL: https://github.com/apache/spark/pull/33683#issuecomment-899150258
**[Test build #142475 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/142475/testReport)** for PR 33683 at commit [`d026ea0`](https://github.com/apache/spark/commit/d026ea05c0ff02a8eb3b35f7a7d65feff91bbdf9).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] xuanyuanking commented on pull request #33683: [SPARK-36041][SS][DOCS] Introduce the RocksDBStateStoreProvider in the programming guide
Posted by GitBox <gi...@apache.org>.
xuanyuanking commented on pull request #33683:
URL: https://github.com/apache/spark/pull/33683#issuecomment-894972015
cc @HeartSaVioR
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] gengliangwang commented on a change in pull request #33683: [SPARK-36041][SS][DOCS] Introduce the RocksDBStateStoreProvider in the programming guide
Posted by GitBox <gi...@apache.org>.
gengliangwang commented on a change in pull request #33683:
URL: https://github.com/apache/spark/pull/33683#discussion_r689232329
##########
File path: docs/structured-streaming-programming-guide.md
##########
@@ -1814,6 +1814,82 @@ Specifically for built-in HDFS state store provider, users can check the state s
it is best if cache missing count is minimized that means Spark won't waste too much time on loading checkpointed state.
User can increase Spark locality waiting configurations to avoid loading state store providers in different executors across batches.
+### State Store
+
+State store is a versioned key-value store which provides both read and write operations. In
+structured streaming, we use the state store provider to handle the state store operations crossing
+batches. There are two build-in state store provider implementations. End users can also implement
+their own state store provider by extending StateStoreProvider interface.
+
+#### HDFS state store provider
+
+The HDFS backend state store provider is the default implementation of [[StateStoreProvider]] and
+[[StateStore]] in which all the data is backed by files in an HDFS-compatible file system. All
+updates to the store has to be done in sets transactionally, and each set of updates increments
Review comment:
has => have
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #33683: [SPARK-36041][SS][DOCS] Introduce the RocksDBStateStoreProvider in the programming guide
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #33683:
URL: https://github.com/apache/spark/pull/33683#issuecomment-894976783
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/142209/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #33683: [SPARK-36041][SS][DOCS] Introduce the RocksDBStateStoreProvider in the programming guide
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #33683:
URL: https://github.com/apache/spark/pull/33683#issuecomment-899157737
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] xuanyuanking commented on pull request #33683: [SPARK-36041][SS][DOCS] Introduce the RocksDBStateStoreProvider in the programming guide
Posted by GitBox <gi...@apache.org>.
xuanyuanking commented on pull request #33683:
URL: https://github.com/apache/spark/pull/33683#issuecomment-899219122
```
The section "State Store and task locality" introduces State Store as well.
```
Thanks @gengliangwang for the detailed check! Yes, I moved the `State Store and task locality` as a child section of `State Store`.
@gengliangwang @viirya @HeartSaVioR All comments addressed, sorry for the delay.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #33683: [SPARK-36041][SS][DOCS] Introduce the RocksDBStateStoreProvider in the programming guide
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #33683:
URL: https://github.com/apache/spark/pull/33683#issuecomment-899157720
Kubernetes integration test status success
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/46981/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] xuanyuanking commented on a change in pull request #33683: [SPARK-36041][SS][DOCS] Introduce the RocksDBStateStoreProvider in the programming guide
Posted by GitBox <gi...@apache.org>.
xuanyuanking commented on a change in pull request #33683:
URL: https://github.com/apache/spark/pull/33683#discussion_r689169441
##########
File path: docs/structured-streaming-programming-guide.md
##########
@@ -1814,6 +1814,23 @@ Specifically for built-in HDFS state store provider, users can check the state s
it is best if cache missing count is minimized that means Spark won't waste too much time on loading checkpointed state.
User can increase Spark locality waiting configurations to avoid loading state store providers in different executors across batches.
+### RocksDB state store implementation
Review comment:
Make sense, done in dec0bae
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] xuanyuanking commented on pull request #33683: [SPARK-36041][SS][DOCS] Introduce the RocksDBStateStoreProvider in the programming guide
Posted by GitBox <gi...@apache.org>.
xuanyuanking commented on pull request #33683:
URL: https://github.com/apache/spark/pull/33683#issuecomment-900412727
Thank all!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] xuanyuanking commented on a change in pull request #33683: [SPARK-36041][SS][DOCS] Introduce the RocksDBStateStoreProvider in the programming guide
Posted by GitBox <gi...@apache.org>.
xuanyuanking commented on a change in pull request #33683:
URL: https://github.com/apache/spark/pull/33683#discussion_r689241848
##########
File path: docs/structured-streaming-programming-guide.md
##########
@@ -1814,6 +1814,82 @@ Specifically for built-in HDFS state store provider, users can check the state s
it is best if cache missing count is minimized that means Spark won't waste too much time on loading checkpointed state.
User can increase Spark locality waiting configurations to avoid loading state store providers in different executors across batches.
+### State Store
+
+State store is a versioned key-value store which provides both read and write operations. In
+structured streaming, we use the state store provider to handle the state store operations crossing
+batches. There are two build-in state store provider implementations. End users can also implement
+their own state store provider by extending StateStoreProvider interface.
+
+#### HDFS state store provider
+
+The HDFS backend state store provider is the default implementation of [[StateStoreProvider]] and
+[[StateStore]] in which all the data is backed by files in an HDFS-compatible file system. All
+updates to the store has to be done in sets transactionally, and each set of updates increments
+the store's version. These versions can be used to re-execute the updates (by retries in RDD
+operations) on the correct version of the store, and regenerate the store version.
+
+### RocksDB state store implementation
Review comment:
Yea, nice catch!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] xuanyuanking commented on a change in pull request #33683: [SPARK-36041][SS][DOCS] Introduce the RocksDBStateStoreProvider in the programming guide
Posted by GitBox <gi...@apache.org>.
xuanyuanking commented on a change in pull request #33683:
URL: https://github.com/apache/spark/pull/33683#discussion_r689241474
##########
File path: docs/structured-streaming-programming-guide.md
##########
@@ -1814,6 +1814,82 @@ Specifically for built-in HDFS state store provider, users can check the state s
it is best if cache missing count is minimized that means Spark won't waste too much time on loading checkpointed state.
User can increase Spark locality waiting configurations to avoid loading state store providers in different executors across batches.
+### State Store
+
+State store is a versioned key-value store which provides both read and write operations. In
+structured streaming, we use the state store provider to handle the state store operations crossing
+batches. There are two build-in state store provider implementations. End users can also implement
Review comment:
Thanks done in 7de4b21.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] gengliangwang commented on pull request #33683: [SPARK-36041][SS][DOCS] Introduce the RocksDBStateStoreProvider in the programming guide
Posted by GitBox <gi...@apache.org>.
gengliangwang commented on pull request #33683:
URL: https://github.com/apache/spark/pull/33683#issuecomment-899950598
Late LGTM, thank you @xuanyuanking
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #33683: [SPARK-36041][SS][DOCS] Introduce the RocksDBStateStoreProvider in the programming guide
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #33683:
URL: https://github.com/apache/spark/pull/33683#issuecomment-895027076
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/46722/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] HeartSaVioR commented on a change in pull request #33683: [SPARK-36041][SS][DOCS] Introduce the RocksDBStateStoreProvider in the programming guide
Posted by GitBox <gi...@apache.org>.
HeartSaVioR commented on a change in pull request #33683:
URL: https://github.com/apache/spark/pull/33683#discussion_r684946042
##########
File path: docs/structured-streaming-programming-guide.md
##########
@@ -1814,6 +1814,23 @@ Specifically for built-in HDFS state store provider, users can check the state s
it is best if cache missing count is minimized that means Spark won't waste too much time on loading checkpointed state.
User can increase Spark locality waiting configurations to avoid loading state store providers in different executors across batches.
+### RocksDB state store implementation
+
+As of Spark 3.2, we add a new build-in state store implementation, RocksDB state store provider.
+
+The current build-in HDFS state store provider has two major drawbacks:
+
+* The amount of state that can be maintained is limited by the heap size of the executors
+* State expiration by watermark and/or timeouts require full scans over all the data
+
+The RocksDB-based State Store implementation can address these drawbacks:
+
+* RocksDB can serve data from the disk with a configurable amount of non-JVM memory.
+* Sorting keys using the appropriate column should avoid full scans to find the to-be-dropped keys.
Review comment:
Please correct me if I'm missing; while this could be something we can evaluate and address, this is not true at least for now. We don't distinguish event time field in state store.
Prefix scan is the only thing we leverage sorted key for now.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] xuanyuanking commented on pull request #33683: [SPARK-36041][SS][DOCS] Introduce the RocksDBStateStoreProvider in the programming guide
Posted by GitBox <gi...@apache.org>.
xuanyuanking commented on pull request #33683:
URL: https://github.com/apache/spark/pull/33683#issuecomment-900412727
Thank all!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #33683: [SPARK-36041][SS][DOCS] Introduce the RocksDBStateStoreProvider in the programming guide
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #33683:
URL: https://github.com/apache/spark/pull/33683#issuecomment-894996488
Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/46722/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] xuanyuanking commented on a change in pull request #33683: [SPARK-36041][SS][DOCS] Introduce the RocksDBStateStoreProvider in the programming guide
Posted by GitBox <gi...@apache.org>.
xuanyuanking commented on a change in pull request #33683:
URL: https://github.com/apache/spark/pull/33683#discussion_r689706005
##########
File path: docs/structured-streaming-programming-guide.md
##########
@@ -1792,7 +1792,85 @@ hence the number is not same as the number of original input rows. You'd like to
There's a known workaround: split your streaming query into multiple queries per stateful operator, and ensure
end-to-end exactly once per query. Ensuring end-to-end exactly once for the last query is optional.
-### State Store and task locality
+### State Store
+
+State store is a versioned key-value store which provides both read and write operations. In
+structured streaming, we use the state store provider to handle the stateful operations across
Review comment:
Make sense, done in 2de0e24
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #33683: [SPARK-36041][SS][DOCS] Introduce the RocksDBStateStoreProvider in the programming guide
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #33683:
URL: https://github.com/apache/spark/pull/33683#issuecomment-899155166
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/142475/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] HeartSaVioR commented on a change in pull request #33683: [SPARK-36041][SS][DOCS] Introduce the RocksDBStateStoreProvider in the programming guide
Posted by GitBox <gi...@apache.org>.
HeartSaVioR commented on a change in pull request #33683:
URL: https://github.com/apache/spark/pull/33683#discussion_r689203232
##########
File path: docs/structured-streaming-programming-guide.md
##########
@@ -1814,6 +1814,82 @@ Specifically for built-in HDFS state store provider, users can check the state s
it is best if cache missing count is minimized that means Spark won't waste too much time on loading checkpointed state.
User can increase Spark locality waiting configurations to avoid loading state store providers in different executors across batches.
+### State Store
+
+State store is a versioned key-value store which provides both read and write operations. In
+structured streaming, we use the state store provider to handle the state store operations crossing
+batches. There are two build-in state store provider implementations. End users can also implement
+their own state store provider by extending StateStoreProvider interface.
+
+#### HDFS state store provider
+
+The HDFS backend state store provider is the default implementation of [[StateStoreProvider]] and
+[[StateStore]] in which all the data is backed by files in an HDFS-compatible file system. All
+updates to the store has to be done in sets transactionally, and each set of updates increments
+the store's version. These versions can be used to re-execute the updates (by retries in RDD
+operations) on the correct version of the store, and regenerate the store version.
+
+### RocksDB state store implementation
+
+As of Spark 3.2, we add a new build-in state store implementation, RocksDB state store provider.
Review comment:
nit: built-in
##########
File path: docs/structured-streaming-programming-guide.md
##########
@@ -1814,6 +1814,82 @@ Specifically for built-in HDFS state store provider, users can check the state s
it is best if cache missing count is minimized that means Spark won't waste too much time on loading checkpointed state.
User can increase Spark locality waiting configurations to avoid loading state store providers in different executors across batches.
+### State Store
+
+State store is a versioned key-value store which provides both read and write operations. In
+structured streaming, we use the state store provider to handle the state store operations crossing
+batches. There are two build-in state store provider implementations. End users can also implement
Review comment:
nit: built-in
##########
File path: docs/structured-streaming-programming-guide.md
##########
@@ -1814,6 +1814,82 @@ Specifically for built-in HDFS state store provider, users can check the state s
it is best if cache missing count is minimized that means Spark won't waste too much time on loading checkpointed state.
User can increase Spark locality waiting configurations to avoid loading state store providers in different executors across batches.
+### State Store
+
+State store is a versioned key-value store which provides both read and write operations. In
+structured streaming, we use the state store provider to handle the state store operations crossing
Review comment:
crossing -> across
##########
File path: docs/structured-streaming-programming-guide.md
##########
@@ -1814,6 +1814,82 @@ Specifically for built-in HDFS state store provider, users can check the state s
it is best if cache missing count is minimized that means Spark won't waste too much time on loading checkpointed state.
User can increase Spark locality waiting configurations to avoid loading state store providers in different executors across batches.
+### State Store
+
+State store is a versioned key-value store which provides both read and write operations. In
+structured streaming, we use the state store provider to handle the state store operations crossing
+batches. There are two build-in state store provider implementations. End users can also implement
+their own state store provider by extending StateStoreProvider interface.
+
+#### HDFS state store provider
+
+The HDFS backend state store provider is the default implementation of [[StateStoreProvider]] and
+[[StateStore]] in which all the data is backed by files in an HDFS-compatible file system. All
+updates to the store has to be done in sets transactionally, and each set of updates increments
+the store's version. These versions can be used to re-execute the updates (by retries in RDD
+operations) on the correct version of the store, and regenerate the store version.
+
+### RocksDB state store implementation
+
+As of Spark 3.2, we add a new build-in state store implementation, RocksDB state store provider.
+
+If you have stateful operations in your streaming query (for example, streaming aggregation,
+streaming dropDuplicates, stream-stream joins, mapGroupsWithState, or flatMapGroupsWithState)
+and you want to maintain millions of keys in the state, then you may face issues related to large
+JVM garbage collection (GC) pauses causing high variations in the micro-batch processing times.
+This occurs because, by default, the state data is maintained in the JVM memory of the executors
Review comment:
Let's directly mention HDFSBackedStateStore here. default can be changed anytime.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] viirya commented on pull request #33683: [SPARK-36041][SS][DOCS] Introduce the RocksDBStateStoreProvider in the programming guide
Posted by GitBox <gi...@apache.org>.
viirya commented on pull request #33683:
URL: https://github.com/apache/spark/pull/33683#issuecomment-899764670
Thanks. Merging to master/3.2.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #33683: [SPARK-36041][SS][DOCS] Introduce the RocksDBStateStoreProvider in the programming guide
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #33683:
URL: https://github.com/apache/spark/pull/33683#issuecomment-899145052
Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/46981/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] viirya commented on a change in pull request #33683: [SPARK-36041][SS][DOCS] Introduce the RocksDBStateStoreProvider in the programming guide
Posted by GitBox <gi...@apache.org>.
viirya commented on a change in pull request #33683:
URL: https://github.com/apache/spark/pull/33683#discussion_r689209780
##########
File path: docs/structured-streaming-programming-guide.md
##########
@@ -1814,6 +1814,82 @@ Specifically for built-in HDFS state store provider, users can check the state s
it is best if cache missing count is minimized that means Spark won't waste too much time on loading checkpointed state.
User can increase Spark locality waiting configurations to avoid loading state store providers in different executors across batches.
+### State Store
+
+State store is a versioned key-value store which provides both read and write operations. In
+structured streaming, we use the state store provider to handle the state store operations crossing
Review comment:
state store operations -> stateful operations
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] xuanyuanking commented on a change in pull request #33683: [SPARK-36041][SS][DOCS] Introduce the RocksDBStateStoreProvider in the programming guide
Posted by GitBox <gi...@apache.org>.
xuanyuanking commented on a change in pull request #33683:
URL: https://github.com/apache/spark/pull/33683#discussion_r689241748
##########
File path: docs/structured-streaming-programming-guide.md
##########
@@ -1814,6 +1814,82 @@ Specifically for built-in HDFS state store provider, users can check the state s
it is best if cache missing count is minimized that means Spark won't waste too much time on loading checkpointed state.
User can increase Spark locality waiting configurations to avoid loading state store providers in different executors across batches.
+### State Store
+
+State store is a versioned key-value store which provides both read and write operations. In
+structured streaming, we use the state store provider to handle the state store operations crossing
+batches. There are two build-in state store provider implementations. End users can also implement
+their own state store provider by extending StateStoreProvider interface.
+
+#### HDFS state store provider
+
+The HDFS backend state store provider is the default implementation of [[StateStoreProvider]] and
+[[StateStore]] in which all the data is backed by files in an HDFS-compatible file system. All
Review comment:
Make sense, we mentioned this drawback in the RocksDB one, but it should also be mentioned here.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #33683: [SPARK-36041][SS][DOCS] Introduce the RocksDBStateStoreProvider in the programming guide
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #33683:
URL: https://github.com/apache/spark/pull/33683#issuecomment-899693897
**[Test build #142517 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/142517/testReport)** for PR 33683 at commit [`2de0e24`](https://github.com/apache/spark/commit/2de0e249f215340c8166256c7dfe6dcee71b7087).
* This patch passes all tests.
* This patch merges cleanly.
* This patch adds no public classes.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #33683: [SPARK-36041][SS][DOCS] Introduce the RocksDBStateStoreProvider in the programming guide
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #33683:
URL: https://github.com/apache/spark/pull/33683#issuecomment-899759254
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/47018/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #33683: [SPARK-36041][SS][DOCS] Introduce the RocksDBStateStoreProvider in the programming guide
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #33683:
URL: https://github.com/apache/spark/pull/33683#issuecomment-899694143
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/142517/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #33683: [SPARK-36041][SS][DOCS] Introduce the RocksDBStateStoreProvider in the programming guide
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #33683:
URL: https://github.com/apache/spark/pull/33683#issuecomment-899135775
**[Test build #142473 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/142473/testReport)** for PR 33683 at commit [`dec0bae`](https://github.com/apache/spark/commit/dec0baee2f9c9981e60945716cbc1ba91545c031).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #33683: [SPARK-36041][SS][DOCS] Introduce the RocksDBStateStoreProvider in the programming guide
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #33683:
URL: https://github.com/apache/spark/pull/33683#issuecomment-899152554
Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/46983/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] HeartSaVioR commented on a change in pull request #33683: [SPARK-36041][SS][DOCS] Introduce the RocksDBStateStoreProvider in the programming guide
Posted by GitBox <gi...@apache.org>.
HeartSaVioR commented on a change in pull request #33683:
URL: https://github.com/apache/spark/pull/33683#discussion_r688837959
##########
File path: docs/structured-streaming-programming-guide.md
##########
@@ -1814,6 +1814,23 @@ Specifically for built-in HDFS state store provider, users can check the state s
it is best if cache missing count is minimized that means Spark won't waste too much time on loading checkpointed state.
User can increase Spark locality waiting configurations to avoid loading state store providers in different executors across batches.
+### RocksDB state store implementation
+
+As of Spark 3.2, we add a new build-in state store implementation, RocksDB state store provider.
+
+The current build-in HDFS state store provider has two major drawbacks:
+
+* The amount of state that can be maintained is limited by the heap size of the executors
+* State expiration by watermark and/or timeouts require full scans over all the data
+
+The RocksDB-based State Store implementation can address these drawbacks:
+
+* RocksDB can serve data from the disk with a configurable amount of non-JVM memory.
+* Sorting keys using the appropriate column should avoid full scans to find the to-be-dropped keys.
Review comment:
@xuanyuanking
Looks like you missed pushing new commit(s). ;)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #33683: [SPARK-36041][SS][DOCS] Introduce the RocksDBStateStoreProvider in the programming guide
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #33683:
URL: https://github.com/apache/spark/pull/33683#issuecomment-895014798
Kubernetes integration test status success
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/46722/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] xuanyuanking commented on pull request #33683: [SPARK-36041][SS][DOCS] Introduce the RocksDBStateStoreProvider in the programming guide
Posted by GitBox <gi...@apache.org>.
xuanyuanking commented on pull request #33683:
URL: https://github.com/apache/spark/pull/33683#issuecomment-899084300
Ah yes, I forgot push on Friday. Sorry for the late, I’ll update this soon!
Liang-Chi Hsieh ***@***.***>于2021年8月15日 周日10:15写道：
> ***@***.**** commented on this pull request.
>
> BTW, I heard it is going to cut RC1 in Monday. If we want to have this
> before RC1, seems we need to update it and merge it asap?
>
> —
> You are receiving this because you were mentioned.
> Reply to this email directly, view it on GitHub
> <https://github.com/apache/spark/pull/33683#pullrequestreview-730205877>,
> or unsubscribe
> <https://github.com/notifications/unsubscribe-auth/ABE4DZPSDKBKBGEYIHBWJXTT47Y4BANCNFSM5BZKX6GA>
> .
> Triage notifications on the go with GitHub Mobile for iOS
> <https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675>
> or Android
> <https://play.google.com/store/apps/details?id=com.github.android&utm_campaign=notification-email>
> .
>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #33683: [SPARK-36041][SS][DOCS] Introduce the RocksDBStateStoreProvider in the programming guide
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #33683:
URL: https://github.com/apache/spark/pull/33683#issuecomment-894976783
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #33683: [SPARK-36041][SS][DOCS] Introduce the RocksDBStateStoreProvider in the programming guide
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #33683:
URL: https://github.com/apache/spark/pull/33683#issuecomment-899138151
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/142473/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] HeartSaVioR commented on a change in pull request #33683: [SPARK-36041][SS][DOCS] Introduce the RocksDBStateStoreProvider in the programming guide
Posted by GitBox <gi...@apache.org>.
HeartSaVioR commented on a change in pull request #33683:
URL: https://github.com/apache/spark/pull/33683#discussion_r684990617
##########
File path: docs/structured-streaming-programming-guide.md
##########
@@ -1814,6 +1814,23 @@ Specifically for built-in HDFS state store provider, users can check the state s
it is best if cache missing count is minimized that means Spark won't waste too much time on loading checkpointed state.
User can increase Spark locality waiting configurations to avoid loading state store providers in different executors across batches.
+### RocksDB state store implementation
+
+As of Spark 3.2, we add a new build-in state store implementation, RocksDB state store provider.
+
+The current build-in HDFS state store provider has two major drawbacks:
+
+* The amount of state that can be maintained is limited by the heap size of the executors
+* State expiration by watermark and/or timeouts require full scans over all the data
+
+The RocksDB-based State Store implementation can address these drawbacks:
+
+* RocksDB can serve data from the disk with a configurable amount of non-JVM memory.
+* Sorting keys using the appropriate column should avoid full scans to find the to-be-dropped keys.
+
+To enable the new build-in state store implementation, set `spark.sql.streaming.stateStore.providerClass`
+to `org.apache.spark.sql.execution.streaming.state.RocksDBStateStoreProvider`.
Review comment:
Additionally, it would be nice if we explain the configs regarding RocksDB state store provider, as these configs are not a part of SQLConf and are not documented.
```
// Configuration that specifies whether to compact the RocksDB data every time data is committed
private val COMPACT_ON_COMMIT_CONF = ConfEntry("compactOnCommit", "false")
private val PAUSE_BG_WORK_FOR_COMMIT_CONF = ConfEntry("pauseBackgroundWorkForCommit", "true")
private val BLOCK_SIZE_KB_CONF = ConfEntry("blockSizeKB", "4")
private val BLOCK_CACHE_SIZE_MB_CONF = ConfEntry("blockCacheSizeMB", "8")
private val LOCK_ACQUIRE_TIMEOUT_MS_CONF = ConfEntry("lockAcquireTimeoutMs", "60000")
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #33683: [SPARK-36041][SS][DOCS] Introduce the RocksDBStateStoreProvider in the programming guide
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #33683:
URL: https://github.com/apache/spark/pull/33683#issuecomment-894972160
**[Test build #142209 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/142209/testReport)** for PR 33683 at commit [`09f6aeb`](https://github.com/apache/spark/commit/09f6aeb529ee390b2e6c61c9e780fe41b89fc41c).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] viirya commented on a change in pull request #33683: [SPARK-36041][SS][DOCS] Introduce the RocksDBStateStoreProvider in the programming guide
Posted by GitBox <gi...@apache.org>.
viirya commented on a change in pull request #33683:
URL: https://github.com/apache/spark/pull/33683#discussion_r689245759
##########
File path: docs/structured-streaming-programming-guide.md
##########
@@ -1792,7 +1792,85 @@ hence the number is not same as the number of original input rows. You'd like to
There's a known workaround: split your streaming query into multiple queries per stateful operator, and ensure
end-to-end exactly once per query. Ensuring end-to-end exactly once for the last query is optional.
-### State Store and task locality
+### State Store
+
+State store is a versioned key-value store which provides both read and write operations. In
+structured streaming, we use the state store provider to handle the stateful operations across
+batches. There are two built-in state store provider implementations. End users can also implement
+their own state store provider by extending StateStoreProvider interface.
+
+#### HDFS state store provider
+
+The HDFS backend state store provider is the default implementation of [[StateStoreProvider]] and
+[[StateStore]] in which all the data is stated in memory map in the first stage, and then backed
Review comment:
stated? stored?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #33683: [SPARK-36041][SS][DOCS] Introduce the RocksDBStateStoreProvider in the programming guide
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #33683:
URL: https://github.com/apache/spark/pull/33683#issuecomment-899233222
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/142488/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #33683: [SPARK-36041][SS][DOCS] Introduce the RocksDBStateStoreProvider in the programming guide
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #33683:
URL: https://github.com/apache/spark/pull/33683#issuecomment-899224072
**[Test build #142488 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/142488/testReport)** for PR 33683 at commit [`7de4b21`](https://github.com/apache/spark/commit/7de4b21d40f3bff2235b1deb9af4a3becb1581f6).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] viirya commented on a change in pull request #33683: [SPARK-36041][SS][DOCS] Introduce the RocksDBStateStoreProvider in the programming guide
Posted by GitBox <gi...@apache.org>.
viirya commented on a change in pull request #33683:
URL: https://github.com/apache/spark/pull/33683#discussion_r689210382
##########
File path: docs/structured-streaming-programming-guide.md
##########
@@ -1814,6 +1814,82 @@ Specifically for built-in HDFS state store provider, users can check the state s
it is best if cache missing count is minimized that means Spark won't waste too much time on loading checkpointed state.
User can increase Spark locality waiting configurations to avoid loading state store providers in different executors across batches.
+### State Store
+
+State store is a versioned key-value store which provides both read and write operations. In
+structured streaming, we use the state store provider to handle the state store operations crossing
+batches. There are two build-in state store provider implementations. End users can also implement
+their own state store provider by extending StateStoreProvider interface.
+
+#### HDFS state store provider
+
+The HDFS backend state store provider is the default implementation of [[StateStoreProvider]] and
+[[StateStore]] in which all the data is backed by files in an HDFS-compatible file system. All
Review comment:
Should we also mention the data is stored in memory map in executors?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #33683: [SPARK-36041][SS][DOCS] Introduce the RocksDBStateStoreProvider in the programming guide
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #33683:
URL: https://github.com/apache/spark/pull/33683#issuecomment-894976646
**[Test build #142209 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/142209/testReport)** for PR 33683 at commit [`09f6aeb`](https://github.com/apache/spark/commit/09f6aeb529ee390b2e6c61c9e780fe41b89fc41c).
* This patch passes all tests.
* This patch merges cleanly.
* This patch adds no public classes.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org