You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@spark.apache.org by do...@apache.org on 2025/01/16 03:47:01 UTC
(spark) branch master updated: [SPARK-50654][SS][TESTS][FOLLOWUP] Fix compile issue of `RocksDBStateStoreCheckpointFormatV2Suite`
This is an automated email from the ASF dual-hosted git repository.
dongjoon pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/spark.git
The following commit(s) were added to refs/heads/master by this push:
new 0ca963337637 [SPARK-50654][SS][TESTS][FOLLOWUP] Fix compile issue of `RocksDBStateStoreCheckpointFormatV2Suite`
0ca963337637 is described below
commit 0ca96333763786ad1409b5ecd0206fe9564739ef
Author: yangjie01 <ya...@baidu.com>
AuthorDate: Wed Jan 15 19:46:48 2025 -0800
[SPARK-50654][SS][TESTS][FOLLOWUP] Fix compile issue of `RocksDBStateStoreCheckpointFormatV2Suite`
### What changes were proposed in this pull request?
This PR replaces `metadata.stateUniqueIds(0)` with `metadata.stateUniqueIds.head(0)` to fix the compilation issue in `RocksDBStateStoreCheckpointFormatV2Suite`. The compilation error occurred because the type of `stateUniqueIds` was changed from `stateUniqueIds: Map[Long, Array[Array[String]]] = Map.empty` to `stateUniqueIds: Option[Map[Long, Array[Array[String]]]] = None` in https://github.com/apache/spark/pull/49278.
### Why are the changes needed?
Fix the compilation error in GitHub Actions:
- https://github.com/apache/spark/actions/runs/12801139497/job/35690169405
- https://github.com/apache/spark/actions/runs/12801160621/job/35690189293
### Does this PR introduce _any_ user-facing change?
No
### How was this patch tested?
- Pass GitHub Actions
- manual check:
```
build/sbt "sql/testOnly *RocksDBStateStoreCheckpointFormatV2Suite"
```
```
[info] Run completed in 1 minute, 33 seconds.
[info] Total number of tests run: 17
[info] Suites: completed 1, aborted 0
[info] Tests: succeeded 17, failed 0, canceled 0, ignored 0, pending 0
[info] All tests passed.
```
### Was this patch authored or co-authored using generative AI tooling?
No
Closes #49523 from LuciferYang/SPARK-50654-FOLLOWUP.
Authored-by: yangjie01 <ya...@baidu.com>
Signed-off-by: Dongjoon Hyun <do...@apache.org>
---
.../streaming/state/RocksDBStateStoreCheckpointFormatV2Suite.scala | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
diff --git a/sql/core/src/test/scala/org/apache/spark/sql/execution/streaming/state/RocksDBStateStoreCheckpointFormatV2Suite.scala b/sql/core/src/test/scala/org/apache/spark/sql/execution/streaming/state/RocksDBStateStoreCheckpointFormatV2Suite.scala
index b5fc88130ef6..d44e2742ad00 100644
--- a/sql/core/src/test/scala/org/apache/spark/sql/execution/streaming/state/RocksDBStateStoreCheckpointFormatV2Suite.scala
+++ b/sql/core/src/test/scala/org/apache/spark/sql/execution/streaming/state/RocksDBStateStoreCheckpointFormatV2Suite.scala
@@ -622,8 +622,8 @@ class RocksDBStateStoreCheckpointFormatV2Suite extends StreamTest
val metadata = commitLog.get(Some(0), Some(1)).map(_._2)
val versionToUniqueIdFromCommitLog = metadata.zipWithIndex.map { case (metadata, idx) =>
- // Use stateUniqueIds(0) because there is only one state operator
- val res2 = metadata.stateUniqueIds(0).map { uniqueIds =>
+ // Use stateUniqueIds.head(0) because there is only one state operator
+ val res2 = metadata.stateUniqueIds.head(0).map { uniqueIds =>
uniqueIds(0)
}
idx -> res2
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@spark.apache.org
For additional commands, e-mail: commits-help@spark.apache.org