You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@spark.apache.org by ur...@apache.org on 2026/09/13 23:45:09 UTC
(spark) branch branch-4.x updated: [SPARK-59291][STREAMING] Add return tags to the public StateSpec methods
This is an automated email from the ASF dual-hosted git repository.
uros-b pushed a commit to branch branch-4.x
in repository https://gitbox.apache.org/repos/asf/spark.git
The following commit(s) were added to refs/heads/branch-4.x by this push:
new f29cd3073d17 [SPARK-59291][STREAMING] Add return tags to the public StateSpec methods
f29cd3073d17 is described below
commit f29cd3073d17ce853320d884398d73f69b26e9b1
Author: Uros <22...@users.noreply.github.com>
AuthorDate: Sun Sep 13 23:44:50 2026 +0000
[SPARK-59291][STREAMING] Add return tags to the public StateSpec methods
### What changes were proposed in this pull request?
This completes the Scaladoc on the public `StateSpec` API:
- Adds `return` tags to all nine public methods that lacked them:
- the four `StateSpec.function(...)` factory overloads (companion object) -- "a new `StateSpec` for the `mapWithState` operation";
- the five builder methods on the abstract class (`initialState` x2, `numPartitions`, `partitioner`, `timeout`), which return `this.type` -- "this `StateSpec` (to allow chaining)".
- Adds the missing `tparam KeyType` to the second and fourth `StateSpec.function(...)` overloads, which previously documented only `ValueType` / `StateType` / `MappedType`.
Scaladoc only; no code changes.
### Why are the changes needed?
It completes the Scaladoc for the public `StateSpec` surface so the generated API docs state what each method returns and document every type parameter. The `function` factories return a freshly built `StateSpec`, and the builder methods return the same instance to allow fluent chaining (as in the class example, `StateSpec.function(mappingFunction).numPartitions(10)`); this closes a small, pre-existing doc gap.
### Does this PR introduce _any_ user-facing change?
No. Scaladoc-only; no API or behavior change.
### How was this patch tested?
N/A -- documentation-only change. The `streaming` module compiles cleanly.
### Was this patch authored or co-authored using generative AI tooling?
Generated-by: Claude Code (Opus 4.8)
Closes #58563 from uros-b/apidoc-statespec-return.
Authored-by: Uros <22...@users.noreply.github.com>
Signed-off-by: Uros Bojanic <22...@users.noreply.github.com>
(cherry picked from commit 2dcbadb3aa878f8100b3574b7cef26b7d16e37f3)
Signed-off-by: Uros Bojanic <22...@users.noreply.github.com>
---
.../scala/org/apache/spark/streaming/StateSpec.scala | 16 ++++++++++++++++
1 file changed, 16 insertions(+)
diff --git a/streaming/src/main/scala/org/apache/spark/streaming/StateSpec.scala b/streaming/src/main/scala/org/apache/spark/streaming/StateSpec.scala
index 52f726290a4b..317868852bf4 100644
--- a/streaming/src/main/scala/org/apache/spark/streaming/StateSpec.scala
+++ b/streaming/src/main/scala/org/apache/spark/streaming/StateSpec.scala
@@ -72,22 +72,30 @@ sealed abstract class StateSpec[KeyType, ValueType, StateType, MappedType] exten
/**
* Set the RDD containing the initial states that will be used by `mapWithState`
+ *
+ * @return this `StateSpec` (to allow chaining)
*/
def initialState(rdd: RDD[(KeyType, StateType)]): this.type
/**
* Set the RDD containing the initial states that will be used by `mapWithState`
+ *
+ * @return this `StateSpec` (to allow chaining)
*/
def initialState(javaPairRDD: JavaPairRDD[KeyType, StateType]): this.type
/**
* Set the number of partitions by which the state RDDs generated by `mapWithState`
* will be partitioned. Hash partitioning will be used.
+ *
+ * @return this `StateSpec` (to allow chaining)
*/
def numPartitions(numPartitions: Int): this.type
/**
* Set the partitioner by which the state RDDs generated by `mapWithState` will be partitioned.
+ *
+ * @return this `StateSpec` (to allow chaining)
*/
def partitioner(partitioner: Partitioner): this.type
@@ -97,6 +105,8 @@ sealed abstract class StateSpec[KeyType, ValueType, StateType, MappedType] exten
* mapping function will be called one final time on the idle states that are going to be
* removed; [[org.apache.spark.streaming.State State.isTimingOut()]] set
* to `true` in that call.
+ *
+ * @return this `StateSpec` (to allow chaining)
*/
def timeout(idleDuration: Duration): this.type
}
@@ -153,6 +163,7 @@ object StateSpec {
* @tparam ValueType Class of the values
* @tparam StateType Class of the states data
* @tparam MappedType Class of the mapped data
+ * @return a new `StateSpec` for the `mapWithState` operation
*/
def function[KeyType, ValueType, StateType, MappedType](
mappingFunction: (Time, KeyType, Option[ValueType], State[StateType]) => Option[MappedType]
@@ -169,9 +180,11 @@ object StateSpec {
*
* @param mappingFunction The function applied on every data item to manage the associated state
* and generate the mapped data
+ * @tparam KeyType Class of the keys
* @tparam ValueType Class of the values
* @tparam StateType Class of the states data
* @tparam MappedType Class of the mapped data
+ * @return a new `StateSpec` for the `mapWithState` operation
*/
def function[KeyType, ValueType, StateType, MappedType](
mappingFunction: (KeyType, Option[ValueType], State[StateType]) => MappedType
@@ -196,6 +209,7 @@ object StateSpec {
* @tparam ValueType Class of the values
* @tparam StateType Class of the states data
* @tparam MappedType Class of the mapped data
+ * @return a new `StateSpec` for the `mapWithState` operation
*/
def function[KeyType, ValueType, StateType, MappedType](mappingFunction:
JFunction4[Time, KeyType, Optional[ValueType], State[StateType], Optional[MappedType]]):
@@ -218,9 +232,11 @@ object StateSpec {
*
* @param mappingFunction The function applied on every data item to manage the associated
* state and generate the mapped data
+ * @tparam KeyType Class of the keys
* @tparam ValueType Class of the values
* @tparam StateType Class of the states data
* @tparam MappedType Class of the mapped data
+ * @return a new `StateSpec` for the `mapWithState` operation
*/
def function[KeyType, ValueType, StateType, MappedType](
mappingFunction: JFunction3[KeyType, Optional[ValueType], State[StateType], MappedType]):
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@spark.apache.org
For additional commands, e-mail: commits-help@spark.apache.org