You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@kafka.apache.org by mj...@apache.org on 2026/08/01 00:09:49 UTC

(kafka) branch trunk updated: KAFKA-20849: Update the headers-aware store docs on suppress() (#23007)

This is an automated email from the ASF dual-hosted git repository.

mjsax pushed a commit to branch trunk
in repository https://gitbox.apache.org/repos/asf/kafka.git


The following commit(s) were added to refs/heads/trunk by this push:
     new e7cba22471f KAFKA-20849: Update the headers-aware store docs on suppress() (#23007)
e7cba22471f is described below

commit e7cba22471f0d2833b850e2b0085c1cfff2efed5
Author: Alieh Saeedi <10...@users.noreply.github.com>
AuthorDate: Sat Aug 1 02:09:41 2026 +0200

    KAFKA-20849: Update the headers-aware store docs on suppress() (#23007)
    
    KAFKA-20413 made the `suppress()` buffer headers-aware when
    `dsl.store.format=HEADERS` is configured
    (`InMemoryTimeOrderedKeyValueChangeBuffer`), but four
    documentation locations still claim that `suppress()` loses headers:
    
    - `dsl-api.md` "Note on headers-aware state stores" on `suppress()`
    - `dsl-api.md` KIP-1285 current-behavior bullets
    - `config-streams.md` `dsl.store.format` "Current limitations"
    - `upgrade-guide.md` KIP-1285 "Current limitations"
    
    Docs only -- no code changes.
    
    Reviewers: Matthias J. Sax <ma...@confluent.io>
---
 docs/streams/developer-guide/config-streams.md | 2 +-
 docs/streams/developer-guide/dsl-api.md        | 4 +---
 docs/streams/upgrade-guide.md                  | 2 ++
 3 files changed, 4 insertions(+), 4 deletions(-)

diff --git a/docs/streams/developer-guide/config-streams.md b/docs/streams/developer-guide/config-streams.md
index 5c006ddaf8a..ec31bb4222c 100644
--- a/docs/streams/developer-guide/config-streams.md
+++ b/docs/streams/developer-guide/config-streams.md
@@ -1407,7 +1407,7 @@ Serde for the inner class of a windowed record. Must implement the `Serde` inter
 >
 > See [KIP-1271](https://cwiki.apache.org/confluence/x/QIM8G) for migration procedures, changelog compatibility, restore behavior, and per-record overhead.
 >
-> **Current limitations**: `dsl.store.format=HEADERS` changes the state store format. It does not define how DSL operators create headers for output records. Some operators write empty headers to their materialized stores, and the buffer stores used by `suppress()` and left/outer stream-stream joins are not headers-aware. See [Stateful transformations](/{version}/streams/developer-guide/dsl-api.html#stateful-transformations) and the [Streams upgrade guide](/{version}/streams/upgrade-guid [...]
+> **Current limitations**: `dsl.store.format=HEADERS` changes the state store format. It does not define how DSL operators create headers for output records. Some operators write empty headers to their materialized stores, and the buffer store used by left/outer stream-stream joins for not-yet-matched records is not headers-aware. See [Stateful transformations](/{version}/streams/developer-guide/dsl-api.html#stateful-transformations) and the [Streams upgrade guide](/{version}/streams/upg [...]
 
 ### ensure.explicit.internal.resource.naming
 
diff --git a/docs/streams/developer-guide/dsl-api.md b/docs/streams/developer-guide/dsl-api.md
index 6ed4e375784..34f18db6e70 100644
--- a/docs/streams/developer-guide/dsl-api.md
+++ b/docs/streams/developer-guide/dsl-api.md
@@ -931,7 +931,7 @@ This config only changes the state store format. It does not define how DSL oper
 
   * aggregations, KTable-KTable joins, materialized `KTable.mapValues`, `KStream.toTable()`, and `StreamsBuilder.table()` write empty headers to their materialized stores
   * KStream-KStream join window stores keep source-record headers, but join result records do not get computed or merged headers; they may carry the headers from the record that triggered the result
-  * `suppress()` and left/outer [KStream-KStream joins](#kstream-kstream-join) use non-headers-aware buffer stores, so records passing through those buffers lose their headers
+  * left/outer [KStream-KStream joins](#kstream-kstream-join) use a non-headers-aware buffer store for not-yet-matched records, so records passing through that buffer lose their headers
 
 A follow-up KIP will define how DSL result headers are computed.
 
@@ -4673,8 +4673,6 @@ The key parts of this program are:
 
 One thing to note is that suppression is just like any other Kafka Streams operator, so you can build a topology with two branches emerging from the `count`, one suppressed, and one not, or even multiple differently configured suppressions. This allows you to apply suppressions where they are needed and otherwise rely on the default continuous update behavior. 
 
-**Note on headers-aware state stores:** `suppress()` uses an in-memory buffer that is not headers-aware. Record headers attached to upstream records are not preserved across the suppression boundary, even when [`dsl.store.format=HEADERS`](../config-streams#dsl-store-format) is set globally per [KIP-1285](https://cwiki.apache.org/confluence/x/4ow8G).
-
 For more detailed information, see the JavaDoc on the `Suppressed` config object and [KIP-328](https://cwiki.apache.org/confluence/x/sQU0BQ "KIP-328"). 
 
 Applying processors (Processor API integration)
diff --git a/docs/streams/upgrade-guide.md b/docs/streams/upgrade-guide.md
index fa6222fc55d..b4bd4c90482 100644
--- a/docs/streams/upgrade-guide.md
+++ b/docs/streams/upgrade-guide.md
@@ -77,6 +77,8 @@ For applications using the Streams Rebalance Protocol (`group.protocol=streams`)
 
 For applications using the Streams Rebalance Protocol (`group.protocol=streams`), the broker-side task assignor is now pluggable ([KIP-1357](https://cwiki.apache.org/confluence/x/NoSnGQ)). Operators register assignors with the new broker configuration `group.streams.assignors`, which accepts a list of built-in assignor names and fully qualified class names of custom `TaskAssignor` implementations; the first entry is the default for groups that do not select one. An individual group picks [...]
 
+The buffer used by `suppress()` is now headers-aware, closing one of the gaps listed under the [KIP-1285 current limitations](#current-limitations) in 4.3.0. With [`dsl.store.format=HEADERS`](/{version}/streams/developer-guide/config-streams.html#dsl-store-format), each buffered value is stored together with the headers of the record it came from, so record headers are preserved across the suppression boundary: the record emitted when a buffered row is evicted carries the headers of the  [...]
+
 ## Streams API changes in 4.3.0
 
 **Note:** Kafka Streams 4.3.0 contains a critical native memory leak in the RocksDB state store layer ([KAFKA-20616](https://issues.apache.org/jira/browse/KAFKA-20616)). The `ColumnFamilyOptions` for the offsets column family is not closed, and column family handles can leak on close-path exceptions, which under cascading task closes (e.g., rebalances or error-triggered recoveries) leads to unbounded off-heap memory growth and eventual OOM. Users running Kafka Streams should consider upg [...]