You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@kafka.apache.org by mj...@apache.org on 2020/07/07 01:10:00 UTC
[kafka] branch trunk updated: MINOR: document timestamped state
stores (#8920)
This is an automated email from the ASF dual-hosted git repository.
mjsax pushed a commit to branch trunk
in repository https://gitbox.apache.org/repos/asf/kafka.git
The following commit(s) were added to refs/heads/trunk by this push:
new 11a8549 MINOR: document timestamped state stores (#8920)
11a8549 is described below
commit 11a8549479431826e64e80e49f1c5e41be8a7fd5
Author: Jim Galasyn <ji...@confluent.io>
AuthorDate: Mon Jul 6 18:09:24 2020 -0700
MINOR: document timestamped state stores (#8920)
Reviewers: Matthias J. Sax <ma...@confluent.io>
---
docs/streams/developer-guide/processor-api.html | 28 +++++++++++++++++++++++++
1 file changed, 28 insertions(+)
diff --git a/docs/streams/developer-guide/processor-api.html b/docs/streams/developer-guide/processor-api.html
index e7c6eac..d314c7d 100644
--- a/docs/streams/developer-guide/processor-api.html
+++ b/docs/streams/developer-guide/processor-api.html
@@ -49,6 +49,7 @@
<li><a class="reference internal" href="#defining-and-creating-a-state-store" id="id4">Defining and creating a State Store</a></li>
<li><a class="reference internal" href="#fault-tolerant-state-stores" id="id5">Fault-tolerant State Stores</a></li>
<li><a class="reference internal" href="#enable-or-disable-fault-tolerance-of-state-stores-store-changelogs" id="id6">Enable or Disable Fault Tolerance of State Stores (Store Changelogs)</a></li>
+ <li><a class="reference internal" href="#timestamped-state-stores" id="id11">Timestamped State Stores</a></li>
<li><a class="reference internal" href="#implementing-custom-state-stores" id="id7">Implementing Custom State Stores</a></li>
</ul>
</li>
@@ -229,6 +230,10 @@
<a class="reference internal" href="config-streams.html#streams-developer-guide-rocksdb-config"><span class="std std-ref">RocksDB configuration</span></a>.</li>
<li>Available <a class="reference external" href="/{{version}}/javadoc/org/apache/kafka/streams/state/Stores.html#persistentKeyValueStore-java.lang.String-">store variants</a>:
time window key-value store, session window key-value store.</li>
+ <li>Use <a class="reference external" href="/{{version}}/javadoc/org/apache/kafka/streams/state/Stores.html#persistentTimestampedKeyValueStore-java.lang.String-">persistentTimestampedKeyValueStore</a>
+ when you need a persistent key-(value/timestamp) store that supports put/get/delete and range queries.</li>
+ <li>Use <a class="reference external" href="/{{version}}/javadoc/org/apache/kafka/streams/state/Stores.html#persistentTimestampedWindowStore-java.lang.String-java.time.Duration-java.time.Duration-boolean-">persistentTimestampedWindowStore</a>
+ when you need a persistent windowedKey-(value/timestamp) store.</li>
</ul>
<div class="highlight-java"><div class="highlight"><pre><span></span><span class="c1">// Creating a persistent key-value store:</span>
<span class="c1">// here, we create a `KeyValueStore<String, Long>` named "persistent-counts".</span>
@@ -260,6 +265,10 @@
in-between app instance restarts.</li>
<li>Available <a class="reference external" href="/{{version}}/javadoc/org/apache/kafka/streams/state/Stores.html#inMemoryKeyValueStore-java.lang.String-">store variants</a>:
time window key-value store, session window key-value store.</li>
+ <li>Use <a class="reference external" href="/{{version}}/javadoc/org/apache/kafka/streams/state/TimestampedKeyValueStore.html">TimestampedKeyValueStore</a>
+ when you need a key-(value/timestamp) store that supports put/get/delete and range queries.</li>
+ <li>Use <a class="reference external" href="/{{version}}/javadoc/org/apache/kafka/streams/state/TimestampedWindowStore.html">TimestampedWindowStore</a>
+ when you need to store windowedKey-(value/timestamp) pairs.</li>
</ul>
<div class="highlight-java"><div class="highlight"><pre><span></span><span class="c1">// Creating an in-memory key-value store:</span>
<span class="c1">// here, we create a `KeyValueStore<String, Long>` named "inmemory-counts".</span>
@@ -343,6 +352,25 @@
</pre></div>
</div>
</div>
+ <div class="section" id="timestamped-state-stores">
+ <span id="streams-developer-guide-state-store-timestamps"></span><h3><a class="toc-backref" href="#id11">Timestamped State Stores</a><a class="headerlink" href="#timestamped-state-stores" title="Permalink to this headline"></a></h3>
+ <p>KTables always store timestamps by default.
+ A timestamped state store improves stream processing semantics and enables
+ handling out-of-order data in source KTables, detecting out-of-order joins and aggregations,
+ and getting the timestamp of the latest update in an Interactive Query.</p>
+ <p>You can query timestamped state stores both with and without a timestamp.</p>
+ <b>Upgrade note:</b> All users upgrade with a single rolling bounce per instance.
+ <ul class="first simple">
+ <li>For Processor API users, nothing changes in existing applications, and you
+ have the option of using the timestamped stores.</li>
+ <li>For DSL operators, store data is upgraded lazily in the background.</li>
+ <li>No upgrade happens if you provide a custom XxxBytesStoreSupplier, but you can opt-in
+ by implementing the <a class="reference external" href="/{{version}}/javadoc/org/apache/kafka/streams/state/TimestampedBytesStore.html">TimestampedBytesStore</a>
+ interface. In this case, the old format is retained, and Streams uses a proxy store
+ that removes/adds timestamps on read/write.</li>
+ </ul>
+ </p>
+ </div>
<div class="section" id="implementing-custom-state-stores">
<span id="streams-developer-guide-state-store-custom"></span><h3><a class="toc-backref" href="#id7">Implementing Custom State Stores</a><a class="headerlink" href="#implementing-custom-state-stores" title="Permalink to this headline"></a></h3>
<p>You can use the <a class="reference internal" href="#streams-developer-guide-state-store-defining"><span class="std std-ref">built-in state store types</span></a> or implement your own.