You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@kafka.apache.org by rh...@apache.org on 2020/07/15 22:33:01 UTC
[kafka-site] branch asf-site updated: Correct documentation content
to be from 2.6.0 release
This is an automated email from the ASF dual-hosted git repository.
rhauch pushed a commit to branch asf-site
in repository https://gitbox.apache.org/repos/asf/kafka-site.git
The following commit(s) were added to refs/heads/asf-site by this push:
new 928afd2 Correct documentation content to be from 2.6.0 release
928afd2 is described below
commit 928afd2d3637b703c1632de0c22a15475512b9c6
Author: Randall Hauch <rh...@gmail.com>
AuthorDate: Wed Jul 15 17:33:51 2020 -0500
Correct documentation content to be from 2.6.0 release
---
26/connect.html | 12 +-
26/generated/kafka_config.html | 10 +-
26/generated/producer_config.html | 2 +-
26/generated/producer_metrics.html | 4 +-
26/generated/protocol_messages.html | 330 +++++++++++++-------------
26/generated/topic_config.html | 4 +-
26/js/templateData.js | 6 +-
26/streams/core-concepts.html | 28 ++-
26/streams/developer-guide/processor-api.html | 28 +++
26/streams/upgrade-guide.html | 8 -
10 files changed, 239 insertions(+), 193 deletions(-)
diff --git a/26/connect.html b/26/connect.html
index 9777b17..797c1fe 100644
--- a/26/connect.html
+++ b/26/connect.html
@@ -331,11 +331,11 @@
<p>Kafka Connect provides error reporting to handle errors encountered along various stages of processing. By default, any error encountered during conversion or within transformations will cause the connector to fail. Each connector configuration can also enable tolerating such errors by skipping them, optionally writing each error and the details of the failed operation and problematic record (with various levels of detail) to the Connect application log. These mechanisms also capt [...]
- <p>To report errors within a connector's converter, transforms, or in specifically the sink connector itself to the log, set <code>errors.log.enable=true</code> in the connector configuration to log details of each error and problem record's topic, partition, and offset. For additional debugging purposes, set <code>errors.log.include.messages=true</code> to also log the problem record key, value, and headers to the log (note this may log sensitive information).
+ <p>To report errors within a connector's converter, transforms, or within the sink connector itself to the log, set <code>errors.log.enable=true</code> in the connector configuration to log details of each error and problem record's topic, partition, and offset. For additional debugging purposes, set <code>errors.log.include.messages=true</code> to also log the problem record key, value, and headers to the log (note this may log sensitive information).</p>
- <p>To report errors within a connector's converter, transforms, or in specifically the sink connector itself to a dead letter queue topic, set <code>errors.deadletterqueue.topic.name</code>, and optionally <code>errors.deadletterqueue.context.headers.enable=true</code>.</p>
+ <p>To report errors within a connector's converter, transforms, or within the sink connector itself to a dead letter queue topic, set <code>errors.deadletterqueue.topic.name</code>, and optionally <code>errors.deadletterqueue.context.headers.enable=true</code>.</p>
- <p>For example, below shows a configuration that will cause a connector will fail immediately upon an error or exception. Although it is not necessary to add extra configuration properties for this behavior, adding the following properties to a sink connector configuration would achieve this "fail fast" behavior:</p>
+ <p>By default connectors exhibit "fail fast" behavior immediately upon an error or exception. This is equivalent to adding the following configuration properties with their defaults to a connector configuration:</p>
<pre class="brush: text;">
# disable retries on failure
@@ -351,7 +351,7 @@
errors.tolerance=none
</pre>
- <p>The following configuration shows how to setup error handling with multiple retries, logging both to the application logs and a Kafka topic with infinite tolerance:</p>
+ <p>These and other related connector configuration properties can be changed to provide different behavior. For example, the following configuration properties can be added to a connector configuration to setup error handling with multiple retries, logging to the application logs and the <code>my-connector-errors</code> Kafka topic, and tolerating all errors by reporting them rather than failing the connector task:</p>
<pre class="brush: text;">
# retry for at most 10 minutes times waiting up to 30 seconds between consecutive failures
@@ -542,7 +542,7 @@
<h5><a id="connect_errantrecordreporter" href="connect_errantrecordreporter">Errant Record Reporter</a></h5>
- <p>The <code>ErrantRecordReporter</code> can be used to report errors encountered after records have been sent to a sink connector. The following is an example implementation and use case of the <code>ErrantRecordReporter</code> in the <code>SinkTask</code> class:</p>
+ <p>When <a href="#connect_errorreporting">error reporting</a> is enabled for a connector, the connector can use an <code>ErrantRecordReporter</code> to report problems with individual records sent to a sink connector. The following example shows how a connector's <code>SinkTask</code> subclass might obtain and use the <code>ErrantRecordReporter</code>, safely handling a null reporter when the DLQ is not enabled or when the connector is installed in an older Connect runtime that doesn [...]
<pre class="brush: java;">
private ErrantRecordReporter reporter;
@@ -562,7 +562,7 @@
public void put(Collection<SinkRecord> records) {
for (SinkRecord record: records) {
try {
- // attempt to send record to data sink
+ // attempt to process and send record to data sink
process(record);
} catch(Exception e) {
if (reporter != null) {
diff --git a/26/generated/kafka_config.html b/26/generated/kafka_config.html
index 26b6d77..acffb7e 100644
--- a/26/generated/kafka_config.html
+++ b/26/generated/kafka_config.html
@@ -584,7 +584,7 @@
</li>
<li>
<h4><a id="replica.fetch.min.bytes" href="#replica.fetch.min.bytes">replica.fetch.min.bytes</a></h4>
-<p>Minimum bytes expected for each fetch response. If not enough bytes, wait up to <code>replica.fetch.wait.max.ms</code> (broker config).</p>
+<p>Minimum bytes expected for each fetch response. If not enough bytes, wait up to replicaMaxWaitTimeMs</p>
<table><tbody>
<tr><th>Type:</th><td>int</td></tr>
<tr><th>Default:</th><td>1</td></tr>
@@ -1049,8 +1049,8 @@
<p>Specify which version of the inter-broker protocol will be used.<br> This is typically bumped after all brokers were upgraded to a new version.<br> Example of some valid values are: 0.8.0, 0.8.1, 0.8.1.1, 0.8.2, 0.8.2.0, 0.8.2.1, 0.9.0.0, 0.9.0.1 Check ApiVersion for the full list.</p>
<table><tbody>
<tr><th>Type:</th><td>string</td></tr>
-<tr><th>Default:</th><td>2.7-IV0</td></tr>
-<tr><th>Valid Values:</th><td>[0.8.0, 0.8.1, 0.8.2, 0.9.0, 0.10.0-IV0, 0.10.0-IV1, 0.10.1-IV0, 0.10.1-IV1, 0.10.1-IV2, 0.10.2-IV0, 0.11.0-IV0, 0.11.0-IV1, 0.11.0-IV2, 1.0-IV0, 1.1-IV0, 2.0-IV0, 2.0-IV1, 2.1-IV0, 2.1-IV1, 2.1-IV2, 2.2-IV0, 2.2-IV1, 2.3-IV0, 2.3-IV1, 2.4-IV0, 2.4-IV1, 2.5-IV0, 2.6-IV0, 2.7-IV0]</td></tr>
+<tr><th>Default:</th><td>2.6-IV0</td></tr>
+<tr><th>Valid Values:</th><td>[0.8.0, 0.8.1, 0.8.2, 0.9.0, 0.10.0-IV0, 0.10.0-IV1, 0.10.1-IV0, 0.10.1-IV1, 0.10.1-IV2, 0.10.2-IV0, 0.11.0-IV0, 0.11.0-IV1, 0.11.0-IV2, 1.0-IV0, 1.1-IV0, 2.0-IV0, 2.0-IV1, 2.1-IV0, 2.1-IV1, 2.1-IV2, 2.2-IV0, 2.2-IV1, 2.3-IV0, 2.3-IV1, 2.4-IV0, 2.4-IV1, 2.5-IV0, 2.6-IV0]</td></tr>
<tr><th>Importance:</th><td>medium</td></tr>
<tr><th>Update Mode:</th><td>read-only</td></tr>
</tbody></table>
@@ -1214,8 +1214,8 @@
<p>Specify the message format version the broker will use to append messages to the logs. The value should be a valid ApiVersion. Some examples are: 0.8.2, 0.9.0.0, 0.10.0, check ApiVersion for more details. By setting a particular message format version, the user is certifying that all the existing messages on disk are smaller or equal than the specified version. Setting this value incorrectly will cause consumers with older versions to break as they will receive messages with a format [...]
<table><tbody>
<tr><th>Type:</th><td>string</td></tr>
-<tr><th>Default:</th><td>2.7-IV0</td></tr>
-<tr><th>Valid Values:</th><td>[0.8.0, 0.8.1, 0.8.2, 0.9.0, 0.10.0-IV0, 0.10.0-IV1, 0.10.1-IV0, 0.10.1-IV1, 0.10.1-IV2, 0.10.2-IV0, 0.11.0-IV0, 0.11.0-IV1, 0.11.0-IV2, 1.0-IV0, 1.1-IV0, 2.0-IV0, 2.0-IV1, 2.1-IV0, 2.1-IV1, 2.1-IV2, 2.2-IV0, 2.2-IV1, 2.3-IV0, 2.3-IV1, 2.4-IV0, 2.4-IV1, 2.5-IV0, 2.6-IV0, 2.7-IV0]</td></tr>
+<tr><th>Default:</th><td>2.6-IV0</td></tr>
+<tr><th>Valid Values:</th><td>[0.8.0, 0.8.1, 0.8.2, 0.9.0, 0.10.0-IV0, 0.10.0-IV1, 0.10.1-IV0, 0.10.1-IV1, 0.10.1-IV2, 0.10.2-IV0, 0.11.0-IV0, 0.11.0-IV1, 0.11.0-IV2, 1.0-IV0, 1.1-IV0, 2.0-IV0, 2.0-IV1, 2.1-IV0, 2.1-IV1, 2.1-IV2, 2.2-IV0, 2.2-IV1, 2.3-IV0, 2.3-IV1, 2.4-IV0, 2.4-IV1, 2.5-IV0, 2.6-IV0]</td></tr>
<tr><th>Importance:</th><td>medium</td></tr>
<tr><th>Update Mode:</th><td>read-only</td></tr>
</tbody></table>
diff --git a/26/generated/producer_config.html b/26/generated/producer_config.html
index e27eb74..58d3fed 100644
--- a/26/generated/producer_config.html
+++ b/26/generated/producer_config.html
@@ -631,7 +631,7 @@
</li>
<li>
<h4><a id="transaction.timeout.ms" href="#transaction.timeout.ms">transaction.timeout.ms</a></h4>
-<p>The maximum amount of time in ms that the transaction coordinator will wait for a transaction status update from the producer before proactively aborting the ongoing transaction.If this value is larger than the transaction.max.timeout.ms setting in the broker, the request will fail with a <code>InvalidTxnTimeoutException</code> error.</p>
+<p>The maximum amount of time in ms that the transaction coordinator will wait for a transaction status update from the producer before proactively aborting the ongoing transaction.If this value is larger than the transaction.max.timeout.ms setting in the broker, the request will fail with a <code>InvalidTransactionTimeout</code> error.</p>
<table><tbody>
<tr><th>Type:</th><td>int</td></tr>
<tr><th>Default:</th><td>60000 (1 minute)</td></tr>
diff --git a/26/generated/producer_metrics.html b/26/generated/producer_metrics.html
index a02defa..deb4078 100644
--- a/26/generated/producer_metrics.html
+++ b/26/generated/producer_metrics.html
@@ -15,7 +15,7 @@
<tr>
<td></td><td>batch-split-total</td><td>The total number of batch splits</td></tr>
<tr>
-<td></td><td>compression-rate-avg</td><td>The average compression rate of record batches, defined as the average ratio of the compressed batch size over the uncompressed size.</td></tr>
+<td></td><td>compression-rate-avg</td><td>The average compression rate of record batches.</td></tr>
<tr>
<td></td><td>metadata-age</td><td>The age in seconds of the current producer metadata being used.</td></tr>
<tr>
@@ -62,7 +62,7 @@
<tr>
<td></td><td>byte-total</td><td>The total number of bytes sent for a topic.</td></tr>
<tr>
-<td></td><td>compression-rate</td><td>The average compression rate of record batches for a topic, defined as the average ratio of the compressed batch size over the uncompressed size.</td></tr>
+<td></td><td>compression-rate</td><td>The average compression rate of record batches for a topic.</td></tr>
<tr>
<td></td><td>record-error-rate</td><td>The average per-second number of record sends that resulted in errors for a topic</td></tr>
<tr>
diff --git a/26/generated/protocol_messages.html b/26/generated/protocol_messages.html
index a0db4f1..7a93844 100644
--- a/26/generated/protocol_messages.html
+++ b/26/generated/protocol_messages.html
@@ -9297,100 +9297,100 @@
<b>Requests:</b><br>
<p><pre>DescribeConfigs Request (Version: 0) => [resources]
- resources => resource_type resource_name [configuration_keys]
+ resources => resource_type resource_name [config_names]
resource_type => INT8
resource_name => STRING
- configuration_keys => STRING
+ config_names => STRING
</pre><table class="data-table"><tbody>
<tr><th>Field</th>
<th>Description</th>
</tr><tr>
-<td>resources</td><td>The resources whose configurations we want to describe.</td></tr>
+<td>resources</td><td>An array of config resources to be returned.</td></tr>
<tr>
-<td>resource_type</td><td>The resource type.</td></tr>
+<td>resource_type</td><td>null</td></tr>
<tr>
-<td>resource_name</td><td>The resource name.</td></tr>
+<td>resource_name</td><td>null</td></tr>
<tr>
-<td>configuration_keys</td><td>The configuration keys to list, or null to list all configuration keys.</td></tr>
+<td>config_names</td><td>null</td></tr>
</table>
</p>
<p><pre>DescribeConfigs Request (Version: 1) => [resources] include_synonyms
- resources => resource_type resource_name [configuration_keys]
+ resources => resource_type resource_name [config_names]
resource_type => INT8
resource_name => STRING
- configuration_keys => STRING
+ config_names => STRING
include_synonyms => BOOLEAN
</pre><table class="data-table"><tbody>
<tr><th>Field</th>
<th>Description</th>
</tr><tr>
-<td>resources</td><td>The resources whose configurations we want to describe.</td></tr>
+<td>resources</td><td>An array of config resources to be returned.</td></tr>
<tr>
-<td>resource_type</td><td>The resource type.</td></tr>
+<td>resource_type</td><td>null</td></tr>
<tr>
-<td>resource_name</td><td>The resource name.</td></tr>
+<td>resource_name</td><td>null</td></tr>
<tr>
-<td>configuration_keys</td><td>The configuration keys to list, or null to list all configuration keys.</td></tr>
+<td>config_names</td><td>null</td></tr>
<tr>
-<td>include_synonyms</td><td>True if we should include all synonyms.</td></tr>
+<td>include_synonyms</td><td>null</td></tr>
</table>
</p>
<p><pre>DescribeConfigs Request (Version: 2) => [resources] include_synonyms
- resources => resource_type resource_name [configuration_keys]
+ resources => resource_type resource_name [config_names]
resource_type => INT8
resource_name => STRING
- configuration_keys => STRING
+ config_names => STRING
include_synonyms => BOOLEAN
</pre><table class="data-table"><tbody>
<tr><th>Field</th>
<th>Description</th>
</tr><tr>
-<td>resources</td><td>The resources whose configurations we want to describe.</td></tr>
+<td>resources</td><td>An array of config resources to be returned.</td></tr>
<tr>
-<td>resource_type</td><td>The resource type.</td></tr>
+<td>resource_type</td><td>null</td></tr>
<tr>
-<td>resource_name</td><td>The resource name.</td></tr>
+<td>resource_name</td><td>null</td></tr>
<tr>
-<td>configuration_keys</td><td>The configuration keys to list, or null to list all configuration keys.</td></tr>
+<td>config_names</td><td>null</td></tr>
<tr>
-<td>include_synonyms</td><td>True if we should include all synonyms.</td></tr>
+<td>include_synonyms</td><td>null</td></tr>
</table>
</p>
<p><pre>DescribeConfigs Request (Version: 3) => [resources] include_synonyms include_documentation
- resources => resource_type resource_name [configuration_keys]
+ resources => resource_type resource_name [config_names]
resource_type => INT8
resource_name => STRING
- configuration_keys => STRING
+ config_names => STRING
include_synonyms => BOOLEAN
include_documentation => BOOLEAN
</pre><table class="data-table"><tbody>
<tr><th>Field</th>
<th>Description</th>
</tr><tr>
-<td>resources</td><td>The resources whose configurations we want to describe.</td></tr>
+<td>resources</td><td>An array of config resources to be returned.</td></tr>
<tr>
-<td>resource_type</td><td>The resource type.</td></tr>
+<td>resource_type</td><td>null</td></tr>
<tr>
-<td>resource_name</td><td>The resource name.</td></tr>
+<td>resource_name</td><td>null</td></tr>
<tr>
-<td>configuration_keys</td><td>The configuration keys to list, or null to list all configuration keys.</td></tr>
+<td>config_names</td><td>null</td></tr>
<tr>
-<td>include_synonyms</td><td>True if we should include all synonyms.</td></tr>
+<td>include_synonyms</td><td>null</td></tr>
<tr>
-<td>include_documentation</td><td>True if we should include configuration documentation.</td></tr>
+<td>include_documentation</td><td>null</td></tr>
</table>
</p>
<b>Responses:</b><br>
-<p><pre>DescribeConfigs Response (Version: 0) => throttle_time_ms [results]
+<p><pre>DescribeConfigs Response (Version: 0) => throttle_time_ms [resources]
throttle_time_ms => INT32
- results => error_code error_message resource_type resource_name [configs]
+ resources => error_code error_message resource_type resource_name [config_entries]
error_code => INT16
error_message => NULLABLE_STRING
resource_type => INT8
resource_name => STRING
- configs => name value read_only is_default is_sensitive
- name => STRING
- value => NULLABLE_STRING
+ config_entries => config_name config_value read_only is_default is_sensitive
+ config_name => STRING
+ config_value => NULLABLE_STRING
read_only => BOOLEAN
is_default => BOOLEAN
is_sensitive => BOOLEAN
@@ -9398,197 +9398,197 @@
<tr><th>Field</th>
<th>Description</th>
</tr><tr>
-<td>throttle_time_ms</td><td>The duration in milliseconds for which the request was throttled due to a quota violation, or zero if the request did not violate any quota.</td></tr>
+<td>throttle_time_ms</td><td>Duration in milliseconds for which the request was throttled due to quota violation (Zero if the request did not violate any quota)</td></tr>
<tr>
-<td>results</td><td>The results for each resource.</td></tr>
+<td>resources</td><td>null</td></tr>
<tr>
-<td>error_code</td><td>The error code, or 0 if we were able to successfully describe the configurations.</td></tr>
+<td>error_code</td><td>Response error code</td></tr>
<tr>
-<td>error_message</td><td>The error message, or null if we were able to successfully describe the configurations.</td></tr>
+<td>error_message</td><td>Response error message</td></tr>
<tr>
-<td>resource_type</td><td>The resource type.</td></tr>
+<td>resource_type</td><td>null</td></tr>
<tr>
-<td>resource_name</td><td>The resource name.</td></tr>
+<td>resource_name</td><td>null</td></tr>
<tr>
-<td>configs</td><td>Each listed configuration.</td></tr>
+<td>config_entries</td><td>null</td></tr>
<tr>
-<td>name</td><td>The configuration name.</td></tr>
+<td>config_name</td><td>null</td></tr>
<tr>
-<td>value</td><td>The configuration value.</td></tr>
+<td>config_value</td><td>null</td></tr>
<tr>
-<td>read_only</td><td>True if the configuration is read-only.</td></tr>
+<td>read_only</td><td>null</td></tr>
<tr>
-<td>is_default</td><td>True if the configuration is not set.</td></tr>
+<td>is_default</td><td>null</td></tr>
<tr>
-<td>is_sensitive</td><td>True if this configuration is sensitive.</td></tr>
+<td>is_sensitive</td><td>null</td></tr>
</table>
</p>
-<p><pre>DescribeConfigs Response (Version: 1) => throttle_time_ms [results]
+<p><pre>DescribeConfigs Response (Version: 1) => throttle_time_ms [resources]
throttle_time_ms => INT32
- results => error_code error_message resource_type resource_name [configs]
+ resources => error_code error_message resource_type resource_name [config_entries]
error_code => INT16
error_message => NULLABLE_STRING
resource_type => INT8
resource_name => STRING
- configs => name value read_only config_source is_sensitive [synonyms]
- name => STRING
- value => NULLABLE_STRING
+ config_entries => config_name config_value read_only config_source is_sensitive [config_synonyms]
+ config_name => STRING
+ config_value => NULLABLE_STRING
read_only => BOOLEAN
config_source => INT8
is_sensitive => BOOLEAN
- synonyms => name value source
- name => STRING
- value => NULLABLE_STRING
- source => INT8
+ config_synonyms => config_name config_value config_source
+ config_name => STRING
+ config_value => NULLABLE_STRING
+ config_source => INT8
</pre><table class="data-table"><tbody>
<tr><th>Field</th>
<th>Description</th>
</tr><tr>
-<td>throttle_time_ms</td><td>The duration in milliseconds for which the request was throttled due to a quota violation, or zero if the request did not violate any quota.</td></tr>
+<td>throttle_time_ms</td><td>Duration in milliseconds for which the request was throttled due to quota violation (Zero if the request did not violate any quota)</td></tr>
<tr>
-<td>results</td><td>The results for each resource.</td></tr>
+<td>resources</td><td>null</td></tr>
<tr>
-<td>error_code</td><td>The error code, or 0 if we were able to successfully describe the configurations.</td></tr>
+<td>error_code</td><td>Response error code</td></tr>
<tr>
-<td>error_message</td><td>The error message, or null if we were able to successfully describe the configurations.</td></tr>
+<td>error_message</td><td>Response error message</td></tr>
<tr>
-<td>resource_type</td><td>The resource type.</td></tr>
+<td>resource_type</td><td>null</td></tr>
<tr>
-<td>resource_name</td><td>The resource name.</td></tr>
+<td>resource_name</td><td>null</td></tr>
<tr>
-<td>configs</td><td>Each listed configuration.</td></tr>
+<td>config_entries</td><td>null</td></tr>
<tr>
-<td>name</td><td>The configuration name.</td></tr>
+<td>config_name</td><td>null</td></tr>
<tr>
-<td>value</td><td>The configuration value.</td></tr>
+<td>config_value</td><td>null</td></tr>
<tr>
-<td>read_only</td><td>True if the configuration is read-only.</td></tr>
+<td>read_only</td><td>null</td></tr>
<tr>
-<td>config_source</td><td>The configuration source.</td></tr>
+<td>config_source</td><td>null</td></tr>
<tr>
-<td>is_sensitive</td><td>True if this configuration is sensitive.</td></tr>
+<td>is_sensitive</td><td>null</td></tr>
<tr>
-<td>synonyms</td><td>The synonyms for this configuration key.</td></tr>
+<td>config_synonyms</td><td>null</td></tr>
<tr>
-<td>name</td><td>The synonym name.</td></tr>
+<td>config_name</td><td>null</td></tr>
<tr>
-<td>value</td><td>The synonym value.</td></tr>
+<td>config_value</td><td>null</td></tr>
<tr>
-<td>source</td><td>The synonym source.</td></tr>
+<td>config_source</td><td>null</td></tr>
</table>
</p>
-<p><pre>DescribeConfigs Response (Version: 2) => throttle_time_ms [results]
+<p><pre>DescribeConfigs Response (Version: 2) => throttle_time_ms [resources]
throttle_time_ms => INT32
- results => error_code error_message resource_type resource_name [configs]
+ resources => error_code error_message resource_type resource_name [config_entries]
error_code => INT16
error_message => NULLABLE_STRING
resource_type => INT8
resource_name => STRING
- configs => name value read_only config_source is_sensitive [synonyms]
- name => STRING
- value => NULLABLE_STRING
+ config_entries => config_name config_value read_only config_source is_sensitive [config_synonyms]
+ config_name => STRING
+ config_value => NULLABLE_STRING
read_only => BOOLEAN
config_source => INT8
is_sensitive => BOOLEAN
- synonyms => name value source
- name => STRING
- value => NULLABLE_STRING
- source => INT8
+ config_synonyms => config_name config_value config_source
+ config_name => STRING
+ config_value => NULLABLE_STRING
+ config_source => INT8
</pre><table class="data-table"><tbody>
<tr><th>Field</th>
<th>Description</th>
</tr><tr>
-<td>throttle_time_ms</td><td>The duration in milliseconds for which the request was throttled due to a quota violation, or zero if the request did not violate any quota.</td></tr>
+<td>throttle_time_ms</td><td>Duration in milliseconds for which the request was throttled due to quota violation (Zero if the request did not violate any quota)</td></tr>
<tr>
-<td>results</td><td>The results for each resource.</td></tr>
+<td>resources</td><td>null</td></tr>
<tr>
-<td>error_code</td><td>The error code, or 0 if we were able to successfully describe the configurations.</td></tr>
+<td>error_code</td><td>Response error code</td></tr>
<tr>
-<td>error_message</td><td>The error message, or null if we were able to successfully describe the configurations.</td></tr>
+<td>error_message</td><td>Response error message</td></tr>
<tr>
-<td>resource_type</td><td>The resource type.</td></tr>
+<td>resource_type</td><td>null</td></tr>
<tr>
-<td>resource_name</td><td>The resource name.</td></tr>
+<td>resource_name</td><td>null</td></tr>
<tr>
-<td>configs</td><td>Each listed configuration.</td></tr>
+<td>config_entries</td><td>null</td></tr>
<tr>
-<td>name</td><td>The configuration name.</td></tr>
+<td>config_name</td><td>null</td></tr>
<tr>
-<td>value</td><td>The configuration value.</td></tr>
+<td>config_value</td><td>null</td></tr>
<tr>
-<td>read_only</td><td>True if the configuration is read-only.</td></tr>
+<td>read_only</td><td>null</td></tr>
<tr>
-<td>config_source</td><td>The configuration source.</td></tr>
+<td>config_source</td><td>null</td></tr>
<tr>
-<td>is_sensitive</td><td>True if this configuration is sensitive.</td></tr>
+<td>is_sensitive</td><td>null</td></tr>
<tr>
-<td>synonyms</td><td>The synonyms for this configuration key.</td></tr>
+<td>config_synonyms</td><td>null</td></tr>
<tr>
-<td>name</td><td>The synonym name.</td></tr>
+<td>config_name</td><td>null</td></tr>
<tr>
-<td>value</td><td>The synonym value.</td></tr>
+<td>config_value</td><td>null</td></tr>
<tr>
-<td>source</td><td>The synonym source.</td></tr>
+<td>config_source</td><td>null</td></tr>
</table>
</p>
-<p><pre>DescribeConfigs Response (Version: 3) => throttle_time_ms [results]
+<p><pre>DescribeConfigs Response (Version: 3) => throttle_time_ms [resources]
throttle_time_ms => INT32
- results => error_code error_message resource_type resource_name [configs]
+ resources => error_code error_message resource_type resource_name [config_entries]
error_code => INT16
error_message => NULLABLE_STRING
resource_type => INT8
resource_name => STRING
- configs => name value read_only config_source is_sensitive [synonyms] config_type documentation
- name => STRING
- value => NULLABLE_STRING
+ config_entries => config_name config_value read_only config_source is_sensitive [config_synonyms] config_type config_documentation
+ config_name => STRING
+ config_value => NULLABLE_STRING
read_only => BOOLEAN
config_source => INT8
is_sensitive => BOOLEAN
- synonyms => name value source
- name => STRING
- value => NULLABLE_STRING
- source => INT8
+ config_synonyms => config_name config_value config_source
+ config_name => STRING
+ config_value => NULLABLE_STRING
+ config_source => INT8
config_type => INT8
- documentation => NULLABLE_STRING
+ config_documentation => NULLABLE_STRING
</pre><table class="data-table"><tbody>
<tr><th>Field</th>
<th>Description</th>
</tr><tr>
-<td>throttle_time_ms</td><td>The duration in milliseconds for which the request was throttled due to a quota violation, or zero if the request did not violate any quota.</td></tr>
+<td>throttle_time_ms</td><td>Duration in milliseconds for which the request was throttled due to quota violation (Zero if the request did not violate any quota)</td></tr>
<tr>
-<td>results</td><td>The results for each resource.</td></tr>
+<td>resources</td><td>null</td></tr>
<tr>
-<td>error_code</td><td>The error code, or 0 if we were able to successfully describe the configurations.</td></tr>
+<td>error_code</td><td>Response error code</td></tr>
<tr>
-<td>error_message</td><td>The error message, or null if we were able to successfully describe the configurations.</td></tr>
+<td>error_message</td><td>Response error message</td></tr>
<tr>
-<td>resource_type</td><td>The resource type.</td></tr>
+<td>resource_type</td><td>null</td></tr>
<tr>
-<td>resource_name</td><td>The resource name.</td></tr>
+<td>resource_name</td><td>null</td></tr>
<tr>
-<td>configs</td><td>Each listed configuration.</td></tr>
+<td>config_entries</td><td>null</td></tr>
<tr>
-<td>name</td><td>The configuration name.</td></tr>
+<td>config_name</td><td>null</td></tr>
<tr>
-<td>value</td><td>The configuration value.</td></tr>
+<td>config_value</td><td>null</td></tr>
<tr>
-<td>read_only</td><td>True if the configuration is read-only.</td></tr>
+<td>read_only</td><td>null</td></tr>
<tr>
-<td>config_source</td><td>The configuration source.</td></tr>
+<td>config_source</td><td>null</td></tr>
<tr>
-<td>is_sensitive</td><td>True if this configuration is sensitive.</td></tr>
+<td>is_sensitive</td><td>null</td></tr>
<tr>
-<td>synonyms</td><td>The synonyms for this configuration key.</td></tr>
+<td>config_synonyms</td><td>null</td></tr>
<tr>
-<td>name</td><td>The synonym name.</td></tr>
+<td>config_name</td><td>null</td></tr>
<tr>
-<td>value</td><td>The synonym value.</td></tr>
+<td>config_value</td><td>null</td></tr>
<tr>
-<td>source</td><td>The synonym source.</td></tr>
+<td>config_source</td><td>null</td></tr>
<tr>
-<td>config_type</td><td>The configuration data type. Type can be one of the following values - BOOLEAN, STRING, INT, SHORT, LONG, DOUBLE, LIST, CLASS, PASSWORD</td></tr>
+<td>config_type</td><td>null</td></tr>
<tr>
-<td>documentation</td><td>The configuration documentation.</td></tr>
+<td>config_documentation</td><td>null</td></tr>
</table>
</p>
<h5><a name="The_Messages_AlterConfigs">AlterConfigs API (Key: 33):</a></h5>
@@ -9700,95 +9700,95 @@
<h5><a name="The_Messages_AlterReplicaLogDirs">AlterReplicaLogDirs API (Key: 34):</a></h5>
<b>Requests:</b><br>
-<p><pre>AlterReplicaLogDirs Request (Version: 0) => [dirs]
- dirs => path [topics]
- path => STRING
- topics => name [partitions]
- name => STRING
+<p><pre>AlterReplicaLogDirs Request (Version: 0) => [log_dirs]
+ log_dirs => log_dir [topics]
+ log_dir => STRING
+ topics => topic [partitions]
+ topic => STRING
partitions => INT32
</pre><table class="data-table"><tbody>
<tr><th>Field</th>
<th>Description</th>
</tr><tr>
-<td>dirs</td><td>The alterations to make for each directory.</td></tr>
+<td>log_dirs</td><td>null</td></tr>
<tr>
-<td>path</td><td>The absolute directory path.</td></tr>
+<td>log_dir</td><td>The absolute log directory path.</td></tr>
<tr>
-<td>topics</td><td>The topics to add to the directory.</td></tr>
+<td>topics</td><td>null</td></tr>
<tr>
-<td>name</td><td>The topic name.</td></tr>
+<td>topic</td><td>Name of topic</td></tr>
<tr>
-<td>partitions</td><td>The partition indexes.</td></tr>
+<td>partitions</td><td>List of partition ids of the topic.</td></tr>
</table>
</p>
-<p><pre>AlterReplicaLogDirs Request (Version: 1) => [dirs]
- dirs => path [topics]
- path => STRING
- topics => name [partitions]
- name => STRING
+<p><pre>AlterReplicaLogDirs Request (Version: 1) => [log_dirs]
+ log_dirs => log_dir [topics]
+ log_dir => STRING
+ topics => topic [partitions]
+ topic => STRING
partitions => INT32
</pre><table class="data-table"><tbody>
<tr><th>Field</th>
<th>Description</th>
</tr><tr>
-<td>dirs</td><td>The alterations to make for each directory.</td></tr>
+<td>log_dirs</td><td>null</td></tr>
<tr>
-<td>path</td><td>The absolute directory path.</td></tr>
+<td>log_dir</td><td>The absolute log directory path.</td></tr>
<tr>
-<td>topics</td><td>The topics to add to the directory.</td></tr>
+<td>topics</td><td>null</td></tr>
<tr>
-<td>name</td><td>The topic name.</td></tr>
+<td>topic</td><td>Name of topic</td></tr>
<tr>
-<td>partitions</td><td>The partition indexes.</td></tr>
+<td>partitions</td><td>List of partition ids of the topic.</td></tr>
</table>
</p>
<b>Responses:</b><br>
-<p><pre>AlterReplicaLogDirs Response (Version: 0) => throttle_time_ms [results]
+<p><pre>AlterReplicaLogDirs Response (Version: 0) => throttle_time_ms [topics]
throttle_time_ms => INT32
- results => topic_name [partitions]
- topic_name => STRING
- partitions => partition_index error_code
- partition_index => INT32
+ topics => topic [partitions]
+ topic => STRING
+ partitions => partition error_code
+ partition => INT32
error_code => INT16
</pre><table class="data-table"><tbody>
<tr><th>Field</th>
<th>Description</th>
</tr><tr>
-<td>throttle_time_ms</td><td>Duration in milliseconds for which the request was throttled due to a quota violation, or zero if the request did not violate any quota.</td></tr>
+<td>throttle_time_ms</td><td>Duration in milliseconds for which the request was throttled due to quota violation (Zero if the request did not violate any quota)</td></tr>
<tr>
-<td>results</td><td>The results for each topic.</td></tr>
+<td>topics</td><td>null</td></tr>
<tr>
-<td>topic_name</td><td>The name of the topic.</td></tr>
+<td>topic</td><td>Name of topic</td></tr>
<tr>
-<td>partitions</td><td>The results for each partition.</td></tr>
+<td>partitions</td><td>null</td></tr>
<tr>
-<td>partition_index</td><td>The partition index.</td></tr>
+<td>partition</td><td>Topic partition id</td></tr>
<tr>
-<td>error_code</td><td>The error code, or 0 if there was no error.</td></tr>
+<td>error_code</td><td>Response error code</td></tr>
</table>
</p>
-<p><pre>AlterReplicaLogDirs Response (Version: 1) => throttle_time_ms [results]
+<p><pre>AlterReplicaLogDirs Response (Version: 1) => throttle_time_ms [topics]
throttle_time_ms => INT32
- results => topic_name [partitions]
- topic_name => STRING
- partitions => partition_index error_code
- partition_index => INT32
+ topics => topic [partitions]
+ topic => STRING
+ partitions => partition error_code
+ partition => INT32
error_code => INT16
</pre><table class="data-table"><tbody>
<tr><th>Field</th>
<th>Description</th>
</tr><tr>
-<td>throttle_time_ms</td><td>Duration in milliseconds for which the request was throttled due to a quota violation, or zero if the request did not violate any quota.</td></tr>
+<td>throttle_time_ms</td><td>Duration in milliseconds for which the request was throttled due to quota violation (Zero if the request did not violate any quota)</td></tr>
<tr>
-<td>results</td><td>The results for each topic.</td></tr>
+<td>topics</td><td>null</td></tr>
<tr>
-<td>topic_name</td><td>The name of the topic.</td></tr>
+<td>topic</td><td>Name of topic</td></tr>
<tr>
-<td>partitions</td><td>The results for each partition.</td></tr>
+<td>partitions</td><td>null</td></tr>
<tr>
-<td>partition_index</td><td>The partition index.</td></tr>
+<td>partition</td><td>Topic partition id</td></tr>
<tr>
-<td>error_code</td><td>The error code, or 0 if there was no error.</td></tr>
+<td>error_code</td><td>Response error code</td></tr>
</table>
</p>
<h5><a name="The_Messages_DescribeLogDirs">DescribeLogDirs API (Key: 35):</a></h5>
diff --git a/26/generated/topic_config.html b/26/generated/topic_config.html
index 12641c0..0bbe4cd 100644
--- a/26/generated/topic_config.html
+++ b/26/generated/topic_config.html
@@ -125,8 +125,8 @@
<p>Specify the message format version the broker will use to append messages to the logs. The value should be a valid ApiVersion. Some examples are: 0.8.2, 0.9.0.0, 0.10.0, check ApiVersion for more details. By setting a particular message format version, the user is certifying that all the existing messages on disk are smaller or equal than the specified version. Setting this value incorrectly will cause consumers with older versions to break as they will receive messages with a format [...]
<table><tbody>
<tr><th>Type:</th><td>string</td></tr>
-<tr><th>Default:</th><td>2.7-IV0</td></tr>
-<tr><th>Valid Values:</th><td>[0.8.0, 0.8.1, 0.8.2, 0.9.0, 0.10.0-IV0, 0.10.0-IV1, 0.10.1-IV0, 0.10.1-IV1, 0.10.1-IV2, 0.10.2-IV0, 0.11.0-IV0, 0.11.0-IV1, 0.11.0-IV2, 1.0-IV0, 1.1-IV0, 2.0-IV0, 2.0-IV1, 2.1-IV0, 2.1-IV1, 2.1-IV2, 2.2-IV0, 2.2-IV1, 2.3-IV0, 2.3-IV1, 2.4-IV0, 2.4-IV1, 2.5-IV0, 2.6-IV0, 2.7-IV0]</td></tr>
+<tr><th>Default:</th><td>2.6-IV0</td></tr>
+<tr><th>Valid Values:</th><td>[0.8.0, 0.8.1, 0.8.2, 0.9.0, 0.10.0-IV0, 0.10.0-IV1, 0.10.1-IV0, 0.10.1-IV1, 0.10.1-IV2, 0.10.2-IV0, 0.11.0-IV0, 0.11.0-IV1, 0.11.0-IV2, 1.0-IV0, 1.1-IV0, 2.0-IV0, 2.0-IV1, 2.1-IV0, 2.1-IV1, 2.1-IV2, 2.2-IV0, 2.2-IV1, 2.3-IV0, 2.3-IV1, 2.4-IV0, 2.4-IV1, 2.5-IV0, 2.6-IV0]</td></tr>
<tr><th>Server Default Property:</th><td>log.message.format.version</td></tr>
<tr><th>Importance:</th><td>medium</td></tr>
</tbody></table>
diff --git a/26/js/templateData.js b/26/js/templateData.js
index 6a7809d..4e51947 100644
--- a/26/js/templateData.js
+++ b/26/js/templateData.js
@@ -17,8 +17,8 @@ limitations under the License.
// Define variables for doc templates
var context={
- "version": "27",
- "dotVersion": "2.7",
- "fullDotVersion": "2.7.0",
+ "version": "26",
+ "dotVersion": "2.6",
+ "fullDotVersion": "2.6.0",
"scalaVersion": "2.13"
};
diff --git a/26/streams/core-concepts.html b/26/streams/core-concepts.html
index 43e1502..d8cbd7a 100644
--- a/26/streams/core-concepts.html
+++ b/26/streams/core-concepts.html
@@ -128,7 +128,33 @@
</ul>
<p>
- Note, that the describe default behavior can be changed in the Processor API by assigning timestamps to output records explicitly when calling <code>#forward()</code>.
+ You can change the default behavior in the Processor API by assigning
+ timestamps to output records explicitly when calling <code>#forward()</code>.
+ </p>
+
+ <p>
+ For aggregations and joins, timestamps are computed by using the
+ following rules.
+ </p>
+
+ <p>
+ <ul>
+ <li>For joins (stream-stream, table-table) that have left and right
+ input records, the timestamp of the output record is assigned
+ <code>max(left.ts, right.ts)</code>.</li>
+ <li>For stream-table joins, the output record is assigned the timestamp
+ from the stream record.</li>
+ <li>For aggregations, Kafka Streams also computes the <code>max</code>
+ timestamp over all records, per key, either globally (for non-windowed)
+ or per-window.</li>
+ <li>For stateless operations, the input record timestamp is passed through.
+ For <code>flatMap</code> and siblings that emit multiple records, all
+ output records inherit the timestamp from the corresponding input record.</li>
+ </ul>
+ </p>
+
+ <p>
+
</p>
<h3><a id="streams_concepts_aggregations" href="#streams_concepts_aggregations">Aggregations</a></h3>
diff --git a/26/streams/developer-guide/processor-api.html b/26/streams/developer-guide/processor-api.html
index e7c6eac..d314c7d 100644
--- a/26/streams/developer-guide/processor-api.html
+++ b/26/streams/developer-guide/processor-api.html
@@ -49,6 +49,7 @@
<li><a class="reference internal" href="#defining-and-creating-a-state-store" id="id4">Defining and creating a State Store</a></li>
<li><a class="reference internal" href="#fault-tolerant-state-stores" id="id5">Fault-tolerant State Stores</a></li>
<li><a class="reference internal" href="#enable-or-disable-fault-tolerance-of-state-stores-store-changelogs" id="id6">Enable or Disable Fault Tolerance of State Stores (Store Changelogs)</a></li>
+ <li><a class="reference internal" href="#timestamped-state-stores" id="id11">Timestamped State Stores</a></li>
<li><a class="reference internal" href="#implementing-custom-state-stores" id="id7">Implementing Custom State Stores</a></li>
</ul>
</li>
@@ -229,6 +230,10 @@
<a class="reference internal" href="config-streams.html#streams-developer-guide-rocksdb-config"><span class="std std-ref">RocksDB configuration</span></a>.</li>
<li>Available <a class="reference external" href="/{{version}}/javadoc/org/apache/kafka/streams/state/Stores.html#persistentKeyValueStore-java.lang.String-">store variants</a>:
time window key-value store, session window key-value store.</li>
+ <li>Use <a class="reference external" href="/{{version}}/javadoc/org/apache/kafka/streams/state/Stores.html#persistentTimestampedKeyValueStore-java.lang.String-">persistentTimestampedKeyValueStore</a>
+ when you need a persistent key-(value/timestamp) store that supports put/get/delete and range queries.</li>
+ <li>Use <a class="reference external" href="/{{version}}/javadoc/org/apache/kafka/streams/state/Stores.html#persistentTimestampedWindowStore-java.lang.String-java.time.Duration-java.time.Duration-boolean-">persistentTimestampedWindowStore</a>
+ when you need a persistent windowedKey-(value/timestamp) store.</li>
</ul>
<div class="highlight-java"><div class="highlight"><pre><span></span><span class="c1">// Creating a persistent key-value store:</span>
<span class="c1">// here, we create a `KeyValueStore<String, Long>` named "persistent-counts".</span>
@@ -260,6 +265,10 @@
in-between app instance restarts.</li>
<li>Available <a class="reference external" href="/{{version}}/javadoc/org/apache/kafka/streams/state/Stores.html#inMemoryKeyValueStore-java.lang.String-">store variants</a>:
time window key-value store, session window key-value store.</li>
+ <li>Use <a class="reference external" href="/{{version}}/javadoc/org/apache/kafka/streams/state/TimestampedKeyValueStore.html">TimestampedKeyValueStore</a>
+ when you need a key-(value/timestamp) store that supports put/get/delete and range queries.</li>
+ <li>Use <a class="reference external" href="/{{version}}/javadoc/org/apache/kafka/streams/state/TimestampedWindowStore.html">TimestampedWindowStore</a>
+ when you need to store windowedKey-(value/timestamp) pairs.</li>
</ul>
<div class="highlight-java"><div class="highlight"><pre><span></span><span class="c1">// Creating an in-memory key-value store:</span>
<span class="c1">// here, we create a `KeyValueStore<String, Long>` named "inmemory-counts".</span>
@@ -343,6 +352,25 @@
</pre></div>
</div>
</div>
+ <div class="section" id="timestamped-state-stores">
+ <span id="streams-developer-guide-state-store-timestamps"></span><h3><a class="toc-backref" href="#id11">Timestamped State Stores</a><a class="headerlink" href="#timestamped-state-stores" title="Permalink to this headline"></a></h3>
+ <p>KTables always store timestamps by default.
+ A timestamped state store improves stream processing semantics and enables
+ handling out-of-order data in source KTables, detecting out-of-order joins and aggregations,
+ and getting the timestamp of the latest update in an Interactive Query.</p>
+ <p>You can query timestamped state stores both with and without a timestamp.</p>
+ <b>Upgrade note:</b> All users upgrade with a single rolling bounce per instance.
+ <ul class="first simple">
+ <li>For Processor API users, nothing changes in existing applications, and you
+ have the option of using the timestamped stores.</li>
+ <li>For DSL operators, store data is upgraded lazily in the background.</li>
+ <li>No upgrade happens if you provide a custom XxxBytesStoreSupplier, but you can opt-in
+ by implementing the <a class="reference external" href="/{{version}}/javadoc/org/apache/kafka/streams/state/TimestampedBytesStore.html">TimestampedBytesStore</a>
+ interface. In this case, the old format is retained, and Streams uses a proxy store
+ that removes/adds timestamps on read/write.</li>
+ </ul>
+ </p>
+ </div>
<div class="section" id="implementing-custom-state-stores">
<span id="streams-developer-guide-state-store-custom"></span><h3><a class="toc-backref" href="#id7">Implementing Custom State Stores</a><a class="headerlink" href="#implementing-custom-state-stores" title="Permalink to this headline"></a></h3>
<p>You can use the <a class="reference internal" href="#streams-developer-guide-state-store-defining"><span class="std std-ref">built-in state store types</span></a> or implement your own.
diff --git a/26/streams/upgrade-guide.html b/26/streams/upgrade-guide.html
index be7a95b..b1af752 100644
--- a/26/streams/upgrade-guide.html
+++ b/26/streams/upgrade-guide.html
@@ -86,14 +86,6 @@
More details about the new config <code>StreamsConfig#TOPOLOGY_OPTIMIZATION</code> can be found in <a href="https://cwiki.apache.org/confluence/display/KAFKA/KIP-295%3A+Add+Streams+Configuration+Allowing+for+Optional+Topology+Optimization">KIP-295</a>.
</p>
- <h3><a id="streams_api_changes_270" href="#streams_api_changes_270">Streams API changes in 2.7.0</a></h3>
- <p>
- The <code>StreamsConfig</code> variable for configuration parameter <code>"topology.optimization"</code>
- is renamed from <code>TOPOLOGY_OPTIMIZATION</code> to <code>TOPOLOGY_OPTIMIZATION_CONFIG</code>.
- The old variable is deprecated. Note, that the parameter name itself is not affected.
- (Cf. <a href="https://cwiki.apache.org/confluence/display/KAFKA/KIP-626%3A+Rename+StreamsConfig+config+variable+name">KIP-629</a>.)
- </p>
-
<h3><a id="streams_api_changes_260" href="#streams_api_changes_260">Streams API changes in 2.6.0</a></h3>
<p>
We added a new processing mode that improves application scalability using exactly-once guarantees