You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@kafka.apache.org by bb...@apache.org on 2020/10/31 12:10:06 UTC
[kafka] branch 2.7 updated: MINOR: Move upgraded docs from site to
kafak docs (#9532)
This is an automated email from the ASF dual-hosted git repository.
bbejeck pushed a commit to branch 2.7
in repository https://gitbox.apache.org/repos/asf/kafka.git
The following commit(s) were added to refs/heads/2.7 by this push:
new c9033ff MINOR: Move upgraded docs from site to kafak docs (#9532)
c9033ff is described below
commit c9033ffddb1aa5c740921137e21a78597788dcd9
Author: Bill Bejeck <bb...@gmail.com>
AuthorDate: Sat Oct 31 07:52:14 2020 -0400
MINOR: Move upgraded docs from site to kafak docs (#9532)
Reviewers: Matthias J. Sax <mj...@apache.org>
---
docs/documentation.html | 45 ++++++++--------
docs/upgrade.html | 137 ++++++++++++++++++++++++++++++++++++++++++++----
2 files changed, 148 insertions(+), 34 deletions(-)
diff --git a/docs/documentation.html b/docs/documentation.html
index 8999be58..afd4955 100644
--- a/docs/documentation.html
+++ b/docs/documentation.html
@@ -22,50 +22,47 @@
<div class="content documentation documentation--current">
- <!--#include virtual="../includes/_nav.htm" -->
- <div class="right">
- <!--#include virtual="../includes/_docs_banner.htm" -->
- <h1>Documentation</h1>
- <h3>Kafka 2.7 Documentation</h3>
- Prior releases: <a href="/07/documentation.html">0.7.x</a>, <a href="/08/documentation.html">0.8.0</a>, <a href="/081/documentation.html">0.8.1.X</a>, <a href="/082/documentation.html">0.8.2.X</a>, <a href="/090/documentation.html">0.9.0.X</a>, <a href="/0100/documentation.html">0.10.0.X</a>, <a href="/0101/documentation.html">0.10.1.X</a>, <a href="/0102/documentation.html">0.10.2.X</a>, <a href="/0110/documentation.html">0.11.0.X</a>, <a href="/10/documentation.html">1.0.X</a>, [...]
+ <!--#include virtual="../includes/_nav.htm" -->
+ <div class="toc-handle-container">
+ <div class="toc-handle"><</div>
+ </div>
+ <div class="docs-nav">
<!--#include virtual="toc.html" -->
+ </div>
+ <div class="right">
+ <!--//#include virtual="../includes/_docs_banner.htm" -->
+
+ <h1>Documentation</h1>
+ <h3>Kafka 2.7 Documentation</h3>
+ Prior releases: <a href="/07/documentation.html">0.7.x</a>, <a href="/08/documentation.html">0.8.0</a>, <a href="/081/documentation.html">0.8.1.X</a>, <a href="/082/documentation.html">0.8.2.X</a>, <a href="/090/documentation.html">0.9.0.X</a>, <a href="/0100/documentation.html">0.10.0.X</a>, <a href="/0101/documentation.html">0.10.1.X</a>, <a href="/0102/documentation.html">0.10.2.X</a>, <a href="/0110/documentation.html">0.11.0.X</a>, <a href="/10/documentation.html">1.0.X</a>, [...]
- <h2><a id="gettingStarted" href="#gettingStarted">1. Getting Started</a></h2>
- <h3><a id="introduction" href="#introduction">1.1 Introduction</a></h3>
- <!--#include virtual="introduction.html" -->
- <h3><a id="uses" href="#uses">1.2 Use Cases</a></h3>
- <!--#include virtual="uses.html" -->
- <h3><a id="quickstart" href="#quickstart">1.3 Quick Start</a></h3>
- <!--#include virtual="quickstart.html" -->
- <h3><a id="ecosystem" href="#ecosystem">1.4 Ecosystem</a></h3>
- <!--#include virtual="ecosystem.html" -->
- <h3><a id="upgrade" href="#upgrade">1.5 Upgrading From Previous Versions</a></h3>
- <!--#include virtual="upgrade.html" -->
+ <h2 class="anchor-heading"><a id="gettingStarted" class="anchor-link"></a><a href="#gettingStarted">1. Getting Started</a></h2>
+ <h3 class="anchor-heading"><a id="introduction" class="anchor-link"></a><a href="#introduction">1.1 Introduction</a></h3>
- <h2><a id="api" href="#api">2. APIs</a></h2>
+ <h2 class="anchor-heading"><a id="api" class="anchor-link"></a><a href="#api">2. APIs</a></h2>
<!--#include virtual="api.html" -->
- <h2><a id="configuration" href="#configuration">3. Configuration</a></h2>
+ <h2 class="anchor-heading"><a id="configuration" class="anchor-link"></a><a href="#configuration">3. Configuration</a></h2>
<!--#include virtual="configuration.html" -->
- <h2><a id="design" href="#design">4. Design</a></h2>
+ <h2 class="anchor-heading"><a id="design" class="anchor-link"></a><a href="#design">4. Design</a></h2>
<!--#include virtual="design.html" -->
- <h2><a id="implementation" href="#implementation">5. Implementation</a></h2>
+ <h2 class="anchor-heading"><a id="implementation" class="anchor-link"></a><a href="#implementation">5. Implementation</a></h2>
<!--#include virtual="implementation.html" -->
- <h2><a id="operations" href="#operations">6. Operations</a></h2>
+ <h2 class="anchor-heading"><a id="operations" class="anchor-link"></a><a href="#operations">6. Operations</a></h2>
<!--#include virtual="ops.html" -->
- <h2><a id="security" href="#security">7. Security</a></h2>
+ <h2 class="anchor-heading"><a id="security" class="anchor-link"></a><a href="#security">7. Security</a></h2>
<!--#include virtual="security.html" -->
- <h2><a id="connect" href="#connect">8. Kafka Connect</a></h2>
+ <h2 class="anchor-heading"><a id="connect" class="anchor-link"></a><a href="#connect">8. Kafka Connect</a></h2>
<!--#include virtual="connect.html" -->
<h2><a id="streams" href="/documentation/streams">9. Kafka Streams</a></h2>
diff --git a/docs/upgrade.html b/docs/upgrade.html
index d877abc..7743a02 100644
--- a/docs/upgrade.html
+++ b/docs/upgrade.html
@@ -19,6 +19,48 @@
<script id="upgrade-template" type="text/x-handlebars-template">
+<h4><a id="upgrade_2_7_0" href="#upgrade_2_7_0">Upgrading to 2.7.0 from any version 0.8.x through 2.6.x</a></h4>
+
+<p><b>If you are upgrading from a version prior to 2.1.x, please see the note below about the change to the schema used to store consumer offsets.
+ Once you have changed the inter.broker.protocol.version to the latest version, it will not be possible to downgrade to a version prior to 2.1.</b></p>
+
+<p><b>For a rolling upgrade:</b></p>
+
+<ol>
+ <li> Update server.properties on all brokers and add the following properties. CURRENT_KAFKA_VERSION refers to the version you
+ are upgrading from. CURRENT_MESSAGE_FORMAT_VERSION refers to the message format version currently in use. If you have previously
+ overridden the message format version, you should keep its current value. Alternatively, if you are upgrading from a version prior
+ to 0.11.0.x, then CURRENT_MESSAGE_FORMAT_VERSION should be set to match CURRENT_KAFKA_VERSION.
+ <ul>
+ <li>inter.broker.protocol.version=CURRENT_KAFKA_VERSION (e.g., <code>2.6</code>, <code>2.5</code>, etc.)</li>
+ <li>log.message.format.version=CURRENT_MESSAGE_FORMAT_VERSION (See <a href="#upgrade_10_performance_impact">potential performance impact
+ following the upgrade</a> for the details on what this configuration does.)</li>
+ </ul>
+ If you are upgrading from version 0.11.0.x or above, and you have not overridden the message format, then you only need to override
+ the inter-broker protocol version.
+ <ul>
+ <li>inter.broker.protocol.version=CURRENT_KAFKA_VERSION (e.g., <code>2.6</code>, <code>2.5</code>, etc.)</li>
+ </ul>
+ </li>
+ <li> Upgrade the brokers one at a time: shut down the broker, update the code, and restart it. Once you have done so, the
+ brokers will be running the latest version and you can verify that the cluster's behavior and performance meets expectations.
+ It is still possible to downgrade at this point if there are any problems.
+ </li>
+ <li> Once the cluster's behavior and performance has been verified, bump the protocol version by editing
+ <code>inter.broker.protocol.version</code> and setting it to <code>2.7</code>.
+ </li>
+ <li> Restart the brokers one by one for the new protocol version to take effect. Once the brokers begin using the latest
+ protocol version, it will no longer be possible to downgrade the cluster to an older version.
+ </li>
+ <li> If you have overridden the message format version as instructed above, then you need to do one more rolling restart to
+ upgrade it to its latest version. Once all (or most) consumers have been upgraded to 0.11.0 or later,
+ change log.message.format.version to 2.7 on each broker and restart them one by one. Note that the older Scala clients,
+ which are no longer maintained, do not support the message format introduced in 0.11, so to avoid conversion costs
+ (or to take advantage of <a href="#upgrade_11_exactly_once_semantics">exactly once semantics</a>),
+ the newer Java clients must be used.
+ </li>
+</ol>
+
<h5><a id="upgrade_270_notable" href="#upgrade_270_notable">Notable changes in 2.7.0</a></h5>
<ul>
<li>
@@ -91,8 +133,49 @@
<a href="https://cwiki.apache.org/confluence/display/KAFKA/KIP-616%3A+Rename+implicit+Serdes+instances+in+kafka-streams-scala">KIP-616</a>
</li>
</ul>
+<h4><a id="upgrade_2_6_0" href="#upgrade_2_6_0">Upgrading to 2.6.0 from any version 0.8.x through 2.5.x</a></h4>
+
+<p><b>If you are upgrading from a version prior to 2.1.x, please see the note below about the change to the schema used to store consumer offsets.
+ Once you have changed the inter.broker.protocol.version to the latest version, it will not be possible to downgrade to a version prior to 2.1.</b></p>
+
+<p><b>For a rolling upgrade:</b></p>
-<h5><a id="upgrade_260_notable" href="#upgrade_260_notable">Notable changes in 2.6.0</a></h5>
+<ol>
+ <li> Update server.properties on all brokers and add the following properties. CURRENT_KAFKA_VERSION refers to the version you
+ are upgrading from. CURRENT_MESSAGE_FORMAT_VERSION refers to the message format version currently in use. If you have previously
+ overridden the message format version, you should keep its current value. Alternatively, if you are upgrading from a version prior
+ to 0.11.0.x, then CURRENT_MESSAGE_FORMAT_VERSION should be set to match CURRENT_KAFKA_VERSION.
+ <ul>
+ <li>inter.broker.protocol.version=CURRENT_KAFKA_VERSION (e.g., <code>2.5</code>, <code>2.4</code>, etc.)</li>
+ <li>log.message.format.version=CURRENT_MESSAGE_FORMAT_VERSION (See <a href="#upgrade_10_performance_impact">potential performance impact
+ following the upgrade</a> for the details on what this configuration does.)</li>
+ </ul>
+ If you are upgrading from version 0.11.0.x or above, and you have not overridden the message format, then you only need to override
+ the inter-broker protocol version.
+ <ul>
+ <li>inter.broker.protocol.version=CURRENT_KAFKA_VERSION (e.g., <code>2.5</code>, <code>2.4</code>, etc.)</li>
+ </ul>
+ </li>
+ <li> Upgrade the brokers one at a time: shut down the broker, update the code, and restart it. Once you have done so, the
+ brokers will be running the latest version and you can verify that the cluster's behavior and performance meets expectations.
+ It is still possible to downgrade at this point if there are any problems.
+ </li>
+ <li> Once the cluster's behavior and performance has been verified, bump the protocol version by editing
+ <code>inter.broker.protocol.version</code> and setting it to <code>2.6</code>.
+ </li>
+ <li> Restart the brokers one by one for the new protocol version to take effect. Once the brokers begin using the latest
+ protocol version, it will no longer be possible to downgrade the cluster to an older version.
+ </li>
+ <li> If you have overridden the message format version as instructed above, then you need to do one more rolling restart to
+ upgrade it to its latest version. Once all (or most) consumers have been upgraded to 0.11.0 or later,
+ change log.message.format.version to 2.6 on each broker and restart them one by one. Note that the older Scala clients,
+ which are no longer maintained, do not support the message format introduced in 0.11, so to avoid conversion costs
+ (or to take advantage of <a href="#upgrade_11_exactly_once_semantics">exactly once semantics</a>),
+ the newer Java clients must be used.
+ </li>
+</ol>
+
+<h5 class="anchor-heading"><a id="upgrade_260_notable" class="anchor-link"></a><a href="#upgrade_260_notable">Notable changes in 2.6.0</a></h5>
<ul>
<li>Kafka Streams adds a new processing mode (requires broker 2.5 or newer) that improves application
scalability using exactly-once guarantees
@@ -112,17 +195,51 @@
Fetch requests and other requests intended only for the leader or follower return NOT_LEADER_OR_FOLLOWER(6) instead of REPLICA_NOT_AVAILABLE(9)
if the broker is not a replica, ensuring that this transient error during reassignments is handled by all clients as a retriable exception.
</li>
- <li>There are several notable changes to the reassignment tool <code>kafka-reassign-partitions.sh</code>
- following the completion of
- <a href="https://cwiki.apache.org/confluence/display/KAFKA/KIP-455%3A+Create+an+Administrative+API+for+Replica+Reassignment">KIP-455</a>.
- This tool now requires the <code>--additional</code> flag to be provided when changing the throttle of an
- active reassignment. Reassignment cancellation is now possible using the
- <code>--cancel</code> command. Finally, reassignment with <code>--zookeeper</code>
- has been deprecated in favor of <code>--bootstrap-server</code>. See the KIP for more detail.
- </li>
</ul>
-<h5><a id="upgrade_250_notable" href="#upgrade_250_notable">Notable changes in 2.5.0</a></h5>
+<h4><a id="upgrade_2_5_0" href="#upgrade_2_5_0">Upgrading to 2.5.0 from any version 0.8.x through 2.4.x</a></h4>
+
+<p><b>If you are upgrading from a version prior to 2.1.x, please see the note below about the change to the schema used to store consumer offsets.
+ Once you have changed the inter.broker.protocol.version to the latest version, it will not be possible to downgrade to a version prior to 2.1.</b></p>
+
+<p><b>For a rolling upgrade:</b></p>
+
+<ol>
+ <li> Update server.properties on all brokers and add the following properties. CURRENT_KAFKA_VERSION refers to the version you
+ are upgrading from. CURRENT_MESSAGE_FORMAT_VERSION refers to the message format version currently in use. If you have previously
+ overridden the message format version, you should keep its current value. Alternatively, if you are upgrading from a version prior
+ to 0.11.0.x, then CURRENT_MESSAGE_FORMAT_VERSION should be set to match CURRENT_KAFKA_VERSION.
+ <ul>
+ <li>inter.broker.protocol.version=CURRENT_KAFKA_VERSION (e.g., <code>2.4</code>, <code>2.3</code>, etc.)</li>
+ <li>log.message.format.version=CURRENT_MESSAGE_FORMAT_VERSION (See <a href="#upgrade_10_performance_impact">potential performance impact
+ following the upgrade</a> for the details on what this configuration does.)</li>
+ </ul>
+ If you are upgrading from version 0.11.0.x or above, and you have not overridden the message format, then you only need to override
+ the inter-broker protocol version.
+ <ul>
+ <li>inter.broker.protocol.version=CURRENT_KAFKA_VERSION (e.g., <code>2.4</code>, <code>2.3</code>, etc.)</li>
+ </ul>
+ </li>
+ <li> Upgrade the brokers one at a time: shut down the broker, update the code, and restart it. Once you have done so, the
+ brokers will be running the latest version and you can verify that the cluster's behavior and performance meets expectations.
+ It is still possible to downgrade at this point if there are any problems.
+ </li>
+ <li> Once the cluster's behavior and performance has been verified, bump the protocol version by editing
+ <code>inter.broker.protocol.version</code> and setting it to <code>2.5</code>.
+ </li>
+ <li> Restart the brokers one by one for the new protocol version to take effect. Once the brokers begin using the latest
+ protocol version, it will no longer be possible to downgrade the cluster to an older version.
+ </li>
+ <li> If you have overridden the message format version as instructed above, then you need to do one more rolling restart to
+ upgrade it to its latest version. Once all (or most) consumers have been upgraded to 0.11.0 or later,
+ change log.message.format.version to 2.5 on each broker and restart them one by one. Note that the older Scala clients,
+ which are no longer maintained, do not support the message format introduced in 0.11, so to avoid conversion costs
+ (or to take advantage of <a href="#upgrade_11_exactly_once_semantics">exactly once semantics</a>),
+ the newer Java clients must be used.
+ </li>
+</ol>
+
+<h5 class="anchor-heading"><a id="upgrade_250_notable" class="anchor-link"></a><a href="#upgrade_250_notable">Notable changes in 2.5.0</a></h5>
<ul>
<li>When <code>RebalanceProtocol#COOPERATIVE</code> is used, <code>Consumer#poll</code> can still return data
while it is in the middle of a rebalance for those partitions still owned by the consumer; in addition