You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@kafka.apache.org by da...@apache.org on 2023/02/08 18:35:17 UTC
[kafka-site] branch asf-site updated: KAFKA-14692 Fix some errors and omissions in the migration docs (#487)
This is an automated email from the ASF dual-hosted git repository.
davidarthur pushed a commit to branch asf-site
in repository https://gitbox.apache.org/repos/asf/kafka-site.git
The following commit(s) were added to refs/heads/asf-site by this push:
new f1cee4a0 KAFKA-14692 Fix some errors and omissions in the migration docs (#487)
f1cee4a0 is described below
commit f1cee4a06e32dc1f4d83917d745ee89dc81508b9
Author: David Arthur <mu...@gmail.com>
AuthorDate: Wed Feb 8 13:35:12 2023 -0500
KAFKA-14692 Fix some errors and omissions in the migration docs (#487)
Reviewers: Mickael Maison <mi...@gmail.com>
---
34/ops.html | 130 ++++++++++++++++++++++++++++++++----------------------------
1 file changed, 70 insertions(+), 60 deletions(-)
diff --git a/34/ops.html b/34/ops.html
index 9f35e49e..24581464 100644
--- a/34/ops.html
+++ b/34/ops.html
@@ -3580,6 +3580,11 @@ foo
<li>Other features <a href="#kraft_missing">not yet supported in KRaft</a></li>
</ul>
+ <p>
+ Please report issues with ZooKeeper to KRaft migration using the
+ <a href="https://issues.apache.org/jira/projects/KAFKA" target="_blank">project JIRA</a> and the "kraft" component.
+ </p>
+
<h3>Terminology</h3>
<p>
In this documentation, we use the term "migration" to refer to the process to changing a Kafka cluster's metadata
@@ -3604,9 +3609,7 @@ foo
be done by adding the following log4j configuration to each KRaft controller's "log4j.properties" file.
</p>
- <pre>
- log4j.logger.org.apache.kafka.metadata.migration=TRACE
- </pre>
+ <pre>log4j.logger.org.apache.kafka.metadata.migration=TRACE</pre>
<p>
It may generally useful to enable DEBUG logging on the KRaft controllers and the ZK brokers during the migration.
@@ -3615,28 +3618,38 @@ foo
<h3>Provisioning the KRaft controller quorum</h3>
<p>
Two things are needed before the migration can begin. The brokers must be configured to support the migration and
- a KRaft controller quorum must be deployed. For the KRaft deployment, please refer to <a href="#kraft_config">the above documentation</a>.
- The KRaft controllers should be provisioned with the latest <code>metadata.version</code> of "3.4".
+ a KRaft controller quorum must be deployed. The KRaft controllers should be provisioned with the same cluster ID as
+ the existing Kafka cluster. This can be found by examining one of the "meta.properties" files in the data directories
+ of the brokers, or by running the following command.
+ </p>
+
+ <pre>./bin/zookeeper-shell.sh localhost:2181 get /cluster/id</pre>
+
+ <p>
+ The KRaft controller quorum should also be provisioned with the latest <code>metadata.version</code> of "3.4".
+ For further instructions on KRaft deployment, please refer to <a href="#kraft_config">the above documentation</a>.
+ </p>
+
+ <p>
In addition to the standard KRaft configuration, the KRaft controllers will need to enable support for the migration
as well as provide ZooKeeper connection configuration.
</p>
<pre>
- # Sample KRaft cluster controller.properties listening on 9093
- process.roles=controller
- node.id=3000
- controller.quorum.voters=1@localhost:9093
- controller.listener.names=CONTROLLER
- listeners=CONTROLLER://:9093
+# Sample KRaft cluster controller.properties listening on 9093
+process.roles=controller
+node.id=3000
+controller.quorum.voters=3000@localhost:9093
+controller.listener.names=CONTROLLER
+listeners=CONTROLLER://:9093
- # Enable the migration
- zookeeper.metadata.migration.enable=true
+# Enable the migration
+zookeeper.metadata.migration.enable=true
- # ZooKeeper client configuration
- zookeeper.connect=localhost:2181
+# ZooKeeper client configuration
+zookeeper.connect=localhost:2181
- # Other configs ...
- </pre>
+# Other configs ...</pre>
<p><em>Note: The KRaft cluster <code>node.id</code> values must be different from any existing ZK broker <code>broker.id</code>.
In KRaft-mode, the brokers and controllers share the same Node ID namespace.</em></p>
@@ -3658,25 +3671,24 @@ foo
<p>Here is a sample config for a broker that is ready for migration:</p>
<pre>
- # Sample ZK broker server.properties listening on 9092
- broker.id=0
- listeners=PLAINTEXT://:9092
- advertised.listeners=PLAINTEXT://localhost:9092
- listener.security.protocol.map=PLAINTEXT:PLAINTEXT,CONTROLLER:PLAINTEXT
+# Sample ZK broker server.properties listening on 9092
+broker.id=0
+listeners=PLAINTEXT://:9092
+advertised.listeners=PLAINTEXT://localhost:9092
+listener.security.protocol.map=PLAINTEXT:PLAINTEXT,CONTROLLER:PLAINTEXT
- # Set the IBP
- inter.broker.protocol.version=3.4
+# Set the IBP
+inter.broker.protocol.version=3.4
- # Enable the migration
- zookeeper.metadata.migration.enable=true
+# Enable the migration
+zookeeper.metadata.migration.enable=true
- # ZooKeeper client configuration
- zookeeper.connect=localhost:2181
+# ZooKeeper client configuration
+zookeeper.connect=localhost:2181
- # KRaft controller quorum configuration
- controller.quorum.voters=3000@localhost:9093
- controller.listener.names=CONTROLLER
- </pre>
+# KRaft controller quorum configuration
+controller.quorum.voters=3000@localhost:9093
+controller.listener.names=CONTROLLER</pre>
<p>
<em>Note: Once the final ZK broker has been restarted with the necessary configuration, the migration will automatically begin.</em>
@@ -3700,26 +3712,25 @@ foo
</p>
<pre>
- # Sample KRaft broker server.properties listening on 9092
- process.roles=broker
- node.id=0
- listeners=PLAINTEXT://:9092
- advertised.listeners=PLAINTEXT://localhost:9092
- listener.security.protocol.map=PLAINTEXT:PLAINTEXT,CONTROLLER:PLAINTEXT
+# Sample KRaft broker server.properties listening on 9092
+process.roles=broker
+node.id=0
+listeners=PLAINTEXT://:9092
+advertised.listeners=PLAINTEXT://localhost:9092
+listener.security.protocol.map=PLAINTEXT:PLAINTEXT,CONTROLLER:PLAINTEXT
- # Don't set the IBP, KRaft uses "metadata.version" feature flag
- # inter.broker.protocol.version=3.4
+# Don't set the IBP, KRaft uses "metadata.version" feature flag
+# inter.broker.protocol.version=3.4
- # Keep the migration enabled
- zookeeper.metadata.migration.enable=true
+# Remove the migration enabled flag
+# zookeeper.metadata.migration.enable=true
- # Remove ZooKeeper client configuration
- # zookeeper.connect=localhost:2181
+# Remove ZooKeeper client configuration
+# zookeeper.connect=localhost:2181
- # Keep the KRaft controller quorum configuration
- controller.quorum.voters=3000@localhost:9093
- controller.listener.names=CONTROLLER
- </pre>
+# Keep the KRaft controller quorum configuration
+controller.quorum.voters=3000@localhost:9093
+controller.listener.names=CONTROLLER</pre>
<p>
Each broker is restarted with a KRaft configuration until the entire cluster is running in KRaft mode.
@@ -3733,21 +3744,20 @@ foo
</p>
<pre>
- # Sample KRaft cluster controller.properties listening on 9093
- process.roles=controller
- node.id=3000
- controller.quorum.voters=1@localhost:9093
- controller.listener.names=CONTROLLER
- listeners=CONTROLLER://:9093
+# Sample KRaft cluster controller.properties listening on 9093
+process.roles=controller
+node.id=3000
+controller.quorum.voters=1@localhost:9093
+controller.listener.names=CONTROLLER
+listeners=CONTROLLER://:9093
- # Disable the migration
- # zookeeper.metadata.migration.enable=true
+# Disable the migration
+# zookeeper.metadata.migration.enable=true
- # Remove ZooKeeper client configuration
- # zookeeper.connect=localhost:2181
+# Remove ZooKeeper client configuration
+# zookeeper.connect=localhost:2181
- # Other configs ...
- </pre>
+# Other configs ...</pre>
</script>