You are viewing a plain text version of this content. The canonical link for it is here.
Posted to jira@kafka.apache.org by GitBox <gi...@apache.org> on 2020/11/03 17:48:23 UTC
[GitHub] [kafka] mjsax commented on a change in pull request #9551: KAFKA-10679: Migrate upgrade changes from site to kafka/docs
mjsax commented on a change in pull request #9551:
URL: https://github.com/apache/kafka/pull/9551#discussion_r516843574
##########
File path: docs/design.html
##########
@@ -508,7 +506,7 @@ <h4><a id="design_compactionbasics" href="#design_compactionbasics">Log Compacti
the log, even if the message with that offset has been compacted away; in this case this position is indistinguishable from the next highest offset that does appear in the log. For example, in the picture above the
offsets 36, 37, and 38 are all equivalent positions and a read beginning at any of these offsets would return a message set beginning with 38.
<p>
- Compaction also allows for deletes. A message with a key and a null payload will be treated as a delete from the log. Such a record is sometimes referred to as a <i>tombstone</i>. This delete marker will cause any prior message with that key to be removed (as would any new
+ Compaction also allows for deletes. A message with a key and a null payload will be treated as a delete from the log. This delete marker will cause any prior message with that key to be removed (as would any new
Review comment:
Why was this sentence removed?
##########
File path: docs/introduction.html
##########
@@ -18,198 +18,203 @@
<script><!--#include virtual="js/templateData.js" --></script>
<script id="introduction-template" type="text/x-handlebars-template">
- <h3> Apache Kafka® is <i>a distributed streaming platform</i>. What exactly does that mean?</h3>
- <p>A streaming platform has three key capabilities:</p>
- <ul>
- <li>Publish and subscribe to streams of records, similar to a message queue or enterprise messaging system.
- <li>Store streams of records in a fault-tolerant durable way.
- <li>Process streams of records as they occur.
- </ul>
- <p>Kafka is generally used for two broad classes of applications:</p>
- <ul>
- <li>Building real-time streaming data pipelines that reliably get data between systems or applications
- <li>Building real-time streaming applications that transform or react to the streams of data
- </ul>
- <p>To understand how Kafka does these things, let's dive in and explore Kafka's capabilities from the bottom up.</p>
- <p>First a few concepts:</p>
- <ul>
- <li>Kafka runs as a cluster on one or more servers that can span multiple datacenters.
- <li>The Kafka cluster stores streams of <i>records</i> in categories called <i>topics</i>.
- <li>Each record consists of a key, a value, and a timestamp.
- </ul>
- <p>Kafka has five core APIs:</p>
- <div style="overflow: hidden;">
- <ul style="float: left; width: 40%;">
- <li>The <a href="/documentation.html#producerapi">Producer API</a> allows an application to publish a stream of records to one or more Kafka topics.
- <li>The <a href="/documentation.html#consumerapi">Consumer API</a> allows an application to subscribe to one or more topics and process the stream of records produced to them.
- <li>The <a href="/documentation/streams">Streams API</a> allows an application to act as a <i>stream processor</i>, consuming an input stream from one or more topics and producing an output stream to one or more output topics, effectively transforming the input streams to output streams.
- <li>The <a href="/documentation.html#connect">Connector API</a> allows building and running reusable producers or consumers that connect Kafka topics to existing applications or data systems. For example, a connector to a relational database might capture every change to a table.
- <li>The <a href="/documentation.html#adminapi">Admin API</a> allows managing and inspecting topics, brokers and other Kafka objects.
- </ul>
- <img src="/{{version}}/images/kafka-apis.png" style="float: right; width: 50%;">
- </div>
- <p>
- In Kafka the communication between the clients and the servers is done with a simple, high-performance, language agnostic <a href="https://kafka.apache.org/protocol.html">TCP protocol</a>. This protocol is versioned and maintains backwards compatibility with older versions. We provide a Java client for Kafka, but clients are available in <a href="https://cwiki.apache.org/confluence/display/KAFKA/Clients">many languages</a>.</p>
-
- <h4><a id="intro_topics" href="#intro_topics">Topics and Logs</a></h4>
- <p>Let's first dive into the core abstraction Kafka provides for a stream of records—the topic.</p>
- <p>A topic is a category or feed name to which records are published. Topics in Kafka are always multi-subscriber; that is, a topic can have zero, one, or many consumers that subscribe to the data written to it.</p>
- <p> For each topic, the Kafka cluster maintains a partitioned log that looks like this: </p>
- <img class="centered" src="/{{version}}/images/log_anatomy.png">
-
- <p> Each partition is an ordered, immutable sequence of records that is continually appended to—a structured commit log. The records in the partitions are each assigned a sequential id number called the <i>offset</i> that uniquely identifies each record within the partition.
- </p>
+ <h4 class="anchor-heading">
Review comment:
Cannot really verify those changes. But I guess, should be ok.
##########
File path: docs/quickstart-docker.html
##########
@@ -0,0 +1,204 @@
+<!--
Review comment:
Cannot really verify this one.
##########
File path: docs/quickstart-zookeeper.html
##########
@@ -0,0 +1,277 @@
+<!--
Review comment:
Cannot really verify this one.
##########
File path: docs/ops.html
##########
@@ -1658,9 +1564,9 @@ <h4><a id="kafka_streams_monitoring" href="#kafka_streams_monitoring">Streams Mo
Use the following configuration option to specify which metrics
you want collected:
-<pre>metrics.recording.level="info"</pre>
+<pre>metrics.recording.level="info"</code></pre>
Review comment:
Missing opening `<code>` tag
##########
File path: docs/design.html
##########
@@ -543,25 +544,25 @@ <h4><a id="design_compactiondetails" href="#design_compactiondetails">Log Compac
(assuming 1k messages).
</ol>
<p>
- <h4><a id="design_compactionconfig" href="#design_compactionconfig">Configuring The Log Cleaner</a></h4>
+ <h4 class="anchor-heading"><a id="design_compactionconfig" class="anchor-link"></a><a href="#design_compactionconfig">Configuring The Log Cleaner</a></h4>
The log cleaner is enabled by default. This will start the pool of cleaner threads.
To enable log cleaning on a particular topic, add the log-specific property
- <pre class="brush: text;"> log.cleanup.policy=compact</pre>
+ <pre class="language-text"> log.cleanup.policy=compact</code></pre>
Review comment:
Seems we are missing a opening `<code>` tag? Similar the next two changes below.
##########
File path: docs/ops.html
##########
@@ -78,37 +68,31 @@ <h4><a id="basic_ops_restarting" href="#basic_ops_restarting">Graceful shutdown<
</ol>
Syncing the logs will happen automatically whenever the server is stopped other than by a hard kill, but the controlled leadership migration requires using a special setting:
- <pre class="brush: text;">
- controlled.shutdown.enable=true
- </pre>
+ <pre class="line-numbers"><code class="language-text"> controlled.shutdown.enable=true</code></pre>
Note that controlled shutdown will only succeed if <i>all</i> the partitions hosted on the broker have replicas (i.e. the replication factor is greater than 1 <i>and</i> at least one of these replicas is alive). This is generally what you want since shutting down the last replica would make that topic partition unavailable.
- <h4><a id="basic_ops_leader_balancing" href="#basic_ops_leader_balancing">Balancing leadership</a></h4>
+ <h4 class="anchor-heading"><a id="basic_ops_leader_balancing" class="anchor-link"></a><a href="#basic_ops_leader_balancing">Balancing leadership</a></h4>
Whenever a broker stops or crashes, leadership for that broker's partitions transfers to other replicas. When the broker is restarted it will only be a follower for all its partitions, meaning it will not be used for client reads and writes.
<p>
To avoid this imbalance, Kafka has a notion of preferred replicas. If the list of replicas for a partition is 1,5,9 then node 1 is preferred as the leader to either node 5 or 9 because it is earlier in the replica list. By default the Kafka cluster will try to restore leadership to the restored replicas. This behaviour is configured with:
- <pre class="brush: text;">
- auto.leader.rebalance.enable=true
- </pre>
+ <pre class="line-numbers"><code class="language-text"> auto.leader.rebalance.enable=true</code></pre>
You can also set this to false, but you will then need to manually restore leadership to the restored replicas by running the command:
- <pre class="brush: bash;">
- > bin/kafka-preferred-replica-election.sh --bootstrap-server broker_host:port
- </pre>
+ <pre class="line-numbers"><code class="language-bash"> > bin/kafka-preferred-replica-election.sh --bootstrap-server broker_host:port</code></pre>
- <h4><a id="basic_ops_racks" href="#basic_ops_racks">Balancing Replicas Across Racks</a></h4>
+ <h4 class="anchor-heading"><a id="basic_ops_racks" class="anchor-link"></a><a href="#basic_ops_racks">Balancing Replicas Across Racks</a></h4>
The rack awareness feature spreads replicas of the same partition across different racks. This extends the guarantees Kafka provides for broker-failure to cover rack-failure, limiting the risk of data loss should all the brokers on a rack fail at once. The feature can also be applied to other broker groupings such as availability zones in EC2.
<p></p>
You can specify that a broker belongs to a particular rack by adding a property to the broker config:
- <pre class="brush: text;"> broker.rack=my-rack-id</pre>
+ <pre class="language-text"> broker.rack=my-rack-id</code></pre>
Review comment:
Missing opening `<code>` tag
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org