You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@kafka.apache.org by jk...@apache.org on 2015/02/12 18:48:28 UTC
svn commit: r1659337 - in /kafka/site/082: design.html ops.html
quickstart.html upgrade.html uses.html
Author: jkreps
Date: Thu Feb 12 17:48:27 2015
New Revision: 1659337
URL: http://svn.apache.org/r1659337
Log:
KAFKA-1938: Fix quickstart references to 0.8.1 and misc. other issues.
Modified:
kafka/site/082/design.html
kafka/site/082/ops.html
kafka/site/082/quickstart.html
kafka/site/082/upgrade.html
kafka/site/082/uses.html
Modified: kafka/site/082/design.html
URL: http://svn.apache.org/viewvc/kafka/site/082/design.html?rev=1659337&r1=1659336&r2=1659337&view=diff
==============================================================================
--- kafka/site/082/design.html (original)
+++ kafka/site/082/design.html Thu Feb 12 17:48:27 2015
@@ -95,9 +95,9 @@ The client controls which partition it p
<h4>Asynchronous send</h4>
<p>
-Batching is one of the big drivers of efficiency, and to enable batching the Kafka producer has an asynchronous mode that accumulates data in memory and sends out larger batches in a single request. The batching can be configured to accumulate no more than a fixed number of messages and to wait no longer than some fixed latency bound (say 100 messages or 5 seconds). This allows the accumulation of more bytes to send, and few larger I/O operations on the servers. Since this buffering happens in the client it obviously reduces the durability as any data buffered in memory and not yet sent will be lost in the event of a producer crash.
+Batching is one of the big drivers of efficiency, and to enable batching the Kafka producer will attempt to accumulate data in memory and to send out larger batches in a single request. The batching can be configured to accumulate no more than a fixed number of messages and to wait no longer than some fixed latency bound (say 64k or 10 ms). This allows the accumulation of more bytes to send, and few larger I/O operations on the servers. This buffering is configurable and gives a mechanism to trade off a small amount of additional latency for better throughput.
<p>
-Note that as of Kafka 0.8.1 the async producer does not have a callback, which could be used to register handlers to catch send errors. Adding such callback functionality is proposed for Kafka 0.9, see <a href="https://cwiki.apache.org/confluence/display/KAFKA/Client+Rewrite#ClientRewrite-ProposedProducerAPI">Proposed Producer API</a>.
+Details on <a href="#newproducerconfigs">configuration and <a href="http://kafka.apache.org/082/javadoc/index.html">api</a> for the producer can be found elsewhere in the documentation.
<h3><a id="theconsumer">4.5 The Consumer</a></h3>
@@ -271,7 +271,7 @@ Let's start by looking at a few use case
<ol>
<li><i>Database change subscription</i>. It is often necessary to have a data set in multiple data systems, and often one of these systems is a database of some kind (either a RDBMS or perhaps a new-fangled key-value store). For example you might have a database, a cache, a search cluster, and a Hadoop cluster. Each change to the database will need to be reflected in the cache, the search cluster, and eventually in Hadoop. In the case that one is only handling the real-time updates you only need recent log. But if you want to be able to reload the cache or restore a failed search node you may need a complete data set.
<li><i>Event sourcing</i>. This is a style of application design which co-locates query processing with application design and uses a log of changes as the primary store for the application.
-<li><i>Journaling for high-availability</i>. A process that does local computation can be made fault-tolerant by logging out changes that it makes to it's local state so another process can reload these changes and carry on if it should fail. A concrete example of this is handling counts, aggregations, and other "group by"-like processing in a stream query system. Samza, a real-time stream-processing framework, <a href="http://samza.incubator.apache.org/learn/documentation/0.7.0/container/state-management.html">uses this feature</a> for exactly this purpose.
+<li><i>Journaling for high-availability</i>. A process that does local computation can be made fault-tolerant by logging out changes that it makes to it's local state so another process can reload these changes and carry on if it should fail. A concrete example of this is handling counts, aggregations, and other "group by"-like processing in a stream query system. Samza, a real-time stream-processing framework, <a href="http://samza.apache.org/learn/documentation/0.7.0/container/state-management.html">uses this feature</a> for exactly this purpose.
</ol>
In each of these cases one needs primarily to handle the real-time feed of changes, but occasionally, when a machine crashes or data needs to be re-loaded or re-processed, one needs to do a full load. Log compaction allows feeding both of these use cases off the same backing topic.
@@ -322,7 +322,7 @@ Log compaction is handled by the log cle
<p>
<h4>Configuring The Log Cleaner</h4>
-As of 0.8.1 the log cleaner is disabled by default. To enable it set the server config
+The log cleaner is disabled by default. To enable it set the server config
<pre> log.cleaner.enable=true</pre>
This will start the pool of cleaner threads. To enable log cleaning on a particular topic you can add the log-specific property
<pre> log.cleanup.policy=compact</pre>
Modified: kafka/site/082/ops.html
URL: http://svn.apache.org/viewvc/kafka/site/082/ops.html?rev=1659337&r1=1659336&r2=1659337&view=diff
==============================================================================
--- kafka/site/082/ops.html (original)
+++ kafka/site/082/ops.html Thu Feb 12 17:48:27 2015
@@ -42,7 +42,8 @@ And finally deleting a topic:
<pre>
> bin/kafka-topics.sh --zookeeper zk_host:port/chroot --delete --topic my_topic_name
</pre>
-WARNING: Delete topic functionality is beta in 0.8.1. Please report any bugs that you encounter on the <a href="mailto: users@kafka.apache.org">mailing list</a> or <a href="https://issues.apache.org/jira/browse/KAFKA">JIRA</a>.
+Topic deletion option is disabled by default. To enable it set the server config
+ <pre>delete.topic.enable=true</pre>
<p>
Kafka does not currently support reducing the number of partitions for a topic or changing the replication factor.
Modified: kafka/site/082/quickstart.html
URL: http://svn.apache.org/viewvc/kafka/site/082/quickstart.html?rev=1659337&r1=1659336&r2=1659337&view=diff
==============================================================================
--- kafka/site/082/quickstart.html (original)
+++ kafka/site/082/quickstart.html Thu Feb 12 17:48:27 2015
@@ -4,11 +4,11 @@ This tutorial assumes you are starting f
<h4> Step 1: Download the code </h4>
-<a href="https://www.apache.org/dyn/closer.cgi?path=/kafka/0.8.1.1/kafka_2.9.2-0.8.1.1.tgz" title="Kafka downloads">Download</a> the 0.8.1.1 release and un-tar it.
+<a href="https://www.apache.org/dyn/closer.cgi?path=/kafka/0.8.2.0/kafka_2.10-0.8.2.0.tgz" title="Kafka downloads">Download</a> the 0.8.2.0 release and un-tar it.
<pre>
-> <b>tar -xzf kafka_2.9.2-0.8.1.1.tgz</b>
-> <b>cd kafka_2.9.2-0.8.1.1</b>
+> <b>tar -xzf kafka_2.10-0.8.2.0.tgz</b>
+> <b>cd kafka_2.10-0.8.2.0</b>
</pre>
<h4>Step 2: Start the server</h4>
Modified: kafka/site/082/upgrade.html
URL: http://svn.apache.org/viewvc/kafka/site/082/upgrade.html?rev=1659337&r1=1659336&r2=1659337&view=diff
==============================================================================
--- kafka/site/082/upgrade.html (original)
+++ kafka/site/082/upgrade.html Thu Feb 12 17:48:27 2015
@@ -1,4 +1,9 @@
<h3><a id="upgrade">1.5 Upgrading From Previous Versions</a></h3>
+
+<h4>Upgrading from 0.8.1 to 0.8.2.0</h4>
+
+0.8.2.0 is fully compatible with 0.8.1. The upgrade can be done one broker at a time by simply bringing it down, updating the code, and restarting it.
+
<h4>Upgrading from 0.8.0 to 0.8.1</h4>
0.8.1 is fully compatible with 0.8. The upgrade can be done one broker at a time by simply bringing it down, updating the code, and restarting it.
Modified: kafka/site/082/uses.html
URL: http://svn.apache.org/viewvc/kafka/site/082/uses.html?rev=1659337&r1=1659336&r2=1659337&view=diff
==============================================================================
--- kafka/site/082/uses.html (original)
+++ kafka/site/082/uses.html Thu Feb 12 17:48:27 2015
@@ -28,7 +28,7 @@ In comparison to log-centric systems lik
<h4>Stream Processing</h4>
-Many users end up doing stage-wise processing of data where data is consumed from topics of raw data and then aggregated, enriched, or otherwise transformed into new Kafka topics for further consumption. For example a processing flow for article recommendation might crawl article content from RSS feeds and publish it to an "articles" topic; further processing might help normalize or deduplicate this content to a topic of cleaned article content; a final stage might attempt to match this content to users. This creates a graph of real-time data flow out of the individual topics. <a href="https://github.com/nathanmarz/storm">Storm</a> and <a href="http://samza.incubator.apache.org/">Samza</a> are popular frameworks for implementing these kinds of transformations.
+Many users end up doing stage-wise processing of data where data is consumed from topics of raw data and then aggregated, enriched, or otherwise transformed into new Kafka topics for further consumption. For example a processing flow for article recommendation might crawl article content from RSS feeds and publish it to an "articles" topic; further processing might help normalize or deduplicate this content to a topic of cleaned article content; a final stage might attempt to match this content to users. This creates a graph of real-time data flow out of the individual topics. <a href="https://storm.apache.org/">Storm</a> and <a href="http://samza.apache.org/">Samza</a> are popular frameworks for implementing these kinds of transformations.
<h4>Event Sourcing</h4>
@@ -36,4 +36,4 @@ Many users end up doing stage-wise proce
<h4>Commit Log</h4>
-Kafka can serve as a kind of external commit-log for a distributed system. The log helps replicate data between nodes and acts as a re-syncing mechanism for failed nodes to restore their data. The <a href="/documentation.html#compaction">log compaction</a> feature in Kafka helps support this usage. In this usage Kafka is similar to <a href="http://zookeeper.apache.org/bookkeeper/">Apache BookKeeper</a> project.
\ No newline at end of file
+Kafka can serve as a kind of external commit-log for a distributed system. The log helps replicate data between nodes and acts as a re-syncing mechanism for failed nodes to restore their data. The <a href="/documentation.html#compaction">log compaction</a> feature in Kafka helps support this usage. In this usage Kafka is similar to <a href="http://zookeeper.apache.org/bookkeeper/">Apache BookKeeper</a> project.