You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@kafka.apache.org by hachikuji <gi...@git.apache.org> on 2017/06/08 21:30:55 UTC
[GitHub] kafka-site pull request #60: Update delivery semantics section for KIP-98
GitHub user hachikuji opened a pull request:
https://github.com/apache/kafka-site/pull/60
Update delivery semantics section for KIP-98
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/hachikuji/kafka-site update-delivery-semantics
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/kafka-site/pull/60.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #60
----
commit 0759f3a22e377e12e857eb6da4977adb62261d30
Author: Jason Gustafson <ja...@confluent.io>
Date: 2017-06-08T21:28:13Z
Update delivery semantics section for KIP-98
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] kafka-site pull request #60: Update delivery semantics section for KIP-98
Posted by ijuma <gi...@git.apache.org>.
Github user ijuma commented on a diff in the pull request:
https://github.com/apache/kafka-site/pull/60#discussion_r122096525
--- Diff: 0110/design.html ---
@@ -264,21 +264,22 @@
messages have a primary key and so the updates are idempotent (receiving the same message twice just overwrites a record with another copy of itself).
</ol>
<p>
- So what about exactly once semantics (i.e. the thing you actually want)? The limitation here is not actually a feature of the messaging system but rather the need to coordinate the consumer's position with
- what is actually stored as output. The classic way of achieving this would be to introduce a two-phase commit between the storage for the consumer position and the storage of the consumers output. But this can be
- handled more simply and generally by simply letting the consumer store its offset in the same place as its output. This is better because many of the output systems a consumer might want to write to will not
- support a two-phase commit. As an example of this, our Hadoop ETL that populates data in HDFS stores its offsets in HDFS with the data it reads so that it is guaranteed that either data and offsets are both updated
- or neither is. We follow similar patterns for many other data systems which require these stronger semantics and for which the messages do not have a primary key to allow for deduplication.
- <p>
- A special case is when the output system is just another Kafka topic (e.g. in a Kafka Streams application). Here we can leverage the new transactional producer capabilities in 0.11.0.0 that were mentioned above.
- Since the consumer's position is stored as a message in a topic, we can ensure that that topic is included in the same transaction as the output topics receiving the processed data. If the transaction is aborted,
- the consumer's position will revert to its old value and none of the output data will be visible to consumers. To enable this, consumers support an "isolation level" to achieve this. In the default
- "read_uncommitted" mode, all messages are visible to consumers even if they were part of an aborted transaction, but in "read_committed" mode, the consumer will only return data from transactions which were committed
- (and any messages which were not part of any transaction).
- <p>
- So effectively Kafka guarantees at-least-once delivery by default, and allows the user to implement at-most-once delivery by disabling retries on the producer and committing its offset prior to processing a batch of
- messages. Exactly-once delivery is supported when processing messages between Kafka topics, such as in Kafka Streams applications. Exactly-once delivery for other destination storage system generally requires
- cooperation with that system, but Kafka provides the offset which makes implementing this straight-forward.
+ So what about exactly once semantics (i.e. the thing you actually want)? When consuming from a Kafka topic and producing to another topic (as in a <a href="https://kafka.apache.org/documentation/streams">Kafka Streams</a>
+ application), we can leverage the new transactional producer capabilities in 0.11.0.0 that were mentioned above. The consumer's position is stored as a message in a topic, so we can write the offset to Kafka in the
+ same transaction as the output topics receiving the processed data. If the transaction is aborted, the consumer's position will revert to its old value and none of the output data will be visible to consumers. Consumers support an "isolation level" configuration
+ to achieve this. In the default "read_uncommitted" mode, all messages are visible to consumers even if they were part of an aborted transaction, but in "read_committed" mode, the consumer will only return data from
+ transactions which were committed (and any messages which were not part of any transaction).
+ <p>
+ When writing to an external system, the limitation is in the need to coordinate the consumer's position with what is actually stored as output. The classic way of achieving this would be to introduce a two-phase
+ commit between the storage for the consumer position and the storage of the consumers output. But this can be handled more simply and generally by simply letting the consumer store its offset in the same place as
+ its output. This is better because many of the output systems a consumer might want to write to will not support a two-phase commit. As an example of this, our Hadoop ETL that populates data in HDFS stores its
+ offsets in HDFS with the data it reads so that it is guaranteed that either data and offsets are both updated or neither is. We follow similar patterns for many other data systems which require these stronger
+ semantics and for which the messages do not have a primary key to allow for deduplication.
+ <p>
+ So effectively Kafka supports exactly-once delivery in <a href="https://kafka.apache.org/documentation/streams">Kafka Streams</a>, and the transactional producer/consumer can be used generally to provide
+ exactly-once delivery when transfering and processing data between Kafka topics. Exactly-once delivery for other destination storage system generally requires cooperation with that system, but Kafka provides the
--- End diff --
"for other destination systems generally requires cooperation such such systems" maybe
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] kafka-site pull request #60: Update delivery semantics section for KIP-98
Posted by hachikuji <gi...@git.apache.org>.
Github user hachikuji commented on a diff in the pull request:
https://github.com/apache/kafka-site/pull/60#discussion_r121249550
--- Diff: 0110/design.html ---
@@ -261,15 +262,23 @@
<li>It can read the messages, process the messages, and finally save its position. In this case there is a possibility that the consumer process crashes after processing messages but before saving its position.
In this case when the new process takes over the first few messages it receives will already have been processed. This corresponds to the "at-least-once" semantics in the case of consumer failure. In many cases
messages have a primary key and so the updates are idempotent (receiving the same message twice just overwrites a record with another copy of itself).
- <li>So what about exactly once semantics (i.e. the thing you actually want)? The limitation here is not actually a feature of the messaging system but rather the need to co-ordinate the consumer's position with
+ </ol>
+ <p>
+ So what about exactly once semantics (i.e. the thing you actually want)? The limitation here is not actually a feature of the messaging system but rather the need to coordinate the consumer's position with
what is actually stored as output. The classic way of achieving this would be to introduce a two-phase commit between the storage for the consumer position and the storage of the consumers output. But this can be
handled more simply and generally by simply letting the consumer store its offset in the same place as its output. This is better because many of the output systems a consumer might want to write to will not
support a two-phase commit. As an example of this, our Hadoop ETL that populates data in HDFS stores its offsets in HDFS with the data it reads so that it is guaranteed that either data and offsets are both updated
or neither is. We follow similar patterns for many other data systems which require these stronger semantics and for which the messages do not have a primary key to allow for deduplication.
- </ol>
<p>
- So effectively Kafka guarantees at-least-once delivery by default and allows the user to implement at most once delivery by disabling retries on the producer and committing its offset prior to processing a batch of
- messages. Exactly-once delivery requires co-operation with the destination storage system but Kafka provides the offset which makes implementing this straight-forward.
+ A special case is when the output system is just another Kafka topic (e.g. in a Kafka Streams application). Here we can leverage the new transactional producer capabilities in 0.11.0.0 that were mentioned above.
+ Since the consumer's position is stored as a message in a topic, we can ensure that that topic is included in the same transaction as the output topics receiving the processed data. If the transaction is aborted,
+ the consumer's position will revert to its old value and none of the output data will be visible to consumers. To enable this, consumers support an "isolation level" to achieve this. In the default
--- End diff --
Thanks. I will try to clarify these lines.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] kafka-site pull request #60: Update delivery semantics section for KIP-98
Posted by ijuma <gi...@git.apache.org>.
Github user ijuma commented on a diff in the pull request:
https://github.com/apache/kafka-site/pull/60#discussion_r121246230
--- Diff: 0110/design.html ---
@@ -261,15 +262,23 @@
<li>It can read the messages, process the messages, and finally save its position. In this case there is a possibility that the consumer process crashes after processing messages but before saving its position.
In this case when the new process takes over the first few messages it receives will already have been processed. This corresponds to the "at-least-once" semantics in the case of consumer failure. In many cases
messages have a primary key and so the updates are idempotent (receiving the same message twice just overwrites a record with another copy of itself).
- <li>So what about exactly once semantics (i.e. the thing you actually want)? The limitation here is not actually a feature of the messaging system but rather the need to co-ordinate the consumer's position with
+ </ol>
+ <p>
+ So what about exactly once semantics (i.e. the thing you actually want)? The limitation here is not actually a feature of the messaging system but rather the need to coordinate the consumer's position with
what is actually stored as output. The classic way of achieving this would be to introduce a two-phase commit between the storage for the consumer position and the storage of the consumers output. But this can be
handled more simply and generally by simply letting the consumer store its offset in the same place as its output. This is better because many of the output systems a consumer might want to write to will not
support a two-phase commit. As an example of this, our Hadoop ETL that populates data in HDFS stores its offsets in HDFS with the data it reads so that it is guaranteed that either data and offsets are both updated
or neither is. We follow similar patterns for many other data systems which require these stronger semantics and for which the messages do not have a primary key to allow for deduplication.
- </ol>
<p>
- So effectively Kafka guarantees at-least-once delivery by default and allows the user to implement at most once delivery by disabling retries on the producer and committing its offset prior to processing a batch of
- messages. Exactly-once delivery requires co-operation with the destination storage system but Kafka provides the offset which makes implementing this straight-forward.
+ A special case is when the output system is just another Kafka topic (e.g. in a Kafka Streams application). Here we can leverage the new transactional producer capabilities in 0.11.0.0 that were mentioned above.
+ Since the consumer's position is stored as a message in a topic, we can ensure that that topic is included in the same transaction as the output topics receiving the processed data. If the transaction is aborted,
+ the consumer's position will revert to its old value and none of the output data will be visible to consumers. To enable this, consumers support an "isolation level" to achieve this. In the default
+ "read_uncommitted" mode, all messages are visible to consumers even if they were part of an aborted transaction, but in "read_committed" mode, the consumer will only return data from transactions which were committed
+ (and any messages which were not part of any transaction).
+ <p>
+ So effectively Kafka guarantees at-least-once delivery by default, and allows the user to implement at-most-once delivery by disabling retries on the producer and committing its offset prior to processing a batch of
--- End diff --
I wonder if we should mention the stronger guarantees first.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] kafka-site pull request #60: Update delivery semantics section for KIP-98
Posted by ijuma <gi...@git.apache.org>.
Github user ijuma commented on a diff in the pull request:
https://github.com/apache/kafka-site/pull/60#discussion_r122096399
--- Diff: 0110/design.html ---
@@ -264,21 +264,22 @@
messages have a primary key and so the updates are idempotent (receiving the same message twice just overwrites a record with another copy of itself).
</ol>
<p>
- So what about exactly once semantics (i.e. the thing you actually want)? The limitation here is not actually a feature of the messaging system but rather the need to coordinate the consumer's position with
- what is actually stored as output. The classic way of achieving this would be to introduce a two-phase commit between the storage for the consumer position and the storage of the consumers output. But this can be
- handled more simply and generally by simply letting the consumer store its offset in the same place as its output. This is better because many of the output systems a consumer might want to write to will not
- support a two-phase commit. As an example of this, our Hadoop ETL that populates data in HDFS stores its offsets in HDFS with the data it reads so that it is guaranteed that either data and offsets are both updated
- or neither is. We follow similar patterns for many other data systems which require these stronger semantics and for which the messages do not have a primary key to allow for deduplication.
- <p>
- A special case is when the output system is just another Kafka topic (e.g. in a Kafka Streams application). Here we can leverage the new transactional producer capabilities in 0.11.0.0 that were mentioned above.
- Since the consumer's position is stored as a message in a topic, we can ensure that that topic is included in the same transaction as the output topics receiving the processed data. If the transaction is aborted,
- the consumer's position will revert to its old value and none of the output data will be visible to consumers. To enable this, consumers support an "isolation level" to achieve this. In the default
- "read_uncommitted" mode, all messages are visible to consumers even if they were part of an aborted transaction, but in "read_committed" mode, the consumer will only return data from transactions which were committed
- (and any messages which were not part of any transaction).
- <p>
- So effectively Kafka guarantees at-least-once delivery by default, and allows the user to implement at-most-once delivery by disabling retries on the producer and committing its offset prior to processing a batch of
- messages. Exactly-once delivery is supported when processing messages between Kafka topics, such as in Kafka Streams applications. Exactly-once delivery for other destination storage system generally requires
- cooperation with that system, but Kafka provides the offset which makes implementing this straight-forward.
+ So what about exactly once semantics (i.e. the thing you actually want)? When consuming from a Kafka topic and producing to another topic (as in a <a href="https://kafka.apache.org/documentation/streams">Kafka Streams</a>
+ application), we can leverage the new transactional producer capabilities in 0.11.0.0 that were mentioned above. The consumer's position is stored as a message in a topic, so we can write the offset to Kafka in the
+ same transaction as the output topics receiving the processed data. If the transaction is aborted, the consumer's position will revert to its old value and none of the output data will be visible to consumers. Consumers support an "isolation level" configuration
+ to achieve this. In the default "read_uncommitted" mode, all messages are visible to consumers even if they were part of an aborted transaction, but in "read_committed" mode, the consumer will only return data from
+ transactions which were committed (and any messages which were not part of any transaction).
+ <p>
+ When writing to an external system, the limitation is in the need to coordinate the consumer's position with what is actually stored as output. The classic way of achieving this would be to introduce a two-phase
+ commit between the storage for the consumer position and the storage of the consumers output. But this can be handled more simply and generally by simply letting the consumer store its offset in the same place as
+ its output. This is better because many of the output systems a consumer might want to write to will not support a two-phase commit. As an example of this, our Hadoop ETL that populates data in HDFS stores its
--- End diff --
`our ETL` doesn't seem right because it's a Confluent Connector right?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] kafka-site pull request #60: Update delivery semantics section for KIP-98
Posted by hachikuji <gi...@git.apache.org>.
Github user hachikuji commented on a diff in the pull request:
https://github.com/apache/kafka-site/pull/60#discussion_r122826323
--- Diff: 0110/design.html ---
@@ -264,21 +264,22 @@
messages have a primary key and so the updates are idempotent (receiving the same message twice just overwrites a record with another copy of itself).
</ol>
<p>
- So what about exactly once semantics (i.e. the thing you actually want)? The limitation here is not actually a feature of the messaging system but rather the need to coordinate the consumer's position with
- what is actually stored as output. The classic way of achieving this would be to introduce a two-phase commit between the storage for the consumer position and the storage of the consumers output. But this can be
- handled more simply and generally by simply letting the consumer store its offset in the same place as its output. This is better because many of the output systems a consumer might want to write to will not
- support a two-phase commit. As an example of this, our Hadoop ETL that populates data in HDFS stores its offsets in HDFS with the data it reads so that it is guaranteed that either data and offsets are both updated
- or neither is. We follow similar patterns for many other data systems which require these stronger semantics and for which the messages do not have a primary key to allow for deduplication.
- <p>
- A special case is when the output system is just another Kafka topic (e.g. in a Kafka Streams application). Here we can leverage the new transactional producer capabilities in 0.11.0.0 that were mentioned above.
- Since the consumer's position is stored as a message in a topic, we can ensure that that topic is included in the same transaction as the output topics receiving the processed data. If the transaction is aborted,
- the consumer's position will revert to its old value and none of the output data will be visible to consumers. To enable this, consumers support an "isolation level" to achieve this. In the default
- "read_uncommitted" mode, all messages are visible to consumers even if they were part of an aborted transaction, but in "read_committed" mode, the consumer will only return data from transactions which were committed
- (and any messages which were not part of any transaction).
- <p>
- So effectively Kafka guarantees at-least-once delivery by default, and allows the user to implement at-most-once delivery by disabling retries on the producer and committing its offset prior to processing a batch of
- messages. Exactly-once delivery is supported when processing messages between Kafka topics, such as in Kafka Streams applications. Exactly-once delivery for other destination storage system generally requires
- cooperation with that system, but Kafka provides the offset which makes implementing this straight-forward.
+ So what about exactly once semantics (i.e. the thing you actually want)? When consuming from a Kafka topic and producing to another topic (as in a <a href="https://kafka.apache.org/documentation/streams">Kafka Streams</a>
+ application), we can leverage the new transactional producer capabilities in 0.11.0.0 that were mentioned above. The consumer's position is stored as a message in a topic, so we can write the offset to Kafka in the
+ same transaction as the output topics receiving the processed data. If the transaction is aborted, the consumer's position will revert to its old value and none of the output data will be visible to consumers. Consumers support an "isolation level" configuration
+ to achieve this. In the default "read_uncommitted" mode, all messages are visible to consumers even if they were part of an aborted transaction, but in "read_committed" mode, the consumer will only return data from
+ transactions which were committed (and any messages which were not part of any transaction).
+ <p>
+ When writing to an external system, the limitation is in the need to coordinate the consumer's position with what is actually stored as output. The classic way of achieving this would be to introduce a two-phase
+ commit between the storage for the consumer position and the storage of the consumers output. But this can be handled more simply and generally by simply letting the consumer store its offset in the same place as
--- End diff --
I'm not sure I understand the second question. Can you elaborate?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] kafka-site pull request #60: Update delivery semantics section for KIP-98
Posted by hachikuji <gi...@git.apache.org>.
Github user hachikuji closed the pull request at:
https://github.com/apache/kafka-site/pull/60
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] kafka-site issue #60: Update delivery semantics section for KIP-98
Posted by hachikuji <gi...@git.apache.org>.
Github user hachikuji commented on the issue:
https://github.com/apache/kafka-site/pull/60
cc @guozhangwang @apurva @ijuma
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] kafka-site pull request #60: Update delivery semantics section for KIP-98
Posted by guozhangwang <gi...@git.apache.org>.
Github user guozhangwang commented on a diff in the pull request:
https://github.com/apache/kafka-site/pull/60#discussion_r121496464
--- Diff: 0110/design.html ---
@@ -261,15 +262,23 @@
<li>It can read the messages, process the messages, and finally save its position. In this case there is a possibility that the consumer process crashes after processing messages but before saving its position.
In this case when the new process takes over the first few messages it receives will already have been processed. This corresponds to the "at-least-once" semantics in the case of consumer failure. In many cases
messages have a primary key and so the updates are idempotent (receiving the same message twice just overwrites a record with another copy of itself).
- <li>So what about exactly once semantics (i.e. the thing you actually want)? The limitation here is not actually a feature of the messaging system but rather the need to co-ordinate the consumer's position with
+ </ol>
+ <p>
+ So what about exactly once semantics (i.e. the thing you actually want)? The limitation here is not actually a feature of the messaging system but rather the need to coordinate the consumer's position with
what is actually stored as output. The classic way of achieving this would be to introduce a two-phase commit between the storage for the consumer position and the storage of the consumers output. But this can be
handled more simply and generally by simply letting the consumer store its offset in the same place as its output. This is better because many of the output systems a consumer might want to write to will not
support a two-phase commit. As an example of this, our Hadoop ETL that populates data in HDFS stores its offsets in HDFS with the data it reads so that it is guaranteed that either data and offsets are both updated
or neither is. We follow similar patterns for many other data systems which require these stronger semantics and for which the messages do not have a primary key to allow for deduplication.
- </ol>
<p>
- So effectively Kafka guarantees at-least-once delivery by default and allows the user to implement at most once delivery by disabling retries on the producer and committing its offset prior to processing a batch of
- messages. Exactly-once delivery requires co-operation with the destination storage system but Kafka provides the offset which makes implementing this straight-forward.
+ A special case is when the output system is just another Kafka topic (e.g. in a Kafka Streams application). Here we can leverage the new transactional producer capabilities in 0.11.0.0 that were mentioned above.
+ Since the consumer's position is stored as a message in a topic, we can ensure that that topic is included in the same transaction as the output topics receiving the processed data. If the transaction is aborted,
+ the consumer's position will revert to its old value and none of the output data will be visible to consumers. To enable this, consumers support an "isolation level" to achieve this. In the default
+ "read_uncommitted" mode, all messages are visible to consumers even if they were part of an aborted transaction, but in "read_committed" mode, the consumer will only return data from transactions which were committed
+ (and any messages which were not part of any transaction).
+ <p>
+ So effectively Kafka guarantees at-least-once delivery by default, and allows the user to implement at-most-once delivery by disabling retries on the producer and committing its offset prior to processing a batch of
+ messages. Exactly-once delivery is supported when processing messages between Kafka topics, such as in Kafka Streams applications. Exactly-once delivery for other destination storage system generally requires
--- End diff --
nit: add a ref link when mentioning Kafka Streams? https://kafka.apache.org/documentation/streams
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] kafka-site pull request #60: Update delivery semantics section for KIP-98
Posted by hachikuji <gi...@git.apache.org>.
Github user hachikuji commented on a diff in the pull request:
https://github.com/apache/kafka-site/pull/60#discussion_r121249508
--- Diff: 0110/design.html ---
@@ -261,15 +262,23 @@
<li>It can read the messages, process the messages, and finally save its position. In this case there is a possibility that the consumer process crashes after processing messages but before saving its position.
In this case when the new process takes over the first few messages it receives will already have been processed. This corresponds to the "at-least-once" semantics in the case of consumer failure. In many cases
messages have a primary key and so the updates are idempotent (receiving the same message twice just overwrites a record with another copy of itself).
- <li>So what about exactly once semantics (i.e. the thing you actually want)? The limitation here is not actually a feature of the messaging system but rather the need to co-ordinate the consumer's position with
+ </ol>
+ <p>
+ So what about exactly once semantics (i.e. the thing you actually want)? The limitation here is not actually a feature of the messaging system but rather the need to coordinate the consumer's position with
what is actually stored as output. The classic way of achieving this would be to introduce a two-phase commit between the storage for the consumer position and the storage of the consumers output. But this can be
handled more simply and generally by simply letting the consumer store its offset in the same place as its output. This is better because many of the output systems a consumer might want to write to will not
support a two-phase commit. As an example of this, our Hadoop ETL that populates data in HDFS stores its offsets in HDFS with the data it reads so that it is guaranteed that either data and offsets are both updated
or neither is. We follow similar patterns for many other data systems which require these stronger semantics and for which the messages do not have a primary key to allow for deduplication.
- </ol>
<p>
- So effectively Kafka guarantees at-least-once delivery by default and allows the user to implement at most once delivery by disabling retries on the producer and committing its offset prior to processing a batch of
- messages. Exactly-once delivery requires co-operation with the destination storage system but Kafka provides the offset which makes implementing this straight-forward.
+ A special case is when the output system is just another Kafka topic (e.g. in a Kafka Streams application). Here we can leverage the new transactional producer capabilities in 0.11.0.0 that were mentioned above.
+ Since the consumer's position is stored as a message in a topic, we can ensure that that topic is included in the same transaction as the output topics receiving the processed data. If the transaction is aborted,
+ the consumer's position will revert to its old value and none of the output data will be visible to consumers. To enable this, consumers support an "isolation level" to achieve this. In the default
+ "read_uncommitted" mode, all messages are visible to consumers even if they were part of an aborted transaction, but in "read_committed" mode, the consumer will only return data from transactions which were committed
+ (and any messages which were not part of any transaction).
+ <p>
+ So effectively Kafka guarantees at-least-once delivery by default, and allows the user to implement at-most-once delivery by disabling retries on the producer and committing its offset prior to processing a batch of
--- End diff --
Good point.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] kafka-site pull request #60: Update delivery semantics section for KIP-98
Posted by ijuma <gi...@git.apache.org>.
Github user ijuma commented on a diff in the pull request:
https://github.com/apache/kafka-site/pull/60#discussion_r122096607
--- Diff: 0110/design.html ---
@@ -264,21 +264,22 @@
messages have a primary key and so the updates are idempotent (receiving the same message twice just overwrites a record with another copy of itself).
</ol>
<p>
- So what about exactly once semantics (i.e. the thing you actually want)? The limitation here is not actually a feature of the messaging system but rather the need to coordinate the consumer's position with
- what is actually stored as output. The classic way of achieving this would be to introduce a two-phase commit between the storage for the consumer position and the storage of the consumers output. But this can be
- handled more simply and generally by simply letting the consumer store its offset in the same place as its output. This is better because many of the output systems a consumer might want to write to will not
- support a two-phase commit. As an example of this, our Hadoop ETL that populates data in HDFS stores its offsets in HDFS with the data it reads so that it is guaranteed that either data and offsets are both updated
- or neither is. We follow similar patterns for many other data systems which require these stronger semantics and for which the messages do not have a primary key to allow for deduplication.
- <p>
- A special case is when the output system is just another Kafka topic (e.g. in a Kafka Streams application). Here we can leverage the new transactional producer capabilities in 0.11.0.0 that were mentioned above.
- Since the consumer's position is stored as a message in a topic, we can ensure that that topic is included in the same transaction as the output topics receiving the processed data. If the transaction is aborted,
- the consumer's position will revert to its old value and none of the output data will be visible to consumers. To enable this, consumers support an "isolation level" to achieve this. In the default
- "read_uncommitted" mode, all messages are visible to consumers even if they were part of an aborted transaction, but in "read_committed" mode, the consumer will only return data from transactions which were committed
- (and any messages which were not part of any transaction).
- <p>
- So effectively Kafka guarantees at-least-once delivery by default, and allows the user to implement at-most-once delivery by disabling retries on the producer and committing its offset prior to processing a batch of
- messages. Exactly-once delivery is supported when processing messages between Kafka topics, such as in Kafka Streams applications. Exactly-once delivery for other destination storage system generally requires
- cooperation with that system, but Kafka provides the offset which makes implementing this straight-forward.
+ So what about exactly once semantics (i.e. the thing you actually want)? When consuming from a Kafka topic and producing to another topic (as in a <a href="https://kafka.apache.org/documentation/streams">Kafka Streams</a>
+ application), we can leverage the new transactional producer capabilities in 0.11.0.0 that were mentioned above. The consumer's position is stored as a message in a topic, so we can write the offset to Kafka in the
+ same transaction as the output topics receiving the processed data. If the transaction is aborted, the consumer's position will revert to its old value and none of the output data will be visible to consumers. Consumers support an "isolation level" configuration
+ to achieve this. In the default "read_uncommitted" mode, all messages are visible to consumers even if they were part of an aborted transaction, but in "read_committed" mode, the consumer will only return data from
+ transactions which were committed (and any messages which were not part of any transaction).
+ <p>
+ When writing to an external system, the limitation is in the need to coordinate the consumer's position with what is actually stored as output. The classic way of achieving this would be to introduce a two-phase
+ commit between the storage for the consumer position and the storage of the consumers output. But this can be handled more simply and generally by simply letting the consumer store its offset in the same place as
+ its output. This is better because many of the output systems a consumer might want to write to will not support a two-phase commit. As an example of this, our Hadoop ETL that populates data in HDFS stores its
+ offsets in HDFS with the data it reads so that it is guaranteed that either data and offsets are both updated or neither is. We follow similar patterns for many other data systems which require these stronger
+ semantics and for which the messages do not have a primary key to allow for deduplication.
+ <p>
+ So effectively Kafka supports exactly-once delivery in <a href="https://kafka.apache.org/documentation/streams">Kafka Streams</a>, and the transactional producer/consumer can be used generally to provide
+ exactly-once delivery when transfering and processing data between Kafka topics. Exactly-once delivery for other destination storage system generally requires cooperation with that system, but Kafka provides the
+ offset which makes implementing this straight-forward. Otherwise, Kafka guarantees at-least-once delivery by default, and allows the user to implement at-most-once delivery by disabling retries on the producer and
--- End diff --
A bit optimistic: "straight-forward" :)
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] kafka-site pull request #60: Update delivery semantics section for KIP-98
Posted by hachikuji <gi...@git.apache.org>.
Github user hachikuji commented on a diff in the pull request:
https://github.com/apache/kafka-site/pull/60#discussion_r122825810
--- Diff: 0110/design.html ---
@@ -264,21 +264,22 @@
messages have a primary key and so the updates are idempotent (receiving the same message twice just overwrites a record with another copy of itself).
</ol>
<p>
- So what about exactly once semantics (i.e. the thing you actually want)? The limitation here is not actually a feature of the messaging system but rather the need to coordinate the consumer's position with
- what is actually stored as output. The classic way of achieving this would be to introduce a two-phase commit between the storage for the consumer position and the storage of the consumers output. But this can be
- handled more simply and generally by simply letting the consumer store its offset in the same place as its output. This is better because many of the output systems a consumer might want to write to will not
- support a two-phase commit. As an example of this, our Hadoop ETL that populates data in HDFS stores its offsets in HDFS with the data it reads so that it is guaranteed that either data and offsets are both updated
- or neither is. We follow similar patterns for many other data systems which require these stronger semantics and for which the messages do not have a primary key to allow for deduplication.
- <p>
- A special case is when the output system is just another Kafka topic (e.g. in a Kafka Streams application). Here we can leverage the new transactional producer capabilities in 0.11.0.0 that were mentioned above.
- Since the consumer's position is stored as a message in a topic, we can ensure that that topic is included in the same transaction as the output topics receiving the processed data. If the transaction is aborted,
- the consumer's position will revert to its old value and none of the output data will be visible to consumers. To enable this, consumers support an "isolation level" to achieve this. In the default
- "read_uncommitted" mode, all messages are visible to consumers even if they were part of an aborted transaction, but in "read_committed" mode, the consumer will only return data from transactions which were committed
- (and any messages which were not part of any transaction).
- <p>
- So effectively Kafka guarantees at-least-once delivery by default, and allows the user to implement at-most-once delivery by disabling retries on the producer and committing its offset prior to processing a batch of
- messages. Exactly-once delivery is supported when processing messages between Kafka topics, such as in Kafka Streams applications. Exactly-once delivery for other destination storage system generally requires
- cooperation with that system, but Kafka provides the offset which makes implementing this straight-forward.
+ So what about exactly once semantics (i.e. the thing you actually want)? When consuming from a Kafka topic and producing to another topic (as in a <a href="https://kafka.apache.org/documentation/streams">Kafka Streams</a>
+ application), we can leverage the new transactional producer capabilities in 0.11.0.0 that were mentioned above. The consumer's position is stored as a message in a topic, so we can write the offset to Kafka in the
+ same transaction as the output topics receiving the processed data. If the transaction is aborted, the consumer's position will revert to its old value and none of the output data will be visible to consumers. Consumers support an "isolation level" configuration
+ to achieve this. In the default "read_uncommitted" mode, all messages are visible to consumers even if they were part of an aborted transaction, but in "read_committed" mode, the consumer will only return data from
+ transactions which were committed (and any messages which were not part of any transaction).
+ <p>
+ When writing to an external system, the limitation is in the need to coordinate the consumer's position with what is actually stored as output. The classic way of achieving this would be to introduce a two-phase
+ commit between the storage for the consumer position and the storage of the consumers output. But this can be handled more simply and generally by simply letting the consumer store its offset in the same place as
+ its output. This is better because many of the output systems a consumer might want to write to will not support a two-phase commit. As an example of this, our Hadoop ETL that populates data in HDFS stores its
+ offsets in HDFS with the data it reads so that it is guaranteed that either data and offsets are both updated or neither is. We follow similar patterns for many other data systems which require these stronger
+ semantics and for which the messages do not have a primary key to allow for deduplication.
+ <p>
+ So effectively Kafka supports exactly-once delivery in <a href="https://kafka.apache.org/documentation/streams">Kafka Streams</a>, and the transactional producer/consumer can be used generally to provide
+ exactly-once delivery when transfering and processing data between Kafka topics. Exactly-once delivery for other destination storage system generally requires cooperation with that system, but Kafka provides the
+ offset which makes implementing this straight-forward. Otherwise, Kafka guarantees at-least-once delivery by default, and allows the user to implement at-most-once delivery by disabling retries on the producer and
--- End diff --
Haha, this was the original language, but I raised an eyebrow when I read that as well. Maybe "feasible" is nearer the mark.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] kafka-site pull request #60: Update delivery semantics section for KIP-98
Posted by guozhangwang <gi...@git.apache.org>.
Github user guozhangwang commented on a diff in the pull request:
https://github.com/apache/kafka-site/pull/60#discussion_r121496370
--- Diff: 0110/design.html ---
@@ -261,15 +262,23 @@
<li>It can read the messages, process the messages, and finally save its position. In this case there is a possibility that the consumer process crashes after processing messages but before saving its position.
In this case when the new process takes over the first few messages it receives will already have been processed. This corresponds to the "at-least-once" semantics in the case of consumer failure. In many cases
messages have a primary key and so the updates are idempotent (receiving the same message twice just overwrites a record with another copy of itself).
- <li>So what about exactly once semantics (i.e. the thing you actually want)? The limitation here is not actually a feature of the messaging system but rather the need to co-ordinate the consumer's position with
+ </ol>
+ <p>
+ So what about exactly once semantics (i.e. the thing you actually want)? The limitation here is not actually a feature of the messaging system but rather the need to coordinate the consumer's position with
what is actually stored as output. The classic way of achieving this would be to introduce a two-phase commit between the storage for the consumer position and the storage of the consumers output. But this can be
handled more simply and generally by simply letting the consumer store its offset in the same place as its output. This is better because many of the output systems a consumer might want to write to will not
support a two-phase commit. As an example of this, our Hadoop ETL that populates data in HDFS stores its offsets in HDFS with the data it reads so that it is guaranteed that either data and offsets are both updated
or neither is. We follow similar patterns for many other data systems which require these stronger semantics and for which the messages do not have a primary key to allow for deduplication.
- </ol>
<p>
- So effectively Kafka guarantees at-least-once delivery by default and allows the user to implement at most once delivery by disabling retries on the producer and committing its offset prior to processing a batch of
- messages. Exactly-once delivery requires co-operation with the destination storage system but Kafka provides the offset which makes implementing this straight-forward.
+ A special case is when the output system is just another Kafka topic (e.g. in a Kafka Streams application). Here we can leverage the new transactional producer capabilities in 0.11.0.0 that were mentioned above.
+ Since the consumer's position is stored as a message in a topic, we can ensure that that topic is included in the same transaction as the output topics receiving the processed data. If the transaction is aborted,
+ the consumer's position will revert to its old value and none of the output data will be visible to consumers. To enable this, consumers support an "isolation level" to achieve this. In the default
+ "read_uncommitted" mode, all messages are visible to consumers even if they were part of an aborted transaction, but in "read_committed" mode, the consumer will only return data from transactions which were committed
+ (and any messages which were not part of any transaction).
+ <p>
+ So effectively Kafka guarantees at-least-once delivery by default, and allows the user to implement at-most-once delivery by disabling retries on the producer and committing its offset prior to processing a batch of
+ messages. Exactly-once delivery is supported when processing messages between Kafka topics, such as in Kafka Streams applications. Exactly-once delivery for other destination storage system generally requires
--- End diff --
By just reading the first sentence it looks like "exactly-once is constraint to only the case when processing messages between Kafka topics" but only the second line points out that e.g. even with an external system like HDFS, Kafka Connect can also achieve EOS by coordinating with the system.
Maybe we can also re-phrase it as "Exactly-once delivery is supported out-of-the-box when ...; for other destination storage system it is generally required to cooperate ... to achieve exactly-once".
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] kafka-site issue #60: Update delivery semantics section for KIP-98
Posted by hachikuji <gi...@git.apache.org>.
Github user hachikuji commented on the issue:
https://github.com/apache/kafka-site/pull/60
@guozhangwang @ijuma @apurvam Pushed a couple changes based on the review. Please take another look.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] kafka-site pull request #60: Update delivery semantics section for KIP-98
Posted by ijuma <gi...@git.apache.org>.
Github user ijuma commented on a diff in the pull request:
https://github.com/apache/kafka-site/pull/60#discussion_r121246131
--- Diff: 0110/design.html ---
@@ -261,15 +262,23 @@
<li>It can read the messages, process the messages, and finally save its position. In this case there is a possibility that the consumer process crashes after processing messages but before saving its position.
In this case when the new process takes over the first few messages it receives will already have been processed. This corresponds to the "at-least-once" semantics in the case of consumer failure. In many cases
messages have a primary key and so the updates are idempotent (receiving the same message twice just overwrites a record with another copy of itself).
- <li>So what about exactly once semantics (i.e. the thing you actually want)? The limitation here is not actually a feature of the messaging system but rather the need to co-ordinate the consumer's position with
+ </ol>
+ <p>
+ So what about exactly once semantics (i.e. the thing you actually want)? The limitation here is not actually a feature of the messaging system but rather the need to coordinate the consumer's position with
what is actually stored as output. The classic way of achieving this would be to introduce a two-phase commit between the storage for the consumer position and the storage of the consumers output. But this can be
handled more simply and generally by simply letting the consumer store its offset in the same place as its output. This is better because many of the output systems a consumer might want to write to will not
support a two-phase commit. As an example of this, our Hadoop ETL that populates data in HDFS stores its offsets in HDFS with the data it reads so that it is guaranteed that either data and offsets are both updated
or neither is. We follow similar patterns for many other data systems which require these stronger semantics and for which the messages do not have a primary key to allow for deduplication.
- </ol>
<p>
- So effectively Kafka guarantees at-least-once delivery by default and allows the user to implement at most once delivery by disabling retries on the producer and committing its offset prior to processing a batch of
- messages. Exactly-once delivery requires co-operation with the destination storage system but Kafka provides the offset which makes implementing this straight-forward.
+ A special case is when the output system is just another Kafka topic (e.g. in a Kafka Streams application). Here we can leverage the new transactional producer capabilities in 0.11.0.0 that were mentioned above.
+ Since the consumer's position is stored as a message in a topic, we can ensure that that topic is included in the same transaction as the output topics receiving the processed data. If the transaction is aborted,
+ the consumer's position will revert to its old value and none of the output data will be visible to consumers. To enable this, consumers support an "isolation level" to achieve this. In the default
--- End diff --
Seems like this sentence could be clarified a little. `we can ensure that that topic is included in the same transaction`, `that` is repeated and it's unclear what it means to include a topic in a transaction. Did you mean the updates to the topic or something along those lines?
Also `the consumer's position will revert to its old value and none of the output data will be visible to consumers`. It may be worth qualifying `consumer` and `consumers` in that sentence.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] kafka-site issue #60: Update delivery semantics section for KIP-98
Posted by hachikuji <gi...@git.apache.org>.
Github user hachikuji commented on the issue:
https://github.com/apache/kafka-site/pull/60
I'm going to reopen this as a PR against kafka since that is where design.html is maintained.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] kafka-site pull request #60: Update delivery semantics section for KIP-98
Posted by ijuma <gi...@git.apache.org>.
Github user ijuma commented on a diff in the pull request:
https://github.com/apache/kafka-site/pull/60#discussion_r122096221
--- Diff: 0110/design.html ---
@@ -264,21 +264,22 @@
messages have a primary key and so the updates are idempotent (receiving the same message twice just overwrites a record with another copy of itself).
</ol>
<p>
- So what about exactly once semantics (i.e. the thing you actually want)? The limitation here is not actually a feature of the messaging system but rather the need to coordinate the consumer's position with
- what is actually stored as output. The classic way of achieving this would be to introduce a two-phase commit between the storage for the consumer position and the storage of the consumers output. But this can be
- handled more simply and generally by simply letting the consumer store its offset in the same place as its output. This is better because many of the output systems a consumer might want to write to will not
- support a two-phase commit. As an example of this, our Hadoop ETL that populates data in HDFS stores its offsets in HDFS with the data it reads so that it is guaranteed that either data and offsets are both updated
- or neither is. We follow similar patterns for many other data systems which require these stronger semantics and for which the messages do not have a primary key to allow for deduplication.
- <p>
- A special case is when the output system is just another Kafka topic (e.g. in a Kafka Streams application). Here we can leverage the new transactional producer capabilities in 0.11.0.0 that were mentioned above.
- Since the consumer's position is stored as a message in a topic, we can ensure that that topic is included in the same transaction as the output topics receiving the processed data. If the transaction is aborted,
- the consumer's position will revert to its old value and none of the output data will be visible to consumers. To enable this, consumers support an "isolation level" to achieve this. In the default
- "read_uncommitted" mode, all messages are visible to consumers even if they were part of an aborted transaction, but in "read_committed" mode, the consumer will only return data from transactions which were committed
- (and any messages which were not part of any transaction).
- <p>
- So effectively Kafka guarantees at-least-once delivery by default, and allows the user to implement at-most-once delivery by disabling retries on the producer and committing its offset prior to processing a batch of
- messages. Exactly-once delivery is supported when processing messages between Kafka topics, such as in Kafka Streams applications. Exactly-once delivery for other destination storage system generally requires
- cooperation with that system, but Kafka provides the offset which makes implementing this straight-forward.
+ So what about exactly once semantics (i.e. the thing you actually want)? When consuming from a Kafka topic and producing to another topic (as in a <a href="https://kafka.apache.org/documentation/streams">Kafka Streams</a>
+ application), we can leverage the new transactional producer capabilities in 0.11.0.0 that were mentioned above. The consumer's position is stored as a message in a topic, so we can write the offset to Kafka in the
+ same transaction as the output topics receiving the processed data. If the transaction is aborted, the consumer's position will revert to its old value and none of the output data will be visible to consumers. Consumers support an "isolation level" configuration
+ to achieve this. In the default "read_uncommitted" mode, all messages are visible to consumers even if they were part of an aborted transaction, but in "read_committed" mode, the consumer will only return data from
+ transactions which were committed (and any messages which were not part of any transaction).
+ <p>
+ When writing to an external system, the limitation is in the need to coordinate the consumer's position with what is actually stored as output. The classic way of achieving this would be to introduce a two-phase
+ commit between the storage for the consumer position and the storage of the consumers output. But this can be handled more simply and generally by simply letting the consumer store its offset in the same place as
--- End diff --
"storage of the consumer position"? Since we talk about consumer position, should we be saying `consumer's output`?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] kafka-site pull request #60: Update delivery semantics section for KIP-98
Posted by guozhangwang <gi...@git.apache.org>.
Github user guozhangwang commented on a diff in the pull request:
https://github.com/apache/kafka-site/pull/60#discussion_r121492476
--- Diff: 0110/design.html ---
@@ -261,15 +262,23 @@
<li>It can read the messages, process the messages, and finally save its position. In this case there is a possibility that the consumer process crashes after processing messages but before saving its position.
In this case when the new process takes over the first few messages it receives will already have been processed. This corresponds to the "at-least-once" semantics in the case of consumer failure. In many cases
messages have a primary key and so the updates are idempotent (receiving the same message twice just overwrites a record with another copy of itself).
- <li>So what about exactly once semantics (i.e. the thing you actually want)? The limitation here is not actually a feature of the messaging system but rather the need to co-ordinate the consumer's position with
+ </ol>
+ <p>
+ So what about exactly once semantics (i.e. the thing you actually want)? The limitation here is not actually a feature of the messaging system but rather the need to coordinate the consumer's position with
what is actually stored as output. The classic way of achieving this would be to introduce a two-phase commit between the storage for the consumer position and the storage of the consumers output. But this can be
handled more simply and generally by simply letting the consumer store its offset in the same place as its output. This is better because many of the output systems a consumer might want to write to will not
support a two-phase commit. As an example of this, our Hadoop ETL that populates data in HDFS stores its offsets in HDFS with the data it reads so that it is guaranteed that either data and offsets are both updated
or neither is. We follow similar patterns for many other data systems which require these stronger semantics and for which the messages do not have a primary key to allow for deduplication.
- </ol>
<p>
- So effectively Kafka guarantees at-least-once delivery by default and allows the user to implement at most once delivery by disabling retries on the producer and committing its offset prior to processing a batch of
- messages. Exactly-once delivery requires co-operation with the destination storage system but Kafka provides the offset which makes implementing this straight-forward.
+ A special case is when the output system is just another Kafka topic (e.g. in a Kafka Streams application). Here we can leverage the new transactional producer capabilities in 0.11.0.0 that were mentioned above.
+ Since the consumer's position is stored as a message in a topic, we can ensure that that topic is included in the same transaction as the output topics receiving the processed data. If the transaction is aborted,
+ the consumer's position will revert to its old value and none of the output data will be visible to consumers. To enable this, consumers support an "isolation level" to achieve this. In the default
+ "read_uncommitted" mode, all messages are visible to consumers even if they were part of an aborted transaction, but in "read_committed" mode, the consumer will only return data from transactions which were committed
+ (and any messages which were not part of any transaction).
+ <p>
+ So effectively Kafka guarantees at-least-once delivery by default, and allows the user to implement at-most-once delivery by disabling retries on the producer and committing its offset prior to processing a batch of
--- End diff --
+1. Maybe we could say "Kafka guarantees exactly-once delivery, but default it is turned off to optimize performance with at-least-once delivery" blah blah.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] kafka-site issue #60: Update delivery semantics section for KIP-98
Posted by guozhangwang <gi...@git.apache.org>.
Github user guozhangwang commented on the issue:
https://github.com/apache/kafka-site/pull/60
lgtm.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---