You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@flume.apache.org by GitBox <gi...@apache.org> on 2022/11/20 16:33:21 UTC
[GitHub] [flume] tmgstevens opened a new pull request, #391: Add Kafka Transactions to KafkaChannel and KafkaSink, including unit tests
tmgstevens opened a new pull request, #391:
URL: https://github.com/apache/flume/pull/391
This PR is to take advantage of Kafka Transactions (https://cwiki.apache.org/confluence/display/KAFKA/KIP-98+-+Exactly+Once+Delivery+and+Transactional+Messaging) as part of KafkaSink and KafkaChannel, which allows cleaner rollback, Exactly Once Semantics (EOS) but also potentially performance improvements (see https://redpanda.com/blog/fast-transactions for how this works).
I have a docs change alongside this, but essentially this is all controlled by setting the producer `transactional.id` and all of the Kafka code works on whether this has been set or not.
Added unit tests to make sure the behaviour is consistent.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@flume.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flume] tmgstevens merged pull request #391: Add Kafka Transactions to KafkaChannel and KafkaSink, including unit tests
Posted by GitBox <gi...@apache.org>.
tmgstevens merged PR #391:
URL: https://github.com/apache/flume/pull/391
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@flume.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flume] tmgstevens commented on pull request #391: Add Kafka Transactions to KafkaChannel and KafkaSink, including unit tests
Posted by GitBox <gi...@apache.org>.
tmgstevens commented on PR #391:
URL: https://github.com/apache/flume/pull/391#issuecomment-1321226639
Thanks for the review @rgoers ! (And a rare green run as well!!!!)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@flume.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org