[ANNOUNCE] Apache Kafka 2.0.0 Released

[Rajini Sivaram <rs...@apache.org>]

The Apache Kafka community is pleased to announce the release for

Apache Kafka 2.0.0.





This is a major release and includes significant new features from

40 KIPs. It contains fixes and improvements from 246 JIRAs, including

a few critical bugs. Here is a summary of some notable changes:

** KIP-290 adds support for prefixed ACLs, simplifying access control
management in large secure deployments. Bulk access to topics,
consumer groups or transactional ids with a prefix can now be granted
using a single rule. Access control for topic creation has also been
improved to enable access to be granted to create specific topics or
topics with a prefix.

** KIP-255 adds a framework for authenticating to Kafka brokers using
OAuth2 bearer tokens. The SASL/OAUTHBEARER implementation is
customizable using callbacks for token retrieval and validation.

**Host name verification is now enabled by default for SSL connections
to ensure that the default SSL configuration is not susceptible to
man-in-the middle attacks. You can disable this verification for
deployments where validation is performed using other mechanisms.

** You can now dynamically update SSL trust stores without broker restart.
You can also configure security for broker listeners in ZooKeeper before
starting brokers, including SSL key store and trust store passwords and
JAAS configuration for SASL. With this new feature, you can store sensitive
password configs in encrypted form in ZooKeeper rather than in cleartext
in the broker properties file.

** The replication protocol has been improved to avoid log divergence
between leader and follower during fast leader failover. We have also
improved resilience of brokers by reducing the memory footprint of
message down-conversions. By using message chunking, both memory
usage and memory reference time have been reduced to avoid
OutOfMemory errors in brokers.

** Kafka clients are now notified of throttling before any throttling is
applied
when quotas are enabled. This enables clients to distinguish between
network errors and large throttle times when quotas are exceeded.

** We have added a configuration option for Kafka consumer to avoid
indefinite blocking in the consumer.

** We have dropped support for Java 7 and removed the previously
deprecated Scala producer and consumer.

** Kafka Connect includes a number of improvements and features.
KIP-298 enables you to control how errors in connectors, transformations
and converters are handled by enabling automatic retries and controlling the
number of errors that are tolerated before the connector is stopped. More
contextual information can be included in the logs to help diagnose problems
and problematic messages consumed by sink connectors can be sent to a
dead letter queue rather than forcing the connector to stop.

** KIP-297 adds a new extension point to move secrets out of connector
configurations and integrate with any external key management system.
The placeholders in connector configurations are only resolved before
sending the configuration to the connector, ensuring that secrets are stored
and managed securely in your preferred key management system and
not exposed over the REST APIs or in log files.

** We have added a thin Scala wrapper API for our Kafka Streams DSL,
which provides better type inference and better type safety during compile
time. Scala users can have less boilerplate in their code, notably regarding
Serdes with new implicit Serdes.

** Message headers are now supported in the Kafka Streams Processor API,
allowing users to add and manipulate headers read from the source topics
and propagate them to the sink topics.

** Windowed aggregations performance in Kafka Streams has been largely
improved (sometimes by an order of magnitude) thanks to the new
single-key-fetch API.

** We have further improved unit testibility of Kafka Streams with the
kafka-streams-testutil artifact.





All of the changes in this release can be found in the release notes:

https://www.apache.org/dist/kafka/2.0.0/RELEASE_NOTES.html





You can download the source and binary release (Scala 2.11 and Scala 2.12)
from:

https://kafka.apache.org/downloads#2.0.0
<https://kafka.apache.org/downloads#2.0.0>



---------------------------------------------------------------------------------------------------





Apache Kafka is a distributed streaming platform with four core APIs:



** The Producer API allows an application to publish a stream records to

one or more Kafka topics.



** The Consumer API allows an application to subscribe to one or more

topics and process the stream of records produced to them.



** The Streams API allows an application to act as a stream processor,

consuming an input stream from one or more topics and producing an

output stream to one or more output topics, effectively transforming the

input streams to output streams.



** The Connector API allows building and running reusable producers or

consumers that connect Kafka topics to existing applications or data

systems. For example, a connector to a relational database might

capture every change to a table.





With these APIs, Kafka can be used for two broad classes of application:



** Building real-time streaming data pipelines that reliably get data

between systems or applications.



** Building real-time streaming applications that transform or react

to the streams of data.







Apache Kafka is in use at large and small companies worldwide, including

Capital One, Goldman Sachs, ING, LinkedIn, Netflix, Pinterest, Rabobank,

Target, The New York Times, Uber, Yelp, and Zalando, among others.







A big thank you for the following 131 contributors to this release!



Adem Efe Gencer, Alex D, Alex Dunayevsky, Allen Wang, Andras Beni,

Andy Bryant, Andy Coates, Anna Povzner, Arjun Satish, asutosh936,

Attila Sasvari, bartdevylder, Benedict Jin, Bill Bejeck, Blake Miller,

Boyang Chen, cburroughs, Chia-Ping Tsai, Chris Egerton, Colin P. Mccabe,

Colin Patrick McCabe, ConcurrencyPractitioner, Damian Guy, dan norwood,

Daniel Shuy, Daniel Wojda, Dark, David Glasser, Debasish Ghosh, Detharon,

Dhruvil Shah, Dmitry Minkovsky, Dong Lin, Edoardo Comar, emmanuel Harel,

Eugene Sevastyanov, Ewen Cheslack-Postava, Fedor Bobin, fedosov-alexander,

Filipe Agapito, Florian Hussonnois, fredfp, Gilles Degols, gitlw, Gitomain,

Guangxian, Gunju Ko, Gunnar Morling, Guozhang Wang, hmcl, huxi, huxihx,

Igor Kostiakov, Ismael Juma, Jacek Laskowski, Jagadesh Adireddi,

Jarek Rudzinski, Jason Gustafson, Jeff Klukas, Jeremy Custenborder,

Jiangjie (Becket) Qin, Jiangjie Qin, JieFang.He, Jimin Hsieh, Joan Goyeau,

Joel Hamill, John Roesler, Jon Lee, Jorge Quilcate Otoya, Jun Rao,

Kamal C, khairy, Koen De Groote, Konstantine Karantasis, Lee Dongjin,

Liju John, Liquan Pei, lisa2lisa, Lucas Wang, Magesh Nandakumar,

Magnus Edenhill, Magnus Reftel, Manikumar Reddy, Manikumar Reddy O,

manjuapu, Mats Julian Olsen, Matthias J. Sax, Max Zheng, maytals,

Michael Arndt, Michael G. Noll, Mickael Maison, nafshartous, Nick Travers,

nixsticks, Paolo Patierno, parafiend, Patrik Erdes, Radai Rosenblatt,

Rajini Sivaram, Randall Hauch, ro7m, Robert Yokota, Roman Khlebnov,

Ron Dagostino, Sandor Murakozi, Sasaki Toru, Sean Glover,

Sebastian Bauersfeld, Siva Santhalingam, Stanislav Kozlovski, Stephane
Maarek,

Stuart Perks, Surabhi Dixit, Sönke Liebau, taekyung, tedyu, Thomas Leplus,

UVN, Vahid Hashemian, Valentino Proietti, Viktor Somogyi, Vitaly Pushkar,

Wladimir Schmidt, wushujames, Xavier Léauté, xin, yaphet,

Yaswanth Kumar, ying-zheng, Yu







We welcome your help and feedback. For more information on how to

report problems, and to get involved, visit the project website at

https://kafka.apache.org/





Thank you!





Regards,



Rajini

Re: [ANNOUNCE] Apache Kafka 2.0.0 Released

[Damian Guy <da...@gmail.com>]

Excellent! Thanks for running the release Rajini!

On Mon, 30 Jul 2018 at 11:25 Rajini Sivaram <rs...@apache.org> wrote:

> The Apache Kafka community is pleased to announce the release for
>
> Apache Kafka 2.0.0.
>
>
>
>
>
> This is a major release and includes significant new features from
>
> 40 KIPs. It contains fixes and improvements from 246 JIRAs, including
>
> a few critical bugs. Here is a summary of some notable changes:
>
> ** KIP-290 adds support for prefixed ACLs, simplifying access control
> management in large secure deployments. Bulk access to topics,
> consumer groups or transactional ids with a prefix can now be granted
> using a single rule. Access control for topic creation has also been
> improved to enable access to be granted to create specific topics or
> topics with a prefix.
>
> ** KIP-255 adds a framework for authenticating to Kafka brokers using
> OAuth2 bearer tokens. The SASL/OAUTHBEARER implementation is
> customizable using callbacks for token retrieval and validation.
>
> **Host name verification is now enabled by default for SSL connections
> to ensure that the default SSL configuration is not susceptible to
> man-in-the middle attacks. You can disable this verification for
> deployments where validation is performed using other mechanisms.
>
> ** You can now dynamically update SSL trust stores without broker restart.
> You can also configure security for broker listeners in ZooKeeper before
> starting brokers, including SSL key store and trust store passwords and
> JAAS configuration for SASL. With this new feature, you can store sensitive
> password configs in encrypted form in ZooKeeper rather than in cleartext
> in the broker properties file.
>
> ** The replication protocol has been improved to avoid log divergence
> between leader and follower during fast leader failover. We have also
> improved resilience of brokers by reducing the memory footprint of
> message down-conversions. By using message chunking, both memory
> usage and memory reference time have been reduced to avoid
> OutOfMemory errors in brokers.
>
> ** Kafka clients are now notified of throttling before any throttling is
> applied
> when quotas are enabled. This enables clients to distinguish between
> network errors and large throttle times when quotas are exceeded.
>
> ** We have added a configuration option for Kafka consumer to avoid
> indefinite blocking in the consumer.
>
> ** We have dropped support for Java 7 and removed the previously
> deprecated Scala producer and consumer.
>
> ** Kafka Connect includes a number of improvements and features.
> KIP-298 enables you to control how errors in connectors, transformations
> and converters are handled by enabling automatic retries and controlling
> the
> number of errors that are tolerated before the connector is stopped. More
> contextual information can be included in the logs to help diagnose
> problems
> and problematic messages consumed by sink connectors can be sent to a
> dead letter queue rather than forcing the connector to stop.
>
> ** KIP-297 adds a new extension point to move secrets out of connector
> configurations and integrate with any external key management system.
> The placeholders in connector configurations are only resolved before
> sending the configuration to the connector, ensuring that secrets are
> stored
> and managed securely in your preferred key management system and
> not exposed over the REST APIs or in log files.
>
> ** We have added a thin Scala wrapper API for our Kafka Streams DSL,
> which provides better type inference and better type safety during compile
> time. Scala users can have less boilerplate in their code, notably
> regarding
> Serdes with new implicit Serdes.
>
> ** Message headers are now supported in the Kafka Streams Processor API,
> allowing users to add and manipulate headers read from the source topics
> and propagate them to the sink topics.
>
> ** Windowed aggregations performance in Kafka Streams has been largely
> improved (sometimes by an order of magnitude) thanks to the new
> single-key-fetch API.
>
> ** We have further improved unit testibility of Kafka Streams with the
> kafka-streams-testutil artifact.
>
>
>
>
>
> All of the changes in this release can be found in the release notes:
>
> https://www.apache.org/dist/kafka/2.0.0/RELEASE_NOTES.html
>
>
>
>
>
> You can download the source and binary release (Scala 2.11 and Scala 2.12)
> from:
>
> https://kafka.apache.org/downloads#2.0.0
> <https://kafka.apache.org/downloads#2.0.0>
>
>
>
>
> ---------------------------------------------------------------------------------------------------
>
>
>
>
>
> Apache Kafka is a distributed streaming platform with four core APIs:
>
>
>
> ** The Producer API allows an application to publish a stream records to
>
> one or more Kafka topics.
>
>
>
> ** The Consumer API allows an application to subscribe to one or more
>
> topics and process the stream of records produced to them.
>
>
>
> ** The Streams API allows an application to act as a stream processor,
>
> consuming an input stream from one or more topics and producing an
>
> output stream to one or more output topics, effectively transforming the
>
> input streams to output streams.
>
>
>
> ** The Connector API allows building and running reusable producers or
>
> consumers that connect Kafka topics to existing applications or data
>
> systems. For example, a connector to a relational database might
>
> capture every change to a table.
>
>
>
>
>
> With these APIs, Kafka can be used for two broad classes of application:
>
>
>
> ** Building real-time streaming data pipelines that reliably get data
>
> between systems or applications.
>
>
>
> ** Building real-time streaming applications that transform or react
>
> to the streams of data.
>
>
>
>
>
>
>
> Apache Kafka is in use at large and small companies worldwide, including
>
> Capital One, Goldman Sachs, ING, LinkedIn, Netflix, Pinterest, Rabobank,
>
> Target, The New York Times, Uber, Yelp, and Zalando, among others.
>
>
>
>
>
>
>
> A big thank you for the following 131 contributors to this release!
>
>
>
> Adem Efe Gencer, Alex D, Alex Dunayevsky, Allen Wang, Andras Beni,
>
> Andy Bryant, Andy Coates, Anna Povzner, Arjun Satish, asutosh936,
>
> Attila Sasvari, bartdevylder, Benedict Jin, Bill Bejeck, Blake Miller,
>
> Boyang Chen, cburroughs, Chia-Ping Tsai, Chris Egerton, Colin P. Mccabe,
>
> Colin Patrick McCabe, ConcurrencyPractitioner, Damian Guy, dan norwood,
>
> Daniel Shuy, Daniel Wojda, Dark, David Glasser, Debasish Ghosh, Detharon,
>
> Dhruvil Shah, Dmitry Minkovsky, Dong Lin, Edoardo Comar, emmanuel Harel,
>
> Eugene Sevastyanov, Ewen Cheslack-Postava, Fedor Bobin, fedosov-alexander,
>
> Filipe Agapito, Florian Hussonnois, fredfp, Gilles Degols, gitlw, Gitomain,
>
> Guangxian, Gunju Ko, Gunnar Morling, Guozhang Wang, hmcl, huxi, huxihx,
>
> Igor Kostiakov, Ismael Juma, Jacek Laskowski, Jagadesh Adireddi,
>
> Jarek Rudzinski, Jason Gustafson, Jeff Klukas, Jeremy Custenborder,
>
> Jiangjie (Becket) Qin, Jiangjie Qin, JieFang.He, Jimin Hsieh, Joan Goyeau,
>
> Joel Hamill, John Roesler, Jon Lee, Jorge Quilcate Otoya, Jun Rao,
>
> Kamal C, khairy, Koen De Groote, Konstantine Karantasis, Lee Dongjin,
>
> Liju John, Liquan Pei, lisa2lisa, Lucas Wang, Magesh Nandakumar,
>
> Magnus Edenhill, Magnus Reftel, Manikumar Reddy, Manikumar Reddy O,
>
> manjuapu, Mats Julian Olsen, Matthias J. Sax, Max Zheng, maytals,
>
> Michael Arndt, Michael G. Noll, Mickael Maison, nafshartous, Nick Travers,
>
> nixsticks, Paolo Patierno, parafiend, Patrik Erdes, Radai Rosenblatt,
>
> Rajini Sivaram, Randall Hauch, ro7m, Robert Yokota, Roman Khlebnov,
>
> Ron Dagostino, Sandor Murakozi, Sasaki Toru, Sean Glover,
>
> Sebastian Bauersfeld, Siva Santhalingam, Stanislav Kozlovski, Stephane
> Maarek,
>
> Stuart Perks, Surabhi Dixit, Sönke Liebau, taekyung, tedyu, Thomas Leplus,
>
> UVN, Vahid Hashemian, Valentino Proietti, Viktor Somogyi, Vitaly Pushkar,
>
> Wladimir Schmidt, wushujames, Xavier Léauté, xin, yaphet,
>
> Yaswanth Kumar, ying-zheng, Yu
>
>
>
>
>
>
>
> We welcome your help and feedback. For more information on how to
>
> report problems, and to get involved, visit the project website at
>
> https://kafka.apache.org/
>
>
>
>
>
> Thank you!
>
>
>
>
>
> Regards,
>
>
>
> Rajini
>

Re: [ANNOUNCE] Apache Kafka 2.0.0 Released

[Dong Lin <li...@gmail.com>]

This is great news! Thanks for driving this Rajini!!

On Mon, Jul 30, 2018 at 3:25 AM, Rajini Sivaram <rs...@apache.org> wrote:

> The Apache Kafka community is pleased to announce the release for
>
> Apache Kafka 2.0.0.
>
>
>
>
>
> This is a major release and includes significant new features from
>
> 40 KIPs. It contains fixes and improvements from 246 JIRAs, including
>
> a few critical bugs. Here is a summary of some notable changes:
>
> ** KIP-290 adds support for prefixed ACLs, simplifying access control
> management in large secure deployments. Bulk access to topics,
> consumer groups or transactional ids with a prefix can now be granted
> using a single rule. Access control for topic creation has also been
> improved to enable access to be granted to create specific topics or
> topics with a prefix.
>
> ** KIP-255 adds a framework for authenticating to Kafka brokers using
> OAuth2 bearer tokens. The SASL/OAUTHBEARER implementation is
> customizable using callbacks for token retrieval and validation.
>
> **Host name verification is now enabled by default for SSL connections
> to ensure that the default SSL configuration is not susceptible to
> man-in-the middle attacks. You can disable this verification for
> deployments where validation is performed using other mechanisms.
>
> ** You can now dynamically update SSL trust stores without broker restart.
> You can also configure security for broker listeners in ZooKeeper before
> starting brokers, including SSL key store and trust store passwords and
> JAAS configuration for SASL. With this new feature, you can store sensitive
> password configs in encrypted form in ZooKeeper rather than in cleartext
> in the broker properties file.
>
> ** The replication protocol has been improved to avoid log divergence
> between leader and follower during fast leader failover. We have also
> improved resilience of brokers by reducing the memory footprint of
> message down-conversions. By using message chunking, both memory
> usage and memory reference time have been reduced to avoid
> OutOfMemory errors in brokers.
>
> ** Kafka clients are now notified of throttling before any throttling is
> applied
> when quotas are enabled. This enables clients to distinguish between
> network errors and large throttle times when quotas are exceeded.
>
> ** We have added a configuration option for Kafka consumer to avoid
> indefinite blocking in the consumer.
>
> ** We have dropped support for Java 7 and removed the previously
> deprecated Scala producer and consumer.
>
> ** Kafka Connect includes a number of improvements and features.
> KIP-298 enables you to control how errors in connectors, transformations
> and converters are handled by enabling automatic retries and controlling
> the
> number of errors that are tolerated before the connector is stopped. More
> contextual information can be included in the logs to help diagnose
> problems
> and problematic messages consumed by sink connectors can be sent to a
> dead letter queue rather than forcing the connector to stop.
>
> ** KIP-297 adds a new extension point to move secrets out of connector
> configurations and integrate with any external key management system.
> The placeholders in connector configurations are only resolved before
> sending the configuration to the connector, ensuring that secrets are
> stored
> and managed securely in your preferred key management system and
> not exposed over the REST APIs or in log files.
>
> ** We have added a thin Scala wrapper API for our Kafka Streams DSL,
> which provides better type inference and better type safety during compile
> time. Scala users can have less boilerplate in their code, notably
> regarding
> Serdes with new implicit Serdes.
>
> ** Message headers are now supported in the Kafka Streams Processor API,
> allowing users to add and manipulate headers read from the source topics
> and propagate them to the sink topics.
>
> ** Windowed aggregations performance in Kafka Streams has been largely
> improved (sometimes by an order of magnitude) thanks to the new
> single-key-fetch API.
>
> ** We have further improved unit testibility of Kafka Streams with the
> kafka-streams-testutil artifact.
>
>
>
>
>
> All of the changes in this release can be found in the release notes:
>
> https://www.apache.org/dist/kafka/2.0.0/RELEASE_NOTES.html
>
>
>
>
>
> You can download the source and binary release (Scala 2.11 and Scala 2.12)
> from:
>
> https://kafka.apache.org/downloads#2.0.0
> <https://kafka.apache.org/downloads#2.0.0>
>
>
>
> ------------------------------------------------------------
> ---------------------------------------
>
>
>
>
>
> Apache Kafka is a distributed streaming platform with four core APIs:
>
>
>
> ** The Producer API allows an application to publish a stream records to
>
> one or more Kafka topics.
>
>
>
> ** The Consumer API allows an application to subscribe to one or more
>
> topics and process the stream of records produced to them.
>
>
>
> ** The Streams API allows an application to act as a stream processor,
>
> consuming an input stream from one or more topics and producing an
>
> output stream to one or more output topics, effectively transforming the
>
> input streams to output streams.
>
>
>
> ** The Connector API allows building and running reusable producers or
>
> consumers that connect Kafka topics to existing applications or data
>
> systems. For example, a connector to a relational database might
>
> capture every change to a table.
>
>
>
>
>
> With these APIs, Kafka can be used for two broad classes of application:
>
>
>
> ** Building real-time streaming data pipelines that reliably get data
>
> between systems or applications.
>
>
>
> ** Building real-time streaming applications that transform or react
>
> to the streams of data.
>
>
>
>
>
>
>
> Apache Kafka is in use at large and small companies worldwide, including
>
> Capital One, Goldman Sachs, ING, LinkedIn, Netflix, Pinterest, Rabobank,
>
> Target, The New York Times, Uber, Yelp, and Zalando, among others.
>
>
>
>
>
>
>
> A big thank you for the following 131 contributors to this release!
>
>
>
> Adem Efe Gencer, Alex D, Alex Dunayevsky, Allen Wang, Andras Beni,
>
> Andy Bryant, Andy Coates, Anna Povzner, Arjun Satish, asutosh936,
>
> Attila Sasvari, bartdevylder, Benedict Jin, Bill Bejeck, Blake Miller,
>
> Boyang Chen, cburroughs, Chia-Ping Tsai, Chris Egerton, Colin P. Mccabe,
>
> Colin Patrick McCabe, ConcurrencyPractitioner, Damian Guy, dan norwood,
>
> Daniel Shuy, Daniel Wojda, Dark, David Glasser, Debasish Ghosh, Detharon,
>
> Dhruvil Shah, Dmitry Minkovsky, Dong Lin, Edoardo Comar, emmanuel Harel,
>
> Eugene Sevastyanov, Ewen Cheslack-Postava, Fedor Bobin, fedosov-alexander,
>
> Filipe Agapito, Florian Hussonnois, fredfp, Gilles Degols, gitlw, Gitomain,
>
> Guangxian, Gunju Ko, Gunnar Morling, Guozhang Wang, hmcl, huxi, huxihx,
>
> Igor Kostiakov, Ismael Juma, Jacek Laskowski, Jagadesh Adireddi,
>
> Jarek Rudzinski, Jason Gustafson, Jeff Klukas, Jeremy Custenborder,
>
> Jiangjie (Becket) Qin, Jiangjie Qin, JieFang.He, Jimin Hsieh, Joan Goyeau,
>
> Joel Hamill, John Roesler, Jon Lee, Jorge Quilcate Otoya, Jun Rao,
>
> Kamal C, khairy, Koen De Groote, Konstantine Karantasis, Lee Dongjin,
>
> Liju John, Liquan Pei, lisa2lisa, Lucas Wang, Magesh Nandakumar,
>
> Magnus Edenhill, Magnus Reftel, Manikumar Reddy, Manikumar Reddy O,
>
> manjuapu, Mats Julian Olsen, Matthias J. Sax, Max Zheng, maytals,
>
> Michael Arndt, Michael G. Noll, Mickael Maison, nafshartous, Nick Travers,
>
> nixsticks, Paolo Patierno, parafiend, Patrik Erdes, Radai Rosenblatt,
>
> Rajini Sivaram, Randall Hauch, ro7m, Robert Yokota, Roman Khlebnov,
>
> Ron Dagostino, Sandor Murakozi, Sasaki Toru, Sean Glover,
>
> Sebastian Bauersfeld, Siva Santhalingam, Stanislav Kozlovski, Stephane
> Maarek,
>
> Stuart Perks, Surabhi Dixit, Sönke Liebau, taekyung, tedyu, Thomas Leplus,
>
> UVN, Vahid Hashemian, Valentino Proietti, Viktor Somogyi, Vitaly Pushkar,
>
> Wladimir Schmidt, wushujames, Xavier Léauté, xin, yaphet,
>
> Yaswanth Kumar, ying-zheng, Yu
>
>
>
>
>
>
>
> We welcome your help and feedback. For more information on how to
>
> report problems, and to get involved, visit the project website at
>
> https://kafka.apache.org/
>
>
>
>
>
> Thank you!
>
>
>
>
>
> Regards,
>
>
>
> Rajini
>

Re: [ANNOUNCE] Apache Kafka 2.0.0 Released

[Guozhang Wang <wa...@gmail.com>]

Great news! Thanks for driving the release Rajini.


Guozhang

On Mon, Jul 30, 2018 at 8:33 AM, Vahid S Hashemian <
vahidhashemian@us.ibm.com> wrote:

> Such a good news on a Monday morning ...
>
> Thank you Rajini for driving the release!
>
> --Vahid
>
>
>
>
> From:   Mickael Maison <mi...@gmail.com>
> To:     Users <us...@kafka.apache.org>
> Cc:     dev <de...@kafka.apache.org>, announce@apache.org, kafka-clients
> <ka...@googlegroups.com>
> Date:   07/30/2018 04:37 AM
> Subject:        Re: [ANNOUNCE] Apache Kafka 2.0.0 Released
>
>
>
> Great news! Thanks for running the release
>
> On Mon, Jul 30, 2018 at 12:20 PM, Manikumar <ma...@gmail.com>
> wrote:
> > Thanks for driving the release!
> >
> >
> >
> > On Mon, Jul 30, 2018 at 3:55 PM Rajini Sivaram <rs...@apache.org>
> wrote:
> >
> >> The Apache Kafka community is pleased to announce the release for
> >>
> >> Apache Kafka 2.0.0.
> >>
> >>
> >>
> >>
> >>
> >> This is a major release and includes significant new features from
> >>
> >> 40 KIPs. It contains fixes and improvements from 246 JIRAs, including
> >>
> >> a few critical bugs. Here is a summary of some notable changes:
> >>
> >> ** KIP-290 adds support for prefixed ACLs, simplifying access control
> >> management in large secure deployments. Bulk access to topics,
> >> consumer groups or transactional ids with a prefix can now be granted
> >> using a single rule. Access control for topic creation has also been
> >> improved to enable access to be granted to create specific topics or
> >> topics with a prefix.
> >>
> >> ** KIP-255 adds a framework for authenticating to Kafka brokers using
> >> OAuth2 bearer tokens. The SASL/OAUTHBEARER implementation is
> >> customizable using callbacks for token retrieval and validation.
> >>
> >> **Host name verification is now enabled by default for SSL connections
> >> to ensure that the default SSL configuration is not susceptible to
> >> man-in-the middle attacks. You can disable this verification for
> >> deployments where validation is performed using other mechanisms.
> >>
> >> ** You can now dynamically update SSL trust stores without broker
> restart.
> >> You can also configure security for broker listeners in ZooKeeper
> before
> >> starting brokers, including SSL key store and trust store passwords and
> >> JAAS configuration for SASL. With this new feature, you can store
> sensitive
> >> password configs in encrypted form in ZooKeeper rather than in
> cleartext
> >> in the broker properties file.
> >>
> >> ** The replication protocol has been improved to avoid log divergence
> >> between leader and follower during fast leader failover. We have also
> >> improved resilience of brokers by reducing the memory footprint of
> >> message down-conversions. By using message chunking, both memory
> >> usage and memory reference time have been reduced to avoid
> >> OutOfMemory errors in brokers.
> >>
> >> ** Kafka clients are now notified of throttling before any throttling
> is
> >> applied
> >> when quotas are enabled. This enables clients to distinguish between
> >> network errors and large throttle times when quotas are exceeded.
> >>
> >> ** We have added a configuration option for Kafka consumer to avoid
> >> indefinite blocking in the consumer.
> >>
> >> ** We have dropped support for Java 7 and removed the previously
> >> deprecated Scala producer and consumer.
> >>
> >> ** Kafka Connect includes a number of improvements and features.
> >> KIP-298 enables you to control how errors in connectors,
> transformations
> >> and converters are handled by enabling automatic retries and
> controlling
> >> the
> >> number of errors that are tolerated before the connector is stopped.
> More
> >> contextual information can be included in the logs to help diagnose
> >> problems
> >> and problematic messages consumed by sink connectors can be sent to a
> >> dead letter queue rather than forcing the connector to stop.
> >>
> >> ** KIP-297 adds a new extension point to move secrets out of connector
> >> configurations and integrate with any external key management system.
> >> The placeholders in connector configurations are only resolved before
> >> sending the configuration to the connector, ensuring that secrets are
> >> stored
> >> and managed securely in your preferred key management system and
> >> not exposed over the REST APIs or in log files.
> >>
> >> ** We have added a thin Scala wrapper API for our Kafka Streams DSL,
> >> which provides better type inference and better type safety during
> compile
> >> time. Scala users can have less boilerplate in their code, notably
> >> regarding
> >> Serdes with new implicit Serdes.
> >>
> >> ** Message headers are now supported in the Kafka Streams Processor
> API,
> >> allowing users to add and manipulate headers read from the source
> topics
> >> and propagate them to the sink topics.
> >>
> >> ** Windowed aggregations performance in Kafka Streams has been largely
> >> improved (sometimes by an order of magnitude) thanks to the new
> >> single-key-fetch API.
> >>
> >> ** We have further improved unit testibility of Kafka Streams with the
> >> kafka-streams-testutil artifact.
> >>
> >>
> >>
> >>
> >>
> >> All of the changes in this release can be found in the release notes:
> >>
> >>
> https://www.apache.org/dist/kafka/2.0.0/RELEASE_NOTES.html
>
> >>
> >>
> >>
> >>
> >>
> >> You can download the source and binary release (Scala 2.11 and Scala
> 2.12)
> >> from:
> >>
> >>
> https://kafka.apache.org/downloads#2.0.0
>
> >> <
> https://kafka.apache.org/downloads#2.0.0
> >
> >>
> >>
> >>
> >>
> >>
> ------------------------------------------------------------
> ---------------------------------------
> >>
> >>
> >>
> >>
> >>
> >> Apache Kafka is a distributed streaming platform with four core APIs:
> >>
> >>
> >>
> >> ** The Producer API allows an application to publish a stream records
> to
> >>
> >> one or more Kafka topics.
> >>
> >>
> >>
> >> ** The Consumer API allows an application to subscribe to one or more
> >>
> >> topics and process the stream of records produced to them.
> >>
> >>
> >>
> >> ** The Streams API allows an application to act as a stream processor,
> >>
> >> consuming an input stream from one or more topics and producing an
> >>
> >> output stream to one or more output topics, effectively transforming
> the
> >>
> >> input streams to output streams.
> >>
> >>
> >>
> >> ** The Connector API allows building and running reusable producers or
> >>
> >> consumers that connect Kafka topics to existing applications or data
> >>
> >> systems. For example, a connector to a relational database might
> >>
> >> capture every change to a table.
> >>
> >>
> >>
> >>
> >>
> >> With these APIs, Kafka can be used for two broad classes of
> application:
> >>
> >>
> >>
> >> ** Building real-time streaming data pipelines that reliably get data
> >>
> >> between systems or applications.
> >>
> >>
> >>
> >> ** Building real-time streaming applications that transform or react
> >>
> >> to the streams of data.
> >>
> >>
> >>
> >>
> >>
> >>
> >>
> >> Apache Kafka is in use at large and small companies worldwide,
> including
> >>
> >> Capital One, Goldman Sachs, ING, LinkedIn, Netflix, Pinterest,
> Rabobank,
> >>
> >> Target, The New York Times, Uber, Yelp, and Zalando, among others.
> >>
> >>
> >>
> >>
> >>
> >>
> >>
> >> A big thank you for the following 131 contributors to this release!
> >>
> >>
> >>
> >> Adem Efe Gencer, Alex D, Alex Dunayevsky, Allen Wang, Andras Beni,
> >>
> >> Andy Bryant, Andy Coates, Anna Povzner, Arjun Satish, asutosh936,
> >>
> >> Attila Sasvari, bartdevylder, Benedict Jin, Bill Bejeck, Blake Miller,
> >>
> >> Boyang Chen, cburroughs, Chia-Ping Tsai, Chris Egerton, Colin P.
> Mccabe,
> >>
> >> Colin Patrick McCabe, ConcurrencyPractitioner, Damian Guy, dan norwood,
> >>
> >> Daniel Shuy, Daniel Wojda, Dark, David Glasser, Debasish Ghosh,
> Detharon,
> >>
> >> Dhruvil Shah, Dmitry Minkovsky, Dong Lin, Edoardo Comar, emmanuel
> Harel,
> >>
> >> Eugene Sevastyanov, Ewen Cheslack-Postava, Fedor Bobin,
> fedosov-alexander,
> >>
> >> Filipe Agapito, Florian Hussonnois, fredfp, Gilles Degols, gitlw,
> Gitomain,
> >>
> >> Guangxian, Gunju Ko, Gunnar Morling, Guozhang Wang, hmcl, huxi, huxihx,
> >>
> >> Igor Kostiakov, Ismael Juma, Jacek Laskowski, Jagadesh Adireddi,
> >>
> >> Jarek Rudzinski, Jason Gustafson, Jeff Klukas, Jeremy Custenborder,
> >>
> >> Jiangjie (Becket) Qin, Jiangjie Qin, JieFang.He, Jimin Hsieh, Joan
> Goyeau,
> >>
> >> Joel Hamill, John Roesler, Jon Lee, Jorge Quilcate Otoya, Jun Rao,
> >>
> >> Kamal C, khairy, Koen De Groote, Konstantine Karantasis, Lee Dongjin,
> >>
> >> Liju John, Liquan Pei, lisa2lisa, Lucas Wang, Magesh Nandakumar,
> >>
> >> Magnus Edenhill, Magnus Reftel, Manikumar Reddy, Manikumar Reddy O,
> >>
> >> manjuapu, Mats Julian Olsen, Matthias J. Sax, Max Zheng, maytals,
> >>
> >> Michael Arndt, Michael G. Noll, Mickael Maison, nafshartous, Nick
> Travers,
> >>
> >> nixsticks, Paolo Patierno, parafiend, Patrik Erdes, Radai Rosenblatt,
> >>
> >> Rajini Sivaram, Randall Hauch, ro7m, Robert Yokota, Roman Khlebnov,
> >>
> >> Ron Dagostino, Sandor Murakozi, Sasaki Toru, Sean Glover,
> >>
> >> Sebastian Bauersfeld, Siva Santhalingam, Stanislav Kozlovski, Stephane
> >> Maarek,
> >>
> >> Stuart Perks, Surabhi Dixit, Sönke Liebau, taekyung, tedyu, Thomas
> Leplus,
> >>
> >> UVN, Vahid Hashemian, Valentino Proietti, Viktor Somogyi, Vitaly
> Pushkar,
> >>
> >> Wladimir Schmidt, wushujames, Xavier Léauté, xin, yaphet,
> >>
> >> Yaswanth Kumar, ying-zheng, Yu
> >>
> >>
> >>
> >>
> >>
> >>
> >>
> >> We welcome your help and feedback. For more information on how to
> >>
> >> report problems, and to get involved, visit the project website at
> >>
> >>
> https://kafka.apache.org/
>
> >>
> >>
> >>
> >>
> >>
> >> Thank you!
> >>
> >>
> >>
> >>
> >>
> >> Regards,
> >>
> >>
> >>
> >> Rajini
> >>
>
>
>
>
>
>


-- 
-- Guozhang

Re: [ANNOUNCE] Apache Kafka 2.0.0 Released

[Vahid S Hashemian <va...@us.ibm.com>]

Such a good news on a Monday morning ...

Thank you Rajini for driving the release!

--Vahid




From:   Mickael Maison <mi...@gmail.com>
To:     Users <us...@kafka.apache.org>
Cc:     dev <de...@kafka.apache.org>, announce@apache.org, kafka-clients 
<ka...@googlegroups.com>
Date:   07/30/2018 04:37 AM
Subject:        Re: [ANNOUNCE] Apache Kafka 2.0.0 Released



Great news! Thanks for running the release

On Mon, Jul 30, 2018 at 12:20 PM, Manikumar <ma...@gmail.com> 
wrote:
> Thanks for driving the release!
>
>
>
> On Mon, Jul 30, 2018 at 3:55 PM Rajini Sivaram <rs...@apache.org> 
wrote:
>
>> The Apache Kafka community is pleased to announce the release for
>>
>> Apache Kafka 2.0.0.
>>
>>
>>
>>
>>
>> This is a major release and includes significant new features from
>>
>> 40 KIPs. It contains fixes and improvements from 246 JIRAs, including
>>
>> a few critical bugs. Here is a summary of some notable changes:
>>
>> ** KIP-290 adds support for prefixed ACLs, simplifying access control
>> management in large secure deployments. Bulk access to topics,
>> consumer groups or transactional ids with a prefix can now be granted
>> using a single rule. Access control for topic creation has also been
>> improved to enable access to be granted to create specific topics or
>> topics with a prefix.
>>
>> ** KIP-255 adds a framework for authenticating to Kafka brokers using
>> OAuth2 bearer tokens. The SASL/OAUTHBEARER implementation is
>> customizable using callbacks for token retrieval and validation.
>>
>> **Host name verification is now enabled by default for SSL connections
>> to ensure that the default SSL configuration is not susceptible to
>> man-in-the middle attacks. You can disable this verification for
>> deployments where validation is performed using other mechanisms.
>>
>> ** You can now dynamically update SSL trust stores without broker 
restart.
>> You can also configure security for broker listeners in ZooKeeper 
before
>> starting brokers, including SSL key store and trust store passwords and
>> JAAS configuration for SASL. With this new feature, you can store 
sensitive
>> password configs in encrypted form in ZooKeeper rather than in 
cleartext
>> in the broker properties file.
>>
>> ** The replication protocol has been improved to avoid log divergence
>> between leader and follower during fast leader failover. We have also
>> improved resilience of brokers by reducing the memory footprint of
>> message down-conversions. By using message chunking, both memory
>> usage and memory reference time have been reduced to avoid
>> OutOfMemory errors in brokers.
>>
>> ** Kafka clients are now notified of throttling before any throttling 
is
>> applied
>> when quotas are enabled. This enables clients to distinguish between
>> network errors and large throttle times when quotas are exceeded.
>>
>> ** We have added a configuration option for Kafka consumer to avoid
>> indefinite blocking in the consumer.
>>
>> ** We have dropped support for Java 7 and removed the previously
>> deprecated Scala producer and consumer.
>>
>> ** Kafka Connect includes a number of improvements and features.
>> KIP-298 enables you to control how errors in connectors, 
transformations
>> and converters are handled by enabling automatic retries and 
controlling
>> the
>> number of errors that are tolerated before the connector is stopped. 
More
>> contextual information can be included in the logs to help diagnose
>> problems
>> and problematic messages consumed by sink connectors can be sent to a
>> dead letter queue rather than forcing the connector to stop.
>>
>> ** KIP-297 adds a new extension point to move secrets out of connector
>> configurations and integrate with any external key management system.
>> The placeholders in connector configurations are only resolved before
>> sending the configuration to the connector, ensuring that secrets are
>> stored
>> and managed securely in your preferred key management system and
>> not exposed over the REST APIs or in log files.
>>
>> ** We have added a thin Scala wrapper API for our Kafka Streams DSL,
>> which provides better type inference and better type safety during 
compile
>> time. Scala users can have less boilerplate in their code, notably
>> regarding
>> Serdes with new implicit Serdes.
>>
>> ** Message headers are now supported in the Kafka Streams Processor 
API,
>> allowing users to add and manipulate headers read from the source 
topics
>> and propagate them to the sink topics.
>>
>> ** Windowed aggregations performance in Kafka Streams has been largely
>> improved (sometimes by an order of magnitude) thanks to the new
>> single-key-fetch API.
>>
>> ** We have further improved unit testibility of Kafka Streams with the
>> kafka-streams-testutil artifact.
>>
>>
>>
>>
>>
>> All of the changes in this release can be found in the release notes:
>>
>> 
https://www.apache.org/dist/kafka/2.0.0/RELEASE_NOTES.html

>>
>>
>>
>>
>>
>> You can download the source and binary release (Scala 2.11 and Scala 
2.12)
>> from:
>>
>> 
https://kafka.apache.org/downloads#2.0.0

>> <
https://kafka.apache.org/downloads#2.0.0
>
>>
>>
>>
>>
>> 
---------------------------------------------------------------------------------------------------
>>
>>
>>
>>
>>
>> Apache Kafka is a distributed streaming platform with four core APIs:
>>
>>
>>
>> ** The Producer API allows an application to publish a stream records 
to
>>
>> one or more Kafka topics.
>>
>>
>>
>> ** The Consumer API allows an application to subscribe to one or more
>>
>> topics and process the stream of records produced to them.
>>
>>
>>
>> ** The Streams API allows an application to act as a stream processor,
>>
>> consuming an input stream from one or more topics and producing an
>>
>> output stream to one or more output topics, effectively transforming 
the
>>
>> input streams to output streams.
>>
>>
>>
>> ** The Connector API allows building and running reusable producers or
>>
>> consumers that connect Kafka topics to existing applications or data
>>
>> systems. For example, a connector to a relational database might
>>
>> capture every change to a table.
>>
>>
>>
>>
>>
>> With these APIs, Kafka can be used for two broad classes of 
application:
>>
>>
>>
>> ** Building real-time streaming data pipelines that reliably get data
>>
>> between systems or applications.
>>
>>
>>
>> ** Building real-time streaming applications that transform or react
>>
>> to the streams of data.
>>
>>
>>
>>
>>
>>
>>
>> Apache Kafka is in use at large and small companies worldwide, 
including
>>
>> Capital One, Goldman Sachs, ING, LinkedIn, Netflix, Pinterest, 
Rabobank,
>>
>> Target, The New York Times, Uber, Yelp, and Zalando, among others.
>>
>>
>>
>>
>>
>>
>>
>> A big thank you for the following 131 contributors to this release!
>>
>>
>>
>> Adem Efe Gencer, Alex D, Alex Dunayevsky, Allen Wang, Andras Beni,
>>
>> Andy Bryant, Andy Coates, Anna Povzner, Arjun Satish, asutosh936,
>>
>> Attila Sasvari, bartdevylder, Benedict Jin, Bill Bejeck, Blake Miller,
>>
>> Boyang Chen, cburroughs, Chia-Ping Tsai, Chris Egerton, Colin P. 
Mccabe,
>>
>> Colin Patrick McCabe, ConcurrencyPractitioner, Damian Guy, dan norwood,
>>
>> Daniel Shuy, Daniel Wojda, Dark, David Glasser, Debasish Ghosh, 
Detharon,
>>
>> Dhruvil Shah, Dmitry Minkovsky, Dong Lin, Edoardo Comar, emmanuel 
Harel,
>>
>> Eugene Sevastyanov, Ewen Cheslack-Postava, Fedor Bobin, 
fedosov-alexander,
>>
>> Filipe Agapito, Florian Hussonnois, fredfp, Gilles Degols, gitlw, 
Gitomain,
>>
>> Guangxian, Gunju Ko, Gunnar Morling, Guozhang Wang, hmcl, huxi, huxihx,
>>
>> Igor Kostiakov, Ismael Juma, Jacek Laskowski, Jagadesh Adireddi,
>>
>> Jarek Rudzinski, Jason Gustafson, Jeff Klukas, Jeremy Custenborder,
>>
>> Jiangjie (Becket) Qin, Jiangjie Qin, JieFang.He, Jimin Hsieh, Joan 
Goyeau,
>>
>> Joel Hamill, John Roesler, Jon Lee, Jorge Quilcate Otoya, Jun Rao,
>>
>> Kamal C, khairy, Koen De Groote, Konstantine Karantasis, Lee Dongjin,
>>
>> Liju John, Liquan Pei, lisa2lisa, Lucas Wang, Magesh Nandakumar,
>>
>> Magnus Edenhill, Magnus Reftel, Manikumar Reddy, Manikumar Reddy O,
>>
>> manjuapu, Mats Julian Olsen, Matthias J. Sax, Max Zheng, maytals,
>>
>> Michael Arndt, Michael G. Noll, Mickael Maison, nafshartous, Nick 
Travers,
>>
>> nixsticks, Paolo Patierno, parafiend, Patrik Erdes, Radai Rosenblatt,
>>
>> Rajini Sivaram, Randall Hauch, ro7m, Robert Yokota, Roman Khlebnov,
>>
>> Ron Dagostino, Sandor Murakozi, Sasaki Toru, Sean Glover,
>>
>> Sebastian Bauersfeld, Siva Santhalingam, Stanislav Kozlovski, Stephane
>> Maarek,
>>
>> Stuart Perks, Surabhi Dixit, Sönke Liebau, taekyung, tedyu, Thomas 
Leplus,
>>
>> UVN, Vahid Hashemian, Valentino Proietti, Viktor Somogyi, Vitaly 
Pushkar,
>>
>> Wladimir Schmidt, wushujames, Xavier Léauté, xin, yaphet,
>>
>> Yaswanth Kumar, ying-zheng, Yu
>>
>>
>>
>>
>>
>>
>>
>> We welcome your help and feedback. For more information on how to
>>
>> report problems, and to get involved, visit the project website at
>>
>> 
https://kafka.apache.org/

>>
>>
>>
>>
>>
>> Thank you!
>>
>>
>>
>>
>>
>> Regards,
>>
>>
>>
>> Rajini
>>

Re: [ANNOUNCE] Apache Kafka 2.0.0 Released

[Vahid S Hashemian <va...@us.ibm.com>]

Such a good news on a Monday morning ...

Thank you Rajini for driving the release!

--Vahid




From:   Mickael Maison <mi...@gmail.com>
To:     Users <us...@kafka.apache.org>
Cc:     dev <de...@kafka.apache.org>, announce@apache.org, kafka-clients 
<ka...@googlegroups.com>
Date:   07/30/2018 04:37 AM
Subject:        Re: [ANNOUNCE] Apache Kafka 2.0.0 Released



Great news! Thanks for running the release

On Mon, Jul 30, 2018 at 12:20 PM, Manikumar <ma...@gmail.com> 
wrote:
> Thanks for driving the release!
>
>
>
> On Mon, Jul 30, 2018 at 3:55 PM Rajini Sivaram <rs...@apache.org> 
wrote:
>
>> The Apache Kafka community is pleased to announce the release for
>>
>> Apache Kafka 2.0.0.
>>
>>
>>
>>
>>
>> This is a major release and includes significant new features from
>>
>> 40 KIPs. It contains fixes and improvements from 246 JIRAs, including
>>
>> a few critical bugs. Here is a summary of some notable changes:
>>
>> ** KIP-290 adds support for prefixed ACLs, simplifying access control
>> management in large secure deployments. Bulk access to topics,
>> consumer groups or transactional ids with a prefix can now be granted
>> using a single rule. Access control for topic creation has also been
>> improved to enable access to be granted to create specific topics or
>> topics with a prefix.
>>
>> ** KIP-255 adds a framework for authenticating to Kafka brokers using
>> OAuth2 bearer tokens. The SASL/OAUTHBEARER implementation is
>> customizable using callbacks for token retrieval and validation.
>>
>> **Host name verification is now enabled by default for SSL connections
>> to ensure that the default SSL configuration is not susceptible to
>> man-in-the middle attacks. You can disable this verification for
>> deployments where validation is performed using other mechanisms.
>>
>> ** You can now dynamically update SSL trust stores without broker 
restart.
>> You can also configure security for broker listeners in ZooKeeper 
before
>> starting brokers, including SSL key store and trust store passwords and
>> JAAS configuration for SASL. With this new feature, you can store 
sensitive
>> password configs in encrypted form in ZooKeeper rather than in 
cleartext
>> in the broker properties file.
>>
>> ** The replication protocol has been improved to avoid log divergence
>> between leader and follower during fast leader failover. We have also
>> improved resilience of brokers by reducing the memory footprint of
>> message down-conversions. By using message chunking, both memory
>> usage and memory reference time have been reduced to avoid
>> OutOfMemory errors in brokers.
>>
>> ** Kafka clients are now notified of throttling before any throttling 
is
>> applied
>> when quotas are enabled. This enables clients to distinguish between
>> network errors and large throttle times when quotas are exceeded.
>>
>> ** We have added a configuration option for Kafka consumer to avoid
>> indefinite blocking in the consumer.
>>
>> ** We have dropped support for Java 7 and removed the previously
>> deprecated Scala producer and consumer.
>>
>> ** Kafka Connect includes a number of improvements and features.
>> KIP-298 enables you to control how errors in connectors, 
transformations
>> and converters are handled by enabling automatic retries and 
controlling
>> the
>> number of errors that are tolerated before the connector is stopped. 
More
>> contextual information can be included in the logs to help diagnose
>> problems
>> and problematic messages consumed by sink connectors can be sent to a
>> dead letter queue rather than forcing the connector to stop.
>>
>> ** KIP-297 adds a new extension point to move secrets out of connector
>> configurations and integrate with any external key management system.
>> The placeholders in connector configurations are only resolved before
>> sending the configuration to the connector, ensuring that secrets are
>> stored
>> and managed securely in your preferred key management system and
>> not exposed over the REST APIs or in log files.
>>
>> ** We have added a thin Scala wrapper API for our Kafka Streams DSL,
>> which provides better type inference and better type safety during 
compile
>> time. Scala users can have less boilerplate in their code, notably
>> regarding
>> Serdes with new implicit Serdes.
>>
>> ** Message headers are now supported in the Kafka Streams Processor 
API,
>> allowing users to add and manipulate headers read from the source 
topics
>> and propagate them to the sink topics.
>>
>> ** Windowed aggregations performance in Kafka Streams has been largely
>> improved (sometimes by an order of magnitude) thanks to the new
>> single-key-fetch API.
>>
>> ** We have further improved unit testibility of Kafka Streams with the
>> kafka-streams-testutil artifact.
>>
>>
>>
>>
>>
>> All of the changes in this release can be found in the release notes:
>>
>> 
https://www.apache.org/dist/kafka/2.0.0/RELEASE_NOTES.html

>>
>>
>>
>>
>>
>> You can download the source and binary release (Scala 2.11 and Scala 
2.12)
>> from:
>>
>> 
https://kafka.apache.org/downloads#2.0.0

>> <
https://kafka.apache.org/downloads#2.0.0
>
>>
>>
>>
>>
>> 
---------------------------------------------------------------------------------------------------
>>
>>
>>
>>
>>
>> Apache Kafka is a distributed streaming platform with four core APIs:
>>
>>
>>
>> ** The Producer API allows an application to publish a stream records 
to
>>
>> one or more Kafka topics.
>>
>>
>>
>> ** The Consumer API allows an application to subscribe to one or more
>>
>> topics and process the stream of records produced to them.
>>
>>
>>
>> ** The Streams API allows an application to act as a stream processor,
>>
>> consuming an input stream from one or more topics and producing an
>>
>> output stream to one or more output topics, effectively transforming 
the
>>
>> input streams to output streams.
>>
>>
>>
>> ** The Connector API allows building and running reusable producers or
>>
>> consumers that connect Kafka topics to existing applications or data
>>
>> systems. For example, a connector to a relational database might
>>
>> capture every change to a table.
>>
>>
>>
>>
>>
>> With these APIs, Kafka can be used for two broad classes of 
application:
>>
>>
>>
>> ** Building real-time streaming data pipelines that reliably get data
>>
>> between systems or applications.
>>
>>
>>
>> ** Building real-time streaming applications that transform or react
>>
>> to the streams of data.
>>
>>
>>
>>
>>
>>
>>
>> Apache Kafka is in use at large and small companies worldwide, 
including
>>
>> Capital One, Goldman Sachs, ING, LinkedIn, Netflix, Pinterest, 
Rabobank,
>>
>> Target, The New York Times, Uber, Yelp, and Zalando, among others.
>>
>>
>>
>>
>>
>>
>>
>> A big thank you for the following 131 contributors to this release!
>>
>>
>>
>> Adem Efe Gencer, Alex D, Alex Dunayevsky, Allen Wang, Andras Beni,
>>
>> Andy Bryant, Andy Coates, Anna Povzner, Arjun Satish, asutosh936,
>>
>> Attila Sasvari, bartdevylder, Benedict Jin, Bill Bejeck, Blake Miller,
>>
>> Boyang Chen, cburroughs, Chia-Ping Tsai, Chris Egerton, Colin P. 
Mccabe,
>>
>> Colin Patrick McCabe, ConcurrencyPractitioner, Damian Guy, dan norwood,
>>
>> Daniel Shuy, Daniel Wojda, Dark, David Glasser, Debasish Ghosh, 
Detharon,
>>
>> Dhruvil Shah, Dmitry Minkovsky, Dong Lin, Edoardo Comar, emmanuel 
Harel,
>>
>> Eugene Sevastyanov, Ewen Cheslack-Postava, Fedor Bobin, 
fedosov-alexander,
>>
>> Filipe Agapito, Florian Hussonnois, fredfp, Gilles Degols, gitlw, 
Gitomain,
>>
>> Guangxian, Gunju Ko, Gunnar Morling, Guozhang Wang, hmcl, huxi, huxihx,
>>
>> Igor Kostiakov, Ismael Juma, Jacek Laskowski, Jagadesh Adireddi,
>>
>> Jarek Rudzinski, Jason Gustafson, Jeff Klukas, Jeremy Custenborder,
>>
>> Jiangjie (Becket) Qin, Jiangjie Qin, JieFang.He, Jimin Hsieh, Joan 
Goyeau,
>>
>> Joel Hamill, John Roesler, Jon Lee, Jorge Quilcate Otoya, Jun Rao,
>>
>> Kamal C, khairy, Koen De Groote, Konstantine Karantasis, Lee Dongjin,
>>
>> Liju John, Liquan Pei, lisa2lisa, Lucas Wang, Magesh Nandakumar,
>>
>> Magnus Edenhill, Magnus Reftel, Manikumar Reddy, Manikumar Reddy O,
>>
>> manjuapu, Mats Julian Olsen, Matthias J. Sax, Max Zheng, maytals,
>>
>> Michael Arndt, Michael G. Noll, Mickael Maison, nafshartous, Nick 
Travers,
>>
>> nixsticks, Paolo Patierno, parafiend, Patrik Erdes, Radai Rosenblatt,
>>
>> Rajini Sivaram, Randall Hauch, ro7m, Robert Yokota, Roman Khlebnov,
>>
>> Ron Dagostino, Sandor Murakozi, Sasaki Toru, Sean Glover,
>>
>> Sebastian Bauersfeld, Siva Santhalingam, Stanislav Kozlovski, Stephane
>> Maarek,
>>
>> Stuart Perks, Surabhi Dixit, Sönke Liebau, taekyung, tedyu, Thomas 
Leplus,
>>
>> UVN, Vahid Hashemian, Valentino Proietti, Viktor Somogyi, Vitaly 
Pushkar,
>>
>> Wladimir Schmidt, wushujames, Xavier Léauté, xin, yaphet,
>>
>> Yaswanth Kumar, ying-zheng, Yu
>>
>>
>>
>>
>>
>>
>>
>> We welcome your help and feedback. For more information on how to
>>
>> report problems, and to get involved, visit the project website at
>>
>> 
https://kafka.apache.org/

>>
>>
>>
>>
>>
>> Thank you!
>>
>>
>>
>>
>>
>> Regards,
>>
>>
>>
>> Rajini
>>

Re: [ANNOUNCE] Apache Kafka 2.0.0 Released

[Mickael Maison <mi...@gmail.com>]

Great news! Thanks for running the release

On Mon, Jul 30, 2018 at 12:20 PM, Manikumar <ma...@gmail.com> wrote:
> Thanks for driving the release!
>
>
>
> On Mon, Jul 30, 2018 at 3:55 PM Rajini Sivaram <rs...@apache.org> wrote:
>
>> The Apache Kafka community is pleased to announce the release for
>>
>> Apache Kafka 2.0.0.
>>
>>
>>
>>
>>
>> This is a major release and includes significant new features from
>>
>> 40 KIPs. It contains fixes and improvements from 246 JIRAs, including
>>
>> a few critical bugs. Here is a summary of some notable changes:
>>
>> ** KIP-290 adds support for prefixed ACLs, simplifying access control
>> management in large secure deployments. Bulk access to topics,
>> consumer groups or transactional ids with a prefix can now be granted
>> using a single rule. Access control for topic creation has also been
>> improved to enable access to be granted to create specific topics or
>> topics with a prefix.
>>
>> ** KIP-255 adds a framework for authenticating to Kafka brokers using
>> OAuth2 bearer tokens. The SASL/OAUTHBEARER implementation is
>> customizable using callbacks for token retrieval and validation.
>>
>> **Host name verification is now enabled by default for SSL connections
>> to ensure that the default SSL configuration is not susceptible to
>> man-in-the middle attacks. You can disable this verification for
>> deployments where validation is performed using other mechanisms.
>>
>> ** You can now dynamically update SSL trust stores without broker restart.
>> You can also configure security for broker listeners in ZooKeeper before
>> starting brokers, including SSL key store and trust store passwords and
>> JAAS configuration for SASL. With this new feature, you can store sensitive
>> password configs in encrypted form in ZooKeeper rather than in cleartext
>> in the broker properties file.
>>
>> ** The replication protocol has been improved to avoid log divergence
>> between leader and follower during fast leader failover. We have also
>> improved resilience of brokers by reducing the memory footprint of
>> message down-conversions. By using message chunking, both memory
>> usage and memory reference time have been reduced to avoid
>> OutOfMemory errors in brokers.
>>
>> ** Kafka clients are now notified of throttling before any throttling is
>> applied
>> when quotas are enabled. This enables clients to distinguish between
>> network errors and large throttle times when quotas are exceeded.
>>
>> ** We have added a configuration option for Kafka consumer to avoid
>> indefinite blocking in the consumer.
>>
>> ** We have dropped support for Java 7 and removed the previously
>> deprecated Scala producer and consumer.
>>
>> ** Kafka Connect includes a number of improvements and features.
>> KIP-298 enables you to control how errors in connectors, transformations
>> and converters are handled by enabling automatic retries and controlling
>> the
>> number of errors that are tolerated before the connector is stopped. More
>> contextual information can be included in the logs to help diagnose
>> problems
>> and problematic messages consumed by sink connectors can be sent to a
>> dead letter queue rather than forcing the connector to stop.
>>
>> ** KIP-297 adds a new extension point to move secrets out of connector
>> configurations and integrate with any external key management system.
>> The placeholders in connector configurations are only resolved before
>> sending the configuration to the connector, ensuring that secrets are
>> stored
>> and managed securely in your preferred key management system and
>> not exposed over the REST APIs or in log files.
>>
>> ** We have added a thin Scala wrapper API for our Kafka Streams DSL,
>> which provides better type inference and better type safety during compile
>> time. Scala users can have less boilerplate in their code, notably
>> regarding
>> Serdes with new implicit Serdes.
>>
>> ** Message headers are now supported in the Kafka Streams Processor API,
>> allowing users to add and manipulate headers read from the source topics
>> and propagate them to the sink topics.
>>
>> ** Windowed aggregations performance in Kafka Streams has been largely
>> improved (sometimes by an order of magnitude) thanks to the new
>> single-key-fetch API.
>>
>> ** We have further improved unit testibility of Kafka Streams with the
>> kafka-streams-testutil artifact.
>>
>>
>>
>>
>>
>> All of the changes in this release can be found in the release notes:
>>
>> https://www.apache.org/dist/kafka/2.0.0/RELEASE_NOTES.html
>>
>>
>>
>>
>>
>> You can download the source and binary release (Scala 2.11 and Scala 2.12)
>> from:
>>
>> https://kafka.apache.org/downloads#2.0.0
>> <https://kafka.apache.org/downloads#2.0.0>
>>
>>
>>
>>
>> ---------------------------------------------------------------------------------------------------
>>
>>
>>
>>
>>
>> Apache Kafka is a distributed streaming platform with four core APIs:
>>
>>
>>
>> ** The Producer API allows an application to publish a stream records to
>>
>> one or more Kafka topics.
>>
>>
>>
>> ** The Consumer API allows an application to subscribe to one or more
>>
>> topics and process the stream of records produced to them.
>>
>>
>>
>> ** The Streams API allows an application to act as a stream processor,
>>
>> consuming an input stream from one or more topics and producing an
>>
>> output stream to one or more output topics, effectively transforming the
>>
>> input streams to output streams.
>>
>>
>>
>> ** The Connector API allows building and running reusable producers or
>>
>> consumers that connect Kafka topics to existing applications or data
>>
>> systems. For example, a connector to a relational database might
>>
>> capture every change to a table.
>>
>>
>>
>>
>>
>> With these APIs, Kafka can be used for two broad classes of application:
>>
>>
>>
>> ** Building real-time streaming data pipelines that reliably get data
>>
>> between systems or applications.
>>
>>
>>
>> ** Building real-time streaming applications that transform or react
>>
>> to the streams of data.
>>
>>
>>
>>
>>
>>
>>
>> Apache Kafka is in use at large and small companies worldwide, including
>>
>> Capital One, Goldman Sachs, ING, LinkedIn, Netflix, Pinterest, Rabobank,
>>
>> Target, The New York Times, Uber, Yelp, and Zalando, among others.
>>
>>
>>
>>
>>
>>
>>
>> A big thank you for the following 131 contributors to this release!
>>
>>
>>
>> Adem Efe Gencer, Alex D, Alex Dunayevsky, Allen Wang, Andras Beni,
>>
>> Andy Bryant, Andy Coates, Anna Povzner, Arjun Satish, asutosh936,
>>
>> Attila Sasvari, bartdevylder, Benedict Jin, Bill Bejeck, Blake Miller,
>>
>> Boyang Chen, cburroughs, Chia-Ping Tsai, Chris Egerton, Colin P. Mccabe,
>>
>> Colin Patrick McCabe, ConcurrencyPractitioner, Damian Guy, dan norwood,
>>
>> Daniel Shuy, Daniel Wojda, Dark, David Glasser, Debasish Ghosh, Detharon,
>>
>> Dhruvil Shah, Dmitry Minkovsky, Dong Lin, Edoardo Comar, emmanuel Harel,
>>
>> Eugene Sevastyanov, Ewen Cheslack-Postava, Fedor Bobin, fedosov-alexander,
>>
>> Filipe Agapito, Florian Hussonnois, fredfp, Gilles Degols, gitlw, Gitomain,
>>
>> Guangxian, Gunju Ko, Gunnar Morling, Guozhang Wang, hmcl, huxi, huxihx,
>>
>> Igor Kostiakov, Ismael Juma, Jacek Laskowski, Jagadesh Adireddi,
>>
>> Jarek Rudzinski, Jason Gustafson, Jeff Klukas, Jeremy Custenborder,
>>
>> Jiangjie (Becket) Qin, Jiangjie Qin, JieFang.He, Jimin Hsieh, Joan Goyeau,
>>
>> Joel Hamill, John Roesler, Jon Lee, Jorge Quilcate Otoya, Jun Rao,
>>
>> Kamal C, khairy, Koen De Groote, Konstantine Karantasis, Lee Dongjin,
>>
>> Liju John, Liquan Pei, lisa2lisa, Lucas Wang, Magesh Nandakumar,
>>
>> Magnus Edenhill, Magnus Reftel, Manikumar Reddy, Manikumar Reddy O,
>>
>> manjuapu, Mats Julian Olsen, Matthias J. Sax, Max Zheng, maytals,
>>
>> Michael Arndt, Michael G. Noll, Mickael Maison, nafshartous, Nick Travers,
>>
>> nixsticks, Paolo Patierno, parafiend, Patrik Erdes, Radai Rosenblatt,
>>
>> Rajini Sivaram, Randall Hauch, ro7m, Robert Yokota, Roman Khlebnov,
>>
>> Ron Dagostino, Sandor Murakozi, Sasaki Toru, Sean Glover,
>>
>> Sebastian Bauersfeld, Siva Santhalingam, Stanislav Kozlovski, Stephane
>> Maarek,
>>
>> Stuart Perks, Surabhi Dixit, Sönke Liebau, taekyung, tedyu, Thomas Leplus,
>>
>> UVN, Vahid Hashemian, Valentino Proietti, Viktor Somogyi, Vitaly Pushkar,
>>
>> Wladimir Schmidt, wushujames, Xavier Léauté, xin, yaphet,
>>
>> Yaswanth Kumar, ying-zheng, Yu
>>
>>
>>
>>
>>
>>
>>
>> We welcome your help and feedback. For more information on how to
>>
>> report problems, and to get involved, visit the project website at
>>
>> https://kafka.apache.org/
>>
>>
>>
>>
>>
>> Thank you!
>>
>>
>>
>>
>>
>> Regards,
>>
>>
>>
>> Rajini
>>

Re: [ANNOUNCE] Apache Kafka 2.0.0 Released

[Mickael Maison <mi...@gmail.com>]

Great news! Thanks for running the release

On Mon, Jul 30, 2018 at 12:20 PM, Manikumar <ma...@gmail.com> wrote:
> Thanks for driving the release!
>
>
>
> On Mon, Jul 30, 2018 at 3:55 PM Rajini Sivaram <rs...@apache.org> wrote:
>
>> The Apache Kafka community is pleased to announce the release for
>>
>> Apache Kafka 2.0.0.
>>
>>
>>
>>
>>
>> This is a major release and includes significant new features from
>>
>> 40 KIPs. It contains fixes and improvements from 246 JIRAs, including
>>
>> a few critical bugs. Here is a summary of some notable changes:
>>
>> ** KIP-290 adds support for prefixed ACLs, simplifying access control
>> management in large secure deployments. Bulk access to topics,
>> consumer groups or transactional ids with a prefix can now be granted
>> using a single rule. Access control for topic creation has also been
>> improved to enable access to be granted to create specific topics or
>> topics with a prefix.
>>
>> ** KIP-255 adds a framework for authenticating to Kafka brokers using
>> OAuth2 bearer tokens. The SASL/OAUTHBEARER implementation is
>> customizable using callbacks for token retrieval and validation.
>>
>> **Host name verification is now enabled by default for SSL connections
>> to ensure that the default SSL configuration is not susceptible to
>> man-in-the middle attacks. You can disable this verification for
>> deployments where validation is performed using other mechanisms.
>>
>> ** You can now dynamically update SSL trust stores without broker restart.
>> You can also configure security for broker listeners in ZooKeeper before
>> starting brokers, including SSL key store and trust store passwords and
>> JAAS configuration for SASL. With this new feature, you can store sensitive
>> password configs in encrypted form in ZooKeeper rather than in cleartext
>> in the broker properties file.
>>
>> ** The replication protocol has been improved to avoid log divergence
>> between leader and follower during fast leader failover. We have also
>> improved resilience of brokers by reducing the memory footprint of
>> message down-conversions. By using message chunking, both memory
>> usage and memory reference time have been reduced to avoid
>> OutOfMemory errors in brokers.
>>
>> ** Kafka clients are now notified of throttling before any throttling is
>> applied
>> when quotas are enabled. This enables clients to distinguish between
>> network errors and large throttle times when quotas are exceeded.
>>
>> ** We have added a configuration option for Kafka consumer to avoid
>> indefinite blocking in the consumer.
>>
>> ** We have dropped support for Java 7 and removed the previously
>> deprecated Scala producer and consumer.
>>
>> ** Kafka Connect includes a number of improvements and features.
>> KIP-298 enables you to control how errors in connectors, transformations
>> and converters are handled by enabling automatic retries and controlling
>> the
>> number of errors that are tolerated before the connector is stopped. More
>> contextual information can be included in the logs to help diagnose
>> problems
>> and problematic messages consumed by sink connectors can be sent to a
>> dead letter queue rather than forcing the connector to stop.
>>
>> ** KIP-297 adds a new extension point to move secrets out of connector
>> configurations and integrate with any external key management system.
>> The placeholders in connector configurations are only resolved before
>> sending the configuration to the connector, ensuring that secrets are
>> stored
>> and managed securely in your preferred key management system and
>> not exposed over the REST APIs or in log files.
>>
>> ** We have added a thin Scala wrapper API for our Kafka Streams DSL,
>> which provides better type inference and better type safety during compile
>> time. Scala users can have less boilerplate in their code, notably
>> regarding
>> Serdes with new implicit Serdes.
>>
>> ** Message headers are now supported in the Kafka Streams Processor API,
>> allowing users to add and manipulate headers read from the source topics
>> and propagate them to the sink topics.
>>
>> ** Windowed aggregations performance in Kafka Streams has been largely
>> improved (sometimes by an order of magnitude) thanks to the new
>> single-key-fetch API.
>>
>> ** We have further improved unit testibility of Kafka Streams with the
>> kafka-streams-testutil artifact.
>>
>>
>>
>>
>>
>> All of the changes in this release can be found in the release notes:
>>
>> https://www.apache.org/dist/kafka/2.0.0/RELEASE_NOTES.html
>>
>>
>>
>>
>>
>> You can download the source and binary release (Scala 2.11 and Scala 2.12)
>> from:
>>
>> https://kafka.apache.org/downloads#2.0.0
>> <https://kafka.apache.org/downloads#2.0.0>
>>
>>
>>
>>
>> ---------------------------------------------------------------------------------------------------
>>
>>
>>
>>
>>
>> Apache Kafka is a distributed streaming platform with four core APIs:
>>
>>
>>
>> ** The Producer API allows an application to publish a stream records to
>>
>> one or more Kafka topics.
>>
>>
>>
>> ** The Consumer API allows an application to subscribe to one or more
>>
>> topics and process the stream of records produced to them.
>>
>>
>>
>> ** The Streams API allows an application to act as a stream processor,
>>
>> consuming an input stream from one or more topics and producing an
>>
>> output stream to one or more output topics, effectively transforming the
>>
>> input streams to output streams.
>>
>>
>>
>> ** The Connector API allows building and running reusable producers or
>>
>> consumers that connect Kafka topics to existing applications or data
>>
>> systems. For example, a connector to a relational database might
>>
>> capture every change to a table.
>>
>>
>>
>>
>>
>> With these APIs, Kafka can be used for two broad classes of application:
>>
>>
>>
>> ** Building real-time streaming data pipelines that reliably get data
>>
>> between systems or applications.
>>
>>
>>
>> ** Building real-time streaming applications that transform or react
>>
>> to the streams of data.
>>
>>
>>
>>
>>
>>
>>
>> Apache Kafka is in use at large and small companies worldwide, including
>>
>> Capital One, Goldman Sachs, ING, LinkedIn, Netflix, Pinterest, Rabobank,
>>
>> Target, The New York Times, Uber, Yelp, and Zalando, among others.
>>
>>
>>
>>
>>
>>
>>
>> A big thank you for the following 131 contributors to this release!
>>
>>
>>
>> Adem Efe Gencer, Alex D, Alex Dunayevsky, Allen Wang, Andras Beni,
>>
>> Andy Bryant, Andy Coates, Anna Povzner, Arjun Satish, asutosh936,
>>
>> Attila Sasvari, bartdevylder, Benedict Jin, Bill Bejeck, Blake Miller,
>>
>> Boyang Chen, cburroughs, Chia-Ping Tsai, Chris Egerton, Colin P. Mccabe,
>>
>> Colin Patrick McCabe, ConcurrencyPractitioner, Damian Guy, dan norwood,
>>
>> Daniel Shuy, Daniel Wojda, Dark, David Glasser, Debasish Ghosh, Detharon,
>>
>> Dhruvil Shah, Dmitry Minkovsky, Dong Lin, Edoardo Comar, emmanuel Harel,
>>
>> Eugene Sevastyanov, Ewen Cheslack-Postava, Fedor Bobin, fedosov-alexander,
>>
>> Filipe Agapito, Florian Hussonnois, fredfp, Gilles Degols, gitlw, Gitomain,
>>
>> Guangxian, Gunju Ko, Gunnar Morling, Guozhang Wang, hmcl, huxi, huxihx,
>>
>> Igor Kostiakov, Ismael Juma, Jacek Laskowski, Jagadesh Adireddi,
>>
>> Jarek Rudzinski, Jason Gustafson, Jeff Klukas, Jeremy Custenborder,
>>
>> Jiangjie (Becket) Qin, Jiangjie Qin, JieFang.He, Jimin Hsieh, Joan Goyeau,
>>
>> Joel Hamill, John Roesler, Jon Lee, Jorge Quilcate Otoya, Jun Rao,
>>
>> Kamal C, khairy, Koen De Groote, Konstantine Karantasis, Lee Dongjin,
>>
>> Liju John, Liquan Pei, lisa2lisa, Lucas Wang, Magesh Nandakumar,
>>
>> Magnus Edenhill, Magnus Reftel, Manikumar Reddy, Manikumar Reddy O,
>>
>> manjuapu, Mats Julian Olsen, Matthias J. Sax, Max Zheng, maytals,
>>
>> Michael Arndt, Michael G. Noll, Mickael Maison, nafshartous, Nick Travers,
>>
>> nixsticks, Paolo Patierno, parafiend, Patrik Erdes, Radai Rosenblatt,
>>
>> Rajini Sivaram, Randall Hauch, ro7m, Robert Yokota, Roman Khlebnov,
>>
>> Ron Dagostino, Sandor Murakozi, Sasaki Toru, Sean Glover,
>>
>> Sebastian Bauersfeld, Siva Santhalingam, Stanislav Kozlovski, Stephane
>> Maarek,
>>
>> Stuart Perks, Surabhi Dixit, Sönke Liebau, taekyung, tedyu, Thomas Leplus,
>>
>> UVN, Vahid Hashemian, Valentino Proietti, Viktor Somogyi, Vitaly Pushkar,
>>
>> Wladimir Schmidt, wushujames, Xavier Léauté, xin, yaphet,
>>
>> Yaswanth Kumar, ying-zheng, Yu
>>
>>
>>
>>
>>
>>
>>
>> We welcome your help and feedback. For more information on how to
>>
>> report problems, and to get involved, visit the project website at
>>
>> https://kafka.apache.org/
>>
>>
>>
>>
>>
>> Thank you!
>>
>>
>>
>>
>>
>> Regards,
>>
>>
>>
>> Rajini
>>

Re: [ANNOUNCE] Apache Kafka 2.0.0 Released

[Manikumar <ma...@gmail.com>]

Thanks for driving the release!



On Mon, Jul 30, 2018 at 3:55 PM Rajini Sivaram <rs...@apache.org> wrote:

> The Apache Kafka community is pleased to announce the release for
>
> Apache Kafka 2.0.0.
>
>
>
>
>
> This is a major release and includes significant new features from
>
> 40 KIPs. It contains fixes and improvements from 246 JIRAs, including
>
> a few critical bugs. Here is a summary of some notable changes:
>
> ** KIP-290 adds support for prefixed ACLs, simplifying access control
> management in large secure deployments. Bulk access to topics,
> consumer groups or transactional ids with a prefix can now be granted
> using a single rule. Access control for topic creation has also been
> improved to enable access to be granted to create specific topics or
> topics with a prefix.
>
> ** KIP-255 adds a framework for authenticating to Kafka brokers using
> OAuth2 bearer tokens. The SASL/OAUTHBEARER implementation is
> customizable using callbacks for token retrieval and validation.
>
> **Host name verification is now enabled by default for SSL connections
> to ensure that the default SSL configuration is not susceptible to
> man-in-the middle attacks. You can disable this verification for
> deployments where validation is performed using other mechanisms.
>
> ** You can now dynamically update SSL trust stores without broker restart.
> You can also configure security for broker listeners in ZooKeeper before
> starting brokers, including SSL key store and trust store passwords and
> JAAS configuration for SASL. With this new feature, you can store sensitive
> password configs in encrypted form in ZooKeeper rather than in cleartext
> in the broker properties file.
>
> ** The replication protocol has been improved to avoid log divergence
> between leader and follower during fast leader failover. We have also
> improved resilience of brokers by reducing the memory footprint of
> message down-conversions. By using message chunking, both memory
> usage and memory reference time have been reduced to avoid
> OutOfMemory errors in brokers.
>
> ** Kafka clients are now notified of throttling before any throttling is
> applied
> when quotas are enabled. This enables clients to distinguish between
> network errors and large throttle times when quotas are exceeded.
>
> ** We have added a configuration option for Kafka consumer to avoid
> indefinite blocking in the consumer.
>
> ** We have dropped support for Java 7 and removed the previously
> deprecated Scala producer and consumer.
>
> ** Kafka Connect includes a number of improvements and features.
> KIP-298 enables you to control how errors in connectors, transformations
> and converters are handled by enabling automatic retries and controlling
> the
> number of errors that are tolerated before the connector is stopped. More
> contextual information can be included in the logs to help diagnose
> problems
> and problematic messages consumed by sink connectors can be sent to a
> dead letter queue rather than forcing the connector to stop.
>
> ** KIP-297 adds a new extension point to move secrets out of connector
> configurations and integrate with any external key management system.
> The placeholders in connector configurations are only resolved before
> sending the configuration to the connector, ensuring that secrets are
> stored
> and managed securely in your preferred key management system and
> not exposed over the REST APIs or in log files.
>
> ** We have added a thin Scala wrapper API for our Kafka Streams DSL,
> which provides better type inference and better type safety during compile
> time. Scala users can have less boilerplate in their code, notably
> regarding
> Serdes with new implicit Serdes.
>
> ** Message headers are now supported in the Kafka Streams Processor API,
> allowing users to add and manipulate headers read from the source topics
> and propagate them to the sink topics.
>
> ** Windowed aggregations performance in Kafka Streams has been largely
> improved (sometimes by an order of magnitude) thanks to the new
> single-key-fetch API.
>
> ** We have further improved unit testibility of Kafka Streams with the
> kafka-streams-testutil artifact.
>
>
>
>
>
> All of the changes in this release can be found in the release notes:
>
> https://www.apache.org/dist/kafka/2.0.0/RELEASE_NOTES.html
>
>
>
>
>
> You can download the source and binary release (Scala 2.11 and Scala 2.12)
> from:
>
> https://kafka.apache.org/downloads#2.0.0
> <https://kafka.apache.org/downloads#2.0.0>
>
>
>
>
> ---------------------------------------------------------------------------------------------------
>
>
>
>
>
> Apache Kafka is a distributed streaming platform with four core APIs:
>
>
>
> ** The Producer API allows an application to publish a stream records to
>
> one or more Kafka topics.
>
>
>
> ** The Consumer API allows an application to subscribe to one or more
>
> topics and process the stream of records produced to them.
>
>
>
> ** The Streams API allows an application to act as a stream processor,
>
> consuming an input stream from one or more topics and producing an
>
> output stream to one or more output topics, effectively transforming the
>
> input streams to output streams.
>
>
>
> ** The Connector API allows building and running reusable producers or
>
> consumers that connect Kafka topics to existing applications or data
>
> systems. For example, a connector to a relational database might
>
> capture every change to a table.
>
>
>
>
>
> With these APIs, Kafka can be used for two broad classes of application:
>
>
>
> ** Building real-time streaming data pipelines that reliably get data
>
> between systems or applications.
>
>
>
> ** Building real-time streaming applications that transform or react
>
> to the streams of data.
>
>
>
>
>
>
>
> Apache Kafka is in use at large and small companies worldwide, including
>
> Capital One, Goldman Sachs, ING, LinkedIn, Netflix, Pinterest, Rabobank,
>
> Target, The New York Times, Uber, Yelp, and Zalando, among others.
>
>
>
>
>
>
>
> A big thank you for the following 131 contributors to this release!
>
>
>
> Adem Efe Gencer, Alex D, Alex Dunayevsky, Allen Wang, Andras Beni,
>
> Andy Bryant, Andy Coates, Anna Povzner, Arjun Satish, asutosh936,
>
> Attila Sasvari, bartdevylder, Benedict Jin, Bill Bejeck, Blake Miller,
>
> Boyang Chen, cburroughs, Chia-Ping Tsai, Chris Egerton, Colin P. Mccabe,
>
> Colin Patrick McCabe, ConcurrencyPractitioner, Damian Guy, dan norwood,
>
> Daniel Shuy, Daniel Wojda, Dark, David Glasser, Debasish Ghosh, Detharon,
>
> Dhruvil Shah, Dmitry Minkovsky, Dong Lin, Edoardo Comar, emmanuel Harel,
>
> Eugene Sevastyanov, Ewen Cheslack-Postava, Fedor Bobin, fedosov-alexander,
>
> Filipe Agapito, Florian Hussonnois, fredfp, Gilles Degols, gitlw, Gitomain,
>
> Guangxian, Gunju Ko, Gunnar Morling, Guozhang Wang, hmcl, huxi, huxihx,
>
> Igor Kostiakov, Ismael Juma, Jacek Laskowski, Jagadesh Adireddi,
>
> Jarek Rudzinski, Jason Gustafson, Jeff Klukas, Jeremy Custenborder,
>
> Jiangjie (Becket) Qin, Jiangjie Qin, JieFang.He, Jimin Hsieh, Joan Goyeau,
>
> Joel Hamill, John Roesler, Jon Lee, Jorge Quilcate Otoya, Jun Rao,
>
> Kamal C, khairy, Koen De Groote, Konstantine Karantasis, Lee Dongjin,
>
> Liju John, Liquan Pei, lisa2lisa, Lucas Wang, Magesh Nandakumar,
>
> Magnus Edenhill, Magnus Reftel, Manikumar Reddy, Manikumar Reddy O,
>
> manjuapu, Mats Julian Olsen, Matthias J. Sax, Max Zheng, maytals,
>
> Michael Arndt, Michael G. Noll, Mickael Maison, nafshartous, Nick Travers,
>
> nixsticks, Paolo Patierno, parafiend, Patrik Erdes, Radai Rosenblatt,
>
> Rajini Sivaram, Randall Hauch, ro7m, Robert Yokota, Roman Khlebnov,
>
> Ron Dagostino, Sandor Murakozi, Sasaki Toru, Sean Glover,
>
> Sebastian Bauersfeld, Siva Santhalingam, Stanislav Kozlovski, Stephane
> Maarek,
>
> Stuart Perks, Surabhi Dixit, Sönke Liebau, taekyung, tedyu, Thomas Leplus,
>
> UVN, Vahid Hashemian, Valentino Proietti, Viktor Somogyi, Vitaly Pushkar,
>
> Wladimir Schmidt, wushujames, Xavier Léauté, xin, yaphet,
>
> Yaswanth Kumar, ying-zheng, Yu
>
>
>
>
>
>
>
> We welcome your help and feedback. For more information on how to
>
> report problems, and to get involved, visit the project website at
>
> https://kafka.apache.org/
>
>
>
>
>
> Thank you!
>
>
>
>
>
> Regards,
>
>
>
> Rajini
>

Re: [ANNOUNCE] Apache Kafka 2.0.0 Released

[Manikumar <ma...@gmail.com>]

Thanks for driving the release!



On Mon, Jul 30, 2018 at 3:55 PM Rajini Sivaram <rs...@apache.org> wrote:

> The Apache Kafka community is pleased to announce the release for
>
> Apache Kafka 2.0.0.
>
>
>
>
>
> This is a major release and includes significant new features from
>
> 40 KIPs. It contains fixes and improvements from 246 JIRAs, including
>
> a few critical bugs. Here is a summary of some notable changes:
>
> ** KIP-290 adds support for prefixed ACLs, simplifying access control
> management in large secure deployments. Bulk access to topics,
> consumer groups or transactional ids with a prefix can now be granted
> using a single rule. Access control for topic creation has also been
> improved to enable access to be granted to create specific topics or
> topics with a prefix.
>
> ** KIP-255 adds a framework for authenticating to Kafka brokers using
> OAuth2 bearer tokens. The SASL/OAUTHBEARER implementation is
> customizable using callbacks for token retrieval and validation.
>
> **Host name verification is now enabled by default for SSL connections
> to ensure that the default SSL configuration is not susceptible to
> man-in-the middle attacks. You can disable this verification for
> deployments where validation is performed using other mechanisms.
>
> ** You can now dynamically update SSL trust stores without broker restart.
> You can also configure security for broker listeners in ZooKeeper before
> starting brokers, including SSL key store and trust store passwords and
> JAAS configuration for SASL. With this new feature, you can store sensitive
> password configs in encrypted form in ZooKeeper rather than in cleartext
> in the broker properties file.
>
> ** The replication protocol has been improved to avoid log divergence
> between leader and follower during fast leader failover. We have also
> improved resilience of brokers by reducing the memory footprint of
> message down-conversions. By using message chunking, both memory
> usage and memory reference time have been reduced to avoid
> OutOfMemory errors in brokers.
>
> ** Kafka clients are now notified of throttling before any throttling is
> applied
> when quotas are enabled. This enables clients to distinguish between
> network errors and large throttle times when quotas are exceeded.
>
> ** We have added a configuration option for Kafka consumer to avoid
> indefinite blocking in the consumer.
>
> ** We have dropped support for Java 7 and removed the previously
> deprecated Scala producer and consumer.
>
> ** Kafka Connect includes a number of improvements and features.
> KIP-298 enables you to control how errors in connectors, transformations
> and converters are handled by enabling automatic retries and controlling
> the
> number of errors that are tolerated before the connector is stopped. More
> contextual information can be included in the logs to help diagnose
> problems
> and problematic messages consumed by sink connectors can be sent to a
> dead letter queue rather than forcing the connector to stop.
>
> ** KIP-297 adds a new extension point to move secrets out of connector
> configurations and integrate with any external key management system.
> The placeholders in connector configurations are only resolved before
> sending the configuration to the connector, ensuring that secrets are
> stored
> and managed securely in your preferred key management system and
> not exposed over the REST APIs or in log files.
>
> ** We have added a thin Scala wrapper API for our Kafka Streams DSL,
> which provides better type inference and better type safety during compile
> time. Scala users can have less boilerplate in their code, notably
> regarding
> Serdes with new implicit Serdes.
>
> ** Message headers are now supported in the Kafka Streams Processor API,
> allowing users to add and manipulate headers read from the source topics
> and propagate them to the sink topics.
>
> ** Windowed aggregations performance in Kafka Streams has been largely
> improved (sometimes by an order of magnitude) thanks to the new
> single-key-fetch API.
>
> ** We have further improved unit testibility of Kafka Streams with the
> kafka-streams-testutil artifact.
>
>
>
>
>
> All of the changes in this release can be found in the release notes:
>
> https://www.apache.org/dist/kafka/2.0.0/RELEASE_NOTES.html
>
>
>
>
>
> You can download the source and binary release (Scala 2.11 and Scala 2.12)
> from:
>
> https://kafka.apache.org/downloads#2.0.0
> <https://kafka.apache.org/downloads#2.0.0>
>
>
>
>
> ---------------------------------------------------------------------------------------------------
>
>
>
>
>
> Apache Kafka is a distributed streaming platform with four core APIs:
>
>
>
> ** The Producer API allows an application to publish a stream records to
>
> one or more Kafka topics.
>
>
>
> ** The Consumer API allows an application to subscribe to one or more
>
> topics and process the stream of records produced to them.
>
>
>
> ** The Streams API allows an application to act as a stream processor,
>
> consuming an input stream from one or more topics and producing an
>
> output stream to one or more output topics, effectively transforming the
>
> input streams to output streams.
>
>
>
> ** The Connector API allows building and running reusable producers or
>
> consumers that connect Kafka topics to existing applications or data
>
> systems. For example, a connector to a relational database might
>
> capture every change to a table.
>
>
>
>
>
> With these APIs, Kafka can be used for two broad classes of application:
>
>
>
> ** Building real-time streaming data pipelines that reliably get data
>
> between systems or applications.
>
>
>
> ** Building real-time streaming applications that transform or react
>
> to the streams of data.
>
>
>
>
>
>
>
> Apache Kafka is in use at large and small companies worldwide, including
>
> Capital One, Goldman Sachs, ING, LinkedIn, Netflix, Pinterest, Rabobank,
>
> Target, The New York Times, Uber, Yelp, and Zalando, among others.
>
>
>
>
>
>
>
> A big thank you for the following 131 contributors to this release!
>
>
>
> Adem Efe Gencer, Alex D, Alex Dunayevsky, Allen Wang, Andras Beni,
>
> Andy Bryant, Andy Coates, Anna Povzner, Arjun Satish, asutosh936,
>
> Attila Sasvari, bartdevylder, Benedict Jin, Bill Bejeck, Blake Miller,
>
> Boyang Chen, cburroughs, Chia-Ping Tsai, Chris Egerton, Colin P. Mccabe,
>
> Colin Patrick McCabe, ConcurrencyPractitioner, Damian Guy, dan norwood,
>
> Daniel Shuy, Daniel Wojda, Dark, David Glasser, Debasish Ghosh, Detharon,
>
> Dhruvil Shah, Dmitry Minkovsky, Dong Lin, Edoardo Comar, emmanuel Harel,
>
> Eugene Sevastyanov, Ewen Cheslack-Postava, Fedor Bobin, fedosov-alexander,
>
> Filipe Agapito, Florian Hussonnois, fredfp, Gilles Degols, gitlw, Gitomain,
>
> Guangxian, Gunju Ko, Gunnar Morling, Guozhang Wang, hmcl, huxi, huxihx,
>
> Igor Kostiakov, Ismael Juma, Jacek Laskowski, Jagadesh Adireddi,
>
> Jarek Rudzinski, Jason Gustafson, Jeff Klukas, Jeremy Custenborder,
>
> Jiangjie (Becket) Qin, Jiangjie Qin, JieFang.He, Jimin Hsieh, Joan Goyeau,
>
> Joel Hamill, John Roesler, Jon Lee, Jorge Quilcate Otoya, Jun Rao,
>
> Kamal C, khairy, Koen De Groote, Konstantine Karantasis, Lee Dongjin,
>
> Liju John, Liquan Pei, lisa2lisa, Lucas Wang, Magesh Nandakumar,
>
> Magnus Edenhill, Magnus Reftel, Manikumar Reddy, Manikumar Reddy O,
>
> manjuapu, Mats Julian Olsen, Matthias J. Sax, Max Zheng, maytals,
>
> Michael Arndt, Michael G. Noll, Mickael Maison, nafshartous, Nick Travers,
>
> nixsticks, Paolo Patierno, parafiend, Patrik Erdes, Radai Rosenblatt,
>
> Rajini Sivaram, Randall Hauch, ro7m, Robert Yokota, Roman Khlebnov,
>
> Ron Dagostino, Sandor Murakozi, Sasaki Toru, Sean Glover,
>
> Sebastian Bauersfeld, Siva Santhalingam, Stanislav Kozlovski, Stephane
> Maarek,
>
> Stuart Perks, Surabhi Dixit, Sönke Liebau, taekyung, tedyu, Thomas Leplus,
>
> UVN, Vahid Hashemian, Valentino Proietti, Viktor Somogyi, Vitaly Pushkar,
>
> Wladimir Schmidt, wushujames, Xavier Léauté, xin, yaphet,
>
> Yaswanth Kumar, ying-zheng, Yu
>
>
>
>
>
>
>
> We welcome your help and feedback. For more information on how to
>
> report problems, and to get involved, visit the project website at
>
> https://kafka.apache.org/
>
>
>
>
>
> Thank you!
>
>
>
>
>
> Regards,
>
>
>
> Rajini
>

Re: [ANNOUNCE] Apache Kafka 2.0.0 Released

[Jeff Widman <je...@jeffwidman.com>]

Congrats!

Is there any reason I don't see 2.0.0 listed on the docs page?

https://kafka.apache.org/documentation/

On Mon, Jul 30, 2018 at 12:54 PM, Stephane Maarek <
stephane@simplemachines.com.au> wrote:

> Congratulations !
>
> On Mon., 30 Jul. 2018, 11:51 am Sean Glover, <se...@lightbend.com>
> wrote:
>
> > Congrats to everyone involved.  Releasing "2.0.0" is a big achievement :)
> >
> > On Mon, Jul 30, 2018 at 12:35 PM Dongjin Lee <do...@apache.org> wrote:
> >
> > > Thank you for your great works! Thanks again for the commiters and all
> > the
> > > contributors!
> > >
> > > On Tue, Jul 31, 2018 at 1:05 AM Ismael Juma <is...@gmail.com> wrote:
> > >
> > > > Thanks to everyone who contributed to the release (including testing
> > and
> > > > bug reports)! And thank you Rajini for managing the release.
> > > >
> > > > Ismael
> > > >
> > > > On Mon, 30 Jul 2018, 03:25 Rajini Sivaram, <rs...@apache.org>
> > wrote:
> > > >
> > > > > The Apache Kafka community is pleased to announce the release for
> > > > >
> > > > > Apache Kafka 2.0.0.
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > > > This is a major release and includes significant new features from
> > > > >
> > > > > 40 KIPs. It contains fixes and improvements from 246 JIRAs,
> including
> > > > >
> > > > > a few critical bugs. Here is a summary of some notable changes:
> > > > >
> > > > > ** KIP-290 adds support for prefixed ACLs, simplifying access
> control
> > > > > management in large secure deployments. Bulk access to topics,
> > > > > consumer groups or transactional ids with a prefix can now be
> granted
> > > > > using a single rule. Access control for topic creation has also
> been
> > > > > improved to enable access to be granted to create specific topics
> or
> > > > > topics with a prefix.
> > > > >
> > > > > ** KIP-255 adds a framework for authenticating to Kafka brokers
> using
> > > > > OAuth2 bearer tokens. The SASL/OAUTHBEARER implementation is
> > > > > customizable using callbacks for token retrieval and validation.
> > > > >
> > > > > **Host name verification is now enabled by default for SSL
> > connections
> > > > > to ensure that the default SSL configuration is not susceptible to
> > > > > man-in-the middle attacks. You can disable this verification for
> > > > > deployments where validation is performed using other mechanisms.
> > > > >
> > > > > ** You can now dynamically update SSL trust stores without broker
> > > > restart.
> > > > > You can also configure security for broker listeners in ZooKeeper
> > > before
> > > > > starting brokers, including SSL key store and trust store passwords
> > and
> > > > > JAAS configuration for SASL. With this new feature, you can store
> > > > sensitive
> > > > > password configs in encrypted form in ZooKeeper rather than in
> > > cleartext
> > > > > in the broker properties file.
> > > > >
> > > > > ** The replication protocol has been improved to avoid log
> divergence
> > > > > between leader and follower during fast leader failover. We have
> also
> > > > > improved resilience of brokers by reducing the memory footprint of
> > > > > message down-conversions. By using message chunking, both memory
> > > > > usage and memory reference time have been reduced to avoid
> > > > > OutOfMemory errors in brokers.
> > > > >
> > > > > ** Kafka clients are now notified of throttling before any
> throttling
> > > is
> > > > > applied
> > > > > when quotas are enabled. This enables clients to distinguish
> between
> > > > > network errors and large throttle times when quotas are exceeded.
> > > > >
> > > > > ** We have added a configuration option for Kafka consumer to avoid
> > > > > indefinite blocking in the consumer.
> > > > >
> > > > > ** We have dropped support for Java 7 and removed the previously
> > > > > deprecated Scala producer and consumer.
> > > > >
> > > > > ** Kafka Connect includes a number of improvements and features.
> > > > > KIP-298 enables you to control how errors in connectors,
> > > transformations
> > > > > and converters are handled by enabling automatic retries and
> > > controlling
> > > > > the
> > > > > number of errors that are tolerated before the connector is
> stopped.
> > > More
> > > > > contextual information can be included in the logs to help diagnose
> > > > > problems
> > > > > and problematic messages consumed by sink connectors can be sent
> to a
> > > > > dead letter queue rather than forcing the connector to stop.
> > > > >
> > > > > ** KIP-297 adds a new extension point to move secrets out of
> > connector
> > > > > configurations and integrate with any external key management
> system.
> > > > > The placeholders in connector configurations are only resolved
> before
> > > > > sending the configuration to the connector, ensuring that secrets
> are
> > > > > stored
> > > > > and managed securely in your preferred key management system and
> > > > > not exposed over the REST APIs or in log files.
> > > > >
> > > > > ** We have added a thin Scala wrapper API for our Kafka Streams
> DSL,
> > > > > which provides better type inference and better type safety during
> > > > compile
> > > > > time. Scala users can have less boilerplate in their code, notably
> > > > > regarding
> > > > > Serdes with new implicit Serdes.
> > > > >
> > > > > ** Message headers are now supported in the Kafka Streams Processor
> > > API,
> > > > > allowing users to add and manipulate headers read from the source
> > > topics
> > > > > and propagate them to the sink topics.
> > > > >
> > > > > ** Windowed aggregations performance in Kafka Streams has been
> > largely
> > > > > improved (sometimes by an order of magnitude) thanks to the new
> > > > > single-key-fetch API.
> > > > >
> > > > > ** We have further improved unit testibility of Kafka Streams with
> > the
> > > > > kafka-streams-testutil artifact.
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > > > All of the changes in this release can be found in the release
> notes:
> > > > >
> > > > > https://www.apache.org/dist/kafka/2.0.0/RELEASE_NOTES.html
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > > > You can download the source and binary release (Scala 2.11 and
> Scala
> > > > 2.12)
> > > > > from:
> > > > >
> > > > > https://kafka.apache.org/downloads#2.0.0
> > > > > <https://kafka.apache.org/downloads#2.0.0>
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > >
> > >
> > ------------------------------------------------------------
> ---------------------------------------
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > > > Apache Kafka is a distributed streaming platform with four core
> APIs:
> > > > >
> > > > >
> > > > >
> > > > > ** The Producer API allows an application to publish a stream
> records
> > > to
> > > > >
> > > > > one or more Kafka topics.
> > > > >
> > > > >
> > > > >
> > > > > ** The Consumer API allows an application to subscribe to one or
> more
> > > > >
> > > > > topics and process the stream of records produced to them.
> > > > >
> > > > >
> > > > >
> > > > > ** The Streams API allows an application to act as a stream
> > processor,
> > > > >
> > > > > consuming an input stream from one or more topics and producing an
> > > > >
> > > > > output stream to one or more output topics, effectively
> transforming
> > > the
> > > > >
> > > > > input streams to output streams.
> > > > >
> > > > >
> > > > >
> > > > > ** The Connector API allows building and running reusable producers
> > or
> > > > >
> > > > > consumers that connect Kafka topics to existing applications or
> data
> > > > >
> > > > > systems. For example, a connector to a relational database might
> > > > >
> > > > > capture every change to a table.
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > > > With these APIs, Kafka can be used for two broad classes of
> > > application:
> > > > >
> > > > >
> > > > >
> > > > > ** Building real-time streaming data pipelines that reliably get
> data
> > > > >
> > > > > between systems or applications.
> > > > >
> > > > >
> > > > >
> > > > > ** Building real-time streaming applications that transform or
> react
> > > > >
> > > > > to the streams of data.
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > > > Apache Kafka is in use at large and small companies worldwide,
> > > including
> > > > >
> > > > > Capital One, Goldman Sachs, ING, LinkedIn, Netflix, Pinterest,
> > > Rabobank,
> > > > >
> > > > > Target, The New York Times, Uber, Yelp, and Zalando, among others.
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > > > A big thank you for the following 131 contributors to this release!
> > > > >
> > > > >
> > > > >
> > > > > Adem Efe Gencer, Alex D, Alex Dunayevsky, Allen Wang, Andras Beni,
> > > > >
> > > > > Andy Bryant, Andy Coates, Anna Povzner, Arjun Satish, asutosh936,
> > > > >
> > > > > Attila Sasvari, bartdevylder, Benedict Jin, Bill Bejeck, Blake
> > Miller,
> > > > >
> > > > > Boyang Chen, cburroughs, Chia-Ping Tsai, Chris Egerton, Colin P.
> > > Mccabe,
> > > > >
> > > > > Colin Patrick McCabe, ConcurrencyPractitioner, Damian Guy, dan
> > norwood,
> > > > >
> > > > > Daniel Shuy, Daniel Wojda, Dark, David Glasser, Debasish Ghosh,
> > > Detharon,
> > > > >
> > > > > Dhruvil Shah, Dmitry Minkovsky, Dong Lin, Edoardo Comar, emmanuel
> > > Harel,
> > > > >
> > > > > Eugene Sevastyanov, Ewen Cheslack-Postava, Fedor Bobin,
> > > > fedosov-alexander,
> > > > >
> > > > > Filipe Agapito, Florian Hussonnois, fredfp, Gilles Degols, gitlw,
> > > > Gitomain,
> > > > >
> > > > > Guangxian, Gunju Ko, Gunnar Morling, Guozhang Wang, hmcl, huxi,
> > huxihx,
> > > > >
> > > > > Igor Kostiakov, Ismael Juma, Jacek Laskowski, Jagadesh Adireddi,
> > > > >
> > > > > Jarek Rudzinski, Jason Gustafson, Jeff Klukas, Jeremy Custenborder,
> > > > >
> > > > > Jiangjie (Becket) Qin, Jiangjie Qin, JieFang.He, Jimin Hsieh, Joan
> > > > Goyeau,
> > > > >
> > > > > Joel Hamill, John Roesler, Jon Lee, Jorge Quilcate Otoya, Jun Rao,
> > > > >
> > > > > Kamal C, khairy, Koen De Groote, Konstantine Karantasis, Lee
> Dongjin,
> > > > >
> > > > > Liju John, Liquan Pei, lisa2lisa, Lucas Wang, Magesh Nandakumar,
> > > > >
> > > > > Magnus Edenhill, Magnus Reftel, Manikumar Reddy, Manikumar Reddy O,
> > > > >
> > > > > manjuapu, Mats Julian Olsen, Matthias J. Sax, Max Zheng, maytals,
> > > > >
> > > > > Michael Arndt, Michael G. Noll, Mickael Maison, nafshartous, Nick
> > > > Travers,
> > > > >
> > > > > nixsticks, Paolo Patierno, parafiend, Patrik Erdes, Radai
> Rosenblatt,
> > > > >
> > > > > Rajini Sivaram, Randall Hauch, ro7m, Robert Yokota, Roman Khlebnov,
> > > > >
> > > > > Ron Dagostino, Sandor Murakozi, Sasaki Toru, Sean Glover,
> > > > >
> > > > > Sebastian Bauersfeld, Siva Santhalingam, Stanislav Kozlovski,
> > Stephane
> > > > > Maarek,
> > > > >
> > > > > Stuart Perks, Surabhi Dixit, Sönke Liebau, taekyung, tedyu, Thomas
> > > > Leplus,
> > > > >
> > > > > UVN, Vahid Hashemian, Valentino Proietti, Viktor Somogyi, Vitaly
> > > Pushkar,
> > > > >
> > > > > Wladimir Schmidt, wushujames, Xavier Léauté, xin, yaphet,
> > > > >
> > > > > Yaswanth Kumar, ying-zheng, Yu
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > > > We welcome your help and feedback. For more information on how to
> > > > >
> > > > > report problems, and to get involved, visit the project website at
> > > > >
> > > > > https://kafka.apache.org/
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > > > Thank you!
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > > > Regards,
> > > > >
> > > > >
> > > > >
> > > > > Rajini
> > > > >
> > > >
> > > > --
> > > > *Dongjin Lee*
> > > >
> > > > *A hitchhiker in the mathematical world.*
> > > >
> > > > *github:  <http://goog_969573159/>github.com/dongjinleekr
> > > > <http://github.com/dongjinleekr>linkedin:
> > > kr.linkedin.com/in/dongjinleekr
> > > > <http://kr.linkedin.com/in/dongjinleekr>slideshare:
> > > www.slideshare.net/dongjinleekr
> > > > <http://www.slideshare.net/dongjinleekr>*
> > > >
> > >
> >
> >
> > --
> > Senior Software Engineer, Lightbend, Inc.
> >
> > <http://lightbend.com>
> >
> > @seg1o <https://twitter.com/seg1o>, in/seanaglover
> > <https://www.linkedin.com/in/seanaglover/>
> >
>



-- 

*Jeff Widman*
jeffwidman.com <http://www.jeffwidman.com/> | 740-WIDMAN-J (943-6265)
<><

Re: [ANNOUNCE] Apache Kafka 2.0.0 Released

[Stephane Maarek <st...@simplemachines.com.au>]

Congratulations !

On Mon., 30 Jul. 2018, 11:51 am Sean Glover, <se...@lightbend.com>
wrote:

> Congrats to everyone involved.  Releasing "2.0.0" is a big achievement :)
>
> On Mon, Jul 30, 2018 at 12:35 PM Dongjin Lee <do...@apache.org> wrote:
>
> > Thank you for your great works! Thanks again for the commiters and all
> the
> > contributors!
> >
> > On Tue, Jul 31, 2018 at 1:05 AM Ismael Juma <is...@gmail.com> wrote:
> >
> > > Thanks to everyone who contributed to the release (including testing
> and
> > > bug reports)! And thank you Rajini for managing the release.
> > >
> > > Ismael
> > >
> > > On Mon, 30 Jul 2018, 03:25 Rajini Sivaram, <rs...@apache.org>
> wrote:
> > >
> > > > The Apache Kafka community is pleased to announce the release for
> > > >
> > > > Apache Kafka 2.0.0.
> > > >
> > > >
> > > >
> > > >
> > > >
> > > > This is a major release and includes significant new features from
> > > >
> > > > 40 KIPs. It contains fixes and improvements from 246 JIRAs, including
> > > >
> > > > a few critical bugs. Here is a summary of some notable changes:
> > > >
> > > > ** KIP-290 adds support for prefixed ACLs, simplifying access control
> > > > management in large secure deployments. Bulk access to topics,
> > > > consumer groups or transactional ids with a prefix can now be granted
> > > > using a single rule. Access control for topic creation has also been
> > > > improved to enable access to be granted to create specific topics or
> > > > topics with a prefix.
> > > >
> > > > ** KIP-255 adds a framework for authenticating to Kafka brokers using
> > > > OAuth2 bearer tokens. The SASL/OAUTHBEARER implementation is
> > > > customizable using callbacks for token retrieval and validation.
> > > >
> > > > **Host name verification is now enabled by default for SSL
> connections
> > > > to ensure that the default SSL configuration is not susceptible to
> > > > man-in-the middle attacks. You can disable this verification for
> > > > deployments where validation is performed using other mechanisms.
> > > >
> > > > ** You can now dynamically update SSL trust stores without broker
> > > restart.
> > > > You can also configure security for broker listeners in ZooKeeper
> > before
> > > > starting brokers, including SSL key store and trust store passwords
> and
> > > > JAAS configuration for SASL. With this new feature, you can store
> > > sensitive
> > > > password configs in encrypted form in ZooKeeper rather than in
> > cleartext
> > > > in the broker properties file.
> > > >
> > > > ** The replication protocol has been improved to avoid log divergence
> > > > between leader and follower during fast leader failover. We have also
> > > > improved resilience of brokers by reducing the memory footprint of
> > > > message down-conversions. By using message chunking, both memory
> > > > usage and memory reference time have been reduced to avoid
> > > > OutOfMemory errors in brokers.
> > > >
> > > > ** Kafka clients are now notified of throttling before any throttling
> > is
> > > > applied
> > > > when quotas are enabled. This enables clients to distinguish between
> > > > network errors and large throttle times when quotas are exceeded.
> > > >
> > > > ** We have added a configuration option for Kafka consumer to avoid
> > > > indefinite blocking in the consumer.
> > > >
> > > > ** We have dropped support for Java 7 and removed the previously
> > > > deprecated Scala producer and consumer.
> > > >
> > > > ** Kafka Connect includes a number of improvements and features.
> > > > KIP-298 enables you to control how errors in connectors,
> > transformations
> > > > and converters are handled by enabling automatic retries and
> > controlling
> > > > the
> > > > number of errors that are tolerated before the connector is stopped.
> > More
> > > > contextual information can be included in the logs to help diagnose
> > > > problems
> > > > and problematic messages consumed by sink connectors can be sent to a
> > > > dead letter queue rather than forcing the connector to stop.
> > > >
> > > > ** KIP-297 adds a new extension point to move secrets out of
> connector
> > > > configurations and integrate with any external key management system.
> > > > The placeholders in connector configurations are only resolved before
> > > > sending the configuration to the connector, ensuring that secrets are
> > > > stored
> > > > and managed securely in your preferred key management system and
> > > > not exposed over the REST APIs or in log files.
> > > >
> > > > ** We have added a thin Scala wrapper API for our Kafka Streams DSL,
> > > > which provides better type inference and better type safety during
> > > compile
> > > > time. Scala users can have less boilerplate in their code, notably
> > > > regarding
> > > > Serdes with new implicit Serdes.
> > > >
> > > > ** Message headers are now supported in the Kafka Streams Processor
> > API,
> > > > allowing users to add and manipulate headers read from the source
> > topics
> > > > and propagate them to the sink topics.
> > > >
> > > > ** Windowed aggregations performance in Kafka Streams has been
> largely
> > > > improved (sometimes by an order of magnitude) thanks to the new
> > > > single-key-fetch API.
> > > >
> > > > ** We have further improved unit testibility of Kafka Streams with
> the
> > > > kafka-streams-testutil artifact.
> > > >
> > > >
> > > >
> > > >
> > > >
> > > > All of the changes in this release can be found in the release notes:
> > > >
> > > > https://www.apache.org/dist/kafka/2.0.0/RELEASE_NOTES.html
> > > >
> > > >
> > > >
> > > >
> > > >
> > > > You can download the source and binary release (Scala 2.11 and Scala
> > > 2.12)
> > > > from:
> > > >
> > > > https://kafka.apache.org/downloads#2.0.0
> > > > <https://kafka.apache.org/downloads#2.0.0>
> > > >
> > > >
> > > >
> > > >
> > > >
> > >
> >
> ---------------------------------------------------------------------------------------------------
> > > >
> > > >
> > > >
> > > >
> > > >
> > > > Apache Kafka is a distributed streaming platform with four core APIs:
> > > >
> > > >
> > > >
> > > > ** The Producer API allows an application to publish a stream records
> > to
> > > >
> > > > one or more Kafka topics.
> > > >
> > > >
> > > >
> > > > ** The Consumer API allows an application to subscribe to one or more
> > > >
> > > > topics and process the stream of records produced to them.
> > > >
> > > >
> > > >
> > > > ** The Streams API allows an application to act as a stream
> processor,
> > > >
> > > > consuming an input stream from one or more topics and producing an
> > > >
> > > > output stream to one or more output topics, effectively transforming
> > the
> > > >
> > > > input streams to output streams.
> > > >
> > > >
> > > >
> > > > ** The Connector API allows building and running reusable producers
> or
> > > >
> > > > consumers that connect Kafka topics to existing applications or data
> > > >
> > > > systems. For example, a connector to a relational database might
> > > >
> > > > capture every change to a table.
> > > >
> > > >
> > > >
> > > >
> > > >
> > > > With these APIs, Kafka can be used for two broad classes of
> > application:
> > > >
> > > >
> > > >
> > > > ** Building real-time streaming data pipelines that reliably get data
> > > >
> > > > between systems or applications.
> > > >
> > > >
> > > >
> > > > ** Building real-time streaming applications that transform or react
> > > >
> > > > to the streams of data.
> > > >
> > > >
> > > >
> > > >
> > > >
> > > >
> > > >
> > > > Apache Kafka is in use at large and small companies worldwide,
> > including
> > > >
> > > > Capital One, Goldman Sachs, ING, LinkedIn, Netflix, Pinterest,
> > Rabobank,
> > > >
> > > > Target, The New York Times, Uber, Yelp, and Zalando, among others.
> > > >
> > > >
> > > >
> > > >
> > > >
> > > >
> > > >
> > > > A big thank you for the following 131 contributors to this release!
> > > >
> > > >
> > > >
> > > > Adem Efe Gencer, Alex D, Alex Dunayevsky, Allen Wang, Andras Beni,
> > > >
> > > > Andy Bryant, Andy Coates, Anna Povzner, Arjun Satish, asutosh936,
> > > >
> > > > Attila Sasvari, bartdevylder, Benedict Jin, Bill Bejeck, Blake
> Miller,
> > > >
> > > > Boyang Chen, cburroughs, Chia-Ping Tsai, Chris Egerton, Colin P.
> > Mccabe,
> > > >
> > > > Colin Patrick McCabe, ConcurrencyPractitioner, Damian Guy, dan
> norwood,
> > > >
> > > > Daniel Shuy, Daniel Wojda, Dark, David Glasser, Debasish Ghosh,
> > Detharon,
> > > >
> > > > Dhruvil Shah, Dmitry Minkovsky, Dong Lin, Edoardo Comar, emmanuel
> > Harel,
> > > >
> > > > Eugene Sevastyanov, Ewen Cheslack-Postava, Fedor Bobin,
> > > fedosov-alexander,
> > > >
> > > > Filipe Agapito, Florian Hussonnois, fredfp, Gilles Degols, gitlw,
> > > Gitomain,
> > > >
> > > > Guangxian, Gunju Ko, Gunnar Morling, Guozhang Wang, hmcl, huxi,
> huxihx,
> > > >
> > > > Igor Kostiakov, Ismael Juma, Jacek Laskowski, Jagadesh Adireddi,
> > > >
> > > > Jarek Rudzinski, Jason Gustafson, Jeff Klukas, Jeremy Custenborder,
> > > >
> > > > Jiangjie (Becket) Qin, Jiangjie Qin, JieFang.He, Jimin Hsieh, Joan
> > > Goyeau,
> > > >
> > > > Joel Hamill, John Roesler, Jon Lee, Jorge Quilcate Otoya, Jun Rao,
> > > >
> > > > Kamal C, khairy, Koen De Groote, Konstantine Karantasis, Lee Dongjin,
> > > >
> > > > Liju John, Liquan Pei, lisa2lisa, Lucas Wang, Magesh Nandakumar,
> > > >
> > > > Magnus Edenhill, Magnus Reftel, Manikumar Reddy, Manikumar Reddy O,
> > > >
> > > > manjuapu, Mats Julian Olsen, Matthias J. Sax, Max Zheng, maytals,
> > > >
> > > > Michael Arndt, Michael G. Noll, Mickael Maison, nafshartous, Nick
> > > Travers,
> > > >
> > > > nixsticks, Paolo Patierno, parafiend, Patrik Erdes, Radai Rosenblatt,
> > > >
> > > > Rajini Sivaram, Randall Hauch, ro7m, Robert Yokota, Roman Khlebnov,
> > > >
> > > > Ron Dagostino, Sandor Murakozi, Sasaki Toru, Sean Glover,
> > > >
> > > > Sebastian Bauersfeld, Siva Santhalingam, Stanislav Kozlovski,
> Stephane
> > > > Maarek,
> > > >
> > > > Stuart Perks, Surabhi Dixit, Sönke Liebau, taekyung, tedyu, Thomas
> > > Leplus,
> > > >
> > > > UVN, Vahid Hashemian, Valentino Proietti, Viktor Somogyi, Vitaly
> > Pushkar,
> > > >
> > > > Wladimir Schmidt, wushujames, Xavier Léauté, xin, yaphet,
> > > >
> > > > Yaswanth Kumar, ying-zheng, Yu
> > > >
> > > >
> > > >
> > > >
> > > >
> > > >
> > > >
> > > > We welcome your help and feedback. For more information on how to
> > > >
> > > > report problems, and to get involved, visit the project website at
> > > >
> > > > https://kafka.apache.org/
> > > >
> > > >
> > > >
> > > >
> > > >
> > > > Thank you!
> > > >
> > > >
> > > >
> > > >
> > > >
> > > > Regards,
> > > >
> > > >
> > > >
> > > > Rajini
> > > >
> > >
> > > --
> > > *Dongjin Lee*
> > >
> > > *A hitchhiker in the mathematical world.*
> > >
> > > *github:  <http://goog_969573159/>github.com/dongjinleekr
> > > <http://github.com/dongjinleekr>linkedin:
> > kr.linkedin.com/in/dongjinleekr
> > > <http://kr.linkedin.com/in/dongjinleekr>slideshare:
> > www.slideshare.net/dongjinleekr
> > > <http://www.slideshare.net/dongjinleekr>*
> > >
> >
>
>
> --
> Senior Software Engineer, Lightbend, Inc.
>
> <http://lightbend.com>
>
> @seg1o <https://twitter.com/seg1o>, in/seanaglover
> <https://www.linkedin.com/in/seanaglover/>
>

Re: [ANNOUNCE] Apache Kafka 2.0.0 Released

[Sean Glover <se...@lightbend.com>]

Congrats to everyone involved.  Releasing "2.0.0" is a big achievement :)

On Mon, Jul 30, 2018 at 12:35 PM Dongjin Lee <do...@apache.org> wrote:

> Thank you for your great works! Thanks again for the commiters and all the
> contributors!
>
> On Tue, Jul 31, 2018 at 1:05 AM Ismael Juma <is...@gmail.com> wrote:
>
> > Thanks to everyone who contributed to the release (including testing and
> > bug reports)! And thank you Rajini for managing the release.
> >
> > Ismael
> >
> > On Mon, 30 Jul 2018, 03:25 Rajini Sivaram, <rs...@apache.org> wrote:
> >
> > > The Apache Kafka community is pleased to announce the release for
> > >
> > > Apache Kafka 2.0.0.
> > >
> > >
> > >
> > >
> > >
> > > This is a major release and includes significant new features from
> > >
> > > 40 KIPs. It contains fixes and improvements from 246 JIRAs, including
> > >
> > > a few critical bugs. Here is a summary of some notable changes:
> > >
> > > ** KIP-290 adds support for prefixed ACLs, simplifying access control
> > > management in large secure deployments. Bulk access to topics,
> > > consumer groups or transactional ids with a prefix can now be granted
> > > using a single rule. Access control for topic creation has also been
> > > improved to enable access to be granted to create specific topics or
> > > topics with a prefix.
> > >
> > > ** KIP-255 adds a framework for authenticating to Kafka brokers using
> > > OAuth2 bearer tokens. The SASL/OAUTHBEARER implementation is
> > > customizable using callbacks for token retrieval and validation.
> > >
> > > **Host name verification is now enabled by default for SSL connections
> > > to ensure that the default SSL configuration is not susceptible to
> > > man-in-the middle attacks. You can disable this verification for
> > > deployments where validation is performed using other mechanisms.
> > >
> > > ** You can now dynamically update SSL trust stores without broker
> > restart.
> > > You can also configure security for broker listeners in ZooKeeper
> before
> > > starting brokers, including SSL key store and trust store passwords and
> > > JAAS configuration for SASL. With this new feature, you can store
> > sensitive
> > > password configs in encrypted form in ZooKeeper rather than in
> cleartext
> > > in the broker properties file.
> > >
> > > ** The replication protocol has been improved to avoid log divergence
> > > between leader and follower during fast leader failover. We have also
> > > improved resilience of brokers by reducing the memory footprint of
> > > message down-conversions. By using message chunking, both memory
> > > usage and memory reference time have been reduced to avoid
> > > OutOfMemory errors in brokers.
> > >
> > > ** Kafka clients are now notified of throttling before any throttling
> is
> > > applied
> > > when quotas are enabled. This enables clients to distinguish between
> > > network errors and large throttle times when quotas are exceeded.
> > >
> > > ** We have added a configuration option for Kafka consumer to avoid
> > > indefinite blocking in the consumer.
> > >
> > > ** We have dropped support for Java 7 and removed the previously
> > > deprecated Scala producer and consumer.
> > >
> > > ** Kafka Connect includes a number of improvements and features.
> > > KIP-298 enables you to control how errors in connectors,
> transformations
> > > and converters are handled by enabling automatic retries and
> controlling
> > > the
> > > number of errors that are tolerated before the connector is stopped.
> More
> > > contextual information can be included in the logs to help diagnose
> > > problems
> > > and problematic messages consumed by sink connectors can be sent to a
> > > dead letter queue rather than forcing the connector to stop.
> > >
> > > ** KIP-297 adds a new extension point to move secrets out of connector
> > > configurations and integrate with any external key management system.
> > > The placeholders in connector configurations are only resolved before
> > > sending the configuration to the connector, ensuring that secrets are
> > > stored
> > > and managed securely in your preferred key management system and
> > > not exposed over the REST APIs or in log files.
> > >
> > > ** We have added a thin Scala wrapper API for our Kafka Streams DSL,
> > > which provides better type inference and better type safety during
> > compile
> > > time. Scala users can have less boilerplate in their code, notably
> > > regarding
> > > Serdes with new implicit Serdes.
> > >
> > > ** Message headers are now supported in the Kafka Streams Processor
> API,
> > > allowing users to add and manipulate headers read from the source
> topics
> > > and propagate them to the sink topics.
> > >
> > > ** Windowed aggregations performance in Kafka Streams has been largely
> > > improved (sometimes by an order of magnitude) thanks to the new
> > > single-key-fetch API.
> > >
> > > ** We have further improved unit testibility of Kafka Streams with the
> > > kafka-streams-testutil artifact.
> > >
> > >
> > >
> > >
> > >
> > > All of the changes in this release can be found in the release notes:
> > >
> > > https://www.apache.org/dist/kafka/2.0.0/RELEASE_NOTES.html
> > >
> > >
> > >
> > >
> > >
> > > You can download the source and binary release (Scala 2.11 and Scala
> > 2.12)
> > > from:
> > >
> > > https://kafka.apache.org/downloads#2.0.0
> > > <https://kafka.apache.org/downloads#2.0.0>
> > >
> > >
> > >
> > >
> > >
> >
> ---------------------------------------------------------------------------------------------------
> > >
> > >
> > >
> > >
> > >
> > > Apache Kafka is a distributed streaming platform with four core APIs:
> > >
> > >
> > >
> > > ** The Producer API allows an application to publish a stream records
> to
> > >
> > > one or more Kafka topics.
> > >
> > >
> > >
> > > ** The Consumer API allows an application to subscribe to one or more
> > >
> > > topics and process the stream of records produced to them.
> > >
> > >
> > >
> > > ** The Streams API allows an application to act as a stream processor,
> > >
> > > consuming an input stream from one or more topics and producing an
> > >
> > > output stream to one or more output topics, effectively transforming
> the
> > >
> > > input streams to output streams.
> > >
> > >
> > >
> > > ** The Connector API allows building and running reusable producers or
> > >
> > > consumers that connect Kafka topics to existing applications or data
> > >
> > > systems. For example, a connector to a relational database might
> > >
> > > capture every change to a table.
> > >
> > >
> > >
> > >
> > >
> > > With these APIs, Kafka can be used for two broad classes of
> application:
> > >
> > >
> > >
> > > ** Building real-time streaming data pipelines that reliably get data
> > >
> > > between systems or applications.
> > >
> > >
> > >
> > > ** Building real-time streaming applications that transform or react
> > >
> > > to the streams of data.
> > >
> > >
> > >
> > >
> > >
> > >
> > >
> > > Apache Kafka is in use at large and small companies worldwide,
> including
> > >
> > > Capital One, Goldman Sachs, ING, LinkedIn, Netflix, Pinterest,
> Rabobank,
> > >
> > > Target, The New York Times, Uber, Yelp, and Zalando, among others.
> > >
> > >
> > >
> > >
> > >
> > >
> > >
> > > A big thank you for the following 131 contributors to this release!
> > >
> > >
> > >
> > > Adem Efe Gencer, Alex D, Alex Dunayevsky, Allen Wang, Andras Beni,
> > >
> > > Andy Bryant, Andy Coates, Anna Povzner, Arjun Satish, asutosh936,
> > >
> > > Attila Sasvari, bartdevylder, Benedict Jin, Bill Bejeck, Blake Miller,
> > >
> > > Boyang Chen, cburroughs, Chia-Ping Tsai, Chris Egerton, Colin P.
> Mccabe,
> > >
> > > Colin Patrick McCabe, ConcurrencyPractitioner, Damian Guy, dan norwood,
> > >
> > > Daniel Shuy, Daniel Wojda, Dark, David Glasser, Debasish Ghosh,
> Detharon,
> > >
> > > Dhruvil Shah, Dmitry Minkovsky, Dong Lin, Edoardo Comar, emmanuel
> Harel,
> > >
> > > Eugene Sevastyanov, Ewen Cheslack-Postava, Fedor Bobin,
> > fedosov-alexander,
> > >
> > > Filipe Agapito, Florian Hussonnois, fredfp, Gilles Degols, gitlw,
> > Gitomain,
> > >
> > > Guangxian, Gunju Ko, Gunnar Morling, Guozhang Wang, hmcl, huxi, huxihx,
> > >
> > > Igor Kostiakov, Ismael Juma, Jacek Laskowski, Jagadesh Adireddi,
> > >
> > > Jarek Rudzinski, Jason Gustafson, Jeff Klukas, Jeremy Custenborder,
> > >
> > > Jiangjie (Becket) Qin, Jiangjie Qin, JieFang.He, Jimin Hsieh, Joan
> > Goyeau,
> > >
> > > Joel Hamill, John Roesler, Jon Lee, Jorge Quilcate Otoya, Jun Rao,
> > >
> > > Kamal C, khairy, Koen De Groote, Konstantine Karantasis, Lee Dongjin,
> > >
> > > Liju John, Liquan Pei, lisa2lisa, Lucas Wang, Magesh Nandakumar,
> > >
> > > Magnus Edenhill, Magnus Reftel, Manikumar Reddy, Manikumar Reddy O,
> > >
> > > manjuapu, Mats Julian Olsen, Matthias J. Sax, Max Zheng, maytals,
> > >
> > > Michael Arndt, Michael G. Noll, Mickael Maison, nafshartous, Nick
> > Travers,
> > >
> > > nixsticks, Paolo Patierno, parafiend, Patrik Erdes, Radai Rosenblatt,
> > >
> > > Rajini Sivaram, Randall Hauch, ro7m, Robert Yokota, Roman Khlebnov,
> > >
> > > Ron Dagostino, Sandor Murakozi, Sasaki Toru, Sean Glover,
> > >
> > > Sebastian Bauersfeld, Siva Santhalingam, Stanislav Kozlovski, Stephane
> > > Maarek,
> > >
> > > Stuart Perks, Surabhi Dixit, Sönke Liebau, taekyung, tedyu, Thomas
> > Leplus,
> > >
> > > UVN, Vahid Hashemian, Valentino Proietti, Viktor Somogyi, Vitaly
> Pushkar,
> > >
> > > Wladimir Schmidt, wushujames, Xavier Léauté, xin, yaphet,
> > >
> > > Yaswanth Kumar, ying-zheng, Yu
> > >
> > >
> > >
> > >
> > >
> > >
> > >
> > > We welcome your help and feedback. For more information on how to
> > >
> > > report problems, and to get involved, visit the project website at
> > >
> > > https://kafka.apache.org/
> > >
> > >
> > >
> > >
> > >
> > > Thank you!
> > >
> > >
> > >
> > >
> > >
> > > Regards,
> > >
> > >
> > >
> > > Rajini
> > >
> >
> > --
> > *Dongjin Lee*
> >
> > *A hitchhiker in the mathematical world.*
> >
> > *github:  <http://goog_969573159/>github.com/dongjinleekr
> > <http://github.com/dongjinleekr>linkedin:
> kr.linkedin.com/in/dongjinleekr
> > <http://kr.linkedin.com/in/dongjinleekr>slideshare:
> www.slideshare.net/dongjinleekr
> > <http://www.slideshare.net/dongjinleekr>*
> >
>


-- 
Senior Software Engineer, Lightbend, Inc.

<http://lightbend.com>

@seg1o <https://twitter.com/seg1o>, in/seanaglover
<https://www.linkedin.com/in/seanaglover/>

Re: [ANNOUNCE] Apache Kafka 2.0.0 Released

[Dongjin Lee <do...@apache.org>]

Thank you for your great works! Thanks again for the commiters and all the
contributors!

On Tue, Jul 31, 2018 at 1:05 AM Ismael Juma <is...@gmail.com> wrote:

> Thanks to everyone who contributed to the release (including testing and
> bug reports)! And thank you Rajini for managing the release.
>
> Ismael
>
> On Mon, 30 Jul 2018, 03:25 Rajini Sivaram, <rs...@apache.org> wrote:
>
> > The Apache Kafka community is pleased to announce the release for
> >
> > Apache Kafka 2.0.0.
> >
> >
> >
> >
> >
> > This is a major release and includes significant new features from
> >
> > 40 KIPs. It contains fixes and improvements from 246 JIRAs, including
> >
> > a few critical bugs. Here is a summary of some notable changes:
> >
> > ** KIP-290 adds support for prefixed ACLs, simplifying access control
> > management in large secure deployments. Bulk access to topics,
> > consumer groups or transactional ids with a prefix can now be granted
> > using a single rule. Access control for topic creation has also been
> > improved to enable access to be granted to create specific topics or
> > topics with a prefix.
> >
> > ** KIP-255 adds a framework for authenticating to Kafka brokers using
> > OAuth2 bearer tokens. The SASL/OAUTHBEARER implementation is
> > customizable using callbacks for token retrieval and validation.
> >
> > **Host name verification is now enabled by default for SSL connections
> > to ensure that the default SSL configuration is not susceptible to
> > man-in-the middle attacks. You can disable this verification for
> > deployments where validation is performed using other mechanisms.
> >
> > ** You can now dynamically update SSL trust stores without broker
> restart.
> > You can also configure security for broker listeners in ZooKeeper before
> > starting brokers, including SSL key store and trust store passwords and
> > JAAS configuration for SASL. With this new feature, you can store
> sensitive
> > password configs in encrypted form in ZooKeeper rather than in cleartext
> > in the broker properties file.
> >
> > ** The replication protocol has been improved to avoid log divergence
> > between leader and follower during fast leader failover. We have also
> > improved resilience of brokers by reducing the memory footprint of
> > message down-conversions. By using message chunking, both memory
> > usage and memory reference time have been reduced to avoid
> > OutOfMemory errors in brokers.
> >
> > ** Kafka clients are now notified of throttling before any throttling is
> > applied
> > when quotas are enabled. This enables clients to distinguish between
> > network errors and large throttle times when quotas are exceeded.
> >
> > ** We have added a configuration option for Kafka consumer to avoid
> > indefinite blocking in the consumer.
> >
> > ** We have dropped support for Java 7 and removed the previously
> > deprecated Scala producer and consumer.
> >
> > ** Kafka Connect includes a number of improvements and features.
> > KIP-298 enables you to control how errors in connectors, transformations
> > and converters are handled by enabling automatic retries and controlling
> > the
> > number of errors that are tolerated before the connector is stopped. More
> > contextual information can be included in the logs to help diagnose
> > problems
> > and problematic messages consumed by sink connectors can be sent to a
> > dead letter queue rather than forcing the connector to stop.
> >
> > ** KIP-297 adds a new extension point to move secrets out of connector
> > configurations and integrate with any external key management system.
> > The placeholders in connector configurations are only resolved before
> > sending the configuration to the connector, ensuring that secrets are
> > stored
> > and managed securely in your preferred key management system and
> > not exposed over the REST APIs or in log files.
> >
> > ** We have added a thin Scala wrapper API for our Kafka Streams DSL,
> > which provides better type inference and better type safety during
> compile
> > time. Scala users can have less boilerplate in their code, notably
> > regarding
> > Serdes with new implicit Serdes.
> >
> > ** Message headers are now supported in the Kafka Streams Processor API,
> > allowing users to add and manipulate headers read from the source topics
> > and propagate them to the sink topics.
> >
> > ** Windowed aggregations performance in Kafka Streams has been largely
> > improved (sometimes by an order of magnitude) thanks to the new
> > single-key-fetch API.
> >
> > ** We have further improved unit testibility of Kafka Streams with the
> > kafka-streams-testutil artifact.
> >
> >
> >
> >
> >
> > All of the changes in this release can be found in the release notes:
> >
> > https://www.apache.org/dist/kafka/2.0.0/RELEASE_NOTES.html
> >
> >
> >
> >
> >
> > You can download the source and binary release (Scala 2.11 and Scala
> 2.12)
> > from:
> >
> > https://kafka.apache.org/downloads#2.0.0
> > <https://kafka.apache.org/downloads#2.0.0>
> >
> >
> >
> >
> >
> ---------------------------------------------------------------------------------------------------
> >
> >
> >
> >
> >
> > Apache Kafka is a distributed streaming platform with four core APIs:
> >
> >
> >
> > ** The Producer API allows an application to publish a stream records to
> >
> > one or more Kafka topics.
> >
> >
> >
> > ** The Consumer API allows an application to subscribe to one or more
> >
> > topics and process the stream of records produced to them.
> >
> >
> >
> > ** The Streams API allows an application to act as a stream processor,
> >
> > consuming an input stream from one or more topics and producing an
> >
> > output stream to one or more output topics, effectively transforming the
> >
> > input streams to output streams.
> >
> >
> >
> > ** The Connector API allows building and running reusable producers or
> >
> > consumers that connect Kafka topics to existing applications or data
> >
> > systems. For example, a connector to a relational database might
> >
> > capture every change to a table.
> >
> >
> >
> >
> >
> > With these APIs, Kafka can be used for two broad classes of application:
> >
> >
> >
> > ** Building real-time streaming data pipelines that reliably get data
> >
> > between systems or applications.
> >
> >
> >
> > ** Building real-time streaming applications that transform or react
> >
> > to the streams of data.
> >
> >
> >
> >
> >
> >
> >
> > Apache Kafka is in use at large and small companies worldwide, including
> >
> > Capital One, Goldman Sachs, ING, LinkedIn, Netflix, Pinterest, Rabobank,
> >
> > Target, The New York Times, Uber, Yelp, and Zalando, among others.
> >
> >
> >
> >
> >
> >
> >
> > A big thank you for the following 131 contributors to this release!
> >
> >
> >
> > Adem Efe Gencer, Alex D, Alex Dunayevsky, Allen Wang, Andras Beni,
> >
> > Andy Bryant, Andy Coates, Anna Povzner, Arjun Satish, asutosh936,
> >
> > Attila Sasvari, bartdevylder, Benedict Jin, Bill Bejeck, Blake Miller,
> >
> > Boyang Chen, cburroughs, Chia-Ping Tsai, Chris Egerton, Colin P. Mccabe,
> >
> > Colin Patrick McCabe, ConcurrencyPractitioner, Damian Guy, dan norwood,
> >
> > Daniel Shuy, Daniel Wojda, Dark, David Glasser, Debasish Ghosh, Detharon,
> >
> > Dhruvil Shah, Dmitry Minkovsky, Dong Lin, Edoardo Comar, emmanuel Harel,
> >
> > Eugene Sevastyanov, Ewen Cheslack-Postava, Fedor Bobin,
> fedosov-alexander,
> >
> > Filipe Agapito, Florian Hussonnois, fredfp, Gilles Degols, gitlw,
> Gitomain,
> >
> > Guangxian, Gunju Ko, Gunnar Morling, Guozhang Wang, hmcl, huxi, huxihx,
> >
> > Igor Kostiakov, Ismael Juma, Jacek Laskowski, Jagadesh Adireddi,
> >
> > Jarek Rudzinski, Jason Gustafson, Jeff Klukas, Jeremy Custenborder,
> >
> > Jiangjie (Becket) Qin, Jiangjie Qin, JieFang.He, Jimin Hsieh, Joan
> Goyeau,
> >
> > Joel Hamill, John Roesler, Jon Lee, Jorge Quilcate Otoya, Jun Rao,
> >
> > Kamal C, khairy, Koen De Groote, Konstantine Karantasis, Lee Dongjin,
> >
> > Liju John, Liquan Pei, lisa2lisa, Lucas Wang, Magesh Nandakumar,
> >
> > Magnus Edenhill, Magnus Reftel, Manikumar Reddy, Manikumar Reddy O,
> >
> > manjuapu, Mats Julian Olsen, Matthias J. Sax, Max Zheng, maytals,
> >
> > Michael Arndt, Michael G. Noll, Mickael Maison, nafshartous, Nick
> Travers,
> >
> > nixsticks, Paolo Patierno, parafiend, Patrik Erdes, Radai Rosenblatt,
> >
> > Rajini Sivaram, Randall Hauch, ro7m, Robert Yokota, Roman Khlebnov,
> >
> > Ron Dagostino, Sandor Murakozi, Sasaki Toru, Sean Glover,
> >
> > Sebastian Bauersfeld, Siva Santhalingam, Stanislav Kozlovski, Stephane
> > Maarek,
> >
> > Stuart Perks, Surabhi Dixit, Sönke Liebau, taekyung, tedyu, Thomas
> Leplus,
> >
> > UVN, Vahid Hashemian, Valentino Proietti, Viktor Somogyi, Vitaly Pushkar,
> >
> > Wladimir Schmidt, wushujames, Xavier Léauté, xin, yaphet,
> >
> > Yaswanth Kumar, ying-zheng, Yu
> >
> >
> >
> >
> >
> >
> >
> > We welcome your help and feedback. For more information on how to
> >
> > report problems, and to get involved, visit the project website at
> >
> > https://kafka.apache.org/
> >
> >
> >
> >
> >
> > Thank you!
> >
> >
> >
> >
> >
> > Regards,
> >
> >
> >
> > Rajini
> >
>
> --
> *Dongjin Lee*
>
> *A hitchhiker in the mathematical world.*
>
> *github:  <http://goog_969573159/>github.com/dongjinleekr
> <http://github.com/dongjinleekr>linkedin: kr.linkedin.com/in/dongjinleekr
> <http://kr.linkedin.com/in/dongjinleekr>slideshare: www.slideshare.net/dongjinleekr
> <http://www.slideshare.net/dongjinleekr>*
>

Re: [ANNOUNCE] Apache Kafka 2.0.0 Released

[Ismael Juma <is...@gmail.com>]

Thanks to everyone who contributed to the release (including testing and
bug reports)! And thank you Rajini for managing the release.

Ismael

On Mon, 30 Jul 2018, 03:25 Rajini Sivaram, <rs...@apache.org> wrote:

> The Apache Kafka community is pleased to announce the release for
>
> Apache Kafka 2.0.0.
>
>
>
>
>
> This is a major release and includes significant new features from
>
> 40 KIPs. It contains fixes and improvements from 246 JIRAs, including
>
> a few critical bugs. Here is a summary of some notable changes:
>
> ** KIP-290 adds support for prefixed ACLs, simplifying access control
> management in large secure deployments. Bulk access to topics,
> consumer groups or transactional ids with a prefix can now be granted
> using a single rule. Access control for topic creation has also been
> improved to enable access to be granted to create specific topics or
> topics with a prefix.
>
> ** KIP-255 adds a framework for authenticating to Kafka brokers using
> OAuth2 bearer tokens. The SASL/OAUTHBEARER implementation is
> customizable using callbacks for token retrieval and validation.
>
> **Host name verification is now enabled by default for SSL connections
> to ensure that the default SSL configuration is not susceptible to
> man-in-the middle attacks. You can disable this verification for
> deployments where validation is performed using other mechanisms.
>
> ** You can now dynamically update SSL trust stores without broker restart.
> You can also configure security for broker listeners in ZooKeeper before
> starting brokers, including SSL key store and trust store passwords and
> JAAS configuration for SASL. With this new feature, you can store sensitive
> password configs in encrypted form in ZooKeeper rather than in cleartext
> in the broker properties file.
>
> ** The replication protocol has been improved to avoid log divergence
> between leader and follower during fast leader failover. We have also
> improved resilience of brokers by reducing the memory footprint of
> message down-conversions. By using message chunking, both memory
> usage and memory reference time have been reduced to avoid
> OutOfMemory errors in brokers.
>
> ** Kafka clients are now notified of throttling before any throttling is
> applied
> when quotas are enabled. This enables clients to distinguish between
> network errors and large throttle times when quotas are exceeded.
>
> ** We have added a configuration option for Kafka consumer to avoid
> indefinite blocking in the consumer.
>
> ** We have dropped support for Java 7 and removed the previously
> deprecated Scala producer and consumer.
>
> ** Kafka Connect includes a number of improvements and features.
> KIP-298 enables you to control how errors in connectors, transformations
> and converters are handled by enabling automatic retries and controlling
> the
> number of errors that are tolerated before the connector is stopped. More
> contextual information can be included in the logs to help diagnose
> problems
> and problematic messages consumed by sink connectors can be sent to a
> dead letter queue rather than forcing the connector to stop.
>
> ** KIP-297 adds a new extension point to move secrets out of connector
> configurations and integrate with any external key management system.
> The placeholders in connector configurations are only resolved before
> sending the configuration to the connector, ensuring that secrets are
> stored
> and managed securely in your preferred key management system and
> not exposed over the REST APIs or in log files.
>
> ** We have added a thin Scala wrapper API for our Kafka Streams DSL,
> which provides better type inference and better type safety during compile
> time. Scala users can have less boilerplate in their code, notably
> regarding
> Serdes with new implicit Serdes.
>
> ** Message headers are now supported in the Kafka Streams Processor API,
> allowing users to add and manipulate headers read from the source topics
> and propagate them to the sink topics.
>
> ** Windowed aggregations performance in Kafka Streams has been largely
> improved (sometimes by an order of magnitude) thanks to the new
> single-key-fetch API.
>
> ** We have further improved unit testibility of Kafka Streams with the
> kafka-streams-testutil artifact.
>
>
>
>
>
> All of the changes in this release can be found in the release notes:
>
> https://www.apache.org/dist/kafka/2.0.0/RELEASE_NOTES.html
>
>
>
>
>
> You can download the source and binary release (Scala 2.11 and Scala 2.12)
> from:
>
> https://kafka.apache.org/downloads#2.0.0
> <https://kafka.apache.org/downloads#2.0.0>
>
>
>
>
> ---------------------------------------------------------------------------------------------------
>
>
>
>
>
> Apache Kafka is a distributed streaming platform with four core APIs:
>
>
>
> ** The Producer API allows an application to publish a stream records to
>
> one or more Kafka topics.
>
>
>
> ** The Consumer API allows an application to subscribe to one or more
>
> topics and process the stream of records produced to them.
>
>
>
> ** The Streams API allows an application to act as a stream processor,
>
> consuming an input stream from one or more topics and producing an
>
> output stream to one or more output topics, effectively transforming the
>
> input streams to output streams.
>
>
>
> ** The Connector API allows building and running reusable producers or
>
> consumers that connect Kafka topics to existing applications or data
>
> systems. For example, a connector to a relational database might
>
> capture every change to a table.
>
>
>
>
>
> With these APIs, Kafka can be used for two broad classes of application:
>
>
>
> ** Building real-time streaming data pipelines that reliably get data
>
> between systems or applications.
>
>
>
> ** Building real-time streaming applications that transform or react
>
> to the streams of data.
>
>
>
>
>
>
>
> Apache Kafka is in use at large and small companies worldwide, including
>
> Capital One, Goldman Sachs, ING, LinkedIn, Netflix, Pinterest, Rabobank,
>
> Target, The New York Times, Uber, Yelp, and Zalando, among others.
>
>
>
>
>
>
>
> A big thank you for the following 131 contributors to this release!
>
>
>
> Adem Efe Gencer, Alex D, Alex Dunayevsky, Allen Wang, Andras Beni,
>
> Andy Bryant, Andy Coates, Anna Povzner, Arjun Satish, asutosh936,
>
> Attila Sasvari, bartdevylder, Benedict Jin, Bill Bejeck, Blake Miller,
>
> Boyang Chen, cburroughs, Chia-Ping Tsai, Chris Egerton, Colin P. Mccabe,
>
> Colin Patrick McCabe, ConcurrencyPractitioner, Damian Guy, dan norwood,
>
> Daniel Shuy, Daniel Wojda, Dark, David Glasser, Debasish Ghosh, Detharon,
>
> Dhruvil Shah, Dmitry Minkovsky, Dong Lin, Edoardo Comar, emmanuel Harel,
>
> Eugene Sevastyanov, Ewen Cheslack-Postava, Fedor Bobin, fedosov-alexander,
>
> Filipe Agapito, Florian Hussonnois, fredfp, Gilles Degols, gitlw, Gitomain,
>
> Guangxian, Gunju Ko, Gunnar Morling, Guozhang Wang, hmcl, huxi, huxihx,
>
> Igor Kostiakov, Ismael Juma, Jacek Laskowski, Jagadesh Adireddi,
>
> Jarek Rudzinski, Jason Gustafson, Jeff Klukas, Jeremy Custenborder,
>
> Jiangjie (Becket) Qin, Jiangjie Qin, JieFang.He, Jimin Hsieh, Joan Goyeau,
>
> Joel Hamill, John Roesler, Jon Lee, Jorge Quilcate Otoya, Jun Rao,
>
> Kamal C, khairy, Koen De Groote, Konstantine Karantasis, Lee Dongjin,
>
> Liju John, Liquan Pei, lisa2lisa, Lucas Wang, Magesh Nandakumar,
>
> Magnus Edenhill, Magnus Reftel, Manikumar Reddy, Manikumar Reddy O,
>
> manjuapu, Mats Julian Olsen, Matthias J. Sax, Max Zheng, maytals,
>
> Michael Arndt, Michael G. Noll, Mickael Maison, nafshartous, Nick Travers,
>
> nixsticks, Paolo Patierno, parafiend, Patrik Erdes, Radai Rosenblatt,
>
> Rajini Sivaram, Randall Hauch, ro7m, Robert Yokota, Roman Khlebnov,
>
> Ron Dagostino, Sandor Murakozi, Sasaki Toru, Sean Glover,
>
> Sebastian Bauersfeld, Siva Santhalingam, Stanislav Kozlovski, Stephane
> Maarek,
>
> Stuart Perks, Surabhi Dixit, Sönke Liebau, taekyung, tedyu, Thomas Leplus,
>
> UVN, Vahid Hashemian, Valentino Proietti, Viktor Somogyi, Vitaly Pushkar,
>
> Wladimir Schmidt, wushujames, Xavier Léauté, xin, yaphet,
>
> Yaswanth Kumar, ying-zheng, Yu
>
>
>
>
>
>
>
> We welcome your help and feedback. For more information on how to
>
> report problems, and to get involved, visit the project website at
>
> https://kafka.apache.org/
>
>
>
>
>
> Thank you!
>
>
>
>
>
> Regards,
>
>
>
> Rajini
>

Re: [ANNOUNCE] Apache Kafka 2.0.0 Released

[Damian Guy <da...@gmail.com>]

Excellent! Thanks for running the release Rajini!

On Mon, 30 Jul 2018 at 11:25 Rajini Sivaram <rs...@apache.org> wrote:

> The Apache Kafka community is pleased to announce the release for
>
> Apache Kafka 2.0.0.
>
>
>
>
>
> This is a major release and includes significant new features from
>
> 40 KIPs. It contains fixes and improvements from 246 JIRAs, including
>
> a few critical bugs. Here is a summary of some notable changes:
>
> ** KIP-290 adds support for prefixed ACLs, simplifying access control
> management in large secure deployments. Bulk access to topics,
> consumer groups or transactional ids with a prefix can now be granted
> using a single rule. Access control for topic creation has also been
> improved to enable access to be granted to create specific topics or
> topics with a prefix.
>
> ** KIP-255 adds a framework for authenticating to Kafka brokers using
> OAuth2 bearer tokens. The SASL/OAUTHBEARER implementation is
> customizable using callbacks for token retrieval and validation.
>
> **Host name verification is now enabled by default for SSL connections
> to ensure that the default SSL configuration is not susceptible to
> man-in-the middle attacks. You can disable this verification for
> deployments where validation is performed using other mechanisms.
>
> ** You can now dynamically update SSL trust stores without broker restart.
> You can also configure security for broker listeners in ZooKeeper before
> starting brokers, including SSL key store and trust store passwords and
> JAAS configuration for SASL. With this new feature, you can store sensitive
> password configs in encrypted form in ZooKeeper rather than in cleartext
> in the broker properties file.
>
> ** The replication protocol has been improved to avoid log divergence
> between leader and follower during fast leader failover. We have also
> improved resilience of brokers by reducing the memory footprint of
> message down-conversions. By using message chunking, both memory
> usage and memory reference time have been reduced to avoid
> OutOfMemory errors in brokers.
>
> ** Kafka clients are now notified of throttling before any throttling is
> applied
> when quotas are enabled. This enables clients to distinguish between
> network errors and large throttle times when quotas are exceeded.
>
> ** We have added a configuration option for Kafka consumer to avoid
> indefinite blocking in the consumer.
>
> ** We have dropped support for Java 7 and removed the previously
> deprecated Scala producer and consumer.
>
> ** Kafka Connect includes a number of improvements and features.
> KIP-298 enables you to control how errors in connectors, transformations
> and converters are handled by enabling automatic retries and controlling
> the
> number of errors that are tolerated before the connector is stopped. More
> contextual information can be included in the logs to help diagnose
> problems
> and problematic messages consumed by sink connectors can be sent to a
> dead letter queue rather than forcing the connector to stop.
>
> ** KIP-297 adds a new extension point to move secrets out of connector
> configurations and integrate with any external key management system.
> The placeholders in connector configurations are only resolved before
> sending the configuration to the connector, ensuring that secrets are
> stored
> and managed securely in your preferred key management system and
> not exposed over the REST APIs or in log files.
>
> ** We have added a thin Scala wrapper API for our Kafka Streams DSL,
> which provides better type inference and better type safety during compile
> time. Scala users can have less boilerplate in their code, notably
> regarding
> Serdes with new implicit Serdes.
>
> ** Message headers are now supported in the Kafka Streams Processor API,
> allowing users to add and manipulate headers read from the source topics
> and propagate them to the sink topics.
>
> ** Windowed aggregations performance in Kafka Streams has been largely
> improved (sometimes by an order of magnitude) thanks to the new
> single-key-fetch API.
>
> ** We have further improved unit testibility of Kafka Streams with the
> kafka-streams-testutil artifact.
>
>
>
>
>
> All of the changes in this release can be found in the release notes:
>
> https://www.apache.org/dist/kafka/2.0.0/RELEASE_NOTES.html
>
>
>
>
>
> You can download the source and binary release (Scala 2.11 and Scala 2.12)
> from:
>
> https://kafka.apache.org/downloads#2.0.0
> <https://kafka.apache.org/downloads#2.0.0>
>
>
>
>
> ---------------------------------------------------------------------------------------------------
>
>
>
>
>
> Apache Kafka is a distributed streaming platform with four core APIs:
>
>
>
> ** The Producer API allows an application to publish a stream records to
>
> one or more Kafka topics.
>
>
>
> ** The Consumer API allows an application to subscribe to one or more
>
> topics and process the stream of records produced to them.
>
>
>
> ** The Streams API allows an application to act as a stream processor,
>
> consuming an input stream from one or more topics and producing an
>
> output stream to one or more output topics, effectively transforming the
>
> input streams to output streams.
>
>
>
> ** The Connector API allows building and running reusable producers or
>
> consumers that connect Kafka topics to existing applications or data
>
> systems. For example, a connector to a relational database might
>
> capture every change to a table.
>
>
>
>
>
> With these APIs, Kafka can be used for two broad classes of application:
>
>
>
> ** Building real-time streaming data pipelines that reliably get data
>
> between systems or applications.
>
>
>
> ** Building real-time streaming applications that transform or react
>
> to the streams of data.
>
>
>
>
>
>
>
> Apache Kafka is in use at large and small companies worldwide, including
>
> Capital One, Goldman Sachs, ING, LinkedIn, Netflix, Pinterest, Rabobank,
>
> Target, The New York Times, Uber, Yelp, and Zalando, among others.
>
>
>
>
>
>
>
> A big thank you for the following 131 contributors to this release!
>
>
>
> Adem Efe Gencer, Alex D, Alex Dunayevsky, Allen Wang, Andras Beni,
>
> Andy Bryant, Andy Coates, Anna Povzner, Arjun Satish, asutosh936,
>
> Attila Sasvari, bartdevylder, Benedict Jin, Bill Bejeck, Blake Miller,
>
> Boyang Chen, cburroughs, Chia-Ping Tsai, Chris Egerton, Colin P. Mccabe,
>
> Colin Patrick McCabe, ConcurrencyPractitioner, Damian Guy, dan norwood,
>
> Daniel Shuy, Daniel Wojda, Dark, David Glasser, Debasish Ghosh, Detharon,
>
> Dhruvil Shah, Dmitry Minkovsky, Dong Lin, Edoardo Comar, emmanuel Harel,
>
> Eugene Sevastyanov, Ewen Cheslack-Postava, Fedor Bobin, fedosov-alexander,
>
> Filipe Agapito, Florian Hussonnois, fredfp, Gilles Degols, gitlw, Gitomain,
>
> Guangxian, Gunju Ko, Gunnar Morling, Guozhang Wang, hmcl, huxi, huxihx,
>
> Igor Kostiakov, Ismael Juma, Jacek Laskowski, Jagadesh Adireddi,
>
> Jarek Rudzinski, Jason Gustafson, Jeff Klukas, Jeremy Custenborder,
>
> Jiangjie (Becket) Qin, Jiangjie Qin, JieFang.He, Jimin Hsieh, Joan Goyeau,
>
> Joel Hamill, John Roesler, Jon Lee, Jorge Quilcate Otoya, Jun Rao,
>
> Kamal C, khairy, Koen De Groote, Konstantine Karantasis, Lee Dongjin,
>
> Liju John, Liquan Pei, lisa2lisa, Lucas Wang, Magesh Nandakumar,
>
> Magnus Edenhill, Magnus Reftel, Manikumar Reddy, Manikumar Reddy O,
>
> manjuapu, Mats Julian Olsen, Matthias J. Sax, Max Zheng, maytals,
>
> Michael Arndt, Michael G. Noll, Mickael Maison, nafshartous, Nick Travers,
>
> nixsticks, Paolo Patierno, parafiend, Patrik Erdes, Radai Rosenblatt,
>
> Rajini Sivaram, Randall Hauch, ro7m, Robert Yokota, Roman Khlebnov,
>
> Ron Dagostino, Sandor Murakozi, Sasaki Toru, Sean Glover,
>
> Sebastian Bauersfeld, Siva Santhalingam, Stanislav Kozlovski, Stephane
> Maarek,
>
> Stuart Perks, Surabhi Dixit, Sönke Liebau, taekyung, tedyu, Thomas Leplus,
>
> UVN, Vahid Hashemian, Valentino Proietti, Viktor Somogyi, Vitaly Pushkar,
>
> Wladimir Schmidt, wushujames, Xavier Léauté, xin, yaphet,
>
> Yaswanth Kumar, ying-zheng, Yu
>
>
>
>
>
>
>
> We welcome your help and feedback. For more information on how to
>
> report problems, and to get involved, visit the project website at
>
> https://kafka.apache.org/
>
>
>
>
>
> Thank you!
>
>
>
>
>
> Regards,
>
>
>
> Rajini
>

Re: [ANNOUNCE] Apache Kafka 2.0.0 Released

[Ismael Juma <is...@gmail.com>]

Thanks to everyone who contributed to the release (including testing and
bug reports)! And thank you Rajini for managing the release.

Ismael

On Mon, 30 Jul 2018, 03:25 Rajini Sivaram, <rs...@apache.org> wrote:

> The Apache Kafka community is pleased to announce the release for
>
> Apache Kafka 2.0.0.
>
>
>
>
>
> This is a major release and includes significant new features from
>
> 40 KIPs. It contains fixes and improvements from 246 JIRAs, including
>
> a few critical bugs. Here is a summary of some notable changes:
>
> ** KIP-290 adds support for prefixed ACLs, simplifying access control
> management in large secure deployments. Bulk access to topics,
> consumer groups or transactional ids with a prefix can now be granted
> using a single rule. Access control for topic creation has also been
> improved to enable access to be granted to create specific topics or
> topics with a prefix.
>
> ** KIP-255 adds a framework for authenticating to Kafka brokers using
> OAuth2 bearer tokens. The SASL/OAUTHBEARER implementation is
> customizable using callbacks for token retrieval and validation.
>
> **Host name verification is now enabled by default for SSL connections
> to ensure that the default SSL configuration is not susceptible to
> man-in-the middle attacks. You can disable this verification for
> deployments where validation is performed using other mechanisms.
>
> ** You can now dynamically update SSL trust stores without broker restart.
> You can also configure security for broker listeners in ZooKeeper before
> starting brokers, including SSL key store and trust store passwords and
> JAAS configuration for SASL. With this new feature, you can store sensitive
> password configs in encrypted form in ZooKeeper rather than in cleartext
> in the broker properties file.
>
> ** The replication protocol has been improved to avoid log divergence
> between leader and follower during fast leader failover. We have also
> improved resilience of brokers by reducing the memory footprint of
> message down-conversions. By using message chunking, both memory
> usage and memory reference time have been reduced to avoid
> OutOfMemory errors in brokers.
>
> ** Kafka clients are now notified of throttling before any throttling is
> applied
> when quotas are enabled. This enables clients to distinguish between
> network errors and large throttle times when quotas are exceeded.
>
> ** We have added a configuration option for Kafka consumer to avoid
> indefinite blocking in the consumer.
>
> ** We have dropped support for Java 7 and removed the previously
> deprecated Scala producer and consumer.
>
> ** Kafka Connect includes a number of improvements and features.
> KIP-298 enables you to control how errors in connectors, transformations
> and converters are handled by enabling automatic retries and controlling
> the
> number of errors that are tolerated before the connector is stopped. More
> contextual information can be included in the logs to help diagnose
> problems
> and problematic messages consumed by sink connectors can be sent to a
> dead letter queue rather than forcing the connector to stop.
>
> ** KIP-297 adds a new extension point to move secrets out of connector
> configurations and integrate with any external key management system.
> The placeholders in connector configurations are only resolved before
> sending the configuration to the connector, ensuring that secrets are
> stored
> and managed securely in your preferred key management system and
> not exposed over the REST APIs or in log files.
>
> ** We have added a thin Scala wrapper API for our Kafka Streams DSL,
> which provides better type inference and better type safety during compile
> time. Scala users can have less boilerplate in their code, notably
> regarding
> Serdes with new implicit Serdes.
>
> ** Message headers are now supported in the Kafka Streams Processor API,
> allowing users to add and manipulate headers read from the source topics
> and propagate them to the sink topics.
>
> ** Windowed aggregations performance in Kafka Streams has been largely
> improved (sometimes by an order of magnitude) thanks to the new
> single-key-fetch API.
>
> ** We have further improved unit testibility of Kafka Streams with the
> kafka-streams-testutil artifact.
>
>
>
>
>
> All of the changes in this release can be found in the release notes:
>
> https://www.apache.org/dist/kafka/2.0.0/RELEASE_NOTES.html
>
>
>
>
>
> You can download the source and binary release (Scala 2.11 and Scala 2.12)
> from:
>
> https://kafka.apache.org/downloads#2.0.0
> <https://kafka.apache.org/downloads#2.0.0>
>
>
>
>
> ---------------------------------------------------------------------------------------------------
>
>
>
>
>
> Apache Kafka is a distributed streaming platform with four core APIs:
>
>
>
> ** The Producer API allows an application to publish a stream records to
>
> one or more Kafka topics.
>
>
>
> ** The Consumer API allows an application to subscribe to one or more
>
> topics and process the stream of records produced to them.
>
>
>
> ** The Streams API allows an application to act as a stream processor,
>
> consuming an input stream from one or more topics and producing an
>
> output stream to one or more output topics, effectively transforming the
>
> input streams to output streams.
>
>
>
> ** The Connector API allows building and running reusable producers or
>
> consumers that connect Kafka topics to existing applications or data
>
> systems. For example, a connector to a relational database might
>
> capture every change to a table.
>
>
>
>
>
> With these APIs, Kafka can be used for two broad classes of application:
>
>
>
> ** Building real-time streaming data pipelines that reliably get data
>
> between systems or applications.
>
>
>
> ** Building real-time streaming applications that transform or react
>
> to the streams of data.
>
>
>
>
>
>
>
> Apache Kafka is in use at large and small companies worldwide, including
>
> Capital One, Goldman Sachs, ING, LinkedIn, Netflix, Pinterest, Rabobank,
>
> Target, The New York Times, Uber, Yelp, and Zalando, among others.
>
>
>
>
>
>
>
> A big thank you for the following 131 contributors to this release!
>
>
>
> Adem Efe Gencer, Alex D, Alex Dunayevsky, Allen Wang, Andras Beni,
>
> Andy Bryant, Andy Coates, Anna Povzner, Arjun Satish, asutosh936,
>
> Attila Sasvari, bartdevylder, Benedict Jin, Bill Bejeck, Blake Miller,
>
> Boyang Chen, cburroughs, Chia-Ping Tsai, Chris Egerton, Colin P. Mccabe,
>
> Colin Patrick McCabe, ConcurrencyPractitioner, Damian Guy, dan norwood,
>
> Daniel Shuy, Daniel Wojda, Dark, David Glasser, Debasish Ghosh, Detharon,
>
> Dhruvil Shah, Dmitry Minkovsky, Dong Lin, Edoardo Comar, emmanuel Harel,
>
> Eugene Sevastyanov, Ewen Cheslack-Postava, Fedor Bobin, fedosov-alexander,
>
> Filipe Agapito, Florian Hussonnois, fredfp, Gilles Degols, gitlw, Gitomain,
>
> Guangxian, Gunju Ko, Gunnar Morling, Guozhang Wang, hmcl, huxi, huxihx,
>
> Igor Kostiakov, Ismael Juma, Jacek Laskowski, Jagadesh Adireddi,
>
> Jarek Rudzinski, Jason Gustafson, Jeff Klukas, Jeremy Custenborder,
>
> Jiangjie (Becket) Qin, Jiangjie Qin, JieFang.He, Jimin Hsieh, Joan Goyeau,
>
> Joel Hamill, John Roesler, Jon Lee, Jorge Quilcate Otoya, Jun Rao,
>
> Kamal C, khairy, Koen De Groote, Konstantine Karantasis, Lee Dongjin,
>
> Liju John, Liquan Pei, lisa2lisa, Lucas Wang, Magesh Nandakumar,
>
> Magnus Edenhill, Magnus Reftel, Manikumar Reddy, Manikumar Reddy O,
>
> manjuapu, Mats Julian Olsen, Matthias J. Sax, Max Zheng, maytals,
>
> Michael Arndt, Michael G. Noll, Mickael Maison, nafshartous, Nick Travers,
>
> nixsticks, Paolo Patierno, parafiend, Patrik Erdes, Radai Rosenblatt,
>
> Rajini Sivaram, Randall Hauch, ro7m, Robert Yokota, Roman Khlebnov,
>
> Ron Dagostino, Sandor Murakozi, Sasaki Toru, Sean Glover,
>
> Sebastian Bauersfeld, Siva Santhalingam, Stanislav Kozlovski, Stephane
> Maarek,
>
> Stuart Perks, Surabhi Dixit, Sönke Liebau, taekyung, tedyu, Thomas Leplus,
>
> UVN, Vahid Hashemian, Valentino Proietti, Viktor Somogyi, Vitaly Pushkar,
>
> Wladimir Schmidt, wushujames, Xavier Léauté, xin, yaphet,
>
> Yaswanth Kumar, ying-zheng, Yu
>
>
>
>
>
>
>
> We welcome your help and feedback. For more information on how to
>
> report problems, and to get involved, visit the project website at
>
> https://kafka.apache.org/
>
>
>
>
>
> Thank you!
>
>
>
>
>
> Regards,
>
>
>
> Rajini
>

Re: [ANNOUNCE] Apache Kafka 2.0.0 Released

[James Cheng <wu...@gmail.com>]

Congrats and great job, everyone! Thanks Rajini for driving the release!

-James

Sent from my iPhone

> On Jul 30, 2018, at 3:25 AM, Rajini Sivaram <rs...@apache.org> wrote:
> 
> The Apache Kafka community is pleased to announce the release for
> 
> Apache Kafka 2.0.0.
> 
> 
> 
> 
> 
> This is a major release and includes significant new features from
> 
> 40 KIPs. It contains fixes and improvements from 246 JIRAs, including
> 
> a few critical bugs. Here is a summary of some notable changes:
> 
> ** KIP-290 adds support for prefixed ACLs, simplifying access control
> management in large secure deployments. Bulk access to topics,
> consumer groups or transactional ids with a prefix can now be granted
> using a single rule. Access control for topic creation has also been
> improved to enable access to be granted to create specific topics or
> topics with a prefix.
> 
> ** KIP-255 adds a framework for authenticating to Kafka brokers using
> OAuth2 bearer tokens. The SASL/OAUTHBEARER implementation is
> customizable using callbacks for token retrieval and validation.
> 
> **Host name verification is now enabled by default for SSL connections
> to ensure that the default SSL configuration is not susceptible to
> man-in-the middle attacks. You can disable this verification for
> deployments where validation is performed using other mechanisms.
> 
> ** You can now dynamically update SSL trust stores without broker restart.
> You can also configure security for broker listeners in ZooKeeper before
> starting brokers, including SSL key store and trust store passwords and
> JAAS configuration for SASL. With this new feature, you can store sensitive
> password configs in encrypted form in ZooKeeper rather than in cleartext
> in the broker properties file.
> 
> ** The replication protocol has been improved to avoid log divergence
> between leader and follower during fast leader failover. We have also
> improved resilience of brokers by reducing the memory footprint of
> message down-conversions. By using message chunking, both memory
> usage and memory reference time have been reduced to avoid
> OutOfMemory errors in brokers.
> 
> ** Kafka clients are now notified of throttling before any throttling is
> applied
> when quotas are enabled. This enables clients to distinguish between
> network errors and large throttle times when quotas are exceeded.
> 
> ** We have added a configuration option for Kafka consumer to avoid
> indefinite blocking in the consumer.
> 
> ** We have dropped support for Java 7 and removed the previously
> deprecated Scala producer and consumer.
> 
> ** Kafka Connect includes a number of improvements and features.
> KIP-298 enables you to control how errors in connectors, transformations
> and converters are handled by enabling automatic retries and controlling the
> number of errors that are tolerated before the connector is stopped. More
> contextual information can be included in the logs to help diagnose problems
> and problematic messages consumed by sink connectors can be sent to a
> dead letter queue rather than forcing the connector to stop.
> 
> ** KIP-297 adds a new extension point to move secrets out of connector
> configurations and integrate with any external key management system.
> The placeholders in connector configurations are only resolved before
> sending the configuration to the connector, ensuring that secrets are stored
> and managed securely in your preferred key management system and
> not exposed over the REST APIs or in log files.
> 
> ** We have added a thin Scala wrapper API for our Kafka Streams DSL,
> which provides better type inference and better type safety during compile
> time. Scala users can have less boilerplate in their code, notably regarding
> Serdes with new implicit Serdes.
> 
> ** Message headers are now supported in the Kafka Streams Processor API,
> allowing users to add and manipulate headers read from the source topics
> and propagate them to the sink topics.
> 
> ** Windowed aggregations performance in Kafka Streams has been largely
> improved (sometimes by an order of magnitude) thanks to the new
> single-key-fetch API.
> 
> ** We have further improved unit testibility of Kafka Streams with the
> kafka-streams-testutil artifact.
> 
> 
> 
> 
> 
> All of the changes in this release can be found in the release notes:
> 
> https://www.apache.org/dist/kafka/2.0.0/RELEASE_NOTES.html
> 
> 
> 
> 
> 
> You can download the source and binary release (Scala 2.11 and Scala 2.12)
> from:
> 
> https://kafka.apache.org/downloads#2.0.0
> <https://kafka.apache.org/downloads#2.0.0>
> 
> 
> 
> ---------------------------------------------------------------------------------------------------
> 
> 
> 
> 
> 
> Apache Kafka is a distributed streaming platform with four core APIs:
> 
> 
> 
> ** The Producer API allows an application to publish a stream records to
> 
> one or more Kafka topics.
> 
> 
> 
> ** The Consumer API allows an application to subscribe to one or more
> 
> topics and process the stream of records produced to them.
> 
> 
> 
> ** The Streams API allows an application to act as a stream processor,
> 
> consuming an input stream from one or more topics and producing an
> 
> output stream to one or more output topics, effectively transforming the
> 
> input streams to output streams.
> 
> 
> 
> ** The Connector API allows building and running reusable producers or
> 
> consumers that connect Kafka topics to existing applications or data
> 
> systems. For example, a connector to a relational database might
> 
> capture every change to a table.
> 
> 
> 
> 
> 
> With these APIs, Kafka can be used for two broad classes of application:
> 
> 
> 
> ** Building real-time streaming data pipelines that reliably get data
> 
> between systems or applications.
> 
> 
> 
> ** Building real-time streaming applications that transform or react
> 
> to the streams of data.
> 
> 
> 
> 
> 
> 
> 
> Apache Kafka is in use at large and small companies worldwide, including
> 
> Capital One, Goldman Sachs, ING, LinkedIn, Netflix, Pinterest, Rabobank,
> 
> Target, The New York Times, Uber, Yelp, and Zalando, among others.
> 
> 
> 
> 
> 
> 
> 
> A big thank you for the following 131 contributors to this release!
> 
> 
> 
> Adem Efe Gencer, Alex D, Alex Dunayevsky, Allen Wang, Andras Beni,
> 
> Andy Bryant, Andy Coates, Anna Povzner, Arjun Satish, asutosh936,
> 
> Attila Sasvari, bartdevylder, Benedict Jin, Bill Bejeck, Blake Miller,
> 
> Boyang Chen, cburroughs, Chia-Ping Tsai, Chris Egerton, Colin P. Mccabe,
> 
> Colin Patrick McCabe, ConcurrencyPractitioner, Damian Guy, dan norwood,
> 
> Daniel Shuy, Daniel Wojda, Dark, David Glasser, Debasish Ghosh, Detharon,
> 
> Dhruvil Shah, Dmitry Minkovsky, Dong Lin, Edoardo Comar, emmanuel Harel,
> 
> Eugene Sevastyanov, Ewen Cheslack-Postava, Fedor Bobin, fedosov-alexander,
> 
> Filipe Agapito, Florian Hussonnois, fredfp, Gilles Degols, gitlw, Gitomain,
> 
> Guangxian, Gunju Ko, Gunnar Morling, Guozhang Wang, hmcl, huxi, huxihx,
> 
> Igor Kostiakov, Ismael Juma, Jacek Laskowski, Jagadesh Adireddi,
> 
> Jarek Rudzinski, Jason Gustafson, Jeff Klukas, Jeremy Custenborder,
> 
> Jiangjie (Becket) Qin, Jiangjie Qin, JieFang.He, Jimin Hsieh, Joan Goyeau,
> 
> Joel Hamill, John Roesler, Jon Lee, Jorge Quilcate Otoya, Jun Rao,
> 
> Kamal C, khairy, Koen De Groote, Konstantine Karantasis, Lee Dongjin,
> 
> Liju John, Liquan Pei, lisa2lisa, Lucas Wang, Magesh Nandakumar,
> 
> Magnus Edenhill, Magnus Reftel, Manikumar Reddy, Manikumar Reddy O,
> 
> manjuapu, Mats Julian Olsen, Matthias J. Sax, Max Zheng, maytals,
> 
> Michael Arndt, Michael G. Noll, Mickael Maison, nafshartous, Nick Travers,
> 
> nixsticks, Paolo Patierno, parafiend, Patrik Erdes, Radai Rosenblatt,
> 
> Rajini Sivaram, Randall Hauch, ro7m, Robert Yokota, Roman Khlebnov,
> 
> Ron Dagostino, Sandor Murakozi, Sasaki Toru, Sean Glover,
> 
> Sebastian Bauersfeld, Siva Santhalingam, Stanislav Kozlovski, Stephane
> Maarek,
> 
> Stuart Perks, Surabhi Dixit, Sönke Liebau, taekyung, tedyu, Thomas Leplus,
> 
> UVN, Vahid Hashemian, Valentino Proietti, Viktor Somogyi, Vitaly Pushkar,
> 
> Wladimir Schmidt, wushujames, Xavier Léauté, xin, yaphet,
> 
> Yaswanth Kumar, ying-zheng, Yu
> 
> 
> 
> 
> 
> 
> 
> We welcome your help and feedback. For more information on how to
> 
> report problems, and to get involved, visit the project website at
> 
> https://kafka.apache.org/
> 
> 
> 
> 
> 
> Thank you!
> 
> 
> 
> 
> 
> Regards,
> 
> 
> 
> Rajini

Re: [ANNOUNCE] Apache Kafka 2.0.0 Released

[James Cheng <wu...@gmail.com>]

Congrats and great job, everyone! Thanks Rajini for driving the release!

-James

Sent from my iPhone

> On Jul 30, 2018, at 3:25 AM, Rajini Sivaram <rs...@apache.org> wrote:
> 
> The Apache Kafka community is pleased to announce the release for
> 
> Apache Kafka 2.0.0.
> 
> 
> 
> 
> 
> This is a major release and includes significant new features from
> 
> 40 KIPs. It contains fixes and improvements from 246 JIRAs, including
> 
> a few critical bugs. Here is a summary of some notable changes:
> 
> ** KIP-290 adds support for prefixed ACLs, simplifying access control
> management in large secure deployments. Bulk access to topics,
> consumer groups or transactional ids with a prefix can now be granted
> using a single rule. Access control for topic creation has also been
> improved to enable access to be granted to create specific topics or
> topics with a prefix.
> 
> ** KIP-255 adds a framework for authenticating to Kafka brokers using
> OAuth2 bearer tokens. The SASL/OAUTHBEARER implementation is
> customizable using callbacks for token retrieval and validation.
> 
> **Host name verification is now enabled by default for SSL connections
> to ensure that the default SSL configuration is not susceptible to
> man-in-the middle attacks. You can disable this verification for
> deployments where validation is performed using other mechanisms.
> 
> ** You can now dynamically update SSL trust stores without broker restart.
> You can also configure security for broker listeners in ZooKeeper before
> starting brokers, including SSL key store and trust store passwords and
> JAAS configuration for SASL. With this new feature, you can store sensitive
> password configs in encrypted form in ZooKeeper rather than in cleartext
> in the broker properties file.
> 
> ** The replication protocol has been improved to avoid log divergence
> between leader and follower during fast leader failover. We have also
> improved resilience of brokers by reducing the memory footprint of
> message down-conversions. By using message chunking, both memory
> usage and memory reference time have been reduced to avoid
> OutOfMemory errors in brokers.
> 
> ** Kafka clients are now notified of throttling before any throttling is
> applied
> when quotas are enabled. This enables clients to distinguish between
> network errors and large throttle times when quotas are exceeded.
> 
> ** We have added a configuration option for Kafka consumer to avoid
> indefinite blocking in the consumer.
> 
> ** We have dropped support for Java 7 and removed the previously
> deprecated Scala producer and consumer.
> 
> ** Kafka Connect includes a number of improvements and features.
> KIP-298 enables you to control how errors in connectors, transformations
> and converters are handled by enabling automatic retries and controlling the
> number of errors that are tolerated before the connector is stopped. More
> contextual information can be included in the logs to help diagnose problems
> and problematic messages consumed by sink connectors can be sent to a
> dead letter queue rather than forcing the connector to stop.
> 
> ** KIP-297 adds a new extension point to move secrets out of connector
> configurations and integrate with any external key management system.
> The placeholders in connector configurations are only resolved before
> sending the configuration to the connector, ensuring that secrets are stored
> and managed securely in your preferred key management system and
> not exposed over the REST APIs or in log files.
> 
> ** We have added a thin Scala wrapper API for our Kafka Streams DSL,
> which provides better type inference and better type safety during compile
> time. Scala users can have less boilerplate in their code, notably regarding
> Serdes with new implicit Serdes.
> 
> ** Message headers are now supported in the Kafka Streams Processor API,
> allowing users to add and manipulate headers read from the source topics
> and propagate them to the sink topics.
> 
> ** Windowed aggregations performance in Kafka Streams has been largely
> improved (sometimes by an order of magnitude) thanks to the new
> single-key-fetch API.
> 
> ** We have further improved unit testibility of Kafka Streams with the
> kafka-streams-testutil artifact.
> 
> 
> 
> 
> 
> All of the changes in this release can be found in the release notes:
> 
> https://www.apache.org/dist/kafka/2.0.0/RELEASE_NOTES.html
> 
> 
> 
> 
> 
> You can download the source and binary release (Scala 2.11 and Scala 2.12)
> from:
> 
> https://kafka.apache.org/downloads#2.0.0
> <https://kafka.apache.org/downloads#2.0.0>
> 
> 
> 
> ---------------------------------------------------------------------------------------------------
> 
> 
> 
> 
> 
> Apache Kafka is a distributed streaming platform with four core APIs:
> 
> 
> 
> ** The Producer API allows an application to publish a stream records to
> 
> one or more Kafka topics.
> 
> 
> 
> ** The Consumer API allows an application to subscribe to one or more
> 
> topics and process the stream of records produced to them.
> 
> 
> 
> ** The Streams API allows an application to act as a stream processor,
> 
> consuming an input stream from one or more topics and producing an
> 
> output stream to one or more output topics, effectively transforming the
> 
> input streams to output streams.
> 
> 
> 
> ** The Connector API allows building and running reusable producers or
> 
> consumers that connect Kafka topics to existing applications or data
> 
> systems. For example, a connector to a relational database might
> 
> capture every change to a table.
> 
> 
> 
> 
> 
> With these APIs, Kafka can be used for two broad classes of application:
> 
> 
> 
> ** Building real-time streaming data pipelines that reliably get data
> 
> between systems or applications.
> 
> 
> 
> ** Building real-time streaming applications that transform or react
> 
> to the streams of data.
> 
> 
> 
> 
> 
> 
> 
> Apache Kafka is in use at large and small companies worldwide, including
> 
> Capital One, Goldman Sachs, ING, LinkedIn, Netflix, Pinterest, Rabobank,
> 
> Target, The New York Times, Uber, Yelp, and Zalando, among others.
> 
> 
> 
> 
> 
> 
> 
> A big thank you for the following 131 contributors to this release!
> 
> 
> 
> Adem Efe Gencer, Alex D, Alex Dunayevsky, Allen Wang, Andras Beni,
> 
> Andy Bryant, Andy Coates, Anna Povzner, Arjun Satish, asutosh936,
> 
> Attila Sasvari, bartdevylder, Benedict Jin, Bill Bejeck, Blake Miller,
> 
> Boyang Chen, cburroughs, Chia-Ping Tsai, Chris Egerton, Colin P. Mccabe,
> 
> Colin Patrick McCabe, ConcurrencyPractitioner, Damian Guy, dan norwood,
> 
> Daniel Shuy, Daniel Wojda, Dark, David Glasser, Debasish Ghosh, Detharon,
> 
> Dhruvil Shah, Dmitry Minkovsky, Dong Lin, Edoardo Comar, emmanuel Harel,
> 
> Eugene Sevastyanov, Ewen Cheslack-Postava, Fedor Bobin, fedosov-alexander,
> 
> Filipe Agapito, Florian Hussonnois, fredfp, Gilles Degols, gitlw, Gitomain,
> 
> Guangxian, Gunju Ko, Gunnar Morling, Guozhang Wang, hmcl, huxi, huxihx,
> 
> Igor Kostiakov, Ismael Juma, Jacek Laskowski, Jagadesh Adireddi,
> 
> Jarek Rudzinski, Jason Gustafson, Jeff Klukas, Jeremy Custenborder,
> 
> Jiangjie (Becket) Qin, Jiangjie Qin, JieFang.He, Jimin Hsieh, Joan Goyeau,
> 
> Joel Hamill, John Roesler, Jon Lee, Jorge Quilcate Otoya, Jun Rao,
> 
> Kamal C, khairy, Koen De Groote, Konstantine Karantasis, Lee Dongjin,
> 
> Liju John, Liquan Pei, lisa2lisa, Lucas Wang, Magesh Nandakumar,
> 
> Magnus Edenhill, Magnus Reftel, Manikumar Reddy, Manikumar Reddy O,
> 
> manjuapu, Mats Julian Olsen, Matthias J. Sax, Max Zheng, maytals,
> 
> Michael Arndt, Michael G. Noll, Mickael Maison, nafshartous, Nick Travers,
> 
> nixsticks, Paolo Patierno, parafiend, Patrik Erdes, Radai Rosenblatt,
> 
> Rajini Sivaram, Randall Hauch, ro7m, Robert Yokota, Roman Khlebnov,
> 
> Ron Dagostino, Sandor Murakozi, Sasaki Toru, Sean Glover,
> 
> Sebastian Bauersfeld, Siva Santhalingam, Stanislav Kozlovski, Stephane
> Maarek,
> 
> Stuart Perks, Surabhi Dixit, Sönke Liebau, taekyung, tedyu, Thomas Leplus,
> 
> UVN, Vahid Hashemian, Valentino Proietti, Viktor Somogyi, Vitaly Pushkar,
> 
> Wladimir Schmidt, wushujames, Xavier Léauté, xin, yaphet,
> 
> Yaswanth Kumar, ying-zheng, Yu
> 
> 
> 
> 
> 
> 
> 
> We welcome your help and feedback. For more information on how to
> 
> report problems, and to get involved, visit the project website at
> 
> https://kafka.apache.org/
> 
> 
> 
> 
> 
> Thank you!
> 
> 
> 
> 
> 
> Regards,
> 
> 
> 
> Rajini