You are viewing a plain text version of this content. The canonical link for it is here.
Posted to announce@apache.org by David Jacot <da...@apache.org> on 2022/01/24 17:03:56 UTC
[ANNOUNCE] Apache Kafka 3.1.0
The Apache Kafka community is pleased to announce the release for
Apache Kafka 3.1.0.
It is a major release that includes many new features, including:
* Apache Kafka supports Java 17
* The FetchRequest supports Topic IDs (KIP-516)
* Extend SASL/OAUTHBEARER with support for OIDC (KIP-768)
* Add broker count metrics (KIP-748)
* Differentiate consistently metric latency measured in millis and
nanos (KIP-773)
* The eager rebalance protocol is deprecated (KAFKA-13439)
* Add TaskId field to StreamsException (KIP-783)
* Custom partitioners in foreign-key joins (KIP-775)
* Fetch/findSessions queries with open endpoints for
SessionStore/WindowStore (KIP-766)
* Range queries with open endpoints (KIP-763)
* Add total blocked time metric to Streams (KIP-761)
* Add additional configuration to control MirrorMaker2 internal topics
naming convention (KIP-690)
You may read a more detailed list of features in the 3.1.0 blog post:
https://blogs.apache.org/kafka/
All of the changes in this release can be found in the release notes:
https://www.apache.org/dist/kafka/3.1.0/RELEASE_NOTES.html
You can download the source and binary release (Scala 2.12 and 2.13) from:
https://kafka.apache.org/downloads#3.1.0
---------------------------------------------------------------------------------------------------
Apache Kafka is a distributed streaming platform with four core APIs:
** The Producer API allows an application to publish a stream of records to
one or more Kafka topics.
** The Consumer API allows an application to subscribe to one or more
topics and process the stream of records produced to them.
** The Streams API allows an application to act as a stream processor,
consuming an input stream from one or more topics and producing an
output stream to one or more output topics, effectively transforming the
input streams to output streams.
** The Connector API allows building and running reusable producers or
consumers that connect Kafka topics to existing applications or data
systems. For example, a connector to a relational database might
capture every change to a table.
With these APIs, Kafka can be used for two broad classes of application:
** Building real-time streaming data pipelines that reliably get data
between systems or applications.
** Building real-time streaming applications that transform or react
to the streams of data.
Apache Kafka is in use at large and small companies worldwide, including
Capital One, Goldman Sachs, ING, LinkedIn, Netflix, Pinterest, Rabobank,
Target, The New York Times, Uber, Yelp, and Zalando, among others.
A big thank you for the following 114 contributors to this release!
A. Sophie Blee-Goldman, Alexander Iskuskov, Alexander Stohr, Almog
Gavra, Andras Katona, Andrew Patterson, Andy Chambers, Andy Lapidas,
Anna Sophie Blee-Goldman, Antony Stubbs, Arjun Satish, Bill Bejeck,
Boyang Chen, Bruno Cadonna, CHUN-HAO TANG, Cheng Tan, Chia-Ping Tsai,
Chris Egerton, Christo Lolov, Colin P. McCabe, Cong Ding, Daniel
Urban, David Arthur, David Jacot, David Mao, Dmitriy Fishman, Edoardo
Comar, Ewen Cheslack-Postava, Greg Harris, Guozhang Wang, Igor Soarez,
Ismael Juma, Israel Ekpo, Ivan Ponomarev, Jakub Scholz, James Galasyn,
Jason Gustafson, Jeff Kim, Jim Galasyn, JoeCqupt, Joel Hamill, John
Gray, John Roesler, Jongho Jeon, Jorge Esteban Quilcate Otoya, Jose
Sancio, Josep Prat, José Armando García Sancio, Jun Rao, Justine
Olshan, Kalpesh Patel, Kamal Chandraprakash, Kevin Zhang, Kirk True,
Konstantine Karantasis, Kowshik Prakasam, Leah Thomas, Lee Dongjin,
Lucas Bradstreet, Luke Chen, Manikumar Reddy, Matthew Wong, Matthias
J. Sax, Michael Carter, Mickael Maison, Nigel Liang, Niket, Niket
Goel, Oliver Hutchison, Omnia G H Ibrahim, Patrick Stuedi, Phil
Hardwick, Prateek Agarwal, Rajini Sivaram, Randall Hauch, René Kerner,
Richard Yu, Rohan, Ron Dagostino, Ryan Dielhenn, Sanjana Kaundinya,
Satish Duggana, Sergio Peña, Sherzod Mamadaliev, Stanislav Vodetskyi,
Ted Yu, Tom Bentley, Tomas Forsman, Tomer Wizman, Uwe Eisele, Victoria
Xia, Viktor Somogyi-Vass, Vincent Jiang, Walker Carlson, Weisheng
Yang, Xavier Léauté, Yanwen(Jason) Lin, Yi Ding, Zara Lim, andy0x01,
dengziming, feyman2016, ik, ik.lim, jem, jiangyuan, kpatelatwork,
leah, loboya~, lujiefsi, sebbASF, singingMan, vamossagar12,
wenbingshen
We welcome your help and feedback. For more information on how to
report problems, and to get involved, visit the project website at
https://kafka.apache.org/
Thank you!
Regards,
David
Re: [ANNOUNCE] Apache Kafka 3.1.0
Posted by Israel Ekpo <is...@gmail.com>.
Deepti,
KIP-653 missed the cut for 3.1 so it was not part of this release
https://cwiki.apache.org/confluence/display/KAFKA/Release+Plan+3.1.0
https://cwiki.apache.org/confluence/display/KAFKA/KIP-653%3A+Upgrade+log4j+to+log4j2
Israel Ekpo
Lead Instructor, IzzyAcademy.com
https://www.youtube.com/c/izzyacademy
https://izzyacademy.com/
On Mon, Jan 24, 2022 at 12:07 PM Deepti Sharma S
<de...@ericsson.com.invalid> wrote:
> Congratulations!
>
> One thing would like to confirm, if we have upgraded the Log4J version
> from 1.x to 2.x in this release?
>
>
> Regards,
> Deepti Sharma
> PMP® & ITIL
>
>
> -----Original Message-----
> From: David Jacot <da...@apache.org>
> Sent: Monday, January 24, 2022 10:34 PM
> To: announce@apache.org; dev@kafka.apache.org; users@kafka.apache.org;
> kafka-clients@googlegroups.com
> Subject: [ANNOUNCE] Apache Kafka 3.1.0
>
> The Apache Kafka community is pleased to announce the release for Apache
> Kafka 3.1.0.
>
> It is a major release that includes many new features, including:
>
> * Apache Kafka supports Java 17
> * The FetchRequest supports Topic IDs (KIP-516)
> * Extend SASL/OAUTHBEARER with support for OIDC (KIP-768)
> * Add broker count metrics (KIP-748)
> * Differentiate consistently metric latency measured in millis and nanos
> (KIP-773)
> * The eager rebalance protocol is deprecated (KAFKA-13439)
> * Add TaskId field to StreamsException (KIP-783)
> * Custom partitioners in foreign-key joins (KIP-775)
> * Fetch/findSessions queries with open endpoints for
> SessionStore/WindowStore (KIP-766)
> * Range queries with open endpoints (KIP-763)
> * Add total blocked time metric to Streams (KIP-761)
> * Add additional configuration to control MirrorMaker2 internal topics
> naming convention (KIP-690)
>
> You may read a more detailed list of features in the 3.1.0 blog post:
> https://blogs.apache.org/kafka/
>
> All of the changes in this release can be found in the release notes:
> https://www.apache.org/dist/kafka/3.1.0/RELEASE_NOTES.html
>
> You can download the source and binary release (Scala 2.12 and 2.13) from:
> https://kafka.apache.org/downloads#3.1.0
>
>
> ---------------------------------------------------------------------------------------------------
>
>
> Apache Kafka is a distributed streaming platform with four core APIs:
>
> ** The Producer API allows an application to publish a stream of records
> to one or more Kafka topics.
>
> ** The Consumer API allows an application to subscribe to one or more
> topics and process the stream of records produced to them.
>
> ** The Streams API allows an application to act as a stream processor,
> consuming an input stream from one or more topics and producing an output
> stream to one or more output topics, effectively transforming the input
> streams to output streams.
>
> ** The Connector API allows building and running reusable producers or
> consumers that connect Kafka topics to existing applications or data
> systems. For example, a connector to a relational database might capture
> every change to a table.
>
>
> With these APIs, Kafka can be used for two broad classes of application:
>
> ** Building real-time streaming data pipelines that reliably get data
> between systems or applications.
>
> ** Building real-time streaming applications that transform or react to
> the streams of data.
>
>
> Apache Kafka is in use at large and small companies worldwide, including
> Capital One, Goldman Sachs, ING, LinkedIn, Netflix, Pinterest, Rabobank,
> Target, The New York Times, Uber, Yelp, and Zalando, among others.
>
> A big thank you for the following 114 contributors to this release!
>
> A. Sophie Blee-Goldman, Alexander Iskuskov, Alexander Stohr, Almog Gavra,
> Andras Katona, Andrew Patterson, Andy Chambers, Andy Lapidas, Anna Sophie
> Blee-Goldman, Antony Stubbs, Arjun Satish, Bill Bejeck, Boyang Chen, Bruno
> Cadonna, CHUN-HAO TANG, Cheng Tan, Chia-Ping Tsai, Chris Egerton, Christo
> Lolov, Colin P. McCabe, Cong Ding, Daniel Urban, David Arthur, David Jacot,
> David Mao, Dmitriy Fishman, Edoardo Comar, Ewen Cheslack-Postava, Greg
> Harris, Guozhang Wang, Igor Soarez, Ismael Juma, Israel Ekpo, Ivan
> Ponomarev, Jakub Scholz, James Galasyn, Jason Gustafson, Jeff Kim, Jim
> Galasyn, JoeCqupt, Joel Hamill, John Gray, John Roesler, Jongho Jeon, Jorge
> Esteban Quilcate Otoya, Jose Sancio, Josep Prat, José Armando García
> Sancio, Jun Rao, Justine Olshan, Kalpesh Patel, Kamal Chandraprakash, Kevin
> Zhang, Kirk True, Konstantine Karantasis, Kowshik Prakasam, Leah Thomas,
> Lee Dongjin, Lucas Bradstreet, Luke Chen, Manikumar Reddy, Matthew Wong,
> Matthias J. Sax, Michael Carter, Mickael Maison, Nigel Liang, Niket, Niket
> Goel, Oliver Hutchison, Omnia G H Ibrahim, Patrick Stuedi, Phil Hardwick,
> Prateek Agarwal, Rajini Sivaram, Randall Hauch, René Kerner, Richard Yu,
> Rohan, Ron Dagostino, Ryan Dielhenn, Sanjana Kaundinya, Satish Duggana,
> Sergio Peña, Sherzod Mamadaliev, Stanislav Vodetskyi, Ted Yu, Tom Bentley,
> Tomas Forsman, Tomer Wizman, Uwe Eisele, Victoria Xia, Viktor Somogyi-Vass,
> Vincent Jiang, Walker Carlson, Weisheng Yang, Xavier Léauté, Yanwen(Jason)
> Lin, Yi Ding, Zara Lim, andy0x01, dengziming, feyman2016, ik, ik.lim, jem,
> jiangyuan, kpatelatwork, leah, loboya~, lujiefsi, sebbASF, singingMan,
> vamossagar12, wenbingshen
>
> We welcome your help and feedback. For more information on how to report
> problems, and to get involved, visit the project website at
> https://kafka.apache.org/
>
> Thank you!
>
>
> Regards,
>
> David
>
Re: [ANNOUNCE] Apache Kafka 3.1.0
Posted by Mickael Maison <mi...@apache.org>.
Thanks David for driving this release!
On Mon, Jan 24, 2022 at 6:07 PM Deepti Sharma S
<de...@ericsson.com.invalid> wrote:
>
> Congratulations!
>
> One thing would like to confirm, if we have upgraded the Log4J version from 1.x to 2.x in this release?
>
>
> Regards,
> Deepti Sharma
> PMP® & ITIL
>
>
> -----Original Message-----
> From: David Jacot <da...@apache.org>
> Sent: Monday, January 24, 2022 10:34 PM
> To: announce@apache.org; dev@kafka.apache.org; users@kafka.apache.org; kafka-clients@googlegroups.com
> Subject: [ANNOUNCE] Apache Kafka 3.1.0
>
> The Apache Kafka community is pleased to announce the release for Apache Kafka 3.1.0.
>
> It is a major release that includes many new features, including:
>
> * Apache Kafka supports Java 17
> * The FetchRequest supports Topic IDs (KIP-516)
> * Extend SASL/OAUTHBEARER with support for OIDC (KIP-768)
> * Add broker count metrics (KIP-748)
> * Differentiate consistently metric latency measured in millis and nanos (KIP-773)
> * The eager rebalance protocol is deprecated (KAFKA-13439)
> * Add TaskId field to StreamsException (KIP-783)
> * Custom partitioners in foreign-key joins (KIP-775)
> * Fetch/findSessions queries with open endpoints for SessionStore/WindowStore (KIP-766)
> * Range queries with open endpoints (KIP-763)
> * Add total blocked time metric to Streams (KIP-761)
> * Add additional configuration to control MirrorMaker2 internal topics naming convention (KIP-690)
>
> You may read a more detailed list of features in the 3.1.0 blog post:
> https://blogs.apache.org/kafka/
>
> All of the changes in this release can be found in the release notes:
> https://www.apache.org/dist/kafka/3.1.0/RELEASE_NOTES.html
>
> You can download the source and binary release (Scala 2.12 and 2.13) from:
> https://kafka.apache.org/downloads#3.1.0
>
> ---------------------------------------------------------------------------------------------------
>
>
> Apache Kafka is a distributed streaming platform with four core APIs:
>
> ** The Producer API allows an application to publish a stream of records to one or more Kafka topics.
>
> ** The Consumer API allows an application to subscribe to one or more topics and process the stream of records produced to them.
>
> ** The Streams API allows an application to act as a stream processor, consuming an input stream from one or more topics and producing an output stream to one or more output topics, effectively transforming the input streams to output streams.
>
> ** The Connector API allows building and running reusable producers or consumers that connect Kafka topics to existing applications or data systems. For example, a connector to a relational database might capture every change to a table.
>
>
> With these APIs, Kafka can be used for two broad classes of application:
>
> ** Building real-time streaming data pipelines that reliably get data between systems or applications.
>
> ** Building real-time streaming applications that transform or react to the streams of data.
>
>
> Apache Kafka is in use at large and small companies worldwide, including Capital One, Goldman Sachs, ING, LinkedIn, Netflix, Pinterest, Rabobank, Target, The New York Times, Uber, Yelp, and Zalando, among others.
>
> A big thank you for the following 114 contributors to this release!
>
> A. Sophie Blee-Goldman, Alexander Iskuskov, Alexander Stohr, Almog Gavra, Andras Katona, Andrew Patterson, Andy Chambers, Andy Lapidas, Anna Sophie Blee-Goldman, Antony Stubbs, Arjun Satish, Bill Bejeck, Boyang Chen, Bruno Cadonna, CHUN-HAO TANG, Cheng Tan, Chia-Ping Tsai, Chris Egerton, Christo Lolov, Colin P. McCabe, Cong Ding, Daniel Urban, David Arthur, David Jacot, David Mao, Dmitriy Fishman, Edoardo Comar, Ewen Cheslack-Postava, Greg Harris, Guozhang Wang, Igor Soarez, Ismael Juma, Israel Ekpo, Ivan Ponomarev, Jakub Scholz, James Galasyn, Jason Gustafson, Jeff Kim, Jim Galasyn, JoeCqupt, Joel Hamill, John Gray, John Roesler, Jongho Jeon, Jorge Esteban Quilcate Otoya, Jose Sancio, Josep Prat, José Armando García Sancio, Jun Rao, Justine Olshan, Kalpesh Patel, Kamal Chandraprakash, Kevin Zhang, Kirk True, Konstantine Karantasis, Kowshik Prakasam, Leah Thomas, Lee Dongjin, Lucas Bradstreet, Luke Chen, Manikumar Reddy, Matthew Wong, Matthias J. Sax, Michael Carter, Mickael Maison, Nigel Liang, Niket, Niket Goel, Oliver Hutchison, Omnia G H Ibrahim, Patrick Stuedi, Phil Hardwick, Prateek Agarwal, Rajini Sivaram, Randall Hauch, René Kerner, Richard Yu, Rohan, Ron Dagostino, Ryan Dielhenn, Sanjana Kaundinya, Satish Duggana, Sergio Peña, Sherzod Mamadaliev, Stanislav Vodetskyi, Ted Yu, Tom Bentley, Tomas Forsman, Tomer Wizman, Uwe Eisele, Victoria Xia, Viktor Somogyi-Vass, Vincent Jiang, Walker Carlson, Weisheng Yang, Xavier Léauté, Yanwen(Jason) Lin, Yi Ding, Zara Lim, andy0x01, dengziming, feyman2016, ik, ik.lim, jem, jiangyuan, kpatelatwork, leah, loboya~, lujiefsi, sebbASF, singingMan, vamossagar12, wenbingshen
>
> We welcome your help and feedback. For more information on how to report problems, and to get involved, visit the project website at https://kafka.apache.org/
>
> Thank you!
>
>
> Regards,
>
> David
Re: [ANNOUNCE] Apache Kafka 3.1.0
Posted by Mickael Maison <mi...@apache.org>.
Thanks David for driving this release!
On Mon, Jan 24, 2022 at 6:07 PM Deepti Sharma S
<de...@ericsson.com.invalid> wrote:
>
> Congratulations!
>
> One thing would like to confirm, if we have upgraded the Log4J version from 1.x to 2.x in this release?
>
>
> Regards,
> Deepti Sharma
> PMP® & ITIL
>
>
> -----Original Message-----
> From: David Jacot <da...@apache.org>
> Sent: Monday, January 24, 2022 10:34 PM
> To: announce@apache.org; dev@kafka.apache.org; users@kafka.apache.org; kafka-clients@googlegroups.com
> Subject: [ANNOUNCE] Apache Kafka 3.1.0
>
> The Apache Kafka community is pleased to announce the release for Apache Kafka 3.1.0.
>
> It is a major release that includes many new features, including:
>
> * Apache Kafka supports Java 17
> * The FetchRequest supports Topic IDs (KIP-516)
> * Extend SASL/OAUTHBEARER with support for OIDC (KIP-768)
> * Add broker count metrics (KIP-748)
> * Differentiate consistently metric latency measured in millis and nanos (KIP-773)
> * The eager rebalance protocol is deprecated (KAFKA-13439)
> * Add TaskId field to StreamsException (KIP-783)
> * Custom partitioners in foreign-key joins (KIP-775)
> * Fetch/findSessions queries with open endpoints for SessionStore/WindowStore (KIP-766)
> * Range queries with open endpoints (KIP-763)
> * Add total blocked time metric to Streams (KIP-761)
> * Add additional configuration to control MirrorMaker2 internal topics naming convention (KIP-690)
>
> You may read a more detailed list of features in the 3.1.0 blog post:
> https://blogs.apache.org/kafka/
>
> All of the changes in this release can be found in the release notes:
> https://www.apache.org/dist/kafka/3.1.0/RELEASE_NOTES.html
>
> You can download the source and binary release (Scala 2.12 and 2.13) from:
> https://kafka.apache.org/downloads#3.1.0
>
> ---------------------------------------------------------------------------------------------------
>
>
> Apache Kafka is a distributed streaming platform with four core APIs:
>
> ** The Producer API allows an application to publish a stream of records to one or more Kafka topics.
>
> ** The Consumer API allows an application to subscribe to one or more topics and process the stream of records produced to them.
>
> ** The Streams API allows an application to act as a stream processor, consuming an input stream from one or more topics and producing an output stream to one or more output topics, effectively transforming the input streams to output streams.
>
> ** The Connector API allows building and running reusable producers or consumers that connect Kafka topics to existing applications or data systems. For example, a connector to a relational database might capture every change to a table.
>
>
> With these APIs, Kafka can be used for two broad classes of application:
>
> ** Building real-time streaming data pipelines that reliably get data between systems or applications.
>
> ** Building real-time streaming applications that transform or react to the streams of data.
>
>
> Apache Kafka is in use at large and small companies worldwide, including Capital One, Goldman Sachs, ING, LinkedIn, Netflix, Pinterest, Rabobank, Target, The New York Times, Uber, Yelp, and Zalando, among others.
>
> A big thank you for the following 114 contributors to this release!
>
> A. Sophie Blee-Goldman, Alexander Iskuskov, Alexander Stohr, Almog Gavra, Andras Katona, Andrew Patterson, Andy Chambers, Andy Lapidas, Anna Sophie Blee-Goldman, Antony Stubbs, Arjun Satish, Bill Bejeck, Boyang Chen, Bruno Cadonna, CHUN-HAO TANG, Cheng Tan, Chia-Ping Tsai, Chris Egerton, Christo Lolov, Colin P. McCabe, Cong Ding, Daniel Urban, David Arthur, David Jacot, David Mao, Dmitriy Fishman, Edoardo Comar, Ewen Cheslack-Postava, Greg Harris, Guozhang Wang, Igor Soarez, Ismael Juma, Israel Ekpo, Ivan Ponomarev, Jakub Scholz, James Galasyn, Jason Gustafson, Jeff Kim, Jim Galasyn, JoeCqupt, Joel Hamill, John Gray, John Roesler, Jongho Jeon, Jorge Esteban Quilcate Otoya, Jose Sancio, Josep Prat, José Armando García Sancio, Jun Rao, Justine Olshan, Kalpesh Patel, Kamal Chandraprakash, Kevin Zhang, Kirk True, Konstantine Karantasis, Kowshik Prakasam, Leah Thomas, Lee Dongjin, Lucas Bradstreet, Luke Chen, Manikumar Reddy, Matthew Wong, Matthias J. Sax, Michael Carter, Mickael Maison, Nigel Liang, Niket, Niket Goel, Oliver Hutchison, Omnia G H Ibrahim, Patrick Stuedi, Phil Hardwick, Prateek Agarwal, Rajini Sivaram, Randall Hauch, René Kerner, Richard Yu, Rohan, Ron Dagostino, Ryan Dielhenn, Sanjana Kaundinya, Satish Duggana, Sergio Peña, Sherzod Mamadaliev, Stanislav Vodetskyi, Ted Yu, Tom Bentley, Tomas Forsman, Tomer Wizman, Uwe Eisele, Victoria Xia, Viktor Somogyi-Vass, Vincent Jiang, Walker Carlson, Weisheng Yang, Xavier Léauté, Yanwen(Jason) Lin, Yi Ding, Zara Lim, andy0x01, dengziming, feyman2016, ik, ik.lim, jem, jiangyuan, kpatelatwork, leah, loboya~, lujiefsi, sebbASF, singingMan, vamossagar12, wenbingshen
>
> We welcome your help and feedback. For more information on how to report problems, and to get involved, visit the project website at https://kafka.apache.org/
>
> Thank you!
>
>
> Regards,
>
> David
RE: [ANNOUNCE] Apache Kafka 3.1.0
Posted by Deepti Sharma S <de...@ericsson.com.INVALID>.
Congratulations!
One thing would like to confirm, if we have upgraded the Log4J version from 1.x to 2.x in this release?
Regards,
Deepti Sharma
PMP® & ITIL
-----Original Message-----
From: David Jacot <da...@apache.org>
Sent: Monday, January 24, 2022 10:34 PM
To: announce@apache.org; dev@kafka.apache.org; users@kafka.apache.org; kafka-clients@googlegroups.com
Subject: [ANNOUNCE] Apache Kafka 3.1.0
The Apache Kafka community is pleased to announce the release for Apache Kafka 3.1.0.
It is a major release that includes many new features, including:
* Apache Kafka supports Java 17
* The FetchRequest supports Topic IDs (KIP-516)
* Extend SASL/OAUTHBEARER with support for OIDC (KIP-768)
* Add broker count metrics (KIP-748)
* Differentiate consistently metric latency measured in millis and nanos (KIP-773)
* The eager rebalance protocol is deprecated (KAFKA-13439)
* Add TaskId field to StreamsException (KIP-783)
* Custom partitioners in foreign-key joins (KIP-775)
* Fetch/findSessions queries with open endpoints for SessionStore/WindowStore (KIP-766)
* Range queries with open endpoints (KIP-763)
* Add total blocked time metric to Streams (KIP-761)
* Add additional configuration to control MirrorMaker2 internal topics naming convention (KIP-690)
You may read a more detailed list of features in the 3.1.0 blog post:
https://blogs.apache.org/kafka/
All of the changes in this release can be found in the release notes:
https://www.apache.org/dist/kafka/3.1.0/RELEASE_NOTES.html
You can download the source and binary release (Scala 2.12 and 2.13) from:
https://kafka.apache.org/downloads#3.1.0
---------------------------------------------------------------------------------------------------
Apache Kafka is a distributed streaming platform with four core APIs:
** The Producer API allows an application to publish a stream of records to one or more Kafka topics.
** The Consumer API allows an application to subscribe to one or more topics and process the stream of records produced to them.
** The Streams API allows an application to act as a stream processor, consuming an input stream from one or more topics and producing an output stream to one or more output topics, effectively transforming the input streams to output streams.
** The Connector API allows building and running reusable producers or consumers that connect Kafka topics to existing applications or data systems. For example, a connector to a relational database might capture every change to a table.
With these APIs, Kafka can be used for two broad classes of application:
** Building real-time streaming data pipelines that reliably get data between systems or applications.
** Building real-time streaming applications that transform or react to the streams of data.
Apache Kafka is in use at large and small companies worldwide, including Capital One, Goldman Sachs, ING, LinkedIn, Netflix, Pinterest, Rabobank, Target, The New York Times, Uber, Yelp, and Zalando, among others.
A big thank you for the following 114 contributors to this release!
A. Sophie Blee-Goldman, Alexander Iskuskov, Alexander Stohr, Almog Gavra, Andras Katona, Andrew Patterson, Andy Chambers, Andy Lapidas, Anna Sophie Blee-Goldman, Antony Stubbs, Arjun Satish, Bill Bejeck, Boyang Chen, Bruno Cadonna, CHUN-HAO TANG, Cheng Tan, Chia-Ping Tsai, Chris Egerton, Christo Lolov, Colin P. McCabe, Cong Ding, Daniel Urban, David Arthur, David Jacot, David Mao, Dmitriy Fishman, Edoardo Comar, Ewen Cheslack-Postava, Greg Harris, Guozhang Wang, Igor Soarez, Ismael Juma, Israel Ekpo, Ivan Ponomarev, Jakub Scholz, James Galasyn, Jason Gustafson, Jeff Kim, Jim Galasyn, JoeCqupt, Joel Hamill, John Gray, John Roesler, Jongho Jeon, Jorge Esteban Quilcate Otoya, Jose Sancio, Josep Prat, José Armando García Sancio, Jun Rao, Justine Olshan, Kalpesh Patel, Kamal Chandraprakash, Kevin Zhang, Kirk True, Konstantine Karantasis, Kowshik Prakasam, Leah Thomas, Lee Dongjin, Lucas Bradstreet, Luke Chen, Manikumar Reddy, Matthew Wong, Matthias J. Sax, Michael Carter, Mickael Maison, Nigel Liang, Niket, Niket Goel, Oliver Hutchison, Omnia G H Ibrahim, Patrick Stuedi, Phil Hardwick, Prateek Agarwal, Rajini Sivaram, Randall Hauch, René Kerner, Richard Yu, Rohan, Ron Dagostino, Ryan Dielhenn, Sanjana Kaundinya, Satish Duggana, Sergio Peña, Sherzod Mamadaliev, Stanislav Vodetskyi, Ted Yu, Tom Bentley, Tomas Forsman, Tomer Wizman, Uwe Eisele, Victoria Xia, Viktor Somogyi-Vass, Vincent Jiang, Walker Carlson, Weisheng Yang, Xavier Léauté, Yanwen(Jason) Lin, Yi Ding, Zara Lim, andy0x01, dengziming, feyman2016, ik, ik.lim, jem, jiangyuan, kpatelatwork, leah, loboya~, lujiefsi, sebbASF, singingMan, vamossagar12, wenbingshen
We welcome your help and feedback. For more information on how to report problems, and to get involved, visit the project website at https://kafka.apache.org/
Thank you!
Regards,
David
Re: [ANNOUNCE] Apache Kafka 3.1.0
Posted by Luke Chen <sh...@gmail.com>.
Thanks David for driving the release!
@Deepti, no, this release doesn't include the log4j2 upgrade feature.
Luke
On Tue, Jan 25, 2022 at 11:43 AM Gwen Shapira <cs...@gmail.com> wrote:
> Exciting! Thanks for driving the release, David.
>
> On Mon, Jan 24, 2022 at 9:04 AM David Jacot <da...@apache.org> wrote:
> >
> > The Apache Kafka community is pleased to announce the release for
> > Apache Kafka 3.1.0.
> >
> > It is a major release that includes many new features, including:
> >
> > * Apache Kafka supports Java 17
> > * The FetchRequest supports Topic IDs (KIP-516)
> > * Extend SASL/OAUTHBEARER with support for OIDC (KIP-768)
> > * Add broker count metrics (KIP-748)
> > * Differentiate consistently metric latency measured in millis and
> > nanos (KIP-773)
> > * The eager rebalance protocol is deprecated (KAFKA-13439)
> > * Add TaskId field to StreamsException (KIP-783)
> > * Custom partitioners in foreign-key joins (KIP-775)
> > * Fetch/findSessions queries with open endpoints for
> > SessionStore/WindowStore (KIP-766)
> > * Range queries with open endpoints (KIP-763)
> > * Add total blocked time metric to Streams (KIP-761)
> > * Add additional configuration to control MirrorMaker2 internal topics
> > naming convention (KIP-690)
> >
> > You may read a more detailed list of features in the 3.1.0 blog post:
> > https://blogs.apache.org/kafka/
> >
> > All of the changes in this release can be found in the release notes:
> > https://www.apache.org/dist/kafka/3.1.0/RELEASE_NOTES.html
> >
> > You can download the source and binary release (Scala 2.12 and 2.13)
> from:
> > https://kafka.apache.org/downloads#3.1.0
> >
> >
> ---------------------------------------------------------------------------------------------------
> >
> >
> > Apache Kafka is a distributed streaming platform with four core APIs:
> >
> > ** The Producer API allows an application to publish a stream of records
> to
> > one or more Kafka topics.
> >
> > ** The Consumer API allows an application to subscribe to one or more
> > topics and process the stream of records produced to them.
> >
> > ** The Streams API allows an application to act as a stream processor,
> > consuming an input stream from one or more topics and producing an
> > output stream to one or more output topics, effectively transforming the
> > input streams to output streams.
> >
> > ** The Connector API allows building and running reusable producers or
> > consumers that connect Kafka topics to existing applications or data
> > systems. For example, a connector to a relational database might
> > capture every change to a table.
> >
> >
> > With these APIs, Kafka can be used for two broad classes of application:
> >
> > ** Building real-time streaming data pipelines that reliably get data
> > between systems or applications.
> >
> > ** Building real-time streaming applications that transform or react
> > to the streams of data.
> >
> >
> > Apache Kafka is in use at large and small companies worldwide, including
> > Capital One, Goldman Sachs, ING, LinkedIn, Netflix, Pinterest, Rabobank,
> > Target, The New York Times, Uber, Yelp, and Zalando, among others.
> >
> > A big thank you for the following 114 contributors to this release!
> >
> > A. Sophie Blee-Goldman, Alexander Iskuskov, Alexander Stohr, Almog
> > Gavra, Andras Katona, Andrew Patterson, Andy Chambers, Andy Lapidas,
> > Anna Sophie Blee-Goldman, Antony Stubbs, Arjun Satish, Bill Bejeck,
> > Boyang Chen, Bruno Cadonna, CHUN-HAO TANG, Cheng Tan, Chia-Ping Tsai,
> > Chris Egerton, Christo Lolov, Colin P. McCabe, Cong Ding, Daniel
> > Urban, David Arthur, David Jacot, David Mao, Dmitriy Fishman, Edoardo
> > Comar, Ewen Cheslack-Postava, Greg Harris, Guozhang Wang, Igor Soarez,
> > Ismael Juma, Israel Ekpo, Ivan Ponomarev, Jakub Scholz, James Galasyn,
> > Jason Gustafson, Jeff Kim, Jim Galasyn, JoeCqupt, Joel Hamill, John
> > Gray, John Roesler, Jongho Jeon, Jorge Esteban Quilcate Otoya, Jose
> > Sancio, Josep Prat, José Armando García Sancio, Jun Rao, Justine
> > Olshan, Kalpesh Patel, Kamal Chandraprakash, Kevin Zhang, Kirk True,
> > Konstantine Karantasis, Kowshik Prakasam, Leah Thomas, Lee Dongjin,
> > Lucas Bradstreet, Luke Chen, Manikumar Reddy, Matthew Wong, Matthias
> > J. Sax, Michael Carter, Mickael Maison, Nigel Liang, Niket, Niket
> > Goel, Oliver Hutchison, Omnia G H Ibrahim, Patrick Stuedi, Phil
> > Hardwick, Prateek Agarwal, Rajini Sivaram, Randall Hauch, René Kerner,
> > Richard Yu, Rohan, Ron Dagostino, Ryan Dielhenn, Sanjana Kaundinya,
> > Satish Duggana, Sergio Peña, Sherzod Mamadaliev, Stanislav Vodetskyi,
> > Ted Yu, Tom Bentley, Tomas Forsman, Tomer Wizman, Uwe Eisele, Victoria
> > Xia, Viktor Somogyi-Vass, Vincent Jiang, Walker Carlson, Weisheng
> > Yang, Xavier Léauté, Yanwen(Jason) Lin, Yi Ding, Zara Lim, andy0x01,
> > dengziming, feyman2016, ik, ik.lim, jem, jiangyuan, kpatelatwork,
> > leah, loboya~, lujiefsi, sebbASF, singingMan, vamossagar12,
> > wenbingshen
> >
> > We welcome your help and feedback. For more information on how to
> > report problems, and to get involved, visit the project website at
> > https://kafka.apache.org/
> >
> > Thank you!
> >
> >
> > Regards,
> >
> > David
>
Re: [ANNOUNCE] Apache Kafka 3.1.0
Posted by Luke Chen <sh...@gmail.com>.
Thanks David for driving the release!
@Deepti, no, this release doesn't include the log4j2 upgrade feature.
Luke
On Tue, Jan 25, 2022 at 11:43 AM Gwen Shapira <cs...@gmail.com> wrote:
> Exciting! Thanks for driving the release, David.
>
> On Mon, Jan 24, 2022 at 9:04 AM David Jacot <da...@apache.org> wrote:
> >
> > The Apache Kafka community is pleased to announce the release for
> > Apache Kafka 3.1.0.
> >
> > It is a major release that includes many new features, including:
> >
> > * Apache Kafka supports Java 17
> > * The FetchRequest supports Topic IDs (KIP-516)
> > * Extend SASL/OAUTHBEARER with support for OIDC (KIP-768)
> > * Add broker count metrics (KIP-748)
> > * Differentiate consistently metric latency measured in millis and
> > nanos (KIP-773)
> > * The eager rebalance protocol is deprecated (KAFKA-13439)
> > * Add TaskId field to StreamsException (KIP-783)
> > * Custom partitioners in foreign-key joins (KIP-775)
> > * Fetch/findSessions queries with open endpoints for
> > SessionStore/WindowStore (KIP-766)
> > * Range queries with open endpoints (KIP-763)
> > * Add total blocked time metric to Streams (KIP-761)
> > * Add additional configuration to control MirrorMaker2 internal topics
> > naming convention (KIP-690)
> >
> > You may read a more detailed list of features in the 3.1.0 blog post:
> > https://blogs.apache.org/kafka/
> >
> > All of the changes in this release can be found in the release notes:
> > https://www.apache.org/dist/kafka/3.1.0/RELEASE_NOTES.html
> >
> > You can download the source and binary release (Scala 2.12 and 2.13)
> from:
> > https://kafka.apache.org/downloads#3.1.0
> >
> >
> ---------------------------------------------------------------------------------------------------
> >
> >
> > Apache Kafka is a distributed streaming platform with four core APIs:
> >
> > ** The Producer API allows an application to publish a stream of records
> to
> > one or more Kafka topics.
> >
> > ** The Consumer API allows an application to subscribe to one or more
> > topics and process the stream of records produced to them.
> >
> > ** The Streams API allows an application to act as a stream processor,
> > consuming an input stream from one or more topics and producing an
> > output stream to one or more output topics, effectively transforming the
> > input streams to output streams.
> >
> > ** The Connector API allows building and running reusable producers or
> > consumers that connect Kafka topics to existing applications or data
> > systems. For example, a connector to a relational database might
> > capture every change to a table.
> >
> >
> > With these APIs, Kafka can be used for two broad classes of application:
> >
> > ** Building real-time streaming data pipelines that reliably get data
> > between systems or applications.
> >
> > ** Building real-time streaming applications that transform or react
> > to the streams of data.
> >
> >
> > Apache Kafka is in use at large and small companies worldwide, including
> > Capital One, Goldman Sachs, ING, LinkedIn, Netflix, Pinterest, Rabobank,
> > Target, The New York Times, Uber, Yelp, and Zalando, among others.
> >
> > A big thank you for the following 114 contributors to this release!
> >
> > A. Sophie Blee-Goldman, Alexander Iskuskov, Alexander Stohr, Almog
> > Gavra, Andras Katona, Andrew Patterson, Andy Chambers, Andy Lapidas,
> > Anna Sophie Blee-Goldman, Antony Stubbs, Arjun Satish, Bill Bejeck,
> > Boyang Chen, Bruno Cadonna, CHUN-HAO TANG, Cheng Tan, Chia-Ping Tsai,
> > Chris Egerton, Christo Lolov, Colin P. McCabe, Cong Ding, Daniel
> > Urban, David Arthur, David Jacot, David Mao, Dmitriy Fishman, Edoardo
> > Comar, Ewen Cheslack-Postava, Greg Harris, Guozhang Wang, Igor Soarez,
> > Ismael Juma, Israel Ekpo, Ivan Ponomarev, Jakub Scholz, James Galasyn,
> > Jason Gustafson, Jeff Kim, Jim Galasyn, JoeCqupt, Joel Hamill, John
> > Gray, John Roesler, Jongho Jeon, Jorge Esteban Quilcate Otoya, Jose
> > Sancio, Josep Prat, José Armando García Sancio, Jun Rao, Justine
> > Olshan, Kalpesh Patel, Kamal Chandraprakash, Kevin Zhang, Kirk True,
> > Konstantine Karantasis, Kowshik Prakasam, Leah Thomas, Lee Dongjin,
> > Lucas Bradstreet, Luke Chen, Manikumar Reddy, Matthew Wong, Matthias
> > J. Sax, Michael Carter, Mickael Maison, Nigel Liang, Niket, Niket
> > Goel, Oliver Hutchison, Omnia G H Ibrahim, Patrick Stuedi, Phil
> > Hardwick, Prateek Agarwal, Rajini Sivaram, Randall Hauch, René Kerner,
> > Richard Yu, Rohan, Ron Dagostino, Ryan Dielhenn, Sanjana Kaundinya,
> > Satish Duggana, Sergio Peña, Sherzod Mamadaliev, Stanislav Vodetskyi,
> > Ted Yu, Tom Bentley, Tomas Forsman, Tomer Wizman, Uwe Eisele, Victoria
> > Xia, Viktor Somogyi-Vass, Vincent Jiang, Walker Carlson, Weisheng
> > Yang, Xavier Léauté, Yanwen(Jason) Lin, Yi Ding, Zara Lim, andy0x01,
> > dengziming, feyman2016, ik, ik.lim, jem, jiangyuan, kpatelatwork,
> > leah, loboya~, lujiefsi, sebbASF, singingMan, vamossagar12,
> > wenbingshen
> >
> > We welcome your help and feedback. For more information on how to
> > report problems, and to get involved, visit the project website at
> > https://kafka.apache.org/
> >
> > Thank you!
> >
> >
> > Regards,
> >
> > David
>
Re: [ANNOUNCE] Apache Kafka 3.1.0
Posted by Gwen Shapira <cs...@gmail.com>.
Exciting! Thanks for driving the release, David.
On Mon, Jan 24, 2022 at 9:04 AM David Jacot <da...@apache.org> wrote:
>
> The Apache Kafka community is pleased to announce the release for
> Apache Kafka 3.1.0.
>
> It is a major release that includes many new features, including:
>
> * Apache Kafka supports Java 17
> * The FetchRequest supports Topic IDs (KIP-516)
> * Extend SASL/OAUTHBEARER with support for OIDC (KIP-768)
> * Add broker count metrics (KIP-748)
> * Differentiate consistently metric latency measured in millis and
> nanos (KIP-773)
> * The eager rebalance protocol is deprecated (KAFKA-13439)
> * Add TaskId field to StreamsException (KIP-783)
> * Custom partitioners in foreign-key joins (KIP-775)
> * Fetch/findSessions queries with open endpoints for
> SessionStore/WindowStore (KIP-766)
> * Range queries with open endpoints (KIP-763)
> * Add total blocked time metric to Streams (KIP-761)
> * Add additional configuration to control MirrorMaker2 internal topics
> naming convention (KIP-690)
>
> You may read a more detailed list of features in the 3.1.0 blog post:
> https://blogs.apache.org/kafka/
>
> All of the changes in this release can be found in the release notes:
> https://www.apache.org/dist/kafka/3.1.0/RELEASE_NOTES.html
>
> You can download the source and binary release (Scala 2.12 and 2.13) from:
> https://kafka.apache.org/downloads#3.1.0
>
> ---------------------------------------------------------------------------------------------------
>
>
> Apache Kafka is a distributed streaming platform with four core APIs:
>
> ** The Producer API allows an application to publish a stream of records to
> one or more Kafka topics.
>
> ** The Consumer API allows an application to subscribe to one or more
> topics and process the stream of records produced to them.
>
> ** The Streams API allows an application to act as a stream processor,
> consuming an input stream from one or more topics and producing an
> output stream to one or more output topics, effectively transforming the
> input streams to output streams.
>
> ** The Connector API allows building and running reusable producers or
> consumers that connect Kafka topics to existing applications or data
> systems. For example, a connector to a relational database might
> capture every change to a table.
>
>
> With these APIs, Kafka can be used for two broad classes of application:
>
> ** Building real-time streaming data pipelines that reliably get data
> between systems or applications.
>
> ** Building real-time streaming applications that transform or react
> to the streams of data.
>
>
> Apache Kafka is in use at large and small companies worldwide, including
> Capital One, Goldman Sachs, ING, LinkedIn, Netflix, Pinterest, Rabobank,
> Target, The New York Times, Uber, Yelp, and Zalando, among others.
>
> A big thank you for the following 114 contributors to this release!
>
> A. Sophie Blee-Goldman, Alexander Iskuskov, Alexander Stohr, Almog
> Gavra, Andras Katona, Andrew Patterson, Andy Chambers, Andy Lapidas,
> Anna Sophie Blee-Goldman, Antony Stubbs, Arjun Satish, Bill Bejeck,
> Boyang Chen, Bruno Cadonna, CHUN-HAO TANG, Cheng Tan, Chia-Ping Tsai,
> Chris Egerton, Christo Lolov, Colin P. McCabe, Cong Ding, Daniel
> Urban, David Arthur, David Jacot, David Mao, Dmitriy Fishman, Edoardo
> Comar, Ewen Cheslack-Postava, Greg Harris, Guozhang Wang, Igor Soarez,
> Ismael Juma, Israel Ekpo, Ivan Ponomarev, Jakub Scholz, James Galasyn,
> Jason Gustafson, Jeff Kim, Jim Galasyn, JoeCqupt, Joel Hamill, John
> Gray, John Roesler, Jongho Jeon, Jorge Esteban Quilcate Otoya, Jose
> Sancio, Josep Prat, José Armando García Sancio, Jun Rao, Justine
> Olshan, Kalpesh Patel, Kamal Chandraprakash, Kevin Zhang, Kirk True,
> Konstantine Karantasis, Kowshik Prakasam, Leah Thomas, Lee Dongjin,
> Lucas Bradstreet, Luke Chen, Manikumar Reddy, Matthew Wong, Matthias
> J. Sax, Michael Carter, Mickael Maison, Nigel Liang, Niket, Niket
> Goel, Oliver Hutchison, Omnia G H Ibrahim, Patrick Stuedi, Phil
> Hardwick, Prateek Agarwal, Rajini Sivaram, Randall Hauch, René Kerner,
> Richard Yu, Rohan, Ron Dagostino, Ryan Dielhenn, Sanjana Kaundinya,
> Satish Duggana, Sergio Peña, Sherzod Mamadaliev, Stanislav Vodetskyi,
> Ted Yu, Tom Bentley, Tomas Forsman, Tomer Wizman, Uwe Eisele, Victoria
> Xia, Viktor Somogyi-Vass, Vincent Jiang, Walker Carlson, Weisheng
> Yang, Xavier Léauté, Yanwen(Jason) Lin, Yi Ding, Zara Lim, andy0x01,
> dengziming, feyman2016, ik, ik.lim, jem, jiangyuan, kpatelatwork,
> leah, loboya~, lujiefsi, sebbASF, singingMan, vamossagar12,
> wenbingshen
>
> We welcome your help and feedback. For more information on how to
> report problems, and to get involved, visit the project website at
> https://kafka.apache.org/
>
> Thank you!
>
>
> Regards,
>
> David
RE: [ANNOUNCE] Apache Kafka 3.1.0
Posted by Deepti Sharma S <de...@ericsson.com.INVALID>.
Congratulations!
One thing would like to confirm, if we have upgraded the Log4J version from 1.x to 2.x in this release?
Regards,
Deepti Sharma
PMP® & ITIL
-----Original Message-----
From: David Jacot <da...@apache.org>
Sent: Monday, January 24, 2022 10:34 PM
To: announce@apache.org; dev@kafka.apache.org; users@kafka.apache.org; kafka-clients@googlegroups.com
Subject: [ANNOUNCE] Apache Kafka 3.1.0
The Apache Kafka community is pleased to announce the release for Apache Kafka 3.1.0.
It is a major release that includes many new features, including:
* Apache Kafka supports Java 17
* The FetchRequest supports Topic IDs (KIP-516)
* Extend SASL/OAUTHBEARER with support for OIDC (KIP-768)
* Add broker count metrics (KIP-748)
* Differentiate consistently metric latency measured in millis and nanos (KIP-773)
* The eager rebalance protocol is deprecated (KAFKA-13439)
* Add TaskId field to StreamsException (KIP-783)
* Custom partitioners in foreign-key joins (KIP-775)
* Fetch/findSessions queries with open endpoints for SessionStore/WindowStore (KIP-766)
* Range queries with open endpoints (KIP-763)
* Add total blocked time metric to Streams (KIP-761)
* Add additional configuration to control MirrorMaker2 internal topics naming convention (KIP-690)
You may read a more detailed list of features in the 3.1.0 blog post:
https://blogs.apache.org/kafka/
All of the changes in this release can be found in the release notes:
https://www.apache.org/dist/kafka/3.1.0/RELEASE_NOTES.html
You can download the source and binary release (Scala 2.12 and 2.13) from:
https://kafka.apache.org/downloads#3.1.0
---------------------------------------------------------------------------------------------------
Apache Kafka is a distributed streaming platform with four core APIs:
** The Producer API allows an application to publish a stream of records to one or more Kafka topics.
** The Consumer API allows an application to subscribe to one or more topics and process the stream of records produced to them.
** The Streams API allows an application to act as a stream processor, consuming an input stream from one or more topics and producing an output stream to one or more output topics, effectively transforming the input streams to output streams.
** The Connector API allows building and running reusable producers or consumers that connect Kafka topics to existing applications or data systems. For example, a connector to a relational database might capture every change to a table.
With these APIs, Kafka can be used for two broad classes of application:
** Building real-time streaming data pipelines that reliably get data between systems or applications.
** Building real-time streaming applications that transform or react to the streams of data.
Apache Kafka is in use at large and small companies worldwide, including Capital One, Goldman Sachs, ING, LinkedIn, Netflix, Pinterest, Rabobank, Target, The New York Times, Uber, Yelp, and Zalando, among others.
A big thank you for the following 114 contributors to this release!
A. Sophie Blee-Goldman, Alexander Iskuskov, Alexander Stohr, Almog Gavra, Andras Katona, Andrew Patterson, Andy Chambers, Andy Lapidas, Anna Sophie Blee-Goldman, Antony Stubbs, Arjun Satish, Bill Bejeck, Boyang Chen, Bruno Cadonna, CHUN-HAO TANG, Cheng Tan, Chia-Ping Tsai, Chris Egerton, Christo Lolov, Colin P. McCabe, Cong Ding, Daniel Urban, David Arthur, David Jacot, David Mao, Dmitriy Fishman, Edoardo Comar, Ewen Cheslack-Postava, Greg Harris, Guozhang Wang, Igor Soarez, Ismael Juma, Israel Ekpo, Ivan Ponomarev, Jakub Scholz, James Galasyn, Jason Gustafson, Jeff Kim, Jim Galasyn, JoeCqupt, Joel Hamill, John Gray, John Roesler, Jongho Jeon, Jorge Esteban Quilcate Otoya, Jose Sancio, Josep Prat, José Armando García Sancio, Jun Rao, Justine Olshan, Kalpesh Patel, Kamal Chandraprakash, Kevin Zhang, Kirk True, Konstantine Karantasis, Kowshik Prakasam, Leah Thomas, Lee Dongjin, Lucas Bradstreet, Luke Chen, Manikumar Reddy, Matthew Wong, Matthias J. Sax, Michael Carter, Mickael Maison, Nigel Liang, Niket, Niket Goel, Oliver Hutchison, Omnia G H Ibrahim, Patrick Stuedi, Phil Hardwick, Prateek Agarwal, Rajini Sivaram, Randall Hauch, René Kerner, Richard Yu, Rohan, Ron Dagostino, Ryan Dielhenn, Sanjana Kaundinya, Satish Duggana, Sergio Peña, Sherzod Mamadaliev, Stanislav Vodetskyi, Ted Yu, Tom Bentley, Tomas Forsman, Tomer Wizman, Uwe Eisele, Victoria Xia, Viktor Somogyi-Vass, Vincent Jiang, Walker Carlson, Weisheng Yang, Xavier Léauté, Yanwen(Jason) Lin, Yi Ding, Zara Lim, andy0x01, dengziming, feyman2016, ik, ik.lim, jem, jiangyuan, kpatelatwork, leah, loboya~, lujiefsi, sebbASF, singingMan, vamossagar12, wenbingshen
We welcome your help and feedback. For more information on how to report problems, and to get involved, visit the project website at https://kafka.apache.org/
Thank you!
Regards,
David
Re: [ANNOUNCE] Apache Kafka 3.1.0
Posted by Men Lim <zu...@gmail.com>.
Yes thank you Isreal. rmoff also clarified things for me over on reddit.
On Wed, Jan 26, 2022 at 6:41 PM Israel Ekpo <is...@gmail.com> wrote:
> Kafka 3.x is production ready if you are running it with Zookeeper
>
> If you are running it in KRaft mode without Zookeeper that set up is not
> yet recommended for production scenarios
>
> I hope this clarifies your concerns
>
>
>
> On Wed, Jan 26, 2022 at 9:50 AM Men Lim <zu...@gmail.com> wrote:
>
> > i'm curious at what point can 3.x be production ready?
> >
> > On Mon, Jan 24, 2022 at 10:04 AM David Jacot <da...@apache.org> wrote:
> >
> > > The Apache Kafka community is pleased to announce the release for
> > > Apache Kafka 3.1.0.
> > >
> > > It is a major release that includes many new features, including:
> > >
> > > * Apache Kafka supports Java 17
> > > * The FetchRequest supports Topic IDs (KIP-516)
> > > * Extend SASL/OAUTHBEARER with support for OIDC (KIP-768)
> > > * Add broker count metrics (KIP-748)
> > > * Differentiate consistently metric latency measured in millis and
> > > nanos (KIP-773)
> > > * The eager rebalance protocol is deprecated (KAFKA-13439)
> > > * Add TaskId field to StreamsException (KIP-783)
> > > * Custom partitioners in foreign-key joins (KIP-775)
> > > * Fetch/findSessions queries with open endpoints for
> > > SessionStore/WindowStore (KIP-766)
> > > * Range queries with open endpoints (KIP-763)
> > > * Add total blocked time metric to Streams (KIP-761)
> > > * Add additional configuration to control MirrorMaker2 internal topics
> > > naming convention (KIP-690)
> > >
> > > You may read a more detailed list of features in the 3.1.0 blog post:
> > > https://blogs.apache.org/kafka/
> > >
> > > All of the changes in this release can be found in the release notes:
> > > https://www.apache.org/dist/kafka/3.1.0/RELEASE_NOTES.html
> > >
> > > You can download the source and binary release (Scala 2.12 and 2.13)
> > from:
> > > https://kafka.apache.org/downloads#3.1.0
> > >
> > >
> > >
> >
> ---------------------------------------------------------------------------------------------------
> > >
> > >
> > > Apache Kafka is a distributed streaming platform with four core APIs:
> > >
> > > ** The Producer API allows an application to publish a stream of
> records
> > to
> > > one or more Kafka topics.
> > >
> > > ** The Consumer API allows an application to subscribe to one or more
> > > topics and process the stream of records produced to them.
> > >
> > > ** The Streams API allows an application to act as a stream processor,
> > > consuming an input stream from one or more topics and producing an
> > > output stream to one or more output topics, effectively transforming
> the
> > > input streams to output streams.
> > >
> > > ** The Connector API allows building and running reusable producers or
> > > consumers that connect Kafka topics to existing applications or data
> > > systems. For example, a connector to a relational database might
> > > capture every change to a table.
> > >
> > >
> > > With these APIs, Kafka can be used for two broad classes of
> application:
> > >
> > > ** Building real-time streaming data pipelines that reliably get data
> > > between systems or applications.
> > >
> > > ** Building real-time streaming applications that transform or react
> > > to the streams of data.
> > >
> > >
> > > Apache Kafka is in use at large and small companies worldwide,
> including
> > > Capital One, Goldman Sachs, ING, LinkedIn, Netflix, Pinterest,
> Rabobank,
> > > Target, The New York Times, Uber, Yelp, and Zalando, among others.
> > >
> > > A big thank you for the following 114 contributors to this release!
> > >
> > > A. Sophie Blee-Goldman, Alexander Iskuskov, Alexander Stohr, Almog
> > > Gavra, Andras Katona, Andrew Patterson, Andy Chambers, Andy Lapidas,
> > > Anna Sophie Blee-Goldman, Antony Stubbs, Arjun Satish, Bill Bejeck,
> > > Boyang Chen, Bruno Cadonna, CHUN-HAO TANG, Cheng Tan, Chia-Ping Tsai,
> > > Chris Egerton, Christo Lolov, Colin P. McCabe, Cong Ding, Daniel
> > > Urban, David Arthur, David Jacot, David Mao, Dmitriy Fishman, Edoardo
> > > Comar, Ewen Cheslack-Postava, Greg Harris, Guozhang Wang, Igor Soarez,
> > > Ismael Juma, Israel Ekpo, Ivan Ponomarev, Jakub Scholz, James Galasyn,
> > > Jason Gustafson, Jeff Kim, Jim Galasyn, JoeCqupt, Joel Hamill, John
> > > Gray, John Roesler, Jongho Jeon, Jorge Esteban Quilcate Otoya, Jose
> > > Sancio, Josep Prat, José Armando García Sancio, Jun Rao, Justine
> > > Olshan, Kalpesh Patel, Kamal Chandraprakash, Kevin Zhang, Kirk True,
> > > Konstantine Karantasis, Kowshik Prakasam, Leah Thomas, Lee Dongjin,
> > > Lucas Bradstreet, Luke Chen, Manikumar Reddy, Matthew Wong, Matthias
> > > J. Sax, Michael Carter, Mickael Maison, Nigel Liang, Niket, Niket
> > > Goel, Oliver Hutchison, Omnia G H Ibrahim, Patrick Stuedi, Phil
> > > Hardwick, Prateek Agarwal, Rajini Sivaram, Randall Hauch, René Kerner,
> > > Richard Yu, Rohan, Ron Dagostino, Ryan Dielhenn, Sanjana Kaundinya,
> > > Satish Duggana, Sergio Peña, Sherzod Mamadaliev, Stanislav Vodetskyi,
> > > Ted Yu, Tom Bentley, Tomas Forsman, Tomer Wizman, Uwe Eisele, Victoria
> > > Xia, Viktor Somogyi-Vass, Vincent Jiang, Walker Carlson, Weisheng
> > > Yang, Xavier Léauté, Yanwen(Jason) Lin, Yi Ding, Zara Lim, andy0x01,
> > > dengziming, feyman2016, ik, ik.lim, jem, jiangyuan, kpatelatwork,
> > > leah, loboya~, lujiefsi, sebbASF, singingMan, vamossagar12,
> > > wenbingshen
> > >
> > > We welcome your help and feedback. For more information on how to
> > > report problems, and to get involved, visit the project website at
> > > https://kafka.apache.org/
> > >
> > > Thank you!
> > >
> > >
> > > Regards,
> > >
> > > David
> > >
> >
> --
> Israel Ekpo
> Lead Instructor, IzzyAcademy.com
> https://www.youtube.com/c/izzyacademy
> https://izzyacademy.com/
>
Re: [ANNOUNCE] Apache Kafka 3.1.0
Posted by Israel Ekpo <is...@gmail.com>.
Kafka 3.x is production ready if you are running it with Zookeeper
If you are running it in KRaft mode without Zookeeper that set up is not
yet recommended for production scenarios
I hope this clarifies your concerns
On Wed, Jan 26, 2022 at 9:50 AM Men Lim <zu...@gmail.com> wrote:
> i'm curious at what point can 3.x be production ready?
>
> On Mon, Jan 24, 2022 at 10:04 AM David Jacot <da...@apache.org> wrote:
>
> > The Apache Kafka community is pleased to announce the release for
> > Apache Kafka 3.1.0.
> >
> > It is a major release that includes many new features, including:
> >
> > * Apache Kafka supports Java 17
> > * The FetchRequest supports Topic IDs (KIP-516)
> > * Extend SASL/OAUTHBEARER with support for OIDC (KIP-768)
> > * Add broker count metrics (KIP-748)
> > * Differentiate consistently metric latency measured in millis and
> > nanos (KIP-773)
> > * The eager rebalance protocol is deprecated (KAFKA-13439)
> > * Add TaskId field to StreamsException (KIP-783)
> > * Custom partitioners in foreign-key joins (KIP-775)
> > * Fetch/findSessions queries with open endpoints for
> > SessionStore/WindowStore (KIP-766)
> > * Range queries with open endpoints (KIP-763)
> > * Add total blocked time metric to Streams (KIP-761)
> > * Add additional configuration to control MirrorMaker2 internal topics
> > naming convention (KIP-690)
> >
> > You may read a more detailed list of features in the 3.1.0 blog post:
> > https://blogs.apache.org/kafka/
> >
> > All of the changes in this release can be found in the release notes:
> > https://www.apache.org/dist/kafka/3.1.0/RELEASE_NOTES.html
> >
> > You can download the source and binary release (Scala 2.12 and 2.13)
> from:
> > https://kafka.apache.org/downloads#3.1.0
> >
> >
> >
> ---------------------------------------------------------------------------------------------------
> >
> >
> > Apache Kafka is a distributed streaming platform with four core APIs:
> >
> > ** The Producer API allows an application to publish a stream of records
> to
> > one or more Kafka topics.
> >
> > ** The Consumer API allows an application to subscribe to one or more
> > topics and process the stream of records produced to them.
> >
> > ** The Streams API allows an application to act as a stream processor,
> > consuming an input stream from one or more topics and producing an
> > output stream to one or more output topics, effectively transforming the
> > input streams to output streams.
> >
> > ** The Connector API allows building and running reusable producers or
> > consumers that connect Kafka topics to existing applications or data
> > systems. For example, a connector to a relational database might
> > capture every change to a table.
> >
> >
> > With these APIs, Kafka can be used for two broad classes of application:
> >
> > ** Building real-time streaming data pipelines that reliably get data
> > between systems or applications.
> >
> > ** Building real-time streaming applications that transform or react
> > to the streams of data.
> >
> >
> > Apache Kafka is in use at large and small companies worldwide, including
> > Capital One, Goldman Sachs, ING, LinkedIn, Netflix, Pinterest, Rabobank,
> > Target, The New York Times, Uber, Yelp, and Zalando, among others.
> >
> > A big thank you for the following 114 contributors to this release!
> >
> > A. Sophie Blee-Goldman, Alexander Iskuskov, Alexander Stohr, Almog
> > Gavra, Andras Katona, Andrew Patterson, Andy Chambers, Andy Lapidas,
> > Anna Sophie Blee-Goldman, Antony Stubbs, Arjun Satish, Bill Bejeck,
> > Boyang Chen, Bruno Cadonna, CHUN-HAO TANG, Cheng Tan, Chia-Ping Tsai,
> > Chris Egerton, Christo Lolov, Colin P. McCabe, Cong Ding, Daniel
> > Urban, David Arthur, David Jacot, David Mao, Dmitriy Fishman, Edoardo
> > Comar, Ewen Cheslack-Postava, Greg Harris, Guozhang Wang, Igor Soarez,
> > Ismael Juma, Israel Ekpo, Ivan Ponomarev, Jakub Scholz, James Galasyn,
> > Jason Gustafson, Jeff Kim, Jim Galasyn, JoeCqupt, Joel Hamill, John
> > Gray, John Roesler, Jongho Jeon, Jorge Esteban Quilcate Otoya, Jose
> > Sancio, Josep Prat, José Armando García Sancio, Jun Rao, Justine
> > Olshan, Kalpesh Patel, Kamal Chandraprakash, Kevin Zhang, Kirk True,
> > Konstantine Karantasis, Kowshik Prakasam, Leah Thomas, Lee Dongjin,
> > Lucas Bradstreet, Luke Chen, Manikumar Reddy, Matthew Wong, Matthias
> > J. Sax, Michael Carter, Mickael Maison, Nigel Liang, Niket, Niket
> > Goel, Oliver Hutchison, Omnia G H Ibrahim, Patrick Stuedi, Phil
> > Hardwick, Prateek Agarwal, Rajini Sivaram, Randall Hauch, René Kerner,
> > Richard Yu, Rohan, Ron Dagostino, Ryan Dielhenn, Sanjana Kaundinya,
> > Satish Duggana, Sergio Peña, Sherzod Mamadaliev, Stanislav Vodetskyi,
> > Ted Yu, Tom Bentley, Tomas Forsman, Tomer Wizman, Uwe Eisele, Victoria
> > Xia, Viktor Somogyi-Vass, Vincent Jiang, Walker Carlson, Weisheng
> > Yang, Xavier Léauté, Yanwen(Jason) Lin, Yi Ding, Zara Lim, andy0x01,
> > dengziming, feyman2016, ik, ik.lim, jem, jiangyuan, kpatelatwork,
> > leah, loboya~, lujiefsi, sebbASF, singingMan, vamossagar12,
> > wenbingshen
> >
> > We welcome your help and feedback. For more information on how to
> > report problems, and to get involved, visit the project website at
> > https://kafka.apache.org/
> >
> > Thank you!
> >
> >
> > Regards,
> >
> > David
> >
>
--
Israel Ekpo
Lead Instructor, IzzyAcademy.com
https://www.youtube.com/c/izzyacademy
https://izzyacademy.com/
Re: [ANNOUNCE] Apache Kafka 3.1.0
Posted by Men Lim <zu...@gmail.com>.
i'm curious at what point can 3.x be production ready?
On Mon, Jan 24, 2022 at 10:04 AM David Jacot <da...@apache.org> wrote:
> The Apache Kafka community is pleased to announce the release for
> Apache Kafka 3.1.0.
>
> It is a major release that includes many new features, including:
>
> * Apache Kafka supports Java 17
> * The FetchRequest supports Topic IDs (KIP-516)
> * Extend SASL/OAUTHBEARER with support for OIDC (KIP-768)
> * Add broker count metrics (KIP-748)
> * Differentiate consistently metric latency measured in millis and
> nanos (KIP-773)
> * The eager rebalance protocol is deprecated (KAFKA-13439)
> * Add TaskId field to StreamsException (KIP-783)
> * Custom partitioners in foreign-key joins (KIP-775)
> * Fetch/findSessions queries with open endpoints for
> SessionStore/WindowStore (KIP-766)
> * Range queries with open endpoints (KIP-763)
> * Add total blocked time metric to Streams (KIP-761)
> * Add additional configuration to control MirrorMaker2 internal topics
> naming convention (KIP-690)
>
> You may read a more detailed list of features in the 3.1.0 blog post:
> https://blogs.apache.org/kafka/
>
> All of the changes in this release can be found in the release notes:
> https://www.apache.org/dist/kafka/3.1.0/RELEASE_NOTES.html
>
> You can download the source and binary release (Scala 2.12 and 2.13) from:
> https://kafka.apache.org/downloads#3.1.0
>
>
> ---------------------------------------------------------------------------------------------------
>
>
> Apache Kafka is a distributed streaming platform with four core APIs:
>
> ** The Producer API allows an application to publish a stream of records to
> one or more Kafka topics.
>
> ** The Consumer API allows an application to subscribe to one or more
> topics and process the stream of records produced to them.
>
> ** The Streams API allows an application to act as a stream processor,
> consuming an input stream from one or more topics and producing an
> output stream to one or more output topics, effectively transforming the
> input streams to output streams.
>
> ** The Connector API allows building and running reusable producers or
> consumers that connect Kafka topics to existing applications or data
> systems. For example, a connector to a relational database might
> capture every change to a table.
>
>
> With these APIs, Kafka can be used for two broad classes of application:
>
> ** Building real-time streaming data pipelines that reliably get data
> between systems or applications.
>
> ** Building real-time streaming applications that transform or react
> to the streams of data.
>
>
> Apache Kafka is in use at large and small companies worldwide, including
> Capital One, Goldman Sachs, ING, LinkedIn, Netflix, Pinterest, Rabobank,
> Target, The New York Times, Uber, Yelp, and Zalando, among others.
>
> A big thank you for the following 114 contributors to this release!
>
> A. Sophie Blee-Goldman, Alexander Iskuskov, Alexander Stohr, Almog
> Gavra, Andras Katona, Andrew Patterson, Andy Chambers, Andy Lapidas,
> Anna Sophie Blee-Goldman, Antony Stubbs, Arjun Satish, Bill Bejeck,
> Boyang Chen, Bruno Cadonna, CHUN-HAO TANG, Cheng Tan, Chia-Ping Tsai,
> Chris Egerton, Christo Lolov, Colin P. McCabe, Cong Ding, Daniel
> Urban, David Arthur, David Jacot, David Mao, Dmitriy Fishman, Edoardo
> Comar, Ewen Cheslack-Postava, Greg Harris, Guozhang Wang, Igor Soarez,
> Ismael Juma, Israel Ekpo, Ivan Ponomarev, Jakub Scholz, James Galasyn,
> Jason Gustafson, Jeff Kim, Jim Galasyn, JoeCqupt, Joel Hamill, John
> Gray, John Roesler, Jongho Jeon, Jorge Esteban Quilcate Otoya, Jose
> Sancio, Josep Prat, José Armando García Sancio, Jun Rao, Justine
> Olshan, Kalpesh Patel, Kamal Chandraprakash, Kevin Zhang, Kirk True,
> Konstantine Karantasis, Kowshik Prakasam, Leah Thomas, Lee Dongjin,
> Lucas Bradstreet, Luke Chen, Manikumar Reddy, Matthew Wong, Matthias
> J. Sax, Michael Carter, Mickael Maison, Nigel Liang, Niket, Niket
> Goel, Oliver Hutchison, Omnia G H Ibrahim, Patrick Stuedi, Phil
> Hardwick, Prateek Agarwal, Rajini Sivaram, Randall Hauch, René Kerner,
> Richard Yu, Rohan, Ron Dagostino, Ryan Dielhenn, Sanjana Kaundinya,
> Satish Duggana, Sergio Peña, Sherzod Mamadaliev, Stanislav Vodetskyi,
> Ted Yu, Tom Bentley, Tomas Forsman, Tomer Wizman, Uwe Eisele, Victoria
> Xia, Viktor Somogyi-Vass, Vincent Jiang, Walker Carlson, Weisheng
> Yang, Xavier Léauté, Yanwen(Jason) Lin, Yi Ding, Zara Lim, andy0x01,
> dengziming, feyman2016, ik, ik.lim, jem, jiangyuan, kpatelatwork,
> leah, loboya~, lujiefsi, sebbASF, singingMan, vamossagar12,
> wenbingshen
>
> We welcome your help and feedback. For more information on how to
> report problems, and to get involved, visit the project website at
> https://kafka.apache.org/
>
> Thank you!
>
>
> Regards,
>
> David
>
Re: [ANNOUNCE] Apache Kafka 3.1.0
Posted by Gwen Shapira <cs...@gmail.com>.
Exciting! Thanks for driving the release, David.
On Mon, Jan 24, 2022 at 9:04 AM David Jacot <da...@apache.org> wrote:
>
> The Apache Kafka community is pleased to announce the release for
> Apache Kafka 3.1.0.
>
> It is a major release that includes many new features, including:
>
> * Apache Kafka supports Java 17
> * The FetchRequest supports Topic IDs (KIP-516)
> * Extend SASL/OAUTHBEARER with support for OIDC (KIP-768)
> * Add broker count metrics (KIP-748)
> * Differentiate consistently metric latency measured in millis and
> nanos (KIP-773)
> * The eager rebalance protocol is deprecated (KAFKA-13439)
> * Add TaskId field to StreamsException (KIP-783)
> * Custom partitioners in foreign-key joins (KIP-775)
> * Fetch/findSessions queries with open endpoints for
> SessionStore/WindowStore (KIP-766)
> * Range queries with open endpoints (KIP-763)
> * Add total blocked time metric to Streams (KIP-761)
> * Add additional configuration to control MirrorMaker2 internal topics
> naming convention (KIP-690)
>
> You may read a more detailed list of features in the 3.1.0 blog post:
> https://blogs.apache.org/kafka/
>
> All of the changes in this release can be found in the release notes:
> https://www.apache.org/dist/kafka/3.1.0/RELEASE_NOTES.html
>
> You can download the source and binary release (Scala 2.12 and 2.13) from:
> https://kafka.apache.org/downloads#3.1.0
>
> ---------------------------------------------------------------------------------------------------
>
>
> Apache Kafka is a distributed streaming platform with four core APIs:
>
> ** The Producer API allows an application to publish a stream of records to
> one or more Kafka topics.
>
> ** The Consumer API allows an application to subscribe to one or more
> topics and process the stream of records produced to them.
>
> ** The Streams API allows an application to act as a stream processor,
> consuming an input stream from one or more topics and producing an
> output stream to one or more output topics, effectively transforming the
> input streams to output streams.
>
> ** The Connector API allows building and running reusable producers or
> consumers that connect Kafka topics to existing applications or data
> systems. For example, a connector to a relational database might
> capture every change to a table.
>
>
> With these APIs, Kafka can be used for two broad classes of application:
>
> ** Building real-time streaming data pipelines that reliably get data
> between systems or applications.
>
> ** Building real-time streaming applications that transform or react
> to the streams of data.
>
>
> Apache Kafka is in use at large and small companies worldwide, including
> Capital One, Goldman Sachs, ING, LinkedIn, Netflix, Pinterest, Rabobank,
> Target, The New York Times, Uber, Yelp, and Zalando, among others.
>
> A big thank you for the following 114 contributors to this release!
>
> A. Sophie Blee-Goldman, Alexander Iskuskov, Alexander Stohr, Almog
> Gavra, Andras Katona, Andrew Patterson, Andy Chambers, Andy Lapidas,
> Anna Sophie Blee-Goldman, Antony Stubbs, Arjun Satish, Bill Bejeck,
> Boyang Chen, Bruno Cadonna, CHUN-HAO TANG, Cheng Tan, Chia-Ping Tsai,
> Chris Egerton, Christo Lolov, Colin P. McCabe, Cong Ding, Daniel
> Urban, David Arthur, David Jacot, David Mao, Dmitriy Fishman, Edoardo
> Comar, Ewen Cheslack-Postava, Greg Harris, Guozhang Wang, Igor Soarez,
> Ismael Juma, Israel Ekpo, Ivan Ponomarev, Jakub Scholz, James Galasyn,
> Jason Gustafson, Jeff Kim, Jim Galasyn, JoeCqupt, Joel Hamill, John
> Gray, John Roesler, Jongho Jeon, Jorge Esteban Quilcate Otoya, Jose
> Sancio, Josep Prat, José Armando García Sancio, Jun Rao, Justine
> Olshan, Kalpesh Patel, Kamal Chandraprakash, Kevin Zhang, Kirk True,
> Konstantine Karantasis, Kowshik Prakasam, Leah Thomas, Lee Dongjin,
> Lucas Bradstreet, Luke Chen, Manikumar Reddy, Matthew Wong, Matthias
> J. Sax, Michael Carter, Mickael Maison, Nigel Liang, Niket, Niket
> Goel, Oliver Hutchison, Omnia G H Ibrahim, Patrick Stuedi, Phil
> Hardwick, Prateek Agarwal, Rajini Sivaram, Randall Hauch, René Kerner,
> Richard Yu, Rohan, Ron Dagostino, Ryan Dielhenn, Sanjana Kaundinya,
> Satish Duggana, Sergio Peña, Sherzod Mamadaliev, Stanislav Vodetskyi,
> Ted Yu, Tom Bentley, Tomas Forsman, Tomer Wizman, Uwe Eisele, Victoria
> Xia, Viktor Somogyi-Vass, Vincent Jiang, Walker Carlson, Weisheng
> Yang, Xavier Léauté, Yanwen(Jason) Lin, Yi Ding, Zara Lim, andy0x01,
> dengziming, feyman2016, ik, ik.lim, jem, jiangyuan, kpatelatwork,
> leah, loboya~, lujiefsi, sebbASF, singingMan, vamossagar12,
> wenbingshen
>
> We welcome your help and feedback. For more information on how to
> report problems, and to get involved, visit the project website at
> https://kafka.apache.org/
>
> Thank you!
>
>
> Regards,
>
> David
Re: [ANNOUNCE] Apache Kafka 3.1.0
Posted by Israel Ekpo <is...@gmail.com>.
Bitfox,
This is a great question, however it should be sent as a separate query to
the mailing list instead of hijacking the release announcement.
Currently, it might be possible to use these 4 things together but it is
just a coincidence that they are all with similar version numbers. They
version number is not a requirement or proof of compatibility
Take a look at what your project ecosystem components need and then you
will figure out which versions of Kafka, Spark, C#, Rust, JavaScript, Ruby,
Java, Scala or Python to use for your projects.
You can still use Python 3 clients with versions of Kafka in the 2.x
series and it should still work if that is what you need.
Java 8 support has been deprecated and you should be using Java 11 or 17
for the latest versions of Kafka.
I hope this helps but next time if you have a different question from what
the current thread is discussing it is proper etiquette to start a new
thread instead of hijacking the current thread.
This may have been a mistake but I just wanted to point it out.
Israel Ekpo
Lead Instructor, IzzyAcademy.com
https://www.youtube.com/c/izzyacademy
https://izzyacademy.com/
On Mon, Jan 24, 2022 at 11:27 PM Bitfox <bi...@bitfox.top> wrote:
> Must spark3, kafka3, scala3, python3 work together if my project used these
> stacks?
>
> Thanks
>
> On Tue, Jan 25, 2022 at 1:04 AM David Jacot <da...@apache.org> wrote:
>
> > The Apache Kafka community is pleased to announce the release for
> > Apache Kafka 3.1.0.
> >
> > It is a major release that includes many new features, including:
> >
> > * Apache Kafka supports Java 17
> > * The FetchRequest supports Topic IDs (KIP-516)
> > * Extend SASL/OAUTHBEARER with support for OIDC (KIP-768)
> > * Add broker count metrics (KIP-748)
> > * Differentiate consistently metric latency measured in millis and
> > nanos (KIP-773)
> > * The eager rebalance protocol is deprecated (KAFKA-13439)
> > * Add TaskId field to StreamsException (KIP-783)
> > * Custom partitioners in foreign-key joins (KIP-775)
> > * Fetch/findSessions queries with open endpoints for
> > SessionStore/WindowStore (KIP-766)
> > * Range queries with open endpoints (KIP-763)
> > * Add total blocked time metric to Streams (KIP-761)
> > * Add additional configuration to control MirrorMaker2 internal topics
> > naming convention (KIP-690)
> >
> > You may read a more detailed list of features in the 3.1.0 blog post:
> > https://blogs.apache.org/kafka/
> >
> > All of the changes in this release can be found in the release notes:
> > https://www.apache.org/dist/kafka/3.1.0/RELEASE_NOTES.html
> >
> > You can download the source and binary release (Scala 2.12 and 2.13)
> from:
> > https://kafka.apache.org/downloads#3.1.0
> >
> >
> >
> ---------------------------------------------------------------------------------------------------
> >
> >
> > Apache Kafka is a distributed streaming platform with four core APIs:
> >
> > ** The Producer API allows an application to publish a stream of records
> to
> > one or more Kafka topics.
> >
> > ** The Consumer API allows an application to subscribe to one or more
> > topics and process the stream of records produced to them.
> >
> > ** The Streams API allows an application to act as a stream processor,
> > consuming an input stream from one or more topics and producing an
> > output stream to one or more output topics, effectively transforming the
> > input streams to output streams.
> >
> > ** The Connector API allows building and running reusable producers or
> > consumers that connect Kafka topics to existing applications or data
> > systems. For example, a connector to a relational database might
> > capture every change to a table.
> >
> >
> > With these APIs, Kafka can be used for two broad classes of application:
> >
> > ** Building real-time streaming data pipelines that reliably get data
> > between systems or applications.
> >
> > ** Building real-time streaming applications that transform or react
> > to the streams of data.
> >
> >
> > Apache Kafka is in use at large and small companies worldwide, including
> > Capital One, Goldman Sachs, ING, LinkedIn, Netflix, Pinterest, Rabobank,
> > Target, The New York Times, Uber, Yelp, and Zalando, among others.
> >
> > A big thank you for the following 114 contributors to this release!
> >
> > A. Sophie Blee-Goldman, Alexander Iskuskov, Alexander Stohr, Almog
> > Gavra, Andras Katona, Andrew Patterson, Andy Chambers, Andy Lapidas,
> > Anna Sophie Blee-Goldman, Antony Stubbs, Arjun Satish, Bill Bejeck,
> > Boyang Chen, Bruno Cadonna, CHUN-HAO TANG, Cheng Tan, Chia-Ping Tsai,
> > Chris Egerton, Christo Lolov, Colin P. McCabe, Cong Ding, Daniel
> > Urban, David Arthur, David Jacot, David Mao, Dmitriy Fishman, Edoardo
> > Comar, Ewen Cheslack-Postava, Greg Harris, Guozhang Wang, Igor Soarez,
> > Ismael Juma, Israel Ekpo, Ivan Ponomarev, Jakub Scholz, James Galasyn,
> > Jason Gustafson, Jeff Kim, Jim Galasyn, JoeCqupt, Joel Hamill, John
> > Gray, John Roesler, Jongho Jeon, Jorge Esteban Quilcate Otoya, Jose
> > Sancio, Josep Prat, José Armando García Sancio, Jun Rao, Justine
> > Olshan, Kalpesh Patel, Kamal Chandraprakash, Kevin Zhang, Kirk True,
> > Konstantine Karantasis, Kowshik Prakasam, Leah Thomas, Lee Dongjin,
> > Lucas Bradstreet, Luke Chen, Manikumar Reddy, Matthew Wong, Matthias
> > J. Sax, Michael Carter, Mickael Maison, Nigel Liang, Niket, Niket
> > Goel, Oliver Hutchison, Omnia G H Ibrahim, Patrick Stuedi, Phil
> > Hardwick, Prateek Agarwal, Rajini Sivaram, Randall Hauch, René Kerner,
> > Richard Yu, Rohan, Ron Dagostino, Ryan Dielhenn, Sanjana Kaundinya,
> > Satish Duggana, Sergio Peña, Sherzod Mamadaliev, Stanislav Vodetskyi,
> > Ted Yu, Tom Bentley, Tomas Forsman, Tomer Wizman, Uwe Eisele, Victoria
> > Xia, Viktor Somogyi-Vass, Vincent Jiang, Walker Carlson, Weisheng
> > Yang, Xavier Léauté, Yanwen(Jason) Lin, Yi Ding, Zara Lim, andy0x01,
> > dengziming, feyman2016, ik, ik.lim, jem, jiangyuan, kpatelatwork,
> > leah, loboya~, lujiefsi, sebbASF, singingMan, vamossagar12,
> > wenbingshen
> >
> > We welcome your help and feedback. For more information on how to
> > report problems, and to get involved, visit the project website at
> > https://kafka.apache.org/
> >
> > Thank you!
> >
> >
> > Regards,
> >
> > David
> >
>
Re: [ANNOUNCE] Apache Kafka 3.1.0
Posted by Bitfox <bi...@bitfox.top>.
Must spark3, kafka3, scala3, python3 work together if my project used these
stacks?
Thanks
On Tue, Jan 25, 2022 at 1:04 AM David Jacot <da...@apache.org> wrote:
> The Apache Kafka community is pleased to announce the release for
> Apache Kafka 3.1.0.
>
> It is a major release that includes many new features, including:
>
> * Apache Kafka supports Java 17
> * The FetchRequest supports Topic IDs (KIP-516)
> * Extend SASL/OAUTHBEARER with support for OIDC (KIP-768)
> * Add broker count metrics (KIP-748)
> * Differentiate consistently metric latency measured in millis and
> nanos (KIP-773)
> * The eager rebalance protocol is deprecated (KAFKA-13439)
> * Add TaskId field to StreamsException (KIP-783)
> * Custom partitioners in foreign-key joins (KIP-775)
> * Fetch/findSessions queries with open endpoints for
> SessionStore/WindowStore (KIP-766)
> * Range queries with open endpoints (KIP-763)
> * Add total blocked time metric to Streams (KIP-761)
> * Add additional configuration to control MirrorMaker2 internal topics
> naming convention (KIP-690)
>
> You may read a more detailed list of features in the 3.1.0 blog post:
> https://blogs.apache.org/kafka/
>
> All of the changes in this release can be found in the release notes:
> https://www.apache.org/dist/kafka/3.1.0/RELEASE_NOTES.html
>
> You can download the source and binary release (Scala 2.12 and 2.13) from:
> https://kafka.apache.org/downloads#3.1.0
>
>
> ---------------------------------------------------------------------------------------------------
>
>
> Apache Kafka is a distributed streaming platform with four core APIs:
>
> ** The Producer API allows an application to publish a stream of records to
> one or more Kafka topics.
>
> ** The Consumer API allows an application to subscribe to one or more
> topics and process the stream of records produced to them.
>
> ** The Streams API allows an application to act as a stream processor,
> consuming an input stream from one or more topics and producing an
> output stream to one or more output topics, effectively transforming the
> input streams to output streams.
>
> ** The Connector API allows building and running reusable producers or
> consumers that connect Kafka topics to existing applications or data
> systems. For example, a connector to a relational database might
> capture every change to a table.
>
>
> With these APIs, Kafka can be used for two broad classes of application:
>
> ** Building real-time streaming data pipelines that reliably get data
> between systems or applications.
>
> ** Building real-time streaming applications that transform or react
> to the streams of data.
>
>
> Apache Kafka is in use at large and small companies worldwide, including
> Capital One, Goldman Sachs, ING, LinkedIn, Netflix, Pinterest, Rabobank,
> Target, The New York Times, Uber, Yelp, and Zalando, among others.
>
> A big thank you for the following 114 contributors to this release!
>
> A. Sophie Blee-Goldman, Alexander Iskuskov, Alexander Stohr, Almog
> Gavra, Andras Katona, Andrew Patterson, Andy Chambers, Andy Lapidas,
> Anna Sophie Blee-Goldman, Antony Stubbs, Arjun Satish, Bill Bejeck,
> Boyang Chen, Bruno Cadonna, CHUN-HAO TANG, Cheng Tan, Chia-Ping Tsai,
> Chris Egerton, Christo Lolov, Colin P. McCabe, Cong Ding, Daniel
> Urban, David Arthur, David Jacot, David Mao, Dmitriy Fishman, Edoardo
> Comar, Ewen Cheslack-Postava, Greg Harris, Guozhang Wang, Igor Soarez,
> Ismael Juma, Israel Ekpo, Ivan Ponomarev, Jakub Scholz, James Galasyn,
> Jason Gustafson, Jeff Kim, Jim Galasyn, JoeCqupt, Joel Hamill, John
> Gray, John Roesler, Jongho Jeon, Jorge Esteban Quilcate Otoya, Jose
> Sancio, Josep Prat, José Armando García Sancio, Jun Rao, Justine
> Olshan, Kalpesh Patel, Kamal Chandraprakash, Kevin Zhang, Kirk True,
> Konstantine Karantasis, Kowshik Prakasam, Leah Thomas, Lee Dongjin,
> Lucas Bradstreet, Luke Chen, Manikumar Reddy, Matthew Wong, Matthias
> J. Sax, Michael Carter, Mickael Maison, Nigel Liang, Niket, Niket
> Goel, Oliver Hutchison, Omnia G H Ibrahim, Patrick Stuedi, Phil
> Hardwick, Prateek Agarwal, Rajini Sivaram, Randall Hauch, René Kerner,
> Richard Yu, Rohan, Ron Dagostino, Ryan Dielhenn, Sanjana Kaundinya,
> Satish Duggana, Sergio Peña, Sherzod Mamadaliev, Stanislav Vodetskyi,
> Ted Yu, Tom Bentley, Tomas Forsman, Tomer Wizman, Uwe Eisele, Victoria
> Xia, Viktor Somogyi-Vass, Vincent Jiang, Walker Carlson, Weisheng
> Yang, Xavier Léauté, Yanwen(Jason) Lin, Yi Ding, Zara Lim, andy0x01,
> dengziming, feyman2016, ik, ik.lim, jem, jiangyuan, kpatelatwork,
> leah, loboya~, lujiefsi, sebbASF, singingMan, vamossagar12,
> wenbingshen
>
> We welcome your help and feedback. For more information on how to
> report problems, and to get involved, visit the project website at
> https://kafka.apache.org/
>
> Thank you!
>
>
> Regards,
>
> David
>