You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@kafka.apache.org by Bruno Cadonna <br...@confluent.io> on 2020/03/09 20:43:49 UTC
Re: [DISCUSS] Apache Kafka 2.5.0 release
Hi David,
A bug report was filed that can be considered a blocker. Basically,
with this bug all RocksDB metrics reported by Streams are constant
zero. The bug is there since 2.4., so it is not a regression, but a
broken feature.
Here is the ticket: https://issues.apache.org/jira/browse/KAFKA-9675
Here is the fix: https://github.com/apache/kafka/pull/8256
Best,
Bruno
On Wed, Feb 26, 2020 at 11:22 PM Randall Hauch <rh...@gmail.com> wrote:
>
> Thanks, David. The PR has been merged to trunk and 2.5, and I'm backporting
> to earlier branches. I'll resolve
> https://issues.apache.org/jira/browse/KAFKA-9601 when I finish backporting.
>
> On Wed, Feb 26, 2020 at 1:28 PM David Arthur <mu...@gmail.com> wrote:
>
> > Thanks, Randall. Leaking sensitive config to the logs seems fairly
> > severe. I think should include this. Let's proceed with cherry-picking to
> > 2.5.
> >
> > -David
> >
> > On Wed, Feb 26, 2020 at 2:25 PM Randall Hauch <rh...@gmail.com> wrote:
> >
> > > Hi, David.
> > >
> > > If we're still not quite ready for an RC, I'd like to squeeze in
> > > https://issues.apache.org/jira/browse/KAFKA-9601, which removes the raw
> > > connector config properties in a DEBUG level log message. PR is ready
> > (test
> > > failures are unrelated), the risk is very low, and I think it'd be great
> > to
> > > correct this sooner than later.
> > >
> > > Randall
> > >
> > > On Wed, Feb 26, 2020 at 11:26 AM David Arthur <mu...@gmail.com> wrote:
> > >
> > > > Viktor, the change LGTM. I've approved and merged the cherry-pick
> > version
> > > > into 2.5.
> > > >
> > > > Thanks!
> > > > David
> > > >
> > > > On Tue, Feb 25, 2020 at 4:43 AM Viktor Somogyi-Vass <
> > > > viktorsomogyi@gmail.com>
> > > > wrote:
> > > >
> > > > > Hi David,
> > > > >
> > > > > There are two short JIRAs related to KIP-352 that documents the newly
> > > > added
> > > > > metrics. Is it possible to merge them in?
> > > > > https://github.com/apache/kafka/pull/7434 (trunk)
> > > > > https://github.com/apache/kafka/pull/8127 (2.5 cherry-pick)
> > > > >
> > > > > Thanks,
> > > > > Viktor
> > > > >
> > > > >
> > > > > On Mon, Feb 24, 2020 at 7:22 PM David Arthur <mu...@gmail.com>
> > wrote:
> > > > >
> > > > > > Thanks, Tu. I've moved KIP-467 out of the release plan.
> > > > > >
> > > > > > -David
> > > > > >
> > > > > > On Thu, Feb 20, 2020 at 6:00 PM Tu Tran <tu...@confluent.io> wrote:
> > > > > >
> > > > > > > Hi David,
> > > > > > >
> > > > > > > Thanks for being the release main driver. Since the
> > implementation
> > > > for
> > > > > > the
> > > > > > > last part of KIP-467 wasn't finalized prior to Feb 12th, could
> > you
> > > > > remove
> > > > > > > KIP-467 from the list?
> > > > > > >
> > > > > > > Thanks,
> > > > > > > Tu
> > > > > > >
> > > > > > > On Thu, Feb 20, 2020 at 7:18 AM David Arthur <mu...@gmail.com>
> > > > wrote:
> > > > > > >
> > > > > > > > Randall / Konstantine,
> > > > > > > >
> > > > > > > > Sorry for the late reply. Thanks for the fix and for the
> > update!
> > > I
> > > > > see
> > > > > > > this
> > > > > > > > change on the 2.5 branch (@b403c66). Consider this a
> > retroactive
> > > > > > approval
> > > > > > > > for this bugfix :)
> > > > > > > >
> > > > > > > > -David
> > > > > > > >
> > > > > > > > On Fri, Feb 14, 2020 at 2:21 PM Konstantine Karantasis <
> > > > > > > > konstantine@confluent.io> wrote:
> > > > > > > >
> > > > > > > > > Hi David,
> > > > > > > > >
> > > > > > > > > I want to confirm what Randall mentions above. The code fixes
> > > for
> > > > > > > > > KAFKA-9556 were in place before code freeze on Wed, but we
> > > spent
> > > > a
> > > > > > bit
> > > > > > > > more
> > > > > > > > > time hardening the conditions of the integration tests and
> > > fixing
> > > > > > some
> > > > > > > > > jenkins branch builders to run the test on repeat.
> > > > > > > > >
> > > > > > > > > Best,
> > > > > > > > > Konstantine
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > On Fri, Feb 14, 2020 at 7:42 AM Randall Hauch <
> > > rhauch@gmail.com>
> > > > > > > wrote:
> > > > > > > > >
> > > > > > > > > > Hi, David.
> > > > > > > > > >
> > > > > > > > > > I just filed
> > > https://issues.apache.org/jira/browse/KAFKA-9556
> > > > > that
> > > > > > > > > > identifies two pretty minor issues with the new KIP-558
> > that
> > > > adds
> > > > > > new
> > > > > > > > > > Connect REST API endpoints to get the list of topics used
> > by
> > > a
> > > > > > > > connector.
> > > > > > > > > > The impact is high: the feature cannot be fully disabled,
> > and
> > > > > > Connect
> > > > > > > > > does
> > > > > > > > > > not automatically reset the topic set when a connector is
> > > > > deleted.
> > > > > > > > > > https://github.com/apache/kafka/pull/8085 includes the two
> > > > > fixes,
> > > > > > > and
> > > > > > > > > also
> > > > > > > > > > adds more unit and integration tests for this feature.
> > > > Although I
> > > > > > > just
> > > > > > > > > > created the blocker this AM, Konstantine has actually be
> > > > working
> > > > > on
> > > > > > > the
> > > > > > > > > fix
> > > > > > > > > > for four days. Risk of merging this PR is low, since a) the
> > > new
> > > > > > > > > integration
> > > > > > > > > > tests add significant coverage and we've run the new tests
> > > > > numerous
> > > > > > > > > times,
> > > > > > > > > > and b) the fixes help gate the new feature even more and
> > > allow
> > > > > the
> > > > > > > > > feature
> > > > > > > > > > to be completely disabled.
> > > > > > > > > >
> > > > > > > > > > I'd like approve to merge
> > > > > > https://github.com/apache/kafka/pull/8085
> > > > > > > > > >
> > > > > > > > > > Thanks!
> > > > > > > > > > Randall
> > > > > > > > > >
> > > > > > > > > > On Mon, Feb 10, 2020 at 11:31 AM David Arthur <
> > > > mumrah@gmail.com>
> > > > > > > > wrote:
> > > > > > > > > >
> > > > > > > > > > > Just a friendly reminder that this Wednesday, February
> > > 12th,
> > > > is
> > > > > > the
> > > > > > > > > code
> > > > > > > > > > > freeze for the 2.5.0 release. After this time we will
> > only
> > > > > accept
> > > > > > > > > blocker
> > > > > > > > > > > bugs onto the release branch.
> > > > > > > > > > >
> > > > > > > > > > > Thanks!
> > > > > > > > > > > David
> > > > > > > > > > >
> > > > > > > > > > > On Fri, Jan 31, 2020 at 5:13 PM David Arthur <
> > > > mumrah@gmail.com
> > > > > >
> > > > > > > > wrote:
> > > > > > > > > > >
> > > > > > > > > > > > Thanks! I've updated the list.
> > > > > > > > > > > >
> > > > > > > > > > > > On Thu, Jan 30, 2020 at 5:48 PM Konstantine Karantasis
> > <
> > > > > > > > > > > > konstantine@confluent.io> wrote:
> > > > > > > > > > > >
> > > > > > > > > > > >> Hi David,
> > > > > > > > > > > >>
> > > > > > > > > > > >> thanks for driving the release.
> > > > > > > > > > > >>
> > > > > > > > > > > >> Please also remove KIP-158 from the list of KIPs that
> > > you
> > > > > plan
> > > > > > > to
> > > > > > > > > > > include
> > > > > > > > > > > >> in 2.5
> > > > > > > > > > > >> KIP-158 has been accepted, but the implementation is
> > not
> > > > yet
> > > > > > > > final.
> > > > > > > > > It
> > > > > > > > > > > >> will be included in the release that follows 2.5.
> > > > > > > > > > > >>
> > > > > > > > > > > >> Regards,
> > > > > > > > > > > >> Konstantine
> > > > > > > > > > > >>
> > > > > > > > > > > >> On 1/30/20, Matthias J. Sax <ma...@confluent.io>
> > > > wrote:
> > > > > > > > > > > >> > Hi David,
> > > > > > > > > > > >> >
> > > > > > > > > > > >> > the following KIP from the list did not make it:
> > > > > > > > > > > >> >
> > > > > > > > > > > >> > - KIP-216 (no PR yet)
> > > > > > > > > > > >> > - KIP-399 (no PR yet)
> > > > > > > > > > > >> > - KIP-401 (PR not merged yet)
> > > > > > > > > > > >> >
> > > > > > > > > > > >> >
> > > > > > > > > > > >> > KIP-444 should be included as we did make progress,
> > > but
> > > > it
> > > > > > is
> > > > > > > > > still
> > > > > > > > > > > not
> > > > > > > > > > > >> > fully implement and we need to finish in in 2.6
> > > release.
> > > > > > > > > > > >> >
> > > > > > > > > > > >> > KIP-447 is partially implemented in 2.5 (ie, broker
> > > and
> > > > > > > > > > > >> > consumer/producer changes -- the Kafka Streams parts
> > > > slip)
> > > > > > > > > > > >> >
> > > > > > > > > > > >> >
> > > > > > > > > > > >> > -Matthias
> > > > > > > > > > > >> >
> > > > > > > > > > > >> >
> > > > > > > > > > > >> > On 1/29/20 9:05 AM, David Arthur wrote:
> > > > > > > > > > > >> >> Hey everyone, just a quick update on the 2.5
> > release.
> > > > > > > > > > > >> >>
> > > > > > > > > > > >> >> I have updated the list of planned KIPs on the
> > > release
> > > > > wiki
> > > > > > > > page
> > > > > > > > > > > >> >>
> > > > > > > > > > > >>
> > > > > > > > > > >
> > > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> > https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=143428858
> > > > > > > > > > > >> .
> > > > > > > > > > > >> >> If I have missed anything, or there are KIPs
> > included
> > > > in
> > > > > > this
> > > > > > > > > list
> > > > > > > > > > > >> which
> > > > > > > > > > > >> >> should *not* be included in 2.5, please let me
> > know.
> > > > > > > > > > > >> >>
> > > > > > > > > > > >> >> Based on the release schedule, the feature freeze
> > is
> > > > > today,
> > > > > > > Jan
> > > > > > > > > > 29th.
> > > > > > > > > > > >> Any
> > > > > > > > > > > >> >> major feature work that is not already complete
> > will
> > > > need
> > > > > > to
> > > > > > > > push
> > > > > > > > > > out
> > > > > > > > > > > >> to
> > > > > > > > > > > >> >> 2.6. I will work on cutting the release branch
> > during
> > > > the
> > > > > > day
> > > > > > > > > > > tomorrow
> > > > > > > > > > > >> >> (Jan
> > > > > > > > > > > >> >> 30th).
> > > > > > > > > > > >> >>
> > > > > > > > > > > >> >> If you have any questions, please feel free to
> > reach
> > > > out
> > > > > to
> > > > > > > me
> > > > > > > > > > > directly
> > > > > > > > > > > >> >> or
> > > > > > > > > > > >> >> in this thread.
> > > > > > > > > > > >> >>
> > > > > > > > > > > >> >> Thanks!
> > > > > > > > > > > >> >> David
> > > > > > > > > > > >> >>
> > > > > > > > > > > >> >> On Mon, Jan 13, 2020 at 1:35 PM Colin McCabe <
> > > > > > > > cmccabe@apache.org
> > > > > > > > > >
> > > > > > > > > > > >> wrote:
> > > > > > > > > > > >> >>
> > > > > > > > > > > >> >>> +1. Thanks for volunteering, David.
> > > > > > > > > > > >> >>>
> > > > > > > > > > > >> >>> best,
> > > > > > > > > > > >> >>> Colin
> > > > > > > > > > > >> >>>
> > > > > > > > > > > >> >>> On Fri, Dec 20, 2019, at 10:59, David Arthur
> > wrote:
> > > > > > > > > > > >> >>>> Greetings!
> > > > > > > > > > > >> >>>>
> > > > > > > > > > > >> >>>> I'd like to volunteer to be release manager for
> > the
> > > > > next
> > > > > > > > > > time-based
> > > > > > > > > > > >> >>> feature
> > > > > > > > > > > >> >>>> release which will be 2.5. If there are no
> > > > objections,
> > > > > > I'll
> > > > > > > > > send
> > > > > > > > > > > out
> > > > > > > > > > > >> >>>> the
> > > > > > > > > > > >> >>>> release plan in the next few days.
> > > > > > > > > > > >> >>>>
> > > > > > > > > > > >> >>>> Thanks,
> > > > > > > > > > > >> >>>> David Arthur
> > > > > > > > > > > >> >>>>
> > > > > > > > > > > >> >>>
> > > > > > > > > > > >> >>
> > > > > > > > > > > >> >>
> > > > > > > > > > > >> >
> > > > > > > > > > > >> >
> > > > > > > > > > > >>
> > > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > > --
> > > > > > > > > > > > David Arthur
> > > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > --
> > > > > > > > > > > David Arthur
> > > > > > > > > > >
> > > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > > --
> > > > > > > > David Arthur
> > > > > > > >
> > > > > > >
> > > > > >
> > > > > >
> > > > > > --
> > > > > > David Arthur
> > > > > >
> > > > >
> > > >
> > > >
> > > > --
> > > > David Arthur
> > > >
> > >
> >
> >
> > --
> > David Arthur
> >
Re: [DISCUSS] Apache Kafka 2.5.0 release
Posted by Bruno Cadonna <br...@confluent.io>.
Hi Ismael,
I understand your reasoning. I thought, since it is a broken feature,
I better let this thread know about it.
Best,
Bruno
On Mon, Mar 9, 2020 at 10:00 PM Ismael Juma <is...@juma.me.uk> wrote:
>
> Hi Bruno,
>
> 2.4.0 has been out for months. If this was not noticed since then, I don't
> think it qualifies as a blocker for 2.5.0.
>
> Ismael
>
> On Mon, Mar 9, 2020 at 8:44 PM Bruno Cadonna <br...@confluent.io> wrote:
>
> > Hi David,
> >
> > A bug report was filed that can be considered a blocker. Basically,
> > with this bug all RocksDB metrics reported by Streams are constant
> > zero. The bug is there since 2.4., so it is not a regression, but a
> > broken feature.
> >
> > Here is the ticket: https://issues.apache.org/jira/browse/KAFKA-9675
> > Here is the fix: https://github.com/apache/kafka/pull/8256
> >
> > Best,
> > Bruno
> >
> > On Wed, Feb 26, 2020 at 11:22 PM Randall Hauch <rh...@gmail.com> wrote:
> > >
> > > Thanks, David. The PR has been merged to trunk and 2.5, and I'm
> > backporting
> > > to earlier branches. I'll resolve
> > > https://issues.apache.org/jira/browse/KAFKA-9601 when I finish
> > backporting.
> > >
> > > On Wed, Feb 26, 2020 at 1:28 PM David Arthur <mu...@gmail.com> wrote:
> > >
> > > > Thanks, Randall. Leaking sensitive config to the logs seems fairly
> > > > severe. I think should include this. Let's proceed with cherry-picking
> > to
> > > > 2.5.
> > > >
> > > > -David
> > > >
> > > > On Wed, Feb 26, 2020 at 2:25 PM Randall Hauch <rh...@gmail.com>
> > wrote:
> > > >
> > > > > Hi, David.
> > > > >
> > > > > If we're still not quite ready for an RC, I'd like to squeeze in
> > > > > https://issues.apache.org/jira/browse/KAFKA-9601, which removes the
> > raw
> > > > > connector config properties in a DEBUG level log message. PR is ready
> > > > (test
> > > > > failures are unrelated), the risk is very low, and I think it'd be
> > great
> > > > to
> > > > > correct this sooner than later.
> > > > >
> > > > > Randall
> > > > >
> > > > > On Wed, Feb 26, 2020 at 11:26 AM David Arthur <mu...@gmail.com>
> > wrote:
> > > > >
> > > > > > Viktor, the change LGTM. I've approved and merged the cherry-pick
> > > > version
> > > > > > into 2.5.
> > > > > >
> > > > > > Thanks!
> > > > > > David
> > > > > >
> > > > > > On Tue, Feb 25, 2020 at 4:43 AM Viktor Somogyi-Vass <
> > > > > > viktorsomogyi@gmail.com>
> > > > > > wrote:
> > > > > >
> > > > > > > Hi David,
> > > > > > >
> > > > > > > There are two short JIRAs related to KIP-352 that documents the
> > newly
> > > > > > added
> > > > > > > metrics. Is it possible to merge them in?
> > > > > > > https://github.com/apache/kafka/pull/7434 (trunk)
> > > > > > > https://github.com/apache/kafka/pull/8127 (2.5 cherry-pick)
> > > > > > >
> > > > > > > Thanks,
> > > > > > > Viktor
> > > > > > >
> > > > > > >
> > > > > > > On Mon, Feb 24, 2020 at 7:22 PM David Arthur <mu...@gmail.com>
> > > > wrote:
> > > > > > >
> > > > > > > > Thanks, Tu. I've moved KIP-467 out of the release plan.
> > > > > > > >
> > > > > > > > -David
> > > > > > > >
> > > > > > > > On Thu, Feb 20, 2020 at 6:00 PM Tu Tran <tu...@confluent.io>
> > wrote:
> > > > > > > >
> > > > > > > > > Hi David,
> > > > > > > > >
> > > > > > > > > Thanks for being the release main driver. Since the
> > > > implementation
> > > > > > for
> > > > > > > > the
> > > > > > > > > last part of KIP-467 wasn't finalized prior to Feb 12th,
> > could
> > > > you
> > > > > > > remove
> > > > > > > > > KIP-467 from the list?
> > > > > > > > >
> > > > > > > > > Thanks,
> > > > > > > > > Tu
> > > > > > > > >
> > > > > > > > > On Thu, Feb 20, 2020 at 7:18 AM David Arthur <
> > mumrah@gmail.com>
> > > > > > wrote:
> > > > > > > > >
> > > > > > > > > > Randall / Konstantine,
> > > > > > > > > >
> > > > > > > > > > Sorry for the late reply. Thanks for the fix and for the
> > > > update!
> > > > > I
> > > > > > > see
> > > > > > > > > this
> > > > > > > > > > change on the 2.5 branch (@b403c66). Consider this a
> > > > retroactive
> > > > > > > > approval
> > > > > > > > > > for this bugfix :)
> > > > > > > > > >
> > > > > > > > > > -David
> > > > > > > > > >
> > > > > > > > > > On Fri, Feb 14, 2020 at 2:21 PM Konstantine Karantasis <
> > > > > > > > > > konstantine@confluent.io> wrote:
> > > > > > > > > >
> > > > > > > > > > > Hi David,
> > > > > > > > > > >
> > > > > > > > > > > I want to confirm what Randall mentions above. The code
> > fixes
> > > > > for
> > > > > > > > > > > KAFKA-9556 were in place before code freeze on Wed, but
> > we
> > > > > spent
> > > > > > a
> > > > > > > > bit
> > > > > > > > > > more
> > > > > > > > > > > time hardening the conditions of the integration tests
> > and
> > > > > fixing
> > > > > > > > some
> > > > > > > > > > > jenkins branch builders to run the test on repeat.
> > > > > > > > > > >
> > > > > > > > > > > Best,
> > > > > > > > > > > Konstantine
> > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > On Fri, Feb 14, 2020 at 7:42 AM Randall Hauch <
> > > > > rhauch@gmail.com>
> > > > > > > > > wrote:
> > > > > > > > > > >
> > > > > > > > > > > > Hi, David.
> > > > > > > > > > > >
> > > > > > > > > > > > I just filed
> > > > > https://issues.apache.org/jira/browse/KAFKA-9556
> > > > > > > that
> > > > > > > > > > > > identifies two pretty minor issues with the new KIP-558
> > > > that
> > > > > > adds
> > > > > > > > new
> > > > > > > > > > > > Connect REST API endpoints to get the list of topics
> > used
> > > > by
> > > > > a
> > > > > > > > > > connector.
> > > > > > > > > > > > The impact is high: the feature cannot be fully
> > disabled,
> > > > and
> > > > > > > > Connect
> > > > > > > > > > > does
> > > > > > > > > > > > not automatically reset the topic set when a connector
> > is
> > > > > > > deleted.
> > > > > > > > > > > > https://github.com/apache/kafka/pull/8085 includes
> > the two
> > > > > > > fixes,
> > > > > > > > > and
> > > > > > > > > > > also
> > > > > > > > > > > > adds more unit and integration tests for this feature.
> > > > > > Although I
> > > > > > > > > just
> > > > > > > > > > > > created the blocker this AM, Konstantine has actually
> > be
> > > > > > working
> > > > > > > on
> > > > > > > > > the
> > > > > > > > > > > fix
> > > > > > > > > > > > for four days. Risk of merging this PR is low, since
> > a) the
> > > > > new
> > > > > > > > > > > integration
> > > > > > > > > > > > tests add significant coverage and we've run the new
> > tests
> > > > > > > numerous
> > > > > > > > > > > times,
> > > > > > > > > > > > and b) the fixes help gate the new feature even more
> > and
> > > > > allow
> > > > > > > the
> > > > > > > > > > > feature
> > > > > > > > > > > > to be completely disabled.
> > > > > > > > > > > >
> > > > > > > > > > > > I'd like approve to merge
> > > > > > > > https://github.com/apache/kafka/pull/8085
> > > > > > > > > > > >
> > > > > > > > > > > > Thanks!
> > > > > > > > > > > > Randall
> > > > > > > > > > > >
> > > > > > > > > > > > On Mon, Feb 10, 2020 at 11:31 AM David Arthur <
> > > > > > mumrah@gmail.com>
> > > > > > > > > > wrote:
> > > > > > > > > > > >
> > > > > > > > > > > > > Just a friendly reminder that this Wednesday,
> > February
> > > > > 12th,
> > > > > > is
> > > > > > > > the
> > > > > > > > > > > code
> > > > > > > > > > > > > freeze for the 2.5.0 release. After this time we will
> > > > only
> > > > > > > accept
> > > > > > > > > > > blocker
> > > > > > > > > > > > > bugs onto the release branch.
> > > > > > > > > > > > >
> > > > > > > > > > > > > Thanks!
> > > > > > > > > > > > > David
> > > > > > > > > > > > >
> > > > > > > > > > > > > On Fri, Jan 31, 2020 at 5:13 PM David Arthur <
> > > > > > mumrah@gmail.com
> > > > > > > >
> > > > > > > > > > wrote:
> > > > > > > > > > > > >
> > > > > > > > > > > > > > Thanks! I've updated the list.
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > On Thu, Jan 30, 2020 at 5:48 PM Konstantine
> > Karantasis
> > > > <
> > > > > > > > > > > > > > konstantine@confluent.io> wrote:
> > > > > > > > > > > > > >
> > > > > > > > > > > > > >> Hi David,
> > > > > > > > > > > > > >>
> > > > > > > > > > > > > >> thanks for driving the release.
> > > > > > > > > > > > > >>
> > > > > > > > > > > > > >> Please also remove KIP-158 from the list of KIPs
> > that
> > > > > you
> > > > > > > plan
> > > > > > > > > to
> > > > > > > > > > > > > include
> > > > > > > > > > > > > >> in 2.5
> > > > > > > > > > > > > >> KIP-158 has been accepted, but the implementation
> > is
> > > > not
> > > > > > yet
> > > > > > > > > > final.
> > > > > > > > > > > It
> > > > > > > > > > > > > >> will be included in the release that follows 2.5.
> > > > > > > > > > > > > >>
> > > > > > > > > > > > > >> Regards,
> > > > > > > > > > > > > >> Konstantine
> > > > > > > > > > > > > >>
> > > > > > > > > > > > > >> On 1/30/20, Matthias J. Sax <
> > matthias@confluent.io>
> > > > > > wrote:
> > > > > > > > > > > > > >> > Hi David,
> > > > > > > > > > > > > >> >
> > > > > > > > > > > > > >> > the following KIP from the list did not make it:
> > > > > > > > > > > > > >> >
> > > > > > > > > > > > > >> > - KIP-216 (no PR yet)
> > > > > > > > > > > > > >> > - KIP-399 (no PR yet)
> > > > > > > > > > > > > >> > - KIP-401 (PR not merged yet)
> > > > > > > > > > > > > >> >
> > > > > > > > > > > > > >> >
> > > > > > > > > > > > > >> > KIP-444 should be included as we did make
> > progress,
> > > > > but
> > > > > > it
> > > > > > > > is
> > > > > > > > > > > still
> > > > > > > > > > > > > not
> > > > > > > > > > > > > >> > fully implement and we need to finish in in 2.6
> > > > > release.
> > > > > > > > > > > > > >> >
> > > > > > > > > > > > > >> > KIP-447 is partially implemented in 2.5 (ie,
> > broker
> > > > > and
> > > > > > > > > > > > > >> > consumer/producer changes -- the Kafka Streams
> > parts
> > > > > > slip)
> > > > > > > > > > > > > >> >
> > > > > > > > > > > > > >> >
> > > > > > > > > > > > > >> > -Matthias
> > > > > > > > > > > > > >> >
> > > > > > > > > > > > > >> >
> > > > > > > > > > > > > >> > On 1/29/20 9:05 AM, David Arthur wrote:
> > > > > > > > > > > > > >> >> Hey everyone, just a quick update on the 2.5
> > > > release.
> > > > > > > > > > > > > >> >>
> > > > > > > > > > > > > >> >> I have updated the list of planned KIPs on the
> > > > > release
> > > > > > > wiki
> > > > > > > > > > page
> > > > > > > > > > > > > >> >>
> > > > > > > > > > > > > >>
> > > > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=143428858
> > > > > > > > > > > > > >> .
> > > > > > > > > > > > > >> >> If I have missed anything, or there are KIPs
> > > > included
> > > > > > in
> > > > > > > > this
> > > > > > > > > > > list
> > > > > > > > > > > > > >> which
> > > > > > > > > > > > > >> >> should *not* be included in 2.5, please let me
> > > > know.
> > > > > > > > > > > > > >> >>
> > > > > > > > > > > > > >> >> Based on the release schedule, the feature
> > freeze
> > > > is
> > > > > > > today,
> > > > > > > > > Jan
> > > > > > > > > > > > 29th.
> > > > > > > > > > > > > >> Any
> > > > > > > > > > > > > >> >> major feature work that is not already complete
> > > > will
> > > > > > need
> > > > > > > > to
> > > > > > > > > > push
> > > > > > > > > > > > out
> > > > > > > > > > > > > >> to
> > > > > > > > > > > > > >> >> 2.6. I will work on cutting the release branch
> > > > during
> > > > > > the
> > > > > > > > day
> > > > > > > > > > > > > tomorrow
> > > > > > > > > > > > > >> >> (Jan
> > > > > > > > > > > > > >> >> 30th).
> > > > > > > > > > > > > >> >>
> > > > > > > > > > > > > >> >> If you have any questions, please feel free to
> > > > reach
> > > > > > out
> > > > > > > to
> > > > > > > > > me
> > > > > > > > > > > > > directly
> > > > > > > > > > > > > >> >> or
> > > > > > > > > > > > > >> >> in this thread.
> > > > > > > > > > > > > >> >>
> > > > > > > > > > > > > >> >> Thanks!
> > > > > > > > > > > > > >> >> David
> > > > > > > > > > > > > >> >>
> > > > > > > > > > > > > >> >> On Mon, Jan 13, 2020 at 1:35 PM Colin McCabe <
> > > > > > > > > > cmccabe@apache.org
> > > > > > > > > > > >
> > > > > > > > > > > > > >> wrote:
> > > > > > > > > > > > > >> >>
> > > > > > > > > > > > > >> >>> +1. Thanks for volunteering, David.
> > > > > > > > > > > > > >> >>>
> > > > > > > > > > > > > >> >>> best,
> > > > > > > > > > > > > >> >>> Colin
> > > > > > > > > > > > > >> >>>
> > > > > > > > > > > > > >> >>> On Fri, Dec 20, 2019, at 10:59, David Arthur
> > > > wrote:
> > > > > > > > > > > > > >> >>>> Greetings!
> > > > > > > > > > > > > >> >>>>
> > > > > > > > > > > > > >> >>>> I'd like to volunteer to be release manager
> > for
> > > > the
> > > > > > > next
> > > > > > > > > > > > time-based
> > > > > > > > > > > > > >> >>> feature
> > > > > > > > > > > > > >> >>>> release which will be 2.5. If there are no
> > > > > > objections,
> > > > > > > > I'll
> > > > > > > > > > > send
> > > > > > > > > > > > > out
> > > > > > > > > > > > > >> >>>> the
> > > > > > > > > > > > > >> >>>> release plan in the next few days.
> > > > > > > > > > > > > >> >>>>
> > > > > > > > > > > > > >> >>>> Thanks,
> > > > > > > > > > > > > >> >>>> David Arthur
> > > > > > > > > > > > > >> >>>>
> > > > > > > > > > > > > >> >>>
> > > > > > > > > > > > > >> >>
> > > > > > > > > > > > > >> >>
> > > > > > > > > > > > > >> >
> > > > > > > > > > > > > >> >
> > > > > > > > > > > > > >>
> > > > > > > > > > > > > >
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > --
> > > > > > > > > > > > > > David Arthur
> > > > > > > > > > > > > >
> > > > > > > > > > > > >
> > > > > > > > > > > > >
> > > > > > > > > > > > > --
> > > > > > > > > > > > > David Arthur
> > > > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > --
> > > > > > > > > > David Arthur
> > > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > > --
> > > > > > > > David Arthur
> > > > > > > >
> > > > > > >
> > > > > >
> > > > > >
> > > > > > --
> > > > > > David Arthur
> > > > > >
> > > > >
> > > >
> > > >
> > > > --
> > > > David Arthur
> > > >
> >
Re: [DISCUSS] Apache Kafka 2.5.0 release
Posted by Ismael Juma <is...@juma.me.uk>.
Hi Bruno,
2.4.0 has been out for months. If this was not noticed since then, I don't
think it qualifies as a blocker for 2.5.0.
Ismael
On Mon, Mar 9, 2020 at 8:44 PM Bruno Cadonna <br...@confluent.io> wrote:
> Hi David,
>
> A bug report was filed that can be considered a blocker. Basically,
> with this bug all RocksDB metrics reported by Streams are constant
> zero. The bug is there since 2.4., so it is not a regression, but a
> broken feature.
>
> Here is the ticket: https://issues.apache.org/jira/browse/KAFKA-9675
> Here is the fix: https://github.com/apache/kafka/pull/8256
>
> Best,
> Bruno
>
> On Wed, Feb 26, 2020 at 11:22 PM Randall Hauch <rh...@gmail.com> wrote:
> >
> > Thanks, David. The PR has been merged to trunk and 2.5, and I'm
> backporting
> > to earlier branches. I'll resolve
> > https://issues.apache.org/jira/browse/KAFKA-9601 when I finish
> backporting.
> >
> > On Wed, Feb 26, 2020 at 1:28 PM David Arthur <mu...@gmail.com> wrote:
> >
> > > Thanks, Randall. Leaking sensitive config to the logs seems fairly
> > > severe. I think should include this. Let's proceed with cherry-picking
> to
> > > 2.5.
> > >
> > > -David
> > >
> > > On Wed, Feb 26, 2020 at 2:25 PM Randall Hauch <rh...@gmail.com>
> wrote:
> > >
> > > > Hi, David.
> > > >
> > > > If we're still not quite ready for an RC, I'd like to squeeze in
> > > > https://issues.apache.org/jira/browse/KAFKA-9601, which removes the
> raw
> > > > connector config properties in a DEBUG level log message. PR is ready
> > > (test
> > > > failures are unrelated), the risk is very low, and I think it'd be
> great
> > > to
> > > > correct this sooner than later.
> > > >
> > > > Randall
> > > >
> > > > On Wed, Feb 26, 2020 at 11:26 AM David Arthur <mu...@gmail.com>
> wrote:
> > > >
> > > > > Viktor, the change LGTM. I've approved and merged the cherry-pick
> > > version
> > > > > into 2.5.
> > > > >
> > > > > Thanks!
> > > > > David
> > > > >
> > > > > On Tue, Feb 25, 2020 at 4:43 AM Viktor Somogyi-Vass <
> > > > > viktorsomogyi@gmail.com>
> > > > > wrote:
> > > > >
> > > > > > Hi David,
> > > > > >
> > > > > > There are two short JIRAs related to KIP-352 that documents the
> newly
> > > > > added
> > > > > > metrics. Is it possible to merge them in?
> > > > > > https://github.com/apache/kafka/pull/7434 (trunk)
> > > > > > https://github.com/apache/kafka/pull/8127 (2.5 cherry-pick)
> > > > > >
> > > > > > Thanks,
> > > > > > Viktor
> > > > > >
> > > > > >
> > > > > > On Mon, Feb 24, 2020 at 7:22 PM David Arthur <mu...@gmail.com>
> > > wrote:
> > > > > >
> > > > > > > Thanks, Tu. I've moved KIP-467 out of the release plan.
> > > > > > >
> > > > > > > -David
> > > > > > >
> > > > > > > On Thu, Feb 20, 2020 at 6:00 PM Tu Tran <tu...@confluent.io>
> wrote:
> > > > > > >
> > > > > > > > Hi David,
> > > > > > > >
> > > > > > > > Thanks for being the release main driver. Since the
> > > implementation
> > > > > for
> > > > > > > the
> > > > > > > > last part of KIP-467 wasn't finalized prior to Feb 12th,
> could
> > > you
> > > > > > remove
> > > > > > > > KIP-467 from the list?
> > > > > > > >
> > > > > > > > Thanks,
> > > > > > > > Tu
> > > > > > > >
> > > > > > > > On Thu, Feb 20, 2020 at 7:18 AM David Arthur <
> mumrah@gmail.com>
> > > > > wrote:
> > > > > > > >
> > > > > > > > > Randall / Konstantine,
> > > > > > > > >
> > > > > > > > > Sorry for the late reply. Thanks for the fix and for the
> > > update!
> > > > I
> > > > > > see
> > > > > > > > this
> > > > > > > > > change on the 2.5 branch (@b403c66). Consider this a
> > > retroactive
> > > > > > > approval
> > > > > > > > > for this bugfix :)
> > > > > > > > >
> > > > > > > > > -David
> > > > > > > > >
> > > > > > > > > On Fri, Feb 14, 2020 at 2:21 PM Konstantine Karantasis <
> > > > > > > > > konstantine@confluent.io> wrote:
> > > > > > > > >
> > > > > > > > > > Hi David,
> > > > > > > > > >
> > > > > > > > > > I want to confirm what Randall mentions above. The code
> fixes
> > > > for
> > > > > > > > > > KAFKA-9556 were in place before code freeze on Wed, but
> we
> > > > spent
> > > > > a
> > > > > > > bit
> > > > > > > > > more
> > > > > > > > > > time hardening the conditions of the integration tests
> and
> > > > fixing
> > > > > > > some
> > > > > > > > > > jenkins branch builders to run the test on repeat.
> > > > > > > > > >
> > > > > > > > > > Best,
> > > > > > > > > > Konstantine
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > On Fri, Feb 14, 2020 at 7:42 AM Randall Hauch <
> > > > rhauch@gmail.com>
> > > > > > > > wrote:
> > > > > > > > > >
> > > > > > > > > > > Hi, David.
> > > > > > > > > > >
> > > > > > > > > > > I just filed
> > > > https://issues.apache.org/jira/browse/KAFKA-9556
> > > > > > that
> > > > > > > > > > > identifies two pretty minor issues with the new KIP-558
> > > that
> > > > > adds
> > > > > > > new
> > > > > > > > > > > Connect REST API endpoints to get the list of topics
> used
> > > by
> > > > a
> > > > > > > > > connector.
> > > > > > > > > > > The impact is high: the feature cannot be fully
> disabled,
> > > and
> > > > > > > Connect
> > > > > > > > > > does
> > > > > > > > > > > not automatically reset the topic set when a connector
> is
> > > > > > deleted.
> > > > > > > > > > > https://github.com/apache/kafka/pull/8085 includes
> the two
> > > > > > fixes,
> > > > > > > > and
> > > > > > > > > > also
> > > > > > > > > > > adds more unit and integration tests for this feature.
> > > > > Although I
> > > > > > > > just
> > > > > > > > > > > created the blocker this AM, Konstantine has actually
> be
> > > > > working
> > > > > > on
> > > > > > > > the
> > > > > > > > > > fix
> > > > > > > > > > > for four days. Risk of merging this PR is low, since
> a) the
> > > > new
> > > > > > > > > > integration
> > > > > > > > > > > tests add significant coverage and we've run the new
> tests
> > > > > > numerous
> > > > > > > > > > times,
> > > > > > > > > > > and b) the fixes help gate the new feature even more
> and
> > > > allow
> > > > > > the
> > > > > > > > > > feature
> > > > > > > > > > > to be completely disabled.
> > > > > > > > > > >
> > > > > > > > > > > I'd like approve to merge
> > > > > > > https://github.com/apache/kafka/pull/8085
> > > > > > > > > > >
> > > > > > > > > > > Thanks!
> > > > > > > > > > > Randall
> > > > > > > > > > >
> > > > > > > > > > > On Mon, Feb 10, 2020 at 11:31 AM David Arthur <
> > > > > mumrah@gmail.com>
> > > > > > > > > wrote:
> > > > > > > > > > >
> > > > > > > > > > > > Just a friendly reminder that this Wednesday,
> February
> > > > 12th,
> > > > > is
> > > > > > > the
> > > > > > > > > > code
> > > > > > > > > > > > freeze for the 2.5.0 release. After this time we will
> > > only
> > > > > > accept
> > > > > > > > > > blocker
> > > > > > > > > > > > bugs onto the release branch.
> > > > > > > > > > > >
> > > > > > > > > > > > Thanks!
> > > > > > > > > > > > David
> > > > > > > > > > > >
> > > > > > > > > > > > On Fri, Jan 31, 2020 at 5:13 PM David Arthur <
> > > > > mumrah@gmail.com
> > > > > > >
> > > > > > > > > wrote:
> > > > > > > > > > > >
> > > > > > > > > > > > > Thanks! I've updated the list.
> > > > > > > > > > > > >
> > > > > > > > > > > > > On Thu, Jan 30, 2020 at 5:48 PM Konstantine
> Karantasis
> > > <
> > > > > > > > > > > > > konstantine@confluent.io> wrote:
> > > > > > > > > > > > >
> > > > > > > > > > > > >> Hi David,
> > > > > > > > > > > > >>
> > > > > > > > > > > > >> thanks for driving the release.
> > > > > > > > > > > > >>
> > > > > > > > > > > > >> Please also remove KIP-158 from the list of KIPs
> that
> > > > you
> > > > > > plan
> > > > > > > > to
> > > > > > > > > > > > include
> > > > > > > > > > > > >> in 2.5
> > > > > > > > > > > > >> KIP-158 has been accepted, but the implementation
> is
> > > not
> > > > > yet
> > > > > > > > > final.
> > > > > > > > > > It
> > > > > > > > > > > > >> will be included in the release that follows 2.5.
> > > > > > > > > > > > >>
> > > > > > > > > > > > >> Regards,
> > > > > > > > > > > > >> Konstantine
> > > > > > > > > > > > >>
> > > > > > > > > > > > >> On 1/30/20, Matthias J. Sax <
> matthias@confluent.io>
> > > > > wrote:
> > > > > > > > > > > > >> > Hi David,
> > > > > > > > > > > > >> >
> > > > > > > > > > > > >> > the following KIP from the list did not make it:
> > > > > > > > > > > > >> >
> > > > > > > > > > > > >> > - KIP-216 (no PR yet)
> > > > > > > > > > > > >> > - KIP-399 (no PR yet)
> > > > > > > > > > > > >> > - KIP-401 (PR not merged yet)
> > > > > > > > > > > > >> >
> > > > > > > > > > > > >> >
> > > > > > > > > > > > >> > KIP-444 should be included as we did make
> progress,
> > > > but
> > > > > it
> > > > > > > is
> > > > > > > > > > still
> > > > > > > > > > > > not
> > > > > > > > > > > > >> > fully implement and we need to finish in in 2.6
> > > > release.
> > > > > > > > > > > > >> >
> > > > > > > > > > > > >> > KIP-447 is partially implemented in 2.5 (ie,
> broker
> > > > and
> > > > > > > > > > > > >> > consumer/producer changes -- the Kafka Streams
> parts
> > > > > slip)
> > > > > > > > > > > > >> >
> > > > > > > > > > > > >> >
> > > > > > > > > > > > >> > -Matthias
> > > > > > > > > > > > >> >
> > > > > > > > > > > > >> >
> > > > > > > > > > > > >> > On 1/29/20 9:05 AM, David Arthur wrote:
> > > > > > > > > > > > >> >> Hey everyone, just a quick update on the 2.5
> > > release.
> > > > > > > > > > > > >> >>
> > > > > > > > > > > > >> >> I have updated the list of planned KIPs on the
> > > > release
> > > > > > wiki
> > > > > > > > > page
> > > > > > > > > > > > >> >>
> > > > > > > > > > > > >>
> > > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=143428858
> > > > > > > > > > > > >> .
> > > > > > > > > > > > >> >> If I have missed anything, or there are KIPs
> > > included
> > > > > in
> > > > > > > this
> > > > > > > > > > list
> > > > > > > > > > > > >> which
> > > > > > > > > > > > >> >> should *not* be included in 2.5, please let me
> > > know.
> > > > > > > > > > > > >> >>
> > > > > > > > > > > > >> >> Based on the release schedule, the feature
> freeze
> > > is
> > > > > > today,
> > > > > > > > Jan
> > > > > > > > > > > 29th.
> > > > > > > > > > > > >> Any
> > > > > > > > > > > > >> >> major feature work that is not already complete
> > > will
> > > > > need
> > > > > > > to
> > > > > > > > > push
> > > > > > > > > > > out
> > > > > > > > > > > > >> to
> > > > > > > > > > > > >> >> 2.6. I will work on cutting the release branch
> > > during
> > > > > the
> > > > > > > day
> > > > > > > > > > > > tomorrow
> > > > > > > > > > > > >> >> (Jan
> > > > > > > > > > > > >> >> 30th).
> > > > > > > > > > > > >> >>
> > > > > > > > > > > > >> >> If you have any questions, please feel free to
> > > reach
> > > > > out
> > > > > > to
> > > > > > > > me
> > > > > > > > > > > > directly
> > > > > > > > > > > > >> >> or
> > > > > > > > > > > > >> >> in this thread.
> > > > > > > > > > > > >> >>
> > > > > > > > > > > > >> >> Thanks!
> > > > > > > > > > > > >> >> David
> > > > > > > > > > > > >> >>
> > > > > > > > > > > > >> >> On Mon, Jan 13, 2020 at 1:35 PM Colin McCabe <
> > > > > > > > > cmccabe@apache.org
> > > > > > > > > > >
> > > > > > > > > > > > >> wrote:
> > > > > > > > > > > > >> >>
> > > > > > > > > > > > >> >>> +1. Thanks for volunteering, David.
> > > > > > > > > > > > >> >>>
> > > > > > > > > > > > >> >>> best,
> > > > > > > > > > > > >> >>> Colin
> > > > > > > > > > > > >> >>>
> > > > > > > > > > > > >> >>> On Fri, Dec 20, 2019, at 10:59, David Arthur
> > > wrote:
> > > > > > > > > > > > >> >>>> Greetings!
> > > > > > > > > > > > >> >>>>
> > > > > > > > > > > > >> >>>> I'd like to volunteer to be release manager
> for
> > > the
> > > > > > next
> > > > > > > > > > > time-based
> > > > > > > > > > > > >> >>> feature
> > > > > > > > > > > > >> >>>> release which will be 2.5. If there are no
> > > > > objections,
> > > > > > > I'll
> > > > > > > > > > send
> > > > > > > > > > > > out
> > > > > > > > > > > > >> >>>> the
> > > > > > > > > > > > >> >>>> release plan in the next few days.
> > > > > > > > > > > > >> >>>>
> > > > > > > > > > > > >> >>>> Thanks,
> > > > > > > > > > > > >> >>>> David Arthur
> > > > > > > > > > > > >> >>>>
> > > > > > > > > > > > >> >>>
> > > > > > > > > > > > >> >>
> > > > > > > > > > > > >> >>
> > > > > > > > > > > > >> >
> > > > > > > > > > > > >> >
> > > > > > > > > > > > >>
> > > > > > > > > > > > >
> > > > > > > > > > > > >
> > > > > > > > > > > > > --
> > > > > > > > > > > > > David Arthur
> > > > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > > --
> > > > > > > > > > > > David Arthur
> > > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > --
> > > > > > > > > David Arthur
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > > >
> > > > > > > --
> > > > > > > David Arthur
> > > > > > >
> > > > > >
> > > > >
> > > > >
> > > > > --
> > > > > David Arthur
> > > > >
> > > >
> > >
> > >
> > > --
> > > David Arthur
> > >
>