You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@kafka.apache.org by Jun Rao <ju...@confluent.io> on 2017/10/16 21:19:31 UTC
Re: [kafka-clients] Re: [VOTE] 1.0.0 RC1
Hi, Guozhang,
Onur found an existing performance bug in the controller when there are
lots of partitions. The fix is simple (
https://github.com/apache/kafka/pull/4075) and reduces the controlled
shutdown time from 6.5 mins to 30 secs, with
25K partitions, RF=2 and 5 brokers.
It would be useful to include this fix in 1.0.0.
Thanks,
Jun
On Mon, Oct 16, 2017 at 9:55 AM, Guozhang Wang <wa...@gmail.com> wrote:
> Hi Tom,
>
> Thanks for pointing it out. I meant to say Oct. 17th, Tuesday, for a 72
> hours period.
>
> That being said, we need to have a lazy majority to accept a release RC
> according to our bylaws (https://cwiki.apache.org/
> confluence/display/KAFKA/Bylaws). And if we cannot achieve that via
> thorough testing within the period we will automatically extend the voting
> process.
>
>
> Guozhang
>
>
>
> On Mon, Oct 16, 2017 at 5:09 AM, Thomas Crayford <tcrayford@salesforce.com
> > wrote:
>
>> Hi Guozhang,
>>
>> This says the due date on the testing is October 13th, which was the day
>> this email was sent. Is that accurate, or is it meant to read October
>> 17th,
>> which is next Tuesday?
>>
>> I feel like this short a testing window for a 1.0 RC is a little low, as
>> 1.0 is clearly a big announcement of stability, and folk should be given
>> enough time to do thorough testing.
>>
>> Thanks
>>
>> Tom
>>
>> On Fri, Oct 13, 2017 at 9:12 PM, Guozhang Wang <wa...@gmail.com>
>> wrote:
>>
>> > Hello Kafka users, developers and client-developers,
>> >
>> > This is the second candidate for release of Apache Kafka 1.0.0.
>> >
>> > It's worth noting that starting in this version we are using a different
>> > version protocol with three digits: *major.minor.bug-fix*
>> >
>> > Any and all testing is welcome, but the following areas are worth
>> > highlighting:
>> >
>> > 1. Client developers should verify that their clients can
>> produce/consume
>> > to/from 1.0.0 brokers (ideally with compressed and uncompressed data).
>> > 2. Performance and stress testing. Heroku and LinkedIn have helped with
>> > this in the past (and issues have been found and fixed).
>> > 3. End users can verify that their apps work correctly with the new
>> > release.
>> >
>> > This is a major version release of Apache Kafka. It includes 29 new
>> KIPs.
>> > See the release notes and release plan
>> > (*https://cwiki.apache.org/confluence/pages/viewpage.
>> > action?pageId=71764913
>> > <https://cwiki.apache.org/confluence/pages/viewpage.action?
>> pageId=71764913
>> > >*)
>> > for more details. A few feature highlights:
>> >
>> > * Java 9 support with significantly faster TLS and CRC32C
>> implementations
>> > (KIP)
>> > * JBOD improvements: disk failure only disables failed disk but not the
>> > broker (KIP-112/KIP-113)
>> > * Newly added metrics across all the modules (KIP-164, KIP-168, KIP-187,
>> > KIP-188, KIP-196)
>> > * Kafka Streams API improvements (KIP-120 / 130 / 138 / 150 / 160 /
>> 161),
>> > and drop compatibility "Evolving" annotations
>> >
>> > Release notes for the 1.0.0 release:
>> > *http://home.apache.org/~guozhang/kafka-1.0.0-rc1/RELEASE_NOTES.html
>> > <http://home.apache.org/~guozhang/kafka-1.0.0-rc1/RELEASE_NOTES.html>*
>> >
>> >
>> >
>> > *** Please download, test and vote by Tuesday, October 13, 8pm PT
>> >
>> > Kafka's KEYS file containing PGP keys we use to sign the release:
>> > http://kafka.apache.org/KEYS
>> >
>> > * Release artifacts to be voted upon (source and binary):
>> > *http://home.apache.org/~guozhang/kafka-1.0.0-rc1/
>> > <http://home.apache.org/~guozhang/kafka-1.0.0-rc1/>*
>> >
>> > * Maven artifacts to be voted upon:
>> > https://repository.apache.org/content/groups/staging/org/apache/kafka/
>> >
>> > * Javadoc:
>> > *http://home.apache.org/~guozhang/kafka-1.0.0-rc1/javadoc/
>> > <http://home.apache.org/~guozhang/kafka-1.0.0-rc1/javadoc/>*
>> >
>> > * Tag to be voted upon (off 1.0 branch) is the 1.0.0-rc1 tag:
>> >
>> > https://git-wip-us.apache.org/repos/asf?p=kafka.git;a=tag;h=
>> > 9424d29dbf0a3c538215b0b98b1e6b956481e4d5
>> >
>> > * Documentation:
>> > Note the documentation can't be pushed live due to changes that will
>> not go
>> > live until the release. You can manually verify by downloading
>> > http://home.apache.org/~guozhang/kafka-1.0.0-rc1/
>> > kafka_2.11-1.0.0-site-docs.tgz
>> >
>> > * Successful Jenkins builds for the 1.0.0 branch:
>> > Unit/integration tests: https://builds.apache.org/job/
>> kafka-1.0-jdk7/31/
>> > System test: https://jenkins.confluent.io/job/system-test-kafka-1.0/1/
>> >
>> >
>> > /**************************************
>> >
>> >
>> > Thanks,
>> > -- Guozhang
>> >
>>
>
>
>
> --
> -- Guozhang
>
> --
> You received this message because you are subscribed to the Google Groups
> "kafka-clients" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to kafka-clients+unsubscribe@googlegroups.com.
> To post to this group, send email to kafka-clients@googlegroups.com.
> Visit this group at https://groups.google.com/group/kafka-clients.
> To view this discussion on the web visit https://groups.google.com/d/
> msgid/kafka-clients/CAHwHRrUhYTtGH9TmTzub9%2BypaxVBMxfvcxnEuX74Z4X5Km%3D-
> jA%40mail.gmail.com
> <https://groups.google.com/d/msgid/kafka-clients/CAHwHRrUhYTtGH9TmTzub9%2BypaxVBMxfvcxnEuX74Z4X5Km%3D-jA%40mail.gmail.com?utm_medium=email&utm_source=footer>
> .
>
> For more options, visit https://groups.google.com/d/optout.
>
Re: [kafka-clients] Re: [VOTE] 1.0.0 RC1
Posted by Guozhang Wang <wa...@gmail.com>.
Thanks Jun,
I will roll out a new RC for this PR then.
Guozhang
On Mon, Oct 16, 2017 at 2:19 PM, Jun Rao <ju...@confluent.io> wrote:
> Hi, Guozhang,
>
> Onur found an existing performance bug in the controller when there are
> lots of partitions. The fix is simple (https://github.com/apache/
> kafka/pull/4075) and reduces the controlled shutdown time from 6.5 mins
> to 30 secs, with
> 25K partitions, RF=2 and 5 brokers.
>
> It would be useful to include this fix in 1.0.0.
>
> Thanks,
>
> Jun
>
>
> On Mon, Oct 16, 2017 at 9:55 AM, Guozhang Wang <wa...@gmail.com> wrote:
>
>> Hi Tom,
>>
>> Thanks for pointing it out. I meant to say Oct. 17th, Tuesday, for a 72
>> hours period.
>>
>> That being said, we need to have a lazy majority to accept a release RC
>> according to our bylaws (https://cwiki.apache.org/conf
>> luence/display/KAFKA/Bylaws). And if we cannot achieve that via thorough
>> testing within the period we will automatically extend the voting process.
>>
>>
>> Guozhang
>>
>>
>>
>> On Mon, Oct 16, 2017 at 5:09 AM, Thomas Crayford <
>> tcrayford@salesforce.com> wrote:
>>
>>> Hi Guozhang,
>>>
>>> This says the due date on the testing is October 13th, which was the day
>>> this email was sent. Is that accurate, or is it meant to read October
>>> 17th,
>>> which is next Tuesday?
>>>
>>> I feel like this short a testing window for a 1.0 RC is a little low, as
>>> 1.0 is clearly a big announcement of stability, and folk should be given
>>> enough time to do thorough testing.
>>>
>>> Thanks
>>>
>>> Tom
>>>
>>> On Fri, Oct 13, 2017 at 9:12 PM, Guozhang Wang <wa...@gmail.com>
>>> wrote:
>>>
>>> > Hello Kafka users, developers and client-developers,
>>> >
>>> > This is the second candidate for release of Apache Kafka 1.0.0.
>>> >
>>> > It's worth noting that starting in this version we are using a
>>> different
>>> > version protocol with three digits: *major.minor.bug-fix*
>>> >
>>> > Any and all testing is welcome, but the following areas are worth
>>> > highlighting:
>>> >
>>> > 1. Client developers should verify that their clients can
>>> produce/consume
>>> > to/from 1.0.0 brokers (ideally with compressed and uncompressed data).
>>> > 2. Performance and stress testing. Heroku and LinkedIn have helped with
>>> > this in the past (and issues have been found and fixed).
>>> > 3. End users can verify that their apps work correctly with the new
>>> > release.
>>> >
>>> > This is a major version release of Apache Kafka. It includes 29 new
>>> KIPs.
>>> > See the release notes and release plan
>>> > (*https://cwiki.apache.org/confluence/pages/viewpage.
>>> > action?pageId=71764913
>>> > <https://cwiki.apache.org/confluence/pages/viewpage.action?p
>>> ageId=71764913
>>> > >*)
>>> > for more details. A few feature highlights:
>>> >
>>> > * Java 9 support with significantly faster TLS and CRC32C
>>> implementations
>>> > (KIP)
>>> > * JBOD improvements: disk failure only disables failed disk but not the
>>> > broker (KIP-112/KIP-113)
>>> > * Newly added metrics across all the modules (KIP-164, KIP-168,
>>> KIP-187,
>>> > KIP-188, KIP-196)
>>> > * Kafka Streams API improvements (KIP-120 / 130 / 138 / 150 / 160 /
>>> 161),
>>> > and drop compatibility "Evolving" annotations
>>> >
>>> > Release notes for the 1.0.0 release:
>>> > *http://home.apache.org/~guozhang/kafka-1.0.0-rc1/RELEASE_NOTES.html
>>> > <http://home.apache.org/~guozhang/kafka-1.0.0-rc1/RELEASE_NOTES.html>*
>>> >
>>> >
>>> >
>>> > *** Please download, test and vote by Tuesday, October 13, 8pm PT
>>> >
>>> > Kafka's KEYS file containing PGP keys we use to sign the release:
>>> > http://kafka.apache.org/KEYS
>>> >
>>> > * Release artifacts to be voted upon (source and binary):
>>> > *http://home.apache.org/~guozhang/kafka-1.0.0-rc1/
>>> > <http://home.apache.org/~guozhang/kafka-1.0.0-rc1/>*
>>> >
>>> > * Maven artifacts to be voted upon:
>>> > https://repository.apache.org/content/groups/staging/org/apache/kafka/
>>> >
>>> > * Javadoc:
>>> > *http://home.apache.org/~guozhang/kafka-1.0.0-rc1/javadoc/
>>> > <http://home.apache.org/~guozhang/kafka-1.0.0-rc1/javadoc/>*
>>> >
>>> > * Tag to be voted upon (off 1.0 branch) is the 1.0.0-rc1 tag:
>>> >
>>> > https://git-wip-us.apache.org/repos/asf?p=kafka.git;a=tag;h=
>>> > 9424d29dbf0a3c538215b0b98b1e6b956481e4d5
>>> >
>>> > * Documentation:
>>> > Note the documentation can't be pushed live due to changes that will
>>> not go
>>> > live until the release. You can manually verify by downloading
>>> > http://home.apache.org/~guozhang/kafka-1.0.0-rc1/
>>> > kafka_2.11-1.0.0-site-docs.tgz
>>> >
>>> > * Successful Jenkins builds for the 1.0.0 branch:
>>> > Unit/integration tests: https://builds.apache.org/job/
>>> kafka-1.0-jdk7/31/
>>> > System test: https://jenkins.confluent.io/job/system-test-kafka-1.0/1/
>>> >
>>> >
>>> > /**************************************
>>> >
>>> >
>>> > Thanks,
>>> > -- Guozhang
>>> >
>>>
>>
>>
>>
>> --
>> -- Guozhang
>>
>> --
>> You received this message because you are subscribed to the Google Groups
>> "kafka-clients" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to kafka-clients+unsubscribe@googlegroups.com.
>> To post to this group, send email to kafka-clients@googlegroups.com.
>> Visit this group at https://groups.google.com/group/kafka-clients.
>> To view this discussion on the web visit https://groups.google.com/d/ms
>> gid/kafka-clients/CAHwHRrUhYTtGH9TmTzub9%2BypaxVBMxfvcxnEuX7
>> 4Z4X5Km%3D-jA%40mail.gmail.com
>> <https://groups.google.com/d/msgid/kafka-clients/CAHwHRrUhYTtGH9TmTzub9%2BypaxVBMxfvcxnEuX74Z4X5Km%3D-jA%40mail.gmail.com?utm_medium=email&utm_source=footer>
>> .
>>
>> For more options, visit https://groups.google.com/d/optout.
>>
>
>
--
-- Guozhang
Re: [kafka-clients] Re: [VOTE] 1.0.0 RC1
Posted by Guozhang Wang <wa...@gmail.com>.
Thanks Jun,
I will roll out a new RC for this PR then.
Guozhang
On Mon, Oct 16, 2017 at 2:19 PM, Jun Rao <ju...@confluent.io> wrote:
> Hi, Guozhang,
>
> Onur found an existing performance bug in the controller when there are
> lots of partitions. The fix is simple (https://github.com/apache/
> kafka/pull/4075) and reduces the controlled shutdown time from 6.5 mins
> to 30 secs, with
> 25K partitions, RF=2 and 5 brokers.
>
> It would be useful to include this fix in 1.0.0.
>
> Thanks,
>
> Jun
>
>
> On Mon, Oct 16, 2017 at 9:55 AM, Guozhang Wang <wa...@gmail.com> wrote:
>
>> Hi Tom,
>>
>> Thanks for pointing it out. I meant to say Oct. 17th, Tuesday, for a 72
>> hours period.
>>
>> That being said, we need to have a lazy majority to accept a release RC
>> according to our bylaws (https://cwiki.apache.org/conf
>> luence/display/KAFKA/Bylaws). And if we cannot achieve that via thorough
>> testing within the period we will automatically extend the voting process.
>>
>>
>> Guozhang
>>
>>
>>
>> On Mon, Oct 16, 2017 at 5:09 AM, Thomas Crayford <
>> tcrayford@salesforce.com> wrote:
>>
>>> Hi Guozhang,
>>>
>>> This says the due date on the testing is October 13th, which was the day
>>> this email was sent. Is that accurate, or is it meant to read October
>>> 17th,
>>> which is next Tuesday?
>>>
>>> I feel like this short a testing window for a 1.0 RC is a little low, as
>>> 1.0 is clearly a big announcement of stability, and folk should be given
>>> enough time to do thorough testing.
>>>
>>> Thanks
>>>
>>> Tom
>>>
>>> On Fri, Oct 13, 2017 at 9:12 PM, Guozhang Wang <wa...@gmail.com>
>>> wrote:
>>>
>>> > Hello Kafka users, developers and client-developers,
>>> >
>>> > This is the second candidate for release of Apache Kafka 1.0.0.
>>> >
>>> > It's worth noting that starting in this version we are using a
>>> different
>>> > version protocol with three digits: *major.minor.bug-fix*
>>> >
>>> > Any and all testing is welcome, but the following areas are worth
>>> > highlighting:
>>> >
>>> > 1. Client developers should verify that their clients can
>>> produce/consume
>>> > to/from 1.0.0 brokers (ideally with compressed and uncompressed data).
>>> > 2. Performance and stress testing. Heroku and LinkedIn have helped with
>>> > this in the past (and issues have been found and fixed).
>>> > 3. End users can verify that their apps work correctly with the new
>>> > release.
>>> >
>>> > This is a major version release of Apache Kafka. It includes 29 new
>>> KIPs.
>>> > See the release notes and release plan
>>> > (*https://cwiki.apache.org/confluence/pages/viewpage.
>>> > action?pageId=71764913
>>> > <https://cwiki.apache.org/confluence/pages/viewpage.action?p
>>> ageId=71764913
>>> > >*)
>>> > for more details. A few feature highlights:
>>> >
>>> > * Java 9 support with significantly faster TLS and CRC32C
>>> implementations
>>> > (KIP)
>>> > * JBOD improvements: disk failure only disables failed disk but not the
>>> > broker (KIP-112/KIP-113)
>>> > * Newly added metrics across all the modules (KIP-164, KIP-168,
>>> KIP-187,
>>> > KIP-188, KIP-196)
>>> > * Kafka Streams API improvements (KIP-120 / 130 / 138 / 150 / 160 /
>>> 161),
>>> > and drop compatibility "Evolving" annotations
>>> >
>>> > Release notes for the 1.0.0 release:
>>> > *http://home.apache.org/~guozhang/kafka-1.0.0-rc1/RELEASE_NOTES.html
>>> > <http://home.apache.org/~guozhang/kafka-1.0.0-rc1/RELEASE_NOTES.html>*
>>> >
>>> >
>>> >
>>> > *** Please download, test and vote by Tuesday, October 13, 8pm PT
>>> >
>>> > Kafka's KEYS file containing PGP keys we use to sign the release:
>>> > http://kafka.apache.org/KEYS
>>> >
>>> > * Release artifacts to be voted upon (source and binary):
>>> > *http://home.apache.org/~guozhang/kafka-1.0.0-rc1/
>>> > <http://home.apache.org/~guozhang/kafka-1.0.0-rc1/>*
>>> >
>>> > * Maven artifacts to be voted upon:
>>> > https://repository.apache.org/content/groups/staging/org/apache/kafka/
>>> >
>>> > * Javadoc:
>>> > *http://home.apache.org/~guozhang/kafka-1.0.0-rc1/javadoc/
>>> > <http://home.apache.org/~guozhang/kafka-1.0.0-rc1/javadoc/>*
>>> >
>>> > * Tag to be voted upon (off 1.0 branch) is the 1.0.0-rc1 tag:
>>> >
>>> > https://git-wip-us.apache.org/repos/asf?p=kafka.git;a=tag;h=
>>> > 9424d29dbf0a3c538215b0b98b1e6b956481e4d5
>>> >
>>> > * Documentation:
>>> > Note the documentation can't be pushed live due to changes that will
>>> not go
>>> > live until the release. You can manually verify by downloading
>>> > http://home.apache.org/~guozhang/kafka-1.0.0-rc1/
>>> > kafka_2.11-1.0.0-site-docs.tgz
>>> >
>>> > * Successful Jenkins builds for the 1.0.0 branch:
>>> > Unit/integration tests: https://builds.apache.org/job/
>>> kafka-1.0-jdk7/31/
>>> > System test: https://jenkins.confluent.io/job/system-test-kafka-1.0/1/
>>> >
>>> >
>>> > /**************************************
>>> >
>>> >
>>> > Thanks,
>>> > -- Guozhang
>>> >
>>>
>>
>>
>>
>> --
>> -- Guozhang
>>
>> --
>> You received this message because you are subscribed to the Google Groups
>> "kafka-clients" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to kafka-clients+unsubscribe@googlegroups.com.
>> To post to this group, send email to kafka-clients@googlegroups.com.
>> Visit this group at https://groups.google.com/group/kafka-clients.
>> To view this discussion on the web visit https://groups.google.com/d/ms
>> gid/kafka-clients/CAHwHRrUhYTtGH9TmTzub9%2BypaxVBMxfvcxnEuX7
>> 4Z4X5Km%3D-jA%40mail.gmail.com
>> <https://groups.google.com/d/msgid/kafka-clients/CAHwHRrUhYTtGH9TmTzub9%2BypaxVBMxfvcxnEuX74Z4X5Km%3D-jA%40mail.gmail.com?utm_medium=email&utm_source=footer>
>> .
>>
>> For more options, visit https://groups.google.com/d/optout.
>>
>
>
--
-- Guozhang