You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@kafka.apache.org by Jun Rao <ju...@confluent.io> on 2015/08/14 14:15:38 UTC
[DISCUSSION] Kafka 0.8.2.2 release?
Hi, Everyone,
Since the release of Kafka 0.8.2.1, a number of people have reported an
issue with snappy compression (
https://issues.apache.org/jira/browse/KAFKA-2189). Basically, if they use
snappy in 0.8.2.1, they will experience a 2-3X space increase. The issue
has since been fixed in trunk (just a snappy jar upgrade). Since 0.8.3 is
still a few months away, it may make sense to do an 0.8.2.2 release just to
fix this issue. Any objections?
Thanks,
Jun
Re: [DISCUSSION] Kafka 0.8.2.2 release?
Posted by Joe Stein <jo...@stealth.ly>.
Is this something also that should be in 0.8.2.2
https://issues.apache.org/jira/browse/KAFKA-2421 wasn't sure what the lz4
usage has been seems to break on some jdks.
+1 on the snappy fixes
~ Joe Stein
On Fri, Aug 14, 2015 at 5:39 PM, Guozhang Wang <wa...@gmail.com> wrote:
> +1 for both KAFKA-2189 and 2308.
>
> On Fri, Aug 14, 2015 at 7:03 AM, Gwen Shapira <gw...@confluent.io> wrote:
>
> > Will be nice to include Kafka-2308 and fix two critical snappy issues in
> > the maintenance release.
> >
> > Gwen
> > On Aug 14, 2015 6:16 AM, "Grant Henke" <gh...@cloudera.com> wrote:
> >
> > > Just to clarify. Will KAFKA-2189 be the only patch in the release?
> > >
> > > On Fri, Aug 14, 2015 at 7:35 AM, Manikumar Reddy <kumar@nmsworks.co.in
> >
> > > wrote:
> > >
> > > > +1 for 0.8.2.2 release
> > > >
> > > > On Fri, Aug 14, 2015 at 5:49 PM, Ismael Juma <is...@juma.me.uk>
> > wrote:
> > > >
> > > > > I think this is a good idea as the change is minimal on our side
> and
> > it
> > > > has
> > > > > been tested in production for some time by the reporter.
> > > > >
> > > > > Best,
> > > > > Ismael
> > > > >
> > > > > On Fri, Aug 14, 2015 at 1:15 PM, Jun Rao <ju...@confluent.io> wrote:
> > > > >
> > > > > > Hi, Everyone,
> > > > > >
> > > > > > Since the release of Kafka 0.8.2.1, a number of people have
> > reported
> > > an
> > > > > > issue with snappy compression (
> > > > > > https://issues.apache.org/jira/browse/KAFKA-2189). Basically, if
> > > they
> > > > > use
> > > > > > snappy in 0.8.2.1, they will experience a 2-3X space increase.
> The
> > > > issue
> > > > > > has since been fixed in trunk (just a snappy jar upgrade). Since
> > > 0.8.3
> > > > is
> > > > > > still a few months away, it may make sense to do an 0.8.2.2
> release
> > > > just
> > > > > to
> > > > > > fix this issue. Any objections?
> > > > > >
> > > > > > Thanks,
> > > > > >
> > > > > > Jun
> > > > > >
> > > > >
> > > >
> > >
> > >
> > >
> > > --
> > > Grant Henke
> > > Software Engineer | Cloudera
> > > grant@cloudera.com | twitter.com/gchenke | linkedin.com/in/granthenke
> > >
> >
>
>
>
> --
> -- Guozhang
>
Re: [DISCUSSION] Kafka 0.8.2.2 release?
Posted by Abhijith <ab...@gmail.com>.
Thanks. Any idea when we can expect 0.8.3?
Sent from my phone
> On Aug 14, 2015, at 5:39 PM, Guozhang Wang <wa...@gmail.com> wrote:
>
> +1 for both KAFKA-2189 and 2308.
>
>> On Fri, Aug 14, 2015 at 7:03 AM, Gwen Shapira <gw...@confluent.io> wrote:
>>
>> Will be nice to include Kafka-2308 and fix two critical snappy issues in
>> the maintenance release.
>>
>> Gwen
>>> On Aug 14, 2015 6:16 AM, "Grant Henke" <gh...@cloudera.com> wrote:
>>>
>>> Just to clarify. Will KAFKA-2189 be the only patch in the release?
>>>
>>> On Fri, Aug 14, 2015 at 7:35 AM, Manikumar Reddy <ku...@nmsworks.co.in>
>>> wrote:
>>>
>>>> +1 for 0.8.2.2 release
>>>>
>>>> On Fri, Aug 14, 2015 at 5:49 PM, Ismael Juma <is...@juma.me.uk>
>> wrote:
>>>>
>>>>> I think this is a good idea as the change is minimal on our side and
>> it
>>>> has
>>>>> been tested in production for some time by the reporter.
>>>>>
>>>>> Best,
>>>>> Ismael
>>>>>
>>>>>> On Fri, Aug 14, 2015 at 1:15 PM, Jun Rao <ju...@confluent.io> wrote:
>>>>>>
>>>>>> Hi, Everyone,
>>>>>>
>>>>>> Since the release of Kafka 0.8.2.1, a number of people have
>> reported
>>> an
>>>>>> issue with snappy compression (
>>>>>> https://issues.apache.org/jira/browse/KAFKA-2189). Basically, if
>>> they
>>>>> use
>>>>>> snappy in 0.8.2.1, they will experience a 2-3X space increase. The
>>>> issue
>>>>>> has since been fixed in trunk (just a snappy jar upgrade). Since
>>> 0.8.3
>>>> is
>>>>>> still a few months away, it may make sense to do an 0.8.2.2 release
>>>> just
>>>>> to
>>>>>> fix this issue. Any objections?
>>>>>>
>>>>>> Thanks,
>>>>>>
>>>>>> Jun
>>>
>>>
>>>
>>> --
>>> Grant Henke
>>> Software Engineer | Cloudera
>>> grant@cloudera.com | twitter.com/gchenke | linkedin.com/in/granthenke
>
>
>
> --
> -- Guozhang
Re: [DISCUSSION] Kafka 0.8.2.2 release?
Posted by Guozhang Wang <wa...@gmail.com>.
+1 for both KAFKA-2189 and 2308.
On Fri, Aug 14, 2015 at 7:03 AM, Gwen Shapira <gw...@confluent.io> wrote:
> Will be nice to include Kafka-2308 and fix two critical snappy issues in
> the maintenance release.
>
> Gwen
> On Aug 14, 2015 6:16 AM, "Grant Henke" <gh...@cloudera.com> wrote:
>
> > Just to clarify. Will KAFKA-2189 be the only patch in the release?
> >
> > On Fri, Aug 14, 2015 at 7:35 AM, Manikumar Reddy <ku...@nmsworks.co.in>
> > wrote:
> >
> > > +1 for 0.8.2.2 release
> > >
> > > On Fri, Aug 14, 2015 at 5:49 PM, Ismael Juma <is...@juma.me.uk>
> wrote:
> > >
> > > > I think this is a good idea as the change is minimal on our side and
> it
> > > has
> > > > been tested in production for some time by the reporter.
> > > >
> > > > Best,
> > > > Ismael
> > > >
> > > > On Fri, Aug 14, 2015 at 1:15 PM, Jun Rao <ju...@confluent.io> wrote:
> > > >
> > > > > Hi, Everyone,
> > > > >
> > > > > Since the release of Kafka 0.8.2.1, a number of people have
> reported
> > an
> > > > > issue with snappy compression (
> > > > > https://issues.apache.org/jira/browse/KAFKA-2189). Basically, if
> > they
> > > > use
> > > > > snappy in 0.8.2.1, they will experience a 2-3X space increase. The
> > > issue
> > > > > has since been fixed in trunk (just a snappy jar upgrade). Since
> > 0.8.3
> > > is
> > > > > still a few months away, it may make sense to do an 0.8.2.2 release
> > > just
> > > > to
> > > > > fix this issue. Any objections?
> > > > >
> > > > > Thanks,
> > > > >
> > > > > Jun
> > > > >
> > > >
> > >
> >
> >
> >
> > --
> > Grant Henke
> > Software Engineer | Cloudera
> > grant@cloudera.com | twitter.com/gchenke | linkedin.com/in/granthenke
> >
>
--
-- Guozhang
Re: [DISCUSSION] Kafka 0.8.2.2 release?
Posted by Ashish Singh <as...@cloudera.com>.
+1 for 0.8.2.2 release with fixes for KAFKA-2189, 2114 and 2308.
On Mon, Aug 17, 2015 at 9:16 AM, Flavio Junqueira <fp...@apache.org> wrote:
> It is pretty typical that Apache projects have a release manager for each
> release:
>
> http://www.apache.org/dev/release-publishing.html <
> http://www.apache.org/dev/release-publishing.html>
>
> It doesn't have to be the same person every time, though, not even for the
> same branch.
>
> -Flavio
>
>
> > On 17 Aug 2015, at 05:01, Gwen Shapira <gw...@confluent.io> wrote:
> >
> > BTW. I think it will be great for Apache Kafka to have a 0.8.2 "release
> > manager" who's role is to cherrypick low-risk bug-fixes into the 0.8.2
> > branch and once enough bug fixes happened (or if sufficiently critical
> > fixes happened) to roll out a new maintenance release (with every 3 month
> > as a reasonable bugfix release target).
>
>
--
Regards,
Ashish
Re: [DISCUSSION] Kafka 0.8.2.2 release?
Posted by Ashish Singh <as...@cloudera.com>.
+1 for 0.8.2.2 release with fixes for KAFKA-2189, 2114 and 2308.
On Mon, Aug 17, 2015 at 9:16 AM, Flavio Junqueira <fp...@apache.org> wrote:
> It is pretty typical that Apache projects have a release manager for each
> release:
>
> http://www.apache.org/dev/release-publishing.html <
> http://www.apache.org/dev/release-publishing.html>
>
> It doesn't have to be the same person every time, though, not even for the
> same branch.
>
> -Flavio
>
>
> > On 17 Aug 2015, at 05:01, Gwen Shapira <gw...@confluent.io> wrote:
> >
> > BTW. I think it will be great for Apache Kafka to have a 0.8.2 "release
> > manager" who's role is to cherrypick low-risk bug-fixes into the 0.8.2
> > branch and once enough bug fixes happened (or if sufficiently critical
> > fixes happened) to roll out a new maintenance release (with every 3 month
> > as a reasonable bugfix release target).
>
>
--
Regards,
Ashish
Re: [DISCUSSION] Kafka 0.8.2.2 release?
Posted by Flavio Junqueira <fp...@apache.org>.
It is pretty typical that Apache projects have a release manager for each release:
http://www.apache.org/dev/release-publishing.html <http://www.apache.org/dev/release-publishing.html>
It doesn't have to be the same person every time, though, not even for the same branch.
-Flavio
> On 17 Aug 2015, at 05:01, Gwen Shapira <gw...@confluent.io> wrote:
>
> BTW. I think it will be great for Apache Kafka to have a 0.8.2 "release
> manager" who's role is to cherrypick low-risk bug-fixes into the 0.8.2
> branch and once enough bug fixes happened (or if sufficiently critical
> fixes happened) to roll out a new maintenance release (with every 3 month
> as a reasonable bugfix release target).
Re: [DISCUSSION] Kafka 0.8.2.2 release?
Posted by Flavio Junqueira <fp...@apache.org>.
It is pretty typical that Apache projects have a release manager for each release:
http://www.apache.org/dev/release-publishing.html <http://www.apache.org/dev/release-publishing.html>
It doesn't have to be the same person every time, though, not even for the same branch.
-Flavio
> On 17 Aug 2015, at 05:01, Gwen Shapira <gw...@confluent.io> wrote:
>
> BTW. I think it will be great for Apache Kafka to have a 0.8.2 "release
> manager" who's role is to cherrypick low-risk bug-fixes into the 0.8.2
> branch and once enough bug fixes happened (or if sufficiently critical
> fixes happened) to roll out a new maintenance release (with every 3 month
> as a reasonable bugfix release target).
Re: [DISCUSSION] Kafka 0.8.2.2 release?
Posted by Gwen Shapira <gw...@confluent.io>.
but +1 for 0.8.2 patch that marks the new consumer API as unstable (or
unimplemented ;)
On Mon, Aug 17, 2015 at 9:12 AM, Gwen Shapira <gw...@confluent.io> wrote:
> The network refactoring portion was not tested well enough yet for me to
> feel comfortable pushing it into a bugfix release. The new purgatory and
> MirrorMaker changes are also pretty big.
>
> The whole goal of bugfix releases is to make sure it is more stable than
> the previous releases.
>
> On Mon, Aug 17, 2015 at 8:54 AM, Stevo Slavić <ss...@gmail.com> wrote:
>
>> Instead of cherrypicking, why not just make 0.8.2.2 of off current trunk,
>> with new consumer API appropriately annotated/documented as unstable?
>>
>> On Mon, Aug 17, 2015, 17:17 Grant Henke <gh...@cloudera.com> wrote:
>>
>> > +dev
>> >
>> > Adding dev list back in. Somehow it got dropped.
>> >
>> >
>> > On Mon, Aug 17, 2015 at 10:16 AM, Grant Henke <gh...@cloudera.com>
>> wrote:
>> >
>> > > Below is a list of candidate bug fix jiras marked fixed for 0.8.3. I
>> > don't
>> > > suspect all of these will (or should) make it into the release but
>> this
>> > > should be a relatively complete list to work from:
>> > >
>> > > - KAFKA-2114 <https://issues.apache.org/jira/browse/KAFKA-2114>:
>> > Unable
>> > > to change min.insync.replicas default
>> > > - KAFKA-1702 <https://issues.apache.org/jira/browse/KAFKA-1702>:
>> > > Messages silently Lost by producer
>> > > - KAFKA-2012 <https://issues.apache.org/jira/browse/KAFKA-2012>:
>> > > Broker should automatically handle corrupt index files
>> > > - KAFKA-2406 <https://issues.apache.org/jira/browse/KAFKA-2406>:
>> ISR
>> > > propagation should be throttled to avoid overwhelming controller.
>> > > - KAFKA-2336 <https://issues.apache.org/jira/browse/KAFKA-2336>:
>> > > Changing offsets.topic.num.partitions after the offset topic is
>> > created
>> > > breaks consumer group partition assignment
>> > > - KAFKA-2337 <https://issues.apache.org/jira/browse/KAFKA-2337>:
>> > Verify
>> > > that metric names will not collide when creating new topics
>> > > - KAFKA-2393 <https://issues.apache.org/jira/browse/KAFKA-2393>:
>> > > Correctly Handle InvalidTopicException in
>> KafkaApis.getTopicMetadata()
>> > > - KAFKA-2189 <https://issues.apache.org/jira/browse/KAFKA-2189>:
>> > Snappy
>> > > compression of message batches less efficient in 0.8.2.1
>> > > - KAFKA-2308 <https://issues.apache.org/jira/browse/KAFKA-2308>:
>> New
>> > > producer + Snappy face un-compression errors after broker restart
>> > > - KAFKA-2042 <https://issues.apache.org/jira/browse/KAFKA-2042>:
>> New
>> > > producer metadata update always get all topics.
>> > > - KAFKA-1367 <https://issues.apache.org/jira/browse/KAFKA-1367>:
>> > Broker
>> > > topic metadata not kept in sync with ZooKeeper
>> > > - KAFKA-972 <https://issues.apache.org/jira/browse/KAFKA-972>:
>> > MetadataRequest
>> > > returns stale list of brokers
>> > > - KAFKA-1867 <https://issues.apache.org/jira/browse/KAFKA-1867>:
>> > liveBroker
>> > > list not updated on a cluster with no topics
>> > > - KAFKA-1650 <https://issues.apache.org/jira/browse/KAFKA-1650>:
>> > Mirror
>> > > Maker could lose data on unclean shutdown.
>> > > - KAFKA-2009 <https://issues.apache.org/jira/browse/KAFKA-2009>:
>> Fix
>> > > UncheckedOffset.removeOffset synchronization and trace logging
>> issue
>> > in
>> > > mirror maker
>> > > - KAFKA-2407 <https://issues.apache.org/jira/browse/KAFKA-2407>:
>> Only
>> > > create a log directory when it will be used
>> > > - KAFKA-2327 <https://issues.apache.org/jira/browse/KAFKA-2327>:
>> > > broker doesn't start if config defines advertised.host but not
>> > > advertised.port
>> > > - KAFKA-1788: producer record can stay in RecordAccumulator
>> forever if
>> > > leader is no available
>> > > - KAFKA-2234 <https://issues.apache.org/jira/browse/KAFKA-2234>:
>> > > Partition reassignment of a nonexistent topic prevents future
>> > reassignments
>> > > - KAFKA-2096 <https://issues.apache.org/jira/browse/KAFKA-2096>:
>> > > Enable keepalive socket option for broker to prevent socket leak
>> > > - KAFKA-1057 <https://issues.apache.org/jira/browse/KAFKA-1057>:
>> Trim
>> > > whitespaces from user specified configs
>> > > - KAFKA-1641 <https://issues.apache.org/jira/browse/KAFKA-1641>:
>> Log
>> > > cleaner exits if last cleaned offset is lower than earliest offset
>> > > - KAFKA-1648 <https://issues.apache.org/jira/browse/KAFKA-1648>:
>> > Round
>> > > robin consumer balance throws an NPE when there are no topics
>> > > - KAFKA-1724 <https://issues.apache.org/jira/browse/KAFKA-1724>:
>> > > Errors after reboot in single node setup
>> > > - KAFKA-1758 <https://issues.apache.org/jira/browse/KAFKA-1758>:
>> > > corrupt recovery file prevents startup
>> > > - KAFKA-1866 <https://issues.apache.org/jira/browse/KAFKA-1866>:
>> > > LogStartOffset gauge throws exceptions after log.delete()
>> > > - KAFKA-1883 <https://issues.apache.org/jira/browse/KAFKA-1883>:
>> > NullPointerException
>> > > in RequestSendThread
>> > > - KAFKA-1896 <https://issues.apache.org/jira/browse/KAFKA-1896>:
>> > > Record size funcition of record in mirror maker hit NPE when the
>> > message
>> > > value is null.
>> > > - KAFKA-2101 <https://issues.apache.org/jira/browse/KAFKA-2101>:
>> > > Metric metadata-age is reset on a failed update
>> > > - KAFKA-2112 <https://issues.apache.org/jira/browse/KAFKA-2112>:
>> make
>> > > overflowWheel volatile
>> > > - KAFKA-2117 <https://issues.apache.org/jira/browse/KAFKA-2117>:
>> > > OffsetManager uses incorrect field for metadata
>> > > - KAFKA-2164 <https://issues.apache.org/jira/browse/KAFKA-2164>:
>> > > ReplicaFetcherThread: suspicious log message on reset offset
>> > > - KAFKA-1668 <https://issues.apache.org/jira/browse/KAFKA-1668>:
>> > > TopicCommand doesn't warn if --topic argument doesn't match any
>> topics
>> > > - KAFKA-2198 <https://issues.apache.org/jira/browse/KAFKA-2198>:
>> > > kafka-topics.sh exits with 0 status on failures
>> > > - KAFKA-2235 <https://issues.apache.org/jira/browse/KAFKA-2235>:
>> > > LogCleaner offset map overflow
>> > > - KAFKA-2241 <https://issues.apache.org/jira/browse/KAFKA-2241>:
>> > > AbstractFetcherThread.shutdown() should not block on
>> > > ReadableByteChannel.read(buffer)
>> > > - KAFKA-2272 <https://issues.apache.org/jira/browse/KAFKA-2272>:
>> > > listeners endpoint parsing fails if the hostname has capital letter
>> > > - KAFKA-2345 <https://issues.apache.org/jira/browse/KAFKA-2345>:
>> > > Attempt to delete a topic already marked for deletion throws
>> > > ZkNodeExistsException
>> > > - KAFKA-2353 <https://issues.apache.org/jira/browse/KAFKA-2353>:
>> > > SocketServer.Processor should catch exception and close the socket
>> > properly
>> > > in configureNewConnections.
>> > > - KAFKA-1836 <https://issues.apache.org/jira/browse/KAFKA-1836>:
>> > > metadata.fetch.timeout.ms set to zero blocks forever
>> > > - KAFKA-2317 <https://issues.apache.org/jira/browse/KAFKA-2317>:
>> > De-register
>> > > isrChangeNotificationListener on controller resignation
>> > >
>> > > Note: KAFKA-2120 <https://issues.apache.org/jira/browse/KAFKA-2120> &
>> > > KAFKA-2421 <https://issues.apache.org/jira/browse/KAFKA-2421> were
>> > > mentioned in previous emails, but are not in the list because they are
>> > not
>> > > committed yet.
>> > >
>> > > Hope that helps the effort.
>> > >
>> > > Thanks,
>> > > Grant
>> > >
>> > > On Mon, Aug 17, 2015 at 12:09 AM, Grant Henke <gh...@cloudera.com>
>> > wrote:
>> > >
>> > >> +1 to that suggestion. Though I suspect that requires a committer to
>> do.
>> > >> Making it part of the standard commit process could work too.
>> > >> On Aug 16, 2015 11:01 PM, "Gwen Shapira" <gw...@confluent.io> wrote:
>> > >>
>> > >>> BTW. I think it will be great for Apache Kafka to have a 0.8.2
>> "release
>> > >>> manager" who's role is to cherrypick low-risk bug-fixes into the
>> 0.8.2
>> > >>> branch and once enough bug fixes happened (or if sufficiently
>> critical
>> > >>> fixes happened) to roll out a new maintenance release (with every 3
>> > month
>> > >>> as a reasonable bugfix release target).
>> > >>>
>> > >>> This will add some predictability regarding how fast we release
>> fixes
>> > for
>> > >>> bugs.
>> > >>>
>> > >>> Gwen
>> > >>>
>> > >>> On Sun, Aug 16, 2015 at 8:09 PM, Jeff Holoman <
>> jholoman@cloudera.com>
>> > >>> wrote:
>> > >>>
>> > >>> > +1 for the release and also including
>> > >>> >
>> > >>> > https://issues.apache.org/jira/browse/KAFKA-2114
>> > >>> >
>> > >>> > Thanks
>> > >>> >
>> > >>> > Jeff
>> > >>> >
>> > >>> > On Sun, Aug 16, 2015 at 2:51 PM, Stevo Slavić <ss...@gmail.com>
>> > >>> wrote:
>> > >>> >
>> > >>> > > +1 (non-binding) for 0.8.2.2 release
>> > >>> > >
>> > >>> > > Would be nice to include in that release new producer resiliency
>> > bug
>> > >>> > fixes
>> > >>> > > https://issues.apache.org/jira/browse/KAFKA-1788 and
>> > >>> > > https://issues.apache.org/jira/browse/KAFKA-2120
>> > >>> > >
>> > >>> > > On Fri, Aug 14, 2015 at 4:03 PM, Gwen Shapira <
>> gwen@confluent.io>
>> > >>> wrote:
>> > >>> > >
>> > >>> > > > Will be nice to include Kafka-2308 and fix two critical snappy
>> > >>> issues
>> > >>> > in
>> > >>> > > > the maintenance release.
>> > >>> > > >
>> > >>> > > > Gwen
>> > >>> > > > On Aug 14, 2015 6:16 AM, "Grant Henke" <gh...@cloudera.com>
>> > >>> wrote:
>> > >>> > > >
>> > >>> > > > > Just to clarify. Will KAFKA-2189 be the only patch in the
>> > >>> release?
>> > >>> > > > >
>> > >>> > > > > On Fri, Aug 14, 2015 at 7:35 AM, Manikumar Reddy <
>> > >>> > kumar@nmsworks.co.in
>> > >>> > > >
>> > >>> > > > > wrote:
>> > >>> > > > >
>> > >>> > > > > > +1 for 0.8.2.2 release
>> > >>> > > > > >
>> > >>> > > > > > On Fri, Aug 14, 2015 at 5:49 PM, Ismael Juma <
>> > >>> ismael@juma.me.uk>
>> > >>> > > > wrote:
>> > >>> > > > > >
>> > >>> > > > > > > I think this is a good idea as the change is minimal on
>> our
>> > >>> side
>> > >>> > > and
>> > >>> > > > it
>> > >>> > > > > > has
>> > >>> > > > > > > been tested in production for some time by the reporter.
>> > >>> > > > > > >
>> > >>> > > > > > > Best,
>> > >>> > > > > > > Ismael
>> > >>> > > > > > >
>> > >>> > > > > > > On Fri, Aug 14, 2015 at 1:15 PM, Jun Rao <
>> jun@confluent.io
>> > >
>> > >>> > wrote:
>> > >>> > > > > > >
>> > >>> > > > > > > > Hi, Everyone,
>> > >>> > > > > > > >
>> > >>> > > > > > > > Since the release of Kafka 0.8.2.1, a number of people
>> > have
>> > >>> > > > reported
>> > >>> > > > > an
>> > >>> > > > > > > > issue with snappy compression (
>> > >>> > > > > > > > https://issues.apache.org/jira/browse/KAFKA-2189).
>> > >>> Basically,
>> > >>> > if
>> > >>> > > > > they
>> > >>> > > > > > > use
>> > >>> > > > > > > > snappy in 0.8.2.1, they will experience a 2-3X space
>> > >>> increase.
>> > >>> > > The
>> > >>> > > > > > issue
>> > >>> > > > > > > > has since been fixed in trunk (just a snappy jar
>> > upgrade).
>> > >>> > Since
>> > >>> > > > > 0.8.3
>> > >>> > > > > > is
>> > >>> > > > > > > > still a few months away, it may make sense to do an
>> > 0.8.2.2
>> > >>> > > release
>> > >>> > > > > > just
>> > >>> > > > > > > to
>> > >>> > > > > > > > fix this issue. Any objections?
>> > >>> > > > > > > >
>> > >>> > > > > > > > Thanks,
>> > >>> > > > > > > >
>> > >>> > > > > > > > Jun
>> > >>> > > > > > > >
>> > >>> > > > > > >
>> > >>> > > > > >
>> > >>> > > > >
>> > >>> > > > >
>> > >>> > > > >
>> > >>> > > > > --
>> > >>> > > > > Grant Henke
>> > >>> > > > > Software Engineer | Cloudera
>> > >>> > > > > grant@cloudera.com | twitter.com/gchenke |
>> > >>> > linkedin.com/in/granthenke
>> > >>> > > > >
>> > >>> > > >
>> > >>> > >
>> > >>> >
>> > >>> >
>> > >>> >
>> > >>> > --
>> > >>> > Jeff Holoman
>> > >>> > Systems Engineer
>> > >>> >
>> > >>>
>> > >>
>> > >
>> > >
>> > > --
>> > > Grant Henke
>> > > Software Engineer | Cloudera
>> > > grant@cloudera.com | twitter.com/gchenke | linkedin.com/in/granthenke
>> > >
>> >
>> >
>> >
>> > --
>> > Grant Henke
>> > Software Engineer | Cloudera
>> > grant@cloudera.com | twitter.com/gchenke | linkedin.com/in/granthenke
>> >
>>
>
>
Re: [DISCUSSION] Kafka 0.8.2.2 release?
Posted by Gwen Shapira <gw...@confluent.io>.
but +1 for 0.8.2 patch that marks the new consumer API as unstable (or
unimplemented ;)
On Mon, Aug 17, 2015 at 9:12 AM, Gwen Shapira <gw...@confluent.io> wrote:
> The network refactoring portion was not tested well enough yet for me to
> feel comfortable pushing it into a bugfix release. The new purgatory and
> MirrorMaker changes are also pretty big.
>
> The whole goal of bugfix releases is to make sure it is more stable than
> the previous releases.
>
> On Mon, Aug 17, 2015 at 8:54 AM, Stevo Slavić <ss...@gmail.com> wrote:
>
>> Instead of cherrypicking, why not just make 0.8.2.2 of off current trunk,
>> with new consumer API appropriately annotated/documented as unstable?
>>
>> On Mon, Aug 17, 2015, 17:17 Grant Henke <gh...@cloudera.com> wrote:
>>
>> > +dev
>> >
>> > Adding dev list back in. Somehow it got dropped.
>> >
>> >
>> > On Mon, Aug 17, 2015 at 10:16 AM, Grant Henke <gh...@cloudera.com>
>> wrote:
>> >
>> > > Below is a list of candidate bug fix jiras marked fixed for 0.8.3. I
>> > don't
>> > > suspect all of these will (or should) make it into the release but
>> this
>> > > should be a relatively complete list to work from:
>> > >
>> > > - KAFKA-2114 <https://issues.apache.org/jira/browse/KAFKA-2114>:
>> > Unable
>> > > to change min.insync.replicas default
>> > > - KAFKA-1702 <https://issues.apache.org/jira/browse/KAFKA-1702>:
>> > > Messages silently Lost by producer
>> > > - KAFKA-2012 <https://issues.apache.org/jira/browse/KAFKA-2012>:
>> > > Broker should automatically handle corrupt index files
>> > > - KAFKA-2406 <https://issues.apache.org/jira/browse/KAFKA-2406>:
>> ISR
>> > > propagation should be throttled to avoid overwhelming controller.
>> > > - KAFKA-2336 <https://issues.apache.org/jira/browse/KAFKA-2336>:
>> > > Changing offsets.topic.num.partitions after the offset topic is
>> > created
>> > > breaks consumer group partition assignment
>> > > - KAFKA-2337 <https://issues.apache.org/jira/browse/KAFKA-2337>:
>> > Verify
>> > > that metric names will not collide when creating new topics
>> > > - KAFKA-2393 <https://issues.apache.org/jira/browse/KAFKA-2393>:
>> > > Correctly Handle InvalidTopicException in
>> KafkaApis.getTopicMetadata()
>> > > - KAFKA-2189 <https://issues.apache.org/jira/browse/KAFKA-2189>:
>> > Snappy
>> > > compression of message batches less efficient in 0.8.2.1
>> > > - KAFKA-2308 <https://issues.apache.org/jira/browse/KAFKA-2308>:
>> New
>> > > producer + Snappy face un-compression errors after broker restart
>> > > - KAFKA-2042 <https://issues.apache.org/jira/browse/KAFKA-2042>:
>> New
>> > > producer metadata update always get all topics.
>> > > - KAFKA-1367 <https://issues.apache.org/jira/browse/KAFKA-1367>:
>> > Broker
>> > > topic metadata not kept in sync with ZooKeeper
>> > > - KAFKA-972 <https://issues.apache.org/jira/browse/KAFKA-972>:
>> > MetadataRequest
>> > > returns stale list of brokers
>> > > - KAFKA-1867 <https://issues.apache.org/jira/browse/KAFKA-1867>:
>> > liveBroker
>> > > list not updated on a cluster with no topics
>> > > - KAFKA-1650 <https://issues.apache.org/jira/browse/KAFKA-1650>:
>> > Mirror
>> > > Maker could lose data on unclean shutdown.
>> > > - KAFKA-2009 <https://issues.apache.org/jira/browse/KAFKA-2009>:
>> Fix
>> > > UncheckedOffset.removeOffset synchronization and trace logging
>> issue
>> > in
>> > > mirror maker
>> > > - KAFKA-2407 <https://issues.apache.org/jira/browse/KAFKA-2407>:
>> Only
>> > > create a log directory when it will be used
>> > > - KAFKA-2327 <https://issues.apache.org/jira/browse/KAFKA-2327>:
>> > > broker doesn't start if config defines advertised.host but not
>> > > advertised.port
>> > > - KAFKA-1788: producer record can stay in RecordAccumulator
>> forever if
>> > > leader is no available
>> > > - KAFKA-2234 <https://issues.apache.org/jira/browse/KAFKA-2234>:
>> > > Partition reassignment of a nonexistent topic prevents future
>> > reassignments
>> > > - KAFKA-2096 <https://issues.apache.org/jira/browse/KAFKA-2096>:
>> > > Enable keepalive socket option for broker to prevent socket leak
>> > > - KAFKA-1057 <https://issues.apache.org/jira/browse/KAFKA-1057>:
>> Trim
>> > > whitespaces from user specified configs
>> > > - KAFKA-1641 <https://issues.apache.org/jira/browse/KAFKA-1641>:
>> Log
>> > > cleaner exits if last cleaned offset is lower than earliest offset
>> > > - KAFKA-1648 <https://issues.apache.org/jira/browse/KAFKA-1648>:
>> > Round
>> > > robin consumer balance throws an NPE when there are no topics
>> > > - KAFKA-1724 <https://issues.apache.org/jira/browse/KAFKA-1724>:
>> > > Errors after reboot in single node setup
>> > > - KAFKA-1758 <https://issues.apache.org/jira/browse/KAFKA-1758>:
>> > > corrupt recovery file prevents startup
>> > > - KAFKA-1866 <https://issues.apache.org/jira/browse/KAFKA-1866>:
>> > > LogStartOffset gauge throws exceptions after log.delete()
>> > > - KAFKA-1883 <https://issues.apache.org/jira/browse/KAFKA-1883>:
>> > NullPointerException
>> > > in RequestSendThread
>> > > - KAFKA-1896 <https://issues.apache.org/jira/browse/KAFKA-1896>:
>> > > Record size funcition of record in mirror maker hit NPE when the
>> > message
>> > > value is null.
>> > > - KAFKA-2101 <https://issues.apache.org/jira/browse/KAFKA-2101>:
>> > > Metric metadata-age is reset on a failed update
>> > > - KAFKA-2112 <https://issues.apache.org/jira/browse/KAFKA-2112>:
>> make
>> > > overflowWheel volatile
>> > > - KAFKA-2117 <https://issues.apache.org/jira/browse/KAFKA-2117>:
>> > > OffsetManager uses incorrect field for metadata
>> > > - KAFKA-2164 <https://issues.apache.org/jira/browse/KAFKA-2164>:
>> > > ReplicaFetcherThread: suspicious log message on reset offset
>> > > - KAFKA-1668 <https://issues.apache.org/jira/browse/KAFKA-1668>:
>> > > TopicCommand doesn't warn if --topic argument doesn't match any
>> topics
>> > > - KAFKA-2198 <https://issues.apache.org/jira/browse/KAFKA-2198>:
>> > > kafka-topics.sh exits with 0 status on failures
>> > > - KAFKA-2235 <https://issues.apache.org/jira/browse/KAFKA-2235>:
>> > > LogCleaner offset map overflow
>> > > - KAFKA-2241 <https://issues.apache.org/jira/browse/KAFKA-2241>:
>> > > AbstractFetcherThread.shutdown() should not block on
>> > > ReadableByteChannel.read(buffer)
>> > > - KAFKA-2272 <https://issues.apache.org/jira/browse/KAFKA-2272>:
>> > > listeners endpoint parsing fails if the hostname has capital letter
>> > > - KAFKA-2345 <https://issues.apache.org/jira/browse/KAFKA-2345>:
>> > > Attempt to delete a topic already marked for deletion throws
>> > > ZkNodeExistsException
>> > > - KAFKA-2353 <https://issues.apache.org/jira/browse/KAFKA-2353>:
>> > > SocketServer.Processor should catch exception and close the socket
>> > properly
>> > > in configureNewConnections.
>> > > - KAFKA-1836 <https://issues.apache.org/jira/browse/KAFKA-1836>:
>> > > metadata.fetch.timeout.ms set to zero blocks forever
>> > > - KAFKA-2317 <https://issues.apache.org/jira/browse/KAFKA-2317>:
>> > De-register
>> > > isrChangeNotificationListener on controller resignation
>> > >
>> > > Note: KAFKA-2120 <https://issues.apache.org/jira/browse/KAFKA-2120> &
>> > > KAFKA-2421 <https://issues.apache.org/jira/browse/KAFKA-2421> were
>> > > mentioned in previous emails, but are not in the list because they are
>> > not
>> > > committed yet.
>> > >
>> > > Hope that helps the effort.
>> > >
>> > > Thanks,
>> > > Grant
>> > >
>> > > On Mon, Aug 17, 2015 at 12:09 AM, Grant Henke <gh...@cloudera.com>
>> > wrote:
>> > >
>> > >> +1 to that suggestion. Though I suspect that requires a committer to
>> do.
>> > >> Making it part of the standard commit process could work too.
>> > >> On Aug 16, 2015 11:01 PM, "Gwen Shapira" <gw...@confluent.io> wrote:
>> > >>
>> > >>> BTW. I think it will be great for Apache Kafka to have a 0.8.2
>> "release
>> > >>> manager" who's role is to cherrypick low-risk bug-fixes into the
>> 0.8.2
>> > >>> branch and once enough bug fixes happened (or if sufficiently
>> critical
>> > >>> fixes happened) to roll out a new maintenance release (with every 3
>> > month
>> > >>> as a reasonable bugfix release target).
>> > >>>
>> > >>> This will add some predictability regarding how fast we release
>> fixes
>> > for
>> > >>> bugs.
>> > >>>
>> > >>> Gwen
>> > >>>
>> > >>> On Sun, Aug 16, 2015 at 8:09 PM, Jeff Holoman <
>> jholoman@cloudera.com>
>> > >>> wrote:
>> > >>>
>> > >>> > +1 for the release and also including
>> > >>> >
>> > >>> > https://issues.apache.org/jira/browse/KAFKA-2114
>> > >>> >
>> > >>> > Thanks
>> > >>> >
>> > >>> > Jeff
>> > >>> >
>> > >>> > On Sun, Aug 16, 2015 at 2:51 PM, Stevo Slavić <ss...@gmail.com>
>> > >>> wrote:
>> > >>> >
>> > >>> > > +1 (non-binding) for 0.8.2.2 release
>> > >>> > >
>> > >>> > > Would be nice to include in that release new producer resiliency
>> > bug
>> > >>> > fixes
>> > >>> > > https://issues.apache.org/jira/browse/KAFKA-1788 and
>> > >>> > > https://issues.apache.org/jira/browse/KAFKA-2120
>> > >>> > >
>> > >>> > > On Fri, Aug 14, 2015 at 4:03 PM, Gwen Shapira <
>> gwen@confluent.io>
>> > >>> wrote:
>> > >>> > >
>> > >>> > > > Will be nice to include Kafka-2308 and fix two critical snappy
>> > >>> issues
>> > >>> > in
>> > >>> > > > the maintenance release.
>> > >>> > > >
>> > >>> > > > Gwen
>> > >>> > > > On Aug 14, 2015 6:16 AM, "Grant Henke" <gh...@cloudera.com>
>> > >>> wrote:
>> > >>> > > >
>> > >>> > > > > Just to clarify. Will KAFKA-2189 be the only patch in the
>> > >>> release?
>> > >>> > > > >
>> > >>> > > > > On Fri, Aug 14, 2015 at 7:35 AM, Manikumar Reddy <
>> > >>> > kumar@nmsworks.co.in
>> > >>> > > >
>> > >>> > > > > wrote:
>> > >>> > > > >
>> > >>> > > > > > +1 for 0.8.2.2 release
>> > >>> > > > > >
>> > >>> > > > > > On Fri, Aug 14, 2015 at 5:49 PM, Ismael Juma <
>> > >>> ismael@juma.me.uk>
>> > >>> > > > wrote:
>> > >>> > > > > >
>> > >>> > > > > > > I think this is a good idea as the change is minimal on
>> our
>> > >>> side
>> > >>> > > and
>> > >>> > > > it
>> > >>> > > > > > has
>> > >>> > > > > > > been tested in production for some time by the reporter.
>> > >>> > > > > > >
>> > >>> > > > > > > Best,
>> > >>> > > > > > > Ismael
>> > >>> > > > > > >
>> > >>> > > > > > > On Fri, Aug 14, 2015 at 1:15 PM, Jun Rao <
>> jun@confluent.io
>> > >
>> > >>> > wrote:
>> > >>> > > > > > >
>> > >>> > > > > > > > Hi, Everyone,
>> > >>> > > > > > > >
>> > >>> > > > > > > > Since the release of Kafka 0.8.2.1, a number of people
>> > have
>> > >>> > > > reported
>> > >>> > > > > an
>> > >>> > > > > > > > issue with snappy compression (
>> > >>> > > > > > > > https://issues.apache.org/jira/browse/KAFKA-2189).
>> > >>> Basically,
>> > >>> > if
>> > >>> > > > > they
>> > >>> > > > > > > use
>> > >>> > > > > > > > snappy in 0.8.2.1, they will experience a 2-3X space
>> > >>> increase.
>> > >>> > > The
>> > >>> > > > > > issue
>> > >>> > > > > > > > has since been fixed in trunk (just a snappy jar
>> > upgrade).
>> > >>> > Since
>> > >>> > > > > 0.8.3
>> > >>> > > > > > is
>> > >>> > > > > > > > still a few months away, it may make sense to do an
>> > 0.8.2.2
>> > >>> > > release
>> > >>> > > > > > just
>> > >>> > > > > > > to
>> > >>> > > > > > > > fix this issue. Any objections?
>> > >>> > > > > > > >
>> > >>> > > > > > > > Thanks,
>> > >>> > > > > > > >
>> > >>> > > > > > > > Jun
>> > >>> > > > > > > >
>> > >>> > > > > > >
>> > >>> > > > > >
>> > >>> > > > >
>> > >>> > > > >
>> > >>> > > > >
>> > >>> > > > > --
>> > >>> > > > > Grant Henke
>> > >>> > > > > Software Engineer | Cloudera
>> > >>> > > > > grant@cloudera.com | twitter.com/gchenke |
>> > >>> > linkedin.com/in/granthenke
>> > >>> > > > >
>> > >>> > > >
>> > >>> > >
>> > >>> >
>> > >>> >
>> > >>> >
>> > >>> > --
>> > >>> > Jeff Holoman
>> > >>> > Systems Engineer
>> > >>> >
>> > >>>
>> > >>
>> > >
>> > >
>> > > --
>> > > Grant Henke
>> > > Software Engineer | Cloudera
>> > > grant@cloudera.com | twitter.com/gchenke | linkedin.com/in/granthenke
>> > >
>> >
>> >
>> >
>> > --
>> > Grant Henke
>> > Software Engineer | Cloudera
>> > grant@cloudera.com | twitter.com/gchenke | linkedin.com/in/granthenke
>> >
>>
>
>
Re: [DISCUSSION] Kafka 0.8.2.2 release?
Posted by Gwen Shapira <gw...@confluent.io>.
The network refactoring portion was not tested well enough yet for me to
feel comfortable pushing it into a bugfix release. The new purgatory and
MirrorMaker changes are also pretty big.
The whole goal of bugfix releases is to make sure it is more stable than
the previous releases.
On Mon, Aug 17, 2015 at 8:54 AM, Stevo Slavić <ss...@gmail.com> wrote:
> Instead of cherrypicking, why not just make 0.8.2.2 of off current trunk,
> with new consumer API appropriately annotated/documented as unstable?
>
> On Mon, Aug 17, 2015, 17:17 Grant Henke <gh...@cloudera.com> wrote:
>
> > +dev
> >
> > Adding dev list back in. Somehow it got dropped.
> >
> >
> > On Mon, Aug 17, 2015 at 10:16 AM, Grant Henke <gh...@cloudera.com>
> wrote:
> >
> > > Below is a list of candidate bug fix jiras marked fixed for 0.8.3. I
> > don't
> > > suspect all of these will (or should) make it into the release but this
> > > should be a relatively complete list to work from:
> > >
> > > - KAFKA-2114 <https://issues.apache.org/jira/browse/KAFKA-2114>:
> > Unable
> > > to change min.insync.replicas default
> > > - KAFKA-1702 <https://issues.apache.org/jira/browse/KAFKA-1702>:
> > > Messages silently Lost by producer
> > > - KAFKA-2012 <https://issues.apache.org/jira/browse/KAFKA-2012>:
> > > Broker should automatically handle corrupt index files
> > > - KAFKA-2406 <https://issues.apache.org/jira/browse/KAFKA-2406>:
> ISR
> > > propagation should be throttled to avoid overwhelming controller.
> > > - KAFKA-2336 <https://issues.apache.org/jira/browse/KAFKA-2336>:
> > > Changing offsets.topic.num.partitions after the offset topic is
> > created
> > > breaks consumer group partition assignment
> > > - KAFKA-2337 <https://issues.apache.org/jira/browse/KAFKA-2337>:
> > Verify
> > > that metric names will not collide when creating new topics
> > > - KAFKA-2393 <https://issues.apache.org/jira/browse/KAFKA-2393>:
> > > Correctly Handle InvalidTopicException in
> KafkaApis.getTopicMetadata()
> > > - KAFKA-2189 <https://issues.apache.org/jira/browse/KAFKA-2189>:
> > Snappy
> > > compression of message batches less efficient in 0.8.2.1
> > > - KAFKA-2308 <https://issues.apache.org/jira/browse/KAFKA-2308>:
> New
> > > producer + Snappy face un-compression errors after broker restart
> > > - KAFKA-2042 <https://issues.apache.org/jira/browse/KAFKA-2042>:
> New
> > > producer metadata update always get all topics.
> > > - KAFKA-1367 <https://issues.apache.org/jira/browse/KAFKA-1367>:
> > Broker
> > > topic metadata not kept in sync with ZooKeeper
> > > - KAFKA-972 <https://issues.apache.org/jira/browse/KAFKA-972>:
> > MetadataRequest
> > > returns stale list of brokers
> > > - KAFKA-1867 <https://issues.apache.org/jira/browse/KAFKA-1867>:
> > liveBroker
> > > list not updated on a cluster with no topics
> > > - KAFKA-1650 <https://issues.apache.org/jira/browse/KAFKA-1650>:
> > Mirror
> > > Maker could lose data on unclean shutdown.
> > > - KAFKA-2009 <https://issues.apache.org/jira/browse/KAFKA-2009>:
> Fix
> > > UncheckedOffset.removeOffset synchronization and trace logging issue
> > in
> > > mirror maker
> > > - KAFKA-2407 <https://issues.apache.org/jira/browse/KAFKA-2407>:
> Only
> > > create a log directory when it will be used
> > > - KAFKA-2327 <https://issues.apache.org/jira/browse/KAFKA-2327>:
> > > broker doesn't start if config defines advertised.host but not
> > > advertised.port
> > > - KAFKA-1788: producer record can stay in RecordAccumulator forever
> if
> > > leader is no available
> > > - KAFKA-2234 <https://issues.apache.org/jira/browse/KAFKA-2234>:
> > > Partition reassignment of a nonexistent topic prevents future
> > reassignments
> > > - KAFKA-2096 <https://issues.apache.org/jira/browse/KAFKA-2096>:
> > > Enable keepalive socket option for broker to prevent socket leak
> > > - KAFKA-1057 <https://issues.apache.org/jira/browse/KAFKA-1057>:
> Trim
> > > whitespaces from user specified configs
> > > - KAFKA-1641 <https://issues.apache.org/jira/browse/KAFKA-1641>:
> Log
> > > cleaner exits if last cleaned offset is lower than earliest offset
> > > - KAFKA-1648 <https://issues.apache.org/jira/browse/KAFKA-1648>:
> > Round
> > > robin consumer balance throws an NPE when there are no topics
> > > - KAFKA-1724 <https://issues.apache.org/jira/browse/KAFKA-1724>:
> > > Errors after reboot in single node setup
> > > - KAFKA-1758 <https://issues.apache.org/jira/browse/KAFKA-1758>:
> > > corrupt recovery file prevents startup
> > > - KAFKA-1866 <https://issues.apache.org/jira/browse/KAFKA-1866>:
> > > LogStartOffset gauge throws exceptions after log.delete()
> > > - KAFKA-1883 <https://issues.apache.org/jira/browse/KAFKA-1883>:
> > NullPointerException
> > > in RequestSendThread
> > > - KAFKA-1896 <https://issues.apache.org/jira/browse/KAFKA-1896>:
> > > Record size funcition of record in mirror maker hit NPE when the
> > message
> > > value is null.
> > > - KAFKA-2101 <https://issues.apache.org/jira/browse/KAFKA-2101>:
> > > Metric metadata-age is reset on a failed update
> > > - KAFKA-2112 <https://issues.apache.org/jira/browse/KAFKA-2112>:
> make
> > > overflowWheel volatile
> > > - KAFKA-2117 <https://issues.apache.org/jira/browse/KAFKA-2117>:
> > > OffsetManager uses incorrect field for metadata
> > > - KAFKA-2164 <https://issues.apache.org/jira/browse/KAFKA-2164>:
> > > ReplicaFetcherThread: suspicious log message on reset offset
> > > - KAFKA-1668 <https://issues.apache.org/jira/browse/KAFKA-1668>:
> > > TopicCommand doesn't warn if --topic argument doesn't match any
> topics
> > > - KAFKA-2198 <https://issues.apache.org/jira/browse/KAFKA-2198>:
> > > kafka-topics.sh exits with 0 status on failures
> > > - KAFKA-2235 <https://issues.apache.org/jira/browse/KAFKA-2235>:
> > > LogCleaner offset map overflow
> > > - KAFKA-2241 <https://issues.apache.org/jira/browse/KAFKA-2241>:
> > > AbstractFetcherThread.shutdown() should not block on
> > > ReadableByteChannel.read(buffer)
> > > - KAFKA-2272 <https://issues.apache.org/jira/browse/KAFKA-2272>:
> > > listeners endpoint parsing fails if the hostname has capital letter
> > > - KAFKA-2345 <https://issues.apache.org/jira/browse/KAFKA-2345>:
> > > Attempt to delete a topic already marked for deletion throws
> > > ZkNodeExistsException
> > > - KAFKA-2353 <https://issues.apache.org/jira/browse/KAFKA-2353>:
> > > SocketServer.Processor should catch exception and close the socket
> > properly
> > > in configureNewConnections.
> > > - KAFKA-1836 <https://issues.apache.org/jira/browse/KAFKA-1836>:
> > > metadata.fetch.timeout.ms set to zero blocks forever
> > > - KAFKA-2317 <https://issues.apache.org/jira/browse/KAFKA-2317>:
> > De-register
> > > isrChangeNotificationListener on controller resignation
> > >
> > > Note: KAFKA-2120 <https://issues.apache.org/jira/browse/KAFKA-2120> &
> > > KAFKA-2421 <https://issues.apache.org/jira/browse/KAFKA-2421> were
> > > mentioned in previous emails, but are not in the list because they are
> > not
> > > committed yet.
> > >
> > > Hope that helps the effort.
> > >
> > > Thanks,
> > > Grant
> > >
> > > On Mon, Aug 17, 2015 at 12:09 AM, Grant Henke <gh...@cloudera.com>
> > wrote:
> > >
> > >> +1 to that suggestion. Though I suspect that requires a committer to
> do.
> > >> Making it part of the standard commit process could work too.
> > >> On Aug 16, 2015 11:01 PM, "Gwen Shapira" <gw...@confluent.io> wrote:
> > >>
> > >>> BTW. I think it will be great for Apache Kafka to have a 0.8.2
> "release
> > >>> manager" who's role is to cherrypick low-risk bug-fixes into the
> 0.8.2
> > >>> branch and once enough bug fixes happened (or if sufficiently
> critical
> > >>> fixes happened) to roll out a new maintenance release (with every 3
> > month
> > >>> as a reasonable bugfix release target).
> > >>>
> > >>> This will add some predictability regarding how fast we release fixes
> > for
> > >>> bugs.
> > >>>
> > >>> Gwen
> > >>>
> > >>> On Sun, Aug 16, 2015 at 8:09 PM, Jeff Holoman <jholoman@cloudera.com
> >
> > >>> wrote:
> > >>>
> > >>> > +1 for the release and also including
> > >>> >
> > >>> > https://issues.apache.org/jira/browse/KAFKA-2114
> > >>> >
> > >>> > Thanks
> > >>> >
> > >>> > Jeff
> > >>> >
> > >>> > On Sun, Aug 16, 2015 at 2:51 PM, Stevo Slavić <ss...@gmail.com>
> > >>> wrote:
> > >>> >
> > >>> > > +1 (non-binding) for 0.8.2.2 release
> > >>> > >
> > >>> > > Would be nice to include in that release new producer resiliency
> > bug
> > >>> > fixes
> > >>> > > https://issues.apache.org/jira/browse/KAFKA-1788 and
> > >>> > > https://issues.apache.org/jira/browse/KAFKA-2120
> > >>> > >
> > >>> > > On Fri, Aug 14, 2015 at 4:03 PM, Gwen Shapira <gwen@confluent.io
> >
> > >>> wrote:
> > >>> > >
> > >>> > > > Will be nice to include Kafka-2308 and fix two critical snappy
> > >>> issues
> > >>> > in
> > >>> > > > the maintenance release.
> > >>> > > >
> > >>> > > > Gwen
> > >>> > > > On Aug 14, 2015 6:16 AM, "Grant Henke" <gh...@cloudera.com>
> > >>> wrote:
> > >>> > > >
> > >>> > > > > Just to clarify. Will KAFKA-2189 be the only patch in the
> > >>> release?
> > >>> > > > >
> > >>> > > > > On Fri, Aug 14, 2015 at 7:35 AM, Manikumar Reddy <
> > >>> > kumar@nmsworks.co.in
> > >>> > > >
> > >>> > > > > wrote:
> > >>> > > > >
> > >>> > > > > > +1 for 0.8.2.2 release
> > >>> > > > > >
> > >>> > > > > > On Fri, Aug 14, 2015 at 5:49 PM, Ismael Juma <
> > >>> ismael@juma.me.uk>
> > >>> > > > wrote:
> > >>> > > > > >
> > >>> > > > > > > I think this is a good idea as the change is minimal on
> our
> > >>> side
> > >>> > > and
> > >>> > > > it
> > >>> > > > > > has
> > >>> > > > > > > been tested in production for some time by the reporter.
> > >>> > > > > > >
> > >>> > > > > > > Best,
> > >>> > > > > > > Ismael
> > >>> > > > > > >
> > >>> > > > > > > On Fri, Aug 14, 2015 at 1:15 PM, Jun Rao <
> jun@confluent.io
> > >
> > >>> > wrote:
> > >>> > > > > > >
> > >>> > > > > > > > Hi, Everyone,
> > >>> > > > > > > >
> > >>> > > > > > > > Since the release of Kafka 0.8.2.1, a number of people
> > have
> > >>> > > > reported
> > >>> > > > > an
> > >>> > > > > > > > issue with snappy compression (
> > >>> > > > > > > > https://issues.apache.org/jira/browse/KAFKA-2189).
> > >>> Basically,
> > >>> > if
> > >>> > > > > they
> > >>> > > > > > > use
> > >>> > > > > > > > snappy in 0.8.2.1, they will experience a 2-3X space
> > >>> increase.
> > >>> > > The
> > >>> > > > > > issue
> > >>> > > > > > > > has since been fixed in trunk (just a snappy jar
> > upgrade).
> > >>> > Since
> > >>> > > > > 0.8.3
> > >>> > > > > > is
> > >>> > > > > > > > still a few months away, it may make sense to do an
> > 0.8.2.2
> > >>> > > release
> > >>> > > > > > just
> > >>> > > > > > > to
> > >>> > > > > > > > fix this issue. Any objections?
> > >>> > > > > > > >
> > >>> > > > > > > > Thanks,
> > >>> > > > > > > >
> > >>> > > > > > > > Jun
> > >>> > > > > > > >
> > >>> > > > > > >
> > >>> > > > > >
> > >>> > > > >
> > >>> > > > >
> > >>> > > > >
> > >>> > > > > --
> > >>> > > > > Grant Henke
> > >>> > > > > Software Engineer | Cloudera
> > >>> > > > > grant@cloudera.com | twitter.com/gchenke |
> > >>> > linkedin.com/in/granthenke
> > >>> > > > >
> > >>> > > >
> > >>> > >
> > >>> >
> > >>> >
> > >>> >
> > >>> > --
> > >>> > Jeff Holoman
> > >>> > Systems Engineer
> > >>> >
> > >>>
> > >>
> > >
> > >
> > > --
> > > Grant Henke
> > > Software Engineer | Cloudera
> > > grant@cloudera.com | twitter.com/gchenke | linkedin.com/in/granthenke
> > >
> >
> >
> >
> > --
> > Grant Henke
> > Software Engineer | Cloudera
> > grant@cloudera.com | twitter.com/gchenke | linkedin.com/in/granthenke
> >
>
Re: [DISCUSSION] Kafka 0.8.2.2 release?
Posted by Gwen Shapira <gw...@confluent.io>.
The network refactoring portion was not tested well enough yet for me to
feel comfortable pushing it into a bugfix release. The new purgatory and
MirrorMaker changes are also pretty big.
The whole goal of bugfix releases is to make sure it is more stable than
the previous releases.
On Mon, Aug 17, 2015 at 8:54 AM, Stevo Slavić <ss...@gmail.com> wrote:
> Instead of cherrypicking, why not just make 0.8.2.2 of off current trunk,
> with new consumer API appropriately annotated/documented as unstable?
>
> On Mon, Aug 17, 2015, 17:17 Grant Henke <gh...@cloudera.com> wrote:
>
> > +dev
> >
> > Adding dev list back in. Somehow it got dropped.
> >
> >
> > On Mon, Aug 17, 2015 at 10:16 AM, Grant Henke <gh...@cloudera.com>
> wrote:
> >
> > > Below is a list of candidate bug fix jiras marked fixed for 0.8.3. I
> > don't
> > > suspect all of these will (or should) make it into the release but this
> > > should be a relatively complete list to work from:
> > >
> > > - KAFKA-2114 <https://issues.apache.org/jira/browse/KAFKA-2114>:
> > Unable
> > > to change min.insync.replicas default
> > > - KAFKA-1702 <https://issues.apache.org/jira/browse/KAFKA-1702>:
> > > Messages silently Lost by producer
> > > - KAFKA-2012 <https://issues.apache.org/jira/browse/KAFKA-2012>:
> > > Broker should automatically handle corrupt index files
> > > - KAFKA-2406 <https://issues.apache.org/jira/browse/KAFKA-2406>:
> ISR
> > > propagation should be throttled to avoid overwhelming controller.
> > > - KAFKA-2336 <https://issues.apache.org/jira/browse/KAFKA-2336>:
> > > Changing offsets.topic.num.partitions after the offset topic is
> > created
> > > breaks consumer group partition assignment
> > > - KAFKA-2337 <https://issues.apache.org/jira/browse/KAFKA-2337>:
> > Verify
> > > that metric names will not collide when creating new topics
> > > - KAFKA-2393 <https://issues.apache.org/jira/browse/KAFKA-2393>:
> > > Correctly Handle InvalidTopicException in
> KafkaApis.getTopicMetadata()
> > > - KAFKA-2189 <https://issues.apache.org/jira/browse/KAFKA-2189>:
> > Snappy
> > > compression of message batches less efficient in 0.8.2.1
> > > - KAFKA-2308 <https://issues.apache.org/jira/browse/KAFKA-2308>:
> New
> > > producer + Snappy face un-compression errors after broker restart
> > > - KAFKA-2042 <https://issues.apache.org/jira/browse/KAFKA-2042>:
> New
> > > producer metadata update always get all topics.
> > > - KAFKA-1367 <https://issues.apache.org/jira/browse/KAFKA-1367>:
> > Broker
> > > topic metadata not kept in sync with ZooKeeper
> > > - KAFKA-972 <https://issues.apache.org/jira/browse/KAFKA-972>:
> > MetadataRequest
> > > returns stale list of brokers
> > > - KAFKA-1867 <https://issues.apache.org/jira/browse/KAFKA-1867>:
> > liveBroker
> > > list not updated on a cluster with no topics
> > > - KAFKA-1650 <https://issues.apache.org/jira/browse/KAFKA-1650>:
> > Mirror
> > > Maker could lose data on unclean shutdown.
> > > - KAFKA-2009 <https://issues.apache.org/jira/browse/KAFKA-2009>:
> Fix
> > > UncheckedOffset.removeOffset synchronization and trace logging issue
> > in
> > > mirror maker
> > > - KAFKA-2407 <https://issues.apache.org/jira/browse/KAFKA-2407>:
> Only
> > > create a log directory when it will be used
> > > - KAFKA-2327 <https://issues.apache.org/jira/browse/KAFKA-2327>:
> > > broker doesn't start if config defines advertised.host but not
> > > advertised.port
> > > - KAFKA-1788: producer record can stay in RecordAccumulator forever
> if
> > > leader is no available
> > > - KAFKA-2234 <https://issues.apache.org/jira/browse/KAFKA-2234>:
> > > Partition reassignment of a nonexistent topic prevents future
> > reassignments
> > > - KAFKA-2096 <https://issues.apache.org/jira/browse/KAFKA-2096>:
> > > Enable keepalive socket option for broker to prevent socket leak
> > > - KAFKA-1057 <https://issues.apache.org/jira/browse/KAFKA-1057>:
> Trim
> > > whitespaces from user specified configs
> > > - KAFKA-1641 <https://issues.apache.org/jira/browse/KAFKA-1641>:
> Log
> > > cleaner exits if last cleaned offset is lower than earliest offset
> > > - KAFKA-1648 <https://issues.apache.org/jira/browse/KAFKA-1648>:
> > Round
> > > robin consumer balance throws an NPE when there are no topics
> > > - KAFKA-1724 <https://issues.apache.org/jira/browse/KAFKA-1724>:
> > > Errors after reboot in single node setup
> > > - KAFKA-1758 <https://issues.apache.org/jira/browse/KAFKA-1758>:
> > > corrupt recovery file prevents startup
> > > - KAFKA-1866 <https://issues.apache.org/jira/browse/KAFKA-1866>:
> > > LogStartOffset gauge throws exceptions after log.delete()
> > > - KAFKA-1883 <https://issues.apache.org/jira/browse/KAFKA-1883>:
> > NullPointerException
> > > in RequestSendThread
> > > - KAFKA-1896 <https://issues.apache.org/jira/browse/KAFKA-1896>:
> > > Record size funcition of record in mirror maker hit NPE when the
> > message
> > > value is null.
> > > - KAFKA-2101 <https://issues.apache.org/jira/browse/KAFKA-2101>:
> > > Metric metadata-age is reset on a failed update
> > > - KAFKA-2112 <https://issues.apache.org/jira/browse/KAFKA-2112>:
> make
> > > overflowWheel volatile
> > > - KAFKA-2117 <https://issues.apache.org/jira/browse/KAFKA-2117>:
> > > OffsetManager uses incorrect field for metadata
> > > - KAFKA-2164 <https://issues.apache.org/jira/browse/KAFKA-2164>:
> > > ReplicaFetcherThread: suspicious log message on reset offset
> > > - KAFKA-1668 <https://issues.apache.org/jira/browse/KAFKA-1668>:
> > > TopicCommand doesn't warn if --topic argument doesn't match any
> topics
> > > - KAFKA-2198 <https://issues.apache.org/jira/browse/KAFKA-2198>:
> > > kafka-topics.sh exits with 0 status on failures
> > > - KAFKA-2235 <https://issues.apache.org/jira/browse/KAFKA-2235>:
> > > LogCleaner offset map overflow
> > > - KAFKA-2241 <https://issues.apache.org/jira/browse/KAFKA-2241>:
> > > AbstractFetcherThread.shutdown() should not block on
> > > ReadableByteChannel.read(buffer)
> > > - KAFKA-2272 <https://issues.apache.org/jira/browse/KAFKA-2272>:
> > > listeners endpoint parsing fails if the hostname has capital letter
> > > - KAFKA-2345 <https://issues.apache.org/jira/browse/KAFKA-2345>:
> > > Attempt to delete a topic already marked for deletion throws
> > > ZkNodeExistsException
> > > - KAFKA-2353 <https://issues.apache.org/jira/browse/KAFKA-2353>:
> > > SocketServer.Processor should catch exception and close the socket
> > properly
> > > in configureNewConnections.
> > > - KAFKA-1836 <https://issues.apache.org/jira/browse/KAFKA-1836>:
> > > metadata.fetch.timeout.ms set to zero blocks forever
> > > - KAFKA-2317 <https://issues.apache.org/jira/browse/KAFKA-2317>:
> > De-register
> > > isrChangeNotificationListener on controller resignation
> > >
> > > Note: KAFKA-2120 <https://issues.apache.org/jira/browse/KAFKA-2120> &
> > > KAFKA-2421 <https://issues.apache.org/jira/browse/KAFKA-2421> were
> > > mentioned in previous emails, but are not in the list because they are
> > not
> > > committed yet.
> > >
> > > Hope that helps the effort.
> > >
> > > Thanks,
> > > Grant
> > >
> > > On Mon, Aug 17, 2015 at 12:09 AM, Grant Henke <gh...@cloudera.com>
> > wrote:
> > >
> > >> +1 to that suggestion. Though I suspect that requires a committer to
> do.
> > >> Making it part of the standard commit process could work too.
> > >> On Aug 16, 2015 11:01 PM, "Gwen Shapira" <gw...@confluent.io> wrote:
> > >>
> > >>> BTW. I think it will be great for Apache Kafka to have a 0.8.2
> "release
> > >>> manager" who's role is to cherrypick low-risk bug-fixes into the
> 0.8.2
> > >>> branch and once enough bug fixes happened (or if sufficiently
> critical
> > >>> fixes happened) to roll out a new maintenance release (with every 3
> > month
> > >>> as a reasonable bugfix release target).
> > >>>
> > >>> This will add some predictability regarding how fast we release fixes
> > for
> > >>> bugs.
> > >>>
> > >>> Gwen
> > >>>
> > >>> On Sun, Aug 16, 2015 at 8:09 PM, Jeff Holoman <jholoman@cloudera.com
> >
> > >>> wrote:
> > >>>
> > >>> > +1 for the release and also including
> > >>> >
> > >>> > https://issues.apache.org/jira/browse/KAFKA-2114
> > >>> >
> > >>> > Thanks
> > >>> >
> > >>> > Jeff
> > >>> >
> > >>> > On Sun, Aug 16, 2015 at 2:51 PM, Stevo Slavić <ss...@gmail.com>
> > >>> wrote:
> > >>> >
> > >>> > > +1 (non-binding) for 0.8.2.2 release
> > >>> > >
> > >>> > > Would be nice to include in that release new producer resiliency
> > bug
> > >>> > fixes
> > >>> > > https://issues.apache.org/jira/browse/KAFKA-1788 and
> > >>> > > https://issues.apache.org/jira/browse/KAFKA-2120
> > >>> > >
> > >>> > > On Fri, Aug 14, 2015 at 4:03 PM, Gwen Shapira <gwen@confluent.io
> >
> > >>> wrote:
> > >>> > >
> > >>> > > > Will be nice to include Kafka-2308 and fix two critical snappy
> > >>> issues
> > >>> > in
> > >>> > > > the maintenance release.
> > >>> > > >
> > >>> > > > Gwen
> > >>> > > > On Aug 14, 2015 6:16 AM, "Grant Henke" <gh...@cloudera.com>
> > >>> wrote:
> > >>> > > >
> > >>> > > > > Just to clarify. Will KAFKA-2189 be the only patch in the
> > >>> release?
> > >>> > > > >
> > >>> > > > > On Fri, Aug 14, 2015 at 7:35 AM, Manikumar Reddy <
> > >>> > kumar@nmsworks.co.in
> > >>> > > >
> > >>> > > > > wrote:
> > >>> > > > >
> > >>> > > > > > +1 for 0.8.2.2 release
> > >>> > > > > >
> > >>> > > > > > On Fri, Aug 14, 2015 at 5:49 PM, Ismael Juma <
> > >>> ismael@juma.me.uk>
> > >>> > > > wrote:
> > >>> > > > > >
> > >>> > > > > > > I think this is a good idea as the change is minimal on
> our
> > >>> side
> > >>> > > and
> > >>> > > > it
> > >>> > > > > > has
> > >>> > > > > > > been tested in production for some time by the reporter.
> > >>> > > > > > >
> > >>> > > > > > > Best,
> > >>> > > > > > > Ismael
> > >>> > > > > > >
> > >>> > > > > > > On Fri, Aug 14, 2015 at 1:15 PM, Jun Rao <
> jun@confluent.io
> > >
> > >>> > wrote:
> > >>> > > > > > >
> > >>> > > > > > > > Hi, Everyone,
> > >>> > > > > > > >
> > >>> > > > > > > > Since the release of Kafka 0.8.2.1, a number of people
> > have
> > >>> > > > reported
> > >>> > > > > an
> > >>> > > > > > > > issue with snappy compression (
> > >>> > > > > > > > https://issues.apache.org/jira/browse/KAFKA-2189).
> > >>> Basically,
> > >>> > if
> > >>> > > > > they
> > >>> > > > > > > use
> > >>> > > > > > > > snappy in 0.8.2.1, they will experience a 2-3X space
> > >>> increase.
> > >>> > > The
> > >>> > > > > > issue
> > >>> > > > > > > > has since been fixed in trunk (just a snappy jar
> > upgrade).
> > >>> > Since
> > >>> > > > > 0.8.3
> > >>> > > > > > is
> > >>> > > > > > > > still a few months away, it may make sense to do an
> > 0.8.2.2
> > >>> > > release
> > >>> > > > > > just
> > >>> > > > > > > to
> > >>> > > > > > > > fix this issue. Any objections?
> > >>> > > > > > > >
> > >>> > > > > > > > Thanks,
> > >>> > > > > > > >
> > >>> > > > > > > > Jun
> > >>> > > > > > > >
> > >>> > > > > > >
> > >>> > > > > >
> > >>> > > > >
> > >>> > > > >
> > >>> > > > >
> > >>> > > > > --
> > >>> > > > > Grant Henke
> > >>> > > > > Software Engineer | Cloudera
> > >>> > > > > grant@cloudera.com | twitter.com/gchenke |
> > >>> > linkedin.com/in/granthenke
> > >>> > > > >
> > >>> > > >
> > >>> > >
> > >>> >
> > >>> >
> > >>> >
> > >>> > --
> > >>> > Jeff Holoman
> > >>> > Systems Engineer
> > >>> >
> > >>>
> > >>
> > >
> > >
> > > --
> > > Grant Henke
> > > Software Engineer | Cloudera
> > > grant@cloudera.com | twitter.com/gchenke | linkedin.com/in/granthenke
> > >
> >
> >
> >
> > --
> > Grant Henke
> > Software Engineer | Cloudera
> > grant@cloudera.com | twitter.com/gchenke | linkedin.com/in/granthenke
> >
>
Re: [DISCUSSION] Kafka 0.8.2.2 release?
Posted by Stevo Slavić <ss...@gmail.com>.
Instead of cherrypicking, why not just make 0.8.2.2 of off current trunk,
with new consumer API appropriately annotated/documented as unstable?
On Mon, Aug 17, 2015, 17:17 Grant Henke <gh...@cloudera.com> wrote:
> +dev
>
> Adding dev list back in. Somehow it got dropped.
>
>
> On Mon, Aug 17, 2015 at 10:16 AM, Grant Henke <gh...@cloudera.com> wrote:
>
> > Below is a list of candidate bug fix jiras marked fixed for 0.8.3. I
> don't
> > suspect all of these will (or should) make it into the release but this
> > should be a relatively complete list to work from:
> >
> > - KAFKA-2114 <https://issues.apache.org/jira/browse/KAFKA-2114>:
> Unable
> > to change min.insync.replicas default
> > - KAFKA-1702 <https://issues.apache.org/jira/browse/KAFKA-1702>:
> > Messages silently Lost by producer
> > - KAFKA-2012 <https://issues.apache.org/jira/browse/KAFKA-2012>:
> > Broker should automatically handle corrupt index files
> > - KAFKA-2406 <https://issues.apache.org/jira/browse/KAFKA-2406>: ISR
> > propagation should be throttled to avoid overwhelming controller.
> > - KAFKA-2336 <https://issues.apache.org/jira/browse/KAFKA-2336>:
> > Changing offsets.topic.num.partitions after the offset topic is
> created
> > breaks consumer group partition assignment
> > - KAFKA-2337 <https://issues.apache.org/jira/browse/KAFKA-2337>:
> Verify
> > that metric names will not collide when creating new topics
> > - KAFKA-2393 <https://issues.apache.org/jira/browse/KAFKA-2393>:
> > Correctly Handle InvalidTopicException in KafkaApis.getTopicMetadata()
> > - KAFKA-2189 <https://issues.apache.org/jira/browse/KAFKA-2189>:
> Snappy
> > compression of message batches less efficient in 0.8.2.1
> > - KAFKA-2308 <https://issues.apache.org/jira/browse/KAFKA-2308>: New
> > producer + Snappy face un-compression errors after broker restart
> > - KAFKA-2042 <https://issues.apache.org/jira/browse/KAFKA-2042>: New
> > producer metadata update always get all topics.
> > - KAFKA-1367 <https://issues.apache.org/jira/browse/KAFKA-1367>:
> Broker
> > topic metadata not kept in sync with ZooKeeper
> > - KAFKA-972 <https://issues.apache.org/jira/browse/KAFKA-972>:
> MetadataRequest
> > returns stale list of brokers
> > - KAFKA-1867 <https://issues.apache.org/jira/browse/KAFKA-1867>:
> liveBroker
> > list not updated on a cluster with no topics
> > - KAFKA-1650 <https://issues.apache.org/jira/browse/KAFKA-1650>:
> Mirror
> > Maker could lose data on unclean shutdown.
> > - KAFKA-2009 <https://issues.apache.org/jira/browse/KAFKA-2009>: Fix
> > UncheckedOffset.removeOffset synchronization and trace logging issue
> in
> > mirror maker
> > - KAFKA-2407 <https://issues.apache.org/jira/browse/KAFKA-2407>: Only
> > create a log directory when it will be used
> > - KAFKA-2327 <https://issues.apache.org/jira/browse/KAFKA-2327>:
> > broker doesn't start if config defines advertised.host but not
> > advertised.port
> > - KAFKA-1788: producer record can stay in RecordAccumulator forever if
> > leader is no available
> > - KAFKA-2234 <https://issues.apache.org/jira/browse/KAFKA-2234>:
> > Partition reassignment of a nonexistent topic prevents future
> reassignments
> > - KAFKA-2096 <https://issues.apache.org/jira/browse/KAFKA-2096>:
> > Enable keepalive socket option for broker to prevent socket leak
> > - KAFKA-1057 <https://issues.apache.org/jira/browse/KAFKA-1057>: Trim
> > whitespaces from user specified configs
> > - KAFKA-1641 <https://issues.apache.org/jira/browse/KAFKA-1641>: Log
> > cleaner exits if last cleaned offset is lower than earliest offset
> > - KAFKA-1648 <https://issues.apache.org/jira/browse/KAFKA-1648>:
> Round
> > robin consumer balance throws an NPE when there are no topics
> > - KAFKA-1724 <https://issues.apache.org/jira/browse/KAFKA-1724>:
> > Errors after reboot in single node setup
> > - KAFKA-1758 <https://issues.apache.org/jira/browse/KAFKA-1758>:
> > corrupt recovery file prevents startup
> > - KAFKA-1866 <https://issues.apache.org/jira/browse/KAFKA-1866>:
> > LogStartOffset gauge throws exceptions after log.delete()
> > - KAFKA-1883 <https://issues.apache.org/jira/browse/KAFKA-1883>:
> NullPointerException
> > in RequestSendThread
> > - KAFKA-1896 <https://issues.apache.org/jira/browse/KAFKA-1896>:
> > Record size funcition of record in mirror maker hit NPE when the
> message
> > value is null.
> > - KAFKA-2101 <https://issues.apache.org/jira/browse/KAFKA-2101>:
> > Metric metadata-age is reset on a failed update
> > - KAFKA-2112 <https://issues.apache.org/jira/browse/KAFKA-2112>: make
> > overflowWheel volatile
> > - KAFKA-2117 <https://issues.apache.org/jira/browse/KAFKA-2117>:
> > OffsetManager uses incorrect field for metadata
> > - KAFKA-2164 <https://issues.apache.org/jira/browse/KAFKA-2164>:
> > ReplicaFetcherThread: suspicious log message on reset offset
> > - KAFKA-1668 <https://issues.apache.org/jira/browse/KAFKA-1668>:
> > TopicCommand doesn't warn if --topic argument doesn't match any topics
> > - KAFKA-2198 <https://issues.apache.org/jira/browse/KAFKA-2198>:
> > kafka-topics.sh exits with 0 status on failures
> > - KAFKA-2235 <https://issues.apache.org/jira/browse/KAFKA-2235>:
> > LogCleaner offset map overflow
> > - KAFKA-2241 <https://issues.apache.org/jira/browse/KAFKA-2241>:
> > AbstractFetcherThread.shutdown() should not block on
> > ReadableByteChannel.read(buffer)
> > - KAFKA-2272 <https://issues.apache.org/jira/browse/KAFKA-2272>:
> > listeners endpoint parsing fails if the hostname has capital letter
> > - KAFKA-2345 <https://issues.apache.org/jira/browse/KAFKA-2345>:
> > Attempt to delete a topic already marked for deletion throws
> > ZkNodeExistsException
> > - KAFKA-2353 <https://issues.apache.org/jira/browse/KAFKA-2353>:
> > SocketServer.Processor should catch exception and close the socket
> properly
> > in configureNewConnections.
> > - KAFKA-1836 <https://issues.apache.org/jira/browse/KAFKA-1836>:
> > metadata.fetch.timeout.ms set to zero blocks forever
> > - KAFKA-2317 <https://issues.apache.org/jira/browse/KAFKA-2317>:
> De-register
> > isrChangeNotificationListener on controller resignation
> >
> > Note: KAFKA-2120 <https://issues.apache.org/jira/browse/KAFKA-2120> &
> > KAFKA-2421 <https://issues.apache.org/jira/browse/KAFKA-2421> were
> > mentioned in previous emails, but are not in the list because they are
> not
> > committed yet.
> >
> > Hope that helps the effort.
> >
> > Thanks,
> > Grant
> >
> > On Mon, Aug 17, 2015 at 12:09 AM, Grant Henke <gh...@cloudera.com>
> wrote:
> >
> >> +1 to that suggestion. Though I suspect that requires a committer to do.
> >> Making it part of the standard commit process could work too.
> >> On Aug 16, 2015 11:01 PM, "Gwen Shapira" <gw...@confluent.io> wrote:
> >>
> >>> BTW. I think it will be great for Apache Kafka to have a 0.8.2 "release
> >>> manager" who's role is to cherrypick low-risk bug-fixes into the 0.8.2
> >>> branch and once enough bug fixes happened (or if sufficiently critical
> >>> fixes happened) to roll out a new maintenance release (with every 3
> month
> >>> as a reasonable bugfix release target).
> >>>
> >>> This will add some predictability regarding how fast we release fixes
> for
> >>> bugs.
> >>>
> >>> Gwen
> >>>
> >>> On Sun, Aug 16, 2015 at 8:09 PM, Jeff Holoman <jh...@cloudera.com>
> >>> wrote:
> >>>
> >>> > +1 for the release and also including
> >>> >
> >>> > https://issues.apache.org/jira/browse/KAFKA-2114
> >>> >
> >>> > Thanks
> >>> >
> >>> > Jeff
> >>> >
> >>> > On Sun, Aug 16, 2015 at 2:51 PM, Stevo Slavić <ss...@gmail.com>
> >>> wrote:
> >>> >
> >>> > > +1 (non-binding) for 0.8.2.2 release
> >>> > >
> >>> > > Would be nice to include in that release new producer resiliency
> bug
> >>> > fixes
> >>> > > https://issues.apache.org/jira/browse/KAFKA-1788 and
> >>> > > https://issues.apache.org/jira/browse/KAFKA-2120
> >>> > >
> >>> > > On Fri, Aug 14, 2015 at 4:03 PM, Gwen Shapira <gw...@confluent.io>
> >>> wrote:
> >>> > >
> >>> > > > Will be nice to include Kafka-2308 and fix two critical snappy
> >>> issues
> >>> > in
> >>> > > > the maintenance release.
> >>> > > >
> >>> > > > Gwen
> >>> > > > On Aug 14, 2015 6:16 AM, "Grant Henke" <gh...@cloudera.com>
> >>> wrote:
> >>> > > >
> >>> > > > > Just to clarify. Will KAFKA-2189 be the only patch in the
> >>> release?
> >>> > > > >
> >>> > > > > On Fri, Aug 14, 2015 at 7:35 AM, Manikumar Reddy <
> >>> > kumar@nmsworks.co.in
> >>> > > >
> >>> > > > > wrote:
> >>> > > > >
> >>> > > > > > +1 for 0.8.2.2 release
> >>> > > > > >
> >>> > > > > > On Fri, Aug 14, 2015 at 5:49 PM, Ismael Juma <
> >>> ismael@juma.me.uk>
> >>> > > > wrote:
> >>> > > > > >
> >>> > > > > > > I think this is a good idea as the change is minimal on our
> >>> side
> >>> > > and
> >>> > > > it
> >>> > > > > > has
> >>> > > > > > > been tested in production for some time by the reporter.
> >>> > > > > > >
> >>> > > > > > > Best,
> >>> > > > > > > Ismael
> >>> > > > > > >
> >>> > > > > > > On Fri, Aug 14, 2015 at 1:15 PM, Jun Rao <jun@confluent.io
> >
> >>> > wrote:
> >>> > > > > > >
> >>> > > > > > > > Hi, Everyone,
> >>> > > > > > > >
> >>> > > > > > > > Since the release of Kafka 0.8.2.1, a number of people
> have
> >>> > > > reported
> >>> > > > > an
> >>> > > > > > > > issue with snappy compression (
> >>> > > > > > > > https://issues.apache.org/jira/browse/KAFKA-2189).
> >>> Basically,
> >>> > if
> >>> > > > > they
> >>> > > > > > > use
> >>> > > > > > > > snappy in 0.8.2.1, they will experience a 2-3X space
> >>> increase.
> >>> > > The
> >>> > > > > > issue
> >>> > > > > > > > has since been fixed in trunk (just a snappy jar
> upgrade).
> >>> > Since
> >>> > > > > 0.8.3
> >>> > > > > > is
> >>> > > > > > > > still a few months away, it may make sense to do an
> 0.8.2.2
> >>> > > release
> >>> > > > > > just
> >>> > > > > > > to
> >>> > > > > > > > fix this issue. Any objections?
> >>> > > > > > > >
> >>> > > > > > > > Thanks,
> >>> > > > > > > >
> >>> > > > > > > > Jun
> >>> > > > > > > >
> >>> > > > > > >
> >>> > > > > >
> >>> > > > >
> >>> > > > >
> >>> > > > >
> >>> > > > > --
> >>> > > > > Grant Henke
> >>> > > > > Software Engineer | Cloudera
> >>> > > > > grant@cloudera.com | twitter.com/gchenke |
> >>> > linkedin.com/in/granthenke
> >>> > > > >
> >>> > > >
> >>> > >
> >>> >
> >>> >
> >>> >
> >>> > --
> >>> > Jeff Holoman
> >>> > Systems Engineer
> >>> >
> >>>
> >>
> >
> >
> > --
> > Grant Henke
> > Software Engineer | Cloudera
> > grant@cloudera.com | twitter.com/gchenke | linkedin.com/in/granthenke
> >
>
>
>
> --
> Grant Henke
> Software Engineer | Cloudera
> grant@cloudera.com | twitter.com/gchenke | linkedin.com/in/granthenke
>
Re: [DISCUSSION] Kafka 0.8.2.2 release?
Posted by Stevo Slavić <ss...@gmail.com>.
Instead of cherrypicking, why not just make 0.8.2.2 of off current trunk,
with new consumer API appropriately annotated/documented as unstable?
On Mon, Aug 17, 2015, 17:17 Grant Henke <gh...@cloudera.com> wrote:
> +dev
>
> Adding dev list back in. Somehow it got dropped.
>
>
> On Mon, Aug 17, 2015 at 10:16 AM, Grant Henke <gh...@cloudera.com> wrote:
>
> > Below is a list of candidate bug fix jiras marked fixed for 0.8.3. I
> don't
> > suspect all of these will (or should) make it into the release but this
> > should be a relatively complete list to work from:
> >
> > - KAFKA-2114 <https://issues.apache.org/jira/browse/KAFKA-2114>:
> Unable
> > to change min.insync.replicas default
> > - KAFKA-1702 <https://issues.apache.org/jira/browse/KAFKA-1702>:
> > Messages silently Lost by producer
> > - KAFKA-2012 <https://issues.apache.org/jira/browse/KAFKA-2012>:
> > Broker should automatically handle corrupt index files
> > - KAFKA-2406 <https://issues.apache.org/jira/browse/KAFKA-2406>: ISR
> > propagation should be throttled to avoid overwhelming controller.
> > - KAFKA-2336 <https://issues.apache.org/jira/browse/KAFKA-2336>:
> > Changing offsets.topic.num.partitions after the offset topic is
> created
> > breaks consumer group partition assignment
> > - KAFKA-2337 <https://issues.apache.org/jira/browse/KAFKA-2337>:
> Verify
> > that metric names will not collide when creating new topics
> > - KAFKA-2393 <https://issues.apache.org/jira/browse/KAFKA-2393>:
> > Correctly Handle InvalidTopicException in KafkaApis.getTopicMetadata()
> > - KAFKA-2189 <https://issues.apache.org/jira/browse/KAFKA-2189>:
> Snappy
> > compression of message batches less efficient in 0.8.2.1
> > - KAFKA-2308 <https://issues.apache.org/jira/browse/KAFKA-2308>: New
> > producer + Snappy face un-compression errors after broker restart
> > - KAFKA-2042 <https://issues.apache.org/jira/browse/KAFKA-2042>: New
> > producer metadata update always get all topics.
> > - KAFKA-1367 <https://issues.apache.org/jira/browse/KAFKA-1367>:
> Broker
> > topic metadata not kept in sync with ZooKeeper
> > - KAFKA-972 <https://issues.apache.org/jira/browse/KAFKA-972>:
> MetadataRequest
> > returns stale list of brokers
> > - KAFKA-1867 <https://issues.apache.org/jira/browse/KAFKA-1867>:
> liveBroker
> > list not updated on a cluster with no topics
> > - KAFKA-1650 <https://issues.apache.org/jira/browse/KAFKA-1650>:
> Mirror
> > Maker could lose data on unclean shutdown.
> > - KAFKA-2009 <https://issues.apache.org/jira/browse/KAFKA-2009>: Fix
> > UncheckedOffset.removeOffset synchronization and trace logging issue
> in
> > mirror maker
> > - KAFKA-2407 <https://issues.apache.org/jira/browse/KAFKA-2407>: Only
> > create a log directory when it will be used
> > - KAFKA-2327 <https://issues.apache.org/jira/browse/KAFKA-2327>:
> > broker doesn't start if config defines advertised.host but not
> > advertised.port
> > - KAFKA-1788: producer record can stay in RecordAccumulator forever if
> > leader is no available
> > - KAFKA-2234 <https://issues.apache.org/jira/browse/KAFKA-2234>:
> > Partition reassignment of a nonexistent topic prevents future
> reassignments
> > - KAFKA-2096 <https://issues.apache.org/jira/browse/KAFKA-2096>:
> > Enable keepalive socket option for broker to prevent socket leak
> > - KAFKA-1057 <https://issues.apache.org/jira/browse/KAFKA-1057>: Trim
> > whitespaces from user specified configs
> > - KAFKA-1641 <https://issues.apache.org/jira/browse/KAFKA-1641>: Log
> > cleaner exits if last cleaned offset is lower than earliest offset
> > - KAFKA-1648 <https://issues.apache.org/jira/browse/KAFKA-1648>:
> Round
> > robin consumer balance throws an NPE when there are no topics
> > - KAFKA-1724 <https://issues.apache.org/jira/browse/KAFKA-1724>:
> > Errors after reboot in single node setup
> > - KAFKA-1758 <https://issues.apache.org/jira/browse/KAFKA-1758>:
> > corrupt recovery file prevents startup
> > - KAFKA-1866 <https://issues.apache.org/jira/browse/KAFKA-1866>:
> > LogStartOffset gauge throws exceptions after log.delete()
> > - KAFKA-1883 <https://issues.apache.org/jira/browse/KAFKA-1883>:
> NullPointerException
> > in RequestSendThread
> > - KAFKA-1896 <https://issues.apache.org/jira/browse/KAFKA-1896>:
> > Record size funcition of record in mirror maker hit NPE when the
> message
> > value is null.
> > - KAFKA-2101 <https://issues.apache.org/jira/browse/KAFKA-2101>:
> > Metric metadata-age is reset on a failed update
> > - KAFKA-2112 <https://issues.apache.org/jira/browse/KAFKA-2112>: make
> > overflowWheel volatile
> > - KAFKA-2117 <https://issues.apache.org/jira/browse/KAFKA-2117>:
> > OffsetManager uses incorrect field for metadata
> > - KAFKA-2164 <https://issues.apache.org/jira/browse/KAFKA-2164>:
> > ReplicaFetcherThread: suspicious log message on reset offset
> > - KAFKA-1668 <https://issues.apache.org/jira/browse/KAFKA-1668>:
> > TopicCommand doesn't warn if --topic argument doesn't match any topics
> > - KAFKA-2198 <https://issues.apache.org/jira/browse/KAFKA-2198>:
> > kafka-topics.sh exits with 0 status on failures
> > - KAFKA-2235 <https://issues.apache.org/jira/browse/KAFKA-2235>:
> > LogCleaner offset map overflow
> > - KAFKA-2241 <https://issues.apache.org/jira/browse/KAFKA-2241>:
> > AbstractFetcherThread.shutdown() should not block on
> > ReadableByteChannel.read(buffer)
> > - KAFKA-2272 <https://issues.apache.org/jira/browse/KAFKA-2272>:
> > listeners endpoint parsing fails if the hostname has capital letter
> > - KAFKA-2345 <https://issues.apache.org/jira/browse/KAFKA-2345>:
> > Attempt to delete a topic already marked for deletion throws
> > ZkNodeExistsException
> > - KAFKA-2353 <https://issues.apache.org/jira/browse/KAFKA-2353>:
> > SocketServer.Processor should catch exception and close the socket
> properly
> > in configureNewConnections.
> > - KAFKA-1836 <https://issues.apache.org/jira/browse/KAFKA-1836>:
> > metadata.fetch.timeout.ms set to zero blocks forever
> > - KAFKA-2317 <https://issues.apache.org/jira/browse/KAFKA-2317>:
> De-register
> > isrChangeNotificationListener on controller resignation
> >
> > Note: KAFKA-2120 <https://issues.apache.org/jira/browse/KAFKA-2120> &
> > KAFKA-2421 <https://issues.apache.org/jira/browse/KAFKA-2421> were
> > mentioned in previous emails, but are not in the list because they are
> not
> > committed yet.
> >
> > Hope that helps the effort.
> >
> > Thanks,
> > Grant
> >
> > On Mon, Aug 17, 2015 at 12:09 AM, Grant Henke <gh...@cloudera.com>
> wrote:
> >
> >> +1 to that suggestion. Though I suspect that requires a committer to do.
> >> Making it part of the standard commit process could work too.
> >> On Aug 16, 2015 11:01 PM, "Gwen Shapira" <gw...@confluent.io> wrote:
> >>
> >>> BTW. I think it will be great for Apache Kafka to have a 0.8.2 "release
> >>> manager" who's role is to cherrypick low-risk bug-fixes into the 0.8.2
> >>> branch and once enough bug fixes happened (or if sufficiently critical
> >>> fixes happened) to roll out a new maintenance release (with every 3
> month
> >>> as a reasonable bugfix release target).
> >>>
> >>> This will add some predictability regarding how fast we release fixes
> for
> >>> bugs.
> >>>
> >>> Gwen
> >>>
> >>> On Sun, Aug 16, 2015 at 8:09 PM, Jeff Holoman <jh...@cloudera.com>
> >>> wrote:
> >>>
> >>> > +1 for the release and also including
> >>> >
> >>> > https://issues.apache.org/jira/browse/KAFKA-2114
> >>> >
> >>> > Thanks
> >>> >
> >>> > Jeff
> >>> >
> >>> > On Sun, Aug 16, 2015 at 2:51 PM, Stevo Slavić <ss...@gmail.com>
> >>> wrote:
> >>> >
> >>> > > +1 (non-binding) for 0.8.2.2 release
> >>> > >
> >>> > > Would be nice to include in that release new producer resiliency
> bug
> >>> > fixes
> >>> > > https://issues.apache.org/jira/browse/KAFKA-1788 and
> >>> > > https://issues.apache.org/jira/browse/KAFKA-2120
> >>> > >
> >>> > > On Fri, Aug 14, 2015 at 4:03 PM, Gwen Shapira <gw...@confluent.io>
> >>> wrote:
> >>> > >
> >>> > > > Will be nice to include Kafka-2308 and fix two critical snappy
> >>> issues
> >>> > in
> >>> > > > the maintenance release.
> >>> > > >
> >>> > > > Gwen
> >>> > > > On Aug 14, 2015 6:16 AM, "Grant Henke" <gh...@cloudera.com>
> >>> wrote:
> >>> > > >
> >>> > > > > Just to clarify. Will KAFKA-2189 be the only patch in the
> >>> release?
> >>> > > > >
> >>> > > > > On Fri, Aug 14, 2015 at 7:35 AM, Manikumar Reddy <
> >>> > kumar@nmsworks.co.in
> >>> > > >
> >>> > > > > wrote:
> >>> > > > >
> >>> > > > > > +1 for 0.8.2.2 release
> >>> > > > > >
> >>> > > > > > On Fri, Aug 14, 2015 at 5:49 PM, Ismael Juma <
> >>> ismael@juma.me.uk>
> >>> > > > wrote:
> >>> > > > > >
> >>> > > > > > > I think this is a good idea as the change is minimal on our
> >>> side
> >>> > > and
> >>> > > > it
> >>> > > > > > has
> >>> > > > > > > been tested in production for some time by the reporter.
> >>> > > > > > >
> >>> > > > > > > Best,
> >>> > > > > > > Ismael
> >>> > > > > > >
> >>> > > > > > > On Fri, Aug 14, 2015 at 1:15 PM, Jun Rao <jun@confluent.io
> >
> >>> > wrote:
> >>> > > > > > >
> >>> > > > > > > > Hi, Everyone,
> >>> > > > > > > >
> >>> > > > > > > > Since the release of Kafka 0.8.2.1, a number of people
> have
> >>> > > > reported
> >>> > > > > an
> >>> > > > > > > > issue with snappy compression (
> >>> > > > > > > > https://issues.apache.org/jira/browse/KAFKA-2189).
> >>> Basically,
> >>> > if
> >>> > > > > they
> >>> > > > > > > use
> >>> > > > > > > > snappy in 0.8.2.1, they will experience a 2-3X space
> >>> increase.
> >>> > > The
> >>> > > > > > issue
> >>> > > > > > > > has since been fixed in trunk (just a snappy jar
> upgrade).
> >>> > Since
> >>> > > > > 0.8.3
> >>> > > > > > is
> >>> > > > > > > > still a few months away, it may make sense to do an
> 0.8.2.2
> >>> > > release
> >>> > > > > > just
> >>> > > > > > > to
> >>> > > > > > > > fix this issue. Any objections?
> >>> > > > > > > >
> >>> > > > > > > > Thanks,
> >>> > > > > > > >
> >>> > > > > > > > Jun
> >>> > > > > > > >
> >>> > > > > > >
> >>> > > > > >
> >>> > > > >
> >>> > > > >
> >>> > > > >
> >>> > > > > --
> >>> > > > > Grant Henke
> >>> > > > > Software Engineer | Cloudera
> >>> > > > > grant@cloudera.com | twitter.com/gchenke |
> >>> > linkedin.com/in/granthenke
> >>> > > > >
> >>> > > >
> >>> > >
> >>> >
> >>> >
> >>> >
> >>> > --
> >>> > Jeff Holoman
> >>> > Systems Engineer
> >>> >
> >>>
> >>
> >
> >
> > --
> > Grant Henke
> > Software Engineer | Cloudera
> > grant@cloudera.com | twitter.com/gchenke | linkedin.com/in/granthenke
> >
>
>
>
> --
> Grant Henke
> Software Engineer | Cloudera
> grant@cloudera.com | twitter.com/gchenke | linkedin.com/in/granthenke
>
Re: [DISCUSSION] Kafka 0.8.2.2 release?
Posted by Gwen Shapira <gw...@confluent.io>.
I completely agree that community focus on 0.8.3 is important and don't
want to take away from that effort.
That said, not everyone will upgrade to 0.8.3 immediately (unfortunately),
and there is still value in releasing bugfixes on previous releases.
I'm keeping the scope very small, to make the new version safe and make the
release easier and less distracting for us. If this interferes with 0.8.3,
we will adjust course to make sure it doesn't.
Gwen
On Tue, Aug 18, 2015 at 10:54 AM, Neha Narkhede <ne...@confluent.io> wrote:
> How about looking at the scope for the 0.8.3 release first before we cut
> yet another point release off of 0.8.2.2? Each release includes some
> overhead and if there is a larger release in the works, it might be worth
> working on getting that. My take is that the 2 things the community has
> been waiting for is SSL support and the new consumer and we have been
> promising to get 0.8.3 with both those features for several months now.
>
> Looking at the progress on both, it seems we are very close to getting both
> those checked in and it looks like we should get there in another 5-6
> weeks. Furthermore, both of these features are large and I anticipate us
> receiving feedback and bugs that will require a couple of point releases on
> top of 0.8.3 anyway. One possibility is to work on 0.8.3 together now and
> get the community to use the newly released features, gather feedback and
> do point releases incorporating that feedback and iterate on it.
>
> We could absolutely do both 0.8.2.2 and 0.8.3. What I'd ask for is for us
> to look at the 0.8.3 timeline too and make a call whether 0.8.2.2 still
> makes sense.
>
> Thanks,
> Neha
>
> On Tue, Aug 18, 2015 at 10:24 AM, Gwen Shapira <gw...@confluent.io> wrote:
>
> > Thanks Jun.
> >
> > I updated the list with your suggestions.
> > If anyone feels we are missing a critical patch for 0.8.2.2, please speak
> > up.
> >
> > Gwen
> >
> > On Mon, Aug 17, 2015 at 5:40 PM, Jun Rao <ju...@confluent.io> wrote:
> >
> > > Hi, Grant,
> > >
> > > I took a look at that list. None of those is really critical as you
> said.
> > > So, I'd suggest that we not include those to minimize the scope of the
> > > release.
> > >
> > > Thanks,
> > >
> > > Jun
> > >
> > > On Mon, Aug 17, 2015 at 5:16 PM, Grant Henke <gh...@cloudera.com>
> > wrote:
> > >
> > > > Thanks Gwen.
> > > >
> > > > I updated a few small things on the wiki page.
> > > >
> > > > Below is a list of jiras I think could also be marked as included.
> All
> > of
> > > > these, though not super critical, seem like fairly small and low risk
> > > > changes that help avoid potentially confusing issues or errors for
> > users.
> > > >
> > > > KAFKA-2012
> > > > KAFKA-972
> > > > KAFKA-2337 & KAFKA-2393
> > > > KAFKA-1867
> > > > KAFKA-2407
> > > > KAFKA-2234
> > > > KAFKA-1866
> > > > KAFKA-2345 & KAFKA-2355
> > > >
> > > > thoughts?
> > > >
> > > > Thank you,
> > > > Grant
> > > >
> > > > On Mon, Aug 17, 2015 at 4:56 PM, Gwen Shapira <gw...@confluent.io>
> > wrote:
> > > >
> > > > > Thanks for creating a list, Grant!
> > > > >
> > > > > I placed it on the wiki with a quick evaluation of the content and
> > > > whether
> > > > > it should be in 0.8.2.2:
> > > > >
> > > > >
> > > >
> > >
> >
> https://cwiki.apache.org/confluence/display/KAFKA/Proposed+patches+for+0.8.2.2
> > > > >
> > > > > I'm attempting to only cherrypick fixes that are both important for
> > > large
> > > > > number of users (or very critical to some users) and very safe
> > (mostly
> > > > > judged by the size of the change, but not only)
> > > > >
> > > > > If your favorite bugfix is missing from the list, or is there but
> > > marked
> > > > > "No", please let us know (in this thread) what we are missing and
> why
> > > it
> > > > is
> > > > > both important and safe.
> > > > > Also, if I accidentally included something you consider unsafe,
> speak
> > > up!
> > > > >
> > > > > Gwen
> > > > >
> > > > > On Mon, Aug 17, 2015 at 8:17 AM, Grant Henke <gh...@cloudera.com>
> > > > wrote:
> > > > >
> > > > > > +dev
> > > > > >
> > > > > > Adding dev list back in. Somehow it got dropped.
> > > > > >
> > > > > >
> > > > > > On Mon, Aug 17, 2015 at 10:16 AM, Grant Henke <
> ghenke@cloudera.com
> > >
> > > > > wrote:
> > > > > >
> > > > > > > Below is a list of candidate bug fix jiras marked fixed for
> > 0.8.3.
> > > I
> > > > > > don't
> > > > > > > suspect all of these will (or should) make it into the release
> > but
> > > > this
> > > > > > > should be a relatively complete list to work from:
> > > > > > >
> > > > > > > - KAFKA-2114 <
> > https://issues.apache.org/jira/browse/KAFKA-2114
> > > >:
> > > > > > Unable
> > > > > > > to change min.insync.replicas default
> > > > > > > - KAFKA-1702 <
> > https://issues.apache.org/jira/browse/KAFKA-1702
> > > >:
> > > > > > > Messages silently Lost by producer
> > > > > > > - KAFKA-2012 <
> > https://issues.apache.org/jira/browse/KAFKA-2012
> > > >:
> > > > > > > Broker should automatically handle corrupt index files
> > > > > > > - KAFKA-2406 <
> > https://issues.apache.org/jira/browse/KAFKA-2406
> > > >:
> > > > > ISR
> > > > > > > propagation should be throttled to avoid overwhelming
> > > controller.
> > > > > > > - KAFKA-2336 <
> > https://issues.apache.org/jira/browse/KAFKA-2336
> > > >:
> > > > > > > Changing offsets.topic.num.partitions after the offset topic
> > is
> > > > > > created
> > > > > > > breaks consumer group partition assignment
> > > > > > > - KAFKA-2337 <
> > https://issues.apache.org/jira/browse/KAFKA-2337
> > > >:
> > > > > > Verify
> > > > > > > that metric names will not collide when creating new topics
> > > > > > > - KAFKA-2393 <
> > https://issues.apache.org/jira/browse/KAFKA-2393
> > > >:
> > > > > > > Correctly Handle InvalidTopicException in
> > > > > KafkaApis.getTopicMetadata()
> > > > > > > - KAFKA-2189 <
> > https://issues.apache.org/jira/browse/KAFKA-2189
> > > >:
> > > > > > Snappy
> > > > > > > compression of message batches less efficient in 0.8.2.1
> > > > > > > - KAFKA-2308 <
> > https://issues.apache.org/jira/browse/KAFKA-2308
> > > >:
> > > > > New
> > > > > > > producer + Snappy face un-compression errors after broker
> > > restart
> > > > > > > - KAFKA-2042 <
> > https://issues.apache.org/jira/browse/KAFKA-2042
> > > >:
> > > > > New
> > > > > > > producer metadata update always get all topics.
> > > > > > > - KAFKA-1367 <
> > https://issues.apache.org/jira/browse/KAFKA-1367
> > > >:
> > > > > > Broker
> > > > > > > topic metadata not kept in sync with ZooKeeper
> > > > > > > - KAFKA-972 <
> https://issues.apache.org/jira/browse/KAFKA-972
> > >:
> > > > > > MetadataRequest
> > > > > > > returns stale list of brokers
> > > > > > > - KAFKA-1867 <
> > https://issues.apache.org/jira/browse/KAFKA-1867
> > > >:
> > > > > > liveBroker
> > > > > > > list not updated on a cluster with no topics
> > > > > > > - KAFKA-1650 <
> > https://issues.apache.org/jira/browse/KAFKA-1650
> > > >:
> > > > > > Mirror
> > > > > > > Maker could lose data on unclean shutdown.
> > > > > > > - KAFKA-2009 <
> > https://issues.apache.org/jira/browse/KAFKA-2009
> > > >:
> > > > > Fix
> > > > > > > UncheckedOffset.removeOffset synchronization and trace
> logging
> > > > issue
> > > > > > in
> > > > > > > mirror maker
> > > > > > > - KAFKA-2407 <
> > https://issues.apache.org/jira/browse/KAFKA-2407
> > > >:
> > > > > Only
> > > > > > > create a log directory when it will be used
> > > > > > > - KAFKA-2327 <
> > https://issues.apache.org/jira/browse/KAFKA-2327
> > > >:
> > > > > > > broker doesn't start if config defines advertised.host but
> not
> > > > > > > advertised.port
> > > > > > > - KAFKA-1788: producer record can stay in RecordAccumulator
> > > > forever
> > > > > if
> > > > > > > leader is no available
> > > > > > > - KAFKA-2234 <
> > https://issues.apache.org/jira/browse/KAFKA-2234
> > > >:
> > > > > > > Partition reassignment of a nonexistent topic prevents
> future
> > > > > > reassignments
> > > > > > > - KAFKA-2096 <
> > https://issues.apache.org/jira/browse/KAFKA-2096
> > > >:
> > > > > > > Enable keepalive socket option for broker to prevent socket
> > leak
> > > > > > > - KAFKA-1057 <
> > https://issues.apache.org/jira/browse/KAFKA-1057
> > > >:
> > > > > Trim
> > > > > > > whitespaces from user specified configs
> > > > > > > - KAFKA-1641 <
> > https://issues.apache.org/jira/browse/KAFKA-1641
> > > >:
> > > > > Log
> > > > > > > cleaner exits if last cleaned offset is lower than earliest
> > > offset
> > > > > > > - KAFKA-1648 <
> > https://issues.apache.org/jira/browse/KAFKA-1648
> > > >:
> > > > > > Round
> > > > > > > robin consumer balance throws an NPE when there are no
> topics
> > > > > > > - KAFKA-1724 <
> > https://issues.apache.org/jira/browse/KAFKA-1724
> > > >:
> > > > > > > Errors after reboot in single node setup
> > > > > > > - KAFKA-1758 <
> > https://issues.apache.org/jira/browse/KAFKA-1758
> > > >:
> > > > > > > corrupt recovery file prevents startup
> > > > > > > - KAFKA-1866 <
> > https://issues.apache.org/jira/browse/KAFKA-1866
> > > >:
> > > > > > > LogStartOffset gauge throws exceptions after log.delete()
> > > > > > > - KAFKA-1883 <
> > https://issues.apache.org/jira/browse/KAFKA-1883
> > > >:
> > > > > > NullPointerException
> > > > > > > in RequestSendThread
> > > > > > > - KAFKA-1896 <
> > https://issues.apache.org/jira/browse/KAFKA-1896
> > > >:
> > > > > > > Record size funcition of record in mirror maker hit NPE when
> > the
> > > > > > message
> > > > > > > value is null.
> > > > > > > - KAFKA-2101 <
> > https://issues.apache.org/jira/browse/KAFKA-2101
> > > >:
> > > > > > > Metric metadata-age is reset on a failed update
> > > > > > > - KAFKA-2112 <
> > https://issues.apache.org/jira/browse/KAFKA-2112
> > > >:
> > > > > make
> > > > > > > overflowWheel volatile
> > > > > > > - KAFKA-2117 <
> > https://issues.apache.org/jira/browse/KAFKA-2117
> > > >:
> > > > > > > OffsetManager uses incorrect field for metadata
> > > > > > > - KAFKA-2164 <
> > https://issues.apache.org/jira/browse/KAFKA-2164
> > > >:
> > > > > > > ReplicaFetcherThread: suspicious log message on reset offset
> > > > > > > - KAFKA-1668 <
> > https://issues.apache.org/jira/browse/KAFKA-1668
> > > >:
> > > > > > > TopicCommand doesn't warn if --topic argument doesn't match
> > any
> > > > > topics
> > > > > > > - KAFKA-2198 <
> > https://issues.apache.org/jira/browse/KAFKA-2198
> > > >:
> > > > > > > kafka-topics.sh exits with 0 status on failures
> > > > > > > - KAFKA-2235 <
> > https://issues.apache.org/jira/browse/KAFKA-2235
> > > >:
> > > > > > > LogCleaner offset map overflow
> > > > > > > - KAFKA-2241 <
> > https://issues.apache.org/jira/browse/KAFKA-2241
> > > >:
> > > > > > > AbstractFetcherThread.shutdown() should not block on
> > > > > > > ReadableByteChannel.read(buffer)
> > > > > > > - KAFKA-2272 <
> > https://issues.apache.org/jira/browse/KAFKA-2272
> > > >:
> > > > > > > listeners endpoint parsing fails if the hostname has capital
> > > > letter
> > > > > > > - KAFKA-2345 <
> > https://issues.apache.org/jira/browse/KAFKA-2345
> > > >:
> > > > > > > Attempt to delete a topic already marked for deletion throws
> > > > > > > ZkNodeExistsException
> > > > > > > - KAFKA-2353 <
> > https://issues.apache.org/jira/browse/KAFKA-2353
> > > >:
> > > > > > > SocketServer.Processor should catch exception and close the
> > > socket
> > > > > > properly
> > > > > > > in configureNewConnections.
> > > > > > > - KAFKA-1836 <
> > https://issues.apache.org/jira/browse/KAFKA-1836
> > > >:
> > > > > > > metadata.fetch.timeout.ms set to zero blocks forever
> > > > > > > - KAFKA-2317 <
> > https://issues.apache.org/jira/browse/KAFKA-2317
> > > >:
> > > > > > De-register
> > > > > > > isrChangeNotificationListener on controller resignation
> > > > > > >
> > > > > > > Note: KAFKA-2120 <
> > https://issues.apache.org/jira/browse/KAFKA-2120
> > > >
> > > > &
> > > > > > > KAFKA-2421 <https://issues.apache.org/jira/browse/KAFKA-2421>
> > were
> > > > > > > mentioned in previous emails, but are not in the list because
> > they
> > > > are
> > > > > > not
> > > > > > > committed yet.
> > > > > > >
> > > > > > > Hope that helps the effort.
> > > > > > >
> > > > > > > Thanks,
> > > > > > > Grant
> > > > > > >
> > > > > > > On Mon, Aug 17, 2015 at 12:09 AM, Grant Henke <
> > ghenke@cloudera.com
> > > >
> > > > > > wrote:
> > > > > > >
> > > > > > >> +1 to that suggestion. Though I suspect that requires a
> > committer
> > > to
> > > > > do.
> > > > > > >> Making it part of the standard commit process could work too.
> > > > > > >> On Aug 16, 2015 11:01 PM, "Gwen Shapira" <gw...@confluent.io>
> > > wrote:
> > > > > > >>
> > > > > > >>> BTW. I think it will be great for Apache Kafka to have a
> 0.8.2
> > > > > "release
> > > > > > >>> manager" who's role is to cherrypick low-risk bug-fixes into
> > the
> > > > > 0.8.2
> > > > > > >>> branch and once enough bug fixes happened (or if sufficiently
> > > > > critical
> > > > > > >>> fixes happened) to roll out a new maintenance release (with
> > > every 3
> > > > > > month
> > > > > > >>> as a reasonable bugfix release target).
> > > > > > >>>
> > > > > > >>> This will add some predictability regarding how fast we
> release
> > > > fixes
> > > > > > for
> > > > > > >>> bugs.
> > > > > > >>>
> > > > > > >>> Gwen
> > > > > > >>>
> > > > > > >>> On Sun, Aug 16, 2015 at 8:09 PM, Jeff Holoman <
> > > > jholoman@cloudera.com
> > > > > >
> > > > > > >>> wrote:
> > > > > > >>>
> > > > > > >>> > +1 for the release and also including
> > > > > > >>> >
> > > > > > >>> > https://issues.apache.org/jira/browse/KAFKA-2114
> > > > > > >>> >
> > > > > > >>> > Thanks
> > > > > > >>> >
> > > > > > >>> > Jeff
> > > > > > >>> >
> > > > > > >>> > On Sun, Aug 16, 2015 at 2:51 PM, Stevo Slavić <
> > > sslavic@gmail.com
> > > > >
> > > > > > >>> wrote:
> > > > > > >>> >
> > > > > > >>> > > +1 (non-binding) for 0.8.2.2 release
> > > > > > >>> > >
> > > > > > >>> > > Would be nice to include in that release new producer
> > > > resiliency
> > > > > > bug
> > > > > > >>> > fixes
> > > > > > >>> > > https://issues.apache.org/jira/browse/KAFKA-1788 and
> > > > > > >>> > > https://issues.apache.org/jira/browse/KAFKA-2120
> > > > > > >>> > >
> > > > > > >>> > > On Fri, Aug 14, 2015 at 4:03 PM, Gwen Shapira <
> > > > gwen@confluent.io
> > > > > >
> > > > > > >>> wrote:
> > > > > > >>> > >
> > > > > > >>> > > > Will be nice to include Kafka-2308 and fix two critical
> > > > snappy
> > > > > > >>> issues
> > > > > > >>> > in
> > > > > > >>> > > > the maintenance release.
> > > > > > >>> > > >
> > > > > > >>> > > > Gwen
> > > > > > >>> > > > On Aug 14, 2015 6:16 AM, "Grant Henke" <
> > > ghenke@cloudera.com>
> > > > > > >>> wrote:
> > > > > > >>> > > >
> > > > > > >>> > > > > Just to clarify. Will KAFKA-2189 be the only patch in
> > the
> > > > > > >>> release?
> > > > > > >>> > > > >
> > > > > > >>> > > > > On Fri, Aug 14, 2015 at 7:35 AM, Manikumar Reddy <
> > > > > > >>> > kumar@nmsworks.co.in
> > > > > > >>> > > >
> > > > > > >>> > > > > wrote:
> > > > > > >>> > > > >
> > > > > > >>> > > > > > +1 for 0.8.2.2 release
> > > > > > >>> > > > > >
> > > > > > >>> > > > > > On Fri, Aug 14, 2015 at 5:49 PM, Ismael Juma <
> > > > > > >>> ismael@juma.me.uk>
> > > > > > >>> > > > wrote:
> > > > > > >>> > > > > >
> > > > > > >>> > > > > > > I think this is a good idea as the change is
> > minimal
> > > on
> > > > > our
> > > > > > >>> side
> > > > > > >>> > > and
> > > > > > >>> > > > it
> > > > > > >>> > > > > > has
> > > > > > >>> > > > > > > been tested in production for some time by the
> > > > reporter.
> > > > > > >>> > > > > > >
> > > > > > >>> > > > > > > Best,
> > > > > > >>> > > > > > > Ismael
> > > > > > >>> > > > > > >
> > > > > > >>> > > > > > > On Fri, Aug 14, 2015 at 1:15 PM, Jun Rao <
> > > > > jun@confluent.io
> > > > > > >
> > > > > > >>> > wrote:
> > > > > > >>> > > > > > >
> > > > > > >>> > > > > > > > Hi, Everyone,
> > > > > > >>> > > > > > > >
> > > > > > >>> > > > > > > > Since the release of Kafka 0.8.2.1, a number of
> > > > people
> > > > > > have
> > > > > > >>> > > > reported
> > > > > > >>> > > > > an
> > > > > > >>> > > > > > > > issue with snappy compression (
> > > > > > >>> > > > > > > >
> https://issues.apache.org/jira/browse/KAFKA-2189
> > ).
> > > > > > >>> Basically,
> > > > > > >>> > if
> > > > > > >>> > > > > they
> > > > > > >>> > > > > > > use
> > > > > > >>> > > > > > > > snappy in 0.8.2.1, they will experience a 2-3X
> > > space
> > > > > > >>> increase.
> > > > > > >>> > > The
> > > > > > >>> > > > > > issue
> > > > > > >>> > > > > > > > has since been fixed in trunk (just a snappy
> jar
> > > > > > upgrade).
> > > > > > >>> > Since
> > > > > > >>> > > > > 0.8.3
> > > > > > >>> > > > > > is
> > > > > > >>> > > > > > > > still a few months away, it may make sense to
> do
> > an
> > > > > > 0.8.2.2
> > > > > > >>> > > release
> > > > > > >>> > > > > > just
> > > > > > >>> > > > > > > to
> > > > > > >>> > > > > > > > fix this issue. Any objections?
> > > > > > >>> > > > > > > >
> > > > > > >>> > > > > > > > Thanks,
> > > > > > >>> > > > > > > >
> > > > > > >>> > > > > > > > Jun
> > > > > > >>> > > > > > > >
> > > > > > >>> > > > > > >
> > > > > > >>> > > > > >
> > > > > > >>> > > > >
> > > > > > >>> > > > >
> > > > > > >>> > > > >
> > > > > > >>> > > > > --
> > > > > > >>> > > > > Grant Henke
> > > > > > >>> > > > > Software Engineer | Cloudera
> > > > > > >>> > > > > grant@cloudera.com | twitter.com/gchenke |
> > > > > > >>> > linkedin.com/in/granthenke
> > > > > > >>> > > > >
> > > > > > >>> > > >
> > > > > > >>> > >
> > > > > > >>> >
> > > > > > >>> >
> > > > > > >>> >
> > > > > > >>> > --
> > > > > > >>> > Jeff Holoman
> > > > > > >>> > Systems Engineer
> > > > > > >>> >
> > > > > > >>>
> > > > > > >>
> > > > > > >
> > > > > > >
> > > > > > > --
> > > > > > > Grant Henke
> > > > > > > Software Engineer | Cloudera
> > > > > > > grant@cloudera.com | twitter.com/gchenke |
> > > > linkedin.com/in/granthenke
> > > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > > --
> > > > > > Grant Henke
> > > > > > Software Engineer | Cloudera
> > > > > > grant@cloudera.com | twitter.com/gchenke |
> > > linkedin.com/in/granthenke
> > > > > >
> > > > >
> > > >
> > > >
> > > >
> > > > --
> > > > Grant Henke
> > > > Software Engineer | Cloudera
> > > > grant@cloudera.com | twitter.com/gchenke |
> linkedin.com/in/granthenke
> > > >
> > >
> >
>
>
>
> --
> Thanks,
> Neha
>
Re: [DISCUSSION] Kafka 0.8.2.2 release?
Posted by Gwen Shapira <gw...@confluent.io>.
I completely agree that community focus on 0.8.3 is important and don't
want to take away from that effort.
That said, not everyone will upgrade to 0.8.3 immediately (unfortunately),
and there is still value in releasing bugfixes on previous releases.
I'm keeping the scope very small, to make the new version safe and make the
release easier and less distracting for us. If this interferes with 0.8.3,
we will adjust course to make sure it doesn't.
Gwen
On Tue, Aug 18, 2015 at 10:54 AM, Neha Narkhede <ne...@confluent.io> wrote:
> How about looking at the scope for the 0.8.3 release first before we cut
> yet another point release off of 0.8.2.2? Each release includes some
> overhead and if there is a larger release in the works, it might be worth
> working on getting that. My take is that the 2 things the community has
> been waiting for is SSL support and the new consumer and we have been
> promising to get 0.8.3 with both those features for several months now.
>
> Looking at the progress on both, it seems we are very close to getting both
> those checked in and it looks like we should get there in another 5-6
> weeks. Furthermore, both of these features are large and I anticipate us
> receiving feedback and bugs that will require a couple of point releases on
> top of 0.8.3 anyway. One possibility is to work on 0.8.3 together now and
> get the community to use the newly released features, gather feedback and
> do point releases incorporating that feedback and iterate on it.
>
> We could absolutely do both 0.8.2.2 and 0.8.3. What I'd ask for is for us
> to look at the 0.8.3 timeline too and make a call whether 0.8.2.2 still
> makes sense.
>
> Thanks,
> Neha
>
> On Tue, Aug 18, 2015 at 10:24 AM, Gwen Shapira <gw...@confluent.io> wrote:
>
> > Thanks Jun.
> >
> > I updated the list with your suggestions.
> > If anyone feels we are missing a critical patch for 0.8.2.2, please speak
> > up.
> >
> > Gwen
> >
> > On Mon, Aug 17, 2015 at 5:40 PM, Jun Rao <ju...@confluent.io> wrote:
> >
> > > Hi, Grant,
> > >
> > > I took a look at that list. None of those is really critical as you
> said.
> > > So, I'd suggest that we not include those to minimize the scope of the
> > > release.
> > >
> > > Thanks,
> > >
> > > Jun
> > >
> > > On Mon, Aug 17, 2015 at 5:16 PM, Grant Henke <gh...@cloudera.com>
> > wrote:
> > >
> > > > Thanks Gwen.
> > > >
> > > > I updated a few small things on the wiki page.
> > > >
> > > > Below is a list of jiras I think could also be marked as included.
> All
> > of
> > > > these, though not super critical, seem like fairly small and low risk
> > > > changes that help avoid potentially confusing issues or errors for
> > users.
> > > >
> > > > KAFKA-2012
> > > > KAFKA-972
> > > > KAFKA-2337 & KAFKA-2393
> > > > KAFKA-1867
> > > > KAFKA-2407
> > > > KAFKA-2234
> > > > KAFKA-1866
> > > > KAFKA-2345 & KAFKA-2355
> > > >
> > > > thoughts?
> > > >
> > > > Thank you,
> > > > Grant
> > > >
> > > > On Mon, Aug 17, 2015 at 4:56 PM, Gwen Shapira <gw...@confluent.io>
> > wrote:
> > > >
> > > > > Thanks for creating a list, Grant!
> > > > >
> > > > > I placed it on the wiki with a quick evaluation of the content and
> > > > whether
> > > > > it should be in 0.8.2.2:
> > > > >
> > > > >
> > > >
> > >
> >
> https://cwiki.apache.org/confluence/display/KAFKA/Proposed+patches+for+0.8.2.2
> > > > >
> > > > > I'm attempting to only cherrypick fixes that are both important for
> > > large
> > > > > number of users (or very critical to some users) and very safe
> > (mostly
> > > > > judged by the size of the change, but not only)
> > > > >
> > > > > If your favorite bugfix is missing from the list, or is there but
> > > marked
> > > > > "No", please let us know (in this thread) what we are missing and
> why
> > > it
> > > > is
> > > > > both important and safe.
> > > > > Also, if I accidentally included something you consider unsafe,
> speak
> > > up!
> > > > >
> > > > > Gwen
> > > > >
> > > > > On Mon, Aug 17, 2015 at 8:17 AM, Grant Henke <gh...@cloudera.com>
> > > > wrote:
> > > > >
> > > > > > +dev
> > > > > >
> > > > > > Adding dev list back in. Somehow it got dropped.
> > > > > >
> > > > > >
> > > > > > On Mon, Aug 17, 2015 at 10:16 AM, Grant Henke <
> ghenke@cloudera.com
> > >
> > > > > wrote:
> > > > > >
> > > > > > > Below is a list of candidate bug fix jiras marked fixed for
> > 0.8.3.
> > > I
> > > > > > don't
> > > > > > > suspect all of these will (or should) make it into the release
> > but
> > > > this
> > > > > > > should be a relatively complete list to work from:
> > > > > > >
> > > > > > > - KAFKA-2114 <
> > https://issues.apache.org/jira/browse/KAFKA-2114
> > > >:
> > > > > > Unable
> > > > > > > to change min.insync.replicas default
> > > > > > > - KAFKA-1702 <
> > https://issues.apache.org/jira/browse/KAFKA-1702
> > > >:
> > > > > > > Messages silently Lost by producer
> > > > > > > - KAFKA-2012 <
> > https://issues.apache.org/jira/browse/KAFKA-2012
> > > >:
> > > > > > > Broker should automatically handle corrupt index files
> > > > > > > - KAFKA-2406 <
> > https://issues.apache.org/jira/browse/KAFKA-2406
> > > >:
> > > > > ISR
> > > > > > > propagation should be throttled to avoid overwhelming
> > > controller.
> > > > > > > - KAFKA-2336 <
> > https://issues.apache.org/jira/browse/KAFKA-2336
> > > >:
> > > > > > > Changing offsets.topic.num.partitions after the offset topic
> > is
> > > > > > created
> > > > > > > breaks consumer group partition assignment
> > > > > > > - KAFKA-2337 <
> > https://issues.apache.org/jira/browse/KAFKA-2337
> > > >:
> > > > > > Verify
> > > > > > > that metric names will not collide when creating new topics
> > > > > > > - KAFKA-2393 <
> > https://issues.apache.org/jira/browse/KAFKA-2393
> > > >:
> > > > > > > Correctly Handle InvalidTopicException in
> > > > > KafkaApis.getTopicMetadata()
> > > > > > > - KAFKA-2189 <
> > https://issues.apache.org/jira/browse/KAFKA-2189
> > > >:
> > > > > > Snappy
> > > > > > > compression of message batches less efficient in 0.8.2.1
> > > > > > > - KAFKA-2308 <
> > https://issues.apache.org/jira/browse/KAFKA-2308
> > > >:
> > > > > New
> > > > > > > producer + Snappy face un-compression errors after broker
> > > restart
> > > > > > > - KAFKA-2042 <
> > https://issues.apache.org/jira/browse/KAFKA-2042
> > > >:
> > > > > New
> > > > > > > producer metadata update always get all topics.
> > > > > > > - KAFKA-1367 <
> > https://issues.apache.org/jira/browse/KAFKA-1367
> > > >:
> > > > > > Broker
> > > > > > > topic metadata not kept in sync with ZooKeeper
> > > > > > > - KAFKA-972 <
> https://issues.apache.org/jira/browse/KAFKA-972
> > >:
> > > > > > MetadataRequest
> > > > > > > returns stale list of brokers
> > > > > > > - KAFKA-1867 <
> > https://issues.apache.org/jira/browse/KAFKA-1867
> > > >:
> > > > > > liveBroker
> > > > > > > list not updated on a cluster with no topics
> > > > > > > - KAFKA-1650 <
> > https://issues.apache.org/jira/browse/KAFKA-1650
> > > >:
> > > > > > Mirror
> > > > > > > Maker could lose data on unclean shutdown.
> > > > > > > - KAFKA-2009 <
> > https://issues.apache.org/jira/browse/KAFKA-2009
> > > >:
> > > > > Fix
> > > > > > > UncheckedOffset.removeOffset synchronization and trace
> logging
> > > > issue
> > > > > > in
> > > > > > > mirror maker
> > > > > > > - KAFKA-2407 <
> > https://issues.apache.org/jira/browse/KAFKA-2407
> > > >:
> > > > > Only
> > > > > > > create a log directory when it will be used
> > > > > > > - KAFKA-2327 <
> > https://issues.apache.org/jira/browse/KAFKA-2327
> > > >:
> > > > > > > broker doesn't start if config defines advertised.host but
> not
> > > > > > > advertised.port
> > > > > > > - KAFKA-1788: producer record can stay in RecordAccumulator
> > > > forever
> > > > > if
> > > > > > > leader is no available
> > > > > > > - KAFKA-2234 <
> > https://issues.apache.org/jira/browse/KAFKA-2234
> > > >:
> > > > > > > Partition reassignment of a nonexistent topic prevents
> future
> > > > > > reassignments
> > > > > > > - KAFKA-2096 <
> > https://issues.apache.org/jira/browse/KAFKA-2096
> > > >:
> > > > > > > Enable keepalive socket option for broker to prevent socket
> > leak
> > > > > > > - KAFKA-1057 <
> > https://issues.apache.org/jira/browse/KAFKA-1057
> > > >:
> > > > > Trim
> > > > > > > whitespaces from user specified configs
> > > > > > > - KAFKA-1641 <
> > https://issues.apache.org/jira/browse/KAFKA-1641
> > > >:
> > > > > Log
> > > > > > > cleaner exits if last cleaned offset is lower than earliest
> > > offset
> > > > > > > - KAFKA-1648 <
> > https://issues.apache.org/jira/browse/KAFKA-1648
> > > >:
> > > > > > Round
> > > > > > > robin consumer balance throws an NPE when there are no
> topics
> > > > > > > - KAFKA-1724 <
> > https://issues.apache.org/jira/browse/KAFKA-1724
> > > >:
> > > > > > > Errors after reboot in single node setup
> > > > > > > - KAFKA-1758 <
> > https://issues.apache.org/jira/browse/KAFKA-1758
> > > >:
> > > > > > > corrupt recovery file prevents startup
> > > > > > > - KAFKA-1866 <
> > https://issues.apache.org/jira/browse/KAFKA-1866
> > > >:
> > > > > > > LogStartOffset gauge throws exceptions after log.delete()
> > > > > > > - KAFKA-1883 <
> > https://issues.apache.org/jira/browse/KAFKA-1883
> > > >:
> > > > > > NullPointerException
> > > > > > > in RequestSendThread
> > > > > > > - KAFKA-1896 <
> > https://issues.apache.org/jira/browse/KAFKA-1896
> > > >:
> > > > > > > Record size funcition of record in mirror maker hit NPE when
> > the
> > > > > > message
> > > > > > > value is null.
> > > > > > > - KAFKA-2101 <
> > https://issues.apache.org/jira/browse/KAFKA-2101
> > > >:
> > > > > > > Metric metadata-age is reset on a failed update
> > > > > > > - KAFKA-2112 <
> > https://issues.apache.org/jira/browse/KAFKA-2112
> > > >:
> > > > > make
> > > > > > > overflowWheel volatile
> > > > > > > - KAFKA-2117 <
> > https://issues.apache.org/jira/browse/KAFKA-2117
> > > >:
> > > > > > > OffsetManager uses incorrect field for metadata
> > > > > > > - KAFKA-2164 <
> > https://issues.apache.org/jira/browse/KAFKA-2164
> > > >:
> > > > > > > ReplicaFetcherThread: suspicious log message on reset offset
> > > > > > > - KAFKA-1668 <
> > https://issues.apache.org/jira/browse/KAFKA-1668
> > > >:
> > > > > > > TopicCommand doesn't warn if --topic argument doesn't match
> > any
> > > > > topics
> > > > > > > - KAFKA-2198 <
> > https://issues.apache.org/jira/browse/KAFKA-2198
> > > >:
> > > > > > > kafka-topics.sh exits with 0 status on failures
> > > > > > > - KAFKA-2235 <
> > https://issues.apache.org/jira/browse/KAFKA-2235
> > > >:
> > > > > > > LogCleaner offset map overflow
> > > > > > > - KAFKA-2241 <
> > https://issues.apache.org/jira/browse/KAFKA-2241
> > > >:
> > > > > > > AbstractFetcherThread.shutdown() should not block on
> > > > > > > ReadableByteChannel.read(buffer)
> > > > > > > - KAFKA-2272 <
> > https://issues.apache.org/jira/browse/KAFKA-2272
> > > >:
> > > > > > > listeners endpoint parsing fails if the hostname has capital
> > > > letter
> > > > > > > - KAFKA-2345 <
> > https://issues.apache.org/jira/browse/KAFKA-2345
> > > >:
> > > > > > > Attempt to delete a topic already marked for deletion throws
> > > > > > > ZkNodeExistsException
> > > > > > > - KAFKA-2353 <
> > https://issues.apache.org/jira/browse/KAFKA-2353
> > > >:
> > > > > > > SocketServer.Processor should catch exception and close the
> > > socket
> > > > > > properly
> > > > > > > in configureNewConnections.
> > > > > > > - KAFKA-1836 <
> > https://issues.apache.org/jira/browse/KAFKA-1836
> > > >:
> > > > > > > metadata.fetch.timeout.ms set to zero blocks forever
> > > > > > > - KAFKA-2317 <
> > https://issues.apache.org/jira/browse/KAFKA-2317
> > > >:
> > > > > > De-register
> > > > > > > isrChangeNotificationListener on controller resignation
> > > > > > >
> > > > > > > Note: KAFKA-2120 <
> > https://issues.apache.org/jira/browse/KAFKA-2120
> > > >
> > > > &
> > > > > > > KAFKA-2421 <https://issues.apache.org/jira/browse/KAFKA-2421>
> > were
> > > > > > > mentioned in previous emails, but are not in the list because
> > they
> > > > are
> > > > > > not
> > > > > > > committed yet.
> > > > > > >
> > > > > > > Hope that helps the effort.
> > > > > > >
> > > > > > > Thanks,
> > > > > > > Grant
> > > > > > >
> > > > > > > On Mon, Aug 17, 2015 at 12:09 AM, Grant Henke <
> > ghenke@cloudera.com
> > > >
> > > > > > wrote:
> > > > > > >
> > > > > > >> +1 to that suggestion. Though I suspect that requires a
> > committer
> > > to
> > > > > do.
> > > > > > >> Making it part of the standard commit process could work too.
> > > > > > >> On Aug 16, 2015 11:01 PM, "Gwen Shapira" <gw...@confluent.io>
> > > wrote:
> > > > > > >>
> > > > > > >>> BTW. I think it will be great for Apache Kafka to have a
> 0.8.2
> > > > > "release
> > > > > > >>> manager" who's role is to cherrypick low-risk bug-fixes into
> > the
> > > > > 0.8.2
> > > > > > >>> branch and once enough bug fixes happened (or if sufficiently
> > > > > critical
> > > > > > >>> fixes happened) to roll out a new maintenance release (with
> > > every 3
> > > > > > month
> > > > > > >>> as a reasonable bugfix release target).
> > > > > > >>>
> > > > > > >>> This will add some predictability regarding how fast we
> release
> > > > fixes
> > > > > > for
> > > > > > >>> bugs.
> > > > > > >>>
> > > > > > >>> Gwen
> > > > > > >>>
> > > > > > >>> On Sun, Aug 16, 2015 at 8:09 PM, Jeff Holoman <
> > > > jholoman@cloudera.com
> > > > > >
> > > > > > >>> wrote:
> > > > > > >>>
> > > > > > >>> > +1 for the release and also including
> > > > > > >>> >
> > > > > > >>> > https://issues.apache.org/jira/browse/KAFKA-2114
> > > > > > >>> >
> > > > > > >>> > Thanks
> > > > > > >>> >
> > > > > > >>> > Jeff
> > > > > > >>> >
> > > > > > >>> > On Sun, Aug 16, 2015 at 2:51 PM, Stevo Slavić <
> > > sslavic@gmail.com
> > > > >
> > > > > > >>> wrote:
> > > > > > >>> >
> > > > > > >>> > > +1 (non-binding) for 0.8.2.2 release
> > > > > > >>> > >
> > > > > > >>> > > Would be nice to include in that release new producer
> > > > resiliency
> > > > > > bug
> > > > > > >>> > fixes
> > > > > > >>> > > https://issues.apache.org/jira/browse/KAFKA-1788 and
> > > > > > >>> > > https://issues.apache.org/jira/browse/KAFKA-2120
> > > > > > >>> > >
> > > > > > >>> > > On Fri, Aug 14, 2015 at 4:03 PM, Gwen Shapira <
> > > > gwen@confluent.io
> > > > > >
> > > > > > >>> wrote:
> > > > > > >>> > >
> > > > > > >>> > > > Will be nice to include Kafka-2308 and fix two critical
> > > > snappy
> > > > > > >>> issues
> > > > > > >>> > in
> > > > > > >>> > > > the maintenance release.
> > > > > > >>> > > >
> > > > > > >>> > > > Gwen
> > > > > > >>> > > > On Aug 14, 2015 6:16 AM, "Grant Henke" <
> > > ghenke@cloudera.com>
> > > > > > >>> wrote:
> > > > > > >>> > > >
> > > > > > >>> > > > > Just to clarify. Will KAFKA-2189 be the only patch in
> > the
> > > > > > >>> release?
> > > > > > >>> > > > >
> > > > > > >>> > > > > On Fri, Aug 14, 2015 at 7:35 AM, Manikumar Reddy <
> > > > > > >>> > kumar@nmsworks.co.in
> > > > > > >>> > > >
> > > > > > >>> > > > > wrote:
> > > > > > >>> > > > >
> > > > > > >>> > > > > > +1 for 0.8.2.2 release
> > > > > > >>> > > > > >
> > > > > > >>> > > > > > On Fri, Aug 14, 2015 at 5:49 PM, Ismael Juma <
> > > > > > >>> ismael@juma.me.uk>
> > > > > > >>> > > > wrote:
> > > > > > >>> > > > > >
> > > > > > >>> > > > > > > I think this is a good idea as the change is
> > minimal
> > > on
> > > > > our
> > > > > > >>> side
> > > > > > >>> > > and
> > > > > > >>> > > > it
> > > > > > >>> > > > > > has
> > > > > > >>> > > > > > > been tested in production for some time by the
> > > > reporter.
> > > > > > >>> > > > > > >
> > > > > > >>> > > > > > > Best,
> > > > > > >>> > > > > > > Ismael
> > > > > > >>> > > > > > >
> > > > > > >>> > > > > > > On Fri, Aug 14, 2015 at 1:15 PM, Jun Rao <
> > > > > jun@confluent.io
> > > > > > >
> > > > > > >>> > wrote:
> > > > > > >>> > > > > > >
> > > > > > >>> > > > > > > > Hi, Everyone,
> > > > > > >>> > > > > > > >
> > > > > > >>> > > > > > > > Since the release of Kafka 0.8.2.1, a number of
> > > > people
> > > > > > have
> > > > > > >>> > > > reported
> > > > > > >>> > > > > an
> > > > > > >>> > > > > > > > issue with snappy compression (
> > > > > > >>> > > > > > > >
> https://issues.apache.org/jira/browse/KAFKA-2189
> > ).
> > > > > > >>> Basically,
> > > > > > >>> > if
> > > > > > >>> > > > > they
> > > > > > >>> > > > > > > use
> > > > > > >>> > > > > > > > snappy in 0.8.2.1, they will experience a 2-3X
> > > space
> > > > > > >>> increase.
> > > > > > >>> > > The
> > > > > > >>> > > > > > issue
> > > > > > >>> > > > > > > > has since been fixed in trunk (just a snappy
> jar
> > > > > > upgrade).
> > > > > > >>> > Since
> > > > > > >>> > > > > 0.8.3
> > > > > > >>> > > > > > is
> > > > > > >>> > > > > > > > still a few months away, it may make sense to
> do
> > an
> > > > > > 0.8.2.2
> > > > > > >>> > > release
> > > > > > >>> > > > > > just
> > > > > > >>> > > > > > > to
> > > > > > >>> > > > > > > > fix this issue. Any objections?
> > > > > > >>> > > > > > > >
> > > > > > >>> > > > > > > > Thanks,
> > > > > > >>> > > > > > > >
> > > > > > >>> > > > > > > > Jun
> > > > > > >>> > > > > > > >
> > > > > > >>> > > > > > >
> > > > > > >>> > > > > >
> > > > > > >>> > > > >
> > > > > > >>> > > > >
> > > > > > >>> > > > >
> > > > > > >>> > > > > --
> > > > > > >>> > > > > Grant Henke
> > > > > > >>> > > > > Software Engineer | Cloudera
> > > > > > >>> > > > > grant@cloudera.com | twitter.com/gchenke |
> > > > > > >>> > linkedin.com/in/granthenke
> > > > > > >>> > > > >
> > > > > > >>> > > >
> > > > > > >>> > >
> > > > > > >>> >
> > > > > > >>> >
> > > > > > >>> >
> > > > > > >>> > --
> > > > > > >>> > Jeff Holoman
> > > > > > >>> > Systems Engineer
> > > > > > >>> >
> > > > > > >>>
> > > > > > >>
> > > > > > >
> > > > > > >
> > > > > > > --
> > > > > > > Grant Henke
> > > > > > > Software Engineer | Cloudera
> > > > > > > grant@cloudera.com | twitter.com/gchenke |
> > > > linkedin.com/in/granthenke
> > > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > > --
> > > > > > Grant Henke
> > > > > > Software Engineer | Cloudera
> > > > > > grant@cloudera.com | twitter.com/gchenke |
> > > linkedin.com/in/granthenke
> > > > > >
> > > > >
> > > >
> > > >
> > > >
> > > > --
> > > > Grant Henke
> > > > Software Engineer | Cloudera
> > > > grant@cloudera.com | twitter.com/gchenke |
> linkedin.com/in/granthenke
> > > >
> > >
> >
>
>
>
> --
> Thanks,
> Neha
>
Re: [DISCUSSION] Kafka 0.8.2.2 release?
Posted by Andrew Otto <ao...@wikimedia.org>.
I agree: keep it simple :) The latest stable version of Kafka right now has a critical bug in it. Fixing that would be good enough. 0.8.2.2 should probably just a maintenance/bugfix release.
> On Aug 18, 2015, at 14:29, Edward Ribeiro <ed...@gmail.com> wrote:
>
> I sort of follow Daniel Nelson on this issue: cut a 0.8.2.2 but not include
> much thing besides Snappy fixes. I mean, include a couple of additional
> critical bug fixes, if really urgent, and that's it.
>
> On Tue, Aug 18, 2015 at 3:25 PM, Daniel Nelson <da...@vungle.com>
> wrote:
>
>> I am strongly in favor of cutting a 0.8.2.2 release, but I don’t think
>> that it needs to include anything other than the fix for Snappy that kicked
>> off this discussion in the first place. If there are additional critical
>> issues that can be included without delaying the process, I see no downside.
>>
>>> On Aug 18, 2015, at 10:54 AM, Neha Narkhede <ne...@confluent.io> wrote:
>>>
>>> How about looking at the scope for the 0.8.3 release first before we cut
>>> yet another point release off of 0.8.2.2? Each release includes some
>>> overhead and if there is a larger release in the works, it might be worth
>>> working on getting that. My take is that the 2 things the community has
>>> been waiting for is SSL support and the new consumer and we have been
>>> promising to get 0.8.3 with both those features for several months now.
>>>
>>> Looking at the progress on both, it seems we are very close to getting
>> both
>>> those checked in and it looks like we should get there in another 5-6
>>> weeks. Furthermore, both of these features are large and I anticipate us
>>> receiving feedback and bugs that will require a couple of point releases
>> on
>>> top of 0.8.3 anyway. One possibility is to work on 0.8.3 together now and
>>> get the community to use the newly released features, gather feedback and
>>> do point releases incorporating that feedback and iterate on it.
>>>
>>> We could absolutely do both 0.8.2.2 and 0.8.3. What I'd ask for is for us
>>> to look at the 0.8.3 timeline too and make a call whether 0.8.2.2 still
>>> makes sense.
>>>
>>> Thanks,
>>> Neha
>>>
>>> On Tue, Aug 18, 2015 at 10:24 AM, Gwen Shapira <gw...@confluent.io>
>> wrote:
>>>
>>>> Thanks Jun.
>>>>
>>>> I updated the list with your suggestions.
>>>> If anyone feels we are missing a critical patch for 0.8.2.2, please
>> speak
>>>> up.
>>>>
>>>> Gwen
>>>>
>>>> On Mon, Aug 17, 2015 at 5:40 PM, Jun Rao <ju...@confluent.io> wrote:
>>>>
>>>>> Hi, Grant,
>>>>>
>>>>> I took a look at that list. None of those is really critical as you
>> said.
>>>>> So, I'd suggest that we not include those to minimize the scope of the
>>>>> release.
>>>>>
>>>>> Thanks,
>>>>>
>>>>> Jun
>>>>>
>>>>> On Mon, Aug 17, 2015 at 5:16 PM, Grant Henke <gh...@cloudera.com>
>>>> wrote:
>>>>>
>>>>>> Thanks Gwen.
>>>>>>
>>>>>> I updated a few small things on the wiki page.
>>>>>>
>>>>>> Below is a list of jiras I think could also be marked as included. All
>>>> of
>>>>>> these, though not super critical, seem like fairly small and low risk
>>>>>> changes that help avoid potentially confusing issues or errors for
>>>> users.
>>>>>>
>>>>>> KAFKA-2012
>>>>>> KAFKA-972
>>>>>> KAFKA-2337 & KAFKA-2393
>>>>>> KAFKA-1867
>>>>>> KAFKA-2407
>>>>>> KAFKA-2234
>>>>>> KAFKA-1866
>>>>>> KAFKA-2345 & KAFKA-2355
>>>>>>
>>>>>> thoughts?
>>>>>>
>>>>>> Thank you,
>>>>>> Grant
>>>>>>
>>>>>> On Mon, Aug 17, 2015 at 4:56 PM, Gwen Shapira <gw...@confluent.io>
>>>> wrote:
>>>>>>
>>>>>>> Thanks for creating a list, Grant!
>>>>>>>
>>>>>>> I placed it on the wiki with a quick evaluation of the content and
>>>>>> whether
>>>>>>> it should be in 0.8.2.2:
>>>>>>>
>>>>>>>
>>>>>>
>>>>>
>>>>
>> https://cwiki.apache.org/confluence/display/KAFKA/Proposed+patches+for+0.8.2.2
>>>>>>>
>>>>>>> I'm attempting to only cherrypick fixes that are both important for
>>>>> large
>>>>>>> number of users (or very critical to some users) and very safe
>>>> (mostly
>>>>>>> judged by the size of the change, but not only)
>>>>>>>
>>>>>>> If your favorite bugfix is missing from the list, or is there but
>>>>> marked
>>>>>>> "No", please let us know (in this thread) what we are missing and why
>>>>> it
>>>>>> is
>>>>>>> both important and safe.
>>>>>>> Also, if I accidentally included something you consider unsafe, speak
>>>>> up!
>>>>>>>
>>>>>>> Gwen
>>>>>>>
>>>>>>> On Mon, Aug 17, 2015 at 8:17 AM, Grant Henke <gh...@cloudera.com>
>>>>>> wrote:
>>>>>>>
>>>>>>>> +dev
>>>>>>>>
>>>>>>>> Adding dev list back in. Somehow it got dropped.
>>>>>>>>
>>>>>>>>
>>>>>>>> On Mon, Aug 17, 2015 at 10:16 AM, Grant Henke <ghenke@cloudera.com
>>>>>
>>>>>>> wrote:
>>>>>>>>
>>>>>>>>> Below is a list of candidate bug fix jiras marked fixed for
>>>> 0.8.3.
>>>>> I
>>>>>>>> don't
>>>>>>>>> suspect all of these will (or should) make it into the release
>>>> but
>>>>>> this
>>>>>>>>> should be a relatively complete list to work from:
>>>>>>>>>
>>>>>>>>> - KAFKA-2114 <
>>>> https://issues.apache.org/jira/browse/KAFKA-2114
>>>>>> :
>>>>>>>> Unable
>>>>>>>>> to change min.insync.replicas default
>>>>>>>>> - KAFKA-1702 <
>>>> https://issues.apache.org/jira/browse/KAFKA-1702
>>>>>> :
>>>>>>>>> Messages silently Lost by producer
>>>>>>>>> - KAFKA-2012 <
>>>> https://issues.apache.org/jira/browse/KAFKA-2012
>>>>>> :
>>>>>>>>> Broker should automatically handle corrupt index files
>>>>>>>>> - KAFKA-2406 <
>>>> https://issues.apache.org/jira/browse/KAFKA-2406
>>>>>> :
>>>>>>> ISR
>>>>>>>>> propagation should be throttled to avoid overwhelming
>>>>> controller.
>>>>>>>>> - KAFKA-2336 <
>>>> https://issues.apache.org/jira/browse/KAFKA-2336
>>>>>> :
>>>>>>>>> Changing offsets.topic.num.partitions after the offset topic
>>>> is
>>>>>>>> created
>>>>>>>>> breaks consumer group partition assignment
>>>>>>>>> - KAFKA-2337 <
>>>> https://issues.apache.org/jira/browse/KAFKA-2337
>>>>>> :
>>>>>>>> Verify
>>>>>>>>> that metric names will not collide when creating new topics
>>>>>>>>> - KAFKA-2393 <
>>>> https://issues.apache.org/jira/browse/KAFKA-2393
>>>>>> :
>>>>>>>>> Correctly Handle InvalidTopicException in
>>>>>>> KafkaApis.getTopicMetadata()
>>>>>>>>> - KAFKA-2189 <
>>>> https://issues.apache.org/jira/browse/KAFKA-2189
>>>>>> :
>>>>>>>> Snappy
>>>>>>>>> compression of message batches less efficient in 0.8.2.1
>>>>>>>>> - KAFKA-2308 <
>>>> https://issues.apache.org/jira/browse/KAFKA-2308
>>>>>> :
>>>>>>> New
>>>>>>>>> producer + Snappy face un-compression errors after broker
>>>>> restart
>>>>>>>>> - KAFKA-2042 <
>>>> https://issues.apache.org/jira/browse/KAFKA-2042
>>>>>> :
>>>>>>> New
>>>>>>>>> producer metadata update always get all topics.
>>>>>>>>> - KAFKA-1367 <
>>>> https://issues.apache.org/jira/browse/KAFKA-1367
>>>>>> :
>>>>>>>> Broker
>>>>>>>>> topic metadata not kept in sync with ZooKeeper
>>>>>>>>> - KAFKA-972 <https://issues.apache.org/jira/browse/KAFKA-972
>>>>> :
>>>>>>>> MetadataRequest
>>>>>>>>> returns stale list of brokers
>>>>>>>>> - KAFKA-1867 <
>>>> https://issues.apache.org/jira/browse/KAFKA-1867
>>>>>> :
>>>>>>>> liveBroker
>>>>>>>>> list not updated on a cluster with no topics
>>>>>>>>> - KAFKA-1650 <
>>>> https://issues.apache.org/jira/browse/KAFKA-1650
>>>>>> :
>>>>>>>> Mirror
>>>>>>>>> Maker could lose data on unclean shutdown.
>>>>>>>>> - KAFKA-2009 <
>>>> https://issues.apache.org/jira/browse/KAFKA-2009
>>>>>> :
>>>>>>> Fix
>>>>>>>>> UncheckedOffset.removeOffset synchronization and trace logging
>>>>>> issue
>>>>>>>> in
>>>>>>>>> mirror maker
>>>>>>>>> - KAFKA-2407 <
>>>> https://issues.apache.org/jira/browse/KAFKA-2407
>>>>>> :
>>>>>>> Only
>>>>>>>>> create a log directory when it will be used
>>>>>>>>> - KAFKA-2327 <
>>>> https://issues.apache.org/jira/browse/KAFKA-2327
>>>>>> :
>>>>>>>>> broker doesn't start if config defines advertised.host but not
>>>>>>>>> advertised.port
>>>>>>>>> - KAFKA-1788: producer record can stay in RecordAccumulator
>>>>>> forever
>>>>>>> if
>>>>>>>>> leader is no available
>>>>>>>>> - KAFKA-2234 <
>>>> https://issues.apache.org/jira/browse/KAFKA-2234
>>>>>> :
>>>>>>>>> Partition reassignment of a nonexistent topic prevents future
>>>>>>>> reassignments
>>>>>>>>> - KAFKA-2096 <
>>>> https://issues.apache.org/jira/browse/KAFKA-2096
>>>>>> :
>>>>>>>>> Enable keepalive socket option for broker to prevent socket
>>>> leak
>>>>>>>>> - KAFKA-1057 <
>>>> https://issues.apache.org/jira/browse/KAFKA-1057
>>>>>> :
>>>>>>> Trim
>>>>>>>>> whitespaces from user specified configs
>>>>>>>>> - KAFKA-1641 <
>>>> https://issues.apache.org/jira/browse/KAFKA-1641
>>>>>> :
>>>>>>> Log
>>>>>>>>> cleaner exits if last cleaned offset is lower than earliest
>>>>> offset
>>>>>>>>> - KAFKA-1648 <
>>>> https://issues.apache.org/jira/browse/KAFKA-1648
>>>>>> :
>>>>>>>> Round
>>>>>>>>> robin consumer balance throws an NPE when there are no topics
>>>>>>>>> - KAFKA-1724 <
>>>> https://issues.apache.org/jira/browse/KAFKA-1724
>>>>>> :
>>>>>>>>> Errors after reboot in single node setup
>>>>>>>>> - KAFKA-1758 <
>>>> https://issues.apache.org/jira/browse/KAFKA-1758
>>>>>> :
>>>>>>>>> corrupt recovery file prevents startup
>>>>>>>>> - KAFKA-1866 <
>>>> https://issues.apache.org/jira/browse/KAFKA-1866
>>>>>> :
>>>>>>>>> LogStartOffset gauge throws exceptions after log.delete()
>>>>>>>>> - KAFKA-1883 <
>>>> https://issues.apache.org/jira/browse/KAFKA-1883
>>>>>> :
>>>>>>>> NullPointerException
>>>>>>>>> in RequestSendThread
>>>>>>>>> - KAFKA-1896 <
>>>> https://issues.apache.org/jira/browse/KAFKA-1896
>>>>>> :
>>>>>>>>> Record size funcition of record in mirror maker hit NPE when
>>>> the
>>>>>>>> message
>>>>>>>>> value is null.
>>>>>>>>> - KAFKA-2101 <
>>>> https://issues.apache.org/jira/browse/KAFKA-2101
>>>>>> :
>>>>>>>>> Metric metadata-age is reset on a failed update
>>>>>>>>> - KAFKA-2112 <
>>>> https://issues.apache.org/jira/browse/KAFKA-2112
>>>>>> :
>>>>>>> make
>>>>>>>>> overflowWheel volatile
>>>>>>>>> - KAFKA-2117 <
>>>> https://issues.apache.org/jira/browse/KAFKA-2117
>>>>>> :
>>>>>>>>> OffsetManager uses incorrect field for metadata
>>>>>>>>> - KAFKA-2164 <
>>>> https://issues.apache.org/jira/browse/KAFKA-2164
>>>>>> :
>>>>>>>>> ReplicaFetcherThread: suspicious log message on reset offset
>>>>>>>>> - KAFKA-1668 <
>>>> https://issues.apache.org/jira/browse/KAFKA-1668
>>>>>> :
>>>>>>>>> TopicCommand doesn't warn if --topic argument doesn't match
>>>> any
>>>>>>> topics
>>>>>>>>> - KAFKA-2198 <
>>>> https://issues.apache.org/jira/browse/KAFKA-2198
>>>>>> :
>>>>>>>>> kafka-topics.sh exits with 0 status on failures
>>>>>>>>> - KAFKA-2235 <
>>>> https://issues.apache.org/jira/browse/KAFKA-2235
>>>>>> :
>>>>>>>>> LogCleaner offset map overflow
>>>>>>>>> - KAFKA-2241 <
>>>> https://issues.apache.org/jira/browse/KAFKA-2241
>>>>>> :
>>>>>>>>> AbstractFetcherThread.shutdown() should not block on
>>>>>>>>> ReadableByteChannel.read(buffer)
>>>>>>>>> - KAFKA-2272 <
>>>> https://issues.apache.org/jira/browse/KAFKA-2272
>>>>>> :
>>>>>>>>> listeners endpoint parsing fails if the hostname has capital
>>>>>> letter
>>>>>>>>> - KAFKA-2345 <
>>>> https://issues.apache.org/jira/browse/KAFKA-2345
>>>>>> :
>>>>>>>>> Attempt to delete a topic already marked for deletion throws
>>>>>>>>> ZkNodeExistsException
>>>>>>>>> - KAFKA-2353 <
>>>> https://issues.apache.org/jira/browse/KAFKA-2353
>>>>>> :
>>>>>>>>> SocketServer.Processor should catch exception and close the
>>>>> socket
>>>>>>>> properly
>>>>>>>>> in configureNewConnections.
>>>>>>>>> - KAFKA-1836 <
>>>> https://issues.apache.org/jira/browse/KAFKA-1836
>>>>>> :
>>>>>>>>> metadata.fetch.timeout.ms set to zero blocks forever
>>>>>>>>> - KAFKA-2317 <
>>>> https://issues.apache.org/jira/browse/KAFKA-2317
>>>>>> :
>>>>>>>> De-register
>>>>>>>>> isrChangeNotificationListener on controller resignation
>>>>>>>>>
>>>>>>>>> Note: KAFKA-2120 <
>>>> https://issues.apache.org/jira/browse/KAFKA-2120
>>>>>>
>>>>>> &
>>>>>>>>> KAFKA-2421 <https://issues.apache.org/jira/browse/KAFKA-2421>
>>>> were
>>>>>>>>> mentioned in previous emails, but are not in the list because
>>>> they
>>>>>> are
>>>>>>>> not
>>>>>>>>> committed yet.
>>>>>>>>>
>>>>>>>>> Hope that helps the effort.
>>>>>>>>>
>>>>>>>>> Thanks,
>>>>>>>>> Grant
>>>>>>>>>
>>>>>>>>> On Mon, Aug 17, 2015 at 12:09 AM, Grant Henke <
>>>> ghenke@cloudera.com
>>>>>>
>>>>>>>> wrote:
>>>>>>>>>
>>>>>>>>>> +1 to that suggestion. Though I suspect that requires a
>>>> committer
>>>>> to
>>>>>>> do.
>>>>>>>>>> Making it part of the standard commit process could work too.
>>>>>>>>>> On Aug 16, 2015 11:01 PM, "Gwen Shapira" <gw...@confluent.io>
>>>>> wrote:
>>>>>>>>>>
>>>>>>>>>>> BTW. I think it will be great for Apache Kafka to have a 0.8.2
>>>>>>> "release
>>>>>>>>>>> manager" who's role is to cherrypick low-risk bug-fixes into
>>>> the
>>>>>>> 0.8.2
>>>>>>>>>>> branch and once enough bug fixes happened (or if sufficiently
>>>>>>> critical
>>>>>>>>>>> fixes happened) to roll out a new maintenance release (with
>>>>> every 3
>>>>>>>> month
>>>>>>>>>>> as a reasonable bugfix release target).
>>>>>>>>>>>
>>>>>>>>>>> This will add some predictability regarding how fast we release
>>>>>> fixes
>>>>>>>> for
>>>>>>>>>>> bugs.
>>>>>>>>>>>
>>>>>>>>>>> Gwen
>>>>>>>>>>>
>>>>>>>>>>> On Sun, Aug 16, 2015 at 8:09 PM, Jeff Holoman <
>>>>>> jholoman@cloudera.com
>>>>>>>>
>>>>>>>>>>> wrote:
>>>>>>>>>>>
>>>>>>>>>>>> +1 for the release and also including
>>>>>>>>>>>>
>>>>>>>>>>>> https://issues.apache.org/jira/browse/KAFKA-2114
>>>>>>>>>>>>
>>>>>>>>>>>> Thanks
>>>>>>>>>>>>
>>>>>>>>>>>> Jeff
>>>>>>>>>>>>
>>>>>>>>>>>> On Sun, Aug 16, 2015 at 2:51 PM, Stevo Slavić <
>>>>> sslavic@gmail.com
>>>>>>>
>>>>>>>>>>> wrote:
>>>>>>>>>>>>
>>>>>>>>>>>>> +1 (non-binding) for 0.8.2.2 release
>>>>>>>>>>>>>
>>>>>>>>>>>>> Would be nice to include in that release new producer
>>>>>> resiliency
>>>>>>>> bug
>>>>>>>>>>>> fixes
>>>>>>>>>>>>> https://issues.apache.org/jira/browse/KAFKA-1788 and
>>>>>>>>>>>>> https://issues.apache.org/jira/browse/KAFKA-2120
>>>>>>>>>>>>>
>>>>>>>>>>>>> On Fri, Aug 14, 2015 at 4:03 PM, Gwen Shapira <
>>>>>> gwen@confluent.io
>>>>>>>>
>>>>>>>>>>> wrote:
>>>>>>>>>>>>>
>>>>>>>>>>>>>> Will be nice to include Kafka-2308 and fix two critical
>>>>>> snappy
>>>>>>>>>>> issues
>>>>>>>>>>>> in
>>>>>>>>>>>>>> the maintenance release.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Gwen
>>>>>>>>>>>>>> On Aug 14, 2015 6:16 AM, "Grant Henke" <
>>>>> ghenke@cloudera.com>
>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Just to clarify. Will KAFKA-2189 be the only patch in
>>>> the
>>>>>>>>>>> release?
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> On Fri, Aug 14, 2015 at 7:35 AM, Manikumar Reddy <
>>>>>>>>>>>> kumar@nmsworks.co.in
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> +1 for 0.8.2.2 release
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> On Fri, Aug 14, 2015 at 5:49 PM, Ismael Juma <
>>>>>>>>>>> ismael@juma.me.uk>
>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> I think this is a good idea as the change is
>>>> minimal
>>>>> on
>>>>>>> our
>>>>>>>>>>> side
>>>>>>>>>>>>> and
>>>>>>>>>>>>>> it
>>>>>>>>>>>>>>>> has
>>>>>>>>>>>>>>>>> been tested in production for some time by the
>>>>>> reporter.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Best,
>>>>>>>>>>>>>>>>> Ismael
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> On Fri, Aug 14, 2015 at 1:15 PM, Jun Rao <
>>>>>>> jun@confluent.io
>>>>>>>>>
>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> Hi, Everyone,
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> Since the release of Kafka 0.8.2.1, a number of
>>>>>> people
>>>>>>>> have
>>>>>>>>>>>>>> reported
>>>>>>>>>>>>>>> an
>>>>>>>>>>>>>>>>>> issue with snappy compression (
>>>>>>>>>>>>>>>>>> https://issues.apache.org/jira/browse/KAFKA-2189
>>>> ).
>>>>>>>>>>> Basically,
>>>>>>>>>>>> if
>>>>>>>>>>>>>>> they
>>>>>>>>>>>>>>>>> use
>>>>>>>>>>>>>>>>>> snappy in 0.8.2.1, they will experience a 2-3X
>>>>> space
>>>>>>>>>>> increase.
>>>>>>>>>>>>> The
>>>>>>>>>>>>>>>> issue
>>>>>>>>>>>>>>>>>> has since been fixed in trunk (just a snappy jar
>>>>>>>> upgrade).
>>>>>>>>>>>> Since
>>>>>>>>>>>>>>> 0.8.3
>>>>>>>>>>>>>>>> is
>>>>>>>>>>>>>>>>>> still a few months away, it may make sense to do
>>>> an
>>>>>>>> 0.8.2.2
>>>>>>>>>>>>> release
>>>>>>>>>>>>>>>> just
>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>> fix this issue. Any objections?
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> Jun
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>> Grant Henke
>>>>>>>>>>>>>>> Software Engineer | Cloudera
>>>>>>>>>>>>>>> grant@cloudera.com | twitter.com/gchenke |
>>>>>>>>>>>> linkedin.com/in/granthenke
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> --
>>>>>>>>>>>> Jeff Holoman
>>>>>>>>>>>> Systems Engineer
>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> --
>>>>>>>>> Grant Henke
>>>>>>>>> Software Engineer | Cloudera
>>>>>>>>> grant@cloudera.com | twitter.com/gchenke |
>>>>>> linkedin.com/in/granthenke
>>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> --
>>>>>>>> Grant Henke
>>>>>>>> Software Engineer | Cloudera
>>>>>>>> grant@cloudera.com | twitter.com/gchenke |
>>>>> linkedin.com/in/granthenke
>>>>>>>>
>>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> --
>>>>>> Grant Henke
>>>>>> Software Engineer | Cloudera
>>>>>> grant@cloudera.com | twitter.com/gchenke | linkedin.com/in/granthenke
>>>>>>
>>>>>
>>>>
>>>
>>>
>>>
>>> --
>>> Thanks,
>>> Neha
>>
>>
Re: [DISCUSSION] Kafka 0.8.2.2 release?
Posted by Andrew Otto <ao...@wikimedia.org>.
I agree: keep it simple :) The latest stable version of Kafka right now has a critical bug in it. Fixing that would be good enough. 0.8.2.2 should probably just a maintenance/bugfix release.
> On Aug 18, 2015, at 14:29, Edward Ribeiro <ed...@gmail.com> wrote:
>
> I sort of follow Daniel Nelson on this issue: cut a 0.8.2.2 but not include
> much thing besides Snappy fixes. I mean, include a couple of additional
> critical bug fixes, if really urgent, and that's it.
>
> On Tue, Aug 18, 2015 at 3:25 PM, Daniel Nelson <da...@vungle.com>
> wrote:
>
>> I am strongly in favor of cutting a 0.8.2.2 release, but I don’t think
>> that it needs to include anything other than the fix for Snappy that kicked
>> off this discussion in the first place. If there are additional critical
>> issues that can be included without delaying the process, I see no downside.
>>
>>> On Aug 18, 2015, at 10:54 AM, Neha Narkhede <ne...@confluent.io> wrote:
>>>
>>> How about looking at the scope for the 0.8.3 release first before we cut
>>> yet another point release off of 0.8.2.2? Each release includes some
>>> overhead and if there is a larger release in the works, it might be worth
>>> working on getting that. My take is that the 2 things the community has
>>> been waiting for is SSL support and the new consumer and we have been
>>> promising to get 0.8.3 with both those features for several months now.
>>>
>>> Looking at the progress on both, it seems we are very close to getting
>> both
>>> those checked in and it looks like we should get there in another 5-6
>>> weeks. Furthermore, both of these features are large and I anticipate us
>>> receiving feedback and bugs that will require a couple of point releases
>> on
>>> top of 0.8.3 anyway. One possibility is to work on 0.8.3 together now and
>>> get the community to use the newly released features, gather feedback and
>>> do point releases incorporating that feedback and iterate on it.
>>>
>>> We could absolutely do both 0.8.2.2 and 0.8.3. What I'd ask for is for us
>>> to look at the 0.8.3 timeline too and make a call whether 0.8.2.2 still
>>> makes sense.
>>>
>>> Thanks,
>>> Neha
>>>
>>> On Tue, Aug 18, 2015 at 10:24 AM, Gwen Shapira <gw...@confluent.io>
>> wrote:
>>>
>>>> Thanks Jun.
>>>>
>>>> I updated the list with your suggestions.
>>>> If anyone feels we are missing a critical patch for 0.8.2.2, please
>> speak
>>>> up.
>>>>
>>>> Gwen
>>>>
>>>> On Mon, Aug 17, 2015 at 5:40 PM, Jun Rao <ju...@confluent.io> wrote:
>>>>
>>>>> Hi, Grant,
>>>>>
>>>>> I took a look at that list. None of those is really critical as you
>> said.
>>>>> So, I'd suggest that we not include those to minimize the scope of the
>>>>> release.
>>>>>
>>>>> Thanks,
>>>>>
>>>>> Jun
>>>>>
>>>>> On Mon, Aug 17, 2015 at 5:16 PM, Grant Henke <gh...@cloudera.com>
>>>> wrote:
>>>>>
>>>>>> Thanks Gwen.
>>>>>>
>>>>>> I updated a few small things on the wiki page.
>>>>>>
>>>>>> Below is a list of jiras I think could also be marked as included. All
>>>> of
>>>>>> these, though not super critical, seem like fairly small and low risk
>>>>>> changes that help avoid potentially confusing issues or errors for
>>>> users.
>>>>>>
>>>>>> KAFKA-2012
>>>>>> KAFKA-972
>>>>>> KAFKA-2337 & KAFKA-2393
>>>>>> KAFKA-1867
>>>>>> KAFKA-2407
>>>>>> KAFKA-2234
>>>>>> KAFKA-1866
>>>>>> KAFKA-2345 & KAFKA-2355
>>>>>>
>>>>>> thoughts?
>>>>>>
>>>>>> Thank you,
>>>>>> Grant
>>>>>>
>>>>>> On Mon, Aug 17, 2015 at 4:56 PM, Gwen Shapira <gw...@confluent.io>
>>>> wrote:
>>>>>>
>>>>>>> Thanks for creating a list, Grant!
>>>>>>>
>>>>>>> I placed it on the wiki with a quick evaluation of the content and
>>>>>> whether
>>>>>>> it should be in 0.8.2.2:
>>>>>>>
>>>>>>>
>>>>>>
>>>>>
>>>>
>> https://cwiki.apache.org/confluence/display/KAFKA/Proposed+patches+for+0.8.2.2
>>>>>>>
>>>>>>> I'm attempting to only cherrypick fixes that are both important for
>>>>> large
>>>>>>> number of users (or very critical to some users) and very safe
>>>> (mostly
>>>>>>> judged by the size of the change, but not only)
>>>>>>>
>>>>>>> If your favorite bugfix is missing from the list, or is there but
>>>>> marked
>>>>>>> "No", please let us know (in this thread) what we are missing and why
>>>>> it
>>>>>> is
>>>>>>> both important and safe.
>>>>>>> Also, if I accidentally included something you consider unsafe, speak
>>>>> up!
>>>>>>>
>>>>>>> Gwen
>>>>>>>
>>>>>>> On Mon, Aug 17, 2015 at 8:17 AM, Grant Henke <gh...@cloudera.com>
>>>>>> wrote:
>>>>>>>
>>>>>>>> +dev
>>>>>>>>
>>>>>>>> Adding dev list back in. Somehow it got dropped.
>>>>>>>>
>>>>>>>>
>>>>>>>> On Mon, Aug 17, 2015 at 10:16 AM, Grant Henke <ghenke@cloudera.com
>>>>>
>>>>>>> wrote:
>>>>>>>>
>>>>>>>>> Below is a list of candidate bug fix jiras marked fixed for
>>>> 0.8.3.
>>>>> I
>>>>>>>> don't
>>>>>>>>> suspect all of these will (or should) make it into the release
>>>> but
>>>>>> this
>>>>>>>>> should be a relatively complete list to work from:
>>>>>>>>>
>>>>>>>>> - KAFKA-2114 <
>>>> https://issues.apache.org/jira/browse/KAFKA-2114
>>>>>> :
>>>>>>>> Unable
>>>>>>>>> to change min.insync.replicas default
>>>>>>>>> - KAFKA-1702 <
>>>> https://issues.apache.org/jira/browse/KAFKA-1702
>>>>>> :
>>>>>>>>> Messages silently Lost by producer
>>>>>>>>> - KAFKA-2012 <
>>>> https://issues.apache.org/jira/browse/KAFKA-2012
>>>>>> :
>>>>>>>>> Broker should automatically handle corrupt index files
>>>>>>>>> - KAFKA-2406 <
>>>> https://issues.apache.org/jira/browse/KAFKA-2406
>>>>>> :
>>>>>>> ISR
>>>>>>>>> propagation should be throttled to avoid overwhelming
>>>>> controller.
>>>>>>>>> - KAFKA-2336 <
>>>> https://issues.apache.org/jira/browse/KAFKA-2336
>>>>>> :
>>>>>>>>> Changing offsets.topic.num.partitions after the offset topic
>>>> is
>>>>>>>> created
>>>>>>>>> breaks consumer group partition assignment
>>>>>>>>> - KAFKA-2337 <
>>>> https://issues.apache.org/jira/browse/KAFKA-2337
>>>>>> :
>>>>>>>> Verify
>>>>>>>>> that metric names will not collide when creating new topics
>>>>>>>>> - KAFKA-2393 <
>>>> https://issues.apache.org/jira/browse/KAFKA-2393
>>>>>> :
>>>>>>>>> Correctly Handle InvalidTopicException in
>>>>>>> KafkaApis.getTopicMetadata()
>>>>>>>>> - KAFKA-2189 <
>>>> https://issues.apache.org/jira/browse/KAFKA-2189
>>>>>> :
>>>>>>>> Snappy
>>>>>>>>> compression of message batches less efficient in 0.8.2.1
>>>>>>>>> - KAFKA-2308 <
>>>> https://issues.apache.org/jira/browse/KAFKA-2308
>>>>>> :
>>>>>>> New
>>>>>>>>> producer + Snappy face un-compression errors after broker
>>>>> restart
>>>>>>>>> - KAFKA-2042 <
>>>> https://issues.apache.org/jira/browse/KAFKA-2042
>>>>>> :
>>>>>>> New
>>>>>>>>> producer metadata update always get all topics.
>>>>>>>>> - KAFKA-1367 <
>>>> https://issues.apache.org/jira/browse/KAFKA-1367
>>>>>> :
>>>>>>>> Broker
>>>>>>>>> topic metadata not kept in sync with ZooKeeper
>>>>>>>>> - KAFKA-972 <https://issues.apache.org/jira/browse/KAFKA-972
>>>>> :
>>>>>>>> MetadataRequest
>>>>>>>>> returns stale list of brokers
>>>>>>>>> - KAFKA-1867 <
>>>> https://issues.apache.org/jira/browse/KAFKA-1867
>>>>>> :
>>>>>>>> liveBroker
>>>>>>>>> list not updated on a cluster with no topics
>>>>>>>>> - KAFKA-1650 <
>>>> https://issues.apache.org/jira/browse/KAFKA-1650
>>>>>> :
>>>>>>>> Mirror
>>>>>>>>> Maker could lose data on unclean shutdown.
>>>>>>>>> - KAFKA-2009 <
>>>> https://issues.apache.org/jira/browse/KAFKA-2009
>>>>>> :
>>>>>>> Fix
>>>>>>>>> UncheckedOffset.removeOffset synchronization and trace logging
>>>>>> issue
>>>>>>>> in
>>>>>>>>> mirror maker
>>>>>>>>> - KAFKA-2407 <
>>>> https://issues.apache.org/jira/browse/KAFKA-2407
>>>>>> :
>>>>>>> Only
>>>>>>>>> create a log directory when it will be used
>>>>>>>>> - KAFKA-2327 <
>>>> https://issues.apache.org/jira/browse/KAFKA-2327
>>>>>> :
>>>>>>>>> broker doesn't start if config defines advertised.host but not
>>>>>>>>> advertised.port
>>>>>>>>> - KAFKA-1788: producer record can stay in RecordAccumulator
>>>>>> forever
>>>>>>> if
>>>>>>>>> leader is no available
>>>>>>>>> - KAFKA-2234 <
>>>> https://issues.apache.org/jira/browse/KAFKA-2234
>>>>>> :
>>>>>>>>> Partition reassignment of a nonexistent topic prevents future
>>>>>>>> reassignments
>>>>>>>>> - KAFKA-2096 <
>>>> https://issues.apache.org/jira/browse/KAFKA-2096
>>>>>> :
>>>>>>>>> Enable keepalive socket option for broker to prevent socket
>>>> leak
>>>>>>>>> - KAFKA-1057 <
>>>> https://issues.apache.org/jira/browse/KAFKA-1057
>>>>>> :
>>>>>>> Trim
>>>>>>>>> whitespaces from user specified configs
>>>>>>>>> - KAFKA-1641 <
>>>> https://issues.apache.org/jira/browse/KAFKA-1641
>>>>>> :
>>>>>>> Log
>>>>>>>>> cleaner exits if last cleaned offset is lower than earliest
>>>>> offset
>>>>>>>>> - KAFKA-1648 <
>>>> https://issues.apache.org/jira/browse/KAFKA-1648
>>>>>> :
>>>>>>>> Round
>>>>>>>>> robin consumer balance throws an NPE when there are no topics
>>>>>>>>> - KAFKA-1724 <
>>>> https://issues.apache.org/jira/browse/KAFKA-1724
>>>>>> :
>>>>>>>>> Errors after reboot in single node setup
>>>>>>>>> - KAFKA-1758 <
>>>> https://issues.apache.org/jira/browse/KAFKA-1758
>>>>>> :
>>>>>>>>> corrupt recovery file prevents startup
>>>>>>>>> - KAFKA-1866 <
>>>> https://issues.apache.org/jira/browse/KAFKA-1866
>>>>>> :
>>>>>>>>> LogStartOffset gauge throws exceptions after log.delete()
>>>>>>>>> - KAFKA-1883 <
>>>> https://issues.apache.org/jira/browse/KAFKA-1883
>>>>>> :
>>>>>>>> NullPointerException
>>>>>>>>> in RequestSendThread
>>>>>>>>> - KAFKA-1896 <
>>>> https://issues.apache.org/jira/browse/KAFKA-1896
>>>>>> :
>>>>>>>>> Record size funcition of record in mirror maker hit NPE when
>>>> the
>>>>>>>> message
>>>>>>>>> value is null.
>>>>>>>>> - KAFKA-2101 <
>>>> https://issues.apache.org/jira/browse/KAFKA-2101
>>>>>> :
>>>>>>>>> Metric metadata-age is reset on a failed update
>>>>>>>>> - KAFKA-2112 <
>>>> https://issues.apache.org/jira/browse/KAFKA-2112
>>>>>> :
>>>>>>> make
>>>>>>>>> overflowWheel volatile
>>>>>>>>> - KAFKA-2117 <
>>>> https://issues.apache.org/jira/browse/KAFKA-2117
>>>>>> :
>>>>>>>>> OffsetManager uses incorrect field for metadata
>>>>>>>>> - KAFKA-2164 <
>>>> https://issues.apache.org/jira/browse/KAFKA-2164
>>>>>> :
>>>>>>>>> ReplicaFetcherThread: suspicious log message on reset offset
>>>>>>>>> - KAFKA-1668 <
>>>> https://issues.apache.org/jira/browse/KAFKA-1668
>>>>>> :
>>>>>>>>> TopicCommand doesn't warn if --topic argument doesn't match
>>>> any
>>>>>>> topics
>>>>>>>>> - KAFKA-2198 <
>>>> https://issues.apache.org/jira/browse/KAFKA-2198
>>>>>> :
>>>>>>>>> kafka-topics.sh exits with 0 status on failures
>>>>>>>>> - KAFKA-2235 <
>>>> https://issues.apache.org/jira/browse/KAFKA-2235
>>>>>> :
>>>>>>>>> LogCleaner offset map overflow
>>>>>>>>> - KAFKA-2241 <
>>>> https://issues.apache.org/jira/browse/KAFKA-2241
>>>>>> :
>>>>>>>>> AbstractFetcherThread.shutdown() should not block on
>>>>>>>>> ReadableByteChannel.read(buffer)
>>>>>>>>> - KAFKA-2272 <
>>>> https://issues.apache.org/jira/browse/KAFKA-2272
>>>>>> :
>>>>>>>>> listeners endpoint parsing fails if the hostname has capital
>>>>>> letter
>>>>>>>>> - KAFKA-2345 <
>>>> https://issues.apache.org/jira/browse/KAFKA-2345
>>>>>> :
>>>>>>>>> Attempt to delete a topic already marked for deletion throws
>>>>>>>>> ZkNodeExistsException
>>>>>>>>> - KAFKA-2353 <
>>>> https://issues.apache.org/jira/browse/KAFKA-2353
>>>>>> :
>>>>>>>>> SocketServer.Processor should catch exception and close the
>>>>> socket
>>>>>>>> properly
>>>>>>>>> in configureNewConnections.
>>>>>>>>> - KAFKA-1836 <
>>>> https://issues.apache.org/jira/browse/KAFKA-1836
>>>>>> :
>>>>>>>>> metadata.fetch.timeout.ms set to zero blocks forever
>>>>>>>>> - KAFKA-2317 <
>>>> https://issues.apache.org/jira/browse/KAFKA-2317
>>>>>> :
>>>>>>>> De-register
>>>>>>>>> isrChangeNotificationListener on controller resignation
>>>>>>>>>
>>>>>>>>> Note: KAFKA-2120 <
>>>> https://issues.apache.org/jira/browse/KAFKA-2120
>>>>>>
>>>>>> &
>>>>>>>>> KAFKA-2421 <https://issues.apache.org/jira/browse/KAFKA-2421>
>>>> were
>>>>>>>>> mentioned in previous emails, but are not in the list because
>>>> they
>>>>>> are
>>>>>>>> not
>>>>>>>>> committed yet.
>>>>>>>>>
>>>>>>>>> Hope that helps the effort.
>>>>>>>>>
>>>>>>>>> Thanks,
>>>>>>>>> Grant
>>>>>>>>>
>>>>>>>>> On Mon, Aug 17, 2015 at 12:09 AM, Grant Henke <
>>>> ghenke@cloudera.com
>>>>>>
>>>>>>>> wrote:
>>>>>>>>>
>>>>>>>>>> +1 to that suggestion. Though I suspect that requires a
>>>> committer
>>>>> to
>>>>>>> do.
>>>>>>>>>> Making it part of the standard commit process could work too.
>>>>>>>>>> On Aug 16, 2015 11:01 PM, "Gwen Shapira" <gw...@confluent.io>
>>>>> wrote:
>>>>>>>>>>
>>>>>>>>>>> BTW. I think it will be great for Apache Kafka to have a 0.8.2
>>>>>>> "release
>>>>>>>>>>> manager" who's role is to cherrypick low-risk bug-fixes into
>>>> the
>>>>>>> 0.8.2
>>>>>>>>>>> branch and once enough bug fixes happened (or if sufficiently
>>>>>>> critical
>>>>>>>>>>> fixes happened) to roll out a new maintenance release (with
>>>>> every 3
>>>>>>>> month
>>>>>>>>>>> as a reasonable bugfix release target).
>>>>>>>>>>>
>>>>>>>>>>> This will add some predictability regarding how fast we release
>>>>>> fixes
>>>>>>>> for
>>>>>>>>>>> bugs.
>>>>>>>>>>>
>>>>>>>>>>> Gwen
>>>>>>>>>>>
>>>>>>>>>>> On Sun, Aug 16, 2015 at 8:09 PM, Jeff Holoman <
>>>>>> jholoman@cloudera.com
>>>>>>>>
>>>>>>>>>>> wrote:
>>>>>>>>>>>
>>>>>>>>>>>> +1 for the release and also including
>>>>>>>>>>>>
>>>>>>>>>>>> https://issues.apache.org/jira/browse/KAFKA-2114
>>>>>>>>>>>>
>>>>>>>>>>>> Thanks
>>>>>>>>>>>>
>>>>>>>>>>>> Jeff
>>>>>>>>>>>>
>>>>>>>>>>>> On Sun, Aug 16, 2015 at 2:51 PM, Stevo Slavić <
>>>>> sslavic@gmail.com
>>>>>>>
>>>>>>>>>>> wrote:
>>>>>>>>>>>>
>>>>>>>>>>>>> +1 (non-binding) for 0.8.2.2 release
>>>>>>>>>>>>>
>>>>>>>>>>>>> Would be nice to include in that release new producer
>>>>>> resiliency
>>>>>>>> bug
>>>>>>>>>>>> fixes
>>>>>>>>>>>>> https://issues.apache.org/jira/browse/KAFKA-1788 and
>>>>>>>>>>>>> https://issues.apache.org/jira/browse/KAFKA-2120
>>>>>>>>>>>>>
>>>>>>>>>>>>> On Fri, Aug 14, 2015 at 4:03 PM, Gwen Shapira <
>>>>>> gwen@confluent.io
>>>>>>>>
>>>>>>>>>>> wrote:
>>>>>>>>>>>>>
>>>>>>>>>>>>>> Will be nice to include Kafka-2308 and fix two critical
>>>>>> snappy
>>>>>>>>>>> issues
>>>>>>>>>>>> in
>>>>>>>>>>>>>> the maintenance release.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Gwen
>>>>>>>>>>>>>> On Aug 14, 2015 6:16 AM, "Grant Henke" <
>>>>> ghenke@cloudera.com>
>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Just to clarify. Will KAFKA-2189 be the only patch in
>>>> the
>>>>>>>>>>> release?
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> On Fri, Aug 14, 2015 at 7:35 AM, Manikumar Reddy <
>>>>>>>>>>>> kumar@nmsworks.co.in
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> +1 for 0.8.2.2 release
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> On Fri, Aug 14, 2015 at 5:49 PM, Ismael Juma <
>>>>>>>>>>> ismael@juma.me.uk>
>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> I think this is a good idea as the change is
>>>> minimal
>>>>> on
>>>>>>> our
>>>>>>>>>>> side
>>>>>>>>>>>>> and
>>>>>>>>>>>>>> it
>>>>>>>>>>>>>>>> has
>>>>>>>>>>>>>>>>> been tested in production for some time by the
>>>>>> reporter.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Best,
>>>>>>>>>>>>>>>>> Ismael
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> On Fri, Aug 14, 2015 at 1:15 PM, Jun Rao <
>>>>>>> jun@confluent.io
>>>>>>>>>
>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> Hi, Everyone,
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> Since the release of Kafka 0.8.2.1, a number of
>>>>>> people
>>>>>>>> have
>>>>>>>>>>>>>> reported
>>>>>>>>>>>>>>> an
>>>>>>>>>>>>>>>>>> issue with snappy compression (
>>>>>>>>>>>>>>>>>> https://issues.apache.org/jira/browse/KAFKA-2189
>>>> ).
>>>>>>>>>>> Basically,
>>>>>>>>>>>> if
>>>>>>>>>>>>>>> they
>>>>>>>>>>>>>>>>> use
>>>>>>>>>>>>>>>>>> snappy in 0.8.2.1, they will experience a 2-3X
>>>>> space
>>>>>>>>>>> increase.
>>>>>>>>>>>>> The
>>>>>>>>>>>>>>>> issue
>>>>>>>>>>>>>>>>>> has since been fixed in trunk (just a snappy jar
>>>>>>>> upgrade).
>>>>>>>>>>>> Since
>>>>>>>>>>>>>>> 0.8.3
>>>>>>>>>>>>>>>> is
>>>>>>>>>>>>>>>>>> still a few months away, it may make sense to do
>>>> an
>>>>>>>> 0.8.2.2
>>>>>>>>>>>>> release
>>>>>>>>>>>>>>>> just
>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>> fix this issue. Any objections?
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> Jun
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>> Grant Henke
>>>>>>>>>>>>>>> Software Engineer | Cloudera
>>>>>>>>>>>>>>> grant@cloudera.com | twitter.com/gchenke |
>>>>>>>>>>>> linkedin.com/in/granthenke
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> --
>>>>>>>>>>>> Jeff Holoman
>>>>>>>>>>>> Systems Engineer
>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> --
>>>>>>>>> Grant Henke
>>>>>>>>> Software Engineer | Cloudera
>>>>>>>>> grant@cloudera.com | twitter.com/gchenke |
>>>>>> linkedin.com/in/granthenke
>>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> --
>>>>>>>> Grant Henke
>>>>>>>> Software Engineer | Cloudera
>>>>>>>> grant@cloudera.com | twitter.com/gchenke |
>>>>> linkedin.com/in/granthenke
>>>>>>>>
>>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> --
>>>>>> Grant Henke
>>>>>> Software Engineer | Cloudera
>>>>>> grant@cloudera.com | twitter.com/gchenke | linkedin.com/in/granthenke
>>>>>>
>>>>>
>>>>
>>>
>>>
>>>
>>> --
>>> Thanks,
>>> Neha
>>
>>
Re: [DISCUSSION] Kafka 0.8.2.2 release?
Posted by Edward Ribeiro <ed...@gmail.com>.
I sort of follow Daniel Nelson on this issue: cut a 0.8.2.2 but not include
much thing besides Snappy fixes. I mean, include a couple of additional
critical bug fixes, if really urgent, and that's it.
On Tue, Aug 18, 2015 at 3:25 PM, Daniel Nelson <da...@vungle.com>
wrote:
> I am strongly in favor of cutting a 0.8.2.2 release, but I don’t think
> that it needs to include anything other than the fix for Snappy that kicked
> off this discussion in the first place. If there are additional critical
> issues that can be included without delaying the process, I see no downside.
>
> > On Aug 18, 2015, at 10:54 AM, Neha Narkhede <ne...@confluent.io> wrote:
> >
> > How about looking at the scope for the 0.8.3 release first before we cut
> > yet another point release off of 0.8.2.2? Each release includes some
> > overhead and if there is a larger release in the works, it might be worth
> > working on getting that. My take is that the 2 things the community has
> > been waiting for is SSL support and the new consumer and we have been
> > promising to get 0.8.3 with both those features for several months now.
> >
> > Looking at the progress on both, it seems we are very close to getting
> both
> > those checked in and it looks like we should get there in another 5-6
> > weeks. Furthermore, both of these features are large and I anticipate us
> > receiving feedback and bugs that will require a couple of point releases
> on
> > top of 0.8.3 anyway. One possibility is to work on 0.8.3 together now and
> > get the community to use the newly released features, gather feedback and
> > do point releases incorporating that feedback and iterate on it.
> >
> > We could absolutely do both 0.8.2.2 and 0.8.3. What I'd ask for is for us
> > to look at the 0.8.3 timeline too and make a call whether 0.8.2.2 still
> > makes sense.
> >
> > Thanks,
> > Neha
> >
> > On Tue, Aug 18, 2015 at 10:24 AM, Gwen Shapira <gw...@confluent.io>
> wrote:
> >
> >> Thanks Jun.
> >>
> >> I updated the list with your suggestions.
> >> If anyone feels we are missing a critical patch for 0.8.2.2, please
> speak
> >> up.
> >>
> >> Gwen
> >>
> >> On Mon, Aug 17, 2015 at 5:40 PM, Jun Rao <ju...@confluent.io> wrote:
> >>
> >>> Hi, Grant,
> >>>
> >>> I took a look at that list. None of those is really critical as you
> said.
> >>> So, I'd suggest that we not include those to minimize the scope of the
> >>> release.
> >>>
> >>> Thanks,
> >>>
> >>> Jun
> >>>
> >>> On Mon, Aug 17, 2015 at 5:16 PM, Grant Henke <gh...@cloudera.com>
> >> wrote:
> >>>
> >>>> Thanks Gwen.
> >>>>
> >>>> I updated a few small things on the wiki page.
> >>>>
> >>>> Below is a list of jiras I think could also be marked as included. All
> >> of
> >>>> these, though not super critical, seem like fairly small and low risk
> >>>> changes that help avoid potentially confusing issues or errors for
> >> users.
> >>>>
> >>>> KAFKA-2012
> >>>> KAFKA-972
> >>>> KAFKA-2337 & KAFKA-2393
> >>>> KAFKA-1867
> >>>> KAFKA-2407
> >>>> KAFKA-2234
> >>>> KAFKA-1866
> >>>> KAFKA-2345 & KAFKA-2355
> >>>>
> >>>> thoughts?
> >>>>
> >>>> Thank you,
> >>>> Grant
> >>>>
> >>>> On Mon, Aug 17, 2015 at 4:56 PM, Gwen Shapira <gw...@confluent.io>
> >> wrote:
> >>>>
> >>>>> Thanks for creating a list, Grant!
> >>>>>
> >>>>> I placed it on the wiki with a quick evaluation of the content and
> >>>> whether
> >>>>> it should be in 0.8.2.2:
> >>>>>
> >>>>>
> >>>>
> >>>
> >>
> https://cwiki.apache.org/confluence/display/KAFKA/Proposed+patches+for+0.8.2.2
> >>>>>
> >>>>> I'm attempting to only cherrypick fixes that are both important for
> >>> large
> >>>>> number of users (or very critical to some users) and very safe
> >> (mostly
> >>>>> judged by the size of the change, but not only)
> >>>>>
> >>>>> If your favorite bugfix is missing from the list, or is there but
> >>> marked
> >>>>> "No", please let us know (in this thread) what we are missing and why
> >>> it
> >>>> is
> >>>>> both important and safe.
> >>>>> Also, if I accidentally included something you consider unsafe, speak
> >>> up!
> >>>>>
> >>>>> Gwen
> >>>>>
> >>>>> On Mon, Aug 17, 2015 at 8:17 AM, Grant Henke <gh...@cloudera.com>
> >>>> wrote:
> >>>>>
> >>>>>> +dev
> >>>>>>
> >>>>>> Adding dev list back in. Somehow it got dropped.
> >>>>>>
> >>>>>>
> >>>>>> On Mon, Aug 17, 2015 at 10:16 AM, Grant Henke <ghenke@cloudera.com
> >>>
> >>>>> wrote:
> >>>>>>
> >>>>>>> Below is a list of candidate bug fix jiras marked fixed for
> >> 0.8.3.
> >>> I
> >>>>>> don't
> >>>>>>> suspect all of these will (or should) make it into the release
> >> but
> >>>> this
> >>>>>>> should be a relatively complete list to work from:
> >>>>>>>
> >>>>>>> - KAFKA-2114 <
> >> https://issues.apache.org/jira/browse/KAFKA-2114
> >>>> :
> >>>>>> Unable
> >>>>>>> to change min.insync.replicas default
> >>>>>>> - KAFKA-1702 <
> >> https://issues.apache.org/jira/browse/KAFKA-1702
> >>>> :
> >>>>>>> Messages silently Lost by producer
> >>>>>>> - KAFKA-2012 <
> >> https://issues.apache.org/jira/browse/KAFKA-2012
> >>>> :
> >>>>>>> Broker should automatically handle corrupt index files
> >>>>>>> - KAFKA-2406 <
> >> https://issues.apache.org/jira/browse/KAFKA-2406
> >>>> :
> >>>>> ISR
> >>>>>>> propagation should be throttled to avoid overwhelming
> >>> controller.
> >>>>>>> - KAFKA-2336 <
> >> https://issues.apache.org/jira/browse/KAFKA-2336
> >>>> :
> >>>>>>> Changing offsets.topic.num.partitions after the offset topic
> >> is
> >>>>>> created
> >>>>>>> breaks consumer group partition assignment
> >>>>>>> - KAFKA-2337 <
> >> https://issues.apache.org/jira/browse/KAFKA-2337
> >>>> :
> >>>>>> Verify
> >>>>>>> that metric names will not collide when creating new topics
> >>>>>>> - KAFKA-2393 <
> >> https://issues.apache.org/jira/browse/KAFKA-2393
> >>>> :
> >>>>>>> Correctly Handle InvalidTopicException in
> >>>>> KafkaApis.getTopicMetadata()
> >>>>>>> - KAFKA-2189 <
> >> https://issues.apache.org/jira/browse/KAFKA-2189
> >>>> :
> >>>>>> Snappy
> >>>>>>> compression of message batches less efficient in 0.8.2.1
> >>>>>>> - KAFKA-2308 <
> >> https://issues.apache.org/jira/browse/KAFKA-2308
> >>>> :
> >>>>> New
> >>>>>>> producer + Snappy face un-compression errors after broker
> >>> restart
> >>>>>>> - KAFKA-2042 <
> >> https://issues.apache.org/jira/browse/KAFKA-2042
> >>>> :
> >>>>> New
> >>>>>>> producer metadata update always get all topics.
> >>>>>>> - KAFKA-1367 <
> >> https://issues.apache.org/jira/browse/KAFKA-1367
> >>>> :
> >>>>>> Broker
> >>>>>>> topic metadata not kept in sync with ZooKeeper
> >>>>>>> - KAFKA-972 <https://issues.apache.org/jira/browse/KAFKA-972
> >>> :
> >>>>>> MetadataRequest
> >>>>>>> returns stale list of brokers
> >>>>>>> - KAFKA-1867 <
> >> https://issues.apache.org/jira/browse/KAFKA-1867
> >>>> :
> >>>>>> liveBroker
> >>>>>>> list not updated on a cluster with no topics
> >>>>>>> - KAFKA-1650 <
> >> https://issues.apache.org/jira/browse/KAFKA-1650
> >>>> :
> >>>>>> Mirror
> >>>>>>> Maker could lose data on unclean shutdown.
> >>>>>>> - KAFKA-2009 <
> >> https://issues.apache.org/jira/browse/KAFKA-2009
> >>>> :
> >>>>> Fix
> >>>>>>> UncheckedOffset.removeOffset synchronization and trace logging
> >>>> issue
> >>>>>> in
> >>>>>>> mirror maker
> >>>>>>> - KAFKA-2407 <
> >> https://issues.apache.org/jira/browse/KAFKA-2407
> >>>> :
> >>>>> Only
> >>>>>>> create a log directory when it will be used
> >>>>>>> - KAFKA-2327 <
> >> https://issues.apache.org/jira/browse/KAFKA-2327
> >>>> :
> >>>>>>> broker doesn't start if config defines advertised.host but not
> >>>>>>> advertised.port
> >>>>>>> - KAFKA-1788: producer record can stay in RecordAccumulator
> >>>> forever
> >>>>> if
> >>>>>>> leader is no available
> >>>>>>> - KAFKA-2234 <
> >> https://issues.apache.org/jira/browse/KAFKA-2234
> >>>> :
> >>>>>>> Partition reassignment of a nonexistent topic prevents future
> >>>>>> reassignments
> >>>>>>> - KAFKA-2096 <
> >> https://issues.apache.org/jira/browse/KAFKA-2096
> >>>> :
> >>>>>>> Enable keepalive socket option for broker to prevent socket
> >> leak
> >>>>>>> - KAFKA-1057 <
> >> https://issues.apache.org/jira/browse/KAFKA-1057
> >>>> :
> >>>>> Trim
> >>>>>>> whitespaces from user specified configs
> >>>>>>> - KAFKA-1641 <
> >> https://issues.apache.org/jira/browse/KAFKA-1641
> >>>> :
> >>>>> Log
> >>>>>>> cleaner exits if last cleaned offset is lower than earliest
> >>> offset
> >>>>>>> - KAFKA-1648 <
> >> https://issues.apache.org/jira/browse/KAFKA-1648
> >>>> :
> >>>>>> Round
> >>>>>>> robin consumer balance throws an NPE when there are no topics
> >>>>>>> - KAFKA-1724 <
> >> https://issues.apache.org/jira/browse/KAFKA-1724
> >>>> :
> >>>>>>> Errors after reboot in single node setup
> >>>>>>> - KAFKA-1758 <
> >> https://issues.apache.org/jira/browse/KAFKA-1758
> >>>> :
> >>>>>>> corrupt recovery file prevents startup
> >>>>>>> - KAFKA-1866 <
> >> https://issues.apache.org/jira/browse/KAFKA-1866
> >>>> :
> >>>>>>> LogStartOffset gauge throws exceptions after log.delete()
> >>>>>>> - KAFKA-1883 <
> >> https://issues.apache.org/jira/browse/KAFKA-1883
> >>>> :
> >>>>>> NullPointerException
> >>>>>>> in RequestSendThread
> >>>>>>> - KAFKA-1896 <
> >> https://issues.apache.org/jira/browse/KAFKA-1896
> >>>> :
> >>>>>>> Record size funcition of record in mirror maker hit NPE when
> >> the
> >>>>>> message
> >>>>>>> value is null.
> >>>>>>> - KAFKA-2101 <
> >> https://issues.apache.org/jira/browse/KAFKA-2101
> >>>> :
> >>>>>>> Metric metadata-age is reset on a failed update
> >>>>>>> - KAFKA-2112 <
> >> https://issues.apache.org/jira/browse/KAFKA-2112
> >>>> :
> >>>>> make
> >>>>>>> overflowWheel volatile
> >>>>>>> - KAFKA-2117 <
> >> https://issues.apache.org/jira/browse/KAFKA-2117
> >>>> :
> >>>>>>> OffsetManager uses incorrect field for metadata
> >>>>>>> - KAFKA-2164 <
> >> https://issues.apache.org/jira/browse/KAFKA-2164
> >>>> :
> >>>>>>> ReplicaFetcherThread: suspicious log message on reset offset
> >>>>>>> - KAFKA-1668 <
> >> https://issues.apache.org/jira/browse/KAFKA-1668
> >>>> :
> >>>>>>> TopicCommand doesn't warn if --topic argument doesn't match
> >> any
> >>>>> topics
> >>>>>>> - KAFKA-2198 <
> >> https://issues.apache.org/jira/browse/KAFKA-2198
> >>>> :
> >>>>>>> kafka-topics.sh exits with 0 status on failures
> >>>>>>> - KAFKA-2235 <
> >> https://issues.apache.org/jira/browse/KAFKA-2235
> >>>> :
> >>>>>>> LogCleaner offset map overflow
> >>>>>>> - KAFKA-2241 <
> >> https://issues.apache.org/jira/browse/KAFKA-2241
> >>>> :
> >>>>>>> AbstractFetcherThread.shutdown() should not block on
> >>>>>>> ReadableByteChannel.read(buffer)
> >>>>>>> - KAFKA-2272 <
> >> https://issues.apache.org/jira/browse/KAFKA-2272
> >>>> :
> >>>>>>> listeners endpoint parsing fails if the hostname has capital
> >>>> letter
> >>>>>>> - KAFKA-2345 <
> >> https://issues.apache.org/jira/browse/KAFKA-2345
> >>>> :
> >>>>>>> Attempt to delete a topic already marked for deletion throws
> >>>>>>> ZkNodeExistsException
> >>>>>>> - KAFKA-2353 <
> >> https://issues.apache.org/jira/browse/KAFKA-2353
> >>>> :
> >>>>>>> SocketServer.Processor should catch exception and close the
> >>> socket
> >>>>>> properly
> >>>>>>> in configureNewConnections.
> >>>>>>> - KAFKA-1836 <
> >> https://issues.apache.org/jira/browse/KAFKA-1836
> >>>> :
> >>>>>>> metadata.fetch.timeout.ms set to zero blocks forever
> >>>>>>> - KAFKA-2317 <
> >> https://issues.apache.org/jira/browse/KAFKA-2317
> >>>> :
> >>>>>> De-register
> >>>>>>> isrChangeNotificationListener on controller resignation
> >>>>>>>
> >>>>>>> Note: KAFKA-2120 <
> >> https://issues.apache.org/jira/browse/KAFKA-2120
> >>>>
> >>>> &
> >>>>>>> KAFKA-2421 <https://issues.apache.org/jira/browse/KAFKA-2421>
> >> were
> >>>>>>> mentioned in previous emails, but are not in the list because
> >> they
> >>>> are
> >>>>>> not
> >>>>>>> committed yet.
> >>>>>>>
> >>>>>>> Hope that helps the effort.
> >>>>>>>
> >>>>>>> Thanks,
> >>>>>>> Grant
> >>>>>>>
> >>>>>>> On Mon, Aug 17, 2015 at 12:09 AM, Grant Henke <
> >> ghenke@cloudera.com
> >>>>
> >>>>>> wrote:
> >>>>>>>
> >>>>>>>> +1 to that suggestion. Though I suspect that requires a
> >> committer
> >>> to
> >>>>> do.
> >>>>>>>> Making it part of the standard commit process could work too.
> >>>>>>>> On Aug 16, 2015 11:01 PM, "Gwen Shapira" <gw...@confluent.io>
> >>> wrote:
> >>>>>>>>
> >>>>>>>>> BTW. I think it will be great for Apache Kafka to have a 0.8.2
> >>>>> "release
> >>>>>>>>> manager" who's role is to cherrypick low-risk bug-fixes into
> >> the
> >>>>> 0.8.2
> >>>>>>>>> branch and once enough bug fixes happened (or if sufficiently
> >>>>> critical
> >>>>>>>>> fixes happened) to roll out a new maintenance release (with
> >>> every 3
> >>>>>> month
> >>>>>>>>> as a reasonable bugfix release target).
> >>>>>>>>>
> >>>>>>>>> This will add some predictability regarding how fast we release
> >>>> fixes
> >>>>>> for
> >>>>>>>>> bugs.
> >>>>>>>>>
> >>>>>>>>> Gwen
> >>>>>>>>>
> >>>>>>>>> On Sun, Aug 16, 2015 at 8:09 PM, Jeff Holoman <
> >>>> jholoman@cloudera.com
> >>>>>>
> >>>>>>>>> wrote:
> >>>>>>>>>
> >>>>>>>>>> +1 for the release and also including
> >>>>>>>>>>
> >>>>>>>>>> https://issues.apache.org/jira/browse/KAFKA-2114
> >>>>>>>>>>
> >>>>>>>>>> Thanks
> >>>>>>>>>>
> >>>>>>>>>> Jeff
> >>>>>>>>>>
> >>>>>>>>>> On Sun, Aug 16, 2015 at 2:51 PM, Stevo Slavić <
> >>> sslavic@gmail.com
> >>>>>
> >>>>>>>>> wrote:
> >>>>>>>>>>
> >>>>>>>>>>> +1 (non-binding) for 0.8.2.2 release
> >>>>>>>>>>>
> >>>>>>>>>>> Would be nice to include in that release new producer
> >>>> resiliency
> >>>>>> bug
> >>>>>>>>>> fixes
> >>>>>>>>>>> https://issues.apache.org/jira/browse/KAFKA-1788 and
> >>>>>>>>>>> https://issues.apache.org/jira/browse/KAFKA-2120
> >>>>>>>>>>>
> >>>>>>>>>>> On Fri, Aug 14, 2015 at 4:03 PM, Gwen Shapira <
> >>>> gwen@confluent.io
> >>>>>>
> >>>>>>>>> wrote:
> >>>>>>>>>>>
> >>>>>>>>>>>> Will be nice to include Kafka-2308 and fix two critical
> >>>> snappy
> >>>>>>>>> issues
> >>>>>>>>>> in
> >>>>>>>>>>>> the maintenance release.
> >>>>>>>>>>>>
> >>>>>>>>>>>> Gwen
> >>>>>>>>>>>> On Aug 14, 2015 6:16 AM, "Grant Henke" <
> >>> ghenke@cloudera.com>
> >>>>>>>>> wrote:
> >>>>>>>>>>>>
> >>>>>>>>>>>>> Just to clarify. Will KAFKA-2189 be the only patch in
> >> the
> >>>>>>>>> release?
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> On Fri, Aug 14, 2015 at 7:35 AM, Manikumar Reddy <
> >>>>>>>>>> kumar@nmsworks.co.in
> >>>>>>>>>>>>
> >>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>> +1 for 0.8.2.2 release
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> On Fri, Aug 14, 2015 at 5:49 PM, Ismael Juma <
> >>>>>>>>> ismael@juma.me.uk>
> >>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> I think this is a good idea as the change is
> >> minimal
> >>> on
> >>>>> our
> >>>>>>>>> side
> >>>>>>>>>>> and
> >>>>>>>>>>>> it
> >>>>>>>>>>>>>> has
> >>>>>>>>>>>>>>> been tested in production for some time by the
> >>>> reporter.
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> Best,
> >>>>>>>>>>>>>>> Ismael
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> On Fri, Aug 14, 2015 at 1:15 PM, Jun Rao <
> >>>>> jun@confluent.io
> >>>>>>>
> >>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> Hi, Everyone,
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> Since the release of Kafka 0.8.2.1, a number of
> >>>> people
> >>>>>> have
> >>>>>>>>>>>> reported
> >>>>>>>>>>>>> an
> >>>>>>>>>>>>>>>> issue with snappy compression (
> >>>>>>>>>>>>>>>> https://issues.apache.org/jira/browse/KAFKA-2189
> >> ).
> >>>>>>>>> Basically,
> >>>>>>>>>> if
> >>>>>>>>>>>>> they
> >>>>>>>>>>>>>>> use
> >>>>>>>>>>>>>>>> snappy in 0.8.2.1, they will experience a 2-3X
> >>> space
> >>>>>>>>> increase.
> >>>>>>>>>>> The
> >>>>>>>>>>>>>> issue
> >>>>>>>>>>>>>>>> has since been fixed in trunk (just a snappy jar
> >>>>>> upgrade).
> >>>>>>>>>> Since
> >>>>>>>>>>>>> 0.8.3
> >>>>>>>>>>>>>> is
> >>>>>>>>>>>>>>>> still a few months away, it may make sense to do
> >> an
> >>>>>> 0.8.2.2
> >>>>>>>>>>> release
> >>>>>>>>>>>>>> just
> >>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>> fix this issue. Any objections?
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> Thanks,
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> Jun
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> --
> >>>>>>>>>>>>> Grant Henke
> >>>>>>>>>>>>> Software Engineer | Cloudera
> >>>>>>>>>>>>> grant@cloudera.com | twitter.com/gchenke |
> >>>>>>>>>> linkedin.com/in/granthenke
> >>>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>> --
> >>>>>>>>>> Jeff Holoman
> >>>>>>>>>> Systems Engineer
> >>>>>>>>>>
> >>>>>>>>>
> >>>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>> --
> >>>>>>> Grant Henke
> >>>>>>> Software Engineer | Cloudera
> >>>>>>> grant@cloudera.com | twitter.com/gchenke |
> >>>> linkedin.com/in/granthenke
> >>>>>>>
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>> --
> >>>>>> Grant Henke
> >>>>>> Software Engineer | Cloudera
> >>>>>> grant@cloudera.com | twitter.com/gchenke |
> >>> linkedin.com/in/granthenke
> >>>>>>
> >>>>>
> >>>>
> >>>>
> >>>>
> >>>> --
> >>>> Grant Henke
> >>>> Software Engineer | Cloudera
> >>>> grant@cloudera.com | twitter.com/gchenke | linkedin.com/in/granthenke
> >>>>
> >>>
> >>
> >
> >
> >
> > --
> > Thanks,
> > Neha
>
>
Re: [DISCUSSION] Kafka 0.8.2.2 release?
Posted by Edward Ribeiro <ed...@gmail.com>.
I sort of follow Daniel Nelson on this issue: cut a 0.8.2.2 but not include
much thing besides Snappy fixes. I mean, include a couple of additional
critical bug fixes, if really urgent, and that's it.
On Tue, Aug 18, 2015 at 3:25 PM, Daniel Nelson <da...@vungle.com>
wrote:
> I am strongly in favor of cutting a 0.8.2.2 release, but I don’t think
> that it needs to include anything other than the fix for Snappy that kicked
> off this discussion in the first place. If there are additional critical
> issues that can be included without delaying the process, I see no downside.
>
> > On Aug 18, 2015, at 10:54 AM, Neha Narkhede <ne...@confluent.io> wrote:
> >
> > How about looking at the scope for the 0.8.3 release first before we cut
> > yet another point release off of 0.8.2.2? Each release includes some
> > overhead and if there is a larger release in the works, it might be worth
> > working on getting that. My take is that the 2 things the community has
> > been waiting for is SSL support and the new consumer and we have been
> > promising to get 0.8.3 with both those features for several months now.
> >
> > Looking at the progress on both, it seems we are very close to getting
> both
> > those checked in and it looks like we should get there in another 5-6
> > weeks. Furthermore, both of these features are large and I anticipate us
> > receiving feedback and bugs that will require a couple of point releases
> on
> > top of 0.8.3 anyway. One possibility is to work on 0.8.3 together now and
> > get the community to use the newly released features, gather feedback and
> > do point releases incorporating that feedback and iterate on it.
> >
> > We could absolutely do both 0.8.2.2 and 0.8.3. What I'd ask for is for us
> > to look at the 0.8.3 timeline too and make a call whether 0.8.2.2 still
> > makes sense.
> >
> > Thanks,
> > Neha
> >
> > On Tue, Aug 18, 2015 at 10:24 AM, Gwen Shapira <gw...@confluent.io>
> wrote:
> >
> >> Thanks Jun.
> >>
> >> I updated the list with your suggestions.
> >> If anyone feels we are missing a critical patch for 0.8.2.2, please
> speak
> >> up.
> >>
> >> Gwen
> >>
> >> On Mon, Aug 17, 2015 at 5:40 PM, Jun Rao <ju...@confluent.io> wrote:
> >>
> >>> Hi, Grant,
> >>>
> >>> I took a look at that list. None of those is really critical as you
> said.
> >>> So, I'd suggest that we not include those to minimize the scope of the
> >>> release.
> >>>
> >>> Thanks,
> >>>
> >>> Jun
> >>>
> >>> On Mon, Aug 17, 2015 at 5:16 PM, Grant Henke <gh...@cloudera.com>
> >> wrote:
> >>>
> >>>> Thanks Gwen.
> >>>>
> >>>> I updated a few small things on the wiki page.
> >>>>
> >>>> Below is a list of jiras I think could also be marked as included. All
> >> of
> >>>> these, though not super critical, seem like fairly small and low risk
> >>>> changes that help avoid potentially confusing issues or errors for
> >> users.
> >>>>
> >>>> KAFKA-2012
> >>>> KAFKA-972
> >>>> KAFKA-2337 & KAFKA-2393
> >>>> KAFKA-1867
> >>>> KAFKA-2407
> >>>> KAFKA-2234
> >>>> KAFKA-1866
> >>>> KAFKA-2345 & KAFKA-2355
> >>>>
> >>>> thoughts?
> >>>>
> >>>> Thank you,
> >>>> Grant
> >>>>
> >>>> On Mon, Aug 17, 2015 at 4:56 PM, Gwen Shapira <gw...@confluent.io>
> >> wrote:
> >>>>
> >>>>> Thanks for creating a list, Grant!
> >>>>>
> >>>>> I placed it on the wiki with a quick evaluation of the content and
> >>>> whether
> >>>>> it should be in 0.8.2.2:
> >>>>>
> >>>>>
> >>>>
> >>>
> >>
> https://cwiki.apache.org/confluence/display/KAFKA/Proposed+patches+for+0.8.2.2
> >>>>>
> >>>>> I'm attempting to only cherrypick fixes that are both important for
> >>> large
> >>>>> number of users (or very critical to some users) and very safe
> >> (mostly
> >>>>> judged by the size of the change, but not only)
> >>>>>
> >>>>> If your favorite bugfix is missing from the list, or is there but
> >>> marked
> >>>>> "No", please let us know (in this thread) what we are missing and why
> >>> it
> >>>> is
> >>>>> both important and safe.
> >>>>> Also, if I accidentally included something you consider unsafe, speak
> >>> up!
> >>>>>
> >>>>> Gwen
> >>>>>
> >>>>> On Mon, Aug 17, 2015 at 8:17 AM, Grant Henke <gh...@cloudera.com>
> >>>> wrote:
> >>>>>
> >>>>>> +dev
> >>>>>>
> >>>>>> Adding dev list back in. Somehow it got dropped.
> >>>>>>
> >>>>>>
> >>>>>> On Mon, Aug 17, 2015 at 10:16 AM, Grant Henke <ghenke@cloudera.com
> >>>
> >>>>> wrote:
> >>>>>>
> >>>>>>> Below is a list of candidate bug fix jiras marked fixed for
> >> 0.8.3.
> >>> I
> >>>>>> don't
> >>>>>>> suspect all of these will (or should) make it into the release
> >> but
> >>>> this
> >>>>>>> should be a relatively complete list to work from:
> >>>>>>>
> >>>>>>> - KAFKA-2114 <
> >> https://issues.apache.org/jira/browse/KAFKA-2114
> >>>> :
> >>>>>> Unable
> >>>>>>> to change min.insync.replicas default
> >>>>>>> - KAFKA-1702 <
> >> https://issues.apache.org/jira/browse/KAFKA-1702
> >>>> :
> >>>>>>> Messages silently Lost by producer
> >>>>>>> - KAFKA-2012 <
> >> https://issues.apache.org/jira/browse/KAFKA-2012
> >>>> :
> >>>>>>> Broker should automatically handle corrupt index files
> >>>>>>> - KAFKA-2406 <
> >> https://issues.apache.org/jira/browse/KAFKA-2406
> >>>> :
> >>>>> ISR
> >>>>>>> propagation should be throttled to avoid overwhelming
> >>> controller.
> >>>>>>> - KAFKA-2336 <
> >> https://issues.apache.org/jira/browse/KAFKA-2336
> >>>> :
> >>>>>>> Changing offsets.topic.num.partitions after the offset topic
> >> is
> >>>>>> created
> >>>>>>> breaks consumer group partition assignment
> >>>>>>> - KAFKA-2337 <
> >> https://issues.apache.org/jira/browse/KAFKA-2337
> >>>> :
> >>>>>> Verify
> >>>>>>> that metric names will not collide when creating new topics
> >>>>>>> - KAFKA-2393 <
> >> https://issues.apache.org/jira/browse/KAFKA-2393
> >>>> :
> >>>>>>> Correctly Handle InvalidTopicException in
> >>>>> KafkaApis.getTopicMetadata()
> >>>>>>> - KAFKA-2189 <
> >> https://issues.apache.org/jira/browse/KAFKA-2189
> >>>> :
> >>>>>> Snappy
> >>>>>>> compression of message batches less efficient in 0.8.2.1
> >>>>>>> - KAFKA-2308 <
> >> https://issues.apache.org/jira/browse/KAFKA-2308
> >>>> :
> >>>>> New
> >>>>>>> producer + Snappy face un-compression errors after broker
> >>> restart
> >>>>>>> - KAFKA-2042 <
> >> https://issues.apache.org/jira/browse/KAFKA-2042
> >>>> :
> >>>>> New
> >>>>>>> producer metadata update always get all topics.
> >>>>>>> - KAFKA-1367 <
> >> https://issues.apache.org/jira/browse/KAFKA-1367
> >>>> :
> >>>>>> Broker
> >>>>>>> topic metadata not kept in sync with ZooKeeper
> >>>>>>> - KAFKA-972 <https://issues.apache.org/jira/browse/KAFKA-972
> >>> :
> >>>>>> MetadataRequest
> >>>>>>> returns stale list of brokers
> >>>>>>> - KAFKA-1867 <
> >> https://issues.apache.org/jira/browse/KAFKA-1867
> >>>> :
> >>>>>> liveBroker
> >>>>>>> list not updated on a cluster with no topics
> >>>>>>> - KAFKA-1650 <
> >> https://issues.apache.org/jira/browse/KAFKA-1650
> >>>> :
> >>>>>> Mirror
> >>>>>>> Maker could lose data on unclean shutdown.
> >>>>>>> - KAFKA-2009 <
> >> https://issues.apache.org/jira/browse/KAFKA-2009
> >>>> :
> >>>>> Fix
> >>>>>>> UncheckedOffset.removeOffset synchronization and trace logging
> >>>> issue
> >>>>>> in
> >>>>>>> mirror maker
> >>>>>>> - KAFKA-2407 <
> >> https://issues.apache.org/jira/browse/KAFKA-2407
> >>>> :
> >>>>> Only
> >>>>>>> create a log directory when it will be used
> >>>>>>> - KAFKA-2327 <
> >> https://issues.apache.org/jira/browse/KAFKA-2327
> >>>> :
> >>>>>>> broker doesn't start if config defines advertised.host but not
> >>>>>>> advertised.port
> >>>>>>> - KAFKA-1788: producer record can stay in RecordAccumulator
> >>>> forever
> >>>>> if
> >>>>>>> leader is no available
> >>>>>>> - KAFKA-2234 <
> >> https://issues.apache.org/jira/browse/KAFKA-2234
> >>>> :
> >>>>>>> Partition reassignment of a nonexistent topic prevents future
> >>>>>> reassignments
> >>>>>>> - KAFKA-2096 <
> >> https://issues.apache.org/jira/browse/KAFKA-2096
> >>>> :
> >>>>>>> Enable keepalive socket option for broker to prevent socket
> >> leak
> >>>>>>> - KAFKA-1057 <
> >> https://issues.apache.org/jira/browse/KAFKA-1057
> >>>> :
> >>>>> Trim
> >>>>>>> whitespaces from user specified configs
> >>>>>>> - KAFKA-1641 <
> >> https://issues.apache.org/jira/browse/KAFKA-1641
> >>>> :
> >>>>> Log
> >>>>>>> cleaner exits if last cleaned offset is lower than earliest
> >>> offset
> >>>>>>> - KAFKA-1648 <
> >> https://issues.apache.org/jira/browse/KAFKA-1648
> >>>> :
> >>>>>> Round
> >>>>>>> robin consumer balance throws an NPE when there are no topics
> >>>>>>> - KAFKA-1724 <
> >> https://issues.apache.org/jira/browse/KAFKA-1724
> >>>> :
> >>>>>>> Errors after reboot in single node setup
> >>>>>>> - KAFKA-1758 <
> >> https://issues.apache.org/jira/browse/KAFKA-1758
> >>>> :
> >>>>>>> corrupt recovery file prevents startup
> >>>>>>> - KAFKA-1866 <
> >> https://issues.apache.org/jira/browse/KAFKA-1866
> >>>> :
> >>>>>>> LogStartOffset gauge throws exceptions after log.delete()
> >>>>>>> - KAFKA-1883 <
> >> https://issues.apache.org/jira/browse/KAFKA-1883
> >>>> :
> >>>>>> NullPointerException
> >>>>>>> in RequestSendThread
> >>>>>>> - KAFKA-1896 <
> >> https://issues.apache.org/jira/browse/KAFKA-1896
> >>>> :
> >>>>>>> Record size funcition of record in mirror maker hit NPE when
> >> the
> >>>>>> message
> >>>>>>> value is null.
> >>>>>>> - KAFKA-2101 <
> >> https://issues.apache.org/jira/browse/KAFKA-2101
> >>>> :
> >>>>>>> Metric metadata-age is reset on a failed update
> >>>>>>> - KAFKA-2112 <
> >> https://issues.apache.org/jira/browse/KAFKA-2112
> >>>> :
> >>>>> make
> >>>>>>> overflowWheel volatile
> >>>>>>> - KAFKA-2117 <
> >> https://issues.apache.org/jira/browse/KAFKA-2117
> >>>> :
> >>>>>>> OffsetManager uses incorrect field for metadata
> >>>>>>> - KAFKA-2164 <
> >> https://issues.apache.org/jira/browse/KAFKA-2164
> >>>> :
> >>>>>>> ReplicaFetcherThread: suspicious log message on reset offset
> >>>>>>> - KAFKA-1668 <
> >> https://issues.apache.org/jira/browse/KAFKA-1668
> >>>> :
> >>>>>>> TopicCommand doesn't warn if --topic argument doesn't match
> >> any
> >>>>> topics
> >>>>>>> - KAFKA-2198 <
> >> https://issues.apache.org/jira/browse/KAFKA-2198
> >>>> :
> >>>>>>> kafka-topics.sh exits with 0 status on failures
> >>>>>>> - KAFKA-2235 <
> >> https://issues.apache.org/jira/browse/KAFKA-2235
> >>>> :
> >>>>>>> LogCleaner offset map overflow
> >>>>>>> - KAFKA-2241 <
> >> https://issues.apache.org/jira/browse/KAFKA-2241
> >>>> :
> >>>>>>> AbstractFetcherThread.shutdown() should not block on
> >>>>>>> ReadableByteChannel.read(buffer)
> >>>>>>> - KAFKA-2272 <
> >> https://issues.apache.org/jira/browse/KAFKA-2272
> >>>> :
> >>>>>>> listeners endpoint parsing fails if the hostname has capital
> >>>> letter
> >>>>>>> - KAFKA-2345 <
> >> https://issues.apache.org/jira/browse/KAFKA-2345
> >>>> :
> >>>>>>> Attempt to delete a topic already marked for deletion throws
> >>>>>>> ZkNodeExistsException
> >>>>>>> - KAFKA-2353 <
> >> https://issues.apache.org/jira/browse/KAFKA-2353
> >>>> :
> >>>>>>> SocketServer.Processor should catch exception and close the
> >>> socket
> >>>>>> properly
> >>>>>>> in configureNewConnections.
> >>>>>>> - KAFKA-1836 <
> >> https://issues.apache.org/jira/browse/KAFKA-1836
> >>>> :
> >>>>>>> metadata.fetch.timeout.ms set to zero blocks forever
> >>>>>>> - KAFKA-2317 <
> >> https://issues.apache.org/jira/browse/KAFKA-2317
> >>>> :
> >>>>>> De-register
> >>>>>>> isrChangeNotificationListener on controller resignation
> >>>>>>>
> >>>>>>> Note: KAFKA-2120 <
> >> https://issues.apache.org/jira/browse/KAFKA-2120
> >>>>
> >>>> &
> >>>>>>> KAFKA-2421 <https://issues.apache.org/jira/browse/KAFKA-2421>
> >> were
> >>>>>>> mentioned in previous emails, but are not in the list because
> >> they
> >>>> are
> >>>>>> not
> >>>>>>> committed yet.
> >>>>>>>
> >>>>>>> Hope that helps the effort.
> >>>>>>>
> >>>>>>> Thanks,
> >>>>>>> Grant
> >>>>>>>
> >>>>>>> On Mon, Aug 17, 2015 at 12:09 AM, Grant Henke <
> >> ghenke@cloudera.com
> >>>>
> >>>>>> wrote:
> >>>>>>>
> >>>>>>>> +1 to that suggestion. Though I suspect that requires a
> >> committer
> >>> to
> >>>>> do.
> >>>>>>>> Making it part of the standard commit process could work too.
> >>>>>>>> On Aug 16, 2015 11:01 PM, "Gwen Shapira" <gw...@confluent.io>
> >>> wrote:
> >>>>>>>>
> >>>>>>>>> BTW. I think it will be great for Apache Kafka to have a 0.8.2
> >>>>> "release
> >>>>>>>>> manager" who's role is to cherrypick low-risk bug-fixes into
> >> the
> >>>>> 0.8.2
> >>>>>>>>> branch and once enough bug fixes happened (or if sufficiently
> >>>>> critical
> >>>>>>>>> fixes happened) to roll out a new maintenance release (with
> >>> every 3
> >>>>>> month
> >>>>>>>>> as a reasonable bugfix release target).
> >>>>>>>>>
> >>>>>>>>> This will add some predictability regarding how fast we release
> >>>> fixes
> >>>>>> for
> >>>>>>>>> bugs.
> >>>>>>>>>
> >>>>>>>>> Gwen
> >>>>>>>>>
> >>>>>>>>> On Sun, Aug 16, 2015 at 8:09 PM, Jeff Holoman <
> >>>> jholoman@cloudera.com
> >>>>>>
> >>>>>>>>> wrote:
> >>>>>>>>>
> >>>>>>>>>> +1 for the release and also including
> >>>>>>>>>>
> >>>>>>>>>> https://issues.apache.org/jira/browse/KAFKA-2114
> >>>>>>>>>>
> >>>>>>>>>> Thanks
> >>>>>>>>>>
> >>>>>>>>>> Jeff
> >>>>>>>>>>
> >>>>>>>>>> On Sun, Aug 16, 2015 at 2:51 PM, Stevo Slavić <
> >>> sslavic@gmail.com
> >>>>>
> >>>>>>>>> wrote:
> >>>>>>>>>>
> >>>>>>>>>>> +1 (non-binding) for 0.8.2.2 release
> >>>>>>>>>>>
> >>>>>>>>>>> Would be nice to include in that release new producer
> >>>> resiliency
> >>>>>> bug
> >>>>>>>>>> fixes
> >>>>>>>>>>> https://issues.apache.org/jira/browse/KAFKA-1788 and
> >>>>>>>>>>> https://issues.apache.org/jira/browse/KAFKA-2120
> >>>>>>>>>>>
> >>>>>>>>>>> On Fri, Aug 14, 2015 at 4:03 PM, Gwen Shapira <
> >>>> gwen@confluent.io
> >>>>>>
> >>>>>>>>> wrote:
> >>>>>>>>>>>
> >>>>>>>>>>>> Will be nice to include Kafka-2308 and fix two critical
> >>>> snappy
> >>>>>>>>> issues
> >>>>>>>>>> in
> >>>>>>>>>>>> the maintenance release.
> >>>>>>>>>>>>
> >>>>>>>>>>>> Gwen
> >>>>>>>>>>>> On Aug 14, 2015 6:16 AM, "Grant Henke" <
> >>> ghenke@cloudera.com>
> >>>>>>>>> wrote:
> >>>>>>>>>>>>
> >>>>>>>>>>>>> Just to clarify. Will KAFKA-2189 be the only patch in
> >> the
> >>>>>>>>> release?
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> On Fri, Aug 14, 2015 at 7:35 AM, Manikumar Reddy <
> >>>>>>>>>> kumar@nmsworks.co.in
> >>>>>>>>>>>>
> >>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>> +1 for 0.8.2.2 release
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> On Fri, Aug 14, 2015 at 5:49 PM, Ismael Juma <
> >>>>>>>>> ismael@juma.me.uk>
> >>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> I think this is a good idea as the change is
> >> minimal
> >>> on
> >>>>> our
> >>>>>>>>> side
> >>>>>>>>>>> and
> >>>>>>>>>>>> it
> >>>>>>>>>>>>>> has
> >>>>>>>>>>>>>>> been tested in production for some time by the
> >>>> reporter.
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> Best,
> >>>>>>>>>>>>>>> Ismael
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> On Fri, Aug 14, 2015 at 1:15 PM, Jun Rao <
> >>>>> jun@confluent.io
> >>>>>>>
> >>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> Hi, Everyone,
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> Since the release of Kafka 0.8.2.1, a number of
> >>>> people
> >>>>>> have
> >>>>>>>>>>>> reported
> >>>>>>>>>>>>> an
> >>>>>>>>>>>>>>>> issue with snappy compression (
> >>>>>>>>>>>>>>>> https://issues.apache.org/jira/browse/KAFKA-2189
> >> ).
> >>>>>>>>> Basically,
> >>>>>>>>>> if
> >>>>>>>>>>>>> they
> >>>>>>>>>>>>>>> use
> >>>>>>>>>>>>>>>> snappy in 0.8.2.1, they will experience a 2-3X
> >>> space
> >>>>>>>>> increase.
> >>>>>>>>>>> The
> >>>>>>>>>>>>>> issue
> >>>>>>>>>>>>>>>> has since been fixed in trunk (just a snappy jar
> >>>>>> upgrade).
> >>>>>>>>>> Since
> >>>>>>>>>>>>> 0.8.3
> >>>>>>>>>>>>>> is
> >>>>>>>>>>>>>>>> still a few months away, it may make sense to do
> >> an
> >>>>>> 0.8.2.2
> >>>>>>>>>>> release
> >>>>>>>>>>>>>> just
> >>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>> fix this issue. Any objections?
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> Thanks,
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> Jun
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> --
> >>>>>>>>>>>>> Grant Henke
> >>>>>>>>>>>>> Software Engineer | Cloudera
> >>>>>>>>>>>>> grant@cloudera.com | twitter.com/gchenke |
> >>>>>>>>>> linkedin.com/in/granthenke
> >>>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>> --
> >>>>>>>>>> Jeff Holoman
> >>>>>>>>>> Systems Engineer
> >>>>>>>>>>
> >>>>>>>>>
> >>>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>> --
> >>>>>>> Grant Henke
> >>>>>>> Software Engineer | Cloudera
> >>>>>>> grant@cloudera.com | twitter.com/gchenke |
> >>>> linkedin.com/in/granthenke
> >>>>>>>
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>> --
> >>>>>> Grant Henke
> >>>>>> Software Engineer | Cloudera
> >>>>>> grant@cloudera.com | twitter.com/gchenke |
> >>> linkedin.com/in/granthenke
> >>>>>>
> >>>>>
> >>>>
> >>>>
> >>>>
> >>>> --
> >>>> Grant Henke
> >>>> Software Engineer | Cloudera
> >>>> grant@cloudera.com | twitter.com/gchenke | linkedin.com/in/granthenke
> >>>>
> >>>
> >>
> >
> >
> >
> > --
> > Thanks,
> > Neha
>
>
Re: [DISCUSSION] Kafka 0.8.2.2 release?
Posted by Daniel Nelson <da...@vungle.com>.
I am strongly in favor of cutting a 0.8.2.2 release, but I don’t think that it needs to include anything other than the fix for Snappy that kicked off this discussion in the first place. If there are additional critical issues that can be included without delaying the process, I see no downside.
> On Aug 18, 2015, at 10:54 AM, Neha Narkhede <ne...@confluent.io> wrote:
>
> How about looking at the scope for the 0.8.3 release first before we cut
> yet another point release off of 0.8.2.2? Each release includes some
> overhead and if there is a larger release in the works, it might be worth
> working on getting that. My take is that the 2 things the community has
> been waiting for is SSL support and the new consumer and we have been
> promising to get 0.8.3 with both those features for several months now.
>
> Looking at the progress on both, it seems we are very close to getting both
> those checked in and it looks like we should get there in another 5-6
> weeks. Furthermore, both of these features are large and I anticipate us
> receiving feedback and bugs that will require a couple of point releases on
> top of 0.8.3 anyway. One possibility is to work on 0.8.3 together now and
> get the community to use the newly released features, gather feedback and
> do point releases incorporating that feedback and iterate on it.
>
> We could absolutely do both 0.8.2.2 and 0.8.3. What I'd ask for is for us
> to look at the 0.8.3 timeline too and make a call whether 0.8.2.2 still
> makes sense.
>
> Thanks,
> Neha
>
> On Tue, Aug 18, 2015 at 10:24 AM, Gwen Shapira <gw...@confluent.io> wrote:
>
>> Thanks Jun.
>>
>> I updated the list with your suggestions.
>> If anyone feels we are missing a critical patch for 0.8.2.2, please speak
>> up.
>>
>> Gwen
>>
>> On Mon, Aug 17, 2015 at 5:40 PM, Jun Rao <ju...@confluent.io> wrote:
>>
>>> Hi, Grant,
>>>
>>> I took a look at that list. None of those is really critical as you said.
>>> So, I'd suggest that we not include those to minimize the scope of the
>>> release.
>>>
>>> Thanks,
>>>
>>> Jun
>>>
>>> On Mon, Aug 17, 2015 at 5:16 PM, Grant Henke <gh...@cloudera.com>
>> wrote:
>>>
>>>> Thanks Gwen.
>>>>
>>>> I updated a few small things on the wiki page.
>>>>
>>>> Below is a list of jiras I think could also be marked as included. All
>> of
>>>> these, though not super critical, seem like fairly small and low risk
>>>> changes that help avoid potentially confusing issues or errors for
>> users.
>>>>
>>>> KAFKA-2012
>>>> KAFKA-972
>>>> KAFKA-2337 & KAFKA-2393
>>>> KAFKA-1867
>>>> KAFKA-2407
>>>> KAFKA-2234
>>>> KAFKA-1866
>>>> KAFKA-2345 & KAFKA-2355
>>>>
>>>> thoughts?
>>>>
>>>> Thank you,
>>>> Grant
>>>>
>>>> On Mon, Aug 17, 2015 at 4:56 PM, Gwen Shapira <gw...@confluent.io>
>> wrote:
>>>>
>>>>> Thanks for creating a list, Grant!
>>>>>
>>>>> I placed it on the wiki with a quick evaluation of the content and
>>>> whether
>>>>> it should be in 0.8.2.2:
>>>>>
>>>>>
>>>>
>>>
>> https://cwiki.apache.org/confluence/display/KAFKA/Proposed+patches+for+0.8.2.2
>>>>>
>>>>> I'm attempting to only cherrypick fixes that are both important for
>>> large
>>>>> number of users (or very critical to some users) and very safe
>> (mostly
>>>>> judged by the size of the change, but not only)
>>>>>
>>>>> If your favorite bugfix is missing from the list, or is there but
>>> marked
>>>>> "No", please let us know (in this thread) what we are missing and why
>>> it
>>>> is
>>>>> both important and safe.
>>>>> Also, if I accidentally included something you consider unsafe, speak
>>> up!
>>>>>
>>>>> Gwen
>>>>>
>>>>> On Mon, Aug 17, 2015 at 8:17 AM, Grant Henke <gh...@cloudera.com>
>>>> wrote:
>>>>>
>>>>>> +dev
>>>>>>
>>>>>> Adding dev list back in. Somehow it got dropped.
>>>>>>
>>>>>>
>>>>>> On Mon, Aug 17, 2015 at 10:16 AM, Grant Henke <ghenke@cloudera.com
>>>
>>>>> wrote:
>>>>>>
>>>>>>> Below is a list of candidate bug fix jiras marked fixed for
>> 0.8.3.
>>> I
>>>>>> don't
>>>>>>> suspect all of these will (or should) make it into the release
>> but
>>>> this
>>>>>>> should be a relatively complete list to work from:
>>>>>>>
>>>>>>> - KAFKA-2114 <
>> https://issues.apache.org/jira/browse/KAFKA-2114
>>>> :
>>>>>> Unable
>>>>>>> to change min.insync.replicas default
>>>>>>> - KAFKA-1702 <
>> https://issues.apache.org/jira/browse/KAFKA-1702
>>>> :
>>>>>>> Messages silently Lost by producer
>>>>>>> - KAFKA-2012 <
>> https://issues.apache.org/jira/browse/KAFKA-2012
>>>> :
>>>>>>> Broker should automatically handle corrupt index files
>>>>>>> - KAFKA-2406 <
>> https://issues.apache.org/jira/browse/KAFKA-2406
>>>> :
>>>>> ISR
>>>>>>> propagation should be throttled to avoid overwhelming
>>> controller.
>>>>>>> - KAFKA-2336 <
>> https://issues.apache.org/jira/browse/KAFKA-2336
>>>> :
>>>>>>> Changing offsets.topic.num.partitions after the offset topic
>> is
>>>>>> created
>>>>>>> breaks consumer group partition assignment
>>>>>>> - KAFKA-2337 <
>> https://issues.apache.org/jira/browse/KAFKA-2337
>>>> :
>>>>>> Verify
>>>>>>> that metric names will not collide when creating new topics
>>>>>>> - KAFKA-2393 <
>> https://issues.apache.org/jira/browse/KAFKA-2393
>>>> :
>>>>>>> Correctly Handle InvalidTopicException in
>>>>> KafkaApis.getTopicMetadata()
>>>>>>> - KAFKA-2189 <
>> https://issues.apache.org/jira/browse/KAFKA-2189
>>>> :
>>>>>> Snappy
>>>>>>> compression of message batches less efficient in 0.8.2.1
>>>>>>> - KAFKA-2308 <
>> https://issues.apache.org/jira/browse/KAFKA-2308
>>>> :
>>>>> New
>>>>>>> producer + Snappy face un-compression errors after broker
>>> restart
>>>>>>> - KAFKA-2042 <
>> https://issues.apache.org/jira/browse/KAFKA-2042
>>>> :
>>>>> New
>>>>>>> producer metadata update always get all topics.
>>>>>>> - KAFKA-1367 <
>> https://issues.apache.org/jira/browse/KAFKA-1367
>>>> :
>>>>>> Broker
>>>>>>> topic metadata not kept in sync with ZooKeeper
>>>>>>> - KAFKA-972 <https://issues.apache.org/jira/browse/KAFKA-972
>>> :
>>>>>> MetadataRequest
>>>>>>> returns stale list of brokers
>>>>>>> - KAFKA-1867 <
>> https://issues.apache.org/jira/browse/KAFKA-1867
>>>> :
>>>>>> liveBroker
>>>>>>> list not updated on a cluster with no topics
>>>>>>> - KAFKA-1650 <
>> https://issues.apache.org/jira/browse/KAFKA-1650
>>>> :
>>>>>> Mirror
>>>>>>> Maker could lose data on unclean shutdown.
>>>>>>> - KAFKA-2009 <
>> https://issues.apache.org/jira/browse/KAFKA-2009
>>>> :
>>>>> Fix
>>>>>>> UncheckedOffset.removeOffset synchronization and trace logging
>>>> issue
>>>>>> in
>>>>>>> mirror maker
>>>>>>> - KAFKA-2407 <
>> https://issues.apache.org/jira/browse/KAFKA-2407
>>>> :
>>>>> Only
>>>>>>> create a log directory when it will be used
>>>>>>> - KAFKA-2327 <
>> https://issues.apache.org/jira/browse/KAFKA-2327
>>>> :
>>>>>>> broker doesn't start if config defines advertised.host but not
>>>>>>> advertised.port
>>>>>>> - KAFKA-1788: producer record can stay in RecordAccumulator
>>>> forever
>>>>> if
>>>>>>> leader is no available
>>>>>>> - KAFKA-2234 <
>> https://issues.apache.org/jira/browse/KAFKA-2234
>>>> :
>>>>>>> Partition reassignment of a nonexistent topic prevents future
>>>>>> reassignments
>>>>>>> - KAFKA-2096 <
>> https://issues.apache.org/jira/browse/KAFKA-2096
>>>> :
>>>>>>> Enable keepalive socket option for broker to prevent socket
>> leak
>>>>>>> - KAFKA-1057 <
>> https://issues.apache.org/jira/browse/KAFKA-1057
>>>> :
>>>>> Trim
>>>>>>> whitespaces from user specified configs
>>>>>>> - KAFKA-1641 <
>> https://issues.apache.org/jira/browse/KAFKA-1641
>>>> :
>>>>> Log
>>>>>>> cleaner exits if last cleaned offset is lower than earliest
>>> offset
>>>>>>> - KAFKA-1648 <
>> https://issues.apache.org/jira/browse/KAFKA-1648
>>>> :
>>>>>> Round
>>>>>>> robin consumer balance throws an NPE when there are no topics
>>>>>>> - KAFKA-1724 <
>> https://issues.apache.org/jira/browse/KAFKA-1724
>>>> :
>>>>>>> Errors after reboot in single node setup
>>>>>>> - KAFKA-1758 <
>> https://issues.apache.org/jira/browse/KAFKA-1758
>>>> :
>>>>>>> corrupt recovery file prevents startup
>>>>>>> - KAFKA-1866 <
>> https://issues.apache.org/jira/browse/KAFKA-1866
>>>> :
>>>>>>> LogStartOffset gauge throws exceptions after log.delete()
>>>>>>> - KAFKA-1883 <
>> https://issues.apache.org/jira/browse/KAFKA-1883
>>>> :
>>>>>> NullPointerException
>>>>>>> in RequestSendThread
>>>>>>> - KAFKA-1896 <
>> https://issues.apache.org/jira/browse/KAFKA-1896
>>>> :
>>>>>>> Record size funcition of record in mirror maker hit NPE when
>> the
>>>>>> message
>>>>>>> value is null.
>>>>>>> - KAFKA-2101 <
>> https://issues.apache.org/jira/browse/KAFKA-2101
>>>> :
>>>>>>> Metric metadata-age is reset on a failed update
>>>>>>> - KAFKA-2112 <
>> https://issues.apache.org/jira/browse/KAFKA-2112
>>>> :
>>>>> make
>>>>>>> overflowWheel volatile
>>>>>>> - KAFKA-2117 <
>> https://issues.apache.org/jira/browse/KAFKA-2117
>>>> :
>>>>>>> OffsetManager uses incorrect field for metadata
>>>>>>> - KAFKA-2164 <
>> https://issues.apache.org/jira/browse/KAFKA-2164
>>>> :
>>>>>>> ReplicaFetcherThread: suspicious log message on reset offset
>>>>>>> - KAFKA-1668 <
>> https://issues.apache.org/jira/browse/KAFKA-1668
>>>> :
>>>>>>> TopicCommand doesn't warn if --topic argument doesn't match
>> any
>>>>> topics
>>>>>>> - KAFKA-2198 <
>> https://issues.apache.org/jira/browse/KAFKA-2198
>>>> :
>>>>>>> kafka-topics.sh exits with 0 status on failures
>>>>>>> - KAFKA-2235 <
>> https://issues.apache.org/jira/browse/KAFKA-2235
>>>> :
>>>>>>> LogCleaner offset map overflow
>>>>>>> - KAFKA-2241 <
>> https://issues.apache.org/jira/browse/KAFKA-2241
>>>> :
>>>>>>> AbstractFetcherThread.shutdown() should not block on
>>>>>>> ReadableByteChannel.read(buffer)
>>>>>>> - KAFKA-2272 <
>> https://issues.apache.org/jira/browse/KAFKA-2272
>>>> :
>>>>>>> listeners endpoint parsing fails if the hostname has capital
>>>> letter
>>>>>>> - KAFKA-2345 <
>> https://issues.apache.org/jira/browse/KAFKA-2345
>>>> :
>>>>>>> Attempt to delete a topic already marked for deletion throws
>>>>>>> ZkNodeExistsException
>>>>>>> - KAFKA-2353 <
>> https://issues.apache.org/jira/browse/KAFKA-2353
>>>> :
>>>>>>> SocketServer.Processor should catch exception and close the
>>> socket
>>>>>> properly
>>>>>>> in configureNewConnections.
>>>>>>> - KAFKA-1836 <
>> https://issues.apache.org/jira/browse/KAFKA-1836
>>>> :
>>>>>>> metadata.fetch.timeout.ms set to zero blocks forever
>>>>>>> - KAFKA-2317 <
>> https://issues.apache.org/jira/browse/KAFKA-2317
>>>> :
>>>>>> De-register
>>>>>>> isrChangeNotificationListener on controller resignation
>>>>>>>
>>>>>>> Note: KAFKA-2120 <
>> https://issues.apache.org/jira/browse/KAFKA-2120
>>>>
>>>> &
>>>>>>> KAFKA-2421 <https://issues.apache.org/jira/browse/KAFKA-2421>
>> were
>>>>>>> mentioned in previous emails, but are not in the list because
>> they
>>>> are
>>>>>> not
>>>>>>> committed yet.
>>>>>>>
>>>>>>> Hope that helps the effort.
>>>>>>>
>>>>>>> Thanks,
>>>>>>> Grant
>>>>>>>
>>>>>>> On Mon, Aug 17, 2015 at 12:09 AM, Grant Henke <
>> ghenke@cloudera.com
>>>>
>>>>>> wrote:
>>>>>>>
>>>>>>>> +1 to that suggestion. Though I suspect that requires a
>> committer
>>> to
>>>>> do.
>>>>>>>> Making it part of the standard commit process could work too.
>>>>>>>> On Aug 16, 2015 11:01 PM, "Gwen Shapira" <gw...@confluent.io>
>>> wrote:
>>>>>>>>
>>>>>>>>> BTW. I think it will be great for Apache Kafka to have a 0.8.2
>>>>> "release
>>>>>>>>> manager" who's role is to cherrypick low-risk bug-fixes into
>> the
>>>>> 0.8.2
>>>>>>>>> branch and once enough bug fixes happened (or if sufficiently
>>>>> critical
>>>>>>>>> fixes happened) to roll out a new maintenance release (with
>>> every 3
>>>>>> month
>>>>>>>>> as a reasonable bugfix release target).
>>>>>>>>>
>>>>>>>>> This will add some predictability regarding how fast we release
>>>> fixes
>>>>>> for
>>>>>>>>> bugs.
>>>>>>>>>
>>>>>>>>> Gwen
>>>>>>>>>
>>>>>>>>> On Sun, Aug 16, 2015 at 8:09 PM, Jeff Holoman <
>>>> jholoman@cloudera.com
>>>>>>
>>>>>>>>> wrote:
>>>>>>>>>
>>>>>>>>>> +1 for the release and also including
>>>>>>>>>>
>>>>>>>>>> https://issues.apache.org/jira/browse/KAFKA-2114
>>>>>>>>>>
>>>>>>>>>> Thanks
>>>>>>>>>>
>>>>>>>>>> Jeff
>>>>>>>>>>
>>>>>>>>>> On Sun, Aug 16, 2015 at 2:51 PM, Stevo Slavić <
>>> sslavic@gmail.com
>>>>>
>>>>>>>>> wrote:
>>>>>>>>>>
>>>>>>>>>>> +1 (non-binding) for 0.8.2.2 release
>>>>>>>>>>>
>>>>>>>>>>> Would be nice to include in that release new producer
>>>> resiliency
>>>>>> bug
>>>>>>>>>> fixes
>>>>>>>>>>> https://issues.apache.org/jira/browse/KAFKA-1788 and
>>>>>>>>>>> https://issues.apache.org/jira/browse/KAFKA-2120
>>>>>>>>>>>
>>>>>>>>>>> On Fri, Aug 14, 2015 at 4:03 PM, Gwen Shapira <
>>>> gwen@confluent.io
>>>>>>
>>>>>>>>> wrote:
>>>>>>>>>>>
>>>>>>>>>>>> Will be nice to include Kafka-2308 and fix two critical
>>>> snappy
>>>>>>>>> issues
>>>>>>>>>> in
>>>>>>>>>>>> the maintenance release.
>>>>>>>>>>>>
>>>>>>>>>>>> Gwen
>>>>>>>>>>>> On Aug 14, 2015 6:16 AM, "Grant Henke" <
>>> ghenke@cloudera.com>
>>>>>>>>> wrote:
>>>>>>>>>>>>
>>>>>>>>>>>>> Just to clarify. Will KAFKA-2189 be the only patch in
>> the
>>>>>>>>> release?
>>>>>>>>>>>>>
>>>>>>>>>>>>> On Fri, Aug 14, 2015 at 7:35 AM, Manikumar Reddy <
>>>>>>>>>> kumar@nmsworks.co.in
>>>>>>>>>>>>
>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>
>>>>>>>>>>>>>> +1 for 0.8.2.2 release
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> On Fri, Aug 14, 2015 at 5:49 PM, Ismael Juma <
>>>>>>>>> ismael@juma.me.uk>
>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> I think this is a good idea as the change is
>> minimal
>>> on
>>>>> our
>>>>>>>>> side
>>>>>>>>>>> and
>>>>>>>>>>>> it
>>>>>>>>>>>>>> has
>>>>>>>>>>>>>>> been tested in production for some time by the
>>>> reporter.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Best,
>>>>>>>>>>>>>>> Ismael
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> On Fri, Aug 14, 2015 at 1:15 PM, Jun Rao <
>>>>> jun@confluent.io
>>>>>>>
>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Hi, Everyone,
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Since the release of Kafka 0.8.2.1, a number of
>>>> people
>>>>>> have
>>>>>>>>>>>> reported
>>>>>>>>>>>>> an
>>>>>>>>>>>>>>>> issue with snappy compression (
>>>>>>>>>>>>>>>> https://issues.apache.org/jira/browse/KAFKA-2189
>> ).
>>>>>>>>> Basically,
>>>>>>>>>> if
>>>>>>>>>>>>> they
>>>>>>>>>>>>>>> use
>>>>>>>>>>>>>>>> snappy in 0.8.2.1, they will experience a 2-3X
>>> space
>>>>>>>>> increase.
>>>>>>>>>>> The
>>>>>>>>>>>>>> issue
>>>>>>>>>>>>>>>> has since been fixed in trunk (just a snappy jar
>>>>>> upgrade).
>>>>>>>>>> Since
>>>>>>>>>>>>> 0.8.3
>>>>>>>>>>>>>> is
>>>>>>>>>>>>>>>> still a few months away, it may make sense to do
>> an
>>>>>> 0.8.2.2
>>>>>>>>>>> release
>>>>>>>>>>>>>> just
>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>> fix this issue. Any objections?
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Jun
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> --
>>>>>>>>>>>>> Grant Henke
>>>>>>>>>>>>> Software Engineer | Cloudera
>>>>>>>>>>>>> grant@cloudera.com | twitter.com/gchenke |
>>>>>>>>>> linkedin.com/in/granthenke
>>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> --
>>>>>>>>>> Jeff Holoman
>>>>>>>>>> Systems Engineer
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> --
>>>>>>> Grant Henke
>>>>>>> Software Engineer | Cloudera
>>>>>>> grant@cloudera.com | twitter.com/gchenke |
>>>> linkedin.com/in/granthenke
>>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> --
>>>>>> Grant Henke
>>>>>> Software Engineer | Cloudera
>>>>>> grant@cloudera.com | twitter.com/gchenke |
>>> linkedin.com/in/granthenke
>>>>>>
>>>>>
>>>>
>>>>
>>>>
>>>> --
>>>> Grant Henke
>>>> Software Engineer | Cloudera
>>>> grant@cloudera.com | twitter.com/gchenke | linkedin.com/in/granthenke
>>>>
>>>
>>
>
>
>
> --
> Thanks,
> Neha
Re: [DISCUSSION] Kafka 0.8.2.2 release?
Posted by Daniel Nelson <da...@vungle.com>.
I am strongly in favor of cutting a 0.8.2.2 release, but I don’t think that it needs to include anything other than the fix for Snappy that kicked off this discussion in the first place. If there are additional critical issues that can be included without delaying the process, I see no downside.
> On Aug 18, 2015, at 10:54 AM, Neha Narkhede <ne...@confluent.io> wrote:
>
> How about looking at the scope for the 0.8.3 release first before we cut
> yet another point release off of 0.8.2.2? Each release includes some
> overhead and if there is a larger release in the works, it might be worth
> working on getting that. My take is that the 2 things the community has
> been waiting for is SSL support and the new consumer and we have been
> promising to get 0.8.3 with both those features for several months now.
>
> Looking at the progress on both, it seems we are very close to getting both
> those checked in and it looks like we should get there in another 5-6
> weeks. Furthermore, both of these features are large and I anticipate us
> receiving feedback and bugs that will require a couple of point releases on
> top of 0.8.3 anyway. One possibility is to work on 0.8.3 together now and
> get the community to use the newly released features, gather feedback and
> do point releases incorporating that feedback and iterate on it.
>
> We could absolutely do both 0.8.2.2 and 0.8.3. What I'd ask for is for us
> to look at the 0.8.3 timeline too and make a call whether 0.8.2.2 still
> makes sense.
>
> Thanks,
> Neha
>
> On Tue, Aug 18, 2015 at 10:24 AM, Gwen Shapira <gw...@confluent.io> wrote:
>
>> Thanks Jun.
>>
>> I updated the list with your suggestions.
>> If anyone feels we are missing a critical patch for 0.8.2.2, please speak
>> up.
>>
>> Gwen
>>
>> On Mon, Aug 17, 2015 at 5:40 PM, Jun Rao <ju...@confluent.io> wrote:
>>
>>> Hi, Grant,
>>>
>>> I took a look at that list. None of those is really critical as you said.
>>> So, I'd suggest that we not include those to minimize the scope of the
>>> release.
>>>
>>> Thanks,
>>>
>>> Jun
>>>
>>> On Mon, Aug 17, 2015 at 5:16 PM, Grant Henke <gh...@cloudera.com>
>> wrote:
>>>
>>>> Thanks Gwen.
>>>>
>>>> I updated a few small things on the wiki page.
>>>>
>>>> Below is a list of jiras I think could also be marked as included. All
>> of
>>>> these, though not super critical, seem like fairly small and low risk
>>>> changes that help avoid potentially confusing issues or errors for
>> users.
>>>>
>>>> KAFKA-2012
>>>> KAFKA-972
>>>> KAFKA-2337 & KAFKA-2393
>>>> KAFKA-1867
>>>> KAFKA-2407
>>>> KAFKA-2234
>>>> KAFKA-1866
>>>> KAFKA-2345 & KAFKA-2355
>>>>
>>>> thoughts?
>>>>
>>>> Thank you,
>>>> Grant
>>>>
>>>> On Mon, Aug 17, 2015 at 4:56 PM, Gwen Shapira <gw...@confluent.io>
>> wrote:
>>>>
>>>>> Thanks for creating a list, Grant!
>>>>>
>>>>> I placed it on the wiki with a quick evaluation of the content and
>>>> whether
>>>>> it should be in 0.8.2.2:
>>>>>
>>>>>
>>>>
>>>
>> https://cwiki.apache.org/confluence/display/KAFKA/Proposed+patches+for+0.8.2.2
>>>>>
>>>>> I'm attempting to only cherrypick fixes that are both important for
>>> large
>>>>> number of users (or very critical to some users) and very safe
>> (mostly
>>>>> judged by the size of the change, but not only)
>>>>>
>>>>> If your favorite bugfix is missing from the list, or is there but
>>> marked
>>>>> "No", please let us know (in this thread) what we are missing and why
>>> it
>>>> is
>>>>> both important and safe.
>>>>> Also, if I accidentally included something you consider unsafe, speak
>>> up!
>>>>>
>>>>> Gwen
>>>>>
>>>>> On Mon, Aug 17, 2015 at 8:17 AM, Grant Henke <gh...@cloudera.com>
>>>> wrote:
>>>>>
>>>>>> +dev
>>>>>>
>>>>>> Adding dev list back in. Somehow it got dropped.
>>>>>>
>>>>>>
>>>>>> On Mon, Aug 17, 2015 at 10:16 AM, Grant Henke <ghenke@cloudera.com
>>>
>>>>> wrote:
>>>>>>
>>>>>>> Below is a list of candidate bug fix jiras marked fixed for
>> 0.8.3.
>>> I
>>>>>> don't
>>>>>>> suspect all of these will (or should) make it into the release
>> but
>>>> this
>>>>>>> should be a relatively complete list to work from:
>>>>>>>
>>>>>>> - KAFKA-2114 <
>> https://issues.apache.org/jira/browse/KAFKA-2114
>>>> :
>>>>>> Unable
>>>>>>> to change min.insync.replicas default
>>>>>>> - KAFKA-1702 <
>> https://issues.apache.org/jira/browse/KAFKA-1702
>>>> :
>>>>>>> Messages silently Lost by producer
>>>>>>> - KAFKA-2012 <
>> https://issues.apache.org/jira/browse/KAFKA-2012
>>>> :
>>>>>>> Broker should automatically handle corrupt index files
>>>>>>> - KAFKA-2406 <
>> https://issues.apache.org/jira/browse/KAFKA-2406
>>>> :
>>>>> ISR
>>>>>>> propagation should be throttled to avoid overwhelming
>>> controller.
>>>>>>> - KAFKA-2336 <
>> https://issues.apache.org/jira/browse/KAFKA-2336
>>>> :
>>>>>>> Changing offsets.topic.num.partitions after the offset topic
>> is
>>>>>> created
>>>>>>> breaks consumer group partition assignment
>>>>>>> - KAFKA-2337 <
>> https://issues.apache.org/jira/browse/KAFKA-2337
>>>> :
>>>>>> Verify
>>>>>>> that metric names will not collide when creating new topics
>>>>>>> - KAFKA-2393 <
>> https://issues.apache.org/jira/browse/KAFKA-2393
>>>> :
>>>>>>> Correctly Handle InvalidTopicException in
>>>>> KafkaApis.getTopicMetadata()
>>>>>>> - KAFKA-2189 <
>> https://issues.apache.org/jira/browse/KAFKA-2189
>>>> :
>>>>>> Snappy
>>>>>>> compression of message batches less efficient in 0.8.2.1
>>>>>>> - KAFKA-2308 <
>> https://issues.apache.org/jira/browse/KAFKA-2308
>>>> :
>>>>> New
>>>>>>> producer + Snappy face un-compression errors after broker
>>> restart
>>>>>>> - KAFKA-2042 <
>> https://issues.apache.org/jira/browse/KAFKA-2042
>>>> :
>>>>> New
>>>>>>> producer metadata update always get all topics.
>>>>>>> - KAFKA-1367 <
>> https://issues.apache.org/jira/browse/KAFKA-1367
>>>> :
>>>>>> Broker
>>>>>>> topic metadata not kept in sync with ZooKeeper
>>>>>>> - KAFKA-972 <https://issues.apache.org/jira/browse/KAFKA-972
>>> :
>>>>>> MetadataRequest
>>>>>>> returns stale list of brokers
>>>>>>> - KAFKA-1867 <
>> https://issues.apache.org/jira/browse/KAFKA-1867
>>>> :
>>>>>> liveBroker
>>>>>>> list not updated on a cluster with no topics
>>>>>>> - KAFKA-1650 <
>> https://issues.apache.org/jira/browse/KAFKA-1650
>>>> :
>>>>>> Mirror
>>>>>>> Maker could lose data on unclean shutdown.
>>>>>>> - KAFKA-2009 <
>> https://issues.apache.org/jira/browse/KAFKA-2009
>>>> :
>>>>> Fix
>>>>>>> UncheckedOffset.removeOffset synchronization and trace logging
>>>> issue
>>>>>> in
>>>>>>> mirror maker
>>>>>>> - KAFKA-2407 <
>> https://issues.apache.org/jira/browse/KAFKA-2407
>>>> :
>>>>> Only
>>>>>>> create a log directory when it will be used
>>>>>>> - KAFKA-2327 <
>> https://issues.apache.org/jira/browse/KAFKA-2327
>>>> :
>>>>>>> broker doesn't start if config defines advertised.host but not
>>>>>>> advertised.port
>>>>>>> - KAFKA-1788: producer record can stay in RecordAccumulator
>>>> forever
>>>>> if
>>>>>>> leader is no available
>>>>>>> - KAFKA-2234 <
>> https://issues.apache.org/jira/browse/KAFKA-2234
>>>> :
>>>>>>> Partition reassignment of a nonexistent topic prevents future
>>>>>> reassignments
>>>>>>> - KAFKA-2096 <
>> https://issues.apache.org/jira/browse/KAFKA-2096
>>>> :
>>>>>>> Enable keepalive socket option for broker to prevent socket
>> leak
>>>>>>> - KAFKA-1057 <
>> https://issues.apache.org/jira/browse/KAFKA-1057
>>>> :
>>>>> Trim
>>>>>>> whitespaces from user specified configs
>>>>>>> - KAFKA-1641 <
>> https://issues.apache.org/jira/browse/KAFKA-1641
>>>> :
>>>>> Log
>>>>>>> cleaner exits if last cleaned offset is lower than earliest
>>> offset
>>>>>>> - KAFKA-1648 <
>> https://issues.apache.org/jira/browse/KAFKA-1648
>>>> :
>>>>>> Round
>>>>>>> robin consumer balance throws an NPE when there are no topics
>>>>>>> - KAFKA-1724 <
>> https://issues.apache.org/jira/browse/KAFKA-1724
>>>> :
>>>>>>> Errors after reboot in single node setup
>>>>>>> - KAFKA-1758 <
>> https://issues.apache.org/jira/browse/KAFKA-1758
>>>> :
>>>>>>> corrupt recovery file prevents startup
>>>>>>> - KAFKA-1866 <
>> https://issues.apache.org/jira/browse/KAFKA-1866
>>>> :
>>>>>>> LogStartOffset gauge throws exceptions after log.delete()
>>>>>>> - KAFKA-1883 <
>> https://issues.apache.org/jira/browse/KAFKA-1883
>>>> :
>>>>>> NullPointerException
>>>>>>> in RequestSendThread
>>>>>>> - KAFKA-1896 <
>> https://issues.apache.org/jira/browse/KAFKA-1896
>>>> :
>>>>>>> Record size funcition of record in mirror maker hit NPE when
>> the
>>>>>> message
>>>>>>> value is null.
>>>>>>> - KAFKA-2101 <
>> https://issues.apache.org/jira/browse/KAFKA-2101
>>>> :
>>>>>>> Metric metadata-age is reset on a failed update
>>>>>>> - KAFKA-2112 <
>> https://issues.apache.org/jira/browse/KAFKA-2112
>>>> :
>>>>> make
>>>>>>> overflowWheel volatile
>>>>>>> - KAFKA-2117 <
>> https://issues.apache.org/jira/browse/KAFKA-2117
>>>> :
>>>>>>> OffsetManager uses incorrect field for metadata
>>>>>>> - KAFKA-2164 <
>> https://issues.apache.org/jira/browse/KAFKA-2164
>>>> :
>>>>>>> ReplicaFetcherThread: suspicious log message on reset offset
>>>>>>> - KAFKA-1668 <
>> https://issues.apache.org/jira/browse/KAFKA-1668
>>>> :
>>>>>>> TopicCommand doesn't warn if --topic argument doesn't match
>> any
>>>>> topics
>>>>>>> - KAFKA-2198 <
>> https://issues.apache.org/jira/browse/KAFKA-2198
>>>> :
>>>>>>> kafka-topics.sh exits with 0 status on failures
>>>>>>> - KAFKA-2235 <
>> https://issues.apache.org/jira/browse/KAFKA-2235
>>>> :
>>>>>>> LogCleaner offset map overflow
>>>>>>> - KAFKA-2241 <
>> https://issues.apache.org/jira/browse/KAFKA-2241
>>>> :
>>>>>>> AbstractFetcherThread.shutdown() should not block on
>>>>>>> ReadableByteChannel.read(buffer)
>>>>>>> - KAFKA-2272 <
>> https://issues.apache.org/jira/browse/KAFKA-2272
>>>> :
>>>>>>> listeners endpoint parsing fails if the hostname has capital
>>>> letter
>>>>>>> - KAFKA-2345 <
>> https://issues.apache.org/jira/browse/KAFKA-2345
>>>> :
>>>>>>> Attempt to delete a topic already marked for deletion throws
>>>>>>> ZkNodeExistsException
>>>>>>> - KAFKA-2353 <
>> https://issues.apache.org/jira/browse/KAFKA-2353
>>>> :
>>>>>>> SocketServer.Processor should catch exception and close the
>>> socket
>>>>>> properly
>>>>>>> in configureNewConnections.
>>>>>>> - KAFKA-1836 <
>> https://issues.apache.org/jira/browse/KAFKA-1836
>>>> :
>>>>>>> metadata.fetch.timeout.ms set to zero blocks forever
>>>>>>> - KAFKA-2317 <
>> https://issues.apache.org/jira/browse/KAFKA-2317
>>>> :
>>>>>> De-register
>>>>>>> isrChangeNotificationListener on controller resignation
>>>>>>>
>>>>>>> Note: KAFKA-2120 <
>> https://issues.apache.org/jira/browse/KAFKA-2120
>>>>
>>>> &
>>>>>>> KAFKA-2421 <https://issues.apache.org/jira/browse/KAFKA-2421>
>> were
>>>>>>> mentioned in previous emails, but are not in the list because
>> they
>>>> are
>>>>>> not
>>>>>>> committed yet.
>>>>>>>
>>>>>>> Hope that helps the effort.
>>>>>>>
>>>>>>> Thanks,
>>>>>>> Grant
>>>>>>>
>>>>>>> On Mon, Aug 17, 2015 at 12:09 AM, Grant Henke <
>> ghenke@cloudera.com
>>>>
>>>>>> wrote:
>>>>>>>
>>>>>>>> +1 to that suggestion. Though I suspect that requires a
>> committer
>>> to
>>>>> do.
>>>>>>>> Making it part of the standard commit process could work too.
>>>>>>>> On Aug 16, 2015 11:01 PM, "Gwen Shapira" <gw...@confluent.io>
>>> wrote:
>>>>>>>>
>>>>>>>>> BTW. I think it will be great for Apache Kafka to have a 0.8.2
>>>>> "release
>>>>>>>>> manager" who's role is to cherrypick low-risk bug-fixes into
>> the
>>>>> 0.8.2
>>>>>>>>> branch and once enough bug fixes happened (or if sufficiently
>>>>> critical
>>>>>>>>> fixes happened) to roll out a new maintenance release (with
>>> every 3
>>>>>> month
>>>>>>>>> as a reasonable bugfix release target).
>>>>>>>>>
>>>>>>>>> This will add some predictability regarding how fast we release
>>>> fixes
>>>>>> for
>>>>>>>>> bugs.
>>>>>>>>>
>>>>>>>>> Gwen
>>>>>>>>>
>>>>>>>>> On Sun, Aug 16, 2015 at 8:09 PM, Jeff Holoman <
>>>> jholoman@cloudera.com
>>>>>>
>>>>>>>>> wrote:
>>>>>>>>>
>>>>>>>>>> +1 for the release and also including
>>>>>>>>>>
>>>>>>>>>> https://issues.apache.org/jira/browse/KAFKA-2114
>>>>>>>>>>
>>>>>>>>>> Thanks
>>>>>>>>>>
>>>>>>>>>> Jeff
>>>>>>>>>>
>>>>>>>>>> On Sun, Aug 16, 2015 at 2:51 PM, Stevo Slavić <
>>> sslavic@gmail.com
>>>>>
>>>>>>>>> wrote:
>>>>>>>>>>
>>>>>>>>>>> +1 (non-binding) for 0.8.2.2 release
>>>>>>>>>>>
>>>>>>>>>>> Would be nice to include in that release new producer
>>>> resiliency
>>>>>> bug
>>>>>>>>>> fixes
>>>>>>>>>>> https://issues.apache.org/jira/browse/KAFKA-1788 and
>>>>>>>>>>> https://issues.apache.org/jira/browse/KAFKA-2120
>>>>>>>>>>>
>>>>>>>>>>> On Fri, Aug 14, 2015 at 4:03 PM, Gwen Shapira <
>>>> gwen@confluent.io
>>>>>>
>>>>>>>>> wrote:
>>>>>>>>>>>
>>>>>>>>>>>> Will be nice to include Kafka-2308 and fix two critical
>>>> snappy
>>>>>>>>> issues
>>>>>>>>>> in
>>>>>>>>>>>> the maintenance release.
>>>>>>>>>>>>
>>>>>>>>>>>> Gwen
>>>>>>>>>>>> On Aug 14, 2015 6:16 AM, "Grant Henke" <
>>> ghenke@cloudera.com>
>>>>>>>>> wrote:
>>>>>>>>>>>>
>>>>>>>>>>>>> Just to clarify. Will KAFKA-2189 be the only patch in
>> the
>>>>>>>>> release?
>>>>>>>>>>>>>
>>>>>>>>>>>>> On Fri, Aug 14, 2015 at 7:35 AM, Manikumar Reddy <
>>>>>>>>>> kumar@nmsworks.co.in
>>>>>>>>>>>>
>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>
>>>>>>>>>>>>>> +1 for 0.8.2.2 release
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> On Fri, Aug 14, 2015 at 5:49 PM, Ismael Juma <
>>>>>>>>> ismael@juma.me.uk>
>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> I think this is a good idea as the change is
>> minimal
>>> on
>>>>> our
>>>>>>>>> side
>>>>>>>>>>> and
>>>>>>>>>>>> it
>>>>>>>>>>>>>> has
>>>>>>>>>>>>>>> been tested in production for some time by the
>>>> reporter.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Best,
>>>>>>>>>>>>>>> Ismael
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> On Fri, Aug 14, 2015 at 1:15 PM, Jun Rao <
>>>>> jun@confluent.io
>>>>>>>
>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Hi, Everyone,
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Since the release of Kafka 0.8.2.1, a number of
>>>> people
>>>>>> have
>>>>>>>>>>>> reported
>>>>>>>>>>>>> an
>>>>>>>>>>>>>>>> issue with snappy compression (
>>>>>>>>>>>>>>>> https://issues.apache.org/jira/browse/KAFKA-2189
>> ).
>>>>>>>>> Basically,
>>>>>>>>>> if
>>>>>>>>>>>>> they
>>>>>>>>>>>>>>> use
>>>>>>>>>>>>>>>> snappy in 0.8.2.1, they will experience a 2-3X
>>> space
>>>>>>>>> increase.
>>>>>>>>>>> The
>>>>>>>>>>>>>> issue
>>>>>>>>>>>>>>>> has since been fixed in trunk (just a snappy jar
>>>>>> upgrade).
>>>>>>>>>> Since
>>>>>>>>>>>>> 0.8.3
>>>>>>>>>>>>>> is
>>>>>>>>>>>>>>>> still a few months away, it may make sense to do
>> an
>>>>>> 0.8.2.2
>>>>>>>>>>> release
>>>>>>>>>>>>>> just
>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>> fix this issue. Any objections?
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Jun
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> --
>>>>>>>>>>>>> Grant Henke
>>>>>>>>>>>>> Software Engineer | Cloudera
>>>>>>>>>>>>> grant@cloudera.com | twitter.com/gchenke |
>>>>>>>>>> linkedin.com/in/granthenke
>>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> --
>>>>>>>>>> Jeff Holoman
>>>>>>>>>> Systems Engineer
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> --
>>>>>>> Grant Henke
>>>>>>> Software Engineer | Cloudera
>>>>>>> grant@cloudera.com | twitter.com/gchenke |
>>>> linkedin.com/in/granthenke
>>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> --
>>>>>> Grant Henke
>>>>>> Software Engineer | Cloudera
>>>>>> grant@cloudera.com | twitter.com/gchenke |
>>> linkedin.com/in/granthenke
>>>>>>
>>>>>
>>>>
>>>>
>>>>
>>>> --
>>>> Grant Henke
>>>> Software Engineer | Cloudera
>>>> grant@cloudera.com | twitter.com/gchenke | linkedin.com/in/granthenke
>>>>
>>>
>>
>
>
>
> --
> Thanks,
> Neha
Re: [DISCUSSION] Kafka 0.8.2.2 release?
Posted by Neha Narkhede <ne...@confluent.io>.
How about looking at the scope for the 0.8.3 release first before we cut
yet another point release off of 0.8.2.2? Each release includes some
overhead and if there is a larger release in the works, it might be worth
working on getting that. My take is that the 2 things the community has
been waiting for is SSL support and the new consumer and we have been
promising to get 0.8.3 with both those features for several months now.
Looking at the progress on both, it seems we are very close to getting both
those checked in and it looks like we should get there in another 5-6
weeks. Furthermore, both of these features are large and I anticipate us
receiving feedback and bugs that will require a couple of point releases on
top of 0.8.3 anyway. One possibility is to work on 0.8.3 together now and
get the community to use the newly released features, gather feedback and
do point releases incorporating that feedback and iterate on it.
We could absolutely do both 0.8.2.2 and 0.8.3. What I'd ask for is for us
to look at the 0.8.3 timeline too and make a call whether 0.8.2.2 still
makes sense.
Thanks,
Neha
On Tue, Aug 18, 2015 at 10:24 AM, Gwen Shapira <gw...@confluent.io> wrote:
> Thanks Jun.
>
> I updated the list with your suggestions.
> If anyone feels we are missing a critical patch for 0.8.2.2, please speak
> up.
>
> Gwen
>
> On Mon, Aug 17, 2015 at 5:40 PM, Jun Rao <ju...@confluent.io> wrote:
>
> > Hi, Grant,
> >
> > I took a look at that list. None of those is really critical as you said.
> > So, I'd suggest that we not include those to minimize the scope of the
> > release.
> >
> > Thanks,
> >
> > Jun
> >
> > On Mon, Aug 17, 2015 at 5:16 PM, Grant Henke <gh...@cloudera.com>
> wrote:
> >
> > > Thanks Gwen.
> > >
> > > I updated a few small things on the wiki page.
> > >
> > > Below is a list of jiras I think could also be marked as included. All
> of
> > > these, though not super critical, seem like fairly small and low risk
> > > changes that help avoid potentially confusing issues or errors for
> users.
> > >
> > > KAFKA-2012
> > > KAFKA-972
> > > KAFKA-2337 & KAFKA-2393
> > > KAFKA-1867
> > > KAFKA-2407
> > > KAFKA-2234
> > > KAFKA-1866
> > > KAFKA-2345 & KAFKA-2355
> > >
> > > thoughts?
> > >
> > > Thank you,
> > > Grant
> > >
> > > On Mon, Aug 17, 2015 at 4:56 PM, Gwen Shapira <gw...@confluent.io>
> wrote:
> > >
> > > > Thanks for creating a list, Grant!
> > > >
> > > > I placed it on the wiki with a quick evaluation of the content and
> > > whether
> > > > it should be in 0.8.2.2:
> > > >
> > > >
> > >
> >
> https://cwiki.apache.org/confluence/display/KAFKA/Proposed+patches+for+0.8.2.2
> > > >
> > > > I'm attempting to only cherrypick fixes that are both important for
> > large
> > > > number of users (or very critical to some users) and very safe
> (mostly
> > > > judged by the size of the change, but not only)
> > > >
> > > > If your favorite bugfix is missing from the list, or is there but
> > marked
> > > > "No", please let us know (in this thread) what we are missing and why
> > it
> > > is
> > > > both important and safe.
> > > > Also, if I accidentally included something you consider unsafe, speak
> > up!
> > > >
> > > > Gwen
> > > >
> > > > On Mon, Aug 17, 2015 at 8:17 AM, Grant Henke <gh...@cloudera.com>
> > > wrote:
> > > >
> > > > > +dev
> > > > >
> > > > > Adding dev list back in. Somehow it got dropped.
> > > > >
> > > > >
> > > > > On Mon, Aug 17, 2015 at 10:16 AM, Grant Henke <ghenke@cloudera.com
> >
> > > > wrote:
> > > > >
> > > > > > Below is a list of candidate bug fix jiras marked fixed for
> 0.8.3.
> > I
> > > > > don't
> > > > > > suspect all of these will (or should) make it into the release
> but
> > > this
> > > > > > should be a relatively complete list to work from:
> > > > > >
> > > > > > - KAFKA-2114 <
> https://issues.apache.org/jira/browse/KAFKA-2114
> > >:
> > > > > Unable
> > > > > > to change min.insync.replicas default
> > > > > > - KAFKA-1702 <
> https://issues.apache.org/jira/browse/KAFKA-1702
> > >:
> > > > > > Messages silently Lost by producer
> > > > > > - KAFKA-2012 <
> https://issues.apache.org/jira/browse/KAFKA-2012
> > >:
> > > > > > Broker should automatically handle corrupt index files
> > > > > > - KAFKA-2406 <
> https://issues.apache.org/jira/browse/KAFKA-2406
> > >:
> > > > ISR
> > > > > > propagation should be throttled to avoid overwhelming
> > controller.
> > > > > > - KAFKA-2336 <
> https://issues.apache.org/jira/browse/KAFKA-2336
> > >:
> > > > > > Changing offsets.topic.num.partitions after the offset topic
> is
> > > > > created
> > > > > > breaks consumer group partition assignment
> > > > > > - KAFKA-2337 <
> https://issues.apache.org/jira/browse/KAFKA-2337
> > >:
> > > > > Verify
> > > > > > that metric names will not collide when creating new topics
> > > > > > - KAFKA-2393 <
> https://issues.apache.org/jira/browse/KAFKA-2393
> > >:
> > > > > > Correctly Handle InvalidTopicException in
> > > > KafkaApis.getTopicMetadata()
> > > > > > - KAFKA-2189 <
> https://issues.apache.org/jira/browse/KAFKA-2189
> > >:
> > > > > Snappy
> > > > > > compression of message batches less efficient in 0.8.2.1
> > > > > > - KAFKA-2308 <
> https://issues.apache.org/jira/browse/KAFKA-2308
> > >:
> > > > New
> > > > > > producer + Snappy face un-compression errors after broker
> > restart
> > > > > > - KAFKA-2042 <
> https://issues.apache.org/jira/browse/KAFKA-2042
> > >:
> > > > New
> > > > > > producer metadata update always get all topics.
> > > > > > - KAFKA-1367 <
> https://issues.apache.org/jira/browse/KAFKA-1367
> > >:
> > > > > Broker
> > > > > > topic metadata not kept in sync with ZooKeeper
> > > > > > - KAFKA-972 <https://issues.apache.org/jira/browse/KAFKA-972
> >:
> > > > > MetadataRequest
> > > > > > returns stale list of brokers
> > > > > > - KAFKA-1867 <
> https://issues.apache.org/jira/browse/KAFKA-1867
> > >:
> > > > > liveBroker
> > > > > > list not updated on a cluster with no topics
> > > > > > - KAFKA-1650 <
> https://issues.apache.org/jira/browse/KAFKA-1650
> > >:
> > > > > Mirror
> > > > > > Maker could lose data on unclean shutdown.
> > > > > > - KAFKA-2009 <
> https://issues.apache.org/jira/browse/KAFKA-2009
> > >:
> > > > Fix
> > > > > > UncheckedOffset.removeOffset synchronization and trace logging
> > > issue
> > > > > in
> > > > > > mirror maker
> > > > > > - KAFKA-2407 <
> https://issues.apache.org/jira/browse/KAFKA-2407
> > >:
> > > > Only
> > > > > > create a log directory when it will be used
> > > > > > - KAFKA-2327 <
> https://issues.apache.org/jira/browse/KAFKA-2327
> > >:
> > > > > > broker doesn't start if config defines advertised.host but not
> > > > > > advertised.port
> > > > > > - KAFKA-1788: producer record can stay in RecordAccumulator
> > > forever
> > > > if
> > > > > > leader is no available
> > > > > > - KAFKA-2234 <
> https://issues.apache.org/jira/browse/KAFKA-2234
> > >:
> > > > > > Partition reassignment of a nonexistent topic prevents future
> > > > > reassignments
> > > > > > - KAFKA-2096 <
> https://issues.apache.org/jira/browse/KAFKA-2096
> > >:
> > > > > > Enable keepalive socket option for broker to prevent socket
> leak
> > > > > > - KAFKA-1057 <
> https://issues.apache.org/jira/browse/KAFKA-1057
> > >:
> > > > Trim
> > > > > > whitespaces from user specified configs
> > > > > > - KAFKA-1641 <
> https://issues.apache.org/jira/browse/KAFKA-1641
> > >:
> > > > Log
> > > > > > cleaner exits if last cleaned offset is lower than earliest
> > offset
> > > > > > - KAFKA-1648 <
> https://issues.apache.org/jira/browse/KAFKA-1648
> > >:
> > > > > Round
> > > > > > robin consumer balance throws an NPE when there are no topics
> > > > > > - KAFKA-1724 <
> https://issues.apache.org/jira/browse/KAFKA-1724
> > >:
> > > > > > Errors after reboot in single node setup
> > > > > > - KAFKA-1758 <
> https://issues.apache.org/jira/browse/KAFKA-1758
> > >:
> > > > > > corrupt recovery file prevents startup
> > > > > > - KAFKA-1866 <
> https://issues.apache.org/jira/browse/KAFKA-1866
> > >:
> > > > > > LogStartOffset gauge throws exceptions after log.delete()
> > > > > > - KAFKA-1883 <
> https://issues.apache.org/jira/browse/KAFKA-1883
> > >:
> > > > > NullPointerException
> > > > > > in RequestSendThread
> > > > > > - KAFKA-1896 <
> https://issues.apache.org/jira/browse/KAFKA-1896
> > >:
> > > > > > Record size funcition of record in mirror maker hit NPE when
> the
> > > > > message
> > > > > > value is null.
> > > > > > - KAFKA-2101 <
> https://issues.apache.org/jira/browse/KAFKA-2101
> > >:
> > > > > > Metric metadata-age is reset on a failed update
> > > > > > - KAFKA-2112 <
> https://issues.apache.org/jira/browse/KAFKA-2112
> > >:
> > > > make
> > > > > > overflowWheel volatile
> > > > > > - KAFKA-2117 <
> https://issues.apache.org/jira/browse/KAFKA-2117
> > >:
> > > > > > OffsetManager uses incorrect field for metadata
> > > > > > - KAFKA-2164 <
> https://issues.apache.org/jira/browse/KAFKA-2164
> > >:
> > > > > > ReplicaFetcherThread: suspicious log message on reset offset
> > > > > > - KAFKA-1668 <
> https://issues.apache.org/jira/browse/KAFKA-1668
> > >:
> > > > > > TopicCommand doesn't warn if --topic argument doesn't match
> any
> > > > topics
> > > > > > - KAFKA-2198 <
> https://issues.apache.org/jira/browse/KAFKA-2198
> > >:
> > > > > > kafka-topics.sh exits with 0 status on failures
> > > > > > - KAFKA-2235 <
> https://issues.apache.org/jira/browse/KAFKA-2235
> > >:
> > > > > > LogCleaner offset map overflow
> > > > > > - KAFKA-2241 <
> https://issues.apache.org/jira/browse/KAFKA-2241
> > >:
> > > > > > AbstractFetcherThread.shutdown() should not block on
> > > > > > ReadableByteChannel.read(buffer)
> > > > > > - KAFKA-2272 <
> https://issues.apache.org/jira/browse/KAFKA-2272
> > >:
> > > > > > listeners endpoint parsing fails if the hostname has capital
> > > letter
> > > > > > - KAFKA-2345 <
> https://issues.apache.org/jira/browse/KAFKA-2345
> > >:
> > > > > > Attempt to delete a topic already marked for deletion throws
> > > > > > ZkNodeExistsException
> > > > > > - KAFKA-2353 <
> https://issues.apache.org/jira/browse/KAFKA-2353
> > >:
> > > > > > SocketServer.Processor should catch exception and close the
> > socket
> > > > > properly
> > > > > > in configureNewConnections.
> > > > > > - KAFKA-1836 <
> https://issues.apache.org/jira/browse/KAFKA-1836
> > >:
> > > > > > metadata.fetch.timeout.ms set to zero blocks forever
> > > > > > - KAFKA-2317 <
> https://issues.apache.org/jira/browse/KAFKA-2317
> > >:
> > > > > De-register
> > > > > > isrChangeNotificationListener on controller resignation
> > > > > >
> > > > > > Note: KAFKA-2120 <
> https://issues.apache.org/jira/browse/KAFKA-2120
> > >
> > > &
> > > > > > KAFKA-2421 <https://issues.apache.org/jira/browse/KAFKA-2421>
> were
> > > > > > mentioned in previous emails, but are not in the list because
> they
> > > are
> > > > > not
> > > > > > committed yet.
> > > > > >
> > > > > > Hope that helps the effort.
> > > > > >
> > > > > > Thanks,
> > > > > > Grant
> > > > > >
> > > > > > On Mon, Aug 17, 2015 at 12:09 AM, Grant Henke <
> ghenke@cloudera.com
> > >
> > > > > wrote:
> > > > > >
> > > > > >> +1 to that suggestion. Though I suspect that requires a
> committer
> > to
> > > > do.
> > > > > >> Making it part of the standard commit process could work too.
> > > > > >> On Aug 16, 2015 11:01 PM, "Gwen Shapira" <gw...@confluent.io>
> > wrote:
> > > > > >>
> > > > > >>> BTW. I think it will be great for Apache Kafka to have a 0.8.2
> > > > "release
> > > > > >>> manager" who's role is to cherrypick low-risk bug-fixes into
> the
> > > > 0.8.2
> > > > > >>> branch and once enough bug fixes happened (or if sufficiently
> > > > critical
> > > > > >>> fixes happened) to roll out a new maintenance release (with
> > every 3
> > > > > month
> > > > > >>> as a reasonable bugfix release target).
> > > > > >>>
> > > > > >>> This will add some predictability regarding how fast we release
> > > fixes
> > > > > for
> > > > > >>> bugs.
> > > > > >>>
> > > > > >>> Gwen
> > > > > >>>
> > > > > >>> On Sun, Aug 16, 2015 at 8:09 PM, Jeff Holoman <
> > > jholoman@cloudera.com
> > > > >
> > > > > >>> wrote:
> > > > > >>>
> > > > > >>> > +1 for the release and also including
> > > > > >>> >
> > > > > >>> > https://issues.apache.org/jira/browse/KAFKA-2114
> > > > > >>> >
> > > > > >>> > Thanks
> > > > > >>> >
> > > > > >>> > Jeff
> > > > > >>> >
> > > > > >>> > On Sun, Aug 16, 2015 at 2:51 PM, Stevo Slavić <
> > sslavic@gmail.com
> > > >
> > > > > >>> wrote:
> > > > > >>> >
> > > > > >>> > > +1 (non-binding) for 0.8.2.2 release
> > > > > >>> > >
> > > > > >>> > > Would be nice to include in that release new producer
> > > resiliency
> > > > > bug
> > > > > >>> > fixes
> > > > > >>> > > https://issues.apache.org/jira/browse/KAFKA-1788 and
> > > > > >>> > > https://issues.apache.org/jira/browse/KAFKA-2120
> > > > > >>> > >
> > > > > >>> > > On Fri, Aug 14, 2015 at 4:03 PM, Gwen Shapira <
> > > gwen@confluent.io
> > > > >
> > > > > >>> wrote:
> > > > > >>> > >
> > > > > >>> > > > Will be nice to include Kafka-2308 and fix two critical
> > > snappy
> > > > > >>> issues
> > > > > >>> > in
> > > > > >>> > > > the maintenance release.
> > > > > >>> > > >
> > > > > >>> > > > Gwen
> > > > > >>> > > > On Aug 14, 2015 6:16 AM, "Grant Henke" <
> > ghenke@cloudera.com>
> > > > > >>> wrote:
> > > > > >>> > > >
> > > > > >>> > > > > Just to clarify. Will KAFKA-2189 be the only patch in
> the
> > > > > >>> release?
> > > > > >>> > > > >
> > > > > >>> > > > > On Fri, Aug 14, 2015 at 7:35 AM, Manikumar Reddy <
> > > > > >>> > kumar@nmsworks.co.in
> > > > > >>> > > >
> > > > > >>> > > > > wrote:
> > > > > >>> > > > >
> > > > > >>> > > > > > +1 for 0.8.2.2 release
> > > > > >>> > > > > >
> > > > > >>> > > > > > On Fri, Aug 14, 2015 at 5:49 PM, Ismael Juma <
> > > > > >>> ismael@juma.me.uk>
> > > > > >>> > > > wrote:
> > > > > >>> > > > > >
> > > > > >>> > > > > > > I think this is a good idea as the change is
> minimal
> > on
> > > > our
> > > > > >>> side
> > > > > >>> > > and
> > > > > >>> > > > it
> > > > > >>> > > > > > has
> > > > > >>> > > > > > > been tested in production for some time by the
> > > reporter.
> > > > > >>> > > > > > >
> > > > > >>> > > > > > > Best,
> > > > > >>> > > > > > > Ismael
> > > > > >>> > > > > > >
> > > > > >>> > > > > > > On Fri, Aug 14, 2015 at 1:15 PM, Jun Rao <
> > > > jun@confluent.io
> > > > > >
> > > > > >>> > wrote:
> > > > > >>> > > > > > >
> > > > > >>> > > > > > > > Hi, Everyone,
> > > > > >>> > > > > > > >
> > > > > >>> > > > > > > > Since the release of Kafka 0.8.2.1, a number of
> > > people
> > > > > have
> > > > > >>> > > > reported
> > > > > >>> > > > > an
> > > > > >>> > > > > > > > issue with snappy compression (
> > > > > >>> > > > > > > > https://issues.apache.org/jira/browse/KAFKA-2189
> ).
> > > > > >>> Basically,
> > > > > >>> > if
> > > > > >>> > > > > they
> > > > > >>> > > > > > > use
> > > > > >>> > > > > > > > snappy in 0.8.2.1, they will experience a 2-3X
> > space
> > > > > >>> increase.
> > > > > >>> > > The
> > > > > >>> > > > > > issue
> > > > > >>> > > > > > > > has since been fixed in trunk (just a snappy jar
> > > > > upgrade).
> > > > > >>> > Since
> > > > > >>> > > > > 0.8.3
> > > > > >>> > > > > > is
> > > > > >>> > > > > > > > still a few months away, it may make sense to do
> an
> > > > > 0.8.2.2
> > > > > >>> > > release
> > > > > >>> > > > > > just
> > > > > >>> > > > > > > to
> > > > > >>> > > > > > > > fix this issue. Any objections?
> > > > > >>> > > > > > > >
> > > > > >>> > > > > > > > Thanks,
> > > > > >>> > > > > > > >
> > > > > >>> > > > > > > > Jun
> > > > > >>> > > > > > > >
> > > > > >>> > > > > > >
> > > > > >>> > > > > >
> > > > > >>> > > > >
> > > > > >>> > > > >
> > > > > >>> > > > >
> > > > > >>> > > > > --
> > > > > >>> > > > > Grant Henke
> > > > > >>> > > > > Software Engineer | Cloudera
> > > > > >>> > > > > grant@cloudera.com | twitter.com/gchenke |
> > > > > >>> > linkedin.com/in/granthenke
> > > > > >>> > > > >
> > > > > >>> > > >
> > > > > >>> > >
> > > > > >>> >
> > > > > >>> >
> > > > > >>> >
> > > > > >>> > --
> > > > > >>> > Jeff Holoman
> > > > > >>> > Systems Engineer
> > > > > >>> >
> > > > > >>>
> > > > > >>
> > > > > >
> > > > > >
> > > > > > --
> > > > > > Grant Henke
> > > > > > Software Engineer | Cloudera
> > > > > > grant@cloudera.com | twitter.com/gchenke |
> > > linkedin.com/in/granthenke
> > > > > >
> > > > >
> > > > >
> > > > >
> > > > > --
> > > > > Grant Henke
> > > > > Software Engineer | Cloudera
> > > > > grant@cloudera.com | twitter.com/gchenke |
> > linkedin.com/in/granthenke
> > > > >
> > > >
> > >
> > >
> > >
> > > --
> > > Grant Henke
> > > Software Engineer | Cloudera
> > > grant@cloudera.com | twitter.com/gchenke | linkedin.com/in/granthenke
> > >
> >
>
--
Thanks,
Neha
Re: [DISCUSSION] Kafka 0.8.2.2 release?
Posted by Neha Narkhede <ne...@confluent.io>.
How about looking at the scope for the 0.8.3 release first before we cut
yet another point release off of 0.8.2.2? Each release includes some
overhead and if there is a larger release in the works, it might be worth
working on getting that. My take is that the 2 things the community has
been waiting for is SSL support and the new consumer and we have been
promising to get 0.8.3 with both those features for several months now.
Looking at the progress on both, it seems we are very close to getting both
those checked in and it looks like we should get there in another 5-6
weeks. Furthermore, both of these features are large and I anticipate us
receiving feedback and bugs that will require a couple of point releases on
top of 0.8.3 anyway. One possibility is to work on 0.8.3 together now and
get the community to use the newly released features, gather feedback and
do point releases incorporating that feedback and iterate on it.
We could absolutely do both 0.8.2.2 and 0.8.3. What I'd ask for is for us
to look at the 0.8.3 timeline too and make a call whether 0.8.2.2 still
makes sense.
Thanks,
Neha
On Tue, Aug 18, 2015 at 10:24 AM, Gwen Shapira <gw...@confluent.io> wrote:
> Thanks Jun.
>
> I updated the list with your suggestions.
> If anyone feels we are missing a critical patch for 0.8.2.2, please speak
> up.
>
> Gwen
>
> On Mon, Aug 17, 2015 at 5:40 PM, Jun Rao <ju...@confluent.io> wrote:
>
> > Hi, Grant,
> >
> > I took a look at that list. None of those is really critical as you said.
> > So, I'd suggest that we not include those to minimize the scope of the
> > release.
> >
> > Thanks,
> >
> > Jun
> >
> > On Mon, Aug 17, 2015 at 5:16 PM, Grant Henke <gh...@cloudera.com>
> wrote:
> >
> > > Thanks Gwen.
> > >
> > > I updated a few small things on the wiki page.
> > >
> > > Below is a list of jiras I think could also be marked as included. All
> of
> > > these, though not super critical, seem like fairly small and low risk
> > > changes that help avoid potentially confusing issues or errors for
> users.
> > >
> > > KAFKA-2012
> > > KAFKA-972
> > > KAFKA-2337 & KAFKA-2393
> > > KAFKA-1867
> > > KAFKA-2407
> > > KAFKA-2234
> > > KAFKA-1866
> > > KAFKA-2345 & KAFKA-2355
> > >
> > > thoughts?
> > >
> > > Thank you,
> > > Grant
> > >
> > > On Mon, Aug 17, 2015 at 4:56 PM, Gwen Shapira <gw...@confluent.io>
> wrote:
> > >
> > > > Thanks for creating a list, Grant!
> > > >
> > > > I placed it on the wiki with a quick evaluation of the content and
> > > whether
> > > > it should be in 0.8.2.2:
> > > >
> > > >
> > >
> >
> https://cwiki.apache.org/confluence/display/KAFKA/Proposed+patches+for+0.8.2.2
> > > >
> > > > I'm attempting to only cherrypick fixes that are both important for
> > large
> > > > number of users (or very critical to some users) and very safe
> (mostly
> > > > judged by the size of the change, but not only)
> > > >
> > > > If your favorite bugfix is missing from the list, or is there but
> > marked
> > > > "No", please let us know (in this thread) what we are missing and why
> > it
> > > is
> > > > both important and safe.
> > > > Also, if I accidentally included something you consider unsafe, speak
> > up!
> > > >
> > > > Gwen
> > > >
> > > > On Mon, Aug 17, 2015 at 8:17 AM, Grant Henke <gh...@cloudera.com>
> > > wrote:
> > > >
> > > > > +dev
> > > > >
> > > > > Adding dev list back in. Somehow it got dropped.
> > > > >
> > > > >
> > > > > On Mon, Aug 17, 2015 at 10:16 AM, Grant Henke <ghenke@cloudera.com
> >
> > > > wrote:
> > > > >
> > > > > > Below is a list of candidate bug fix jiras marked fixed for
> 0.8.3.
> > I
> > > > > don't
> > > > > > suspect all of these will (or should) make it into the release
> but
> > > this
> > > > > > should be a relatively complete list to work from:
> > > > > >
> > > > > > - KAFKA-2114 <
> https://issues.apache.org/jira/browse/KAFKA-2114
> > >:
> > > > > Unable
> > > > > > to change min.insync.replicas default
> > > > > > - KAFKA-1702 <
> https://issues.apache.org/jira/browse/KAFKA-1702
> > >:
> > > > > > Messages silently Lost by producer
> > > > > > - KAFKA-2012 <
> https://issues.apache.org/jira/browse/KAFKA-2012
> > >:
> > > > > > Broker should automatically handle corrupt index files
> > > > > > - KAFKA-2406 <
> https://issues.apache.org/jira/browse/KAFKA-2406
> > >:
> > > > ISR
> > > > > > propagation should be throttled to avoid overwhelming
> > controller.
> > > > > > - KAFKA-2336 <
> https://issues.apache.org/jira/browse/KAFKA-2336
> > >:
> > > > > > Changing offsets.topic.num.partitions after the offset topic
> is
> > > > > created
> > > > > > breaks consumer group partition assignment
> > > > > > - KAFKA-2337 <
> https://issues.apache.org/jira/browse/KAFKA-2337
> > >:
> > > > > Verify
> > > > > > that metric names will not collide when creating new topics
> > > > > > - KAFKA-2393 <
> https://issues.apache.org/jira/browse/KAFKA-2393
> > >:
> > > > > > Correctly Handle InvalidTopicException in
> > > > KafkaApis.getTopicMetadata()
> > > > > > - KAFKA-2189 <
> https://issues.apache.org/jira/browse/KAFKA-2189
> > >:
> > > > > Snappy
> > > > > > compression of message batches less efficient in 0.8.2.1
> > > > > > - KAFKA-2308 <
> https://issues.apache.org/jira/browse/KAFKA-2308
> > >:
> > > > New
> > > > > > producer + Snappy face un-compression errors after broker
> > restart
> > > > > > - KAFKA-2042 <
> https://issues.apache.org/jira/browse/KAFKA-2042
> > >:
> > > > New
> > > > > > producer metadata update always get all topics.
> > > > > > - KAFKA-1367 <
> https://issues.apache.org/jira/browse/KAFKA-1367
> > >:
> > > > > Broker
> > > > > > topic metadata not kept in sync with ZooKeeper
> > > > > > - KAFKA-972 <https://issues.apache.org/jira/browse/KAFKA-972
> >:
> > > > > MetadataRequest
> > > > > > returns stale list of brokers
> > > > > > - KAFKA-1867 <
> https://issues.apache.org/jira/browse/KAFKA-1867
> > >:
> > > > > liveBroker
> > > > > > list not updated on a cluster with no topics
> > > > > > - KAFKA-1650 <
> https://issues.apache.org/jira/browse/KAFKA-1650
> > >:
> > > > > Mirror
> > > > > > Maker could lose data on unclean shutdown.
> > > > > > - KAFKA-2009 <
> https://issues.apache.org/jira/browse/KAFKA-2009
> > >:
> > > > Fix
> > > > > > UncheckedOffset.removeOffset synchronization and trace logging
> > > issue
> > > > > in
> > > > > > mirror maker
> > > > > > - KAFKA-2407 <
> https://issues.apache.org/jira/browse/KAFKA-2407
> > >:
> > > > Only
> > > > > > create a log directory when it will be used
> > > > > > - KAFKA-2327 <
> https://issues.apache.org/jira/browse/KAFKA-2327
> > >:
> > > > > > broker doesn't start if config defines advertised.host but not
> > > > > > advertised.port
> > > > > > - KAFKA-1788: producer record can stay in RecordAccumulator
> > > forever
> > > > if
> > > > > > leader is no available
> > > > > > - KAFKA-2234 <
> https://issues.apache.org/jira/browse/KAFKA-2234
> > >:
> > > > > > Partition reassignment of a nonexistent topic prevents future
> > > > > reassignments
> > > > > > - KAFKA-2096 <
> https://issues.apache.org/jira/browse/KAFKA-2096
> > >:
> > > > > > Enable keepalive socket option for broker to prevent socket
> leak
> > > > > > - KAFKA-1057 <
> https://issues.apache.org/jira/browse/KAFKA-1057
> > >:
> > > > Trim
> > > > > > whitespaces from user specified configs
> > > > > > - KAFKA-1641 <
> https://issues.apache.org/jira/browse/KAFKA-1641
> > >:
> > > > Log
> > > > > > cleaner exits if last cleaned offset is lower than earliest
> > offset
> > > > > > - KAFKA-1648 <
> https://issues.apache.org/jira/browse/KAFKA-1648
> > >:
> > > > > Round
> > > > > > robin consumer balance throws an NPE when there are no topics
> > > > > > - KAFKA-1724 <
> https://issues.apache.org/jira/browse/KAFKA-1724
> > >:
> > > > > > Errors after reboot in single node setup
> > > > > > - KAFKA-1758 <
> https://issues.apache.org/jira/browse/KAFKA-1758
> > >:
> > > > > > corrupt recovery file prevents startup
> > > > > > - KAFKA-1866 <
> https://issues.apache.org/jira/browse/KAFKA-1866
> > >:
> > > > > > LogStartOffset gauge throws exceptions after log.delete()
> > > > > > - KAFKA-1883 <
> https://issues.apache.org/jira/browse/KAFKA-1883
> > >:
> > > > > NullPointerException
> > > > > > in RequestSendThread
> > > > > > - KAFKA-1896 <
> https://issues.apache.org/jira/browse/KAFKA-1896
> > >:
> > > > > > Record size funcition of record in mirror maker hit NPE when
> the
> > > > > message
> > > > > > value is null.
> > > > > > - KAFKA-2101 <
> https://issues.apache.org/jira/browse/KAFKA-2101
> > >:
> > > > > > Metric metadata-age is reset on a failed update
> > > > > > - KAFKA-2112 <
> https://issues.apache.org/jira/browse/KAFKA-2112
> > >:
> > > > make
> > > > > > overflowWheel volatile
> > > > > > - KAFKA-2117 <
> https://issues.apache.org/jira/browse/KAFKA-2117
> > >:
> > > > > > OffsetManager uses incorrect field for metadata
> > > > > > - KAFKA-2164 <
> https://issues.apache.org/jira/browse/KAFKA-2164
> > >:
> > > > > > ReplicaFetcherThread: suspicious log message on reset offset
> > > > > > - KAFKA-1668 <
> https://issues.apache.org/jira/browse/KAFKA-1668
> > >:
> > > > > > TopicCommand doesn't warn if --topic argument doesn't match
> any
> > > > topics
> > > > > > - KAFKA-2198 <
> https://issues.apache.org/jira/browse/KAFKA-2198
> > >:
> > > > > > kafka-topics.sh exits with 0 status on failures
> > > > > > - KAFKA-2235 <
> https://issues.apache.org/jira/browse/KAFKA-2235
> > >:
> > > > > > LogCleaner offset map overflow
> > > > > > - KAFKA-2241 <
> https://issues.apache.org/jira/browse/KAFKA-2241
> > >:
> > > > > > AbstractFetcherThread.shutdown() should not block on
> > > > > > ReadableByteChannel.read(buffer)
> > > > > > - KAFKA-2272 <
> https://issues.apache.org/jira/browse/KAFKA-2272
> > >:
> > > > > > listeners endpoint parsing fails if the hostname has capital
> > > letter
> > > > > > - KAFKA-2345 <
> https://issues.apache.org/jira/browse/KAFKA-2345
> > >:
> > > > > > Attempt to delete a topic already marked for deletion throws
> > > > > > ZkNodeExistsException
> > > > > > - KAFKA-2353 <
> https://issues.apache.org/jira/browse/KAFKA-2353
> > >:
> > > > > > SocketServer.Processor should catch exception and close the
> > socket
> > > > > properly
> > > > > > in configureNewConnections.
> > > > > > - KAFKA-1836 <
> https://issues.apache.org/jira/browse/KAFKA-1836
> > >:
> > > > > > metadata.fetch.timeout.ms set to zero blocks forever
> > > > > > - KAFKA-2317 <
> https://issues.apache.org/jira/browse/KAFKA-2317
> > >:
> > > > > De-register
> > > > > > isrChangeNotificationListener on controller resignation
> > > > > >
> > > > > > Note: KAFKA-2120 <
> https://issues.apache.org/jira/browse/KAFKA-2120
> > >
> > > &
> > > > > > KAFKA-2421 <https://issues.apache.org/jira/browse/KAFKA-2421>
> were
> > > > > > mentioned in previous emails, but are not in the list because
> they
> > > are
> > > > > not
> > > > > > committed yet.
> > > > > >
> > > > > > Hope that helps the effort.
> > > > > >
> > > > > > Thanks,
> > > > > > Grant
> > > > > >
> > > > > > On Mon, Aug 17, 2015 at 12:09 AM, Grant Henke <
> ghenke@cloudera.com
> > >
> > > > > wrote:
> > > > > >
> > > > > >> +1 to that suggestion. Though I suspect that requires a
> committer
> > to
> > > > do.
> > > > > >> Making it part of the standard commit process could work too.
> > > > > >> On Aug 16, 2015 11:01 PM, "Gwen Shapira" <gw...@confluent.io>
> > wrote:
> > > > > >>
> > > > > >>> BTW. I think it will be great for Apache Kafka to have a 0.8.2
> > > > "release
> > > > > >>> manager" who's role is to cherrypick low-risk bug-fixes into
> the
> > > > 0.8.2
> > > > > >>> branch and once enough bug fixes happened (or if sufficiently
> > > > critical
> > > > > >>> fixes happened) to roll out a new maintenance release (with
> > every 3
> > > > > month
> > > > > >>> as a reasonable bugfix release target).
> > > > > >>>
> > > > > >>> This will add some predictability regarding how fast we release
> > > fixes
> > > > > for
> > > > > >>> bugs.
> > > > > >>>
> > > > > >>> Gwen
> > > > > >>>
> > > > > >>> On Sun, Aug 16, 2015 at 8:09 PM, Jeff Holoman <
> > > jholoman@cloudera.com
> > > > >
> > > > > >>> wrote:
> > > > > >>>
> > > > > >>> > +1 for the release and also including
> > > > > >>> >
> > > > > >>> > https://issues.apache.org/jira/browse/KAFKA-2114
> > > > > >>> >
> > > > > >>> > Thanks
> > > > > >>> >
> > > > > >>> > Jeff
> > > > > >>> >
> > > > > >>> > On Sun, Aug 16, 2015 at 2:51 PM, Stevo Slavić <
> > sslavic@gmail.com
> > > >
> > > > > >>> wrote:
> > > > > >>> >
> > > > > >>> > > +1 (non-binding) for 0.8.2.2 release
> > > > > >>> > >
> > > > > >>> > > Would be nice to include in that release new producer
> > > resiliency
> > > > > bug
> > > > > >>> > fixes
> > > > > >>> > > https://issues.apache.org/jira/browse/KAFKA-1788 and
> > > > > >>> > > https://issues.apache.org/jira/browse/KAFKA-2120
> > > > > >>> > >
> > > > > >>> > > On Fri, Aug 14, 2015 at 4:03 PM, Gwen Shapira <
> > > gwen@confluent.io
> > > > >
> > > > > >>> wrote:
> > > > > >>> > >
> > > > > >>> > > > Will be nice to include Kafka-2308 and fix two critical
> > > snappy
> > > > > >>> issues
> > > > > >>> > in
> > > > > >>> > > > the maintenance release.
> > > > > >>> > > >
> > > > > >>> > > > Gwen
> > > > > >>> > > > On Aug 14, 2015 6:16 AM, "Grant Henke" <
> > ghenke@cloudera.com>
> > > > > >>> wrote:
> > > > > >>> > > >
> > > > > >>> > > > > Just to clarify. Will KAFKA-2189 be the only patch in
> the
> > > > > >>> release?
> > > > > >>> > > > >
> > > > > >>> > > > > On Fri, Aug 14, 2015 at 7:35 AM, Manikumar Reddy <
> > > > > >>> > kumar@nmsworks.co.in
> > > > > >>> > > >
> > > > > >>> > > > > wrote:
> > > > > >>> > > > >
> > > > > >>> > > > > > +1 for 0.8.2.2 release
> > > > > >>> > > > > >
> > > > > >>> > > > > > On Fri, Aug 14, 2015 at 5:49 PM, Ismael Juma <
> > > > > >>> ismael@juma.me.uk>
> > > > > >>> > > > wrote:
> > > > > >>> > > > > >
> > > > > >>> > > > > > > I think this is a good idea as the change is
> minimal
> > on
> > > > our
> > > > > >>> side
> > > > > >>> > > and
> > > > > >>> > > > it
> > > > > >>> > > > > > has
> > > > > >>> > > > > > > been tested in production for some time by the
> > > reporter.
> > > > > >>> > > > > > >
> > > > > >>> > > > > > > Best,
> > > > > >>> > > > > > > Ismael
> > > > > >>> > > > > > >
> > > > > >>> > > > > > > On Fri, Aug 14, 2015 at 1:15 PM, Jun Rao <
> > > > jun@confluent.io
> > > > > >
> > > > > >>> > wrote:
> > > > > >>> > > > > > >
> > > > > >>> > > > > > > > Hi, Everyone,
> > > > > >>> > > > > > > >
> > > > > >>> > > > > > > > Since the release of Kafka 0.8.2.1, a number of
> > > people
> > > > > have
> > > > > >>> > > > reported
> > > > > >>> > > > > an
> > > > > >>> > > > > > > > issue with snappy compression (
> > > > > >>> > > > > > > > https://issues.apache.org/jira/browse/KAFKA-2189
> ).
> > > > > >>> Basically,
> > > > > >>> > if
> > > > > >>> > > > > they
> > > > > >>> > > > > > > use
> > > > > >>> > > > > > > > snappy in 0.8.2.1, they will experience a 2-3X
> > space
> > > > > >>> increase.
> > > > > >>> > > The
> > > > > >>> > > > > > issue
> > > > > >>> > > > > > > > has since been fixed in trunk (just a snappy jar
> > > > > upgrade).
> > > > > >>> > Since
> > > > > >>> > > > > 0.8.3
> > > > > >>> > > > > > is
> > > > > >>> > > > > > > > still a few months away, it may make sense to do
> an
> > > > > 0.8.2.2
> > > > > >>> > > release
> > > > > >>> > > > > > just
> > > > > >>> > > > > > > to
> > > > > >>> > > > > > > > fix this issue. Any objections?
> > > > > >>> > > > > > > >
> > > > > >>> > > > > > > > Thanks,
> > > > > >>> > > > > > > >
> > > > > >>> > > > > > > > Jun
> > > > > >>> > > > > > > >
> > > > > >>> > > > > > >
> > > > > >>> > > > > >
> > > > > >>> > > > >
> > > > > >>> > > > >
> > > > > >>> > > > >
> > > > > >>> > > > > --
> > > > > >>> > > > > Grant Henke
> > > > > >>> > > > > Software Engineer | Cloudera
> > > > > >>> > > > > grant@cloudera.com | twitter.com/gchenke |
> > > > > >>> > linkedin.com/in/granthenke
> > > > > >>> > > > >
> > > > > >>> > > >
> > > > > >>> > >
> > > > > >>> >
> > > > > >>> >
> > > > > >>> >
> > > > > >>> > --
> > > > > >>> > Jeff Holoman
> > > > > >>> > Systems Engineer
> > > > > >>> >
> > > > > >>>
> > > > > >>
> > > > > >
> > > > > >
> > > > > > --
> > > > > > Grant Henke
> > > > > > Software Engineer | Cloudera
> > > > > > grant@cloudera.com | twitter.com/gchenke |
> > > linkedin.com/in/granthenke
> > > > > >
> > > > >
> > > > >
> > > > >
> > > > > --
> > > > > Grant Henke
> > > > > Software Engineer | Cloudera
> > > > > grant@cloudera.com | twitter.com/gchenke |
> > linkedin.com/in/granthenke
> > > > >
> > > >
> > >
> > >
> > >
> > > --
> > > Grant Henke
> > > Software Engineer | Cloudera
> > > grant@cloudera.com | twitter.com/gchenke | linkedin.com/in/granthenke
> > >
> >
>
--
Thanks,
Neha
Re: [DISCUSSION] Kafka 0.8.2.2 release?
Posted by Gwen Shapira <gw...@confluent.io>.
Thanks Jun.
I updated the list with your suggestions.
If anyone feels we are missing a critical patch for 0.8.2.2, please speak
up.
Gwen
On Mon, Aug 17, 2015 at 5:40 PM, Jun Rao <ju...@confluent.io> wrote:
> Hi, Grant,
>
> I took a look at that list. None of those is really critical as you said.
> So, I'd suggest that we not include those to minimize the scope of the
> release.
>
> Thanks,
>
> Jun
>
> On Mon, Aug 17, 2015 at 5:16 PM, Grant Henke <gh...@cloudera.com> wrote:
>
> > Thanks Gwen.
> >
> > I updated a few small things on the wiki page.
> >
> > Below is a list of jiras I think could also be marked as included. All of
> > these, though not super critical, seem like fairly small and low risk
> > changes that help avoid potentially confusing issues or errors for users.
> >
> > KAFKA-2012
> > KAFKA-972
> > KAFKA-2337 & KAFKA-2393
> > KAFKA-1867
> > KAFKA-2407
> > KAFKA-2234
> > KAFKA-1866
> > KAFKA-2345 & KAFKA-2355
> >
> > thoughts?
> >
> > Thank you,
> > Grant
> >
> > On Mon, Aug 17, 2015 at 4:56 PM, Gwen Shapira <gw...@confluent.io> wrote:
> >
> > > Thanks for creating a list, Grant!
> > >
> > > I placed it on the wiki with a quick evaluation of the content and
> > whether
> > > it should be in 0.8.2.2:
> > >
> > >
> >
> https://cwiki.apache.org/confluence/display/KAFKA/Proposed+patches+for+0.8.2.2
> > >
> > > I'm attempting to only cherrypick fixes that are both important for
> large
> > > number of users (or very critical to some users) and very safe (mostly
> > > judged by the size of the change, but not only)
> > >
> > > If your favorite bugfix is missing from the list, or is there but
> marked
> > > "No", please let us know (in this thread) what we are missing and why
> it
> > is
> > > both important and safe.
> > > Also, if I accidentally included something you consider unsafe, speak
> up!
> > >
> > > Gwen
> > >
> > > On Mon, Aug 17, 2015 at 8:17 AM, Grant Henke <gh...@cloudera.com>
> > wrote:
> > >
> > > > +dev
> > > >
> > > > Adding dev list back in. Somehow it got dropped.
> > > >
> > > >
> > > > On Mon, Aug 17, 2015 at 10:16 AM, Grant Henke <gh...@cloudera.com>
> > > wrote:
> > > >
> > > > > Below is a list of candidate bug fix jiras marked fixed for 0.8.3.
> I
> > > > don't
> > > > > suspect all of these will (or should) make it into the release but
> > this
> > > > > should be a relatively complete list to work from:
> > > > >
> > > > > - KAFKA-2114 <https://issues.apache.org/jira/browse/KAFKA-2114
> >:
> > > > Unable
> > > > > to change min.insync.replicas default
> > > > > - KAFKA-1702 <https://issues.apache.org/jira/browse/KAFKA-1702
> >:
> > > > > Messages silently Lost by producer
> > > > > - KAFKA-2012 <https://issues.apache.org/jira/browse/KAFKA-2012
> >:
> > > > > Broker should automatically handle corrupt index files
> > > > > - KAFKA-2406 <https://issues.apache.org/jira/browse/KAFKA-2406
> >:
> > > ISR
> > > > > propagation should be throttled to avoid overwhelming
> controller.
> > > > > - KAFKA-2336 <https://issues.apache.org/jira/browse/KAFKA-2336
> >:
> > > > > Changing offsets.topic.num.partitions after the offset topic is
> > > > created
> > > > > breaks consumer group partition assignment
> > > > > - KAFKA-2337 <https://issues.apache.org/jira/browse/KAFKA-2337
> >:
> > > > Verify
> > > > > that metric names will not collide when creating new topics
> > > > > - KAFKA-2393 <https://issues.apache.org/jira/browse/KAFKA-2393
> >:
> > > > > Correctly Handle InvalidTopicException in
> > > KafkaApis.getTopicMetadata()
> > > > > - KAFKA-2189 <https://issues.apache.org/jira/browse/KAFKA-2189
> >:
> > > > Snappy
> > > > > compression of message batches less efficient in 0.8.2.1
> > > > > - KAFKA-2308 <https://issues.apache.org/jira/browse/KAFKA-2308
> >:
> > > New
> > > > > producer + Snappy face un-compression errors after broker
> restart
> > > > > - KAFKA-2042 <https://issues.apache.org/jira/browse/KAFKA-2042
> >:
> > > New
> > > > > producer metadata update always get all topics.
> > > > > - KAFKA-1367 <https://issues.apache.org/jira/browse/KAFKA-1367
> >:
> > > > Broker
> > > > > topic metadata not kept in sync with ZooKeeper
> > > > > - KAFKA-972 <https://issues.apache.org/jira/browse/KAFKA-972>:
> > > > MetadataRequest
> > > > > returns stale list of brokers
> > > > > - KAFKA-1867 <https://issues.apache.org/jira/browse/KAFKA-1867
> >:
> > > > liveBroker
> > > > > list not updated on a cluster with no topics
> > > > > - KAFKA-1650 <https://issues.apache.org/jira/browse/KAFKA-1650
> >:
> > > > Mirror
> > > > > Maker could lose data on unclean shutdown.
> > > > > - KAFKA-2009 <https://issues.apache.org/jira/browse/KAFKA-2009
> >:
> > > Fix
> > > > > UncheckedOffset.removeOffset synchronization and trace logging
> > issue
> > > > in
> > > > > mirror maker
> > > > > - KAFKA-2407 <https://issues.apache.org/jira/browse/KAFKA-2407
> >:
> > > Only
> > > > > create a log directory when it will be used
> > > > > - KAFKA-2327 <https://issues.apache.org/jira/browse/KAFKA-2327
> >:
> > > > > broker doesn't start if config defines advertised.host but not
> > > > > advertised.port
> > > > > - KAFKA-1788: producer record can stay in RecordAccumulator
> > forever
> > > if
> > > > > leader is no available
> > > > > - KAFKA-2234 <https://issues.apache.org/jira/browse/KAFKA-2234
> >:
> > > > > Partition reassignment of a nonexistent topic prevents future
> > > > reassignments
> > > > > - KAFKA-2096 <https://issues.apache.org/jira/browse/KAFKA-2096
> >:
> > > > > Enable keepalive socket option for broker to prevent socket leak
> > > > > - KAFKA-1057 <https://issues.apache.org/jira/browse/KAFKA-1057
> >:
> > > Trim
> > > > > whitespaces from user specified configs
> > > > > - KAFKA-1641 <https://issues.apache.org/jira/browse/KAFKA-1641
> >:
> > > Log
> > > > > cleaner exits if last cleaned offset is lower than earliest
> offset
> > > > > - KAFKA-1648 <https://issues.apache.org/jira/browse/KAFKA-1648
> >:
> > > > Round
> > > > > robin consumer balance throws an NPE when there are no topics
> > > > > - KAFKA-1724 <https://issues.apache.org/jira/browse/KAFKA-1724
> >:
> > > > > Errors after reboot in single node setup
> > > > > - KAFKA-1758 <https://issues.apache.org/jira/browse/KAFKA-1758
> >:
> > > > > corrupt recovery file prevents startup
> > > > > - KAFKA-1866 <https://issues.apache.org/jira/browse/KAFKA-1866
> >:
> > > > > LogStartOffset gauge throws exceptions after log.delete()
> > > > > - KAFKA-1883 <https://issues.apache.org/jira/browse/KAFKA-1883
> >:
> > > > NullPointerException
> > > > > in RequestSendThread
> > > > > - KAFKA-1896 <https://issues.apache.org/jira/browse/KAFKA-1896
> >:
> > > > > Record size funcition of record in mirror maker hit NPE when the
> > > > message
> > > > > value is null.
> > > > > - KAFKA-2101 <https://issues.apache.org/jira/browse/KAFKA-2101
> >:
> > > > > Metric metadata-age is reset on a failed update
> > > > > - KAFKA-2112 <https://issues.apache.org/jira/browse/KAFKA-2112
> >:
> > > make
> > > > > overflowWheel volatile
> > > > > - KAFKA-2117 <https://issues.apache.org/jira/browse/KAFKA-2117
> >:
> > > > > OffsetManager uses incorrect field for metadata
> > > > > - KAFKA-2164 <https://issues.apache.org/jira/browse/KAFKA-2164
> >:
> > > > > ReplicaFetcherThread: suspicious log message on reset offset
> > > > > - KAFKA-1668 <https://issues.apache.org/jira/browse/KAFKA-1668
> >:
> > > > > TopicCommand doesn't warn if --topic argument doesn't match any
> > > topics
> > > > > - KAFKA-2198 <https://issues.apache.org/jira/browse/KAFKA-2198
> >:
> > > > > kafka-topics.sh exits with 0 status on failures
> > > > > - KAFKA-2235 <https://issues.apache.org/jira/browse/KAFKA-2235
> >:
> > > > > LogCleaner offset map overflow
> > > > > - KAFKA-2241 <https://issues.apache.org/jira/browse/KAFKA-2241
> >:
> > > > > AbstractFetcherThread.shutdown() should not block on
> > > > > ReadableByteChannel.read(buffer)
> > > > > - KAFKA-2272 <https://issues.apache.org/jira/browse/KAFKA-2272
> >:
> > > > > listeners endpoint parsing fails if the hostname has capital
> > letter
> > > > > - KAFKA-2345 <https://issues.apache.org/jira/browse/KAFKA-2345
> >:
> > > > > Attempt to delete a topic already marked for deletion throws
> > > > > ZkNodeExistsException
> > > > > - KAFKA-2353 <https://issues.apache.org/jira/browse/KAFKA-2353
> >:
> > > > > SocketServer.Processor should catch exception and close the
> socket
> > > > properly
> > > > > in configureNewConnections.
> > > > > - KAFKA-1836 <https://issues.apache.org/jira/browse/KAFKA-1836
> >:
> > > > > metadata.fetch.timeout.ms set to zero blocks forever
> > > > > - KAFKA-2317 <https://issues.apache.org/jira/browse/KAFKA-2317
> >:
> > > > De-register
> > > > > isrChangeNotificationListener on controller resignation
> > > > >
> > > > > Note: KAFKA-2120 <https://issues.apache.org/jira/browse/KAFKA-2120
> >
> > &
> > > > > KAFKA-2421 <https://issues.apache.org/jira/browse/KAFKA-2421> were
> > > > > mentioned in previous emails, but are not in the list because they
> > are
> > > > not
> > > > > committed yet.
> > > > >
> > > > > Hope that helps the effort.
> > > > >
> > > > > Thanks,
> > > > > Grant
> > > > >
> > > > > On Mon, Aug 17, 2015 at 12:09 AM, Grant Henke <ghenke@cloudera.com
> >
> > > > wrote:
> > > > >
> > > > >> +1 to that suggestion. Though I suspect that requires a committer
> to
> > > do.
> > > > >> Making it part of the standard commit process could work too.
> > > > >> On Aug 16, 2015 11:01 PM, "Gwen Shapira" <gw...@confluent.io>
> wrote:
> > > > >>
> > > > >>> BTW. I think it will be great for Apache Kafka to have a 0.8.2
> > > "release
> > > > >>> manager" who's role is to cherrypick low-risk bug-fixes into the
> > > 0.8.2
> > > > >>> branch and once enough bug fixes happened (or if sufficiently
> > > critical
> > > > >>> fixes happened) to roll out a new maintenance release (with
> every 3
> > > > month
> > > > >>> as a reasonable bugfix release target).
> > > > >>>
> > > > >>> This will add some predictability regarding how fast we release
> > fixes
> > > > for
> > > > >>> bugs.
> > > > >>>
> > > > >>> Gwen
> > > > >>>
> > > > >>> On Sun, Aug 16, 2015 at 8:09 PM, Jeff Holoman <
> > jholoman@cloudera.com
> > > >
> > > > >>> wrote:
> > > > >>>
> > > > >>> > +1 for the release and also including
> > > > >>> >
> > > > >>> > https://issues.apache.org/jira/browse/KAFKA-2114
> > > > >>> >
> > > > >>> > Thanks
> > > > >>> >
> > > > >>> > Jeff
> > > > >>> >
> > > > >>> > On Sun, Aug 16, 2015 at 2:51 PM, Stevo Slavić <
> sslavic@gmail.com
> > >
> > > > >>> wrote:
> > > > >>> >
> > > > >>> > > +1 (non-binding) for 0.8.2.2 release
> > > > >>> > >
> > > > >>> > > Would be nice to include in that release new producer
> > resiliency
> > > > bug
> > > > >>> > fixes
> > > > >>> > > https://issues.apache.org/jira/browse/KAFKA-1788 and
> > > > >>> > > https://issues.apache.org/jira/browse/KAFKA-2120
> > > > >>> > >
> > > > >>> > > On Fri, Aug 14, 2015 at 4:03 PM, Gwen Shapira <
> > gwen@confluent.io
> > > >
> > > > >>> wrote:
> > > > >>> > >
> > > > >>> > > > Will be nice to include Kafka-2308 and fix two critical
> > snappy
> > > > >>> issues
> > > > >>> > in
> > > > >>> > > > the maintenance release.
> > > > >>> > > >
> > > > >>> > > > Gwen
> > > > >>> > > > On Aug 14, 2015 6:16 AM, "Grant Henke" <
> ghenke@cloudera.com>
> > > > >>> wrote:
> > > > >>> > > >
> > > > >>> > > > > Just to clarify. Will KAFKA-2189 be the only patch in the
> > > > >>> release?
> > > > >>> > > > >
> > > > >>> > > > > On Fri, Aug 14, 2015 at 7:35 AM, Manikumar Reddy <
> > > > >>> > kumar@nmsworks.co.in
> > > > >>> > > >
> > > > >>> > > > > wrote:
> > > > >>> > > > >
> > > > >>> > > > > > +1 for 0.8.2.2 release
> > > > >>> > > > > >
> > > > >>> > > > > > On Fri, Aug 14, 2015 at 5:49 PM, Ismael Juma <
> > > > >>> ismael@juma.me.uk>
> > > > >>> > > > wrote:
> > > > >>> > > > > >
> > > > >>> > > > > > > I think this is a good idea as the change is minimal
> on
> > > our
> > > > >>> side
> > > > >>> > > and
> > > > >>> > > > it
> > > > >>> > > > > > has
> > > > >>> > > > > > > been tested in production for some time by the
> > reporter.
> > > > >>> > > > > > >
> > > > >>> > > > > > > Best,
> > > > >>> > > > > > > Ismael
> > > > >>> > > > > > >
> > > > >>> > > > > > > On Fri, Aug 14, 2015 at 1:15 PM, Jun Rao <
> > > jun@confluent.io
> > > > >
> > > > >>> > wrote:
> > > > >>> > > > > > >
> > > > >>> > > > > > > > Hi, Everyone,
> > > > >>> > > > > > > >
> > > > >>> > > > > > > > Since the release of Kafka 0.8.2.1, a number of
> > people
> > > > have
> > > > >>> > > > reported
> > > > >>> > > > > an
> > > > >>> > > > > > > > issue with snappy compression (
> > > > >>> > > > > > > > https://issues.apache.org/jira/browse/KAFKA-2189).
> > > > >>> Basically,
> > > > >>> > if
> > > > >>> > > > > they
> > > > >>> > > > > > > use
> > > > >>> > > > > > > > snappy in 0.8.2.1, they will experience a 2-3X
> space
> > > > >>> increase.
> > > > >>> > > The
> > > > >>> > > > > > issue
> > > > >>> > > > > > > > has since been fixed in trunk (just a snappy jar
> > > > upgrade).
> > > > >>> > Since
> > > > >>> > > > > 0.8.3
> > > > >>> > > > > > is
> > > > >>> > > > > > > > still a few months away, it may make sense to do an
> > > > 0.8.2.2
> > > > >>> > > release
> > > > >>> > > > > > just
> > > > >>> > > > > > > to
> > > > >>> > > > > > > > fix this issue. Any objections?
> > > > >>> > > > > > > >
> > > > >>> > > > > > > > Thanks,
> > > > >>> > > > > > > >
> > > > >>> > > > > > > > Jun
> > > > >>> > > > > > > >
> > > > >>> > > > > > >
> > > > >>> > > > > >
> > > > >>> > > > >
> > > > >>> > > > >
> > > > >>> > > > >
> > > > >>> > > > > --
> > > > >>> > > > > Grant Henke
> > > > >>> > > > > Software Engineer | Cloudera
> > > > >>> > > > > grant@cloudera.com | twitter.com/gchenke |
> > > > >>> > linkedin.com/in/granthenke
> > > > >>> > > > >
> > > > >>> > > >
> > > > >>> > >
> > > > >>> >
> > > > >>> >
> > > > >>> >
> > > > >>> > --
> > > > >>> > Jeff Holoman
> > > > >>> > Systems Engineer
> > > > >>> >
> > > > >>>
> > > > >>
> > > > >
> > > > >
> > > > > --
> > > > > Grant Henke
> > > > > Software Engineer | Cloudera
> > > > > grant@cloudera.com | twitter.com/gchenke |
> > linkedin.com/in/granthenke
> > > > >
> > > >
> > > >
> > > >
> > > > --
> > > > Grant Henke
> > > > Software Engineer | Cloudera
> > > > grant@cloudera.com | twitter.com/gchenke |
> linkedin.com/in/granthenke
> > > >
> > >
> >
> >
> >
> > --
> > Grant Henke
> > Software Engineer | Cloudera
> > grant@cloudera.com | twitter.com/gchenke | linkedin.com/in/granthenke
> >
>
Re: [DISCUSSION] Kafka 0.8.2.2 release?
Posted by Gwen Shapira <gw...@confluent.io>.
Thanks Jun.
I updated the list with your suggestions.
If anyone feels we are missing a critical patch for 0.8.2.2, please speak
up.
Gwen
On Mon, Aug 17, 2015 at 5:40 PM, Jun Rao <ju...@confluent.io> wrote:
> Hi, Grant,
>
> I took a look at that list. None of those is really critical as you said.
> So, I'd suggest that we not include those to minimize the scope of the
> release.
>
> Thanks,
>
> Jun
>
> On Mon, Aug 17, 2015 at 5:16 PM, Grant Henke <gh...@cloudera.com> wrote:
>
> > Thanks Gwen.
> >
> > I updated a few small things on the wiki page.
> >
> > Below is a list of jiras I think could also be marked as included. All of
> > these, though not super critical, seem like fairly small and low risk
> > changes that help avoid potentially confusing issues or errors for users.
> >
> > KAFKA-2012
> > KAFKA-972
> > KAFKA-2337 & KAFKA-2393
> > KAFKA-1867
> > KAFKA-2407
> > KAFKA-2234
> > KAFKA-1866
> > KAFKA-2345 & KAFKA-2355
> >
> > thoughts?
> >
> > Thank you,
> > Grant
> >
> > On Mon, Aug 17, 2015 at 4:56 PM, Gwen Shapira <gw...@confluent.io> wrote:
> >
> > > Thanks for creating a list, Grant!
> > >
> > > I placed it on the wiki with a quick evaluation of the content and
> > whether
> > > it should be in 0.8.2.2:
> > >
> > >
> >
> https://cwiki.apache.org/confluence/display/KAFKA/Proposed+patches+for+0.8.2.2
> > >
> > > I'm attempting to only cherrypick fixes that are both important for
> large
> > > number of users (or very critical to some users) and very safe (mostly
> > > judged by the size of the change, but not only)
> > >
> > > If your favorite bugfix is missing from the list, or is there but
> marked
> > > "No", please let us know (in this thread) what we are missing and why
> it
> > is
> > > both important and safe.
> > > Also, if I accidentally included something you consider unsafe, speak
> up!
> > >
> > > Gwen
> > >
> > > On Mon, Aug 17, 2015 at 8:17 AM, Grant Henke <gh...@cloudera.com>
> > wrote:
> > >
> > > > +dev
> > > >
> > > > Adding dev list back in. Somehow it got dropped.
> > > >
> > > >
> > > > On Mon, Aug 17, 2015 at 10:16 AM, Grant Henke <gh...@cloudera.com>
> > > wrote:
> > > >
> > > > > Below is a list of candidate bug fix jiras marked fixed for 0.8.3.
> I
> > > > don't
> > > > > suspect all of these will (or should) make it into the release but
> > this
> > > > > should be a relatively complete list to work from:
> > > > >
> > > > > - KAFKA-2114 <https://issues.apache.org/jira/browse/KAFKA-2114
> >:
> > > > Unable
> > > > > to change min.insync.replicas default
> > > > > - KAFKA-1702 <https://issues.apache.org/jira/browse/KAFKA-1702
> >:
> > > > > Messages silently Lost by producer
> > > > > - KAFKA-2012 <https://issues.apache.org/jira/browse/KAFKA-2012
> >:
> > > > > Broker should automatically handle corrupt index files
> > > > > - KAFKA-2406 <https://issues.apache.org/jira/browse/KAFKA-2406
> >:
> > > ISR
> > > > > propagation should be throttled to avoid overwhelming
> controller.
> > > > > - KAFKA-2336 <https://issues.apache.org/jira/browse/KAFKA-2336
> >:
> > > > > Changing offsets.topic.num.partitions after the offset topic is
> > > > created
> > > > > breaks consumer group partition assignment
> > > > > - KAFKA-2337 <https://issues.apache.org/jira/browse/KAFKA-2337
> >:
> > > > Verify
> > > > > that metric names will not collide when creating new topics
> > > > > - KAFKA-2393 <https://issues.apache.org/jira/browse/KAFKA-2393
> >:
> > > > > Correctly Handle InvalidTopicException in
> > > KafkaApis.getTopicMetadata()
> > > > > - KAFKA-2189 <https://issues.apache.org/jira/browse/KAFKA-2189
> >:
> > > > Snappy
> > > > > compression of message batches less efficient in 0.8.2.1
> > > > > - KAFKA-2308 <https://issues.apache.org/jira/browse/KAFKA-2308
> >:
> > > New
> > > > > producer + Snappy face un-compression errors after broker
> restart
> > > > > - KAFKA-2042 <https://issues.apache.org/jira/browse/KAFKA-2042
> >:
> > > New
> > > > > producer metadata update always get all topics.
> > > > > - KAFKA-1367 <https://issues.apache.org/jira/browse/KAFKA-1367
> >:
> > > > Broker
> > > > > topic metadata not kept in sync with ZooKeeper
> > > > > - KAFKA-972 <https://issues.apache.org/jira/browse/KAFKA-972>:
> > > > MetadataRequest
> > > > > returns stale list of brokers
> > > > > - KAFKA-1867 <https://issues.apache.org/jira/browse/KAFKA-1867
> >:
> > > > liveBroker
> > > > > list not updated on a cluster with no topics
> > > > > - KAFKA-1650 <https://issues.apache.org/jira/browse/KAFKA-1650
> >:
> > > > Mirror
> > > > > Maker could lose data on unclean shutdown.
> > > > > - KAFKA-2009 <https://issues.apache.org/jira/browse/KAFKA-2009
> >:
> > > Fix
> > > > > UncheckedOffset.removeOffset synchronization and trace logging
> > issue
> > > > in
> > > > > mirror maker
> > > > > - KAFKA-2407 <https://issues.apache.org/jira/browse/KAFKA-2407
> >:
> > > Only
> > > > > create a log directory when it will be used
> > > > > - KAFKA-2327 <https://issues.apache.org/jira/browse/KAFKA-2327
> >:
> > > > > broker doesn't start if config defines advertised.host but not
> > > > > advertised.port
> > > > > - KAFKA-1788: producer record can stay in RecordAccumulator
> > forever
> > > if
> > > > > leader is no available
> > > > > - KAFKA-2234 <https://issues.apache.org/jira/browse/KAFKA-2234
> >:
> > > > > Partition reassignment of a nonexistent topic prevents future
> > > > reassignments
> > > > > - KAFKA-2096 <https://issues.apache.org/jira/browse/KAFKA-2096
> >:
> > > > > Enable keepalive socket option for broker to prevent socket leak
> > > > > - KAFKA-1057 <https://issues.apache.org/jira/browse/KAFKA-1057
> >:
> > > Trim
> > > > > whitespaces from user specified configs
> > > > > - KAFKA-1641 <https://issues.apache.org/jira/browse/KAFKA-1641
> >:
> > > Log
> > > > > cleaner exits if last cleaned offset is lower than earliest
> offset
> > > > > - KAFKA-1648 <https://issues.apache.org/jira/browse/KAFKA-1648
> >:
> > > > Round
> > > > > robin consumer balance throws an NPE when there are no topics
> > > > > - KAFKA-1724 <https://issues.apache.org/jira/browse/KAFKA-1724
> >:
> > > > > Errors after reboot in single node setup
> > > > > - KAFKA-1758 <https://issues.apache.org/jira/browse/KAFKA-1758
> >:
> > > > > corrupt recovery file prevents startup
> > > > > - KAFKA-1866 <https://issues.apache.org/jira/browse/KAFKA-1866
> >:
> > > > > LogStartOffset gauge throws exceptions after log.delete()
> > > > > - KAFKA-1883 <https://issues.apache.org/jira/browse/KAFKA-1883
> >:
> > > > NullPointerException
> > > > > in RequestSendThread
> > > > > - KAFKA-1896 <https://issues.apache.org/jira/browse/KAFKA-1896
> >:
> > > > > Record size funcition of record in mirror maker hit NPE when the
> > > > message
> > > > > value is null.
> > > > > - KAFKA-2101 <https://issues.apache.org/jira/browse/KAFKA-2101
> >:
> > > > > Metric metadata-age is reset on a failed update
> > > > > - KAFKA-2112 <https://issues.apache.org/jira/browse/KAFKA-2112
> >:
> > > make
> > > > > overflowWheel volatile
> > > > > - KAFKA-2117 <https://issues.apache.org/jira/browse/KAFKA-2117
> >:
> > > > > OffsetManager uses incorrect field for metadata
> > > > > - KAFKA-2164 <https://issues.apache.org/jira/browse/KAFKA-2164
> >:
> > > > > ReplicaFetcherThread: suspicious log message on reset offset
> > > > > - KAFKA-1668 <https://issues.apache.org/jira/browse/KAFKA-1668
> >:
> > > > > TopicCommand doesn't warn if --topic argument doesn't match any
> > > topics
> > > > > - KAFKA-2198 <https://issues.apache.org/jira/browse/KAFKA-2198
> >:
> > > > > kafka-topics.sh exits with 0 status on failures
> > > > > - KAFKA-2235 <https://issues.apache.org/jira/browse/KAFKA-2235
> >:
> > > > > LogCleaner offset map overflow
> > > > > - KAFKA-2241 <https://issues.apache.org/jira/browse/KAFKA-2241
> >:
> > > > > AbstractFetcherThread.shutdown() should not block on
> > > > > ReadableByteChannel.read(buffer)
> > > > > - KAFKA-2272 <https://issues.apache.org/jira/browse/KAFKA-2272
> >:
> > > > > listeners endpoint parsing fails if the hostname has capital
> > letter
> > > > > - KAFKA-2345 <https://issues.apache.org/jira/browse/KAFKA-2345
> >:
> > > > > Attempt to delete a topic already marked for deletion throws
> > > > > ZkNodeExistsException
> > > > > - KAFKA-2353 <https://issues.apache.org/jira/browse/KAFKA-2353
> >:
> > > > > SocketServer.Processor should catch exception and close the
> socket
> > > > properly
> > > > > in configureNewConnections.
> > > > > - KAFKA-1836 <https://issues.apache.org/jira/browse/KAFKA-1836
> >:
> > > > > metadata.fetch.timeout.ms set to zero blocks forever
> > > > > - KAFKA-2317 <https://issues.apache.org/jira/browse/KAFKA-2317
> >:
> > > > De-register
> > > > > isrChangeNotificationListener on controller resignation
> > > > >
> > > > > Note: KAFKA-2120 <https://issues.apache.org/jira/browse/KAFKA-2120
> >
> > &
> > > > > KAFKA-2421 <https://issues.apache.org/jira/browse/KAFKA-2421> were
> > > > > mentioned in previous emails, but are not in the list because they
> > are
> > > > not
> > > > > committed yet.
> > > > >
> > > > > Hope that helps the effort.
> > > > >
> > > > > Thanks,
> > > > > Grant
> > > > >
> > > > > On Mon, Aug 17, 2015 at 12:09 AM, Grant Henke <ghenke@cloudera.com
> >
> > > > wrote:
> > > > >
> > > > >> +1 to that suggestion. Though I suspect that requires a committer
> to
> > > do.
> > > > >> Making it part of the standard commit process could work too.
> > > > >> On Aug 16, 2015 11:01 PM, "Gwen Shapira" <gw...@confluent.io>
> wrote:
> > > > >>
> > > > >>> BTW. I think it will be great for Apache Kafka to have a 0.8.2
> > > "release
> > > > >>> manager" who's role is to cherrypick low-risk bug-fixes into the
> > > 0.8.2
> > > > >>> branch and once enough bug fixes happened (or if sufficiently
> > > critical
> > > > >>> fixes happened) to roll out a new maintenance release (with
> every 3
> > > > month
> > > > >>> as a reasonable bugfix release target).
> > > > >>>
> > > > >>> This will add some predictability regarding how fast we release
> > fixes
> > > > for
> > > > >>> bugs.
> > > > >>>
> > > > >>> Gwen
> > > > >>>
> > > > >>> On Sun, Aug 16, 2015 at 8:09 PM, Jeff Holoman <
> > jholoman@cloudera.com
> > > >
> > > > >>> wrote:
> > > > >>>
> > > > >>> > +1 for the release and also including
> > > > >>> >
> > > > >>> > https://issues.apache.org/jira/browse/KAFKA-2114
> > > > >>> >
> > > > >>> > Thanks
> > > > >>> >
> > > > >>> > Jeff
> > > > >>> >
> > > > >>> > On Sun, Aug 16, 2015 at 2:51 PM, Stevo Slavić <
> sslavic@gmail.com
> > >
> > > > >>> wrote:
> > > > >>> >
> > > > >>> > > +1 (non-binding) for 0.8.2.2 release
> > > > >>> > >
> > > > >>> > > Would be nice to include in that release new producer
> > resiliency
> > > > bug
> > > > >>> > fixes
> > > > >>> > > https://issues.apache.org/jira/browse/KAFKA-1788 and
> > > > >>> > > https://issues.apache.org/jira/browse/KAFKA-2120
> > > > >>> > >
> > > > >>> > > On Fri, Aug 14, 2015 at 4:03 PM, Gwen Shapira <
> > gwen@confluent.io
> > > >
> > > > >>> wrote:
> > > > >>> > >
> > > > >>> > > > Will be nice to include Kafka-2308 and fix two critical
> > snappy
> > > > >>> issues
> > > > >>> > in
> > > > >>> > > > the maintenance release.
> > > > >>> > > >
> > > > >>> > > > Gwen
> > > > >>> > > > On Aug 14, 2015 6:16 AM, "Grant Henke" <
> ghenke@cloudera.com>
> > > > >>> wrote:
> > > > >>> > > >
> > > > >>> > > > > Just to clarify. Will KAFKA-2189 be the only patch in the
> > > > >>> release?
> > > > >>> > > > >
> > > > >>> > > > > On Fri, Aug 14, 2015 at 7:35 AM, Manikumar Reddy <
> > > > >>> > kumar@nmsworks.co.in
> > > > >>> > > >
> > > > >>> > > > > wrote:
> > > > >>> > > > >
> > > > >>> > > > > > +1 for 0.8.2.2 release
> > > > >>> > > > > >
> > > > >>> > > > > > On Fri, Aug 14, 2015 at 5:49 PM, Ismael Juma <
> > > > >>> ismael@juma.me.uk>
> > > > >>> > > > wrote:
> > > > >>> > > > > >
> > > > >>> > > > > > > I think this is a good idea as the change is minimal
> on
> > > our
> > > > >>> side
> > > > >>> > > and
> > > > >>> > > > it
> > > > >>> > > > > > has
> > > > >>> > > > > > > been tested in production for some time by the
> > reporter.
> > > > >>> > > > > > >
> > > > >>> > > > > > > Best,
> > > > >>> > > > > > > Ismael
> > > > >>> > > > > > >
> > > > >>> > > > > > > On Fri, Aug 14, 2015 at 1:15 PM, Jun Rao <
> > > jun@confluent.io
> > > > >
> > > > >>> > wrote:
> > > > >>> > > > > > >
> > > > >>> > > > > > > > Hi, Everyone,
> > > > >>> > > > > > > >
> > > > >>> > > > > > > > Since the release of Kafka 0.8.2.1, a number of
> > people
> > > > have
> > > > >>> > > > reported
> > > > >>> > > > > an
> > > > >>> > > > > > > > issue with snappy compression (
> > > > >>> > > > > > > > https://issues.apache.org/jira/browse/KAFKA-2189).
> > > > >>> Basically,
> > > > >>> > if
> > > > >>> > > > > they
> > > > >>> > > > > > > use
> > > > >>> > > > > > > > snappy in 0.8.2.1, they will experience a 2-3X
> space
> > > > >>> increase.
> > > > >>> > > The
> > > > >>> > > > > > issue
> > > > >>> > > > > > > > has since been fixed in trunk (just a snappy jar
> > > > upgrade).
> > > > >>> > Since
> > > > >>> > > > > 0.8.3
> > > > >>> > > > > > is
> > > > >>> > > > > > > > still a few months away, it may make sense to do an
> > > > 0.8.2.2
> > > > >>> > > release
> > > > >>> > > > > > just
> > > > >>> > > > > > > to
> > > > >>> > > > > > > > fix this issue. Any objections?
> > > > >>> > > > > > > >
> > > > >>> > > > > > > > Thanks,
> > > > >>> > > > > > > >
> > > > >>> > > > > > > > Jun
> > > > >>> > > > > > > >
> > > > >>> > > > > > >
> > > > >>> > > > > >
> > > > >>> > > > >
> > > > >>> > > > >
> > > > >>> > > > >
> > > > >>> > > > > --
> > > > >>> > > > > Grant Henke
> > > > >>> > > > > Software Engineer | Cloudera
> > > > >>> > > > > grant@cloudera.com | twitter.com/gchenke |
> > > > >>> > linkedin.com/in/granthenke
> > > > >>> > > > >
> > > > >>> > > >
> > > > >>> > >
> > > > >>> >
> > > > >>> >
> > > > >>> >
> > > > >>> > --
> > > > >>> > Jeff Holoman
> > > > >>> > Systems Engineer
> > > > >>> >
> > > > >>>
> > > > >>
> > > > >
> > > > >
> > > > > --
> > > > > Grant Henke
> > > > > Software Engineer | Cloudera
> > > > > grant@cloudera.com | twitter.com/gchenke |
> > linkedin.com/in/granthenke
> > > > >
> > > >
> > > >
> > > >
> > > > --
> > > > Grant Henke
> > > > Software Engineer | Cloudera
> > > > grant@cloudera.com | twitter.com/gchenke |
> linkedin.com/in/granthenke
> > > >
> > >
> >
> >
> >
> > --
> > Grant Henke
> > Software Engineer | Cloudera
> > grant@cloudera.com | twitter.com/gchenke | linkedin.com/in/granthenke
> >
>
Re: [DISCUSSION] Kafka 0.8.2.2 release?
Posted by Jun Rao <ju...@confluent.io>.
Hi, Grant,
I took a look at that list. None of those is really critical as you said.
So, I'd suggest that we not include those to minimize the scope of the
release.
Thanks,
Jun
On Mon, Aug 17, 2015 at 5:16 PM, Grant Henke <gh...@cloudera.com> wrote:
> Thanks Gwen.
>
> I updated a few small things on the wiki page.
>
> Below is a list of jiras I think could also be marked as included. All of
> these, though not super critical, seem like fairly small and low risk
> changes that help avoid potentially confusing issues or errors for users.
>
> KAFKA-2012
> KAFKA-972
> KAFKA-2337 & KAFKA-2393
> KAFKA-1867
> KAFKA-2407
> KAFKA-2234
> KAFKA-1866
> KAFKA-2345 & KAFKA-2355
>
> thoughts?
>
> Thank you,
> Grant
>
> On Mon, Aug 17, 2015 at 4:56 PM, Gwen Shapira <gw...@confluent.io> wrote:
>
> > Thanks for creating a list, Grant!
> >
> > I placed it on the wiki with a quick evaluation of the content and
> whether
> > it should be in 0.8.2.2:
> >
> >
> https://cwiki.apache.org/confluence/display/KAFKA/Proposed+patches+for+0.8.2.2
> >
> > I'm attempting to only cherrypick fixes that are both important for large
> > number of users (or very critical to some users) and very safe (mostly
> > judged by the size of the change, but not only)
> >
> > If your favorite bugfix is missing from the list, or is there but marked
> > "No", please let us know (in this thread) what we are missing and why it
> is
> > both important and safe.
> > Also, if I accidentally included something you consider unsafe, speak up!
> >
> > Gwen
> >
> > On Mon, Aug 17, 2015 at 8:17 AM, Grant Henke <gh...@cloudera.com>
> wrote:
> >
> > > +dev
> > >
> > > Adding dev list back in. Somehow it got dropped.
> > >
> > >
> > > On Mon, Aug 17, 2015 at 10:16 AM, Grant Henke <gh...@cloudera.com>
> > wrote:
> > >
> > > > Below is a list of candidate bug fix jiras marked fixed for 0.8.3. I
> > > don't
> > > > suspect all of these will (or should) make it into the release but
> this
> > > > should be a relatively complete list to work from:
> > > >
> > > > - KAFKA-2114 <https://issues.apache.org/jira/browse/KAFKA-2114>:
> > > Unable
> > > > to change min.insync.replicas default
> > > > - KAFKA-1702 <https://issues.apache.org/jira/browse/KAFKA-1702>:
> > > > Messages silently Lost by producer
> > > > - KAFKA-2012 <https://issues.apache.org/jira/browse/KAFKA-2012>:
> > > > Broker should automatically handle corrupt index files
> > > > - KAFKA-2406 <https://issues.apache.org/jira/browse/KAFKA-2406>:
> > ISR
> > > > propagation should be throttled to avoid overwhelming controller.
> > > > - KAFKA-2336 <https://issues.apache.org/jira/browse/KAFKA-2336>:
> > > > Changing offsets.topic.num.partitions after the offset topic is
> > > created
> > > > breaks consumer group partition assignment
> > > > - KAFKA-2337 <https://issues.apache.org/jira/browse/KAFKA-2337>:
> > > Verify
> > > > that metric names will not collide when creating new topics
> > > > - KAFKA-2393 <https://issues.apache.org/jira/browse/KAFKA-2393>:
> > > > Correctly Handle InvalidTopicException in
> > KafkaApis.getTopicMetadata()
> > > > - KAFKA-2189 <https://issues.apache.org/jira/browse/KAFKA-2189>:
> > > Snappy
> > > > compression of message batches less efficient in 0.8.2.1
> > > > - KAFKA-2308 <https://issues.apache.org/jira/browse/KAFKA-2308>:
> > New
> > > > producer + Snappy face un-compression errors after broker restart
> > > > - KAFKA-2042 <https://issues.apache.org/jira/browse/KAFKA-2042>:
> > New
> > > > producer metadata update always get all topics.
> > > > - KAFKA-1367 <https://issues.apache.org/jira/browse/KAFKA-1367>:
> > > Broker
> > > > topic metadata not kept in sync with ZooKeeper
> > > > - KAFKA-972 <https://issues.apache.org/jira/browse/KAFKA-972>:
> > > MetadataRequest
> > > > returns stale list of brokers
> > > > - KAFKA-1867 <https://issues.apache.org/jira/browse/KAFKA-1867>:
> > > liveBroker
> > > > list not updated on a cluster with no topics
> > > > - KAFKA-1650 <https://issues.apache.org/jira/browse/KAFKA-1650>:
> > > Mirror
> > > > Maker could lose data on unclean shutdown.
> > > > - KAFKA-2009 <https://issues.apache.org/jira/browse/KAFKA-2009>:
> > Fix
> > > > UncheckedOffset.removeOffset synchronization and trace logging
> issue
> > > in
> > > > mirror maker
> > > > - KAFKA-2407 <https://issues.apache.org/jira/browse/KAFKA-2407>:
> > Only
> > > > create a log directory when it will be used
> > > > - KAFKA-2327 <https://issues.apache.org/jira/browse/KAFKA-2327>:
> > > > broker doesn't start if config defines advertised.host but not
> > > > advertised.port
> > > > - KAFKA-1788: producer record can stay in RecordAccumulator
> forever
> > if
> > > > leader is no available
> > > > - KAFKA-2234 <https://issues.apache.org/jira/browse/KAFKA-2234>:
> > > > Partition reassignment of a nonexistent topic prevents future
> > > reassignments
> > > > - KAFKA-2096 <https://issues.apache.org/jira/browse/KAFKA-2096>:
> > > > Enable keepalive socket option for broker to prevent socket leak
> > > > - KAFKA-1057 <https://issues.apache.org/jira/browse/KAFKA-1057>:
> > Trim
> > > > whitespaces from user specified configs
> > > > - KAFKA-1641 <https://issues.apache.org/jira/browse/KAFKA-1641>:
> > Log
> > > > cleaner exits if last cleaned offset is lower than earliest offset
> > > > - KAFKA-1648 <https://issues.apache.org/jira/browse/KAFKA-1648>:
> > > Round
> > > > robin consumer balance throws an NPE when there are no topics
> > > > - KAFKA-1724 <https://issues.apache.org/jira/browse/KAFKA-1724>:
> > > > Errors after reboot in single node setup
> > > > - KAFKA-1758 <https://issues.apache.org/jira/browse/KAFKA-1758>:
> > > > corrupt recovery file prevents startup
> > > > - KAFKA-1866 <https://issues.apache.org/jira/browse/KAFKA-1866>:
> > > > LogStartOffset gauge throws exceptions after log.delete()
> > > > - KAFKA-1883 <https://issues.apache.org/jira/browse/KAFKA-1883>:
> > > NullPointerException
> > > > in RequestSendThread
> > > > - KAFKA-1896 <https://issues.apache.org/jira/browse/KAFKA-1896>:
> > > > Record size funcition of record in mirror maker hit NPE when the
> > > message
> > > > value is null.
> > > > - KAFKA-2101 <https://issues.apache.org/jira/browse/KAFKA-2101>:
> > > > Metric metadata-age is reset on a failed update
> > > > - KAFKA-2112 <https://issues.apache.org/jira/browse/KAFKA-2112>:
> > make
> > > > overflowWheel volatile
> > > > - KAFKA-2117 <https://issues.apache.org/jira/browse/KAFKA-2117>:
> > > > OffsetManager uses incorrect field for metadata
> > > > - KAFKA-2164 <https://issues.apache.org/jira/browse/KAFKA-2164>:
> > > > ReplicaFetcherThread: suspicious log message on reset offset
> > > > - KAFKA-1668 <https://issues.apache.org/jira/browse/KAFKA-1668>:
> > > > TopicCommand doesn't warn if --topic argument doesn't match any
> > topics
> > > > - KAFKA-2198 <https://issues.apache.org/jira/browse/KAFKA-2198>:
> > > > kafka-topics.sh exits with 0 status on failures
> > > > - KAFKA-2235 <https://issues.apache.org/jira/browse/KAFKA-2235>:
> > > > LogCleaner offset map overflow
> > > > - KAFKA-2241 <https://issues.apache.org/jira/browse/KAFKA-2241>:
> > > > AbstractFetcherThread.shutdown() should not block on
> > > > ReadableByteChannel.read(buffer)
> > > > - KAFKA-2272 <https://issues.apache.org/jira/browse/KAFKA-2272>:
> > > > listeners endpoint parsing fails if the hostname has capital
> letter
> > > > - KAFKA-2345 <https://issues.apache.org/jira/browse/KAFKA-2345>:
> > > > Attempt to delete a topic already marked for deletion throws
> > > > ZkNodeExistsException
> > > > - KAFKA-2353 <https://issues.apache.org/jira/browse/KAFKA-2353>:
> > > > SocketServer.Processor should catch exception and close the socket
> > > properly
> > > > in configureNewConnections.
> > > > - KAFKA-1836 <https://issues.apache.org/jira/browse/KAFKA-1836>:
> > > > metadata.fetch.timeout.ms set to zero blocks forever
> > > > - KAFKA-2317 <https://issues.apache.org/jira/browse/KAFKA-2317>:
> > > De-register
> > > > isrChangeNotificationListener on controller resignation
> > > >
> > > > Note: KAFKA-2120 <https://issues.apache.org/jira/browse/KAFKA-2120>
> &
> > > > KAFKA-2421 <https://issues.apache.org/jira/browse/KAFKA-2421> were
> > > > mentioned in previous emails, but are not in the list because they
> are
> > > not
> > > > committed yet.
> > > >
> > > > Hope that helps the effort.
> > > >
> > > > Thanks,
> > > > Grant
> > > >
> > > > On Mon, Aug 17, 2015 at 12:09 AM, Grant Henke <gh...@cloudera.com>
> > > wrote:
> > > >
> > > >> +1 to that suggestion. Though I suspect that requires a committer to
> > do.
> > > >> Making it part of the standard commit process could work too.
> > > >> On Aug 16, 2015 11:01 PM, "Gwen Shapira" <gw...@confluent.io> wrote:
> > > >>
> > > >>> BTW. I think it will be great for Apache Kafka to have a 0.8.2
> > "release
> > > >>> manager" who's role is to cherrypick low-risk bug-fixes into the
> > 0.8.2
> > > >>> branch and once enough bug fixes happened (or if sufficiently
> > critical
> > > >>> fixes happened) to roll out a new maintenance release (with every 3
> > > month
> > > >>> as a reasonable bugfix release target).
> > > >>>
> > > >>> This will add some predictability regarding how fast we release
> fixes
> > > for
> > > >>> bugs.
> > > >>>
> > > >>> Gwen
> > > >>>
> > > >>> On Sun, Aug 16, 2015 at 8:09 PM, Jeff Holoman <
> jholoman@cloudera.com
> > >
> > > >>> wrote:
> > > >>>
> > > >>> > +1 for the release and also including
> > > >>> >
> > > >>> > https://issues.apache.org/jira/browse/KAFKA-2114
> > > >>> >
> > > >>> > Thanks
> > > >>> >
> > > >>> > Jeff
> > > >>> >
> > > >>> > On Sun, Aug 16, 2015 at 2:51 PM, Stevo Slavić <sslavic@gmail.com
> >
> > > >>> wrote:
> > > >>> >
> > > >>> > > +1 (non-binding) for 0.8.2.2 release
> > > >>> > >
> > > >>> > > Would be nice to include in that release new producer
> resiliency
> > > bug
> > > >>> > fixes
> > > >>> > > https://issues.apache.org/jira/browse/KAFKA-1788 and
> > > >>> > > https://issues.apache.org/jira/browse/KAFKA-2120
> > > >>> > >
> > > >>> > > On Fri, Aug 14, 2015 at 4:03 PM, Gwen Shapira <
> gwen@confluent.io
> > >
> > > >>> wrote:
> > > >>> > >
> > > >>> > > > Will be nice to include Kafka-2308 and fix two critical
> snappy
> > > >>> issues
> > > >>> > in
> > > >>> > > > the maintenance release.
> > > >>> > > >
> > > >>> > > > Gwen
> > > >>> > > > On Aug 14, 2015 6:16 AM, "Grant Henke" <gh...@cloudera.com>
> > > >>> wrote:
> > > >>> > > >
> > > >>> > > > > Just to clarify. Will KAFKA-2189 be the only patch in the
> > > >>> release?
> > > >>> > > > >
> > > >>> > > > > On Fri, Aug 14, 2015 at 7:35 AM, Manikumar Reddy <
> > > >>> > kumar@nmsworks.co.in
> > > >>> > > >
> > > >>> > > > > wrote:
> > > >>> > > > >
> > > >>> > > > > > +1 for 0.8.2.2 release
> > > >>> > > > > >
> > > >>> > > > > > On Fri, Aug 14, 2015 at 5:49 PM, Ismael Juma <
> > > >>> ismael@juma.me.uk>
> > > >>> > > > wrote:
> > > >>> > > > > >
> > > >>> > > > > > > I think this is a good idea as the change is minimal on
> > our
> > > >>> side
> > > >>> > > and
> > > >>> > > > it
> > > >>> > > > > > has
> > > >>> > > > > > > been tested in production for some time by the
> reporter.
> > > >>> > > > > > >
> > > >>> > > > > > > Best,
> > > >>> > > > > > > Ismael
> > > >>> > > > > > >
> > > >>> > > > > > > On Fri, Aug 14, 2015 at 1:15 PM, Jun Rao <
> > jun@confluent.io
> > > >
> > > >>> > wrote:
> > > >>> > > > > > >
> > > >>> > > > > > > > Hi, Everyone,
> > > >>> > > > > > > >
> > > >>> > > > > > > > Since the release of Kafka 0.8.2.1, a number of
> people
> > > have
> > > >>> > > > reported
> > > >>> > > > > an
> > > >>> > > > > > > > issue with snappy compression (
> > > >>> > > > > > > > https://issues.apache.org/jira/browse/KAFKA-2189).
> > > >>> Basically,
> > > >>> > if
> > > >>> > > > > they
> > > >>> > > > > > > use
> > > >>> > > > > > > > snappy in 0.8.2.1, they will experience a 2-3X space
> > > >>> increase.
> > > >>> > > The
> > > >>> > > > > > issue
> > > >>> > > > > > > > has since been fixed in trunk (just a snappy jar
> > > upgrade).
> > > >>> > Since
> > > >>> > > > > 0.8.3
> > > >>> > > > > > is
> > > >>> > > > > > > > still a few months away, it may make sense to do an
> > > 0.8.2.2
> > > >>> > > release
> > > >>> > > > > > just
> > > >>> > > > > > > to
> > > >>> > > > > > > > fix this issue. Any objections?
> > > >>> > > > > > > >
> > > >>> > > > > > > > Thanks,
> > > >>> > > > > > > >
> > > >>> > > > > > > > Jun
> > > >>> > > > > > > >
> > > >>> > > > > > >
> > > >>> > > > > >
> > > >>> > > > >
> > > >>> > > > >
> > > >>> > > > >
> > > >>> > > > > --
> > > >>> > > > > Grant Henke
> > > >>> > > > > Software Engineer | Cloudera
> > > >>> > > > > grant@cloudera.com | twitter.com/gchenke |
> > > >>> > linkedin.com/in/granthenke
> > > >>> > > > >
> > > >>> > > >
> > > >>> > >
> > > >>> >
> > > >>> >
> > > >>> >
> > > >>> > --
> > > >>> > Jeff Holoman
> > > >>> > Systems Engineer
> > > >>> >
> > > >>>
> > > >>
> > > >
> > > >
> > > > --
> > > > Grant Henke
> > > > Software Engineer | Cloudera
> > > > grant@cloudera.com | twitter.com/gchenke |
> linkedin.com/in/granthenke
> > > >
> > >
> > >
> > >
> > > --
> > > Grant Henke
> > > Software Engineer | Cloudera
> > > grant@cloudera.com | twitter.com/gchenke | linkedin.com/in/granthenke
> > >
> >
>
>
>
> --
> Grant Henke
> Software Engineer | Cloudera
> grant@cloudera.com | twitter.com/gchenke | linkedin.com/in/granthenke
>
Re: [DISCUSSION] Kafka 0.8.2.2 release?
Posted by Jun Rao <ju...@confluent.io>.
Hi, Grant,
I took a look at that list. None of those is really critical as you said.
So, I'd suggest that we not include those to minimize the scope of the
release.
Thanks,
Jun
On Mon, Aug 17, 2015 at 5:16 PM, Grant Henke <gh...@cloudera.com> wrote:
> Thanks Gwen.
>
> I updated a few small things on the wiki page.
>
> Below is a list of jiras I think could also be marked as included. All of
> these, though not super critical, seem like fairly small and low risk
> changes that help avoid potentially confusing issues or errors for users.
>
> KAFKA-2012
> KAFKA-972
> KAFKA-2337 & KAFKA-2393
> KAFKA-1867
> KAFKA-2407
> KAFKA-2234
> KAFKA-1866
> KAFKA-2345 & KAFKA-2355
>
> thoughts?
>
> Thank you,
> Grant
>
> On Mon, Aug 17, 2015 at 4:56 PM, Gwen Shapira <gw...@confluent.io> wrote:
>
> > Thanks for creating a list, Grant!
> >
> > I placed it on the wiki with a quick evaluation of the content and
> whether
> > it should be in 0.8.2.2:
> >
> >
> https://cwiki.apache.org/confluence/display/KAFKA/Proposed+patches+for+0.8.2.2
> >
> > I'm attempting to only cherrypick fixes that are both important for large
> > number of users (or very critical to some users) and very safe (mostly
> > judged by the size of the change, but not only)
> >
> > If your favorite bugfix is missing from the list, or is there but marked
> > "No", please let us know (in this thread) what we are missing and why it
> is
> > both important and safe.
> > Also, if I accidentally included something you consider unsafe, speak up!
> >
> > Gwen
> >
> > On Mon, Aug 17, 2015 at 8:17 AM, Grant Henke <gh...@cloudera.com>
> wrote:
> >
> > > +dev
> > >
> > > Adding dev list back in. Somehow it got dropped.
> > >
> > >
> > > On Mon, Aug 17, 2015 at 10:16 AM, Grant Henke <gh...@cloudera.com>
> > wrote:
> > >
> > > > Below is a list of candidate bug fix jiras marked fixed for 0.8.3. I
> > > don't
> > > > suspect all of these will (or should) make it into the release but
> this
> > > > should be a relatively complete list to work from:
> > > >
> > > > - KAFKA-2114 <https://issues.apache.org/jira/browse/KAFKA-2114>:
> > > Unable
> > > > to change min.insync.replicas default
> > > > - KAFKA-1702 <https://issues.apache.org/jira/browse/KAFKA-1702>:
> > > > Messages silently Lost by producer
> > > > - KAFKA-2012 <https://issues.apache.org/jira/browse/KAFKA-2012>:
> > > > Broker should automatically handle corrupt index files
> > > > - KAFKA-2406 <https://issues.apache.org/jira/browse/KAFKA-2406>:
> > ISR
> > > > propagation should be throttled to avoid overwhelming controller.
> > > > - KAFKA-2336 <https://issues.apache.org/jira/browse/KAFKA-2336>:
> > > > Changing offsets.topic.num.partitions after the offset topic is
> > > created
> > > > breaks consumer group partition assignment
> > > > - KAFKA-2337 <https://issues.apache.org/jira/browse/KAFKA-2337>:
> > > Verify
> > > > that metric names will not collide when creating new topics
> > > > - KAFKA-2393 <https://issues.apache.org/jira/browse/KAFKA-2393>:
> > > > Correctly Handle InvalidTopicException in
> > KafkaApis.getTopicMetadata()
> > > > - KAFKA-2189 <https://issues.apache.org/jira/browse/KAFKA-2189>:
> > > Snappy
> > > > compression of message batches less efficient in 0.8.2.1
> > > > - KAFKA-2308 <https://issues.apache.org/jira/browse/KAFKA-2308>:
> > New
> > > > producer + Snappy face un-compression errors after broker restart
> > > > - KAFKA-2042 <https://issues.apache.org/jira/browse/KAFKA-2042>:
> > New
> > > > producer metadata update always get all topics.
> > > > - KAFKA-1367 <https://issues.apache.org/jira/browse/KAFKA-1367>:
> > > Broker
> > > > topic metadata not kept in sync with ZooKeeper
> > > > - KAFKA-972 <https://issues.apache.org/jira/browse/KAFKA-972>:
> > > MetadataRequest
> > > > returns stale list of brokers
> > > > - KAFKA-1867 <https://issues.apache.org/jira/browse/KAFKA-1867>:
> > > liveBroker
> > > > list not updated on a cluster with no topics
> > > > - KAFKA-1650 <https://issues.apache.org/jira/browse/KAFKA-1650>:
> > > Mirror
> > > > Maker could lose data on unclean shutdown.
> > > > - KAFKA-2009 <https://issues.apache.org/jira/browse/KAFKA-2009>:
> > Fix
> > > > UncheckedOffset.removeOffset synchronization and trace logging
> issue
> > > in
> > > > mirror maker
> > > > - KAFKA-2407 <https://issues.apache.org/jira/browse/KAFKA-2407>:
> > Only
> > > > create a log directory when it will be used
> > > > - KAFKA-2327 <https://issues.apache.org/jira/browse/KAFKA-2327>:
> > > > broker doesn't start if config defines advertised.host but not
> > > > advertised.port
> > > > - KAFKA-1788: producer record can stay in RecordAccumulator
> forever
> > if
> > > > leader is no available
> > > > - KAFKA-2234 <https://issues.apache.org/jira/browse/KAFKA-2234>:
> > > > Partition reassignment of a nonexistent topic prevents future
> > > reassignments
> > > > - KAFKA-2096 <https://issues.apache.org/jira/browse/KAFKA-2096>:
> > > > Enable keepalive socket option for broker to prevent socket leak
> > > > - KAFKA-1057 <https://issues.apache.org/jira/browse/KAFKA-1057>:
> > Trim
> > > > whitespaces from user specified configs
> > > > - KAFKA-1641 <https://issues.apache.org/jira/browse/KAFKA-1641>:
> > Log
> > > > cleaner exits if last cleaned offset is lower than earliest offset
> > > > - KAFKA-1648 <https://issues.apache.org/jira/browse/KAFKA-1648>:
> > > Round
> > > > robin consumer balance throws an NPE when there are no topics
> > > > - KAFKA-1724 <https://issues.apache.org/jira/browse/KAFKA-1724>:
> > > > Errors after reboot in single node setup
> > > > - KAFKA-1758 <https://issues.apache.org/jira/browse/KAFKA-1758>:
> > > > corrupt recovery file prevents startup
> > > > - KAFKA-1866 <https://issues.apache.org/jira/browse/KAFKA-1866>:
> > > > LogStartOffset gauge throws exceptions after log.delete()
> > > > - KAFKA-1883 <https://issues.apache.org/jira/browse/KAFKA-1883>:
> > > NullPointerException
> > > > in RequestSendThread
> > > > - KAFKA-1896 <https://issues.apache.org/jira/browse/KAFKA-1896>:
> > > > Record size funcition of record in mirror maker hit NPE when the
> > > message
> > > > value is null.
> > > > - KAFKA-2101 <https://issues.apache.org/jira/browse/KAFKA-2101>:
> > > > Metric metadata-age is reset on a failed update
> > > > - KAFKA-2112 <https://issues.apache.org/jira/browse/KAFKA-2112>:
> > make
> > > > overflowWheel volatile
> > > > - KAFKA-2117 <https://issues.apache.org/jira/browse/KAFKA-2117>:
> > > > OffsetManager uses incorrect field for metadata
> > > > - KAFKA-2164 <https://issues.apache.org/jira/browse/KAFKA-2164>:
> > > > ReplicaFetcherThread: suspicious log message on reset offset
> > > > - KAFKA-1668 <https://issues.apache.org/jira/browse/KAFKA-1668>:
> > > > TopicCommand doesn't warn if --topic argument doesn't match any
> > topics
> > > > - KAFKA-2198 <https://issues.apache.org/jira/browse/KAFKA-2198>:
> > > > kafka-topics.sh exits with 0 status on failures
> > > > - KAFKA-2235 <https://issues.apache.org/jira/browse/KAFKA-2235>:
> > > > LogCleaner offset map overflow
> > > > - KAFKA-2241 <https://issues.apache.org/jira/browse/KAFKA-2241>:
> > > > AbstractFetcherThread.shutdown() should not block on
> > > > ReadableByteChannel.read(buffer)
> > > > - KAFKA-2272 <https://issues.apache.org/jira/browse/KAFKA-2272>:
> > > > listeners endpoint parsing fails if the hostname has capital
> letter
> > > > - KAFKA-2345 <https://issues.apache.org/jira/browse/KAFKA-2345>:
> > > > Attempt to delete a topic already marked for deletion throws
> > > > ZkNodeExistsException
> > > > - KAFKA-2353 <https://issues.apache.org/jira/browse/KAFKA-2353>:
> > > > SocketServer.Processor should catch exception and close the socket
> > > properly
> > > > in configureNewConnections.
> > > > - KAFKA-1836 <https://issues.apache.org/jira/browse/KAFKA-1836>:
> > > > metadata.fetch.timeout.ms set to zero blocks forever
> > > > - KAFKA-2317 <https://issues.apache.org/jira/browse/KAFKA-2317>:
> > > De-register
> > > > isrChangeNotificationListener on controller resignation
> > > >
> > > > Note: KAFKA-2120 <https://issues.apache.org/jira/browse/KAFKA-2120>
> &
> > > > KAFKA-2421 <https://issues.apache.org/jira/browse/KAFKA-2421> were
> > > > mentioned in previous emails, but are not in the list because they
> are
> > > not
> > > > committed yet.
> > > >
> > > > Hope that helps the effort.
> > > >
> > > > Thanks,
> > > > Grant
> > > >
> > > > On Mon, Aug 17, 2015 at 12:09 AM, Grant Henke <gh...@cloudera.com>
> > > wrote:
> > > >
> > > >> +1 to that suggestion. Though I suspect that requires a committer to
> > do.
> > > >> Making it part of the standard commit process could work too.
> > > >> On Aug 16, 2015 11:01 PM, "Gwen Shapira" <gw...@confluent.io> wrote:
> > > >>
> > > >>> BTW. I think it will be great for Apache Kafka to have a 0.8.2
> > "release
> > > >>> manager" who's role is to cherrypick low-risk bug-fixes into the
> > 0.8.2
> > > >>> branch and once enough bug fixes happened (or if sufficiently
> > critical
> > > >>> fixes happened) to roll out a new maintenance release (with every 3
> > > month
> > > >>> as a reasonable bugfix release target).
> > > >>>
> > > >>> This will add some predictability regarding how fast we release
> fixes
> > > for
> > > >>> bugs.
> > > >>>
> > > >>> Gwen
> > > >>>
> > > >>> On Sun, Aug 16, 2015 at 8:09 PM, Jeff Holoman <
> jholoman@cloudera.com
> > >
> > > >>> wrote:
> > > >>>
> > > >>> > +1 for the release and also including
> > > >>> >
> > > >>> > https://issues.apache.org/jira/browse/KAFKA-2114
> > > >>> >
> > > >>> > Thanks
> > > >>> >
> > > >>> > Jeff
> > > >>> >
> > > >>> > On Sun, Aug 16, 2015 at 2:51 PM, Stevo Slavić <sslavic@gmail.com
> >
> > > >>> wrote:
> > > >>> >
> > > >>> > > +1 (non-binding) for 0.8.2.2 release
> > > >>> > >
> > > >>> > > Would be nice to include in that release new producer
> resiliency
> > > bug
> > > >>> > fixes
> > > >>> > > https://issues.apache.org/jira/browse/KAFKA-1788 and
> > > >>> > > https://issues.apache.org/jira/browse/KAFKA-2120
> > > >>> > >
> > > >>> > > On Fri, Aug 14, 2015 at 4:03 PM, Gwen Shapira <
> gwen@confluent.io
> > >
> > > >>> wrote:
> > > >>> > >
> > > >>> > > > Will be nice to include Kafka-2308 and fix two critical
> snappy
> > > >>> issues
> > > >>> > in
> > > >>> > > > the maintenance release.
> > > >>> > > >
> > > >>> > > > Gwen
> > > >>> > > > On Aug 14, 2015 6:16 AM, "Grant Henke" <gh...@cloudera.com>
> > > >>> wrote:
> > > >>> > > >
> > > >>> > > > > Just to clarify. Will KAFKA-2189 be the only patch in the
> > > >>> release?
> > > >>> > > > >
> > > >>> > > > > On Fri, Aug 14, 2015 at 7:35 AM, Manikumar Reddy <
> > > >>> > kumar@nmsworks.co.in
> > > >>> > > >
> > > >>> > > > > wrote:
> > > >>> > > > >
> > > >>> > > > > > +1 for 0.8.2.2 release
> > > >>> > > > > >
> > > >>> > > > > > On Fri, Aug 14, 2015 at 5:49 PM, Ismael Juma <
> > > >>> ismael@juma.me.uk>
> > > >>> > > > wrote:
> > > >>> > > > > >
> > > >>> > > > > > > I think this is a good idea as the change is minimal on
> > our
> > > >>> side
> > > >>> > > and
> > > >>> > > > it
> > > >>> > > > > > has
> > > >>> > > > > > > been tested in production for some time by the
> reporter.
> > > >>> > > > > > >
> > > >>> > > > > > > Best,
> > > >>> > > > > > > Ismael
> > > >>> > > > > > >
> > > >>> > > > > > > On Fri, Aug 14, 2015 at 1:15 PM, Jun Rao <
> > jun@confluent.io
> > > >
> > > >>> > wrote:
> > > >>> > > > > > >
> > > >>> > > > > > > > Hi, Everyone,
> > > >>> > > > > > > >
> > > >>> > > > > > > > Since the release of Kafka 0.8.2.1, a number of
> people
> > > have
> > > >>> > > > reported
> > > >>> > > > > an
> > > >>> > > > > > > > issue with snappy compression (
> > > >>> > > > > > > > https://issues.apache.org/jira/browse/KAFKA-2189).
> > > >>> Basically,
> > > >>> > if
> > > >>> > > > > they
> > > >>> > > > > > > use
> > > >>> > > > > > > > snappy in 0.8.2.1, they will experience a 2-3X space
> > > >>> increase.
> > > >>> > > The
> > > >>> > > > > > issue
> > > >>> > > > > > > > has since been fixed in trunk (just a snappy jar
> > > upgrade).
> > > >>> > Since
> > > >>> > > > > 0.8.3
> > > >>> > > > > > is
> > > >>> > > > > > > > still a few months away, it may make sense to do an
> > > 0.8.2.2
> > > >>> > > release
> > > >>> > > > > > just
> > > >>> > > > > > > to
> > > >>> > > > > > > > fix this issue. Any objections?
> > > >>> > > > > > > >
> > > >>> > > > > > > > Thanks,
> > > >>> > > > > > > >
> > > >>> > > > > > > > Jun
> > > >>> > > > > > > >
> > > >>> > > > > > >
> > > >>> > > > > >
> > > >>> > > > >
> > > >>> > > > >
> > > >>> > > > >
> > > >>> > > > > --
> > > >>> > > > > Grant Henke
> > > >>> > > > > Software Engineer | Cloudera
> > > >>> > > > > grant@cloudera.com | twitter.com/gchenke |
> > > >>> > linkedin.com/in/granthenke
> > > >>> > > > >
> > > >>> > > >
> > > >>> > >
> > > >>> >
> > > >>> >
> > > >>> >
> > > >>> > --
> > > >>> > Jeff Holoman
> > > >>> > Systems Engineer
> > > >>> >
> > > >>>
> > > >>
> > > >
> > > >
> > > > --
> > > > Grant Henke
> > > > Software Engineer | Cloudera
> > > > grant@cloudera.com | twitter.com/gchenke |
> linkedin.com/in/granthenke
> > > >
> > >
> > >
> > >
> > > --
> > > Grant Henke
> > > Software Engineer | Cloudera
> > > grant@cloudera.com | twitter.com/gchenke | linkedin.com/in/granthenke
> > >
> >
>
>
>
> --
> Grant Henke
> Software Engineer | Cloudera
> grant@cloudera.com | twitter.com/gchenke | linkedin.com/in/granthenke
>
Re: [DISCUSSION] Kafka 0.8.2.2 release?
Posted by Grant Henke <gh...@cloudera.com>.
Thanks Gwen.
I updated a few small things on the wiki page.
Below is a list of jiras I think could also be marked as included. All of
these, though not super critical, seem like fairly small and low risk
changes that help avoid potentially confusing issues or errors for users.
KAFKA-2012
KAFKA-972
KAFKA-2337 & KAFKA-2393
KAFKA-1867
KAFKA-2407
KAFKA-2234
KAFKA-1866
KAFKA-2345 & KAFKA-2355
thoughts?
Thank you,
Grant
On Mon, Aug 17, 2015 at 4:56 PM, Gwen Shapira <gw...@confluent.io> wrote:
> Thanks for creating a list, Grant!
>
> I placed it on the wiki with a quick evaluation of the content and whether
> it should be in 0.8.2.2:
>
> https://cwiki.apache.org/confluence/display/KAFKA/Proposed+patches+for+0.8.2.2
>
> I'm attempting to only cherrypick fixes that are both important for large
> number of users (or very critical to some users) and very safe (mostly
> judged by the size of the change, but not only)
>
> If your favorite bugfix is missing from the list, or is there but marked
> "No", please let us know (in this thread) what we are missing and why it is
> both important and safe.
> Also, if I accidentally included something you consider unsafe, speak up!
>
> Gwen
>
> On Mon, Aug 17, 2015 at 8:17 AM, Grant Henke <gh...@cloudera.com> wrote:
>
> > +dev
> >
> > Adding dev list back in. Somehow it got dropped.
> >
> >
> > On Mon, Aug 17, 2015 at 10:16 AM, Grant Henke <gh...@cloudera.com>
> wrote:
> >
> > > Below is a list of candidate bug fix jiras marked fixed for 0.8.3. I
> > don't
> > > suspect all of these will (or should) make it into the release but this
> > > should be a relatively complete list to work from:
> > >
> > > - KAFKA-2114 <https://issues.apache.org/jira/browse/KAFKA-2114>:
> > Unable
> > > to change min.insync.replicas default
> > > - KAFKA-1702 <https://issues.apache.org/jira/browse/KAFKA-1702>:
> > > Messages silently Lost by producer
> > > - KAFKA-2012 <https://issues.apache.org/jira/browse/KAFKA-2012>:
> > > Broker should automatically handle corrupt index files
> > > - KAFKA-2406 <https://issues.apache.org/jira/browse/KAFKA-2406>:
> ISR
> > > propagation should be throttled to avoid overwhelming controller.
> > > - KAFKA-2336 <https://issues.apache.org/jira/browse/KAFKA-2336>:
> > > Changing offsets.topic.num.partitions after the offset topic is
> > created
> > > breaks consumer group partition assignment
> > > - KAFKA-2337 <https://issues.apache.org/jira/browse/KAFKA-2337>:
> > Verify
> > > that metric names will not collide when creating new topics
> > > - KAFKA-2393 <https://issues.apache.org/jira/browse/KAFKA-2393>:
> > > Correctly Handle InvalidTopicException in
> KafkaApis.getTopicMetadata()
> > > - KAFKA-2189 <https://issues.apache.org/jira/browse/KAFKA-2189>:
> > Snappy
> > > compression of message batches less efficient in 0.8.2.1
> > > - KAFKA-2308 <https://issues.apache.org/jira/browse/KAFKA-2308>:
> New
> > > producer + Snappy face un-compression errors after broker restart
> > > - KAFKA-2042 <https://issues.apache.org/jira/browse/KAFKA-2042>:
> New
> > > producer metadata update always get all topics.
> > > - KAFKA-1367 <https://issues.apache.org/jira/browse/KAFKA-1367>:
> > Broker
> > > topic metadata not kept in sync with ZooKeeper
> > > - KAFKA-972 <https://issues.apache.org/jira/browse/KAFKA-972>:
> > MetadataRequest
> > > returns stale list of brokers
> > > - KAFKA-1867 <https://issues.apache.org/jira/browse/KAFKA-1867>:
> > liveBroker
> > > list not updated on a cluster with no topics
> > > - KAFKA-1650 <https://issues.apache.org/jira/browse/KAFKA-1650>:
> > Mirror
> > > Maker could lose data on unclean shutdown.
> > > - KAFKA-2009 <https://issues.apache.org/jira/browse/KAFKA-2009>:
> Fix
> > > UncheckedOffset.removeOffset synchronization and trace logging issue
> > in
> > > mirror maker
> > > - KAFKA-2407 <https://issues.apache.org/jira/browse/KAFKA-2407>:
> Only
> > > create a log directory when it will be used
> > > - KAFKA-2327 <https://issues.apache.org/jira/browse/KAFKA-2327>:
> > > broker doesn't start if config defines advertised.host but not
> > > advertised.port
> > > - KAFKA-1788: producer record can stay in RecordAccumulator forever
> if
> > > leader is no available
> > > - KAFKA-2234 <https://issues.apache.org/jira/browse/KAFKA-2234>:
> > > Partition reassignment of a nonexistent topic prevents future
> > reassignments
> > > - KAFKA-2096 <https://issues.apache.org/jira/browse/KAFKA-2096>:
> > > Enable keepalive socket option for broker to prevent socket leak
> > > - KAFKA-1057 <https://issues.apache.org/jira/browse/KAFKA-1057>:
> Trim
> > > whitespaces from user specified configs
> > > - KAFKA-1641 <https://issues.apache.org/jira/browse/KAFKA-1641>:
> Log
> > > cleaner exits if last cleaned offset is lower than earliest offset
> > > - KAFKA-1648 <https://issues.apache.org/jira/browse/KAFKA-1648>:
> > Round
> > > robin consumer balance throws an NPE when there are no topics
> > > - KAFKA-1724 <https://issues.apache.org/jira/browse/KAFKA-1724>:
> > > Errors after reboot in single node setup
> > > - KAFKA-1758 <https://issues.apache.org/jira/browse/KAFKA-1758>:
> > > corrupt recovery file prevents startup
> > > - KAFKA-1866 <https://issues.apache.org/jira/browse/KAFKA-1866>:
> > > LogStartOffset gauge throws exceptions after log.delete()
> > > - KAFKA-1883 <https://issues.apache.org/jira/browse/KAFKA-1883>:
> > NullPointerException
> > > in RequestSendThread
> > > - KAFKA-1896 <https://issues.apache.org/jira/browse/KAFKA-1896>:
> > > Record size funcition of record in mirror maker hit NPE when the
> > message
> > > value is null.
> > > - KAFKA-2101 <https://issues.apache.org/jira/browse/KAFKA-2101>:
> > > Metric metadata-age is reset on a failed update
> > > - KAFKA-2112 <https://issues.apache.org/jira/browse/KAFKA-2112>:
> make
> > > overflowWheel volatile
> > > - KAFKA-2117 <https://issues.apache.org/jira/browse/KAFKA-2117>:
> > > OffsetManager uses incorrect field for metadata
> > > - KAFKA-2164 <https://issues.apache.org/jira/browse/KAFKA-2164>:
> > > ReplicaFetcherThread: suspicious log message on reset offset
> > > - KAFKA-1668 <https://issues.apache.org/jira/browse/KAFKA-1668>:
> > > TopicCommand doesn't warn if --topic argument doesn't match any
> topics
> > > - KAFKA-2198 <https://issues.apache.org/jira/browse/KAFKA-2198>:
> > > kafka-topics.sh exits with 0 status on failures
> > > - KAFKA-2235 <https://issues.apache.org/jira/browse/KAFKA-2235>:
> > > LogCleaner offset map overflow
> > > - KAFKA-2241 <https://issues.apache.org/jira/browse/KAFKA-2241>:
> > > AbstractFetcherThread.shutdown() should not block on
> > > ReadableByteChannel.read(buffer)
> > > - KAFKA-2272 <https://issues.apache.org/jira/browse/KAFKA-2272>:
> > > listeners endpoint parsing fails if the hostname has capital letter
> > > - KAFKA-2345 <https://issues.apache.org/jira/browse/KAFKA-2345>:
> > > Attempt to delete a topic already marked for deletion throws
> > > ZkNodeExistsException
> > > - KAFKA-2353 <https://issues.apache.org/jira/browse/KAFKA-2353>:
> > > SocketServer.Processor should catch exception and close the socket
> > properly
> > > in configureNewConnections.
> > > - KAFKA-1836 <https://issues.apache.org/jira/browse/KAFKA-1836>:
> > > metadata.fetch.timeout.ms set to zero blocks forever
> > > - KAFKA-2317 <https://issues.apache.org/jira/browse/KAFKA-2317>:
> > De-register
> > > isrChangeNotificationListener on controller resignation
> > >
> > > Note: KAFKA-2120 <https://issues.apache.org/jira/browse/KAFKA-2120> &
> > > KAFKA-2421 <https://issues.apache.org/jira/browse/KAFKA-2421> were
> > > mentioned in previous emails, but are not in the list because they are
> > not
> > > committed yet.
> > >
> > > Hope that helps the effort.
> > >
> > > Thanks,
> > > Grant
> > >
> > > On Mon, Aug 17, 2015 at 12:09 AM, Grant Henke <gh...@cloudera.com>
> > wrote:
> > >
> > >> +1 to that suggestion. Though I suspect that requires a committer to
> do.
> > >> Making it part of the standard commit process could work too.
> > >> On Aug 16, 2015 11:01 PM, "Gwen Shapira" <gw...@confluent.io> wrote:
> > >>
> > >>> BTW. I think it will be great for Apache Kafka to have a 0.8.2
> "release
> > >>> manager" who's role is to cherrypick low-risk bug-fixes into the
> 0.8.2
> > >>> branch and once enough bug fixes happened (or if sufficiently
> critical
> > >>> fixes happened) to roll out a new maintenance release (with every 3
> > month
> > >>> as a reasonable bugfix release target).
> > >>>
> > >>> This will add some predictability regarding how fast we release fixes
> > for
> > >>> bugs.
> > >>>
> > >>> Gwen
> > >>>
> > >>> On Sun, Aug 16, 2015 at 8:09 PM, Jeff Holoman <jholoman@cloudera.com
> >
> > >>> wrote:
> > >>>
> > >>> > +1 for the release and also including
> > >>> >
> > >>> > https://issues.apache.org/jira/browse/KAFKA-2114
> > >>> >
> > >>> > Thanks
> > >>> >
> > >>> > Jeff
> > >>> >
> > >>> > On Sun, Aug 16, 2015 at 2:51 PM, Stevo Slavić <ss...@gmail.com>
> > >>> wrote:
> > >>> >
> > >>> > > +1 (non-binding) for 0.8.2.2 release
> > >>> > >
> > >>> > > Would be nice to include in that release new producer resiliency
> > bug
> > >>> > fixes
> > >>> > > https://issues.apache.org/jira/browse/KAFKA-1788 and
> > >>> > > https://issues.apache.org/jira/browse/KAFKA-2120
> > >>> > >
> > >>> > > On Fri, Aug 14, 2015 at 4:03 PM, Gwen Shapira <gwen@confluent.io
> >
> > >>> wrote:
> > >>> > >
> > >>> > > > Will be nice to include Kafka-2308 and fix two critical snappy
> > >>> issues
> > >>> > in
> > >>> > > > the maintenance release.
> > >>> > > >
> > >>> > > > Gwen
> > >>> > > > On Aug 14, 2015 6:16 AM, "Grant Henke" <gh...@cloudera.com>
> > >>> wrote:
> > >>> > > >
> > >>> > > > > Just to clarify. Will KAFKA-2189 be the only patch in the
> > >>> release?
> > >>> > > > >
> > >>> > > > > On Fri, Aug 14, 2015 at 7:35 AM, Manikumar Reddy <
> > >>> > kumar@nmsworks.co.in
> > >>> > > >
> > >>> > > > > wrote:
> > >>> > > > >
> > >>> > > > > > +1 for 0.8.2.2 release
> > >>> > > > > >
> > >>> > > > > > On Fri, Aug 14, 2015 at 5:49 PM, Ismael Juma <
> > >>> ismael@juma.me.uk>
> > >>> > > > wrote:
> > >>> > > > > >
> > >>> > > > > > > I think this is a good idea as the change is minimal on
> our
> > >>> side
> > >>> > > and
> > >>> > > > it
> > >>> > > > > > has
> > >>> > > > > > > been tested in production for some time by the reporter.
> > >>> > > > > > >
> > >>> > > > > > > Best,
> > >>> > > > > > > Ismael
> > >>> > > > > > >
> > >>> > > > > > > On Fri, Aug 14, 2015 at 1:15 PM, Jun Rao <
> jun@confluent.io
> > >
> > >>> > wrote:
> > >>> > > > > > >
> > >>> > > > > > > > Hi, Everyone,
> > >>> > > > > > > >
> > >>> > > > > > > > Since the release of Kafka 0.8.2.1, a number of people
> > have
> > >>> > > > reported
> > >>> > > > > an
> > >>> > > > > > > > issue with snappy compression (
> > >>> > > > > > > > https://issues.apache.org/jira/browse/KAFKA-2189).
> > >>> Basically,
> > >>> > if
> > >>> > > > > they
> > >>> > > > > > > use
> > >>> > > > > > > > snappy in 0.8.2.1, they will experience a 2-3X space
> > >>> increase.
> > >>> > > The
> > >>> > > > > > issue
> > >>> > > > > > > > has since been fixed in trunk (just a snappy jar
> > upgrade).
> > >>> > Since
> > >>> > > > > 0.8.3
> > >>> > > > > > is
> > >>> > > > > > > > still a few months away, it may make sense to do an
> > 0.8.2.2
> > >>> > > release
> > >>> > > > > > just
> > >>> > > > > > > to
> > >>> > > > > > > > fix this issue. Any objections?
> > >>> > > > > > > >
> > >>> > > > > > > > Thanks,
> > >>> > > > > > > >
> > >>> > > > > > > > Jun
> > >>> > > > > > > >
> > >>> > > > > > >
> > >>> > > > > >
> > >>> > > > >
> > >>> > > > >
> > >>> > > > >
> > >>> > > > > --
> > >>> > > > > Grant Henke
> > >>> > > > > Software Engineer | Cloudera
> > >>> > > > > grant@cloudera.com | twitter.com/gchenke |
> > >>> > linkedin.com/in/granthenke
> > >>> > > > >
> > >>> > > >
> > >>> > >
> > >>> >
> > >>> >
> > >>> >
> > >>> > --
> > >>> > Jeff Holoman
> > >>> > Systems Engineer
> > >>> >
> > >>>
> > >>
> > >
> > >
> > > --
> > > Grant Henke
> > > Software Engineer | Cloudera
> > > grant@cloudera.com | twitter.com/gchenke | linkedin.com/in/granthenke
> > >
> >
> >
> >
> > --
> > Grant Henke
> > Software Engineer | Cloudera
> > grant@cloudera.com | twitter.com/gchenke | linkedin.com/in/granthenke
> >
>
--
Grant Henke
Software Engineer | Cloudera
grant@cloudera.com | twitter.com/gchenke | linkedin.com/in/granthenke
Re: [DISCUSSION] Kafka 0.8.2.2 release?
Posted by Jun Rao <ju...@confluent.io>.
Ok, thanks everyone for the feedback. Based on the feedback, I'd recommend
that we do an 0.8.2.2 release including just the two snappy bug fixes.
Jun
On Tue, Aug 18, 2015 at 12:59 PM, Gwen Shapira <gw...@confluent.io> wrote:
> Any objections if I leave KAFKA-2114 (setting min.insync.replicas default)
> out?
>
> The test code is using changes that were done after 0.8.2.x cut-off, which
> makes it difficult to cherry-pick.
>
> Gwen
>
>
>
> On Tue, Aug 18, 2015 at 12:16 PM, Gwen Shapira <gw...@confluent.io> wrote:
>
> > Jun,
> >
> > KAFKA-2147 doesn't seem to have a commit associated with it, so I can't
> > cherrypick just this fix.
> > I suggest leaving this out since there is a 0.8.2.x workaround in the
> JIRA.
> >
> > Gwen
> >
> > On Mon, Aug 17, 2015 at 5:24 PM, Jun Rao <ju...@confluent.io> wrote:
> >
> >> Gwen,
> >>
> >> Thanks for putting the list together.
> >>
> >> I'd recommend that we exclude the following:
> >> KAFKA-1702: This is for the old producer and is only a problem if there
> >> are
> >> some unexpected exceptions (e.g. UnknownClass).
> >> KAFKA-2336: Most people don't change offsets.topic.num.partitions.
> >> KAFKA-1724: The patch there is never committed since the fix is included
> >> in
> >> another jira (a much larger patch).
> >> KAFKA-2241: This doesn't seem be a common problem. It only happens when
> >> the
> >> fetch request blocks on the broker for an extended period of time, which
> >> should be rare.
> >>
> >> I'd also recommend that we include the following:
> >> KAFKA-2147: This impacts the memory size of the purgatory and a number
> of
> >> people have experienced that. The fix is small and has been tested in
> >> production usage. It hasn't been committed though since the issue is
> >> already fixed in trunk and we weren't planning for an 0.8.2.2 release
> >> then.
> >>
> >> Thanks,
> >>
> >> Jun
> >>
> >> On Mon, Aug 17, 2015 at 2:56 PM, Gwen Shapira <gw...@confluent.io>
> wrote:
> >>
> >> > Thanks for creating a list, Grant!
> >> >
> >> > I placed it on the wiki with a quick evaluation of the content and
> >> whether
> >> > it should be in 0.8.2.2:
> >> >
> >> >
> >>
> https://cwiki.apache.org/confluence/display/KAFKA/Proposed+patches+for+0.8.2.2
> >> >
> >> > I'm attempting to only cherrypick fixes that are both important for
> >> large
> >> > number of users (or very critical to some users) and very safe (mostly
> >> > judged by the size of the change, but not only)
> >> >
> >> > If your favorite bugfix is missing from the list, or is there but
> marked
> >> > "No", please let us know (in this thread) what we are missing and why
> >> it is
> >> > both important and safe.
> >> > Also, if I accidentally included something you consider unsafe, speak
> >> up!
> >> >
> >> > Gwen
> >> >
> >> > On Mon, Aug 17, 2015 at 8:17 AM, Grant Henke <gh...@cloudera.com>
> >> wrote:
> >> >
> >> > > +dev
> >> > >
> >> > > Adding dev list back in. Somehow it got dropped.
> >> > >
> >> > >
> >> > > On Mon, Aug 17, 2015 at 10:16 AM, Grant Henke <gh...@cloudera.com>
> >> > wrote:
> >> > >
> >> > > > Below is a list of candidate bug fix jiras marked fixed for
> 0.8.3. I
> >> > > don't
> >> > > > suspect all of these will (or should) make it into the release but
> >> this
> >> > > > should be a relatively complete list to work from:
> >> > > >
> >> > > > - KAFKA-2114 <https://issues.apache.org/jira/browse/KAFKA-2114
> >:
> >> > > Unable
> >> > > > to change min.insync.replicas default
> >> > > > - KAFKA-1702 <https://issues.apache.org/jira/browse/KAFKA-1702
> >:
> >> > > > Messages silently Lost by producer
> >> > > > - KAFKA-2012 <https://issues.apache.org/jira/browse/KAFKA-2012
> >:
> >> > > > Broker should automatically handle corrupt index files
> >> > > > - KAFKA-2406 <https://issues.apache.org/jira/browse/KAFKA-2406
> >:
> >> > ISR
> >> > > > propagation should be throttled to avoid overwhelming
> controller.
> >> > > > - KAFKA-2336 <https://issues.apache.org/jira/browse/KAFKA-2336
> >:
> >> > > > Changing offsets.topic.num.partitions after the offset topic is
> >> > > created
> >> > > > breaks consumer group partition assignment
> >> > > > - KAFKA-2337 <https://issues.apache.org/jira/browse/KAFKA-2337
> >:
> >> > > Verify
> >> > > > that metric names will not collide when creating new topics
> >> > > > - KAFKA-2393 <https://issues.apache.org/jira/browse/KAFKA-2393
> >:
> >> > > > Correctly Handle InvalidTopicException in
> >> > KafkaApis.getTopicMetadata()
> >> > > > - KAFKA-2189 <https://issues.apache.org/jira/browse/KAFKA-2189
> >:
> >> > > Snappy
> >> > > > compression of message batches less efficient in 0.8.2.1
> >> > > > - KAFKA-2308 <https://issues.apache.org/jira/browse/KAFKA-2308
> >:
> >> > New
> >> > > > producer + Snappy face un-compression errors after broker
> restart
> >> > > > - KAFKA-2042 <https://issues.apache.org/jira/browse/KAFKA-2042
> >:
> >> > New
> >> > > > producer metadata update always get all topics.
> >> > > > - KAFKA-1367 <https://issues.apache.org/jira/browse/KAFKA-1367
> >:
> >> > > Broker
> >> > > > topic metadata not kept in sync with ZooKeeper
> >> > > > - KAFKA-972 <https://issues.apache.org/jira/browse/KAFKA-972>:
> >> > > MetadataRequest
> >> > > > returns stale list of brokers
> >> > > > - KAFKA-1867 <https://issues.apache.org/jira/browse/KAFKA-1867
> >:
> >> > > liveBroker
> >> > > > list not updated on a cluster with no topics
> >> > > > - KAFKA-1650 <https://issues.apache.org/jira/browse/KAFKA-1650
> >:
> >> > > Mirror
> >> > > > Maker could lose data on unclean shutdown.
> >> > > > - KAFKA-2009 <https://issues.apache.org/jira/browse/KAFKA-2009
> >:
> >> > Fix
> >> > > > UncheckedOffset.removeOffset synchronization and trace logging
> >> issue
> >> > > in
> >> > > > mirror maker
> >> > > > - KAFKA-2407 <https://issues.apache.org/jira/browse/KAFKA-2407
> >:
> >> > Only
> >> > > > create a log directory when it will be used
> >> > > > - KAFKA-2327 <https://issues.apache.org/jira/browse/KAFKA-2327
> >:
> >> > > > broker doesn't start if config defines advertised.host but not
> >> > > > advertised.port
> >> > > > - KAFKA-1788: producer record can stay in RecordAccumulator
> >> forever
> >> > if
> >> > > > leader is no available
> >> > > > - KAFKA-2234 <https://issues.apache.org/jira/browse/KAFKA-2234
> >:
> >> > > > Partition reassignment of a nonexistent topic prevents future
> >> > > reassignments
> >> > > > - KAFKA-2096 <https://issues.apache.org/jira/browse/KAFKA-2096
> >:
> >> > > > Enable keepalive socket option for broker to prevent socket
> leak
> >> > > > - KAFKA-1057 <https://issues.apache.org/jira/browse/KAFKA-1057
> >:
> >> > Trim
> >> > > > whitespaces from user specified configs
> >> > > > - KAFKA-1641 <https://issues.apache.org/jira/browse/KAFKA-1641
> >:
> >> > Log
> >> > > > cleaner exits if last cleaned offset is lower than earliest
> >> offset
> >> > > > - KAFKA-1648 <https://issues.apache.org/jira/browse/KAFKA-1648
> >:
> >> > > Round
> >> > > > robin consumer balance throws an NPE when there are no topics
> >> > > > - KAFKA-1724 <https://issues.apache.org/jira/browse/KAFKA-1724
> >:
> >> > > > Errors after reboot in single node setup
> >> > > > - KAFKA-1758 <https://issues.apache.org/jira/browse/KAFKA-1758
> >:
> >> > > > corrupt recovery file prevents startup
> >> > > > - KAFKA-1866 <https://issues.apache.org/jira/browse/KAFKA-1866
> >:
> >> > > > LogStartOffset gauge throws exceptions after log.delete()
> >> > > > - KAFKA-1883 <https://issues.apache.org/jira/browse/KAFKA-1883
> >:
> >> > > NullPointerException
> >> > > > in RequestSendThread
> >> > > > - KAFKA-1896 <https://issues.apache.org/jira/browse/KAFKA-1896
> >:
> >> > > > Record size funcition of record in mirror maker hit NPE when
> the
> >> > > message
> >> > > > value is null.
> >> > > > - KAFKA-2101 <https://issues.apache.org/jira/browse/KAFKA-2101
> >:
> >> > > > Metric metadata-age is reset on a failed update
> >> > > > - KAFKA-2112 <https://issues.apache.org/jira/browse/KAFKA-2112
> >:
> >> > make
> >> > > > overflowWheel volatile
> >> > > > - KAFKA-2117 <https://issues.apache.org/jira/browse/KAFKA-2117
> >:
> >> > > > OffsetManager uses incorrect field for metadata
> >> > > > - KAFKA-2164 <https://issues.apache.org/jira/browse/KAFKA-2164
> >:
> >> > > > ReplicaFetcherThread: suspicious log message on reset offset
> >> > > > - KAFKA-1668 <https://issues.apache.org/jira/browse/KAFKA-1668
> >:
> >> > > > TopicCommand doesn't warn if --topic argument doesn't match any
> >> > topics
> >> > > > - KAFKA-2198 <https://issues.apache.org/jira/browse/KAFKA-2198
> >:
> >> > > > kafka-topics.sh exits with 0 status on failures
> >> > > > - KAFKA-2235 <https://issues.apache.org/jira/browse/KAFKA-2235
> >:
> >> > > > LogCleaner offset map overflow
> >> > > > - KAFKA-2241 <https://issues.apache.org/jira/browse/KAFKA-2241
> >:
> >> > > > AbstractFetcherThread.shutdown() should not block on
> >> > > > ReadableByteChannel.read(buffer)
> >> > > > - KAFKA-2272 <https://issues.apache.org/jira/browse/KAFKA-2272
> >:
> >> > > > listeners endpoint parsing fails if the hostname has capital
> >> letter
> >> > > > - KAFKA-2345 <https://issues.apache.org/jira/browse/KAFKA-2345
> >:
> >> > > > Attempt to delete a topic already marked for deletion throws
> >> > > > ZkNodeExistsException
> >> > > > - KAFKA-2353 <https://issues.apache.org/jira/browse/KAFKA-2353
> >:
> >> > > > SocketServer.Processor should catch exception and close the
> >> socket
> >> > > properly
> >> > > > in configureNewConnections.
> >> > > > - KAFKA-1836 <https://issues.apache.org/jira/browse/KAFKA-1836
> >:
> >> > > > metadata.fetch.timeout.ms set to zero blocks forever
> >> > > > - KAFKA-2317 <https://issues.apache.org/jira/browse/KAFKA-2317
> >:
> >> > > De-register
> >> > > > isrChangeNotificationListener on controller resignation
> >> > > >
> >> > > > Note: KAFKA-2120 <
> https://issues.apache.org/jira/browse/KAFKA-2120>
> >> &
> >> > > > KAFKA-2421 <https://issues.apache.org/jira/browse/KAFKA-2421>
> were
> >> > > > mentioned in previous emails, but are not in the list because they
> >> are
> >> > > not
> >> > > > committed yet.
> >> > > >
> >> > > > Hope that helps the effort.
> >> > > >
> >> > > > Thanks,
> >> > > > Grant
> >> > > >
> >> > > > On Mon, Aug 17, 2015 at 12:09 AM, Grant Henke <
> ghenke@cloudera.com>
> >> > > wrote:
> >> > > >
> >> > > >> +1 to that suggestion. Though I suspect that requires a committer
> >> to
> >> > do.
> >> > > >> Making it part of the standard commit process could work too.
> >> > > >> On Aug 16, 2015 11:01 PM, "Gwen Shapira" <gw...@confluent.io>
> >> wrote:
> >> > > >>
> >> > > >>> BTW. I think it will be great for Apache Kafka to have a 0.8.2
> >> > "release
> >> > > >>> manager" who's role is to cherrypick low-risk bug-fixes into the
> >> > 0.8.2
> >> > > >>> branch and once enough bug fixes happened (or if sufficiently
> >> > critical
> >> > > >>> fixes happened) to roll out a new maintenance release (with
> every
> >> 3
> >> > > month
> >> > > >>> as a reasonable bugfix release target).
> >> > > >>>
> >> > > >>> This will add some predictability regarding how fast we release
> >> fixes
> >> > > for
> >> > > >>> bugs.
> >> > > >>>
> >> > > >>> Gwen
> >> > > >>>
> >> > > >>> On Sun, Aug 16, 2015 at 8:09 PM, Jeff Holoman <
> >> jholoman@cloudera.com
> >> > >
> >> > > >>> wrote:
> >> > > >>>
> >> > > >>> > +1 for the release and also including
> >> > > >>> >
> >> > > >>> > https://issues.apache.org/jira/browse/KAFKA-2114
> >> > > >>> >
> >> > > >>> > Thanks
> >> > > >>> >
> >> > > >>> > Jeff
> >> > > >>> >
> >> > > >>> > On Sun, Aug 16, 2015 at 2:51 PM, Stevo Slavić <
> >> sslavic@gmail.com>
> >> > > >>> wrote:
> >> > > >>> >
> >> > > >>> > > +1 (non-binding) for 0.8.2.2 release
> >> > > >>> > >
> >> > > >>> > > Would be nice to include in that release new producer
> >> resiliency
> >> > > bug
> >> > > >>> > fixes
> >> > > >>> > > https://issues.apache.org/jira/browse/KAFKA-1788 and
> >> > > >>> > > https://issues.apache.org/jira/browse/KAFKA-2120
> >> > > >>> > >
> >> > > >>> > > On Fri, Aug 14, 2015 at 4:03 PM, Gwen Shapira <
> >> gwen@confluent.io
> >> > >
> >> > > >>> wrote:
> >> > > >>> > >
> >> > > >>> > > > Will be nice to include Kafka-2308 and fix two critical
> >> snappy
> >> > > >>> issues
> >> > > >>> > in
> >> > > >>> > > > the maintenance release.
> >> > > >>> > > >
> >> > > >>> > > > Gwen
> >> > > >>> > > > On Aug 14, 2015 6:16 AM, "Grant Henke" <
> ghenke@cloudera.com
> >> >
> >> > > >>> wrote:
> >> > > >>> > > >
> >> > > >>> > > > > Just to clarify. Will KAFKA-2189 be the only patch in
> the
> >> > > >>> release?
> >> > > >>> > > > >
> >> > > >>> > > > > On Fri, Aug 14, 2015 at 7:35 AM, Manikumar Reddy <
> >> > > >>> > kumar@nmsworks.co.in
> >> > > >>> > > >
> >> > > >>> > > > > wrote:
> >> > > >>> > > > >
> >> > > >>> > > > > > +1 for 0.8.2.2 release
> >> > > >>> > > > > >
> >> > > >>> > > > > > On Fri, Aug 14, 2015 at 5:49 PM, Ismael Juma <
> >> > > >>> ismael@juma.me.uk>
> >> > > >>> > > > wrote:
> >> > > >>> > > > > >
> >> > > >>> > > > > > > I think this is a good idea as the change is minimal
> >> on
> >> > our
> >> > > >>> side
> >> > > >>> > > and
> >> > > >>> > > > it
> >> > > >>> > > > > > has
> >> > > >>> > > > > > > been tested in production for some time by the
> >> reporter.
> >> > > >>> > > > > > >
> >> > > >>> > > > > > > Best,
> >> > > >>> > > > > > > Ismael
> >> > > >>> > > > > > >
> >> > > >>> > > > > > > On Fri, Aug 14, 2015 at 1:15 PM, Jun Rao <
> >> > jun@confluent.io
> >> > > >
> >> > > >>> > wrote:
> >> > > >>> > > > > > >
> >> > > >>> > > > > > > > Hi, Everyone,
> >> > > >>> > > > > > > >
> >> > > >>> > > > > > > > Since the release of Kafka 0.8.2.1, a number of
> >> people
> >> > > have
> >> > > >>> > > > reported
> >> > > >>> > > > > an
> >> > > >>> > > > > > > > issue with snappy compression (
> >> > > >>> > > > > > > > https://issues.apache.org/jira/browse/KAFKA-2189
> ).
> >> > > >>> Basically,
> >> > > >>> > if
> >> > > >>> > > > > they
> >> > > >>> > > > > > > use
> >> > > >>> > > > > > > > snappy in 0.8.2.1, they will experience a 2-3X
> space
> >> > > >>> increase.
> >> > > >>> > > The
> >> > > >>> > > > > > issue
> >> > > >>> > > > > > > > has since been fixed in trunk (just a snappy jar
> >> > > upgrade).
> >> > > >>> > Since
> >> > > >>> > > > > 0.8.3
> >> > > >>> > > > > > is
> >> > > >>> > > > > > > > still a few months away, it may make sense to do
> an
> >> > > 0.8.2.2
> >> > > >>> > > release
> >> > > >>> > > > > > just
> >> > > >>> > > > > > > to
> >> > > >>> > > > > > > > fix this issue. Any objections?
> >> > > >>> > > > > > > >
> >> > > >>> > > > > > > > Thanks,
> >> > > >>> > > > > > > >
> >> > > >>> > > > > > > > Jun
> >> > > >>> > > > > > > >
> >> > > >>> > > > > > >
> >> > > >>> > > > > >
> >> > > >>> > > > >
> >> > > >>> > > > >
> >> > > >>> > > > >
> >> > > >>> > > > > --
> >> > > >>> > > > > Grant Henke
> >> > > >>> > > > > Software Engineer | Cloudera
> >> > > >>> > > > > grant@cloudera.com | twitter.com/gchenke |
> >> > > >>> > linkedin.com/in/granthenke
> >> > > >>> > > > >
> >> > > >>> > > >
> >> > > >>> > >
> >> > > >>> >
> >> > > >>> >
> >> > > >>> >
> >> > > >>> > --
> >> > > >>> > Jeff Holoman
> >> > > >>> > Systems Engineer
> >> > > >>> >
> >> > > >>>
> >> > > >>
> >> > > >
> >> > > >
> >> > > > --
> >> > > > Grant Henke
> >> > > > Software Engineer | Cloudera
> >> > > > grant@cloudera.com | twitter.com/gchenke |
> >> linkedin.com/in/granthenke
> >> > > >
> >> > >
> >> > >
> >> > >
> >> > > --
> >> > > Grant Henke
> >> > > Software Engineer | Cloudera
> >> > > grant@cloudera.com | twitter.com/gchenke |
> linkedin.com/in/granthenke
> >> > >
> >> >
> >>
> >
> >
>
Re: [DISCUSSION] Kafka 0.8.2.2 release?
Posted by Jun Rao <ju...@confluent.io>.
Ok, thanks everyone for the feedback. Based on the feedback, I'd recommend
that we do an 0.8.2.2 release including just the two snappy bug fixes.
Jun
On Tue, Aug 18, 2015 at 12:59 PM, Gwen Shapira <gw...@confluent.io> wrote:
> Any objections if I leave KAFKA-2114 (setting min.insync.replicas default)
> out?
>
> The test code is using changes that were done after 0.8.2.x cut-off, which
> makes it difficult to cherry-pick.
>
> Gwen
>
>
>
> On Tue, Aug 18, 2015 at 12:16 PM, Gwen Shapira <gw...@confluent.io> wrote:
>
> > Jun,
> >
> > KAFKA-2147 doesn't seem to have a commit associated with it, so I can't
> > cherrypick just this fix.
> > I suggest leaving this out since there is a 0.8.2.x workaround in the
> JIRA.
> >
> > Gwen
> >
> > On Mon, Aug 17, 2015 at 5:24 PM, Jun Rao <ju...@confluent.io> wrote:
> >
> >> Gwen,
> >>
> >> Thanks for putting the list together.
> >>
> >> I'd recommend that we exclude the following:
> >> KAFKA-1702: This is for the old producer and is only a problem if there
> >> are
> >> some unexpected exceptions (e.g. UnknownClass).
> >> KAFKA-2336: Most people don't change offsets.topic.num.partitions.
> >> KAFKA-1724: The patch there is never committed since the fix is included
> >> in
> >> another jira (a much larger patch).
> >> KAFKA-2241: This doesn't seem be a common problem. It only happens when
> >> the
> >> fetch request blocks on the broker for an extended period of time, which
> >> should be rare.
> >>
> >> I'd also recommend that we include the following:
> >> KAFKA-2147: This impacts the memory size of the purgatory and a number
> of
> >> people have experienced that. The fix is small and has been tested in
> >> production usage. It hasn't been committed though since the issue is
> >> already fixed in trunk and we weren't planning for an 0.8.2.2 release
> >> then.
> >>
> >> Thanks,
> >>
> >> Jun
> >>
> >> On Mon, Aug 17, 2015 at 2:56 PM, Gwen Shapira <gw...@confluent.io>
> wrote:
> >>
> >> > Thanks for creating a list, Grant!
> >> >
> >> > I placed it on the wiki with a quick evaluation of the content and
> >> whether
> >> > it should be in 0.8.2.2:
> >> >
> >> >
> >>
> https://cwiki.apache.org/confluence/display/KAFKA/Proposed+patches+for+0.8.2.2
> >> >
> >> > I'm attempting to only cherrypick fixes that are both important for
> >> large
> >> > number of users (or very critical to some users) and very safe (mostly
> >> > judged by the size of the change, but not only)
> >> >
> >> > If your favorite bugfix is missing from the list, or is there but
> marked
> >> > "No", please let us know (in this thread) what we are missing and why
> >> it is
> >> > both important and safe.
> >> > Also, if I accidentally included something you consider unsafe, speak
> >> up!
> >> >
> >> > Gwen
> >> >
> >> > On Mon, Aug 17, 2015 at 8:17 AM, Grant Henke <gh...@cloudera.com>
> >> wrote:
> >> >
> >> > > +dev
> >> > >
> >> > > Adding dev list back in. Somehow it got dropped.
> >> > >
> >> > >
> >> > > On Mon, Aug 17, 2015 at 10:16 AM, Grant Henke <gh...@cloudera.com>
> >> > wrote:
> >> > >
> >> > > > Below is a list of candidate bug fix jiras marked fixed for
> 0.8.3. I
> >> > > don't
> >> > > > suspect all of these will (or should) make it into the release but
> >> this
> >> > > > should be a relatively complete list to work from:
> >> > > >
> >> > > > - KAFKA-2114 <https://issues.apache.org/jira/browse/KAFKA-2114
> >:
> >> > > Unable
> >> > > > to change min.insync.replicas default
> >> > > > - KAFKA-1702 <https://issues.apache.org/jira/browse/KAFKA-1702
> >:
> >> > > > Messages silently Lost by producer
> >> > > > - KAFKA-2012 <https://issues.apache.org/jira/browse/KAFKA-2012
> >:
> >> > > > Broker should automatically handle corrupt index files
> >> > > > - KAFKA-2406 <https://issues.apache.org/jira/browse/KAFKA-2406
> >:
> >> > ISR
> >> > > > propagation should be throttled to avoid overwhelming
> controller.
> >> > > > - KAFKA-2336 <https://issues.apache.org/jira/browse/KAFKA-2336
> >:
> >> > > > Changing offsets.topic.num.partitions after the offset topic is
> >> > > created
> >> > > > breaks consumer group partition assignment
> >> > > > - KAFKA-2337 <https://issues.apache.org/jira/browse/KAFKA-2337
> >:
> >> > > Verify
> >> > > > that metric names will not collide when creating new topics
> >> > > > - KAFKA-2393 <https://issues.apache.org/jira/browse/KAFKA-2393
> >:
> >> > > > Correctly Handle InvalidTopicException in
> >> > KafkaApis.getTopicMetadata()
> >> > > > - KAFKA-2189 <https://issues.apache.org/jira/browse/KAFKA-2189
> >:
> >> > > Snappy
> >> > > > compression of message batches less efficient in 0.8.2.1
> >> > > > - KAFKA-2308 <https://issues.apache.org/jira/browse/KAFKA-2308
> >:
> >> > New
> >> > > > producer + Snappy face un-compression errors after broker
> restart
> >> > > > - KAFKA-2042 <https://issues.apache.org/jira/browse/KAFKA-2042
> >:
> >> > New
> >> > > > producer metadata update always get all topics.
> >> > > > - KAFKA-1367 <https://issues.apache.org/jira/browse/KAFKA-1367
> >:
> >> > > Broker
> >> > > > topic metadata not kept in sync with ZooKeeper
> >> > > > - KAFKA-972 <https://issues.apache.org/jira/browse/KAFKA-972>:
> >> > > MetadataRequest
> >> > > > returns stale list of brokers
> >> > > > - KAFKA-1867 <https://issues.apache.org/jira/browse/KAFKA-1867
> >:
> >> > > liveBroker
> >> > > > list not updated on a cluster with no topics
> >> > > > - KAFKA-1650 <https://issues.apache.org/jira/browse/KAFKA-1650
> >:
> >> > > Mirror
> >> > > > Maker could lose data on unclean shutdown.
> >> > > > - KAFKA-2009 <https://issues.apache.org/jira/browse/KAFKA-2009
> >:
> >> > Fix
> >> > > > UncheckedOffset.removeOffset synchronization and trace logging
> >> issue
> >> > > in
> >> > > > mirror maker
> >> > > > - KAFKA-2407 <https://issues.apache.org/jira/browse/KAFKA-2407
> >:
> >> > Only
> >> > > > create a log directory when it will be used
> >> > > > - KAFKA-2327 <https://issues.apache.org/jira/browse/KAFKA-2327
> >:
> >> > > > broker doesn't start if config defines advertised.host but not
> >> > > > advertised.port
> >> > > > - KAFKA-1788: producer record can stay in RecordAccumulator
> >> forever
> >> > if
> >> > > > leader is no available
> >> > > > - KAFKA-2234 <https://issues.apache.org/jira/browse/KAFKA-2234
> >:
> >> > > > Partition reassignment of a nonexistent topic prevents future
> >> > > reassignments
> >> > > > - KAFKA-2096 <https://issues.apache.org/jira/browse/KAFKA-2096
> >:
> >> > > > Enable keepalive socket option for broker to prevent socket
> leak
> >> > > > - KAFKA-1057 <https://issues.apache.org/jira/browse/KAFKA-1057
> >:
> >> > Trim
> >> > > > whitespaces from user specified configs
> >> > > > - KAFKA-1641 <https://issues.apache.org/jira/browse/KAFKA-1641
> >:
> >> > Log
> >> > > > cleaner exits if last cleaned offset is lower than earliest
> >> offset
> >> > > > - KAFKA-1648 <https://issues.apache.org/jira/browse/KAFKA-1648
> >:
> >> > > Round
> >> > > > robin consumer balance throws an NPE when there are no topics
> >> > > > - KAFKA-1724 <https://issues.apache.org/jira/browse/KAFKA-1724
> >:
> >> > > > Errors after reboot in single node setup
> >> > > > - KAFKA-1758 <https://issues.apache.org/jira/browse/KAFKA-1758
> >:
> >> > > > corrupt recovery file prevents startup
> >> > > > - KAFKA-1866 <https://issues.apache.org/jira/browse/KAFKA-1866
> >:
> >> > > > LogStartOffset gauge throws exceptions after log.delete()
> >> > > > - KAFKA-1883 <https://issues.apache.org/jira/browse/KAFKA-1883
> >:
> >> > > NullPointerException
> >> > > > in RequestSendThread
> >> > > > - KAFKA-1896 <https://issues.apache.org/jira/browse/KAFKA-1896
> >:
> >> > > > Record size funcition of record in mirror maker hit NPE when
> the
> >> > > message
> >> > > > value is null.
> >> > > > - KAFKA-2101 <https://issues.apache.org/jira/browse/KAFKA-2101
> >:
> >> > > > Metric metadata-age is reset on a failed update
> >> > > > - KAFKA-2112 <https://issues.apache.org/jira/browse/KAFKA-2112
> >:
> >> > make
> >> > > > overflowWheel volatile
> >> > > > - KAFKA-2117 <https://issues.apache.org/jira/browse/KAFKA-2117
> >:
> >> > > > OffsetManager uses incorrect field for metadata
> >> > > > - KAFKA-2164 <https://issues.apache.org/jira/browse/KAFKA-2164
> >:
> >> > > > ReplicaFetcherThread: suspicious log message on reset offset
> >> > > > - KAFKA-1668 <https://issues.apache.org/jira/browse/KAFKA-1668
> >:
> >> > > > TopicCommand doesn't warn if --topic argument doesn't match any
> >> > topics
> >> > > > - KAFKA-2198 <https://issues.apache.org/jira/browse/KAFKA-2198
> >:
> >> > > > kafka-topics.sh exits with 0 status on failures
> >> > > > - KAFKA-2235 <https://issues.apache.org/jira/browse/KAFKA-2235
> >:
> >> > > > LogCleaner offset map overflow
> >> > > > - KAFKA-2241 <https://issues.apache.org/jira/browse/KAFKA-2241
> >:
> >> > > > AbstractFetcherThread.shutdown() should not block on
> >> > > > ReadableByteChannel.read(buffer)
> >> > > > - KAFKA-2272 <https://issues.apache.org/jira/browse/KAFKA-2272
> >:
> >> > > > listeners endpoint parsing fails if the hostname has capital
> >> letter
> >> > > > - KAFKA-2345 <https://issues.apache.org/jira/browse/KAFKA-2345
> >:
> >> > > > Attempt to delete a topic already marked for deletion throws
> >> > > > ZkNodeExistsException
> >> > > > - KAFKA-2353 <https://issues.apache.org/jira/browse/KAFKA-2353
> >:
> >> > > > SocketServer.Processor should catch exception and close the
> >> socket
> >> > > properly
> >> > > > in configureNewConnections.
> >> > > > - KAFKA-1836 <https://issues.apache.org/jira/browse/KAFKA-1836
> >:
> >> > > > metadata.fetch.timeout.ms set to zero blocks forever
> >> > > > - KAFKA-2317 <https://issues.apache.org/jira/browse/KAFKA-2317
> >:
> >> > > De-register
> >> > > > isrChangeNotificationListener on controller resignation
> >> > > >
> >> > > > Note: KAFKA-2120 <
> https://issues.apache.org/jira/browse/KAFKA-2120>
> >> &
> >> > > > KAFKA-2421 <https://issues.apache.org/jira/browse/KAFKA-2421>
> were
> >> > > > mentioned in previous emails, but are not in the list because they
> >> are
> >> > > not
> >> > > > committed yet.
> >> > > >
> >> > > > Hope that helps the effort.
> >> > > >
> >> > > > Thanks,
> >> > > > Grant
> >> > > >
> >> > > > On Mon, Aug 17, 2015 at 12:09 AM, Grant Henke <
> ghenke@cloudera.com>
> >> > > wrote:
> >> > > >
> >> > > >> +1 to that suggestion. Though I suspect that requires a committer
> >> to
> >> > do.
> >> > > >> Making it part of the standard commit process could work too.
> >> > > >> On Aug 16, 2015 11:01 PM, "Gwen Shapira" <gw...@confluent.io>
> >> wrote:
> >> > > >>
> >> > > >>> BTW. I think it will be great for Apache Kafka to have a 0.8.2
> >> > "release
> >> > > >>> manager" who's role is to cherrypick low-risk bug-fixes into the
> >> > 0.8.2
> >> > > >>> branch and once enough bug fixes happened (or if sufficiently
> >> > critical
> >> > > >>> fixes happened) to roll out a new maintenance release (with
> every
> >> 3
> >> > > month
> >> > > >>> as a reasonable bugfix release target).
> >> > > >>>
> >> > > >>> This will add some predictability regarding how fast we release
> >> fixes
> >> > > for
> >> > > >>> bugs.
> >> > > >>>
> >> > > >>> Gwen
> >> > > >>>
> >> > > >>> On Sun, Aug 16, 2015 at 8:09 PM, Jeff Holoman <
> >> jholoman@cloudera.com
> >> > >
> >> > > >>> wrote:
> >> > > >>>
> >> > > >>> > +1 for the release and also including
> >> > > >>> >
> >> > > >>> > https://issues.apache.org/jira/browse/KAFKA-2114
> >> > > >>> >
> >> > > >>> > Thanks
> >> > > >>> >
> >> > > >>> > Jeff
> >> > > >>> >
> >> > > >>> > On Sun, Aug 16, 2015 at 2:51 PM, Stevo Slavić <
> >> sslavic@gmail.com>
> >> > > >>> wrote:
> >> > > >>> >
> >> > > >>> > > +1 (non-binding) for 0.8.2.2 release
> >> > > >>> > >
> >> > > >>> > > Would be nice to include in that release new producer
> >> resiliency
> >> > > bug
> >> > > >>> > fixes
> >> > > >>> > > https://issues.apache.org/jira/browse/KAFKA-1788 and
> >> > > >>> > > https://issues.apache.org/jira/browse/KAFKA-2120
> >> > > >>> > >
> >> > > >>> > > On Fri, Aug 14, 2015 at 4:03 PM, Gwen Shapira <
> >> gwen@confluent.io
> >> > >
> >> > > >>> wrote:
> >> > > >>> > >
> >> > > >>> > > > Will be nice to include Kafka-2308 and fix two critical
> >> snappy
> >> > > >>> issues
> >> > > >>> > in
> >> > > >>> > > > the maintenance release.
> >> > > >>> > > >
> >> > > >>> > > > Gwen
> >> > > >>> > > > On Aug 14, 2015 6:16 AM, "Grant Henke" <
> ghenke@cloudera.com
> >> >
> >> > > >>> wrote:
> >> > > >>> > > >
> >> > > >>> > > > > Just to clarify. Will KAFKA-2189 be the only patch in
> the
> >> > > >>> release?
> >> > > >>> > > > >
> >> > > >>> > > > > On Fri, Aug 14, 2015 at 7:35 AM, Manikumar Reddy <
> >> > > >>> > kumar@nmsworks.co.in
> >> > > >>> > > >
> >> > > >>> > > > > wrote:
> >> > > >>> > > > >
> >> > > >>> > > > > > +1 for 0.8.2.2 release
> >> > > >>> > > > > >
> >> > > >>> > > > > > On Fri, Aug 14, 2015 at 5:49 PM, Ismael Juma <
> >> > > >>> ismael@juma.me.uk>
> >> > > >>> > > > wrote:
> >> > > >>> > > > > >
> >> > > >>> > > > > > > I think this is a good idea as the change is minimal
> >> on
> >> > our
> >> > > >>> side
> >> > > >>> > > and
> >> > > >>> > > > it
> >> > > >>> > > > > > has
> >> > > >>> > > > > > > been tested in production for some time by the
> >> reporter.
> >> > > >>> > > > > > >
> >> > > >>> > > > > > > Best,
> >> > > >>> > > > > > > Ismael
> >> > > >>> > > > > > >
> >> > > >>> > > > > > > On Fri, Aug 14, 2015 at 1:15 PM, Jun Rao <
> >> > jun@confluent.io
> >> > > >
> >> > > >>> > wrote:
> >> > > >>> > > > > > >
> >> > > >>> > > > > > > > Hi, Everyone,
> >> > > >>> > > > > > > >
> >> > > >>> > > > > > > > Since the release of Kafka 0.8.2.1, a number of
> >> people
> >> > > have
> >> > > >>> > > > reported
> >> > > >>> > > > > an
> >> > > >>> > > > > > > > issue with snappy compression (
> >> > > >>> > > > > > > > https://issues.apache.org/jira/browse/KAFKA-2189
> ).
> >> > > >>> Basically,
> >> > > >>> > if
> >> > > >>> > > > > they
> >> > > >>> > > > > > > use
> >> > > >>> > > > > > > > snappy in 0.8.2.1, they will experience a 2-3X
> space
> >> > > >>> increase.
> >> > > >>> > > The
> >> > > >>> > > > > > issue
> >> > > >>> > > > > > > > has since been fixed in trunk (just a snappy jar
> >> > > upgrade).
> >> > > >>> > Since
> >> > > >>> > > > > 0.8.3
> >> > > >>> > > > > > is
> >> > > >>> > > > > > > > still a few months away, it may make sense to do
> an
> >> > > 0.8.2.2
> >> > > >>> > > release
> >> > > >>> > > > > > just
> >> > > >>> > > > > > > to
> >> > > >>> > > > > > > > fix this issue. Any objections?
> >> > > >>> > > > > > > >
> >> > > >>> > > > > > > > Thanks,
> >> > > >>> > > > > > > >
> >> > > >>> > > > > > > > Jun
> >> > > >>> > > > > > > >
> >> > > >>> > > > > > >
> >> > > >>> > > > > >
> >> > > >>> > > > >
> >> > > >>> > > > >
> >> > > >>> > > > >
> >> > > >>> > > > > --
> >> > > >>> > > > > Grant Henke
> >> > > >>> > > > > Software Engineer | Cloudera
> >> > > >>> > > > > grant@cloudera.com | twitter.com/gchenke |
> >> > > >>> > linkedin.com/in/granthenke
> >> > > >>> > > > >
> >> > > >>> > > >
> >> > > >>> > >
> >> > > >>> >
> >> > > >>> >
> >> > > >>> >
> >> > > >>> > --
> >> > > >>> > Jeff Holoman
> >> > > >>> > Systems Engineer
> >> > > >>> >
> >> > > >>>
> >> > > >>
> >> > > >
> >> > > >
> >> > > > --
> >> > > > Grant Henke
> >> > > > Software Engineer | Cloudera
> >> > > > grant@cloudera.com | twitter.com/gchenke |
> >> linkedin.com/in/granthenke
> >> > > >
> >> > >
> >> > >
> >> > >
> >> > > --
> >> > > Grant Henke
> >> > > Software Engineer | Cloudera
> >> > > grant@cloudera.com | twitter.com/gchenke |
> linkedin.com/in/granthenke
> >> > >
> >> >
> >>
> >
> >
>
Re: [DISCUSSION] Kafka 0.8.2.2 release?
Posted by Jun Rao <ju...@confluent.io>.
Yes, I think it's fine to leave out both KAFKA-2147 and KAFKA-2114.
Thanks,
Jun
On Tue, Aug 18, 2015 at 12:59 PM, Gwen Shapira <gw...@confluent.io> wrote:
> Any objections if I leave KAFKA-2114 (setting min.insync.replicas default)
> out?
>
> The test code is using changes that were done after 0.8.2.x cut-off, which
> makes it difficult to cherry-pick.
>
> Gwen
>
>
>
> On Tue, Aug 18, 2015 at 12:16 PM, Gwen Shapira <gw...@confluent.io> wrote:
>
> > Jun,
> >
> > KAFKA-2147 doesn't seem to have a commit associated with it, so I can't
> > cherrypick just this fix.
> > I suggest leaving this out since there is a 0.8.2.x workaround in the
> JIRA.
> >
> > Gwen
> >
> > On Mon, Aug 17, 2015 at 5:24 PM, Jun Rao <ju...@confluent.io> wrote:
> >
> >> Gwen,
> >>
> >> Thanks for putting the list together.
> >>
> >> I'd recommend that we exclude the following:
> >> KAFKA-1702: This is for the old producer and is only a problem if there
> >> are
> >> some unexpected exceptions (e.g. UnknownClass).
> >> KAFKA-2336: Most people don't change offsets.topic.num.partitions.
> >> KAFKA-1724: The patch there is never committed since the fix is included
> >> in
> >> another jira (a much larger patch).
> >> KAFKA-2241: This doesn't seem be a common problem. It only happens when
> >> the
> >> fetch request blocks on the broker for an extended period of time, which
> >> should be rare.
> >>
> >> I'd also recommend that we include the following:
> >> KAFKA-2147: This impacts the memory size of the purgatory and a number
> of
> >> people have experienced that. The fix is small and has been tested in
> >> production usage. It hasn't been committed though since the issue is
> >> already fixed in trunk and we weren't planning for an 0.8.2.2 release
> >> then.
> >>
> >> Thanks,
> >>
> >> Jun
> >>
> >> On Mon, Aug 17, 2015 at 2:56 PM, Gwen Shapira <gw...@confluent.io>
> wrote:
> >>
> >> > Thanks for creating a list, Grant!
> >> >
> >> > I placed it on the wiki with a quick evaluation of the content and
> >> whether
> >> > it should be in 0.8.2.2:
> >> >
> >> >
> >>
> https://cwiki.apache.org/confluence/display/KAFKA/Proposed+patches+for+0.8.2.2
> >> >
> >> > I'm attempting to only cherrypick fixes that are both important for
> >> large
> >> > number of users (or very critical to some users) and very safe (mostly
> >> > judged by the size of the change, but not only)
> >> >
> >> > If your favorite bugfix is missing from the list, or is there but
> marked
> >> > "No", please let us know (in this thread) what we are missing and why
> >> it is
> >> > both important and safe.
> >> > Also, if I accidentally included something you consider unsafe, speak
> >> up!
> >> >
> >> > Gwen
> >> >
> >> > On Mon, Aug 17, 2015 at 8:17 AM, Grant Henke <gh...@cloudera.com>
> >> wrote:
> >> >
> >> > > +dev
> >> > >
> >> > > Adding dev list back in. Somehow it got dropped.
> >> > >
> >> > >
> >> > > On Mon, Aug 17, 2015 at 10:16 AM, Grant Henke <gh...@cloudera.com>
> >> > wrote:
> >> > >
> >> > > > Below is a list of candidate bug fix jiras marked fixed for
> 0.8.3. I
> >> > > don't
> >> > > > suspect all of these will (or should) make it into the release but
> >> this
> >> > > > should be a relatively complete list to work from:
> >> > > >
> >> > > > - KAFKA-2114 <https://issues.apache.org/jira/browse/KAFKA-2114
> >:
> >> > > Unable
> >> > > > to change min.insync.replicas default
> >> > > > - KAFKA-1702 <https://issues.apache.org/jira/browse/KAFKA-1702
> >:
> >> > > > Messages silently Lost by producer
> >> > > > - KAFKA-2012 <https://issues.apache.org/jira/browse/KAFKA-2012
> >:
> >> > > > Broker should automatically handle corrupt index files
> >> > > > - KAFKA-2406 <https://issues.apache.org/jira/browse/KAFKA-2406
> >:
> >> > ISR
> >> > > > propagation should be throttled to avoid overwhelming
> controller.
> >> > > > - KAFKA-2336 <https://issues.apache.org/jira/browse/KAFKA-2336
> >:
> >> > > > Changing offsets.topic.num.partitions after the offset topic is
> >> > > created
> >> > > > breaks consumer group partition assignment
> >> > > > - KAFKA-2337 <https://issues.apache.org/jira/browse/KAFKA-2337
> >:
> >> > > Verify
> >> > > > that metric names will not collide when creating new topics
> >> > > > - KAFKA-2393 <https://issues.apache.org/jira/browse/KAFKA-2393
> >:
> >> > > > Correctly Handle InvalidTopicException in
> >> > KafkaApis.getTopicMetadata()
> >> > > > - KAFKA-2189 <https://issues.apache.org/jira/browse/KAFKA-2189
> >:
> >> > > Snappy
> >> > > > compression of message batches less efficient in 0.8.2.1
> >> > > > - KAFKA-2308 <https://issues.apache.org/jira/browse/KAFKA-2308
> >:
> >> > New
> >> > > > producer + Snappy face un-compression errors after broker
> restart
> >> > > > - KAFKA-2042 <https://issues.apache.org/jira/browse/KAFKA-2042
> >:
> >> > New
> >> > > > producer metadata update always get all topics.
> >> > > > - KAFKA-1367 <https://issues.apache.org/jira/browse/KAFKA-1367
> >:
> >> > > Broker
> >> > > > topic metadata not kept in sync with ZooKeeper
> >> > > > - KAFKA-972 <https://issues.apache.org/jira/browse/KAFKA-972>:
> >> > > MetadataRequest
> >> > > > returns stale list of brokers
> >> > > > - KAFKA-1867 <https://issues.apache.org/jira/browse/KAFKA-1867
> >:
> >> > > liveBroker
> >> > > > list not updated on a cluster with no topics
> >> > > > - KAFKA-1650 <https://issues.apache.org/jira/browse/KAFKA-1650
> >:
> >> > > Mirror
> >> > > > Maker could lose data on unclean shutdown.
> >> > > > - KAFKA-2009 <https://issues.apache.org/jira/browse/KAFKA-2009
> >:
> >> > Fix
> >> > > > UncheckedOffset.removeOffset synchronization and trace logging
> >> issue
> >> > > in
> >> > > > mirror maker
> >> > > > - KAFKA-2407 <https://issues.apache.org/jira/browse/KAFKA-2407
> >:
> >> > Only
> >> > > > create a log directory when it will be used
> >> > > > - KAFKA-2327 <https://issues.apache.org/jira/browse/KAFKA-2327
> >:
> >> > > > broker doesn't start if config defines advertised.host but not
> >> > > > advertised.port
> >> > > > - KAFKA-1788: producer record can stay in RecordAccumulator
> >> forever
> >> > if
> >> > > > leader is no available
> >> > > > - KAFKA-2234 <https://issues.apache.org/jira/browse/KAFKA-2234
> >:
> >> > > > Partition reassignment of a nonexistent topic prevents future
> >> > > reassignments
> >> > > > - KAFKA-2096 <https://issues.apache.org/jira/browse/KAFKA-2096
> >:
> >> > > > Enable keepalive socket option for broker to prevent socket
> leak
> >> > > > - KAFKA-1057 <https://issues.apache.org/jira/browse/KAFKA-1057
> >:
> >> > Trim
> >> > > > whitespaces from user specified configs
> >> > > > - KAFKA-1641 <https://issues.apache.org/jira/browse/KAFKA-1641
> >:
> >> > Log
> >> > > > cleaner exits if last cleaned offset is lower than earliest
> >> offset
> >> > > > - KAFKA-1648 <https://issues.apache.org/jira/browse/KAFKA-1648
> >:
> >> > > Round
> >> > > > robin consumer balance throws an NPE when there are no topics
> >> > > > - KAFKA-1724 <https://issues.apache.org/jira/browse/KAFKA-1724
> >:
> >> > > > Errors after reboot in single node setup
> >> > > > - KAFKA-1758 <https://issues.apache.org/jira/browse/KAFKA-1758
> >:
> >> > > > corrupt recovery file prevents startup
> >> > > > - KAFKA-1866 <https://issues.apache.org/jira/browse/KAFKA-1866
> >:
> >> > > > LogStartOffset gauge throws exceptions after log.delete()
> >> > > > - KAFKA-1883 <https://issues.apache.org/jira/browse/KAFKA-1883
> >:
> >> > > NullPointerException
> >> > > > in RequestSendThread
> >> > > > - KAFKA-1896 <https://issues.apache.org/jira/browse/KAFKA-1896
> >:
> >> > > > Record size funcition of record in mirror maker hit NPE when
> the
> >> > > message
> >> > > > value is null.
> >> > > > - KAFKA-2101 <https://issues.apache.org/jira/browse/KAFKA-2101
> >:
> >> > > > Metric metadata-age is reset on a failed update
> >> > > > - KAFKA-2112 <https://issues.apache.org/jira/browse/KAFKA-2112
> >:
> >> > make
> >> > > > overflowWheel volatile
> >> > > > - KAFKA-2117 <https://issues.apache.org/jira/browse/KAFKA-2117
> >:
> >> > > > OffsetManager uses incorrect field for metadata
> >> > > > - KAFKA-2164 <https://issues.apache.org/jira/browse/KAFKA-2164
> >:
> >> > > > ReplicaFetcherThread: suspicious log message on reset offset
> >> > > > - KAFKA-1668 <https://issues.apache.org/jira/browse/KAFKA-1668
> >:
> >> > > > TopicCommand doesn't warn if --topic argument doesn't match any
> >> > topics
> >> > > > - KAFKA-2198 <https://issues.apache.org/jira/browse/KAFKA-2198
> >:
> >> > > > kafka-topics.sh exits with 0 status on failures
> >> > > > - KAFKA-2235 <https://issues.apache.org/jira/browse/KAFKA-2235
> >:
> >> > > > LogCleaner offset map overflow
> >> > > > - KAFKA-2241 <https://issues.apache.org/jira/browse/KAFKA-2241
> >:
> >> > > > AbstractFetcherThread.shutdown() should not block on
> >> > > > ReadableByteChannel.read(buffer)
> >> > > > - KAFKA-2272 <https://issues.apache.org/jira/browse/KAFKA-2272
> >:
> >> > > > listeners endpoint parsing fails if the hostname has capital
> >> letter
> >> > > > - KAFKA-2345 <https://issues.apache.org/jira/browse/KAFKA-2345
> >:
> >> > > > Attempt to delete a topic already marked for deletion throws
> >> > > > ZkNodeExistsException
> >> > > > - KAFKA-2353 <https://issues.apache.org/jira/browse/KAFKA-2353
> >:
> >> > > > SocketServer.Processor should catch exception and close the
> >> socket
> >> > > properly
> >> > > > in configureNewConnections.
> >> > > > - KAFKA-1836 <https://issues.apache.org/jira/browse/KAFKA-1836
> >:
> >> > > > metadata.fetch.timeout.ms set to zero blocks forever
> >> > > > - KAFKA-2317 <https://issues.apache.org/jira/browse/KAFKA-2317
> >:
> >> > > De-register
> >> > > > isrChangeNotificationListener on controller resignation
> >> > > >
> >> > > > Note: KAFKA-2120 <
> https://issues.apache.org/jira/browse/KAFKA-2120>
> >> &
> >> > > > KAFKA-2421 <https://issues.apache.org/jira/browse/KAFKA-2421>
> were
> >> > > > mentioned in previous emails, but are not in the list because they
> >> are
> >> > > not
> >> > > > committed yet.
> >> > > >
> >> > > > Hope that helps the effort.
> >> > > >
> >> > > > Thanks,
> >> > > > Grant
> >> > > >
> >> > > > On Mon, Aug 17, 2015 at 12:09 AM, Grant Henke <
> ghenke@cloudera.com>
> >> > > wrote:
> >> > > >
> >> > > >> +1 to that suggestion. Though I suspect that requires a committer
> >> to
> >> > do.
> >> > > >> Making it part of the standard commit process could work too.
> >> > > >> On Aug 16, 2015 11:01 PM, "Gwen Shapira" <gw...@confluent.io>
> >> wrote:
> >> > > >>
> >> > > >>> BTW. I think it will be great for Apache Kafka to have a 0.8.2
> >> > "release
> >> > > >>> manager" who's role is to cherrypick low-risk bug-fixes into the
> >> > 0.8.2
> >> > > >>> branch and once enough bug fixes happened (or if sufficiently
> >> > critical
> >> > > >>> fixes happened) to roll out a new maintenance release (with
> every
> >> 3
> >> > > month
> >> > > >>> as a reasonable bugfix release target).
> >> > > >>>
> >> > > >>> This will add some predictability regarding how fast we release
> >> fixes
> >> > > for
> >> > > >>> bugs.
> >> > > >>>
> >> > > >>> Gwen
> >> > > >>>
> >> > > >>> On Sun, Aug 16, 2015 at 8:09 PM, Jeff Holoman <
> >> jholoman@cloudera.com
> >> > >
> >> > > >>> wrote:
> >> > > >>>
> >> > > >>> > +1 for the release and also including
> >> > > >>> >
> >> > > >>> > https://issues.apache.org/jira/browse/KAFKA-2114
> >> > > >>> >
> >> > > >>> > Thanks
> >> > > >>> >
> >> > > >>> > Jeff
> >> > > >>> >
> >> > > >>> > On Sun, Aug 16, 2015 at 2:51 PM, Stevo Slavić <
> >> sslavic@gmail.com>
> >> > > >>> wrote:
> >> > > >>> >
> >> > > >>> > > +1 (non-binding) for 0.8.2.2 release
> >> > > >>> > >
> >> > > >>> > > Would be nice to include in that release new producer
> >> resiliency
> >> > > bug
> >> > > >>> > fixes
> >> > > >>> > > https://issues.apache.org/jira/browse/KAFKA-1788 and
> >> > > >>> > > https://issues.apache.org/jira/browse/KAFKA-2120
> >> > > >>> > >
> >> > > >>> > > On Fri, Aug 14, 2015 at 4:03 PM, Gwen Shapira <
> >> gwen@confluent.io
> >> > >
> >> > > >>> wrote:
> >> > > >>> > >
> >> > > >>> > > > Will be nice to include Kafka-2308 and fix two critical
> >> snappy
> >> > > >>> issues
> >> > > >>> > in
> >> > > >>> > > > the maintenance release.
> >> > > >>> > > >
> >> > > >>> > > > Gwen
> >> > > >>> > > > On Aug 14, 2015 6:16 AM, "Grant Henke" <
> ghenke@cloudera.com
> >> >
> >> > > >>> wrote:
> >> > > >>> > > >
> >> > > >>> > > > > Just to clarify. Will KAFKA-2189 be the only patch in
> the
> >> > > >>> release?
> >> > > >>> > > > >
> >> > > >>> > > > > On Fri, Aug 14, 2015 at 7:35 AM, Manikumar Reddy <
> >> > > >>> > kumar@nmsworks.co.in
> >> > > >>> > > >
> >> > > >>> > > > > wrote:
> >> > > >>> > > > >
> >> > > >>> > > > > > +1 for 0.8.2.2 release
> >> > > >>> > > > > >
> >> > > >>> > > > > > On Fri, Aug 14, 2015 at 5:49 PM, Ismael Juma <
> >> > > >>> ismael@juma.me.uk>
> >> > > >>> > > > wrote:
> >> > > >>> > > > > >
> >> > > >>> > > > > > > I think this is a good idea as the change is minimal
> >> on
> >> > our
> >> > > >>> side
> >> > > >>> > > and
> >> > > >>> > > > it
> >> > > >>> > > > > > has
> >> > > >>> > > > > > > been tested in production for some time by the
> >> reporter.
> >> > > >>> > > > > > >
> >> > > >>> > > > > > > Best,
> >> > > >>> > > > > > > Ismael
> >> > > >>> > > > > > >
> >> > > >>> > > > > > > On Fri, Aug 14, 2015 at 1:15 PM, Jun Rao <
> >> > jun@confluent.io
> >> > > >
> >> > > >>> > wrote:
> >> > > >>> > > > > > >
> >> > > >>> > > > > > > > Hi, Everyone,
> >> > > >>> > > > > > > >
> >> > > >>> > > > > > > > Since the release of Kafka 0.8.2.1, a number of
> >> people
> >> > > have
> >> > > >>> > > > reported
> >> > > >>> > > > > an
> >> > > >>> > > > > > > > issue with snappy compression (
> >> > > >>> > > > > > > > https://issues.apache.org/jira/browse/KAFKA-2189
> ).
> >> > > >>> Basically,
> >> > > >>> > if
> >> > > >>> > > > > they
> >> > > >>> > > > > > > use
> >> > > >>> > > > > > > > snappy in 0.8.2.1, they will experience a 2-3X
> space
> >> > > >>> increase.
> >> > > >>> > > The
> >> > > >>> > > > > > issue
> >> > > >>> > > > > > > > has since been fixed in trunk (just a snappy jar
> >> > > upgrade).
> >> > > >>> > Since
> >> > > >>> > > > > 0.8.3
> >> > > >>> > > > > > is
> >> > > >>> > > > > > > > still a few months away, it may make sense to do
> an
> >> > > 0.8.2.2
> >> > > >>> > > release
> >> > > >>> > > > > > just
> >> > > >>> > > > > > > to
> >> > > >>> > > > > > > > fix this issue. Any objections?
> >> > > >>> > > > > > > >
> >> > > >>> > > > > > > > Thanks,
> >> > > >>> > > > > > > >
> >> > > >>> > > > > > > > Jun
> >> > > >>> > > > > > > >
> >> > > >>> > > > > > >
> >> > > >>> > > > > >
> >> > > >>> > > > >
> >> > > >>> > > > >
> >> > > >>> > > > >
> >> > > >>> > > > > --
> >> > > >>> > > > > Grant Henke
> >> > > >>> > > > > Software Engineer | Cloudera
> >> > > >>> > > > > grant@cloudera.com | twitter.com/gchenke |
> >> > > >>> > linkedin.com/in/granthenke
> >> > > >>> > > > >
> >> > > >>> > > >
> >> > > >>> > >
> >> > > >>> >
> >> > > >>> >
> >> > > >>> >
> >> > > >>> > --
> >> > > >>> > Jeff Holoman
> >> > > >>> > Systems Engineer
> >> > > >>> >
> >> > > >>>
> >> > > >>
> >> > > >
> >> > > >
> >> > > > --
> >> > > > Grant Henke
> >> > > > Software Engineer | Cloudera
> >> > > > grant@cloudera.com | twitter.com/gchenke |
> >> linkedin.com/in/granthenke
> >> > > >
> >> > >
> >> > >
> >> > >
> >> > > --
> >> > > Grant Henke
> >> > > Software Engineer | Cloudera
> >> > > grant@cloudera.com | twitter.com/gchenke |
> linkedin.com/in/granthenke
> >> > >
> >> >
> >>
> >
> >
>
Re: [DISCUSSION] Kafka 0.8.2.2 release?
Posted by Gwen Shapira <gw...@confluent.io>.
Any objections if I leave KAFKA-2114 (setting min.insync.replicas default)
out?
The test code is using changes that were done after 0.8.2.x cut-off, which
makes it difficult to cherry-pick.
Gwen
On Tue, Aug 18, 2015 at 12:16 PM, Gwen Shapira <gw...@confluent.io> wrote:
> Jun,
>
> KAFKA-2147 doesn't seem to have a commit associated with it, so I can't
> cherrypick just this fix.
> I suggest leaving this out since there is a 0.8.2.x workaround in the JIRA.
>
> Gwen
>
> On Mon, Aug 17, 2015 at 5:24 PM, Jun Rao <ju...@confluent.io> wrote:
>
>> Gwen,
>>
>> Thanks for putting the list together.
>>
>> I'd recommend that we exclude the following:
>> KAFKA-1702: This is for the old producer and is only a problem if there
>> are
>> some unexpected exceptions (e.g. UnknownClass).
>> KAFKA-2336: Most people don't change offsets.topic.num.partitions.
>> KAFKA-1724: The patch there is never committed since the fix is included
>> in
>> another jira (a much larger patch).
>> KAFKA-2241: This doesn't seem be a common problem. It only happens when
>> the
>> fetch request blocks on the broker for an extended period of time, which
>> should be rare.
>>
>> I'd also recommend that we include the following:
>> KAFKA-2147: This impacts the memory size of the purgatory and a number of
>> people have experienced that. The fix is small and has been tested in
>> production usage. It hasn't been committed though since the issue is
>> already fixed in trunk and we weren't planning for an 0.8.2.2 release
>> then.
>>
>> Thanks,
>>
>> Jun
>>
>> On Mon, Aug 17, 2015 at 2:56 PM, Gwen Shapira <gw...@confluent.io> wrote:
>>
>> > Thanks for creating a list, Grant!
>> >
>> > I placed it on the wiki with a quick evaluation of the content and
>> whether
>> > it should be in 0.8.2.2:
>> >
>> >
>> https://cwiki.apache.org/confluence/display/KAFKA/Proposed+patches+for+0.8.2.2
>> >
>> > I'm attempting to only cherrypick fixes that are both important for
>> large
>> > number of users (or very critical to some users) and very safe (mostly
>> > judged by the size of the change, but not only)
>> >
>> > If your favorite bugfix is missing from the list, or is there but marked
>> > "No", please let us know (in this thread) what we are missing and why
>> it is
>> > both important and safe.
>> > Also, if I accidentally included something you consider unsafe, speak
>> up!
>> >
>> > Gwen
>> >
>> > On Mon, Aug 17, 2015 at 8:17 AM, Grant Henke <gh...@cloudera.com>
>> wrote:
>> >
>> > > +dev
>> > >
>> > > Adding dev list back in. Somehow it got dropped.
>> > >
>> > >
>> > > On Mon, Aug 17, 2015 at 10:16 AM, Grant Henke <gh...@cloudera.com>
>> > wrote:
>> > >
>> > > > Below is a list of candidate bug fix jiras marked fixed for 0.8.3. I
>> > > don't
>> > > > suspect all of these will (or should) make it into the release but
>> this
>> > > > should be a relatively complete list to work from:
>> > > >
>> > > > - KAFKA-2114 <https://issues.apache.org/jira/browse/KAFKA-2114>:
>> > > Unable
>> > > > to change min.insync.replicas default
>> > > > - KAFKA-1702 <https://issues.apache.org/jira/browse/KAFKA-1702>:
>> > > > Messages silently Lost by producer
>> > > > - KAFKA-2012 <https://issues.apache.org/jira/browse/KAFKA-2012>:
>> > > > Broker should automatically handle corrupt index files
>> > > > - KAFKA-2406 <https://issues.apache.org/jira/browse/KAFKA-2406>:
>> > ISR
>> > > > propagation should be throttled to avoid overwhelming controller.
>> > > > - KAFKA-2336 <https://issues.apache.org/jira/browse/KAFKA-2336>:
>> > > > Changing offsets.topic.num.partitions after the offset topic is
>> > > created
>> > > > breaks consumer group partition assignment
>> > > > - KAFKA-2337 <https://issues.apache.org/jira/browse/KAFKA-2337>:
>> > > Verify
>> > > > that metric names will not collide when creating new topics
>> > > > - KAFKA-2393 <https://issues.apache.org/jira/browse/KAFKA-2393>:
>> > > > Correctly Handle InvalidTopicException in
>> > KafkaApis.getTopicMetadata()
>> > > > - KAFKA-2189 <https://issues.apache.org/jira/browse/KAFKA-2189>:
>> > > Snappy
>> > > > compression of message batches less efficient in 0.8.2.1
>> > > > - KAFKA-2308 <https://issues.apache.org/jira/browse/KAFKA-2308>:
>> > New
>> > > > producer + Snappy face un-compression errors after broker restart
>> > > > - KAFKA-2042 <https://issues.apache.org/jira/browse/KAFKA-2042>:
>> > New
>> > > > producer metadata update always get all topics.
>> > > > - KAFKA-1367 <https://issues.apache.org/jira/browse/KAFKA-1367>:
>> > > Broker
>> > > > topic metadata not kept in sync with ZooKeeper
>> > > > - KAFKA-972 <https://issues.apache.org/jira/browse/KAFKA-972>:
>> > > MetadataRequest
>> > > > returns stale list of brokers
>> > > > - KAFKA-1867 <https://issues.apache.org/jira/browse/KAFKA-1867>:
>> > > liveBroker
>> > > > list not updated on a cluster with no topics
>> > > > - KAFKA-1650 <https://issues.apache.org/jira/browse/KAFKA-1650>:
>> > > Mirror
>> > > > Maker could lose data on unclean shutdown.
>> > > > - KAFKA-2009 <https://issues.apache.org/jira/browse/KAFKA-2009>:
>> > Fix
>> > > > UncheckedOffset.removeOffset synchronization and trace logging
>> issue
>> > > in
>> > > > mirror maker
>> > > > - KAFKA-2407 <https://issues.apache.org/jira/browse/KAFKA-2407>:
>> > Only
>> > > > create a log directory when it will be used
>> > > > - KAFKA-2327 <https://issues.apache.org/jira/browse/KAFKA-2327>:
>> > > > broker doesn't start if config defines advertised.host but not
>> > > > advertised.port
>> > > > - KAFKA-1788: producer record can stay in RecordAccumulator
>> forever
>> > if
>> > > > leader is no available
>> > > > - KAFKA-2234 <https://issues.apache.org/jira/browse/KAFKA-2234>:
>> > > > Partition reassignment of a nonexistent topic prevents future
>> > > reassignments
>> > > > - KAFKA-2096 <https://issues.apache.org/jira/browse/KAFKA-2096>:
>> > > > Enable keepalive socket option for broker to prevent socket leak
>> > > > - KAFKA-1057 <https://issues.apache.org/jira/browse/KAFKA-1057>:
>> > Trim
>> > > > whitespaces from user specified configs
>> > > > - KAFKA-1641 <https://issues.apache.org/jira/browse/KAFKA-1641>:
>> > Log
>> > > > cleaner exits if last cleaned offset is lower than earliest
>> offset
>> > > > - KAFKA-1648 <https://issues.apache.org/jira/browse/KAFKA-1648>:
>> > > Round
>> > > > robin consumer balance throws an NPE when there are no topics
>> > > > - KAFKA-1724 <https://issues.apache.org/jira/browse/KAFKA-1724>:
>> > > > Errors after reboot in single node setup
>> > > > - KAFKA-1758 <https://issues.apache.org/jira/browse/KAFKA-1758>:
>> > > > corrupt recovery file prevents startup
>> > > > - KAFKA-1866 <https://issues.apache.org/jira/browse/KAFKA-1866>:
>> > > > LogStartOffset gauge throws exceptions after log.delete()
>> > > > - KAFKA-1883 <https://issues.apache.org/jira/browse/KAFKA-1883>:
>> > > NullPointerException
>> > > > in RequestSendThread
>> > > > - KAFKA-1896 <https://issues.apache.org/jira/browse/KAFKA-1896>:
>> > > > Record size funcition of record in mirror maker hit NPE when the
>> > > message
>> > > > value is null.
>> > > > - KAFKA-2101 <https://issues.apache.org/jira/browse/KAFKA-2101>:
>> > > > Metric metadata-age is reset on a failed update
>> > > > - KAFKA-2112 <https://issues.apache.org/jira/browse/KAFKA-2112>:
>> > make
>> > > > overflowWheel volatile
>> > > > - KAFKA-2117 <https://issues.apache.org/jira/browse/KAFKA-2117>:
>> > > > OffsetManager uses incorrect field for metadata
>> > > > - KAFKA-2164 <https://issues.apache.org/jira/browse/KAFKA-2164>:
>> > > > ReplicaFetcherThread: suspicious log message on reset offset
>> > > > - KAFKA-1668 <https://issues.apache.org/jira/browse/KAFKA-1668>:
>> > > > TopicCommand doesn't warn if --topic argument doesn't match any
>> > topics
>> > > > - KAFKA-2198 <https://issues.apache.org/jira/browse/KAFKA-2198>:
>> > > > kafka-topics.sh exits with 0 status on failures
>> > > > - KAFKA-2235 <https://issues.apache.org/jira/browse/KAFKA-2235>:
>> > > > LogCleaner offset map overflow
>> > > > - KAFKA-2241 <https://issues.apache.org/jira/browse/KAFKA-2241>:
>> > > > AbstractFetcherThread.shutdown() should not block on
>> > > > ReadableByteChannel.read(buffer)
>> > > > - KAFKA-2272 <https://issues.apache.org/jira/browse/KAFKA-2272>:
>> > > > listeners endpoint parsing fails if the hostname has capital
>> letter
>> > > > - KAFKA-2345 <https://issues.apache.org/jira/browse/KAFKA-2345>:
>> > > > Attempt to delete a topic already marked for deletion throws
>> > > > ZkNodeExistsException
>> > > > - KAFKA-2353 <https://issues.apache.org/jira/browse/KAFKA-2353>:
>> > > > SocketServer.Processor should catch exception and close the
>> socket
>> > > properly
>> > > > in configureNewConnections.
>> > > > - KAFKA-1836 <https://issues.apache.org/jira/browse/KAFKA-1836>:
>> > > > metadata.fetch.timeout.ms set to zero blocks forever
>> > > > - KAFKA-2317 <https://issues.apache.org/jira/browse/KAFKA-2317>:
>> > > De-register
>> > > > isrChangeNotificationListener on controller resignation
>> > > >
>> > > > Note: KAFKA-2120 <https://issues.apache.org/jira/browse/KAFKA-2120>
>> &
>> > > > KAFKA-2421 <https://issues.apache.org/jira/browse/KAFKA-2421> were
>> > > > mentioned in previous emails, but are not in the list because they
>> are
>> > > not
>> > > > committed yet.
>> > > >
>> > > > Hope that helps the effort.
>> > > >
>> > > > Thanks,
>> > > > Grant
>> > > >
>> > > > On Mon, Aug 17, 2015 at 12:09 AM, Grant Henke <gh...@cloudera.com>
>> > > wrote:
>> > > >
>> > > >> +1 to that suggestion. Though I suspect that requires a committer
>> to
>> > do.
>> > > >> Making it part of the standard commit process could work too.
>> > > >> On Aug 16, 2015 11:01 PM, "Gwen Shapira" <gw...@confluent.io>
>> wrote:
>> > > >>
>> > > >>> BTW. I think it will be great for Apache Kafka to have a 0.8.2
>> > "release
>> > > >>> manager" who's role is to cherrypick low-risk bug-fixes into the
>> > 0.8.2
>> > > >>> branch and once enough bug fixes happened (or if sufficiently
>> > critical
>> > > >>> fixes happened) to roll out a new maintenance release (with every
>> 3
>> > > month
>> > > >>> as a reasonable bugfix release target).
>> > > >>>
>> > > >>> This will add some predictability regarding how fast we release
>> fixes
>> > > for
>> > > >>> bugs.
>> > > >>>
>> > > >>> Gwen
>> > > >>>
>> > > >>> On Sun, Aug 16, 2015 at 8:09 PM, Jeff Holoman <
>> jholoman@cloudera.com
>> > >
>> > > >>> wrote:
>> > > >>>
>> > > >>> > +1 for the release and also including
>> > > >>> >
>> > > >>> > https://issues.apache.org/jira/browse/KAFKA-2114
>> > > >>> >
>> > > >>> > Thanks
>> > > >>> >
>> > > >>> > Jeff
>> > > >>> >
>> > > >>> > On Sun, Aug 16, 2015 at 2:51 PM, Stevo Slavić <
>> sslavic@gmail.com>
>> > > >>> wrote:
>> > > >>> >
>> > > >>> > > +1 (non-binding) for 0.8.2.2 release
>> > > >>> > >
>> > > >>> > > Would be nice to include in that release new producer
>> resiliency
>> > > bug
>> > > >>> > fixes
>> > > >>> > > https://issues.apache.org/jira/browse/KAFKA-1788 and
>> > > >>> > > https://issues.apache.org/jira/browse/KAFKA-2120
>> > > >>> > >
>> > > >>> > > On Fri, Aug 14, 2015 at 4:03 PM, Gwen Shapira <
>> gwen@confluent.io
>> > >
>> > > >>> wrote:
>> > > >>> > >
>> > > >>> > > > Will be nice to include Kafka-2308 and fix two critical
>> snappy
>> > > >>> issues
>> > > >>> > in
>> > > >>> > > > the maintenance release.
>> > > >>> > > >
>> > > >>> > > > Gwen
>> > > >>> > > > On Aug 14, 2015 6:16 AM, "Grant Henke" <ghenke@cloudera.com
>> >
>> > > >>> wrote:
>> > > >>> > > >
>> > > >>> > > > > Just to clarify. Will KAFKA-2189 be the only patch in the
>> > > >>> release?
>> > > >>> > > > >
>> > > >>> > > > > On Fri, Aug 14, 2015 at 7:35 AM, Manikumar Reddy <
>> > > >>> > kumar@nmsworks.co.in
>> > > >>> > > >
>> > > >>> > > > > wrote:
>> > > >>> > > > >
>> > > >>> > > > > > +1 for 0.8.2.2 release
>> > > >>> > > > > >
>> > > >>> > > > > > On Fri, Aug 14, 2015 at 5:49 PM, Ismael Juma <
>> > > >>> ismael@juma.me.uk>
>> > > >>> > > > wrote:
>> > > >>> > > > > >
>> > > >>> > > > > > > I think this is a good idea as the change is minimal
>> on
>> > our
>> > > >>> side
>> > > >>> > > and
>> > > >>> > > > it
>> > > >>> > > > > > has
>> > > >>> > > > > > > been tested in production for some time by the
>> reporter.
>> > > >>> > > > > > >
>> > > >>> > > > > > > Best,
>> > > >>> > > > > > > Ismael
>> > > >>> > > > > > >
>> > > >>> > > > > > > On Fri, Aug 14, 2015 at 1:15 PM, Jun Rao <
>> > jun@confluent.io
>> > > >
>> > > >>> > wrote:
>> > > >>> > > > > > >
>> > > >>> > > > > > > > Hi, Everyone,
>> > > >>> > > > > > > >
>> > > >>> > > > > > > > Since the release of Kafka 0.8.2.1, a number of
>> people
>> > > have
>> > > >>> > > > reported
>> > > >>> > > > > an
>> > > >>> > > > > > > > issue with snappy compression (
>> > > >>> > > > > > > > https://issues.apache.org/jira/browse/KAFKA-2189).
>> > > >>> Basically,
>> > > >>> > if
>> > > >>> > > > > they
>> > > >>> > > > > > > use
>> > > >>> > > > > > > > snappy in 0.8.2.1, they will experience a 2-3X space
>> > > >>> increase.
>> > > >>> > > The
>> > > >>> > > > > > issue
>> > > >>> > > > > > > > has since been fixed in trunk (just a snappy jar
>> > > upgrade).
>> > > >>> > Since
>> > > >>> > > > > 0.8.3
>> > > >>> > > > > > is
>> > > >>> > > > > > > > still a few months away, it may make sense to do an
>> > > 0.8.2.2
>> > > >>> > > release
>> > > >>> > > > > > just
>> > > >>> > > > > > > to
>> > > >>> > > > > > > > fix this issue. Any objections?
>> > > >>> > > > > > > >
>> > > >>> > > > > > > > Thanks,
>> > > >>> > > > > > > >
>> > > >>> > > > > > > > Jun
>> > > >>> > > > > > > >
>> > > >>> > > > > > >
>> > > >>> > > > > >
>> > > >>> > > > >
>> > > >>> > > > >
>> > > >>> > > > >
>> > > >>> > > > > --
>> > > >>> > > > > Grant Henke
>> > > >>> > > > > Software Engineer | Cloudera
>> > > >>> > > > > grant@cloudera.com | twitter.com/gchenke |
>> > > >>> > linkedin.com/in/granthenke
>> > > >>> > > > >
>> > > >>> > > >
>> > > >>> > >
>> > > >>> >
>> > > >>> >
>> > > >>> >
>> > > >>> > --
>> > > >>> > Jeff Holoman
>> > > >>> > Systems Engineer
>> > > >>> >
>> > > >>>
>> > > >>
>> > > >
>> > > >
>> > > > --
>> > > > Grant Henke
>> > > > Software Engineer | Cloudera
>> > > > grant@cloudera.com | twitter.com/gchenke |
>> linkedin.com/in/granthenke
>> > > >
>> > >
>> > >
>> > >
>> > > --
>> > > Grant Henke
>> > > Software Engineer | Cloudera
>> > > grant@cloudera.com | twitter.com/gchenke | linkedin.com/in/granthenke
>> > >
>> >
>>
>
>
Re: [DISCUSSION] Kafka 0.8.2.2 release?
Posted by Gwen Shapira <gw...@confluent.io>.
Any objections if I leave KAFKA-2114 (setting min.insync.replicas default)
out?
The test code is using changes that were done after 0.8.2.x cut-off, which
makes it difficult to cherry-pick.
Gwen
On Tue, Aug 18, 2015 at 12:16 PM, Gwen Shapira <gw...@confluent.io> wrote:
> Jun,
>
> KAFKA-2147 doesn't seem to have a commit associated with it, so I can't
> cherrypick just this fix.
> I suggest leaving this out since there is a 0.8.2.x workaround in the JIRA.
>
> Gwen
>
> On Mon, Aug 17, 2015 at 5:24 PM, Jun Rao <ju...@confluent.io> wrote:
>
>> Gwen,
>>
>> Thanks for putting the list together.
>>
>> I'd recommend that we exclude the following:
>> KAFKA-1702: This is for the old producer and is only a problem if there
>> are
>> some unexpected exceptions (e.g. UnknownClass).
>> KAFKA-2336: Most people don't change offsets.topic.num.partitions.
>> KAFKA-1724: The patch there is never committed since the fix is included
>> in
>> another jira (a much larger patch).
>> KAFKA-2241: This doesn't seem be a common problem. It only happens when
>> the
>> fetch request blocks on the broker for an extended period of time, which
>> should be rare.
>>
>> I'd also recommend that we include the following:
>> KAFKA-2147: This impacts the memory size of the purgatory and a number of
>> people have experienced that. The fix is small and has been tested in
>> production usage. It hasn't been committed though since the issue is
>> already fixed in trunk and we weren't planning for an 0.8.2.2 release
>> then.
>>
>> Thanks,
>>
>> Jun
>>
>> On Mon, Aug 17, 2015 at 2:56 PM, Gwen Shapira <gw...@confluent.io> wrote:
>>
>> > Thanks for creating a list, Grant!
>> >
>> > I placed it on the wiki with a quick evaluation of the content and
>> whether
>> > it should be in 0.8.2.2:
>> >
>> >
>> https://cwiki.apache.org/confluence/display/KAFKA/Proposed+patches+for+0.8.2.2
>> >
>> > I'm attempting to only cherrypick fixes that are both important for
>> large
>> > number of users (or very critical to some users) and very safe (mostly
>> > judged by the size of the change, but not only)
>> >
>> > If your favorite bugfix is missing from the list, or is there but marked
>> > "No", please let us know (in this thread) what we are missing and why
>> it is
>> > both important and safe.
>> > Also, if I accidentally included something you consider unsafe, speak
>> up!
>> >
>> > Gwen
>> >
>> > On Mon, Aug 17, 2015 at 8:17 AM, Grant Henke <gh...@cloudera.com>
>> wrote:
>> >
>> > > +dev
>> > >
>> > > Adding dev list back in. Somehow it got dropped.
>> > >
>> > >
>> > > On Mon, Aug 17, 2015 at 10:16 AM, Grant Henke <gh...@cloudera.com>
>> > wrote:
>> > >
>> > > > Below is a list of candidate bug fix jiras marked fixed for 0.8.3. I
>> > > don't
>> > > > suspect all of these will (or should) make it into the release but
>> this
>> > > > should be a relatively complete list to work from:
>> > > >
>> > > > - KAFKA-2114 <https://issues.apache.org/jira/browse/KAFKA-2114>:
>> > > Unable
>> > > > to change min.insync.replicas default
>> > > > - KAFKA-1702 <https://issues.apache.org/jira/browse/KAFKA-1702>:
>> > > > Messages silently Lost by producer
>> > > > - KAFKA-2012 <https://issues.apache.org/jira/browse/KAFKA-2012>:
>> > > > Broker should automatically handle corrupt index files
>> > > > - KAFKA-2406 <https://issues.apache.org/jira/browse/KAFKA-2406>:
>> > ISR
>> > > > propagation should be throttled to avoid overwhelming controller.
>> > > > - KAFKA-2336 <https://issues.apache.org/jira/browse/KAFKA-2336>:
>> > > > Changing offsets.topic.num.partitions after the offset topic is
>> > > created
>> > > > breaks consumer group partition assignment
>> > > > - KAFKA-2337 <https://issues.apache.org/jira/browse/KAFKA-2337>:
>> > > Verify
>> > > > that metric names will not collide when creating new topics
>> > > > - KAFKA-2393 <https://issues.apache.org/jira/browse/KAFKA-2393>:
>> > > > Correctly Handle InvalidTopicException in
>> > KafkaApis.getTopicMetadata()
>> > > > - KAFKA-2189 <https://issues.apache.org/jira/browse/KAFKA-2189>:
>> > > Snappy
>> > > > compression of message batches less efficient in 0.8.2.1
>> > > > - KAFKA-2308 <https://issues.apache.org/jira/browse/KAFKA-2308>:
>> > New
>> > > > producer + Snappy face un-compression errors after broker restart
>> > > > - KAFKA-2042 <https://issues.apache.org/jira/browse/KAFKA-2042>:
>> > New
>> > > > producer metadata update always get all topics.
>> > > > - KAFKA-1367 <https://issues.apache.org/jira/browse/KAFKA-1367>:
>> > > Broker
>> > > > topic metadata not kept in sync with ZooKeeper
>> > > > - KAFKA-972 <https://issues.apache.org/jira/browse/KAFKA-972>:
>> > > MetadataRequest
>> > > > returns stale list of brokers
>> > > > - KAFKA-1867 <https://issues.apache.org/jira/browse/KAFKA-1867>:
>> > > liveBroker
>> > > > list not updated on a cluster with no topics
>> > > > - KAFKA-1650 <https://issues.apache.org/jira/browse/KAFKA-1650>:
>> > > Mirror
>> > > > Maker could lose data on unclean shutdown.
>> > > > - KAFKA-2009 <https://issues.apache.org/jira/browse/KAFKA-2009>:
>> > Fix
>> > > > UncheckedOffset.removeOffset synchronization and trace logging
>> issue
>> > > in
>> > > > mirror maker
>> > > > - KAFKA-2407 <https://issues.apache.org/jira/browse/KAFKA-2407>:
>> > Only
>> > > > create a log directory when it will be used
>> > > > - KAFKA-2327 <https://issues.apache.org/jira/browse/KAFKA-2327>:
>> > > > broker doesn't start if config defines advertised.host but not
>> > > > advertised.port
>> > > > - KAFKA-1788: producer record can stay in RecordAccumulator
>> forever
>> > if
>> > > > leader is no available
>> > > > - KAFKA-2234 <https://issues.apache.org/jira/browse/KAFKA-2234>:
>> > > > Partition reassignment of a nonexistent topic prevents future
>> > > reassignments
>> > > > - KAFKA-2096 <https://issues.apache.org/jira/browse/KAFKA-2096>:
>> > > > Enable keepalive socket option for broker to prevent socket leak
>> > > > - KAFKA-1057 <https://issues.apache.org/jira/browse/KAFKA-1057>:
>> > Trim
>> > > > whitespaces from user specified configs
>> > > > - KAFKA-1641 <https://issues.apache.org/jira/browse/KAFKA-1641>:
>> > Log
>> > > > cleaner exits if last cleaned offset is lower than earliest
>> offset
>> > > > - KAFKA-1648 <https://issues.apache.org/jira/browse/KAFKA-1648>:
>> > > Round
>> > > > robin consumer balance throws an NPE when there are no topics
>> > > > - KAFKA-1724 <https://issues.apache.org/jira/browse/KAFKA-1724>:
>> > > > Errors after reboot in single node setup
>> > > > - KAFKA-1758 <https://issues.apache.org/jira/browse/KAFKA-1758>:
>> > > > corrupt recovery file prevents startup
>> > > > - KAFKA-1866 <https://issues.apache.org/jira/browse/KAFKA-1866>:
>> > > > LogStartOffset gauge throws exceptions after log.delete()
>> > > > - KAFKA-1883 <https://issues.apache.org/jira/browse/KAFKA-1883>:
>> > > NullPointerException
>> > > > in RequestSendThread
>> > > > - KAFKA-1896 <https://issues.apache.org/jira/browse/KAFKA-1896>:
>> > > > Record size funcition of record in mirror maker hit NPE when the
>> > > message
>> > > > value is null.
>> > > > - KAFKA-2101 <https://issues.apache.org/jira/browse/KAFKA-2101>:
>> > > > Metric metadata-age is reset on a failed update
>> > > > - KAFKA-2112 <https://issues.apache.org/jira/browse/KAFKA-2112>:
>> > make
>> > > > overflowWheel volatile
>> > > > - KAFKA-2117 <https://issues.apache.org/jira/browse/KAFKA-2117>:
>> > > > OffsetManager uses incorrect field for metadata
>> > > > - KAFKA-2164 <https://issues.apache.org/jira/browse/KAFKA-2164>:
>> > > > ReplicaFetcherThread: suspicious log message on reset offset
>> > > > - KAFKA-1668 <https://issues.apache.org/jira/browse/KAFKA-1668>:
>> > > > TopicCommand doesn't warn if --topic argument doesn't match any
>> > topics
>> > > > - KAFKA-2198 <https://issues.apache.org/jira/browse/KAFKA-2198>:
>> > > > kafka-topics.sh exits with 0 status on failures
>> > > > - KAFKA-2235 <https://issues.apache.org/jira/browse/KAFKA-2235>:
>> > > > LogCleaner offset map overflow
>> > > > - KAFKA-2241 <https://issues.apache.org/jira/browse/KAFKA-2241>:
>> > > > AbstractFetcherThread.shutdown() should not block on
>> > > > ReadableByteChannel.read(buffer)
>> > > > - KAFKA-2272 <https://issues.apache.org/jira/browse/KAFKA-2272>:
>> > > > listeners endpoint parsing fails if the hostname has capital
>> letter
>> > > > - KAFKA-2345 <https://issues.apache.org/jira/browse/KAFKA-2345>:
>> > > > Attempt to delete a topic already marked for deletion throws
>> > > > ZkNodeExistsException
>> > > > - KAFKA-2353 <https://issues.apache.org/jira/browse/KAFKA-2353>:
>> > > > SocketServer.Processor should catch exception and close the
>> socket
>> > > properly
>> > > > in configureNewConnections.
>> > > > - KAFKA-1836 <https://issues.apache.org/jira/browse/KAFKA-1836>:
>> > > > metadata.fetch.timeout.ms set to zero blocks forever
>> > > > - KAFKA-2317 <https://issues.apache.org/jira/browse/KAFKA-2317>:
>> > > De-register
>> > > > isrChangeNotificationListener on controller resignation
>> > > >
>> > > > Note: KAFKA-2120 <https://issues.apache.org/jira/browse/KAFKA-2120>
>> &
>> > > > KAFKA-2421 <https://issues.apache.org/jira/browse/KAFKA-2421> were
>> > > > mentioned in previous emails, but are not in the list because they
>> are
>> > > not
>> > > > committed yet.
>> > > >
>> > > > Hope that helps the effort.
>> > > >
>> > > > Thanks,
>> > > > Grant
>> > > >
>> > > > On Mon, Aug 17, 2015 at 12:09 AM, Grant Henke <gh...@cloudera.com>
>> > > wrote:
>> > > >
>> > > >> +1 to that suggestion. Though I suspect that requires a committer
>> to
>> > do.
>> > > >> Making it part of the standard commit process could work too.
>> > > >> On Aug 16, 2015 11:01 PM, "Gwen Shapira" <gw...@confluent.io>
>> wrote:
>> > > >>
>> > > >>> BTW. I think it will be great for Apache Kafka to have a 0.8.2
>> > "release
>> > > >>> manager" who's role is to cherrypick low-risk bug-fixes into the
>> > 0.8.2
>> > > >>> branch and once enough bug fixes happened (or if sufficiently
>> > critical
>> > > >>> fixes happened) to roll out a new maintenance release (with every
>> 3
>> > > month
>> > > >>> as a reasonable bugfix release target).
>> > > >>>
>> > > >>> This will add some predictability regarding how fast we release
>> fixes
>> > > for
>> > > >>> bugs.
>> > > >>>
>> > > >>> Gwen
>> > > >>>
>> > > >>> On Sun, Aug 16, 2015 at 8:09 PM, Jeff Holoman <
>> jholoman@cloudera.com
>> > >
>> > > >>> wrote:
>> > > >>>
>> > > >>> > +1 for the release and also including
>> > > >>> >
>> > > >>> > https://issues.apache.org/jira/browse/KAFKA-2114
>> > > >>> >
>> > > >>> > Thanks
>> > > >>> >
>> > > >>> > Jeff
>> > > >>> >
>> > > >>> > On Sun, Aug 16, 2015 at 2:51 PM, Stevo Slavić <
>> sslavic@gmail.com>
>> > > >>> wrote:
>> > > >>> >
>> > > >>> > > +1 (non-binding) for 0.8.2.2 release
>> > > >>> > >
>> > > >>> > > Would be nice to include in that release new producer
>> resiliency
>> > > bug
>> > > >>> > fixes
>> > > >>> > > https://issues.apache.org/jira/browse/KAFKA-1788 and
>> > > >>> > > https://issues.apache.org/jira/browse/KAFKA-2120
>> > > >>> > >
>> > > >>> > > On Fri, Aug 14, 2015 at 4:03 PM, Gwen Shapira <
>> gwen@confluent.io
>> > >
>> > > >>> wrote:
>> > > >>> > >
>> > > >>> > > > Will be nice to include Kafka-2308 and fix two critical
>> snappy
>> > > >>> issues
>> > > >>> > in
>> > > >>> > > > the maintenance release.
>> > > >>> > > >
>> > > >>> > > > Gwen
>> > > >>> > > > On Aug 14, 2015 6:16 AM, "Grant Henke" <ghenke@cloudera.com
>> >
>> > > >>> wrote:
>> > > >>> > > >
>> > > >>> > > > > Just to clarify. Will KAFKA-2189 be the only patch in the
>> > > >>> release?
>> > > >>> > > > >
>> > > >>> > > > > On Fri, Aug 14, 2015 at 7:35 AM, Manikumar Reddy <
>> > > >>> > kumar@nmsworks.co.in
>> > > >>> > > >
>> > > >>> > > > > wrote:
>> > > >>> > > > >
>> > > >>> > > > > > +1 for 0.8.2.2 release
>> > > >>> > > > > >
>> > > >>> > > > > > On Fri, Aug 14, 2015 at 5:49 PM, Ismael Juma <
>> > > >>> ismael@juma.me.uk>
>> > > >>> > > > wrote:
>> > > >>> > > > > >
>> > > >>> > > > > > > I think this is a good idea as the change is minimal
>> on
>> > our
>> > > >>> side
>> > > >>> > > and
>> > > >>> > > > it
>> > > >>> > > > > > has
>> > > >>> > > > > > > been tested in production for some time by the
>> reporter.
>> > > >>> > > > > > >
>> > > >>> > > > > > > Best,
>> > > >>> > > > > > > Ismael
>> > > >>> > > > > > >
>> > > >>> > > > > > > On Fri, Aug 14, 2015 at 1:15 PM, Jun Rao <
>> > jun@confluent.io
>> > > >
>> > > >>> > wrote:
>> > > >>> > > > > > >
>> > > >>> > > > > > > > Hi, Everyone,
>> > > >>> > > > > > > >
>> > > >>> > > > > > > > Since the release of Kafka 0.8.2.1, a number of
>> people
>> > > have
>> > > >>> > > > reported
>> > > >>> > > > > an
>> > > >>> > > > > > > > issue with snappy compression (
>> > > >>> > > > > > > > https://issues.apache.org/jira/browse/KAFKA-2189).
>> > > >>> Basically,
>> > > >>> > if
>> > > >>> > > > > they
>> > > >>> > > > > > > use
>> > > >>> > > > > > > > snappy in 0.8.2.1, they will experience a 2-3X space
>> > > >>> increase.
>> > > >>> > > The
>> > > >>> > > > > > issue
>> > > >>> > > > > > > > has since been fixed in trunk (just a snappy jar
>> > > upgrade).
>> > > >>> > Since
>> > > >>> > > > > 0.8.3
>> > > >>> > > > > > is
>> > > >>> > > > > > > > still a few months away, it may make sense to do an
>> > > 0.8.2.2
>> > > >>> > > release
>> > > >>> > > > > > just
>> > > >>> > > > > > > to
>> > > >>> > > > > > > > fix this issue. Any objections?
>> > > >>> > > > > > > >
>> > > >>> > > > > > > > Thanks,
>> > > >>> > > > > > > >
>> > > >>> > > > > > > > Jun
>> > > >>> > > > > > > >
>> > > >>> > > > > > >
>> > > >>> > > > > >
>> > > >>> > > > >
>> > > >>> > > > >
>> > > >>> > > > >
>> > > >>> > > > > --
>> > > >>> > > > > Grant Henke
>> > > >>> > > > > Software Engineer | Cloudera
>> > > >>> > > > > grant@cloudera.com | twitter.com/gchenke |
>> > > >>> > linkedin.com/in/granthenke
>> > > >>> > > > >
>> > > >>> > > >
>> > > >>> > >
>> > > >>> >
>> > > >>> >
>> > > >>> >
>> > > >>> > --
>> > > >>> > Jeff Holoman
>> > > >>> > Systems Engineer
>> > > >>> >
>> > > >>>
>> > > >>
>> > > >
>> > > >
>> > > > --
>> > > > Grant Henke
>> > > > Software Engineer | Cloudera
>> > > > grant@cloudera.com | twitter.com/gchenke |
>> linkedin.com/in/granthenke
>> > > >
>> > >
>> > >
>> > >
>> > > --
>> > > Grant Henke
>> > > Software Engineer | Cloudera
>> > > grant@cloudera.com | twitter.com/gchenke | linkedin.com/in/granthenke
>> > >
>> >
>>
>
>
Re: [DISCUSSION] Kafka 0.8.2.2 release?
Posted by Gwen Shapira <gw...@confluent.io>.
Jun,
KAFKA-2147 doesn't seem to have a commit associated with it, so I can't
cherrypick just this fix.
I suggest leaving this out since there is a 0.8.2.x workaround in the JIRA.
Gwen
On Mon, Aug 17, 2015 at 5:24 PM, Jun Rao <ju...@confluent.io> wrote:
> Gwen,
>
> Thanks for putting the list together.
>
> I'd recommend that we exclude the following:
> KAFKA-1702: This is for the old producer and is only a problem if there are
> some unexpected exceptions (e.g. UnknownClass).
> KAFKA-2336: Most people don't change offsets.topic.num.partitions.
> KAFKA-1724: The patch there is never committed since the fix is included in
> another jira (a much larger patch).
> KAFKA-2241: This doesn't seem be a common problem. It only happens when the
> fetch request blocks on the broker for an extended period of time, which
> should be rare.
>
> I'd also recommend that we include the following:
> KAFKA-2147: This impacts the memory size of the purgatory and a number of
> people have experienced that. The fix is small and has been tested in
> production usage. It hasn't been committed though since the issue is
> already fixed in trunk and we weren't planning for an 0.8.2.2 release then.
>
> Thanks,
>
> Jun
>
> On Mon, Aug 17, 2015 at 2:56 PM, Gwen Shapira <gw...@confluent.io> wrote:
>
> > Thanks for creating a list, Grant!
> >
> > I placed it on the wiki with a quick evaluation of the content and
> whether
> > it should be in 0.8.2.2:
> >
> >
> https://cwiki.apache.org/confluence/display/KAFKA/Proposed+patches+for+0.8.2.2
> >
> > I'm attempting to only cherrypick fixes that are both important for large
> > number of users (or very critical to some users) and very safe (mostly
> > judged by the size of the change, but not only)
> >
> > If your favorite bugfix is missing from the list, or is there but marked
> > "No", please let us know (in this thread) what we are missing and why it
> is
> > both important and safe.
> > Also, if I accidentally included something you consider unsafe, speak up!
> >
> > Gwen
> >
> > On Mon, Aug 17, 2015 at 8:17 AM, Grant Henke <gh...@cloudera.com>
> wrote:
> >
> > > +dev
> > >
> > > Adding dev list back in. Somehow it got dropped.
> > >
> > >
> > > On Mon, Aug 17, 2015 at 10:16 AM, Grant Henke <gh...@cloudera.com>
> > wrote:
> > >
> > > > Below is a list of candidate bug fix jiras marked fixed for 0.8.3. I
> > > don't
> > > > suspect all of these will (or should) make it into the release but
> this
> > > > should be a relatively complete list to work from:
> > > >
> > > > - KAFKA-2114 <https://issues.apache.org/jira/browse/KAFKA-2114>:
> > > Unable
> > > > to change min.insync.replicas default
> > > > - KAFKA-1702 <https://issues.apache.org/jira/browse/KAFKA-1702>:
> > > > Messages silently Lost by producer
> > > > - KAFKA-2012 <https://issues.apache.org/jira/browse/KAFKA-2012>:
> > > > Broker should automatically handle corrupt index files
> > > > - KAFKA-2406 <https://issues.apache.org/jira/browse/KAFKA-2406>:
> > ISR
> > > > propagation should be throttled to avoid overwhelming controller.
> > > > - KAFKA-2336 <https://issues.apache.org/jira/browse/KAFKA-2336>:
> > > > Changing offsets.topic.num.partitions after the offset topic is
> > > created
> > > > breaks consumer group partition assignment
> > > > - KAFKA-2337 <https://issues.apache.org/jira/browse/KAFKA-2337>:
> > > Verify
> > > > that metric names will not collide when creating new topics
> > > > - KAFKA-2393 <https://issues.apache.org/jira/browse/KAFKA-2393>:
> > > > Correctly Handle InvalidTopicException in
> > KafkaApis.getTopicMetadata()
> > > > - KAFKA-2189 <https://issues.apache.org/jira/browse/KAFKA-2189>:
> > > Snappy
> > > > compression of message batches less efficient in 0.8.2.1
> > > > - KAFKA-2308 <https://issues.apache.org/jira/browse/KAFKA-2308>:
> > New
> > > > producer + Snappy face un-compression errors after broker restart
> > > > - KAFKA-2042 <https://issues.apache.org/jira/browse/KAFKA-2042>:
> > New
> > > > producer metadata update always get all topics.
> > > > - KAFKA-1367 <https://issues.apache.org/jira/browse/KAFKA-1367>:
> > > Broker
> > > > topic metadata not kept in sync with ZooKeeper
> > > > - KAFKA-972 <https://issues.apache.org/jira/browse/KAFKA-972>:
> > > MetadataRequest
> > > > returns stale list of brokers
> > > > - KAFKA-1867 <https://issues.apache.org/jira/browse/KAFKA-1867>:
> > > liveBroker
> > > > list not updated on a cluster with no topics
> > > > - KAFKA-1650 <https://issues.apache.org/jira/browse/KAFKA-1650>:
> > > Mirror
> > > > Maker could lose data on unclean shutdown.
> > > > - KAFKA-2009 <https://issues.apache.org/jira/browse/KAFKA-2009>:
> > Fix
> > > > UncheckedOffset.removeOffset synchronization and trace logging
> issue
> > > in
> > > > mirror maker
> > > > - KAFKA-2407 <https://issues.apache.org/jira/browse/KAFKA-2407>:
> > Only
> > > > create a log directory when it will be used
> > > > - KAFKA-2327 <https://issues.apache.org/jira/browse/KAFKA-2327>:
> > > > broker doesn't start if config defines advertised.host but not
> > > > advertised.port
> > > > - KAFKA-1788: producer record can stay in RecordAccumulator
> forever
> > if
> > > > leader is no available
> > > > - KAFKA-2234 <https://issues.apache.org/jira/browse/KAFKA-2234>:
> > > > Partition reassignment of a nonexistent topic prevents future
> > > reassignments
> > > > - KAFKA-2096 <https://issues.apache.org/jira/browse/KAFKA-2096>:
> > > > Enable keepalive socket option for broker to prevent socket leak
> > > > - KAFKA-1057 <https://issues.apache.org/jira/browse/KAFKA-1057>:
> > Trim
> > > > whitespaces from user specified configs
> > > > - KAFKA-1641 <https://issues.apache.org/jira/browse/KAFKA-1641>:
> > Log
> > > > cleaner exits if last cleaned offset is lower than earliest offset
> > > > - KAFKA-1648 <https://issues.apache.org/jira/browse/KAFKA-1648>:
> > > Round
> > > > robin consumer balance throws an NPE when there are no topics
> > > > - KAFKA-1724 <https://issues.apache.org/jira/browse/KAFKA-1724>:
> > > > Errors after reboot in single node setup
> > > > - KAFKA-1758 <https://issues.apache.org/jira/browse/KAFKA-1758>:
> > > > corrupt recovery file prevents startup
> > > > - KAFKA-1866 <https://issues.apache.org/jira/browse/KAFKA-1866>:
> > > > LogStartOffset gauge throws exceptions after log.delete()
> > > > - KAFKA-1883 <https://issues.apache.org/jira/browse/KAFKA-1883>:
> > > NullPointerException
> > > > in RequestSendThread
> > > > - KAFKA-1896 <https://issues.apache.org/jira/browse/KAFKA-1896>:
> > > > Record size funcition of record in mirror maker hit NPE when the
> > > message
> > > > value is null.
> > > > - KAFKA-2101 <https://issues.apache.org/jira/browse/KAFKA-2101>:
> > > > Metric metadata-age is reset on a failed update
> > > > - KAFKA-2112 <https://issues.apache.org/jira/browse/KAFKA-2112>:
> > make
> > > > overflowWheel volatile
> > > > - KAFKA-2117 <https://issues.apache.org/jira/browse/KAFKA-2117>:
> > > > OffsetManager uses incorrect field for metadata
> > > > - KAFKA-2164 <https://issues.apache.org/jira/browse/KAFKA-2164>:
> > > > ReplicaFetcherThread: suspicious log message on reset offset
> > > > - KAFKA-1668 <https://issues.apache.org/jira/browse/KAFKA-1668>:
> > > > TopicCommand doesn't warn if --topic argument doesn't match any
> > topics
> > > > - KAFKA-2198 <https://issues.apache.org/jira/browse/KAFKA-2198>:
> > > > kafka-topics.sh exits with 0 status on failures
> > > > - KAFKA-2235 <https://issues.apache.org/jira/browse/KAFKA-2235>:
> > > > LogCleaner offset map overflow
> > > > - KAFKA-2241 <https://issues.apache.org/jira/browse/KAFKA-2241>:
> > > > AbstractFetcherThread.shutdown() should not block on
> > > > ReadableByteChannel.read(buffer)
> > > > - KAFKA-2272 <https://issues.apache.org/jira/browse/KAFKA-2272>:
> > > > listeners endpoint parsing fails if the hostname has capital
> letter
> > > > - KAFKA-2345 <https://issues.apache.org/jira/browse/KAFKA-2345>:
> > > > Attempt to delete a topic already marked for deletion throws
> > > > ZkNodeExistsException
> > > > - KAFKA-2353 <https://issues.apache.org/jira/browse/KAFKA-2353>:
> > > > SocketServer.Processor should catch exception and close the socket
> > > properly
> > > > in configureNewConnections.
> > > > - KAFKA-1836 <https://issues.apache.org/jira/browse/KAFKA-1836>:
> > > > metadata.fetch.timeout.ms set to zero blocks forever
> > > > - KAFKA-2317 <https://issues.apache.org/jira/browse/KAFKA-2317>:
> > > De-register
> > > > isrChangeNotificationListener on controller resignation
> > > >
> > > > Note: KAFKA-2120 <https://issues.apache.org/jira/browse/KAFKA-2120>
> &
> > > > KAFKA-2421 <https://issues.apache.org/jira/browse/KAFKA-2421> were
> > > > mentioned in previous emails, but are not in the list because they
> are
> > > not
> > > > committed yet.
> > > >
> > > > Hope that helps the effort.
> > > >
> > > > Thanks,
> > > > Grant
> > > >
> > > > On Mon, Aug 17, 2015 at 12:09 AM, Grant Henke <gh...@cloudera.com>
> > > wrote:
> > > >
> > > >> +1 to that suggestion. Though I suspect that requires a committer to
> > do.
> > > >> Making it part of the standard commit process could work too.
> > > >> On Aug 16, 2015 11:01 PM, "Gwen Shapira" <gw...@confluent.io> wrote:
> > > >>
> > > >>> BTW. I think it will be great for Apache Kafka to have a 0.8.2
> > "release
> > > >>> manager" who's role is to cherrypick low-risk bug-fixes into the
> > 0.8.2
> > > >>> branch and once enough bug fixes happened (or if sufficiently
> > critical
> > > >>> fixes happened) to roll out a new maintenance release (with every 3
> > > month
> > > >>> as a reasonable bugfix release target).
> > > >>>
> > > >>> This will add some predictability regarding how fast we release
> fixes
> > > for
> > > >>> bugs.
> > > >>>
> > > >>> Gwen
> > > >>>
> > > >>> On Sun, Aug 16, 2015 at 8:09 PM, Jeff Holoman <
> jholoman@cloudera.com
> > >
> > > >>> wrote:
> > > >>>
> > > >>> > +1 for the release and also including
> > > >>> >
> > > >>> > https://issues.apache.org/jira/browse/KAFKA-2114
> > > >>> >
> > > >>> > Thanks
> > > >>> >
> > > >>> > Jeff
> > > >>> >
> > > >>> > On Sun, Aug 16, 2015 at 2:51 PM, Stevo Slavić <sslavic@gmail.com
> >
> > > >>> wrote:
> > > >>> >
> > > >>> > > +1 (non-binding) for 0.8.2.2 release
> > > >>> > >
> > > >>> > > Would be nice to include in that release new producer
> resiliency
> > > bug
> > > >>> > fixes
> > > >>> > > https://issues.apache.org/jira/browse/KAFKA-1788 and
> > > >>> > > https://issues.apache.org/jira/browse/KAFKA-2120
> > > >>> > >
> > > >>> > > On Fri, Aug 14, 2015 at 4:03 PM, Gwen Shapira <
> gwen@confluent.io
> > >
> > > >>> wrote:
> > > >>> > >
> > > >>> > > > Will be nice to include Kafka-2308 and fix two critical
> snappy
> > > >>> issues
> > > >>> > in
> > > >>> > > > the maintenance release.
> > > >>> > > >
> > > >>> > > > Gwen
> > > >>> > > > On Aug 14, 2015 6:16 AM, "Grant Henke" <gh...@cloudera.com>
> > > >>> wrote:
> > > >>> > > >
> > > >>> > > > > Just to clarify. Will KAFKA-2189 be the only patch in the
> > > >>> release?
> > > >>> > > > >
> > > >>> > > > > On Fri, Aug 14, 2015 at 7:35 AM, Manikumar Reddy <
> > > >>> > kumar@nmsworks.co.in
> > > >>> > > >
> > > >>> > > > > wrote:
> > > >>> > > > >
> > > >>> > > > > > +1 for 0.8.2.2 release
> > > >>> > > > > >
> > > >>> > > > > > On Fri, Aug 14, 2015 at 5:49 PM, Ismael Juma <
> > > >>> ismael@juma.me.uk>
> > > >>> > > > wrote:
> > > >>> > > > > >
> > > >>> > > > > > > I think this is a good idea as the change is minimal on
> > our
> > > >>> side
> > > >>> > > and
> > > >>> > > > it
> > > >>> > > > > > has
> > > >>> > > > > > > been tested in production for some time by the
> reporter.
> > > >>> > > > > > >
> > > >>> > > > > > > Best,
> > > >>> > > > > > > Ismael
> > > >>> > > > > > >
> > > >>> > > > > > > On Fri, Aug 14, 2015 at 1:15 PM, Jun Rao <
> > jun@confluent.io
> > > >
> > > >>> > wrote:
> > > >>> > > > > > >
> > > >>> > > > > > > > Hi, Everyone,
> > > >>> > > > > > > >
> > > >>> > > > > > > > Since the release of Kafka 0.8.2.1, a number of
> people
> > > have
> > > >>> > > > reported
> > > >>> > > > > an
> > > >>> > > > > > > > issue with snappy compression (
> > > >>> > > > > > > > https://issues.apache.org/jira/browse/KAFKA-2189).
> > > >>> Basically,
> > > >>> > if
> > > >>> > > > > they
> > > >>> > > > > > > use
> > > >>> > > > > > > > snappy in 0.8.2.1, they will experience a 2-3X space
> > > >>> increase.
> > > >>> > > The
> > > >>> > > > > > issue
> > > >>> > > > > > > > has since been fixed in trunk (just a snappy jar
> > > upgrade).
> > > >>> > Since
> > > >>> > > > > 0.8.3
> > > >>> > > > > > is
> > > >>> > > > > > > > still a few months away, it may make sense to do an
> > > 0.8.2.2
> > > >>> > > release
> > > >>> > > > > > just
> > > >>> > > > > > > to
> > > >>> > > > > > > > fix this issue. Any objections?
> > > >>> > > > > > > >
> > > >>> > > > > > > > Thanks,
> > > >>> > > > > > > >
> > > >>> > > > > > > > Jun
> > > >>> > > > > > > >
> > > >>> > > > > > >
> > > >>> > > > > >
> > > >>> > > > >
> > > >>> > > > >
> > > >>> > > > >
> > > >>> > > > > --
> > > >>> > > > > Grant Henke
> > > >>> > > > > Software Engineer | Cloudera
> > > >>> > > > > grant@cloudera.com | twitter.com/gchenke |
> > > >>> > linkedin.com/in/granthenke
> > > >>> > > > >
> > > >>> > > >
> > > >>> > >
> > > >>> >
> > > >>> >
> > > >>> >
> > > >>> > --
> > > >>> > Jeff Holoman
> > > >>> > Systems Engineer
> > > >>> >
> > > >>>
> > > >>
> > > >
> > > >
> > > > --
> > > > Grant Henke
> > > > Software Engineer | Cloudera
> > > > grant@cloudera.com | twitter.com/gchenke |
> linkedin.com/in/granthenke
> > > >
> > >
> > >
> > >
> > > --
> > > Grant Henke
> > > Software Engineer | Cloudera
> > > grant@cloudera.com | twitter.com/gchenke | linkedin.com/in/granthenke
> > >
> >
>
Re: [DISCUSSION] Kafka 0.8.2.2 release?
Posted by Gwen Shapira <gw...@confluent.io>.
Jun,
KAFKA-2147 doesn't seem to have a commit associated with it, so I can't
cherrypick just this fix.
I suggest leaving this out since there is a 0.8.2.x workaround in the JIRA.
Gwen
On Mon, Aug 17, 2015 at 5:24 PM, Jun Rao <ju...@confluent.io> wrote:
> Gwen,
>
> Thanks for putting the list together.
>
> I'd recommend that we exclude the following:
> KAFKA-1702: This is for the old producer and is only a problem if there are
> some unexpected exceptions (e.g. UnknownClass).
> KAFKA-2336: Most people don't change offsets.topic.num.partitions.
> KAFKA-1724: The patch there is never committed since the fix is included in
> another jira (a much larger patch).
> KAFKA-2241: This doesn't seem be a common problem. It only happens when the
> fetch request blocks on the broker for an extended period of time, which
> should be rare.
>
> I'd also recommend that we include the following:
> KAFKA-2147: This impacts the memory size of the purgatory and a number of
> people have experienced that. The fix is small and has been tested in
> production usage. It hasn't been committed though since the issue is
> already fixed in trunk and we weren't planning for an 0.8.2.2 release then.
>
> Thanks,
>
> Jun
>
> On Mon, Aug 17, 2015 at 2:56 PM, Gwen Shapira <gw...@confluent.io> wrote:
>
> > Thanks for creating a list, Grant!
> >
> > I placed it on the wiki with a quick evaluation of the content and
> whether
> > it should be in 0.8.2.2:
> >
> >
> https://cwiki.apache.org/confluence/display/KAFKA/Proposed+patches+for+0.8.2.2
> >
> > I'm attempting to only cherrypick fixes that are both important for large
> > number of users (or very critical to some users) and very safe (mostly
> > judged by the size of the change, but not only)
> >
> > If your favorite bugfix is missing from the list, or is there but marked
> > "No", please let us know (in this thread) what we are missing and why it
> is
> > both important and safe.
> > Also, if I accidentally included something you consider unsafe, speak up!
> >
> > Gwen
> >
> > On Mon, Aug 17, 2015 at 8:17 AM, Grant Henke <gh...@cloudera.com>
> wrote:
> >
> > > +dev
> > >
> > > Adding dev list back in. Somehow it got dropped.
> > >
> > >
> > > On Mon, Aug 17, 2015 at 10:16 AM, Grant Henke <gh...@cloudera.com>
> > wrote:
> > >
> > > > Below is a list of candidate bug fix jiras marked fixed for 0.8.3. I
> > > don't
> > > > suspect all of these will (or should) make it into the release but
> this
> > > > should be a relatively complete list to work from:
> > > >
> > > > - KAFKA-2114 <https://issues.apache.org/jira/browse/KAFKA-2114>:
> > > Unable
> > > > to change min.insync.replicas default
> > > > - KAFKA-1702 <https://issues.apache.org/jira/browse/KAFKA-1702>:
> > > > Messages silently Lost by producer
> > > > - KAFKA-2012 <https://issues.apache.org/jira/browse/KAFKA-2012>:
> > > > Broker should automatically handle corrupt index files
> > > > - KAFKA-2406 <https://issues.apache.org/jira/browse/KAFKA-2406>:
> > ISR
> > > > propagation should be throttled to avoid overwhelming controller.
> > > > - KAFKA-2336 <https://issues.apache.org/jira/browse/KAFKA-2336>:
> > > > Changing offsets.topic.num.partitions after the offset topic is
> > > created
> > > > breaks consumer group partition assignment
> > > > - KAFKA-2337 <https://issues.apache.org/jira/browse/KAFKA-2337>:
> > > Verify
> > > > that metric names will not collide when creating new topics
> > > > - KAFKA-2393 <https://issues.apache.org/jira/browse/KAFKA-2393>:
> > > > Correctly Handle InvalidTopicException in
> > KafkaApis.getTopicMetadata()
> > > > - KAFKA-2189 <https://issues.apache.org/jira/browse/KAFKA-2189>:
> > > Snappy
> > > > compression of message batches less efficient in 0.8.2.1
> > > > - KAFKA-2308 <https://issues.apache.org/jira/browse/KAFKA-2308>:
> > New
> > > > producer + Snappy face un-compression errors after broker restart
> > > > - KAFKA-2042 <https://issues.apache.org/jira/browse/KAFKA-2042>:
> > New
> > > > producer metadata update always get all topics.
> > > > - KAFKA-1367 <https://issues.apache.org/jira/browse/KAFKA-1367>:
> > > Broker
> > > > topic metadata not kept in sync with ZooKeeper
> > > > - KAFKA-972 <https://issues.apache.org/jira/browse/KAFKA-972>:
> > > MetadataRequest
> > > > returns stale list of brokers
> > > > - KAFKA-1867 <https://issues.apache.org/jira/browse/KAFKA-1867>:
> > > liveBroker
> > > > list not updated on a cluster with no topics
> > > > - KAFKA-1650 <https://issues.apache.org/jira/browse/KAFKA-1650>:
> > > Mirror
> > > > Maker could lose data on unclean shutdown.
> > > > - KAFKA-2009 <https://issues.apache.org/jira/browse/KAFKA-2009>:
> > Fix
> > > > UncheckedOffset.removeOffset synchronization and trace logging
> issue
> > > in
> > > > mirror maker
> > > > - KAFKA-2407 <https://issues.apache.org/jira/browse/KAFKA-2407>:
> > Only
> > > > create a log directory when it will be used
> > > > - KAFKA-2327 <https://issues.apache.org/jira/browse/KAFKA-2327>:
> > > > broker doesn't start if config defines advertised.host but not
> > > > advertised.port
> > > > - KAFKA-1788: producer record can stay in RecordAccumulator
> forever
> > if
> > > > leader is no available
> > > > - KAFKA-2234 <https://issues.apache.org/jira/browse/KAFKA-2234>:
> > > > Partition reassignment of a nonexistent topic prevents future
> > > reassignments
> > > > - KAFKA-2096 <https://issues.apache.org/jira/browse/KAFKA-2096>:
> > > > Enable keepalive socket option for broker to prevent socket leak
> > > > - KAFKA-1057 <https://issues.apache.org/jira/browse/KAFKA-1057>:
> > Trim
> > > > whitespaces from user specified configs
> > > > - KAFKA-1641 <https://issues.apache.org/jira/browse/KAFKA-1641>:
> > Log
> > > > cleaner exits if last cleaned offset is lower than earliest offset
> > > > - KAFKA-1648 <https://issues.apache.org/jira/browse/KAFKA-1648>:
> > > Round
> > > > robin consumer balance throws an NPE when there are no topics
> > > > - KAFKA-1724 <https://issues.apache.org/jira/browse/KAFKA-1724>:
> > > > Errors after reboot in single node setup
> > > > - KAFKA-1758 <https://issues.apache.org/jira/browse/KAFKA-1758>:
> > > > corrupt recovery file prevents startup
> > > > - KAFKA-1866 <https://issues.apache.org/jira/browse/KAFKA-1866>:
> > > > LogStartOffset gauge throws exceptions after log.delete()
> > > > - KAFKA-1883 <https://issues.apache.org/jira/browse/KAFKA-1883>:
> > > NullPointerException
> > > > in RequestSendThread
> > > > - KAFKA-1896 <https://issues.apache.org/jira/browse/KAFKA-1896>:
> > > > Record size funcition of record in mirror maker hit NPE when the
> > > message
> > > > value is null.
> > > > - KAFKA-2101 <https://issues.apache.org/jira/browse/KAFKA-2101>:
> > > > Metric metadata-age is reset on a failed update
> > > > - KAFKA-2112 <https://issues.apache.org/jira/browse/KAFKA-2112>:
> > make
> > > > overflowWheel volatile
> > > > - KAFKA-2117 <https://issues.apache.org/jira/browse/KAFKA-2117>:
> > > > OffsetManager uses incorrect field for metadata
> > > > - KAFKA-2164 <https://issues.apache.org/jira/browse/KAFKA-2164>:
> > > > ReplicaFetcherThread: suspicious log message on reset offset
> > > > - KAFKA-1668 <https://issues.apache.org/jira/browse/KAFKA-1668>:
> > > > TopicCommand doesn't warn if --topic argument doesn't match any
> > topics
> > > > - KAFKA-2198 <https://issues.apache.org/jira/browse/KAFKA-2198>:
> > > > kafka-topics.sh exits with 0 status on failures
> > > > - KAFKA-2235 <https://issues.apache.org/jira/browse/KAFKA-2235>:
> > > > LogCleaner offset map overflow
> > > > - KAFKA-2241 <https://issues.apache.org/jira/browse/KAFKA-2241>:
> > > > AbstractFetcherThread.shutdown() should not block on
> > > > ReadableByteChannel.read(buffer)
> > > > - KAFKA-2272 <https://issues.apache.org/jira/browse/KAFKA-2272>:
> > > > listeners endpoint parsing fails if the hostname has capital
> letter
> > > > - KAFKA-2345 <https://issues.apache.org/jira/browse/KAFKA-2345>:
> > > > Attempt to delete a topic already marked for deletion throws
> > > > ZkNodeExistsException
> > > > - KAFKA-2353 <https://issues.apache.org/jira/browse/KAFKA-2353>:
> > > > SocketServer.Processor should catch exception and close the socket
> > > properly
> > > > in configureNewConnections.
> > > > - KAFKA-1836 <https://issues.apache.org/jira/browse/KAFKA-1836>:
> > > > metadata.fetch.timeout.ms set to zero blocks forever
> > > > - KAFKA-2317 <https://issues.apache.org/jira/browse/KAFKA-2317>:
> > > De-register
> > > > isrChangeNotificationListener on controller resignation
> > > >
> > > > Note: KAFKA-2120 <https://issues.apache.org/jira/browse/KAFKA-2120>
> &
> > > > KAFKA-2421 <https://issues.apache.org/jira/browse/KAFKA-2421> were
> > > > mentioned in previous emails, but are not in the list because they
> are
> > > not
> > > > committed yet.
> > > >
> > > > Hope that helps the effort.
> > > >
> > > > Thanks,
> > > > Grant
> > > >
> > > > On Mon, Aug 17, 2015 at 12:09 AM, Grant Henke <gh...@cloudera.com>
> > > wrote:
> > > >
> > > >> +1 to that suggestion. Though I suspect that requires a committer to
> > do.
> > > >> Making it part of the standard commit process could work too.
> > > >> On Aug 16, 2015 11:01 PM, "Gwen Shapira" <gw...@confluent.io> wrote:
> > > >>
> > > >>> BTW. I think it will be great for Apache Kafka to have a 0.8.2
> > "release
> > > >>> manager" who's role is to cherrypick low-risk bug-fixes into the
> > 0.8.2
> > > >>> branch and once enough bug fixes happened (or if sufficiently
> > critical
> > > >>> fixes happened) to roll out a new maintenance release (with every 3
> > > month
> > > >>> as a reasonable bugfix release target).
> > > >>>
> > > >>> This will add some predictability regarding how fast we release
> fixes
> > > for
> > > >>> bugs.
> > > >>>
> > > >>> Gwen
> > > >>>
> > > >>> On Sun, Aug 16, 2015 at 8:09 PM, Jeff Holoman <
> jholoman@cloudera.com
> > >
> > > >>> wrote:
> > > >>>
> > > >>> > +1 for the release and also including
> > > >>> >
> > > >>> > https://issues.apache.org/jira/browse/KAFKA-2114
> > > >>> >
> > > >>> > Thanks
> > > >>> >
> > > >>> > Jeff
> > > >>> >
> > > >>> > On Sun, Aug 16, 2015 at 2:51 PM, Stevo Slavić <sslavic@gmail.com
> >
> > > >>> wrote:
> > > >>> >
> > > >>> > > +1 (non-binding) for 0.8.2.2 release
> > > >>> > >
> > > >>> > > Would be nice to include in that release new producer
> resiliency
> > > bug
> > > >>> > fixes
> > > >>> > > https://issues.apache.org/jira/browse/KAFKA-1788 and
> > > >>> > > https://issues.apache.org/jira/browse/KAFKA-2120
> > > >>> > >
> > > >>> > > On Fri, Aug 14, 2015 at 4:03 PM, Gwen Shapira <
> gwen@confluent.io
> > >
> > > >>> wrote:
> > > >>> > >
> > > >>> > > > Will be nice to include Kafka-2308 and fix two critical
> snappy
> > > >>> issues
> > > >>> > in
> > > >>> > > > the maintenance release.
> > > >>> > > >
> > > >>> > > > Gwen
> > > >>> > > > On Aug 14, 2015 6:16 AM, "Grant Henke" <gh...@cloudera.com>
> > > >>> wrote:
> > > >>> > > >
> > > >>> > > > > Just to clarify. Will KAFKA-2189 be the only patch in the
> > > >>> release?
> > > >>> > > > >
> > > >>> > > > > On Fri, Aug 14, 2015 at 7:35 AM, Manikumar Reddy <
> > > >>> > kumar@nmsworks.co.in
> > > >>> > > >
> > > >>> > > > > wrote:
> > > >>> > > > >
> > > >>> > > > > > +1 for 0.8.2.2 release
> > > >>> > > > > >
> > > >>> > > > > > On Fri, Aug 14, 2015 at 5:49 PM, Ismael Juma <
> > > >>> ismael@juma.me.uk>
> > > >>> > > > wrote:
> > > >>> > > > > >
> > > >>> > > > > > > I think this is a good idea as the change is minimal on
> > our
> > > >>> side
> > > >>> > > and
> > > >>> > > > it
> > > >>> > > > > > has
> > > >>> > > > > > > been tested in production for some time by the
> reporter.
> > > >>> > > > > > >
> > > >>> > > > > > > Best,
> > > >>> > > > > > > Ismael
> > > >>> > > > > > >
> > > >>> > > > > > > On Fri, Aug 14, 2015 at 1:15 PM, Jun Rao <
> > jun@confluent.io
> > > >
> > > >>> > wrote:
> > > >>> > > > > > >
> > > >>> > > > > > > > Hi, Everyone,
> > > >>> > > > > > > >
> > > >>> > > > > > > > Since the release of Kafka 0.8.2.1, a number of
> people
> > > have
> > > >>> > > > reported
> > > >>> > > > > an
> > > >>> > > > > > > > issue with snappy compression (
> > > >>> > > > > > > > https://issues.apache.org/jira/browse/KAFKA-2189).
> > > >>> Basically,
> > > >>> > if
> > > >>> > > > > they
> > > >>> > > > > > > use
> > > >>> > > > > > > > snappy in 0.8.2.1, they will experience a 2-3X space
> > > >>> increase.
> > > >>> > > The
> > > >>> > > > > > issue
> > > >>> > > > > > > > has since been fixed in trunk (just a snappy jar
> > > upgrade).
> > > >>> > Since
> > > >>> > > > > 0.8.3
> > > >>> > > > > > is
> > > >>> > > > > > > > still a few months away, it may make sense to do an
> > > 0.8.2.2
> > > >>> > > release
> > > >>> > > > > > just
> > > >>> > > > > > > to
> > > >>> > > > > > > > fix this issue. Any objections?
> > > >>> > > > > > > >
> > > >>> > > > > > > > Thanks,
> > > >>> > > > > > > >
> > > >>> > > > > > > > Jun
> > > >>> > > > > > > >
> > > >>> > > > > > >
> > > >>> > > > > >
> > > >>> > > > >
> > > >>> > > > >
> > > >>> > > > >
> > > >>> > > > > --
> > > >>> > > > > Grant Henke
> > > >>> > > > > Software Engineer | Cloudera
> > > >>> > > > > grant@cloudera.com | twitter.com/gchenke |
> > > >>> > linkedin.com/in/granthenke
> > > >>> > > > >
> > > >>> > > >
> > > >>> > >
> > > >>> >
> > > >>> >
> > > >>> >
> > > >>> > --
> > > >>> > Jeff Holoman
> > > >>> > Systems Engineer
> > > >>> >
> > > >>>
> > > >>
> > > >
> > > >
> > > > --
> > > > Grant Henke
> > > > Software Engineer | Cloudera
> > > > grant@cloudera.com | twitter.com/gchenke |
> linkedin.com/in/granthenke
> > > >
> > >
> > >
> > >
> > > --
> > > Grant Henke
> > > Software Engineer | Cloudera
> > > grant@cloudera.com | twitter.com/gchenke | linkedin.com/in/granthenke
> > >
> >
>
Re: [DISCUSSION] Kafka 0.8.2.2 release?
Posted by Jun Rao <ju...@confluent.io>.
Gwen,
Thanks for putting the list together.
I'd recommend that we exclude the following:
KAFKA-1702: This is for the old producer and is only a problem if there are
some unexpected exceptions (e.g. UnknownClass).
KAFKA-2336: Most people don't change offsets.topic.num.partitions.
KAFKA-1724: The patch there is never committed since the fix is included in
another jira (a much larger patch).
KAFKA-2241: This doesn't seem be a common problem. It only happens when the
fetch request blocks on the broker for an extended period of time, which
should be rare.
I'd also recommend that we include the following:
KAFKA-2147: This impacts the memory size of the purgatory and a number of
people have experienced that. The fix is small and has been tested in
production usage. It hasn't been committed though since the issue is
already fixed in trunk and we weren't planning for an 0.8.2.2 release then.
Thanks,
Jun
On Mon, Aug 17, 2015 at 2:56 PM, Gwen Shapira <gw...@confluent.io> wrote:
> Thanks for creating a list, Grant!
>
> I placed it on the wiki with a quick evaluation of the content and whether
> it should be in 0.8.2.2:
>
> https://cwiki.apache.org/confluence/display/KAFKA/Proposed+patches+for+0.8.2.2
>
> I'm attempting to only cherrypick fixes that are both important for large
> number of users (or very critical to some users) and very safe (mostly
> judged by the size of the change, but not only)
>
> If your favorite bugfix is missing from the list, or is there but marked
> "No", please let us know (in this thread) what we are missing and why it is
> both important and safe.
> Also, if I accidentally included something you consider unsafe, speak up!
>
> Gwen
>
> On Mon, Aug 17, 2015 at 8:17 AM, Grant Henke <gh...@cloudera.com> wrote:
>
> > +dev
> >
> > Adding dev list back in. Somehow it got dropped.
> >
> >
> > On Mon, Aug 17, 2015 at 10:16 AM, Grant Henke <gh...@cloudera.com>
> wrote:
> >
> > > Below is a list of candidate bug fix jiras marked fixed for 0.8.3. I
> > don't
> > > suspect all of these will (or should) make it into the release but this
> > > should be a relatively complete list to work from:
> > >
> > > - KAFKA-2114 <https://issues.apache.org/jira/browse/KAFKA-2114>:
> > Unable
> > > to change min.insync.replicas default
> > > - KAFKA-1702 <https://issues.apache.org/jira/browse/KAFKA-1702>:
> > > Messages silently Lost by producer
> > > - KAFKA-2012 <https://issues.apache.org/jira/browse/KAFKA-2012>:
> > > Broker should automatically handle corrupt index files
> > > - KAFKA-2406 <https://issues.apache.org/jira/browse/KAFKA-2406>:
> ISR
> > > propagation should be throttled to avoid overwhelming controller.
> > > - KAFKA-2336 <https://issues.apache.org/jira/browse/KAFKA-2336>:
> > > Changing offsets.topic.num.partitions after the offset topic is
> > created
> > > breaks consumer group partition assignment
> > > - KAFKA-2337 <https://issues.apache.org/jira/browse/KAFKA-2337>:
> > Verify
> > > that metric names will not collide when creating new topics
> > > - KAFKA-2393 <https://issues.apache.org/jira/browse/KAFKA-2393>:
> > > Correctly Handle InvalidTopicException in
> KafkaApis.getTopicMetadata()
> > > - KAFKA-2189 <https://issues.apache.org/jira/browse/KAFKA-2189>:
> > Snappy
> > > compression of message batches less efficient in 0.8.2.1
> > > - KAFKA-2308 <https://issues.apache.org/jira/browse/KAFKA-2308>:
> New
> > > producer + Snappy face un-compression errors after broker restart
> > > - KAFKA-2042 <https://issues.apache.org/jira/browse/KAFKA-2042>:
> New
> > > producer metadata update always get all topics.
> > > - KAFKA-1367 <https://issues.apache.org/jira/browse/KAFKA-1367>:
> > Broker
> > > topic metadata not kept in sync with ZooKeeper
> > > - KAFKA-972 <https://issues.apache.org/jira/browse/KAFKA-972>:
> > MetadataRequest
> > > returns stale list of brokers
> > > - KAFKA-1867 <https://issues.apache.org/jira/browse/KAFKA-1867>:
> > liveBroker
> > > list not updated on a cluster with no topics
> > > - KAFKA-1650 <https://issues.apache.org/jira/browse/KAFKA-1650>:
> > Mirror
> > > Maker could lose data on unclean shutdown.
> > > - KAFKA-2009 <https://issues.apache.org/jira/browse/KAFKA-2009>:
> Fix
> > > UncheckedOffset.removeOffset synchronization and trace logging issue
> > in
> > > mirror maker
> > > - KAFKA-2407 <https://issues.apache.org/jira/browse/KAFKA-2407>:
> Only
> > > create a log directory when it will be used
> > > - KAFKA-2327 <https://issues.apache.org/jira/browse/KAFKA-2327>:
> > > broker doesn't start if config defines advertised.host but not
> > > advertised.port
> > > - KAFKA-1788: producer record can stay in RecordAccumulator forever
> if
> > > leader is no available
> > > - KAFKA-2234 <https://issues.apache.org/jira/browse/KAFKA-2234>:
> > > Partition reassignment of a nonexistent topic prevents future
> > reassignments
> > > - KAFKA-2096 <https://issues.apache.org/jira/browse/KAFKA-2096>:
> > > Enable keepalive socket option for broker to prevent socket leak
> > > - KAFKA-1057 <https://issues.apache.org/jira/browse/KAFKA-1057>:
> Trim
> > > whitespaces from user specified configs
> > > - KAFKA-1641 <https://issues.apache.org/jira/browse/KAFKA-1641>:
> Log
> > > cleaner exits if last cleaned offset is lower than earliest offset
> > > - KAFKA-1648 <https://issues.apache.org/jira/browse/KAFKA-1648>:
> > Round
> > > robin consumer balance throws an NPE when there are no topics
> > > - KAFKA-1724 <https://issues.apache.org/jira/browse/KAFKA-1724>:
> > > Errors after reboot in single node setup
> > > - KAFKA-1758 <https://issues.apache.org/jira/browse/KAFKA-1758>:
> > > corrupt recovery file prevents startup
> > > - KAFKA-1866 <https://issues.apache.org/jira/browse/KAFKA-1866>:
> > > LogStartOffset gauge throws exceptions after log.delete()
> > > - KAFKA-1883 <https://issues.apache.org/jira/browse/KAFKA-1883>:
> > NullPointerException
> > > in RequestSendThread
> > > - KAFKA-1896 <https://issues.apache.org/jira/browse/KAFKA-1896>:
> > > Record size funcition of record in mirror maker hit NPE when the
> > message
> > > value is null.
> > > - KAFKA-2101 <https://issues.apache.org/jira/browse/KAFKA-2101>:
> > > Metric metadata-age is reset on a failed update
> > > - KAFKA-2112 <https://issues.apache.org/jira/browse/KAFKA-2112>:
> make
> > > overflowWheel volatile
> > > - KAFKA-2117 <https://issues.apache.org/jira/browse/KAFKA-2117>:
> > > OffsetManager uses incorrect field for metadata
> > > - KAFKA-2164 <https://issues.apache.org/jira/browse/KAFKA-2164>:
> > > ReplicaFetcherThread: suspicious log message on reset offset
> > > - KAFKA-1668 <https://issues.apache.org/jira/browse/KAFKA-1668>:
> > > TopicCommand doesn't warn if --topic argument doesn't match any
> topics
> > > - KAFKA-2198 <https://issues.apache.org/jira/browse/KAFKA-2198>:
> > > kafka-topics.sh exits with 0 status on failures
> > > - KAFKA-2235 <https://issues.apache.org/jira/browse/KAFKA-2235>:
> > > LogCleaner offset map overflow
> > > - KAFKA-2241 <https://issues.apache.org/jira/browse/KAFKA-2241>:
> > > AbstractFetcherThread.shutdown() should not block on
> > > ReadableByteChannel.read(buffer)
> > > - KAFKA-2272 <https://issues.apache.org/jira/browse/KAFKA-2272>:
> > > listeners endpoint parsing fails if the hostname has capital letter
> > > - KAFKA-2345 <https://issues.apache.org/jira/browse/KAFKA-2345>:
> > > Attempt to delete a topic already marked for deletion throws
> > > ZkNodeExistsException
> > > - KAFKA-2353 <https://issues.apache.org/jira/browse/KAFKA-2353>:
> > > SocketServer.Processor should catch exception and close the socket
> > properly
> > > in configureNewConnections.
> > > - KAFKA-1836 <https://issues.apache.org/jira/browse/KAFKA-1836>:
> > > metadata.fetch.timeout.ms set to zero blocks forever
> > > - KAFKA-2317 <https://issues.apache.org/jira/browse/KAFKA-2317>:
> > De-register
> > > isrChangeNotificationListener on controller resignation
> > >
> > > Note: KAFKA-2120 <https://issues.apache.org/jira/browse/KAFKA-2120> &
> > > KAFKA-2421 <https://issues.apache.org/jira/browse/KAFKA-2421> were
> > > mentioned in previous emails, but are not in the list because they are
> > not
> > > committed yet.
> > >
> > > Hope that helps the effort.
> > >
> > > Thanks,
> > > Grant
> > >
> > > On Mon, Aug 17, 2015 at 12:09 AM, Grant Henke <gh...@cloudera.com>
> > wrote:
> > >
> > >> +1 to that suggestion. Though I suspect that requires a committer to
> do.
> > >> Making it part of the standard commit process could work too.
> > >> On Aug 16, 2015 11:01 PM, "Gwen Shapira" <gw...@confluent.io> wrote:
> > >>
> > >>> BTW. I think it will be great for Apache Kafka to have a 0.8.2
> "release
> > >>> manager" who's role is to cherrypick low-risk bug-fixes into the
> 0.8.2
> > >>> branch and once enough bug fixes happened (or if sufficiently
> critical
> > >>> fixes happened) to roll out a new maintenance release (with every 3
> > month
> > >>> as a reasonable bugfix release target).
> > >>>
> > >>> This will add some predictability regarding how fast we release fixes
> > for
> > >>> bugs.
> > >>>
> > >>> Gwen
> > >>>
> > >>> On Sun, Aug 16, 2015 at 8:09 PM, Jeff Holoman <jholoman@cloudera.com
> >
> > >>> wrote:
> > >>>
> > >>> > +1 for the release and also including
> > >>> >
> > >>> > https://issues.apache.org/jira/browse/KAFKA-2114
> > >>> >
> > >>> > Thanks
> > >>> >
> > >>> > Jeff
> > >>> >
> > >>> > On Sun, Aug 16, 2015 at 2:51 PM, Stevo Slavić <ss...@gmail.com>
> > >>> wrote:
> > >>> >
> > >>> > > +1 (non-binding) for 0.8.2.2 release
> > >>> > >
> > >>> > > Would be nice to include in that release new producer resiliency
> > bug
> > >>> > fixes
> > >>> > > https://issues.apache.org/jira/browse/KAFKA-1788 and
> > >>> > > https://issues.apache.org/jira/browse/KAFKA-2120
> > >>> > >
> > >>> > > On Fri, Aug 14, 2015 at 4:03 PM, Gwen Shapira <gwen@confluent.io
> >
> > >>> wrote:
> > >>> > >
> > >>> > > > Will be nice to include Kafka-2308 and fix two critical snappy
> > >>> issues
> > >>> > in
> > >>> > > > the maintenance release.
> > >>> > > >
> > >>> > > > Gwen
> > >>> > > > On Aug 14, 2015 6:16 AM, "Grant Henke" <gh...@cloudera.com>
> > >>> wrote:
> > >>> > > >
> > >>> > > > > Just to clarify. Will KAFKA-2189 be the only patch in the
> > >>> release?
> > >>> > > > >
> > >>> > > > > On Fri, Aug 14, 2015 at 7:35 AM, Manikumar Reddy <
> > >>> > kumar@nmsworks.co.in
> > >>> > > >
> > >>> > > > > wrote:
> > >>> > > > >
> > >>> > > > > > +1 for 0.8.2.2 release
> > >>> > > > > >
> > >>> > > > > > On Fri, Aug 14, 2015 at 5:49 PM, Ismael Juma <
> > >>> ismael@juma.me.uk>
> > >>> > > > wrote:
> > >>> > > > > >
> > >>> > > > > > > I think this is a good idea as the change is minimal on
> our
> > >>> side
> > >>> > > and
> > >>> > > > it
> > >>> > > > > > has
> > >>> > > > > > > been tested in production for some time by the reporter.
> > >>> > > > > > >
> > >>> > > > > > > Best,
> > >>> > > > > > > Ismael
> > >>> > > > > > >
> > >>> > > > > > > On Fri, Aug 14, 2015 at 1:15 PM, Jun Rao <
> jun@confluent.io
> > >
> > >>> > wrote:
> > >>> > > > > > >
> > >>> > > > > > > > Hi, Everyone,
> > >>> > > > > > > >
> > >>> > > > > > > > Since the release of Kafka 0.8.2.1, a number of people
> > have
> > >>> > > > reported
> > >>> > > > > an
> > >>> > > > > > > > issue with snappy compression (
> > >>> > > > > > > > https://issues.apache.org/jira/browse/KAFKA-2189).
> > >>> Basically,
> > >>> > if
> > >>> > > > > they
> > >>> > > > > > > use
> > >>> > > > > > > > snappy in 0.8.2.1, they will experience a 2-3X space
> > >>> increase.
> > >>> > > The
> > >>> > > > > > issue
> > >>> > > > > > > > has since been fixed in trunk (just a snappy jar
> > upgrade).
> > >>> > Since
> > >>> > > > > 0.8.3
> > >>> > > > > > is
> > >>> > > > > > > > still a few months away, it may make sense to do an
> > 0.8.2.2
> > >>> > > release
> > >>> > > > > > just
> > >>> > > > > > > to
> > >>> > > > > > > > fix this issue. Any objections?
> > >>> > > > > > > >
> > >>> > > > > > > > Thanks,
> > >>> > > > > > > >
> > >>> > > > > > > > Jun
> > >>> > > > > > > >
> > >>> > > > > > >
> > >>> > > > > >
> > >>> > > > >
> > >>> > > > >
> > >>> > > > >
> > >>> > > > > --
> > >>> > > > > Grant Henke
> > >>> > > > > Software Engineer | Cloudera
> > >>> > > > > grant@cloudera.com | twitter.com/gchenke |
> > >>> > linkedin.com/in/granthenke
> > >>> > > > >
> > >>> > > >
> > >>> > >
> > >>> >
> > >>> >
> > >>> >
> > >>> > --
> > >>> > Jeff Holoman
> > >>> > Systems Engineer
> > >>> >
> > >>>
> > >>
> > >
> > >
> > > --
> > > Grant Henke
> > > Software Engineer | Cloudera
> > > grant@cloudera.com | twitter.com/gchenke | linkedin.com/in/granthenke
> > >
> >
> >
> >
> > --
> > Grant Henke
> > Software Engineer | Cloudera
> > grant@cloudera.com | twitter.com/gchenke | linkedin.com/in/granthenke
> >
>
Re: [DISCUSSION] Kafka 0.8.2.2 release?
Posted by Jun Rao <ju...@confluent.io>.
Gwen,
Thanks for putting the list together.
I'd recommend that we exclude the following:
KAFKA-1702: This is for the old producer and is only a problem if there are
some unexpected exceptions (e.g. UnknownClass).
KAFKA-2336: Most people don't change offsets.topic.num.partitions.
KAFKA-1724: The patch there is never committed since the fix is included in
another jira (a much larger patch).
KAFKA-2241: This doesn't seem be a common problem. It only happens when the
fetch request blocks on the broker for an extended period of time, which
should be rare.
I'd also recommend that we include the following:
KAFKA-2147: This impacts the memory size of the purgatory and a number of
people have experienced that. The fix is small and has been tested in
production usage. It hasn't been committed though since the issue is
already fixed in trunk and we weren't planning for an 0.8.2.2 release then.
Thanks,
Jun
On Mon, Aug 17, 2015 at 2:56 PM, Gwen Shapira <gw...@confluent.io> wrote:
> Thanks for creating a list, Grant!
>
> I placed it on the wiki with a quick evaluation of the content and whether
> it should be in 0.8.2.2:
>
> https://cwiki.apache.org/confluence/display/KAFKA/Proposed+patches+for+0.8.2.2
>
> I'm attempting to only cherrypick fixes that are both important for large
> number of users (or very critical to some users) and very safe (mostly
> judged by the size of the change, but not only)
>
> If your favorite bugfix is missing from the list, or is there but marked
> "No", please let us know (in this thread) what we are missing and why it is
> both important and safe.
> Also, if I accidentally included something you consider unsafe, speak up!
>
> Gwen
>
> On Mon, Aug 17, 2015 at 8:17 AM, Grant Henke <gh...@cloudera.com> wrote:
>
> > +dev
> >
> > Adding dev list back in. Somehow it got dropped.
> >
> >
> > On Mon, Aug 17, 2015 at 10:16 AM, Grant Henke <gh...@cloudera.com>
> wrote:
> >
> > > Below is a list of candidate bug fix jiras marked fixed for 0.8.3. I
> > don't
> > > suspect all of these will (or should) make it into the release but this
> > > should be a relatively complete list to work from:
> > >
> > > - KAFKA-2114 <https://issues.apache.org/jira/browse/KAFKA-2114>:
> > Unable
> > > to change min.insync.replicas default
> > > - KAFKA-1702 <https://issues.apache.org/jira/browse/KAFKA-1702>:
> > > Messages silently Lost by producer
> > > - KAFKA-2012 <https://issues.apache.org/jira/browse/KAFKA-2012>:
> > > Broker should automatically handle corrupt index files
> > > - KAFKA-2406 <https://issues.apache.org/jira/browse/KAFKA-2406>:
> ISR
> > > propagation should be throttled to avoid overwhelming controller.
> > > - KAFKA-2336 <https://issues.apache.org/jira/browse/KAFKA-2336>:
> > > Changing offsets.topic.num.partitions after the offset topic is
> > created
> > > breaks consumer group partition assignment
> > > - KAFKA-2337 <https://issues.apache.org/jira/browse/KAFKA-2337>:
> > Verify
> > > that metric names will not collide when creating new topics
> > > - KAFKA-2393 <https://issues.apache.org/jira/browse/KAFKA-2393>:
> > > Correctly Handle InvalidTopicException in
> KafkaApis.getTopicMetadata()
> > > - KAFKA-2189 <https://issues.apache.org/jira/browse/KAFKA-2189>:
> > Snappy
> > > compression of message batches less efficient in 0.8.2.1
> > > - KAFKA-2308 <https://issues.apache.org/jira/browse/KAFKA-2308>:
> New
> > > producer + Snappy face un-compression errors after broker restart
> > > - KAFKA-2042 <https://issues.apache.org/jira/browse/KAFKA-2042>:
> New
> > > producer metadata update always get all topics.
> > > - KAFKA-1367 <https://issues.apache.org/jira/browse/KAFKA-1367>:
> > Broker
> > > topic metadata not kept in sync with ZooKeeper
> > > - KAFKA-972 <https://issues.apache.org/jira/browse/KAFKA-972>:
> > MetadataRequest
> > > returns stale list of brokers
> > > - KAFKA-1867 <https://issues.apache.org/jira/browse/KAFKA-1867>:
> > liveBroker
> > > list not updated on a cluster with no topics
> > > - KAFKA-1650 <https://issues.apache.org/jira/browse/KAFKA-1650>:
> > Mirror
> > > Maker could lose data on unclean shutdown.
> > > - KAFKA-2009 <https://issues.apache.org/jira/browse/KAFKA-2009>:
> Fix
> > > UncheckedOffset.removeOffset synchronization and trace logging issue
> > in
> > > mirror maker
> > > - KAFKA-2407 <https://issues.apache.org/jira/browse/KAFKA-2407>:
> Only
> > > create a log directory when it will be used
> > > - KAFKA-2327 <https://issues.apache.org/jira/browse/KAFKA-2327>:
> > > broker doesn't start if config defines advertised.host but not
> > > advertised.port
> > > - KAFKA-1788: producer record can stay in RecordAccumulator forever
> if
> > > leader is no available
> > > - KAFKA-2234 <https://issues.apache.org/jira/browse/KAFKA-2234>:
> > > Partition reassignment of a nonexistent topic prevents future
> > reassignments
> > > - KAFKA-2096 <https://issues.apache.org/jira/browse/KAFKA-2096>:
> > > Enable keepalive socket option for broker to prevent socket leak
> > > - KAFKA-1057 <https://issues.apache.org/jira/browse/KAFKA-1057>:
> Trim
> > > whitespaces from user specified configs
> > > - KAFKA-1641 <https://issues.apache.org/jira/browse/KAFKA-1641>:
> Log
> > > cleaner exits if last cleaned offset is lower than earliest offset
> > > - KAFKA-1648 <https://issues.apache.org/jira/browse/KAFKA-1648>:
> > Round
> > > robin consumer balance throws an NPE when there are no topics
> > > - KAFKA-1724 <https://issues.apache.org/jira/browse/KAFKA-1724>:
> > > Errors after reboot in single node setup
> > > - KAFKA-1758 <https://issues.apache.org/jira/browse/KAFKA-1758>:
> > > corrupt recovery file prevents startup
> > > - KAFKA-1866 <https://issues.apache.org/jira/browse/KAFKA-1866>:
> > > LogStartOffset gauge throws exceptions after log.delete()
> > > - KAFKA-1883 <https://issues.apache.org/jira/browse/KAFKA-1883>:
> > NullPointerException
> > > in RequestSendThread
> > > - KAFKA-1896 <https://issues.apache.org/jira/browse/KAFKA-1896>:
> > > Record size funcition of record in mirror maker hit NPE when the
> > message
> > > value is null.
> > > - KAFKA-2101 <https://issues.apache.org/jira/browse/KAFKA-2101>:
> > > Metric metadata-age is reset on a failed update
> > > - KAFKA-2112 <https://issues.apache.org/jira/browse/KAFKA-2112>:
> make
> > > overflowWheel volatile
> > > - KAFKA-2117 <https://issues.apache.org/jira/browse/KAFKA-2117>:
> > > OffsetManager uses incorrect field for metadata
> > > - KAFKA-2164 <https://issues.apache.org/jira/browse/KAFKA-2164>:
> > > ReplicaFetcherThread: suspicious log message on reset offset
> > > - KAFKA-1668 <https://issues.apache.org/jira/browse/KAFKA-1668>:
> > > TopicCommand doesn't warn if --topic argument doesn't match any
> topics
> > > - KAFKA-2198 <https://issues.apache.org/jira/browse/KAFKA-2198>:
> > > kafka-topics.sh exits with 0 status on failures
> > > - KAFKA-2235 <https://issues.apache.org/jira/browse/KAFKA-2235>:
> > > LogCleaner offset map overflow
> > > - KAFKA-2241 <https://issues.apache.org/jira/browse/KAFKA-2241>:
> > > AbstractFetcherThread.shutdown() should not block on
> > > ReadableByteChannel.read(buffer)
> > > - KAFKA-2272 <https://issues.apache.org/jira/browse/KAFKA-2272>:
> > > listeners endpoint parsing fails if the hostname has capital letter
> > > - KAFKA-2345 <https://issues.apache.org/jira/browse/KAFKA-2345>:
> > > Attempt to delete a topic already marked for deletion throws
> > > ZkNodeExistsException
> > > - KAFKA-2353 <https://issues.apache.org/jira/browse/KAFKA-2353>:
> > > SocketServer.Processor should catch exception and close the socket
> > properly
> > > in configureNewConnections.
> > > - KAFKA-1836 <https://issues.apache.org/jira/browse/KAFKA-1836>:
> > > metadata.fetch.timeout.ms set to zero blocks forever
> > > - KAFKA-2317 <https://issues.apache.org/jira/browse/KAFKA-2317>:
> > De-register
> > > isrChangeNotificationListener on controller resignation
> > >
> > > Note: KAFKA-2120 <https://issues.apache.org/jira/browse/KAFKA-2120> &
> > > KAFKA-2421 <https://issues.apache.org/jira/browse/KAFKA-2421> were
> > > mentioned in previous emails, but are not in the list because they are
> > not
> > > committed yet.
> > >
> > > Hope that helps the effort.
> > >
> > > Thanks,
> > > Grant
> > >
> > > On Mon, Aug 17, 2015 at 12:09 AM, Grant Henke <gh...@cloudera.com>
> > wrote:
> > >
> > >> +1 to that suggestion. Though I suspect that requires a committer to
> do.
> > >> Making it part of the standard commit process could work too.
> > >> On Aug 16, 2015 11:01 PM, "Gwen Shapira" <gw...@confluent.io> wrote:
> > >>
> > >>> BTW. I think it will be great for Apache Kafka to have a 0.8.2
> "release
> > >>> manager" who's role is to cherrypick low-risk bug-fixes into the
> 0.8.2
> > >>> branch and once enough bug fixes happened (or if sufficiently
> critical
> > >>> fixes happened) to roll out a new maintenance release (with every 3
> > month
> > >>> as a reasonable bugfix release target).
> > >>>
> > >>> This will add some predictability regarding how fast we release fixes
> > for
> > >>> bugs.
> > >>>
> > >>> Gwen
> > >>>
> > >>> On Sun, Aug 16, 2015 at 8:09 PM, Jeff Holoman <jholoman@cloudera.com
> >
> > >>> wrote:
> > >>>
> > >>> > +1 for the release and also including
> > >>> >
> > >>> > https://issues.apache.org/jira/browse/KAFKA-2114
> > >>> >
> > >>> > Thanks
> > >>> >
> > >>> > Jeff
> > >>> >
> > >>> > On Sun, Aug 16, 2015 at 2:51 PM, Stevo Slavić <ss...@gmail.com>
> > >>> wrote:
> > >>> >
> > >>> > > +1 (non-binding) for 0.8.2.2 release
> > >>> > >
> > >>> > > Would be nice to include in that release new producer resiliency
> > bug
> > >>> > fixes
> > >>> > > https://issues.apache.org/jira/browse/KAFKA-1788 and
> > >>> > > https://issues.apache.org/jira/browse/KAFKA-2120
> > >>> > >
> > >>> > > On Fri, Aug 14, 2015 at 4:03 PM, Gwen Shapira <gwen@confluent.io
> >
> > >>> wrote:
> > >>> > >
> > >>> > > > Will be nice to include Kafka-2308 and fix two critical snappy
> > >>> issues
> > >>> > in
> > >>> > > > the maintenance release.
> > >>> > > >
> > >>> > > > Gwen
> > >>> > > > On Aug 14, 2015 6:16 AM, "Grant Henke" <gh...@cloudera.com>
> > >>> wrote:
> > >>> > > >
> > >>> > > > > Just to clarify. Will KAFKA-2189 be the only patch in the
> > >>> release?
> > >>> > > > >
> > >>> > > > > On Fri, Aug 14, 2015 at 7:35 AM, Manikumar Reddy <
> > >>> > kumar@nmsworks.co.in
> > >>> > > >
> > >>> > > > > wrote:
> > >>> > > > >
> > >>> > > > > > +1 for 0.8.2.2 release
> > >>> > > > > >
> > >>> > > > > > On Fri, Aug 14, 2015 at 5:49 PM, Ismael Juma <
> > >>> ismael@juma.me.uk>
> > >>> > > > wrote:
> > >>> > > > > >
> > >>> > > > > > > I think this is a good idea as the change is minimal on
> our
> > >>> side
> > >>> > > and
> > >>> > > > it
> > >>> > > > > > has
> > >>> > > > > > > been tested in production for some time by the reporter.
> > >>> > > > > > >
> > >>> > > > > > > Best,
> > >>> > > > > > > Ismael
> > >>> > > > > > >
> > >>> > > > > > > On Fri, Aug 14, 2015 at 1:15 PM, Jun Rao <
> jun@confluent.io
> > >
> > >>> > wrote:
> > >>> > > > > > >
> > >>> > > > > > > > Hi, Everyone,
> > >>> > > > > > > >
> > >>> > > > > > > > Since the release of Kafka 0.8.2.1, a number of people
> > have
> > >>> > > > reported
> > >>> > > > > an
> > >>> > > > > > > > issue with snappy compression (
> > >>> > > > > > > > https://issues.apache.org/jira/browse/KAFKA-2189).
> > >>> Basically,
> > >>> > if
> > >>> > > > > they
> > >>> > > > > > > use
> > >>> > > > > > > > snappy in 0.8.2.1, they will experience a 2-3X space
> > >>> increase.
> > >>> > > The
> > >>> > > > > > issue
> > >>> > > > > > > > has since been fixed in trunk (just a snappy jar
> > upgrade).
> > >>> > Since
> > >>> > > > > 0.8.3
> > >>> > > > > > is
> > >>> > > > > > > > still a few months away, it may make sense to do an
> > 0.8.2.2
> > >>> > > release
> > >>> > > > > > just
> > >>> > > > > > > to
> > >>> > > > > > > > fix this issue. Any objections?
> > >>> > > > > > > >
> > >>> > > > > > > > Thanks,
> > >>> > > > > > > >
> > >>> > > > > > > > Jun
> > >>> > > > > > > >
> > >>> > > > > > >
> > >>> > > > > >
> > >>> > > > >
> > >>> > > > >
> > >>> > > > >
> > >>> > > > > --
> > >>> > > > > Grant Henke
> > >>> > > > > Software Engineer | Cloudera
> > >>> > > > > grant@cloudera.com | twitter.com/gchenke |
> > >>> > linkedin.com/in/granthenke
> > >>> > > > >
> > >>> > > >
> > >>> > >
> > >>> >
> > >>> >
> > >>> >
> > >>> > --
> > >>> > Jeff Holoman
> > >>> > Systems Engineer
> > >>> >
> > >>>
> > >>
> > >
> > >
> > > --
> > > Grant Henke
> > > Software Engineer | Cloudera
> > > grant@cloudera.com | twitter.com/gchenke | linkedin.com/in/granthenke
> > >
> >
> >
> >
> > --
> > Grant Henke
> > Software Engineer | Cloudera
> > grant@cloudera.com | twitter.com/gchenke | linkedin.com/in/granthenke
> >
>
Re: [DISCUSSION] Kafka 0.8.2.2 release?
Posted by Grant Henke <gh...@cloudera.com>.
Thanks Gwen.
I updated a few small things on the wiki page.
Below is a list of jiras I think could also be marked as included. All of
these, though not super critical, seem like fairly small and low risk
changes that help avoid potentially confusing issues or errors for users.
KAFKA-2012
KAFKA-972
KAFKA-2337 & KAFKA-2393
KAFKA-1867
KAFKA-2407
KAFKA-2234
KAFKA-1866
KAFKA-2345 & KAFKA-2355
thoughts?
Thank you,
Grant
On Mon, Aug 17, 2015 at 4:56 PM, Gwen Shapira <gw...@confluent.io> wrote:
> Thanks for creating a list, Grant!
>
> I placed it on the wiki with a quick evaluation of the content and whether
> it should be in 0.8.2.2:
>
> https://cwiki.apache.org/confluence/display/KAFKA/Proposed+patches+for+0.8.2.2
>
> I'm attempting to only cherrypick fixes that are both important for large
> number of users (or very critical to some users) and very safe (mostly
> judged by the size of the change, but not only)
>
> If your favorite bugfix is missing from the list, or is there but marked
> "No", please let us know (in this thread) what we are missing and why it is
> both important and safe.
> Also, if I accidentally included something you consider unsafe, speak up!
>
> Gwen
>
> On Mon, Aug 17, 2015 at 8:17 AM, Grant Henke <gh...@cloudera.com> wrote:
>
> > +dev
> >
> > Adding dev list back in. Somehow it got dropped.
> >
> >
> > On Mon, Aug 17, 2015 at 10:16 AM, Grant Henke <gh...@cloudera.com>
> wrote:
> >
> > > Below is a list of candidate bug fix jiras marked fixed for 0.8.3. I
> > don't
> > > suspect all of these will (or should) make it into the release but this
> > > should be a relatively complete list to work from:
> > >
> > > - KAFKA-2114 <https://issues.apache.org/jira/browse/KAFKA-2114>:
> > Unable
> > > to change min.insync.replicas default
> > > - KAFKA-1702 <https://issues.apache.org/jira/browse/KAFKA-1702>:
> > > Messages silently Lost by producer
> > > - KAFKA-2012 <https://issues.apache.org/jira/browse/KAFKA-2012>:
> > > Broker should automatically handle corrupt index files
> > > - KAFKA-2406 <https://issues.apache.org/jira/browse/KAFKA-2406>:
> ISR
> > > propagation should be throttled to avoid overwhelming controller.
> > > - KAFKA-2336 <https://issues.apache.org/jira/browse/KAFKA-2336>:
> > > Changing offsets.topic.num.partitions after the offset topic is
> > created
> > > breaks consumer group partition assignment
> > > - KAFKA-2337 <https://issues.apache.org/jira/browse/KAFKA-2337>:
> > Verify
> > > that metric names will not collide when creating new topics
> > > - KAFKA-2393 <https://issues.apache.org/jira/browse/KAFKA-2393>:
> > > Correctly Handle InvalidTopicException in
> KafkaApis.getTopicMetadata()
> > > - KAFKA-2189 <https://issues.apache.org/jira/browse/KAFKA-2189>:
> > Snappy
> > > compression of message batches less efficient in 0.8.2.1
> > > - KAFKA-2308 <https://issues.apache.org/jira/browse/KAFKA-2308>:
> New
> > > producer + Snappy face un-compression errors after broker restart
> > > - KAFKA-2042 <https://issues.apache.org/jira/browse/KAFKA-2042>:
> New
> > > producer metadata update always get all topics.
> > > - KAFKA-1367 <https://issues.apache.org/jira/browse/KAFKA-1367>:
> > Broker
> > > topic metadata not kept in sync with ZooKeeper
> > > - KAFKA-972 <https://issues.apache.org/jira/browse/KAFKA-972>:
> > MetadataRequest
> > > returns stale list of brokers
> > > - KAFKA-1867 <https://issues.apache.org/jira/browse/KAFKA-1867>:
> > liveBroker
> > > list not updated on a cluster with no topics
> > > - KAFKA-1650 <https://issues.apache.org/jira/browse/KAFKA-1650>:
> > Mirror
> > > Maker could lose data on unclean shutdown.
> > > - KAFKA-2009 <https://issues.apache.org/jira/browse/KAFKA-2009>:
> Fix
> > > UncheckedOffset.removeOffset synchronization and trace logging issue
> > in
> > > mirror maker
> > > - KAFKA-2407 <https://issues.apache.org/jira/browse/KAFKA-2407>:
> Only
> > > create a log directory when it will be used
> > > - KAFKA-2327 <https://issues.apache.org/jira/browse/KAFKA-2327>:
> > > broker doesn't start if config defines advertised.host but not
> > > advertised.port
> > > - KAFKA-1788: producer record can stay in RecordAccumulator forever
> if
> > > leader is no available
> > > - KAFKA-2234 <https://issues.apache.org/jira/browse/KAFKA-2234>:
> > > Partition reassignment of a nonexistent topic prevents future
> > reassignments
> > > - KAFKA-2096 <https://issues.apache.org/jira/browse/KAFKA-2096>:
> > > Enable keepalive socket option for broker to prevent socket leak
> > > - KAFKA-1057 <https://issues.apache.org/jira/browse/KAFKA-1057>:
> Trim
> > > whitespaces from user specified configs
> > > - KAFKA-1641 <https://issues.apache.org/jira/browse/KAFKA-1641>:
> Log
> > > cleaner exits if last cleaned offset is lower than earliest offset
> > > - KAFKA-1648 <https://issues.apache.org/jira/browse/KAFKA-1648>:
> > Round
> > > robin consumer balance throws an NPE when there are no topics
> > > - KAFKA-1724 <https://issues.apache.org/jira/browse/KAFKA-1724>:
> > > Errors after reboot in single node setup
> > > - KAFKA-1758 <https://issues.apache.org/jira/browse/KAFKA-1758>:
> > > corrupt recovery file prevents startup
> > > - KAFKA-1866 <https://issues.apache.org/jira/browse/KAFKA-1866>:
> > > LogStartOffset gauge throws exceptions after log.delete()
> > > - KAFKA-1883 <https://issues.apache.org/jira/browse/KAFKA-1883>:
> > NullPointerException
> > > in RequestSendThread
> > > - KAFKA-1896 <https://issues.apache.org/jira/browse/KAFKA-1896>:
> > > Record size funcition of record in mirror maker hit NPE when the
> > message
> > > value is null.
> > > - KAFKA-2101 <https://issues.apache.org/jira/browse/KAFKA-2101>:
> > > Metric metadata-age is reset on a failed update
> > > - KAFKA-2112 <https://issues.apache.org/jira/browse/KAFKA-2112>:
> make
> > > overflowWheel volatile
> > > - KAFKA-2117 <https://issues.apache.org/jira/browse/KAFKA-2117>:
> > > OffsetManager uses incorrect field for metadata
> > > - KAFKA-2164 <https://issues.apache.org/jira/browse/KAFKA-2164>:
> > > ReplicaFetcherThread: suspicious log message on reset offset
> > > - KAFKA-1668 <https://issues.apache.org/jira/browse/KAFKA-1668>:
> > > TopicCommand doesn't warn if --topic argument doesn't match any
> topics
> > > - KAFKA-2198 <https://issues.apache.org/jira/browse/KAFKA-2198>:
> > > kafka-topics.sh exits with 0 status on failures
> > > - KAFKA-2235 <https://issues.apache.org/jira/browse/KAFKA-2235>:
> > > LogCleaner offset map overflow
> > > - KAFKA-2241 <https://issues.apache.org/jira/browse/KAFKA-2241>:
> > > AbstractFetcherThread.shutdown() should not block on
> > > ReadableByteChannel.read(buffer)
> > > - KAFKA-2272 <https://issues.apache.org/jira/browse/KAFKA-2272>:
> > > listeners endpoint parsing fails if the hostname has capital letter
> > > - KAFKA-2345 <https://issues.apache.org/jira/browse/KAFKA-2345>:
> > > Attempt to delete a topic already marked for deletion throws
> > > ZkNodeExistsException
> > > - KAFKA-2353 <https://issues.apache.org/jira/browse/KAFKA-2353>:
> > > SocketServer.Processor should catch exception and close the socket
> > properly
> > > in configureNewConnections.
> > > - KAFKA-1836 <https://issues.apache.org/jira/browse/KAFKA-1836>:
> > > metadata.fetch.timeout.ms set to zero blocks forever
> > > - KAFKA-2317 <https://issues.apache.org/jira/browse/KAFKA-2317>:
> > De-register
> > > isrChangeNotificationListener on controller resignation
> > >
> > > Note: KAFKA-2120 <https://issues.apache.org/jira/browse/KAFKA-2120> &
> > > KAFKA-2421 <https://issues.apache.org/jira/browse/KAFKA-2421> were
> > > mentioned in previous emails, but are not in the list because they are
> > not
> > > committed yet.
> > >
> > > Hope that helps the effort.
> > >
> > > Thanks,
> > > Grant
> > >
> > > On Mon, Aug 17, 2015 at 12:09 AM, Grant Henke <gh...@cloudera.com>
> > wrote:
> > >
> > >> +1 to that suggestion. Though I suspect that requires a committer to
> do.
> > >> Making it part of the standard commit process could work too.
> > >> On Aug 16, 2015 11:01 PM, "Gwen Shapira" <gw...@confluent.io> wrote:
> > >>
> > >>> BTW. I think it will be great for Apache Kafka to have a 0.8.2
> "release
> > >>> manager" who's role is to cherrypick low-risk bug-fixes into the
> 0.8.2
> > >>> branch and once enough bug fixes happened (or if sufficiently
> critical
> > >>> fixes happened) to roll out a new maintenance release (with every 3
> > month
> > >>> as a reasonable bugfix release target).
> > >>>
> > >>> This will add some predictability regarding how fast we release fixes
> > for
> > >>> bugs.
> > >>>
> > >>> Gwen
> > >>>
> > >>> On Sun, Aug 16, 2015 at 8:09 PM, Jeff Holoman <jholoman@cloudera.com
> >
> > >>> wrote:
> > >>>
> > >>> > +1 for the release and also including
> > >>> >
> > >>> > https://issues.apache.org/jira/browse/KAFKA-2114
> > >>> >
> > >>> > Thanks
> > >>> >
> > >>> > Jeff
> > >>> >
> > >>> > On Sun, Aug 16, 2015 at 2:51 PM, Stevo Slavić <ss...@gmail.com>
> > >>> wrote:
> > >>> >
> > >>> > > +1 (non-binding) for 0.8.2.2 release
> > >>> > >
> > >>> > > Would be nice to include in that release new producer resiliency
> > bug
> > >>> > fixes
> > >>> > > https://issues.apache.org/jira/browse/KAFKA-1788 and
> > >>> > > https://issues.apache.org/jira/browse/KAFKA-2120
> > >>> > >
> > >>> > > On Fri, Aug 14, 2015 at 4:03 PM, Gwen Shapira <gwen@confluent.io
> >
> > >>> wrote:
> > >>> > >
> > >>> > > > Will be nice to include Kafka-2308 and fix two critical snappy
> > >>> issues
> > >>> > in
> > >>> > > > the maintenance release.
> > >>> > > >
> > >>> > > > Gwen
> > >>> > > > On Aug 14, 2015 6:16 AM, "Grant Henke" <gh...@cloudera.com>
> > >>> wrote:
> > >>> > > >
> > >>> > > > > Just to clarify. Will KAFKA-2189 be the only patch in the
> > >>> release?
> > >>> > > > >
> > >>> > > > > On Fri, Aug 14, 2015 at 7:35 AM, Manikumar Reddy <
> > >>> > kumar@nmsworks.co.in
> > >>> > > >
> > >>> > > > > wrote:
> > >>> > > > >
> > >>> > > > > > +1 for 0.8.2.2 release
> > >>> > > > > >
> > >>> > > > > > On Fri, Aug 14, 2015 at 5:49 PM, Ismael Juma <
> > >>> ismael@juma.me.uk>
> > >>> > > > wrote:
> > >>> > > > > >
> > >>> > > > > > > I think this is a good idea as the change is minimal on
> our
> > >>> side
> > >>> > > and
> > >>> > > > it
> > >>> > > > > > has
> > >>> > > > > > > been tested in production for some time by the reporter.
> > >>> > > > > > >
> > >>> > > > > > > Best,
> > >>> > > > > > > Ismael
> > >>> > > > > > >
> > >>> > > > > > > On Fri, Aug 14, 2015 at 1:15 PM, Jun Rao <
> jun@confluent.io
> > >
> > >>> > wrote:
> > >>> > > > > > >
> > >>> > > > > > > > Hi, Everyone,
> > >>> > > > > > > >
> > >>> > > > > > > > Since the release of Kafka 0.8.2.1, a number of people
> > have
> > >>> > > > reported
> > >>> > > > > an
> > >>> > > > > > > > issue with snappy compression (
> > >>> > > > > > > > https://issues.apache.org/jira/browse/KAFKA-2189).
> > >>> Basically,
> > >>> > if
> > >>> > > > > they
> > >>> > > > > > > use
> > >>> > > > > > > > snappy in 0.8.2.1, they will experience a 2-3X space
> > >>> increase.
> > >>> > > The
> > >>> > > > > > issue
> > >>> > > > > > > > has since been fixed in trunk (just a snappy jar
> > upgrade).
> > >>> > Since
> > >>> > > > > 0.8.3
> > >>> > > > > > is
> > >>> > > > > > > > still a few months away, it may make sense to do an
> > 0.8.2.2
> > >>> > > release
> > >>> > > > > > just
> > >>> > > > > > > to
> > >>> > > > > > > > fix this issue. Any objections?
> > >>> > > > > > > >
> > >>> > > > > > > > Thanks,
> > >>> > > > > > > >
> > >>> > > > > > > > Jun
> > >>> > > > > > > >
> > >>> > > > > > >
> > >>> > > > > >
> > >>> > > > >
> > >>> > > > >
> > >>> > > > >
> > >>> > > > > --
> > >>> > > > > Grant Henke
> > >>> > > > > Software Engineer | Cloudera
> > >>> > > > > grant@cloudera.com | twitter.com/gchenke |
> > >>> > linkedin.com/in/granthenke
> > >>> > > > >
> > >>> > > >
> > >>> > >
> > >>> >
> > >>> >
> > >>> >
> > >>> > --
> > >>> > Jeff Holoman
> > >>> > Systems Engineer
> > >>> >
> > >>>
> > >>
> > >
> > >
> > > --
> > > Grant Henke
> > > Software Engineer | Cloudera
> > > grant@cloudera.com | twitter.com/gchenke | linkedin.com/in/granthenke
> > >
> >
> >
> >
> > --
> > Grant Henke
> > Software Engineer | Cloudera
> > grant@cloudera.com | twitter.com/gchenke | linkedin.com/in/granthenke
> >
>
--
Grant Henke
Software Engineer | Cloudera
grant@cloudera.com | twitter.com/gchenke | linkedin.com/in/granthenke
Re: [DISCUSSION] Kafka 0.8.2.2 release?
Posted by Gwen Shapira <gw...@confluent.io>.
Thanks for creating a list, Grant!
I placed it on the wiki with a quick evaluation of the content and whether
it should be in 0.8.2.2:
https://cwiki.apache.org/confluence/display/KAFKA/Proposed+patches+for+0.8.2.2
I'm attempting to only cherrypick fixes that are both important for large
number of users (or very critical to some users) and very safe (mostly
judged by the size of the change, but not only)
If your favorite bugfix is missing from the list, or is there but marked
"No", please let us know (in this thread) what we are missing and why it is
both important and safe.
Also, if I accidentally included something you consider unsafe, speak up!
Gwen
On Mon, Aug 17, 2015 at 8:17 AM, Grant Henke <gh...@cloudera.com> wrote:
> +dev
>
> Adding dev list back in. Somehow it got dropped.
>
>
> On Mon, Aug 17, 2015 at 10:16 AM, Grant Henke <gh...@cloudera.com> wrote:
>
> > Below is a list of candidate bug fix jiras marked fixed for 0.8.3. I
> don't
> > suspect all of these will (or should) make it into the release but this
> > should be a relatively complete list to work from:
> >
> > - KAFKA-2114 <https://issues.apache.org/jira/browse/KAFKA-2114>:
> Unable
> > to change min.insync.replicas default
> > - KAFKA-1702 <https://issues.apache.org/jira/browse/KAFKA-1702>:
> > Messages silently Lost by producer
> > - KAFKA-2012 <https://issues.apache.org/jira/browse/KAFKA-2012>:
> > Broker should automatically handle corrupt index files
> > - KAFKA-2406 <https://issues.apache.org/jira/browse/KAFKA-2406>: ISR
> > propagation should be throttled to avoid overwhelming controller.
> > - KAFKA-2336 <https://issues.apache.org/jira/browse/KAFKA-2336>:
> > Changing offsets.topic.num.partitions after the offset topic is
> created
> > breaks consumer group partition assignment
> > - KAFKA-2337 <https://issues.apache.org/jira/browse/KAFKA-2337>:
> Verify
> > that metric names will not collide when creating new topics
> > - KAFKA-2393 <https://issues.apache.org/jira/browse/KAFKA-2393>:
> > Correctly Handle InvalidTopicException in KafkaApis.getTopicMetadata()
> > - KAFKA-2189 <https://issues.apache.org/jira/browse/KAFKA-2189>:
> Snappy
> > compression of message batches less efficient in 0.8.2.1
> > - KAFKA-2308 <https://issues.apache.org/jira/browse/KAFKA-2308>: New
> > producer + Snappy face un-compression errors after broker restart
> > - KAFKA-2042 <https://issues.apache.org/jira/browse/KAFKA-2042>: New
> > producer metadata update always get all topics.
> > - KAFKA-1367 <https://issues.apache.org/jira/browse/KAFKA-1367>:
> Broker
> > topic metadata not kept in sync with ZooKeeper
> > - KAFKA-972 <https://issues.apache.org/jira/browse/KAFKA-972>:
> MetadataRequest
> > returns stale list of brokers
> > - KAFKA-1867 <https://issues.apache.org/jira/browse/KAFKA-1867>:
> liveBroker
> > list not updated on a cluster with no topics
> > - KAFKA-1650 <https://issues.apache.org/jira/browse/KAFKA-1650>:
> Mirror
> > Maker could lose data on unclean shutdown.
> > - KAFKA-2009 <https://issues.apache.org/jira/browse/KAFKA-2009>: Fix
> > UncheckedOffset.removeOffset synchronization and trace logging issue
> in
> > mirror maker
> > - KAFKA-2407 <https://issues.apache.org/jira/browse/KAFKA-2407>: Only
> > create a log directory when it will be used
> > - KAFKA-2327 <https://issues.apache.org/jira/browse/KAFKA-2327>:
> > broker doesn't start if config defines advertised.host but not
> > advertised.port
> > - KAFKA-1788: producer record can stay in RecordAccumulator forever if
> > leader is no available
> > - KAFKA-2234 <https://issues.apache.org/jira/browse/KAFKA-2234>:
> > Partition reassignment of a nonexistent topic prevents future
> reassignments
> > - KAFKA-2096 <https://issues.apache.org/jira/browse/KAFKA-2096>:
> > Enable keepalive socket option for broker to prevent socket leak
> > - KAFKA-1057 <https://issues.apache.org/jira/browse/KAFKA-1057>: Trim
> > whitespaces from user specified configs
> > - KAFKA-1641 <https://issues.apache.org/jira/browse/KAFKA-1641>: Log
> > cleaner exits if last cleaned offset is lower than earliest offset
> > - KAFKA-1648 <https://issues.apache.org/jira/browse/KAFKA-1648>:
> Round
> > robin consumer balance throws an NPE when there are no topics
> > - KAFKA-1724 <https://issues.apache.org/jira/browse/KAFKA-1724>:
> > Errors after reboot in single node setup
> > - KAFKA-1758 <https://issues.apache.org/jira/browse/KAFKA-1758>:
> > corrupt recovery file prevents startup
> > - KAFKA-1866 <https://issues.apache.org/jira/browse/KAFKA-1866>:
> > LogStartOffset gauge throws exceptions after log.delete()
> > - KAFKA-1883 <https://issues.apache.org/jira/browse/KAFKA-1883>:
> NullPointerException
> > in RequestSendThread
> > - KAFKA-1896 <https://issues.apache.org/jira/browse/KAFKA-1896>:
> > Record size funcition of record in mirror maker hit NPE when the
> message
> > value is null.
> > - KAFKA-2101 <https://issues.apache.org/jira/browse/KAFKA-2101>:
> > Metric metadata-age is reset on a failed update
> > - KAFKA-2112 <https://issues.apache.org/jira/browse/KAFKA-2112>: make
> > overflowWheel volatile
> > - KAFKA-2117 <https://issues.apache.org/jira/browse/KAFKA-2117>:
> > OffsetManager uses incorrect field for metadata
> > - KAFKA-2164 <https://issues.apache.org/jira/browse/KAFKA-2164>:
> > ReplicaFetcherThread: suspicious log message on reset offset
> > - KAFKA-1668 <https://issues.apache.org/jira/browse/KAFKA-1668>:
> > TopicCommand doesn't warn if --topic argument doesn't match any topics
> > - KAFKA-2198 <https://issues.apache.org/jira/browse/KAFKA-2198>:
> > kafka-topics.sh exits with 0 status on failures
> > - KAFKA-2235 <https://issues.apache.org/jira/browse/KAFKA-2235>:
> > LogCleaner offset map overflow
> > - KAFKA-2241 <https://issues.apache.org/jira/browse/KAFKA-2241>:
> > AbstractFetcherThread.shutdown() should not block on
> > ReadableByteChannel.read(buffer)
> > - KAFKA-2272 <https://issues.apache.org/jira/browse/KAFKA-2272>:
> > listeners endpoint parsing fails if the hostname has capital letter
> > - KAFKA-2345 <https://issues.apache.org/jira/browse/KAFKA-2345>:
> > Attempt to delete a topic already marked for deletion throws
> > ZkNodeExistsException
> > - KAFKA-2353 <https://issues.apache.org/jira/browse/KAFKA-2353>:
> > SocketServer.Processor should catch exception and close the socket
> properly
> > in configureNewConnections.
> > - KAFKA-1836 <https://issues.apache.org/jira/browse/KAFKA-1836>:
> > metadata.fetch.timeout.ms set to zero blocks forever
> > - KAFKA-2317 <https://issues.apache.org/jira/browse/KAFKA-2317>:
> De-register
> > isrChangeNotificationListener on controller resignation
> >
> > Note: KAFKA-2120 <https://issues.apache.org/jira/browse/KAFKA-2120> &
> > KAFKA-2421 <https://issues.apache.org/jira/browse/KAFKA-2421> were
> > mentioned in previous emails, but are not in the list because they are
> not
> > committed yet.
> >
> > Hope that helps the effort.
> >
> > Thanks,
> > Grant
> >
> > On Mon, Aug 17, 2015 at 12:09 AM, Grant Henke <gh...@cloudera.com>
> wrote:
> >
> >> +1 to that suggestion. Though I suspect that requires a committer to do.
> >> Making it part of the standard commit process could work too.
> >> On Aug 16, 2015 11:01 PM, "Gwen Shapira" <gw...@confluent.io> wrote:
> >>
> >>> BTW. I think it will be great for Apache Kafka to have a 0.8.2 "release
> >>> manager" who's role is to cherrypick low-risk bug-fixes into the 0.8.2
> >>> branch and once enough bug fixes happened (or if sufficiently critical
> >>> fixes happened) to roll out a new maintenance release (with every 3
> month
> >>> as a reasonable bugfix release target).
> >>>
> >>> This will add some predictability regarding how fast we release fixes
> for
> >>> bugs.
> >>>
> >>> Gwen
> >>>
> >>> On Sun, Aug 16, 2015 at 8:09 PM, Jeff Holoman <jh...@cloudera.com>
> >>> wrote:
> >>>
> >>> > +1 for the release and also including
> >>> >
> >>> > https://issues.apache.org/jira/browse/KAFKA-2114
> >>> >
> >>> > Thanks
> >>> >
> >>> > Jeff
> >>> >
> >>> > On Sun, Aug 16, 2015 at 2:51 PM, Stevo Slavić <ss...@gmail.com>
> >>> wrote:
> >>> >
> >>> > > +1 (non-binding) for 0.8.2.2 release
> >>> > >
> >>> > > Would be nice to include in that release new producer resiliency
> bug
> >>> > fixes
> >>> > > https://issues.apache.org/jira/browse/KAFKA-1788 and
> >>> > > https://issues.apache.org/jira/browse/KAFKA-2120
> >>> > >
> >>> > > On Fri, Aug 14, 2015 at 4:03 PM, Gwen Shapira <gw...@confluent.io>
> >>> wrote:
> >>> > >
> >>> > > > Will be nice to include Kafka-2308 and fix two critical snappy
> >>> issues
> >>> > in
> >>> > > > the maintenance release.
> >>> > > >
> >>> > > > Gwen
> >>> > > > On Aug 14, 2015 6:16 AM, "Grant Henke" <gh...@cloudera.com>
> >>> wrote:
> >>> > > >
> >>> > > > > Just to clarify. Will KAFKA-2189 be the only patch in the
> >>> release?
> >>> > > > >
> >>> > > > > On Fri, Aug 14, 2015 at 7:35 AM, Manikumar Reddy <
> >>> > kumar@nmsworks.co.in
> >>> > > >
> >>> > > > > wrote:
> >>> > > > >
> >>> > > > > > +1 for 0.8.2.2 release
> >>> > > > > >
> >>> > > > > > On Fri, Aug 14, 2015 at 5:49 PM, Ismael Juma <
> >>> ismael@juma.me.uk>
> >>> > > > wrote:
> >>> > > > > >
> >>> > > > > > > I think this is a good idea as the change is minimal on our
> >>> side
> >>> > > and
> >>> > > > it
> >>> > > > > > has
> >>> > > > > > > been tested in production for some time by the reporter.
> >>> > > > > > >
> >>> > > > > > > Best,
> >>> > > > > > > Ismael
> >>> > > > > > >
> >>> > > > > > > On Fri, Aug 14, 2015 at 1:15 PM, Jun Rao <jun@confluent.io
> >
> >>> > wrote:
> >>> > > > > > >
> >>> > > > > > > > Hi, Everyone,
> >>> > > > > > > >
> >>> > > > > > > > Since the release of Kafka 0.8.2.1, a number of people
> have
> >>> > > > reported
> >>> > > > > an
> >>> > > > > > > > issue with snappy compression (
> >>> > > > > > > > https://issues.apache.org/jira/browse/KAFKA-2189).
> >>> Basically,
> >>> > if
> >>> > > > > they
> >>> > > > > > > use
> >>> > > > > > > > snappy in 0.8.2.1, they will experience a 2-3X space
> >>> increase.
> >>> > > The
> >>> > > > > > issue
> >>> > > > > > > > has since been fixed in trunk (just a snappy jar
> upgrade).
> >>> > Since
> >>> > > > > 0.8.3
> >>> > > > > > is
> >>> > > > > > > > still a few months away, it may make sense to do an
> 0.8.2.2
> >>> > > release
> >>> > > > > > just
> >>> > > > > > > to
> >>> > > > > > > > fix this issue. Any objections?
> >>> > > > > > > >
> >>> > > > > > > > Thanks,
> >>> > > > > > > >
> >>> > > > > > > > Jun
> >>> > > > > > > >
> >>> > > > > > >
> >>> > > > > >
> >>> > > > >
> >>> > > > >
> >>> > > > >
> >>> > > > > --
> >>> > > > > Grant Henke
> >>> > > > > Software Engineer | Cloudera
> >>> > > > > grant@cloudera.com | twitter.com/gchenke |
> >>> > linkedin.com/in/granthenke
> >>> > > > >
> >>> > > >
> >>> > >
> >>> >
> >>> >
> >>> >
> >>> > --
> >>> > Jeff Holoman
> >>> > Systems Engineer
> >>> >
> >>>
> >>
> >
> >
> > --
> > Grant Henke
> > Software Engineer | Cloudera
> > grant@cloudera.com | twitter.com/gchenke | linkedin.com/in/granthenke
> >
>
>
>
> --
> Grant Henke
> Software Engineer | Cloudera
> grant@cloudera.com | twitter.com/gchenke | linkedin.com/in/granthenke
>
Re: [DISCUSSION] Kafka 0.8.2.2 release?
Posted by Gwen Shapira <gw...@confluent.io>.
Thanks for creating a list, Grant!
I placed it on the wiki with a quick evaluation of the content and whether
it should be in 0.8.2.2:
https://cwiki.apache.org/confluence/display/KAFKA/Proposed+patches+for+0.8.2.2
I'm attempting to only cherrypick fixes that are both important for large
number of users (or very critical to some users) and very safe (mostly
judged by the size of the change, but not only)
If your favorite bugfix is missing from the list, or is there but marked
"No", please let us know (in this thread) what we are missing and why it is
both important and safe.
Also, if I accidentally included something you consider unsafe, speak up!
Gwen
On Mon, Aug 17, 2015 at 8:17 AM, Grant Henke <gh...@cloudera.com> wrote:
> +dev
>
> Adding dev list back in. Somehow it got dropped.
>
>
> On Mon, Aug 17, 2015 at 10:16 AM, Grant Henke <gh...@cloudera.com> wrote:
>
> > Below is a list of candidate bug fix jiras marked fixed for 0.8.3. I
> don't
> > suspect all of these will (or should) make it into the release but this
> > should be a relatively complete list to work from:
> >
> > - KAFKA-2114 <https://issues.apache.org/jira/browse/KAFKA-2114>:
> Unable
> > to change min.insync.replicas default
> > - KAFKA-1702 <https://issues.apache.org/jira/browse/KAFKA-1702>:
> > Messages silently Lost by producer
> > - KAFKA-2012 <https://issues.apache.org/jira/browse/KAFKA-2012>:
> > Broker should automatically handle corrupt index files
> > - KAFKA-2406 <https://issues.apache.org/jira/browse/KAFKA-2406>: ISR
> > propagation should be throttled to avoid overwhelming controller.
> > - KAFKA-2336 <https://issues.apache.org/jira/browse/KAFKA-2336>:
> > Changing offsets.topic.num.partitions after the offset topic is
> created
> > breaks consumer group partition assignment
> > - KAFKA-2337 <https://issues.apache.org/jira/browse/KAFKA-2337>:
> Verify
> > that metric names will not collide when creating new topics
> > - KAFKA-2393 <https://issues.apache.org/jira/browse/KAFKA-2393>:
> > Correctly Handle InvalidTopicException in KafkaApis.getTopicMetadata()
> > - KAFKA-2189 <https://issues.apache.org/jira/browse/KAFKA-2189>:
> Snappy
> > compression of message batches less efficient in 0.8.2.1
> > - KAFKA-2308 <https://issues.apache.org/jira/browse/KAFKA-2308>: New
> > producer + Snappy face un-compression errors after broker restart
> > - KAFKA-2042 <https://issues.apache.org/jira/browse/KAFKA-2042>: New
> > producer metadata update always get all topics.
> > - KAFKA-1367 <https://issues.apache.org/jira/browse/KAFKA-1367>:
> Broker
> > topic metadata not kept in sync with ZooKeeper
> > - KAFKA-972 <https://issues.apache.org/jira/browse/KAFKA-972>:
> MetadataRequest
> > returns stale list of brokers
> > - KAFKA-1867 <https://issues.apache.org/jira/browse/KAFKA-1867>:
> liveBroker
> > list not updated on a cluster with no topics
> > - KAFKA-1650 <https://issues.apache.org/jira/browse/KAFKA-1650>:
> Mirror
> > Maker could lose data on unclean shutdown.
> > - KAFKA-2009 <https://issues.apache.org/jira/browse/KAFKA-2009>: Fix
> > UncheckedOffset.removeOffset synchronization and trace logging issue
> in
> > mirror maker
> > - KAFKA-2407 <https://issues.apache.org/jira/browse/KAFKA-2407>: Only
> > create a log directory when it will be used
> > - KAFKA-2327 <https://issues.apache.org/jira/browse/KAFKA-2327>:
> > broker doesn't start if config defines advertised.host but not
> > advertised.port
> > - KAFKA-1788: producer record can stay in RecordAccumulator forever if
> > leader is no available
> > - KAFKA-2234 <https://issues.apache.org/jira/browse/KAFKA-2234>:
> > Partition reassignment of a nonexistent topic prevents future
> reassignments
> > - KAFKA-2096 <https://issues.apache.org/jira/browse/KAFKA-2096>:
> > Enable keepalive socket option for broker to prevent socket leak
> > - KAFKA-1057 <https://issues.apache.org/jira/browse/KAFKA-1057>: Trim
> > whitespaces from user specified configs
> > - KAFKA-1641 <https://issues.apache.org/jira/browse/KAFKA-1641>: Log
> > cleaner exits if last cleaned offset is lower than earliest offset
> > - KAFKA-1648 <https://issues.apache.org/jira/browse/KAFKA-1648>:
> Round
> > robin consumer balance throws an NPE when there are no topics
> > - KAFKA-1724 <https://issues.apache.org/jira/browse/KAFKA-1724>:
> > Errors after reboot in single node setup
> > - KAFKA-1758 <https://issues.apache.org/jira/browse/KAFKA-1758>:
> > corrupt recovery file prevents startup
> > - KAFKA-1866 <https://issues.apache.org/jira/browse/KAFKA-1866>:
> > LogStartOffset gauge throws exceptions after log.delete()
> > - KAFKA-1883 <https://issues.apache.org/jira/browse/KAFKA-1883>:
> NullPointerException
> > in RequestSendThread
> > - KAFKA-1896 <https://issues.apache.org/jira/browse/KAFKA-1896>:
> > Record size funcition of record in mirror maker hit NPE when the
> message
> > value is null.
> > - KAFKA-2101 <https://issues.apache.org/jira/browse/KAFKA-2101>:
> > Metric metadata-age is reset on a failed update
> > - KAFKA-2112 <https://issues.apache.org/jira/browse/KAFKA-2112>: make
> > overflowWheel volatile
> > - KAFKA-2117 <https://issues.apache.org/jira/browse/KAFKA-2117>:
> > OffsetManager uses incorrect field for metadata
> > - KAFKA-2164 <https://issues.apache.org/jira/browse/KAFKA-2164>:
> > ReplicaFetcherThread: suspicious log message on reset offset
> > - KAFKA-1668 <https://issues.apache.org/jira/browse/KAFKA-1668>:
> > TopicCommand doesn't warn if --topic argument doesn't match any topics
> > - KAFKA-2198 <https://issues.apache.org/jira/browse/KAFKA-2198>:
> > kafka-topics.sh exits with 0 status on failures
> > - KAFKA-2235 <https://issues.apache.org/jira/browse/KAFKA-2235>:
> > LogCleaner offset map overflow
> > - KAFKA-2241 <https://issues.apache.org/jira/browse/KAFKA-2241>:
> > AbstractFetcherThread.shutdown() should not block on
> > ReadableByteChannel.read(buffer)
> > - KAFKA-2272 <https://issues.apache.org/jira/browse/KAFKA-2272>:
> > listeners endpoint parsing fails if the hostname has capital letter
> > - KAFKA-2345 <https://issues.apache.org/jira/browse/KAFKA-2345>:
> > Attempt to delete a topic already marked for deletion throws
> > ZkNodeExistsException
> > - KAFKA-2353 <https://issues.apache.org/jira/browse/KAFKA-2353>:
> > SocketServer.Processor should catch exception and close the socket
> properly
> > in configureNewConnections.
> > - KAFKA-1836 <https://issues.apache.org/jira/browse/KAFKA-1836>:
> > metadata.fetch.timeout.ms set to zero blocks forever
> > - KAFKA-2317 <https://issues.apache.org/jira/browse/KAFKA-2317>:
> De-register
> > isrChangeNotificationListener on controller resignation
> >
> > Note: KAFKA-2120 <https://issues.apache.org/jira/browse/KAFKA-2120> &
> > KAFKA-2421 <https://issues.apache.org/jira/browse/KAFKA-2421> were
> > mentioned in previous emails, but are not in the list because they are
> not
> > committed yet.
> >
> > Hope that helps the effort.
> >
> > Thanks,
> > Grant
> >
> > On Mon, Aug 17, 2015 at 12:09 AM, Grant Henke <gh...@cloudera.com>
> wrote:
> >
> >> +1 to that suggestion. Though I suspect that requires a committer to do.
> >> Making it part of the standard commit process could work too.
> >> On Aug 16, 2015 11:01 PM, "Gwen Shapira" <gw...@confluent.io> wrote:
> >>
> >>> BTW. I think it will be great for Apache Kafka to have a 0.8.2 "release
> >>> manager" who's role is to cherrypick low-risk bug-fixes into the 0.8.2
> >>> branch and once enough bug fixes happened (or if sufficiently critical
> >>> fixes happened) to roll out a new maintenance release (with every 3
> month
> >>> as a reasonable bugfix release target).
> >>>
> >>> This will add some predictability regarding how fast we release fixes
> for
> >>> bugs.
> >>>
> >>> Gwen
> >>>
> >>> On Sun, Aug 16, 2015 at 8:09 PM, Jeff Holoman <jh...@cloudera.com>
> >>> wrote:
> >>>
> >>> > +1 for the release and also including
> >>> >
> >>> > https://issues.apache.org/jira/browse/KAFKA-2114
> >>> >
> >>> > Thanks
> >>> >
> >>> > Jeff
> >>> >
> >>> > On Sun, Aug 16, 2015 at 2:51 PM, Stevo Slavić <ss...@gmail.com>
> >>> wrote:
> >>> >
> >>> > > +1 (non-binding) for 0.8.2.2 release
> >>> > >
> >>> > > Would be nice to include in that release new producer resiliency
> bug
> >>> > fixes
> >>> > > https://issues.apache.org/jira/browse/KAFKA-1788 and
> >>> > > https://issues.apache.org/jira/browse/KAFKA-2120
> >>> > >
> >>> > > On Fri, Aug 14, 2015 at 4:03 PM, Gwen Shapira <gw...@confluent.io>
> >>> wrote:
> >>> > >
> >>> > > > Will be nice to include Kafka-2308 and fix two critical snappy
> >>> issues
> >>> > in
> >>> > > > the maintenance release.
> >>> > > >
> >>> > > > Gwen
> >>> > > > On Aug 14, 2015 6:16 AM, "Grant Henke" <gh...@cloudera.com>
> >>> wrote:
> >>> > > >
> >>> > > > > Just to clarify. Will KAFKA-2189 be the only patch in the
> >>> release?
> >>> > > > >
> >>> > > > > On Fri, Aug 14, 2015 at 7:35 AM, Manikumar Reddy <
> >>> > kumar@nmsworks.co.in
> >>> > > >
> >>> > > > > wrote:
> >>> > > > >
> >>> > > > > > +1 for 0.8.2.2 release
> >>> > > > > >
> >>> > > > > > On Fri, Aug 14, 2015 at 5:49 PM, Ismael Juma <
> >>> ismael@juma.me.uk>
> >>> > > > wrote:
> >>> > > > > >
> >>> > > > > > > I think this is a good idea as the change is minimal on our
> >>> side
> >>> > > and
> >>> > > > it
> >>> > > > > > has
> >>> > > > > > > been tested in production for some time by the reporter.
> >>> > > > > > >
> >>> > > > > > > Best,
> >>> > > > > > > Ismael
> >>> > > > > > >
> >>> > > > > > > On Fri, Aug 14, 2015 at 1:15 PM, Jun Rao <jun@confluent.io
> >
> >>> > wrote:
> >>> > > > > > >
> >>> > > > > > > > Hi, Everyone,
> >>> > > > > > > >
> >>> > > > > > > > Since the release of Kafka 0.8.2.1, a number of people
> have
> >>> > > > reported
> >>> > > > > an
> >>> > > > > > > > issue with snappy compression (
> >>> > > > > > > > https://issues.apache.org/jira/browse/KAFKA-2189).
> >>> Basically,
> >>> > if
> >>> > > > > they
> >>> > > > > > > use
> >>> > > > > > > > snappy in 0.8.2.1, they will experience a 2-3X space
> >>> increase.
> >>> > > The
> >>> > > > > > issue
> >>> > > > > > > > has since been fixed in trunk (just a snappy jar
> upgrade).
> >>> > Since
> >>> > > > > 0.8.3
> >>> > > > > > is
> >>> > > > > > > > still a few months away, it may make sense to do an
> 0.8.2.2
> >>> > > release
> >>> > > > > > just
> >>> > > > > > > to
> >>> > > > > > > > fix this issue. Any objections?
> >>> > > > > > > >
> >>> > > > > > > > Thanks,
> >>> > > > > > > >
> >>> > > > > > > > Jun
> >>> > > > > > > >
> >>> > > > > > >
> >>> > > > > >
> >>> > > > >
> >>> > > > >
> >>> > > > >
> >>> > > > > --
> >>> > > > > Grant Henke
> >>> > > > > Software Engineer | Cloudera
> >>> > > > > grant@cloudera.com | twitter.com/gchenke |
> >>> > linkedin.com/in/granthenke
> >>> > > > >
> >>> > > >
> >>> > >
> >>> >
> >>> >
> >>> >
> >>> > --
> >>> > Jeff Holoman
> >>> > Systems Engineer
> >>> >
> >>>
> >>
> >
> >
> > --
> > Grant Henke
> > Software Engineer | Cloudera
> > grant@cloudera.com | twitter.com/gchenke | linkedin.com/in/granthenke
> >
>
>
>
> --
> Grant Henke
> Software Engineer | Cloudera
> grant@cloudera.com | twitter.com/gchenke | linkedin.com/in/granthenke
>
Re: [DISCUSSION] Kafka 0.8.2.2 release?
Posted by Grant Henke <gh...@cloudera.com>.
+dev
Adding dev list back in. Somehow it got dropped.
On Mon, Aug 17, 2015 at 10:16 AM, Grant Henke <gh...@cloudera.com> wrote:
> Below is a list of candidate bug fix jiras marked fixed for 0.8.3. I don't
> suspect all of these will (or should) make it into the release but this
> should be a relatively complete list to work from:
>
> - KAFKA-2114 <https://issues.apache.org/jira/browse/KAFKA-2114>: Unable
> to change min.insync.replicas default
> - KAFKA-1702 <https://issues.apache.org/jira/browse/KAFKA-1702>:
> Messages silently Lost by producer
> - KAFKA-2012 <https://issues.apache.org/jira/browse/KAFKA-2012>:
> Broker should automatically handle corrupt index files
> - KAFKA-2406 <https://issues.apache.org/jira/browse/KAFKA-2406>: ISR
> propagation should be throttled to avoid overwhelming controller.
> - KAFKA-2336 <https://issues.apache.org/jira/browse/KAFKA-2336>:
> Changing offsets.topic.num.partitions after the offset topic is created
> breaks consumer group partition assignment
> - KAFKA-2337 <https://issues.apache.org/jira/browse/KAFKA-2337>: Verify
> that metric names will not collide when creating new topics
> - KAFKA-2393 <https://issues.apache.org/jira/browse/KAFKA-2393>:
> Correctly Handle InvalidTopicException in KafkaApis.getTopicMetadata()
> - KAFKA-2189 <https://issues.apache.org/jira/browse/KAFKA-2189>: Snappy
> compression of message batches less efficient in 0.8.2.1
> - KAFKA-2308 <https://issues.apache.org/jira/browse/KAFKA-2308>: New
> producer + Snappy face un-compression errors after broker restart
> - KAFKA-2042 <https://issues.apache.org/jira/browse/KAFKA-2042>: New
> producer metadata update always get all topics.
> - KAFKA-1367 <https://issues.apache.org/jira/browse/KAFKA-1367>: Broker
> topic metadata not kept in sync with ZooKeeper
> - KAFKA-972 <https://issues.apache.org/jira/browse/KAFKA-972>: MetadataRequest
> returns stale list of brokers
> - KAFKA-1867 <https://issues.apache.org/jira/browse/KAFKA-1867>: liveBroker
> list not updated on a cluster with no topics
> - KAFKA-1650 <https://issues.apache.org/jira/browse/KAFKA-1650>: Mirror
> Maker could lose data on unclean shutdown.
> - KAFKA-2009 <https://issues.apache.org/jira/browse/KAFKA-2009>: Fix
> UncheckedOffset.removeOffset synchronization and trace logging issue in
> mirror maker
> - KAFKA-2407 <https://issues.apache.org/jira/browse/KAFKA-2407>: Only
> create a log directory when it will be used
> - KAFKA-2327 <https://issues.apache.org/jira/browse/KAFKA-2327>:
> broker doesn't start if config defines advertised.host but not
> advertised.port
> - KAFKA-1788: producer record can stay in RecordAccumulator forever if
> leader is no available
> - KAFKA-2234 <https://issues.apache.org/jira/browse/KAFKA-2234>:
> Partition reassignment of a nonexistent topic prevents future reassignments
> - KAFKA-2096 <https://issues.apache.org/jira/browse/KAFKA-2096>:
> Enable keepalive socket option for broker to prevent socket leak
> - KAFKA-1057 <https://issues.apache.org/jira/browse/KAFKA-1057>: Trim
> whitespaces from user specified configs
> - KAFKA-1641 <https://issues.apache.org/jira/browse/KAFKA-1641>: Log
> cleaner exits if last cleaned offset is lower than earliest offset
> - KAFKA-1648 <https://issues.apache.org/jira/browse/KAFKA-1648>: Round
> robin consumer balance throws an NPE when there are no topics
> - KAFKA-1724 <https://issues.apache.org/jira/browse/KAFKA-1724>:
> Errors after reboot in single node setup
> - KAFKA-1758 <https://issues.apache.org/jira/browse/KAFKA-1758>:
> corrupt recovery file prevents startup
> - KAFKA-1866 <https://issues.apache.org/jira/browse/KAFKA-1866>:
> LogStartOffset gauge throws exceptions after log.delete()
> - KAFKA-1883 <https://issues.apache.org/jira/browse/KAFKA-1883>: NullPointerException
> in RequestSendThread
> - KAFKA-1896 <https://issues.apache.org/jira/browse/KAFKA-1896>:
> Record size funcition of record in mirror maker hit NPE when the message
> value is null.
> - KAFKA-2101 <https://issues.apache.org/jira/browse/KAFKA-2101>:
> Metric metadata-age is reset on a failed update
> - KAFKA-2112 <https://issues.apache.org/jira/browse/KAFKA-2112>: make
> overflowWheel volatile
> - KAFKA-2117 <https://issues.apache.org/jira/browse/KAFKA-2117>:
> OffsetManager uses incorrect field for metadata
> - KAFKA-2164 <https://issues.apache.org/jira/browse/KAFKA-2164>:
> ReplicaFetcherThread: suspicious log message on reset offset
> - KAFKA-1668 <https://issues.apache.org/jira/browse/KAFKA-1668>:
> TopicCommand doesn't warn if --topic argument doesn't match any topics
> - KAFKA-2198 <https://issues.apache.org/jira/browse/KAFKA-2198>:
> kafka-topics.sh exits with 0 status on failures
> - KAFKA-2235 <https://issues.apache.org/jira/browse/KAFKA-2235>:
> LogCleaner offset map overflow
> - KAFKA-2241 <https://issues.apache.org/jira/browse/KAFKA-2241>:
> AbstractFetcherThread.shutdown() should not block on
> ReadableByteChannel.read(buffer)
> - KAFKA-2272 <https://issues.apache.org/jira/browse/KAFKA-2272>:
> listeners endpoint parsing fails if the hostname has capital letter
> - KAFKA-2345 <https://issues.apache.org/jira/browse/KAFKA-2345>:
> Attempt to delete a topic already marked for deletion throws
> ZkNodeExistsException
> - KAFKA-2353 <https://issues.apache.org/jira/browse/KAFKA-2353>:
> SocketServer.Processor should catch exception and close the socket properly
> in configureNewConnections.
> - KAFKA-1836 <https://issues.apache.org/jira/browse/KAFKA-1836>:
> metadata.fetch.timeout.ms set to zero blocks forever
> - KAFKA-2317 <https://issues.apache.org/jira/browse/KAFKA-2317>: De-register
> isrChangeNotificationListener on controller resignation
>
> Note: KAFKA-2120 <https://issues.apache.org/jira/browse/KAFKA-2120> &
> KAFKA-2421 <https://issues.apache.org/jira/browse/KAFKA-2421> were
> mentioned in previous emails, but are not in the list because they are not
> committed yet.
>
> Hope that helps the effort.
>
> Thanks,
> Grant
>
> On Mon, Aug 17, 2015 at 12:09 AM, Grant Henke <gh...@cloudera.com> wrote:
>
>> +1 to that suggestion. Though I suspect that requires a committer to do.
>> Making it part of the standard commit process could work too.
>> On Aug 16, 2015 11:01 PM, "Gwen Shapira" <gw...@confluent.io> wrote:
>>
>>> BTW. I think it will be great for Apache Kafka to have a 0.8.2 "release
>>> manager" who's role is to cherrypick low-risk bug-fixes into the 0.8.2
>>> branch and once enough bug fixes happened (or if sufficiently critical
>>> fixes happened) to roll out a new maintenance release (with every 3 month
>>> as a reasonable bugfix release target).
>>>
>>> This will add some predictability regarding how fast we release fixes for
>>> bugs.
>>>
>>> Gwen
>>>
>>> On Sun, Aug 16, 2015 at 8:09 PM, Jeff Holoman <jh...@cloudera.com>
>>> wrote:
>>>
>>> > +1 for the release and also including
>>> >
>>> > https://issues.apache.org/jira/browse/KAFKA-2114
>>> >
>>> > Thanks
>>> >
>>> > Jeff
>>> >
>>> > On Sun, Aug 16, 2015 at 2:51 PM, Stevo Slavić <ss...@gmail.com>
>>> wrote:
>>> >
>>> > > +1 (non-binding) for 0.8.2.2 release
>>> > >
>>> > > Would be nice to include in that release new producer resiliency bug
>>> > fixes
>>> > > https://issues.apache.org/jira/browse/KAFKA-1788 and
>>> > > https://issues.apache.org/jira/browse/KAFKA-2120
>>> > >
>>> > > On Fri, Aug 14, 2015 at 4:03 PM, Gwen Shapira <gw...@confluent.io>
>>> wrote:
>>> > >
>>> > > > Will be nice to include Kafka-2308 and fix two critical snappy
>>> issues
>>> > in
>>> > > > the maintenance release.
>>> > > >
>>> > > > Gwen
>>> > > > On Aug 14, 2015 6:16 AM, "Grant Henke" <gh...@cloudera.com>
>>> wrote:
>>> > > >
>>> > > > > Just to clarify. Will KAFKA-2189 be the only patch in the
>>> release?
>>> > > > >
>>> > > > > On Fri, Aug 14, 2015 at 7:35 AM, Manikumar Reddy <
>>> > kumar@nmsworks.co.in
>>> > > >
>>> > > > > wrote:
>>> > > > >
>>> > > > > > +1 for 0.8.2.2 release
>>> > > > > >
>>> > > > > > On Fri, Aug 14, 2015 at 5:49 PM, Ismael Juma <
>>> ismael@juma.me.uk>
>>> > > > wrote:
>>> > > > > >
>>> > > > > > > I think this is a good idea as the change is minimal on our
>>> side
>>> > > and
>>> > > > it
>>> > > > > > has
>>> > > > > > > been tested in production for some time by the reporter.
>>> > > > > > >
>>> > > > > > > Best,
>>> > > > > > > Ismael
>>> > > > > > >
>>> > > > > > > On Fri, Aug 14, 2015 at 1:15 PM, Jun Rao <ju...@confluent.io>
>>> > wrote:
>>> > > > > > >
>>> > > > > > > > Hi, Everyone,
>>> > > > > > > >
>>> > > > > > > > Since the release of Kafka 0.8.2.1, a number of people have
>>> > > > reported
>>> > > > > an
>>> > > > > > > > issue with snappy compression (
>>> > > > > > > > https://issues.apache.org/jira/browse/KAFKA-2189).
>>> Basically,
>>> > if
>>> > > > > they
>>> > > > > > > use
>>> > > > > > > > snappy in 0.8.2.1, they will experience a 2-3X space
>>> increase.
>>> > > The
>>> > > > > > issue
>>> > > > > > > > has since been fixed in trunk (just a snappy jar upgrade).
>>> > Since
>>> > > > > 0.8.3
>>> > > > > > is
>>> > > > > > > > still a few months away, it may make sense to do an 0.8.2.2
>>> > > release
>>> > > > > > just
>>> > > > > > > to
>>> > > > > > > > fix this issue. Any objections?
>>> > > > > > > >
>>> > > > > > > > Thanks,
>>> > > > > > > >
>>> > > > > > > > Jun
>>> > > > > > > >
>>> > > > > > >
>>> > > > > >
>>> > > > >
>>> > > > >
>>> > > > >
>>> > > > > --
>>> > > > > Grant Henke
>>> > > > > Software Engineer | Cloudera
>>> > > > > grant@cloudera.com | twitter.com/gchenke |
>>> > linkedin.com/in/granthenke
>>> > > > >
>>> > > >
>>> > >
>>> >
>>> >
>>> >
>>> > --
>>> > Jeff Holoman
>>> > Systems Engineer
>>> >
>>>
>>
>
>
> --
> Grant Henke
> Software Engineer | Cloudera
> grant@cloudera.com | twitter.com/gchenke | linkedin.com/in/granthenke
>
--
Grant Henke
Software Engineer | Cloudera
grant@cloudera.com | twitter.com/gchenke | linkedin.com/in/granthenke
Re: [DISCUSSION] Kafka 0.8.2.2 release?
Posted by Grant Henke <gh...@cloudera.com>.
+dev
Adding dev list back in. Somehow it got dropped.
On Mon, Aug 17, 2015 at 10:16 AM, Grant Henke <gh...@cloudera.com> wrote:
> Below is a list of candidate bug fix jiras marked fixed for 0.8.3. I don't
> suspect all of these will (or should) make it into the release but this
> should be a relatively complete list to work from:
>
> - KAFKA-2114 <https://issues.apache.org/jira/browse/KAFKA-2114>: Unable
> to change min.insync.replicas default
> - KAFKA-1702 <https://issues.apache.org/jira/browse/KAFKA-1702>:
> Messages silently Lost by producer
> - KAFKA-2012 <https://issues.apache.org/jira/browse/KAFKA-2012>:
> Broker should automatically handle corrupt index files
> - KAFKA-2406 <https://issues.apache.org/jira/browse/KAFKA-2406>: ISR
> propagation should be throttled to avoid overwhelming controller.
> - KAFKA-2336 <https://issues.apache.org/jira/browse/KAFKA-2336>:
> Changing offsets.topic.num.partitions after the offset topic is created
> breaks consumer group partition assignment
> - KAFKA-2337 <https://issues.apache.org/jira/browse/KAFKA-2337>: Verify
> that metric names will not collide when creating new topics
> - KAFKA-2393 <https://issues.apache.org/jira/browse/KAFKA-2393>:
> Correctly Handle InvalidTopicException in KafkaApis.getTopicMetadata()
> - KAFKA-2189 <https://issues.apache.org/jira/browse/KAFKA-2189>: Snappy
> compression of message batches less efficient in 0.8.2.1
> - KAFKA-2308 <https://issues.apache.org/jira/browse/KAFKA-2308>: New
> producer + Snappy face un-compression errors after broker restart
> - KAFKA-2042 <https://issues.apache.org/jira/browse/KAFKA-2042>: New
> producer metadata update always get all topics.
> - KAFKA-1367 <https://issues.apache.org/jira/browse/KAFKA-1367>: Broker
> topic metadata not kept in sync with ZooKeeper
> - KAFKA-972 <https://issues.apache.org/jira/browse/KAFKA-972>: MetadataRequest
> returns stale list of brokers
> - KAFKA-1867 <https://issues.apache.org/jira/browse/KAFKA-1867>: liveBroker
> list not updated on a cluster with no topics
> - KAFKA-1650 <https://issues.apache.org/jira/browse/KAFKA-1650>: Mirror
> Maker could lose data on unclean shutdown.
> - KAFKA-2009 <https://issues.apache.org/jira/browse/KAFKA-2009>: Fix
> UncheckedOffset.removeOffset synchronization and trace logging issue in
> mirror maker
> - KAFKA-2407 <https://issues.apache.org/jira/browse/KAFKA-2407>: Only
> create a log directory when it will be used
> - KAFKA-2327 <https://issues.apache.org/jira/browse/KAFKA-2327>:
> broker doesn't start if config defines advertised.host but not
> advertised.port
> - KAFKA-1788: producer record can stay in RecordAccumulator forever if
> leader is no available
> - KAFKA-2234 <https://issues.apache.org/jira/browse/KAFKA-2234>:
> Partition reassignment of a nonexistent topic prevents future reassignments
> - KAFKA-2096 <https://issues.apache.org/jira/browse/KAFKA-2096>:
> Enable keepalive socket option for broker to prevent socket leak
> - KAFKA-1057 <https://issues.apache.org/jira/browse/KAFKA-1057>: Trim
> whitespaces from user specified configs
> - KAFKA-1641 <https://issues.apache.org/jira/browse/KAFKA-1641>: Log
> cleaner exits if last cleaned offset is lower than earliest offset
> - KAFKA-1648 <https://issues.apache.org/jira/browse/KAFKA-1648>: Round
> robin consumer balance throws an NPE when there are no topics
> - KAFKA-1724 <https://issues.apache.org/jira/browse/KAFKA-1724>:
> Errors after reboot in single node setup
> - KAFKA-1758 <https://issues.apache.org/jira/browse/KAFKA-1758>:
> corrupt recovery file prevents startup
> - KAFKA-1866 <https://issues.apache.org/jira/browse/KAFKA-1866>:
> LogStartOffset gauge throws exceptions after log.delete()
> - KAFKA-1883 <https://issues.apache.org/jira/browse/KAFKA-1883>: NullPointerException
> in RequestSendThread
> - KAFKA-1896 <https://issues.apache.org/jira/browse/KAFKA-1896>:
> Record size funcition of record in mirror maker hit NPE when the message
> value is null.
> - KAFKA-2101 <https://issues.apache.org/jira/browse/KAFKA-2101>:
> Metric metadata-age is reset on a failed update
> - KAFKA-2112 <https://issues.apache.org/jira/browse/KAFKA-2112>: make
> overflowWheel volatile
> - KAFKA-2117 <https://issues.apache.org/jira/browse/KAFKA-2117>:
> OffsetManager uses incorrect field for metadata
> - KAFKA-2164 <https://issues.apache.org/jira/browse/KAFKA-2164>:
> ReplicaFetcherThread: suspicious log message on reset offset
> - KAFKA-1668 <https://issues.apache.org/jira/browse/KAFKA-1668>:
> TopicCommand doesn't warn if --topic argument doesn't match any topics
> - KAFKA-2198 <https://issues.apache.org/jira/browse/KAFKA-2198>:
> kafka-topics.sh exits with 0 status on failures
> - KAFKA-2235 <https://issues.apache.org/jira/browse/KAFKA-2235>:
> LogCleaner offset map overflow
> - KAFKA-2241 <https://issues.apache.org/jira/browse/KAFKA-2241>:
> AbstractFetcherThread.shutdown() should not block on
> ReadableByteChannel.read(buffer)
> - KAFKA-2272 <https://issues.apache.org/jira/browse/KAFKA-2272>:
> listeners endpoint parsing fails if the hostname has capital letter
> - KAFKA-2345 <https://issues.apache.org/jira/browse/KAFKA-2345>:
> Attempt to delete a topic already marked for deletion throws
> ZkNodeExistsException
> - KAFKA-2353 <https://issues.apache.org/jira/browse/KAFKA-2353>:
> SocketServer.Processor should catch exception and close the socket properly
> in configureNewConnections.
> - KAFKA-1836 <https://issues.apache.org/jira/browse/KAFKA-1836>:
> metadata.fetch.timeout.ms set to zero blocks forever
> - KAFKA-2317 <https://issues.apache.org/jira/browse/KAFKA-2317>: De-register
> isrChangeNotificationListener on controller resignation
>
> Note: KAFKA-2120 <https://issues.apache.org/jira/browse/KAFKA-2120> &
> KAFKA-2421 <https://issues.apache.org/jira/browse/KAFKA-2421> were
> mentioned in previous emails, but are not in the list because they are not
> committed yet.
>
> Hope that helps the effort.
>
> Thanks,
> Grant
>
> On Mon, Aug 17, 2015 at 12:09 AM, Grant Henke <gh...@cloudera.com> wrote:
>
>> +1 to that suggestion. Though I suspect that requires a committer to do.
>> Making it part of the standard commit process could work too.
>> On Aug 16, 2015 11:01 PM, "Gwen Shapira" <gw...@confluent.io> wrote:
>>
>>> BTW. I think it will be great for Apache Kafka to have a 0.8.2 "release
>>> manager" who's role is to cherrypick low-risk bug-fixes into the 0.8.2
>>> branch and once enough bug fixes happened (or if sufficiently critical
>>> fixes happened) to roll out a new maintenance release (with every 3 month
>>> as a reasonable bugfix release target).
>>>
>>> This will add some predictability regarding how fast we release fixes for
>>> bugs.
>>>
>>> Gwen
>>>
>>> On Sun, Aug 16, 2015 at 8:09 PM, Jeff Holoman <jh...@cloudera.com>
>>> wrote:
>>>
>>> > +1 for the release and also including
>>> >
>>> > https://issues.apache.org/jira/browse/KAFKA-2114
>>> >
>>> > Thanks
>>> >
>>> > Jeff
>>> >
>>> > On Sun, Aug 16, 2015 at 2:51 PM, Stevo Slavić <ss...@gmail.com>
>>> wrote:
>>> >
>>> > > +1 (non-binding) for 0.8.2.2 release
>>> > >
>>> > > Would be nice to include in that release new producer resiliency bug
>>> > fixes
>>> > > https://issues.apache.org/jira/browse/KAFKA-1788 and
>>> > > https://issues.apache.org/jira/browse/KAFKA-2120
>>> > >
>>> > > On Fri, Aug 14, 2015 at 4:03 PM, Gwen Shapira <gw...@confluent.io>
>>> wrote:
>>> > >
>>> > > > Will be nice to include Kafka-2308 and fix two critical snappy
>>> issues
>>> > in
>>> > > > the maintenance release.
>>> > > >
>>> > > > Gwen
>>> > > > On Aug 14, 2015 6:16 AM, "Grant Henke" <gh...@cloudera.com>
>>> wrote:
>>> > > >
>>> > > > > Just to clarify. Will KAFKA-2189 be the only patch in the
>>> release?
>>> > > > >
>>> > > > > On Fri, Aug 14, 2015 at 7:35 AM, Manikumar Reddy <
>>> > kumar@nmsworks.co.in
>>> > > >
>>> > > > > wrote:
>>> > > > >
>>> > > > > > +1 for 0.8.2.2 release
>>> > > > > >
>>> > > > > > On Fri, Aug 14, 2015 at 5:49 PM, Ismael Juma <
>>> ismael@juma.me.uk>
>>> > > > wrote:
>>> > > > > >
>>> > > > > > > I think this is a good idea as the change is minimal on our
>>> side
>>> > > and
>>> > > > it
>>> > > > > > has
>>> > > > > > > been tested in production for some time by the reporter.
>>> > > > > > >
>>> > > > > > > Best,
>>> > > > > > > Ismael
>>> > > > > > >
>>> > > > > > > On Fri, Aug 14, 2015 at 1:15 PM, Jun Rao <ju...@confluent.io>
>>> > wrote:
>>> > > > > > >
>>> > > > > > > > Hi, Everyone,
>>> > > > > > > >
>>> > > > > > > > Since the release of Kafka 0.8.2.1, a number of people have
>>> > > > reported
>>> > > > > an
>>> > > > > > > > issue with snappy compression (
>>> > > > > > > > https://issues.apache.org/jira/browse/KAFKA-2189).
>>> Basically,
>>> > if
>>> > > > > they
>>> > > > > > > use
>>> > > > > > > > snappy in 0.8.2.1, they will experience a 2-3X space
>>> increase.
>>> > > The
>>> > > > > > issue
>>> > > > > > > > has since been fixed in trunk (just a snappy jar upgrade).
>>> > Since
>>> > > > > 0.8.3
>>> > > > > > is
>>> > > > > > > > still a few months away, it may make sense to do an 0.8.2.2
>>> > > release
>>> > > > > > just
>>> > > > > > > to
>>> > > > > > > > fix this issue. Any objections?
>>> > > > > > > >
>>> > > > > > > > Thanks,
>>> > > > > > > >
>>> > > > > > > > Jun
>>> > > > > > > >
>>> > > > > > >
>>> > > > > >
>>> > > > >
>>> > > > >
>>> > > > >
>>> > > > > --
>>> > > > > Grant Henke
>>> > > > > Software Engineer | Cloudera
>>> > > > > grant@cloudera.com | twitter.com/gchenke |
>>> > linkedin.com/in/granthenke
>>> > > > >
>>> > > >
>>> > >
>>> >
>>> >
>>> >
>>> > --
>>> > Jeff Holoman
>>> > Systems Engineer
>>> >
>>>
>>
>
>
> --
> Grant Henke
> Software Engineer | Cloudera
> grant@cloudera.com | twitter.com/gchenke | linkedin.com/in/granthenke
>
--
Grant Henke
Software Engineer | Cloudera
grant@cloudera.com | twitter.com/gchenke | linkedin.com/in/granthenke
Re: [DISCUSSION] Kafka 0.8.2.2 release?
Posted by Grant Henke <gh...@cloudera.com>.
Below is a list of candidate bug fix jiras marked fixed for 0.8.3. I don't
suspect all of these will (or should) make it into the release but this
should be a relatively complete list to work from:
- KAFKA-2114 <https://issues.apache.org/jira/browse/KAFKA-2114>: Unable
to change min.insync.replicas default
- KAFKA-1702 <https://issues.apache.org/jira/browse/KAFKA-1702>:
Messages silently Lost by producer
- KAFKA-2012 <https://issues.apache.org/jira/browse/KAFKA-2012>: Broker
should automatically handle corrupt index files
- KAFKA-2406 <https://issues.apache.org/jira/browse/KAFKA-2406>: ISR
propagation should be throttled to avoid overwhelming controller.
- KAFKA-2336 <https://issues.apache.org/jira/browse/KAFKA-2336>:
Changing offsets.topic.num.partitions after the offset topic is created
breaks consumer group partition assignment
- KAFKA-2337 <https://issues.apache.org/jira/browse/KAFKA-2337>: Verify
that metric names will not collide when creating new topics
- KAFKA-2393 <https://issues.apache.org/jira/browse/KAFKA-2393>:
Correctly Handle InvalidTopicException in KafkaApis.getTopicMetadata()
- KAFKA-2189 <https://issues.apache.org/jira/browse/KAFKA-2189>: Snappy
compression of message batches less efficient in 0.8.2.1
- KAFKA-2308 <https://issues.apache.org/jira/browse/KAFKA-2308>: New
producer + Snappy face un-compression errors after broker restart
- KAFKA-2042 <https://issues.apache.org/jira/browse/KAFKA-2042>: New
producer metadata update always get all topics.
- KAFKA-1367 <https://issues.apache.org/jira/browse/KAFKA-1367>: Broker
topic metadata not kept in sync with ZooKeeper
- KAFKA-972 <https://issues.apache.org/jira/browse/KAFKA-972>:
MetadataRequest
returns stale list of brokers
- KAFKA-1867 <https://issues.apache.org/jira/browse/KAFKA-1867>: liveBroker
list not updated on a cluster with no topics
- KAFKA-1650 <https://issues.apache.org/jira/browse/KAFKA-1650>: Mirror
Maker could lose data on unclean shutdown.
- KAFKA-2009 <https://issues.apache.org/jira/browse/KAFKA-2009>: Fix
UncheckedOffset.removeOffset synchronization and trace logging issue in
mirror maker
- KAFKA-2407 <https://issues.apache.org/jira/browse/KAFKA-2407>: Only
create a log directory when it will be used
- KAFKA-2327 <https://issues.apache.org/jira/browse/KAFKA-2327>: broker
doesn't start if config defines advertised.host but not advertised.port
- KAFKA-1788: producer record can stay in RecordAccumulator forever if
leader is no available
- KAFKA-2234 <https://issues.apache.org/jira/browse/KAFKA-2234>:
Partition reassignment of a nonexistent topic prevents future reassignments
- KAFKA-2096 <https://issues.apache.org/jira/browse/KAFKA-2096>: Enable
keepalive socket option for broker to prevent socket leak
- KAFKA-1057 <https://issues.apache.org/jira/browse/KAFKA-1057>: Trim
whitespaces from user specified configs
- KAFKA-1641 <https://issues.apache.org/jira/browse/KAFKA-1641>: Log
cleaner exits if last cleaned offset is lower than earliest offset
- KAFKA-1648 <https://issues.apache.org/jira/browse/KAFKA-1648>: Round
robin consumer balance throws an NPE when there are no topics
- KAFKA-1724 <https://issues.apache.org/jira/browse/KAFKA-1724>: Errors
after reboot in single node setup
- KAFKA-1758 <https://issues.apache.org/jira/browse/KAFKA-1758>: corrupt
recovery file prevents startup
- KAFKA-1866 <https://issues.apache.org/jira/browse/KAFKA-1866>:
LogStartOffset gauge throws exceptions after log.delete()
- KAFKA-1883 <https://issues.apache.org/jira/browse/KAFKA-1883>:
NullPointerException
in RequestSendThread
- KAFKA-1896 <https://issues.apache.org/jira/browse/KAFKA-1896>: Record
size funcition of record in mirror maker hit NPE when the message value is
null.
- KAFKA-2101 <https://issues.apache.org/jira/browse/KAFKA-2101>: Metric
metadata-age is reset on a failed update
- KAFKA-2112 <https://issues.apache.org/jira/browse/KAFKA-2112>: make
overflowWheel volatile
- KAFKA-2117 <https://issues.apache.org/jira/browse/KAFKA-2117>:
OffsetManager uses incorrect field for metadata
- KAFKA-2164 <https://issues.apache.org/jira/browse/KAFKA-2164>:
ReplicaFetcherThread: suspicious log message on reset offset
- KAFKA-1668 <https://issues.apache.org/jira/browse/KAFKA-1668>:
TopicCommand doesn't warn if --topic argument doesn't match any topics
- KAFKA-2198 <https://issues.apache.org/jira/browse/KAFKA-2198>:
kafka-topics.sh exits with 0 status on failures
- KAFKA-2235 <https://issues.apache.org/jira/browse/KAFKA-2235>:
LogCleaner offset map overflow
- KAFKA-2241 <https://issues.apache.org/jira/browse/KAFKA-2241>:
AbstractFetcherThread.shutdown() should not block on
ReadableByteChannel.read(buffer)
- KAFKA-2272 <https://issues.apache.org/jira/browse/KAFKA-2272>:
listeners endpoint parsing fails if the hostname has capital letter
- KAFKA-2345 <https://issues.apache.org/jira/browse/KAFKA-2345>: Attempt
to delete a topic already marked for deletion throws ZkNodeExistsException
- KAFKA-2353 <https://issues.apache.org/jira/browse/KAFKA-2353>:
SocketServer.Processor should catch exception and close the socket properly
in configureNewConnections.
- KAFKA-1836 <https://issues.apache.org/jira/browse/KAFKA-1836>:
metadata.fetch.timeout.ms set to zero blocks forever
- KAFKA-2317 <https://issues.apache.org/jira/browse/KAFKA-2317>: De-register
isrChangeNotificationListener on controller resignation
Note: KAFKA-2120 <https://issues.apache.org/jira/browse/KAFKA-2120> &
KAFKA-2421 <https://issues.apache.org/jira/browse/KAFKA-2421> were
mentioned in previous emails, but are not in the list because they are not
committed yet.
Hope that helps the effort.
Thanks,
Grant
On Mon, Aug 17, 2015 at 12:09 AM, Grant Henke <gh...@cloudera.com> wrote:
> +1 to that suggestion. Though I suspect that requires a committer to do.
> Making it part of the standard commit process could work too.
> On Aug 16, 2015 11:01 PM, "Gwen Shapira" <gw...@confluent.io> wrote:
>
>> BTW. I think it will be great for Apache Kafka to have a 0.8.2 "release
>> manager" who's role is to cherrypick low-risk bug-fixes into the 0.8.2
>> branch and once enough bug fixes happened (or if sufficiently critical
>> fixes happened) to roll out a new maintenance release (with every 3 month
>> as a reasonable bugfix release target).
>>
>> This will add some predictability regarding how fast we release fixes for
>> bugs.
>>
>> Gwen
>>
>> On Sun, Aug 16, 2015 at 8:09 PM, Jeff Holoman <jh...@cloudera.com>
>> wrote:
>>
>> > +1 for the release and also including
>> >
>> > https://issues.apache.org/jira/browse/KAFKA-2114
>> >
>> > Thanks
>> >
>> > Jeff
>> >
>> > On Sun, Aug 16, 2015 at 2:51 PM, Stevo Slavić <ss...@gmail.com>
>> wrote:
>> >
>> > > +1 (non-binding) for 0.8.2.2 release
>> > >
>> > > Would be nice to include in that release new producer resiliency bug
>> > fixes
>> > > https://issues.apache.org/jira/browse/KAFKA-1788 and
>> > > https://issues.apache.org/jira/browse/KAFKA-2120
>> > >
>> > > On Fri, Aug 14, 2015 at 4:03 PM, Gwen Shapira <gw...@confluent.io>
>> wrote:
>> > >
>> > > > Will be nice to include Kafka-2308 and fix two critical snappy
>> issues
>> > in
>> > > > the maintenance release.
>> > > >
>> > > > Gwen
>> > > > On Aug 14, 2015 6:16 AM, "Grant Henke" <gh...@cloudera.com> wrote:
>> > > >
>> > > > > Just to clarify. Will KAFKA-2189 be the only patch in the release?
>> > > > >
>> > > > > On Fri, Aug 14, 2015 at 7:35 AM, Manikumar Reddy <
>> > kumar@nmsworks.co.in
>> > > >
>> > > > > wrote:
>> > > > >
>> > > > > > +1 for 0.8.2.2 release
>> > > > > >
>> > > > > > On Fri, Aug 14, 2015 at 5:49 PM, Ismael Juma <ismael@juma.me.uk
>> >
>> > > > wrote:
>> > > > > >
>> > > > > > > I think this is a good idea as the change is minimal on our
>> side
>> > > and
>> > > > it
>> > > > > > has
>> > > > > > > been tested in production for some time by the reporter.
>> > > > > > >
>> > > > > > > Best,
>> > > > > > > Ismael
>> > > > > > >
>> > > > > > > On Fri, Aug 14, 2015 at 1:15 PM, Jun Rao <ju...@confluent.io>
>> > wrote:
>> > > > > > >
>> > > > > > > > Hi, Everyone,
>> > > > > > > >
>> > > > > > > > Since the release of Kafka 0.8.2.1, a number of people have
>> > > > reported
>> > > > > an
>> > > > > > > > issue with snappy compression (
>> > > > > > > > https://issues.apache.org/jira/browse/KAFKA-2189).
>> Basically,
>> > if
>> > > > > they
>> > > > > > > use
>> > > > > > > > snappy in 0.8.2.1, they will experience a 2-3X space
>> increase.
>> > > The
>> > > > > > issue
>> > > > > > > > has since been fixed in trunk (just a snappy jar upgrade).
>> > Since
>> > > > > 0.8.3
>> > > > > > is
>> > > > > > > > still a few months away, it may make sense to do an 0.8.2.2
>> > > release
>> > > > > > just
>> > > > > > > to
>> > > > > > > > fix this issue. Any objections?
>> > > > > > > >
>> > > > > > > > Thanks,
>> > > > > > > >
>> > > > > > > > Jun
>> > > > > > > >
>> > > > > > >
>> > > > > >
>> > > > >
>> > > > >
>> > > > >
>> > > > > --
>> > > > > Grant Henke
>> > > > > Software Engineer | Cloudera
>> > > > > grant@cloudera.com | twitter.com/gchenke |
>> > linkedin.com/in/granthenke
>> > > > >
>> > > >
>> > >
>> >
>> >
>> >
>> > --
>> > Jeff Holoman
>> > Systems Engineer
>> >
>>
>
--
Grant Henke
Software Engineer | Cloudera
grant@cloudera.com | twitter.com/gchenke | linkedin.com/in/granthenke
Re: [DISCUSSION] Kafka 0.8.2.2 release?
Posted by Grant Henke <gh...@cloudera.com>.
+1 to that suggestion. Though I suspect that requires a committer to do.
Making it part of the standard commit process could work too.
On Aug 16, 2015 11:01 PM, "Gwen Shapira" <gw...@confluent.io> wrote:
> BTW. I think it will be great for Apache Kafka to have a 0.8.2 "release
> manager" who's role is to cherrypick low-risk bug-fixes into the 0.8.2
> branch and once enough bug fixes happened (or if sufficiently critical
> fixes happened) to roll out a new maintenance release (with every 3 month
> as a reasonable bugfix release target).
>
> This will add some predictability regarding how fast we release fixes for
> bugs.
>
> Gwen
>
> On Sun, Aug 16, 2015 at 8:09 PM, Jeff Holoman <jh...@cloudera.com>
> wrote:
>
> > +1 for the release and also including
> >
> > https://issues.apache.org/jira/browse/KAFKA-2114
> >
> > Thanks
> >
> > Jeff
> >
> > On Sun, Aug 16, 2015 at 2:51 PM, Stevo Slavić <ss...@gmail.com> wrote:
> >
> > > +1 (non-binding) for 0.8.2.2 release
> > >
> > > Would be nice to include in that release new producer resiliency bug
> > fixes
> > > https://issues.apache.org/jira/browse/KAFKA-1788 and
> > > https://issues.apache.org/jira/browse/KAFKA-2120
> > >
> > > On Fri, Aug 14, 2015 at 4:03 PM, Gwen Shapira <gw...@confluent.io>
> wrote:
> > >
> > > > Will be nice to include Kafka-2308 and fix two critical snappy issues
> > in
> > > > the maintenance release.
> > > >
> > > > Gwen
> > > > On Aug 14, 2015 6:16 AM, "Grant Henke" <gh...@cloudera.com> wrote:
> > > >
> > > > > Just to clarify. Will KAFKA-2189 be the only patch in the release?
> > > > >
> > > > > On Fri, Aug 14, 2015 at 7:35 AM, Manikumar Reddy <
> > kumar@nmsworks.co.in
> > > >
> > > > > wrote:
> > > > >
> > > > > > +1 for 0.8.2.2 release
> > > > > >
> > > > > > On Fri, Aug 14, 2015 at 5:49 PM, Ismael Juma <is...@juma.me.uk>
> > > > wrote:
> > > > > >
> > > > > > > I think this is a good idea as the change is minimal on our
> side
> > > and
> > > > it
> > > > > > has
> > > > > > > been tested in production for some time by the reporter.
> > > > > > >
> > > > > > > Best,
> > > > > > > Ismael
> > > > > > >
> > > > > > > On Fri, Aug 14, 2015 at 1:15 PM, Jun Rao <ju...@confluent.io>
> > wrote:
> > > > > > >
> > > > > > > > Hi, Everyone,
> > > > > > > >
> > > > > > > > Since the release of Kafka 0.8.2.1, a number of people have
> > > > reported
> > > > > an
> > > > > > > > issue with snappy compression (
> > > > > > > > https://issues.apache.org/jira/browse/KAFKA-2189).
> Basically,
> > if
> > > > > they
> > > > > > > use
> > > > > > > > snappy in 0.8.2.1, they will experience a 2-3X space
> increase.
> > > The
> > > > > > issue
> > > > > > > > has since been fixed in trunk (just a snappy jar upgrade).
> > Since
> > > > > 0.8.3
> > > > > > is
> > > > > > > > still a few months away, it may make sense to do an 0.8.2.2
> > > release
> > > > > > just
> > > > > > > to
> > > > > > > > fix this issue. Any objections?
> > > > > > > >
> > > > > > > > Thanks,
> > > > > > > >
> > > > > > > > Jun
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > > >
> > > > >
> > > > > --
> > > > > Grant Henke
> > > > > Software Engineer | Cloudera
> > > > > grant@cloudera.com | twitter.com/gchenke |
> > linkedin.com/in/granthenke
> > > > >
> > > >
> > >
> >
> >
> >
> > --
> > Jeff Holoman
> > Systems Engineer
> >
>
Re: [DISCUSSION] Kafka 0.8.2.2 release?
Posted by Gwen Shapira <gw...@confluent.io>.
BTW. I think it will be great for Apache Kafka to have a 0.8.2 "release
manager" who's role is to cherrypick low-risk bug-fixes into the 0.8.2
branch and once enough bug fixes happened (or if sufficiently critical
fixes happened) to roll out a new maintenance release (with every 3 month
as a reasonable bugfix release target).
This will add some predictability regarding how fast we release fixes for
bugs.
Gwen
On Sun, Aug 16, 2015 at 8:09 PM, Jeff Holoman <jh...@cloudera.com> wrote:
> +1 for the release and also including
>
> https://issues.apache.org/jira/browse/KAFKA-2114
>
> Thanks
>
> Jeff
>
> On Sun, Aug 16, 2015 at 2:51 PM, Stevo Slavić <ss...@gmail.com> wrote:
>
> > +1 (non-binding) for 0.8.2.2 release
> >
> > Would be nice to include in that release new producer resiliency bug
> fixes
> > https://issues.apache.org/jira/browse/KAFKA-1788 and
> > https://issues.apache.org/jira/browse/KAFKA-2120
> >
> > On Fri, Aug 14, 2015 at 4:03 PM, Gwen Shapira <gw...@confluent.io> wrote:
> >
> > > Will be nice to include Kafka-2308 and fix two critical snappy issues
> in
> > > the maintenance release.
> > >
> > > Gwen
> > > On Aug 14, 2015 6:16 AM, "Grant Henke" <gh...@cloudera.com> wrote:
> > >
> > > > Just to clarify. Will KAFKA-2189 be the only patch in the release?
> > > >
> > > > On Fri, Aug 14, 2015 at 7:35 AM, Manikumar Reddy <
> kumar@nmsworks.co.in
> > >
> > > > wrote:
> > > >
> > > > > +1 for 0.8.2.2 release
> > > > >
> > > > > On Fri, Aug 14, 2015 at 5:49 PM, Ismael Juma <is...@juma.me.uk>
> > > wrote:
> > > > >
> > > > > > I think this is a good idea as the change is minimal on our side
> > and
> > > it
> > > > > has
> > > > > > been tested in production for some time by the reporter.
> > > > > >
> > > > > > Best,
> > > > > > Ismael
> > > > > >
> > > > > > On Fri, Aug 14, 2015 at 1:15 PM, Jun Rao <ju...@confluent.io>
> wrote:
> > > > > >
> > > > > > > Hi, Everyone,
> > > > > > >
> > > > > > > Since the release of Kafka 0.8.2.1, a number of people have
> > > reported
> > > > an
> > > > > > > issue with snappy compression (
> > > > > > > https://issues.apache.org/jira/browse/KAFKA-2189). Basically,
> if
> > > > they
> > > > > > use
> > > > > > > snappy in 0.8.2.1, they will experience a 2-3X space increase.
> > The
> > > > > issue
> > > > > > > has since been fixed in trunk (just a snappy jar upgrade).
> Since
> > > > 0.8.3
> > > > > is
> > > > > > > still a few months away, it may make sense to do an 0.8.2.2
> > release
> > > > > just
> > > > > > to
> > > > > > > fix this issue. Any objections?
> > > > > > >
> > > > > > > Thanks,
> > > > > > >
> > > > > > > Jun
> > > > > > >
> > > > > >
> > > > >
> > > >
> > > >
> > > >
> > > > --
> > > > Grant Henke
> > > > Software Engineer | Cloudera
> > > > grant@cloudera.com | twitter.com/gchenke |
> linkedin.com/in/granthenke
> > > >
> > >
> >
>
>
>
> --
> Jeff Holoman
> Systems Engineer
>
Re: [DISCUSSION] Kafka 0.8.2.2 release?
Posted by Gwen Shapira <gw...@confluent.io>.
BTW. I think it will be great for Apache Kafka to have a 0.8.2 "release
manager" who's role is to cherrypick low-risk bug-fixes into the 0.8.2
branch and once enough bug fixes happened (or if sufficiently critical
fixes happened) to roll out a new maintenance release (with every 3 month
as a reasonable bugfix release target).
This will add some predictability regarding how fast we release fixes for
bugs.
Gwen
On Sun, Aug 16, 2015 at 8:09 PM, Jeff Holoman <jh...@cloudera.com> wrote:
> +1 for the release and also including
>
> https://issues.apache.org/jira/browse/KAFKA-2114
>
> Thanks
>
> Jeff
>
> On Sun, Aug 16, 2015 at 2:51 PM, Stevo Slavić <ss...@gmail.com> wrote:
>
> > +1 (non-binding) for 0.8.2.2 release
> >
> > Would be nice to include in that release new producer resiliency bug
> fixes
> > https://issues.apache.org/jira/browse/KAFKA-1788 and
> > https://issues.apache.org/jira/browse/KAFKA-2120
> >
> > On Fri, Aug 14, 2015 at 4:03 PM, Gwen Shapira <gw...@confluent.io> wrote:
> >
> > > Will be nice to include Kafka-2308 and fix two critical snappy issues
> in
> > > the maintenance release.
> > >
> > > Gwen
> > > On Aug 14, 2015 6:16 AM, "Grant Henke" <gh...@cloudera.com> wrote:
> > >
> > > > Just to clarify. Will KAFKA-2189 be the only patch in the release?
> > > >
> > > > On Fri, Aug 14, 2015 at 7:35 AM, Manikumar Reddy <
> kumar@nmsworks.co.in
> > >
> > > > wrote:
> > > >
> > > > > +1 for 0.8.2.2 release
> > > > >
> > > > > On Fri, Aug 14, 2015 at 5:49 PM, Ismael Juma <is...@juma.me.uk>
> > > wrote:
> > > > >
> > > > > > I think this is a good idea as the change is minimal on our side
> > and
> > > it
> > > > > has
> > > > > > been tested in production for some time by the reporter.
> > > > > >
> > > > > > Best,
> > > > > > Ismael
> > > > > >
> > > > > > On Fri, Aug 14, 2015 at 1:15 PM, Jun Rao <ju...@confluent.io>
> wrote:
> > > > > >
> > > > > > > Hi, Everyone,
> > > > > > >
> > > > > > > Since the release of Kafka 0.8.2.1, a number of people have
> > > reported
> > > > an
> > > > > > > issue with snappy compression (
> > > > > > > https://issues.apache.org/jira/browse/KAFKA-2189). Basically,
> if
> > > > they
> > > > > > use
> > > > > > > snappy in 0.8.2.1, they will experience a 2-3X space increase.
> > The
> > > > > issue
> > > > > > > has since been fixed in trunk (just a snappy jar upgrade).
> Since
> > > > 0.8.3
> > > > > is
> > > > > > > still a few months away, it may make sense to do an 0.8.2.2
> > release
> > > > > just
> > > > > > to
> > > > > > > fix this issue. Any objections?
> > > > > > >
> > > > > > > Thanks,
> > > > > > >
> > > > > > > Jun
> > > > > > >
> > > > > >
> > > > >
> > > >
> > > >
> > > >
> > > > --
> > > > Grant Henke
> > > > Software Engineer | Cloudera
> > > > grant@cloudera.com | twitter.com/gchenke |
> linkedin.com/in/granthenke
> > > >
> > >
> >
>
>
>
> --
> Jeff Holoman
> Systems Engineer
>
Re: [DISCUSSION] Kafka 0.8.2.2 release?
Posted by Jeff Holoman <jh...@cloudera.com>.
+1 for the release and also including
https://issues.apache.org/jira/browse/KAFKA-2114
Thanks
Jeff
On Sun, Aug 16, 2015 at 2:51 PM, Stevo Slavić <ss...@gmail.com> wrote:
> +1 (non-binding) for 0.8.2.2 release
>
> Would be nice to include in that release new producer resiliency bug fixes
> https://issues.apache.org/jira/browse/KAFKA-1788 and
> https://issues.apache.org/jira/browse/KAFKA-2120
>
> On Fri, Aug 14, 2015 at 4:03 PM, Gwen Shapira <gw...@confluent.io> wrote:
>
> > Will be nice to include Kafka-2308 and fix two critical snappy issues in
> > the maintenance release.
> >
> > Gwen
> > On Aug 14, 2015 6:16 AM, "Grant Henke" <gh...@cloudera.com> wrote:
> >
> > > Just to clarify. Will KAFKA-2189 be the only patch in the release?
> > >
> > > On Fri, Aug 14, 2015 at 7:35 AM, Manikumar Reddy <kumar@nmsworks.co.in
> >
> > > wrote:
> > >
> > > > +1 for 0.8.2.2 release
> > > >
> > > > On Fri, Aug 14, 2015 at 5:49 PM, Ismael Juma <is...@juma.me.uk>
> > wrote:
> > > >
> > > > > I think this is a good idea as the change is minimal on our side
> and
> > it
> > > > has
> > > > > been tested in production for some time by the reporter.
> > > > >
> > > > > Best,
> > > > > Ismael
> > > > >
> > > > > On Fri, Aug 14, 2015 at 1:15 PM, Jun Rao <ju...@confluent.io> wrote:
> > > > >
> > > > > > Hi, Everyone,
> > > > > >
> > > > > > Since the release of Kafka 0.8.2.1, a number of people have
> > reported
> > > an
> > > > > > issue with snappy compression (
> > > > > > https://issues.apache.org/jira/browse/KAFKA-2189). Basically, if
> > > they
> > > > > use
> > > > > > snappy in 0.8.2.1, they will experience a 2-3X space increase.
> The
> > > > issue
> > > > > > has since been fixed in trunk (just a snappy jar upgrade). Since
> > > 0.8.3
> > > > is
> > > > > > still a few months away, it may make sense to do an 0.8.2.2
> release
> > > > just
> > > > > to
> > > > > > fix this issue. Any objections?
> > > > > >
> > > > > > Thanks,
> > > > > >
> > > > > > Jun
> > > > > >
> > > > >
> > > >
> > >
> > >
> > >
> > > --
> > > Grant Henke
> > > Software Engineer | Cloudera
> > > grant@cloudera.com | twitter.com/gchenke | linkedin.com/in/granthenke
> > >
> >
>
--
Jeff Holoman
Systems Engineer
Re: [DISCUSSION] Kafka 0.8.2.2 release?
Posted by Jeff Holoman <jh...@cloudera.com>.
+1 for the release and also including
https://issues.apache.org/jira/browse/KAFKA-2114
Thanks
Jeff
On Sun, Aug 16, 2015 at 2:51 PM, Stevo Slavić <ss...@gmail.com> wrote:
> +1 (non-binding) for 0.8.2.2 release
>
> Would be nice to include in that release new producer resiliency bug fixes
> https://issues.apache.org/jira/browse/KAFKA-1788 and
> https://issues.apache.org/jira/browse/KAFKA-2120
>
> On Fri, Aug 14, 2015 at 4:03 PM, Gwen Shapira <gw...@confluent.io> wrote:
>
> > Will be nice to include Kafka-2308 and fix two critical snappy issues in
> > the maintenance release.
> >
> > Gwen
> > On Aug 14, 2015 6:16 AM, "Grant Henke" <gh...@cloudera.com> wrote:
> >
> > > Just to clarify. Will KAFKA-2189 be the only patch in the release?
> > >
> > > On Fri, Aug 14, 2015 at 7:35 AM, Manikumar Reddy <kumar@nmsworks.co.in
> >
> > > wrote:
> > >
> > > > +1 for 0.8.2.2 release
> > > >
> > > > On Fri, Aug 14, 2015 at 5:49 PM, Ismael Juma <is...@juma.me.uk>
> > wrote:
> > > >
> > > > > I think this is a good idea as the change is minimal on our side
> and
> > it
> > > > has
> > > > > been tested in production for some time by the reporter.
> > > > >
> > > > > Best,
> > > > > Ismael
> > > > >
> > > > > On Fri, Aug 14, 2015 at 1:15 PM, Jun Rao <ju...@confluent.io> wrote:
> > > > >
> > > > > > Hi, Everyone,
> > > > > >
> > > > > > Since the release of Kafka 0.8.2.1, a number of people have
> > reported
> > > an
> > > > > > issue with snappy compression (
> > > > > > https://issues.apache.org/jira/browse/KAFKA-2189). Basically, if
> > > they
> > > > > use
> > > > > > snappy in 0.8.2.1, they will experience a 2-3X space increase.
> The
> > > > issue
> > > > > > has since been fixed in trunk (just a snappy jar upgrade). Since
> > > 0.8.3
> > > > is
> > > > > > still a few months away, it may make sense to do an 0.8.2.2
> release
> > > > just
> > > > > to
> > > > > > fix this issue. Any objections?
> > > > > >
> > > > > > Thanks,
> > > > > >
> > > > > > Jun
> > > > > >
> > > > >
> > > >
> > >
> > >
> > >
> > > --
> > > Grant Henke
> > > Software Engineer | Cloudera
> > > grant@cloudera.com | twitter.com/gchenke | linkedin.com/in/granthenke
> > >
> >
>
--
Jeff Holoman
Systems Engineer
Re: [DISCUSSION] Kafka 0.8.2.2 release?
Posted by Stevo Slavić <ss...@gmail.com>.
+1 (non-binding) for 0.8.2.2 release
Would be nice to include in that release new producer resiliency bug fixes
https://issues.apache.org/jira/browse/KAFKA-1788 and
https://issues.apache.org/jira/browse/KAFKA-2120
On Fri, Aug 14, 2015 at 4:03 PM, Gwen Shapira <gw...@confluent.io> wrote:
> Will be nice to include Kafka-2308 and fix two critical snappy issues in
> the maintenance release.
>
> Gwen
> On Aug 14, 2015 6:16 AM, "Grant Henke" <gh...@cloudera.com> wrote:
>
> > Just to clarify. Will KAFKA-2189 be the only patch in the release?
> >
> > On Fri, Aug 14, 2015 at 7:35 AM, Manikumar Reddy <ku...@nmsworks.co.in>
> > wrote:
> >
> > > +1 for 0.8.2.2 release
> > >
> > > On Fri, Aug 14, 2015 at 5:49 PM, Ismael Juma <is...@juma.me.uk>
> wrote:
> > >
> > > > I think this is a good idea as the change is minimal on our side and
> it
> > > has
> > > > been tested in production for some time by the reporter.
> > > >
> > > > Best,
> > > > Ismael
> > > >
> > > > On Fri, Aug 14, 2015 at 1:15 PM, Jun Rao <ju...@confluent.io> wrote:
> > > >
> > > > > Hi, Everyone,
> > > > >
> > > > > Since the release of Kafka 0.8.2.1, a number of people have
> reported
> > an
> > > > > issue with snappy compression (
> > > > > https://issues.apache.org/jira/browse/KAFKA-2189). Basically, if
> > they
> > > > use
> > > > > snappy in 0.8.2.1, they will experience a 2-3X space increase. The
> > > issue
> > > > > has since been fixed in trunk (just a snappy jar upgrade). Since
> > 0.8.3
> > > is
> > > > > still a few months away, it may make sense to do an 0.8.2.2 release
> > > just
> > > > to
> > > > > fix this issue. Any objections?
> > > > >
> > > > > Thanks,
> > > > >
> > > > > Jun
> > > > >
> > > >
> > >
> >
> >
> >
> > --
> > Grant Henke
> > Software Engineer | Cloudera
> > grant@cloudera.com | twitter.com/gchenke | linkedin.com/in/granthenke
> >
>
Re: [DISCUSSION] Kafka 0.8.2.2 release?
Posted by Stevo Slavić <ss...@gmail.com>.
+1 (non-binding) for 0.8.2.2 release
Would be nice to include in that release new producer resiliency bug fixes
https://issues.apache.org/jira/browse/KAFKA-1788 and
https://issues.apache.org/jira/browse/KAFKA-2120
On Fri, Aug 14, 2015 at 4:03 PM, Gwen Shapira <gw...@confluent.io> wrote:
> Will be nice to include Kafka-2308 and fix two critical snappy issues in
> the maintenance release.
>
> Gwen
> On Aug 14, 2015 6:16 AM, "Grant Henke" <gh...@cloudera.com> wrote:
>
> > Just to clarify. Will KAFKA-2189 be the only patch in the release?
> >
> > On Fri, Aug 14, 2015 at 7:35 AM, Manikumar Reddy <ku...@nmsworks.co.in>
> > wrote:
> >
> > > +1 for 0.8.2.2 release
> > >
> > > On Fri, Aug 14, 2015 at 5:49 PM, Ismael Juma <is...@juma.me.uk>
> wrote:
> > >
> > > > I think this is a good idea as the change is minimal on our side and
> it
> > > has
> > > > been tested in production for some time by the reporter.
> > > >
> > > > Best,
> > > > Ismael
> > > >
> > > > On Fri, Aug 14, 2015 at 1:15 PM, Jun Rao <ju...@confluent.io> wrote:
> > > >
> > > > > Hi, Everyone,
> > > > >
> > > > > Since the release of Kafka 0.8.2.1, a number of people have
> reported
> > an
> > > > > issue with snappy compression (
> > > > > https://issues.apache.org/jira/browse/KAFKA-2189). Basically, if
> > they
> > > > use
> > > > > snappy in 0.8.2.1, they will experience a 2-3X space increase. The
> > > issue
> > > > > has since been fixed in trunk (just a snappy jar upgrade). Since
> > 0.8.3
> > > is
> > > > > still a few months away, it may make sense to do an 0.8.2.2 release
> > > just
> > > > to
> > > > > fix this issue. Any objections?
> > > > >
> > > > > Thanks,
> > > > >
> > > > > Jun
> > > > >
> > > >
> > >
> >
> >
> >
> > --
> > Grant Henke
> > Software Engineer | Cloudera
> > grant@cloudera.com | twitter.com/gchenke | linkedin.com/in/granthenke
> >
>
Re: [DISCUSSION] Kafka 0.8.2.2 release?
Posted by Gwen Shapira <gw...@confluent.io>.
Will be nice to include Kafka-2308 and fix two critical snappy issues in
the maintenance release.
Gwen
On Aug 14, 2015 6:16 AM, "Grant Henke" <gh...@cloudera.com> wrote:
> Just to clarify. Will KAFKA-2189 be the only patch in the release?
>
> On Fri, Aug 14, 2015 at 7:35 AM, Manikumar Reddy <ku...@nmsworks.co.in>
> wrote:
>
> > +1 for 0.8.2.2 release
> >
> > On Fri, Aug 14, 2015 at 5:49 PM, Ismael Juma <is...@juma.me.uk> wrote:
> >
> > > I think this is a good idea as the change is minimal on our side and it
> > has
> > > been tested in production for some time by the reporter.
> > >
> > > Best,
> > > Ismael
> > >
> > > On Fri, Aug 14, 2015 at 1:15 PM, Jun Rao <ju...@confluent.io> wrote:
> > >
> > > > Hi, Everyone,
> > > >
> > > > Since the release of Kafka 0.8.2.1, a number of people have reported
> an
> > > > issue with snappy compression (
> > > > https://issues.apache.org/jira/browse/KAFKA-2189). Basically, if
> they
> > > use
> > > > snappy in 0.8.2.1, they will experience a 2-3X space increase. The
> > issue
> > > > has since been fixed in trunk (just a snappy jar upgrade). Since
> 0.8.3
> > is
> > > > still a few months away, it may make sense to do an 0.8.2.2 release
> > just
> > > to
> > > > fix this issue. Any objections?
> > > >
> > > > Thanks,
> > > >
> > > > Jun
> > > >
> > >
> >
>
>
>
> --
> Grant Henke
> Software Engineer | Cloudera
> grant@cloudera.com | twitter.com/gchenke | linkedin.com/in/granthenke
>
Re: [DISCUSSION] Kafka 0.8.2.2 release?
Posted by Gwen Shapira <gw...@confluent.io>.
Will be nice to include Kafka-2308 and fix two critical snappy issues in
the maintenance release.
Gwen
On Aug 14, 2015 6:16 AM, "Grant Henke" <gh...@cloudera.com> wrote:
> Just to clarify. Will KAFKA-2189 be the only patch in the release?
>
> On Fri, Aug 14, 2015 at 7:35 AM, Manikumar Reddy <ku...@nmsworks.co.in>
> wrote:
>
> > +1 for 0.8.2.2 release
> >
> > On Fri, Aug 14, 2015 at 5:49 PM, Ismael Juma <is...@juma.me.uk> wrote:
> >
> > > I think this is a good idea as the change is minimal on our side and it
> > has
> > > been tested in production for some time by the reporter.
> > >
> > > Best,
> > > Ismael
> > >
> > > On Fri, Aug 14, 2015 at 1:15 PM, Jun Rao <ju...@confluent.io> wrote:
> > >
> > > > Hi, Everyone,
> > > >
> > > > Since the release of Kafka 0.8.2.1, a number of people have reported
> an
> > > > issue with snappy compression (
> > > > https://issues.apache.org/jira/browse/KAFKA-2189). Basically, if
> they
> > > use
> > > > snappy in 0.8.2.1, they will experience a 2-3X space increase. The
> > issue
> > > > has since been fixed in trunk (just a snappy jar upgrade). Since
> 0.8.3
> > is
> > > > still a few months away, it may make sense to do an 0.8.2.2 release
> > just
> > > to
> > > > fix this issue. Any objections?
> > > >
> > > > Thanks,
> > > >
> > > > Jun
> > > >
> > >
> >
>
>
>
> --
> Grant Henke
> Software Engineer | Cloudera
> grant@cloudera.com | twitter.com/gchenke | linkedin.com/in/granthenke
>
Re: [DISCUSSION] Kafka 0.8.2.2 release?
Posted by Grant Henke <gh...@cloudera.com>.
Just to clarify. Will KAFKA-2189 be the only patch in the release?
On Fri, Aug 14, 2015 at 7:35 AM, Manikumar Reddy <ku...@nmsworks.co.in>
wrote:
> +1 for 0.8.2.2 release
>
> On Fri, Aug 14, 2015 at 5:49 PM, Ismael Juma <is...@juma.me.uk> wrote:
>
> > I think this is a good idea as the change is minimal on our side and it
> has
> > been tested in production for some time by the reporter.
> >
> > Best,
> > Ismael
> >
> > On Fri, Aug 14, 2015 at 1:15 PM, Jun Rao <ju...@confluent.io> wrote:
> >
> > > Hi, Everyone,
> > >
> > > Since the release of Kafka 0.8.2.1, a number of people have reported an
> > > issue with snappy compression (
> > > https://issues.apache.org/jira/browse/KAFKA-2189). Basically, if they
> > use
> > > snappy in 0.8.2.1, they will experience a 2-3X space increase. The
> issue
> > > has since been fixed in trunk (just a snappy jar upgrade). Since 0.8.3
> is
> > > still a few months away, it may make sense to do an 0.8.2.2 release
> just
> > to
> > > fix this issue. Any objections?
> > >
> > > Thanks,
> > >
> > > Jun
> > >
> >
>
--
Grant Henke
Software Engineer | Cloudera
grant@cloudera.com | twitter.com/gchenke | linkedin.com/in/granthenke
Re: [DISCUSSION] Kafka 0.8.2.2 release?
Posted by Grant Henke <gh...@cloudera.com>.
Just to clarify. Will KAFKA-2189 be the only patch in the release?
On Fri, Aug 14, 2015 at 7:35 AM, Manikumar Reddy <ku...@nmsworks.co.in>
wrote:
> +1 for 0.8.2.2 release
>
> On Fri, Aug 14, 2015 at 5:49 PM, Ismael Juma <is...@juma.me.uk> wrote:
>
> > I think this is a good idea as the change is minimal on our side and it
> has
> > been tested in production for some time by the reporter.
> >
> > Best,
> > Ismael
> >
> > On Fri, Aug 14, 2015 at 1:15 PM, Jun Rao <ju...@confluent.io> wrote:
> >
> > > Hi, Everyone,
> > >
> > > Since the release of Kafka 0.8.2.1, a number of people have reported an
> > > issue with snappy compression (
> > > https://issues.apache.org/jira/browse/KAFKA-2189). Basically, if they
> > use
> > > snappy in 0.8.2.1, they will experience a 2-3X space increase. The
> issue
> > > has since been fixed in trunk (just a snappy jar upgrade). Since 0.8.3
> is
> > > still a few months away, it may make sense to do an 0.8.2.2 release
> just
> > to
> > > fix this issue. Any objections?
> > >
> > > Thanks,
> > >
> > > Jun
> > >
> >
>
--
Grant Henke
Software Engineer | Cloudera
grant@cloudera.com | twitter.com/gchenke | linkedin.com/in/granthenke
Re: [DISCUSSION] Kafka 0.8.2.2 release?
Posted by Manikumar Reddy <ku...@nmsworks.co.in>.
+1 for 0.8.2.2 release
On Fri, Aug 14, 2015 at 5:49 PM, Ismael Juma <is...@juma.me.uk> wrote:
> I think this is a good idea as the change is minimal on our side and it has
> been tested in production for some time by the reporter.
>
> Best,
> Ismael
>
> On Fri, Aug 14, 2015 at 1:15 PM, Jun Rao <ju...@confluent.io> wrote:
>
> > Hi, Everyone,
> >
> > Since the release of Kafka 0.8.2.1, a number of people have reported an
> > issue with snappy compression (
> > https://issues.apache.org/jira/browse/KAFKA-2189). Basically, if they
> use
> > snappy in 0.8.2.1, they will experience a 2-3X space increase. The issue
> > has since been fixed in trunk (just a snappy jar upgrade). Since 0.8.3 is
> > still a few months away, it may make sense to do an 0.8.2.2 release just
> to
> > fix this issue. Any objections?
> >
> > Thanks,
> >
> > Jun
> >
>
Re: [DISCUSSION] Kafka 0.8.2.2 release?
Posted by Manikumar Reddy <ku...@nmsworks.co.in>.
+1 for 0.8.2.2 release
On Fri, Aug 14, 2015 at 5:49 PM, Ismael Juma <is...@juma.me.uk> wrote:
> I think this is a good idea as the change is minimal on our side and it has
> been tested in production for some time by the reporter.
>
> Best,
> Ismael
>
> On Fri, Aug 14, 2015 at 1:15 PM, Jun Rao <ju...@confluent.io> wrote:
>
> > Hi, Everyone,
> >
> > Since the release of Kafka 0.8.2.1, a number of people have reported an
> > issue with snappy compression (
> > https://issues.apache.org/jira/browse/KAFKA-2189). Basically, if they
> use
> > snappy in 0.8.2.1, they will experience a 2-3X space increase. The issue
> > has since been fixed in trunk (just a snappy jar upgrade). Since 0.8.3 is
> > still a few months away, it may make sense to do an 0.8.2.2 release just
> to
> > fix this issue. Any objections?
> >
> > Thanks,
> >
> > Jun
> >
>
Re: [DISCUSSION] Kafka 0.8.2.2 release?
Posted by Ismael Juma <is...@juma.me.uk>.
I think this is a good idea as the change is minimal on our side and it has
been tested in production for some time by the reporter.
Best,
Ismael
On Fri, Aug 14, 2015 at 1:15 PM, Jun Rao <ju...@confluent.io> wrote:
> Hi, Everyone,
>
> Since the release of Kafka 0.8.2.1, a number of people have reported an
> issue with snappy compression (
> https://issues.apache.org/jira/browse/KAFKA-2189). Basically, if they use
> snappy in 0.8.2.1, they will experience a 2-3X space increase. The issue
> has since been fixed in trunk (just a snappy jar upgrade). Since 0.8.3 is
> still a few months away, it may make sense to do an 0.8.2.2 release just to
> fix this issue. Any objections?
>
> Thanks,
>
> Jun
>
Re: [DISCUSSION] Kafka 0.8.2.2 release?
Posted by Ismael Juma <is...@juma.me.uk>.
I think this is a good idea as the change is minimal on our side and it has
been tested in production for some time by the reporter.
Best,
Ismael
On Fri, Aug 14, 2015 at 1:15 PM, Jun Rao <ju...@confluent.io> wrote:
> Hi, Everyone,
>
> Since the release of Kafka 0.8.2.1, a number of people have reported an
> issue with snappy compression (
> https://issues.apache.org/jira/browse/KAFKA-2189). Basically, if they use
> snappy in 0.8.2.1, they will experience a 2-3X space increase. The issue
> has since been fixed in trunk (just a snappy jar upgrade). Since 0.8.3 is
> still a few months away, it may make sense to do an 0.8.2.2 release just to
> fix this issue. Any objections?
>
> Thanks,
>
> Jun
>