You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@kafka.apache.org by John Yost <ho...@gmail.com> on 2017/09/19 13:41:46 UTC
Upgraded brokers from 0.9.0.1 -> 0.10.0.1: how to upgrade message
format to 0.10.0.1?
Hi Everyone,
We recently upgraded our cluster from 0.9.0.1 to 0.10.0.1 but had to keep
our Kafka clients at 0.9.0.1. We now want to upgrade our clients and,
concurrently, the message version to 0.10.0.1.
When we did the 0.9.0.1 -> 0.10.0.1 broker upgrade we were not able to
upgrade the kafka clients to 0.10.0.1, and we did not set the message
format. As a result the 0.9.0.1 -> 0.10.0.1 format conversion for both
incoming out outgoing messages caused our memory heap requirements to go
from 6 GB up to 12 GB, resulting in long GC pauses that caused our brokers
to crash.
Once I explicitly set the message format to 0.9.0.1 everything was fine.
However, now that I want to convert to the 0.10.0.1 message format, I am
looking for guidance as to the best way to do this. I can switch our Kafka
clients to the 0.10.0.1 release, no problem.
But...our challenge is that we occasionally have to replay topics, and I am
concerned that we will be back to the issue of converting outgoing messages
from 0.9.0.1 to 0.10.0.1 in these replay scenarios as well as in
inter-broker replication.
Please confirm the best way to upgrade our clients and message format to
0.10.0.1 while preserving the stability of our cluster as well as the
ability to replay topics that will have both 0.9.0.1 and 0.10.0.1-formatted
messages.
Thanks
--John
Re: Upgraded brokers from 0.9.0.1 -> 0.10.0.1: how to upgrade message
format to 0.10.0.1?
Posted by John Yost <ho...@gmail.com>.
Ah, cool, thanks Ismael!
--John
On Tue, Sep 19, 2017 at 10:20 AM, Ismael Juma <is...@juma.me.uk> wrote:
> 0.10.0.1 consumers understand the older formats. So, the conversion only
> happens when the message format is newer than what the consumer
> understands. For the producer side, the conversion is not particularly
> costly since the data is in the heap and, if you use compression, 0.9.0.x
> would do recompression either way.
>
> Ismael
>
> On Tue, Sep 19, 2017 at 2:41 PM, John Yost <ho...@gmail.com> wrote:
>
> > Hi Everyone,
> >
> > We recently upgraded our cluster from 0.9.0.1 to 0.10.0.1 but had to keep
> > our Kafka clients at 0.9.0.1. We now want to upgrade our clients and,
> > concurrently, the message version to 0.10.0.1.
> >
> > When we did the 0.9.0.1 -> 0.10.0.1 broker upgrade we were not able to
> > upgrade the kafka clients to 0.10.0.1, and we did not set the message
> > format. As a result the 0.9.0.1 -> 0.10.0.1 format conversion for both
> > incoming out outgoing messages caused our memory heap requirements to go
> > from 6 GB up to 12 GB, resulting in long GC pauses that caused our
> brokers
> > to crash.
> >
> > Once I explicitly set the message format to 0.9.0.1 everything was fine.
> > However, now that I want to convert to the 0.10.0.1 message format, I am
> > looking for guidance as to the best way to do this. I can switch our
> Kafka
> > clients to the 0.10.0.1 release, no problem.
> >
> > But...our challenge is that we occasionally have to replay topics, and I
> am
> > concerned that we will be back to the issue of converting outgoing
> messages
> > from 0.9.0.1 to 0.10.0.1 in these replay scenarios as well as in
> > inter-broker replication.
> >
> > Please confirm the best way to upgrade our clients and message format to
> > 0.10.0.1 while preserving the stability of our cluster as well as the
> > ability to replay topics that will have both 0.9.0.1 and
> 0.10.0.1-formatted
> > messages.
> >
> > Thanks
> >
> > --John
> >
>
Re: Upgraded brokers from 0.9.0.1 -> 0.10.0.1: how to upgrade message
format to 0.10.0.1?
Posted by Ismael Juma <is...@juma.me.uk>.
0.10.0.1 consumers understand the older formats. So, the conversion only
happens when the message format is newer than what the consumer
understands. For the producer side, the conversion is not particularly
costly since the data is in the heap and, if you use compression, 0.9.0.x
would do recompression either way.
Ismael
On Tue, Sep 19, 2017 at 2:41 PM, John Yost <ho...@gmail.com> wrote:
> Hi Everyone,
>
> We recently upgraded our cluster from 0.9.0.1 to 0.10.0.1 but had to keep
> our Kafka clients at 0.9.0.1. We now want to upgrade our clients and,
> concurrently, the message version to 0.10.0.1.
>
> When we did the 0.9.0.1 -> 0.10.0.1 broker upgrade we were not able to
> upgrade the kafka clients to 0.10.0.1, and we did not set the message
> format. As a result the 0.9.0.1 -> 0.10.0.1 format conversion for both
> incoming out outgoing messages caused our memory heap requirements to go
> from 6 GB up to 12 GB, resulting in long GC pauses that caused our brokers
> to crash.
>
> Once I explicitly set the message format to 0.9.0.1 everything was fine.
> However, now that I want to convert to the 0.10.0.1 message format, I am
> looking for guidance as to the best way to do this. I can switch our Kafka
> clients to the 0.10.0.1 release, no problem.
>
> But...our challenge is that we occasionally have to replay topics, and I am
> concerned that we will be back to the issue of converting outgoing messages
> from 0.9.0.1 to 0.10.0.1 in these replay scenarios as well as in
> inter-broker replication.
>
> Please confirm the best way to upgrade our clients and message format to
> 0.10.0.1 while preserving the stability of our cluster as well as the
> ability to replay topics that will have both 0.9.0.1 and 0.10.0.1-formatted
> messages.
>
> Thanks
>
> --John
>