Re: [EXTERNAL] Kafka upgrade from 0.10 to 2.3.x

[Jose Manuel Vega Monroy <jo...@williamhill.com>]

Hi Vitalii,

About log format, if clients are old ones will start to fail since they couldn't understand the messages.

So bump them if you planning upgrade the log format version.

Thanks

Get Outlook for Android<https://aka.ms/ghei36>

________________________________
From: Vitalii Stoianov <vi...@gmail.com>
Sent: Thursday, November 21, 2019 10:31:22 PM
To: users@kafka.apache.org <us...@kafka.apache.org>
Subject: [EXTERNAL] Kafka upgrade from 0.10 to 2.3.x

Hi All,

All the docs that I was able to find describe the rolling upgrade.
But I didn't find any docs that describe how to perform no-rolling upgrade.

So if system can afford downtime how in this case to upgrade kafka form
version 0.10.0 to 2.3.x.
Is it still required to do few restarts or we can just upgrade all in one
go?

For example:
0. Stop producers -> consumer -> brokers.
1. Upgrade brokers, clients.
2. Upgrade brokers, inter.broker.protocol.version.
3. Upgrade brokers, log.message.format.version.
4. Start brokers.
5. Start consumers.
6. Start producers.

In case of upgrade from 0.10.0 to 2.3.x Docs states that message format has
changed since version 0.10.0
https://urldefense.proofpoint.com/v2/url?u=https-3A__kafka.apache.org_22_documentation.html-23messageset&d=DwIBaQ&c=pWn2jKJ-j-AhxLuiRFe-Qw&r=i5Pk4pirVCmwsmddZqplM1jyQtVWeoOOb-vkuqku5P8&m=1Bbq_fCBGYdws-Lil0jrt3sHYfBeO69HPka-rrOZbH4&s=B_4bFQr79wzHjrL7LRtsG7wOCcTJqmcQqOUfoAC80gc&e=  (old format that
is used by 0.10.0) and in 0.11.0 was introduced new format that is used
till now 2.3.x (as I understand).

What will happen with kafka brokers data logs on disk after message format
change change ?
And if data on disk is not changed during change of message format: What
will happen if new consumer receive message in old message format because
it was not read by consumer before upgrade?

Regards,
Vitalii.
Confidentiality: The contents of this e-mail and any attachments transmitted with it are intended to be confidential to the intended recipient; and may be privileged or otherwise protected from disclosure. If you are not an intended recipient of this e-mail, do not duplicate or redistribute it by any means. Please delete it and any attachments and notify the sender that you have received it in error. This e-mail is sent by a William Hill PLC group company. The William Hill group companies include, among others, William Hill PLC (registered number 4212563), William Hill Organization Limited (registered number 278208), William Hill US HoldCo Inc, WHG (International) Limited (registered number 99191) and Mr Green Limited (registered number C43260). Each of William Hill PLC and William Hill Organization Limited is registered in England and Wales and has its registered office at 1 Bedford Avenue, London, WC1B 3AU, UK. William Hill U.S. HoldCo, Inc. is registered in Delaware and has its registered office at 1007 N. Orange Street, 9 Floor, Wilmington, New Castle County DE 19801 Delaware, United States of America. WHG (International) Limited is registered in Gibraltar and has its registered office at 6/1 Waterport Place, Gibraltar. Mr Green Limited is registered in Malta and has its registered office at Tagliaferro Business Centre, Level 7, 14 High Street, Sliema SLM 1549, Malta. Unless specifically indicated otherwise, the contents of this e-mail are subject to contract; and are not an official statement, and do not necessarily represent the views, of William Hill PLC, its subsidiaries or affiliated companies. Please note that neither William Hill PLC, nor its subsidiaries and affiliated companies can accept any responsibility for any viruses contained within this e-mail and it is your responsibility to scan any emails and their attachments. William Hill PLC, its subsidiaries and affiliated companies may monitor e-mail traffic data and also the content of e-mails for effective operation of the e-mail system, or for security, purposes.

Re: [EXTERNAL] Kafka upgrade from 0.10 to 2.3.x

[Vitalii Stoianov <vi...@gmail.com>]

On Fri, Nov 22, 2019 at 10:18 AM Jose Manuel Vega Monroy <
jose.monroy@williamhill.com> wrote:

> Hi Vitalii,
>
> About log format, if clients are old ones will start to fail since they
> couldn't understand the messages.
>
> So bump them if you planning upgrade the log format version.
>
> Thanks
>
> Get Outlook for Android<https://aka.ms/ghei36>
>
> ________________________________
> From: Vitalii Stoianov <vi...@gmail.com>
> Sent: Thursday, November 21, 2019 10:31:22 PM
> To: users@kafka.apache.org <us...@kafka.apache.org>
> Subject: [EXTERNAL] Kafka upgrade from 0.10 to 2.3.x
>
> Hi All,
>
> All the docs that I was able to find describe the rolling upgrade.
> But I didn't find any docs that describe how to perform no-rolling upgrade.
>
> So if system can afford downtime how in this case to upgrade kafka form
> version 0.10.0 to 2.3.x.
> Is it still required to do few restarts or we can just upgrade all in one
> go?
>
> For example:
> 0. Stop producers -> consumer -> brokers.
> 1. Upgrade brokers, clients.
> 2. Upgrade brokers, inter.broker.protocol.version.
> 3. Upgrade brokers, log.message.format.version.
> 4. Start brokers.
> 5. Start consumers.
> 6. Start producers.
>
> In case of upgrade from 0.10.0 to 2.3.x Docs states that message format has
> changed since version 0.10.0
>
> https://urldefense.proofpoint.com/v2/url?u=https-3A__kafka.apache.org_22_documentation.html-23messageset&d=DwIBaQ&c=pWn2jKJ-j-AhxLuiRFe-Qw&r=i5Pk4pirVCmwsmddZqplM1jyQtVWeoOOb-vkuqku5P8&m=1Bbq_fCBGYdws-Lil0jrt3sHYfBeO69HPka-rrOZbH4&s=B_4bFQr79wzHjrL7LRtsG7wOCcTJqmcQqOUfoAC80gc&e=
> (old format that
> is used by 0.10.0) and in 0.11.0 was introduced new format that is used
> till now 2.3.x (as I understand).
>
> What will happen with kafka brokers data logs on disk after message format
> change change ?
> And if data on disk is not changed during change of message format: What
> will happen if new consumer receive message in old message format because
> it was not read by consumer before upgrade?
>
> Regards,
> Vitalii.
> Confidentiality: The contents of this e-mail and any attachments
> transmitted with it are intended to be confidential to the intended
> recipient; and may be privileged or otherwise protected from disclosure. If
> you are not an intended recipient of this e-mail, do not duplicate or
> redistribute it by any means. Please delete it and any attachments and
> notify the sender that you have received it in error. This e-mail is sent
> by a William Hill PLC group company. The William Hill group companies
> include, among others, William Hill PLC (registered number 4212563),
> William Hill Organization Limited (registered number 278208), William Hill
> US HoldCo Inc, WHG (International) Limited (registered number 99191) and Mr
> Green Limited (registered number C43260). Each of William Hill PLC and
> William Hill Organization Limited is registered in England and Wales and
> has its registered office at 1 Bedford Avenue, London, WC1B 3AU, UK.
> William Hill U.S. HoldCo, Inc. is registered in Delaware and has its
> registered office at 1007 N. Orange Street, 9 Floor, Wilmington, New Castle
> County DE 19801 Delaware, United States of America. WHG (International)
> Limited is registered in Gibraltar and has its registered office at 6/1
> Waterport Place, Gibraltar. Mr Green Limited is registered in Malta and has
> its registered office at Tagliaferro Business Centre, Level 7, 14 High
> Street, Sliema SLM 1549, Malta. Unless specifically indicated otherwise,
> the contents of this e-mail are subject to contract; and are not an
> official statement, and do not necessarily represent the views, of William
> Hill PLC, its subsidiaries or affiliated companies. Please note that
> neither William Hill PLC, nor its subsidiaries and affiliated companies can
> accept any responsibility for any viruses contained within this e-mail and
> it is your responsibility to scan any emails and their attachments. William
> Hill PLC, its subsidiaries and affiliated companies may monitor e-mail
> traffic data and also the content of e-mails for effective operation of the
> e-mail system, or for security, purposes.
>