[SURVEY] How do people upgrade their Flink applications?

[Konstantinos Kallas <ko...@hotmail.com>]

Hi everyone,

I am interested in how people safely upgrade their Flink applications 
when they apply a patch, or when they develop a new feature. How do you 
ensure that no bug was introduced?

Do you run two versions of the application simultaneously (the old and 
the upgraded one), monitoring their results to make sure that they are 
consistent? Do you run a set of unit and integration tests and then 
deploy if they all succeed? Do you have some other method?

Best regards,
Konstantinos Kallas

Re: [SURVEY] How do people upgrade their Flink applications?

[Konstantinos Kallas <ko...@hotmail.com>]

Thank you for the thoughts Yang. If you don't mind, I would be interested to know how you deploy the upgraded application to the pre-production environment.

Do you collect historical data and then run the upgraded application using that, or do you duplicate new incoming data on the fly, running the new and the old application simultaneously?

Also, how can one ensure that the output of the upgraded application is consistent with the old one? Do you inspect the output of the application manually?

Thanks,

Konstantinos

On 10/8/19 10:36 PM, Yang Wang wrote:
Just to share some thoughts.

If a Flink application need to upgrade, including user jar or Flink version, the following instructions
need to be done.
1. Unit test, integration test, stability test, benchmark test
2. Deploy the upgraded application to test environment with fake inputs/outputs to avoid impact on
production applications
3. Deploy the upgraded application to pre-production environment with copy of production inputs/outputs.
It should be running for more than one cycle, usually one day or one week.
4. Deploy it the production environment



Best,
Yang

Konstantinos Kallas <ko...@hotmail.com>> 于2019年10月8日周二 上午4:07写道：
Hi everyone,

I am interested in how people safely upgrade their Flink applications
when they apply a patch, or when they develop a new feature. How do you
ensure that no bug was introduced?

Do you run two versions of the application simultaneously (the old and
the upgraded one), monitoring their results to make sure that they are
consistent? Do you run a set of unit and integration tests and then
deploy if they all succeed? Do you have some other method?

Best regards,
Konstantinos Kallas

Re: [SURVEY] How do people upgrade their Flink applications?

[Yang Wang <da...@gmail.com>]

Just to share some thoughts.

If a Flink application need to upgrade, including user jar or Flink
version, the following instructions
need to be done.
1. Unit test, integration test, stability test, benchmark test
2. Deploy the upgraded application to test environment with fake
inputs/outputs to avoid impact on
production applications
3. Deploy the upgraded application to pre-production environment with copy
of production inputs/outputs.
It should be running for more than one cycle, usually one day or one week.
4. Deploy it the production environment



Best,
Yang

Konstantinos Kallas <ko...@hotmail.com> 于2019年10月8日周二
上午4:07写道：

> Hi everyone,
>
> I am interested in how people safely upgrade their Flink applications
> when they apply a patch, or when they develop a new feature. How do you
> ensure that no bug was introduced?
>
> Do you run two versions of the application simultaneously (the old and
> the upgraded one), monitoring their results to make sure that they are
> consistent? Do you run a set of unit and integration tests and then
> deploy if they all succeed? Do you have some other method?
>
> Best regards,
> Konstantinos Kallas
>
>