[NEW] ZK upgrade test tool

[Andor Molnar <an...@cloudera.com>]

Hi,

I thought it could be interesting for some of you that I started to work on
an automated upgrade testing tool for ZooKeeper.

https://github.com/anmolnar/zk-upgrade-test

Reasoning behind it is that later, when 3.5 becomes stable, we'd like to
validate if we can upgrade our 3.4 instances without problems.

Currently it's in very early stage, there're a still a lot of moving parts
and it's only a basic command line tool, but I'd like to get some community
feedback to decide which approach could be the best for such tool and how
many of you find it useful.

The scaffolding is pretty-much ready:

By default, the tool creates a CentOS7-based docker image and downloads the
source and the target version of ZooKeeper inside.

Next, it fires up a dedicated docker network and starts as many containers
as the number of nodes in the test ensemble.

Testing steps are currently as basic as it could be:
1. Create config
2. Start 'source' version
3. Run smoke test (create node)
4. Stop / start 'target' version
5. Run smoke test (read the original node)

What's coming?
- config side: I'd like to have a way to inject custom configuration files
somehow instead of hardcoding it.
- config transformation: it'd be useful to do some config transformation
before the upgrade, because it's very likely needed in real world scenario,
- smoke tests: add more smoke tests to cover more functionality,
- rolling upgrade: shouldn't upgrade all nodes at once. Rather I'd like to
upgrade one node at a time and run smoke tests in each iteration.

Regards,
Andor

Re: [NEW] ZK upgrade test tool

[Abraham Fine <af...@apache.org>]

Great work Andor!

This will be super valuable going forward, especially the rolling upgrade stuff.

Abe

On Tue, Feb 6, 2018, at 04:44, Andor Molnar wrote:
> Hi,
> 
> I thought it could be interesting for some of you that I started to work on
> an automated upgrade testing tool for ZooKeeper.
> 
> https://github.com/anmolnar/zk-upgrade-test
> 
> Reasoning behind it is that later, when 3.5 becomes stable, we'd like to
> validate if we can upgrade our 3.4 instances without problems.
> 
> Currently it's in very early stage, there're a still a lot of moving parts
> and it's only a basic command line tool, but I'd like to get some community
> feedback to decide which approach could be the best for such tool and how
> many of you find it useful.
> 
> The scaffolding is pretty-much ready:
> 
> By default, the tool creates a CentOS7-based docker image and downloads the
> source and the target version of ZooKeeper inside.
> 
> Next, it fires up a dedicated docker network and starts as many containers
> as the number of nodes in the test ensemble.
> 
> Testing steps are currently as basic as it could be:
> 1. Create config
> 2. Start 'source' version
> 3. Run smoke test (create node)
> 4. Stop / start 'target' version
> 5. Run smoke test (read the original node)
> 
> What's coming?
> - config side: I'd like to have a way to inject custom configuration files
> somehow instead of hardcoding it.
> - config transformation: it'd be useful to do some config transformation
> before the upgrade, because it's very likely needed in real world scenario,
> - smoke tests: add more smoke tests to cover more functionality,
> - rolling upgrade: shouldn't upgrade all nodes at once. Rather I'd like to
> upgrade one node at a time and run smoke tests in each iteration.
> 
> Regards,
> Andor