Side-by-side migration for 0.7 to 0.8

[Patrick McBryde <pa...@curalate.com>]

Hello,
Does anyone have tips or advice for side-by-side migration from 0.7 to 0.8
clusters? We're in the process of migrating, and are bringing up our 0.8
cluster and first producer & consumer now. We're running into issues with
getting the 0.7 and 0.8 jar's to play nice, as it seems like there's no way
to do a piecemeal migration since the two jar's cannot coexist in a
project. So we can't connect to our existing ops infrastructure, can't
build tests side by side, etc.

Our current plan is to bring up a small service which enqueues onto our 0.8
cluster to abstract 0.8 away from our existing infrastructure, and at some
point migrate all producers and consumers in one go. This feels like a very
risky method for a migration of production resources, though, so we're
hoping somebody has a better method for getting the two API versions to
work together.

Thanks,
Patrick

Re: Side-by-side migration for 0.7 to 0.8

[Jason Rosenberg <jb...@squareup.com>]

Hi Patrick,

When we went through this, we 'shaded' the old kafka jar, so the 2 could
co-exist in the same app.  We use maven, and there's a maven 'shade
plugin', etc.

In our case, it was intractable to try to update all producers and
consumers "in one go" as you suggest, so we had to have a way in our code
base for the old and new to co-exist.

Jason

On Fri, Apr 3, 2015 at 8:48 PM, Patrick McBryde <pa...@curalate.com>
wrote:

> Hello,
> Does anyone have tips or advice for side-by-side migration from 0.7 to 0.8
> clusters? We're in the process of migrating, and are bringing up our 0.8
> cluster and first producer & consumer now. We're running into issues with
> getting the 0.7 and 0.8 jar's to play nice, as it seems like there's no way
> to do a piecemeal migration since the two jar's cannot coexist in a
> project. So we can't connect to our existing ops infrastructure, can't
> build tests side by side, etc.
>
> Our current plan is to bring up a small service which enqueues onto our 0.8
> cluster to abstract 0.8 away from our existing infrastructure, and at some
> point migrate all producers and consumers in one go. This feels like a very
> risky method for a migration of production resources, though, so we're
> hoping somebody has a better method for getting the two API versions to
> work together.
>
> Thanks,
> Patrick
>