Flume 1.6 support on Kafka 0.8.2

[Avi Asulin <av...@plarium.com>]

Hi ,
We want to upgrade Cloudera to CDH 5.7 while keeping our current kafka
version 0.8.2
We are getting this warning

[ConsumerFetcherThread-FlumeBI_DEV_cloudera3.localdomain-1468857491433-15876e87-0-3],
Error in fetch kafka.consumer.ConsumerFetcherThread$FetchRequest@60e8e67f.
Possible cause: java.lang.IllegalArgumentException

its working fine with kafka 0.9
is flume 1.6 supporting kafka 0.8 version ?

Thanks

Re: Flume 1.6 support on Kafka 0.8.2

[Gonzalo Herreros <gh...@gmail.com>]

Apache Flume 1.6 runs on kafka 0.8, what you have is a branch from Cloudera.

I would advise against that. The whole point of using a distribution is
that you use component versions they have integrated and tested together.
Either use kafka 0.9 or don't upgrade to 5.7 until you are ready

If you still want to go ahead, you need to update the Flume classpath so
the 0.8 libraries are taken before the ones shipped with the distribution
and hope everything works.

Gonzalo

On 20 July 2016 at 13:42, Avi Asulin <av...@plarium.com> wrote:

> Hi ,
> We want to upgrade Cloudera to CDH 5.7 while keeping our current kafka
> version 0.8.2
> We are getting this warning
>
> [ConsumerFetcherThread-FlumeBI_DEV_cloudera3.localdomain-1468857491433-15876e87-0-3],
> Error in fetch kafka.consumer.ConsumerFetcherThread$FetchRequest@60e8e67f.
> Possible cause: java.lang.IllegalArgumentException
>
> its working fine with kafka 0.9
> is flume 1.6 supporting kafka 0.8 version ?
>
> Thanks
>