You are viewing a plain text version of this content. The canonical link for it is here.

Posted to users@kafka.apache.org by Yosi Botzer <yo...@gmail.com> on 2015/04/08 17:13:25 UTC

offset-management-in-kafka

Hi,

I have seen this presentation:
http://www.slideshare.net/jjkoshy/offset-management-in-kafka

describing a new approach for offset management.

I wanted to know from which kafka version this option will be available and
will it be available by default

Thanks
Yosi

Re: offset-management-in-kafka

Posted by Gwen Shapira <gs...@cloudera.com>.

You are looking at the newer, and not-ready-yet consumer there.

The configuration you are looking for is defined here:
core/src/main/scala/kafka/consumer/ConsumerConfig.scala

Gwen

On Wed, Apr 8, 2015 at 9:25 AM, Yosi Botzer <yo...@gmail.com> wrote:

> Hi,
>
> I am looking at the java
> class org.apache.kafka.clients.consumer.ConsumerConfig and I do not see
> there a constant for offsets.storage
>
> Am I missing something?
>
> This is my pom dependency definition:
>
> <dependency>
> <groupId>org.apache.kafka</groupId>
> <artifactId>kafka_2.10</artifactId>
> <version>0.8.2.1</version>
> </dependency>
>
> On Wed, Apr 8, 2015 at 6:29 PM, Gwen Shapira <gs...@cloudera.com>
> wrote:
>
> > This is available from 0.8.2.0, and is enabled on server by default. The
> > consumer needs to specify offsets.storage parameter - the default is
> still
> > zookeeper, so the consumers should set it to 'kafka'.
> >
> > The documentation also explain how to migrate from zookeeper offsets to
> > kafka offsets.
> >
> > Gwen
> >
> > On Wed, Apr 8, 2015 at 8:13 AM, Yosi Botzer <yo...@gmail.com>
> wrote:
> >
> > > Hi,
> > >
> > > I have seen this presentation:
> > > http://www.slideshare.net/jjkoshy/offset-management-in-kafka
> > >
> > > describing a new approach for offset management.
> > >
> > > I wanted to know from which kafka version this option will be available
> > and
> > > will it be available by default
> > >
> > > Thanks
> > > Yosi
> > >
> >
>

Re: offset-management-in-kafka

Posted by Yosi Botzer <yo...@gmail.com>.

Hi,

I am looking at the java
class org.apache.kafka.clients.consumer.ConsumerConfig and I do not see
there a constant for offsets.storage

Am I missing something?

This is my pom dependency definition:

<dependency>
<groupId>org.apache.kafka</groupId>
<artifactId>kafka_2.10</artifactId>
<version>0.8.2.1</version>
</dependency>

On Wed, Apr 8, 2015 at 6:29 PM, Gwen Shapira <gs...@cloudera.com> wrote:

> This is available from 0.8.2.0, and is enabled on server by default. The
> consumer needs to specify offsets.storage parameter - the default is still
> zookeeper, so the consumers should set it to 'kafka'.
>
> The documentation also explain how to migrate from zookeeper offsets to
> kafka offsets.
>
> Gwen
>
> On Wed, Apr 8, 2015 at 8:13 AM, Yosi Botzer <yo...@gmail.com> wrote:
>
> > Hi,
> >
> > I have seen this presentation:
> > http://www.slideshare.net/jjkoshy/offset-management-in-kafka
> >
> > describing a new approach for offset management.
> >
> > I wanted to know from which kafka version this option will be available
> and
> > will it be available by default
> >
> > Thanks
> > Yosi
> >
>

Re: offset-management-in-kafka

Posted by Yosi Botzer <yo...@gmail.com>.

Thanks Gwan,

Are there any plans to change the default from zookeeper to kafka in the
future?


On Wed, Apr 8, 2015 at 6:29 PM, Gwen Shapira <gs...@cloudera.com> wrote:

> This is available from 0.8.2.0, and is enabled on server by default. The
> consumer needs to specify offsets.storage parameter - the default is still
> zookeeper, so the consumers should set it to 'kafka'.
>
> The documentation also explain how to migrate from zookeeper offsets to
> kafka offsets.
>
> Gwen
>
> On Wed, Apr 8, 2015 at 8:13 AM, Yosi Botzer <yo...@gmail.com> wrote:
>
> > Hi,
> >
> > I have seen this presentation:
> > http://www.slideshare.net/jjkoshy/offset-management-in-kafka
> >
> > describing a new approach for offset management.
> >
> > I wanted to know from which kafka version this option will be available
> and
> > will it be available by default
> >
> > Thanks
> > Yosi
> >
>

Re: offset-management-in-kafka

Posted by Gwen Shapira <gs...@cloudera.com>.

This is available from 0.8.2.0, and is enabled on server by default. The
consumer needs to specify offsets.storage parameter - the default is still
zookeeper, so the consumers should set it to 'kafka'.

The documentation also explain how to migrate from zookeeper offsets to
kafka offsets.

Gwen

On Wed, Apr 8, 2015 at 8:13 AM, Yosi Botzer <yo...@gmail.com> wrote:

> Hi,
>
> I have seen this presentation:
> http://www.slideshare.net/jjkoshy/offset-management-in-kafka
>
> describing a new approach for offset management.
>
> I wanted to know from which kafka version this option will be available and
> will it be available by default
>
> Thanks
> Yosi
>