You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@kafka.apache.org by Surendranauth Hiraman <su...@velos.io> on 2014/12/12 15:33:48 UTC
Client Offset Storage
My team is using Kafka 0.8.1 and we may not be able to upgrade to 0.8.2 to
take advantage of the broker-side commit of client offsets.
Is anyone aware of a Java/Scala library for client offset storage outside
of ZK?
-- Suren
SUREN HIRAMAN, VP TECHNOLOGY
Velos
Accelerating Machine Learning
440 NINTH AVENUE, 11TH FLOOR
NEW YORK, NY 10001
O: (917) 525-2466 ext. 105
F: 646.349.4063
E: suren.hiraman@v <su...@sociocast.com>elos.io
W: www.velos.io
Re: Client Offset Storage
Posted by Stevo Slavić <ss...@gmail.com>.
I have no experience with, but https://github.com/gerritjvv/kafka-fast
seems to fit your description.
Kind regards,
Stevo Slavic
On Fri, Dec 12, 2014 at 7:18 PM, Surendranauth Hiraman <
suren.hiraman@velos.io> wrote:
>
> Basically, don't want to use ZK, for the reasons driving the new client
> offset features in 0.8.2.
>
> So we are looking for a library we could use that provides an alternative
> client offset implementation (broker-side, redis, local file, anything
> else). We could roll our own and put it behind the client offset api but I
> wanted to see if anyone had an implementation.
>
> -Suren
>
>
> On Fri, Dec 12, 2014 at 12:41 PM, Steve Morin <st...@stevemorin.com>
> wrote:
> >
> > Suren,
> > Like out of the box storage or roll your own?
> > -Steve
> >
> > On Fri, Dec 12, 2014 at 6:33 AM, Surendranauth Hiraman <
> > suren.hiraman@velos.io> wrote:
> >
> > > My team is using Kafka 0.8.1 and we may not be able to upgrade to 0.8.2
> > to
> > > take advantage of the broker-side commit of client offsets.
> > >
> > > Is anyone aware of a Java/Scala library for client offset storage
> outside
> > > of ZK?
> > >
> > >
> > > -- Suren
> > >
> > >
> > > SUREN HIRAMAN, VP TECHNOLOGY
> > > Velos
> > > Accelerating Machine Learning
> > >
> > > 440 NINTH AVENUE, 11TH FLOOR
> > > NEW YORK, NY 10001
> > > O: (917) 525-2466 ext. 105
> > > F: 646.349.4063
> > > E: suren.hiraman@v <su...@sociocast.com>elos.io
> > > W: www.velos.io
> > >
> >
>
>
> --
>
> SUREN HIRAMAN, VP TECHNOLOGY
> Velos
> Accelerating Machine Learning
>
> 440 NINTH AVENUE, 11TH FLOOR
> NEW YORK, NY 10001
> O: (917) 525-2466 ext. 105
> F: 646.349.4063
> E: suren.hiraman@v <su...@sociocast.com>elos.io
> W: www.velos.io
>
Re: Client Offset Storage
Posted by Surendranauth Hiraman <su...@velos.io>.
Basically, don't want to use ZK, for the reasons driving the new client
offset features in 0.8.2.
So we are looking for a library we could use that provides an alternative
client offset implementation (broker-side, redis, local file, anything
else). We could roll our own and put it behind the client offset api but I
wanted to see if anyone had an implementation.
-Suren
On Fri, Dec 12, 2014 at 12:41 PM, Steve Morin <st...@stevemorin.com> wrote:
>
> Suren,
> Like out of the box storage or roll your own?
> -Steve
>
> On Fri, Dec 12, 2014 at 6:33 AM, Surendranauth Hiraman <
> suren.hiraman@velos.io> wrote:
>
> > My team is using Kafka 0.8.1 and we may not be able to upgrade to 0.8.2
> to
> > take advantage of the broker-side commit of client offsets.
> >
> > Is anyone aware of a Java/Scala library for client offset storage outside
> > of ZK?
> >
> >
> > -- Suren
> >
> >
> > SUREN HIRAMAN, VP TECHNOLOGY
> > Velos
> > Accelerating Machine Learning
> >
> > 440 NINTH AVENUE, 11TH FLOOR
> > NEW YORK, NY 10001
> > O: (917) 525-2466 ext. 105
> > F: 646.349.4063
> > E: suren.hiraman@v <su...@sociocast.com>elos.io
> > W: www.velos.io
> >
>
--
SUREN HIRAMAN, VP TECHNOLOGY
Velos
Accelerating Machine Learning
440 NINTH AVENUE, 11TH FLOOR
NEW YORK, NY 10001
O: (917) 525-2466 ext. 105
F: 646.349.4063
E: suren.hiraman@v <su...@sociocast.com>elos.io
W: www.velos.io
Re: Client Offset Storage
Posted by Steve Morin <st...@stevemorin.com>.
Suren,
Like out of the box storage or roll your own?
-Steve
On Fri, Dec 12, 2014 at 6:33 AM, Surendranauth Hiraman <
suren.hiraman@velos.io> wrote:
> My team is using Kafka 0.8.1 and we may not be able to upgrade to 0.8.2 to
> take advantage of the broker-side commit of client offsets.
>
> Is anyone aware of a Java/Scala library for client offset storage outside
> of ZK?
>
>
> -- Suren
>
>
> SUREN HIRAMAN, VP TECHNOLOGY
> Velos
> Accelerating Machine Learning
>
> 440 NINTH AVENUE, 11TH FLOOR
> NEW YORK, NY 10001
> O: (917) 525-2466 ext. 105
> F: 646.349.4063
> E: suren.hiraman@v <su...@sociocast.com>elos.io
> W: www.velos.io
>