You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@kafka.apache.org by Nick DeCoursin <n....@foodpanda.com> on 2017/01/25 08:43:52 UTC
Track Latest Count - Unique Key
From the documentation
<http://docs.confluent.io/3.1.1/streams/developer-guide.html#id3>:
The counting operation for record streams is trivial to implement: you can
> maintain a local state store that tracks the latest count for each key,
> and, upon receiving a new record, update the corresponding key by
> incrementing its count by one.
How? Is there any examples of this online? To me, it doesn't seem so
trivial because there's no such thing as a transaction in Kafka,
Thank you,
Nick
--
Nick DeCoursin
Software Engineer
foodpanda
Tel | +1 920 450 5434
Mail | n.decoursin@foodpanda.com
Skype | nick.foodpanda
Foodpanda GmbH | Schreiberhauer Str. 30 | 10317 Berlin | Germany
Sitz der Gesellschaft | Berlin, AG Charlottenburg | HRB 138224 B |
USt-ID-Nr | DE 283789080
Geschäftsführer | Benjamin Bauer, Felix Plog, Ralf Wenzel
CONFIDENTIALITY NOTICE: This message (including any attachments) is
confidential and may be privileged. It may be read, copied and used only by
the intended recipient. If you have received it in error please contact the
sender (by return e-mail) immediately and delete this message. Any
unauthorized use or dissemination of this message in whole or in parts is
strictly prohibited.
Re: Track Latest Count - Unique Key
Posted by Nick DeCoursin <n....@foodpanda.com>.
@Damian Thank you!!
On 25 January 2017 at 13:37, Damian Guy <da...@gmail.com> wrote:
> You can have a look at
> https://github.com/apache/kafka/blob/trunk/streams/src/
> main/java/org/apache/kafka/streams/kstream/internals/
> KGroupedStreamImpl.java#L150
> for an example. Obviously exactly-once semantics for Kafka haven't been
> completed, so this would be at-least-once.
>
> On Wed, 25 Jan 2017 at 08:43 Nick DeCoursin <n....@foodpanda.com>
> wrote:
>
> > From the documentation
> > <http://docs.confluent.io/3.1.1/streams/developer-guide.html#id3>:
> >
> > The counting operation for record streams is trivial to implement: you
> can
> > > maintain a local state store that tracks the latest count for each key,
> > > and, upon receiving a new record, update the corresponding key by
> > > incrementing its count by one.
> >
> >
> > How? Is there any examples of this online? To me, it doesn't seem so
> > trivial because there's no such thing as a transaction in Kafka,
> >
> > Thank you,
> > Nick
> >
> > --
> >
> > Nick DeCoursin
> > Software Engineer
> > foodpanda
> >
> > Tel | +1 920 450 5434 <(920)%20450-5434>
> >
> > Mail | n.decoursin@foodpanda.com
> >
> > Skype | nick.foodpanda
> >
> > Foodpanda GmbH | Schreiberhauer Str. 30 | 10317 Berlin | Germany
> > Sitz der Gesellschaft | Berlin, AG Charlottenburg | HRB 138224 B |
> > USt-ID-Nr | DE 283789080
> > Geschäftsführer | Benjamin Bauer, Felix Plog, Ralf Wenzel
> >
> > CONFIDENTIALITY NOTICE: This message (including any attachments) is
> > confidential and may be privileged. It may be read, copied and used only
> by
> > the intended recipient. If you have received it in error please contact
> the
> > sender (by return e-mail) immediately and delete this message. Any
> > unauthorized use or dissemination of this message in whole or in parts is
> > strictly prohibited.
> >
>
--
Nick DeCoursin
Software Engineer
foodpanda
Tel | +1 920 450 5434
Mail | n.decoursin@foodpanda.com
Skype | nick.foodpanda
Foodpanda GmbH | Schreiberhauer Str. 30 | 10317 Berlin | Germany
Sitz der Gesellschaft | Berlin, AG Charlottenburg | HRB 138224 B |
USt-ID-Nr | DE 283789080
Geschäftsführer | Benjamin Bauer, Felix Plog, Ralf Wenzel
CONFIDENTIALITY NOTICE: This message (including any attachments) is
confidential and may be privileged. It may be read, copied and used only by
the intended recipient. If you have received it in error please contact the
sender (by return e-mail) immediately and delete this message. Any
unauthorized use or dissemination of this message in whole or in parts is
strictly prohibited.
Re: Track Latest Count - Unique Key
Posted by Damian Guy <da...@gmail.com>.
You can have a look at
https://github.com/apache/kafka/blob/trunk/streams/src/main/java/org/apache/kafka/streams/kstream/internals/KGroupedStreamImpl.java#L150
for an example. Obviously exactly-once semantics for Kafka haven't been
completed, so this would be at-least-once.
On Wed, 25 Jan 2017 at 08:43 Nick DeCoursin <n....@foodpanda.com>
wrote:
> From the documentation
> <http://docs.confluent.io/3.1.1/streams/developer-guide.html#id3>:
>
> The counting operation for record streams is trivial to implement: you can
> > maintain a local state store that tracks the latest count for each key,
> > and, upon receiving a new record, update the corresponding key by
> > incrementing its count by one.
>
>
> How? Is there any examples of this online? To me, it doesn't seem so
> trivial because there's no such thing as a transaction in Kafka,
>
> Thank you,
> Nick
>
> --
>
> Nick DeCoursin
> Software Engineer
> foodpanda
>
> Tel | +1 920 450 5434 <(920)%20450-5434>
>
> Mail | n.decoursin@foodpanda.com
>
> Skype | nick.foodpanda
>
> Foodpanda GmbH | Schreiberhauer Str. 30 | 10317 Berlin | Germany
> Sitz der Gesellschaft | Berlin, AG Charlottenburg | HRB 138224 B |
> USt-ID-Nr | DE 283789080
> Geschäftsführer | Benjamin Bauer, Felix Plog, Ralf Wenzel
>
> CONFIDENTIALITY NOTICE: This message (including any attachments) is
> confidential and may be privileged. It may be read, copied and used only by
> the intended recipient. If you have received it in error please contact the
> sender (by return e-mail) immediately and delete this message. Any
> unauthorized use or dissemination of this message in whole or in parts is
> strictly prohibited.
>