You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@kafka.apache.org by Hamza HACHANI <ha...@supcom.tn> on 2016/08/04 03:53:44 UTC
Re : A specific use case
Hi,
Yes in fact .
And ï found à solution.
It was in editing the method punctuate in kafka stream processor.
----- Message de réponse -----
De : "Guozhang Wang" <wa...@gmail.com>
Pour : "users@kafka.apache.org" <us...@kafka.apache.org>
Objet : A specific use case
Date : mer., août 3, 2016 23:38
Hello Hamza,
By saying "broker" I think you are actually referring to a Kafka Streams
instance?
Guozhang
On Mon, Aug 1, 2016 at 1:01 AM, Hamza HACHANI <ha...@supcom.tn>
wrote:
> Good morning,
>
> I'm working on a specific use case. In fact i'm receiving messages from an
> operator network and trying to do statistics on their number per
> minute,perhour,per day ...
>
> I would like to create a broker that receives the messages and generates a
> message every minute. These producted messages are consumed by a consumer
> from in one hand and also se,t to an other topic which receives them and
> generates messages every minute.
>
> I've been doing that for a while without a success. In fact the first
> broker in any time it receives a messages ,it produces one and send it to
> the other topic.
>
> My question is ,what i'm trying to do,Is it possible without passing by an
> intermediate java processus which is out of kafka.
>
> If yes , How ?
>
> Thanks In advance.
>
--
-- Guozhang
RE: Re : A specific use case
Posted by Hamza HACHANI <ha...@supcom.tn>.
Thanks Guozhang Wang.
Hamza
________________________________
De : Guozhang Wang <wa...@gmail.com>
Envoyé : jeudi 4 août 2016 06:58:22
À : users@kafka.apache.org
Objet : Re: Re : A specific use case
Yeah, if you can buffer yourself in the process() function and then rely on
punctuate() for generating the outputs that would resolve your issue.
Remember that punctuate() function itself is event-time driven so if you do
not have any data coming in then it may not be triggered. Details:
https://github.com/apache/kafka/pull/1689
Guozhang
On Wed, Aug 3, 2016 at 8:53 PM, Hamza HACHANI <ha...@supcom.tn>
wrote:
> Hi,
> Yes in fact .
> And ï found à solution.
> It was in editing the method punctuate in kafka stream processor.
>
> ----- Message de réponse -----
> De : "Guozhang Wang" <wa...@gmail.com>
> Pour : "users@kafka.apache.org" <us...@kafka.apache.org>
> Objet : A specific use case
> Date : mer., août 3, 2016 23:38
>
> Hello Hamza,
>
> By saying "broker" I think you are actually referring to a Kafka Streams
> instance?
>
>
> Guozhang
>
> On Mon, Aug 1, 2016 at 1:01 AM, Hamza HACHANI <ha...@supcom.tn>
> wrote:
>
> > Good morning,
> >
> > I'm working on a specific use case. In fact i'm receiving messages from
> an
> > operator network and trying to do statistics on their number per
> > minute,perhour,per day ...
> >
> > I would like to create a broker that receives the messages and generates
> a
> > message every minute. These producted messages are consumed by a consumer
> > from in one hand and also se,t to an other topic which receives them and
> > generates messages every minute.
> >
> > I've been doing that for a while without a success. In fact the first
> > broker in any time it receives a messages ,it produces one and send it to
> > the other topic.
> >
> > My question is ,what i'm trying to do,Is it possible without passing by
> an
> > intermediate java processus which is out of kafka.
> >
> > If yes , How ?
> >
> > Thanks In advance.
> >
>
>
>
> --
> -- Guozhang
>
--
-- Guozhang
Re: Re : A specific use case
Posted by Guozhang Wang <wa...@gmail.com>.
Yeah, if you can buffer yourself in the process() function and then rely on
punctuate() for generating the outputs that would resolve your issue.
Remember that punctuate() function itself is event-time driven so if you do
not have any data coming in then it may not be triggered. Details:
https://github.com/apache/kafka/pull/1689
Guozhang
On Wed, Aug 3, 2016 at 8:53 PM, Hamza HACHANI <ha...@supcom.tn>
wrote:
> Hi,
> Yes in fact .
> And ï found à solution.
> It was in editing the method punctuate in kafka stream processor.
>
> ----- Message de réponse -----
> De : "Guozhang Wang" <wa...@gmail.com>
> Pour : "users@kafka.apache.org" <us...@kafka.apache.org>
> Objet : A specific use case
> Date : mer., août 3, 2016 23:38
>
> Hello Hamza,
>
> By saying "broker" I think you are actually referring to a Kafka Streams
> instance?
>
>
> Guozhang
>
> On Mon, Aug 1, 2016 at 1:01 AM, Hamza HACHANI <ha...@supcom.tn>
> wrote:
>
> > Good morning,
> >
> > I'm working on a specific use case. In fact i'm receiving messages from
> an
> > operator network and trying to do statistics on their number per
> > minute,perhour,per day ...
> >
> > I would like to create a broker that receives the messages and generates
> a
> > message every minute. These producted messages are consumed by a consumer
> > from in one hand and also se,t to an other topic which receives them and
> > generates messages every minute.
> >
> > I've been doing that for a while without a success. In fact the first
> > broker in any time it receives a messages ,it produces one and send it to
> > the other topic.
> >
> > My question is ,what i'm trying to do,Is it possible without passing by
> an
> > intermediate java processus which is out of kafka.
> >
> > If yes , How ?
> >
> > Thanks In advance.
> >
>
>
>
> --
> -- Guozhang
>
--
-- Guozhang