You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@samza.apache.org by ramesh shanmugam <rs...@12digitmedia.com> on 2016/02/05 01:41:21 UTC

Samza Throttling clarification

Hi Team,

We have use case that destination system accept only 160 messages per
minute.

Planning to use Samza with Kafka. Looking for out of box throttling feature
but not able locate in documenttion.

Can you please suggest how throttling can be achieved in Samza?.

-- 
Thanks
Ramesh

Re: Samza Throttling clarification

Posted by Rick Mangi <ri...@chartbeat.com>.
Can I ask why you’re using samza and kafka for this?


> On Feb 4, 2016, at 7:41 PM, ramesh shanmugam <rs...@12digitmedia.com> wrote:
> 
> Hi Team,
> 
> We have use case that destination system accept only 160 messages per
> minute.
> 
> Planning to use Samza with Kafka. Looking for out of box throttling feature
> but not able locate in documenttion.
> 
> Can you please suggest how throttling can be achieved in Samza?.
> 
> --
> Thanks
> Ramesh

Re: Samza Throttling clarification

Posted by Benjamin Edwards <ed...@gmail.com>.
Recommend token bukets! Here is a solid implementation:
https://github.com/bbeck/token-bucket

On Fri, 5 Feb 2016 at 03:33 Jagadish Venkatraman <ja...@gmail.com>
wrote:

> Hi Ramesh,
>
> I understand that you want to limit the calls to the destination service.
> Currently, there's no framework level support for throttling in Samza
> (yet). But, since Samza is single threaded, you can trivially implement
> some form of throttling by calling Thread.sleep() in the StreamTask.process
> method.
>
> There's a Jira open for this: https://wiki.apache.org/samza/Throttling .
> Also please see https://issues.apache.org/jira/browse/SAMZA-24 for a
> discussion on this feature.
>
> Thanks,
> Jagadish
>
> On Thu, Feb 4, 2016 at 4:41 PM, ramesh shanmugam <
> rshanmugam@12digitmedia.com> wrote:
>
> > Hi Team,
> >
> > We have use case that destination system accept only 160 messages per
> > minute.
> >
> > Planning to use Samza with Kafka. Looking for out of box throttling
> feature
> > but not able locate in documenttion.
> >
> > Can you please suggest how throttling can be achieved in Samza?.
> >
> > --
> > Thanks
> > Ramesh
> >
>
>
>
> --
> Jagadish V,
> Graduate Student,
> Department of Computer Science,
> Stanford University
>

Re: Samza Throttling clarification

Posted by Jagadish Venkatraman <ja...@gmail.com>.
Hi Ramesh,

I understand that you want to limit the calls to the destination service.
Currently, there's no framework level support for throttling in Samza
(yet). But, since Samza is single threaded, you can trivially implement
some form of throttling by calling Thread.sleep() in the StreamTask.process
method.

There's a Jira open for this: https://wiki.apache.org/samza/Throttling .
Also please see https://issues.apache.org/jira/browse/SAMZA-24 for a
discussion on this feature.

Thanks,
Jagadish

On Thu, Feb 4, 2016 at 4:41 PM, ramesh shanmugam <
rshanmugam@12digitmedia.com> wrote:

> Hi Team,
>
> We have use case that destination system accept only 160 messages per
> minute.
>
> Planning to use Samza with Kafka. Looking for out of box throttling feature
> but not able locate in documenttion.
>
> Can you please suggest how throttling can be achieved in Samza?.
>
> --
> Thanks
> Ramesh
>



-- 
Jagadish V,
Graduate Student,
Department of Computer Science,
Stanford University