You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@flume.apache.org by pravesh suyal <pr...@gmail.com> on 2014/05/06 09:05:45 UTC
Flume-OG Chokes alternatives in Flume-NG to control data transfer limit
Dear Flume community,
Is there is any alternative to the Flume-OG Chokes in the Flume-NG?
Chokes served the purpose well to control the data transfer rate limit
across the Sources-to-Sinks.
Is this been dropped from Flume-NG purposely? OR there is an alternative?
Regards
--
Pravesh Suyal
Re: Flume-OG Chokes alternatives in Flume-NG to control data transfer limit
Posted by Arvind Prabhakar <ar...@apache.org>.
At the moment, the only way to control the drain rate will be by the number
of active sinks. Within a sink though, there is no control the drain rate.
Having a drain rate throttle within the sink would be an overkill and may
not add much value since the sink itself is bound by batch size and single
thread transfer semantics.
Regards,
Arvind Prabhakar
On Thu, May 8, 2014 at 5:47 AM, pravesh suyal <pr...@gmail.com>wrote:
> Thanx Arvind.
>
> Is the steady rate configurable through the flume configurations?
> Otherwise, that would mean we need to adjust through adjusting the total
> number of Sinks? In case I just need a single Source/Sink pairing and still
> would want to control the drain rate?
>
> Regards
> Pravesh
>
>
> On Tue, May 6, 2014 at 9:44 PM, Arvind Prabhakar <ar...@apache.org>wrote:
>
>> This is indirectly available in Flume (NG) via the number of sinks you
>> setup. Each sink operates on a single thread that drains the channel at a
>> steady rate. If you want to have a lower drain rate, reduce the number of
>> sinks; and conversely increase them for higher drain rate.
>>
>> Regards,
>> Arvind Prabhakar
>>
>>
>> On Tue, May 6, 2014 at 12:05 AM, pravesh suyal <pr...@gmail.com>wrote:
>>
>>>
>>> Dear Flume community,
>>>
>>> Is there is any alternative to the Flume-OG Chokes in the Flume-NG?
>>>
>>> Chokes served the purpose well to control the data transfer rate limit
>>> across the Sources-to-Sinks.
>>>
>>> Is this been dropped from Flume-NG purposely? OR there is an alternative?
>>>
>>> Regards
>>> --
>>> Pravesh Suyal
>>>
>>
>>
>
>
> --
> Pravesh Suyal
>
Re: Flume-OG Chokes alternatives in Flume-NG to control data transfer limit
Posted by pravesh suyal <pr...@gmail.com>.
Thanx Arvind.
Is the steady rate configurable through the flume configurations?
Otherwise, that would mean we need to adjust through adjusting the total
number of Sinks? In case I just need a single Source/Sink pairing and still
would want to control the drain rate?
Regards
Pravesh
On Tue, May 6, 2014 at 9:44 PM, Arvind Prabhakar <ar...@apache.org> wrote:
> This is indirectly available in Flume (NG) via the number of sinks you
> setup. Each sink operates on a single thread that drains the channel at a
> steady rate. If you want to have a lower drain rate, reduce the number of
> sinks; and conversely increase them for higher drain rate.
>
> Regards,
> Arvind Prabhakar
>
>
> On Tue, May 6, 2014 at 12:05 AM, pravesh suyal <pr...@gmail.com>wrote:
>
>>
>> Dear Flume community,
>>
>> Is there is any alternative to the Flume-OG Chokes in the Flume-NG?
>>
>> Chokes served the purpose well to control the data transfer rate limit
>> across the Sources-to-Sinks.
>>
>> Is this been dropped from Flume-NG purposely? OR there is an alternative?
>>
>> Regards
>> --
>> Pravesh Suyal
>>
>
>
--
Pravesh Suyal
Re: Flume-OG Chokes alternatives in Flume-NG to control data transfer limit
Posted by Arvind Prabhakar <ar...@apache.org>.
This is indirectly available in Flume (NG) via the number of sinks you
setup. Each sink operates on a single thread that drains the channel at a
steady rate. If you want to have a lower drain rate, reduce the number of
sinks; and conversely increase them for higher drain rate.
Regards,
Arvind Prabhakar
On Tue, May 6, 2014 at 12:05 AM, pravesh suyal <pr...@gmail.com>wrote:
>
> Dear Flume community,
>
> Is there is any alternative to the Flume-OG Chokes in the Flume-NG?
>
> Chokes served the purpose well to control the data transfer rate limit
> across the Sources-to-Sinks.
>
> Is this been dropped from Flume-NG purposely? OR there is an alternative?
>
> Regards
> --
> Pravesh Suyal
>