[jira] [Commented] (APEXMALHAR-2045) Add bandwidth control feature to Apex

["Priyanka Gugale (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/APEXMALHAR-2045?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15226158#comment-15226158 ] 

Priyanka Gugale commented on APEXMALHAR-2045:
---------------------------------------------

high level design we are considering, We would have following components:
1. BandwidthManager
This keeps track of current bandwidth usage of system and takes decision if requested data bandwidth can be used right away or not. To do this it used Leaky bucket algorithm where it emits data as long as it has not overused bandwidth (i.e. bandwidth consumption is >=0) and then wait to accumulate bandwidth for a while (till bandwidth goes from -ve value to +ve).

2. BandwidthLimitingInputOperator
Any Input operator which want to implement bandwidth restriction should implement BandwidthLimitingInputOperator. The operator have abstract method  to initialize instance of BandwidthManager and a method to emit tuple with bandwidth restriction to emit tuples as per available bandwidth.

3. BandwidthPartitioner
Bandwidth partitioner is introduced for static partitioning. If static partitioning is used by default StatelessPartitioner class is initialized. With bandwidth restriction we want to equally divide bandwidth amongst available partitions. BandwidthPartitioner should take care of it. It extends StatelessPartitioner, it just sets right bandwidth on all partitions after StatelessPartitioner creates/deletes partitiolns. In case of dynamic partitioning the operator implementing definePartitions, should take care of bandwidth distribution.

This design takes care of basic bandwidth restriction, also takes care of partitions by equally distributing available bandwidth among all partitions. Also this is open enough to do further modifications to take care of complex situations. 

> Add bandwidth control feature to Apex
> -------------------------------------
>
>                 Key: APEXMALHAR-2045
>                 URL: https://issues.apache.org/jira/browse/APEXMALHAR-2045
>             Project: Apache Apex Malhar
>          Issue Type: Task
>            Reporter: Priyanka Gugale
>            Assignee: Priyanka Gugale
>
>  bandwidth restrictions on input operator for number of bytes to be consumed per second.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)