You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@flume.apache.org by "Juhani Connolly (JIRA)" <ji...@apache.org> on 2012/09/06 06:06:07 UTC
[jira] [Created] (FLUME-1541) Implement a SinkSelector for
LoadBalancingSinkProcessor that includes failover mechanics
Juhani Connolly created FLUME-1541:
--------------------------------------
Summary: Implement a SinkSelector for LoadBalancingSinkProcessor that includes failover mechanics
Key: FLUME-1541
URL: https://issues.apache.org/jira/browse/FLUME-1541
Project: Flume
Issue Type: Bug
Reporter: Juhani Connolly
Implement a new sink selector with the following behaviors:
- Round robin usage of sinks
- Temporarily removing sinks from the round robin pool when they fail
- Increasing timeout mechanics on sequential failures
- Spreading load equally over surviving sinks(current Round Robin sends all the failed sinks events to the next sink in line)
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (FLUME-1541) Implement a SinkSelector for
LoadBalancingSinkProcessor that includes failover mechanics
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/FLUME-1541?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13450264#comment-13450264 ]
Hudson commented on FLUME-1541:
-------------------------------
Integrated in flume-trunk #293 (See [https://builds.apache.org/job/flume-trunk/293/])
FLUME-1541. Implement failover for LoadBalancingSinkProcessor. (Revision d36861bcea4e3afeb08bfddd46ce0f7533186f00)
Result = UNSTABLE
mpercy : http://git-wip-us.apache.org/repos/asf/flume/repo?p=flume.git;a=summary&a=commit&h=d36861bcea4e3afeb08bfddd46ce0f7533186f00
Files :
* flume-ng-core/src/test/java/org/apache/flume/sink/TestLoadBalancingSinkProcessor.java
* flume-ng-core/src/main/java/org/apache/flume/sink/LoadBalancingSinkProcessor.java
* flume-ng-doc/sphinx/FlumeUserGuide.rst
* flume-ng-core/src/main/java/org/apache/flume/sink/AbstractSinkSelector.java
> Implement a SinkSelector for LoadBalancingSinkProcessor that includes failover mechanics
> ----------------------------------------------------------------------------------------
>
> Key: FLUME-1541
> URL: https://issues.apache.org/jira/browse/FLUME-1541
> Project: Flume
> Issue Type: Bug
> Components: Sinks+Sources
> Affects Versions: v1.3.0
> Reporter: Juhani Connolly
> Assignee: Juhani Connolly
> Fix For: v1.3.0
>
> Attachments: FLUME-1541-3.patch
>
>
> Implement a new sink selector with the following behaviors:
> - Round robin usage of sinks
> - Temporarily removing sinks from the round robin pool when they fail
> - Increasing timeout mechanics on sequential failures
> - Spreading load equally over surviving sinks(current Round Robin sends all the failed sinks events to the next sink in line)
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (FLUME-1541) Implement a SinkSelector for
LoadBalancingSinkProcessor that includes failover mechanics
Posted by "Juhani Connolly (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/FLUME-1541?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Juhani Connolly updated FLUME-1541:
-----------------------------------
Component/s: Sinks+Sources
Affects Version/s: v1.3.0
Fix Version/s: v1.3.0
> Implement a SinkSelector for LoadBalancingSinkProcessor that includes failover mechanics
> ----------------------------------------------------------------------------------------
>
> Key: FLUME-1541
> URL: https://issues.apache.org/jira/browse/FLUME-1541
> Project: Flume
> Issue Type: Bug
> Components: Sinks+Sources
> Affects Versions: v1.3.0
> Reporter: Juhani Connolly
> Fix For: v1.3.0
>
>
> Implement a new sink selector with the following behaviors:
> - Round robin usage of sinks
> - Temporarily removing sinks from the round robin pool when they fail
> - Increasing timeout mechanics on sequential failures
> - Spreading load equally over surviving sinks(current Round Robin sends all the failed sinks events to the next sink in line)
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (FLUME-1541) Implement a SinkSelector for
LoadBalancingSinkProcessor that includes failover mechanics
Posted by "Hari Shreedharan (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/FLUME-1541?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13449415#comment-13449415 ]
Hari Shreedharan commented on FLUME-1541:
-----------------------------------------
Juhani,
This is excellent. Do you think you could add this backoff mechanism to the SDK load balancing Rpc client also. It might just be copy paste code for now, but it would be good to refactor the load balancing code in the RpcClient and Sinks to use the same libraries.
> Implement a SinkSelector for LoadBalancingSinkProcessor that includes failover mechanics
> ----------------------------------------------------------------------------------------
>
> Key: FLUME-1541
> URL: https://issues.apache.org/jira/browse/FLUME-1541
> Project: Flume
> Issue Type: Bug
> Components: Sinks+Sources
> Affects Versions: v1.3.0
> Reporter: Juhani Connolly
> Fix For: v1.3.0
>
>
> Implement a new sink selector with the following behaviors:
> - Round robin usage of sinks
> - Temporarily removing sinks from the round robin pool when they fail
> - Increasing timeout mechanics on sequential failures
> - Spreading load equally over surviving sinks(current Round Robin sends all the failed sinks events to the next sink in line)
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (FLUME-1541) Implement a SinkSelector for
LoadBalancingSinkProcessor that includes failover mechanics
Posted by "Juhani Connolly (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/FLUME-1541?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Juhani Connolly updated FLUME-1541:
-----------------------------------
Attachment: FLUME-1541-3.patch
> Implement a SinkSelector for LoadBalancingSinkProcessor that includes failover mechanics
> ----------------------------------------------------------------------------------------
>
> Key: FLUME-1541
> URL: https://issues.apache.org/jira/browse/FLUME-1541
> Project: Flume
> Issue Type: Bug
> Components: Sinks+Sources
> Affects Versions: v1.3.0
> Reporter: Juhani Connolly
> Fix For: v1.3.0
>
> Attachments: FLUME-1541-3.patch
>
>
> Implement a new sink selector with the following behaviors:
> - Round robin usage of sinks
> - Temporarily removing sinks from the round robin pool when they fail
> - Increasing timeout mechanics on sequential failures
> - Spreading load equally over surviving sinks(current Round Robin sends all the failed sinks events to the next sink in line)
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (FLUME-1541) Implement a SinkSelector for
LoadBalancingSinkProcessor that includes failover mechanics
Posted by "Juhani Connolly (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/FLUME-1541?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13449462#comment-13449462 ]
Juhani Connolly commented on FLUME-1541:
----------------------------------------
I can have a look into it when I can get a bit of time. I haven't really touched the sdk at all, but I'm sure it should be ok
> Implement a SinkSelector for LoadBalancingSinkProcessor that includes failover mechanics
> ----------------------------------------------------------------------------------------
>
> Key: FLUME-1541
> URL: https://issues.apache.org/jira/browse/FLUME-1541
> Project: Flume
> Issue Type: Bug
> Components: Sinks+Sources
> Affects Versions: v1.3.0
> Reporter: Juhani Connolly
> Fix For: v1.3.0
>
>
> Implement a new sink selector with the following behaviors:
> - Round robin usage of sinks
> - Temporarily removing sinks from the round robin pool when they fail
> - Increasing timeout mechanics on sequential failures
> - Spreading load equally over surviving sinks(current Round Robin sends all the failed sinks events to the next sink in line)
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (FLUME-1541) Implement a SinkSelector for
LoadBalancingSinkProcessor that includes failover mechanics
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/FLUME-1541?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13450251#comment-13450251 ]
Hudson commented on FLUME-1541:
-------------------------------
Integrated in flume-1.3.0 #7 (See [https://builds.apache.org/job/flume-1.3.0/7/])
FLUME-1541. Implement failover for LoadBalancingSinkProcessor. (Revision 752f70f4fd053fa53650c0d7673a99c7610464f9)
Result = FAILURE
mpercy : http://git-wip-us.apache.org/repos/asf/flume/repo?p=flume.git;a=summary&a=commit&h=752f70f4fd053fa53650c0d7673a99c7610464f9
Files :
* flume-ng-core/src/main/java/org/apache/flume/sink/AbstractSinkSelector.java
* flume-ng-doc/sphinx/FlumeUserGuide.rst
* flume-ng-core/src/test/java/org/apache/flume/sink/TestLoadBalancingSinkProcessor.java
* flume-ng-core/src/main/java/org/apache/flume/sink/LoadBalancingSinkProcessor.java
> Implement a SinkSelector for LoadBalancingSinkProcessor that includes failover mechanics
> ----------------------------------------------------------------------------------------
>
> Key: FLUME-1541
> URL: https://issues.apache.org/jira/browse/FLUME-1541
> Project: Flume
> Issue Type: Bug
> Components: Sinks+Sources
> Affects Versions: v1.3.0
> Reporter: Juhani Connolly
> Assignee: Juhani Connolly
> Fix For: v1.3.0
>
> Attachments: FLUME-1541-3.patch
>
>
> Implement a new sink selector with the following behaviors:
> - Round robin usage of sinks
> - Temporarily removing sinks from the round robin pool when they fail
> - Increasing timeout mechanics on sequential failures
> - Spreading load equally over surviving sinks(current Round Robin sends all the failed sinks events to the next sink in line)
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira