You are viewing a plain text version of this content. The canonical link for it is here.

Posted to dev@flume.apache.org by "Juhani Connolly (Updated) (JIRA)" <ji...@apache.org> on 2012/02/03 07:41:53 UTC

[jira] [Updated] (FLUME-865) Implement failover sink

     [ https://issues.apache.org/jira/browse/FLUME-865?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Juhani Connolly updated FLUME-865:
----------------------------------

    Attachment: Flume-865.patch

I'm attaching my preliminary implementation for this if anyone wants to look at it, though I'm not posting to review board yet because this does not include changes to account for the work in progress from 949.

More importantly, I would like some input on the contract that sinks will need to agree to:
- For this runner to succesfully failover, there must be a common agreement on when that should be done. Right now some sinks will send EventDeliveryException, but some(like AvroSource), never give a clear signal of when they have died, instead throwing AvroRemoteException

My current implementation will switch as soon as an EventDeliveryException is thrown.
Furthermore, when dealing with retry, there will need to be some kind of mechanism to retry a failed sink. My current implementation just throws failed sinks into a deadSinks bin, and once it runs out, returns all of them to active use, letting them fail again until one is able to receive events.
                
> Implement failover sink 
> ------------------------
>
>                 Key: FLUME-865
>                 URL: https://issues.apache.org/jira/browse/FLUME-865
>             Project: Flume
>          Issue Type: New Feature
>          Components: Sinks+Sources
>    Affects Versions: NG alpha 2
>            Reporter: Jarek Jarcec Cecho
>            Assignee: Juhani Connolly
>         Attachments: Flume-865.patch
>
>
> It would be nice if the flume-ng would have ability to failover to different sink in case that the active one is not responding (e.g. before failing the transaction).

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira