You are viewing a plain text version of this content. The canonical link for it is here.

Posted to dev@flume.apache.org by "Hari Shreedharan (JIRA)" <ji...@apache.org> on 2013/03/14 01:22:12 UTC

[jira] [Updated] (FLUME-1947) ExecSource should optionally block if the channel is full.

     [ https://issues.apache.org/jira/browse/FLUME-1947?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Hari Shreedharan updated FLUME-1947:
------------------------------------

    Attachment: FLUME-1947.patch

Made multiple changes to the exec source:

1. Flush the events to the channel, based on a configurable timeout - even when a full batch was not received.
2. Block the spawned process until channel exceptions are cleared up by not reading the stdout buffer. Backoff exponentially (subject to an upper limit), till events are available.
                
> ExecSource should optionally block if the channel is full.
> ----------------------------------------------------------
>
>                 Key: FLUME-1947
>                 URL: https://issues.apache.org/jira/browse/FLUME-1947
>             Project: Flume
>          Issue Type: Bug
>            Reporter: Hari Shreedharan
>         Attachments: FLUME-1947.patch
>
>
> This is so that the process executed blocks till the channel clears up. This is exceptionally useful if the process is a tail -F or something like that. 
> In addition, the exec source does have issues with respect to the last batch not being sent. We could also specify a timeout at the end of which the batch should be flushed to the channel if there are no new events.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira