Re: filter out events

[NerdyNick <ne...@gmail.com>]

Feature Jira is put in https://issues.apache.org/jira/browse/FLUME-723

On Fri, Jul 29, 2011 at 9:55 AM, Lior Harel <ha...@gmail.com> wrote:
>> Hi Nick,
>> That's definitely sounds like a good idea, a moment after submitting my questions I noticed there are similar questions in the flume users group. So I guess such functionality will be valuable.
>> Lior
> On Jul 27, 2011, at 6:19 PM, NerdyNick wrote:
>
>> Seeing as this idea of filtering events comes up a lot do you think a
>> decorator built into Flume that takes a RegEx and if it matches it
>> doesn't send the event would solve most use-cases out there. If this
>> sounds like a good idea I'll throw a Jira task up to do it as well as
>> make notes about the limitations to make sure they get into the docs
>> about the decorator. Of course we got to wait tell the new Jira is
>> ready.
>>
>> On Mon, Jul 25, 2011 at 4:41 PM, Jonathan Hsieh <jo...@cloudera.com> wrote:
>>> [Please subscribe to new flume-user@incubator.apache.org list,
>>> bcc flume-user@cloudera.org, cc flume-user@incubator.apache.org]
>>> Lior,
>>> You could write a decorator plugin in that only propagates the message that
>>> meet a particular criteria and not send the others.
>>> If you used BE or DFO mode, you could put this at the agent or the
>>> collector.
>>> If you are using this with the e2e mode, this would need to go to the agent.
>>>  This is because you would need to make sure that this filter happens
>>> *before* it gets to the agentE2E.  If it is after, you will remove messages
>>> that are needed to calculate a proper checksum!
>>> Jon.
>>>
>>> On Tue, Jul 12, 2011 at 11:21 PM, Lior H <ha...@gmail.com> wrote:
>>>>
>>>> I am looking for a way to drop events from flowing down the flume
>>>> stream.
>>>> For example, when tailing a directory of log files from different
>>>> processes, how would one collect errors and warnings only?
>>>
>>>
>>>
>>> --
>>> // Jonathan Hsieh (shay)
>>> // Software Engineer, Cloudera
>>> // jon@cloudera.com
>>>
>>>
>>
>>
>>
>> --
>> Nick Verbeck - NerdyNick
>> ----------------------------------------------------
>> NerdyNick.com
>> Coloco.ubuntu-rocks.org
>
>



-- 
Nick Verbeck - NerdyNick
----------------------------------------------------
NerdyNick.com
Coloco.ubuntu-rocks.org