You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@flume.apache.org by Ronald Van De Kuil <ro...@gmail.com> on 2016/04/15 12:38:41 UTC
Priority and Expiry time of flume events
Hello,
Can anyone confirm my understanding that there is neither priority of flume events nor expiry of events in flume 1.6.0?
I could not find it in Steve Hoffman's book nor in the user manual.
If so, are there plans to add it in the future?
Met vriendelijke groet,
Ronald van de Kuil
Re: Priority and Expiry time of flume events
Posted by Chris Horrocks <ch...@hor.rocks>.
The Kafka channel would allow you to set event retention within Kafka.
> On 15 Apr 2016, at 12:54, Gonzalo Herreros <gh...@gmail.com> wrote:
>
> That would depend on the channel.
> AFAIK, all the channels provided are FIFO without expiration but technically you could implement a channel that does that.
>
> You could achieve some priority management using multiplexing.
>
> Gonzalo
>
>> On 15 April 2016 at 11:38, Ronald Van De Kuil <ro...@gmail.com> wrote:
>> Hello,
>>
>> Can anyone confirm my understanding that there is neither priority of flume events nor expiry of events in flume 1.6.0?
>>
>> I could not find it in Steve Hoffman's book nor in the user manual.
>>
>> If so, are there plans to add it in the future?
>>
>> Met vriendelijke groet,
>> Ronald van de Kuil
>
Re: Priority and Expiry time of flume events
Posted by Ronald Van De Kuil <ro...@gmail.com>.
Thank you all for the tips.
Met vriendelijke groet,
Ronald van de Kuil
> Op 15 apr. 2016 om 14:31 heeft Gonzalo Herreros <gh...@gmail.com> het volgende geschreven:
>
> I think so.
> I hadn't considered kafka retention, however it might not have enough precision for this purpose if you want accurate expiry dates
>
>> On 15 April 2016 at 13:27, Ronald van de Kuil <ro...@gmail.com> wrote:
>> I see, so there could be a channel that is routed to if some header matches (the priority lane), and if there is a nomatch then it would route to the default channel. And if I would need more then I would need to make a code mod, right?
>>
>>
>> 2016-04-15 13:54 GMT+02:00 Gonzalo Herreros <gh...@gmail.com>:
>>> That would depend on the channel.
>>> AFAIK, all the channels provided are FIFO without expiration but technically you could implement a channel that does that.
>>>
>>> You could achieve some priority management using multiplexing.
>>>
>>> Gonzalo
>>>
>>>> On 15 April 2016 at 11:38, Ronald Van De Kuil <ro...@gmail.com> wrote:
>>>> Hello,
>>>>
>>>> Can anyone confirm my understanding that there is neither priority of flume events nor expiry of events in flume 1.6.0?
>>>>
>>>> I could not find it in Steve Hoffman's book nor in the user manual.
>>>>
>>>> If so, are there plans to add it in the future?
>>>>
>>>> Met vriendelijke groet,
>>>> Ronald van de Kuil
>
Re: Priority and Expiry time of flume events
Posted by Gonzalo Herreros <gh...@gmail.com>.
I think so.
I hadn't considered kafka retention, however it might not have enough
precision for this purpose if you want accurate expiry dates
On 15 April 2016 at 13:27, Ronald van de Kuil <ro...@gmail.com>
wrote:
> I see, so there could be a channel that is routed to if some header
> matches (the priority lane), and if there is a nomatch then it would route
> to the default channel. And if I would need more then I would need to make
> a code mod, right?
>
>
> 2016-04-15 13:54 GMT+02:00 Gonzalo Herreros <gh...@gmail.com>:
>
>> That would depend on the channel.
>> AFAIK, all the channels provided are FIFO without expiration but
>> technically you could implement a channel that does that.
>>
>> You could achieve some priority management using multiplexing.
>>
>> Gonzalo
>>
>> On 15 April 2016 at 11:38, Ronald Van De Kuil <
>> ronald.van.de.kuil@gmail.com> wrote:
>>
>>> Hello,
>>>
>>> Can anyone confirm my understanding that there is neither priority of
>>> flume events nor expiry of events in flume 1.6.0?
>>>
>>> I could not find it in Steve Hoffman's book nor in the user manual.
>>>
>>> If so, are there plans to add it in the future?
>>>
>>> Met vriendelijke groet,
>>>
>>> Ronald van de Kuil
>>>
>>>
>>
>
Re: Priority and Expiry time of flume events
Posted by Ronald van de Kuil <ro...@gmail.com>.
I see, so there could be a channel that is routed to if some header matches
(the priority lane), and if there is a nomatch then it would route to the
default channel. And if I would need more then I would need to make a code
mod, right?
2016-04-15 13:54 GMT+02:00 Gonzalo Herreros <gh...@gmail.com>:
> That would depend on the channel.
> AFAIK, all the channels provided are FIFO without expiration but
> technically you could implement a channel that does that.
>
> You could achieve some priority management using multiplexing.
>
> Gonzalo
>
> On 15 April 2016 at 11:38, Ronald Van De Kuil <
> ronald.van.de.kuil@gmail.com> wrote:
>
>> Hello,
>>
>> Can anyone confirm my understanding that there is neither priority of
>> flume events nor expiry of events in flume 1.6.0?
>>
>> I could not find it in Steve Hoffman's book nor in the user manual.
>>
>> If so, are there plans to add it in the future?
>>
>> Met vriendelijke groet,
>>
>> Ronald van de Kuil
>>
>>
>
Re: Priority and Expiry time of flume events
Posted by Gonzalo Herreros <gh...@gmail.com>.
That would depend on the channel.
AFAIK, all the channels provided are FIFO without expiration but
technically you could implement a channel that does that.
You could achieve some priority management using multiplexing.
Gonzalo
On 15 April 2016 at 11:38, Ronald Van De Kuil <ro...@gmail.com>
wrote:
> Hello,
>
> Can anyone confirm my understanding that there is neither priority of
> flume events nor expiry of events in flume 1.6.0?
>
> I could not find it in Steve Hoffman's book nor in the user manual.
>
> If so, are there plans to add it in the future?
>
> Met vriendelijke groet,
>
> Ronald van de Kuil
>
>