diskfailover

[Torsten Curdt <tc...@vafer.org>]

I am not sure I understand the semantics of the diskfailover decorator
correctly.

  { decorator(...) => { diskFailover(10000) => sink(...) }};

If the sink throws an exception, the diskFailover should store the
events to disk.
That seems to work fine. But what about retry and replaying the events
from disk?
I thought I saw that working in 0.9.1 ...not anymore in 0.9.4

Looking at the source I found that the diskFailover decorator supports
two arguments, maxMillis and checkMillis. Unfortunately it's also not
quite obvious what that means exactly.

I assume/hope one of them is the delay between retries?

Is that somewhere in the docs and I didn't find it?

cheers,
Torsten

Re: diskfailover

[Torsten Curdt <tc...@vafer.org>]

Anyone?

On Wed, Jul 27, 2011 at 10:22 PM, Torsten Curdt <tc...@vafer.org> wrote:
> I am not sure I understand the semantics of the diskfailover decorator
> correctly.
>
>  { decorator(...) => { diskFailover(10000) => sink(...) }};
>
> If the sink throws an exception, the diskFailover should store the
> events to disk.
> That seems to work fine. But what about retry and replaying the events
> from disk?
> I thought I saw that working in 0.9.1 ...not anymore in 0.9.4
>
> Looking at the source I found that the diskFailover decorator supports
> two arguments, maxMillis and checkMillis. Unfortunately it's also not
> quite obvious what that means exactly.
>
> I assume/hope one of them is the delay between retries?
>
> Is that somewhere in the docs and I didn't find it?
>
> cheers,
> Torsten
>

Re: diskfailover

[Jonathan Hsieh <jo...@cloudera.com>]

Hi Torsten,

Ideally, you would be using a agentDFOSink instead of the diskFailover.
  diskFailover decorator is used used internally with in it.  I've created a
page on the new wiki that talks about feature status and the intent of
certain sources and sinks.

Here's the feature status page:
https://cwiki.apache.org/FLUME/feature-status.html

Jon.


On Wed, Jul 27, 2011 at 1:22 PM, Torsten Curdt <tc...@vafer.org> wrote:

> I am not sure I understand the semantics of the diskfailover decorator
> correctly.
>
>  { decorator(...) => { diskFailover(10000) => sink(...) }};
>
> If the sink throws an exception, the diskFailover should store the
> events to disk.
> That seems to work fine. But what about retry and replaying the events
> from disk?
> I thought I saw that working in 0.9.1 ...not anymore in 0.9.4
>
> Looking at the source I found that the diskFailover decorator supports
> two arguments, maxMillis and checkMillis. Unfortunately it's also not
> quite obvious what that means exactly.
>
> I assume/hope one of them is the delay between retries?
>
> Is that somewhere in the docs and I didn't find it?
>
> cheers,
> Torsten
>



-- 
// Jonathan Hsieh (shay)
// Software Engineer, Cloudera
// jon@cloudera.com