Re: Is Memory Channel data lost on process stop?

[Otis Gospodnetic <ot...@gmail.com>]

Thanks for the info, Mike.  Added
https://issues.apache.org/jira/browse/FLUME-2396 to address persisting
in-memory data to disk on (clean) shutdown.

Otis
--
Performance Monitoring * Log Analytics * Search Analytics
Solr & Elasticsearch Support * http://sematext.com/


On Wed, May 28, 2014 at 5:54 PM, Mike Percy <mp...@apache.org> wrote:

> The traditional memory channel does not store anything to disk on process
> shutdown, so you lose the data if you kill the process. However, a plain
> reconfiguration will not reinitialize or clear the channel as long as the
> channel name remains the same.
>
> The recoverable memory channel thing (FLUME-896) was never stable and was
> removed some time back.
>
> As far as I know, the spillable memory channel does not today spill to
> disk on shutdown, but one could conceive of adding this feature, since it
> uses its own internal file channel instance to "spill" when its memory
> capacity has been exceeded.
>
> Mike
>
>
> On Wed, May 28, 2014 at 1:44 PM, Christopher Shannon <
> cshannon108@gmail.com> wrote:
>
>> Good question; I look forward to someone cogently answering it.
>>
>> Related, and you probably know this already, the only signals that invoke
>> the JVM shutdown hooks (clean shutdown) are HUP and TERM. (kill -HUP <PID>;
>> kill -TERM <PID>).
>> On May 28, 2014 3:37 PM, "Otis Gospodnetic" <ot...@gmail.com>
>> wrote:
>>
>>> Hi,
>>>
>>> What happens with events buffered in the Memory channel (say via the
>>> Spillable version - https://issues.apache.org/jira/browse/FLUME-1227 or
>>> via its Durable cousin - https://issues.apache.org/jira/browse/FLUME-896
>>> ) when the process is stopped?
>>>
>>> Say the process is stopped gently and appropriately, via an init.d
>>> script or a gentle kill (not kill -9).
>>>
>>> Is the data that was in memory still lost?
>>> Or is there a Shutdown hook that gets run to preserve data on disk?
>>>
>>> Thanks,
>>> Otis
>>> --
>>> Performance Monitoring * Log Analytics * Search Analytics
>>> Solr & Elasticsearch Support * http://sematext.com/
>>>
>>
>