You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@qpid.apache.org by Ramayan Tiwari <ra...@gmail.com> on 2016/08/23 02:02:16 UTC
Flow to disk behavior with In Memory messages
Hi all,
As I understand, flow to disk tries to protect Direct Memory by sending new
messages to disk when using some form of persistent.
What is the behavior when virtual host node type is "Memory". I was looking
at the implementation, StoredMemoryMessage doesn't seem to do anything in
case of flowToDisk(). So does that mean that Broker will ultimately shut
down with OOM if we keep sending messages ?
Thanks
Ramayan
Re: Flow to disk behavior with In Memory messages
Posted by Ramayan Tiwari <ra...@gmail.com>.
Thanks Lorenz for explaining that.
We are planning to use Memory VirtualHost(Node) in production as we do our
own persistence using database. We also monitor broker heap and stop
sending messages to broker at a certain threshold to protect broker against
OOM. Now that monitoring heap is not sufficient with the 6.0.x we are also
exploring alternatives to monitor Direct Memory as well.
Is there any shortcomings with using Memory vs say BDB in production
environment, other than of course persistence?
- Ramayan
On Tue, Aug 23, 2016 at 1:08 AM, Lorenz Quack <qu...@gmail.com>
wrote:
> Hi Ramayan,
>
> That is not entirely correct.
> Flow to Disk is only indirectly related to persistence.
>
> Persistent messages are *always* written to the message store
> before the message is acknowledged or the transaction completes.
> However, for performance reasons the broker will keep a copy of
> the message in memory. When flow to disk kicks in two things
> happen:
>
> 1) The in memory copy of new persistent messages will be
> discarded
>
> 2) New transient messages will also be written to the message
> store
>
> Note that I always wrote that the messages will be written to the
> message store and not to disk. In case of a Memory Virtual Host
> the message store is held in memory so there won't be any
> persistence. Not even for persistent messages. And, yes,
> eventually your broker will run out of memory.
>
> Memory VirtualHost(Node)s are more for testing purposes than
> anything else.
>
>
> Kind Regards,
> Lorenz
>
>
>
> On 23/08/16 03:02, Ramayan Tiwari wrote:
>
>> Hi all,
>>
>> As I understand, flow to disk tries to protect Direct Memory by sending
>> new
>> messages to disk when using some form of persistent.
>>
>> What is the behavior when virtual host node type is "Memory". I was
>> looking
>> at the implementation, StoredMemoryMessage doesn't seem to do anything in
>> case of flowToDisk(). So does that mean that Broker will ultimately shut
>> down with OOM if we keep sending messages ?
>>
>> Thanks
>> Ramayan
>>
>>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
> For additional commands, e-mail: users-help@qpid.apache.org
>
>
Re: Flow to disk behavior with In Memory messages
Posted by Lorenz Quack <qu...@gmail.com>.
Hi Ramayan,
That is not entirely correct.
Flow to Disk is only indirectly related to persistence.
Persistent messages are *always* written to the message store
before the message is acknowledged or the transaction completes.
However, for performance reasons the broker will keep a copy of
the message in memory. When flow to disk kicks in two things
happen:
1) The in memory copy of new persistent messages will be
discarded
2) New transient messages will also be written to the message
store
Note that I always wrote that the messages will be written to the
message store and not to disk. In case of a Memory Virtual Host
the message store is held in memory so there won't be any
persistence. Not even for persistent messages. And, yes,
eventually your broker will run out of memory.
Memory VirtualHost(Node)s are more for testing purposes than
anything else.
Kind Regards,
Lorenz
On 23/08/16 03:02, Ramayan Tiwari wrote:
> Hi all,
>
> As I understand, flow to disk tries to protect Direct Memory by sending new
> messages to disk when using some form of persistent.
>
> What is the behavior when virtual host node type is "Memory". I was looking
> at the implementation, StoredMemoryMessage doesn't seem to do anything in
> case of flowToDisk(). So does that mean that Broker will ultimately shut
> down with OOM if we keep sending messages ?
>
> Thanks
> Ramayan
>
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
For additional commands, e-mail: users-help@qpid.apache.org