Question on C++ broker per message marginal memory cost

[Robert Greig <ro...@gmail.com>]

Hi C++ devs,

I have recently been looking at the footprint of the Java broker, in
particular when using persistent messaging, and how much memory each
message will take up (assuming no consumers in the system).

What does the C++ broker do here?

RG

Re: Question on C++ broker per message marginal memory cost

[Carl Trieloff <cc...@redhat.com>]

Robert Greig wrote:
> 2008/5/15 Carl Trieloff <cc...@redhat.com>:
>
>   
>> 1. keep a reference to each message in memory  (so this is the limiting
>> factor)
>> 2. then we can release to content of the message (for both durable or
>> transient messages)
>>
>> 3. there are two policies for releasing a message, they are
>>   - the size of any given message (i.e. larger than X bytes then release
>> content)
>>   - the depth of a queue is more than Y, then start releasing all messages
>> to disk.
>>     
>
> Yes, this is the general idea in Java too but the crux of the issue is
> how you deal with the enqueuing information etc as well as message
> payload. You start out with a couple of pointer but quickly on a 64
> bit machine you can end up storing 128 bytes per message or more.
>
> RG
>   

yea, we have one holder object per message that says in memory for the 
current impl even
when all the content has been released. So yes, in the current impl we 
still eat a few bytes per
message regardless of released content. could probably add a policy to 
release this too (as the store
provides queue order effieciently) if durable but we don't do that today.

Carl.

Re: Question on C++ broker per message marginal memory cost

[Robert Greig <ro...@gmail.com>]

2008/5/15 Carl Trieloff <cc...@redhat.com>:

> 1. keep a reference to each message in memory  (so this is the limiting
> factor)
> 2. then we can release to content of the message (for both durable or
> transient messages)
>
> 3. there are two policies for releasing a message, they are
>   - the size of any given message (i.e. larger than X bytes then release
> content)
>   - the depth of a queue is more than Y, then start releasing all messages
> to disk.

Yes, this is the general idea in Java too but the crux of the issue is
how you deal with the enqueuing information etc as well as message
payload. You start out with a couple of pointer but quickly on a 64
bit machine you can end up storing 128 bytes per message or more.

RG

Re: Question on C++ broker per message marginal memory cost

[Carl Trieloff <cc...@redhat.com>]

Robert Greig wrote:
> Hi C++ devs,
>
> I have recently been looking at the footprint of the Java broker, in
> particular when using persistent messaging, and how much memory each
> message will take up (assuming no consumers in the system).
>
> What does the C++ broker do here?
>
> RG
>   


Few things, Gordon wrote that piece but is out so I will provide some 
details which
he might add to when back.

1. keep a reference to each message in memory  (so this is the limiting 
factor)
2. then we can release to content of the message (for both durable or 
transient messages)

3. there are two policies for releasing a message, they are
    - the size of any given message (i.e. larger than X bytes then 
release content)
    - the depth of a queue is more than Y, then start releasing all 
messages to disk.


Hope that helps
Carl.