You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@qpid.apache.org by Praveen M <le...@gmail.com> on 2012/01/14 03:27:26 UTC
override to turn BDBMessageStore syncCommit to false?
Hi,
I would like to exploit the Berkeley DB's nosyncCommit (lazy commit)
option to exploit enqueue performance.
I see that in the current BDBMessageStore implementation there is no way to
override the syncCommit Boolean value and
would like to know if this could be made into an option.
Also, that said It'll be nice to know if someone had used the
nosyncCommit() option that BDB provides and if using it does any invisible
harm. (DB corruptions etc.).
Now, to why do I want to override that value to false? (I do realize that
turning it to false, doesn't guarantee that the message was written to disk)
In my use case. I have a database outside Qpid which persists all the
messages enqueued (for durability and transactionality with message
callback operations.).
My sequence of operations is,
1) Enqueue messages -> persist message in my database outside Qpid ->
enqueue to Qpid in persistant mode
Since my message is already persisted in my database, I'm ok with losing a
message in BDB, and would like to enhance throughput of my overall enqueue
operation by using the nosyncCommit option.
The reason I'm not using non-persistant mode in Qpid is because I don't
want to overwhelm the memory available by flooding the broker with messages.
In the event of a broker crash, I plan to restore the messages from my
database outside Qpid. (I realize this could turn out a little expensive).
I've been thinking
that if there is a replication/HA solution to have messages go to a primary
and secondary broker (in a 2 instance cluster), in the event of a crash,
I'll have to
only restore the most recent messages from my database outside Qpid upto a
certain checkpoint to which the primary and secondary are sync'ed (Not sure
if this is achievable.).
I do realize that my architecture is a little complicated. Please do let me
know if you have more questions about what I'm trying to achieve here, I'll
be happy to
clarify doubts.
I'd really like to know if I can add an option to override the syncCommit
to false without any major side-effects apart from what I expect.
Thank you,
--
-Praveen
Re: override to turn BDBMessageStore syncCommit to false?
Posted by Robbie Gemmell <ro...@gmail.com>.
Hi Praveen,
Doing asynchronous commits to the store in the way implemented for the
non-transactional sessions is basically not compatible with the use of
transactions. Completion of the transaction is the guarantee that the
persistent message was put on disk when using a persistent store
(which is really the key reason to use persistence+transactions in the
first place, guarantees) so the only way to achieve any performance
increase in that situation would basically be for the broker to
outright lie, which gets you back to having almost the same guarantees
(i.e none really) as the non-transactional case.
By removing that guarantee the message was safe on disk you wouldnt
really be in much/any better a position than using a non-transacted
publishing session, because although you could definitively say the
message had hit the broker for processing you would still never
actually know whether a given message had made it to disk and onto a
queue if the broker were to go to down. Furthermore, you could quite
likely still be committing the message to your own database before it
was on disk at the broker, in which case you may as well do that
before you ever try sending it and just use non-transactional
publishing. Whether a message was to be lost during transit to the
broker or after reaching the broker but before being safe on disk if
using async store commits, your application would still need to be
capable of replaying messages from the database and possibly being
able to handle duplicates, so it doesnt really change anything from an
application implementation point of view either.
Given all that, it seems like the changes Rob made to the Java brokers
handling of persistent messages on non-transacted sessions should suit
your usage just as well if not better than introducing the
irregular(/broken from most viewpoints) transactional behaviour would.
In fact, the behaviour there now could be noticably faster than your
proposal in many cases due to the lack of transactional round trips,
again seeming to make it more suited to the general desire of
increasing persistent messaging performance with some acceptability of
losses.
Robbie
On 28 January 2012 01:42, Praveen M <le...@gmail.com> wrote:
> Hi Rob,
>
>
> Sorry for the late reply. I finally got around to reading your changelist
> and also playing with it.
> I realize that the change you've done is for sessions with auto_commit mode.
>
> I was wondering if the async override could be applied to transacted mode
> too.
>
> If you read my earlier mail,
>
> " My sequence of operations is,
> 1) Enqueue messages -> persist message in my database outside Qpid ->
> enqueue to Qpid in persistant mode
> Since my message is already persisted in my database, I'm ok
> with losing amessage in BDB, and would like to enhance throughput of my
> overall enqueue"
>
> Though persisting the message in my database, the reason why I'm using
> transacted session is to protect against a case like below.
>
> If I use auto_commit, the sequeue of operations are.
> 1) my enqueue server -> writes to my database outside qpid -> enqueue to
> qpid (auto_commit) -> commit my database write outside qpid
>
> But there is a case where, after i do the enqueue to qpid and commit my
> database write, my enqueue server can crash and the auto_committed message
> might be lost, as there is
> no guarantee that it has reached the broker (could be still in network
> buffer or in transit..but not yet reached the broker).
>
> To avoid this scenario I was thinking of two options.
>
> To use publish_sync option with auto_commit
> 1) my enqueue server -> writes to my database outside qpid -> enqueue to
> qpid (auto_commit with publish_sync) -> commit my database write outside
> qpid
>
> The downside to this is, publish_sync isn't really a JMS terminology and
> also, on the client side, there is a lock held on the connection with
> publish_sync option, avoiding other sessions sharing the connection to make
> progress. I spoke to Robbie about this earlier, and his suggestion was to
> try and stick to JMS terminology if possible.
>
> 2) Use transacted mode commit
> my enqueue server -> writes to my database outside qpid -> enqueue to qpid
> (transacted_mode) -> commit qpid session-> commit my database write
> outside qpid
>
> This looked pretty reliable and conforms to JMS, and I was hoping to use
> this, as there is a guarantee that the message is pushed to the broker
> before I write to my own database.
> The one enhancement that I thought will improve the performance of the
> roundtrip for this case was, to set syncCommit to false, so that I get the
> guarantee that the broker got my message, but also return back quickly than
> wait for the write to happen(which takes most of the roundtrip time in the
> transaction).
>
> I do understand that what I am trying to achieve is quite different from
> normal usage.
>
> I'd like to hear what you think about this problem and what you would
> suggest as my best option in this case.
>
> Thanks,
> Praveen
>
>
>
> Thanks,
> Praveen
>
>
> On Sat, Jan 21, 2012 at 12:00 PM, Rob Godfrey <ro...@gmail.com>wrote:
>
>> Hi Praveen,
>>
>> I've checked in my changes on trunk if you want to have a play around...
>> let me know if you run into any issues,
>>
>> Cheers,
>> Rob
>>
>> On 20 January 2012 16:57, Praveen M <le...@gmail.com> wrote:
>>
>> > Hi Rob,
>> >
>> > Thanks for writing. Please see inline.
>> >
>> > On Fri, Jan 20, 2012 at 1:27 AM, Rob Godfrey <rob.j.godfrey@gmail.com
>> > >wrote:
>> >
>> > > Hi Praveen,
>> > >
>> > > apologies for the slow response here...
>> > >
>> > > On 14 January 2012 03:27, Praveen M <le...@gmail.com> wrote:
>> > >
>> > > > Hi,
>> > > >
>> > > > I would like to exploit the Berkeley DB's nosyncCommit (lazy
>> commit)
>> > > > option to exploit enqueue performance.
>> > > > I see that in the current BDBMessageStore implementation there is no
>> > way
>> > > to
>> > > > override the syncCommit Boolean value and
>> > > > would like to know if this could be made into an option.
>> > > >
>> > > > Also, that said It'll be nice to know if someone had used the
>> > > > nosyncCommit() option that BDB provides and if using it does any
>> > > invisible
>> > > > harm. (DB corruptions etc.).
>> > > >
>> > > >
>> > > So, I have coincidentally been working on a patch to increase
>> throughput
>> > > for non transactional messages which makes greater use of no-sync
>> > commits.
>> > > The patch is basically to allow a certain amount of "out-of-order"
>> > > processing while still maintaining the guarantees on durability that
>> the
>> > > AMQP specification states that you should get. For non transactional
>> > > durable messages this is giving me something like a 6.5x increase in
>> max
>> > > total throughput.
>> > >
>> >
>> > *Ah, this is great. Exactly what I was looking for. *
>> >
>> > >
>> > > Unfortunately this patch did uncover a couple of issues, one being the
>> > need
>> > > to have some code in Qpid to handle occasional BDB lock timeouts... and
>> > the
>> > > other a very occasional bug in BDB itself which they guys at Oracle now
>> > > have a fix for, but won't be publicly released until their regular
>> > update.
>> > > There is a reasonably easy workaround for this which I shall
>> incorporate
>> > > into my patch, but it may take me a couple more days to get it all
>> > > complete.
>> > >
>> > > Note that my patch is a little different to what you are suggesting -
>> in
>> > > your case a message may go on to a queue in memory and then be
>> delivered
>> > > without the guarantee that it was sync'd to disk... in my patch the
>> > message
>> > > won;t be released into the in-memory queue until it is sync'd, however
>> I
>> > am
>> > > batching up more such enqueues to get better performance. In my tests
>> > the
>> > > total max throughput with these two possible solutions is pretty much
>> > > identical, however the latency on individual messages is likely to be a
>> > > little different.
>> > >
>> >
>> > *That sounds even better. Your solution fits the bill perfectly.*
>> > *I'd prefer the message getting sync'd to disk first before delivery
>> too.*
>> > * *
>> >
>> > >
>> > > Now, to why do I want to override that value to false? (I do realize
>> that
>> > > > turning it to false, doesn't guarantee that the message was written
>> to
>> > > > disk)
>> > > > In my use case. I have a database outside Qpid which persists all the
>> > > > messages enqueued (for durability and transactionality with message
>> > > > callback operations.).
>> > > >
>> > > > My sequence of operations is,
>> > > >
>> > > > 1) Enqueue messages -> persist message in my database outside Qpid ->
>> > > > enqueue to Qpid in persistant mode
>> > > > Since my message is already persisted in my database, I'm ok with
>> > losing
>> > > a
>> > > > message in BDB, and would like to enhance throughput of my overall
>> > > enqueue
>> > > > operation by using the nosyncCommit option.
>> > > > The reason I'm not using non-persistant mode in Qpid is because I
>> don't
>> > > > want to overwhelm the memory available by flooding the broker with
>> > > > messages.
>> > > >
>> > > > In the event of a broker crash, I plan to restore the messages from
>> my
>> > > > database outside Qpid. (I realize this could turn out a little
>> > > expensive).
>> > > > I've been thinking
>> > > > that if there is a replication/HA solution to have messages go to a
>> > > primary
>> > > > and secondary broker (in a 2 instance cluster), in the event of a
>> > crash,
>> > > > I'll have to
>> > > > only restore the most recent messages from my database outside Qpid
>> > upto
>> > > a
>> > > > certain checkpoint to which the primary and secondary are sync'ed
>> (Not
>> > > sure
>> > > > if this is achievable.).
>> > > >
>> > > > I do realize that my architecture is a little complicated. Please do
>> > let
>> > > me
>> > > > know if you have more questions about what I'm trying to achieve
>> here,
>> > > I'll
>> > > > be happy to
>> > > > clarify doubts.
>> > > >
>> > > > I'd really like to know if I can add an option to override the
>> > syncCommit
>> > > > to false without any major side-effects apart from what I expect.
>> > > >
>> > > >
>> > > >
>> > > Does my patch seem like it would satisfy your requirements?
>> > >
>> >
>> > *Yes. It's quite exactly what I was looking for. I have a couple of
>> > questions on the HA*
>> > *state of things though. Please do see our other thread on that. It will
>> be
>> > really *
>> > *valuable if you mention what you think.*
>> > *
>> > *
>> > *Thank you Rob.*
>> > * *
>> >
>> > >
>> > > Cheers,
>> > > Rob
>> > >
>> >
>> >
>> >
>> > --
>> > -Praveen
>> >
>>
>
>
>
> --
> -Praveen
---------------------------------------------------------------------
Apache Qpid - AMQP Messaging Implementation
Project: http://qpid.apache.org
Use/Interact: mailto:users-subscribe@qpid.apache.org
Re: override to turn BDBMessageStore syncCommit to false?
Posted by Praveen M <le...@gmail.com>.
Hi Rob,
Sorry for the late reply. I finally got around to reading your changelist
and also playing with it.
I realize that the change you've done is for sessions with auto_commit mode.
I was wondering if the async override could be applied to transacted mode
too.
If you read my earlier mail,
" My sequence of operations is,
1) Enqueue messages -> persist message in my database outside Qpid ->
enqueue to Qpid in persistant mode
Since my message is already persisted in my database, I'm ok
with losing amessage in BDB, and would like to enhance throughput of my
overall enqueue"
Though persisting the message in my database, the reason why I'm using
transacted session is to protect against a case like below.
If I use auto_commit, the sequeue of operations are.
1) my enqueue server -> writes to my database outside qpid -> enqueue to
qpid (auto_commit) -> commit my database write outside qpid
But there is a case where, after i do the enqueue to qpid and commit my
database write, my enqueue server can crash and the auto_committed message
might be lost, as there is
no guarantee that it has reached the broker (could be still in network
buffer or in transit..but not yet reached the broker).
To avoid this scenario I was thinking of two options.
To use publish_sync option with auto_commit
1) my enqueue server -> writes to my database outside qpid -> enqueue to
qpid (auto_commit with publish_sync) -> commit my database write outside
qpid
The downside to this is, publish_sync isn't really a JMS terminology and
also, on the client side, there is a lock held on the connection with
publish_sync option, avoiding other sessions sharing the connection to make
progress. I spoke to Robbie about this earlier, and his suggestion was to
try and stick to JMS terminology if possible.
2) Use transacted mode commit
my enqueue server -> writes to my database outside qpid -> enqueue to qpid
(transacted_mode) -> commit qpid session-> commit my database write
outside qpid
This looked pretty reliable and conforms to JMS, and I was hoping to use
this, as there is a guarantee that the message is pushed to the broker
before I write to my own database.
The one enhancement that I thought will improve the performance of the
roundtrip for this case was, to set syncCommit to false, so that I get the
guarantee that the broker got my message, but also return back quickly than
wait for the write to happen(which takes most of the roundtrip time in the
transaction).
I do understand that what I am trying to achieve is quite different from
normal usage.
I'd like to hear what you think about this problem and what you would
suggest as my best option in this case.
Thanks,
Praveen
Thanks,
Praveen
On Sat, Jan 21, 2012 at 12:00 PM, Rob Godfrey <ro...@gmail.com>wrote:
> Hi Praveen,
>
> I've checked in my changes on trunk if you want to have a play around...
> let me know if you run into any issues,
>
> Cheers,
> Rob
>
> On 20 January 2012 16:57, Praveen M <le...@gmail.com> wrote:
>
> > Hi Rob,
> >
> > Thanks for writing. Please see inline.
> >
> > On Fri, Jan 20, 2012 at 1:27 AM, Rob Godfrey <rob.j.godfrey@gmail.com
> > >wrote:
> >
> > > Hi Praveen,
> > >
> > > apologies for the slow response here...
> > >
> > > On 14 January 2012 03:27, Praveen M <le...@gmail.com> wrote:
> > >
> > > > Hi,
> > > >
> > > > I would like to exploit the Berkeley DB's nosyncCommit (lazy
> commit)
> > > > option to exploit enqueue performance.
> > > > I see that in the current BDBMessageStore implementation there is no
> > way
> > > to
> > > > override the syncCommit Boolean value and
> > > > would like to know if this could be made into an option.
> > > >
> > > > Also, that said It'll be nice to know if someone had used the
> > > > nosyncCommit() option that BDB provides and if using it does any
> > > invisible
> > > > harm. (DB corruptions etc.).
> > > >
> > > >
> > > So, I have coincidentally been working on a patch to increase
> throughput
> > > for non transactional messages which makes greater use of no-sync
> > commits.
> > > The patch is basically to allow a certain amount of "out-of-order"
> > > processing while still maintaining the guarantees on durability that
> the
> > > AMQP specification states that you should get. For non transactional
> > > durable messages this is giving me something like a 6.5x increase in
> max
> > > total throughput.
> > >
> >
> > *Ah, this is great. Exactly what I was looking for. *
> >
> > >
> > > Unfortunately this patch did uncover a couple of issues, one being the
> > need
> > > to have some code in Qpid to handle occasional BDB lock timeouts... and
> > the
> > > other a very occasional bug in BDB itself which they guys at Oracle now
> > > have a fix for, but won't be publicly released until their regular
> > update.
> > > There is a reasonably easy workaround for this which I shall
> incorporate
> > > into my patch, but it may take me a couple more days to get it all
> > > complete.
> > >
> > > Note that my patch is a little different to what you are suggesting -
> in
> > > your case a message may go on to a queue in memory and then be
> delivered
> > > without the guarantee that it was sync'd to disk... in my patch the
> > message
> > > won;t be released into the in-memory queue until it is sync'd, however
> I
> > am
> > > batching up more such enqueues to get better performance. In my tests
> > the
> > > total max throughput with these two possible solutions is pretty much
> > > identical, however the latency on individual messages is likely to be a
> > > little different.
> > >
> >
> > *That sounds even better. Your solution fits the bill perfectly.*
> > *I'd prefer the message getting sync'd to disk first before delivery
> too.*
> > * *
> >
> > >
> > > Now, to why do I want to override that value to false? (I do realize
> that
> > > > turning it to false, doesn't guarantee that the message was written
> to
> > > > disk)
> > > > In my use case. I have a database outside Qpid which persists all the
> > > > messages enqueued (for durability and transactionality with message
> > > > callback operations.).
> > > >
> > > > My sequence of operations is,
> > > >
> > > > 1) Enqueue messages -> persist message in my database outside Qpid ->
> > > > enqueue to Qpid in persistant mode
> > > > Since my message is already persisted in my database, I'm ok with
> > losing
> > > a
> > > > message in BDB, and would like to enhance throughput of my overall
> > > enqueue
> > > > operation by using the nosyncCommit option.
> > > > The reason I'm not using non-persistant mode in Qpid is because I
> don't
> > > > want to overwhelm the memory available by flooding the broker with
> > > > messages.
> > > >
> > > > In the event of a broker crash, I plan to restore the messages from
> my
> > > > database outside Qpid. (I realize this could turn out a little
> > > expensive).
> > > > I've been thinking
> > > > that if there is a replication/HA solution to have messages go to a
> > > primary
> > > > and secondary broker (in a 2 instance cluster), in the event of a
> > crash,
> > > > I'll have to
> > > > only restore the most recent messages from my database outside Qpid
> > upto
> > > a
> > > > certain checkpoint to which the primary and secondary are sync'ed
> (Not
> > > sure
> > > > if this is achievable.).
> > > >
> > > > I do realize that my architecture is a little complicated. Please do
> > let
> > > me
> > > > know if you have more questions about what I'm trying to achieve
> here,
> > > I'll
> > > > be happy to
> > > > clarify doubts.
> > > >
> > > > I'd really like to know if I can add an option to override the
> > syncCommit
> > > > to false without any major side-effects apart from what I expect.
> > > >
> > > >
> > > >
> > > Does my patch seem like it would satisfy your requirements?
> > >
> >
> > *Yes. It's quite exactly what I was looking for. I have a couple of
> > questions on the HA*
> > *state of things though. Please do see our other thread on that. It will
> be
> > really *
> > *valuable if you mention what you think.*
> > *
> > *
> > *Thank you Rob.*
> > * *
> >
> > >
> > > Cheers,
> > > Rob
> > >
> >
> >
> >
> > --
> > -Praveen
> >
>
--
-Praveen
Re: override to turn BDBMessageStore syncCommit to false?
Posted by Rob Godfrey <ro...@gmail.com>.
Hi Praveen,
I've checked in my changes on trunk if you want to have a play around...
let me know if you run into any issues,
Cheers,
Rob
On 20 January 2012 16:57, Praveen M <le...@gmail.com> wrote:
> Hi Rob,
>
> Thanks for writing. Please see inline.
>
> On Fri, Jan 20, 2012 at 1:27 AM, Rob Godfrey <rob.j.godfrey@gmail.com
> >wrote:
>
> > Hi Praveen,
> >
> > apologies for the slow response here...
> >
> > On 14 January 2012 03:27, Praveen M <le...@gmail.com> wrote:
> >
> > > Hi,
> > >
> > > I would like to exploit the Berkeley DB's nosyncCommit (lazy commit)
> > > option to exploit enqueue performance.
> > > I see that in the current BDBMessageStore implementation there is no
> way
> > to
> > > override the syncCommit Boolean value and
> > > would like to know if this could be made into an option.
> > >
> > > Also, that said It'll be nice to know if someone had used the
> > > nosyncCommit() option that BDB provides and if using it does any
> > invisible
> > > harm. (DB corruptions etc.).
> > >
> > >
> > So, I have coincidentally been working on a patch to increase throughput
> > for non transactional messages which makes greater use of no-sync
> commits.
> > The patch is basically to allow a certain amount of "out-of-order"
> > processing while still maintaining the guarantees on durability that the
> > AMQP specification states that you should get. For non transactional
> > durable messages this is giving me something like a 6.5x increase in max
> > total throughput.
> >
>
> *Ah, this is great. Exactly what I was looking for. *
>
> >
> > Unfortunately this patch did uncover a couple of issues, one being the
> need
> > to have some code in Qpid to handle occasional BDB lock timeouts... and
> the
> > other a very occasional bug in BDB itself which they guys at Oracle now
> > have a fix for, but won't be publicly released until their regular
> update.
> > There is a reasonably easy workaround for this which I shall incorporate
> > into my patch, but it may take me a couple more days to get it all
> > complete.
> >
> > Note that my patch is a little different to what you are suggesting - in
> > your case a message may go on to a queue in memory and then be delivered
> > without the guarantee that it was sync'd to disk... in my patch the
> message
> > won;t be released into the in-memory queue until it is sync'd, however I
> am
> > batching up more such enqueues to get better performance. In my tests
> the
> > total max throughput with these two possible solutions is pretty much
> > identical, however the latency on individual messages is likely to be a
> > little different.
> >
>
> *That sounds even better. Your solution fits the bill perfectly.*
> *I'd prefer the message getting sync'd to disk first before delivery too.*
> * *
>
> >
> > Now, to why do I want to override that value to false? (I do realize that
> > > turning it to false, doesn't guarantee that the message was written to
> > > disk)
> > > In my use case. I have a database outside Qpid which persists all the
> > > messages enqueued (for durability and transactionality with message
> > > callback operations.).
> > >
> > > My sequence of operations is,
> > >
> > > 1) Enqueue messages -> persist message in my database outside Qpid ->
> > > enqueue to Qpid in persistant mode
> > > Since my message is already persisted in my database, I'm ok with
> losing
> > a
> > > message in BDB, and would like to enhance throughput of my overall
> > enqueue
> > > operation by using the nosyncCommit option.
> > > The reason I'm not using non-persistant mode in Qpid is because I don't
> > > want to overwhelm the memory available by flooding the broker with
> > > messages.
> > >
> > > In the event of a broker crash, I plan to restore the messages from my
> > > database outside Qpid. (I realize this could turn out a little
> > expensive).
> > > I've been thinking
> > > that if there is a replication/HA solution to have messages go to a
> > primary
> > > and secondary broker (in a 2 instance cluster), in the event of a
> crash,
> > > I'll have to
> > > only restore the most recent messages from my database outside Qpid
> upto
> > a
> > > certain checkpoint to which the primary and secondary are sync'ed (Not
> > sure
> > > if this is achievable.).
> > >
> > > I do realize that my architecture is a little complicated. Please do
> let
> > me
> > > know if you have more questions about what I'm trying to achieve here,
> > I'll
> > > be happy to
> > > clarify doubts.
> > >
> > > I'd really like to know if I can add an option to override the
> syncCommit
> > > to false without any major side-effects apart from what I expect.
> > >
> > >
> > >
> > Does my patch seem like it would satisfy your requirements?
> >
>
> *Yes. It's quite exactly what I was looking for. I have a couple of
> questions on the HA*
> *state of things though. Please do see our other thread on that. It will be
> really *
> *valuable if you mention what you think.*
> *
> *
> *Thank you Rob.*
> * *
>
> >
> > Cheers,
> > Rob
> >
>
>
>
> --
> -Praveen
>
Re: override to turn BDBMessageStore syncCommit to false?
Posted by Praveen M <le...@gmail.com>.
Hi Rob,
Thanks for writing. Please see inline.
On Fri, Jan 20, 2012 at 1:27 AM, Rob Godfrey <ro...@gmail.com>wrote:
> Hi Praveen,
>
> apologies for the slow response here...
>
> On 14 January 2012 03:27, Praveen M <le...@gmail.com> wrote:
>
> > Hi,
> >
> > I would like to exploit the Berkeley DB's nosyncCommit (lazy commit)
> > option to exploit enqueue performance.
> > I see that in the current BDBMessageStore implementation there is no way
> to
> > override the syncCommit Boolean value and
> > would like to know if this could be made into an option.
> >
> > Also, that said It'll be nice to know if someone had used the
> > nosyncCommit() option that BDB provides and if using it does any
> invisible
> > harm. (DB corruptions etc.).
> >
> >
> So, I have coincidentally been working on a patch to increase throughput
> for non transactional messages which makes greater use of no-sync commits.
> The patch is basically to allow a certain amount of "out-of-order"
> processing while still maintaining the guarantees on durability that the
> AMQP specification states that you should get. For non transactional
> durable messages this is giving me something like a 6.5x increase in max
> total throughput.
>
*Ah, this is great. Exactly what I was looking for. *
>
> Unfortunately this patch did uncover a couple of issues, one being the need
> to have some code in Qpid to handle occasional BDB lock timeouts... and the
> other a very occasional bug in BDB itself which they guys at Oracle now
> have a fix for, but won't be publicly released until their regular update.
> There is a reasonably easy workaround for this which I shall incorporate
> into my patch, but it may take me a couple more days to get it all
> complete.
>
> Note that my patch is a little different to what you are suggesting - in
> your case a message may go on to a queue in memory and then be delivered
> without the guarantee that it was sync'd to disk... in my patch the message
> won;t be released into the in-memory queue until it is sync'd, however I am
> batching up more such enqueues to get better performance. In my tests the
> total max throughput with these two possible solutions is pretty much
> identical, however the latency on individual messages is likely to be a
> little different.
>
*That sounds even better. Your solution fits the bill perfectly.*
*I'd prefer the message getting sync'd to disk first before delivery too.*
* *
>
> Now, to why do I want to override that value to false? (I do realize that
> > turning it to false, doesn't guarantee that the message was written to
> > disk)
> > In my use case. I have a database outside Qpid which persists all the
> > messages enqueued (for durability and transactionality with message
> > callback operations.).
> >
> > My sequence of operations is,
> >
> > 1) Enqueue messages -> persist message in my database outside Qpid ->
> > enqueue to Qpid in persistant mode
> > Since my message is already persisted in my database, I'm ok with losing
> a
> > message in BDB, and would like to enhance throughput of my overall
> enqueue
> > operation by using the nosyncCommit option.
> > The reason I'm not using non-persistant mode in Qpid is because I don't
> > want to overwhelm the memory available by flooding the broker with
> > messages.
> >
> > In the event of a broker crash, I plan to restore the messages from my
> > database outside Qpid. (I realize this could turn out a little
> expensive).
> > I've been thinking
> > that if there is a replication/HA solution to have messages go to a
> primary
> > and secondary broker (in a 2 instance cluster), in the event of a crash,
> > I'll have to
> > only restore the most recent messages from my database outside Qpid upto
> a
> > certain checkpoint to which the primary and secondary are sync'ed (Not
> sure
> > if this is achievable.).
> >
> > I do realize that my architecture is a little complicated. Please do let
> me
> > know if you have more questions about what I'm trying to achieve here,
> I'll
> > be happy to
> > clarify doubts.
> >
> > I'd really like to know if I can add an option to override the syncCommit
> > to false without any major side-effects apart from what I expect.
> >
> >
> >
> Does my patch seem like it would satisfy your requirements?
>
*Yes. It's quite exactly what I was looking for. I have a couple of
questions on the HA*
*state of things though. Please do see our other thread on that. It will be
really *
*valuable if you mention what you think.*
*
*
*Thank you Rob.*
* *
>
> Cheers,
> Rob
>
--
-Praveen
Re: override to turn BDBMessageStore syncCommit to false?
Posted by Rob Godfrey <ro...@gmail.com>.
Hi Praveen,
apologies for the slow response here...
On 14 January 2012 03:27, Praveen M <le...@gmail.com> wrote:
> Hi,
>
> I would like to exploit the Berkeley DB's nosyncCommit (lazy commit)
> option to exploit enqueue performance.
> I see that in the current BDBMessageStore implementation there is no way to
> override the syncCommit Boolean value and
> would like to know if this could be made into an option.
>
> Also, that said It'll be nice to know if someone had used the
> nosyncCommit() option that BDB provides and if using it does any invisible
> harm. (DB corruptions etc.).
>
>
So, I have coincidentally been working on a patch to increase throughput
for non transactional messages which makes greater use of no-sync commits.
The patch is basically to allow a certain amount of "out-of-order"
processing while still maintaining the guarantees on durability that the
AMQP specification states that you should get. For non transactional
durable messages this is giving me something like a 6.5x increase in max
total throughput.
Unfortunately this patch did uncover a couple of issues, one being the need
to have some code in Qpid to handle occasional BDB lock timeouts... and the
other a very occasional bug in BDB itself which they guys at Oracle now
have a fix for, but won't be publicly released until their regular update.
There is a reasonably easy workaround for this which I shall incorporate
into my patch, but it may take me a couple more days to get it all complete.
Note that my patch is a little different to what you are suggesting - in
your case a message may go on to a queue in memory and then be delivered
without the guarantee that it was sync'd to disk... in my patch the message
won;t be released into the in-memory queue until it is sync'd, however I am
batching up more such enqueues to get better performance. In my tests the
total max throughput with these two possible solutions is pretty much
identical, however the latency on individual messages is likely to be a
little different.
Now, to why do I want to override that value to false? (I do realize that
> turning it to false, doesn't guarantee that the message was written to
> disk)
> In my use case. I have a database outside Qpid which persists all the
> messages enqueued (for durability and transactionality with message
> callback operations.).
>
> My sequence of operations is,
>
> 1) Enqueue messages -> persist message in my database outside Qpid ->
> enqueue to Qpid in persistant mode
> Since my message is already persisted in my database, I'm ok with losing a
> message in BDB, and would like to enhance throughput of my overall enqueue
> operation by using the nosyncCommit option.
> The reason I'm not using non-persistant mode in Qpid is because I don't
> want to overwhelm the memory available by flooding the broker with
> messages.
>
> In the event of a broker crash, I plan to restore the messages from my
> database outside Qpid. (I realize this could turn out a little expensive).
> I've been thinking
> that if there is a replication/HA solution to have messages go to a primary
> and secondary broker (in a 2 instance cluster), in the event of a crash,
> I'll have to
> only restore the most recent messages from my database outside Qpid upto a
> certain checkpoint to which the primary and secondary are sync'ed (Not sure
> if this is achievable.).
>
> I do realize that my architecture is a little complicated. Please do let me
> know if you have more questions about what I'm trying to achieve here, I'll
> be happy to
> clarify doubts.
>
> I'd really like to know if I can add an option to override the syncCommit
> to false without any major side-effects apart from what I expect.
>
>
>
Does my patch seem like it would satisfy your requirements?
Cheers,
Rob