You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@qpid.apache.org by Andy Grove <an...@codesuccess.com> on 2007/10/27 12:00:54 UTC
QPID M2 and Berkley DB
Hi,
I'd like to use QPID with Berkeley DB rather than the in-memory message
store.
I downloaded the bdbstore.jar and followed the instructions on the following
page:
http://cwiki.apache.org/qpid/3rd-party-libraries.html
I also downloaded Berkeley DB Java Edition version 3.2.44.
When I run qpid-server I get this stack trace:
Exception in thread "main" java.lang.AbstractMethodError:
org.apache.qpid.server.store.berkeleydb.BDBMessageStore.configure
(Lorg/apache/qpid/server/virtualhost/VirtualHost;Ljava/lang/String;Lorg/apache/commons/configuration/Configuration;)V
at
org.apache.qpid.server.virtualhost.VirtualHost.initialiseMessageStore(
VirtualHost.java:190)
at org.apache.qpid.server.virtualhost.VirtualHost.<init>(
VirtualHost.java:162)
If I understand this correctly, the bdbstore.jar I downloaded is not
compatible with the M2 release. Is there a newer bdbstore available or can I
build one from source?
Thanks,
Andy.
Re: QPID M2 and Berkley DB
Posted by Carl Trieloff <cc...@redhat.com>.
Andy Grove wrote:
> Sure. That makes sense. In fact, I'm working on a product/solution where
> I'll be using messaging and databases in a transactional way and it's
> imperative that the performance of messaging (with persistent messages,
> durable subscribers and transactions) is faster than the database
> operations. I'm in the process of performance testing a prototype of the
> solution and I'll be using QPID M2 and Berkeley DB for now and will see how
> the performance looks.
>
>
Andy,
few things I can note from the M3 work on AMQP 0-10. AMQP 0-10 allows
for a reliable
publish, thus TX only adds the ability to commit 'sets' of messages, and
DTX adds ACID coordination.
So if all you need is the reliability of the individual messages then
there are gains that can be had. In this
use case I am able to sustain durable rates of 2/3 that of the transient
rate using a SAN. If you are forcing
a TX commit on every message I believe you are no more reliable thus
just using the confirm modes of
AMQP 0-10. However, if you are committing small sets of messages then
are effectively forcing some
head of line blocking.
regards
Carl.
Re: QPID M2 and Berkley DB
Posted by Robert Greig <ro...@gmail.com>.
On 13/11/2007, Andy Grove <an...@codesuccess.com> wrote:
> Yes, that's right. I'm using pub/sub with persistent messages, so the client
> isn't waiting for the consumer(s) to process the messages. However, the
> consumers are processing > 9,000 messages per second.
>
> I'm not using RAID - I'm using 250GB SATA-II drives with 8MB cache.
OK that's interesting. Some final questions: is the publisher using
transactions? If so how many messages in a single transactions? And
what acknowledge mode are the consumers using? AUTO_ACK?
Thanks,
Robert
RE: QPID M2 and Berkley DB
Posted by Andy Grove <an...@codesuccess.com>.
Yes, that's right. I'm using pub/sub with persistent messages, so the client
isn't waiting for the consumer(s) to process the messages. However, the
consumers are processing > 9,000 messages per second.
I'm not using RAID - I'm using 250GB SATA-II drives with 8MB cache.
Hope that helps.
Andy.
-----Original Message-----
From: Rupert Smith [mailto:rupertlssmith@googlemail.com]
Sent: 13 November 2007 13:22
To: qpid-users@incubator.apache.org
Subject: Re: QPID M2 and Berkley DB
Is it just timing how long it takes for the the messages to be sent (not
waiting for them to be consumed)?
On 13/11/2007, Robert Greig <ro...@gmail.com> wrote:
>
> On 13/11/2007, Andy Grove <an...@codesuccess.com> wrote:
> > I'm actually measuring the performance on the client side, so it's
> 12,000
> > messages sent to the broker each second. I have two subscribers, running
> on
> > separate machines. The subscribers are not transactional.
>
> Is this a sustained test? Also what kind of disk subsystem are you
> using? RAID 0?
>
> These numbers are in fact better (much better!) than we have seen so I
> am keen to understand your setup.
>
> RG
>
Re: QPID M2 and Berkley DB
Posted by Rupert Smith <ru...@googlemail.com>.
Is it just timing how long it takes for the the messages to be sent (not
waiting for them to be consumed)?
On 13/11/2007, Robert Greig <ro...@gmail.com> wrote:
>
> On 13/11/2007, Andy Grove <an...@codesuccess.com> wrote:
> > I'm actually measuring the performance on the client side, so it's
> 12,000
> > messages sent to the broker each second. I have two subscribers, running
> on
> > separate machines. The subscribers are not transactional.
>
> Is this a sustained test? Also what kind of disk subsystem are you
> using? RAID 0?
>
> These numbers are in fact better (much better!) than we have seen so I
> am keen to understand your setup.
>
> RG
>
Re: QPID M2 and Berkley DB
Posted by Robert Greig <ro...@gmail.com>.
On 13/11/2007, Andy Grove <an...@codesuccess.com> wrote:
> I'm actually measuring the performance on the client side, so it's 12,000
> messages sent to the broker each second. I have two subscribers, running on
> separate machines. The subscribers are not transactional.
Is this a sustained test? Also what kind of disk subsystem are you
using? RAID 0?
These numbers are in fact better (much better!) than we have seen so I
am keen to understand your setup.
RG
RE: QPID M2 and Berkley DB
Posted by Andy Grove <an...@codesuccess.com>.
I'm actually measuring the performance on the client side, so it's 12,000
messages sent to the broker each second. I have two subscribers, running on
separate machines. The subscribers are not transactional.
Thanks,
Andy.
-----Original Message-----
From: Robert Greig [mailto:robert.j.greig@gmail.com]
Sent: 13 November 2007 11:51
To: qpid-users@incubator.apache.org
Subject: Re: QPID M2 and Berkley DB
On 13/11/2007, Andy Grove <an...@codesuccess.com> wrote:
> I'm more interested in scalability than raw performance per client. I'm
> currently seeing maximum throughput of around 12,000 messages per second
for
> the particular test that I am running (sending 1k persistent transactional
> messages to durable subscribers on a single topic with the publisher,
broker
> and consumer each running on separate quad-core 2.4ghz Intel boxes running
> Fedora 8).
That's interesting. How are you measuring the 12,000 per second? Is
that 12,000 received per second per consumer?
RG
Re: QPID M2 and Berkley DB
Posted by Robert Greig <ro...@gmail.com>.
On 13/11/2007, Andy Grove <an...@codesuccess.com> wrote:
> I'm more interested in scalability than raw performance per client. I'm
> currently seeing maximum throughput of around 12,000 messages per second for
> the particular test that I am running (sending 1k persistent transactional
> messages to durable subscribers on a single topic with the publisher, broker
> and consumer each running on separate quad-core 2.4ghz Intel boxes running
> Fedora 8).
That's interesting. How are you measuring the 12,000 per second? Is
that 12,000 received per second per consumer?
RG
RE: QPID M2 and Berkley DB
Posted by Andy Grove <an...@codesuccess.com>.
Carl,
Sorry for the slow response. I'm working part time on this particular
project at the moment. Yes, I would be very interested to see some numbers.
I'm more interested in scalability than raw performance per client. I'm
currently seeing maximum throughput of around 12,000 messages per second for
the particular test that I am running (sending 1k persistent transactional
messages to durable subscribers on a single topic with the publisher, broker
and consumer each running on separate quad-core 2.4ghz Intel boxes running
Fedora 8). Despite the good performance Java QPID/BDB is currently the
bottleneck in this application and I would like to be able to test the C++
broker.
Thanks,
Andy.
-----Original Message-----
From: Carl Trieloff [mailto:cctrieloff@redhat.com]
Sent: 05 November 2007 02:08
To: qpid-users@incubator.apache.org
Subject: Re: QPID M2 and Berkley DB
Andy,
In a few weeks it might be worth giving the C++ broker with the async
store a run. We are still
tracking some bugs but are able to beat bdb performance by orders. If
you like I can post some
numbers later this week.
Carl.
Robert Greig wrote:
> On 04/11/2007, Andy Grove <an...@codesuccess.com> wrote:
>
>
>> Sure. That makes sense. In fact, I'm working on a product/solution where
>> I'll be using messaging and databases in a transactional way and it's
>> imperative that the performance of messaging (with persistent messages,
>> durable subscribers and transactions) is faster than the database
>> operations. I'm in the process of performance testing a prototype of the
>> solution and I'll be using QPID M2 and Berkeley DB for now and will see
how
>> the performance looks.
>>
>
> It will be interesting if you could share your findings with us. In
> our performance testing (admittedly with a SAN) we have seen around
> 700 msgs/second (single commit per message).on a simple point to point
> test case.
>
>
>> I don't have experience of HOWL but have worked with low-level
transaction
>> code in the past (implementing an EJB container). I guess there's no
reason
>> why I couldn't just have a go at creating an implementation of the
>> MessageStore interface - that would at least be a fast way to familiarise
>> myself with the work involved and see if it makes sense for me to
contribute
>> towards this.
>>
>
> Yes definitely. The BDB store is probably a good example to follow.
>
> RG
>
Re: QPID M2 and Berkley DB
Posted by Carl Trieloff <cc...@redhat.com>.
Andy,
In a few weeks it might be worth giving the C++ broker with the async
store a run. We are still
tracking some bugs but are able to beat bdb performance by orders. If
you like I can post some
numbers later this week.
Carl.
Robert Greig wrote:
> On 04/11/2007, Andy Grove <an...@codesuccess.com> wrote:
>
>
>> Sure. That makes sense. In fact, I'm working on a product/solution where
>> I'll be using messaging and databases in a transactional way and it's
>> imperative that the performance of messaging (with persistent messages,
>> durable subscribers and transactions) is faster than the database
>> operations. I'm in the process of performance testing a prototype of the
>> solution and I'll be using QPID M2 and Berkeley DB for now and will see how
>> the performance looks.
>>
>
> It will be interesting if you could share your findings with us. In
> our performance testing (admittedly with a SAN) we have seen around
> 700 msgs/second (single commit per message).on a simple point to point
> test case.
>
>
>> I don't have experience of HOWL but have worked with low-level transaction
>> code in the past (implementing an EJB container). I guess there's no reason
>> why I couldn't just have a go at creating an implementation of the
>> MessageStore interface - that would at least be a fast way to familiarise
>> myself with the work involved and see if it makes sense for me to contribute
>> towards this.
>>
>
> Yes definitely. The BDB store is probably a good example to follow.
>
> RG
>
Re: QPID M2 and Berkley DB
Posted by Robert Greig <ro...@gmail.com>.
On 04/11/2007, Andy Grove <an...@codesuccess.com> wrote:
> Sure. That makes sense. In fact, I'm working on a product/solution where
> I'll be using messaging and databases in a transactional way and it's
> imperative that the performance of messaging (with persistent messages,
> durable subscribers and transactions) is faster than the database
> operations. I'm in the process of performance testing a prototype of the
> solution and I'll be using QPID M2 and Berkeley DB for now and will see how
> the performance looks.
It will be interesting if you could share your findings with us. In
our performance testing (admittedly with a SAN) we have seen around
700 msgs/second (single commit per message).on a simple point to point
test case.
> I don't have experience of HOWL but have worked with low-level transaction
> code in the past (implementing an EJB container). I guess there's no reason
> why I couldn't just have a go at creating an implementation of the
> MessageStore interface - that would at least be a fast way to familiarise
> myself with the work involved and see if it makes sense for me to contribute
> towards this.
Yes definitely. The BDB store is probably a good example to follow.
RG
RE: QPID M2 and Berkley DB
Posted by Andy Grove <an...@codesuccess.com>.
Sure. That makes sense. In fact, I'm working on a product/solution where
I'll be using messaging and databases in a transactional way and it's
imperative that the performance of messaging (with persistent messages,
durable subscribers and transactions) is faster than the database
operations. I'm in the process of performance testing a prototype of the
solution and I'll be using QPID M2 and Berkeley DB for now and will see how
the performance looks.
I don't have experience of HOWL but have worked with low-level transaction
code in the past (implementing an EJB container). I guess there's no reason
why I couldn't just have a go at creating an implementation of the
MessageStore interface - that would at least be a fast way to familiarise
myself with the work involved and see if it makes sense for me to contribute
towards this.
Thanks,
Andy.
-----Original Message-----
From: Robert Greig [mailto:robert.j.greig@gmail.com]
Sent: 04 November 2007 10:36
To: qpid-users@incubator.apache.org
Subject: Re: QPID M2 and Berkley DB
On 04/11/2007, Andy Grove <an...@codesuccess.com> wrote:
> Thanks Robert. I have this building now.
Great.
> I would be interested in contributing to an Apache friendly message store
if
> help is needed. I'm still getting up to speed with AMQP but have extensive
> database and JDBC experience.
Do you have any experience with HOWL? That would be an interesting one
to create for comparison with BDB.
One reason why a JDBC (i.e. relational database backed) store has not
really been progressed is because basically it will perform like a
3-legged giraffe. A persistent message store doesn't really require a
relational database - given the nature of the problem being solved a
much simpler high performance logger is all that is required.
RG
Re: QPID M2 and Berkley DB
Posted by Robert Greig <ro...@gmail.com>.
On 04/11/2007, Andy Grove <an...@codesuccess.com> wrote:
> Thanks Robert. I have this building now.
Great.
> I would be interested in contributing to an Apache friendly message store if
> help is needed. I'm still getting up to speed with AMQP but have extensive
> database and JDBC experience.
Do you have any experience with HOWL? That would be an interesting one
to create for comparison with BDB.
One reason why a JDBC (i.e. relational database backed) store has not
really been progressed is because basically it will perform like a
3-legged giraffe. A persistent message store doesn't really require a
relational database - given the nature of the problem being solved a
much simpler high performance logger is all that is required.
RG
RE: QPID M2 and Berkley DB
Posted by Andy Grove <an...@codesuccess.com>.
Thanks Robert. I have this building now.
I would be interested in contributing to an Apache friendly message store if
help is needed. I'm still getting up to speed with AMQP but have extensive
database and JDBC experience.
Thanks,
Andy.
-----Original Message-----
From: Robert Greig [mailto:robert.j.greig@gmail.com]
Sent: 04 November 2007 08:14
To: qpid-users@incubator.apache.org
Subject: Re: QPID M2 and Berkley DB
On 03/11/2007, Andy Grove <an...@codesuccess.com> wrote:
> I tried building from source but there are some source incompatibilities
between bdbstore and QPID M2. I made the following minor changes to be able
to compile QPID M2 and bdbstore.
I'm really sorry but it appears that Martin has given you the wrong
URL. The URL he gave was for M2.1 not M2.
If you download from
https://svn.jboss.org/repos/rhmessaging/store/branches/java/M2/java/bdbstore
it should work. I just tried it and it worked - all I needed to do was
remove SNAPSHOT from the pom.xml in the bdbstore directory since M2
qpid has now been made ready for release (i.e. snapshot removed from
it).
Can someone fix the pom in the bdbstore not to reference snapshot? I
tried to commit my change but it appears I don't have the right
permissions on the RH svn repo.
> I don't yet understand the relationship between Apache and Red Hat on
these projects. Red Hat obviously have their own broker implementation based
on QPID. Is the intention for bdbstore to be backwards compatible with
Apache QPID? Should I file bugs with Apache and/or Red Hat for this issue?
The RH svn contains some modules that cannot be hosted on apache for
licence reasons. The RH svn does not contain any code that forks qpid.
For issues with the BDB store you should ideally raise them on the RH
mailing lists. For M3 there are plans to create a persistent store
that is apache-friendly; in fact I believe a JDBC store may already
exist. The BDB store offers very good performance, although a
specialised store should be able to improve on it further and I know
that this is something that some people are interested in exploring
further.
I should point out that the BDB store is certainly production quality
- it is being used in production today for several key systems that I
know about and has been extensively tested.
Again, apologies for the trouble you have had getting this to work.
RG
Re: QPID M2 and Berkley DB
Posted by Robert Greig <ro...@gmail.com>.
On 03/11/2007, Andy Grove <an...@codesuccess.com> wrote:
> I tried building from source but there are some source incompatibilities between bdbstore and QPID M2. I made the following minor changes to be able to compile QPID M2 and bdbstore.
I'm really sorry but it appears that Martin has given you the wrong
URL. The URL he gave was for M2.1 not M2.
If you download from
https://svn.jboss.org/repos/rhmessaging/store/branches/java/M2/java/bdbstore
it should work. I just tried it and it worked - all I needed to do was
remove SNAPSHOT from the pom.xml in the bdbstore directory since M2
qpid has now been made ready for release (i.e. snapshot removed from
it).
Can someone fix the pom in the bdbstore not to reference snapshot? I
tried to commit my change but it appears I don't have the right
permissions on the RH svn repo.
> I don't yet understand the relationship between Apache and Red Hat on these projects. Red Hat obviously have their own broker implementation based on QPID. Is the intention for bdbstore to be backwards compatible with Apache QPID? Should I file bugs with Apache and/or Red Hat for this issue?
The RH svn contains some modules that cannot be hosted on apache for
licence reasons. The RH svn does not contain any code that forks qpid.
For issues with the BDB store you should ideally raise them on the RH
mailing lists. For M3 there are plans to create a persistent store
that is apache-friendly; in fact I believe a JDBC store may already
exist. The BDB store offers very good performance, although a
specialised store should be able to improve on it further and I know
that this is something that some people are interested in exploring
further.
I should point out that the BDB store is certainly production quality
- it is being used in production today for several key systems that I
know about and has been extensively tested.
Again, apologies for the trouble you have had getting this to work.
RG
RE: QPID M2 and Berkley DB
Posted by Andy Grove <an...@codesuccess.com>.
Thanks Martin.
I tried building from source but there are some source incompatibilities between bdbstore and QPID M2. I made the following minor changes to be able to compile QPID M2 and bdbstore.
1. Add additional constructors to QPID AMQException that take a message and an exception
2. Change bdbstore to use the new AMQException constructors
3. Change scope of QPID AMQMessage.incrementReference() from protected to public
However, all of the bdbstore tests failed with a NPE:
java.lang.NullPointerException
at org.apache.qpid.server.exchange.DefaultExchangeFactory.initialise(DefaultExchangeFactory.java:73)
at org.apache.qpid.server.virtualhost.VirtualHost.&init&(VirtualHost.java:148)
at org.apache.qpid.server.virtualhost.VirtualHost.&init&(VirtualHost.java:122)
at org.apache.qpid.server.store.berkeleydb.BDBStoreTest.setUp(BDBStoreTest.java:83)
The initiating code in BDBStoreTest is:
_virtualHost = new VirtualHost("test", _store);
That constructor is implemented in QPID as:
public VirtualHost(String name, MessageStore store) throws Exception
{
this(name, null, store);
}
This results in a null hostConfig being passed to an overloaded constructor and hence causes the DefaultExchangeFactory to fail because it attempts to call methods on the hostConfig object.
I don't yet understand the relationship between Apache and Red Hat on these projects. Red Hat obviously have their own broker implementation based on QPID. Is the intention for bdbstore to be backwards compatible with Apache QPID? Should I file bugs with Apache and/or Red Hat for this issue?
Thanks,
Andy.
-----Original Message-----
From: martin.a.ritchie@googlemail.com [mailto:martin.a.ritchie@googlemail.com] On Behalf Of Martin Ritchie
Sent: 29 October 2007 10:31
To: qpid-users@incubator.apache.org
Subject: Re: QPID M2 and Berkley DB
Yes, that page has not been updated since the BDB Code moved repository.
You can download the source code here:
https://svn.jboss.org/repos/rhmessaging/store/branches/java/M2.1/java/bdbstore/
if you place the bdbstore directory in your apache java directory it
will build using mvn from within the bdbstore directory.
I don't believe we have a binary build as yet. I'm sure when M2 has
been released there will be a corresponding package made.
On 27/10/2007, Andy Grove <an...@codesuccess.com> wrote:
> Hi,
>
> I'd like to use QPID with Berkeley DB rather than the in-memory message
> store.
>
> I downloaded the bdbstore.jar and followed the instructions on the following
> page:
>
> http://cwiki.apache.org/qpid/3rd-party-libraries.html
>
> I also downloaded Berkeley DB Java Edition version 3.2.44.
>
> When I run qpid-server I get this stack trace:
>
> Exception in thread "main" java.lang.AbstractMethodError:
> org.apache.qpid.server.store.berkeleydb.BDBMessageStore.configure
> (Lorg/apache/qpid/server/virtualhost/VirtualHost;Ljava/lang/String;Lorg/apache/commons/configuration/Configuration;)V
> at
> org.apache.qpid.server.virtualhost.VirtualHost.initialiseMessageStore(
> VirtualHost.java:190)
> at org.apache.qpid.server.virtualhost.VirtualHost.<init>(
> VirtualHost.java:162)
>
> If I understand this correctly, the bdbstore.jar I downloaded is not
> compatible with the M2 release. Is there a newer bdbstore available or can I
> build one from source?
>
> Thanks,
>
> Andy.
>
--
Martin Ritchie
Re: QPID M2 and Berkley DB
Posted by Martin Ritchie <ri...@apache.org>.
Yes, that page has not been updated since the BDB Code moved repository.
You can download the source code here:
https://svn.jboss.org/repos/rhmessaging/store/branches/java/M2.1/java/bdbstore/
if you place the bdbstore directory in your apache java directory it
will build using mvn from within the bdbstore directory.
I don't believe we have a binary build as yet. I'm sure when M2 has
been released there will be a corresponding package made.
On 27/10/2007, Andy Grove <an...@codesuccess.com> wrote:
> Hi,
>
> I'd like to use QPID with Berkeley DB rather than the in-memory message
> store.
>
> I downloaded the bdbstore.jar and followed the instructions on the following
> page:
>
> http://cwiki.apache.org/qpid/3rd-party-libraries.html
>
> I also downloaded Berkeley DB Java Edition version 3.2.44.
>
> When I run qpid-server I get this stack trace:
>
> Exception in thread "main" java.lang.AbstractMethodError:
> org.apache.qpid.server.store.berkeleydb.BDBMessageStore.configure
> (Lorg/apache/qpid/server/virtualhost/VirtualHost;Ljava/lang/String;Lorg/apache/commons/configuration/Configuration;)V
> at
> org.apache.qpid.server.virtualhost.VirtualHost.initialiseMessageStore(
> VirtualHost.java:190)
> at org.apache.qpid.server.virtualhost.VirtualHost.<init>(
> VirtualHost.java:162)
>
> If I understand this correctly, the bdbstore.jar I downloaded is not
> compatible with the M2 release. Is there a newer bdbstore available or can I
> build one from source?
>
> Thanks,
>
> Andy.
>
--
Martin Ritchie