beginner needs help with topology

[nkiesel <nk...@tbdnetworks.com>]

Hi all,

this is my first attempt to use ActiveMQ, so please bear with me if I'm
asking stupid questions.  I did try to find answers in the FAQ, the
docu, and the forums, but I'm still undecided how/what to do.

Requirements:
I have two (identical) Java applications running on tho machines
connected over the Internet.  These two applications act in a
master/slave mode: every action done on the master must be repeated on
the slave.  In case either the network or the slave is down, actions
should be queued so that they can be processed once the slave is online
again. On top of that, there is some transaction logic around these
actions: a group of actions should be either executed completely or not
at all by both the master and the slave. Finally, master and slave must
be able to switch roles (by restarting master and slave with e.g. a
command-line option).

Implementation idea:
I have a central place in the application where all actions pass
through. Basic idea is now to splice in a durable queue or topic (see
below) at this point that sends all actions over to the slave. The
transaction logic will be implemented by adding "begin" and "end"
actions. On startup, both apps look at their queue and either purge (if
they find a "begin" without an "end") or process the queue before
resuming execution.

Questions:
1) Should I use embedded or standalone brokers? Also what is the
recommended way to XML-configure an embedded broker?
2) Should I use a topic or a composite destination?
3) Should I use brokers on both machines with a network connector for
bridging them or have the slave app connect to the master broker?
4) Can I find out if there is an active sender/publisher for a
queue/topic (so that the slave can decide on startup whether there is
still a chance to get an "end" if he sees a "begin").

While testing, I had some problems with the "two brokers" approach:
after stopping/starting the slave broker, the slave app would no longer
receive messages although both brokers successfully reconnected. This
was even more the case with durable topics.  Is this a known issue?

Sorry for the overlong question!

Best,
  Norbert


-- 
View this message in context: http://www.nabble.com/beginner-needs-help-with-topology-tf2073837.html#a5710466
Sent from the ActiveMQ - User forum at Nabble.com.

Re: beginner needs help with topology

[nkiesel <nk...@tbdnetworks.com>]

nkiesel wrote:
> 
> Ok, I got a step further using
> "vm://localhost?brokerConfig=xbean:activemq.xml". However, this now says:
> "class path resource [activemq.xml] cannot be opened because it does not
> exist" although activemq.xml is in the local directory and '.' is in
> classpath as well
> 

I finally figured it out: the connection URL that works is
"vm://localhost?brokerConfig=xbean:file:///path/to/my/activemq.xml"

-- 
View this message in context: http://www.nabble.com/beginner-needs-help-with-topology-tf2073837.html#a5873893
Sent from the ActiveMQ - User forum at Nabble.com.

Re: beginner needs help with topology

[nkiesel <nk...@tbdnetworks.com>]

Ok, I got a step further using
"vm://localhost?brokerConfig=xbean:activemq.xml". However, this now says:
"class path resource [activemq.xml] cannot be opened because it does not
exist" although activemq.xml is in the local directory and '.' is in
classpath as well
-- 
View this message in context: http://www.nabble.com/beginner-needs-help-with-topology-tf2073837.html#a5873184
Sent from the ActiveMQ - User forum at Nabble.com.

Re: beginner needs help with topology

[nkiesel <nk...@tbdnetworks.com>]

Ok, I admit I'm defeated.

All I wanted to do is to configure an embedded AQ broker using an XML
configuration in a Java app that does not use Spring.  My naive approach was
to use something like 
... new
ActiveMQConnectionFactory("vm://localhost?brokerConfig=/my/path/to/config.xml")
but I can't make it work.  After reading the docu and the source code I
figured I have to use "?broker.brokerConfig=file:///my/path/to/config/xml"
but that gives error messages (and does not work).

I'm using AQ 4.0.1 with Sun JDK 1.5 under Debian Linux.

I would be really graceful for someone hitting me with the cluestick.

Best,
  Norbert

P.S.: On a side-track, I tried to find an example how to get e.g. #items in
a queue from within my Java program. From the FAQ "How do I find the size of
a Queue" I clicked down to "How can I see what destinations are used". But
that only talks about how to get a list of all active destinations.  The
link provided points to the Java API doc that does not give any methods for
getting statistics (or, more likely, I'm missing something here).  --nk

-- 
View this message in context: http://www.nabble.com/beginner-needs-help-with-topology-tf2073837.html#a5871302
Sent from the ActiveMQ - User forum at Nabble.com.

Re: beginner needs help with topology

[James Strachan <ja...@gmail.com>]

On 8/9/06, nkiesel <nk...@tbdnetworks.com> wrote:
>
> >> Also what is the
> >> recommended way to XML-configure an embedded broker?
>
> >http://incubator.apache.org/activemq/how-do-i-embed-a-broker-inside-a-connection.html
>
> I read this, but the XML config require XBean/Spring which I currently don't
> use.

If you are not using the XML configuration then I'd suggest reading
the JavaDoc - starting wtih BrokerService...
http://incubator.apache.org/activemq/maven/activemq-core/apidocs/org/apache/activemq/broker/BrokerService.html

also read this which shows the basic Java code to use..
http://incubator.apache.org/activemq/how-do-i-embed-a-broker-inside-a-connection.html


>  I also found
>
> http://svn.apache.org/repos/asf/incubator/activemq/trunk/assembly/src/test/java/org/apache/activemq/config/BrokerXmlConfigTest.java
> but there the nice
> connectionFactory.setBrokerXmlConfig("file:src/sample-conf/default.xml") is
> commented out and I also don't find it in the API.  So my code currently
> does
>   ActiveMQConnectionFactory aqcf = new
> ActiveMQConnectionFactory("vm://localhost");
>   Properties prop = new Properties();
>   prop.setProperty("brokerXmlConfig", "file:myCondig.xml"));
>   aqcf.populateProperties(prop);
> I did not try to run this yet though.

See this for more help
http://incubator.apache.org/activemq/how-do-i-embed-a-broker-inside-a-connection.html


> Based on your advise, I simplified my topology to use an embedded broker in
> the master app and have the slave app read it's queue from there.  I
> currently don't envision more than one slave, so that should be ok.
>
> Thanks again for your comments.  Is there a place (wiki?) where I should
> post a summary of my setup once I'm done and it's actually working?

Sure that'd be great. I've created a page for example configurations..

http://goopen.org/confluence/display/ACTIVEMQ/User+Submitted+Configurations

There's a link to this page from here
http://activemq.org/site/xml-configuration.html
-- 

James
-------
http://radio.weblogs.com/0112098/

Re: beginner needs help with topology

[nkiesel <nk...@tbdnetworks.com>]

>> Also what is the
>> recommended way to XML-configure an embedded broker?

>http://incubator.apache.org/activemq/how-do-i-embed-a-broker-inside-a-connection.html

I read this, but the XML config require XBean/Spring which I currently don't
use.  I also found

http://svn.apache.org/repos/asf/incubator/activemq/trunk/assembly/src/test/java/org/apache/activemq/config/BrokerXmlConfigTest.java
but there the nice
connectionFactory.setBrokerXmlConfig("file:src/sample-conf/default.xml") is
commented out and I also don't find it in the API.  So my code currently
does
  ActiveMQConnectionFactory aqcf = new
ActiveMQConnectionFactory("vm://localhost");
  Properties prop = new Properties();
  prop.setProperty("brokerXmlConfig", "file:myCondig.xml"));
  aqcf.populateProperties(prop);
I did not try to run this yet though.

Based on your advise, I simplified my topology to use an embedded broker in
the master app and have the slave app read it's queue from there.  I
currently don't envision more than one slave, so that should be ok.

Thanks again for your comments.  Is there a place (wiki?) where I should
post a summary of my setup once I'm done and it's actually working?

Best,
  Norbert


-- 
View this message in context: http://www.nabble.com/beginner-needs-help-with-topology-tf2073837.html#a5725469
Sent from the ActiveMQ - User forum at Nabble.com.

Re: beginner needs help with topology

[James Strachan <ja...@gmail.com>]

On 8/8/06, nkiesel <nk...@tbdnetworks.com> wrote:
> Hi all,
>
> this is my first attempt to use ActiveMQ, so please bear with me if I'm
> asking stupid questions.

Welcome! :)


>  I did try to find answers in the FAQ, the
> docu, and the forums, but I'm still undecided how/what to do.
>
> Requirements:
> I have two (identical) Java applications running on tho machines
> connected over the Internet.  These two applications act in a
> master/slave mode: every action done on the master must be repeated on
> the slave.  In case either the network or the slave is down, actions
> should be queued so that they can be processed once the slave is online
> again. On top of that, there is some transaction logic around these
> actions: a group of actions should be either executed completely or not
> at all by both the master and the slave. Finally, master and slave must
> be able to switch roles (by restarting master and slave with e.g. a
> command-line option).
>
> Implementation idea:
> I have a central place in the application where all actions pass
> through. Basic idea is now to splice in a durable queue or topic (see
> below) at this point that sends all actions over to the slave. The
> transaction logic will be implemented by adding "begin" and "end"
> actions. On startup, both apps look at their queue and either purge (if
> they find a "begin" without an "end") or process the queue before
> resuming execution.

I don't follow why you need a 'begin' and 'end' message? Is it to
process many messages in a single transaction? If so just have some
kinda marker on the last message (e.g. using Message Groups)

http://incubator.apache.org/activemq/message-groups.html

so that the JMSXGroupSeq = 0 for the last message

Then if JMSXGroupSeq is specified as a header you can call
session.commit() if the value is zero to atomically process all of the
messages.


> Questions:
> 1) Should I use embedded or standalone brokers?

It depends on your persistence strategy and how many JMS clients you
have. If you have lots of clients or want reliable persistence I'd
recommend a stand alone broker as its a bit easier to manage and
backup etc


> Also what is the
> recommended way to XML-configure an embedded broker?

http://incubator.apache.org/activemq/how-do-i-embed-a-broker-inside-a-connection.html


> 2) Should I use a topic or a composite destination?

Queues are generally easier to work with (you can browse then and load
balance them etc). So I'd recommend using 2 queues using either
Virtual Destinations or Composite Destinations. Though if you don't
need to load balance requests on the queue then durable topics is
fine.


> 3) Should I use brokers on both machines with a network connector for
> bridging them or have the slave app connect to the master broker?

I'd just run 1 broker - depending on your requirements maybe a
master/slave pair of brokers
http://incubator.apache.org/activemq/masterslave.html

> 4) Can I find out if there is an active sender/publisher for a
> queue/topic (so that the slave can decide on startup whether there is
> still a chance to get an "end" if he sees a "begin").

You can browse active consumers via JMX
http://incubator.apache.org/activemq/jmx.html

or receive advisories
http://incubator.apache.org/activemq/advisory-message.html


> While testing, I had some problems with the "two brokers" approach:
> after stopping/starting the slave broker, the slave app would no longer
> receive messages although both brokers successfully reconnected. This
> was even more the case with durable topics.  Is this a known issue?

Not sure - it could be.

-- 

James
-------
http://radio.weblogs.com/0112098/