Design Guidance

[Will Hartung <wi...@msoft.com>]

I'm looking for some design tips here.

Here's the basics. Thick client written in .NET talking to a host server
through Webservices that needs to be "continually updated".

Obvious choice would be to use ActiveMQ and the .NET client library,
subscribe to an appropriate topic/queue, and live happily ever after.

To be blunt, that simply isn't going to happen. Just close your eyes and
think of England, it's not in the cards.

Rather, we're going to have to Poll the server regularly to get our changes.

But that doesn't mean that we cannot post our updates to a queue/topic, and
then leverage that on the web server to send the updates to the client when
they call.

So, here's my thinking. Every time the client calls the web server. it
connects to the JMS server using a Durable Subscription. It then slurps down
all of the pending update messages (or some max number of them), then
forwards them down to the client and disconnecting from the JMS server.

The problem is that we don't have any real sense of transactions here.
There's no obvious way to make sure the client actually recieved and
processed those changes that the server forwarded to it.

One thought was that when the client requests an update, it includes a
timestamp representing the "last time" it got an update, which would be the
timestamp of the last message that it got. Then, when the server is fetching
messages from the JMS server, it would simply ACK the ones that are before
that timestamp, and then simply discard them. But for the current messages,
it would not ACK or Commit them, and leave them on the server. Another
thought is that after the client does a refresh and update, it "commits" the
request by sending a "Ok up to *timestamp*" and the server then refetches
and discards those older messages.

Either solution requires fetching the messages twice from the JMS server.

So, with my limited knowledge of ActiveMQ and JMS in particular, is that a
good solution? Is there some functionality that would make this more
efficient? I can't see any way to get around using the durable subscription.

Overall, I'm thinking of having a small JMS cluster (for reliability moreso
than performance) backing the web services tier, then the app tier funnels
update events in to the JMS cluster. Can ActiveMQ be gateway with Oracle
Queues, so that, say, a stored procedure can post an update event to an
Orcale Queue that then forwards it in to ActiveMQ?

Thanx for any thoughts on this.

Regards,

Will Hartung
(willh@msoft.com)

--
View this message in context: http://www.nabble.com/Design-Guidance-t1447633.html#a3910797
Sent from the ActiveMQ - User forum at Nabble.com.

Re: Design Guidance

[James Strachan <ja...@gmail.com>]

On 4/18/06, Will Hartung <wi...@msoft.com> wrote:
> >> To be blunt, that simply isn't going to happen. Just close your eyes and
> >> think of England, it's not in the cards.
>
> > Why is it never gonna happen?
>
> Design mostly, the web service is the interface.

So the only possible technology is some web service API plus a SOAP
over HTTP stack right?

If so you could look at WS-Notification
http://servicemix.org/WS+Notification

> > I'm still at a bit of a loss to completely understand what you are
> > trying to do. Is there some requirement explicitly disallowing a .Net
> > client library (which does everything you need already)?
>
> > Or is the idea to use some kind of RESTful API to ActiveMQ to avoid
> > using client libraries?
>
> Essentially, keeping the client as agnostic as practical.

Note that the C# client supports pluggable transports so it should be
possible to use any transport/marshalling layer inside it. Plus it
should be possible to integrate ActiveMQ.Net into WCF to further hide
the C# client.


> > We have REST support so you can just use HTTP POST/GET; you can also
> > browse the available messages on a queue as an XML document (or RSS
> > feed). Would that help?
>
> I'll have to look. I thought the REST support was for publishing, not
> necessarily subscribing.

The weakness is in the consumption side; its hard to do RESTful
consumption really well - so we're not 100% pure REST (we use GET to
consume by default) - or you can use GET to browse queues then a POST
to delete messages (but we have no message locking to do pure RESTful
load balancing).


> > You could bridge from ActiveMQ to Oracle Queues; but I don't see what
> > value Oracle Queues brings to the table.
>
> I'm actually more interested in going the other way, from Oracle to
> ActiveMQ. The idea being that not all changes will necessarily come from the
> app server, some may source from the database. If the stored procedures can
> post to an Oracle Queue, and then that Queue can be forwarded to an ActiveMQ
> Topic, that would satisfy that requirement quite nicely.

We've a JMS bridge in both ActiveMQ and ServiceMix; we're just missing
support for Oracle AQ; though we could treat it as a JMS provider
right?

--

James
-------
http://radio.weblogs.com/0112098/

Re: Design Guidance

[Will Hartung <wi...@msoft.com>]

>> To be blunt, that simply isn't going to happen. Just close your eyes and
>> think of England, it's not in the cards.

> Why is it never gonna happen?

Design mostly, the web service is the interface. The fact that ActiveMQ (or
any other solution) is behind the scenes providing the updates is simply a
detail. 


> I'm still at a bit of a loss to completely understand what you are
> trying to do. Is there some requirement explicitly disallowing a .Net
> client library (which does everything you need already)?

> Or is the idea to use some kind of RESTful API to ActiveMQ to avoid
> using client libraries?

Essentially, keeping the client as agnostic as practical.

> We have REST support so you can just use HTTP POST/GET; you can also
> browse the available messages on a queue as an XML document (or RSS
> feed). Would that help?

I'll have to look. I thought the REST support was for publishing, not
necessarily subscribing.

> You could bridge from ActiveMQ to Oracle Queues; but I don't see what
> value Oracle Queues brings to the table.

I'm actually more interested in going the other way, from Oracle to
ActiveMQ. The idea being that not all changes will necessarily come from the
app server, some may source from the database. If the stored procedures can
post to an Oracle Queue, and then that Queue can be forwarded to an ActiveMQ
Topic, that would satisfy that requirement quite nicely.

Thanx!

Regards,

Will Hartung
(willh@msoft.com)



--
View this message in context: http://www.nabble.com/Design-Guidance-t1447633.html#a3970484
Sent from the ActiveMQ - User forum at Nabble.com.

Re: Design Guidance

[James Strachan <ja...@gmail.com>]

On 4/14/06, Will Hartung <wi...@msoft.com> wrote:
>
> I'm looking for some design tips here.
>
> Here's the basics. Thick client written in .NET talking to a host server
> through Webservices that needs to be "continually updated".
>
> Obvious choice would be to use ActiveMQ and the .NET client library,
> subscribe to an appropriate topic/queue, and live happily ever after.
>
> To be blunt, that simply isn't going to happen. Just close your eyes and
> think of England, it's not in the cards.

Why is it never gonna happen?


> Rather, we're going to have to Poll the server regularly to get our changes.
>
> But that doesn't mean that we cannot post our updates to a queue/topic, and
> then leverage that on the web server to send the updates to the client when
> they call.
>
> So, here's my thinking. Every time the client calls the web server. it
> connects to the JMS server using a Durable Subscription. It then slurps down
> all of the pending update messages (or some max number of them), then
> forwards them down to the client and disconnecting from the JMS server.
>
> The problem is that we don't have any real sense of transactions here.
> There's no obvious way to make sure the client actually recieved and
> processed those changes that the server forwarded to it.
>
> One thought was that when the client requests an update, it includes a
> timestamp representing the "last time" it got an update, which would be the
> timestamp of the last message that it got. Then, when the server is fetching
> messages from the JMS server, it would simply ACK the ones that are before
> that timestamp, and then simply discard them. But for the current messages,
> it would not ACK or Commit them, and leave them on the server. Another
> thought is that after the client does a refresh and update, it "commits" the
> request by sending a "Ok up to *timestamp*" and the server then refetches
> and discards those older messages.
>
> Either solution requires fetching the messages twice from the JMS server.
>
> So, with my limited knowledge of ActiveMQ and JMS in particular, is that a
> good solution? Is there some functionality that would make this more
> efficient? I can't see any way to get around using the durable subscription.

I'm still at a bit of a loss to completely understand what you are
trying to do. Is there some requirement explicitly disallowing a .Net
client library (which does everything you need already)?

Or is the idea to use some kind of RESTful API to ActiveMQ to avoid
using client libraries?

We have REST support so you can just use HTTP POST/GET; you can also
browse the available messages on a queue as an XML document (or RSS
feed). Would that help?


> Overall, I'm thinking of having a small JMS cluster (for reliability moreso
> than performance) backing the web services tier, then the app tier funnels
> update events in to the JMS cluster. Can ActiveMQ be gateway with Oracle
> Queues, so that, say, a stored procedure can post an update event to an
> Orcale Queue that then forwards it in to ActiveMQ?

You could bridge from ActiveMQ to Oracle Queues; but I don't see what
value Oracle Queues brings to the table.

--

James
-------
http://radio.weblogs.com/0112098/