Updating state in a route.

[Corey Nolet <cj...@gmail.com>]

I am working on a throttling system for my ingest that will check to see if
my upstream JMS broker is backed up to a particular threshold and, if it is
backed up, begin to route messages to disk instead of sending them to the
database.

I'm wondering the best way to implement this using the correct EIP
paradigms. I'll need to periodically (maybe every 2 minutes) check on the
state of the broker and I was thinking of implementing a quartz route that
would check the state and send a message into my main ingest routes with a
special header (like throttle=true/false).

I was thinking my main ingest routes can maintain their own state and
further set some type of header like "shouldThrottle" on each of their
messages so that I can use content based routing to determine where the
messages should go:

(header.get(shouldThrottle) == true) ? to('disk') : to('database')

Does this seem like a reasonable solution or am I going about this the
wrong way? I know EIP is all about trying to keep state in the streams
instead of the processors. I'm not sure if there's a better solution than
having the "state checker" send a message to the main ingest processors so
they can set their own state.


Thanks much!

Re: Updating state in a route.

[Raul Kripalani <ra...@evosent.com>]

Sounds reasonable to me.
To prepare the system for HA, you can broadcast the state on a JMS
topic to which all nodes subscribe to. Contrary to making the route
itself check every two minutes, this method allows you to run the
checking logic in a single place inside the cluster, in an HA
scenario.
The only handicap is what happens when a node initializes and receives
messages before having received its first "heartbeat" from the
checker. Perhaps you'll need to implement a a handshake or bootstrap
mechanism when a node explicitly asks the state checker to broadcast
the last state.

Or better even, you could use AMQ Last Image subscription policy. This
will work pretty well.

Regards,
Raúl.

> On 19 Mar 2014, at 02:52, Corey Nolet <cj...@gmail.com> wrote:
>
> I am working on a throttling system for my ingest that will check to see if
> my upstream JMS broker is backed up to a particular threshold and, if it is
> backed up, begin to route messages to disk instead of sending them to the
> database.
>
> I'm wondering the best way to implement this using the correct EIP
> paradigms. I'll need to periodically (maybe every 2 minutes) check on the
> state of the broker and I was thinking of implementing a quartz route that
> would check the state and send a message into my main ingest routes with a
> special header (like throttle=true/false).
>
> I was thinking my main ingest routes can maintain their own state and
> further set some type of header like "shouldThrottle" on each of their
> messages so that I can use content based routing to determine where the
> messages should go:
>
> (header.get(shouldThrottle) == true) ? to('disk') : to('database')
>
> Does this seem like a reasonable solution or am I going about this the
> wrong way? I know EIP is all about trying to keep state in the streams
> instead of the processors. I'm not sure if there's a better solution than
> having the "state checker" send a message to the main ingest processors so
> they can set their own state.
>
>
> Thanks much!