You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@camel.apache.org by "christian.ohr" <ch...@googlemail.com> on 2011/08/21 22:26:37 UTC
Camel Runtime Management : Exchange Load
Hi,
I consider runtime monitoring and manageability as one of the key features
for any real-world usage of "headless" integration solutions. Camel has good
JMX support (and improving), but the exposed attributes may not always be as
usable as possible. Often it's only about details, e.g. min/max/avg
processing time should omit at least the first exchange - usually there's
extra initialization cost that is not representative for all subsequent
exchanges and distorts the statistic to some degree.
At this time I'd like to propose an additional runtime attribute: an
exchange load. It can be seen as an equivalent to the Unix process load
(obtained e.g. with the "w" command) and is an EWMA (exponentially weighted
moving average) of the inflight exchanges of each individual route or the
whole Camel context. It averages the number of exchanges over a certain
period of time (e.g. 1, 5, 15 minutes), giving bigger weight to more recent
readings, and smaller weight to older readings. At any time, you would get a
good impression about how loaded the Camel machinery has recently been.
If this seems useful, I'm happy to open a JIRA issue and provide a patch
that adds this statistic.
cheers
Christian
--
View this message in context: http://camel.465427.n5.nabble.com/Camel-Runtime-Management-Exchange-Load-tp4721211p4721211.html
Sent from the Camel - Users mailing list archive at Nabble.com.
Re: Camel Runtime Management : Exchange Load
Posted by Willem Jiang <wi...@gmail.com>.
It sounds good. We can make this load calculation strategy be plug able,
then it could be easy to replaced :)
On 8/22/11 4:26 AM, christian.ohr wrote:
> Hi,
>
> I consider runtime monitoring and manageability as one of the key features
> for any real-world usage of "headless" integration solutions. Camel has good
> JMX support (and improving), but the exposed attributes may not always be as
> usable as possible. Often it's only about details, e.g. min/max/avg
> processing time should omit at least the first exchange - usually there's
> extra initialization cost that is not representative for all subsequent
> exchanges and distorts the statistic to some degree.
>
> At this time I'd like to propose an additional runtime attribute: an
> exchange load. It can be seen as an equivalent to the Unix process load
> (obtained e.g. with the "w" command) and is an EWMA (exponentially weighted
> moving average) of the inflight exchanges of each individual route or the
> whole Camel context. It averages the number of exchanges over a certain
> period of time (e.g. 1, 5, 15 minutes), giving bigger weight to more recent
> readings, and smaller weight to older readings. At any time, you would get a
> good impression about how loaded the Camel machinery has recently been.
>
> If this seems useful, I'm happy to open a JIRA issue and provide a patch
> that adds this statistic.
>
> cheers
> Christian
>
>
> --
> View this message in context: http://camel.465427.n5.nabble.com/Camel-Runtime-Management-Exchange-Load-tp4721211p4721211.html
> Sent from the Camel - Users mailing list archive at Nabble.com.
>
--
Willem
----------------------------------
FuseSource
Web: http://www.fusesource.com
Blog: http://willemjiang.blogspot.com (English)
http://jnn.javaeye.com (Chinese)
Twitter: willemjiang
Weibo: willemjiang
Re: Camel Runtime Management : Exchange Load
Posted by Claus Ibsen <cl...@gmail.com>.
On Mon, Aug 22, 2011 at 3:56 PM, christian.ohr
<ch...@googlemail.com> wrote:
> Claus, in fact I agree with your concern. I would e.g. never keep historic
> data, calculate trends etc. within Camel, or more generally, observing
> runtime data and derive "higher-order" information from it.
> The load statistic I have in mind is pretty basic, fast to calculate
> (thousands of updates in in few ms) and, as mentioned, compiled into every
> Linux kernel. Anyway I'll open an JIRA issue and attach a patch with some
> code to it. If you then still have doubts, be assured I'm not religious
> about it ;-)
>
Christian sounds good. Looking forward to the contribution.
> Willem, for the time being I'll keep it simple and stupid (i.e. not
> pluggable) just like calculating the other JMX statistics are not pluggable
> either.
>
Yeah that is fine by me.
> Christian
>
> --
> View this message in context: http://camel.465427.n5.nabble.com/Camel-Runtime-Management-Exchange-Load-tp4721211p4723514.html
> Sent from the Camel - Users mailing list archive at Nabble.com.
>
--
Claus Ibsen
-----------------
FuseSource
Email: cibsen@fusesource.com
Web: http://fusesource.com
Twitter: davsclaus, fusenews
Blog: http://davsclaus.blogspot.com/
Author of Camel in Action: http://www.manning.com/ibsen/
Re: Camel Runtime Management : Exchange Load
Posted by "christian.ohr" <ch...@googlemail.com>.
CAMEL-4366
--
View this message in context: http://camel.465427.n5.nabble.com/Camel-Runtime-Management-Exchange-Load-tp4721211p4724158.html
Sent from the Camel - Users mailing list archive at Nabble.com.
Re: Camel Runtime Management : Exchange Load
Posted by "christian.ohr" <ch...@googlemail.com>.
Claus, in fact I agree with your concern. I would e.g. never keep historic
data, calculate trends etc. within Camel, or more generally, observing
runtime data and derive "higher-order" information from it.
The load statistic I have in mind is pretty basic, fast to calculate
(thousands of updates in in few ms) and, as mentioned, compiled into every
Linux kernel. Anyway I'll open an JIRA issue and attach a patch with some
code to it. If you then still have doubts, be assured I'm not religious
about it ;-)
Willem, for the time being I'll keep it simple and stupid (i.e. not
pluggable) just like calculating the other JMX statistics are not pluggable
either.
Christian
--
View this message in context: http://camel.465427.n5.nabble.com/Camel-Runtime-Management-Exchange-Load-tp4721211p4723514.html
Sent from the Camel - Users mailing list archive at Nabble.com.
Re: Camel Runtime Management : Exchange Load
Posted by Claus Ibsen <cl...@gmail.com>.
On Sun, Aug 21, 2011 at 10:26 PM, christian.ohr
<ch...@googlemail.com> wrote:
> Hi,
>
> I consider runtime monitoring and manageability as one of the key features
> for any real-world usage of "headless" integration solutions. Camel has good
> JMX support (and improving), but the exposed attributes may not always be as
> usable as possible. Often it's only about details, e.g. min/max/avg
> processing time should omit at least the first exchange - usually there's
> extra initialization cost that is not representative for all subsequent
> exchanges and distorts the statistic to some degree.
>
> At this time I'd like to propose an additional runtime attribute: an
> exchange load. It can be seen as an equivalent to the Unix process load
> (obtained e.g. with the "w" command) and is an EWMA (exponentially weighted
> moving average) of the inflight exchanges of each individual route or the
> whole Camel context. It averages the number of exchanges over a certain
> period of time (e.g. 1, 5, 15 minutes), giving bigger weight to more recent
> readings, and smaller weight to older readings. At any time, you would get a
> good impression about how loaded the Camel machinery has recently been.
>
> If this seems useful, I'm happy to open a JIRA issue and provide a patch
> that adds this statistic.
>
Well it sounds at first like a good idea, but then maybe not.
Camel provides basic information which is fast and easy to calculate
on the fly, without any contention points etc.
Very often any capable monitoring tooling is able to provide you with
loads of such kind of details based on the stats it can extract from
Camel over JMX. And they allow you to customize your dashboard and
what kind of details you want.
For example tooling such as: Nagios, Hyperic, FuseHQ and from the
bigger vendors as well.
So I am just a bit concerned that Camel does not cross the line and
starts to provide stats which people ought to get from a management
tool.
And if this new load stat is added, then people might want to be able
fully customize the stat according to their needs. By which the
management tool already offers such insights. So I am just a bit
cautions to now move down a path where Camel should not go.
As we love contributions, and that a "load" stat would be nice to
have, then I am sure to go for it Christian. And its easier to discuss
when there is some code at hand to look at as well.
> cheers
> Christian
>
>
> --
> View this message in context: http://camel.465427.n5.nabble.com/Camel-Runtime-Management-Exchange-Load-tp4721211p4721211.html
> Sent from the Camel - Users mailing list archive at Nabble.com.
>
--
Claus Ibsen
-----------------
FuseSource
Email: cibsen@fusesource.com
Web: http://fusesource.com
Twitter: davsclaus, fusenews
Blog: http://davsclaus.blogspot.com/
Author of Camel in Action: http://www.manning.com/ibsen/