[HEADS UP] - Improved lifecycle and management of routes in Camel 2.5

[Claus Ibsen <cl...@gmail.com>]

Hi

I am working a bit on improving the lifecycle and management of routes in Camel.

The bigger goal is to support suspension of routes / camel. For
example you can press suspend on CamelContext mbean and have it
temporary suspend all routes.
Then you can resume again by pressing resume, and the suspended routes
will be resumed. For example if there is a problem and you need to
just suspend Camel.
BTW Camel will only resume the routes which was suspended. So if you
have manually stopped a route beforehand, this route will remain
stopped.

Also we introduce management of routes so you can suspend an
individual route as well.
There will be a change in behavior of the stop route operation in JMX.
Previously it would force stop a route, disregarding any in flight
exchanges.
Now the stop operation will use the graceful shutdown feature in Camel
and stop the route more reliable. If you want to force stop a route
you can pass in a timeout value, such as a couple of seconds. Which
triggers Camel to force stopping when the timeout was hit.

Routes can now support suspension, which is done by letting it's
consumer implement the SuspendableService interface. This gives those
consumers a fine grained control of suspension.
There are doSuspend, doResume methods readily available from
ServiceSupport. Which means the DefaultConsumer got it already. For
example the camel-jms component leverages this in its JmsConsumer.
This allows it more gently to suspend listening on a JMS queue.

Likewise camel-jetty consumer will return HTTP status 503 if its
suspended. If you stop it, then it will unbind the port listener. So
suspension is much quicker and gentler to use if you just temporary
need the route to be inactive.

Component writers is encouraged to support this if its possible. For
example this would be great for the many HTTP based consumers to
support.

On the same time we are fixing the JMX management to always list all
routes (also if they have been configured with autoStartup=false).
This allows you actually to manually start those routes from JMX :).

And there will be a shutdown operation for routes. It's to be used for
removing a route from management. The idea would be for you have to
stop the route first before you can shutdown and remove the route.
Maybe we should rename the shutdown operation name to remove?

You can see some background information here:
https://cwiki.apache.org/confluence/display/CAMEL/Lifecycle

Which has a diagram of the lifecycle states.


Any thoughts?


-- 
Claus Ibsen
Apache Camel Committer

Author of Camel in Action: http://www.manning.com/ibsen/
Open Source Integration: http://fusesource.com
Blog: http://davsclaus.blogspot.com/
Twitter: http://twitter.com/davsclaus

Re: [HEADS UP] - Improved lifecycle and management of routes in Camel 2.5

[Martin Krasser <de...@martin-krasser.de>]

Hi Claus,

I really like that, great work! Makes it easier to bind the lifecycle of 
routes to the lifecycle of components from other frameworks (e.g. to the 
lifecycle of Akka actors: these can be restarted and during a restart 
one might want to suspend routes to those actors).

Cheers!

Claus Ibsen schrieb:
> Hi
>
> I am working a bit on improving the lifecycle and management of routes in Camel.
>
> The bigger goal is to support suspension of routes / camel. For
> example you can press suspend on CamelContext mbean and have it
> temporary suspend all routes.
> Then you can resume again by pressing resume, and the suspended routes
> will be resumed. For example if there is a problem and you need to
> just suspend Camel.
> BTW Camel will only resume the routes which was suspended. So if you
> have manually stopped a route beforehand, this route will remain
> stopped.
>
> Also we introduce management of routes so you can suspend an
> individual route as well.
> There will be a change in behavior of the stop route operation in JMX.
> Previously it would force stop a route, disregarding any in flight
> exchanges.
> Now the stop operation will use the graceful shutdown feature in Camel
> and stop the route more reliable. If you want to force stop a route
> you can pass in a timeout value, such as a couple of seconds. Which
> triggers Camel to force stopping when the timeout was hit.
>
> Routes can now support suspension, which is done by letting it's
> consumer implement the SuspendableService interface. This gives those
> consumers a fine grained control of suspension.
> There are doSuspend, doResume methods readily available from
> ServiceSupport. Which means the DefaultConsumer got it already. For
> example the camel-jms component leverages this in its JmsConsumer.
> This allows it more gently to suspend listening on a JMS queue.
>
> Likewise camel-jetty consumer will return HTTP status 503 if its
> suspended. If you stop it, then it will unbind the port listener. So
> suspension is much quicker and gentler to use if you just temporary
> need the route to be inactive.
>
> Component writers is encouraged to support this if its possible. For
> example this would be great for the many HTTP based consumers to
> support.
>
> On the same time we are fixing the JMX management to always list all
> routes (also if they have been configured with autoStartup=false).
> This allows you actually to manually start those routes from JMX :).
>
> And there will be a shutdown operation for routes. It's to be used for
> removing a route from management. The idea would be for you have to
> stop the route first before you can shutdown and remove the route.
> Maybe we should rename the shutdown operation name to remove?
>
> You can see some background information here:
> https://cwiki.apache.org/confluence/display/CAMEL/Lifecycle
>
> Which has a diagram of the lifecycle states.
>
>
> Any thoughts?
>
>
>   

Re: [HEADS UP] - Improved lifecycle and management of routes in Camel 2.5

[Willem Jiang <wi...@gmail.com>]

Christian Schneider wrote:
> 
> 
> Am 03.08.2010 06:04, schrieb Claus Ibsen:
>>
>> Yes you need to differentiate between resume/start in your business 
>> logic.
>> The consumer most often need to different logic. For example a HTTP
>> consumer may have to setup and start a HTTP server.
>> Where as in resume it just need to accept requests instead of sending
>> back http status 503 (temporary out of service).
>>
>> Keep the states/methods separated makes it easier to implement and 
>> understand.
>>
>> If you extend ServiceSupport then you got these callbacks you can 
>> override:
>> - doStart
>> - doStop
>> - doResume
>> - doSuspend
>> - doShutdown (special when CamelContext is shutting down)
>>
> That is a good idea to keep these methods simple.
If we don't want to support manage the route dynamically, doStart and 
doStop are enough, but we are not.

doResume and doSuspend are trying to provides a way let the camel 
context hold the endpoint states, and don't destroy or create the 
consumer or producer again.

>>
>> But the logic in the startRoute, resumeRoute has built in so they
>> handle if you invoke either one.
>> So if a route is suspended and your invoke startRoute then it knows
>> that it should resume the route. And vice versa. So from a management
>> point of view in JMX it should not be confusing.
>>
> This also sounds good. The only problem is then that users will probably 
> understand the diagram in the documentation
> as how the management methods work. So it could be confusing that e.g. 
> startRoute also works from suspended state.
> But as I guess that all the transitions in the diagram work as explained 
> the diagram does not explain anything wrong.
>> stop ->  suspend is a bit harder, due you would need to start it and
>> then suspend it immediately, but there is a "window" where the route
>> is actually started, and it could potentially take in messages. And
>> routes can most often start very fast. Well there may be some WS
>> components who parse WSDL files etc which may take some time. We could
>> introduce a doWarmUp callback which is invoked when CamelContext is
>> starting up. It would then allow to pre parse the WSDL files etc. so
>> the consumer is "almost ready".
>>
> I do not think this is really necessary. I only proposed the transition 
> for completenes. A warmup method makes the interface more complex and 
> offers
> not so much value as camel is already starting quite fast.
> 

+1 for not adding the doWarmUp API, as we don't gain much benefit from it.

> Btw. I also think shutdown could be renamed to remove. Without any more 
> knowledge I would interpret shutdown as something that can be reversed 
> by start. So remove is much clearer.
> 
> Greetings
> 
> Christian
>>
>>
>>
> 

Willem

Re: [HEADS UP] - Improved lifecycle and management of routes in Camel 2.5

[Claus Ibsen <cl...@gmail.com>]

On Tue, Aug 3, 2010 at 10:30 PM, Christian Schneider
<ch...@die-schneider.net> wrote:
>
>
> Am 03.08.2010 06:04, schrieb Claus Ibsen:
>>
>> Yes you need to differentiate between resume/start in your business logic.
>> The consumer most often need to different logic. For example a HTTP
>> consumer may have to setup and start a HTTP server.
>> Where as in resume it just need to accept requests instead of sending
>> back http status 503 (temporary out of service).
>>
>> Keep the states/methods separated makes it easier to implement and
>> understand.
>>
>> If you extend ServiceSupport then you got these callbacks you can
>> override:
>> - doStart
>> - doStop
>> - doResume
>> - doSuspend
>> - doShutdown (special when CamelContext is shutting down)
>>
> That is a good idea to keep these methods simple.
>>
>> But the logic in the startRoute, resumeRoute has built in so they
>> handle if you invoke either one.
>> So if a route is suspended and your invoke startRoute then it knows
>> that it should resume the route. And vice versa. So from a management
>> point of view in JMX it should not be confusing.
>>
> This also sounds good. The only problem is then that users will probably
> understand the diagram in the documentation
> as how the management methods work. So it could be confusing that e.g.
> startRoute also works from suspended state.
> But as I guess that all the transitions in the diagram work as explained the
> diagram does not explain anything wrong.
>>

I will add a note in the diagram that JMX operations can figure out to
invoke either resume/start depending on the state.


>> stop ->  suspend is a bit harder, due you would need to start it and
>> then suspend it immediately, but there is a "window" where the route
>> is actually started, and it could potentially take in messages. And
>> routes can most often start very fast. Well there may be some WS
>> components who parse WSDL files etc which may take some time. We could
>> introduce a doWarmUp callback which is invoked when CamelContext is
>> starting up. It would then allow to pre parse the WSDL files etc. so
>> the consumer is "almost ready".
>>
> I do not think this is really necessary. I only proposed the transition for
> completenes. A warmup method makes the interface more complex and offers
> not so much value as camel is already starting quite fast.
>
> Btw. I also think shutdown could be renamed to remove. Without any more
> knowledge I would interpret shutdown as something that can be reversed by
> start. So remove is much clearer.
>

The JDK API uses shutdown for its thread pools, which means you can
actually not stat them again.
I think it may be the same for other 3rd party APIs such as quartz.

But remove is more meaningful as the service/route must be stopped
before you shutdown.
I have added removeRoute and marked shutdownRoute as @deprecated.


> Greetings
>
> Christian
>>
>>
>>
>
> --
> ----
> http://www.liquid-reality.de
>
>



-- 
Claus Ibsen
Apache Camel Committer

Author of Camel in Action: http://www.manning.com/ibsen/
Open Source Integration: http://fusesource.com
Blog: http://davsclaus.blogspot.com/
Twitter: http://twitter.com/davsclaus

Re: [HEADS UP] - Improved lifecycle and management of routes in Camel 2.5

[Christian Schneider <ch...@die-schneider.net>]

Am 03.08.2010 06:04, schrieb Claus Ibsen:
>
> Yes you need to differentiate between resume/start in your business logic.
> The consumer most often need to different logic. For example a HTTP
> consumer may have to setup and start a HTTP server.
> Where as in resume it just need to accept requests instead of sending
> back http status 503 (temporary out of service).
>
> Keep the states/methods separated makes it easier to implement and understand.
>
> If you extend ServiceSupport then you got these callbacks you can override:
> - doStart
> - doStop
> - doResume
> - doSuspend
> - doShutdown (special when CamelContext is shutting down)
>
That is a good idea to keep these methods simple.
>
> But the logic in the startRoute, resumeRoute has built in so they
> handle if you invoke either one.
> So if a route is suspended and your invoke startRoute then it knows
> that it should resume the route. And vice versa. So from a management
> point of view in JMX it should not be confusing.
>
This also sounds good. The only problem is then that users will probably 
understand the diagram in the documentation
as how the management methods work. So it could be confusing that e.g. 
startRoute also works from suspended state.
But as I guess that all the transitions in the diagram work as explained 
the diagram does not explain anything wrong.
> stop ->  suspend is a bit harder, due you would need to start it and
> then suspend it immediately, but there is a "window" where the route
> is actually started, and it could potentially take in messages. And
> routes can most often start very fast. Well there may be some WS
> components who parse WSDL files etc which may take some time. We could
> introduce a doWarmUp callback which is invoked when CamelContext is
> starting up. It would then allow to pre parse the WSDL files etc. so
> the consumer is "almost ready".
>
I do not think this is really necessary. I only proposed the transition 
for completenes. A warmup method makes the interface more complex and offers
not so much value as camel is already starting quite fast.

Btw. I also think shutdown could be renamed to remove. Without any more 
knowledge I would interpret shutdown as something that can be reversed 
by start. So remove is much clearer.

Greetings

Christian
>
>
>

-- 
----
http://www.liquid-reality.de

Re: [HEADS UP] - Improved lifecycle and management of routes in Camel 2.5

[Claus Ibsen <cl...@gmail.com>]

On Mon, Aug 2, 2010 at 11:04 PM, Christian Schneider
<ch...@die-schneider.net> wrote:
>  Hi Claus,
>
> is it necessary to have a resume operation? I would propose to have only the
> start operation that can be called from stopped and from suspended state.
> Also I propose to allow the suspend command from stop to bring the route in
> a state where it can start faster.
>
> My proposed changes would allow to call any method from almost any state
> which makes the model easier to use and understand.
>

Yes you need to differentiate between resume/start in your business logic.
The consumer most often need to different logic. For example a HTTP
consumer may have to setup and start a HTTP server.
Where as in resume it just need to accept requests instead of sending
back http status 503 (temporary out of service).

Keep the states/methods separated makes it easier to implement and understand.

If you extend ServiceSupport then you got these callbacks you can override:
- doStart
- doStop
- doResume
- doSuspend
- doShutdown (special when CamelContext is shutting down)



But the logic in the startRoute, resumeRoute has built in so they
handle if you invoke either one.
So if a route is suspended and your invoke startRoute then it knows
that it should resume the route. And vice versa. So from a management
point of view in JMX it should not be confusing.


stop -> suspend is a bit harder, due you would need to start it and
then suspend it immediately, but there is a "window" where the route
is actually started, and it could potentially take in messages. And
routes can most often start very fast. Well there may be some WS
components who parse WSDL files etc which may take some time. We could
introduce a doWarmUp callback which is invoked when CamelContext is
starting up. It would then allow to pre parse the WSDL files etc. so
the consumer is "almost ready".



> Greetings
>
> Christian
>
>
> Am 02.08.2010 08:35, schrieb Claus Ibsen:
>>
>> Hi
>>
>> I am working a bit on improving the lifecycle and management of routes in
>> Camel.
>>
>> The bigger goal is to support suspension of routes / camel. For
>> example you can press suspend on CamelContext mbean and have it
>> temporary suspend all routes.
>> Then you can resume again by pressing resume, and the suspended routes
>> will be resumed. For example if there is a problem and you need to
>> just suspend Camel.
>> BTW Camel will only resume the routes which was suspended. So if you
>> have manually stopped a route beforehand, this route will remain
>> stopped.
>>
>> Also we introduce management of routes so you can suspend an
>> individual route as well.
>> There will be a change in behavior of the stop route operation in JMX.
>> Previously it would force stop a route, disregarding any in flight
>> exchanges.
>> Now the stop operation will use the graceful shutdown feature in Camel
>> and stop the route more reliable. If you want to force stop a route
>> you can pass in a timeout value, such as a couple of seconds. Which
>> triggers Camel to force stopping when the timeout was hit.
>>
>> Routes can now support suspension, which is done by letting it's
>> consumer implement the SuspendableService interface. This gives those
>> consumers a fine grained control of suspension.
>> There are doSuspend, doResume methods readily available from
>> ServiceSupport. Which means the DefaultConsumer got it already. For
>> example the camel-jms component leverages this in its JmsConsumer.
>> This allows it more gently to suspend listening on a JMS queue.
>>
>> Likewise camel-jetty consumer will return HTTP status 503 if its
>> suspended. If you stop it, then it will unbind the port listener. So
>> suspension is much quicker and gentler to use if you just temporary
>> need the route to be inactive.
>>
>> Component writers is encouraged to support this if its possible. For
>> example this would be great for the many HTTP based consumers to
>> support.
>>
>> On the same time we are fixing the JMX management to always list all
>> routes (also if they have been configured with autoStartup=false).
>> This allows you actually to manually start those routes from JMX :).
>>
>> And there will be a shutdown operation for routes. It's to be used for
>> removing a route from management. The idea would be for you have to
>> stop the route first before you can shutdown and remove the route.
>> Maybe we should rename the shutdown operation name to remove?
>>
>> You can see some background information here:
>> https://cwiki.apache.org/confluence/display/CAMEL/Lifecycle
>>
>> Which has a diagram of the lifecycle states.
>>
>>
>> Any thoughts?
>>
>>
>
> --
> ----
> http://www.liquid-reality.de
>
>



-- 
Claus Ibsen
Apache Camel Committer

Author of Camel in Action: http://www.manning.com/ibsen/
Open Source Integration: http://fusesource.com
Blog: http://davsclaus.blogspot.com/
Twitter: http://twitter.com/davsclaus

Re: [HEADS UP] - Improved lifecycle and management of routes in Camel 2.5

[Christian Schneider <ch...@die-schneider.net>]

  Hi Claus,

is it necessary to have a resume operation? I would propose to have only 
the start operation that can be called from stopped and from suspended 
state.
Also I propose to allow the suspend command from stop to bring the route 
in a state where it can start faster.

My proposed changes would allow to call any method from almost any state 
which makes the model easier to use and understand.

Greetings

Christian


Am 02.08.2010 08:35, schrieb Claus Ibsen:
> Hi
>
> I am working a bit on improving the lifecycle and management of routes in Camel.
>
> The bigger goal is to support suspension of routes / camel. For
> example you can press suspend on CamelContext mbean and have it
> temporary suspend all routes.
> Then you can resume again by pressing resume, and the suspended routes
> will be resumed. For example if there is a problem and you need to
> just suspend Camel.
> BTW Camel will only resume the routes which was suspended. So if you
> have manually stopped a route beforehand, this route will remain
> stopped.
>
> Also we introduce management of routes so you can suspend an
> individual route as well.
> There will be a change in behavior of the stop route operation in JMX.
> Previously it would force stop a route, disregarding any in flight
> exchanges.
> Now the stop operation will use the graceful shutdown feature in Camel
> and stop the route more reliable. If you want to force stop a route
> you can pass in a timeout value, such as a couple of seconds. Which
> triggers Camel to force stopping when the timeout was hit.
>
> Routes can now support suspension, which is done by letting it's
> consumer implement the SuspendableService interface. This gives those
> consumers a fine grained control of suspension.
> There are doSuspend, doResume methods readily available from
> ServiceSupport. Which means the DefaultConsumer got it already. For
> example the camel-jms component leverages this in its JmsConsumer.
> This allows it more gently to suspend listening on a JMS queue.
>
> Likewise camel-jetty consumer will return HTTP status 503 if its
> suspended. If you stop it, then it will unbind the port listener. So
> suspension is much quicker and gentler to use if you just temporary
> need the route to be inactive.
>
> Component writers is encouraged to support this if its possible. For
> example this would be great for the many HTTP based consumers to
> support.
>
> On the same time we are fixing the JMX management to always list all
> routes (also if they have been configured with autoStartup=false).
> This allows you actually to manually start those routes from JMX :).
>
> And there will be a shutdown operation for routes. It's to be used for
> removing a route from management. The idea would be for you have to
> stop the route first before you can shutdown and remove the route.
> Maybe we should rename the shutdown operation name to remove?
>
> You can see some background information here:
> https://cwiki.apache.org/confluence/display/CAMEL/Lifecycle
>
> Which has a diagram of the lifecycle states.
>
>
> Any thoughts?
>
>

-- 
----
http://www.liquid-reality.de