shutdown strategy is set on context level / way of route level?

[onders <on...@gmail.com>]

Hi,

As per my understanding, shutdown strategy is set per camelcontext.
Is there a way / other ways where we can apply shutdown strategy or custom
shutdown implementation per route(s) in a camel context?

In my use case, i have number of events stored in database already. In order
to avoid reprocessing a single event multiple times, i use onConsume option
in sql component and mark their status e.g status=1 meaning 'processing'
where initially records were in the db with status=0 meaning 'initial'. What
i want to do is that when the PID of my camel context and routes is signaled
like 'kill -9 <pid>', i want to reset the status of records back to status=0
'initial' in order not to miss processing those records when the process is
restarted.

Or is there any other way doing what i want to achieve?

Thanks 




--
View this message in context: http://camel.465427.n5.nabble.com/shutdown-strategy-is-set-on-context-level-way-of-route-level-tp5808094.html
Sent from the Camel - Users mailing list archive at Nabble.com.

Re: shutdown strategy is set on context level / way of route level?

[Onder SEZGIN <on...@gmail.com>]

Thanks Zoran.
Idempotent consumer is nice option but it brings (persistence is essential
for me not to lose messages) persistence problem and using a repository to
write the messages and idempotent consumer will do check against the
repository. This will bring a delay for the probably. (potential
performance problem)

SELECT FOR UPDATE SKIP LOCKED seems to be tricky from the blogposts . one
example is this :
https://markjbobak.wordpress.com/2010/04/06/unintended-consequences/

Well apart from your great suggestions (gave me looking at the problem from
a different angle)  from my previous experience(for a similar but in a
different application), my route with following java DSLs

.shutdownRoute(ShutdownRoute.Defer)
...
.completeAllOnStop()
.forceCompletionOnStop()

helped me to get some time space to complete all running transactions in a
route to get completed before the application goes down when it got SIGKILL
signal.

At that time, it felt a bit tricky. So still i feel if i apply such similar
solution, it will again be tricky solution and i am not %100 sure if the
solution will work out.

I will give your suggestions tries :)

Seems all the 3 different option got tricky paths.

Let's see how i get around.
Will keep you posted about the consequence.

If any suggestions pops out, feel free to ping me in this thread :)

Regards
Onder

On Thu, Jul 27, 2017 at 11:34 AM, Zoran Regvart <zo...@regvart.com> wrote:

> Hi Önder,
> I think it might be worth looking into idempotent consumer pattern[1],
> or roll your own logic based on not committing the transaction that
> updates the status to 1 until finished and locking the row beforehand,
> and some databases (Oracle IIRC) have a SKIP LOCKED clause that you
> can use to fetch all unlocked rows.
>
> So in pseudo code:
>
> SELECT FOR UPDATE ... WHERE STATUS = 0 SKIP LOCKED
> do work
> UPDATE ... SET STATUS = 1 WHERE ...
> COMMIT
>
> But yeah, I would rely on the database functionality to keep track of
> the status, I don't think shutdown strategy would be as helpful with
> SIGKILL, as the whole JVM would be forced to quit,
>
> zoran
>
> [1] https://camel.apache.org/idempotent-consumer.html
>
> On Thu, Jul 27, 2017 at 6:43 AM, onders <on...@gmail.com>
> wrote:
> > Hi,
> >
> > As per my understanding, shutdown strategy is set per camelcontext.
> > Is there a way / other ways where we can apply shutdown strategy or
> custom
> > shutdown implementation per route(s) in a camel context?
> >
> > In my use case, i have number of events stored in database already. In
> order
> > to avoid reprocessing a single event multiple times, i use onConsume
> option
> > in sql component and mark their status e.g status=1 meaning 'processing'
> > where initially records were in the db with status=0 meaning 'initial'.
> What
> > i want to do is that when the PID of my camel context and routes is
> signaled
> > like 'kill -9 <pid>', i want to reset the status of records back to
> status=0
> > 'initial' in order not to miss processing those records when the process
> is
> > restarted.
> >
> > Or is there any other way doing what i want to achieve?
>
>
> --
> Zoran Regvart
>

Re: shutdown strategy is set on context level / way of route level?

[Zoran Regvart <zo...@regvart.com>]

Hi Önder,
I think it might be worth looking into idempotent consumer pattern[1],
or roll your own logic based on not committing the transaction that
updates the status to 1 until finished and locking the row beforehand,
and some databases (Oracle IIRC) have a SKIP LOCKED clause that you
can use to fetch all unlocked rows.

So in pseudo code:

SELECT FOR UPDATE ... WHERE STATUS = 0 SKIP LOCKED
do work
UPDATE ... SET STATUS = 1 WHERE ...
COMMIT

But yeah, I would rely on the database functionality to keep track of
the status, I don't think shutdown strategy would be as helpful with
SIGKILL, as the whole JVM would be forced to quit,

zoran

[1] https://camel.apache.org/idempotent-consumer.html

On Thu, Jul 27, 2017 at 6:43 AM, onders <on...@gmail.com> wrote:
> Hi,
>
> As per my understanding, shutdown strategy is set per camelcontext.
> Is there a way / other ways where we can apply shutdown strategy or custom
> shutdown implementation per route(s) in a camel context?
>
> In my use case, i have number of events stored in database already. In order
> to avoid reprocessing a single event multiple times, i use onConsume option
> in sql component and mark their status e.g status=1 meaning 'processing'
> where initially records were in the db with status=0 meaning 'initial'. What
> i want to do is that when the PID of my camel context and routes is signaled
> like 'kill -9 <pid>', i want to reset the status of records back to status=0
> 'initial' in order not to miss processing those records when the process is
> restarted.
>
> Or is there any other way doing what i want to achieve?


-- 
Zoran Regvart