Exponential backoff for Scheduler Component

[Damir Murat <da...@gmail.com>]

Hi,

I'm using backoff support in scheduler component and it works nicely.
However, I'm missing exponential backoff support. Is there any reason (beside lack of time, of course) it is not there?
I think it would be very useful addition to the scheduler.

Regards,
Damir Murat

Re: Exponential backoff for Scheduler Component

[Damir Murat <da...@gmail.com>]

After some reading and experimenting, I've learned that DLC redelivery policy
actually prevents scheduler from firing. So for error cases, I'm covered. I
tried and everything works beautifully :-)

For the idle case, though, it would be great to have simple uri-based exponential
backoff support. Maybe with differentiation between error and idle scenarios,
if possible. Is this JIRA worthy material?

Regards,
Damir Murat

Re: Exponential backoff for Scheduler Component

[Damir Murat <da...@gmail.com>]

Thank you my friend for taking time to look into this. Glad to hear from you, even so publicly :-)

Although I'm not complete camel novice, I'm using it after very long time. So it is quite possible that
I misunderstood or forgot some important concepts.

With that aside, let me explain my use case in some more details.

I have a scheduled polling consumer which looks into a database for records with appropriate status.
If those are not found, polling consumer stays idle and does not route exchange any further. If error
occurs, I think it is enough to backoff and try again without any meaningful error handling:

// code in groovy
from("scheduler:timer?delay=1s&backoffMultiplier=10&backoffIdleThreshold=1&backoffErrorThreshold=1")
    .routeId("eupEventPublisherRoute")
    .process({ Exchange exchange ->
        String serviceBeanName = EupEventPublisherForCamelService.simpleName.uncapitalize()
        EupEventPublisherForCamelService eupEventPublisherForCamelService = exchange.context.registry.lookupByNameAndType(serviceBeanName, EupEventPublisherForCamelService)

        List<EupEventPublisherEntity> eventList = eupEventPublisherForCamelService.findAllNonPublishedEventsForCamelRoute()
        exchange.in.body = eventList

        if (eventList.size() < 1) {
          // Indicate to scheduler that there was no results, so it can backoff
          exchange.setProperty(Exchange.SCHEDULER_POLLED_MESSAGES, false)

          // Stop routing further since we do not want empty exchanges to end up in jms broker.
          exchange.setProperty(Exchange.ROUTE_STOP, true)
        }
    })
.split(body())
.to("log:camelLog?showAll=true")
...

Maybe I can throw some exception to signal idle case, but it looks too harsh to do. In addition,
I don't really know how DLC redelivery and scheduler interplay in such scenario. I guess DLC
redelivery will not stop scheduler from firing.

I thought it would be easier to have exponential backoff in scheduler itself, so I can gradually backoff
in idle case. It would be even better to be able to configure idle and error cases independently (something
like backoffIdleMultiplier and backoffErrorMultiplier) since, in case of errors, I would like to have
a different and bigger multiplier then for idle case.

Hope this makes some sense.

Regards,
Damir Murat

Re: Exponential backoff for Scheduler Component

[Zoran Regvart <zo...@regvart.com>]

Hi Damir,
that might be a good thing to add to the scheduler component. There is
support for exponential backoff in the RedeliveryPolicy[1] that might
fit your use case.

zoran

[1] https://github.com/apache/camel/blob/master/camel-core/src/main/docs/eips/dead-letter-channel.adoc#redelivery

On Fri, Jun 1, 2018 at 10:58 AM, Damir Murat <da...@gmail.com> wrote:
> Hi,
>
> I'm using backoff support in scheduler component and it works nicely.
> However, I'm missing exponential backoff support. Is there any reason (beside lack of time, of course) it is not there?
> I think it would be very useful addition to the scheduler.
>
> Regards,
> Damir Murat
>



-- 
Zoran Regvart