You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@camel.apache.org by GitBox <gi...@apache.org> on 2022/08/25 11:34:15 UTC
[GitHub] [camel-k] brunoNetId opened a new issue, #3576: How to run a Camel process as a cron job (other than a timer)
brunoNetId opened a new issue, #3576:
URL: https://github.com/apache/camel-k/issues/3576
With the following use case as an example:
- process the daily Kafka messages produced in a topic
How could we run a Cron job with Camel K that connects to Kafka, consumes the latest messages, and shuts down when no more messages are available (e.g. setting a timeout).
I've created a proposed example (attached) where I use an aggregator with a timeout. When the aggregator completes, let's say when no new messages arrive in a specific window of time, a shutdown process trigger to stop the CamelContext and the Java process.
Cron could see the process has stopped and schedule the next run the next day or as per a Camel timer definition.
For simplicity, the example uses HTTP instead of Kafka.
[cron-example.tar.gz](https://github.com/apache/camel-k/files/9424194/cron-example.tar.gz)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@camel.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [camel-k] davsclaus commented on issue #3576: How to run a Camel process as a cron job (other than a timer)
Posted by GitBox <gi...@apache.org>.
davsclaus commented on issue #3576:
URL: https://github.com/apache/camel-k/issues/3576#issuecomment-1227172846
You can use the camel.main.durationXXX options where you can say that if Camel is idle after 30 seconds (then terminate). You can then use from kafka in the route, that would then drain the topic until there are no more messages. Then Camel becomes idle and after X period it terminates itself.
And use the cron trait to make it run once per day or what schedule you need.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@camel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [camel-k] github-actions[bot] closed issue #3576: How to run a Camel process as a cron job (other than a timer)
Posted by GitBox <gi...@apache.org>.
github-actions[bot] closed issue #3576: How to run a Camel process as a cron job (other than a timer)
URL: https://github.com/apache/camel-k/issues/3576
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@camel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [camel-k] brunoNetId commented on issue #3576: How to run a Camel process as a cron job (other than a timer)
Posted by GitBox <gi...@apache.org>.
brunoNetId commented on issue #3576:
URL: https://github.com/apache/camel-k/issues/3576#issuecomment-1227212869
As per @lburgazzoli, the problem there is that all Camel exchanges may have terminated, but not the aggregator... Camel may think it is idle, but it is actually not, the aggregator is still ongoing.
Another problem is that currently the cron trait only works with timer and passive components... not if a kafka consumer is defined (I think)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@camel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [camel-k] brunoNetId commented on issue #3576: How to run a Camel process as a cron job (other than a timer)
Posted by GitBox <gi...@apache.org>.
brunoNetId commented on issue #3576:
URL: https://github.com/apache/camel-k/issues/3576#issuecomment-1227147906
Yes... unfortunately, the trait seems limited and not able to cover the proposed use case.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@camel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [camel-k] github-actions[bot] commented on issue #3576: How to run a Camel process as a cron job (other than a timer)
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on issue #3576:
URL: https://github.com/apache/camel-k/issues/3576#issuecomment-1325796065
This issue has been automatically marked as stale due to 90 days of inactivity.
It will be closed if no further activity occurs within 15 days.
If you think that’s incorrect or the issue should never stale, please simply write any comment.
Thanks for your contributions!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@camel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [camel-k] lburgazzoli commented on issue #3576: How to run a Camel process as a cron job (other than a timer)
Posted by GitBox <gi...@apache.org>.
lburgazzoli commented on issue #3576:
URL: https://github.com/apache/camel-k/issues/3576#issuecomment-1227153108
The issue with the cron trait for this particular case is that the camel context is stopped after the first exchange completes, however if an aggregation policy is defined, then the cron would kill the pod before the aggregator completes.
We should probably enhance the [shutdown strategy](https://github.com/apache/camel-k-runtime/blob/main/camel-k-cron/impl/src/main/java/org/apache/camel/k/cron/CronSourceLoaderInterceptor.java#L111-L136) to handle this case
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@camel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [camel-k] christophd commented on issue #3576: How to run a Camel process as a cron job (other than a timer)
Posted by GitBox <gi...@apache.org>.
christophd commented on issue #3576:
URL: https://github.com/apache/camel-k/issues/3576#issuecomment-1227144954
there is a cron trait that may be useful for this: https://camel.apache.org/camel-k/1.9.x/traits/cron.html
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@camel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org