You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@camel.apache.org by Ron Cecchini <ro...@comcast.net> on 2019/09/27 19:14:15 UTC
One CamelContext vs. multiple CamelContexts
TL;DR: 1 CamelContext with 100 Routes vs. 100 CamelContexts each with 1 Route
Say I need to ingest data from a hundred sensors or data sources, over TCP or JMS, and get it written to a central database or JMS.
The messages are asynchronous and don't require a response or any processing. We just have to suck in all that data and write it out to a DB or JMS.
It would be really nice to keep these 100 very simple routes in a single config / RouteBuilder. But that's not the smart thing to do... By the time you reach a 100 routes you'd probably need an app server and access to a cluster. But I don't think spinning up a new CamelContext / app for 100 single Routes is the way to go either. Or maybe it is? Maybe you containerize every single Route with Docker and manage it with Kubernetes (or whatever)?
I guess I'm just looking to see if anyone has experimented with this and did some performance comparisons - like, how many Routes were you able to cram into your CamelContext / Spring Boot app before it started degrading? And how folks managed a scenario like this where they had to pull in data from many sources.
If you don't have a cluster, and have to keep everything on a single beefy host, I guess the question is moot and you have to do as much as you can in one CamelContext until you hit a scalability limit...
Thanks and have a good weekend.
Ron
Re: One CamelContext vs. multiple CamelContexts
Posted by Jerry Smith <je...@gmail.com>.
Building on what Claus said regarding 'deployment unit', I deploy my
'application' to Apache Karaf. In total it consists of 17 bundles(contexts)
containing 642 routes. How you organize it all is going to depend on what
makes sense for your application to be able to start/stop and update
independently from other pieces of your app.
On Mon, Sep 30, 2019 at 4:07 AM Claus Ibsen <cl...@gmail.com> wrote:
> Hi
>
> Camel has no limitations on the number of routes per context. However
> you should have 1 camel context per "app" (deployment unit), so they
> have the same lifecycle etc.
> So a rule of thumb is to have routes that belong together, and share
> same lifecycle / change together in your app.
>
> So if your app has 100 routes then that is fine. Just mind starting up
> and shutting down 100 routes may take a little time.
>
> In container world then 1 route per container is surely overkill, as
> jvm overhead is still signifcant. Well camel-quarkus will help a lot
> on this when you can AOT compile to native, but there are still some
> way to go for 3rd party to fully support this and the graalvm project
> itself.
>
>
> On Fri, Sep 27, 2019 at 9:14 PM Ron Cecchini <ro...@comcast.net>
> wrote:
> >
> > TL;DR: 1 CamelContext with 100 Routes vs. 100 CamelContexts each with 1
> Route
> >
> > Say I need to ingest data from a hundred sensors or data sources, over
> TCP or JMS, and get it written to a central database or JMS.
> >
> > The messages are asynchronous and don't require a response or any
> processing. We just have to suck in all that data and write it out to a DB
> or JMS.
> >
> > It would be really nice to keep these 100 very simple routes in a single
> config / RouteBuilder. But that's not the smart thing to do... By the
> time you reach a 100 routes you'd probably need an app server and access to
> a cluster. But I don't think spinning up a new CamelContext / app for 100
> single Routes is the way to go either. Or maybe it is? Maybe you
> containerize every single Route with Docker and manage it with Kubernetes
> (or whatever)?
> >
> > I guess I'm just looking to see if anyone has experimented with this and
> did some performance comparisons - like, how many Routes were you able to
> cram into your CamelContext / Spring Boot app before it started degrading?
> And how folks managed a scenario like this where they had to pull in data
> from many sources.
> >
> > If you don't have a cluster, and have to keep everything on a single
> beefy host, I guess the question is moot and you have to do as much as you
> can in one CamelContext until you hit a scalability limit...
> >
> > Thanks and have a good weekend.
> >
> > Ron
>
>
>
> --
> Claus Ibsen
> -----------------
> http://davsclaus.com @davsclaus
> Camel in Action 2: https://www.manning.com/ibsen2
>
Re: One CamelContext vs. multiple CamelContexts
Posted by Claus Ibsen <cl...@gmail.com>.
Hi
Camel has no limitations on the number of routes per context. However
you should have 1 camel context per "app" (deployment unit), so they
have the same lifecycle etc.
So a rule of thumb is to have routes that belong together, and share
same lifecycle / change together in your app.
So if your app has 100 routes then that is fine. Just mind starting up
and shutting down 100 routes may take a little time.
In container world then 1 route per container is surely overkill, as
jvm overhead is still signifcant. Well camel-quarkus will help a lot
on this when you can AOT compile to native, but there are still some
way to go for 3rd party to fully support this and the graalvm project
itself.
On Fri, Sep 27, 2019 at 9:14 PM Ron Cecchini <ro...@comcast.net> wrote:
>
> TL;DR: 1 CamelContext with 100 Routes vs. 100 CamelContexts each with 1 Route
>
> Say I need to ingest data from a hundred sensors or data sources, over TCP or JMS, and get it written to a central database or JMS.
>
> The messages are asynchronous and don't require a response or any processing. We just have to suck in all that data and write it out to a DB or JMS.
>
> It would be really nice to keep these 100 very simple routes in a single config / RouteBuilder. But that's not the smart thing to do... By the time you reach a 100 routes you'd probably need an app server and access to a cluster. But I don't think spinning up a new CamelContext / app for 100 single Routes is the way to go either. Or maybe it is? Maybe you containerize every single Route with Docker and manage it with Kubernetes (or whatever)?
>
> I guess I'm just looking to see if anyone has experimented with this and did some performance comparisons - like, how many Routes were you able to cram into your CamelContext / Spring Boot app before it started degrading? And how folks managed a scenario like this where they had to pull in data from many sources.
>
> If you don't have a cluster, and have to keep everything on a single beefy host, I guess the question is moot and you have to do as much as you can in one CamelContext until you hit a scalability limit...
>
> Thanks and have a good weekend.
>
> Ron
--
Claus Ibsen
-----------------
http://davsclaus.com @davsclaus
Camel in Action 2: https://www.manning.com/ibsen2
Re: One CamelContext vs. multiple CamelContexts
Posted by Jan Bednář <ma...@janbednar.eu>.
Hi Ron,
Mixing multiple CamelContext per application is not recomended approach.
Also support for multiple contexts is removed in Camel 3:
https://camel.apache.org/manual/latest/camel-3-migration-guide.html#_multiple_camelcontexts_per_application_not_supported
Dne 27.9.2019 v 21:19 Mantas Gridinas napsal(a):
> You're going to be fine. My current project runs 400+ routes in single context.
>
> On Fri, Sep 27, 2019 at 7:14 PM Ron Cecchini <ro...@comcast.net> wrote:
>> TL;DR: 1 CamelContext with 100 Routes vs. 100 CamelContexts each with 1 Route
>>
>> Say I need to ingest data from a hundred sensors or data sources, over TCP or JMS, and get it written to a central database or JMS.
>>
>> The messages are asynchronous and don't require a response or any processing. We just have to suck in all that data and write it out to a DB or JMS.
>>
>> It would be really nice to keep these 100 very simple routes in a single config / RouteBuilder. But that's not the smart thing to do... By the time you reach a 100 routes you'd probably need an app server and access to a cluster. But I don't think spinning up a new CamelContext / app for 100 single Routes is the way to go either. Or maybe it is? Maybe you containerize every single Route with Docker and manage it with Kubernetes (or whatever)?
>>
>> I guess I'm just looking to see if anyone has experimented with this and did some performance comparisons - like, how many Routes were you able to cram into your CamelContext / Spring Boot app before it started degrading? And how folks managed a scenario like this where they had to pull in data from many sources.
>>
>> If you don't have a cluster, and have to keep everything on a single beefy host, I guess the question is moot and you have to do as much as you can in one CamelContext until you hit a scalability limit...
>>
>> Thanks and have a good weekend.
>>
>> Ron
Re: One CamelContext vs. multiple CamelContexts
Posted by Mantas Gridinas <mg...@gmail.com>.
You're going to be fine. My current project runs 400+ routes in single context.
On Fri, Sep 27, 2019 at 7:14 PM Ron Cecchini <ro...@comcast.net> wrote:
>
> TL;DR: 1 CamelContext with 100 Routes vs. 100 CamelContexts each with 1 Route
>
> Say I need to ingest data from a hundred sensors or data sources, over TCP or JMS, and get it written to a central database or JMS.
>
> The messages are asynchronous and don't require a response or any processing. We just have to suck in all that data and write it out to a DB or JMS.
>
> It would be really nice to keep these 100 very simple routes in a single config / RouteBuilder. But that's not the smart thing to do... By the time you reach a 100 routes you'd probably need an app server and access to a cluster. But I don't think spinning up a new CamelContext / app for 100 single Routes is the way to go either. Or maybe it is? Maybe you containerize every single Route with Docker and manage it with Kubernetes (or whatever)?
>
> I guess I'm just looking to see if anyone has experimented with this and did some performance comparisons - like, how many Routes were you able to cram into your CamelContext / Spring Boot app before it started degrading? And how folks managed a scenario like this where they had to pull in data from many sources.
>
> If you don't have a cluster, and have to keep everything on a single beefy host, I guess the question is moot and you have to do as much as you can in one CamelContext until you hit a scalability limit...
>
> Thanks and have a good weekend.
>
> Ron