Load route steps dynamically

[ski n <ra...@gmail.com>]

Hi all,

I have a question how to load (add or remove) route steps dynamically with
Java/Java DSL.

For example, I have a route with around 20 steps. The users can configure
which steps are used and which not.

Now the creating of the route has the following steps:

1) Write code (Java DSL)
2) Compile code
3) Load (Add) Route
4) Start Route
5) Run Route

I have the option to write multiple routes and load the route based on user
preferences. But this is not desirable because I need to maintain tens of
routes (instead of a few). The other possibility is that I add a choice
step and let it happen in runtime (but this takes unneeded performance, as
it already known on load if these steps are needed or not.

What are my options of add/remove steps on loading a route?

Things I came up with (but I am not sure if they are possible):

1) Change from Java to XML and manipulate the XML (Preferable not)

2) Add construction in route to load or not. For example:

split("mySplit").loadStep(true/false)

When false this step is removed on load.

3) Create the route as string and load it dynamically on start (something
like RouteLoader). Of course there are multiple instances running with
different configuration.

4) Use a RouteTemplate and somehow configure if a step is used or not based
on parameters.

5) Use some aspect programming, just like unit tests: weaveById(id).remove()

So I liked to avoid the first option (using XML), because and then need to
rewrite several routes. Is one of the other possible? Or is there another
possibility where I am unaware of?

Greets,

Raymond

Re: Load route steps dynamically

[Claus Ibsen <cl...@gmail.com>]

Ad 4)

We designed route templates on purpose to be static and simple to keep
the concept easy while we evolve and improve its implementation.
The recent additon of Kamelets to the core, and with the many new DSLs
makes it important that the route templates work great across all of
those.

However if there comes up more advanced use cases for route templates
/ kamelets, then some kind of programming control flow may be
something we need to introduce.


Ad 5)
Yes you can always do something similar to what we have with advice
with to manipulate the route.
And then add the result of that operation as the route.


Ad 2)
You can always in Java DSL, have a base route builder class which has
getter/setter to turn on "steps", and the use that to add dynamic
routes.
Then in the configure method you just use regular Java code with if
... elseif .. blocks to build the route DSL.

The route DSL does not have to be one single continuation of stacked
method calls. You can break it up and use if { ... } blocks
and in there use the DSL to do "stuff" on the route.

On Sat, Apr 17, 2021 at 10:00 AM ski n <ra...@gmail.com> wrote:
>
> Hi all,
>
> I have a question how to load (add or remove) route steps dynamically with
> Java/Java DSL.
>
> For example, I have a route with around 20 steps. The users can configure
> which steps are used and which not.
>
> Now the creating of the route has the following steps:
>
> 1) Write code (Java DSL)
> 2) Compile code
> 3) Load (Add) Route
> 4) Start Route
> 5) Run Route
>
> I have the option to write multiple routes and load the route based on user
> preferences. But this is not desirable because I need to maintain tens of
> routes (instead of a few). The other possibility is that I add a choice
> step and let it happen in runtime (but this takes unneeded performance, as
> it already known on load if these steps are needed or not.
>
> What are my options of add/remove steps on loading a route?
>
> Things I came up with (but I am not sure if they are possible):
>
> 1) Change from Java to XML and manipulate the XML (Preferable not)
>
> 2) Add construction in route to load or not. For example:
>
> split("mySplit").loadStep(true/false)
>
> When false this step is removed on load.
>
> 3) Create the route as string and load it dynamically on start (something
> like RouteLoader). Of course there are multiple instances running with
> different configuration.
>
> 4) Use a RouteTemplate and somehow configure if a step is used or not based
> on parameters.
>
> 5) Use some aspect programming, just like unit tests: weaveById(id).remove()
>
> So I liked to avoid the first option (using XML), because and then need to
> rewrite several routes. Is one of the other possible? Or is there another
> possibility where I am unaware of?
>
> Greets,
>
> Raymond



-- 
Claus Ibsen
-----------------
http://davsclaus.com @davsclaus
Camel in Action 2: https://www.manning.com/ibsen2

Re: Load route steps dynamically

[ski n <ra...@gmail.com>]

Yes, I already use the choice blocks. I wondered that there maybe was a
performance penalty, while the choices aren't really necessary in runtime.
Did a small load test between minimal route (8 steps) vs full route (94
steps). Done with Camel 3.7.3. The test was processing 100 MB of messages
with both routes.

The difference between the two was that the minimal route was 6% faster.
This shows how efficient Camel's routing engine is. So I am not too
worried. On the other hand it is best to go for the optimal result. It
would be nice if the routeTemplate would support optional steps.

Raymond

Op za 17 apr. 2021 om 13:51 schreef Mantas Gridinas <mg...@gmail.com>:

> Sounds like you're programming in your routes rather than integrating
> processes/systems. I think the simplest solution would be using
> feature flags in choice blocks.
>
> On Sat, Apr 17, 2021 at 8:00 AM ski n <ra...@gmail.com> wrote:
> >
> > Hi all,
> >
> > I have a question how to load (add or remove) route steps dynamically
> with
> > Java/Java DSL.
> >
> > For example, I have a route with around 20 steps. The users can configure
> > which steps are used and which not.
> >
> > Now the creating of the route has the following steps:
> >
> > 1) Write code (Java DSL)
> > 2) Compile code
> > 3) Load (Add) Route
> > 4) Start Route
> > 5) Run Route
> >
> > I have the option to write multiple routes and load the route based on
> user
> > preferences. But this is not desirable because I need to maintain tens of
> > routes (instead of a few). The other possibility is that I add a choice
> > step and let it happen in runtime (but this takes unneeded performance,
> as
> > it already known on load if these steps are needed or not.
> >
> > What are my options of add/remove steps on loading a route?
> >
> > Things I came up with (but I am not sure if they are possible):
> >
> > 1) Change from Java to XML and manipulate the XML (Preferable not)
> >
> > 2) Add construction in route to load or not. For example:
> >
> > split("mySplit").loadStep(true/false)
> >
> > When false this step is removed on load.
> >
> > 3) Create the route as string and load it dynamically on start (something
> > like RouteLoader). Of course there are multiple instances running with
> > different configuration.
> >
> > 4) Use a RouteTemplate and somehow configure if a step is used or not
> based
> > on parameters.
> >
> > 5) Use some aspect programming, just like unit tests:
> weaveById(id).remove()
> >
> > So I liked to avoid the first option (using XML), because and then need
> to
> > rewrite several routes. Is one of the other possible? Or is there another
> > possibility where I am unaware of?
> >
> > Greets,
> >
> > Raymond
>

Re: Load route steps dynamically

[Mantas Gridinas <mg...@gmail.com>]

Sounds like you're programming in your routes rather than integrating
processes/systems. I think the simplest solution would be using
feature flags in choice blocks.

On Sat, Apr 17, 2021 at 8:00 AM ski n <ra...@gmail.com> wrote:
>
> Hi all,
>
> I have a question how to load (add or remove) route steps dynamically with
> Java/Java DSL.
>
> For example, I have a route with around 20 steps. The users can configure
> which steps are used and which not.
>
> Now the creating of the route has the following steps:
>
> 1) Write code (Java DSL)
> 2) Compile code
> 3) Load (Add) Route
> 4) Start Route
> 5) Run Route
>
> I have the option to write multiple routes and load the route based on user
> preferences. But this is not desirable because I need to maintain tens of
> routes (instead of a few). The other possibility is that I add a choice
> step and let it happen in runtime (but this takes unneeded performance, as
> it already known on load if these steps are needed or not.
>
> What are my options of add/remove steps on loading a route?
>
> Things I came up with (but I am not sure if they are possible):
>
> 1) Change from Java to XML and manipulate the XML (Preferable not)
>
> 2) Add construction in route to load or not. For example:
>
> split("mySplit").loadStep(true/false)
>
> When false this step is removed on load.
>
> 3) Create the route as string and load it dynamically on start (something
> like RouteLoader). Of course there are multiple instances running with
> different configuration.
>
> 4) Use a RouteTemplate and somehow configure if a step is used or not based
> on parameters.
>
> 5) Use some aspect programming, just like unit tests: weaveById(id).remove()
>
> So I liked to avoid the first option (using XML), because and then need to
> rewrite several routes. Is one of the other possible? Or is there another
> possibility where I am unaware of?
>
> Greets,
>
> Raymond