You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@camel.apache.org by Steve973 <st...@gmail.com> on 2022/11/05 15:05:33 UTC
Request for input re: Dynamic Router EIP component
Hello. I introduced a Dynamic Router EIP component in 3.15 because I had
uses for a dynamic router that was a bit different than what the core
component offers. I would like some feedback about what features to
potentially add that would make it even more useful. First, just in case
you are not familiar with its capabilities, I will describe how I am using
it in a software system that I am building at work, which exercises it
fairly well. We have a little fewer than 10 spring boot modules that
handle specific tasks of the system. One of the modules is a "routing
engine" that is mostly a wrapper for Camel, along with some
business-logic-specific integration code. This is where I instantiate the
Dynamic Router component, and create a JMS route that can feed it
subscriber messages. The other modules register their subscriptions with
the dynamic router by sending messages over JMS to the router. These
messages contain a predicate that allows the dynamic router to evaluate
messages that pass through it to determine one, or more, recipients. With
their subscription messages, they also include a destination URI for
applicable messages to be routed to. This allows for a somewhat advanced
version of the command pattern in my system.
Can any of you think of what this is missing? My first implementation used
a list for keeping track of subscribers, but that allowed a module to
register its commands multiple times. So, that has been changed to a map
that maps by subscriber ID. The other thing, and I have not developed a
solution for this, but in the event of an error, the "routing table" of
routes and endpoints is not available that I know of. I am not sure how to
handle that, or if there is something built into Camel that might
facilitate that type of tracing/debugging information, of it I might need
to keep track of it in a table within the component.
Anyway, this is already long enough, and I don't want to bore you with too
many details. If you have time, please provide some feedback, and I will
get more features into the Dynamic Router EIP.
Thanks,
Steve
Re: Request for input re: Dynamic Router EIP component
Posted by Steve973 <st...@gmail.com>.
I am glad that my explanation helped. So, let's get into your questions
right away:
1. This is where you have to decide how you want your clients to
subscribe. I set up a JMS queue, and set up a very simple route that reads
from that queue and sends the message right to the control channel. It is
far less useful in the same JVM, I would think.
2. That way would be much less dynamic. You have to know your routes at
compile time. The example that you are giving is much like the original
dynamic router in the Camel DSL.
3. Camel has plenty of documentation on Predicates. When I created the
documentation, I intentionally skipped a tutorial on Predicate usage,
because it would be redundant. Using the "Simple" language is a good and
pretty straightforward way to go with predicates.
On Sat, Nov 5, 2022 at 3:05 PM ski n <ra...@gmail.com> wrote:
> Yeah, that makes it clearer.
>
> Some remarks:
>
> 1) How does the channel work? Does it work only within the CamelContext
> (like SEDA), in the whole virtual machine (like VM) or can it be used
> distributed (like JMS)?
> 2) Wouldn't be easier if you separate it in two components: "publish" and
> "subscribe" and then do something like:
>
> from("direct:a")
> .to("publish:mychannel");
>
> from("subcribe:mychannel?id=1&priority=1&predicate=<text predicate>")
> .to("someUri");
>
> from("subcribe:mychannel?id=2&priority=2predicate=<text predicate>")
> .to("someUri");
>
> from("subcribe:mychannel?id=3&priority=3&predicate=<text predicate>")
> .to("someUri");
>
> 3) In the documentation maybe could to add an example with subscribe while
> using different expression languages/predicates.
>
>
> Raymond
>
>
>
>
>
>
>
>
>
>
> On Sat, Nov 5, 2022 at 7:01 PM Steve973 <st...@gmail.com> wrote:
>
> > Hi, Raymond, and thank you so much for your response! I appreciate it,
> and
> > I appreciate that you included those two links that I probably should
> have
> > included.
> >
> > I will do my best to answer your questions, but since I am not the best
> > teacher, please let me know where I need to explain better:
> >
> > 1. The best way I would generally describe the use case for this EIP
> > component is when you want dynamic orchestration between modules of
> your
> > system:
> > a. You have services that come and go over time, possibly because your
> > system watches for something to occur, and then it spins up services
> > to handle whatever follows. This allows them to plug into routing,
> > perform
> > their tasks, and then unplug from routing when they are going to stop
> > and
> > go away.
> > b. You have a way to programmatically enable and disable features of
> > your system. Enable them, and they subscribe for routing. Disable
> > them,
> > and they unsubscribe.
> > c. You want a way to decentralize your routing logic and keep
> > module-specific routing in the applicable module. The router does not
> > have
> > to know the system's list of commands, or how to handle any of them
> > prior
> > to runtime.
> > 2. Your understanding of the channels is correct. This allows for
> > logical separation of routing. One channel has one set of subscribers
> > that
> > is separate from any other channel that has its own (separate) set of
> > subscribers.
> > 3. You can use an object for subscribing or unsubscribing, and you can
> > also use the URI (see
> >
> >
> https://camel.apache.org/components/3.18.x/dynamic-router-component.html#_method_2_uri_only
> > to see what I mean). Use of the component in a route is as simple as
> > from("<some uri>").to("dynamic-router:<channel name>") so, at least in
> > that
> > part, you don't have to worry about any java code, since using the
> > dynamic
> > router component is just like using any other camel component. When
> you
> > are subscribing/unsubscribing is where using code might apply, to
> create
> > the subscription message and then send it, for example, by a producer
> > template.
> >
> > Please let me know if this helps. I am always open to expanding its
> > usefulness, or simplifying its use. So, if you have any suggestions, I
> > would be happy to see how I could get it integrated into the component.
> >
> > Thanks again,
> > Steve
> >
> > On Sat, Nov 5, 2022 at 11:44 AM ski n <ra...@gmail.com> wrote:
> >
> > > For those interested. Here are some more resources:
> > >
> > > Background:
> > >
> > > https://camel.apache.org/blog/2022/01/dynamic-router-eip-component/
> > >
> > > Documentation:
> > >
> > >
> https://camel.apache.org/components/3.18.x/dynamic-router-component.html
> > >
> > > -------------------
> > >
> > > For me, reading through the documentation, it seems like a powerful
> > > component. However, there are some things I am failing to grasp:
> > >
> > > 1. What are the use cases this can be used for. And with kind of use
> > cases
> > > I mean business use cases explained in simple terms for end-users.
> > > 2. As I understand it, the dynamic router provides 'topics' similar to
> > > Kafka topics or JMS topics, but then build into Camel. Is this correct?
> > > 3. In the documentation examples I see a lot of plain Java code. Things
> > > like unsubscribing:
> > >
> > > // Send message to the Dynamic Router "billing" channel// to
> > > unsubscribe from billing messagesDynamicRouterControlMessage
> > > unsubscribeMsg = new UnsubscribeMessageBuilder()
> > > .id("billingSubscription")
> > > .channel("billing")
> > > .build();
> > > template.sendBody("dynamic-router:control", unsubscribeMsg);
> > >
> > > I try to use the Route DSL's as much as possible. (especially XML and
> > YAML,
> > > because they are easily to manipulate). How can I use the component
> > within
> > > regular routes without things like Predicates and other Java code?
> > >
> > > Kind regards,
> > >
> > > Raymond
> > >
> > >
> > >
> > >
> > >
> > >
> > >
> > >
> > >
> > > On Sat, Nov 5, 2022 at 4:06 PM Steve973 <st...@gmail.com> wrote:
> > >
> > > > Hello. I introduced a Dynamic Router EIP component in 3.15 because I
> > had
> > > > uses for a dynamic router that was a bit different than what the core
> > > > component offers. I would like some feedback about what features to
> > > > potentially add that would make it even more useful. First, just in
> > case
> > > > you are not familiar with its capabilities, I will describe how I am
> > > using
> > > > it in a software system that I am building at work, which exercises
> it
> > > > fairly well. We have a little fewer than 10 spring boot modules that
> > > > handle specific tasks of the system. One of the modules is a
> "routing
> > > > engine" that is mostly a wrapper for Camel, along with some
> > > > business-logic-specific integration code. This is where I
> instantiate
> > > the
> > > > Dynamic Router component, and create a JMS route that can feed it
> > > > subscriber messages. The other modules register their subscriptions
> > with
> > > > the dynamic router by sending messages over JMS to the router. These
> > > > messages contain a predicate that allows the dynamic router to
> evaluate
> > > > messages that pass through it to determine one, or more, recipients.
> > > With
> > > > their subscription messages, they also include a destination URI for
> > > > applicable messages to be routed to. This allows for a somewhat
> > advanced
> > > > version of the command pattern in my system.
> > > >
> > > > Can any of you think of what this is missing? My first
> implementation
> > > used
> > > > a list for keeping track of subscribers, but that allowed a module to
> > > > register its commands multiple times. So, that has been changed to a
> > map
> > > > that maps by subscriber ID. The other thing, and I have not
> developed
> > a
> > > > solution for this, but in the event of an error, the "routing table"
> of
> > > > routes and endpoints is not available that I know of. I am not sure
> > how
> > > to
> > > > handle that, or if there is something built into Camel that might
> > > > facilitate that type of tracing/debugging information, of it I might
> > need
> > > > to keep track of it in a table within the component.
> > > >
> > > > Anyway, this is already long enough, and I don't want to bore you
> with
> > > too
> > > > many details. If you have time, please provide some feedback, and I
> > will
> > > > get more features into the Dynamic Router EIP.
> > > >
> > > > Thanks,
> > > > Steve
> > > >
> > >
> >
>
Re: Request for input re: Dynamic Router EIP component
Posted by ski n <ra...@gmail.com>.
Yeah, that makes it clearer.
Some remarks:
1) How does the channel work? Does it work only within the CamelContext
(like SEDA), in the whole virtual machine (like VM) or can it be used
distributed (like JMS)?
2) Wouldn't be easier if you separate it in two components: "publish" and
"subscribe" and then do something like:
from("direct:a")
.to("publish:mychannel");
from("subcribe:mychannel?id=1&priority=1&predicate=<text predicate>")
.to("someUri");
from("subcribe:mychannel?id=2&priority=2predicate=<text predicate>")
.to("someUri");
from("subcribe:mychannel?id=3&priority=3&predicate=<text predicate>")
.to("someUri");
3) In the documentation maybe could to add an example with subscribe while
using different expression languages/predicates.
Raymond
On Sat, Nov 5, 2022 at 7:01 PM Steve973 <st...@gmail.com> wrote:
> Hi, Raymond, and thank you so much for your response! I appreciate it, and
> I appreciate that you included those two links that I probably should have
> included.
>
> I will do my best to answer your questions, but since I am not the best
> teacher, please let me know where I need to explain better:
>
> 1. The best way I would generally describe the use case for this EIP
> component is when you want dynamic orchestration between modules of your
> system:
> a. You have services that come and go over time, possibly because your
> system watches for something to occur, and then it spins up services
> to handle whatever follows. This allows them to plug into routing,
> perform
> their tasks, and then unplug from routing when they are going to stop
> and
> go away.
> b. You have a way to programmatically enable and disable features of
> your system. Enable them, and they subscribe for routing. Disable
> them,
> and they unsubscribe.
> c. You want a way to decentralize your routing logic and keep
> module-specific routing in the applicable module. The router does not
> have
> to know the system's list of commands, or how to handle any of them
> prior
> to runtime.
> 2. Your understanding of the channels is correct. This allows for
> logical separation of routing. One channel has one set of subscribers
> that
> is separate from any other channel that has its own (separate) set of
> subscribers.
> 3. You can use an object for subscribing or unsubscribing, and you can
> also use the URI (see
>
> https://camel.apache.org/components/3.18.x/dynamic-router-component.html#_method_2_uri_only
> to see what I mean). Use of the component in a route is as simple as
> from("<some uri>").to("dynamic-router:<channel name>") so, at least in
> that
> part, you don't have to worry about any java code, since using the
> dynamic
> router component is just like using any other camel component. When you
> are subscribing/unsubscribing is where using code might apply, to create
> the subscription message and then send it, for example, by a producer
> template.
>
> Please let me know if this helps. I am always open to expanding its
> usefulness, or simplifying its use. So, if you have any suggestions, I
> would be happy to see how I could get it integrated into the component.
>
> Thanks again,
> Steve
>
> On Sat, Nov 5, 2022 at 11:44 AM ski n <ra...@gmail.com> wrote:
>
> > For those interested. Here are some more resources:
> >
> > Background:
> >
> > https://camel.apache.org/blog/2022/01/dynamic-router-eip-component/
> >
> > Documentation:
> >
> > https://camel.apache.org/components/3.18.x/dynamic-router-component.html
> >
> > -------------------
> >
> > For me, reading through the documentation, it seems like a powerful
> > component. However, there are some things I am failing to grasp:
> >
> > 1. What are the use cases this can be used for. And with kind of use
> cases
> > I mean business use cases explained in simple terms for end-users.
> > 2. As I understand it, the dynamic router provides 'topics' similar to
> > Kafka topics or JMS topics, but then build into Camel. Is this correct?
> > 3. In the documentation examples I see a lot of plain Java code. Things
> > like unsubscribing:
> >
> > // Send message to the Dynamic Router "billing" channel// to
> > unsubscribe from billing messagesDynamicRouterControlMessage
> > unsubscribeMsg = new UnsubscribeMessageBuilder()
> > .id("billingSubscription")
> > .channel("billing")
> > .build();
> > template.sendBody("dynamic-router:control", unsubscribeMsg);
> >
> > I try to use the Route DSL's as much as possible. (especially XML and
> YAML,
> > because they are easily to manipulate). How can I use the component
> within
> > regular routes without things like Predicates and other Java code?
> >
> > Kind regards,
> >
> > Raymond
> >
> >
> >
> >
> >
> >
> >
> >
> >
> > On Sat, Nov 5, 2022 at 4:06 PM Steve973 <st...@gmail.com> wrote:
> >
> > > Hello. I introduced a Dynamic Router EIP component in 3.15 because I
> had
> > > uses for a dynamic router that was a bit different than what the core
> > > component offers. I would like some feedback about what features to
> > > potentially add that would make it even more useful. First, just in
> case
> > > you are not familiar with its capabilities, I will describe how I am
> > using
> > > it in a software system that I am building at work, which exercises it
> > > fairly well. We have a little fewer than 10 spring boot modules that
> > > handle specific tasks of the system. One of the modules is a "routing
> > > engine" that is mostly a wrapper for Camel, along with some
> > > business-logic-specific integration code. This is where I instantiate
> > the
> > > Dynamic Router component, and create a JMS route that can feed it
> > > subscriber messages. The other modules register their subscriptions
> with
> > > the dynamic router by sending messages over JMS to the router. These
> > > messages contain a predicate that allows the dynamic router to evaluate
> > > messages that pass through it to determine one, or more, recipients.
> > With
> > > their subscription messages, they also include a destination URI for
> > > applicable messages to be routed to. This allows for a somewhat
> advanced
> > > version of the command pattern in my system.
> > >
> > > Can any of you think of what this is missing? My first implementation
> > used
> > > a list for keeping track of subscribers, but that allowed a module to
> > > register its commands multiple times. So, that has been changed to a
> map
> > > that maps by subscriber ID. The other thing, and I have not developed
> a
> > > solution for this, but in the event of an error, the "routing table" of
> > > routes and endpoints is not available that I know of. I am not sure
> how
> > to
> > > handle that, or if there is something built into Camel that might
> > > facilitate that type of tracing/debugging information, of it I might
> need
> > > to keep track of it in a table within the component.
> > >
> > > Anyway, this is already long enough, and I don't want to bore you with
> > too
> > > many details. If you have time, please provide some feedback, and I
> will
> > > get more features into the Dynamic Router EIP.
> > >
> > > Thanks,
> > > Steve
> > >
> >
>
Re: Request for input re: Dynamic Router EIP component
Posted by Steve973 <st...@gmail.com>.
Hi, Raymond, and thank you so much for your response! I appreciate it, and
I appreciate that you included those two links that I probably should have
included.
I will do my best to answer your questions, but since I am not the best
teacher, please let me know where I need to explain better:
1. The best way I would generally describe the use case for this EIP
component is when you want dynamic orchestration between modules of your
system:
a. You have services that come and go over time, possibly because your
system watches for something to occur, and then it spins up services
to handle whatever follows. This allows them to plug into routing, perform
their tasks, and then unplug from routing when they are going to stop and
go away.
b. You have a way to programmatically enable and disable features of
your system. Enable them, and they subscribe for routing. Disable them,
and they unsubscribe.
c. You want a way to decentralize your routing logic and keep
module-specific routing in the applicable module. The router does not have
to know the system's list of commands, or how to handle any of them prior
to runtime.
2. Your understanding of the channels is correct. This allows for
logical separation of routing. One channel has one set of subscribers that
is separate from any other channel that has its own (separate) set of
subscribers.
3. You can use an object for subscribing or unsubscribing, and you can
also use the URI (see
https://camel.apache.org/components/3.18.x/dynamic-router-component.html#_method_2_uri_only
to see what I mean). Use of the component in a route is as simple as
from("<some uri>").to("dynamic-router:<channel name>") so, at least in that
part, you don't have to worry about any java code, since using the dynamic
router component is just like using any other camel component. When you
are subscribing/unsubscribing is where using code might apply, to create
the subscription message and then send it, for example, by a producer
template.
Please let me know if this helps. I am always open to expanding its
usefulness, or simplifying its use. So, if you have any suggestions, I
would be happy to see how I could get it integrated into the component.
Thanks again,
Steve
On Sat, Nov 5, 2022 at 11:44 AM ski n <ra...@gmail.com> wrote:
> For those interested. Here are some more resources:
>
> Background:
>
> https://camel.apache.org/blog/2022/01/dynamic-router-eip-component/
>
> Documentation:
>
> https://camel.apache.org/components/3.18.x/dynamic-router-component.html
>
> -------------------
>
> For me, reading through the documentation, it seems like a powerful
> component. However, there are some things I am failing to grasp:
>
> 1. What are the use cases this can be used for. And with kind of use cases
> I mean business use cases explained in simple terms for end-users.
> 2. As I understand it, the dynamic router provides 'topics' similar to
> Kafka topics or JMS topics, but then build into Camel. Is this correct?
> 3. In the documentation examples I see a lot of plain Java code. Things
> like unsubscribing:
>
> // Send message to the Dynamic Router "billing" channel// to
> unsubscribe from billing messagesDynamicRouterControlMessage
> unsubscribeMsg = new UnsubscribeMessageBuilder()
> .id("billingSubscription")
> .channel("billing")
> .build();
> template.sendBody("dynamic-router:control", unsubscribeMsg);
>
> I try to use the Route DSL's as much as possible. (especially XML and YAML,
> because they are easily to manipulate). How can I use the component within
> regular routes without things like Predicates and other Java code?
>
> Kind regards,
>
> Raymond
>
>
>
>
>
>
>
>
>
> On Sat, Nov 5, 2022 at 4:06 PM Steve973 <st...@gmail.com> wrote:
>
> > Hello. I introduced a Dynamic Router EIP component in 3.15 because I had
> > uses for a dynamic router that was a bit different than what the core
> > component offers. I would like some feedback about what features to
> > potentially add that would make it even more useful. First, just in case
> > you are not familiar with its capabilities, I will describe how I am
> using
> > it in a software system that I am building at work, which exercises it
> > fairly well. We have a little fewer than 10 spring boot modules that
> > handle specific tasks of the system. One of the modules is a "routing
> > engine" that is mostly a wrapper for Camel, along with some
> > business-logic-specific integration code. This is where I instantiate
> the
> > Dynamic Router component, and create a JMS route that can feed it
> > subscriber messages. The other modules register their subscriptions with
> > the dynamic router by sending messages over JMS to the router. These
> > messages contain a predicate that allows the dynamic router to evaluate
> > messages that pass through it to determine one, or more, recipients.
> With
> > their subscription messages, they also include a destination URI for
> > applicable messages to be routed to. This allows for a somewhat advanced
> > version of the command pattern in my system.
> >
> > Can any of you think of what this is missing? My first implementation
> used
> > a list for keeping track of subscribers, but that allowed a module to
> > register its commands multiple times. So, that has been changed to a map
> > that maps by subscriber ID. The other thing, and I have not developed a
> > solution for this, but in the event of an error, the "routing table" of
> > routes and endpoints is not available that I know of. I am not sure how
> to
> > handle that, or if there is something built into Camel that might
> > facilitate that type of tracing/debugging information, of it I might need
> > to keep track of it in a table within the component.
> >
> > Anyway, this is already long enough, and I don't want to bore you with
> too
> > many details. If you have time, please provide some feedback, and I will
> > get more features into the Dynamic Router EIP.
> >
> > Thanks,
> > Steve
> >
>
Re: Request for input re: Dynamic Router EIP component
Posted by ski n <ra...@gmail.com>.
For those interested. Here are some more resources:
Background:
https://camel.apache.org/blog/2022/01/dynamic-router-eip-component/
Documentation:
https://camel.apache.org/components/3.18.x/dynamic-router-component.html
-------------------
For me, reading through the documentation, it seems like a powerful
component. However, there are some things I am failing to grasp:
1. What are the use cases this can be used for. And with kind of use cases
I mean business use cases explained in simple terms for end-users.
2. As I understand it, the dynamic router provides 'topics' similar to
Kafka topics or JMS topics, but then build into Camel. Is this correct?
3. In the documentation examples I see a lot of plain Java code. Things
like unsubscribing:
// Send message to the Dynamic Router "billing" channel// to
unsubscribe from billing messagesDynamicRouterControlMessage
unsubscribeMsg = new UnsubscribeMessageBuilder()
.id("billingSubscription")
.channel("billing")
.build();
template.sendBody("dynamic-router:control", unsubscribeMsg);
I try to use the Route DSL's as much as possible. (especially XML and YAML,
because they are easily to manipulate). How can I use the component within
regular routes without things like Predicates and other Java code?
Kind regards,
Raymond
On Sat, Nov 5, 2022 at 4:06 PM Steve973 <st...@gmail.com> wrote:
> Hello. I introduced a Dynamic Router EIP component in 3.15 because I had
> uses for a dynamic router that was a bit different than what the core
> component offers. I would like some feedback about what features to
> potentially add that would make it even more useful. First, just in case
> you are not familiar with its capabilities, I will describe how I am using
> it in a software system that I am building at work, which exercises it
> fairly well. We have a little fewer than 10 spring boot modules that
> handle specific tasks of the system. One of the modules is a "routing
> engine" that is mostly a wrapper for Camel, along with some
> business-logic-specific integration code. This is where I instantiate the
> Dynamic Router component, and create a JMS route that can feed it
> subscriber messages. The other modules register their subscriptions with
> the dynamic router by sending messages over JMS to the router. These
> messages contain a predicate that allows the dynamic router to evaluate
> messages that pass through it to determine one, or more, recipients. With
> their subscription messages, they also include a destination URI for
> applicable messages to be routed to. This allows for a somewhat advanced
> version of the command pattern in my system.
>
> Can any of you think of what this is missing? My first implementation used
> a list for keeping track of subscribers, but that allowed a module to
> register its commands multiple times. So, that has been changed to a map
> that maps by subscriber ID. The other thing, and I have not developed a
> solution for this, but in the event of an error, the "routing table" of
> routes and endpoints is not available that I know of. I am not sure how to
> handle that, or if there is something built into Camel that might
> facilitate that type of tracing/debugging information, of it I might need
> to keep track of it in a table within the component.
>
> Anyway, this is already long enough, and I don't want to bore you with too
> many details. If you have time, please provide some feedback, and I will
> get more features into the Dynamic Router EIP.
>
> Thanks,
> Steve
>