You are viewing a plain text version of this content. The canonical link for it is here.

Posted to dev@camel.apache.org by Kishan Gajjar <ki...@flipkart.com.INVALID> on 2020/05/14 09:27:07 UTC

Feature requirements on Camel

Hi All,

I have been evaluating Apache Camel for my Orchestration usecases. And have
realised that there are few features which are required to be built in
Camel to satisfy my requirements. I wanted to walk you through those
requirements and understand if it merges with Camel’s roadmap.

Below is the feature list we additionally require:

Route Management: Our usecase is to cater various kinds of orchestration
flows which needs to be executed on their trigger event. In Camel’s term we
need capability to capture the multiple routes which can be used as a
centralised place for managing them.
2.

Route Versioning: Any new change in the existing Route will be
rolled-out in the gradual fashion, which means there will be multiple
versions of a route which needs to be supported and executed in A/B manner.
3.

Data exchange control across the processor: Camel exchanges generic data
across the processors, which can lead to change of data in unexpected
processor boundaries. To restrict the data access boundary needs a
type-safe way of data sharing across the processors.
4.

Data driven routing. Camel provides the way to route control based on
static and dynamic across the processors. We need additional capability to
route processors based on the data dependency.
5.

Mapping with Json DSL: Mapping the route definition with JSON DSL. So
far Camel supports XML and java DSL for route definition.

Let me your thoughts on the requirements, which all make sense to be
added/extended with the Camel. We can work together to contribute.

--
Kishan Gajjar

*-----------------------------------------------------------------------------------------*

*This email and any files transmitted with it are confidential and
intended solely for the use of the individual or entity to whom they are
addressed. If you have received this email in error, please notify the
system manager. This message contains confidential information and is
intended only for the individual named. If you are not the named addressee,
you should not disseminate, distribute or copy this email. Please notify
the sender immediately by email if you have received this email by mistake
and delete this email from your system. If you are not the intended
recipient, you are notified that disclosing, copying, distributing or
taking any action in reliance on the contents of this information is
strictly prohibited.*****

****

*Any views or opinions presented in this
email are solely those of the author and do not necessarily represent those
of the organization. Any information on shares, debentures or similar
instruments, recommended product pricing, valuations and the like are for
information purposes only. It is not meant to be an instruction or
recommendation, as the case may be, to buy or to sell securities, products,
services nor an offer to buy or sell securities, products or services
unless specifically stated to be so on behalf of the Flipkart group.
Employees of the Flipkart group of companies are expressly required not to
make defamatory statements and not to infringe or authorise any
infringement of copyright or any other legal right by email communications.
Any such communication is contrary to organizational policy and outside the
scope of the employment of the individual concerned. The organization will
not accept any liability in respect of such communication, and the employee
responsible will be personally liable for any damages or other liability
arising.*****

****

*Our organization accepts no liability for the
content of this email, or for the consequences of any actions taken on the
basis of the information *provided,* unless that information is
subsequently confirmed in writing. If you are not the intended recipient,
you are notified that disclosing, copying, distributing or taking any
action in reliance on the contents of this information is strictly
prohibited.*

_-----------------------------------------------------------------------------------------_

Re: Feature requirements on Camel

Posted by Kishan Gajjar <ki...@flipkart.com.INVALID>.

Thank you Onder for detailed feedback, I will try to respond to your
queries.

camel can be thought to act (up to a certain point) similar to
some BPM tools. especially bullet 1 and 2 more BPM(ish)
 --- BPM tools are focused more on the business driven execution. Our
requirement is focused on microservice orchestration, that's one of the
reasons we were looking into the Camel approach. Adding more to 1/2 points:
we want a host a platform where different stakeholders define their
orchestration. Where the platform can also be flexible to update/rollout
the definition of the orchestration. That's where we thought of persisting
and managing the route definition.

3 4 sound reasonable. some are already in place depending on how&what you
use in/with camel.
it also IMHO needs a bit more concrete scenarios after a bit of trials
--- Will add a concrete scenario for 3: We want to guard-rail the data
being passed to each processor as per its functional scope. For eg: we have
an Order entity which consist if Items bought, Payment transactions and
Fulfillment details, where a Processor meant to derive Refund amount *should
consume only* Payment transactions so that it cannot modify anything in
Items and Fulfillments.
-- For 4th point: Data driven execution we have need where based on User's
provided data we decide the subsequent flow.

And for the 5th point: our requirement is to cater a Platform which
provides easy on-boarding to any new usecase(Route definition in this
case). Where JSON has better interactions, easy to create dynamically
compared to XML.

--
Kishan Gajjar


On Thu, May 14, 2020 at 8:50 PM Onder SEZGIN <on...@apache.org> wrote:

> Hi,
>
> thanks for the overview look of what you are trying to achieve.
> at the high level, some parts sound achievable with BPM tools.
> of course, camel can be thought to act (up to a certain point) similar to
> some BPM tools. especially bullet 1 and 2 more BPM(ish)
>
> 3 4 sound reasonable. some are already in place depending on how&what you
> use in/with camel.
> it also IMHO needs a bit more concrete scenarios after a bit of trials. 5
> is a choice. personally i don't see a reason of using JSON dsl as JSON is
> used for data/metadata representation.
> but if there is an effort, community would consider i believe, but unless
> there is a usecase, hard to find a reason to back it up implementing it.
>
> Cheers
> Onder
>
> On Thu, May 14, 2020 at 1:54 PM Kishan Gajjar
> <ki...@flipkart.com.invalid> wrote:
>
> > Hi All,
> >
> > I have been evaluating Apache Camel for my Orchestration usecases. And
> have
> > realised that there are few features which are required to be built in
> > Camel to satisfy my requirements. I wanted to walk you through those
> > requirements and understand if it merges with Camel’s roadmap.
> >
> > Below is the feature list we additionally require:
> >
> >    1.
> >
> >    Route Management: Our usecase is to cater various kinds of
> orchestration
> >    flows which needs to be executed on their trigger event. In Camel’s
> > term we
> >    need capability to capture the multiple routes which can be used as a
> >    centralised place for managing them.
> >    2.
> >
> >    Route Versioning: Any new change in the existing Route will be
> >    rolled-out in the gradual fashion, which means there will be multiple
> >    versions of a route which needs to be supported and executed in A/B
> > manner.
> >    3.
> >
> >    Data exchange control across the processor: Camel exchanges generic
> data
> >    across the processors, which can lead to change of data in unexpected
> >    processor boundaries. To restrict the data access boundary needs a
> >    type-safe way of data sharing across the processors.
> >    4.
> >
> >    Data driven routing. Camel provides the way to route control based on
> >    static and dynamic across the processors. We need additional
> capability
> > to
> >    route processors based on the data dependency.
> >    5.
> >
> >    Mapping with Json DSL: Mapping the route definition with JSON DSL. So
> >    far Camel supports XML and java DSL for route definition.
> >
> >
> > Let me your thoughts on the requirements, which all make sense to be
> > added/extended with the Camel. We can work together to contribute.
> >
> > --
> > Kishan Gajjar
> >
> > --
> >
> >
> >
> >
> >
> *-----------------------------------------------------------------------------------------*
> >
> >
> > *This email and any files transmitted with it are confidential and
> > intended solely for the use of the individual or entity to whom they are
> > addressed. If you have received this email in error, please notify the
> > system manager. This message contains confidential information and is
> > intended only for the individual named. If you are not the named
> > addressee,
> > you should not disseminate, distribute or copy this email. Please notify
> > the sender immediately by email if you have received this email by
> mistake
> > and delete this email from your system. If you are not the intended
> > recipient, you are notified that disclosing, copying, distributing or
> > taking any action in reliance on the contents of this information is
> > strictly prohibited.*****
> >
> >  ****
> >
> > *Any views or opinions presented in this
> > email are solely those of the author and do not necessarily represent
> > those
> > of the organization. Any information on shares, debentures or similar
> > instruments, recommended product pricing, valuations and the like are for
> > information purposes only. It is not meant to be an instruction or
> > recommendation, as the case may be, to buy or to sell securities,
> > products,
> > services nor an offer to buy or sell securities, products or services
> > unless specifically stated to be so on behalf of the Flipkart group.
> > Employees of the Flipkart group of companies are expressly required not
> to
> > make defamatory statements and not to infringe or authorise any
> > infringement of copyright or any other legal right by email
> > communications.
> > Any such communication is contrary to organizational policy and outside
> > the
> > scope of the employment of the individual concerned. The organization
> will
> > not accept any liability in respect of such communication, and the
> > employee
> > responsible will be personally liable for any damages or other liability
> > arising.*****
> >
> >  ****
> >
> > *Our organization accepts no liability for the
> > content of this email, or for the consequences of any actions taken on
> the
> > basis of the information *provided,* unless that information is
> > subsequently confirmed in writing. If you are not the intended recipient,
> > you are notified that disclosing, copying, distributing or taking any
> > action in reliance on the contents of this information is strictly
> > prohibited.*
> >
> >
> >
> >
> _-----------------------------------------------------------------------------------------_
> >
> >
>

-- 



*-----------------------------------------------------------------------------------------*


*This email and any files transmitted with it are confidential and 
intended solely for the use of the individual or entity to whom they are 
addressed. If you have received this email in error, please notify the 
system manager. This message contains confidential information and is 
intended only for the individual named. If you are not the named addressee, 
you should not disseminate, distribute or copy this email. Please notify 
the sender immediately by email if you have received this email by mistake 
and delete this email from your system. If you are not the intended 
recipient, you are notified that disclosing, copying, distributing or 
taking any action in reliance on the contents of this information is 
strictly prohibited.*****

 ****

*Any views or opinions presented in this 
email are solely those of the author and do not necessarily represent those 
of the organization. Any information on shares, debentures or similar 
instruments, recommended product pricing, valuations and the like are for 
information purposes only. It is not meant to be an instruction or 
recommendation, as the case may be, to buy or to sell securities, products, 
services nor an offer to buy or sell securities, products or services 
unless specifically stated to be so on behalf of the Flipkart group. 
Employees of the Flipkart group of companies are expressly required not to 
make defamatory statements and not to infringe or authorise any 
infringement of copyright or any other legal right by email communications. 
Any such communication is contrary to organizational policy and outside the 
scope of the employment of the individual concerned. The organization will 
not accept any liability in respect of such communication, and the employee 
responsible will be personally liable for any damages or other liability 
arising.*****

 ****

*Our organization accepts no liability for the 
content of this email, or for the consequences of any actions taken on the 
basis of the information *provided,* unless that information is 
subsequently confirmed in writing. If you are not the intended recipient, 
you are notified that disclosing, copying, distributing or taking any 
action in reliance on the contents of this information is strictly 
prohibited.*


_-----------------------------------------------------------------------------------------_

Re: Feature requirements on Camel

Posted by Onder SEZGIN <on...@apache.org>.

Hi,

thanks for the overview look of what you are trying to achieve.
at the high level, some parts sound achievable with BPM tools.
of course, camel can be thought to act (up to a certain point) similar to
some BPM tools. especially bullet 1 and 2 more BPM(ish)

3 4 sound reasonable. some are already in place depending on how&what you
use in/with camel.
it also IMHO needs a bit more concrete scenarios after a bit of trials. 5
is a choice. personally i don't see a reason of using JSON dsl as JSON is
used for data/metadata representation.
but if there is an effort, community would consider i believe, but unless
there is a usecase, hard to find a reason to back it up implementing it.

Cheers
Onder

On Thu, May 14, 2020 at 1:54 PM Kishan Gajjar
<ki...@flipkart.com.invalid> wrote:

> Hi All,
>
> I have been evaluating Apache Camel for my Orchestration usecases. And have
> realised that there are few features which are required to be built in
> Camel to satisfy my requirements. I wanted to walk you through those
> requirements and understand if it merges with Camel’s roadmap.
>
> Below is the feature list we additionally require:
>
>    1.
>
>    Route Management: Our usecase is to cater various kinds of orchestration
>    flows which needs to be executed on their trigger event. In Camel’s
> term we
>    need capability to capture the multiple routes which can be used as a
>    centralised place for managing them.
>    2.
>
>    Route Versioning: Any new change in the existing Route will be
>    rolled-out in the gradual fashion, which means there will be multiple
>    versions of a route which needs to be supported and executed in A/B
> manner.
>    3.
>
>    Data exchange control across the processor: Camel exchanges generic data
>    across the processors, which can lead to change of data in unexpected
>    processor boundaries. To restrict the data access boundary needs a
>    type-safe way of data sharing across the processors.
>    4.
>
>    Data driven routing. Camel provides the way to route control based on
>    static and dynamic across the processors. We need additional capability
> to
>    route processors based on the data dependency.
>    5.
>
>    Mapping with Json DSL: Mapping the route definition with JSON DSL. So
>    far Camel supports XML and java DSL for route definition.
>
>
> Let me your thoughts on the requirements, which all make sense to be
> added/extended with the Camel. We can work together to contribute.
>
> --
> Kishan Gajjar
>
> --
>
>
>
>
> *-----------------------------------------------------------------------------------------*
>
>
> *This email and any files transmitted with it are confidential and
> intended solely for the use of the individual or entity to whom they are
> addressed. If you have received this email in error, please notify the
> system manager. This message contains confidential information and is
> intended only for the individual named. If you are not the named
> addressee,
> you should not disseminate, distribute or copy this email. Please notify
> the sender immediately by email if you have received this email by mistake
> and delete this email from your system. If you are not the intended
> recipient, you are notified that disclosing, copying, distributing or
> taking any action in reliance on the contents of this information is
> strictly prohibited.*****
>
>  ****
>
> *Any views or opinions presented in this
> email are solely those of the author and do not necessarily represent
> those
> of the organization. Any information on shares, debentures or similar
> instruments, recommended product pricing, valuations and the like are for
> information purposes only. It is not meant to be an instruction or
> recommendation, as the case may be, to buy or to sell securities,
> products,
> services nor an offer to buy or sell securities, products or services
> unless specifically stated to be so on behalf of the Flipkart group.
> Employees of the Flipkart group of companies are expressly required not to
> make defamatory statements and not to infringe or authorise any
> infringement of copyright or any other legal right by email
> communications.
> Any such communication is contrary to organizational policy and outside
> the
> scope of the employment of the individual concerned. The organization will
> not accept any liability in respect of such communication, and the
> employee
> responsible will be personally liable for any damages or other liability
> arising.*****
>
>  ****
>
> *Our organization accepts no liability for the
> content of this email, or for the consequences of any actions taken on the
> basis of the information *provided,* unless that information is
> subsequently confirmed in writing. If you are not the intended recipient,
> you are notified that disclosing, copying, distributing or taking any
> action in reliance on the contents of this information is strictly
> prohibited.*
>
>
>
> _-----------------------------------------------------------------------------------------_
>
>

Fwd: Feature requirements on Camel

Posted by Kishan Gajjar <ki...@flipkart.com.INVALID>.

Hi All,

Below is the feature list we additionally require:

Data driven routing. Camel provides the way to route control based on
static and dynamic across the processors. We need additional capability to
route processors based on the data dependency.
5.

Mapping with Json DSL: Mapping the route definition with JSON DSL. So
far Camel supports XML and java DSL for route definition.

Let me your thoughts on the requirements, which all make sense to be
added/extended with the Camel. We can work together to contribute.

--
Kishan Gajjar

*-----------------------------------------------------------------------------------------*

****

_-----------------------------------------------------------------------------------------_

Re: Feature requirements on Camel

Posted by Zoran Regvart <zo...@regvart.com>.

Hi Kishan,
some thoughts inline

On Thu, May 14, 2020 at 12:58 PM Kishan Gajjar
<ki...@flipkart.com.invalid> wrote:
>    Route Management: Our usecase is to cater various kinds of orchestration
>    flows which needs to be executed on their trigger event. In Camel’s term we
>    need capability to capture the multiple routes which can be used as a
>    centralised place for managing them.

Are you aware of Hawt.io (https://hawt.io/) it's often used to provide
graphical UI to manage Camel. About your last remarks, I'm not exactly
clear what you mean by "capture", if you mean ability to run multiple
routes within one deployment -- you can have as many as you need
routes in a single Camel context, there are EIPs for routing and
splitting the payload if you need to pass them to other routes.

>    Route Versioning: Any new change in the existing Route will be
>    rolled-out in the gradual fashion, which means there will be multiple
>    versions of a route which needs to be supported and executed in A/B manner.

Your best option here is to version the artefact you deploy, if that's
a Camel route definition have two of them with different IDs. Naming
conventions come in handy.

>    Data exchange control across the processor: Camel exchanges generic data
>    across the processors, which can lead to change of data in unexpected
>    processor boundaries. To restrict the data access boundary needs a
>    type-safe way of data sharing across the processors.

If you need strong typing define a Java domain model defined and put
only objects from that domain model into the message body. If using a
Java model is not what you want you can enforce schema checks for many
common data formats.

>    Data driven routing. Camel provides the way to route control based on
>    static and dynamic across the processors. We need additional capability to
>    route processors based on the data dependency.

I'm not really clear on what you mean with data dependency, but have a
look at content based router EIP and see if that fits your use case.

>    Mapping with Json DSL: Mapping the route definition with JSON DSL. So
>    far Camel supports XML and java DSL for route definition.

I think there's an effort for a YAML based DSL, not sure if either
JSON or YAML is expressive enough to achieve fidelity you can have
with the XML and Java DSLs, the effort needed to put another DSL on
par with XML and Java DSLs is substantial and I (personally) don't see
much benefit in having one.

zoran
-- 
Zoran Regvart