Re: Adding something new into Camel DSL -- but how?

[Steve973 <st...@gmail.com>]

Hi, Claus, and thanks for the reply.  I created the ticket on 31 Oct:
https://issues.apache.org/jira/projects/CAMEL/issues/CAMEL-17154 and I am
working on it in the evenings as time allows.  I have finished the initial
implementation, and I am working on tests and the asciidoc stuff.  So far,
I have gotten through at least half of the happy-path unit tests, and I
have some questions about aspects of the asciidoc format and content,
mostly related to the links that I see in some of the other adoc files that
I am using as examples.  I just added my notes about camel-core, and I
added a comment with some of my design and implementation ideas.  Any of my
questions can wait until the 3.14 LTS release is complete.

Thanks again,
Steve

On Tue, Nov 30, 2021 at 3:28 PM Claus Ibsen <cl...@gmail.com> wrote:

> Hi Steven
>
>
> On Sun, Nov 28, 2021 at 5:01 PM Steve973 <st...@gmail.com> wrote:
> >
> > Hi, Claus.  I have done what I hope was a fairly full tracing through the
> > code of camel-core, and I can now say that I appreciate your concern
> about
> > adding things to core!  I also took a closer look at the Saga and Kamelet
> > components, and that helped to correct a misconception that I had --
> > components are not solely for implementing things like transport
> > implementations into Camel, and that it is (arguably) a more suitable
> > mechanism for adding anything that does not have a strict and solid
> reason
> > for existing in core.  So I will implement this EIP as a component, and
> not
> > inside of core.  I created a markdown file with my notes when I traced
> > through core.  If you, or anyone else, might be interested, I can attach
> it
> > to a reply here.
> >
>
> Thanks for your patience and impressive to hear that you dived into
> the deep source code and
> found the good, bad, "and ugly" of our huge code base.
>
> One "bad" about EIPs is that the model is defined tightly into the
> same JAR which means
> that adding new EIPs is harder - or that users can't develop custom
> EIPs and plugin as they need.
>
> But on the other side then everybody of Camel has the same DSL, and
> therefore its "standard" and familiar.
>
> About your new dynamic eip, then did you create a JIRA for it? If not
> you are welcome to do so.
> And then attach the notes in the JIRA so its not lost, and we have it
> tracked for everybody to see,
> and also easier find in years to come.
>
> I am a bit busy this week with Red Hat virtual company event the
> entire week. And on the same time I must finish some work on health
> checks.
> However I should get some "clear timer" later in December.
>
> And we need Camel 3.14 LTS released first also. So have some patience,
> and then we have better clear road for 3.15 onwards to add new EIPs.
>
>
> > Thanks,
> > Steve
> >
> > On Wed, Nov 24, 2021 at 10:20 AM Claus Ibsen <cl...@gmail.com>
> wrote:
> >
> > > Hi
> > >
> > > Adding EIPs or changes to the DSL is not a Camel end user task, and is
> > > not documented thoroughly.
> > > Adding new EIPs happens more rarely and there is maybe a little
> > > hesitation to add because when its in the DSL it stays for a long
> > > time.
> > > Also the DSL can become bloated - there are some mistakes from Camel
> > > v1 and v2 that we have today.
> > >
> > > That said to add a new EIP there are 2 ways
> > >
> > > - The EIP is implemented fully in the core, so all its code are in
> core.
> > > - The EIP is implemeted in a component and are located in camel-xxx in
> > > the components folder. Examples are circuit breakers, saga, kamelet
> > > and maybe others I forgot.
> > >
> > > So first you need to find out if its the former or the latter. However
> > > for both cases, the the DSL model are in the core and there is a fair
> > > bit of code to write to add a new model.
> > > Therefore its a good idea to find a simple EIP and see where its used
> > > in the code - can be ConvertBodyDefinition (are in core). Or if its
> > > outside then maybe KameletDefinition which is the latest EIP added.
> > >
> > > The reifer is the "guy" that sits between the model (DSL) and the
> > > processor (implementation), so all 3 are separated in their own JARs.
> > >
> > > And on top of that then there is also the new YAML DSL, so adding a
> > > new EIP can require some coding there too, but that can be done
> > > afterwards.
> > > At first is to get the regular Java and XML DSLs to work. The XML DSL
> > > is source generated with a maven plugin, and it can "have weird
> > > problems" if the DSL model is wrong.
> > > So if you have a very complex EIP then it can be tricky sometimes.
> > >
> > >
> > >
> > > On Mon, Nov 22, 2021 at 10:16 PM Steve973 <st...@gmail.com> wrote:
> > > >
> > > > Hi, all.  I am working on a new version of the Dynamic Router
> processor
> > > > that implements the pattern spec much more closely, and I have the
> > > initial
> > > > implementation complete, but I want to test it (among other ways) by
> > > using
> > > > it in the DSL.  I haven't seen any documentation on how to do this,
> or
> > > any
> > > > tutorials.  Have I missed something?  Could anyone give some kind of
> > > > overview?  I could probably use a different processor and trace up
> > > through
> > > > the classes to see what all is involved, but I don't want to miss
> > > anything,
> > > > and this would be great documentation to have.  If it would be
> helpful
> > > for
> > > > other contributors, I would not mind putting it together for
> inclusion in
> > > > the overall documentation.
> > > >
> > > > Thanks,
> > > > Steve
> > >
> > >
> > >
> > > --
> > > Claus Ibsen
> > > -----------------
> > > http://davsclaus.com @davsclaus
> > > Camel in Action 2: https://www.manning.com/ibsen2
> > >
>
>
>
> --
> Claus Ibsen
> -----------------
> http://davsclaus.com @davsclaus
> Camel in Action 2: https://www.manning.com/ibsen2
>

Re: Adding something new into Camel DSL -- but how?

[Claus Ibsen <cl...@gmail.com>]

Hi Steven

Good to know. Sorry I am too focused on 3.14 LTS release, a few last
things to clear up.

About ascii doc links, then they can be tricky. There has been some
improvements/updates to the website build and whatnot,
so its a bit in "flux" at the moment. So if you need to continue then
do not add links with xref yet, but add a TODO etc in the doc then we
can look at that later.

On Sat, Dec 4, 2021 at 4:51 PM Steve973 <st...@gmail.com> wrote:
>
> Hi, Claus, and thanks for the reply.  I created the ticket on 31 Oct:
> https://issues.apache.org/jira/projects/CAMEL/issues/CAMEL-17154 and I am
> working on it in the evenings as time allows.  I have finished the initial
> implementation, and I am working on tests and the asciidoc stuff.  So far,
> I have gotten through at least half of the happy-path unit tests, and I
> have some questions about aspects of the asciidoc format and content,
> mostly related to the links that I see in some of the other adoc files that
> I am using as examples.  I just added my notes about camel-core, and I
> added a comment with some of my design and implementation ideas.  Any of my
> questions can wait until the 3.14 LTS release is complete.
>
> Thanks again,
> Steve
>
> On Tue, Nov 30, 2021 at 3:28 PM Claus Ibsen <cl...@gmail.com> wrote:
>
> > Hi Steven
> >
> >
> > On Sun, Nov 28, 2021 at 5:01 PM Steve973 <st...@gmail.com> wrote:
> > >
> > > Hi, Claus.  I have done what I hope was a fairly full tracing through the
> > > code of camel-core, and I can now say that I appreciate your concern
> > about
> > > adding things to core!  I also took a closer look at the Saga and Kamelet
> > > components, and that helped to correct a misconception that I had --
> > > components are not solely for implementing things like transport
> > > implementations into Camel, and that it is (arguably) a more suitable
> > > mechanism for adding anything that does not have a strict and solid
> > reason
> > > for existing in core.  So I will implement this EIP as a component, and
> > not
> > > inside of core.  I created a markdown file with my notes when I traced
> > > through core.  If you, or anyone else, might be interested, I can attach
> > it
> > > to a reply here.
> > >
> >
> > Thanks for your patience and impressive to hear that you dived into
> > the deep source code and
> > found the good, bad, "and ugly" of our huge code base.
> >
> > One "bad" about EIPs is that the model is defined tightly into the
> > same JAR which means
> > that adding new EIPs is harder - or that users can't develop custom
> > EIPs and plugin as they need.
> >
> > But on the other side then everybody of Camel has the same DSL, and
> > therefore its "standard" and familiar.
> >
> > About your new dynamic eip, then did you create a JIRA for it? If not
> > you are welcome to do so.
> > And then attach the notes in the JIRA so its not lost, and we have it
> > tracked for everybody to see,
> > and also easier find in years to come.
> >
> > I am a bit busy this week with Red Hat virtual company event the
> > entire week. And on the same time I must finish some work on health
> > checks.
> > However I should get some "clear timer" later in December.
> >
> > And we need Camel 3.14 LTS released first also. So have some patience,
> > and then we have better clear road for 3.15 onwards to add new EIPs.
> >
> >
> > > Thanks,
> > > Steve
> > >
> > > On Wed, Nov 24, 2021 at 10:20 AM Claus Ibsen <cl...@gmail.com>
> > wrote:
> > >
> > > > Hi
> > > >
> > > > Adding EIPs or changes to the DSL is not a Camel end user task, and is
> > > > not documented thoroughly.
> > > > Adding new EIPs happens more rarely and there is maybe a little
> > > > hesitation to add because when its in the DSL it stays for a long
> > > > time.
> > > > Also the DSL can become bloated - there are some mistakes from Camel
> > > > v1 and v2 that we have today.
> > > >
> > > > That said to add a new EIP there are 2 ways
> > > >
> > > > - The EIP is implemented fully in the core, so all its code are in
> > core.
> > > > - The EIP is implemeted in a component and are located in camel-xxx in
> > > > the components folder. Examples are circuit breakers, saga, kamelet
> > > > and maybe others I forgot.
> > > >
> > > > So first you need to find out if its the former or the latter. However
> > > > for both cases, the the DSL model are in the core and there is a fair
> > > > bit of code to write to add a new model.
> > > > Therefore its a good idea to find a simple EIP and see where its used
> > > > in the code - can be ConvertBodyDefinition (are in core). Or if its
> > > > outside then maybe KameletDefinition which is the latest EIP added.
> > > >
> > > > The reifer is the "guy" that sits between the model (DSL) and the
> > > > processor (implementation), so all 3 are separated in their own JARs.
> > > >
> > > > And on top of that then there is also the new YAML DSL, so adding a
> > > > new EIP can require some coding there too, but that can be done
> > > > afterwards.
> > > > At first is to get the regular Java and XML DSLs to work. The XML DSL
> > > > is source generated with a maven plugin, and it can "have weird
> > > > problems" if the DSL model is wrong.
> > > > So if you have a very complex EIP then it can be tricky sometimes.
> > > >
> > > >
> > > >
> > > > On Mon, Nov 22, 2021 at 10:16 PM Steve973 <st...@gmail.com> wrote:
> > > > >
> > > > > Hi, all.  I am working on a new version of the Dynamic Router
> > processor
> > > > > that implements the pattern spec much more closely, and I have the
> > > > initial
> > > > > implementation complete, but I want to test it (among other ways) by
> > > > using
> > > > > it in the DSL.  I haven't seen any documentation on how to do this,
> > or
> > > > any
> > > > > tutorials.  Have I missed something?  Could anyone give some kind of
> > > > > overview?  I could probably use a different processor and trace up
> > > > through
> > > > > the classes to see what all is involved, but I don't want to miss
> > > > anything,
> > > > > and this would be great documentation to have.  If it would be
> > helpful
> > > > for
> > > > > other contributors, I would not mind putting it together for
> > inclusion in
> > > > > the overall documentation.
> > > > >
> > > > > Thanks,
> > > > > Steve
> > > >
> > > >
> > > >
> > > > --
> > > > Claus Ibsen
> > > > -----------------
> > > > http://davsclaus.com @davsclaus
> > > > Camel in Action 2: https://www.manning.com/ibsen2
> > > >
> >
> >
> >
> > --
> > Claus Ibsen
> > -----------------
> > http://davsclaus.com @davsclaus
> > Camel in Action 2: https://www.manning.com/ibsen2
> >



-- 
Claus Ibsen
-----------------
http://davsclaus.com @davsclaus
Camel in Action 2: https://www.manning.com/ibsen2