You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@camel.apache.org by Andrea <an...@tarocch.it> on 2021/09/18 20:52:39 UTC
[HEADS-UP][DISCUSS] migrating Camel Kafka Connector to use kamelets
Hello,
finally the switch to use kamelets in Camel Kafka Connector project is in an usable state:
https://github.com/apache/camel-kafka-connector/tree/kamelets
this branch has refactored the existing core in reusable kamelets and has added generation of kafka connectors using the kamelet catalog as well. The generation happens using the following criteria:
- one kafka connector is generated for each kamelet in the kamelet catalog.
- one kafka connector is generated for each camel component ("the old way"), excluding the component used in at least one kamelet.
Going forward, as the kamelet catalog will add more kamelets, I personally prefer to further reduce the number of kafka connectors generated "the old way" (most probably selecting just the most meaningful/used/that makes sense).
These would made the project more maintainable by reducing number of connectors and improving their quality by reusing kamelets that are the preferred way in camel to define "connectors" and are used/will be used across all runtimes.
I would like to give everyone interested some time to look at the work and think about improvements/concerns before merging it to main and shortly after releasing a new version of Camel Kafka Connector.
Regards,
Andrea.
Re: [HEADS-UP][DISCUSS] migrating Camel Kafka Connector to use kamelets
Posted by Otavio Rodolfo Piske <an...@gmail.com>.
Hi Andrea, that looks great!
I am looking forward to seeing this on the main tree. I think the idea of
releasing 0.11 based on Camel 3.11 is great and should give some room for
the users to migrate.
I hope the tests didn't give too much of a headache to convert o.O
Kind regards
On Mon, Sep 20, 2021 at 2:34 PM Luca Burgazzoli <lb...@gmail.com>
wrote:
> On Mon, Sep 20, 2021 at 9:12 AM Claus Ibsen <cl...@gmail.com> wrote:
>
> >
> > If a old connector is needed we could ship the maven plugin so CKC
> > users can generate a custom connector themseleves as "fallback"
> >
> >
> Wonder if the generation of a camel kafka connector could also be a target
> for https://github.com/apache/camel-kameleon
>
--
Otavio R. Piske
http://orpiske.net
Re: [HEADS-UP][DISCUSS] migrating Camel Kafka Connector to use kamelets
Posted by Andrea <an...@tarocch.it>.
AH!
Interesting, yes with some minor changes into the plugins potentially we could give the option to generate a connector "the old way" through kameleon.
Should I open an issue to that subproject to keep track of this idea?
On Mon, Sep 20, 2021, at 14:33, Luca Burgazzoli wrote:
> On Mon, Sep 20, 2021 at 9:12 AM Claus Ibsen <cl...@gmail.com> wrote:
>
> >
> > If a old connector is needed we could ship the maven plugin so CKC
> > users can generate a custom connector themseleves as "fallback"
> >
> >
> Wonder if the generation of a camel kafka connector could also be a target
> for https://github.com/apache/camel-kameleon
>
Re: [HEADS-UP][DISCUSS] migrating Camel Kafka Connector to use kamelets
Posted by Luca Burgazzoli <lb...@gmail.com>.
On Mon, Sep 20, 2021 at 9:12 AM Claus Ibsen <cl...@gmail.com> wrote:
>
> If a old connector is needed we could ship the maven plugin so CKC
> users can generate a custom connector themseleves as "fallback"
>
>
Wonder if the generation of a camel kafka connector could also be a target
for https://github.com/apache/camel-kameleon
Re: [HEADS-UP][DISCUSS] migrating Camel Kafka Connector to use kamelets
Posted by Zineb Bendhiba <be...@gmail.com>.
Hello Andrea,
I like this idea.
Zineb
Le lun. 20 sept. 2021 à 09:57, Claus Ibsen <cl...@gmail.com> a écrit :
> On Mon, Sep 20, 2021 at 9:53 AM Andrea <an...@tarocch.it> wrote:
> >
> > Oh if I understand correctly you idea:
> >
> > - release ckc 0.11.0 based on camel 3.11 generating connectors the old
> way
> > - merge kamelets into main and generate connectors just from kamelets
> from 3.12 onward
> >
> > Is that what you have in mand? If yes you have my +1 :)
> >
>
> Yes correct
>
>
> > On Mon, Sep 20, 2021, at 09:12, Claus Ibsen wrote:
> > > Hi
> > >
> > > Great to see progress on CKC.
> > >
> > > Before doing the "kamelet" thingy. I wonder if we can get a 0.11.0
> > > release out that is just "the old way" but for the 3.11 LTS release.
> > > Then users have that stable Camel version to stay on.
> > >
> > > Then the kamalet work can be merged after that and go into releases
> > > for Camel 3.12 onwards.
> > >
> > > And yeah I agree that we should get to a point where its only
> > > generated from kamelets. Then that also ensures that users that more
> > > easily jump from CKC to Camel-K.
> > > We could also do a release without the old way as nobody needs all the
> > > 300 components in Camel. But then the CKC community can give feedback
> > > on connectors needed and we build kamelets for that.
> > >
> > > If a old connector is needed we could ship the maven plugin so CKC
> > > users can generate a custom connector themseleves as "fallback"
> > >
> > > On Sat, Sep 18, 2021 at 10:53 PM Andrea <an...@tarocch.it> wrote:
> > > >
> > > > Hello,
> > > >
> > > > finally the switch to use kamelets in Camel Kafka Connector project
> is in an usable state:
> > > >
> > > > https://github.com/apache/camel-kafka-connector/tree/kamelets
> > > >
> > > > this branch has refactored the existing core in reusable kamelets
> and has added generation of kafka connectors using the kamelet catalog as
> well. The generation happens using the following criteria:
> > > >
> > > > - one kafka connector is generated for each kamelet in the kamelet
> catalog.
> > > > - one kafka connector is generated for each camel component ("the
> old way"), excluding the component used in at least one kamelet.
> > > >
> > > > Going forward, as the kamelet catalog will add more kamelets, I
> personally prefer to further reduce the number of kafka connectors
> generated "the old way" (most probably selecting just the most
> meaningful/used/that makes sense).
> > > >
> > > > These would made the project more maintainable by reducing number of
> connectors and improving their quality by reusing kamelets that are the
> preferred way in camel to define "connectors" and are used/will be used
> across all runtimes.
> > > >
> > > > I would like to give everyone interested some time to look at the
> work and think about improvements/concerns before merging it to main and
> shortly after releasing a new version of Camel Kafka Connector.
> > > >
> > > > Regards,
> > > > Andrea.
> > >
> > >
> > >
> > > --
> > > Claus Ibsen
> > > -----------------
> > > http://davsclaus.com @davsclaus
> > > Camel in Action 2: https://www.manning.com/ibsen2
> > >
>
>
>
> --
> Claus Ibsen
> -----------------
> http://davsclaus.com @davsclaus
> Camel in Action 2: https://www.manning.com/ibsen2
>
--
Zineb
Re: [HEADS-UP][DISCUSS] migrating Camel Kafka Connector to use kamelets
Posted by Claus Ibsen <cl...@gmail.com>.
On Mon, Sep 20, 2021 at 9:53 AM Andrea <an...@tarocch.it> wrote:
>
> Oh if I understand correctly you idea:
>
> - release ckc 0.11.0 based on camel 3.11 generating connectors the old way
> - merge kamelets into main and generate connectors just from kamelets from 3.12 onward
>
> Is that what you have in mand? If yes you have my +1 :)
>
Yes correct
> On Mon, Sep 20, 2021, at 09:12, Claus Ibsen wrote:
> > Hi
> >
> > Great to see progress on CKC.
> >
> > Before doing the "kamelet" thingy. I wonder if we can get a 0.11.0
> > release out that is just "the old way" but for the 3.11 LTS release.
> > Then users have that stable Camel version to stay on.
> >
> > Then the kamalet work can be merged after that and go into releases
> > for Camel 3.12 onwards.
> >
> > And yeah I agree that we should get to a point where its only
> > generated from kamelets. Then that also ensures that users that more
> > easily jump from CKC to Camel-K.
> > We could also do a release without the old way as nobody needs all the
> > 300 components in Camel. But then the CKC community can give feedback
> > on connectors needed and we build kamelets for that.
> >
> > If a old connector is needed we could ship the maven plugin so CKC
> > users can generate a custom connector themseleves as "fallback"
> >
> > On Sat, Sep 18, 2021 at 10:53 PM Andrea <an...@tarocch.it> wrote:
> > >
> > > Hello,
> > >
> > > finally the switch to use kamelets in Camel Kafka Connector project is in an usable state:
> > >
> > > https://github.com/apache/camel-kafka-connector/tree/kamelets
> > >
> > > this branch has refactored the existing core in reusable kamelets and has added generation of kafka connectors using the kamelet catalog as well. The generation happens using the following criteria:
> > >
> > > - one kafka connector is generated for each kamelet in the kamelet catalog.
> > > - one kafka connector is generated for each camel component ("the old way"), excluding the component used in at least one kamelet.
> > >
> > > Going forward, as the kamelet catalog will add more kamelets, I personally prefer to further reduce the number of kafka connectors generated "the old way" (most probably selecting just the most meaningful/used/that makes sense).
> > >
> > > These would made the project more maintainable by reducing number of connectors and improving their quality by reusing kamelets that are the preferred way in camel to define "connectors" and are used/will be used across all runtimes.
> > >
> > > I would like to give everyone interested some time to look at the work and think about improvements/concerns before merging it to main and shortly after releasing a new version of Camel Kafka Connector.
> > >
> > > Regards,
> > > Andrea.
> >
> >
> >
> > --
> > Claus Ibsen
> > -----------------
> > http://davsclaus.com @davsclaus
> > Camel in Action 2: https://www.manning.com/ibsen2
> >
--
Claus Ibsen
-----------------
http://davsclaus.com @davsclaus
Camel in Action 2: https://www.manning.com/ibsen2
Re: [HEADS-UP][DISCUSS] migrating Camel Kafka Connector to use kamelets
Posted by Andrea <an...@tarocch.it>.
Oh if I understand correctly you idea:
- release ckc 0.11.0 based on camel 3.11 generating connectors the old way
- merge kamelets into main and generate connectors just from kamelets from 3.12 onward
Is that what you have in mand? If yes you have my +1 :)
On Mon, Sep 20, 2021, at 09:12, Claus Ibsen wrote:
> Hi
>
> Great to see progress on CKC.
>
> Before doing the "kamelet" thingy. I wonder if we can get a 0.11.0
> release out that is just "the old way" but for the 3.11 LTS release.
> Then users have that stable Camel version to stay on.
>
> Then the kamalet work can be merged after that and go into releases
> for Camel 3.12 onwards.
>
> And yeah I agree that we should get to a point where its only
> generated from kamelets. Then that also ensures that users that more
> easily jump from CKC to Camel-K.
> We could also do a release without the old way as nobody needs all the
> 300 components in Camel. But then the CKC community can give feedback
> on connectors needed and we build kamelets for that.
>
> If a old connector is needed we could ship the maven plugin so CKC
> users can generate a custom connector themseleves as "fallback"
>
> On Sat, Sep 18, 2021 at 10:53 PM Andrea <an...@tarocch.it> wrote:
> >
> > Hello,
> >
> > finally the switch to use kamelets in Camel Kafka Connector project is in an usable state:
> >
> > https://github.com/apache/camel-kafka-connector/tree/kamelets
> >
> > this branch has refactored the existing core in reusable kamelets and has added generation of kafka connectors using the kamelet catalog as well. The generation happens using the following criteria:
> >
> > - one kafka connector is generated for each kamelet in the kamelet catalog.
> > - one kafka connector is generated for each camel component ("the old way"), excluding the component used in at least one kamelet.
> >
> > Going forward, as the kamelet catalog will add more kamelets, I personally prefer to further reduce the number of kafka connectors generated "the old way" (most probably selecting just the most meaningful/used/that makes sense).
> >
> > These would made the project more maintainable by reducing number of connectors and improving their quality by reusing kamelets that are the preferred way in camel to define "connectors" and are used/will be used across all runtimes.
> >
> > I would like to give everyone interested some time to look at the work and think about improvements/concerns before merging it to main and shortly after releasing a new version of Camel Kafka Connector.
> >
> > Regards,
> > Andrea.
>
>
>
> --
> Claus Ibsen
> -----------------
> http://davsclaus.com @davsclaus
> Camel in Action 2: https://www.manning.com/ibsen2
>
Re: [HEADS-UP][DISCUSS] migrating Camel Kafka Connector to use kamelets
Posted by Claus Ibsen <cl...@gmail.com>.
Hi
Great to see progress on CKC.
Before doing the "kamelet" thingy. I wonder if we can get a 0.11.0
release out that is just "the old way" but for the 3.11 LTS release.
Then users have that stable Camel version to stay on.
Then the kamalet work can be merged after that and go into releases
for Camel 3.12 onwards.
And yeah I agree that we should get to a point where its only
generated from kamelets. Then that also ensures that users that more
easily jump from CKC to Camel-K.
We could also do a release without the old way as nobody needs all the
300 components in Camel. But then the CKC community can give feedback
on connectors needed and we build kamelets for that.
If a old connector is needed we could ship the maven plugin so CKC
users can generate a custom connector themseleves as "fallback"
On Sat, Sep 18, 2021 at 10:53 PM Andrea <an...@tarocch.it> wrote:
>
> Hello,
>
> finally the switch to use kamelets in Camel Kafka Connector project is in an usable state:
>
> https://github.com/apache/camel-kafka-connector/tree/kamelets
>
> this branch has refactored the existing core in reusable kamelets and has added generation of kafka connectors using the kamelet catalog as well. The generation happens using the following criteria:
>
> - one kafka connector is generated for each kamelet in the kamelet catalog.
> - one kafka connector is generated for each camel component ("the old way"), excluding the component used in at least one kamelet.
>
> Going forward, as the kamelet catalog will add more kamelets, I personally prefer to further reduce the number of kafka connectors generated "the old way" (most probably selecting just the most meaningful/used/that makes sense).
>
> These would made the project more maintainable by reducing number of connectors and improving their quality by reusing kamelets that are the preferred way in camel to define "connectors" and are used/will be used across all runtimes.
>
> I would like to give everyone interested some time to look at the work and think about improvements/concerns before merging it to main and shortly after releasing a new version of Camel Kafka Connector.
>
> Regards,
> Andrea.
--
Claus Ibsen
-----------------
http://davsclaus.com @davsclaus
Camel in Action 2: https://www.manning.com/ibsen2
Re: [HEADS-UP][DISCUSS] migrating Camel Kafka Connector to use
kamelets
Posted by Andrea Cosentino <an...@yahoo.com.INVALID>.
Hello,
It makes totally sense from my point of view.
I will be much more easier to maintain the project basing it on kamelets where available.
Thanks
--
Andrea Cosentino
----------------------------------
Apache Camel PMC Chair
Apache Karaf Committer
Apache Servicemix PMC Member
Email: ancosen1985@yahoo.com
Twitter: @oscerd2
Github: oscerd
On Saturday, September 18, 2021, 10:53:19 PM GMT+2, Andrea <an...@tarocch.it> wrote:
Hello,
finally the switch to use kamelets in Camel Kafka Connector project is in an usable state:
https://github.com/apache/camel-kafka-connector/tree/kamelets
this branch has refactored the existing core in reusable kamelets and has added generation of kafka connectors using the kamelet catalog as well. The generation happens using the following criteria:
- one kafka connector is generated for each kamelet in the kamelet catalog.
- one kafka connector is generated for each camel component ("the old way"), excluding the component used in at least one kamelet.
Going forward, as the kamelet catalog will add more kamelets, I personally prefer to further reduce the number of kafka connectors generated "the old way" (most probably selecting just the most meaningful/used/that makes sense).
These would made the project more maintainable by reducing number of connectors and improving their quality by reusing kamelets that are the preferred way in camel to define "connectors" and are used/will be used across all runtimes.
I would like to give everyone interested some time to look at the work and think about improvements/concerns before merging it to main and shortly after releasing a new version of Camel Kafka Connector.
Regards,
Andrea.