You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@flink.apache.org by Etienne Chauchot <ec...@apache.org> on 2022/10/14 14:18:14 UTC
[Cassandra] source connector
Hi all,
As promised, I have developed the Cassandra source connector based on
FLIP-27. I've just submitted the
PR:https://github.com/apache/flink/pull/21073
Best
Etienne
Re: [Cassandra] source connector
Posted by Etienne Chauchot <ec...@apache.org>.
Hi all,
Would anyone have time to review the PR mentioned in this thread ?
Thanks in advance !
Best
Etienne
Le 25/10/2022 à 16:38, Etienne Chauchot a écrit :
> Ok thanks everyone.
>
> Etienne
>
> Le 24/10/2022 à 20:03, Martijn Visser a écrit :
>> Hi all,
>>
>> @Etienne many thanks for the PR for the Cassandra Source. Hopefully
>> we can
>> make this available after the 1.16 release.
>>
>> With regards to the connector externalization comments, while the
>> wiki for
>> the release plan for 1.17 is not available yet, the externalization of
>> connectors is definitely a big topic for the 1.17 release. I see no
>> harm in
>> already reviewing this PR and not necessarily merging it in the Flink
>> main
>> repo, but preparing for the dedicated Cassandra connector repository.
>> There
>> are dedicated threads on the externalization of connectors, if there are
>> questions or concerns then those should be discussed in those threads.
>>
>> Best regards,
>>
>> Martijn
>>
>> On Mon, Oct 24, 2022 at 6:28 PM Jing Ge <ji...@ververica.com> wrote:
>>
>>> Hi,
>>>
>>> The plan was and the ideal process is to try the externalizing
>>> connector
>>> development and release with the elastic connector first and make it
>>> stable
>>> before starting the migration of other connectors. There are already
>>> many
>>> connectors like Iceberg and AWS connectors that are trying
>>> externalizing.
>>> It is risky and might be out of control to start so many
>>> connectors externalizing in parallel without the proven stable process.
>>> Please be aware of the maintenance effort and consider a feasible
>>> timing to
>>> start the externalization.
>>>
>>> Best regards,
>>> Jing
>>>
>>> On Fri, Oct 21, 2022 at 4:14 PM Ryan Skraba
>>> <ry...@aiven.io.invalid>
>>> wrote:
>>>
>>>> There's definitely consensus on externalizing the flink
>>>> connectors! I've
>>>> been tracking the progress and I'd be happy to provide support on
>>> Cassandra
>>>> if you'd like.
>>>>
>>>> There's some new information at
>>>>
>>>>
>>> https://cwiki.apache.org/confluence/display/FLINK/Externalized+Connector+development
>>>
>>>> The first step to externalizing Cassandra would be for a PMC member to
>>>> create the flink-connector-cassandra repository for the project.
>>>>
>>>> If I understand correctly, this shouldn't block your PR, since the
>>>> core
>>>> connector (inside apache/flink) and the external one
>>>> (apache/flink-cassandra-connector) should be kept in sync for one
>>>> major
>>>> release cycle. For my own benefit, I can start by reviewing your PR!
>>>>
>>>> All my best, Ryan
>>>>
>>>> On Fri, Oct 21, 2022 at 11:51 AM Etienne Chauchot
>>>> <ec...@apache.org>
>>>> wrote:
>>>>
>>>>> Hi,
>>>>>
>>>>> Yes sure, if there is a consensus on moving, just tell me where to
>>>>> move
>>>>> my PR to.
>>>>>
>>>>> Best
>>>>>
>>>>> Etienne
>>>>>
>>>>> Le 19/10/2022 à 18:20, Alexander Fedulov a écrit :
>>>>>> Hi Etienne,
>>>>>>
>>>>>> thanks for your contribution. In light of the current efforts to
>>>>>> externalize connectors, do you think we could maybe combine the new
>>>>>> implementation with moving it into an external repository instead of
>>>>>> merging into Flink main?
>>>>>>
>>>>>> Best,
>>>>>> Alexander Fedulov
>>>>>>
>>>>>> On Fri, Oct 14, 2022 at 4:18 PM Etienne Chauchot <
>>> echauchot@apache.org
>>>>>> wrote:
>>>>>>
>>>>>>> Hi all,
>>>>>>>
>>>>>>> As promised, I have developed the Cassandra source connector based
>>> on
>>>>>>> FLIP-27. I've just submitted the
>>>>>>> PR:https://github.com/apache/flink/pull/21073
>>>>>>>
>>>>>>>
>>>>>>> Best
>>>>>>>
>>>>>>> Etienne
>>>>>>>
Re: [Cassandra] source connector
Posted by Etienne Chauchot <ec...@apache.org>.
Ok thanks everyone.
Etienne
Le 24/10/2022 à 20:03, Martijn Visser a écrit :
> Hi all,
>
> @Etienne many thanks for the PR for the Cassandra Source. Hopefully we can
> make this available after the 1.16 release.
>
> With regards to the connector externalization comments, while the wiki for
> the release plan for 1.17 is not available yet, the externalization of
> connectors is definitely a big topic for the 1.17 release. I see no harm in
> already reviewing this PR and not necessarily merging it in the Flink main
> repo, but preparing for the dedicated Cassandra connector repository. There
> are dedicated threads on the externalization of connectors, if there are
> questions or concerns then those should be discussed in those threads.
>
> Best regards,
>
> Martijn
>
> On Mon, Oct 24, 2022 at 6:28 PM Jing Ge <ji...@ververica.com> wrote:
>
>> Hi,
>>
>> The plan was and the ideal process is to try the externalizing connector
>> development and release with the elastic connector first and make it stable
>> before starting the migration of other connectors. There are already many
>> connectors like Iceberg and AWS connectors that are trying externalizing.
>> It is risky and might be out of control to start so many
>> connectors externalizing in parallel without the proven stable process.
>> Please be aware of the maintenance effort and consider a feasible timing to
>> start the externalization.
>>
>> Best regards,
>> Jing
>>
>> On Fri, Oct 21, 2022 at 4:14 PM Ryan Skraba <ry...@aiven.io.invalid>
>> wrote:
>>
>>> There's definitely consensus on externalizing the flink connectors! I've
>>> been tracking the progress and I'd be happy to provide support on
>> Cassandra
>>> if you'd like.
>>>
>>> There's some new information at
>>>
>>>
>> https://cwiki.apache.org/confluence/display/FLINK/Externalized+Connector+development
>>> The first step to externalizing Cassandra would be for a PMC member to
>>> create the flink-connector-cassandra repository for the project.
>>>
>>> If I understand correctly, this shouldn't block your PR, since the core
>>> connector (inside apache/flink) and the external one
>>> (apache/flink-cassandra-connector) should be kept in sync for one major
>>> release cycle. For my own benefit, I can start by reviewing your PR!
>>>
>>> All my best, Ryan
>>>
>>> On Fri, Oct 21, 2022 at 11:51 AM Etienne Chauchot <ec...@apache.org>
>>> wrote:
>>>
>>>> Hi,
>>>>
>>>> Yes sure, if there is a consensus on moving, just tell me where to move
>>>> my PR to.
>>>>
>>>> Best
>>>>
>>>> Etienne
>>>>
>>>> Le 19/10/2022 à 18:20, Alexander Fedulov a écrit :
>>>>> Hi Etienne,
>>>>>
>>>>> thanks for your contribution. In light of the current efforts to
>>>>> externalize connectors, do you think we could maybe combine the new
>>>>> implementation with moving it into an external repository instead of
>>>>> merging into Flink main?
>>>>>
>>>>> Best,
>>>>> Alexander Fedulov
>>>>>
>>>>> On Fri, Oct 14, 2022 at 4:18 PM Etienne Chauchot <
>> echauchot@apache.org
>>>>> wrote:
>>>>>
>>>>>> Hi all,
>>>>>>
>>>>>> As promised, I have developed the Cassandra source connector based
>> on
>>>>>> FLIP-27. I've just submitted the
>>>>>> PR:https://github.com/apache/flink/pull/21073
>>>>>>
>>>>>>
>>>>>> Best
>>>>>>
>>>>>> Etienne
>>>>>>
Re: [Cassandra] source connector
Posted by Martijn Visser <ma...@apache.org>.
Hi all,
@Etienne many thanks for the PR for the Cassandra Source. Hopefully we can
make this available after the 1.16 release.
With regards to the connector externalization comments, while the wiki for
the release plan for 1.17 is not available yet, the externalization of
connectors is definitely a big topic for the 1.17 release. I see no harm in
already reviewing this PR and not necessarily merging it in the Flink main
repo, but preparing for the dedicated Cassandra connector repository. There
are dedicated threads on the externalization of connectors, if there are
questions or concerns then those should be discussed in those threads.
Best regards,
Martijn
On Mon, Oct 24, 2022 at 6:28 PM Jing Ge <ji...@ververica.com> wrote:
> Hi,
>
> The plan was and the ideal process is to try the externalizing connector
> development and release with the elastic connector first and make it stable
> before starting the migration of other connectors. There are already many
> connectors like Iceberg and AWS connectors that are trying externalizing.
> It is risky and might be out of control to start so many
> connectors externalizing in parallel without the proven stable process.
> Please be aware of the maintenance effort and consider a feasible timing to
> start the externalization.
>
> Best regards,
> Jing
>
> On Fri, Oct 21, 2022 at 4:14 PM Ryan Skraba <ry...@aiven.io.invalid>
> wrote:
>
> > There's definitely consensus on externalizing the flink connectors! I've
> > been tracking the progress and I'd be happy to provide support on
> Cassandra
> > if you'd like.
> >
> > There's some new information at
> >
> >
> https://cwiki.apache.org/confluence/display/FLINK/Externalized+Connector+development
> >
> > The first step to externalizing Cassandra would be for a PMC member to
> > create the flink-connector-cassandra repository for the project.
> >
> > If I understand correctly, this shouldn't block your PR, since the core
> > connector (inside apache/flink) and the external one
> > (apache/flink-cassandra-connector) should be kept in sync for one major
> > release cycle. For my own benefit, I can start by reviewing your PR!
> >
> > All my best, Ryan
> >
> > On Fri, Oct 21, 2022 at 11:51 AM Etienne Chauchot <ec...@apache.org>
> > wrote:
> >
> > > Hi,
> > >
> > > Yes sure, if there is a consensus on moving, just tell me where to move
> > > my PR to.
> > >
> > > Best
> > >
> > > Etienne
> > >
> > > Le 19/10/2022 à 18:20, Alexander Fedulov a écrit :
> > > > Hi Etienne,
> > > >
> > > > thanks for your contribution. In light of the current efforts to
> > > > externalize connectors, do you think we could maybe combine the new
> > > > implementation with moving it into an external repository instead of
> > > > merging into Flink main?
> > > >
> > > > Best,
> > > > Alexander Fedulov
> > > >
> > > > On Fri, Oct 14, 2022 at 4:18 PM Etienne Chauchot <
> echauchot@apache.org
> > >
> > > > wrote:
> > > >
> > > >> Hi all,
> > > >>
> > > >> As promised, I have developed the Cassandra source connector based
> on
> > > >> FLIP-27. I've just submitted the
> > > >> PR:https://github.com/apache/flink/pull/21073
> > > >>
> > > >>
> > > >> Best
> > > >>
> > > >> Etienne
> > > >>
> > >
> >
>
Re: [Cassandra] source connector
Posted by Jing Ge <ji...@ververica.com>.
Hi,
The plan was and the ideal process is to try the externalizing connector
development and release with the elastic connector first and make it stable
before starting the migration of other connectors. There are already many
connectors like Iceberg and AWS connectors that are trying externalizing.
It is risky and might be out of control to start so many
connectors externalizing in parallel without the proven stable process.
Please be aware of the maintenance effort and consider a feasible timing to
start the externalization.
Best regards,
Jing
On Fri, Oct 21, 2022 at 4:14 PM Ryan Skraba <ry...@aiven.io.invalid>
wrote:
> There's definitely consensus on externalizing the flink connectors! I've
> been tracking the progress and I'd be happy to provide support on Cassandra
> if you'd like.
>
> There's some new information at
>
> https://cwiki.apache.org/confluence/display/FLINK/Externalized+Connector+development
>
> The first step to externalizing Cassandra would be for a PMC member to
> create the flink-connector-cassandra repository for the project.
>
> If I understand correctly, this shouldn't block your PR, since the core
> connector (inside apache/flink) and the external one
> (apache/flink-cassandra-connector) should be kept in sync for one major
> release cycle. For my own benefit, I can start by reviewing your PR!
>
> All my best, Ryan
>
> On Fri, Oct 21, 2022 at 11:51 AM Etienne Chauchot <ec...@apache.org>
> wrote:
>
> > Hi,
> >
> > Yes sure, if there is a consensus on moving, just tell me where to move
> > my PR to.
> >
> > Best
> >
> > Etienne
> >
> > Le 19/10/2022 à 18:20, Alexander Fedulov a écrit :
> > > Hi Etienne,
> > >
> > > thanks for your contribution. In light of the current efforts to
> > > externalize connectors, do you think we could maybe combine the new
> > > implementation with moving it into an external repository instead of
> > > merging into Flink main?
> > >
> > > Best,
> > > Alexander Fedulov
> > >
> > > On Fri, Oct 14, 2022 at 4:18 PM Etienne Chauchot <echauchot@apache.org
> >
> > > wrote:
> > >
> > >> Hi all,
> > >>
> > >> As promised, I have developed the Cassandra source connector based on
> > >> FLIP-27. I've just submitted the
> > >> PR:https://github.com/apache/flink/pull/21073
> > >>
> > >>
> > >> Best
> > >>
> > >> Etienne
> > >>
> >
>
Re: [Cassandra] source connector
Posted by Ryan Skraba <ry...@aiven.io.INVALID>.
There's definitely consensus on externalizing the flink connectors! I've
been tracking the progress and I'd be happy to provide support on Cassandra
if you'd like.
There's some new information at
https://cwiki.apache.org/confluence/display/FLINK/Externalized+Connector+development
The first step to externalizing Cassandra would be for a PMC member to
create the flink-connector-cassandra repository for the project.
If I understand correctly, this shouldn't block your PR, since the core
connector (inside apache/flink) and the external one
(apache/flink-cassandra-connector) should be kept in sync for one major
release cycle. For my own benefit, I can start by reviewing your PR!
All my best, Ryan
On Fri, Oct 21, 2022 at 11:51 AM Etienne Chauchot <ec...@apache.org>
wrote:
> Hi,
>
> Yes sure, if there is a consensus on moving, just tell me where to move
> my PR to.
>
> Best
>
> Etienne
>
> Le 19/10/2022 à 18:20, Alexander Fedulov a écrit :
> > Hi Etienne,
> >
> > thanks for your contribution. In light of the current efforts to
> > externalize connectors, do you think we could maybe combine the new
> > implementation with moving it into an external repository instead of
> > merging into Flink main?
> >
> > Best,
> > Alexander Fedulov
> >
> > On Fri, Oct 14, 2022 at 4:18 PM Etienne Chauchot <ec...@apache.org>
> > wrote:
> >
> >> Hi all,
> >>
> >> As promised, I have developed the Cassandra source connector based on
> >> FLIP-27. I've just submitted the
> >> PR:https://github.com/apache/flink/pull/21073
> >>
> >>
> >> Best
> >>
> >> Etienne
> >>
>
Re: [Cassandra] source connector
Posted by Etienne Chauchot <ec...@apache.org>.
Hi,
Yes sure, if there is a consensus on moving, just tell me where to move
my PR to.
Best
Etienne
Le 19/10/2022 à 18:20, Alexander Fedulov a écrit :
> Hi Etienne,
>
> thanks for your contribution. In light of the current efforts to
> externalize connectors, do you think we could maybe combine the new
> implementation with moving it into an external repository instead of
> merging into Flink main?
>
> Best,
> Alexander Fedulov
>
> On Fri, Oct 14, 2022 at 4:18 PM Etienne Chauchot <ec...@apache.org>
> wrote:
>
>> Hi all,
>>
>> As promised, I have developed the Cassandra source connector based on
>> FLIP-27. I've just submitted the
>> PR:https://github.com/apache/flink/pull/21073
>>
>>
>> Best
>>
>> Etienne
>>
Re: [Cassandra] source connector
Posted by Alexander Fedulov <al...@deltastream.io>.
Hi Etienne,
thanks for your contribution. In light of the current efforts to
externalize connectors, do you think we could maybe combine the new
implementation with moving it into an external repository instead of
merging into Flink main?
Best,
Alexander Fedulov
On Fri, Oct 14, 2022 at 4:18 PM Etienne Chauchot <ec...@apache.org>
wrote:
> Hi all,
>
> As promised, I have developed the Cassandra source connector based on
> FLIP-27. I've just submitted the
> PR:https://github.com/apache/flink/pull/21073
>
>
> Best
>
> Etienne
>