You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@flink.apache.org by Rich Bowen <rb...@apache.org> on 2025/12/19 18:57:44 UTC
Vendor-specific repos (re: flink-connector-aws)
Hey friends.
How does Flink feel about vendor-specific connectors, in general?
The reason that I'm asking is that I'm working with a team at work about the https://github.com/apache/flink-connector-aws repository, and the (very understandable!) difficulty of getting non-AWS employees interested/engaged/passionate in issues around that repo. And, in turn, the difficulty of getting reviews/votes on releases for stuff from that repo.
It also seems to be the only such vendor-specific repo (I suppose you could count elasticsearch there, kinda). Does the project prefer that such work happen elsewhere, in the aws Github org, or is there value to the community in having it in the apache/flink-* namespace?
If the former, that can be very easily arranged. If the latter, how do you recommend that we operate to deliver quickly for our customers, but also engage in reasonable ways with the community?
Re: Vendor-specific repos (re: flink-connector-aws)
Posted by Rich Bowen <rb...@apache.org>.
On 2026/01/09 15:59:23 Robert Metzger wrote:
> Hey Rich,
> sorry for the late reply.
>
> Looking back at the initial discussion to introduce flink-connector-aws,
> the reasoning was a not around organizational or vendor-specific arguments,
> rather technical around dependencies of shared utilities:
> https://lists.apache.org/thread/swp4bs8407gtsgn2gh0k3wx1m4o3kqqp
>
> I think the problem with the AWS connector repo in particular is that many
> of the committers / PMC members who initially maintained the repo have
> moved on to other Flink-related employers, and now have a different focus.
> In general, the Flink connectors suffer from lack of review capacity. I
> think we'll need to get new committers involved in the project to fix that
> issue. The PMC had a short discussion about this problem at the end of last
> year.
>
> Do you have any specific PRs you'd like to get merged?
Thanks so much for your response here. This is very helpful. As you know, I'm an outsider to this project, but trying to provide mentorship/guidance for AWS contributors looking to get more involved. That (not just Flink in particular, but Apache projects in general) is a large part of my role at AWS. And each ASF project has a different culture, which requires some time to understand. So appreciate your patience in explaining.
I sincerely hope that we can begin to rebuild our participation here, but of course we do understand that earning trust and visibility takes time. And so this is, as you mention, a whole new group of people that are trying to get up to speed not just with the code, but with the community dynamics.
Thank you!
--Rich
Re: Vendor-specific repos (re: flink-connector-aws)
Posted by tison <wa...@gmail.com>.
> how do you recommend that we operate to deliver quickly for our customers, but also engage in reasonable ways with the community?
In short, either you find a PMC liaison[^1] to help with the release, or
have a PMC member as your employee. The latter can be done by that
employee contributing to the connector, and the Flink PMC considers
him/her a good candidate.
I can't speak on behalf of the Flink PMC, but this seems to be a
common pattern that happens several times.
[^1] can be different each time
Best,
tison.
tison <wa...@gmail.com> 于2026年1月10日周六 02:06写道：
>
> Hi,
>
> Just to share some of my experience -
>
> I'm a Flink committer but not a PMC member, so I can merge patches but
> can hardly drive a release.
>
> During one of my previous careers, I was assigned to help in
> maintaining flink-connector-pulsar [1].
>
> [1] https://github.com/apache/flink-connector-pulsar
>
> My colleague (at that time), Yufan Sheng (@syhily) was the major
> developer then. I helped him review patches and merge them.
>
> When we need a new release, since I know several Flink PMC members
> well, I contact them privately to help drive a release on the list.
>
> After we leave the company, we will hardly have time to allocate to
> that repo. As a result, there is no new release since v4.1.0 (Dec.
> 2023). I can see a few new commits, however.
>
> This is not a bad thing. ASF projects are driven by "volunteers": no
> "interest", no effort.
>
> One thing that could be helpful is, if the related PMC members are
> willing, registering themselves as potential coordinators for the
> connector patch review and releases, so that any new contributor
> wouldn't be scared off by thinking the connector repo is discontinued.
>
> Best,
> tison.
>
> Robert Metzger <rm...@apache.org> 于2026年1月9日周五 23:59写道：
> >
> > Hey Rich,
> > sorry for the late reply.
> >
> > Looking back at the initial discussion to introduce flink-connector-aws,
> > the reasoning was a not around organizational or vendor-specific arguments,
> > rather technical around dependencies of shared utilities:
> > https://lists.apache.org/thread/swp4bs8407gtsgn2gh0k3wx1m4o3kqqp
> >
> > I think the problem with the AWS connector repo in particular is that many
> > of the committers / PMC members who initially maintained the repo have
> > moved on to other Flink-related employers, and now have a different focus.
> > In general, the Flink connectors suffer from lack of review capacity. I
> > think we'll need to get new committers involved in the project to fix that
> > issue. The PMC had a short discussion about this problem at the end of last
> > year.
> >
> > Do you have any specific PRs you'd like to get merged?
> >
> > I personally prefer if core connectors are maintained by Flink, and I
> > believe at least the AWS Kinesis connector is pretty core to Flink. So I'm
> > personally in favor of supporting AWS in getting their changes in,
> > establishing new committers at the project.
> >
> > I'll think more about this issue, and I'll try to get a conclusion from the
> > PMC discussion on the state of the connectors in general.
> >
> >
> > On Fri, Dec 19, 2025 at 7:58 PM Rich Bowen <rb...@apache.org> wrote:
> >
> > > Hey friends.
> > >
> > > How does Flink feel about vendor-specific connectors, in general?
> > >
> > > The reason that I'm asking is that I'm working with a team at work about
> > > the https://github.com/apache/flink-connector-aws repository, and the
> > > (very understandable!) difficulty of getting non-AWS employees
> > > interested/engaged/passionate in issues around that repo. And, in turn, the
> > > difficulty of getting reviews/votes on releases for stuff from that repo.
> > >
> > > It also seems to be the only such vendor-specific repo (I suppose you
> > > could count elasticsearch there, kinda). Does the project prefer that such
> > > work happen elsewhere, in the aws Github org, or is there value to the
> > > community in having it in the apache/flink-* namespace?
> > >
> > > If the former, that can be very easily arranged. If the latter, how do you
> > > recommend that we operate to deliver quickly for our customers, but also
> > > engage in reasonable ways with the community?
> > >
Re: Vendor-specific repos (re: flink-connector-aws)
Posted by tison <wa...@gmail.com>.
Hi,
Just to share some of my experience -
I'm a Flink committer but not a PMC member, so I can merge patches but
can hardly drive a release.
During one of my previous careers, I was assigned to help in
maintaining flink-connector-pulsar [1].
[1] https://github.com/apache/flink-connector-pulsar
My colleague (at that time), Yufan Sheng (@syhily) was the major
developer then. I helped him review patches and merge them.
When we need a new release, since I know several Flink PMC members
well, I contact them privately to help drive a release on the list.
After we leave the company, we will hardly have time to allocate to
that repo. As a result, there is no new release since v4.1.0 (Dec.
2023). I can see a few new commits, however.
This is not a bad thing. ASF projects are driven by "volunteers": no
"interest", no effort.
One thing that could be helpful is, if the related PMC members are
willing, registering themselves as potential coordinators for the
connector patch review and releases, so that any new contributor
wouldn't be scared off by thinking the connector repo is discontinued.
Best,
tison.
Robert Metzger <rm...@apache.org> 于2026年1月9日周五 23:59写道：
>
> Hey Rich,
> sorry for the late reply.
>
> Looking back at the initial discussion to introduce flink-connector-aws,
> the reasoning was a not around organizational or vendor-specific arguments,
> rather technical around dependencies of shared utilities:
> https://lists.apache.org/thread/swp4bs8407gtsgn2gh0k3wx1m4o3kqqp
>
> I think the problem with the AWS connector repo in particular is that many
> of the committers / PMC members who initially maintained the repo have
> moved on to other Flink-related employers, and now have a different focus.
> In general, the Flink connectors suffer from lack of review capacity. I
> think we'll need to get new committers involved in the project to fix that
> issue. The PMC had a short discussion about this problem at the end of last
> year.
>
> Do you have any specific PRs you'd like to get merged?
>
> I personally prefer if core connectors are maintained by Flink, and I
> believe at least the AWS Kinesis connector is pretty core to Flink. So I'm
> personally in favor of supporting AWS in getting their changes in,
> establishing new committers at the project.
>
> I'll think more about this issue, and I'll try to get a conclusion from the
> PMC discussion on the state of the connectors in general.
>
>
> On Fri, Dec 19, 2025 at 7:58 PM Rich Bowen <rb...@apache.org> wrote:
>
> > Hey friends.
> >
> > How does Flink feel about vendor-specific connectors, in general?
> >
> > The reason that I'm asking is that I'm working with a team at work about
> > the https://github.com/apache/flink-connector-aws repository, and the
> > (very understandable!) difficulty of getting non-AWS employees
> > interested/engaged/passionate in issues around that repo. And, in turn, the
> > difficulty of getting reviews/votes on releases for stuff from that repo.
> >
> > It also seems to be the only such vendor-specific repo (I suppose you
> > could count elasticsearch there, kinda). Does the project prefer that such
> > work happen elsewhere, in the aws Github org, or is there value to the
> > community in having it in the apache/flink-* namespace?
> >
> > If the former, that can be very easily arranged. If the latter, how do you
> > recommend that we operate to deliver quickly for our customers, but also
> > engage in reasonable ways with the community?
> >
Re: Vendor-specific repos (re: flink-connector-aws)
Posted by Robert Metzger <rm...@apache.org>.
Hey Rich,
sorry for the late reply.
Looking back at the initial discussion to introduce flink-connector-aws,
the reasoning was a not around organizational or vendor-specific arguments,
rather technical around dependencies of shared utilities:
https://lists.apache.org/thread/swp4bs8407gtsgn2gh0k3wx1m4o3kqqp
I think the problem with the AWS connector repo in particular is that many
of the committers / PMC members who initially maintained the repo have
moved on to other Flink-related employers, and now have a different focus.
In general, the Flink connectors suffer from lack of review capacity. I
think we'll need to get new committers involved in the project to fix that
issue. The PMC had a short discussion about this problem at the end of last
year.
Do you have any specific PRs you'd like to get merged?
I personally prefer if core connectors are maintained by Flink, and I
believe at least the AWS Kinesis connector is pretty core to Flink. So I'm
personally in favor of supporting AWS in getting their changes in,
establishing new committers at the project.
I'll think more about this issue, and I'll try to get a conclusion from the
PMC discussion on the state of the connectors in general.
On Fri, Dec 19, 2025 at 7:58 PM Rich Bowen <rb...@apache.org> wrote:
> Hey friends.
>
> How does Flink feel about vendor-specific connectors, in general?
>
> The reason that I'm asking is that I'm working with a team at work about
> the https://github.com/apache/flink-connector-aws repository, and the
> (very understandable!) difficulty of getting non-AWS employees
> interested/engaged/passionate in issues around that repo. And, in turn, the
> difficulty of getting reviews/votes on releases for stuff from that repo.
>
> It also seems to be the only such vendor-specific repo (I suppose you
> could count elasticsearch there, kinda). Does the project prefer that such
> work happen elsewhere, in the aws Github org, or is there value to the
> community in having it in the apache/flink-* namespace?
>
> If the former, that can be very easily arranged. If the latter, how do you
> recommend that we operate to deliver quickly for our customers, but also
> engage in reasonable ways with the community?
>