Re: Create a public open GitHub org for Flink ecosystem projects.

[Becket Qin <be...@gmail.com>]

Hi all,

I am happy to let you know that the Flink PMC has granted the permission to
use *flink-extended* as the name of the GitHub organization.

This name was chosen because:
1. It is generic enough to host almost any ecosystem project of Flink.
2. The name is clear that the hosted projects aim to extend the capability
of Flink, and is not a part of Apache Flink.

This GitHub organization <https://github.com/flink-extended> has been
created and I will add a few projects to it shortly!

Thanks,

Jiangjie (Becket) Qin

On Thu, Sep 23, 2021 at 5:41 PM Becket Qin <be...@gmail.com> wrote:

> Thanks Martijn, looking forward to the connector ecosystem proposal!
>
> Also thanks Yu and Guowei for the positive feedback :)
>
> On Thu, Sep 23, 2021 at 4:14 PM Guowei Ma <gu...@gmail.com> wrote:
>
>> Hi, Becket
>>
>> Thank you very much for promoting this work.
>> After 1.14 is released, we have plans to open source Remote Shuffle
>> Service
>> for Flink. If we have this, we can put it there.
>>
>> Best,
>> Guowei
>>
>>
>> On Thu, Sep 23, 2021 at 4:07 PM Martijn Visser <ma...@ververica.com>
>> wrote:
>>
>> > Hi all,
>> >
>> > Thanks Becket for the initiative. With the example projects mentioned, I
>> > agree that it makes a lot of sense to improve the ecosystem for projects
>> > that can be used to extend Flink.
>> >
>> > I would like to mention that Arvid and I have been working on a proposal
>> > with regards to the connectors and the connector ecosystem, which we
>> want
>> > to share with the community probably next week or so.
>> >
>> > Best regards,
>> >
>> > Martijn
>> >
>> > On Thu, 23 Sept 2021 at 10:00, Becket Qin <be...@gmail.com> wrote:
>> >
>> > > Thanks Leonard. Glad to know it helps.
>> > >
>> > > That said, I also want to mention that for connectors, another option
>> is
>> > to
>> > > contribute them to Apache Bahir. I think letting the project stay in
>> this
>> > > GitHub org for some time to ensure it is well maintained before
>> > > contributing that to an Apache project seems a good approach.
>> > >
>> > > Cheers,
>> > >
>> > > Jiangjie (Becket) Qin
>> > >
>> > > On Thu, Sep 23, 2021 at 3:26 PM Leonard Xu <xb...@gmail.com> wrote:
>> > >
>> > > > Thanks  Becket and PMC members for the effort.
>> > > >
>> > > > There’s one developer is willing to contribute MongoDB connector to
>> > > > community in user-zh mail list,
>> > > >  he has developed an internal version in their company, I think this
>> > > > project is pretty proper for such kind of contributions.
>> > > >
>> > > > Best,
>> > > > Leonard
>> > > > [1]
>> > > >
>> > >
>> >
>> https://lists.apache.org/thread.html/r82c376fea440100f1cb3050026afa07cdc99b94454f608e05e4102f1%40%3Cuser-zh.flink.apache.org%3E
>> > > >
>> > > > > 在 2021年9月23日，14:59，Ingo Bürk <in...@ververica.com> 写道：
>> > > > >
>> > > > > Hi,
>> > > > >
>> > > > > thank you (and the PMC) for the initiative on such a community
>> > effort.
>> > > > > Are there already projects expected/known to move to such an
>> > > > organization?
>> > > > > I think it would make sense to have at least a couple projects
>> lined
>> > up
>> > > > so
>> > > > > the org doesn't start out empty.
>> > > > >
>> > > > >
>> > > > > Best
>> > > > > Ingo
>> > > > >
>> > > > > On Thu, Sep 23, 2021 at 8:43 AM Becket Qin <be...@gmail.com>
>> > > wrote:
>> > > > >
>> > > > >> Hi Flink devs,
>> > > > >>
>> > > > >> Recently we had some discussion in the Flink PMC about creating a
>> > > public
>> > > > >> open GitHub organization to host the code repo of some Flink
>> > ecosystem
>> > > > >> projects. Instead of the Flink PMC doing this, we found that it
>> is
>> > > more
>> > > > >> suitable for someone in the Flink community to do this on their
>> > > personal
>> > > > >> behalf. So here I would love to see if people would be
>> interested in
>> > > > coming
>> > > > >> together to help create and maintain this GitHub organization as
>> a
>> > > > >> community effort.
>> > > > >>
>> > > > >> *** Motivation*
>> > > > >>
>> > > > >> Currently, usually an ecosystem project is hosted in a company's
>> > > GitHub
>> > > > >> repo. However, this does not always work well for those who want
>> to
>> > > > >> collaborate on the projects..
>> > > > >>
>> > > > >>   1. Some employers may have concerns if their employees
>> contribute
>> > > code
>> > > > >>   to another company's repo. Instead, they would rather fork and
>> > > develop
>> > > > >> in
>> > > > >>   their own repo. This results in split efforts instead of joint
>> > force
>> > > > to
>> > > > >>   develop the project.
>> > > > >>   2. Sometimes a company's policy disallows granting repo
>> > permissions
>> > > to
>> > > > >>   external contributors.
>> > > > >>   3. Sometimes a company does not have a GitHub repo and is also
>> not
>> > > > >>   willing to open source a project in a personal repo.
>> > > > >>
>> > > > >> Therefore a public open GitHub organization would provide a
>> > *neutral*
>> > > > place
>> > > > >> helpful to facilitate the sharing and collaboration on the Flink
>> > > > ecosystem
>> > > > >> projects for developers in these situations.
>> > > > >>
>> > > > >> *** How does it work?*
>> > > > >>
>> > > > >>   1. The public ecosystem GitHub org would be created and
>> maintained
>> > > by
>> > > > a
>> > > > >>   few volunteers.
>> > > > >>   2. The volunteers who maintain the org are only responsible for
>> > > > creating
>> > > > >>   and deleting the individual project code repositories upon the
>> > > > requests
>> > > > >>   from the project owners.
>> > > > >>   3. When someone wants to put a Flink ecosystem project in this
>> > > > >>   organization, a new GitHub repo will be created to host that
>> > > project.
>> > > > >>   4. The owners of each individual project will maintain the code
>> > repo
>> > > > of
>> > > > >>   that project, including merging PRs, granting commit
>> permissions
>> > to
>> > > > >> other
>> > > > >>   contributors, publishing releases, etc.
>> > > > >>
>> > > > >> *Note that this open GitHub org is NOT affiliated with ASF or the
>> > > Apache
>> > > > >> Flink project, although the volunteers who maintain the org may
>> be
>> > > Flink
>> > > > >> committers or PMC members.*
>> > > > >>
>> > > > >> *** What's next*
>> > > > >> If people find the public GitHub org for the ecosystem projects
>> > > useful,
>> > > > we
>> > > > >> will do the following:
>> > > > >>
>> > > > >>   1. Create the org together with the volunteers helping maintain
>> > the
>> > > > org.
>> > > > >>   (PMC permission is required if the org name contains "Flink",
>> > which
>> > > is
>> > > > >>   likely).
>> > > > >>   2. Welcome people to put their projects in the org as they see
>> > fit.
>> > > > >>   3. Link the projects to flink-packages.org
>> > > > >>
>> > > > >>
>> > > > >> To give some additional context, the following questions were
>> raised
>> > > and
>> > > > >> answered in the earlier PMC discussion.
>> > > > >> *** Why not host the projects in Apache?*
>> > > > >> One suggestion brought up was asking the ecosystem project
>> owners to
>> > > > >> contribute their projects to Apache, either to an existing
>> project
>> > > such
>> > > > as
>> > > > >> Apache Bahir, or through the incubator.
>> > > > >>
>> > > > >> The ASF is a great place to host organization neutral projects.
>> > > However,
>> > > > >> being a part of an Apache project has a high standard and demands
>> > long
>> > > > term
>> > > > >> commitments. People also have to be committers, use jira, mailing
>> > > lists.
>> > > > >> The projects might also have to go through incubation which is
>> even
>> > > more
>> > > > >> involved.
>> > > > >>
>> > > > >> Some of the ecosystem projects might not be ready to become a
>> part
>> > of
>> > > > >> Apache in some cases. The project may still be in the starting
>> stage
>> > > and
>> > > > >> people don't know if it will last. Some of the projects may want
>> > more
>> > > > >> flexible coordination, etc.
>> > > > >>
>> > > > >> Therefore, an open GitHub organization would solve the issues
>> > > mentioned
>> > > > in
>> > > > >> the motivation with the least restrictions.
>> > > > >>
>> > > > >> *** The relationship with flink-packages.org <
>> > > http://flink-packages.org
>> > > > >*
>> > > > >>
>> > > > >> As people may already know that we have a flink-packages.org
>> > website
>> > > > >> listing the Flink ecosystem projects so users can easily find
>> them.
>> > > One
>> > > > >> reasonable question is how the new open GitHub org relates to the
>> > > > website.
>> > > > >>
>> > > > >> While flink-package.org aims to connect the project developers
>> to
>> > the
>> > > > end
>> > > > >> users, the GitHub org focuses on helping with the collaboration
>> > > between
>> > > > >> developers of the projects. The projects in the GitHub org will
>> be
>> > > > linked
>> > > > >> to and listed on the website.
>> > > > >>
>> > > > >> Thanks,
>> > > > >>
>> > > > >> Jiangjie (Becket) Qin
>> > > > >>
>> > > >
>> > > >
>> > >
>> >
>>
>