You are viewing a plain text version of this content. The canonical link for it is here.

Posted to dev@flink.apache.org by Xintong Song <to...@gmail.com> on 2022/04/28 05:39:09 UTC

[DISCUSS] DockerHub repository maintainers

Hi devs,

I'd like to start a discussion about maintainers for DockerHub repositories
under the *apache* namespace [1].

Currently, the Flink community maintains various repositories (flink,
flink-statefun, flink-statefun-playground, and flink-kubernetes-operator)
on DockerHub under the *apache* namespace. There's a limitation on how many
members the *apache* namespace can add, and recently INFRA is complaining
about Flink taking too many places [2][3]. They would like us to reduce our
maintainers from 20 now to 5.

Jingsong and I would like to volunteer as two of the maintainers, and we
would like to learn who else wants to join us. While any committer may
serve as one of the maintainers, it's probably nice to also involve at
least one maintainer from statefun and one from kubernetes-operator.

What do you think?

Thank you~

Xintong Song


[1] https://hub.docker.com/orgs/apache

[2] https://issues.apache.org/jira/browse/INFRA-23208

[3] https://issues.apache.org/jira/browse/INFRA-23213

Re: [DISCUSS] DockerHub repository maintainers

Posted by Xintong Song <to...@gmail.com>.

Thanks for volunteering, Marton.

Since the discussion has been open for quite long and no one else is
volunteering, I'll reply to the INFRA with the list of 4 maintainers
(Xintong, Jinsong, Till, Marton).

Thank you~

Xintong Song



On Sat, May 7, 2022 at 3:15 PM Márton Balassi <ba...@gmail.com>
wrote:

> Hi team,
>
> I volunteer for the flink-kubernetes-operator repo.
>
> On Fri, May 6, 2022 at 1:42 PM Xintong Song <to...@gmail.com> wrote:
>
> > @Till,
> >
> > Thanks for volunteering.
> >
> > @Konstantin,
> >
> > From my experience, the effort that requires DockerHub access in the main
> > project release process is quite limited. I helped Yun Gao on releasing
> the
> > 1.15.0 images, and what I did was just check out the `flink-docker` repo
> > and run the release script, that's it. If all the sub-projects are as
> easy
> > as the main project, then it's probably ok that only a small group of
> > people have access. Concerning the redundancy, if a maintainer from a
> > sub-project is temporarily unreachable, I believe the other maintainers
> > would be glad to help.
> >
> > It would of course be good to have more seats. I just haven't come up
> with
> > good reasons to persuade the INFRA folks. What's your suggestions?
> >
> >
> > Thank you~
> >
> > Xintong Song
> >
> >
> >
> > On Fri, May 6, 2022 at 6:38 PM Konstantin Knauf <kn...@apache.org>
> wrote:
> >
> > > Hi Xintong,
> > >
> > > it is a pity that we can only have 5 maintainers. Every (patch) release
> > of
> > > flink, flink-statefun, the flink-kubernetes-operator requires a
> > maintainer
> > > to publish the image then, if I am not mistaken. As its mostly
> different
> > > groups managing the sub-projects, this is quite the bottleneck. If we
> > give
> > > one seat to flink-statefun maintainers, one to the
> > > flink-kubernetes-operator maintainers, this leaves three seats for
> Apache
> > > Flink core, and there is no redundancy for the other projects. When I
> > > managed the last two patch releases, the DockerHub access was also the
> > > biggest hurdle. Maybe we can talk to the INFRA people again. We can
> > > certainly reduce it, but 5 is very little.
> > >
> > > Cheers,
> > >
> > > Konstantin
> > >
> > >
> > >
> > >
> > >
> > >
> > > Am Fr., 6. Mai 2022 um 09:00 Uhr schrieb Till Rohrmann <
> > > trohrmann@apache.org
> > > >:
> > >
> > > > Hi everyone,
> > > >
> > > > thanks for starting this discussion Xintong. I would volunteer as a
> > > > maintainer of the flink-statefun Docker repository if you need one.
> > > >
> > > > Cheers,
> > > > Till
> > > >
> > > > On Fri, May 6, 2022 at 6:22 AM Xintong Song <to...@gmail.com>
> > > wrote:
> > > >
> > > >> It seems to me we at least don't have a consensus on dropping the
> use
> > of
> > > >> apache namespace, which means we need to decide on a list of
> > maintainers
> > > >> anyway. So maybe we can get the discussion back to the maintainers.
> We
> > > may
> > > >> continue the official-image vs. apache-namespace in a separate
> thread
> > if
> > > >> necessary.
> > > >>
> > > >> As mentioned previously, we need to reduce the number of maintainers
> > > from
> > > >> 20 to 5, as required by INFRA. Jingsong and I would like to
> volunteer
> > > as 2
> > > >> of the 5, and we would like to learn who else wants to join us. Of
> > > course
> > > >> the list of maintainers can be modified later.
> > > >>
> > > >> *This also means the current maintainers may be removed from the
> > list.*
> > > >> Please let us know if you still need that privilege. CC-ed all the
> > > current
> > > >> maintainers for attention.
> > > >>
> > > >> Thank you~
> > > >>
> > > >> Xintong Song
> > > >>
> > > >>
> > > >>
> > > >> On Wed, May 4, 2022 at 3:14 PM Chesnay Schepler <chesnay@apache.org
> >
> > > >> wrote:
> > > >>
> > > >> > One advantage is that the images are periodically rebuilt to get
> > > >> > security fixes.
> > > >> >
> > > >> > The operator is a different story anyway because it is AFAIK only
> > > >> > supposed to be used via docker
> > > >> > (i.e., no standalone mode), which alleviates concerns about
> keeping
> > > the
> > > >> > logic within the image
> > > >> > to a minimum (which bit us in the past on the flink side).
> > > >> >
> > > >> > On 03/05/2022 16:09, Yang Wang wrote:
> > > >> > > The flink-kubernetes-operator project is only published
> > > >> > > via apache/flink-kubernetes-operator on docker hub and github
> > > >> packages.
> > > >> > > We do not find the obvious advantages by using docker hub
> official
> > > >> > images.
> > > >> > >
> > > >> > > Best,
> > > >> > > Yang
> > > >> > >
> > > >> > > Xintong Song <to...@gmail.com> 于2022年4月28日周四 19:27写道：
> > > >> > >
> > > >> > >> I agree with you that doing QA for the image after the release
> > has
> > > >> been
> > > >> > >> finalized doesn't feel right. IIUR, that is mostly because
> > official
> > > >> > image
> > > >> > >> PR needs 1) the binary release being deployed and propagated
> and
> > 2)
> > > >> the
> > > >> > >> corresponding git commit being specified. I'm not completely
> sure
> > > >> about
> > > >> > >> this. Maybe we can improve the process by investigating more
> > about
> > > >> the
> > > >> > >> feasibility of pre-verifying an official image PR before
> > finalizing
> > > >> the
> > > >> > >> release. It's definitely a good thing to do if possible.
> > > >> > >>
> > > >> > >> I also agree that QA from DockerHub folks is valuable to us.
> > > >> > >>
> > > >> > >> I'm not against publishing official-images, and I'm not against
> > > >> working
> > > >> > >> closely with the DockerHub folks to improve the process of
> > > delivering
> > > >> > the
> > > >> > >> official image. However, I don't think these should become
> > reasons
> > > >> that
> > > >> > we
> > > >> > >> don't release our own apache/flink images.
> > > >> > >>
> > > >> > >> Taking the 1.12.0 as an example, admittedly it would be nice
> for
> > us
> > > >> to
> > > >> > >> comply with the DockerHub folks' standards and not have a
> > > >> > >> just-for-kubernetes command in our entrypoint. However, this is
> > IMO
> > > >> far
> > > >> > >> less important compared to delivering the image to our users
> > > timely.
> > > >> I
> > > >> > >> guess that's where the DockerHub folks and us have different
> > > >> > >> priorities, and that's why I think we should have a path that
> is
> > > >> fully
> > > >> > >> controlled by this community to deliver images. We could take
> > their
> > > >> > >> valuable inputs and improve afterwards. Actually, that's what
> we
> > > did
> > > >> for
> > > >> > >> 1.12.0 by starting to release to apache/flink.
> > > >> > >>
> > > >> > >> Thank you~
> > > >> > >>
> > > >> > >> Xintong Song
> > > >> > >>
> > > >> > >>
> > > >> > >>
> > > >> > >> On Thu, Apr 28, 2022 at 6:30 PM Chesnay Schepler <
> > > chesnay@apache.org
> > > >> >
> > > >> > >> wrote:
> > > >> > >>
> > > >> > >>> I still think that's mostly a process issue.
> > > >> > >>> Of course we can be blind-sided if we do the QA for a release
> > > >> artifact
> > > >> > >>> after the release has been finalized.
> > > >> > >>> But that's a clearly broken process from the get-go.
> > > >> > >>>
> > > >> > >>> At the very least we should already open a PR when the RC is
> > > >> created to
> > > >> > >>> get earlier feedback.
> > > >> > >>>
> > > >> > >>> Moreover, nowadays the docker images are way slimmer and we
> are
> > > much
> > > >> > >>> more careful on what is actually added to the scripts.
> > > >> > >>>
> > > >> > >>> Finally, the problems they found did show that their QA is
> very
> > > >> > valuable
> > > >> > >>> to us. And side-stepping that for such an essential piece of a
> > > >> release
> > > >> > >>> isn't a good idea imo.
> > > >> > >>>
> > > >> > >>> On 28/04/2022 11:31, Xintong Song wrote:
> > > >> > >>>> I'm overall against only releasing to official-images.
> > > >> > >>>>
> > > >> > >>>> We started releasing to apache/flink, in addition to the
> > > >> > >> official-image,
> > > >> > >>> in
> > > >> > >>>> 1.12.0. That was because releasing the official-image needs
> > > >> approval
> > > >> > >> from
> > > >> > >>>> the DockerHub folks, which is not under control of the Flink
> > > >> > community.
> > > >> > >>> For
> > > >> > >>>> 1.12.0 there were unfortunately some divergences between us
> and
> > > the
> > > >> > >>>> DockerHub folks, and it ended-up taking us nearly 2 months to
> > get
> > > >> that
> > > >> > >>>> official-image PR merged [1][2]. Many users, especially those
> > who
> > > >> need
> > > >> > >>>> Flink's K8s & Native-K8s deployment modes, were asking for
> the
> > > >> image
> > > >> > >>> after
> > > >> > >>>> 1.12.0 was announced.
> > > >> > >>>>
> > > >> > >>>> One could argue that what happened for 1.12.0 is not a
> regular
> > > >> case.
> > > >> > >>>> However, I'd like to point out that the docker images are not
> > > >> > something
> > > >> > >>>> nice-to-have, but a practically necessary piece of the
> release
> > > for
> > > >> the
> > > >> > >>> k8s
> > > >> > >>>> / native-k8s deployments to work. I'm strongly against a
> > release
> > > >> > >> process
> > > >> > >>>> where such an important piece depends on the approval of a
> 3rd
> > > >> party.
> > > >> > >>>>
> > > >> > >>>> Thank you~
> > > >> > >>>>
> > > >> > >>>> Xintong Song
> > > >> > >>>>
> > > >> > >>>>
> > > >> > >>>> [1] https://issues.apache.org/jira/browse/FLINK-20650
> > > >> > >>>>
> > > >> > >>>> [2]
> > https://github.com/docker-library/official-images/pull/9249
> > > >> > >>>>
> > > >> > >>>>
> > > >> > >>>>
> > > >> > >>>> On Thu, Apr 28, 2022 at 2:43 PM Chesnay Schepler <
> > > >> chesnay@apache.org>
> > > >> > >>> wrote:
> > > >> > >>>>> We could just stop releasing to apache/flink and only go for
> > the
> > > >> > >>>>> official-images route.
> > > >> > >>>>>
> > > >> > >>>>> On 28/04/2022 07:43, Xintong Song wrote:
> > > >> > >>>>>> Forgot to mention that, we have also proposed to use one
> > shared
> > > >> > >> account
> > > >> > >>>>> and
> > > >> > >>>>>> limit its access to the PMC members, like what we do with
> the
> > > >> PyPI
> > > >> > >>>>> account.
> > > >> > >>>>>> Unfortunately, INFRA rejected this proposal [1].
> > > >> > >>>>>>
> > > >> > >>>>>>
> > > >> > >>>>>> Thank you~
> > > >> > >>>>>>
> > > >> > >>>>>> Xintong Song
> > > >> > >>>>>>
> > > >> > >>>>>>
> > > >> > >>>>>> [1] https://issues.apache.org/jira/browse/INFRA-23208
> > > >> > >>>>>>
> > > >> > >>>>>> On Thu, Apr 28, 2022 at 1:39 PM Xintong Song <
> > > >> tonysong820@gmail.com
> > > >> > >
> > > >> > >>>>> wrote:
> > > >> > >>>>>>> Hi devs,
> > > >> > >>>>>>>
> > > >> > >>>>>>> I'd like to start a discussion about maintainers for
> > DockerHub
> > > >> > >>>>>>> repositories under the *apache* namespace [1].
> > > >> > >>>>>>>
> > > >> > >>>>>>> Currently, the Flink community maintains various
> > repositories
> > > >> > >> (flink,
> > > >> > >>>>>>> flink-statefun, flink-statefun-playground, and
> > > >> > >>>>> flink-kubernetes-operator)
> > > >> > >>>>>>> on DockerHub under the *apache* namespace. There's a
> > > limitation
> > > >> on
> > > >> > >> how
> > > >> > >>>>> many
> > > >> > >>>>>>> members the *apache* namespace can add, and recently INFRA
> > is
> > > >> > >>>>> complaining
> > > >> > >>>>>>> about Flink taking too many places [2][3]. They would like
> > us
> > > to
> > > >> > >>> reduce
> > > >> > >>>>> our
> > > >> > >>>>>>> maintainers from 20 now to 5.
> > > >> > >>>>>>>
> > > >> > >>>>>>> Jingsong and I would like to volunteer as two of the
> > > >> maintainers,
> > > >> > >> and
> > > >> > >>> we
> > > >> > >>>>>>> would like to learn who else wants to join us. While any
> > > >> committer
> > > >> > >> may
> > > >> > >>>>>>> serve as one of the maintainers, it's probably nice to
> also
> > > >> involve
> > > >> > >> at
> > > >> > >>>>>>> least one maintainer from statefun and one from
> > > >> > kubernetes-operator.
> > > >> > >>>>>>>
> > > >> > >>>>>>> What do you think?
> > > >> > >>>>>>>
> > > >> > >>>>>>> Thank you~
> > > >> > >>>>>>>
> > > >> > >>>>>>> Xintong Song
> > > >> > >>>>>>>
> > > >> > >>>>>>>
> > > >> > >>>>>>> [1] https://hub.docker.com/orgs/apache
> > > >> > >>>>>>>
> > > >> > >>>>>>> [2] https://issues.apache.org/jira/browse/INFRA-23208
> > > >> > >>>>>>>
> > > >> > >>>>>>> [3] https://issues.apache.org/jira/browse/INFRA-23213
> > > >> > >>>>>>>
> > > >> > >>>>>>>
> > > >> > >>>
> > > >> >
> > > >> >
> > > >>
> > > >
> > >
> > > --
> > > https://twitter.com/snntrable
> > > https://github.com/knaufk
> > >
> >
>

Re: [DISCUSS] DockerHub repository maintainers

Posted by Márton Balassi <ba...@gmail.com>.

Hi team,

I volunteer for the flink-kubernetes-operator repo.

On Fri, May 6, 2022 at 1:42 PM Xintong Song <to...@gmail.com> wrote:

> @Till,
>
> Thanks for volunteering.
>
> @Konstantin,
>
> From my experience, the effort that requires DockerHub access in the main
> project release process is quite limited. I helped Yun Gao on releasing the
> 1.15.0 images, and what I did was just check out the `flink-docker` repo
> and run the release script, that's it. If all the sub-projects are as easy
> as the main project, then it's probably ok that only a small group of
> people have access. Concerning the redundancy, if a maintainer from a
> sub-project is temporarily unreachable, I believe the other maintainers
> would be glad to help.
>
> It would of course be good to have more seats. I just haven't come up with
> good reasons to persuade the INFRA folks. What's your suggestions?
>
>
> Thank you~
>
> Xintong Song
>
>
>
> On Fri, May 6, 2022 at 6:38 PM Konstantin Knauf <kn...@apache.org> wrote:
>
> > Hi Xintong,
> >
> > it is a pity that we can only have 5 maintainers. Every (patch) release
> of
> > flink, flink-statefun, the flink-kubernetes-operator requires a
> maintainer
> > to publish the image then, if I am not mistaken. As its mostly different
> > groups managing the sub-projects, this is quite the bottleneck. If we
> give
> > one seat to flink-statefun maintainers, one to the
> > flink-kubernetes-operator maintainers, this leaves three seats for Apache
> > Flink core, and there is no redundancy for the other projects. When I
> > managed the last two patch releases, the DockerHub access was also the
> > biggest hurdle. Maybe we can talk to the INFRA people again. We can
> > certainly reduce it, but 5 is very little.
> >
> > Cheers,
> >
> > Konstantin
> >
> >
> >
> >
> >
> >
> > Am Fr., 6. Mai 2022 um 09:00 Uhr schrieb Till Rohrmann <
> > trohrmann@apache.org
> > >:
> >
> > > Hi everyone,
> > >
> > > thanks for starting this discussion Xintong. I would volunteer as a
> > > maintainer of the flink-statefun Docker repository if you need one.
> > >
> > > Cheers,
> > > Till
> > >
> > > On Fri, May 6, 2022 at 6:22 AM Xintong Song <to...@gmail.com>
> > wrote:
> > >
> > >> It seems to me we at least don't have a consensus on dropping the use
> of
> > >> apache namespace, which means we need to decide on a list of
> maintainers
> > >> anyway. So maybe we can get the discussion back to the maintainers. We
> > may
> > >> continue the official-image vs. apache-namespace in a separate thread
> if
> > >> necessary.
> > >>
> > >> As mentioned previously, we need to reduce the number of maintainers
> > from
> > >> 20 to 5, as required by INFRA. Jingsong and I would like to volunteer
> > as 2
> > >> of the 5, and we would like to learn who else wants to join us. Of
> > course
> > >> the list of maintainers can be modified later.
> > >>
> > >> *This also means the current maintainers may be removed from the
> list.*
> > >> Please let us know if you still need that privilege. CC-ed all the
> > current
> > >> maintainers for attention.
> > >>
> > >> Thank you~
> > >>
> > >> Xintong Song
> > >>
> > >>
> > >>
> > >> On Wed, May 4, 2022 at 3:14 PM Chesnay Schepler <ch...@apache.org>
> > >> wrote:
> > >>
> > >> > One advantage is that the images are periodically rebuilt to get
> > >> > security fixes.
> > >> >
> > >> > The operator is a different story anyway because it is AFAIK only
> > >> > supposed to be used via docker
> > >> > (i.e., no standalone mode), which alleviates concerns about keeping
> > the
> > >> > logic within the image
> > >> > to a minimum (which bit us in the past on the flink side).
> > >> >
> > >> > On 03/05/2022 16:09, Yang Wang wrote:
> > >> > > The flink-kubernetes-operator project is only published
> > >> > > via apache/flink-kubernetes-operator on docker hub and github
> > >> packages.
> > >> > > We do not find the obvious advantages by using docker hub official
> > >> > images.
> > >> > >
> > >> > > Best,
> > >> > > Yang
> > >> > >
> > >> > > Xintong Song <to...@gmail.com> 于2022年4月28日周四 19:27写道：
> > >> > >
> > >> > >> I agree with you that doing QA for the image after the release
> has
> > >> been
> > >> > >> finalized doesn't feel right. IIUR, that is mostly because
> official
> > >> > image
> > >> > >> PR needs 1) the binary release being deployed and propagated and
> 2)
> > >> the
> > >> > >> corresponding git commit being specified. I'm not completely sure
> > >> about
> > >> > >> this. Maybe we can improve the process by investigating more
> about
> > >> the
> > >> > >> feasibility of pre-verifying an official image PR before
> finalizing
> > >> the
> > >> > >> release. It's definitely a good thing to do if possible.
> > >> > >>
> > >> > >> I also agree that QA from DockerHub folks is valuable to us.
> > >> > >>
> > >> > >> I'm not against publishing official-images, and I'm not against
> > >> working
> > >> > >> closely with the DockerHub folks to improve the process of
> > delivering
> > >> > the
> > >> > >> official image. However, I don't think these should become
> reasons
> > >> that
> > >> > we
> > >> > >> don't release our own apache/flink images.
> > >> > >>
> > >> > >> Taking the 1.12.0 as an example, admittedly it would be nice for
> us
> > >> to
> > >> > >> comply with the DockerHub folks' standards and not have a
> > >> > >> just-for-kubernetes command in our entrypoint. However, this is
> IMO
> > >> far
> > >> > >> less important compared to delivering the image to our users
> > timely.
> > >> I
> > >> > >> guess that's where the DockerHub folks and us have different
> > >> > >> priorities, and that's why I think we should have a path that is
> > >> fully
> > >> > >> controlled by this community to deliver images. We could take
> their
> > >> > >> valuable inputs and improve afterwards. Actually, that's what we
> > did
> > >> for
> > >> > >> 1.12.0 by starting to release to apache/flink.
> > >> > >>
> > >> > >> Thank you~
> > >> > >>
> > >> > >> Xintong Song
> > >> > >>
> > >> > >>
> > >> > >>
> > >> > >> On Thu, Apr 28, 2022 at 6:30 PM Chesnay Schepler <
> > chesnay@apache.org
> > >> >
> > >> > >> wrote:
> > >> > >>
> > >> > >>> I still think that's mostly a process issue.
> > >> > >>> Of course we can be blind-sided if we do the QA for a release
> > >> artifact
> > >> > >>> after the release has been finalized.
> > >> > >>> But that's a clearly broken process from the get-go.
> > >> > >>>
> > >> > >>> At the very least we should already open a PR when the RC is
> > >> created to
> > >> > >>> get earlier feedback.
> > >> > >>>
> > >> > >>> Moreover, nowadays the docker images are way slimmer and we are
> > much
> > >> > >>> more careful on what is actually added to the scripts.
> > >> > >>>
> > >> > >>> Finally, the problems they found did show that their QA is very
> > >> > valuable
> > >> > >>> to us. And side-stepping that for such an essential piece of a
> > >> release
> > >> > >>> isn't a good idea imo.
> > >> > >>>
> > >> > >>> On 28/04/2022 11:31, Xintong Song wrote:
> > >> > >>>> I'm overall against only releasing to official-images.
> > >> > >>>>
> > >> > >>>> We started releasing to apache/flink, in addition to the
> > >> > >> official-image,
> > >> > >>> in
> > >> > >>>> 1.12.0. That was because releasing the official-image needs
> > >> approval
> > >> > >> from
> > >> > >>>> the DockerHub folks, which is not under control of the Flink
> > >> > community.
> > >> > >>> For
> > >> > >>>> 1.12.0 there were unfortunately some divergences between us and
> > the
> > >> > >>>> DockerHub folks, and it ended-up taking us nearly 2 months to
> get
> > >> that
> > >> > >>>> official-image PR merged [1][2]. Many users, especially those
> who
> > >> need
> > >> > >>>> Flink's K8s & Native-K8s deployment modes, were asking for the
> > >> image
> > >> > >>> after
> > >> > >>>> 1.12.0 was announced.
> > >> > >>>>
> > >> > >>>> One could argue that what happened for 1.12.0 is not a regular
> > >> case.
> > >> > >>>> However, I'd like to point out that the docker images are not
> > >> > something
> > >> > >>>> nice-to-have, but a practically necessary piece of the release
> > for
> > >> the
> > >> > >>> k8s
> > >> > >>>> / native-k8s deployments to work. I'm strongly against a
> release
> > >> > >> process
> > >> > >>>> where such an important piece depends on the approval of a 3rd
> > >> party.
> > >> > >>>>
> > >> > >>>> Thank you~
> > >> > >>>>
> > >> > >>>> Xintong Song
> > >> > >>>>
> > >> > >>>>
> > >> > >>>> [1] https://issues.apache.org/jira/browse/FLINK-20650
> > >> > >>>>
> > >> > >>>> [2]
> https://github.com/docker-library/official-images/pull/9249
> > >> > >>>>
> > >> > >>>>
> > >> > >>>>
> > >> > >>>> On Thu, Apr 28, 2022 at 2:43 PM Chesnay Schepler <
> > >> chesnay@apache.org>
> > >> > >>> wrote:
> > >> > >>>>> We could just stop releasing to apache/flink and only go for
> the
> > >> > >>>>> official-images route.
> > >> > >>>>>
> > >> > >>>>> On 28/04/2022 07:43, Xintong Song wrote:
> > >> > >>>>>> Forgot to mention that, we have also proposed to use one
> shared
> > >> > >> account
> > >> > >>>>> and
> > >> > >>>>>> limit its access to the PMC members, like what we do with the
> > >> PyPI
> > >> > >>>>> account.
> > >> > >>>>>> Unfortunately, INFRA rejected this proposal [1].
> > >> > >>>>>>
> > >> > >>>>>>
> > >> > >>>>>> Thank you~
> > >> > >>>>>>
> > >> > >>>>>> Xintong Song
> > >> > >>>>>>
> > >> > >>>>>>
> > >> > >>>>>> [1] https://issues.apache.org/jira/browse/INFRA-23208
> > >> > >>>>>>
> > >> > >>>>>> On Thu, Apr 28, 2022 at 1:39 PM Xintong Song <
> > >> tonysong820@gmail.com
> > >> > >
> > >> > >>>>> wrote:
> > >> > >>>>>>> Hi devs,
> > >> > >>>>>>>
> > >> > >>>>>>> I'd like to start a discussion about maintainers for
> DockerHub
> > >> > >>>>>>> repositories under the *apache* namespace [1].
> > >> > >>>>>>>
> > >> > >>>>>>> Currently, the Flink community maintains various
> repositories
> > >> > >> (flink,
> > >> > >>>>>>> flink-statefun, flink-statefun-playground, and
> > >> > >>>>> flink-kubernetes-operator)
> > >> > >>>>>>> on DockerHub under the *apache* namespace. There's a
> > limitation
> > >> on
> > >> > >> how
> > >> > >>>>> many
> > >> > >>>>>>> members the *apache* namespace can add, and recently INFRA
> is
> > >> > >>>>> complaining
> > >> > >>>>>>> about Flink taking too many places [2][3]. They would like
> us
> > to
> > >> > >>> reduce
> > >> > >>>>> our
> > >> > >>>>>>> maintainers from 20 now to 5.
> > >> > >>>>>>>
> > >> > >>>>>>> Jingsong and I would like to volunteer as two of the
> > >> maintainers,
> > >> > >> and
> > >> > >>> we
> > >> > >>>>>>> would like to learn who else wants to join us. While any
> > >> committer
> > >> > >> may
> > >> > >>>>>>> serve as one of the maintainers, it's probably nice to also
> > >> involve
> > >> > >> at
> > >> > >>>>>>> least one maintainer from statefun and one from
> > >> > kubernetes-operator.
> > >> > >>>>>>>
> > >> > >>>>>>> What do you think?
> > >> > >>>>>>>
> > >> > >>>>>>> Thank you~
> > >> > >>>>>>>
> > >> > >>>>>>> Xintong Song
> > >> > >>>>>>>
> > >> > >>>>>>>
> > >> > >>>>>>> [1] https://hub.docker.com/orgs/apache
> > >> > >>>>>>>
> > >> > >>>>>>> [2] https://issues.apache.org/jira/browse/INFRA-23208
> > >> > >>>>>>>
> > >> > >>>>>>> [3] https://issues.apache.org/jira/browse/INFRA-23213
> > >> > >>>>>>>
> > >> > >>>>>>>
> > >> > >>>
> > >> >
> > >> >
> > >>
> > >
> >
> > --
> > https://twitter.com/snntrable
> > https://github.com/knaufk
> >
>

Re: [DISCUSS] DockerHub repository maintainers

Posted by Xintong Song <to...@gmail.com>.

@Till,

Thanks for volunteering.

@Konstantin,

From my experience, the effort that requires DockerHub access in the main
project release process is quite limited. I helped Yun Gao on releasing the
1.15.0 images, and what I did was just check out the `flink-docker` repo
and run the release script, that's it. If all the sub-projects are as easy
as the main project, then it's probably ok that only a small group of
people have access. Concerning the redundancy, if a maintainer from a
sub-project is temporarily unreachable, I believe the other maintainers
would be glad to help.

It would of course be good to have more seats. I just haven't come up with
good reasons to persuade the INFRA folks. What's your suggestions?


Thank you~

Xintong Song



On Fri, May 6, 2022 at 6:38 PM Konstantin Knauf <kn...@apache.org> wrote:

> Hi Xintong,
>
> it is a pity that we can only have 5 maintainers. Every (patch) release of
> flink, flink-statefun, the flink-kubernetes-operator requires a maintainer
> to publish the image then, if I am not mistaken. As its mostly different
> groups managing the sub-projects, this is quite the bottleneck. If we give
> one seat to flink-statefun maintainers, one to the
> flink-kubernetes-operator maintainers, this leaves three seats for Apache
> Flink core, and there is no redundancy for the other projects. When I
> managed the last two patch releases, the DockerHub access was also the
> biggest hurdle. Maybe we can talk to the INFRA people again. We can
> certainly reduce it, but 5 is very little.
>
> Cheers,
>
> Konstantin
>
>
>
>
>
>
> Am Fr., 6. Mai 2022 um 09:00 Uhr schrieb Till Rohrmann <
> trohrmann@apache.org
> >:
>
> > Hi everyone,
> >
> > thanks for starting this discussion Xintong. I would volunteer as a
> > maintainer of the flink-statefun Docker repository if you need one.
> >
> > Cheers,
> > Till
> >
> > On Fri, May 6, 2022 at 6:22 AM Xintong Song <to...@gmail.com>
> wrote:
> >
> >> It seems to me we at least don't have a consensus on dropping the use of
> >> apache namespace, which means we need to decide on a list of maintainers
> >> anyway. So maybe we can get the discussion back to the maintainers. We
> may
> >> continue the official-image vs. apache-namespace in a separate thread if
> >> necessary.
> >>
> >> As mentioned previously, we need to reduce the number of maintainers
> from
> >> 20 to 5, as required by INFRA. Jingsong and I would like to volunteer
> as 2
> >> of the 5, and we would like to learn who else wants to join us. Of
> course
> >> the list of maintainers can be modified later.
> >>
> >> *This also means the current maintainers may be removed from the list.*
> >> Please let us know if you still need that privilege. CC-ed all the
> current
> >> maintainers for attention.
> >>
> >> Thank you~
> >>
> >> Xintong Song
> >>
> >>
> >>
> >> On Wed, May 4, 2022 at 3:14 PM Chesnay Schepler <ch...@apache.org>
> >> wrote:
> >>
> >> > One advantage is that the images are periodically rebuilt to get
> >> > security fixes.
> >> >
> >> > The operator is a different story anyway because it is AFAIK only
> >> > supposed to be used via docker
> >> > (i.e., no standalone mode), which alleviates concerns about keeping
> the
> >> > logic within the image
> >> > to a minimum (which bit us in the past on the flink side).
> >> >
> >> > On 03/05/2022 16:09, Yang Wang wrote:
> >> > > The flink-kubernetes-operator project is only published
> >> > > via apache/flink-kubernetes-operator on docker hub and github
> >> packages.
> >> > > We do not find the obvious advantages by using docker hub official
> >> > images.
> >> > >
> >> > > Best,
> >> > > Yang
> >> > >
> >> > > Xintong Song <to...@gmail.com> 于2022年4月28日周四 19:27写道：
> >> > >
> >> > >> I agree with you that doing QA for the image after the release has
> >> been
> >> > >> finalized doesn't feel right. IIUR, that is mostly because official
> >> > image
> >> > >> PR needs 1) the binary release being deployed and propagated and 2)
> >> the
> >> > >> corresponding git commit being specified. I'm not completely sure
> >> about
> >> > >> this. Maybe we can improve the process by investigating more about
> >> the
> >> > >> feasibility of pre-verifying an official image PR before finalizing
> >> the
> >> > >> release. It's definitely a good thing to do if possible.
> >> > >>
> >> > >> I also agree that QA from DockerHub folks is valuable to us.
> >> > >>
> >> > >> I'm not against publishing official-images, and I'm not against
> >> working
> >> > >> closely with the DockerHub folks to improve the process of
> delivering
> >> > the
> >> > >> official image. However, I don't think these should become reasons
> >> that
> >> > we
> >> > >> don't release our own apache/flink images.
> >> > >>
> >> > >> Taking the 1.12.0 as an example, admittedly it would be nice for us
> >> to
> >> > >> comply with the DockerHub folks' standards and not have a
> >> > >> just-for-kubernetes command in our entrypoint. However, this is IMO
> >> far
> >> > >> less important compared to delivering the image to our users
> timely.
> >> I
> >> > >> guess that's where the DockerHub folks and us have different
> >> > >> priorities, and that's why I think we should have a path that is
> >> fully
> >> > >> controlled by this community to deliver images. We could take their
> >> > >> valuable inputs and improve afterwards. Actually, that's what we
> did
> >> for
> >> > >> 1.12.0 by starting to release to apache/flink.
> >> > >>
> >> > >> Thank you~
> >> > >>
> >> > >> Xintong Song
> >> > >>
> >> > >>
> >> > >>
> >> > >> On Thu, Apr 28, 2022 at 6:30 PM Chesnay Schepler <
> chesnay@apache.org
> >> >
> >> > >> wrote:
> >> > >>
> >> > >>> I still think that's mostly a process issue.
> >> > >>> Of course we can be blind-sided if we do the QA for a release
> >> artifact
> >> > >>> after the release has been finalized.
> >> > >>> But that's a clearly broken process from the get-go.
> >> > >>>
> >> > >>> At the very least we should already open a PR when the RC is
> >> created to
> >> > >>> get earlier feedback.
> >> > >>>
> >> > >>> Moreover, nowadays the docker images are way slimmer and we are
> much
> >> > >>> more careful on what is actually added to the scripts.
> >> > >>>
> >> > >>> Finally, the problems they found did show that their QA is very
> >> > valuable
> >> > >>> to us. And side-stepping that for such an essential piece of a
> >> release
> >> > >>> isn't a good idea imo.
> >> > >>>
> >> > >>> On 28/04/2022 11:31, Xintong Song wrote:
> >> > >>>> I'm overall against only releasing to official-images.
> >> > >>>>
> >> > >>>> We started releasing to apache/flink, in addition to the
> >> > >> official-image,
> >> > >>> in
> >> > >>>> 1.12.0. That was because releasing the official-image needs
> >> approval
> >> > >> from
> >> > >>>> the DockerHub folks, which is not under control of the Flink
> >> > community.
> >> > >>> For
> >> > >>>> 1.12.0 there were unfortunately some divergences between us and
> the
> >> > >>>> DockerHub folks, and it ended-up taking us nearly 2 months to get
> >> that
> >> > >>>> official-image PR merged [1][2]. Many users, especially those who
> >> need
> >> > >>>> Flink's K8s & Native-K8s deployment modes, were asking for the
> >> image
> >> > >>> after
> >> > >>>> 1.12.0 was announced.
> >> > >>>>
> >> > >>>> One could argue that what happened for 1.12.0 is not a regular
> >> case.
> >> > >>>> However, I'd like to point out that the docker images are not
> >> > something
> >> > >>>> nice-to-have, but a practically necessary piece of the release
> for
> >> the
> >> > >>> k8s
> >> > >>>> / native-k8s deployments to work. I'm strongly against a release
> >> > >> process
> >> > >>>> where such an important piece depends on the approval of a 3rd
> >> party.
> >> > >>>>
> >> > >>>> Thank you~
> >> > >>>>
> >> > >>>> Xintong Song
> >> > >>>>
> >> > >>>>
> >> > >>>> [1] https://issues.apache.org/jira/browse/FLINK-20650
> >> > >>>>
> >> > >>>> [2] https://github.com/docker-library/official-images/pull/9249
> >> > >>>>
> >> > >>>>
> >> > >>>>
> >> > >>>> On Thu, Apr 28, 2022 at 2:43 PM Chesnay Schepler <
> >> chesnay@apache.org>
> >> > >>> wrote:
> >> > >>>>> We could just stop releasing to apache/flink and only go for the
> >> > >>>>> official-images route.
> >> > >>>>>
> >> > >>>>> On 28/04/2022 07:43, Xintong Song wrote:
> >> > >>>>>> Forgot to mention that, we have also proposed to use one shared
> >> > >> account
> >> > >>>>> and
> >> > >>>>>> limit its access to the PMC members, like what we do with the
> >> PyPI
> >> > >>>>> account.
> >> > >>>>>> Unfortunately, INFRA rejected this proposal [1].
> >> > >>>>>>
> >> > >>>>>>
> >> > >>>>>> Thank you~
> >> > >>>>>>
> >> > >>>>>> Xintong Song
> >> > >>>>>>
> >> > >>>>>>
> >> > >>>>>> [1] https://issues.apache.org/jira/browse/INFRA-23208
> >> > >>>>>>
> >> > >>>>>> On Thu, Apr 28, 2022 at 1:39 PM Xintong Song <
> >> tonysong820@gmail.com
> >> > >
> >> > >>>>> wrote:
> >> > >>>>>>> Hi devs,
> >> > >>>>>>>
> >> > >>>>>>> I'd like to start a discussion about maintainers for DockerHub
> >> > >>>>>>> repositories under the *apache* namespace [1].
> >> > >>>>>>>
> >> > >>>>>>> Currently, the Flink community maintains various repositories
> >> > >> (flink,
> >> > >>>>>>> flink-statefun, flink-statefun-playground, and
> >> > >>>>> flink-kubernetes-operator)
> >> > >>>>>>> on DockerHub under the *apache* namespace. There's a
> limitation
> >> on
> >> > >> how
> >> > >>>>> many
> >> > >>>>>>> members the *apache* namespace can add, and recently INFRA is
> >> > >>>>> complaining
> >> > >>>>>>> about Flink taking too many places [2][3]. They would like us
> to
> >> > >>> reduce
> >> > >>>>> our
> >> > >>>>>>> maintainers from 20 now to 5.
> >> > >>>>>>>
> >> > >>>>>>> Jingsong and I would like to volunteer as two of the
> >> maintainers,
> >> > >> and
> >> > >>> we
> >> > >>>>>>> would like to learn who else wants to join us. While any
> >> committer
> >> > >> may
> >> > >>>>>>> serve as one of the maintainers, it's probably nice to also
> >> involve
> >> > >> at
> >> > >>>>>>> least one maintainer from statefun and one from
> >> > kubernetes-operator.
> >> > >>>>>>>
> >> > >>>>>>> What do you think?
> >> > >>>>>>>
> >> > >>>>>>> Thank you~
> >> > >>>>>>>
> >> > >>>>>>> Xintong Song
> >> > >>>>>>>
> >> > >>>>>>>
> >> > >>>>>>> [1] https://hub.docker.com/orgs/apache
> >> > >>>>>>>
> >> > >>>>>>> [2] https://issues.apache.org/jira/browse/INFRA-23208
> >> > >>>>>>>
> >> > >>>>>>> [3] https://issues.apache.org/jira/browse/INFRA-23213
> >> > >>>>>>>
> >> > >>>>>>>
> >> > >>>
> >> >
> >> >
> >>
> >
>
> --
> https://twitter.com/snntrable
> https://github.com/knaufk
>

Re: [DISCUSS] DockerHub repository maintainers

Posted by Konstantin Knauf <kn...@apache.org>.

Hi Xintong,

it is a pity that we can only have 5 maintainers. Every (patch) release of
flink, flink-statefun, the flink-kubernetes-operator requires a maintainer
to publish the image then, if I am not mistaken. As its mostly different
groups managing the sub-projects, this is quite the bottleneck. If we give
one seat to flink-statefun maintainers, one to the
flink-kubernetes-operator maintainers, this leaves three seats for Apache
Flink core, and there is no redundancy for the other projects. When I
managed the last two patch releases, the DockerHub access was also the
biggest hurdle. Maybe we can talk to the INFRA people again. We can
certainly reduce it, but 5 is very little.

Cheers,

Konstantin






Am Fr., 6. Mai 2022 um 09:00 Uhr schrieb Till Rohrmann <trohrmann@apache.org
>:

> Hi everyone,
>
> thanks for starting this discussion Xintong. I would volunteer as a
> maintainer of the flink-statefun Docker repository if you need one.
>
> Cheers,
> Till
>
> On Fri, May 6, 2022 at 6:22 AM Xintong Song <to...@gmail.com> wrote:
>
>> It seems to me we at least don't have a consensus on dropping the use of
>> apache namespace, which means we need to decide on a list of maintainers
>> anyway. So maybe we can get the discussion back to the maintainers. We may
>> continue the official-image vs. apache-namespace in a separate thread if
>> necessary.
>>
>> As mentioned previously, we need to reduce the number of maintainers from
>> 20 to 5, as required by INFRA. Jingsong and I would like to volunteer as 2
>> of the 5, and we would like to learn who else wants to join us. Of course
>> the list of maintainers can be modified later.
>>
>> *This also means the current maintainers may be removed from the list.*
>> Please let us know if you still need that privilege. CC-ed all the current
>> maintainers for attention.
>>
>> Thank you~
>>
>> Xintong Song
>>
>>
>>
>> On Wed, May 4, 2022 at 3:14 PM Chesnay Schepler <ch...@apache.org>
>> wrote:
>>
>> > One advantage is that the images are periodically rebuilt to get
>> > security fixes.
>> >
>> > The operator is a different story anyway because it is AFAIK only
>> > supposed to be used via docker
>> > (i.e., no standalone mode), which alleviates concerns about keeping the
>> > logic within the image
>> > to a minimum (which bit us in the past on the flink side).
>> >
>> > On 03/05/2022 16:09, Yang Wang wrote:
>> > > The flink-kubernetes-operator project is only published
>> > > via apache/flink-kubernetes-operator on docker hub and github
>> packages.
>> > > We do not find the obvious advantages by using docker hub official
>> > images.
>> > >
>> > > Best,
>> > > Yang
>> > >
>> > > Xintong Song <to...@gmail.com> 于2022年4月28日周四 19:27写道：
>> > >
>> > >> I agree with you that doing QA for the image after the release has
>> been
>> > >> finalized doesn't feel right. IIUR, that is mostly because official
>> > image
>> > >> PR needs 1) the binary release being deployed and propagated and 2)
>> the
>> > >> corresponding git commit being specified. I'm not completely sure
>> about
>> > >> this. Maybe we can improve the process by investigating more about
>> the
>> > >> feasibility of pre-verifying an official image PR before finalizing
>> the
>> > >> release. It's definitely a good thing to do if possible.
>> > >>
>> > >> I also agree that QA from DockerHub folks is valuable to us.
>> > >>
>> > >> I'm not against publishing official-images, and I'm not against
>> working
>> > >> closely with the DockerHub folks to improve the process of delivering
>> > the
>> > >> official image. However, I don't think these should become reasons
>> that
>> > we
>> > >> don't release our own apache/flink images.
>> > >>
>> > >> Taking the 1.12.0 as an example, admittedly it would be nice for us
>> to
>> > >> comply with the DockerHub folks' standards and not have a
>> > >> just-for-kubernetes command in our entrypoint. However, this is IMO
>> far
>> > >> less important compared to delivering the image to our users timely.
>> I
>> > >> guess that's where the DockerHub folks and us have different
>> > >> priorities, and that's why I think we should have a path that is
>> fully
>> > >> controlled by this community to deliver images. We could take their
>> > >> valuable inputs and improve afterwards. Actually, that's what we did
>> for
>> > >> 1.12.0 by starting to release to apache/flink.
>> > >>
>> > >> Thank you~
>> > >>
>> > >> Xintong Song
>> > >>
>> > >>
>> > >>
>> > >> On Thu, Apr 28, 2022 at 6:30 PM Chesnay Schepler <chesnay@apache.org
>> >
>> > >> wrote:
>> > >>
>> > >>> I still think that's mostly a process issue.
>> > >>> Of course we can be blind-sided if we do the QA for a release
>> artifact
>> > >>> after the release has been finalized.
>> > >>> But that's a clearly broken process from the get-go.
>> > >>>
>> > >>> At the very least we should already open a PR when the RC is
>> created to
>> > >>> get earlier feedback.
>> > >>>
>> > >>> Moreover, nowadays the docker images are way slimmer and we are much
>> > >>> more careful on what is actually added to the scripts.
>> > >>>
>> > >>> Finally, the problems they found did show that their QA is very
>> > valuable
>> > >>> to us. And side-stepping that for such an essential piece of a
>> release
>> > >>> isn't a good idea imo.
>> > >>>
>> > >>> On 28/04/2022 11:31, Xintong Song wrote:
>> > >>>> I'm overall against only releasing to official-images.
>> > >>>>
>> > >>>> We started releasing to apache/flink, in addition to the
>> > >> official-image,
>> > >>> in
>> > >>>> 1.12.0. That was because releasing the official-image needs
>> approval
>> > >> from
>> > >>>> the DockerHub folks, which is not under control of the Flink
>> > community.
>> > >>> For
>> > >>>> 1.12.0 there were unfortunately some divergences between us and the
>> > >>>> DockerHub folks, and it ended-up taking us nearly 2 months to get
>> that
>> > >>>> official-image PR merged [1][2]. Many users, especially those who
>> need
>> > >>>> Flink's K8s & Native-K8s deployment modes, were asking for the
>> image
>> > >>> after
>> > >>>> 1.12.0 was announced.
>> > >>>>
>> > >>>> One could argue that what happened for 1.12.0 is not a regular
>> case.
>> > >>>> However, I'd like to point out that the docker images are not
>> > something
>> > >>>> nice-to-have, but a practically necessary piece of the release for
>> the
>> > >>> k8s
>> > >>>> / native-k8s deployments to work. I'm strongly against a release
>> > >> process
>> > >>>> where such an important piece depends on the approval of a 3rd
>> party.
>> > >>>>
>> > >>>> Thank you~
>> > >>>>
>> > >>>> Xintong Song
>> > >>>>
>> > >>>>
>> > >>>> [1] https://issues.apache.org/jira/browse/FLINK-20650
>> > >>>>
>> > >>>> [2] https://github.com/docker-library/official-images/pull/9249
>> > >>>>
>> > >>>>
>> > >>>>
>> > >>>> On Thu, Apr 28, 2022 at 2:43 PM Chesnay Schepler <
>> chesnay@apache.org>
>> > >>> wrote:
>> > >>>>> We could just stop releasing to apache/flink and only go for the
>> > >>>>> official-images route.
>> > >>>>>
>> > >>>>> On 28/04/2022 07:43, Xintong Song wrote:
>> > >>>>>> Forgot to mention that, we have also proposed to use one shared
>> > >> account
>> > >>>>> and
>> > >>>>>> limit its access to the PMC members, like what we do with the
>> PyPI
>> > >>>>> account.
>> > >>>>>> Unfortunately, INFRA rejected this proposal [1].
>> > >>>>>>
>> > >>>>>>
>> > >>>>>> Thank you~
>> > >>>>>>
>> > >>>>>> Xintong Song
>> > >>>>>>
>> > >>>>>>
>> > >>>>>> [1] https://issues.apache.org/jira/browse/INFRA-23208
>> > >>>>>>
>> > >>>>>> On Thu, Apr 28, 2022 at 1:39 PM Xintong Song <
>> tonysong820@gmail.com
>> > >
>> > >>>>> wrote:
>> > >>>>>>> Hi devs,
>> > >>>>>>>
>> > >>>>>>> I'd like to start a discussion about maintainers for DockerHub
>> > >>>>>>> repositories under the *apache* namespace [1].
>> > >>>>>>>
>> > >>>>>>> Currently, the Flink community maintains various repositories
>> > >> (flink,
>> > >>>>>>> flink-statefun, flink-statefun-playground, and
>> > >>>>> flink-kubernetes-operator)
>> > >>>>>>> on DockerHub under the *apache* namespace. There's a limitation
>> on
>> > >> how
>> > >>>>> many
>> > >>>>>>> members the *apache* namespace can add, and recently INFRA is
>> > >>>>> complaining
>> > >>>>>>> about Flink taking too many places [2][3]. They would like us to
>> > >>> reduce
>> > >>>>> our
>> > >>>>>>> maintainers from 20 now to 5.
>> > >>>>>>>
>> > >>>>>>> Jingsong and I would like to volunteer as two of the
>> maintainers,
>> > >> and
>> > >>> we
>> > >>>>>>> would like to learn who else wants to join us. While any
>> committer
>> > >> may
>> > >>>>>>> serve as one of the maintainers, it's probably nice to also
>> involve
>> > >> at
>> > >>>>>>> least one maintainer from statefun and one from
>> > kubernetes-operator.
>> > >>>>>>>
>> > >>>>>>> What do you think?
>> > >>>>>>>
>> > >>>>>>> Thank you~
>> > >>>>>>>
>> > >>>>>>> Xintong Song
>> > >>>>>>>
>> > >>>>>>>
>> > >>>>>>> [1] https://hub.docker.com/orgs/apache
>> > >>>>>>>
>> > >>>>>>> [2] https://issues.apache.org/jira/browse/INFRA-23208
>> > >>>>>>>
>> > >>>>>>> [3] https://issues.apache.org/jira/browse/INFRA-23213
>> > >>>>>>>
>> > >>>>>>>
>> > >>>
>> >
>> >
>>
>

-- 
https://twitter.com/snntrable
https://github.com/knaufk

Re: [DISCUSS] DockerHub repository maintainers

Posted by Till Rohrmann <tr...@apache.org>.

Hi everyone,

thanks for starting this discussion Xintong. I would volunteer as a
maintainer of the flink-statefun Docker repository if you need one.

Cheers,
Till

On Fri, May 6, 2022 at 6:22 AM Xintong Song <to...@gmail.com> wrote:

> It seems to me we at least don't have a consensus on dropping the use of
> apache namespace, which means we need to decide on a list of maintainers
> anyway. So maybe we can get the discussion back to the maintainers. We may
> continue the official-image vs. apache-namespace in a separate thread if
> necessary.
>
> As mentioned previously, we need to reduce the number of maintainers from
> 20 to 5, as required by INFRA. Jingsong and I would like to volunteer as 2
> of the 5, and we would like to learn who else wants to join us. Of course
> the list of maintainers can be modified later.
>
> *This also means the current maintainers may be removed from the list.*
> Please let us know if you still need that privilege. CC-ed all the current
> maintainers for attention.
>
> Thank you~
>
> Xintong Song
>
>
>
> On Wed, May 4, 2022 at 3:14 PM Chesnay Schepler <ch...@apache.org>
> wrote:
>
> > One advantage is that the images are periodically rebuilt to get
> > security fixes.
> >
> > The operator is a different story anyway because it is AFAIK only
> > supposed to be used via docker
> > (i.e., no standalone mode), which alleviates concerns about keeping the
> > logic within the image
> > to a minimum (which bit us in the past on the flink side).
> >
> > On 03/05/2022 16:09, Yang Wang wrote:
> > > The flink-kubernetes-operator project is only published
> > > via apache/flink-kubernetes-operator on docker hub and github packages.
> > > We do not find the obvious advantages by using docker hub official
> > images.
> > >
> > > Best,
> > > Yang
> > >
> > > Xintong Song <to...@gmail.com> 于2022年4月28日周四 19:27写道：
> > >
> > >> I agree with you that doing QA for the image after the release has
> been
> > >> finalized doesn't feel right. IIUR, that is mostly because official
> > image
> > >> PR needs 1) the binary release being deployed and propagated and 2)
> the
> > >> corresponding git commit being specified. I'm not completely sure
> about
> > >> this. Maybe we can improve the process by investigating more about the
> > >> feasibility of pre-verifying an official image PR before finalizing
> the
> > >> release. It's definitely a good thing to do if possible.
> > >>
> > >> I also agree that QA from DockerHub folks is valuable to us.
> > >>
> > >> I'm not against publishing official-images, and I'm not against
> working
> > >> closely with the DockerHub folks to improve the process of delivering
> > the
> > >> official image. However, I don't think these should become reasons
> that
> > we
> > >> don't release our own apache/flink images.
> > >>
> > >> Taking the 1.12.0 as an example, admittedly it would be nice for us to
> > >> comply with the DockerHub folks' standards and not have a
> > >> just-for-kubernetes command in our entrypoint. However, this is IMO
> far
> > >> less important compared to delivering the image to our users timely. I
> > >> guess that's where the DockerHub folks and us have different
> > >> priorities, and that's why I think we should have a path that is fully
> > >> controlled by this community to deliver images. We could take their
> > >> valuable inputs and improve afterwards. Actually, that's what we did
> for
> > >> 1.12.0 by starting to release to apache/flink.
> > >>
> > >> Thank you~
> > >>
> > >> Xintong Song
> > >>
> > >>
> > >>
> > >> On Thu, Apr 28, 2022 at 6:30 PM Chesnay Schepler <ch...@apache.org>
> > >> wrote:
> > >>
> > >>> I still think that's mostly a process issue.
> > >>> Of course we can be blind-sided if we do the QA for a release
> artifact
> > >>> after the release has been finalized.
> > >>> But that's a clearly broken process from the get-go.
> > >>>
> > >>> At the very least we should already open a PR when the RC is created
> to
> > >>> get earlier feedback.
> > >>>
> > >>> Moreover, nowadays the docker images are way slimmer and we are much
> > >>> more careful on what is actually added to the scripts.
> > >>>
> > >>> Finally, the problems they found did show that their QA is very
> > valuable
> > >>> to us. And side-stepping that for such an essential piece of a
> release
> > >>> isn't a good idea imo.
> > >>>
> > >>> On 28/04/2022 11:31, Xintong Song wrote:
> > >>>> I'm overall against only releasing to official-images.
> > >>>>
> > >>>> We started releasing to apache/flink, in addition to the
> > >> official-image,
> > >>> in
> > >>>> 1.12.0. That was because releasing the official-image needs approval
> > >> from
> > >>>> the DockerHub folks, which is not under control of the Flink
> > community.
> > >>> For
> > >>>> 1.12.0 there were unfortunately some divergences between us and the
> > >>>> DockerHub folks, and it ended-up taking us nearly 2 months to get
> that
> > >>>> official-image PR merged [1][2]. Many users, especially those who
> need
> > >>>> Flink's K8s & Native-K8s deployment modes, were asking for the image
> > >>> after
> > >>>> 1.12.0 was announced.
> > >>>>
> > >>>> One could argue that what happened for 1.12.0 is not a regular case.
> > >>>> However, I'd like to point out that the docker images are not
> > something
> > >>>> nice-to-have, but a practically necessary piece of the release for
> the
> > >>> k8s
> > >>>> / native-k8s deployments to work. I'm strongly against a release
> > >> process
> > >>>> where such an important piece depends on the approval of a 3rd
> party.
> > >>>>
> > >>>> Thank you~
> > >>>>
> > >>>> Xintong Song
> > >>>>
> > >>>>
> > >>>> [1] https://issues.apache.org/jira/browse/FLINK-20650
> > >>>>
> > >>>> [2] https://github.com/docker-library/official-images/pull/9249
> > >>>>
> > >>>>
> > >>>>
> > >>>> On Thu, Apr 28, 2022 at 2:43 PM Chesnay Schepler <
> chesnay@apache.org>
> > >>> wrote:
> > >>>>> We could just stop releasing to apache/flink and only go for the
> > >>>>> official-images route.
> > >>>>>
> > >>>>> On 28/04/2022 07:43, Xintong Song wrote:
> > >>>>>> Forgot to mention that, we have also proposed to use one shared
> > >> account
> > >>>>> and
> > >>>>>> limit its access to the PMC members, like what we do with the PyPI
> > >>>>> account.
> > >>>>>> Unfortunately, INFRA rejected this proposal [1].
> > >>>>>>
> > >>>>>>
> > >>>>>> Thank you~
> > >>>>>>
> > >>>>>> Xintong Song
> > >>>>>>
> > >>>>>>
> > >>>>>> [1] https://issues.apache.org/jira/browse/INFRA-23208
> > >>>>>>
> > >>>>>> On Thu, Apr 28, 2022 at 1:39 PM Xintong Song <
> tonysong820@gmail.com
> > >
> > >>>>> wrote:
> > >>>>>>> Hi devs,
> > >>>>>>>
> > >>>>>>> I'd like to start a discussion about maintainers for DockerHub
> > >>>>>>> repositories under the *apache* namespace [1].
> > >>>>>>>
> > >>>>>>> Currently, the Flink community maintains various repositories
> > >> (flink,
> > >>>>>>> flink-statefun, flink-statefun-playground, and
> > >>>>> flink-kubernetes-operator)
> > >>>>>>> on DockerHub under the *apache* namespace. There's a limitation
> on
> > >> how
> > >>>>> many
> > >>>>>>> members the *apache* namespace can add, and recently INFRA is
> > >>>>> complaining
> > >>>>>>> about Flink taking too many places [2][3]. They would like us to
> > >>> reduce
> > >>>>> our
> > >>>>>>> maintainers from 20 now to 5.
> > >>>>>>>
> > >>>>>>> Jingsong and I would like to volunteer as two of the maintainers,
> > >> and
> > >>> we
> > >>>>>>> would like to learn who else wants to join us. While any
> committer
> > >> may
> > >>>>>>> serve as one of the maintainers, it's probably nice to also
> involve
> > >> at
> > >>>>>>> least one maintainer from statefun and one from
> > kubernetes-operator.
> > >>>>>>>
> > >>>>>>> What do you think?
> > >>>>>>>
> > >>>>>>> Thank you~
> > >>>>>>>
> > >>>>>>> Xintong Song
> > >>>>>>>
> > >>>>>>>
> > >>>>>>> [1] https://hub.docker.com/orgs/apache
> > >>>>>>>
> > >>>>>>> [2] https://issues.apache.org/jira/browse/INFRA-23208
> > >>>>>>>
> > >>>>>>> [3] https://issues.apache.org/jira/browse/INFRA-23213
> > >>>>>>>
> > >>>>>>>
> > >>>
> >
> >
>

Re: [DISCUSS] DockerHub repository maintainers

Posted by Xintong Song <to...@gmail.com>.

It seems to me we at least don't have a consensus on dropping the use of
apache namespace, which means we need to decide on a list of maintainers
anyway. So maybe we can get the discussion back to the maintainers. We may
continue the official-image vs. apache-namespace in a separate thread if
necessary.

As mentioned previously, we need to reduce the number of maintainers from
20 to 5, as required by INFRA. Jingsong and I would like to volunteer as 2
of the 5, and we would like to learn who else wants to join us. Of course
the list of maintainers can be modified later.

*This also means the current maintainers may be removed from the list.*
Please let us know if you still need that privilege. CC-ed all the current
maintainers for attention.

Thank you~

Xintong Song



On Wed, May 4, 2022 at 3:14 PM Chesnay Schepler <ch...@apache.org> wrote:

> One advantage is that the images are periodically rebuilt to get
> security fixes.
>
> The operator is a different story anyway because it is AFAIK only
> supposed to be used via docker
> (i.e., no standalone mode), which alleviates concerns about keeping the
> logic within the image
> to a minimum (which bit us in the past on the flink side).
>
> On 03/05/2022 16:09, Yang Wang wrote:
> > The flink-kubernetes-operator project is only published
> > via apache/flink-kubernetes-operator on docker hub and github packages.
> > We do not find the obvious advantages by using docker hub official
> images.
> >
> > Best,
> > Yang
> >
> > Xintong Song <to...@gmail.com> 于2022年4月28日周四 19:27写道：
> >
> >> I agree with you that doing QA for the image after the release has been
> >> finalized doesn't feel right. IIUR, that is mostly because official
> image
> >> PR needs 1) the binary release being deployed and propagated and 2) the
> >> corresponding git commit being specified. I'm not completely sure about
> >> this. Maybe we can improve the process by investigating more about the
> >> feasibility of pre-verifying an official image PR before finalizing the
> >> release. It's definitely a good thing to do if possible.
> >>
> >> I also agree that QA from DockerHub folks is valuable to us.
> >>
> >> I'm not against publishing official-images, and I'm not against working
> >> closely with the DockerHub folks to improve the process of delivering
> the
> >> official image. However, I don't think these should become reasons that
> we
> >> don't release our own apache/flink images.
> >>
> >> Taking the 1.12.0 as an example, admittedly it would be nice for us to
> >> comply with the DockerHub folks' standards and not have a
> >> just-for-kubernetes command in our entrypoint. However, this is IMO far
> >> less important compared to delivering the image to our users timely. I
> >> guess that's where the DockerHub folks and us have different
> >> priorities, and that's why I think we should have a path that is fully
> >> controlled by this community to deliver images. We could take their
> >> valuable inputs and improve afterwards. Actually, that's what we did for
> >> 1.12.0 by starting to release to apache/flink.
> >>
> >> Thank you~
> >>
> >> Xintong Song
> >>
> >>
> >>
> >> On Thu, Apr 28, 2022 at 6:30 PM Chesnay Schepler <ch...@apache.org>
> >> wrote:
> >>
> >>> I still think that's mostly a process issue.
> >>> Of course we can be blind-sided if we do the QA for a release artifact
> >>> after the release has been finalized.
> >>> But that's a clearly broken process from the get-go.
> >>>
> >>> At the very least we should already open a PR when the RC is created to
> >>> get earlier feedback.
> >>>
> >>> Moreover, nowadays the docker images are way slimmer and we are much
> >>> more careful on what is actually added to the scripts.
> >>>
> >>> Finally, the problems they found did show that their QA is very
> valuable
> >>> to us. And side-stepping that for such an essential piece of a release
> >>> isn't a good idea imo.
> >>>
> >>> On 28/04/2022 11:31, Xintong Song wrote:
> >>>> I'm overall against only releasing to official-images.
> >>>>
> >>>> We started releasing to apache/flink, in addition to the
> >> official-image,
> >>> in
> >>>> 1.12.0. That was because releasing the official-image needs approval
> >> from
> >>>> the DockerHub folks, which is not under control of the Flink
> community.
> >>> For
> >>>> 1.12.0 there were unfortunately some divergences between us and the
> >>>> DockerHub folks, and it ended-up taking us nearly 2 months to get that
> >>>> official-image PR merged [1][2]. Many users, especially those who need
> >>>> Flink's K8s & Native-K8s deployment modes, were asking for the image
> >>> after
> >>>> 1.12.0 was announced.
> >>>>
> >>>> One could argue that what happened for 1.12.0 is not a regular case.
> >>>> However, I'd like to point out that the docker images are not
> something
> >>>> nice-to-have, but a practically necessary piece of the release for the
> >>> k8s
> >>>> / native-k8s deployments to work. I'm strongly against a release
> >> process
> >>>> where such an important piece depends on the approval of a 3rd party.
> >>>>
> >>>> Thank you~
> >>>>
> >>>> Xintong Song
> >>>>
> >>>>
> >>>> [1] https://issues.apache.org/jira/browse/FLINK-20650
> >>>>
> >>>> [2] https://github.com/docker-library/official-images/pull/9249
> >>>>
> >>>>
> >>>>
> >>>> On Thu, Apr 28, 2022 at 2:43 PM Chesnay Schepler <ch...@apache.org>
> >>> wrote:
> >>>>> We could just stop releasing to apache/flink and only go for the
> >>>>> official-images route.
> >>>>>
> >>>>> On 28/04/2022 07:43, Xintong Song wrote:
> >>>>>> Forgot to mention that, we have also proposed to use one shared
> >> account
> >>>>> and
> >>>>>> limit its access to the PMC members, like what we do with the PyPI
> >>>>> account.
> >>>>>> Unfortunately, INFRA rejected this proposal [1].
> >>>>>>
> >>>>>>
> >>>>>> Thank you~
> >>>>>>
> >>>>>> Xintong Song
> >>>>>>
> >>>>>>
> >>>>>> [1] https://issues.apache.org/jira/browse/INFRA-23208
> >>>>>>
> >>>>>> On Thu, Apr 28, 2022 at 1:39 PM Xintong Song <tonysong820@gmail.com
> >
> >>>>> wrote:
> >>>>>>> Hi devs,
> >>>>>>>
> >>>>>>> I'd like to start a discussion about maintainers for DockerHub
> >>>>>>> repositories under the *apache* namespace [1].
> >>>>>>>
> >>>>>>> Currently, the Flink community maintains various repositories
> >> (flink,
> >>>>>>> flink-statefun, flink-statefun-playground, and
> >>>>> flink-kubernetes-operator)
> >>>>>>> on DockerHub under the *apache* namespace. There's a limitation on
> >> how
> >>>>> many
> >>>>>>> members the *apache* namespace can add, and recently INFRA is
> >>>>> complaining
> >>>>>>> about Flink taking too many places [2][3]. They would like us to
> >>> reduce
> >>>>> our
> >>>>>>> maintainers from 20 now to 5.
> >>>>>>>
> >>>>>>> Jingsong and I would like to volunteer as two of the maintainers,
> >> and
> >>> we
> >>>>>>> would like to learn who else wants to join us. While any committer
> >> may
> >>>>>>> serve as one of the maintainers, it's probably nice to also involve
> >> at
> >>>>>>> least one maintainer from statefun and one from
> kubernetes-operator.
> >>>>>>>
> >>>>>>> What do you think?
> >>>>>>>
> >>>>>>> Thank you~
> >>>>>>>
> >>>>>>> Xintong Song
> >>>>>>>
> >>>>>>>
> >>>>>>> [1] https://hub.docker.com/orgs/apache
> >>>>>>>
> >>>>>>> [2] https://issues.apache.org/jira/browse/INFRA-23208
> >>>>>>>
> >>>>>>> [3] https://issues.apache.org/jira/browse/INFRA-23213
> >>>>>>>
> >>>>>>>
> >>>
>
>

Re: [DISCUSS] DockerHub repository maintainers

Posted by Chesnay Schepler <ch...@apache.org>.

One advantage is that the images are periodically rebuilt to get 
security fixes.

The operator is a different story anyway because it is AFAIK only 
supposed to be used via docker
(i.e., no standalone mode), which alleviates concerns about keeping the 
logic within the image
to a minimum (which bit us in the past on the flink side).

On 03/05/2022 16:09, Yang Wang wrote:
> The flink-kubernetes-operator project is only published
> via apache/flink-kubernetes-operator on docker hub and github packages.
> We do not find the obvious advantages by using docker hub official images.
>
> Best,
> Yang
>
> Xintong Song <to...@gmail.com> 于2022年4月28日周四 19:27写道：
>
>> I agree with you that doing QA for the image after the release has been
>> finalized doesn't feel right. IIUR, that is mostly because official image
>> PR needs 1) the binary release being deployed and propagated and 2) the
>> corresponding git commit being specified. I'm not completely sure about
>> this. Maybe we can improve the process by investigating more about the
>> feasibility of pre-verifying an official image PR before finalizing the
>> release. It's definitely a good thing to do if possible.
>>
>> I also agree that QA from DockerHub folks is valuable to us.
>>
>> I'm not against publishing official-images, and I'm not against working
>> closely with the DockerHub folks to improve the process of delivering the
>> official image. However, I don't think these should become reasons that we
>> don't release our own apache/flink images.
>>
>> Taking the 1.12.0 as an example, admittedly it would be nice for us to
>> comply with the DockerHub folks' standards and not have a
>> just-for-kubernetes command in our entrypoint. However, this is IMO far
>> less important compared to delivering the image to our users timely. I
>> guess that's where the DockerHub folks and us have different
>> priorities, and that's why I think we should have a path that is fully
>> controlled by this community to deliver images. We could take their
>> valuable inputs and improve afterwards. Actually, that's what we did for
>> 1.12.0 by starting to release to apache/flink.
>>
>> Thank you~
>>
>> Xintong Song
>>
>>
>>
>> On Thu, Apr 28, 2022 at 6:30 PM Chesnay Schepler <ch...@apache.org>
>> wrote:
>>
>>> I still think that's mostly a process issue.
>>> Of course we can be blind-sided if we do the QA for a release artifact
>>> after the release has been finalized.
>>> But that's a clearly broken process from the get-go.
>>>
>>> At the very least we should already open a PR when the RC is created to
>>> get earlier feedback.
>>>
>>> Moreover, nowadays the docker images are way slimmer and we are much
>>> more careful on what is actually added to the scripts.
>>>
>>> Finally, the problems they found did show that their QA is very valuable
>>> to us. And side-stepping that for such an essential piece of a release
>>> isn't a good idea imo.
>>>
>>> On 28/04/2022 11:31, Xintong Song wrote:
>>>> I'm overall against only releasing to official-images.
>>>>
>>>> We started releasing to apache/flink, in addition to the
>> official-image,
>>> in
>>>> 1.12.0. That was because releasing the official-image needs approval
>> from
>>>> the DockerHub folks, which is not under control of the Flink community.
>>> For
>>>> 1.12.0 there were unfortunately some divergences between us and the
>>>> DockerHub folks, and it ended-up taking us nearly 2 months to get that
>>>> official-image PR merged [1][2]. Many users, especially those who need
>>>> Flink's K8s & Native-K8s deployment modes, were asking for the image
>>> after
>>>> 1.12.0 was announced.
>>>>
>>>> One could argue that what happened for 1.12.0 is not a regular case.
>>>> However, I'd like to point out that the docker images are not something
>>>> nice-to-have, but a practically necessary piece of the release for the
>>> k8s
>>>> / native-k8s deployments to work. I'm strongly against a release
>> process
>>>> where such an important piece depends on the approval of a 3rd party.
>>>>
>>>> Thank you~
>>>>
>>>> Xintong Song
>>>>
>>>>
>>>> [1] https://issues.apache.org/jira/browse/FLINK-20650
>>>>
>>>> [2] https://github.com/docker-library/official-images/pull/9249
>>>>
>>>>
>>>>
>>>> On Thu, Apr 28, 2022 at 2:43 PM Chesnay Schepler <ch...@apache.org>
>>> wrote:
>>>>> We could just stop releasing to apache/flink and only go for the
>>>>> official-images route.
>>>>>
>>>>> On 28/04/2022 07:43, Xintong Song wrote:
>>>>>> Forgot to mention that, we have also proposed to use one shared
>> account
>>>>> and
>>>>>> limit its access to the PMC members, like what we do with the PyPI
>>>>> account.
>>>>>> Unfortunately, INFRA rejected this proposal [1].
>>>>>>
>>>>>>
>>>>>> Thank you~
>>>>>>
>>>>>> Xintong Song
>>>>>>
>>>>>>
>>>>>> [1] https://issues.apache.org/jira/browse/INFRA-23208
>>>>>>
>>>>>> On Thu, Apr 28, 2022 at 1:39 PM Xintong Song <to...@gmail.com>
>>>>> wrote:
>>>>>>> Hi devs,
>>>>>>>
>>>>>>> I'd like to start a discussion about maintainers for DockerHub
>>>>>>> repositories under the *apache* namespace [1].
>>>>>>>
>>>>>>> Currently, the Flink community maintains various repositories
>> (flink,
>>>>>>> flink-statefun, flink-statefun-playground, and
>>>>> flink-kubernetes-operator)
>>>>>>> on DockerHub under the *apache* namespace. There's a limitation on
>> how
>>>>> many
>>>>>>> members the *apache* namespace can add, and recently INFRA is
>>>>> complaining
>>>>>>> about Flink taking too many places [2][3]. They would like us to
>>> reduce
>>>>> our
>>>>>>> maintainers from 20 now to 5.
>>>>>>>
>>>>>>> Jingsong and I would like to volunteer as two of the maintainers,
>> and
>>> we
>>>>>>> would like to learn who else wants to join us. While any committer
>> may
>>>>>>> serve as one of the maintainers, it's probably nice to also involve
>> at
>>>>>>> least one maintainer from statefun and one from kubernetes-operator.
>>>>>>>
>>>>>>> What do you think?
>>>>>>>
>>>>>>> Thank you~
>>>>>>>
>>>>>>> Xintong Song
>>>>>>>
>>>>>>>
>>>>>>> [1] https://hub.docker.com/orgs/apache
>>>>>>>
>>>>>>> [2] https://issues.apache.org/jira/browse/INFRA-23208
>>>>>>>
>>>>>>> [3] https://issues.apache.org/jira/browse/INFRA-23213
>>>>>>>
>>>>>>>
>>>

Re: [DISCUSS] DockerHub repository maintainers

Posted by Yang Wang <da...@gmail.com>.

The flink-kubernetes-operator project is only published
via apache/flink-kubernetes-operator on docker hub and github packages.
We do not find the obvious advantages by using docker hub official images.

Best,
Yang

Xintong Song <to...@gmail.com> 于2022年4月28日周四 19:27写道：

> I agree with you that doing QA for the image after the release has been
> finalized doesn't feel right. IIUR, that is mostly because official image
> PR needs 1) the binary release being deployed and propagated and 2) the
> corresponding git commit being specified. I'm not completely sure about
> this. Maybe we can improve the process by investigating more about the
> feasibility of pre-verifying an official image PR before finalizing the
> release. It's definitely a good thing to do if possible.
>
> I also agree that QA from DockerHub folks is valuable to us.
>
> I'm not against publishing official-images, and I'm not against working
> closely with the DockerHub folks to improve the process of delivering the
> official image. However, I don't think these should become reasons that we
> don't release our own apache/flink images.
>
> Taking the 1.12.0 as an example, admittedly it would be nice for us to
> comply with the DockerHub folks' standards and not have a
> just-for-kubernetes command in our entrypoint. However, this is IMO far
> less important compared to delivering the image to our users timely. I
> guess that's where the DockerHub folks and us have different
> priorities, and that's why I think we should have a path that is fully
> controlled by this community to deliver images. We could take their
> valuable inputs and improve afterwards. Actually, that's what we did for
> 1.12.0 by starting to release to apache/flink.
>
> Thank you~
>
> Xintong Song
>
>
>
> On Thu, Apr 28, 2022 at 6:30 PM Chesnay Schepler <ch...@apache.org>
> wrote:
>
> > I still think that's mostly a process issue.
> > Of course we can be blind-sided if we do the QA for a release artifact
> > after the release has been finalized.
> > But that's a clearly broken process from the get-go.
> >
> > At the very least we should already open a PR when the RC is created to
> > get earlier feedback.
> >
> > Moreover, nowadays the docker images are way slimmer and we are much
> > more careful on what is actually added to the scripts.
> >
> > Finally, the problems they found did show that their QA is very valuable
> > to us. And side-stepping that for such an essential piece of a release
> > isn't a good idea imo.
> >
> > On 28/04/2022 11:31, Xintong Song wrote:
> > > I'm overall against only releasing to official-images.
> > >
> > > We started releasing to apache/flink, in addition to the
> official-image,
> > in
> > > 1.12.0. That was because releasing the official-image needs approval
> from
> > > the DockerHub folks, which is not under control of the Flink community.
> > For
> > > 1.12.0 there were unfortunately some divergences between us and the
> > > DockerHub folks, and it ended-up taking us nearly 2 months to get that
> > > official-image PR merged [1][2]. Many users, especially those who need
> > > Flink's K8s & Native-K8s deployment modes, were asking for the image
> > after
> > > 1.12.0 was announced.
> > >
> > > One could argue that what happened for 1.12.0 is not a regular case.
> > > However, I'd like to point out that the docker images are not something
> > > nice-to-have, but a practically necessary piece of the release for the
> > k8s
> > > / native-k8s deployments to work. I'm strongly against a release
> process
> > > where such an important piece depends on the approval of a 3rd party.
> > >
> > > Thank you~
> > >
> > > Xintong Song
> > >
> > >
> > > [1] https://issues.apache.org/jira/browse/FLINK-20650
> > >
> > > [2] https://github.com/docker-library/official-images/pull/9249
> > >
> > >
> > >
> > > On Thu, Apr 28, 2022 at 2:43 PM Chesnay Schepler <ch...@apache.org>
> > wrote:
> > >
> > >> We could just stop releasing to apache/flink and only go for the
> > >> official-images route.
> > >>
> > >> On 28/04/2022 07:43, Xintong Song wrote:
> > >>> Forgot to mention that, we have also proposed to use one shared
> account
> > >> and
> > >>> limit its access to the PMC members, like what we do with the PyPI
> > >> account.
> > >>> Unfortunately, INFRA rejected this proposal [1].
> > >>>
> > >>>
> > >>> Thank you~
> > >>>
> > >>> Xintong Song
> > >>>
> > >>>
> > >>> [1] https://issues.apache.org/jira/browse/INFRA-23208
> > >>>
> > >>> On Thu, Apr 28, 2022 at 1:39 PM Xintong Song <to...@gmail.com>
> > >> wrote:
> > >>>> Hi devs,
> > >>>>
> > >>>> I'd like to start a discussion about maintainers for DockerHub
> > >>>> repositories under the *apache* namespace [1].
> > >>>>
> > >>>> Currently, the Flink community maintains various repositories
> (flink,
> > >>>> flink-statefun, flink-statefun-playground, and
> > >> flink-kubernetes-operator)
> > >>>> on DockerHub under the *apache* namespace. There's a limitation on
> how
> > >> many
> > >>>> members the *apache* namespace can add, and recently INFRA is
> > >> complaining
> > >>>> about Flink taking too many places [2][3]. They would like us to
> > reduce
> > >> our
> > >>>> maintainers from 20 now to 5.
> > >>>>
> > >>>> Jingsong and I would like to volunteer as two of the maintainers,
> and
> > we
> > >>>> would like to learn who else wants to join us. While any committer
> may
> > >>>> serve as one of the maintainers, it's probably nice to also involve
> at
> > >>>> least one maintainer from statefun and one from kubernetes-operator.
> > >>>>
> > >>>> What do you think?
> > >>>>
> > >>>> Thank you~
> > >>>>
> > >>>> Xintong Song
> > >>>>
> > >>>>
> > >>>> [1] https://hub.docker.com/orgs/apache
> > >>>>
> > >>>> [2] https://issues.apache.org/jira/browse/INFRA-23208
> > >>>>
> > >>>> [3] https://issues.apache.org/jira/browse/INFRA-23213
> > >>>>
> > >>>>
> > >>
> >
> >
>

Re: [DISCUSS] DockerHub repository maintainers

Posted by Xintong Song <to...@gmail.com>.

I agree with you that doing QA for the image after the release has been
finalized doesn't feel right. IIUR, that is mostly because official image
PR needs 1) the binary release being deployed and propagated and 2) the
corresponding git commit being specified. I'm not completely sure about
this. Maybe we can improve the process by investigating more about the
feasibility of pre-verifying an official image PR before finalizing the
release. It's definitely a good thing to do if possible.

I also agree that QA from DockerHub folks is valuable to us.

I'm not against publishing official-images, and I'm not against working
closely with the DockerHub folks to improve the process of delivering the
official image. However, I don't think these should become reasons that we
don't release our own apache/flink images.

Taking the 1.12.0 as an example, admittedly it would be nice for us to
comply with the DockerHub folks' standards and not have a
just-for-kubernetes command in our entrypoint. However, this is IMO far
less important compared to delivering the image to our users timely. I
guess that's where the DockerHub folks and us have different
priorities, and that's why I think we should have a path that is fully
controlled by this community to deliver images. We could take their
valuable inputs and improve afterwards. Actually, that's what we did for
1.12.0 by starting to release to apache/flink.

Thank you~

Xintong Song



On Thu, Apr 28, 2022 at 6:30 PM Chesnay Schepler <ch...@apache.org> wrote:

> I still think that's mostly a process issue.
> Of course we can be blind-sided if we do the QA for a release artifact
> after the release has been finalized.
> But that's a clearly broken process from the get-go.
>
> At the very least we should already open a PR when the RC is created to
> get earlier feedback.
>
> Moreover, nowadays the docker images are way slimmer and we are much
> more careful on what is actually added to the scripts.
>
> Finally, the problems they found did show that their QA is very valuable
> to us. And side-stepping that for such an essential piece of a release
> isn't a good idea imo.
>
> On 28/04/2022 11:31, Xintong Song wrote:
> > I'm overall against only releasing to official-images.
> >
> > We started releasing to apache/flink, in addition to the official-image,
> in
> > 1.12.0. That was because releasing the official-image needs approval from
> > the DockerHub folks, which is not under control of the Flink community.
> For
> > 1.12.0 there were unfortunately some divergences between us and the
> > DockerHub folks, and it ended-up taking us nearly 2 months to get that
> > official-image PR merged [1][2]. Many users, especially those who need
> > Flink's K8s & Native-K8s deployment modes, were asking for the image
> after
> > 1.12.0 was announced.
> >
> > One could argue that what happened for 1.12.0 is not a regular case.
> > However, I'd like to point out that the docker images are not something
> > nice-to-have, but a practically necessary piece of the release for the
> k8s
> > / native-k8s deployments to work. I'm strongly against a release process
> > where such an important piece depends on the approval of a 3rd party.
> >
> > Thank you~
> >
> > Xintong Song
> >
> >
> > [1] https://issues.apache.org/jira/browse/FLINK-20650
> >
> > [2] https://github.com/docker-library/official-images/pull/9249
> >
> >
> >
> > On Thu, Apr 28, 2022 at 2:43 PM Chesnay Schepler <ch...@apache.org>
> wrote:
> >
> >> We could just stop releasing to apache/flink and only go for the
> >> official-images route.
> >>
> >> On 28/04/2022 07:43, Xintong Song wrote:
> >>> Forgot to mention that, we have also proposed to use one shared account
> >> and
> >>> limit its access to the PMC members, like what we do with the PyPI
> >> account.
> >>> Unfortunately, INFRA rejected this proposal [1].
> >>>
> >>>
> >>> Thank you~
> >>>
> >>> Xintong Song
> >>>
> >>>
> >>> [1] https://issues.apache.org/jira/browse/INFRA-23208
> >>>
> >>> On Thu, Apr 28, 2022 at 1:39 PM Xintong Song <to...@gmail.com>
> >> wrote:
> >>>> Hi devs,
> >>>>
> >>>> I'd like to start a discussion about maintainers for DockerHub
> >>>> repositories under the *apache* namespace [1].
> >>>>
> >>>> Currently, the Flink community maintains various repositories (flink,
> >>>> flink-statefun, flink-statefun-playground, and
> >> flink-kubernetes-operator)
> >>>> on DockerHub under the *apache* namespace. There's a limitation on how
> >> many
> >>>> members the *apache* namespace can add, and recently INFRA is
> >> complaining
> >>>> about Flink taking too many places [2][3]. They would like us to
> reduce
> >> our
> >>>> maintainers from 20 now to 5.
> >>>>
> >>>> Jingsong and I would like to volunteer as two of the maintainers, and
> we
> >>>> would like to learn who else wants to join us. While any committer may
> >>>> serve as one of the maintainers, it's probably nice to also involve at
> >>>> least one maintainer from statefun and one from kubernetes-operator.
> >>>>
> >>>> What do you think?
> >>>>
> >>>> Thank you~
> >>>>
> >>>> Xintong Song
> >>>>
> >>>>
> >>>> [1] https://hub.docker.com/orgs/apache
> >>>>
> >>>> [2] https://issues.apache.org/jira/browse/INFRA-23208
> >>>>
> >>>> [3] https://issues.apache.org/jira/browse/INFRA-23213
> >>>>
> >>>>
> >>
>
>

Re: [DISCUSS] DockerHub repository maintainers

Posted by Chesnay Schepler <ch...@apache.org>.

I still think that's mostly a process issue.
Of course we can be blind-sided if we do the QA for a release artifact 
after the release has been finalized.
But that's a clearly broken process from the get-go.

At the very least we should already open a PR when the RC is created to 
get earlier feedback.

Moreover, nowadays the docker images are way slimmer and we are much 
more careful on what is actually added to the scripts.

Finally, the problems they found did show that their QA is very valuable 
to us. And side-stepping that for such an essential piece of a release 
isn't a good idea imo.

On 28/04/2022 11:31, Xintong Song wrote:
> I'm overall against only releasing to official-images.
>
> We started releasing to apache/flink, in addition to the official-image, in
> 1.12.0. That was because releasing the official-image needs approval from
> the DockerHub folks, which is not under control of the Flink community. For
> 1.12.0 there were unfortunately some divergences between us and the
> DockerHub folks, and it ended-up taking us nearly 2 months to get that
> official-image PR merged [1][2]. Many users, especially those who need
> Flink's K8s & Native-K8s deployment modes, were asking for the image after
> 1.12.0 was announced.
>
> One could argue that what happened for 1.12.0 is not a regular case.
> However, I'd like to point out that the docker images are not something
> nice-to-have, but a practically necessary piece of the release for the k8s
> / native-k8s deployments to work. I'm strongly against a release process
> where such an important piece depends on the approval of a 3rd party.
>
> Thank you~
>
> Xintong Song
>
>
> [1] https://issues.apache.org/jira/browse/FLINK-20650
>
> [2] https://github.com/docker-library/official-images/pull/9249
>
>
>
> On Thu, Apr 28, 2022 at 2:43 PM Chesnay Schepler <ch...@apache.org> wrote:
>
>> We could just stop releasing to apache/flink and only go for the
>> official-images route.
>>
>> On 28/04/2022 07:43, Xintong Song wrote:
>>> Forgot to mention that, we have also proposed to use one shared account
>> and
>>> limit its access to the PMC members, like what we do with the PyPI
>> account.
>>> Unfortunately, INFRA rejected this proposal [1].
>>>
>>>
>>> Thank you~
>>>
>>> Xintong Song
>>>
>>>
>>> [1] https://issues.apache.org/jira/browse/INFRA-23208
>>>
>>> On Thu, Apr 28, 2022 at 1:39 PM Xintong Song <to...@gmail.com>
>> wrote:
>>>> Hi devs,
>>>>
>>>> I'd like to start a discussion about maintainers for DockerHub
>>>> repositories under the *apache* namespace [1].
>>>>
>>>> Currently, the Flink community maintains various repositories (flink,
>>>> flink-statefun, flink-statefun-playground, and
>> flink-kubernetes-operator)
>>>> on DockerHub under the *apache* namespace. There's a limitation on how
>> many
>>>> members the *apache* namespace can add, and recently INFRA is
>> complaining
>>>> about Flink taking too many places [2][3]. They would like us to reduce
>> our
>>>> maintainers from 20 now to 5.
>>>>
>>>> Jingsong and I would like to volunteer as two of the maintainers, and we
>>>> would like to learn who else wants to join us. While any committer may
>>>> serve as one of the maintainers, it's probably nice to also involve at
>>>> least one maintainer from statefun and one from kubernetes-operator.
>>>>
>>>> What do you think?
>>>>
>>>> Thank you~
>>>>
>>>> Xintong Song
>>>>
>>>>
>>>> [1] https://hub.docker.com/orgs/apache
>>>>
>>>> [2] https://issues.apache.org/jira/browse/INFRA-23208
>>>>
>>>> [3] https://issues.apache.org/jira/browse/INFRA-23213
>>>>
>>>>
>>

Re: [DISCUSS] DockerHub repository maintainers

Posted by Xintong Song <to...@gmail.com>.

I'm overall against only releasing to official-images.

We started releasing to apache/flink, in addition to the official-image, in
1.12.0. That was because releasing the official-image needs approval from
the DockerHub folks, which is not under control of the Flink community. For
1.12.0 there were unfortunately some divergences between us and the
DockerHub folks, and it ended-up taking us nearly 2 months to get that
official-image PR merged [1][2]. Many users, especially those who need
Flink's K8s & Native-K8s deployment modes, were asking for the image after
1.12.0 was announced.

One could argue that what happened for 1.12.0 is not a regular case.
However, I'd like to point out that the docker images are not something
nice-to-have, but a practically necessary piece of the release for the k8s
/ native-k8s deployments to work. I'm strongly against a release process
where such an important piece depends on the approval of a 3rd party.

Thank you~

Xintong Song

[1] https://issues.apache.org/jira/browse/FLINK-20650

[2] https://github.com/docker-library/official-images/pull/9249

On Thu, Apr 28, 2022 at 2:43 PM Chesnay Schepler <ch...@apache.org> wrote:

> We could just stop releasing to apache/flink and only go for the
> official-images route.
>
> On 28/04/2022 07:43, Xintong Song wrote:
> > Forgot to mention that, we have also proposed to use one shared account
> and
> > limit its access to the PMC members, like what we do with the PyPI
> account.
> > Unfortunately, INFRA rejected this proposal [1].
> >
> >
> > Thank you~
> >
> > Xintong Song
> >
> >
> > [1] https://issues.apache.org/jira/browse/INFRA-23208
> >
> > On Thu, Apr 28, 2022 at 1:39 PM Xintong Song <to...@gmail.com>
> wrote:
> >
> >> Hi devs,
> >>
> >> I'd like to start a discussion about maintainers for DockerHub
> >> repositories under the *apache* namespace [1].
> >>
> >> Currently, the Flink community maintains various repositories (flink,
> >> flink-statefun, flink-statefun-playground, and
> flink-kubernetes-operator)
> >> on DockerHub under the *apache* namespace. There's a limitation on how
> many
> >> members the *apache* namespace can add, and recently INFRA is
> complaining
> >> about Flink taking too many places [2][3]. They would like us to reduce
> our
> >> maintainers from 20 now to 5.
> >>
> >> Jingsong and I would like to volunteer as two of the maintainers, and we
> >> would like to learn who else wants to join us. While any committer may
> >> serve as one of the maintainers, it's probably nice to also involve at
> >> least one maintainer from statefun and one from kubernetes-operator.
> >>
> >> What do you think?
> >>
> >> Thank you~
> >>
> >> Xintong Song
> >>
> >>
> >> [1] https://hub.docker.com/orgs/apache
> >>
> >> [2] https://issues.apache.org/jira/browse/INFRA-23208
> >>
> >> [3] https://issues.apache.org/jira/browse/INFRA-23213
> >>
> >>
>
>

Re: [DISCUSS] DockerHub repository maintainers

Posted by Chesnay Schepler <ch...@apache.org>.

We could just stop releasing to apache/flink and only go for the 
official-images route.

On 28/04/2022 07:43, Xintong Song wrote:
> Forgot to mention that, we have also proposed to use one shared account and
> limit its access to the PMC members, like what we do with the PyPI account.
> Unfortunately, INFRA rejected this proposal [1].
>
>
> Thank you~
>
> Xintong Song
>
>
> [1] https://issues.apache.org/jira/browse/INFRA-23208
>
> On Thu, Apr 28, 2022 at 1:39 PM Xintong Song <to...@gmail.com> wrote:
>
>> Hi devs,
>>
>> I'd like to start a discussion about maintainers for DockerHub
>> repositories under the *apache* namespace [1].
>>
>> Currently, the Flink community maintains various repositories (flink,
>> flink-statefun, flink-statefun-playground, and flink-kubernetes-operator)
>> on DockerHub under the *apache* namespace. There's a limitation on how many
>> members the *apache* namespace can add, and recently INFRA is complaining
>> about Flink taking too many places [2][3]. They would like us to reduce our
>> maintainers from 20 now to 5.
>>
>> Jingsong and I would like to volunteer as two of the maintainers, and we
>> would like to learn who else wants to join us. While any committer may
>> serve as one of the maintainers, it's probably nice to also involve at
>> least one maintainer from statefun and one from kubernetes-operator.
>>
>> What do you think?
>>
>> Thank you~
>>
>> Xintong Song
>>
>>
>> [1] https://hub.docker.com/orgs/apache
>>
>> [2] https://issues.apache.org/jira/browse/INFRA-23208
>>
>> [3] https://issues.apache.org/jira/browse/INFRA-23213
>>
>>

Re: [DISCUSS] DockerHub repository maintainers

Posted by Xintong Song <to...@gmail.com>.

Forgot to mention that, we have also proposed to use one shared account and
limit its access to the PMC members, like what we do with the PyPI account.
Unfortunately, INFRA rejected this proposal [1].


Thank you~

Xintong Song


[1] https://issues.apache.org/jira/browse/INFRA-23208

On Thu, Apr 28, 2022 at 1:39 PM Xintong Song <to...@gmail.com> wrote:

> Hi devs,
>
> I'd like to start a discussion about maintainers for DockerHub
> repositories under the *apache* namespace [1].
>
> Currently, the Flink community maintains various repositories (flink,
> flink-statefun, flink-statefun-playground, and flink-kubernetes-operator)
> on DockerHub under the *apache* namespace. There's a limitation on how many
> members the *apache* namespace can add, and recently INFRA is complaining
> about Flink taking too many places [2][3]. They would like us to reduce our
> maintainers from 20 now to 5.
>
> Jingsong and I would like to volunteer as two of the maintainers, and we
> would like to learn who else wants to join us. While any committer may
> serve as one of the maintainers, it's probably nice to also involve at
> least one maintainer from statefun and one from kubernetes-operator.
>
> What do you think?
>
> Thank you~
>
> Xintong Song
>
>
> [1] https://hub.docker.com/orgs/apache
>
> [2] https://issues.apache.org/jira/browse/INFRA-23208
>
> [3] https://issues.apache.org/jira/browse/INFRA-23213
>
>