[PROPOSAL] Transition released containers to the official ASF dockerhub organization

[Ahmet Altay <al...@google.com>]

Hi all,

I saw recent progress on the containers and wanted to bring this question
to the attention of the dev list.

Would it be possible to use the official ASF dockerhub organization for new
Beam container releases? Concretely, starting from 2.19 could we release
Beam containers to https://hub.docker.com/u/apache instead of
https://hub.docker.com/u/apachebeam ?

Ahmet

Re: [PROPOSAL] Transition released containers to the official ASF dockerhub organization

[Kenneth Knowles <ke...@apache.org>]

Perhaps we can make this part of committer onboarding. And maybe other one
time steps of the release guide.

Kenn

On Fri, Feb 14, 2020 at 10:12 AM Ahmet Altay <al...@google.com> wrote:

> How long does it take to add a new person to the list?
>
> On Fri, Feb 14, 2020 at 9:11 AM Hannah Jiang <ha...@google.com>
> wrote:
>
>> Could we ask them to buik add a list of people to add to the list? We
>>> could add all PMC members and previous release managers to the list. That
>>> might cover a good chunk of the future releases.
>>
>> Adding PMC members and previous release managers will not solve the long
>> term permission issue, because we need to add new release managers to the
>> maintainers group and only the infra team has permission to add/remove
>> members from whatever group.
>> I will add upcoming release managers to the maintainers group as well so
>> we don't need to deal with this issue at least the next three releases.
>> Let's try with this approach first and revisit it later if needed.
>>
>> On Thu, Feb 13, 2020 at 9:55 AM Robert Burke <ro...@frantil.com> wrote:
>>
>>> +1 to a bulk add. Shared account removes all accouttabillity and is at
>>> risk for abuse.
>>>
>>> As it stands, the release managers could abuse their privilege, but we'd
>>> have the opportunity to know about whodunnit.
>>>
>>> On Thu, Feb 13, 2020, 9:51 AM Robert Bradshaw <ro...@google.com>
>>> wrote:
>>>
>>>> +1, granting permission to individual accounts is preferable to trying
>>>> to share a single account.
>>>>
>>>> On Thu, Feb 13, 2020 at 9:44 AM Ahmet Altay <al...@google.com> wrote:
>>>> >
>>>> > Could we ask them to buik add a list of people to add to the list? We
>>>> could add all PMC members and previous release managers to the list. That
>>>> might cover a good chunk of the future releases.
>>>> >
>>>> > On Wed, Feb 12, 2020 at 10:10 PM Hannah Jiang <ha...@google.com>
>>>> wrote:
>>>> >>
>>>> >> Thanks everyone for supporting it.
>>>> >>
>>>> >> Yes, it's very slow to get tickets resolved by infra. I propose a
>>>> minor improvement to reduce interactions with infra.
>>>> >>
>>>> >> So far, we have granted maintainer permission(read & write) to
>>>> release managers' personal accounts. This step needs help from infra to add
>>>> new members to the group for every new release manager.
>>>> >> In order to avoid this, I proposed that we create a new account for
>>>> release purpose only and share it with release managers. The new account
>>>> will have read & write permissions to all Apache Beam docker repositories.
>>>> A password will be shared on an as-needed basis and we can change the
>>>> password periodically if needed, which is in our control. Are there any
>>>> concerns which I am not aware of with the sharing account approach?
>>>> >>
>>>> >> Thanks,
>>>> >> Hannah
>>>> >>
>>>> >>
>>>> >> On Thu, Jan 16, 2020 at 10:41 AM Kenneth Knowles <ke...@apache.org>
>>>> wrote:
>>>> >>>
>>>> >>> +1 very nice explanation
>>>> >>>
>>>> >>> On Wed, Jan 15, 2020 at 1:57 PM Ahmet Altay <al...@google.com>
>>>> wrote:
>>>> >>>>
>>>> >>>> +1 - Thank you for driving this!
>>>> >>>>
>>>> >>>> On Wed, Jan 15, 2020 at 1:55 PM Thomas Weise <th...@apache.org>
>>>> wrote:
>>>> >>>>>
>>>> >>>>> +1 for the namespace proposal.
>>>> >>>>>
>>>> >>>>> It is similar to github repos. Top-level is the org, then single
>>>> level for repo (beam-abc, beam-xzy, ..)
>>>> >>>>>
>>>> >>>>>
>>>> >>>>>
>>>> >>>>> On Wed, Jan 15, 2020 at 1:45 PM Robert Bradshaw <
>>>> robertwb@google.com> wrote:
>>>> >>>>>>
>>>> >>>>>> Various tags of the same image should at least logically be the
>>>> same
>>>> >>>>>> thing, so I agree that we should not be trying to share a single
>>>> >>>>>> repository in that way. A full suite of apache/beam-{image_desc}
>>>> >>>>>> repositories, if apache is fine with that, seems like the best
>>>> >>>>>> approach.
>>>> >>>>>>
>>>> >>>>>> On Wed, Jan 15, 2020 at 1:32 PM Kyle Weaver <kc...@google.com>
>>>> wrote:
>>>> >>>>>> >
>>>> >>>>>> > +1, agree that moving current image name to tags is a
>>>> non-starter. Thanks for driving this Hannah. Let us know what they say
>>>> about repo creation.
>>>> >>>>>> >
>>>> >>>>>> > On Wed, Jan 15, 2020 at 1:16 PM Udi Meiri <eh...@google.com>
>>>> wrote:
>>>> >>>>>> >>
>>>> >>>>>> >> SG +1
>>>> >>>>>> >>
>>>> >>>>>> >> On Wed, Jan 15, 2020 at 12:59 PM Hannah Jiang <
>>>> hannahjiang@google.com> wrote:
>>>> >>>>>> >>>
>>>> >>>>>> >>> I have done some research about images released under apache
>>>> namespace at docker hub, and here is my proposal.
>>>> >>>>>> >>>
>>>> >>>>>> >>> Currently, we are using apachebeam as our namespace and each
>>>> image has its own repository. Version number is used to tag the images.
>>>> >>>>>> >>> ie: apachebeam/python2.7_sdk:2.19.0,
>>>> apachebeam/flink1.9_job_server:2.19.0
>>>> >>>>>> >>>
>>>> >>>>>> >>> Now we are migrating to apache namespace and docker hub
>>>> doesn't support nested repository names, so we cannot use
>>>> apache/beam/{image-desc}:{version}.
>>>> >>>>>> >>> Instead, I propose to use apache/beam-{image_desc}:{version}
>>>> as our repository name.
>>>> >>>>>> >>> ie: apache/beam-python2.7_sdk:2.19.0,
>>>> apache/beam-flink1.9_job_server:2.19.0
>>>> >>>>>> >>> => When a user searches for apache/beam at docker hub, it
>>>> will list all the repositories we deployed with apache/beam-, so no
>>>> concerns that some released images are missed by users.
>>>> >>>>>> >>> => Repository names give insights to the users which
>>>> repositories they should use.
>>>> >>>>>> >>> => A downside with this approach is we need to create a new
>>>> repository whenever we release a new image, time and effort needed for this
>>>> is pending, I am contacting apache docker hub management team.
>>>> >>>>>> >>>
>>>> >>>>>> >>> I have considered using beam as repository name and moving
>>>> image name and version to tags, (ie: apache/beam:python3.7_sdk_2.19.0),
>>>> which means put all images to a single repository, however, this approach
>>>> has some downsides.
>>>> >>>>>> >>> => When a user searches for apache/beam, only one repository
>>>> is returned. Users need to use tags to identify which images they should
>>>> use. Since we release images with new tags for each version, it will
>>>> overwhelm the users and give them an impression that the images are not
>>>> organized well. It's also difficult to know what kind of images we deployed.
>>>> >>>>>> >>> => With both image name and version included at tags, it is
>>>> a little bit more complicated to maintain the code.
>>>> >>>>>> >>> => There is no correct answer which image the latest tag
>>>> should point to.
>>>> >>>>>> >>>
>>>> >>>>>> >>> Are there any concerns with this proposal?
>>>> >>>>>> >>>
>>>> >>>>>> >>> Thanks,
>>>> >>>>>> >>> Hannah
>>>> >>>>>> >>>
>>>> >>>>>> >>>
>>>> >>>>>> >>>
>>>> >>>>>> >>>
>>>> >>>>>> >>> On Fri, Jan 10, 2020 at 4:19 PM Ahmet Altay <
>>>> altay@google.com> wrote:
>>>> >>>>>> >>>>
>>>> >>>>>> >>>>
>>>> >>>>>> >>>>
>>>> >>>>>> >>>> On Fri, Jan 10, 2020 at 3:33 PM Ahmet Altay <
>>>> altay@google.com> wrote:
>>>> >>>>>> >>>>>
>>>> >>>>>> >>>>>
>>>> >>>>>> >>>>>
>>>> >>>>>> >>>>> On Fri, Jan 10, 2020 at 3:32 PM Ankur Goenka <
>>>> goenka@google.com> wrote:
>>>> >>>>>> >>>>>>
>>>> >>>>>> >>>>>> Also curious to know if apache provide any infra support
>>>> fro projects under Apache umbrella and any quota limits they might have.
>>>> >>>>>> >>>>
>>>> >>>>>> >>>>
>>>> >>>>>> >>>> Maybe Hannah can ask with an infra ticket?
>>>> >>>>>> >>>>
>>>> >>>>>> >>>>>>
>>>> >>>>>> >>>>>>
>>>> >>>>>> >>>>>> On Fri, Jan 10, 2020, 2:26 PM Robert Bradshaw <
>>>> robertwb@google.com> wrote:
>>>> >>>>>> >>>>>>>
>>>> >>>>>> >>>>>>> One downside is that, unlike many of these projects, we
>>>> release a
>>>> >>>>>> >>>>>>> dozen or so containers. Is there exactly (and only) one
>>>> level of
>>>> >>>>>> >>>>>>> namespacing/nesting we can leverage here? (This isn't a
>>>> blocker, but
>>>> >>>>>> >>>>>>> something to consider.)
>>>> >>>>>> >>>>>
>>>> >>>>>> >>>>>
>>>> >>>>>> >>>>> After a quick search, I could not find a way to use more
>>>> than one level of repositories. We can use the naming scheme we currently
>>>> use to help with. Our repositories are named as apachebeam/X, we could
>>>> start using apache/beam/X.
>>>> >>>>>> >>>>>
>>>> >>>>>> >>>>>>>
>>>> >>>>>> >>>>>>>
>>>> >>>>>> >>>>>>> On Fri, Jan 10, 2020 at 2:06 PM Hannah Jiang <
>>>> hannahjiang@google.com> wrote:
>>>> >>>>>> >>>>>>> >
>>>> >>>>>> >>>>>>> > Thanks Ahmet for proposing it.
>>>> >>>>>> >>>>>>> > I will take it and work towards v2.19.
>>>> >>>>>> >>>>
>>>> >>>>>> >>>>
>>>> >>>>>> >>>> Missed this part. Thank you Hannah!
>>>> >>>>>> >>>>
>>>> >>>>>> >>>>>>>
>>>> >>>>>> >>>>>>> >
>>>> >>>>>> >>>>>>> > Hannah
>>>> >>>>>> >>>>>>> >
>>>> >>>>>> >>>>>>> > On Fri, Jan 10, 2020 at 1:50 PM Kyle Weaver <
>>>> kcweaver@google.com> wrote:
>>>> >>>>>> >>>>>>> >>
>>>> >>>>>> >>>>>>> >> It'd be nice to have the clout/official sheen of
>>>> apache attached to our containers. Although getting the required
>>>> permissions might add some small overhead to the release process. For
>>>> example, yesterday, when we needed to create new repositories (not just
>>>> update existing ones), since we have top-level ownership of the apachebeam
>>>> organization, it was quick and easy to add them. I imagine we'd have had to
>>>> get approval from someone outside the project to do that under the apache
>>>> org. But this won't need to happen very often, so it's probably not that
>>>> big a deal.
>>>> >>>>>> >>>>>>> >>
>>>> >>>>>> >>>>>>> >> On Fri, Jan 10, 2020 at 1:40 PM Ahmet Altay <
>>>> altay@google.com> wrote:
>>>> >>>>>> >>>>>>> >>>
>>>> >>>>>> >>>>>>> >>> Hi all,
>>>> >>>>>> >>>>>>> >>>
>>>> >>>>>> >>>>>>> >>> I saw recent progress on the containers and wanted
>>>> to bring this question to the attention of the dev list.
>>>> >>>>>> >>>>>>> >>>
>>>> >>>>>> >>>>>>> >>> Would it be possible to use the official ASF
>>>> dockerhub organization for new Beam container releases? Concretely,
>>>> starting from 2.19 could we release Beam containers to
>>>> https://hub.docker.com/u/apache instead of
>>>> https://hub.docker.com/u/apachebeam ?
>>>> >>>>>> >>>>>>> >>>
>>>> >>>>>> >>>>>>> >>> Ahmet
>>>>
>>>

Re: [PROPOSAL] Transition released containers to the official ASF dockerhub organization

[Ahmet Altay <al...@google.com>]

How long does it take to add a new person to the list?

On Fri, Feb 14, 2020 at 9:11 AM Hannah Jiang <ha...@google.com> wrote:

> Could we ask them to buik add a list of people to add to the list? We
>> could add all PMC members and previous release managers to the list. That
>> might cover a good chunk of the future releases.
>
> Adding PMC members and previous release managers will not solve the long
> term permission issue, because we need to add new release managers to the
> maintainers group and only the infra team has permission to add/remove
> members from whatever group.
> I will add upcoming release managers to the maintainers group as well so
> we don't need to deal with this issue at least the next three releases.
> Let's try with this approach first and revisit it later if needed.
>
> On Thu, Feb 13, 2020 at 9:55 AM Robert Burke <ro...@frantil.com> wrote:
>
>> +1 to a bulk add. Shared account removes all accouttabillity and is at
>> risk for abuse.
>>
>> As it stands, the release managers could abuse their privilege, but we'd
>> have the opportunity to know about whodunnit.
>>
>> On Thu, Feb 13, 2020, 9:51 AM Robert Bradshaw <ro...@google.com>
>> wrote:
>>
>>> +1, granting permission to individual accounts is preferable to trying
>>> to share a single account.
>>>
>>> On Thu, Feb 13, 2020 at 9:44 AM Ahmet Altay <al...@google.com> wrote:
>>> >
>>> > Could we ask them to buik add a list of people to add to the list? We
>>> could add all PMC members and previous release managers to the list. That
>>> might cover a good chunk of the future releases.
>>> >
>>> > On Wed, Feb 12, 2020 at 10:10 PM Hannah Jiang <ha...@google.com>
>>> wrote:
>>> >>
>>> >> Thanks everyone for supporting it.
>>> >>
>>> >> Yes, it's very slow to get tickets resolved by infra. I propose a
>>> minor improvement to reduce interactions with infra.
>>> >>
>>> >> So far, we have granted maintainer permission(read & write) to
>>> release managers' personal accounts. This step needs help from infra to add
>>> new members to the group for every new release manager.
>>> >> In order to avoid this, I proposed that we create a new account for
>>> release purpose only and share it with release managers. The new account
>>> will have read & write permissions to all Apache Beam docker repositories.
>>> A password will be shared on an as-needed basis and we can change the
>>> password periodically if needed, which is in our control. Are there any
>>> concerns which I am not aware of with the sharing account approach?
>>> >>
>>> >> Thanks,
>>> >> Hannah
>>> >>
>>> >>
>>> >> On Thu, Jan 16, 2020 at 10:41 AM Kenneth Knowles <ke...@apache.org>
>>> wrote:
>>> >>>
>>> >>> +1 very nice explanation
>>> >>>
>>> >>> On Wed, Jan 15, 2020 at 1:57 PM Ahmet Altay <al...@google.com>
>>> wrote:
>>> >>>>
>>> >>>> +1 - Thank you for driving this!
>>> >>>>
>>> >>>> On Wed, Jan 15, 2020 at 1:55 PM Thomas Weise <th...@apache.org>
>>> wrote:
>>> >>>>>
>>> >>>>> +1 for the namespace proposal.
>>> >>>>>
>>> >>>>> It is similar to github repos. Top-level is the org, then single
>>> level for repo (beam-abc, beam-xzy, ..)
>>> >>>>>
>>> >>>>>
>>> >>>>>
>>> >>>>> On Wed, Jan 15, 2020 at 1:45 PM Robert Bradshaw <
>>> robertwb@google.com> wrote:
>>> >>>>>>
>>> >>>>>> Various tags of the same image should at least logically be the
>>> same
>>> >>>>>> thing, so I agree that we should not be trying to share a single
>>> >>>>>> repository in that way. A full suite of apache/beam-{image_desc}
>>> >>>>>> repositories, if apache is fine with that, seems like the best
>>> >>>>>> approach.
>>> >>>>>>
>>> >>>>>> On Wed, Jan 15, 2020 at 1:32 PM Kyle Weaver <kc...@google.com>
>>> wrote:
>>> >>>>>> >
>>> >>>>>> > +1, agree that moving current image name to tags is a
>>> non-starter. Thanks for driving this Hannah. Let us know what they say
>>> about repo creation.
>>> >>>>>> >
>>> >>>>>> > On Wed, Jan 15, 2020 at 1:16 PM Udi Meiri <eh...@google.com>
>>> wrote:
>>> >>>>>> >>
>>> >>>>>> >> SG +1
>>> >>>>>> >>
>>> >>>>>> >> On Wed, Jan 15, 2020 at 12:59 PM Hannah Jiang <
>>> hannahjiang@google.com> wrote:
>>> >>>>>> >>>
>>> >>>>>> >>> I have done some research about images released under apache
>>> namespace at docker hub, and here is my proposal.
>>> >>>>>> >>>
>>> >>>>>> >>> Currently, we are using apachebeam as our namespace and each
>>> image has its own repository. Version number is used to tag the images.
>>> >>>>>> >>> ie: apachebeam/python2.7_sdk:2.19.0,
>>> apachebeam/flink1.9_job_server:2.19.0
>>> >>>>>> >>>
>>> >>>>>> >>> Now we are migrating to apache namespace and docker hub
>>> doesn't support nested repository names, so we cannot use
>>> apache/beam/{image-desc}:{version}.
>>> >>>>>> >>> Instead, I propose to use apache/beam-{image_desc}:{version}
>>> as our repository name.
>>> >>>>>> >>> ie: apache/beam-python2.7_sdk:2.19.0,
>>> apache/beam-flink1.9_job_server:2.19.0
>>> >>>>>> >>> => When a user searches for apache/beam at docker hub, it
>>> will list all the repositories we deployed with apache/beam-, so no
>>> concerns that some released images are missed by users.
>>> >>>>>> >>> => Repository names give insights to the users which
>>> repositories they should use.
>>> >>>>>> >>> => A downside with this approach is we need to create a new
>>> repository whenever we release a new image, time and effort needed for this
>>> is pending, I am contacting apache docker hub management team.
>>> >>>>>> >>>
>>> >>>>>> >>> I have considered using beam as repository name and moving
>>> image name and version to tags, (ie: apache/beam:python3.7_sdk_2.19.0),
>>> which means put all images to a single repository, however, this approach
>>> has some downsides.
>>> >>>>>> >>> => When a user searches for apache/beam, only one repository
>>> is returned. Users need to use tags to identify which images they should
>>> use. Since we release images with new tags for each version, it will
>>> overwhelm the users and give them an impression that the images are not
>>> organized well. It's also difficult to know what kind of images we deployed.
>>> >>>>>> >>> => With both image name and version included at tags, it is a
>>> little bit more complicated to maintain the code.
>>> >>>>>> >>> => There is no correct answer which image the latest tag
>>> should point to.
>>> >>>>>> >>>
>>> >>>>>> >>> Are there any concerns with this proposal?
>>> >>>>>> >>>
>>> >>>>>> >>> Thanks,
>>> >>>>>> >>> Hannah
>>> >>>>>> >>>
>>> >>>>>> >>>
>>> >>>>>> >>>
>>> >>>>>> >>>
>>> >>>>>> >>> On Fri, Jan 10, 2020 at 4:19 PM Ahmet Altay <al...@google.com>
>>> wrote:
>>> >>>>>> >>>>
>>> >>>>>> >>>>
>>> >>>>>> >>>>
>>> >>>>>> >>>> On Fri, Jan 10, 2020 at 3:33 PM Ahmet Altay <
>>> altay@google.com> wrote:
>>> >>>>>> >>>>>
>>> >>>>>> >>>>>
>>> >>>>>> >>>>>
>>> >>>>>> >>>>> On Fri, Jan 10, 2020 at 3:32 PM Ankur Goenka <
>>> goenka@google.com> wrote:
>>> >>>>>> >>>>>>
>>> >>>>>> >>>>>> Also curious to know if apache provide any infra support
>>> fro projects under Apache umbrella and any quota limits they might have.
>>> >>>>>> >>>>
>>> >>>>>> >>>>
>>> >>>>>> >>>> Maybe Hannah can ask with an infra ticket?
>>> >>>>>> >>>>
>>> >>>>>> >>>>>>
>>> >>>>>> >>>>>>
>>> >>>>>> >>>>>> On Fri, Jan 10, 2020, 2:26 PM Robert Bradshaw <
>>> robertwb@google.com> wrote:
>>> >>>>>> >>>>>>>
>>> >>>>>> >>>>>>> One downside is that, unlike many of these projects, we
>>> release a
>>> >>>>>> >>>>>>> dozen or so containers. Is there exactly (and only) one
>>> level of
>>> >>>>>> >>>>>>> namespacing/nesting we can leverage here? (This isn't a
>>> blocker, but
>>> >>>>>> >>>>>>> something to consider.)
>>> >>>>>> >>>>>
>>> >>>>>> >>>>>
>>> >>>>>> >>>>> After a quick search, I could not find a way to use more
>>> than one level of repositories. We can use the naming scheme we currently
>>> use to help with. Our repositories are named as apachebeam/X, we could
>>> start using apache/beam/X.
>>> >>>>>> >>>>>
>>> >>>>>> >>>>>>>
>>> >>>>>> >>>>>>>
>>> >>>>>> >>>>>>> On Fri, Jan 10, 2020 at 2:06 PM Hannah Jiang <
>>> hannahjiang@google.com> wrote:
>>> >>>>>> >>>>>>> >
>>> >>>>>> >>>>>>> > Thanks Ahmet for proposing it.
>>> >>>>>> >>>>>>> > I will take it and work towards v2.19.
>>> >>>>>> >>>>
>>> >>>>>> >>>>
>>> >>>>>> >>>> Missed this part. Thank you Hannah!
>>> >>>>>> >>>>
>>> >>>>>> >>>>>>>
>>> >>>>>> >>>>>>> >
>>> >>>>>> >>>>>>> > Hannah
>>> >>>>>> >>>>>>> >
>>> >>>>>> >>>>>>> > On Fri, Jan 10, 2020 at 1:50 PM Kyle Weaver <
>>> kcweaver@google.com> wrote:
>>> >>>>>> >>>>>>> >>
>>> >>>>>> >>>>>>> >> It'd be nice to have the clout/official sheen of
>>> apache attached to our containers. Although getting the required
>>> permissions might add some small overhead to the release process. For
>>> example, yesterday, when we needed to create new repositories (not just
>>> update existing ones), since we have top-level ownership of the apachebeam
>>> organization, it was quick and easy to add them. I imagine we'd have had to
>>> get approval from someone outside the project to do that under the apache
>>> org. But this won't need to happen very often, so it's probably not that
>>> big a deal.
>>> >>>>>> >>>>>>> >>
>>> >>>>>> >>>>>>> >> On Fri, Jan 10, 2020 at 1:40 PM Ahmet Altay <
>>> altay@google.com> wrote:
>>> >>>>>> >>>>>>> >>>
>>> >>>>>> >>>>>>> >>> Hi all,
>>> >>>>>> >>>>>>> >>>
>>> >>>>>> >>>>>>> >>> I saw recent progress on the containers and wanted to
>>> bring this question to the attention of the dev list.
>>> >>>>>> >>>>>>> >>>
>>> >>>>>> >>>>>>> >>> Would it be possible to use the official ASF
>>> dockerhub organization for new Beam container releases? Concretely,
>>> starting from 2.19 could we release Beam containers to
>>> https://hub.docker.com/u/apache instead of
>>> https://hub.docker.com/u/apachebeam ?
>>> >>>>>> >>>>>>> >>>
>>> >>>>>> >>>>>>> >>> Ahmet
>>>
>>

Re: [PROPOSAL] Transition released containers to the official ASF dockerhub organization

[Hannah Jiang <ha...@google.com>]

>
> Could we ask them to buik add a list of people to add to the list? We
> could add all PMC members and previous release managers to the list. That
> might cover a good chunk of the future releases.

Adding PMC members and previous release managers will not solve the long
term permission issue, because we need to add new release managers to the
maintainers group and only the infra team has permission to add/remove
members from whatever group.
I will add upcoming release managers to the maintainers group as well so we
don't need to deal with this issue at least the next three releases.
Let's try with this approach first and revisit it later if needed.

On Thu, Feb 13, 2020 at 9:55 AM Robert Burke <ro...@frantil.com> wrote:

> +1 to a bulk add. Shared account removes all accouttabillity and is at
> risk for abuse.
>
> As it stands, the release managers could abuse their privilege, but we'd
> have the opportunity to know about whodunnit.
>
> On Thu, Feb 13, 2020, 9:51 AM Robert Bradshaw <ro...@google.com> wrote:
>
>> +1, granting permission to individual accounts is preferable to trying
>> to share a single account.
>>
>> On Thu, Feb 13, 2020 at 9:44 AM Ahmet Altay <al...@google.com> wrote:
>> >
>> > Could we ask them to buik add a list of people to add to the list? We
>> could add all PMC members and previous release managers to the list. That
>> might cover a good chunk of the future releases.
>> >
>> > On Wed, Feb 12, 2020 at 10:10 PM Hannah Jiang <ha...@google.com>
>> wrote:
>> >>
>> >> Thanks everyone for supporting it.
>> >>
>> >> Yes, it's very slow to get tickets resolved by infra. I propose a
>> minor improvement to reduce interactions with infra.
>> >>
>> >> So far, we have granted maintainer permission(read & write) to release
>> managers' personal accounts. This step needs help from infra to add new
>> members to the group for every new release manager.
>> >> In order to avoid this, I proposed that we create a new account for
>> release purpose only and share it with release managers. The new account
>> will have read & write permissions to all Apache Beam docker repositories.
>> A password will be shared on an as-needed basis and we can change the
>> password periodically if needed, which is in our control. Are there any
>> concerns which I am not aware of with the sharing account approach?
>> >>
>> >> Thanks,
>> >> Hannah
>> >>
>> >>
>> >> On Thu, Jan 16, 2020 at 10:41 AM Kenneth Knowles <ke...@apache.org>
>> wrote:
>> >>>
>> >>> +1 very nice explanation
>> >>>
>> >>> On Wed, Jan 15, 2020 at 1:57 PM Ahmet Altay <al...@google.com> wrote:
>> >>>>
>> >>>> +1 - Thank you for driving this!
>> >>>>
>> >>>> On Wed, Jan 15, 2020 at 1:55 PM Thomas Weise <th...@apache.org> wrote:
>> >>>>>
>> >>>>> +1 for the namespace proposal.
>> >>>>>
>> >>>>> It is similar to github repos. Top-level is the org, then single
>> level for repo (beam-abc, beam-xzy, ..)
>> >>>>>
>> >>>>>
>> >>>>>
>> >>>>> On Wed, Jan 15, 2020 at 1:45 PM Robert Bradshaw <
>> robertwb@google.com> wrote:
>> >>>>>>
>> >>>>>> Various tags of the same image should at least logically be the
>> same
>> >>>>>> thing, so I agree that we should not be trying to share a single
>> >>>>>> repository in that way. A full suite of apache/beam-{image_desc}
>> >>>>>> repositories, if apache is fine with that, seems like the best
>> >>>>>> approach.
>> >>>>>>
>> >>>>>> On Wed, Jan 15, 2020 at 1:32 PM Kyle Weaver <kc...@google.com>
>> wrote:
>> >>>>>> >
>> >>>>>> > +1, agree that moving current image name to tags is a
>> non-starter. Thanks for driving this Hannah. Let us know what they say
>> about repo creation.
>> >>>>>> >
>> >>>>>> > On Wed, Jan 15, 2020 at 1:16 PM Udi Meiri <eh...@google.com>
>> wrote:
>> >>>>>> >>
>> >>>>>> >> SG +1
>> >>>>>> >>
>> >>>>>> >> On Wed, Jan 15, 2020 at 12:59 PM Hannah Jiang <
>> hannahjiang@google.com> wrote:
>> >>>>>> >>>
>> >>>>>> >>> I have done some research about images released under apache
>> namespace at docker hub, and here is my proposal.
>> >>>>>> >>>
>> >>>>>> >>> Currently, we are using apachebeam as our namespace and each
>> image has its own repository. Version number is used to tag the images.
>> >>>>>> >>> ie: apachebeam/python2.7_sdk:2.19.0,
>> apachebeam/flink1.9_job_server:2.19.0
>> >>>>>> >>>
>> >>>>>> >>> Now we are migrating to apache namespace and docker hub
>> doesn't support nested repository names, so we cannot use
>> apache/beam/{image-desc}:{version}.
>> >>>>>> >>> Instead, I propose to use apache/beam-{image_desc}:{version}
>> as our repository name.
>> >>>>>> >>> ie: apache/beam-python2.7_sdk:2.19.0,
>> apache/beam-flink1.9_job_server:2.19.0
>> >>>>>> >>> => When a user searches for apache/beam at docker hub, it will
>> list all the repositories we deployed with apache/beam-, so no concerns
>> that some released images are missed by users.
>> >>>>>> >>> => Repository names give insights to the users which
>> repositories they should use.
>> >>>>>> >>> => A downside with this approach is we need to create a new
>> repository whenever we release a new image, time and effort needed for this
>> is pending, I am contacting apache docker hub management team.
>> >>>>>> >>>
>> >>>>>> >>> I have considered using beam as repository name and moving
>> image name and version to tags, (ie: apache/beam:python3.7_sdk_2.19.0),
>> which means put all images to a single repository, however, this approach
>> has some downsides.
>> >>>>>> >>> => When a user searches for apache/beam, only one repository
>> is returned. Users need to use tags to identify which images they should
>> use. Since we release images with new tags for each version, it will
>> overwhelm the users and give them an impression that the images are not
>> organized well. It's also difficult to know what kind of images we deployed.
>> >>>>>> >>> => With both image name and version included at tags, it is a
>> little bit more complicated to maintain the code.
>> >>>>>> >>> => There is no correct answer which image the latest tag
>> should point to.
>> >>>>>> >>>
>> >>>>>> >>> Are there any concerns with this proposal?
>> >>>>>> >>>
>> >>>>>> >>> Thanks,
>> >>>>>> >>> Hannah
>> >>>>>> >>>
>> >>>>>> >>>
>> >>>>>> >>>
>> >>>>>> >>>
>> >>>>>> >>> On Fri, Jan 10, 2020 at 4:19 PM Ahmet Altay <al...@google.com>
>> wrote:
>> >>>>>> >>>>
>> >>>>>> >>>>
>> >>>>>> >>>>
>> >>>>>> >>>> On Fri, Jan 10, 2020 at 3:33 PM Ahmet Altay <al...@google.com>
>> wrote:
>> >>>>>> >>>>>
>> >>>>>> >>>>>
>> >>>>>> >>>>>
>> >>>>>> >>>>> On Fri, Jan 10, 2020 at 3:32 PM Ankur Goenka <
>> goenka@google.com> wrote:
>> >>>>>> >>>>>>
>> >>>>>> >>>>>> Also curious to know if apache provide any infra support
>> fro projects under Apache umbrella and any quota limits they might have.
>> >>>>>> >>>>
>> >>>>>> >>>>
>> >>>>>> >>>> Maybe Hannah can ask with an infra ticket?
>> >>>>>> >>>>
>> >>>>>> >>>>>>
>> >>>>>> >>>>>>
>> >>>>>> >>>>>> On Fri, Jan 10, 2020, 2:26 PM Robert Bradshaw <
>> robertwb@google.com> wrote:
>> >>>>>> >>>>>>>
>> >>>>>> >>>>>>> One downside is that, unlike many of these projects, we
>> release a
>> >>>>>> >>>>>>> dozen or so containers. Is there exactly (and only) one
>> level of
>> >>>>>> >>>>>>> namespacing/nesting we can leverage here? (This isn't a
>> blocker, but
>> >>>>>> >>>>>>> something to consider.)
>> >>>>>> >>>>>
>> >>>>>> >>>>>
>> >>>>>> >>>>> After a quick search, I could not find a way to use more
>> than one level of repositories. We can use the naming scheme we currently
>> use to help with. Our repositories are named as apachebeam/X, we could
>> start using apache/beam/X.
>> >>>>>> >>>>>
>> >>>>>> >>>>>>>
>> >>>>>> >>>>>>>
>> >>>>>> >>>>>>> On Fri, Jan 10, 2020 at 2:06 PM Hannah Jiang <
>> hannahjiang@google.com> wrote:
>> >>>>>> >>>>>>> >
>> >>>>>> >>>>>>> > Thanks Ahmet for proposing it.
>> >>>>>> >>>>>>> > I will take it and work towards v2.19.
>> >>>>>> >>>>
>> >>>>>> >>>>
>> >>>>>> >>>> Missed this part. Thank you Hannah!
>> >>>>>> >>>>
>> >>>>>> >>>>>>>
>> >>>>>> >>>>>>> >
>> >>>>>> >>>>>>> > Hannah
>> >>>>>> >>>>>>> >
>> >>>>>> >>>>>>> > On Fri, Jan 10, 2020 at 1:50 PM Kyle Weaver <
>> kcweaver@google.com> wrote:
>> >>>>>> >>>>>>> >>
>> >>>>>> >>>>>>> >> It'd be nice to have the clout/official sheen of apache
>> attached to our containers. Although getting the required permissions might
>> add some small overhead to the release process. For example, yesterday,
>> when we needed to create new repositories (not just update existing ones),
>> since we have top-level ownership of the apachebeam organization, it was
>> quick and easy to add them. I imagine we'd have had to get approval from
>> someone outside the project to do that under the apache org. But this won't
>> need to happen very often, so it's probably not that big a deal.
>> >>>>>> >>>>>>> >>
>> >>>>>> >>>>>>> >> On Fri, Jan 10, 2020 at 1:40 PM Ahmet Altay <
>> altay@google.com> wrote:
>> >>>>>> >>>>>>> >>>
>> >>>>>> >>>>>>> >>> Hi all,
>> >>>>>> >>>>>>> >>>
>> >>>>>> >>>>>>> >>> I saw recent progress on the containers and wanted to
>> bring this question to the attention of the dev list.
>> >>>>>> >>>>>>> >>>
>> >>>>>> >>>>>>> >>> Would it be possible to use the official ASF dockerhub
>> organization for new Beam container releases? Concretely, starting from
>> 2.19 could we release Beam containers to https://hub.docker.com/u/apache
>> instead of https://hub.docker.com/u/apachebeam ?
>> >>>>>> >>>>>>> >>>
>> >>>>>> >>>>>>> >>> Ahmet
>>
>

Re: [PROPOSAL] Transition released containers to the official ASF dockerhub organization

[Robert Burke <ro...@frantil.com>]

+1 to a bulk add. Shared account removes all accouttabillity and is at risk
for abuse.

As it stands, the release managers could abuse their privilege, but we'd
have the opportunity to know about whodunnit.

On Thu, Feb 13, 2020, 9:51 AM Robert Bradshaw <ro...@google.com> wrote:

> +1, granting permission to individual accounts is preferable to trying
> to share a single account.
>
> On Thu, Feb 13, 2020 at 9:44 AM Ahmet Altay <al...@google.com> wrote:
> >
> > Could we ask them to buik add a list of people to add to the list? We
> could add all PMC members and previous release managers to the list. That
> might cover a good chunk of the future releases.
> >
> > On Wed, Feb 12, 2020 at 10:10 PM Hannah Jiang <ha...@google.com>
> wrote:
> >>
> >> Thanks everyone for supporting it.
> >>
> >> Yes, it's very slow to get tickets resolved by infra. I propose a minor
> improvement to reduce interactions with infra.
> >>
> >> So far, we have granted maintainer permission(read & write) to release
> managers' personal accounts. This step needs help from infra to add new
> members to the group for every new release manager.
> >> In order to avoid this, I proposed that we create a new account for
> release purpose only and share it with release managers. The new account
> will have read & write permissions to all Apache Beam docker repositories.
> A password will be shared on an as-needed basis and we can change the
> password periodically if needed, which is in our control. Are there any
> concerns which I am not aware of with the sharing account approach?
> >>
> >> Thanks,
> >> Hannah
> >>
> >>
> >> On Thu, Jan 16, 2020 at 10:41 AM Kenneth Knowles <ke...@apache.org>
> wrote:
> >>>
> >>> +1 very nice explanation
> >>>
> >>> On Wed, Jan 15, 2020 at 1:57 PM Ahmet Altay <al...@google.com> wrote:
> >>>>
> >>>> +1 - Thank you for driving this!
> >>>>
> >>>> On Wed, Jan 15, 2020 at 1:55 PM Thomas Weise <th...@apache.org> wrote:
> >>>>>
> >>>>> +1 for the namespace proposal.
> >>>>>
> >>>>> It is similar to github repos. Top-level is the org, then single
> level for repo (beam-abc, beam-xzy, ..)
> >>>>>
> >>>>>
> >>>>>
> >>>>> On Wed, Jan 15, 2020 at 1:45 PM Robert Bradshaw <ro...@google.com>
> wrote:
> >>>>>>
> >>>>>> Various tags of the same image should at least logically be the same
> >>>>>> thing, so I agree that we should not be trying to share a single
> >>>>>> repository in that way. A full suite of apache/beam-{image_desc}
> >>>>>> repositories, if apache is fine with that, seems like the best
> >>>>>> approach.
> >>>>>>
> >>>>>> On Wed, Jan 15, 2020 at 1:32 PM Kyle Weaver <kc...@google.com>
> wrote:
> >>>>>> >
> >>>>>> > +1, agree that moving current image name to tags is a
> non-starter. Thanks for driving this Hannah. Let us know what they say
> about repo creation.
> >>>>>> >
> >>>>>> > On Wed, Jan 15, 2020 at 1:16 PM Udi Meiri <eh...@google.com>
> wrote:
> >>>>>> >>
> >>>>>> >> SG +1
> >>>>>> >>
> >>>>>> >> On Wed, Jan 15, 2020 at 12:59 PM Hannah Jiang <
> hannahjiang@google.com> wrote:
> >>>>>> >>>
> >>>>>> >>> I have done some research about images released under apache
> namespace at docker hub, and here is my proposal.
> >>>>>> >>>
> >>>>>> >>> Currently, we are using apachebeam as our namespace and each
> image has its own repository. Version number is used to tag the images.
> >>>>>> >>> ie: apachebeam/python2.7_sdk:2.19.0,
> apachebeam/flink1.9_job_server:2.19.0
> >>>>>> >>>
> >>>>>> >>> Now we are migrating to apache namespace and docker hub doesn't
> support nested repository names, so we cannot use
> apache/beam/{image-desc}:{version}.
> >>>>>> >>> Instead, I propose to use apache/beam-{image_desc}:{version} as
> our repository name.
> >>>>>> >>> ie: apache/beam-python2.7_sdk:2.19.0,
> apache/beam-flink1.9_job_server:2.19.0
> >>>>>> >>> => When a user searches for apache/beam at docker hub, it will
> list all the repositories we deployed with apache/beam-, so no concerns
> that some released images are missed by users.
> >>>>>> >>> => Repository names give insights to the users which
> repositories they should use.
> >>>>>> >>> => A downside with this approach is we need to create a new
> repository whenever we release a new image, time and effort needed for this
> is pending, I am contacting apache docker hub management team.
> >>>>>> >>>
> >>>>>> >>> I have considered using beam as repository name and moving
> image name and version to tags, (ie: apache/beam:python3.7_sdk_2.19.0),
> which means put all images to a single repository, however, this approach
> has some downsides.
> >>>>>> >>> => When a user searches for apache/beam, only one repository is
> returned. Users need to use tags to identify which images they should use.
> Since we release images with new tags for each version, it will overwhelm
> the users and give them an impression that the images are not organized
> well. It's also difficult to know what kind of images we deployed.
> >>>>>> >>> => With both image name and version included at tags, it is a
> little bit more complicated to maintain the code.
> >>>>>> >>> => There is no correct answer which image the latest tag should
> point to.
> >>>>>> >>>
> >>>>>> >>> Are there any concerns with this proposal?
> >>>>>> >>>
> >>>>>> >>> Thanks,
> >>>>>> >>> Hannah
> >>>>>> >>>
> >>>>>> >>>
> >>>>>> >>>
> >>>>>> >>>
> >>>>>> >>> On Fri, Jan 10, 2020 at 4:19 PM Ahmet Altay <al...@google.com>
> wrote:
> >>>>>> >>>>
> >>>>>> >>>>
> >>>>>> >>>>
> >>>>>> >>>> On Fri, Jan 10, 2020 at 3:33 PM Ahmet Altay <al...@google.com>
> wrote:
> >>>>>> >>>>>
> >>>>>> >>>>>
> >>>>>> >>>>>
> >>>>>> >>>>> On Fri, Jan 10, 2020 at 3:32 PM Ankur Goenka <
> goenka@google.com> wrote:
> >>>>>> >>>>>>
> >>>>>> >>>>>> Also curious to know if apache provide any infra support fro
> projects under Apache umbrella and any quota limits they might have.
> >>>>>> >>>>
> >>>>>> >>>>
> >>>>>> >>>> Maybe Hannah can ask with an infra ticket?
> >>>>>> >>>>
> >>>>>> >>>>>>
> >>>>>> >>>>>>
> >>>>>> >>>>>> On Fri, Jan 10, 2020, 2:26 PM Robert Bradshaw <
> robertwb@google.com> wrote:
> >>>>>> >>>>>>>
> >>>>>> >>>>>>> One downside is that, unlike many of these projects, we
> release a
> >>>>>> >>>>>>> dozen or so containers. Is there exactly (and only) one
> level of
> >>>>>> >>>>>>> namespacing/nesting we can leverage here? (This isn't a
> blocker, but
> >>>>>> >>>>>>> something to consider.)
> >>>>>> >>>>>
> >>>>>> >>>>>
> >>>>>> >>>>> After a quick search, I could not find a way to use more than
> one level of repositories. We can use the naming scheme we currently use to
> help with. Our repositories are named as apachebeam/X, we could start using
> apache/beam/X.
> >>>>>> >>>>>
> >>>>>> >>>>>>>
> >>>>>> >>>>>>>
> >>>>>> >>>>>>> On Fri, Jan 10, 2020 at 2:06 PM Hannah Jiang <
> hannahjiang@google.com> wrote:
> >>>>>> >>>>>>> >
> >>>>>> >>>>>>> > Thanks Ahmet for proposing it.
> >>>>>> >>>>>>> > I will take it and work towards v2.19.
> >>>>>> >>>>
> >>>>>> >>>>
> >>>>>> >>>> Missed this part. Thank you Hannah!
> >>>>>> >>>>
> >>>>>> >>>>>>>
> >>>>>> >>>>>>> >
> >>>>>> >>>>>>> > Hannah
> >>>>>> >>>>>>> >
> >>>>>> >>>>>>> > On Fri, Jan 10, 2020 at 1:50 PM Kyle Weaver <
> kcweaver@google.com> wrote:
> >>>>>> >>>>>>> >>
> >>>>>> >>>>>>> >> It'd be nice to have the clout/official sheen of apache
> attached to our containers. Although getting the required permissions might
> add some small overhead to the release process. For example, yesterday,
> when we needed to create new repositories (not just update existing ones),
> since we have top-level ownership of the apachebeam organization, it was
> quick and easy to add them. I imagine we'd have had to get approval from
> someone outside the project to do that under the apache org. But this won't
> need to happen very often, so it's probably not that big a deal.
> >>>>>> >>>>>>> >>
> >>>>>> >>>>>>> >> On Fri, Jan 10, 2020 at 1:40 PM Ahmet Altay <
> altay@google.com> wrote:
> >>>>>> >>>>>>> >>>
> >>>>>> >>>>>>> >>> Hi all,
> >>>>>> >>>>>>> >>>
> >>>>>> >>>>>>> >>> I saw recent progress on the containers and wanted to
> bring this question to the attention of the dev list.
> >>>>>> >>>>>>> >>>
> >>>>>> >>>>>>> >>> Would it be possible to use the official ASF dockerhub
> organization for new Beam container releases? Concretely, starting from
> 2.19 could we release Beam containers to https://hub.docker.com/u/apache
> instead of https://hub.docker.com/u/apachebeam ?
> >>>>>> >>>>>>> >>>
> >>>>>> >>>>>>> >>> Ahmet
>

Re: [PROPOSAL] Transition released containers to the official ASF dockerhub organization

[Robert Bradshaw <ro...@google.com>]

+1, granting permission to individual accounts is preferable to trying
to share a single account.

On Thu, Feb 13, 2020 at 9:44 AM Ahmet Altay <al...@google.com> wrote:
>
> Could we ask them to buik add a list of people to add to the list? We could add all PMC members and previous release managers to the list. That might cover a good chunk of the future releases.
>
> On Wed, Feb 12, 2020 at 10:10 PM Hannah Jiang <ha...@google.com> wrote:
>>
>> Thanks everyone for supporting it.
>>
>> Yes, it's very slow to get tickets resolved by infra. I propose a minor improvement to reduce interactions with infra.
>>
>> So far, we have granted maintainer permission(read & write) to release managers' personal accounts. This step needs help from infra to add new members to the group for every new release manager.
>> In order to avoid this, I proposed that we create a new account for release purpose only and share it with release managers. The new account will have read & write permissions to all Apache Beam docker repositories. A password will be shared on an as-needed basis and we can change the password periodically if needed, which is in our control. Are there any concerns which I am not aware of with the sharing account approach?
>>
>> Thanks,
>> Hannah
>>
>>
>> On Thu, Jan 16, 2020 at 10:41 AM Kenneth Knowles <ke...@apache.org> wrote:
>>>
>>> +1 very nice explanation
>>>
>>> On Wed, Jan 15, 2020 at 1:57 PM Ahmet Altay <al...@google.com> wrote:
>>>>
>>>> +1 - Thank you for driving this!
>>>>
>>>> On Wed, Jan 15, 2020 at 1:55 PM Thomas Weise <th...@apache.org> wrote:
>>>>>
>>>>> +1 for the namespace proposal.
>>>>>
>>>>> It is similar to github repos. Top-level is the org, then single level for repo (beam-abc, beam-xzy, ..)
>>>>>
>>>>>
>>>>>
>>>>> On Wed, Jan 15, 2020 at 1:45 PM Robert Bradshaw <ro...@google.com> wrote:
>>>>>>
>>>>>> Various tags of the same image should at least logically be the same
>>>>>> thing, so I agree that we should not be trying to share a single
>>>>>> repository in that way. A full suite of apache/beam-{image_desc}
>>>>>> repositories, if apache is fine with that, seems like the best
>>>>>> approach.
>>>>>>
>>>>>> On Wed, Jan 15, 2020 at 1:32 PM Kyle Weaver <kc...@google.com> wrote:
>>>>>> >
>>>>>> > +1, agree that moving current image name to tags is a non-starter. Thanks for driving this Hannah. Let us know what they say about repo creation.
>>>>>> >
>>>>>> > On Wed, Jan 15, 2020 at 1:16 PM Udi Meiri <eh...@google.com> wrote:
>>>>>> >>
>>>>>> >> SG +1
>>>>>> >>
>>>>>> >> On Wed, Jan 15, 2020 at 12:59 PM Hannah Jiang <ha...@google.com> wrote:
>>>>>> >>>
>>>>>> >>> I have done some research about images released under apache namespace at docker hub, and here is my proposal.
>>>>>> >>>
>>>>>> >>> Currently, we are using apachebeam as our namespace and each image has its own repository. Version number is used to tag the images.
>>>>>> >>> ie: apachebeam/python2.7_sdk:2.19.0, apachebeam/flink1.9_job_server:2.19.0
>>>>>> >>>
>>>>>> >>> Now we are migrating to apache namespace and docker hub doesn't support nested repository names, so we cannot use apache/beam/{image-desc}:{version}.
>>>>>> >>> Instead, I propose to use apache/beam-{image_desc}:{version} as our repository name.
>>>>>> >>> ie: apache/beam-python2.7_sdk:2.19.0, apache/beam-flink1.9_job_server:2.19.0
>>>>>> >>> => When a user searches for apache/beam at docker hub, it will list all the repositories we deployed with apache/beam-, so no concerns that some released images are missed by users.
>>>>>> >>> => Repository names give insights to the users which repositories they should use.
>>>>>> >>> => A downside with this approach is we need to create a new repository whenever we release a new image, time and effort needed for this is pending, I am contacting apache docker hub management team.
>>>>>> >>>
>>>>>> >>> I have considered using beam as repository name and moving image name and version to tags, (ie: apache/beam:python3.7_sdk_2.19.0), which means put all images to a single repository, however, this approach has some downsides.
>>>>>> >>> => When a user searches for apache/beam, only one repository is returned. Users need to use tags to identify which images they should use. Since we release images with new tags for each version, it will overwhelm the users and give them an impression that the images are not organized well. It's also difficult to know what kind of images we deployed.
>>>>>> >>> => With both image name and version included at tags, it is a little bit more complicated to maintain the code.
>>>>>> >>> => There is no correct answer which image the latest tag should point to.
>>>>>> >>>
>>>>>> >>> Are there any concerns with this proposal?
>>>>>> >>>
>>>>>> >>> Thanks,
>>>>>> >>> Hannah
>>>>>> >>>
>>>>>> >>>
>>>>>> >>>
>>>>>> >>>
>>>>>> >>> On Fri, Jan 10, 2020 at 4:19 PM Ahmet Altay <al...@google.com> wrote:
>>>>>> >>>>
>>>>>> >>>>
>>>>>> >>>>
>>>>>> >>>> On Fri, Jan 10, 2020 at 3:33 PM Ahmet Altay <al...@google.com> wrote:
>>>>>> >>>>>
>>>>>> >>>>>
>>>>>> >>>>>
>>>>>> >>>>> On Fri, Jan 10, 2020 at 3:32 PM Ankur Goenka <go...@google.com> wrote:
>>>>>> >>>>>>
>>>>>> >>>>>> Also curious to know if apache provide any infra support fro projects under Apache umbrella and any quota limits they might have.
>>>>>> >>>>
>>>>>> >>>>
>>>>>> >>>> Maybe Hannah can ask with an infra ticket?
>>>>>> >>>>
>>>>>> >>>>>>
>>>>>> >>>>>>
>>>>>> >>>>>> On Fri, Jan 10, 2020, 2:26 PM Robert Bradshaw <ro...@google.com> wrote:
>>>>>> >>>>>>>
>>>>>> >>>>>>> One downside is that, unlike many of these projects, we release a
>>>>>> >>>>>>> dozen or so containers. Is there exactly (and only) one level of
>>>>>> >>>>>>> namespacing/nesting we can leverage here? (This isn't a blocker, but
>>>>>> >>>>>>> something to consider.)
>>>>>> >>>>>
>>>>>> >>>>>
>>>>>> >>>>> After a quick search, I could not find a way to use more than one level of repositories. We can use the naming scheme we currently use to help with. Our repositories are named as apachebeam/X, we could start using apache/beam/X.
>>>>>> >>>>>
>>>>>> >>>>>>>
>>>>>> >>>>>>>
>>>>>> >>>>>>> On Fri, Jan 10, 2020 at 2:06 PM Hannah Jiang <ha...@google.com> wrote:
>>>>>> >>>>>>> >
>>>>>> >>>>>>> > Thanks Ahmet for proposing it.
>>>>>> >>>>>>> > I will take it and work towards v2.19.
>>>>>> >>>>
>>>>>> >>>>
>>>>>> >>>> Missed this part. Thank you Hannah!
>>>>>> >>>>
>>>>>> >>>>>>>
>>>>>> >>>>>>> >
>>>>>> >>>>>>> > Hannah
>>>>>> >>>>>>> >
>>>>>> >>>>>>> > On Fri, Jan 10, 2020 at 1:50 PM Kyle Weaver <kc...@google.com> wrote:
>>>>>> >>>>>>> >>
>>>>>> >>>>>>> >> It'd be nice to have the clout/official sheen of apache attached to our containers. Although getting the required permissions might add some small overhead to the release process. For example, yesterday, when we needed to create new repositories (not just update existing ones), since we have top-level ownership of the apachebeam organization, it was quick and easy to add them. I imagine we'd have had to get approval from someone outside the project to do that under the apache org. But this won't need to happen very often, so it's probably not that big a deal.
>>>>>> >>>>>>> >>
>>>>>> >>>>>>> >> On Fri, Jan 10, 2020 at 1:40 PM Ahmet Altay <al...@google.com> wrote:
>>>>>> >>>>>>> >>>
>>>>>> >>>>>>> >>> Hi all,
>>>>>> >>>>>>> >>>
>>>>>> >>>>>>> >>> I saw recent progress on the containers and wanted to bring this question to the attention of the dev list.
>>>>>> >>>>>>> >>>
>>>>>> >>>>>>> >>> Would it be possible to use the official ASF dockerhub organization for new Beam container releases? Concretely, starting from 2.19 could we release Beam containers to https://hub.docker.com/u/apache instead of https://hub.docker.com/u/apachebeam ?
>>>>>> >>>>>>> >>>
>>>>>> >>>>>>> >>> Ahmet

Re: [PROPOSAL] Transition released containers to the official ASF dockerhub organization

[Ahmet Altay <al...@google.com>]

Could we ask them to buik add a list of people to add to the list? We could
add all PMC members and previous release managers to the list. That might
cover a good chunk of the future releases.

On Wed, Feb 12, 2020 at 10:10 PM Hannah Jiang <ha...@google.com>
wrote:

> Thanks everyone for supporting it.
>
> Yes, it's very slow to get tickets resolved by infra. I propose a minor
> improvement to reduce interactions with infra.
>
> So far, we have granted maintainer permission(read & write) to release
> managers' personal accounts. This step needs help from infra to add new
> members to the group for every new release manager.
> In order to avoid this, I proposed that we create a new account for
> release purpose only and share it with release managers. The new account
> will have read & write permissions to all Apache Beam docker repositories.
> A password will be shared on an as-needed basis and we can change the
> password periodically if needed, which is in our control. Are there any
> concerns which I am not aware of with the sharing account approach?
>
> Thanks,
> Hannah
>
>
> On Thu, Jan 16, 2020 at 10:41 AM Kenneth Knowles <ke...@apache.org> wrote:
>
>> +1 very nice explanation
>>
>> On Wed, Jan 15, 2020 at 1:57 PM Ahmet Altay <al...@google.com> wrote:
>>
>>> +1 - Thank you for driving this!
>>>
>>> On Wed, Jan 15, 2020 at 1:55 PM Thomas Weise <th...@apache.org> wrote:
>>>
>>>> +1 for the namespace proposal.
>>>>
>>>> It is similar to github repos. Top-level is the org, then single level
>>>> for repo (beam-abc, beam-xzy, ..)
>>>>
>>>>
>>>>
>>>> On Wed, Jan 15, 2020 at 1:45 PM Robert Bradshaw <ro...@google.com>
>>>> wrote:
>>>>
>>>>> Various tags of the same image should at least logically be the same
>>>>> thing, so I agree that we should not be trying to share a single
>>>>> repository in that way. A full suite of apache/beam-{image_desc}
>>>>> repositories, if apache is fine with that, seems like the best
>>>>> approach.
>>>>>
>>>>> On Wed, Jan 15, 2020 at 1:32 PM Kyle Weaver <kc...@google.com>
>>>>> wrote:
>>>>> >
>>>>> > +1, agree that moving current image name to tags is a non-starter.
>>>>> Thanks for driving this Hannah. Let us know what they say about repo
>>>>> creation.
>>>>> >
>>>>> > On Wed, Jan 15, 2020 at 1:16 PM Udi Meiri <eh...@google.com> wrote:
>>>>> >>
>>>>> >> SG +1
>>>>> >>
>>>>> >> On Wed, Jan 15, 2020 at 12:59 PM Hannah Jiang <
>>>>> hannahjiang@google.com> wrote:
>>>>> >>>
>>>>> >>> I have done some research about images released under apache
>>>>> namespace at docker hub, and here is my proposal.
>>>>> >>>
>>>>> >>> Currently, we are using apachebeam as our namespace and each image
>>>>> has its own repository. Version number is used to tag the images.
>>>>> >>> ie: apachebeam/python2.7_sdk:2.19.0,
>>>>> apachebeam/flink1.9_job_server:2.19.0
>>>>> >>>
>>>>> >>> Now we are migrating to apache namespace and docker hub doesn't
>>>>> support nested repository names, so we cannot use
>>>>> apache/beam/{image-desc}:{version}.
>>>>> >>> Instead, I propose to use apache/beam-{image_desc}:{version} as
>>>>> our repository name.
>>>>> >>> ie: apache/beam-python2.7_sdk:2.19.0,
>>>>> apache/beam-flink1.9_job_server:2.19.0
>>>>> >>> => When a user searches for apache/beam at docker hub, it will
>>>>> list all the repositories we deployed with apache/beam-, so no concerns
>>>>> that some released images are missed by users.
>>>>> >>> => Repository names give insights to the users which repositories
>>>>> they should use.
>>>>> >>> => A downside with this approach is we need to create a new
>>>>> repository whenever we release a new image, time and effort needed for this
>>>>> is pending, I am contacting apache docker hub management team.
>>>>> >>>
>>>>> >>> I have considered using beam as repository name and moving image
>>>>> name and version to tags, (ie: apache/beam:python3.7_sdk_2.19.0), which
>>>>> means put all images to a single repository, however, this approach has
>>>>> some downsides.
>>>>> >>> => When a user searches for apache/beam, only one repository is
>>>>> returned. Users need to use tags to identify which images they should use.
>>>>> Since we release images with new tags for each version, it will overwhelm
>>>>> the users and give them an impression that the images are not organized
>>>>> well. It's also difficult to know what kind of images we deployed.
>>>>> >>> => With both image name and version included at tags, it is a
>>>>> little bit more complicated to maintain the code.
>>>>> >>> => There is no correct answer which image the latest tag should
>>>>> point to.
>>>>> >>>
>>>>> >>> Are there any concerns with this proposal?
>>>>> >>>
>>>>> >>> Thanks,
>>>>> >>> Hannah
>>>>> >>>
>>>>> >>>
>>>>> >>>
>>>>> >>>
>>>>> >>> On Fri, Jan 10, 2020 at 4:19 PM Ahmet Altay <al...@google.com>
>>>>> wrote:
>>>>> >>>>
>>>>> >>>>
>>>>> >>>>
>>>>> >>>> On Fri, Jan 10, 2020 at 3:33 PM Ahmet Altay <al...@google.com>
>>>>> wrote:
>>>>> >>>>>
>>>>> >>>>>
>>>>> >>>>>
>>>>> >>>>> On Fri, Jan 10, 2020 at 3:32 PM Ankur Goenka <go...@google.com>
>>>>> wrote:
>>>>> >>>>>>
>>>>> >>>>>> Also curious to know if apache provide any infra support fro
>>>>> projects under Apache umbrella and any quota limits they might have.
>>>>> >>>>
>>>>> >>>>
>>>>> >>>> Maybe Hannah can ask with an infra ticket?
>>>>> >>>>
>>>>> >>>>>>
>>>>> >>>>>>
>>>>> >>>>>> On Fri, Jan 10, 2020, 2:26 PM Robert Bradshaw <
>>>>> robertwb@google.com> wrote:
>>>>> >>>>>>>
>>>>> >>>>>>> One downside is that, unlike many of these projects, we
>>>>> release a
>>>>> >>>>>>> dozen or so containers. Is there exactly (and only) one level
>>>>> of
>>>>> >>>>>>> namespacing/nesting we can leverage here? (This isn't a
>>>>> blocker, but
>>>>> >>>>>>> something to consider.)
>>>>> >>>>>
>>>>> >>>>>
>>>>> >>>>> After a quick search, I could not find a way to use more than
>>>>> one level of repositories. We can use the naming scheme we currently use to
>>>>> help with. Our repositories are named as apachebeam/X, we could start using
>>>>> apache/beam/X.
>>>>> >>>>>
>>>>> >>>>>>>
>>>>> >>>>>>>
>>>>> >>>>>>> On Fri, Jan 10, 2020 at 2:06 PM Hannah Jiang <
>>>>> hannahjiang@google.com> wrote:
>>>>> >>>>>>> >
>>>>> >>>>>>> > Thanks Ahmet for proposing it.
>>>>> >>>>>>> > I will take it and work towards v2.19.
>>>>> >>>>
>>>>> >>>>
>>>>> >>>> Missed this part. Thank you Hannah!
>>>>> >>>>
>>>>> >>>>>>>
>>>>> >>>>>>> >
>>>>> >>>>>>> > Hannah
>>>>> >>>>>>> >
>>>>> >>>>>>> > On Fri, Jan 10, 2020 at 1:50 PM Kyle Weaver <
>>>>> kcweaver@google.com> wrote:
>>>>> >>>>>>> >>
>>>>> >>>>>>> >> It'd be nice to have the clout/official sheen of apache
>>>>> attached to our containers. Although getting the required permissions might
>>>>> add some small overhead to the release process. For example, yesterday,
>>>>> when we needed to create new repositories (not just update existing ones),
>>>>> since we have top-level ownership of the apachebeam organization, it was
>>>>> quick and easy to add them. I imagine we'd have had to get approval from
>>>>> someone outside the project to do that under the apache org. But this won't
>>>>> need to happen very often, so it's probably not that big a deal.
>>>>> >>>>>>> >>
>>>>> >>>>>>> >> On Fri, Jan 10, 2020 at 1:40 PM Ahmet Altay <
>>>>> altay@google.com> wrote:
>>>>> >>>>>>> >>>
>>>>> >>>>>>> >>> Hi all,
>>>>> >>>>>>> >>>
>>>>> >>>>>>> >>> I saw recent progress on the containers and wanted to
>>>>> bring this question to the attention of the dev list.
>>>>> >>>>>>> >>>
>>>>> >>>>>>> >>> Would it be possible to use the official ASF dockerhub
>>>>> organization for new Beam container releases? Concretely, starting from
>>>>> 2.19 could we release Beam containers to
>>>>> https://hub.docker.com/u/apache instead of
>>>>> https://hub.docker.com/u/apachebeam ?
>>>>> >>>>>>> >>>
>>>>> >>>>>>> >>> Ahmet
>>>>>
>>>>

Re: [PROPOSAL] Transition released containers to the official ASF dockerhub organization

[Hannah Jiang <ha...@google.com>]

Thanks everyone for supporting it.

Yes, it's very slow to get tickets resolved by infra. I propose a minor
improvement to reduce interactions with infra.

So far, we have granted maintainer permission(read & write) to release
managers' personal accounts. This step needs help from infra to add new
members to the group for every new release manager.
In order to avoid this, I proposed that we create a new account for release
purpose only and share it with release managers. The new account will have
read & write permissions to all Apache Beam docker repositories. A password
will be shared on an as-needed basis and we can change the password
periodically if needed, which is in our control. Are there any concerns
which I am not aware of with the sharing account approach?

Thanks,
Hannah


On Thu, Jan 16, 2020 at 10:41 AM Kenneth Knowles <ke...@apache.org> wrote:

> +1 very nice explanation
>
> On Wed, Jan 15, 2020 at 1:57 PM Ahmet Altay <al...@google.com> wrote:
>
>> +1 - Thank you for driving this!
>>
>> On Wed, Jan 15, 2020 at 1:55 PM Thomas Weise <th...@apache.org> wrote:
>>
>>> +1 for the namespace proposal.
>>>
>>> It is similar to github repos. Top-level is the org, then single level
>>> for repo (beam-abc, beam-xzy, ..)
>>>
>>>
>>>
>>> On Wed, Jan 15, 2020 at 1:45 PM Robert Bradshaw <ro...@google.com>
>>> wrote:
>>>
>>>> Various tags of the same image should at least logically be the same
>>>> thing, so I agree that we should not be trying to share a single
>>>> repository in that way. A full suite of apache/beam-{image_desc}
>>>> repositories, if apache is fine with that, seems like the best
>>>> approach.
>>>>
>>>> On Wed, Jan 15, 2020 at 1:32 PM Kyle Weaver <kc...@google.com>
>>>> wrote:
>>>> >
>>>> > +1, agree that moving current image name to tags is a non-starter.
>>>> Thanks for driving this Hannah. Let us know what they say about repo
>>>> creation.
>>>> >
>>>> > On Wed, Jan 15, 2020 at 1:16 PM Udi Meiri <eh...@google.com> wrote:
>>>> >>
>>>> >> SG +1
>>>> >>
>>>> >> On Wed, Jan 15, 2020 at 12:59 PM Hannah Jiang <
>>>> hannahjiang@google.com> wrote:
>>>> >>>
>>>> >>> I have done some research about images released under apache
>>>> namespace at docker hub, and here is my proposal.
>>>> >>>
>>>> >>> Currently, we are using apachebeam as our namespace and each image
>>>> has its own repository. Version number is used to tag the images.
>>>> >>> ie: apachebeam/python2.7_sdk:2.19.0,
>>>> apachebeam/flink1.9_job_server:2.19.0
>>>> >>>
>>>> >>> Now we are migrating to apache namespace and docker hub doesn't
>>>> support nested repository names, so we cannot use
>>>> apache/beam/{image-desc}:{version}.
>>>> >>> Instead, I propose to use apache/beam-{image_desc}:{version} as our
>>>> repository name.
>>>> >>> ie: apache/beam-python2.7_sdk:2.19.0,
>>>> apache/beam-flink1.9_job_server:2.19.0
>>>> >>> => When a user searches for apache/beam at docker hub, it will list
>>>> all the repositories we deployed with apache/beam-, so no concerns that
>>>> some released images are missed by users.
>>>> >>> => Repository names give insights to the users which repositories
>>>> they should use.
>>>> >>> => A downside with this approach is we need to create a new
>>>> repository whenever we release a new image, time and effort needed for this
>>>> is pending, I am contacting apache docker hub management team.
>>>> >>>
>>>> >>> I have considered using beam as repository name and moving image
>>>> name and version to tags, (ie: apache/beam:python3.7_sdk_2.19.0), which
>>>> means put all images to a single repository, however, this approach has
>>>> some downsides.
>>>> >>> => When a user searches for apache/beam, only one repository is
>>>> returned. Users need to use tags to identify which images they should use.
>>>> Since we release images with new tags for each version, it will overwhelm
>>>> the users and give them an impression that the images are not organized
>>>> well. It's also difficult to know what kind of images we deployed.
>>>> >>> => With both image name and version included at tags, it is a
>>>> little bit more complicated to maintain the code.
>>>> >>> => There is no correct answer which image the latest tag should
>>>> point to.
>>>> >>>
>>>> >>> Are there any concerns with this proposal?
>>>> >>>
>>>> >>> Thanks,
>>>> >>> Hannah
>>>> >>>
>>>> >>>
>>>> >>>
>>>> >>>
>>>> >>> On Fri, Jan 10, 2020 at 4:19 PM Ahmet Altay <al...@google.com>
>>>> wrote:
>>>> >>>>
>>>> >>>>
>>>> >>>>
>>>> >>>> On Fri, Jan 10, 2020 at 3:33 PM Ahmet Altay <al...@google.com>
>>>> wrote:
>>>> >>>>>
>>>> >>>>>
>>>> >>>>>
>>>> >>>>> On Fri, Jan 10, 2020 at 3:32 PM Ankur Goenka <go...@google.com>
>>>> wrote:
>>>> >>>>>>
>>>> >>>>>> Also curious to know if apache provide any infra support fro
>>>> projects under Apache umbrella and any quota limits they might have.
>>>> >>>>
>>>> >>>>
>>>> >>>> Maybe Hannah can ask with an infra ticket?
>>>> >>>>
>>>> >>>>>>
>>>> >>>>>>
>>>> >>>>>> On Fri, Jan 10, 2020, 2:26 PM Robert Bradshaw <
>>>> robertwb@google.com> wrote:
>>>> >>>>>>>
>>>> >>>>>>> One downside is that, unlike many of these projects, we release
>>>> a
>>>> >>>>>>> dozen or so containers. Is there exactly (and only) one level of
>>>> >>>>>>> namespacing/nesting we can leverage here? (This isn't a
>>>> blocker, but
>>>> >>>>>>> something to consider.)
>>>> >>>>>
>>>> >>>>>
>>>> >>>>> After a quick search, I could not find a way to use more than one
>>>> level of repositories. We can use the naming scheme we currently use to
>>>> help with. Our repositories are named as apachebeam/X, we could start using
>>>> apache/beam/X.
>>>> >>>>>
>>>> >>>>>>>
>>>> >>>>>>>
>>>> >>>>>>> On Fri, Jan 10, 2020 at 2:06 PM Hannah Jiang <
>>>> hannahjiang@google.com> wrote:
>>>> >>>>>>> >
>>>> >>>>>>> > Thanks Ahmet for proposing it.
>>>> >>>>>>> > I will take it and work towards v2.19.
>>>> >>>>
>>>> >>>>
>>>> >>>> Missed this part. Thank you Hannah!
>>>> >>>>
>>>> >>>>>>>
>>>> >>>>>>> >
>>>> >>>>>>> > Hannah
>>>> >>>>>>> >
>>>> >>>>>>> > On Fri, Jan 10, 2020 at 1:50 PM Kyle Weaver <
>>>> kcweaver@google.com> wrote:
>>>> >>>>>>> >>
>>>> >>>>>>> >> It'd be nice to have the clout/official sheen of apache
>>>> attached to our containers. Although getting the required permissions might
>>>> add some small overhead to the release process. For example, yesterday,
>>>> when we needed to create new repositories (not just update existing ones),
>>>> since we have top-level ownership of the apachebeam organization, it was
>>>> quick and easy to add them. I imagine we'd have had to get approval from
>>>> someone outside the project to do that under the apache org. But this won't
>>>> need to happen very often, so it's probably not that big a deal.
>>>> >>>>>>> >>
>>>> >>>>>>> >> On Fri, Jan 10, 2020 at 1:40 PM Ahmet Altay <
>>>> altay@google.com> wrote:
>>>> >>>>>>> >>>
>>>> >>>>>>> >>> Hi all,
>>>> >>>>>>> >>>
>>>> >>>>>>> >>> I saw recent progress on the containers and wanted to bring
>>>> this question to the attention of the dev list.
>>>> >>>>>>> >>>
>>>> >>>>>>> >>> Would it be possible to use the official ASF dockerhub
>>>> organization for new Beam container releases? Concretely, starting from
>>>> 2.19 could we release Beam containers to
>>>> https://hub.docker.com/u/apache instead of
>>>> https://hub.docker.com/u/apachebeam ?
>>>> >>>>>>> >>>
>>>> >>>>>>> >>> Ahmet
>>>>
>>>

Re: [PROPOSAL] Transition released containers to the official ASF dockerhub organization

[Kenneth Knowles <ke...@apache.org>]

+1 very nice explanation

On Wed, Jan 15, 2020 at 1:57 PM Ahmet Altay <al...@google.com> wrote:

> +1 - Thank you for driving this!
>
> On Wed, Jan 15, 2020 at 1:55 PM Thomas Weise <th...@apache.org> wrote:
>
>> +1 for the namespace proposal.
>>
>> It is similar to github repos. Top-level is the org, then single level
>> for repo (beam-abc, beam-xzy, ..)
>>
>>
>>
>> On Wed, Jan 15, 2020 at 1:45 PM Robert Bradshaw <ro...@google.com>
>> wrote:
>>
>>> Various tags of the same image should at least logically be the same
>>> thing, so I agree that we should not be trying to share a single
>>> repository in that way. A full suite of apache/beam-{image_desc}
>>> repositories, if apache is fine with that, seems like the best
>>> approach.
>>>
>>> On Wed, Jan 15, 2020 at 1:32 PM Kyle Weaver <kc...@google.com> wrote:
>>> >
>>> > +1, agree that moving current image name to tags is a non-starter.
>>> Thanks for driving this Hannah. Let us know what they say about repo
>>> creation.
>>> >
>>> > On Wed, Jan 15, 2020 at 1:16 PM Udi Meiri <eh...@google.com> wrote:
>>> >>
>>> >> SG +1
>>> >>
>>> >> On Wed, Jan 15, 2020 at 12:59 PM Hannah Jiang <ha...@google.com>
>>> wrote:
>>> >>>
>>> >>> I have done some research about images released under apache
>>> namespace at docker hub, and here is my proposal.
>>> >>>
>>> >>> Currently, we are using apachebeam as our namespace and each image
>>> has its own repository. Version number is used to tag the images.
>>> >>> ie: apachebeam/python2.7_sdk:2.19.0,
>>> apachebeam/flink1.9_job_server:2.19.0
>>> >>>
>>> >>> Now we are migrating to apache namespace and docker hub doesn't
>>> support nested repository names, so we cannot use
>>> apache/beam/{image-desc}:{version}.
>>> >>> Instead, I propose to use apache/beam-{image_desc}:{version} as our
>>> repository name.
>>> >>> ie: apache/beam-python2.7_sdk:2.19.0,
>>> apache/beam-flink1.9_job_server:2.19.0
>>> >>> => When a user searches for apache/beam at docker hub, it will list
>>> all the repositories we deployed with apache/beam-, so no concerns that
>>> some released images are missed by users.
>>> >>> => Repository names give insights to the users which repositories
>>> they should use.
>>> >>> => A downside with this approach is we need to create a new
>>> repository whenever we release a new image, time and effort needed for this
>>> is pending, I am contacting apache docker hub management team.
>>> >>>
>>> >>> I have considered using beam as repository name and moving image
>>> name and version to tags, (ie: apache/beam:python3.7_sdk_2.19.0), which
>>> means put all images to a single repository, however, this approach has
>>> some downsides.
>>> >>> => When a user searches for apache/beam, only one repository is
>>> returned. Users need to use tags to identify which images they should use.
>>> Since we release images with new tags for each version, it will overwhelm
>>> the users and give them an impression that the images are not organized
>>> well. It's also difficult to know what kind of images we deployed.
>>> >>> => With both image name and version included at tags, it is a little
>>> bit more complicated to maintain the code.
>>> >>> => There is no correct answer which image the latest tag should
>>> point to.
>>> >>>
>>> >>> Are there any concerns with this proposal?
>>> >>>
>>> >>> Thanks,
>>> >>> Hannah
>>> >>>
>>> >>>
>>> >>>
>>> >>>
>>> >>> On Fri, Jan 10, 2020 at 4:19 PM Ahmet Altay <al...@google.com>
>>> wrote:
>>> >>>>
>>> >>>>
>>> >>>>
>>> >>>> On Fri, Jan 10, 2020 at 3:33 PM Ahmet Altay <al...@google.com>
>>> wrote:
>>> >>>>>
>>> >>>>>
>>> >>>>>
>>> >>>>> On Fri, Jan 10, 2020 at 3:32 PM Ankur Goenka <go...@google.com>
>>> wrote:
>>> >>>>>>
>>> >>>>>> Also curious to know if apache provide any infra support fro
>>> projects under Apache umbrella and any quota limits they might have.
>>> >>>>
>>> >>>>
>>> >>>> Maybe Hannah can ask with an infra ticket?
>>> >>>>
>>> >>>>>>
>>> >>>>>>
>>> >>>>>> On Fri, Jan 10, 2020, 2:26 PM Robert Bradshaw <
>>> robertwb@google.com> wrote:
>>> >>>>>>>
>>> >>>>>>> One downside is that, unlike many of these projects, we release a
>>> >>>>>>> dozen or so containers. Is there exactly (and only) one level of
>>> >>>>>>> namespacing/nesting we can leverage here? (This isn't a blocker,
>>> but
>>> >>>>>>> something to consider.)
>>> >>>>>
>>> >>>>>
>>> >>>>> After a quick search, I could not find a way to use more than one
>>> level of repositories. We can use the naming scheme we currently use to
>>> help with. Our repositories are named as apachebeam/X, we could start using
>>> apache/beam/X.
>>> >>>>>
>>> >>>>>>>
>>> >>>>>>>
>>> >>>>>>> On Fri, Jan 10, 2020 at 2:06 PM Hannah Jiang <
>>> hannahjiang@google.com> wrote:
>>> >>>>>>> >
>>> >>>>>>> > Thanks Ahmet for proposing it.
>>> >>>>>>> > I will take it and work towards v2.19.
>>> >>>>
>>> >>>>
>>> >>>> Missed this part. Thank you Hannah!
>>> >>>>
>>> >>>>>>>
>>> >>>>>>> >
>>> >>>>>>> > Hannah
>>> >>>>>>> >
>>> >>>>>>> > On Fri, Jan 10, 2020 at 1:50 PM Kyle Weaver <
>>> kcweaver@google.com> wrote:
>>> >>>>>>> >>
>>> >>>>>>> >> It'd be nice to have the clout/official sheen of apache
>>> attached to our containers. Although getting the required permissions might
>>> add some small overhead to the release process. For example, yesterday,
>>> when we needed to create new repositories (not just update existing ones),
>>> since we have top-level ownership of the apachebeam organization, it was
>>> quick and easy to add them. I imagine we'd have had to get approval from
>>> someone outside the project to do that under the apache org. But this won't
>>> need to happen very often, so it's probably not that big a deal.
>>> >>>>>>> >>
>>> >>>>>>> >> On Fri, Jan 10, 2020 at 1:40 PM Ahmet Altay <al...@google.com>
>>> wrote:
>>> >>>>>>> >>>
>>> >>>>>>> >>> Hi all,
>>> >>>>>>> >>>
>>> >>>>>>> >>> I saw recent progress on the containers and wanted to bring
>>> this question to the attention of the dev list.
>>> >>>>>>> >>>
>>> >>>>>>> >>> Would it be possible to use the official ASF dockerhub
>>> organization for new Beam container releases? Concretely, starting from
>>> 2.19 could we release Beam containers to https://hub.docker.com/u/apache
>>> instead of https://hub.docker.com/u/apachebeam ?
>>> >>>>>>> >>>
>>> >>>>>>> >>> Ahmet
>>>
>>

Re: [PROPOSAL] Transition released containers to the official ASF dockerhub organization

[Ahmet Altay <al...@google.com>]

+1 - Thank you for driving this!

On Wed, Jan 15, 2020 at 1:55 PM Thomas Weise <th...@apache.org> wrote:

> +1 for the namespace proposal.
>
> It is similar to github repos. Top-level is the org, then single level for
> repo (beam-abc, beam-xzy, ..)
>
>
>
> On Wed, Jan 15, 2020 at 1:45 PM Robert Bradshaw <ro...@google.com>
> wrote:
>
>> Various tags of the same image should at least logically be the same
>> thing, so I agree that we should not be trying to share a single
>> repository in that way. A full suite of apache/beam-{image_desc}
>> repositories, if apache is fine with that, seems like the best
>> approach.
>>
>> On Wed, Jan 15, 2020 at 1:32 PM Kyle Weaver <kc...@google.com> wrote:
>> >
>> > +1, agree that moving current image name to tags is a non-starter.
>> Thanks for driving this Hannah. Let us know what they say about repo
>> creation.
>> >
>> > On Wed, Jan 15, 2020 at 1:16 PM Udi Meiri <eh...@google.com> wrote:
>> >>
>> >> SG +1
>> >>
>> >> On Wed, Jan 15, 2020 at 12:59 PM Hannah Jiang <ha...@google.com>
>> wrote:
>> >>>
>> >>> I have done some research about images released under apache
>> namespace at docker hub, and here is my proposal.
>> >>>
>> >>> Currently, we are using apachebeam as our namespace and each image
>> has its own repository. Version number is used to tag the images.
>> >>> ie: apachebeam/python2.7_sdk:2.19.0,
>> apachebeam/flink1.9_job_server:2.19.0
>> >>>
>> >>> Now we are migrating to apache namespace and docker hub doesn't
>> support nested repository names, so we cannot use
>> apache/beam/{image-desc}:{version}.
>> >>> Instead, I propose to use apache/beam-{image_desc}:{version} as our
>> repository name.
>> >>> ie: apache/beam-python2.7_sdk:2.19.0,
>> apache/beam-flink1.9_job_server:2.19.0
>> >>> => When a user searches for apache/beam at docker hub, it will list
>> all the repositories we deployed with apache/beam-, so no concerns that
>> some released images are missed by users.
>> >>> => Repository names give insights to the users which repositories
>> they should use.
>> >>> => A downside with this approach is we need to create a new
>> repository whenever we release a new image, time and effort needed for this
>> is pending, I am contacting apache docker hub management team.
>> >>>
>> >>> I have considered using beam as repository name and moving image name
>> and version to tags, (ie: apache/beam:python3.7_sdk_2.19.0), which means
>> put all images to a single repository, however, this approach has some
>> downsides.
>> >>> => When a user searches for apache/beam, only one repository is
>> returned. Users need to use tags to identify which images they should use.
>> Since we release images with new tags for each version, it will overwhelm
>> the users and give them an impression that the images are not organized
>> well. It's also difficult to know what kind of images we deployed.
>> >>> => With both image name and version included at tags, it is a little
>> bit more complicated to maintain the code.
>> >>> => There is no correct answer which image the latest tag should point
>> to.
>> >>>
>> >>> Are there any concerns with this proposal?
>> >>>
>> >>> Thanks,
>> >>> Hannah
>> >>>
>> >>>
>> >>>
>> >>>
>> >>> On Fri, Jan 10, 2020 at 4:19 PM Ahmet Altay <al...@google.com> wrote:
>> >>>>
>> >>>>
>> >>>>
>> >>>> On Fri, Jan 10, 2020 at 3:33 PM Ahmet Altay <al...@google.com>
>> wrote:
>> >>>>>
>> >>>>>
>> >>>>>
>> >>>>> On Fri, Jan 10, 2020 at 3:32 PM Ankur Goenka <go...@google.com>
>> wrote:
>> >>>>>>
>> >>>>>> Also curious to know if apache provide any infra support fro
>> projects under Apache umbrella and any quota limits they might have.
>> >>>>
>> >>>>
>> >>>> Maybe Hannah can ask with an infra ticket?
>> >>>>
>> >>>>>>
>> >>>>>>
>> >>>>>> On Fri, Jan 10, 2020, 2:26 PM Robert Bradshaw <ro...@google.com>
>> wrote:
>> >>>>>>>
>> >>>>>>> One downside is that, unlike many of these projects, we release a
>> >>>>>>> dozen or so containers. Is there exactly (and only) one level of
>> >>>>>>> namespacing/nesting we can leverage here? (This isn't a blocker,
>> but
>> >>>>>>> something to consider.)
>> >>>>>
>> >>>>>
>> >>>>> After a quick search, I could not find a way to use more than one
>> level of repositories. We can use the naming scheme we currently use to
>> help with. Our repositories are named as apachebeam/X, we could start using
>> apache/beam/X.
>> >>>>>
>> >>>>>>>
>> >>>>>>>
>> >>>>>>> On Fri, Jan 10, 2020 at 2:06 PM Hannah Jiang <
>> hannahjiang@google.com> wrote:
>> >>>>>>> >
>> >>>>>>> > Thanks Ahmet for proposing it.
>> >>>>>>> > I will take it and work towards v2.19.
>> >>>>
>> >>>>
>> >>>> Missed this part. Thank you Hannah!
>> >>>>
>> >>>>>>>
>> >>>>>>> >
>> >>>>>>> > Hannah
>> >>>>>>> >
>> >>>>>>> > On Fri, Jan 10, 2020 at 1:50 PM Kyle Weaver <
>> kcweaver@google.com> wrote:
>> >>>>>>> >>
>> >>>>>>> >> It'd be nice to have the clout/official sheen of apache
>> attached to our containers. Although getting the required permissions might
>> add some small overhead to the release process. For example, yesterday,
>> when we needed to create new repositories (not just update existing ones),
>> since we have top-level ownership of the apachebeam organization, it was
>> quick and easy to add them. I imagine we'd have had to get approval from
>> someone outside the project to do that under the apache org. But this won't
>> need to happen very often, so it's probably not that big a deal.
>> >>>>>>> >>
>> >>>>>>> >> On Fri, Jan 10, 2020 at 1:40 PM Ahmet Altay <al...@google.com>
>> wrote:
>> >>>>>>> >>>
>> >>>>>>> >>> Hi all,
>> >>>>>>> >>>
>> >>>>>>> >>> I saw recent progress on the containers and wanted to bring
>> this question to the attention of the dev list.
>> >>>>>>> >>>
>> >>>>>>> >>> Would it be possible to use the official ASF dockerhub
>> organization for new Beam container releases? Concretely, starting from
>> 2.19 could we release Beam containers to https://hub.docker.com/u/apache
>> instead of https://hub.docker.com/u/apachebeam ?
>> >>>>>>> >>>
>> >>>>>>> >>> Ahmet
>>
>

Re: [PROPOSAL] Transition released containers to the official ASF dockerhub organization

[Thomas Weise <th...@apache.org>]

+1 for the namespace proposal.

It is similar to github repos. Top-level is the org, then single level for
repo (beam-abc, beam-xzy, ..)



On Wed, Jan 15, 2020 at 1:45 PM Robert Bradshaw <ro...@google.com> wrote:

> Various tags of the same image should at least logically be the same
> thing, so I agree that we should not be trying to share a single
> repository in that way. A full suite of apache/beam-{image_desc}
> repositories, if apache is fine with that, seems like the best
> approach.
>
> On Wed, Jan 15, 2020 at 1:32 PM Kyle Weaver <kc...@google.com> wrote:
> >
> > +1, agree that moving current image name to tags is a non-starter.
> Thanks for driving this Hannah. Let us know what they say about repo
> creation.
> >
> > On Wed, Jan 15, 2020 at 1:16 PM Udi Meiri <eh...@google.com> wrote:
> >>
> >> SG +1
> >>
> >> On Wed, Jan 15, 2020 at 12:59 PM Hannah Jiang <ha...@google.com>
> wrote:
> >>>
> >>> I have done some research about images released under apache namespace
> at docker hub, and here is my proposal.
> >>>
> >>> Currently, we are using apachebeam as our namespace and each image has
> its own repository. Version number is used to tag the images.
> >>> ie: apachebeam/python2.7_sdk:2.19.0,
> apachebeam/flink1.9_job_server:2.19.0
> >>>
> >>> Now we are migrating to apache namespace and docker hub doesn't
> support nested repository names, so we cannot use
> apache/beam/{image-desc}:{version}.
> >>> Instead, I propose to use apache/beam-{image_desc}:{version} as our
> repository name.
> >>> ie: apache/beam-python2.7_sdk:2.19.0,
> apache/beam-flink1.9_job_server:2.19.0
> >>> => When a user searches for apache/beam at docker hub, it will list
> all the repositories we deployed with apache/beam-, so no concerns that
> some released images are missed by users.
> >>> => Repository names give insights to the users which repositories they
> should use.
> >>> => A downside with this approach is we need to create a new repository
> whenever we release a new image, time and effort needed for this is
> pending, I am contacting apache docker hub management team.
> >>>
> >>> I have considered using beam as repository name and moving image name
> and version to tags, (ie: apache/beam:python3.7_sdk_2.19.0), which means
> put all images to a single repository, however, this approach has some
> downsides.
> >>> => When a user searches for apache/beam, only one repository is
> returned. Users need to use tags to identify which images they should use.
> Since we release images with new tags for each version, it will overwhelm
> the users and give them an impression that the images are not organized
> well. It's also difficult to know what kind of images we deployed.
> >>> => With both image name and version included at tags, it is a little
> bit more complicated to maintain the code.
> >>> => There is no correct answer which image the latest tag should point
> to.
> >>>
> >>> Are there any concerns with this proposal?
> >>>
> >>> Thanks,
> >>> Hannah
> >>>
> >>>
> >>>
> >>>
> >>> On Fri, Jan 10, 2020 at 4:19 PM Ahmet Altay <al...@google.com> wrote:
> >>>>
> >>>>
> >>>>
> >>>> On Fri, Jan 10, 2020 at 3:33 PM Ahmet Altay <al...@google.com> wrote:
> >>>>>
> >>>>>
> >>>>>
> >>>>> On Fri, Jan 10, 2020 at 3:32 PM Ankur Goenka <go...@google.com>
> wrote:
> >>>>>>
> >>>>>> Also curious to know if apache provide any infra support fro
> projects under Apache umbrella and any quota limits they might have.
> >>>>
> >>>>
> >>>> Maybe Hannah can ask with an infra ticket?
> >>>>
> >>>>>>
> >>>>>>
> >>>>>> On Fri, Jan 10, 2020, 2:26 PM Robert Bradshaw <ro...@google.com>
> wrote:
> >>>>>>>
> >>>>>>> One downside is that, unlike many of these projects, we release a
> >>>>>>> dozen or so containers. Is there exactly (and only) one level of
> >>>>>>> namespacing/nesting we can leverage here? (This isn't a blocker,
> but
> >>>>>>> something to consider.)
> >>>>>
> >>>>>
> >>>>> After a quick search, I could not find a way to use more than one
> level of repositories. We can use the naming scheme we currently use to
> help with. Our repositories are named as apachebeam/X, we could start using
> apache/beam/X.
> >>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>> On Fri, Jan 10, 2020 at 2:06 PM Hannah Jiang <
> hannahjiang@google.com> wrote:
> >>>>>>> >
> >>>>>>> > Thanks Ahmet for proposing it.
> >>>>>>> > I will take it and work towards v2.19.
> >>>>
> >>>>
> >>>> Missed this part. Thank you Hannah!
> >>>>
> >>>>>>>
> >>>>>>> >
> >>>>>>> > Hannah
> >>>>>>> >
> >>>>>>> > On Fri, Jan 10, 2020 at 1:50 PM Kyle Weaver <kc...@google.com>
> wrote:
> >>>>>>> >>
> >>>>>>> >> It'd be nice to have the clout/official sheen of apache
> attached to our containers. Although getting the required permissions might
> add some small overhead to the release process. For example, yesterday,
> when we needed to create new repositories (not just update existing ones),
> since we have top-level ownership of the apachebeam organization, it was
> quick and easy to add them. I imagine we'd have had to get approval from
> someone outside the project to do that under the apache org. But this won't
> need to happen very often, so it's probably not that big a deal.
> >>>>>>> >>
> >>>>>>> >> On Fri, Jan 10, 2020 at 1:40 PM Ahmet Altay <al...@google.com>
> wrote:
> >>>>>>> >>>
> >>>>>>> >>> Hi all,
> >>>>>>> >>>
> >>>>>>> >>> I saw recent progress on the containers and wanted to bring
> this question to the attention of the dev list.
> >>>>>>> >>>
> >>>>>>> >>> Would it be possible to use the official ASF dockerhub
> organization for new Beam container releases? Concretely, starting from
> 2.19 could we release Beam containers to https://hub.docker.com/u/apache
> instead of https://hub.docker.com/u/apachebeam ?
> >>>>>>> >>>
> >>>>>>> >>> Ahmet
>

Re: [PROPOSAL] Transition released containers to the official ASF dockerhub organization

[Robert Bradshaw <ro...@google.com>]

Various tags of the same image should at least logically be the same
thing, so I agree that we should not be trying to share a single
repository in that way. A full suite of apache/beam-{image_desc}
repositories, if apache is fine with that, seems like the best
approach.

On Wed, Jan 15, 2020 at 1:32 PM Kyle Weaver <kc...@google.com> wrote:
>
> +1, agree that moving current image name to tags is a non-starter. Thanks for driving this Hannah. Let us know what they say about repo creation.
>
> On Wed, Jan 15, 2020 at 1:16 PM Udi Meiri <eh...@google.com> wrote:
>>
>> SG +1
>>
>> On Wed, Jan 15, 2020 at 12:59 PM Hannah Jiang <ha...@google.com> wrote:
>>>
>>> I have done some research about images released under apache namespace at docker hub, and here is my proposal.
>>>
>>> Currently, we are using apachebeam as our namespace and each image has its own repository. Version number is used to tag the images.
>>> ie: apachebeam/python2.7_sdk:2.19.0, apachebeam/flink1.9_job_server:2.19.0
>>>
>>> Now we are migrating to apache namespace and docker hub doesn't support nested repository names, so we cannot use apache/beam/{image-desc}:{version}.
>>> Instead, I propose to use apache/beam-{image_desc}:{version} as our repository name.
>>> ie: apache/beam-python2.7_sdk:2.19.0, apache/beam-flink1.9_job_server:2.19.0
>>> => When a user searches for apache/beam at docker hub, it will list all the repositories we deployed with apache/beam-, so no concerns that some released images are missed by users.
>>> => Repository names give insights to the users which repositories they should use.
>>> => A downside with this approach is we need to create a new repository whenever we release a new image, time and effort needed for this is pending, I am contacting apache docker hub management team.
>>>
>>> I have considered using beam as repository name and moving image name and version to tags, (ie: apache/beam:python3.7_sdk_2.19.0), which means put all images to a single repository, however, this approach has some downsides.
>>> => When a user searches for apache/beam, only one repository is returned. Users need to use tags to identify which images they should use. Since we release images with new tags for each version, it will overwhelm the users and give them an impression that the images are not organized well. It's also difficult to know what kind of images we deployed.
>>> => With both image name and version included at tags, it is a little bit more complicated to maintain the code.
>>> => There is no correct answer which image the latest tag should point to.
>>>
>>> Are there any concerns with this proposal?
>>>
>>> Thanks,
>>> Hannah
>>>
>>>
>>>
>>>
>>> On Fri, Jan 10, 2020 at 4:19 PM Ahmet Altay <al...@google.com> wrote:
>>>>
>>>>
>>>>
>>>> On Fri, Jan 10, 2020 at 3:33 PM Ahmet Altay <al...@google.com> wrote:
>>>>>
>>>>>
>>>>>
>>>>> On Fri, Jan 10, 2020 at 3:32 PM Ankur Goenka <go...@google.com> wrote:
>>>>>>
>>>>>> Also curious to know if apache provide any infra support fro projects under Apache umbrella and any quota limits they might have.
>>>>
>>>>
>>>> Maybe Hannah can ask with an infra ticket?
>>>>
>>>>>>
>>>>>>
>>>>>> On Fri, Jan 10, 2020, 2:26 PM Robert Bradshaw <ro...@google.com> wrote:
>>>>>>>
>>>>>>> One downside is that, unlike many of these projects, we release a
>>>>>>> dozen or so containers. Is there exactly (and only) one level of
>>>>>>> namespacing/nesting we can leverage here? (This isn't a blocker, but
>>>>>>> something to consider.)
>>>>>
>>>>>
>>>>> After a quick search, I could not find a way to use more than one level of repositories. We can use the naming scheme we currently use to help with. Our repositories are named as apachebeam/X, we could start using apache/beam/X.
>>>>>
>>>>>>>
>>>>>>>
>>>>>>> On Fri, Jan 10, 2020 at 2:06 PM Hannah Jiang <ha...@google.com> wrote:
>>>>>>> >
>>>>>>> > Thanks Ahmet for proposing it.
>>>>>>> > I will take it and work towards v2.19.
>>>>
>>>>
>>>> Missed this part. Thank you Hannah!
>>>>
>>>>>>>
>>>>>>> >
>>>>>>> > Hannah
>>>>>>> >
>>>>>>> > On Fri, Jan 10, 2020 at 1:50 PM Kyle Weaver <kc...@google.com> wrote:
>>>>>>> >>
>>>>>>> >> It'd be nice to have the clout/official sheen of apache attached to our containers. Although getting the required permissions might add some small overhead to the release process. For example, yesterday, when we needed to create new repositories (not just update existing ones), since we have top-level ownership of the apachebeam organization, it was quick and easy to add them. I imagine we'd have had to get approval from someone outside the project to do that under the apache org. But this won't need to happen very often, so it's probably not that big a deal.
>>>>>>> >>
>>>>>>> >> On Fri, Jan 10, 2020 at 1:40 PM Ahmet Altay <al...@google.com> wrote:
>>>>>>> >>>
>>>>>>> >>> Hi all,
>>>>>>> >>>
>>>>>>> >>> I saw recent progress on the containers and wanted to bring this question to the attention of the dev list.
>>>>>>> >>>
>>>>>>> >>> Would it be possible to use the official ASF dockerhub organization for new Beam container releases? Concretely, starting from 2.19 could we release Beam containers to https://hub.docker.com/u/apache instead of https://hub.docker.com/u/apachebeam ?
>>>>>>> >>>
>>>>>>> >>> Ahmet

Re: [PROPOSAL] Transition released containers to the official ASF dockerhub organization

[Kyle Weaver <kc...@google.com>]

+1, agree that moving current image name to tags is a non-starter. Thanks
for driving this Hannah. Let us know what they say about repo creation.

On Wed, Jan 15, 2020 at 1:16 PM Udi Meiri <eh...@google.com> wrote:

> SG +1
>
> On Wed, Jan 15, 2020 at 12:59 PM Hannah Jiang <ha...@google.com>
> wrote:
>
>> I have done some research about images released under apache namespace at
>> docker hub, and here is my proposal.
>>
>> Currently, we are using apachebeam as our namespace and each image has
>> its own repository. Version number is used to tag the images.
>> ie: apachebeam/python2.7_sdk:2.19.0, apachebeam/flink1.9_job_server:2.19.0
>>
>> Now we are migrating to apache namespace and docker hub doesn't support
>> nested repository names, so we cannot use
>> apache/beam/{image-desc}:{version}.
>> Instead, I propose to use *apache/beam-{image_desc}:{version}* as our
>> repository name.
>> ie: apache/beam-python2.7_sdk:2.19.0,
>> apache/beam-flink1.9_job_server:2.19.0
>> => When a user searches for *apache/beam* at docker hub, it will list
>> all the repositories we deployed with apache/beam-, so no concerns that
>> some released images are missed by users.
>> => Repository names give insights to the users which repositories they
>> should use.
>> => A downside with this approach is we need to create a new repository
>> whenever we release a new image, time and effort needed for this is
>> pending, I am contacting apache docker hub management team.
>>
>> I have considered using beam as repository name and moving image name and
>> version to tags, (ie: apache/beam:python3.7_sdk_2.19.0), which means put
>> all images to a single repository, however, this approach has some
>> downsides.
>> => When a user searches for apache/beam, only one repository is returned.
>> Users need to use tags to identify which images they should use. Since we
>> release images with new tags for each version, it will overwhelm the users
>> and give them an impression that the images are not organized well. It's
>> also difficult to know what kind of images we deployed.
>> => With both image name and version included at tags, it is a little bit
>> more complicated to maintain the code.
>> => There is no correct answer which image the latest tag should point to.
>>
>> Are there any concerns with this proposal?
>>
>> Thanks,
>> Hannah
>>
>>
>>
>>
>> On Fri, Jan 10, 2020 at 4:19 PM Ahmet Altay <al...@google.com> wrote:
>>
>>>
>>>
>>> On Fri, Jan 10, 2020 at 3:33 PM Ahmet Altay <al...@google.com> wrote:
>>>
>>>>
>>>>
>>>> On Fri, Jan 10, 2020 at 3:32 PM Ankur Goenka <go...@google.com> wrote:
>>>>
>>>>> Also curious to know if apache provide any infra support fro projects
>>>>> under Apache umbrella and any quota limits they might have.
>>>>>
>>>>
>>> Maybe Hannah can ask with an infra ticket?
>>>
>>>
>>>>
>>>>> On Fri, Jan 10, 2020, 2:26 PM Robert Bradshaw <ro...@google.com>
>>>>> wrote:
>>>>>
>>>>>> One downside is that, unlike many of these projects, we release a
>>>>>> dozen or so containers. Is there exactly (and only) one level of
>>>>>> namespacing/nesting we can leverage here? (This isn't a blocker, but
>>>>>> something to consider.)
>>>>>>
>>>>>
>>>> After a quick search, I could not find a way to use more than one level
>>>> of repositories. We can use the naming scheme we currently use to help
>>>> with. Our repositories are named as apachebeam/X, we could start using
>>>> apache/beam/X.
>>>>
>>>>
>>>>>
>>>>>> On Fri, Jan 10, 2020 at 2:06 PM Hannah Jiang <ha...@google.com>
>>>>>> wrote:
>>>>>> >
>>>>>> > Thanks Ahmet for proposing it.
>>>>>> > I will take it and work towards v2.19.
>>>>>>
>>>>>
>>> Missed this part. Thank you Hannah!
>>>
>>>
>>>> >
>>>>>> > Hannah
>>>>>> >
>>>>>> > On Fri, Jan 10, 2020 at 1:50 PM Kyle Weaver <kc...@google.com>
>>>>>> wrote:
>>>>>> >>
>>>>>> >> It'd be nice to have the clout/official sheen of apache attached
>>>>>> to our containers. Although getting the required permissions might add some
>>>>>> small overhead to the release process. For example, yesterday, when we
>>>>>> needed to create new repositories (not just update existing ones), since we
>>>>>> have top-level ownership of the apachebeam organization, it was quick and
>>>>>> easy to add them. I imagine we'd have had to get approval from someone
>>>>>> outside the project to do that under the apache org. But this won't need to
>>>>>> happen very often, so it's probably not that big a deal.
>>>>>> >>
>>>>>> >> On Fri, Jan 10, 2020 at 1:40 PM Ahmet Altay <al...@google.com>
>>>>>> wrote:
>>>>>> >>>
>>>>>> >>> Hi all,
>>>>>> >>>
>>>>>> >>> I saw recent progress on the containers and wanted to bring this
>>>>>> question to the attention of the dev list.
>>>>>> >>>
>>>>>> >>> Would it be possible to use the official ASF dockerhub
>>>>>> organization for new Beam container releases? Concretely, starting from
>>>>>> 2.19 could we release Beam containers to
>>>>>> https://hub.docker.com/u/apache instead of
>>>>>> https://hub.docker.com/u/apachebeam ?
>>>>>> >>>
>>>>>> >>> Ahmet
>>>>>>
>>>>>

Re: [PROPOSAL] Transition released containers to the official ASF dockerhub organization

[Udi Meiri <eh...@google.com>]

SG +1

On Wed, Jan 15, 2020 at 12:59 PM Hannah Jiang <ha...@google.com>
wrote:

> I have done some research about images released under apache namespace at
> docker hub, and here is my proposal.
>
> Currently, we are using apachebeam as our namespace and each image has its
> own repository. Version number is used to tag the images.
> ie: apachebeam/python2.7_sdk:2.19.0, apachebeam/flink1.9_job_server:2.19.0
>
> Now we are migrating to apache namespace and docker hub doesn't support
> nested repository names, so we cannot use
> apache/beam/{image-desc}:{version}.
> Instead, I propose to use *apache/beam-{image_desc}:{version}* as our
> repository name.
> ie: apache/beam-python2.7_sdk:2.19.0,
> apache/beam-flink1.9_job_server:2.19.0
> => When a user searches for *apache/beam* at docker hub, it will list all
> the repositories we deployed with apache/beam-, so no concerns that some
> released images are missed by users.
> => Repository names give insights to the users which repositories they
> should use.
> => A downside with this approach is we need to create a new repository
> whenever we release a new image, time and effort needed for this is
> pending, I am contacting apache docker hub management team.
>
> I have considered using beam as repository name and moving image name and
> version to tags, (ie: apache/beam:python3.7_sdk_2.19.0), which means put
> all images to a single repository, however, this approach has some
> downsides.
> => When a user searches for apache/beam, only one repository is returned.
> Users need to use tags to identify which images they should use. Since we
> release images with new tags for each version, it will overwhelm the users
> and give them an impression that the images are not organized well. It's
> also difficult to know what kind of images we deployed.
> => With both image name and version included at tags, it is a little bit
> more complicated to maintain the code.
> => There is no correct answer which image the latest tag should point to.
>
> Are there any concerns with this proposal?
>
> Thanks,
> Hannah
>
>
>
>
> On Fri, Jan 10, 2020 at 4:19 PM Ahmet Altay <al...@google.com> wrote:
>
>>
>>
>> On Fri, Jan 10, 2020 at 3:33 PM Ahmet Altay <al...@google.com> wrote:
>>
>>>
>>>
>>> On Fri, Jan 10, 2020 at 3:32 PM Ankur Goenka <go...@google.com> wrote:
>>>
>>>> Also curious to know if apache provide any infra support fro projects
>>>> under Apache umbrella and any quota limits they might have.
>>>>
>>>
>> Maybe Hannah can ask with an infra ticket?
>>
>>
>>>
>>>> On Fri, Jan 10, 2020, 2:26 PM Robert Bradshaw <ro...@google.com>
>>>> wrote:
>>>>
>>>>> One downside is that, unlike many of these projects, we release a
>>>>> dozen or so containers. Is there exactly (and only) one level of
>>>>> namespacing/nesting we can leverage here? (This isn't a blocker, but
>>>>> something to consider.)
>>>>>
>>>>
>>> After a quick search, I could not find a way to use more than one level
>>> of repositories. We can use the naming scheme we currently use to help
>>> with. Our repositories are named as apachebeam/X, we could start using
>>> apache/beam/X.
>>>
>>>
>>>>
>>>>> On Fri, Jan 10, 2020 at 2:06 PM Hannah Jiang <ha...@google.com>
>>>>> wrote:
>>>>> >
>>>>> > Thanks Ahmet for proposing it.
>>>>> > I will take it and work towards v2.19.
>>>>>
>>>>
>> Missed this part. Thank you Hannah!
>>
>>
>>> >
>>>>> > Hannah
>>>>> >
>>>>> > On Fri, Jan 10, 2020 at 1:50 PM Kyle Weaver <kc...@google.com>
>>>>> wrote:
>>>>> >>
>>>>> >> It'd be nice to have the clout/official sheen of apache attached to
>>>>> our containers. Although getting the required permissions might add some
>>>>> small overhead to the release process. For example, yesterday, when we
>>>>> needed to create new repositories (not just update existing ones), since we
>>>>> have top-level ownership of the apachebeam organization, it was quick and
>>>>> easy to add them. I imagine we'd have had to get approval from someone
>>>>> outside the project to do that under the apache org. But this won't need to
>>>>> happen very often, so it's probably not that big a deal.
>>>>> >>
>>>>> >> On Fri, Jan 10, 2020 at 1:40 PM Ahmet Altay <al...@google.com>
>>>>> wrote:
>>>>> >>>
>>>>> >>> Hi all,
>>>>> >>>
>>>>> >>> I saw recent progress on the containers and wanted to bring this
>>>>> question to the attention of the dev list.
>>>>> >>>
>>>>> >>> Would it be possible to use the official ASF dockerhub
>>>>> organization for new Beam container releases? Concretely, starting from
>>>>> 2.19 could we release Beam containers to
>>>>> https://hub.docker.com/u/apache instead of
>>>>> https://hub.docker.com/u/apachebeam ?
>>>>> >>>
>>>>> >>> Ahmet
>>>>>
>>>>

Re: [PROPOSAL] Transition released containers to the official ASF dockerhub organization

[Hannah Jiang <ha...@google.com>]

I have done some research about images released under apache namespace at
docker hub, and here is my proposal.

Currently, we are using apachebeam as our namespace and each image has its
own repository. Version number is used to tag the images.
ie: apachebeam/python2.7_sdk:2.19.0, apachebeam/flink1.9_job_server:2.19.0

Now we are migrating to apache namespace and docker hub doesn't support
nested repository names, so we cannot use
apache/beam/{image-desc}:{version}.
Instead, I propose to use *apache/beam-{image_desc}:{version}* as our
repository name.
ie: apache/beam-python2.7_sdk:2.19.0, apache/beam-flink1.9_job_server:2.19.0
=> When a user searches for *apache/beam* at docker hub, it will list all
the repositories we deployed with apache/beam-, so no concerns that some
released images are missed by users.
=> Repository names give insights to the users which repositories they
should use.
=> A downside with this approach is we need to create a new repository
whenever we release a new image, time and effort needed for this is
pending, I am contacting apache docker hub management team.

I have considered using beam as repository name and moving image name and
version to tags, (ie: apache/beam:python3.7_sdk_2.19.0), which means put
all images to a single repository, however, this approach has some
downsides.
=> When a user searches for apache/beam, only one repository is returned.
Users need to use tags to identify which images they should use. Since we
release images with new tags for each version, it will overwhelm the users
and give them an impression that the images are not organized well. It's
also difficult to know what kind of images we deployed.
=> With both image name and version included at tags, it is a little bit
more complicated to maintain the code.
=> There is no correct answer which image the latest tag should point to.

Are there any concerns with this proposal?

Thanks,
Hannah




On Fri, Jan 10, 2020 at 4:19 PM Ahmet Altay <al...@google.com> wrote:

>
>
> On Fri, Jan 10, 2020 at 3:33 PM Ahmet Altay <al...@google.com> wrote:
>
>>
>>
>> On Fri, Jan 10, 2020 at 3:32 PM Ankur Goenka <go...@google.com> wrote:
>>
>>> Also curious to know if apache provide any infra support fro projects
>>> under Apache umbrella and any quota limits they might have.
>>>
>>
> Maybe Hannah can ask with an infra ticket?
>
>
>>
>>> On Fri, Jan 10, 2020, 2:26 PM Robert Bradshaw <ro...@google.com>
>>> wrote:
>>>
>>>> One downside is that, unlike many of these projects, we release a
>>>> dozen or so containers. Is there exactly (and only) one level of
>>>> namespacing/nesting we can leverage here? (This isn't a blocker, but
>>>> something to consider.)
>>>>
>>>
>> After a quick search, I could not find a way to use more than one level
>> of repositories. We can use the naming scheme we currently use to help
>> with. Our repositories are named as apachebeam/X, we could start using
>> apache/beam/X.
>>
>>
>>>
>>>> On Fri, Jan 10, 2020 at 2:06 PM Hannah Jiang <ha...@google.com>
>>>> wrote:
>>>> >
>>>> > Thanks Ahmet for proposing it.
>>>> > I will take it and work towards v2.19.
>>>>
>>>
> Missed this part. Thank you Hannah!
>
>
>> >
>>>> > Hannah
>>>> >
>>>> > On Fri, Jan 10, 2020 at 1:50 PM Kyle Weaver <kc...@google.com>
>>>> wrote:
>>>> >>
>>>> >> It'd be nice to have the clout/official sheen of apache attached to
>>>> our containers. Although getting the required permissions might add some
>>>> small overhead to the release process. For example, yesterday, when we
>>>> needed to create new repositories (not just update existing ones), since we
>>>> have top-level ownership of the apachebeam organization, it was quick and
>>>> easy to add them. I imagine we'd have had to get approval from someone
>>>> outside the project to do that under the apache org. But this won't need to
>>>> happen very often, so it's probably not that big a deal.
>>>> >>
>>>> >> On Fri, Jan 10, 2020 at 1:40 PM Ahmet Altay <al...@google.com>
>>>> wrote:
>>>> >>>
>>>> >>> Hi all,
>>>> >>>
>>>> >>> I saw recent progress on the containers and wanted to bring this
>>>> question to the attention of the dev list.
>>>> >>>
>>>> >>> Would it be possible to use the official ASF dockerhub organization
>>>> for new Beam container releases? Concretely, starting from 2.19 could we
>>>> release Beam containers to https://hub.docker.com/u/apache instead of
>>>> https://hub.docker.com/u/apachebeam ?
>>>> >>>
>>>> >>> Ahmet
>>>>
>>>

Re: [PROPOSAL] Transition released containers to the official ASF dockerhub organization

[Ahmet Altay <al...@google.com>]

On Fri, Jan 10, 2020 at 3:33 PM Ahmet Altay <al...@google.com> wrote:

>
>
> On Fri, Jan 10, 2020 at 3:32 PM Ankur Goenka <go...@google.com> wrote:
>
>> Also curious to know if apache provide any infra support fro projects
>> under Apache umbrella and any quota limits they might have.
>>
>
Maybe Hannah can ask with an infra ticket?


>
>> On Fri, Jan 10, 2020, 2:26 PM Robert Bradshaw <ro...@google.com>
>> wrote:
>>
>>> One downside is that, unlike many of these projects, we release a
>>> dozen or so containers. Is there exactly (and only) one level of
>>> namespacing/nesting we can leverage here? (This isn't a blocker, but
>>> something to consider.)
>>>
>>
> After a quick search, I could not find a way to use more than one level of
> repositories. We can use the naming scheme we currently use to help with.
> Our repositories are named as apachebeam/X, we could start using
> apache/beam/X.
>
>
>>
>>> On Fri, Jan 10, 2020 at 2:06 PM Hannah Jiang <ha...@google.com>
>>> wrote:
>>> >
>>> > Thanks Ahmet for proposing it.
>>> > I will take it and work towards v2.19.
>>>
>>
Missed this part. Thank you Hannah!


> >
>>> > Hannah
>>> >
>>> > On Fri, Jan 10, 2020 at 1:50 PM Kyle Weaver <kc...@google.com>
>>> wrote:
>>> >>
>>> >> It'd be nice to have the clout/official sheen of apache attached to
>>> our containers. Although getting the required permissions might add some
>>> small overhead to the release process. For example, yesterday, when we
>>> needed to create new repositories (not just update existing ones), since we
>>> have top-level ownership of the apachebeam organization, it was quick and
>>> easy to add them. I imagine we'd have had to get approval from someone
>>> outside the project to do that under the apache org. But this won't need to
>>> happen very often, so it's probably not that big a deal.
>>> >>
>>> >> On Fri, Jan 10, 2020 at 1:40 PM Ahmet Altay <al...@google.com> wrote:
>>> >>>
>>> >>> Hi all,
>>> >>>
>>> >>> I saw recent progress on the containers and wanted to bring this
>>> question to the attention of the dev list.
>>> >>>
>>> >>> Would it be possible to use the official ASF dockerhub organization
>>> for new Beam container releases? Concretely, starting from 2.19 could we
>>> release Beam containers to https://hub.docker.com/u/apache instead of
>>> https://hub.docker.com/u/apachebeam ?
>>> >>>
>>> >>> Ahmet
>>>
>>

Re: [PROPOSAL] Transition released containers to the official ASF dockerhub organization

[Ahmet Altay <al...@google.com>]

On Fri, Jan 10, 2020 at 3:32 PM Ankur Goenka <go...@google.com> wrote:

> Also curious to know if apache provide any infra support fro projects
> under Apache umbrella and any quota limits they might have.
>
> On Fri, Jan 10, 2020, 2:26 PM Robert Bradshaw <ro...@google.com> wrote:
>
>> One downside is that, unlike many of these projects, we release a
>> dozen or so containers. Is there exactly (and only) one level of
>> namespacing/nesting we can leverage here? (This isn't a blocker, but
>> something to consider.)
>>
>
After a quick search, I could not find a way to use more than one level of
repositories. We can use the naming scheme we currently use to help with.
Our repositories are named as apachebeam/X, we could start using
apache/beam/X.


>
>> On Fri, Jan 10, 2020 at 2:06 PM Hannah Jiang <ha...@google.com>
>> wrote:
>> >
>> > Thanks Ahmet for proposing it.
>> > I will take it and work towards v2.19.
>> >
>> > Hannah
>> >
>> > On Fri, Jan 10, 2020 at 1:50 PM Kyle Weaver <kc...@google.com>
>> wrote:
>> >>
>> >> It'd be nice to have the clout/official sheen of apache attached to
>> our containers. Although getting the required permissions might add some
>> small overhead to the release process. For example, yesterday, when we
>> needed to create new repositories (not just update existing ones), since we
>> have top-level ownership of the apachebeam organization, it was quick and
>> easy to add them. I imagine we'd have had to get approval from someone
>> outside the project to do that under the apache org. But this won't need to
>> happen very often, so it's probably not that big a deal.
>> >>
>> >> On Fri, Jan 10, 2020 at 1:40 PM Ahmet Altay <al...@google.com> wrote:
>> >>>
>> >>> Hi all,
>> >>>
>> >>> I saw recent progress on the containers and wanted to bring this
>> question to the attention of the dev list.
>> >>>
>> >>> Would it be possible to use the official ASF dockerhub organization
>> for new Beam container releases? Concretely, starting from 2.19 could we
>> release Beam containers to https://hub.docker.com/u/apache instead of
>> https://hub.docker.com/u/apachebeam ?
>> >>>
>> >>> Ahmet
>>
>

Re: [PROPOSAL] Transition released containers to the official ASF dockerhub organization

[Ankur Goenka <go...@google.com>]

Also curious to know if apache provide any infra support fro projects under
Apache umbrella and any quota limits they might have.

On Fri, Jan 10, 2020, 2:26 PM Robert Bradshaw <ro...@google.com> wrote:

> One downside is that, unlike many of these projects, we release a
> dozen or so containers. Is there exactly (and only) one level of
> namespacing/nesting we can leverage here? (This isn't a blocker, but
> something to consider.)
>
> On Fri, Jan 10, 2020 at 2:06 PM Hannah Jiang <ha...@google.com>
> wrote:
> >
> > Thanks Ahmet for proposing it.
> > I will take it and work towards v2.19.
> >
> > Hannah
> >
> > On Fri, Jan 10, 2020 at 1:50 PM Kyle Weaver <kc...@google.com> wrote:
> >>
> >> It'd be nice to have the clout/official sheen of apache attached to our
> containers. Although getting the required permissions might add some small
> overhead to the release process. For example, yesterday, when we needed to
> create new repositories (not just update existing ones), since we have
> top-level ownership of the apachebeam organization, it was quick and easy
> to add them. I imagine we'd have had to get approval from someone outside
> the project to do that under the apache org. But this won't need to happen
> very often, so it's probably not that big a deal.
> >>
> >> On Fri, Jan 10, 2020 at 1:40 PM Ahmet Altay <al...@google.com> wrote:
> >>>
> >>> Hi all,
> >>>
> >>> I saw recent progress on the containers and wanted to bring this
> question to the attention of the dev list.
> >>>
> >>> Would it be possible to use the official ASF dockerhub organization
> for new Beam container releases? Concretely, starting from 2.19 could we
> release Beam containers to https://hub.docker.com/u/apache instead of
> https://hub.docker.com/u/apachebeam ?
> >>>
> >>> Ahmet
>

Re: [PROPOSAL] Transition released containers to the official ASF dockerhub organization

[Robert Bradshaw <ro...@google.com>]

One downside is that, unlike many of these projects, we release a
dozen or so containers. Is there exactly (and only) one level of
namespacing/nesting we can leverage here? (This isn't a blocker, but
something to consider.)

On Fri, Jan 10, 2020 at 2:06 PM Hannah Jiang <ha...@google.com> wrote:
>
> Thanks Ahmet for proposing it.
> I will take it and work towards v2.19.
>
> Hannah
>
> On Fri, Jan 10, 2020 at 1:50 PM Kyle Weaver <kc...@google.com> wrote:
>>
>> It'd be nice to have the clout/official sheen of apache attached to our containers. Although getting the required permissions might add some small overhead to the release process. For example, yesterday, when we needed to create new repositories (not just update existing ones), since we have top-level ownership of the apachebeam organization, it was quick and easy to add them. I imagine we'd have had to get approval from someone outside the project to do that under the apache org. But this won't need to happen very often, so it's probably not that big a deal.
>>
>> On Fri, Jan 10, 2020 at 1:40 PM Ahmet Altay <al...@google.com> wrote:
>>>
>>> Hi all,
>>>
>>> I saw recent progress on the containers and wanted to bring this question to the attention of the dev list.
>>>
>>> Would it be possible to use the official ASF dockerhub organization for new Beam container releases? Concretely, starting from 2.19 could we release Beam containers to https://hub.docker.com/u/apache instead of https://hub.docker.com/u/apachebeam ?
>>>
>>> Ahmet

Re: [PROPOSAL] Transition released containers to the official ASF dockerhub organization

[Hannah Jiang <ha...@google.com>]

Thanks Ahmet for proposing it.
I will take it and work towards v2.19.

Hannah

On Fri, Jan 10, 2020 at 1:50 PM Kyle Weaver <kc...@google.com> wrote:

> It'd be nice to have the clout/official sheen of apache attached to our
> containers. Although getting the required permissions might add some small
> overhead to the release process. For example, yesterday, when we needed to
> create new repositories (not just update existing ones), since we have
> top-level ownership of the apachebeam organization, it was quick and easy
> to add them. I imagine we'd have had to get approval from someone outside
> the project to do that under the apache org. But this won't need to happen
> very often, so it's probably not that big a deal.
>
> On Fri, Jan 10, 2020 at 1:40 PM Ahmet Altay <al...@google.com> wrote:
>
>> Hi all,
>>
>> I saw recent progress on the containers and wanted to bring this question
>> to the attention of the dev list.
>>
>> Would it be possible to use the official ASF dockerhub organization for
>> new Beam container releases? Concretely, starting from 2.19 could we
>> release Beam containers to https://hub.docker.com/u/apache instead of
>> https://hub.docker.com/u/apachebeam ?
>>
>> Ahmet
>>
>

Re: [PROPOSAL] Transition released containers to the official ASF dockerhub organization

[Kyle Weaver <kc...@google.com>]

It'd be nice to have the clout/official sheen of apache attached to our
containers. Although getting the required permissions might add some small
overhead to the release process. For example, yesterday, when we needed to
create new repositories (not just update existing ones), since we have
top-level ownership of the apachebeam organization, it was quick and easy
to add them. I imagine we'd have had to get approval from someone outside
the project to do that under the apache org. But this won't need to happen
very often, so it's probably not that big a deal.

On Fri, Jan 10, 2020 at 1:40 PM Ahmet Altay <al...@google.com> wrote:

> Hi all,
>
> I saw recent progress on the containers and wanted to bring this question
> to the attention of the dev list.
>
> Would it be possible to use the official ASF dockerhub organization for
> new Beam container releases? Concretely, starting from 2.19 could we
> release Beam containers to https://hub.docker.com/u/apache instead of
> https://hub.docker.com/u/apachebeam ?
>
> Ahmet
>