You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@geode.apache.org by Sean Busbey <bu...@cloudera.com> on 2015/06/20 06:56:29 UTC
Foundation policy on releases and Geode nightly builds
Hi Folks!
I've been involved with the Apache NiFi (incubating) community and they
recently noticed[1] that y'all have nightly builds set up on the Docker
central registry[2]. They've been wanting to do something similar for some
time, but thus far have been held back by policy.
Reading through the ASF policy on releases[3], this looks to me like a
violation of the policy of only making releases available outside of the
development community.
1) The short version of the ASF policy is that a project must not encourage
end users to use anything other than releases that have been voted on by a
PMC (and for incubating projects the IPMC).
2) The Docker Hub is external to the foundation, generally accessible to
those outside of the development community, and expressly geared towards
pushing to downstream users.
Docker Hub can also be limited to distributing images just within a
development team, however you appear to be pointing those outside of the
dev@geode list at the image.
3) You have a wiki page that details making use of the image on Docker
Hub[4].
That page is a subpage of a wiki section entitled "Develop", so it might be
intended for dev@geode use, but it is not obvious from the page.
Additionally, your public facing twitter account posted a link to said
page[5].
4) You have a public facing blog post that points folks to both the Docker
Hub image and a direct download of a nightly build tarball[6].
----
Please clean all of this up. I can empathize with the desire for a faster
feedback cycle with end users. As many at the ASF, I'm a proponent of tight
feedback cycles; but the apparent conflict between foundation policy and
this kind of publishing means I have to push eager communities (like
yourselves and the NiFi community) to constrain themselves to properly PMC
blessed releases.
As an aside, your main page includes a "Download" link that points people
to the source repository and build instructions (rather than keeping that
information under something like "contribute" or "get the source" or
similar caveats other projects use). That's not against the letter of the
ASF policy, since it is not packaged, but it seems against the spirit.
[1]: http://s.apache.org/c4z
[2]: https://registry.hub.docker.com/u/apachegeode/geode/
[3]: http://www.apache.org/dev/release.html#what
[4]:
https://cwiki.apache.org/confluence/display/GEODE/How+to+use+Geode+on+Docker
[5]: https://twitter.com/ApacheGeode/status/612049863971180544
[6]: https://blogs.apache.org/geode/entry/welcome
--
Sean
Re: Foundation policy on releases and Geode nightly builds
Posted by William Markito <wm...@pivotal.io>.
On Thu, Jul 2, 2015 at 11:06 AM, Roman Shaposhnik <ro...@shaposhnik.org>
wrote:
> On Thu, Jul 2, 2015 at 7:10 AM, Sean Busbey <bu...@cloudera.com> wrote:
> > Thanks for the clean up thus far.
>
> Didn't I say we expect to finish up taking care of actionable
> feedback by the end of the week? ;-)
>
> > A few points of feedback:
> >
> > * The blog post still points folks to the nightly builds. and still says
> > there is an "official image" available at Docker Hub
> >
> > * Given its existing exposure, the wiki page on using Docker with Geode
> > could use a disclaimer that the current instructions are only for
> > developers involved in working on Geode (perhaps with a "get involved"
> > link). Presuming y'all will add user-facing docker instructions later, a
> > ticket that tracks that for followup would be nice to include.
>
> Yup. Those two are definitely on my list. The other one is brining
> Dockerfile into the Geode's repo so we can break the link between
> an ASF-looking Docker repo and a random github project (sorry, William
> ;-)).
>
No problem. As soon as the code drop happens, I'll happily move the
Dockerfile into our repo.
>
> Thanks,
> Roman.
>
--
William Markito Oliveira
-- For questions about Apache Geode, please write to
*dev@geode.incubator.apache.org
<de...@geode.incubator.apache.org>*
Re: Foundation policy on releases and Geode nightly builds
Posted by Roman Shaposhnik <ro...@shaposhnik.org>.
On Thu, Jul 2, 2015 at 7:10 AM, Sean Busbey <bu...@cloudera.com> wrote:
> Thanks for the clean up thus far.
Didn't I say we expect to finish up taking care of actionable
feedback by the end of the week? ;-)
> A few points of feedback:
>
> * The blog post still points folks to the nightly builds. and still says
> there is an "official image" available at Docker Hub
>
> * Given its existing exposure, the wiki page on using Docker with Geode
> could use a disclaimer that the current instructions are only for
> developers involved in working on Geode (perhaps with a "get involved"
> link). Presuming y'all will add user-facing docker instructions later, a
> ticket that tracks that for followup would be nice to include.
Yup. Those two are definitely on my list. The other one is brining
Dockerfile into the Geode's repo so we can break the link between
an ASF-looking Docker repo and a random github project (sorry, William ;-)).
Thanks,
Roman.
Re: Foundation policy on releases and Geode nightly builds
Posted by Sean Busbey <bu...@cloudera.com>.
On Wed, Jul 1, 2015 at 5:19 PM, Roman Shaposhnik <ro...@shaposhnik.org>
wrote:
> On Wed, Jul 1, 2015 at 3:14 PM, Niall Pemberton
> <ni...@gmail.com> wrote:
> > For anyone interested whos not subscribed to
> general@incubator.apache.org -
> > you can see the discussion here:
> >
> > http://markmail.org/message/7ijv774ptan7qs3b
>
> You've beaten me to the punch ;-) We're also taking care of the actionable
> feedback that was provided on the thread. Hope to fully close the loop on
> this
> next week sometime.
>
> Thanks,
> Roman.
>
Thanks for the clean up thus far. A few points of feedback:
* The blog post still points folks to the nightly builds. and still says
there is an "official image" available at Docker Hub
* Given its existing exposure, the wiki page on using Docker with Geode
could use a disclaimer that the current instructions are only for
developers involved in working on Geode (perhaps with a "get involved"
link). Presuming y'all will add user-facing docker instructions later, a
ticket that tracks that for followup would be nice to include.
--
Sean
Re: Foundation policy on releases and Geode nightly builds
Posted by Roman Shaposhnik <ro...@shaposhnik.org>.
On Wed, Jul 1, 2015 at 3:14 PM, Niall Pemberton
<ni...@gmail.com> wrote:
> For anyone interested whos not subscribed to general@incubator.apache.org -
> you can see the discussion here:
>
> http://markmail.org/message/7ijv774ptan7qs3b
You've beaten me to the punch ;-) We're also taking care of the actionable
feedback that was provided on the thread. Hope to fully close the loop on this
next week sometime.
Thanks,
Roman.
Re: Foundation policy on releases and Geode nightly builds
Posted by Niall Pemberton <ni...@gmail.com>.
For anyone interested whos not subscribed to general@incubator.apache.org -
you can see the discussion here:
http://markmail.org/message/7ijv774ptan7qs3b
Niall
On Tue, Jun 23, 2015 at 5:39 AM, Roman Shaposhnik <ro...@shaposhnik.org>
wrote:
> On Sat, Jun 20, 2015 at 3:15 PM, Sean Busbey <bu...@cloudera.com> wrote:
> > It's problematic to reference non-public lists that other folks can't go
> > follow along with.
>
> What's this supposed to mean? I can't really change the fact that this
> discussion *already* happened on the list that not all have access to.
> Still I felt given your membership status it was relevant to mention it
> (since you have no problem accessing it). Same applies to all the other
> mentors here. Not sure what you find 'problematic' there.
>
> > I re-read that thread on infrastructure@, and I don't
> > see anyone bring up the matter of nightly builds. All the support is
> around
> > publishing docker images that contain released software.
>
> I don't understand what gave you that impression. I wasn't really
> subsetting
> that discussion to the 'images that contain released software' but was
> asking an open ended question.
>
> > AFAIK, the current policy would apply equally to SNAPSHOTs put in the
> Maven
> > repo. That is, those SNAPSHOT artifacts are for the development community
> > *only* and they must not be pointed to for downstream users.
>
> This is where we have different opinions interpreting the policy. The best
> way to resolve this disagreement is on general@ and not a poddling mailing
> list. Once this disagreement is resolved either of us can follow up with
> a poddling.
>
> > Your point in that private list about Maven Central and Docker Hub is
> very
> > relevant; I agree they are essentially the same kind of
> > publish-to-the-public access point. While we have SNAPSHOT artifacts
> posted
> > to the ASF maven repo, that repo is not mirrored into Maven Central
> because
> > it would be against foundation policy.
> >
> > What the Geode PMC is currently doing is the equivalent to a project
> > publishing the SNAPSHOT artifacts to Maven Central. I hope we are all in
> > agreement that that would be inappropriate.
>
> Actually no we are not. And like I said an appropriate place to resolve
> this
> disagreement is on general@ at this point. But just to record it here, the
> reason I disagree with your argument is that I see nothing in our policy
> that
> would support the claim that it makes any difference of whether the
> artifacts
> are published on ASF managed INFRA or not. What I see is this:
> ===============================================================
> "If the general public is being instructed to download a package, then
> that package
> has been released."
> ===============================================================
> it matters not where this package is residing.
>
> > I have also seen lots of folks successfully use Docker images to do build
> > automation. That's not related to the matter at hand,
>
> I agree. I was only using it as an example of something that a project
> may want to publish under its 'official' (whatever that means) account
> on Docker hub. The project will be, then, fully within its right to
> communicate
> to the 'general public' that the recommended way of building it is:
> $ docker run FOO
>
> My point here is: even when such a communication happens, I hope
> we both can agree that the build related docker container should NOT
> be considered as part of a project binary release (and shouldn't be covered
> by ASF's release policy)
>
> > which is publishing
> > to the Docker Hub. Nothing other than Geode showed up in a superficial
> > search for nightly builds from ASF projects.
>
> Take a look at https://registry.hub.docker.com/u/bigtop/slaves/ Those
> images
> are supposed to be updated every time the build infra (as defined by
> Bigtop's
> puppet code) changes.
>
> IOW, the content of these images is keyed off of CI that triggers from the
> unreleased Bigtop code checked in.
>
> > The tweet from the Geode PMC, the blog post, and a quick search of
> twitter
> > for additional references makes discussion of possible uses of docker and
> > the hub irrelevant. The docker image on Docker Hub is of non released
> > software and is being used outside of the development community.
>
> Like I said we have different interpretations of 'development community'.
> Yours
> is narrow, mine includes downstream developers integrating with the
> project.
>
> > It needs to be removed.
>
> We may very well end up doing that, but not until there's a legitimate
> discussion
> clarifying the situation.
>
> > I encourage more discussion of this on general@incubator (though the
> > release policy would have to go to legal-discuss),
>
> That is actually not clear to me. From a legal perspective ASF is all about
> open *source* development. Having seen too many "we don't even recognize
> binary convenience artifacts" threads over the years I won't be surprised
> if
> the issue gets punted back to the board/comdev.
>
> I'd be very curious to see how it shakes out, sine I believe it is high
> time
> we finally clarify this part of ASF's policy once and for all.
>
> Once again, thanks for bringing the inconsistencies to light -- I am very
> much looking forward to our productive discussion on general@
>
> Thanks,
> Roman.
>
Re: Foundation policy on releases and Geode nightly builds
Posted by Roman Shaposhnik <ro...@shaposhnik.org>.
On Sat, Jun 20, 2015 at 3:15 PM, Sean Busbey <bu...@cloudera.com> wrote:
> It's problematic to reference non-public lists that other folks can't go
> follow along with.
What's this supposed to mean? I can't really change the fact that this
discussion *already* happened on the list that not all have access to.
Still I felt given your membership status it was relevant to mention it
(since you have no problem accessing it). Same applies to all the other
mentors here. Not sure what you find 'problematic' there.
> I re-read that thread on infrastructure@, and I don't
> see anyone bring up the matter of nightly builds. All the support is around
> publishing docker images that contain released software.
I don't understand what gave you that impression. I wasn't really subsetting
that discussion to the 'images that contain released software' but was
asking an open ended question.
> AFAIK, the current policy would apply equally to SNAPSHOTs put in the Maven
> repo. That is, those SNAPSHOT artifacts are for the development community
> *only* and they must not be pointed to for downstream users.
This is where we have different opinions interpreting the policy. The best
way to resolve this disagreement is on general@ and not a poddling mailing
list. Once this disagreement is resolved either of us can follow up with
a poddling.
> Your point in that private list about Maven Central and Docker Hub is very
> relevant; I agree they are essentially the same kind of
> publish-to-the-public access point. While we have SNAPSHOT artifacts posted
> to the ASF maven repo, that repo is not mirrored into Maven Central because
> it would be against foundation policy.
>
> What the Geode PMC is currently doing is the equivalent to a project
> publishing the SNAPSHOT artifacts to Maven Central. I hope we are all in
> agreement that that would be inappropriate.
Actually no we are not. And like I said an appropriate place to resolve this
disagreement is on general@ at this point. But just to record it here, the
reason I disagree with your argument is that I see nothing in our policy that
would support the claim that it makes any difference of whether the artifacts
are published on ASF managed INFRA or not. What I see is this:
===============================================================
"If the general public is being instructed to download a package, then
that package
has been released."
===============================================================
it matters not where this package is residing.
> I have also seen lots of folks successfully use Docker images to do build
> automation. That's not related to the matter at hand,
I agree. I was only using it as an example of something that a project
may want to publish under its 'official' (whatever that means) account
on Docker hub. The project will be, then, fully within its right to communicate
to the 'general public' that the recommended way of building it is:
$ docker run FOO
My point here is: even when such a communication happens, I hope
we both can agree that the build related docker container should NOT
be considered as part of a project binary release (and shouldn't be covered
by ASF's release policy)
> which is publishing
> to the Docker Hub. Nothing other than Geode showed up in a superficial
> search for nightly builds from ASF projects.
Take a look at https://registry.hub.docker.com/u/bigtop/slaves/ Those images
are supposed to be updated every time the build infra (as defined by Bigtop's
puppet code) changes.
IOW, the content of these images is keyed off of CI that triggers from the
unreleased Bigtop code checked in.
> The tweet from the Geode PMC, the blog post, and a quick search of twitter
> for additional references makes discussion of possible uses of docker and
> the hub irrelevant. The docker image on Docker Hub is of non released
> software and is being used outside of the development community.
Like I said we have different interpretations of 'development community'. Yours
is narrow, mine includes downstream developers integrating with the project.
> It needs to be removed.
We may very well end up doing that, but not until there's a legitimate
discussion
clarifying the situation.
> I encourage more discussion of this on general@incubator (though the
> release policy would have to go to legal-discuss),
That is actually not clear to me. From a legal perspective ASF is all about
open *source* development. Having seen too many "we don't even recognize
binary convenience artifacts" threads over the years I won't be surprised if
the issue gets punted back to the board/comdev.
I'd be very curious to see how it shakes out, sine I believe it is high time
we finally clarify this part of ASF's policy once and for all.
Once again, thanks for bringing the inconsistencies to light -- I am very
much looking forward to our productive discussion on general@
Thanks,
Roman.
Re: Foundation policy on releases and Geode nightly builds
Posted by Sean Busbey <bu...@cloudera.com>.
On Sat, Jun 20, 2015 at 1:58 AM, Roman Shaposhnik <rv...@apache.org> wrote:
> On Fri, Jun 19, 2015 at 9:56 PM, Sean Busbey <bu...@cloudera.com> wrote:
> > Reading through the ASF policy on releases[3], this looks to me like a
> > violation of the policy of only making releases available outside of the
> > development community.
>
> A version of this question was raised when I was asking how soon ASF
> can have its own Docker registry (the discussion was on infra@)
> Basically it boils down to the fact that as a developer and tester on
> the project
> having a Docker image that I can simply docker run to test/etc. has become
> part of my daily routine. This is as useful as having a -NIGHTLY snapshot
in
> the Maven repo.
>
>
It's problematic to reference non-public lists that other folks can't go
follow along with. I re-read that thread on infrastructure@, and I don't
see anyone bring up the matter of nightly builds. All the support is around
publishing docker images that contain released software.
AFAIK, the current policy would apply equally to SNAPSHOTs put in the Maven
repo. That is, those SNAPSHOT artifacts are for the development community
*only* and they must not be pointed to for downstream users. The current
policy makes it clear that it if a non-released artifact is getting used
outside of the development community that is not okay and needs to be
addressed. That would similarly go for SNAPSHOT maven artifacts.
Your point in that private list about Maven Central and Docker Hub is very
relevant; I agree they are essentially the same kind of
publish-to-the-public access point. While we have SNAPSHOT artifacts posted
to the ASF maven repo, that repo is not mirrored into Maven Central because
it would be against foundation policy. I imagine that once the ASF wide
registry provided by Infra goes live, it will similarly have a non-mirrored
space for within-project use and an area that is mirrored out to Docker Hub
for public facing distribution.
What the Geode PMC is currently doing is the equivalent to a project
publishing the SNAPSHOT artifacts to Maven Central. I hope we are all in
agreement that that would be inappropriate.
> > 1) The short version of the ASF policy is that a project must not
encourage
> > end users to use anything other than releases that have been voted on
by a
> > PMC (and for incubating projects the IPMC).
>
> Sure. I'd agree with you that this is the question of labeling. How to
clearly
> label Docker artifacts not intended for downstream consumption the same
> way we do with -SNAPSHOT Maven artifacts would be a good discussion
> to be had on general@incubator
>
> > 2) The Docker Hub is external to the foundation, generally accessible to
> > those outside of the development community, and expressly geared towards
> > pushing to downstream users.
>
> I don't agree with the last statement. In fact, 50% of what I use Docker
> images within ASF projects is build automation. This has nothing to do
> with using the software as a downstream user.
>
I have also seen lots of folks successfully use Docker images to do build
automation. That's not related to the matter at hand, which is publishing
to the Docker Hub. Nothing other than Geode showed up in a superficial
search for nightly builds from ASF projects.
The tweet from the Geode PMC, the blog post, and a quick search of twitter
for additional references makes discussion of possible uses of docker and
the hub irrelevant. The docker image on Docker Hub is of non released
software and is being used outside of the development community. It needs
to be removed.
> > Docker Hub can also be limited to distributing images just within a
> > development team, however you appear to be pointing those outside of the
> > dev@geode list at the image.
>
> That is a good point.
>
> > 3) You have a wiki page that details making use of the image on Docker
> > Hub[4].
> >
> > That page is a subpage of a wiki section entitled "Develop", so it
might be
> > intended for dev@geode use, but it is not obvious from the page.
> > Additionally, your public facing twitter account posted a link to said
> > page[5].
> >
> > 4) You have a public facing blog post that points folks to both the
Docker
> > Hub image and a direct download of a nightly build tarball[6].
> >
> > ----
> >
> > Please clean all of this up.
>
> Some of it will be cleaned up and updated, some of it requires further
> discussion
> on general@ I'll bring the discussion there on Mon or so.
>
> Thanks for bringing this to our attention.
>
>
> > I can empathize with the desire for a faster
> > feedback cycle with end users. As many at the ASF, I'm a proponent of
tight
> > feedback cycles; but the apparent conflict between foundation policy and
> > this kind of publishing means I have to push eager communities (like
> > yourselves and the NiFi community) to constrain themselves to properly
PMC
> > blessed releases.
>
> Personally, I see these cases as an opportunity to make sure that our
policy
> is in support of of the foundation goals AND the goals of the software
> communities
> we serve. I wish NiFi concerns were brought up to the attention of IPMC.
>
I encourage more discussion of this on general@incubator (though the
release policy would have to go to legal-discuss), but the publishing of
non-released software is a foundation policy with a basis in how we meet
our legal obligations. It is not an area where "better to ask forgiveness"
works. The first step of the discussion to change the policy is to comply
with it.
--
Sean
Re: Foundation policy on releases and Geode nightly builds
Posted by Roman Shaposhnik <rv...@apache.org>.
On Fri, Jun 19, 2015 at 9:56 PM, Sean Busbey <bu...@cloudera.com> wrote:
> Reading through the ASF policy on releases[3], this looks to me like a
> violation of the policy of only making releases available outside of the
> development community.
A version of this question was raised when I was asking how soon ASF
can have its own Docker registry (the discussion was on infra@)
Basically it boils down to the fact that as a developer and tester on
the project
having a Docker image that I can simply docker run to test/etc. has become
part of my daily routine. This is as useful as having a -NIGHTLY snapshot in
the Maven repo.
> 1) The short version of the ASF policy is that a project must not encourage
> end users to use anything other than releases that have been voted on by a
> PMC (and for incubating projects the IPMC).
Sure. I'd agree with you that this is the question of labeling. How to clearly
label Docker artifacts not intended for downstream consumption the same
way we do with -SNAPSHOT Maven artifacts would be a good discussion
to be had on general@incubator
> 2) The Docker Hub is external to the foundation, generally accessible to
> those outside of the development community, and expressly geared towards
> pushing to downstream users.
I don't agree with the last statement. In fact, 50% of what I use Docker
images within ASF projects is build automation. This has nothing to do
with using the software as a downstream user.
> Docker Hub can also be limited to distributing images just within a
> development team, however you appear to be pointing those outside of the
> dev@geode list at the image.
That is a good point.
> 3) You have a wiki page that details making use of the image on Docker
> Hub[4].
>
> That page is a subpage of a wiki section entitled "Develop", so it might be
> intended for dev@geode use, but it is not obvious from the page.
> Additionally, your public facing twitter account posted a link to said
> page[5].
>
> 4) You have a public facing blog post that points folks to both the Docker
> Hub image and a direct download of a nightly build tarball[6].
>
> ----
>
> Please clean all of this up.
Some of it will be cleaned up and updated, some of it requires further
discussion
on general@ I'll bring the discussion there on Mon or so.
Thanks for bringing this to our attention.
> I can empathize with the desire for a faster
> feedback cycle with end users. As many at the ASF, I'm a proponent of tight
> feedback cycles; but the apparent conflict between foundation policy and
> this kind of publishing means I have to push eager communities (like
> yourselves and the NiFi community) to constrain themselves to properly PMC
> blessed releases.
Personally, I see these cases as an opportunity to make sure that our policy
is in support of of the foundation goals AND the goals of the software
communities
we serve. I wish NiFi concerns were brought up to the attention of IPMC.
Thanks,
Roman.