You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@spark.apache.org by Sean Owen <sr...@apache.org> on 2019/08/02 04:09:23 UTC

Re: Recognizing non-code contributions

(Let's move this thread to dev@ now as it is a general and important
community question. This was requested on members@)

On Thu, Aug 1, 2019 at 10:20 PM Matei Zaharia <ma...@gmail.com> wrote:
>
> Our text on becoming a committer already says that we want committers who focus on our docs: https://spark.apache.org/committers.html. Just working on docs / books outside the project doesn’t make as much sense IMO.
>
> Matei
>
> On Aug 1, 2019, at 8:10 PM, Reynold Xin <rx...@databricks.com> wrote:
>
> Not sure if ASF allows this but we can start some sort of “MVP” program recognizing non-code contributors...
>
> On Thu, Aug 1, 2019 at 7:18 PM Wenchen Fan <cl...@gmail.com> wrote:
>>
>> I agree with the concerns about commit bit. If he maintains the Spark documentation himself, and doesn't need to commit code/merge PRs, then the commit bit is not needed but only as a form of recognition.
>>
>> Note that, we do have documentation in the Spark codebase. If he starts to reorganize/improve the document in the Spark codebase, and make significant progress, I think the commit bit is necessary and I'm +1 to make him a committer.
>>
>> On Fri, Aug 2, 2019 at 9:54 AM Felix Cheung <fe...@apache.org> wrote:
>>>
>>> Given the feedback here, I’d like to ask
>>>
>>> a) can this kind of documentation example guide be included into spark project proper or spark-website? Or can they be included in the newly mint ASF training repo?
>>>
>>> b) in the absence of “commit” even for documentation, can we come up with ways to recognize contributions to the community?
>>>
>>>
>>>
>>> On Thu, Aug 1, 2019 at 6:13 PM Hyukjin Kwon <gu...@gmail.com> wrote:
>>>>
>>>> I agree with Sean in general, in particular, commit bit.
>>>>
>>>> Personal thought:
>>>>   I think committer should at least be used to the dev at some degree as primary.
>>>>   Other contributions should be counted of course (and I emphasize JIRA management) but it's secondary.
>>>>
>>>>
>>>> 2019년 8월 2일 (금) 오전 1:32, Sean Owen <sr...@apache.org>님이 작성:
>>>>>
>>>>> First let me say this is my opinion and a philosophical question on
>>>>> which people disagree, but I do feel pretty clear on my take, FWIW.
>>>>>
>>>>> If someone does a great job maintaining a Spark tutorial website,
>>>>> writes a good book, runs good Spark meetups, manages social media for
>>>>> the project or something, could such a person become a committer? It's
>>>>> often said on ASF lists, and it came up again this week, that 'yes'
>>>>> this person is 'committed' to the project so can be a 'committer'.
>>>>>
>>>>> I'd say 'no'. The reasoning above is more wordplay than an argument.
>>>>> Making someone a committer does a specific thing: gives them the
>>>>> commit bit. Someone who never commits anything never needs it, so,
>>>>> why? The counter-argument is, what does it hurt then? but I think this
>>>>> cuts the former way. (OK, it also gives them some additional power
>>>>> over others' code changes, but, same argument I think.)
>>>>>
>>>>> Of course the commit bit is widely understood as a form of recognition
>>>>> anyway. I think people really mean, this kind of person deserves
>>>>> recognition, and it's the only real official badge the project can
>>>>> hand out. I personally don't think that justifies use of the commit
>>>>> bit as purely recognition. If a lot of the ASF's legal scaffolding
>>>>> rests on responsible oversight of software releases, I suppose there
>>>>> is a distant and theoretical downside to giving commit access
>>>>> 'unnecessarily'.
>>>>>
>>>>> I'd prefer to simply recognize people in this category. But I admit,
>>>>> there's not a great way to do it other than say 'thank you'.
>>>>>
>>>>>
>>>>> What about someone who primarily writes doc changes, updates the
>>>>> website? maybe fixes tests? I'd say 'yes' that could be a basis for
>>>>> earning the commit bit, because those things require commit access to
>>>>> change. It doesn't have to be code per se. See for example Shane, who
>>>>> mostly tends the build and fixes scripts. Same argument.
>>>>>
>>>>>
>>>>> I'll say that someone's non-code, non-project contributions can factor
>>>>> in to deciding whether they should get the commit bit. Someone who
>>>>> does reviews and proved they can use commit access wisely, and know
>>>>> enough to tend some part of the code well, _and also demonstrates that
>>>>> expertise outside the project_, is a bit better than someone who
>>>>> doesn't. It's not irrelevant, but I think isn't sufficient on its own
>>>>> to justify getting the project code/doc commit bit.

---------------------------------------------------------------------
To unsubscribe e-mail: dev-unsubscribe@spark.apache.org


Re: Recognizing non-code contributions

Posted by Myrle Krantz <my...@apache.org>.
On Sun, Aug 4, 2019 at 7:13 PM Sean Owen <sr...@apache.org> wrote:

> On Sun, Aug 4, 2019 at 11:21 AM Myrle Krantz <my...@apache.org> wrote:
> > Let me make a guess at what you are trying to accomplish with it.
> Correct me please if I'm wrong:
> > * You want to encourage contributions that aren't just code
> contributions.  You recognize for example that good documentation is
> critical to bringing new contributors and committers to your project.  You
> recognize that other non-code contributions are also important to your
> project's success.
>
> No, I think the position here was docs-only contributors _could_ be
> committers. The "only coders can be committers" idea was I think a
> misreading and accidental strawman from the members@ thread.
>

Can you be specific?  This status us supposed to serve as a form of
recognition, right?  For who or what contributions?


>
> > * You want to keep access to the code repositories "tight", ie minimized
> to people who you view as likely to be actively using that access for the
> goals of the project.  In particular, someone who writes tutorials, may not
> even want access to the spark code repositories.
>
> Yes, but in the narrow sense that someone with zero interaction with
> the repo doesn't seem to need any repo access. That may actually be
> uncontroversial?
>
> There are tougher question here which we've always grappled with:
> - What about someone who's stopped contributing but did some good work
> in the past? they probably won't use the commit bit or contribute.
> Generally I think we've recognized 'past-only' merit with the commit
> bit.
>

We have a saying at Apache: merit doesn't expire.  This is especially
important for volunteer-based project work.  People's availability varies.
If the barrier to re-entry is low, you're more likely to get contributions
from these people.  Contributions that can have a particularly high value
because they are informed by a longer history of the project.

But the real question which I feel needs an answer here is: what's the harm
you fear?  With modern source control, surely any harm is reversible?  If
you're worried about people doing things outside their "abilities", then
maybe the real question you need to ask when choosing committers is just:
do they know to ask for help first in fragile parts of the code?  You may
find that someone who is doesn't see themselves as a coder is *less* likely
to break code than other committers.


> - What about someone who's made a lot of code contributions but
> doesn't review changes? have they demonstrated the skill that we need
> people to deploy when they have the commit bit, which is knowing when
> _others_ changes are OK to merge? Again I think we've only distantly
> considered that, and there are committers who are almost entirely
> contributors and not reviewers
>

This is really common in Apache projects.  But even if they never merge
anyone else's PR, giving them the commit bit reduces the load on those who
do merge PR's for their *own* PRs.  And increases the likelihood that they
will eventually look at someone else's PR.  If you're trying to recognize
someone who reviews other people's PRs, maybe you're looking for an honor
*in addition to* committer rather than instead of?


> ... though I don't think we were questioning that here. (Maybe we should)
>
>
> > Other arguments I've heard for this (which you may or may not share)
> > * Fear of "diluting" the social value of committership in your project.
>
> I think the goal is to tease apart being a committer as having a
> particular important power in the repo, and being a committer as
> recognition and valuable social status. I don't think there's a worry
> that too many committers dilute the 'brand'. On the contrary, I'd say
> we _don't_ want mere recognition to be a factor in handing out this
> access? Recognition ought to be more freely available, without
> debating the effects of giving commit access.
>

If the only question is "commitment" to the project, then you shouldn't
need to worry about people breaking things who have a commit bit.  People
who are committed to the project won't break things on purpose, and they'll
quickly learn to not break things on access too.  That's part of the
definition of commitment.

We've never teases these things apart at Apache because it simply hasn't
been necessary.  Just like in SVN there are (or were?  I hope an SVN
committer jumps in here) committers for certain areas of the code.  Those
areas of the code were only ever defined socially.  And that never caused
them problems for *years* of development.  If n is the percent of code that
an sub-module committer can touch, you can also imagine cases where n =
0%.  And I think every project over a certain complexity *implicitly* has
sub-module committers.  People know about certain areas of the code and not
about other areas of the code.  They know to get their stuff reviewed when
they're working with code they're less familiar with.

> And here are some possible effects of creating a second category of
> recognition for non-code contributions which does not include commit access:
> > * things that aren't rewarded with committership are somehow viewed as
> being of lesser value to the project.  (It's already true in the industry
> that documentation writers, marketers, and event-organizers are sometime
> looked down on by programmers.)
>
> Agree, though, again, I don't think anyone is saying docs-only
> contributors can't be committers here.
> Really I just think it's unfortunate how much the commit bit is
> considered a 'reward' - maybe that's the problem.
>

We really shouldn't treat committer status like a reward.  It's just a
permission: permission to participate more fully in the project.  That
permission includes, but is not limited to the commit bit.  We should be
thanking people who accept those permissions because it means they intend
to continue contributing to the project.

>
> You hit on one of my concerns. In theory some "VIP" status is better
> than nothing, but if it's viewed as second-class status more than
> anything, it backfires. That is, does it become perceived as a badge
> of "someone the PMC has definitely decided won't be a committer,
> because they decided to hand out VIP status instead"? It could be
> worse than having no status, which indicates nothing about a PMC
> decision.
>

I hadn't even thought about the possibility of it being a case of "damned
by faint praise".  But you're right.  That is a possible concern.


>
> > * People who want to work on spark may be more likely to chose to fork
> your repository rather than work with it directly.
> > * People who start off in other areas but have minor contributions to
> make to your code may never make those contributions (for example someone
> who does manual QA may never make that first step to automate manual work,
> or improve existing automated tests.  Documentation writers may never
> bother to create that pull request on your grammar errors for your inline
> documentation, so inline documentation may become a less-maintained second
> class citizen.)
>
> Like, if not given the commit bit, they'll work only outside the repo?
> I am not sure this is a risk? we're already talking about people who
> contribute to Spark without being a committer. Would they stop if we
> gave them VIP status but not the commit bit? (the risk there is as
> above I think)
>
> Or is the idea that if one doesn't have the rights to merge one's own
> changes rapidly, they'll stop? That's better solved by ensuring that
> good patches get reviewed and merged quickly, which is a big, separate
> problem.
>

I've seen this happen.  Especially when merges start slowing down, people
either stop contributing altogether or just work from their own fork.  If
you have enough "non-meriging" committers in your volunteer "pipeline" you
have a better chance of people stepping up and becoming "merging"
committers.  And at very least you remove a roadblock to participation when
you make someone a committer.  Otherwise, you could accidentally doom your
project to a lifespan equal to the volunteer burnout cycle of your one most
dedicated volunteer.  Or accidentally couple it too tightly to the economic
interests of that person's employer.  Either way, you'll shorten your
project's lifespan unnecessarily.  And that would be a shame, because Spark
is a really awesome project.

Best,
Myrle
(Sorry Sean, I accidentally sent this to just you first.)

Re: Recognizing non-code contributions

Posted by Nicholas Chammas <ni...@gmail.com>.
On Mon, Aug 5, 2019 at 9:55 AM Sean Owen <sr...@apache.org> wrote:

> On Mon, Aug 5, 2019 at 3:50 AM Myrle Krantz <my...@apache.org> wrote:
> > So... events coordinators?  I'd still make them committers.  I guess I'm
> still struggling to understand what problem making people VIP's without
> giving them committership is trying to solve.
>
> We may just agree to disagree, which is fine, but I think the argument
> is clear enough: such a person has zero need for the commit bit.
> Turning it around, what are we trying to accomplish by giving said
> person a commit bit? I know people say there's no harm, but I think
> there is at least _some_ downside. We're widening access to change
> software artifacts, the main thing that we put ASF process and checks
> around for liability reasons. I know the point is trust, and said
> person is likely to understand to never use the commit bit, but it
> brings us back to the same place. I don't wish to convince anyone else
> of my stance, though I do find it more logical, just that it's
> reasonable within The Apache Way.
>

+1 to this.

Re: Recognizing non-code contributions

Posted by Myrle Krantz <my...@apache.org>.
Hey William,

Thanks for pointing this inconsistency out out.

Best Regards,
Myrle

On Fri, Aug 9, 2019 at 12:22 AM William Shen <wi...@marinsoftware.com>
wrote:

> Not sure where this thread is heading toward, but I find the role
> definition listed on
> http://www.apache.org/foundation/how-it-works.html#roles clarifying and
> well defined, though they seem to vary slightly from what is on
> https://community.apache.org/contributors/. Not sure which one is more
> authoritative.
>
> In the context of this thread, here is the difference as I see it:
>
> On www.apache.org
> <http://www.apache.org/foundation/how-it-works.html#roles>,
>
>> A *committer* is a *developer* that was given write access to the code
>> repository and has a signed Contributor License Agreement (CLA) on file.
>> A *developer* is a *user* who contributes to a project in the form of *code
>> or documentation*.
>
> Hence, a committer is a user, who contributes code or documentation, that
> was given write access to the repository.
>
> On community.apache.org <https://community.apache.org/contributors/>,
>
>> Committer is a term used at the ASF to signify someone who is committed
>> to a particular project. It brings with it the privilege of write access to
>> the project repository and resources.
>> The foundations of an Apache project and how the community contributes to
>> it is sometimes referred to by the acronym CoPDoC: (Co)mmunity - one must
>> interact with others, and share vision and knowledge (P)roject - a clear
>> vision and consensus are needed (Do)cumentation - without it, the stuff
>> remains only in the minds of the authors (C)ode - discussion goes nowhere
>> without code
>> once someone has contributed sufficiently to any area of CoPDoC they can
>> be voted in as a committer.
>
> Hence, a committer is someone, who has contributed sufficiently to *any*
> area of Community, Project, Documentation, and Code, that has been given
> write access to the project repository and resources.
>
> The key difference between the two definitions is: Can someone, who
> contribute significantly in the area of Community or Project without
> contribution to Documentation nor Code, be a committer? According to
> www.apache.org <http://www.apache.org/foundation/how-it-works.html#roles>,
> no; according to community.apache.org
> <https://community.apache.org/contributors/>, yes. And that difference
> seems to be a main point of discussion here.
>
> By www.apache.org
> <http://www.apache.org/foundation/how-it-works.html#roles>, a committer
> seems to be a role designed to create a subset of users to "making
> short-term decisions for the project," and I think that reference to the
> ability to commit a patch for Code or Documentation in the project's
> repository or resources. And I think that is sensible, and may be in
> support of limiting committership to those, who contribute sufficiently
> code or documentation to a project's repository or resources, that the
> project invites to make future commit decisions for the project.
>
> On Wed, Aug 7, 2019 at 6:07 AM Hyukjin Kwon <gu...@gmail.com> wrote:
>
>> > Currently, I have heard some ideas or attitudes that I consider to be
>> overly motivated by fear of unlikely occurrences.
>> > And I've heard some statements disregard widely accepted principles of
>> inclusiveness at the Apache Software Foundation.
>> > But I suspect that there's more to the attitude of not including
>> non-coding committers at Spark.
>>
>> I missed some contexts you mentioned. Yes, SVN and commons look good
>> examples.
>> Also, for clarification, I did not mean to absolutely do not add
>> non-codding committers.
>> Spark already has build/config committer and I am happy with that.
>>
>> I was replaying to "the risk is very small". Given my experience in Spark
>> dev, people (and I) make mistakes
>> which, for instance, blocks the release months. Sometimes it requires to
>> rewrite whole PRs with courtesy
>> (rather than, for instance, reverting). This is already happening and it
>> brings some overhead to the dev.
>> Yes, maybe the volume matters to handle those issues.
>>
>> The point I was trying to make was commit bit could be a too strong sword
>> and might have to be
>> given and used with familiarity and caution.
>>
>> For clarification, I have no issue except one concern above for the fact
>> that someone becomes a non-code committer since Spark already has it.
>>
>>
>> 2019년 8월 7일 (수) 오후 6:04, Myrle Krantz <my...@apache.org>님이 작성:
>>
>>>
>>>
>>> On Tue, Aug 6, 2019 at 7:57 PM Sean Owen <sr...@apache.org> wrote:
>>>
>>>> On Tue, Aug 6, 2019 at 11:45 AM Myrle Krantz <my...@apache.org> wrote:
>>>> > I had understood your position to be that you would be willing to
>>>> make at least some non-coding contributors to committers but that your
>>>> "line" is somewhat different than my own.   My response to you assumed that
>>>> position on your part.  I do not think it's good for a project to accept
>>>> absolutely no non-code committers.  If nothing else, it violates my sense
>>>> of fairness, both towards those contributors, and also towards the ASF
>>>> which relies on a pipeline of non-code contributors who come to us through
>>>> the projects.
>>>>
>>>> Oh OK, I thought this argument was made repeatedly: someone who has
>>>> not and evidently will not ever commit anything to a project doesn't
>>>> seem to need the commit bit. Agree to disagree. That was the
>>>> 'non-code' definition?
>>>>
>>>
>>> That argument was made and acknowledged.  And then answered with:
>>> a.) the commit bit is only part of what makes a committer, and not the
>>> most important part.
>>> b.) including the commit bit in the package is harmless.  The risk
>>> associated with giving someone the commit bit who is not going to use it is
>>> lower than the risk associated with the average pull request.
>>> c.) creating a new package without the commit-bit creates significant
>>> effort and bears significant risks.
>>>
>>>
>>>> Someone who contributes docs to the project? Sure. We actually have
>>>> done this, albeit for a build and config contributions. Agree.
>>>>
>>>> Pardon a complicated analogy to explain my thinking, but: let's say
>>>> the space of acceptable decisions on adding committers at the ASF
>>>> ranges from 1 (Super Aggressive) to 10 (Very Conservative). Most
>>>> project decisions probably fall in, say, 3 to 7. Here we're debating
>>>> whether a project should theoretically at times go all the way to 1,
>>>> or at most 2, and I think that's just not that important. We're pretty
>>>> much agreeing 2 is not out of the question, 1 we agree to disagree.
>>>>
>>>> Spark decisions here are probably 5-7 on average. I'd like it be like
>>>> 4-6 personally. I suspect the real inbound argument is: all projects
>>>> should be making all decisions in 1-3 or else it isn't The Apache Way.
>>>> I accept anecdotes that projects function well in that range, but,
>>>> Spark and Hadoop don't seem to (nor evidently Cassandra). I have a
>>>> hard time rationalizing this. These are, after all, some of the
>>>> biggest and most successful projects at Apache. At times it sounds
>>>> like concern trolling, to 'help' these projects not fall apart.
>>>>
>>>> If so, you read correctly that there is a significant difference of
>>>> opinion here, but that's what it is. Not the theoretical debate above.
>>>>
>>>
>>> I think this misrepresents where the "middle" is in Apache projects.  I
>>> think the middle is probably closer to where OfBiz is: occasionally
>>> offering non-coding contributors committership, but probably not with the
>>> frequency I would like.  But even that occasional committership for
>>> non-coding committers has been extraordinarily important for the ASF as an
>>> organization.  Sharan Foga started as a non-coding contributor for OfBiz,
>>> and is now VP of Community Development at the ASF, and organized the Apache
>>> Roadshow in Berlin last year (where Spark talks were well-received and
>>> probably helped your community). The OfBiz project did us all a huge favor
>>> by providing Sharan with the first step into our organization.
>>>
>>> What you are perceiving as an extreme is the SVN project in which all
>>> you have to do to receive committership is to ask.  Or the commons project
>>> in which in which every ASF member is automatically a committer.  Those
>>> projects can't be claimed to be insensitive to bugs.  SVN is a data
>>> repository: bugs can cause you to lose your code. For commons, programming
>>> mistakes can cause security flaws, compile errors and worse in a massive
>>> number of dependent project.  And yes, I know you've seen all of that
>>> conversation, Sean, but most of it was on members@, so I'm summarizing
>>> here for those who can't see members@.
>>>
>>> If Spark and Hadoop and Cassandra don't include non-coding contributors
>>> in their committer candidate pool, that puts them on the outer extreme of
>>> ASF projects.  These projects are undeniably successful despite this fact.
>>> But I still wouldn't recommend the approach to other ASF projects.  I do
>>> not believe most projects can get away with this and still survive long
>>> term.
>>>
>>>
>>>> Spark should shift, but equally, so should this viewpoint from some at
>>>> the ASF, as I find my caricature of it equally suboptimal.
>>>> Shred that analogy as you like, but it explains what's in my head.
>>>
>>>
>>> I disagree, but if you want to change that position at the ASF, a good
>>> place to start the conversation is on dev@community.
>>>
>>> > For more documentation on the definition of a committer at Apache,
>>>> read here: https://community.apache.org/contributors/  "Being a
>>>> committer does not necessarily mean you commit code, it means you are
>>>> committed to the project and are productively contributing to its success."
>>>>
>>>> Per above, I just don't think this statement should be in the canon,
>>>> and would prefer to clarify it, but hey it is there and I accept it.
>>>> Still: what's committed? I'd define committed to the project, as,
>>>> well, working on the project's output. It just punts the question.
>>>>
>>>
>>> That content is owned by the community project.  The appropriate place
>>> to request a change is on dev@community.
>>>
>>> No project can be successful without QA, design, project management,
>>> user care, documentation, marketing, infrastructure, community building and
>>> more.  Working together to create artifacts is important.  It represents
>>> the community's common goal, and provides a point around which to organize
>>> efforts.  Without non-coding contributions, projects won't be able to
>>> produce those artifacts, or they'll be dependent on corporate patronage, or
>>> they just won't be as good as they could be.
>>>
>>>
>>>> > I also don't yet see a "serious offense" here.  My e-mail to board@
>>>> is simply a heads up, which I do owe the rest of the board when I'm
>>>> interacting with one of our projects.  Here are my exact words: "Most of
>>>> that discussion is fairly harmless.  Some of it, I have found concerning."
>>>> Right now, I'm still trying to approach Spark's position with a
>>>> learning-and-teaching mindset.
>>>>
>>>> I'm nitpicking your words, which are by themselves reasonable. I think
>>>> learning-and-teaching is just the right attitude.
>>>> But have you heard different ideas here that are valid or merely "not
>>>> harmful"? are the ideas you don't share just not your choice or
>>>> "concerning"?
>>>>
>>>
>>> Currently, I have heard some ideas or attitudes that I consider to be
>>> overly motivated by fear of unlikely occurrences.  And I've heard some
>>> statements disregard widely accepted principles of inclusiveness at the
>>> Apache Software Foundation.
>>>
>>> But I suspect that there's more to the attitude of not including
>>> non-coding committers at Spark.  You have a community that is larger than
>>> the average at the ASF.  Dunbar's number is 150 ("humans can comfortably
>>> maintain 150 stable relationships" - check it on wikipedia).  With PMC and
>>> committers, Spark is already well over 100. You may feel like you're losing
>>> the ability to integrate new people into your community at that size.  And
>>> that may be the real reason you are trying to find a more selective
>>> criteria to base community membership decisions on.
>>>
>>> Here's the thing: communities at the ASF decide themselves who to offer
>>> committership to.  This is an important principle, and one we only overrule
>>> in the most extreme of circumstances.  And I don't currently see an
>>> "extreme circumstance" here.  If I dislike the way you are making those
>>> decisions, my best recourse is to try to change prevailing attitudes.  And
>>> in order to do that, I have to try to understand where those attitudes are
>>> coming from.  Calling people names or shouting "first principles" or "The
>>> Apache Way" is not going to be very convincing, amiright?  I'm not inclined
>>> to that sort of behavior anyways.  I have to explain the benefits of the
>>> approach I'm advocating for, and explain why the fears are unfounded.
>>>
>>> But if the real problem is that the community is too large for comfort,
>>> then my arguments don't address that.
>>>
>>> I'm afraid it primes people to drive by to feel good delivering the
>>>> safe, conventional mom-and-apple-pie ideals: what are you afraid of?
>>>> what is your problem with openness? why do you hate freedom and The
>>>> Apache Way? We'll have another round of throw-the-bums-out,
>>>> shut-it-all-down threads. These aren't wrong ideals. It just generates
>>>> no useful discussion, and is patronizing. I find it hard to dissent
>>>> reasonably.
>>>>
>>>
>>> No drive-by's yet.  I can see your concern though, especially since I've
>>> seen that sort of thing happen in other contexts.  It was not my intention
>>> to provoke drive-by's; I tried to phrase my message accordingly.  I simply
>>> wanted to adhere to the principles of openness that are so important at the
>>> ASF.
>>>
>>> We've had complaints on members@ about people learning about important
>>> conversations "too late", and requests to notify board@ about such
>>> conversations.
>>>
>>> Best,
>>> Myrle
>>>
>>

Re: Recognizing non-code contributions

Posted by William Shen <wi...@marinsoftware.com>.
Not sure where this thread is heading toward, but I find the role
definition listed on
http://www.apache.org/foundation/how-it-works.html#roles clarifying and
well defined, though they seem to vary slightly from what is on
https://community.apache.org/contributors/. Not sure which one is more
authoritative.

In the context of this thread, here is the difference as I see it:

On www.apache.org <http://www.apache.org/foundation/how-it-works.html#roles>
,

> A *committer* is a *developer* that was given write access to the code
> repository and has a signed Contributor License Agreement (CLA) on file.
> A *developer* is a *user* who contributes to a project in the form of *code
> or documentation*.

Hence, a committer is a user, who contributes code or documentation, that
was given write access to the repository.

On community.apache.org <https://community.apache.org/contributors/>,

> Committer is a term used at the ASF to signify someone who is committed to
> a particular project. It brings with it the privilege of write access to
> the project repository and resources.
> The foundations of an Apache project and how the community contributes to
> it is sometimes referred to by the acronym CoPDoC: (Co)mmunity - one must
> interact with others, and share vision and knowledge (P)roject - a clear
> vision and consensus are needed (Do)cumentation - without it, the stuff
> remains only in the minds of the authors (C)ode - discussion goes nowhere
> without code
> once someone has contributed sufficiently to any area of CoPDoC they can
> be voted in as a committer.

Hence, a committer is someone, who has contributed sufficiently to *any*
area of Community, Project, Documentation, and Code, that has been given
write access to the project repository and resources.

The key difference between the two definitions is: Can someone, who
contribute significantly in the area of Community or Project without
contribution to Documentation nor Code, be a committer? According to
www.apache.org <http://www.apache.org/foundation/how-it-works.html#roles>,
no; according to community.apache.org
<https://community.apache.org/contributors/>, yes. And that difference
seems to be a main point of discussion here.

By www.apache.org <http://www.apache.org/foundation/how-it-works.html#roles>,
a committer seems to be a role designed to create a subset of users to
"making short-term decisions for the project," and I think that reference
to the ability to commit a patch for Code or Documentation in the project's
repository or resources. And I think that is sensible, and may be in
support of limiting committership to those, who contribute sufficiently
code or documentation to a project's repository or resources, that the
project invites to make future commit decisions for the project.

On Wed, Aug 7, 2019 at 6:07 AM Hyukjin Kwon <gu...@gmail.com> wrote:

> > Currently, I have heard some ideas or attitudes that I consider to be
> overly motivated by fear of unlikely occurrences.
> > And I've heard some statements disregard widely accepted principles of
> inclusiveness at the Apache Software Foundation.
> > But I suspect that there's more to the attitude of not including
> non-coding committers at Spark.
>
> I missed some contexts you mentioned. Yes, SVN and commons look good
> examples.
> Also, for clarification, I did not mean to absolutely do not add
> non-codding committers.
> Spark already has build/config committer and I am happy with that.
>
> I was replaying to "the risk is very small". Given my experience in Spark
> dev, people (and I) make mistakes
> which, for instance, blocks the release months. Sometimes it requires to
> rewrite whole PRs with courtesy
> (rather than, for instance, reverting). This is already happening and it
> brings some overhead to the dev.
> Yes, maybe the volume matters to handle those issues.
>
> The point I was trying to make was commit bit could be a too strong sword
> and might have to be
> given and used with familiarity and caution.
>
> For clarification, I have no issue except one concern above for the fact
> that someone becomes a non-code committer since Spark already has it.
>
>
> 2019년 8월 7일 (수) 오후 6:04, Myrle Krantz <my...@apache.org>님이 작성:
>
>>
>>
>> On Tue, Aug 6, 2019 at 7:57 PM Sean Owen <sr...@apache.org> wrote:
>>
>>> On Tue, Aug 6, 2019 at 11:45 AM Myrle Krantz <my...@apache.org> wrote:
>>> > I had understood your position to be that you would be willing to make
>>> at least some non-coding contributors to committers but that your "line" is
>>> somewhat different than my own.   My response to you assumed that position
>>> on your part.  I do not think it's good for a project to accept absolutely
>>> no non-code committers.  If nothing else, it violates my sense of fairness,
>>> both towards those contributors, and also towards the ASF which relies on a
>>> pipeline of non-code contributors who come to us through the projects.
>>>
>>> Oh OK, I thought this argument was made repeatedly: someone who has
>>> not and evidently will not ever commit anything to a project doesn't
>>> seem to need the commit bit. Agree to disagree. That was the
>>> 'non-code' definition?
>>>
>>
>> That argument was made and acknowledged.  And then answered with:
>> a.) the commit bit is only part of what makes a committer, and not the
>> most important part.
>> b.) including the commit bit in the package is harmless.  The risk
>> associated with giving someone the commit bit who is not going to use it is
>> lower than the risk associated with the average pull request.
>> c.) creating a new package without the commit-bit creates significant
>> effort and bears significant risks.
>>
>>
>>> Someone who contributes docs to the project? Sure. We actually have
>>> done this, albeit for a build and config contributions. Agree.
>>>
>>> Pardon a complicated analogy to explain my thinking, but: let's say
>>> the space of acceptable decisions on adding committers at the ASF
>>> ranges from 1 (Super Aggressive) to 10 (Very Conservative). Most
>>> project decisions probably fall in, say, 3 to 7. Here we're debating
>>> whether a project should theoretically at times go all the way to 1,
>>> or at most 2, and I think that's just not that important. We're pretty
>>> much agreeing 2 is not out of the question, 1 we agree to disagree.
>>>
>>> Spark decisions here are probably 5-7 on average. I'd like it be like
>>> 4-6 personally. I suspect the real inbound argument is: all projects
>>> should be making all decisions in 1-3 or else it isn't The Apache Way.
>>> I accept anecdotes that projects function well in that range, but,
>>> Spark and Hadoop don't seem to (nor evidently Cassandra). I have a
>>> hard time rationalizing this. These are, after all, some of the
>>> biggest and most successful projects at Apache. At times it sounds
>>> like concern trolling, to 'help' these projects not fall apart.
>>>
>>> If so, you read correctly that there is a significant difference of
>>> opinion here, but that's what it is. Not the theoretical debate above.
>>>
>>
>> I think this misrepresents where the "middle" is in Apache projects.  I
>> think the middle is probably closer to where OfBiz is: occasionally
>> offering non-coding contributors committership, but probably not with the
>> frequency I would like.  But even that occasional committership for
>> non-coding committers has been extraordinarily important for the ASF as an
>> organization.  Sharan Foga started as a non-coding contributor for OfBiz,
>> and is now VP of Community Development at the ASF, and organized the Apache
>> Roadshow in Berlin last year (where Spark talks were well-received and
>> probably helped your community). The OfBiz project did us all a huge favor
>> by providing Sharan with the first step into our organization.
>>
>> What you are perceiving as an extreme is the SVN project in which all you
>> have to do to receive committership is to ask.  Or the commons project in
>> which in which every ASF member is automatically a committer.  Those
>> projects can't be claimed to be insensitive to bugs.  SVN is a data
>> repository: bugs can cause you to lose your code. For commons, programming
>> mistakes can cause security flaws, compile errors and worse in a massive
>> number of dependent project.  And yes, I know you've seen all of that
>> conversation, Sean, but most of it was on members@, so I'm summarizing
>> here for those who can't see members@.
>>
>> If Spark and Hadoop and Cassandra don't include non-coding contributors
>> in their committer candidate pool, that puts them on the outer extreme of
>> ASF projects.  These projects are undeniably successful despite this fact.
>> But I still wouldn't recommend the approach to other ASF projects.  I do
>> not believe most projects can get away with this and still survive long
>> term.
>>
>>
>>> Spark should shift, but equally, so should this viewpoint from some at
>>> the ASF, as I find my caricature of it equally suboptimal.
>>> Shred that analogy as you like, but it explains what's in my head.
>>
>>
>> I disagree, but if you want to change that position at the ASF, a good
>> place to start the conversation is on dev@community.
>>
>> > For more documentation on the definition of a committer at Apache, read
>>> here: https://community.apache.org/contributors/  "Being a committer
>>> does not necessarily mean you commit code, it means you are committed to
>>> the project and are productively contributing to its success."
>>>
>>> Per above, I just don't think this statement should be in the canon,
>>> and would prefer to clarify it, but hey it is there and I accept it.
>>> Still: what's committed? I'd define committed to the project, as,
>>> well, working on the project's output. It just punts the question.
>>>
>>
>> That content is owned by the community project.  The appropriate place to
>> request a change is on dev@community.
>>
>> No project can be successful without QA, design, project management, user
>> care, documentation, marketing, infrastructure, community building and
>> more.  Working together to create artifacts is important.  It represents
>> the community's common goal, and provides a point around which to organize
>> efforts.  Without non-coding contributions, projects won't be able to
>> produce those artifacts, or they'll be dependent on corporate patronage, or
>> they just won't be as good as they could be.
>>
>>
>>> > I also don't yet see a "serious offense" here.  My e-mail to board@
>>> is simply a heads up, which I do owe the rest of the board when I'm
>>> interacting with one of our projects.  Here are my exact words: "Most of
>>> that discussion is fairly harmless.  Some of it, I have found concerning."
>>> Right now, I'm still trying to approach Spark's position with a
>>> learning-and-teaching mindset.
>>>
>>> I'm nitpicking your words, which are by themselves reasonable. I think
>>> learning-and-teaching is just the right attitude.
>>> But have you heard different ideas here that are valid or merely "not
>>> harmful"? are the ideas you don't share just not your choice or
>>> "concerning"?
>>>
>>
>> Currently, I have heard some ideas or attitudes that I consider to be
>> overly motivated by fear of unlikely occurrences.  And I've heard some
>> statements disregard widely accepted principles of inclusiveness at the
>> Apache Software Foundation.
>>
>> But I suspect that there's more to the attitude of not including
>> non-coding committers at Spark.  You have a community that is larger than
>> the average at the ASF.  Dunbar's number is 150 ("humans can comfortably
>> maintain 150 stable relationships" - check it on wikipedia).  With PMC and
>> committers, Spark is already well over 100. You may feel like you're losing
>> the ability to integrate new people into your community at that size.  And
>> that may be the real reason you are trying to find a more selective
>> criteria to base community membership decisions on.
>>
>> Here's the thing: communities at the ASF decide themselves who to offer
>> committership to.  This is an important principle, and one we only overrule
>> in the most extreme of circumstances.  And I don't currently see an
>> "extreme circumstance" here.  If I dislike the way you are making those
>> decisions, my best recourse is to try to change prevailing attitudes.  And
>> in order to do that, I have to try to understand where those attitudes are
>> coming from.  Calling people names or shouting "first principles" or "The
>> Apache Way" is not going to be very convincing, amiright?  I'm not inclined
>> to that sort of behavior anyways.  I have to explain the benefits of the
>> approach I'm advocating for, and explain why the fears are unfounded.
>>
>> But if the real problem is that the community is too large for comfort,
>> then my arguments don't address that.
>>
>> I'm afraid it primes people to drive by to feel good delivering the
>>> safe, conventional mom-and-apple-pie ideals: what are you afraid of?
>>> what is your problem with openness? why do you hate freedom and The
>>> Apache Way? We'll have another round of throw-the-bums-out,
>>> shut-it-all-down threads. These aren't wrong ideals. It just generates
>>> no useful discussion, and is patronizing. I find it hard to dissent
>>> reasonably.
>>>
>>
>> No drive-by's yet.  I can see your concern though, especially since I've
>> seen that sort of thing happen in other contexts.  It was not my intention
>> to provoke drive-by's; I tried to phrase my message accordingly.  I simply
>> wanted to adhere to the principles of openness that are so important at the
>> ASF.
>>
>> We've had complaints on members@ about people learning about important
>> conversations "too late", and requests to notify board@ about such
>> conversations.
>>
>> Best,
>> Myrle
>>
>

Re: Recognizing non-code contributions

Posted by Hyukjin Kwon <gu...@gmail.com>.
> Currently, I have heard some ideas or attitudes that I consider to be
overly motivated by fear of unlikely occurrences.
> And I've heard some statements disregard widely accepted principles of
inclusiveness at the Apache Software Foundation.
> But I suspect that there's more to the attitude of not including
non-coding committers at Spark.

I missed some contexts you mentioned. Yes, SVN and commons look good
examples.
Also, for clarification, I did not mean to absolutely do not add
non-codding committers.
Spark already has build/config committer and I am happy with that.

I was replaying to "the risk is very small". Given my experience in Spark
dev, people (and I) make mistakes
which, for instance, blocks the release months. Sometimes it requires to
rewrite whole PRs with courtesy
(rather than, for instance, reverting). This is already happening and it
brings some overhead to the dev.
Yes, maybe the volume matters to handle those issues.

The point I was trying to make was commit bit could be a too strong sword
and might have to be
given and used with familiarity and caution.

For clarification, I have no issue except one concern above for the fact
that someone becomes a non-code committer since Spark already has it.


2019년 8월 7일 (수) 오후 6:04, Myrle Krantz <my...@apache.org>님이 작성:

>
>
> On Tue, Aug 6, 2019 at 7:57 PM Sean Owen <sr...@apache.org> wrote:
>
>> On Tue, Aug 6, 2019 at 11:45 AM Myrle Krantz <my...@apache.org> wrote:
>> > I had understood your position to be that you would be willing to make
>> at least some non-coding contributors to committers but that your "line" is
>> somewhat different than my own.   My response to you assumed that position
>> on your part.  I do not think it's good for a project to accept absolutely
>> no non-code committers.  If nothing else, it violates my sense of fairness,
>> both towards those contributors, and also towards the ASF which relies on a
>> pipeline of non-code contributors who come to us through the projects.
>>
>> Oh OK, I thought this argument was made repeatedly: someone who has
>> not and evidently will not ever commit anything to a project doesn't
>> seem to need the commit bit. Agree to disagree. That was the
>> 'non-code' definition?
>>
>
> That argument was made and acknowledged.  And then answered with:
> a.) the commit bit is only part of what makes a committer, and not the
> most important part.
> b.) including the commit bit in the package is harmless.  The risk
> associated with giving someone the commit bit who is not going to use it is
> lower than the risk associated with the average pull request.
> c.) creating a new package without the commit-bit creates significant
> effort and bears significant risks.
>
>
>> Someone who contributes docs to the project? Sure. We actually have
>> done this, albeit for a build and config contributions. Agree.
>>
>> Pardon a complicated analogy to explain my thinking, but: let's say
>> the space of acceptable decisions on adding committers at the ASF
>> ranges from 1 (Super Aggressive) to 10 (Very Conservative). Most
>> project decisions probably fall in, say, 3 to 7. Here we're debating
>> whether a project should theoretically at times go all the way to 1,
>> or at most 2, and I think that's just not that important. We're pretty
>> much agreeing 2 is not out of the question, 1 we agree to disagree.
>>
>> Spark decisions here are probably 5-7 on average. I'd like it be like
>> 4-6 personally. I suspect the real inbound argument is: all projects
>> should be making all decisions in 1-3 or else it isn't The Apache Way.
>> I accept anecdotes that projects function well in that range, but,
>> Spark and Hadoop don't seem to (nor evidently Cassandra). I have a
>> hard time rationalizing this. These are, after all, some of the
>> biggest and most successful projects at Apache. At times it sounds
>> like concern trolling, to 'help' these projects not fall apart.
>>
>> If so, you read correctly that there is a significant difference of
>> opinion here, but that's what it is. Not the theoretical debate above.
>>
>
> I think this misrepresents where the "middle" is in Apache projects.  I
> think the middle is probably closer to where OfBiz is: occasionally
> offering non-coding contributors committership, but probably not with the
> frequency I would like.  But even that occasional committership for
> non-coding committers has been extraordinarily important for the ASF as an
> organization.  Sharan Foga started as a non-coding contributor for OfBiz,
> and is now VP of Community Development at the ASF, and organized the Apache
> Roadshow in Berlin last year (where Spark talks were well-received and
> probably helped your community). The OfBiz project did us all a huge favor
> by providing Sharan with the first step into our organization.
>
> What you are perceiving as an extreme is the SVN project in which all you
> have to do to receive committership is to ask.  Or the commons project in
> which in which every ASF member is automatically a committer.  Those
> projects can't be claimed to be insensitive to bugs.  SVN is a data
> repository: bugs can cause you to lose your code. For commons, programming
> mistakes can cause security flaws, compile errors and worse in a massive
> number of dependent project.  And yes, I know you've seen all of that
> conversation, Sean, but most of it was on members@, so I'm summarizing
> here for those who can't see members@.
>
> If Spark and Hadoop and Cassandra don't include non-coding contributors in
> their committer candidate pool, that puts them on the outer extreme of ASF
> projects.  These projects are undeniably successful despite this fact.  But
> I still wouldn't recommend the approach to other ASF projects.  I do not
> believe most projects can get away with this and still survive long term.
>
>
>> Spark should shift, but equally, so should this viewpoint from some at
>> the ASF, as I find my caricature of it equally suboptimal.
>> Shred that analogy as you like, but it explains what's in my head.
>
>
> I disagree, but if you want to change that position at the ASF, a good
> place to start the conversation is on dev@community.
>
> > For more documentation on the definition of a committer at Apache, read
>> here: https://community.apache.org/contributors/  "Being a committer
>> does not necessarily mean you commit code, it means you are committed to
>> the project and are productively contributing to its success."
>>
>> Per above, I just don't think this statement should be in the canon,
>> and would prefer to clarify it, but hey it is there and I accept it.
>> Still: what's committed? I'd define committed to the project, as,
>> well, working on the project's output. It just punts the question.
>>
>
> That content is owned by the community project.  The appropriate place to
> request a change is on dev@community.
>
> No project can be successful without QA, design, project management, user
> care, documentation, marketing, infrastructure, community building and
> more.  Working together to create artifacts is important.  It represents
> the community's common goal, and provides a point around which to organize
> efforts.  Without non-coding contributions, projects won't be able to
> produce those artifacts, or they'll be dependent on corporate patronage, or
> they just won't be as good as they could be.
>
>
>> > I also don't yet see a "serious offense" here.  My e-mail to board@ is
>> simply a heads up, which I do owe the rest of the board when I'm
>> interacting with one of our projects.  Here are my exact words: "Most of
>> that discussion is fairly harmless.  Some of it, I have found concerning."
>> Right now, I'm still trying to approach Spark's position with a
>> learning-and-teaching mindset.
>>
>> I'm nitpicking your words, which are by themselves reasonable. I think
>> learning-and-teaching is just the right attitude.
>> But have you heard different ideas here that are valid or merely "not
>> harmful"? are the ideas you don't share just not your choice or
>> "concerning"?
>>
>
> Currently, I have heard some ideas or attitudes that I consider to be
> overly motivated by fear of unlikely occurrences.  And I've heard some
> statements disregard widely accepted principles of inclusiveness at the
> Apache Software Foundation.
>
> But I suspect that there's more to the attitude of not including
> non-coding committers at Spark.  You have a community that is larger than
> the average at the ASF.  Dunbar's number is 150 ("humans can comfortably
> maintain 150 stable relationships" - check it on wikipedia).  With PMC and
> committers, Spark is already well over 100. You may feel like you're losing
> the ability to integrate new people into your community at that size.  And
> that may be the real reason you are trying to find a more selective
> criteria to base community membership decisions on.
>
> Here's the thing: communities at the ASF decide themselves who to offer
> committership to.  This is an important principle, and one we only overrule
> in the most extreme of circumstances.  And I don't currently see an
> "extreme circumstance" here.  If I dislike the way you are making those
> decisions, my best recourse is to try to change prevailing attitudes.  And
> in order to do that, I have to try to understand where those attitudes are
> coming from.  Calling people names or shouting "first principles" or "The
> Apache Way" is not going to be very convincing, amiright?  I'm not inclined
> to that sort of behavior anyways.  I have to explain the benefits of the
> approach I'm advocating for, and explain why the fears are unfounded.
>
> But if the real problem is that the community is too large for comfort,
> then my arguments don't address that.
>
> I'm afraid it primes people to drive by to feel good delivering the
>> safe, conventional mom-and-apple-pie ideals: what are you afraid of?
>> what is your problem with openness? why do you hate freedom and The
>> Apache Way? We'll have another round of throw-the-bums-out,
>> shut-it-all-down threads. These aren't wrong ideals. It just generates
>> no useful discussion, and is patronizing. I find it hard to dissent
>> reasonably.
>>
>
> No drive-by's yet.  I can see your concern though, especially since I've
> seen that sort of thing happen in other contexts.  It was not my intention
> to provoke drive-by's; I tried to phrase my message accordingly.  I simply
> wanted to adhere to the principles of openness that are so important at the
> ASF.
>
> We've had complaints on members@ about people learning about important
> conversations "too late", and requests to notify board@ about such
> conversations.
>
> Best,
> Myrle
>

Re: Recognizing non-code contributions

Posted by Myrle Krantz <my...@apache.org>.
On Tue, Aug 6, 2019 at 7:57 PM Sean Owen <sr...@apache.org> wrote:

> On Tue, Aug 6, 2019 at 11:45 AM Myrle Krantz <my...@apache.org> wrote:
> > I had understood your position to be that you would be willing to make
> at least some non-coding contributors to committers but that your "line" is
> somewhat different than my own.   My response to you assumed that position
> on your part.  I do not think it's good for a project to accept absolutely
> no non-code committers.  If nothing else, it violates my sense of fairness,
> both towards those contributors, and also towards the ASF which relies on a
> pipeline of non-code contributors who come to us through the projects.
>
> Oh OK, I thought this argument was made repeatedly: someone who has
> not and evidently will not ever commit anything to a project doesn't
> seem to need the commit bit. Agree to disagree. That was the
> 'non-code' definition?
>

That argument was made and acknowledged.  And then answered with:
a.) the commit bit is only part of what makes a committer, and not the most
important part.
b.) including the commit bit in the package is harmless.  The risk
associated with giving someone the commit bit who is not going to use it is
lower than the risk associated with the average pull request.
c.) creating a new package without the commit-bit creates significant
effort and bears significant risks.


> Someone who contributes docs to the project? Sure. We actually have
> done this, albeit for a build and config contributions. Agree.
>
> Pardon a complicated analogy to explain my thinking, but: let's say
> the space of acceptable decisions on adding committers at the ASF
> ranges from 1 (Super Aggressive) to 10 (Very Conservative). Most
> project decisions probably fall in, say, 3 to 7. Here we're debating
> whether a project should theoretically at times go all the way to 1,
> or at most 2, and I think that's just not that important. We're pretty
> much agreeing 2 is not out of the question, 1 we agree to disagree.
>
> Spark decisions here are probably 5-7 on average. I'd like it be like
> 4-6 personally. I suspect the real inbound argument is: all projects
> should be making all decisions in 1-3 or else it isn't The Apache Way.
> I accept anecdotes that projects function well in that range, but,
> Spark and Hadoop don't seem to (nor evidently Cassandra). I have a
> hard time rationalizing this. These are, after all, some of the
> biggest and most successful projects at Apache. At times it sounds
> like concern trolling, to 'help' these projects not fall apart.
>
> If so, you read correctly that there is a significant difference of
> opinion here, but that's what it is. Not the theoretical debate above.
>

I think this misrepresents where the "middle" is in Apache projects.  I
think the middle is probably closer to where OfBiz is: occasionally
offering non-coding contributors committership, but probably not with the
frequency I would like.  But even that occasional committership for
non-coding committers has been extraordinarily important for the ASF as an
organization.  Sharan Foga started as a non-coding contributor for OfBiz,
and is now VP of Community Development at the ASF, and organized the Apache
Roadshow in Berlin last year (where Spark talks were well-received and
probably helped your community). The OfBiz project did us all a huge favor
by providing Sharan with the first step into our organization.

What you are perceiving as an extreme is the SVN project in which all you
have to do to receive committership is to ask.  Or the commons project in
which in which every ASF member is automatically a committer.  Those
projects can't be claimed to be insensitive to bugs.  SVN is a data
repository: bugs can cause you to lose your code. For commons, programming
mistakes can cause security flaws, compile errors and worse in a massive
number of dependent project.  And yes, I know you've seen all of that
conversation, Sean, but most of it was on members@, so I'm summarizing here
for those who can't see members@.

If Spark and Hadoop and Cassandra don't include non-coding contributors in
their committer candidate pool, that puts them on the outer extreme of ASF
projects.  These projects are undeniably successful despite this fact.  But
I still wouldn't recommend the approach to other ASF projects.  I do not
believe most projects can get away with this and still survive long term.


> Spark should shift, but equally, so should this viewpoint from some at
> the ASF, as I find my caricature of it equally suboptimal.
> Shred that analogy as you like, but it explains what's in my head.


I disagree, but if you want to change that position at the ASF, a good
place to start the conversation is on dev@community.

> For more documentation on the definition of a committer at Apache, read
> here: https://community.apache.org/contributors/  "Being a committer does
> not necessarily mean you commit code, it means you are committed to the
> project and are productively contributing to its success."
>
> Per above, I just don't think this statement should be in the canon,
> and would prefer to clarify it, but hey it is there and I accept it.
> Still: what's committed? I'd define committed to the project, as,
> well, working on the project's output. It just punts the question.
>

That content is owned by the community project.  The appropriate place to
request a change is on dev@community.

No project can be successful without QA, design, project management, user
care, documentation, marketing, infrastructure, community building and
more.  Working together to create artifacts is important.  It represents
the community's common goal, and provides a point around which to organize
efforts.  Without non-coding contributions, projects won't be able to
produce those artifacts, or they'll be dependent on corporate patronage, or
they just won't be as good as they could be.


> > I also don't yet see a "serious offense" here.  My e-mail to board@ is
> simply a heads up, which I do owe the rest of the board when I'm
> interacting with one of our projects.  Here are my exact words: "Most of
> that discussion is fairly harmless.  Some of it, I have found concerning."
> Right now, I'm still trying to approach Spark's position with a
> learning-and-teaching mindset.
>
> I'm nitpicking your words, which are by themselves reasonable. I think
> learning-and-teaching is just the right attitude.
> But have you heard different ideas here that are valid or merely "not
> harmful"? are the ideas you don't share just not your choice or
> "concerning"?
>

Currently, I have heard some ideas or attitudes that I consider to be
overly motivated by fear of unlikely occurrences.  And I've heard some
statements disregard widely accepted principles of inclusiveness at the
Apache Software Foundation.

But I suspect that there's more to the attitude of not including non-coding
committers at Spark.  You have a community that is larger than the average
at the ASF.  Dunbar's number is 150 ("humans can comfortably maintain 150
stable relationships" - check it on wikipedia).  With PMC and committers,
Spark is already well over 100. You may feel like you're losing the ability
to integrate new people into your community at that size.  And that may be
the real reason you are trying to find a more selective criteria to base
community membership decisions on.

Here's the thing: communities at the ASF decide themselves who to offer
committership to.  This is an important principle, and one we only overrule
in the most extreme of circumstances.  And I don't currently see an
"extreme circumstance" here.  If I dislike the way you are making those
decisions, my best recourse is to try to change prevailing attitudes.  And
in order to do that, I have to try to understand where those attitudes are
coming from.  Calling people names or shouting "first principles" or "The
Apache Way" is not going to be very convincing, amiright?  I'm not inclined
to that sort of behavior anyways.  I have to explain the benefits of the
approach I'm advocating for, and explain why the fears are unfounded.

But if the real problem is that the community is too large for comfort,
then my arguments don't address that.

I'm afraid it primes people to drive by to feel good delivering the
> safe, conventional mom-and-apple-pie ideals: what are you afraid of?
> what is your problem with openness? why do you hate freedom and The
> Apache Way? We'll have another round of throw-the-bums-out,
> shut-it-all-down threads. These aren't wrong ideals. It just generates
> no useful discussion, and is patronizing. I find it hard to dissent
> reasonably.
>

No drive-by's yet.  I can see your concern though, especially since I've
seen that sort of thing happen in other contexts.  It was not my intention
to provoke drive-by's; I tried to phrase my message accordingly.  I simply
wanted to adhere to the principles of openness that are so important at the
ASF.

We've had complaints on members@ about people learning about important
conversations "too late", and requests to notify board@ about such
conversations.

Best,
Myrle

Re: Recognizing non-code contributions

Posted by Sean Owen <sr...@apache.org>.
On Tue, Aug 6, 2019 at 11:45 AM Myrle Krantz <my...@apache.org> wrote:
> I had understood your position to be that you would be willing to make at least some non-coding contributors to committers but that your "line" is somewhat different than my own.   My response to you assumed that position on your part.  I do not think it's good for a project to accept absolutely no non-code committers.  If nothing else, it violates my sense of fairness, both towards those contributors, and also towards the ASF which relies on a pipeline of non-code contributors who come to us through the projects.

Oh OK, I thought this argument was made repeatedly: someone who has
not and evidently will not ever commit anything to a project doesn't
seem to need the commit bit. Agree to disagree. That was the
'non-code' definition?

Someone who contributes docs to the project? Sure. We actually have
done this, albeit for a build and config contributions. Agree.

Pardon a complicated analogy to explain my thinking, but: let's say
the space of acceptable decisions on adding committers at the ASF
ranges from 1 (Super Aggressive) to 10 (Very Conservative). Most
project decisions probably fall in, say, 3 to 7. Here we're debating
whether a project should theoretically at times go all the way to 1,
or at most 2, and I think that's just not that important. We're pretty
much agreeing 2 is not out of the question, 1 we agree to disagree.

Spark decisions here are probably 5-7 on average. I'd like it be like
4-6 personally. I suspect the real inbound argument is: all projects
should be making all decisions in 1-3 or else it isn't The Apache Way.
I accept anecdotes that projects function well in that range, but,
Spark and Hadoop don't seem to (nor evidently Cassandra). I have a
hard time rationalizing this. These are, after all, some of the
biggest and most successful projects at Apache. At times it sounds
like concern trolling, to 'help' these projects not fall apart.

If so, you read correctly that there is a significant difference of
opinion here, but that's what it is. Not the theoretical debate above.

Spark should shift, but equally, so should this viewpoint from some at
the ASF, as I find my caricature of it equally suboptimal.
Shred that analogy as you like, but it explains what's in my head.


> For more documentation on the definition of a committer at Apache, read here: https://community.apache.org/contributors/  "Being a committer does not necessarily mean you commit code, it means you are committed to the project and are productively contributing to its success."

Per above, I just don't think this statement should be in the canon,
and would prefer to clarify it, but hey it is there and I accept it.
Still: what's committed? I'd define committed to the project, as,
well, working on the project's output. It just punts the question.


> I also don't yet see a "serious offense" here.  My e-mail to board@ is simply a heads up, which I do owe the rest of the board when I'm interacting with one of our projects.  Here are my exact words: "Most of that discussion is fairly harmless.  Some of it, I have found concerning."  Right now, I'm still trying to approach Spark's position with a learning-and-teaching mindset.

I'm nitpicking your words, which are by themselves reasonable. I think
learning-and-teaching is just the right attitude.
But have you heard different ideas here that are valid or merely "not
harmful"? are the ideas you don't share just not your choice or
"concerning"?

I'm afraid it primes people to drive by to feel good delivering the
safe, conventional mom-and-apple-pie ideals: what are you afraid of?
what is your problem with openness? why do you hate freedom and The
Apache Way? We'll have another round of throw-the-bums-out,
shut-it-all-down threads. These aren't wrong ideals. It just generates
no useful discussion, and is patronizing. I find it hard to dissent
reasonably.

---------------------------------------------------------------------
To unsubscribe e-mail: dev-unsubscribe@spark.apache.org


Re: Recognizing non-code contributions

Posted by Myrle Krantz <my...@apache.org>.
On Tue, Aug 6, 2019 at 6:11 PM Sean Owen <sr...@apache.org> wrote:

> On Tue, Aug 6, 2019 at 10:46 AM Myrle Krantz <my...@apache.org> wrote:
> >> You can tell there's a range of opinions here. I'm probably less
> >> 'conservative' about adding committers than most on the PMC, right or
> >> wrong, but more conservative than some at the ASF. I think there's
> >> room to inch towards the middle ground here and this is good
> >> discussion informing the thinking.
> >
> >
> > That's not actually my current reading of the Spark community.  My
> current reading based on the responses of Hyukjin, and Jungtaek, is that
> your community wouldn't take a non-coding committer no matter how clear
> their contributions are to the community, and that by extension such a
> person could never become a PMC member.
> >
> > If my reading is correct (and the sample size *is* still quite small,
> and only includes one PMC member), I see that as a serious problem.
>
> Again if "non-code" means "no interaction with the project repo", no I
> do not hear support for making said person a committer for all the
> reasons you've heard here. I don't support it.
>
> Wait, didn't we just get done agreeing that's a reasonable position if
> not one you hold? I'm quite confused.
> It's fine to invite the board, members to come participate here as you
> have just done separately, but you're now portraying this as a serious
> offense, despite your comments here?
>

I think both representations of my position are inaccurate.

I had understood your position to be that you would be willing to make at
least some non-coding contributors to committers but that your "line" is
somewhat different than my own.   My response to you assumed that position
on your part.  I do not think it's good for a project to accept absolutely
no non-code committers.  If nothing else, it violates my sense of fairness,
both towards those contributors, and also towards the ASF which relies on a
pipeline of non-code contributors who come to us through the projects.

For more documentation on the definition of a committer at Apache, read
here: https://community.apache.org/contributors/  "Being a committer does
not necessarily mean you commit code, it means you are committed to the
project and are productively contributing to its success."

I also don't yet see a "serious offense" here.  My e-mail to board@ is
simply a heads up, which I do owe the rest of the board when I'm
interacting with one of our projects.  Here are my exact words: "Most of
that discussion is fairly harmless.  Some of it, I have found concerning."
Right now, I'm still trying to approach Spark's position with a
learning-and-teaching mindset.

Does that make it clearer?

Best Regards,
Myrle

Re: Recognizing non-code contributions

Posted by Sean Owen <sr...@apache.org>.
On Tue, Aug 6, 2019 at 10:46 AM Myrle Krantz <my...@apache.org> wrote:
>> You can tell there's a range of opinions here. I'm probably less
>> 'conservative' about adding committers than most on the PMC, right or
>> wrong, but more conservative than some at the ASF. I think there's
>> room to inch towards the middle ground here and this is good
>> discussion informing the thinking.
>
>
> That's not actually my current reading of the Spark community.  My current reading based on the responses of Hyukjin, and Jungtaek, is that your community wouldn't take a non-coding committer no matter how clear their contributions are to the community, and that by extension such a person could never become a PMC member.
>
> If my reading is correct (and the sample size *is* still quite small, and only includes one PMC member), I see that as a serious problem.

Again if "non-code" means "no interaction with the project repo", no I
do not hear support for making said person a committer for all the
reasons you've heard here. I don't support it.

Wait, didn't we just get done agreeing that's a reasonable position if
not one you hold? I'm quite confused.
It's fine to invite the board, members to come participate here as you
have just done separately, but you're now portraying this as a serious
offense, despite your comments here?

---------------------------------------------------------------------
To unsubscribe e-mail: dev-unsubscribe@spark.apache.org


Re: Recognizing non-code contributions

Posted by Holden Karau <ho...@pigscanfly.ca>.
So I’d like to add non-coding committers, I think there is great value in
both recognizing them and eventually having a broader PMC (eg maybe someone
who’s put a lot of time into teaching Spark has important things to say
about a proposed release, perhaps important enough for a binding vote).

That being said I think that my view is not aligned with the rest of the
PMC and I believe it is important that we work together so I’m ok with
exploring things like Spark VIP (I think in Kafka land there is an MVP of
the year concept but I need to talk with folks over there more for ideas on
how they run it).

I like to think the Apache way is broad enough to allow for variations like
this to be explored in different projects while sharing what has worked and
not worked historically so that we can all build healthy OSS communities.

On Tue, Aug 6, 2019 at 8:46 AM Myrle Krantz <my...@apache.org> wrote:

>
>
> On Tue, Aug 6, 2019 at 5:36 PM Sean Owen <sr...@apache.org> wrote:
>
>> You can tell there's a range of opinions here. I'm probably less
>> 'conservative' about adding committers than most on the PMC, right or
>> wrong, but more conservative than some at the ASF. I think there's
>> room to inch towards the middle ground here and this is good
>> discussion informing the thinking.
>>
>
> That's not actually my current reading of the Spark community.  My current
> reading based on the responses of Hyukjin, and Jungtaek, is that your
> community wouldn't take a non-coding committer no matter how clear their
> contributions are to the community, and that by extension such a person
> could never become a PMC member.
>
> If my reading is correct (and the sample size *is* still quite small, and
> only includes one PMC member), I see that as a serious problem.
>
> How do the other PMC members and community members see this?
>
> Best Regards,
> Myrle
>
-- 
Twitter: https://twitter.com/holdenkarau
Books (Learning Spark, High Performance Spark, etc.):
https://amzn.to/2MaRAG9  <https://amzn.to/2MaRAG9>
YouTube Live Streams: https://www.youtube.com/user/holdenkarau

Re: Recognizing non-code contributions

Posted by Hyukjin Kwon <gu...@gmail.com>.
> I wonder which project nominees non-coding only committers but I at least
know multiple projects. They all have that serious problem then.

I mean It know multiple projects don't do that and according to what you
said, they all have that serious problem.

2019년 8월 7일 (수) 오전 1:05, Hyukjin Kwon <gu...@gmail.com>님이 작성:

> Well, actually I am rather less conservative on adding committers. There
> are multiple people who are active in both non-coding and coding activities.
> I as an example am one of Korean meetup admin and my main focus was to
> management JIRA. In addition, review the PRs that are not being reviewed.
> As I said earlier at the very first time, I think committers should
> ideally be used to the dev at some degrees as primary. Other contributions
> should be counted.
>
> I wonder which project nominees non-coding only committers but I at least
> know multiple projects. They all have that serious problem then.
>
> 2019년 8월 7일 (수) 오전 12:46, Myrle Krantz <my...@apache.org>님이 작성:
>
>>
>>
>> On Tue, Aug 6, 2019 at 5:36 PM Sean Owen <sr...@apache.org> wrote:
>>
>>> You can tell there's a range of opinions here. I'm probably less
>>> 'conservative' about adding committers than most on the PMC, right or
>>> wrong, but more conservative than some at the ASF. I think there's
>>> room to inch towards the middle ground here and this is good
>>> discussion informing the thinking.
>>>
>>
>> That's not actually my current reading of the Spark community.  My
>> current reading based on the responses of Hyukjin, and Jungtaek, is that
>> your community wouldn't take a non-coding committer no matter how clear
>> their contributions are to the community, and that by extension such a
>> person could never become a PMC member.
>>
>> If my reading is correct (and the sample size *is* still quite small, and
>> only includes one PMC member), I see that as a serious problem.
>>
>> How do the other PMC members and community members see this?
>>
>> Best Regards,
>> Myrle
>>
>

Re: Recognizing non-code contributions

Posted by Hyukjin Kwon <gu...@gmail.com>.
Well, actually I am rather less conservative on adding committers. There
are multiple people who are active in both non-coding and coding activities.
I as an example am one of Korean meetup admin and my main focus was to
management JIRA. In addition, review the PRs that are not being reviewed.
As I said earlier at the very first time, I think committers should ideally
be used to the dev at some degrees as primary. Other contributions should
be counted.

I wonder which project nominees non-coding only committers but I at least
know multiple projects. They all have that serious problem then.

2019년 8월 7일 (수) 오전 12:46, Myrle Krantz <my...@apache.org>님이 작성:

>
>
> On Tue, Aug 6, 2019 at 5:36 PM Sean Owen <sr...@apache.org> wrote:
>
>> You can tell there's a range of opinions here. I'm probably less
>> 'conservative' about adding committers than most on the PMC, right or
>> wrong, but more conservative than some at the ASF. I think there's
>> room to inch towards the middle ground here and this is good
>> discussion informing the thinking.
>>
>
> That's not actually my current reading of the Spark community.  My current
> reading based on the responses of Hyukjin, and Jungtaek, is that your
> community wouldn't take a non-coding committer no matter how clear their
> contributions are to the community, and that by extension such a person
> could never become a PMC member.
>
> If my reading is correct (and the sample size *is* still quite small, and
> only includes one PMC member), I see that as a serious problem.
>
> How do the other PMC members and community members see this?
>
> Best Regards,
> Myrle
>

Re: Recognizing non-code contributions

Posted by Myrle Krantz <my...@apache.org>.
On Tue, Aug 6, 2019 at 5:36 PM Sean Owen <sr...@apache.org> wrote:

> You can tell there's a range of opinions here. I'm probably less
> 'conservative' about adding committers than most on the PMC, right or
> wrong, but more conservative than some at the ASF. I think there's
> room to inch towards the middle ground here and this is good
> discussion informing the thinking.
>

That's not actually my current reading of the Spark community.  My current
reading based on the responses of Hyukjin, and Jungtaek, is that your
community wouldn't take a non-coding committer no matter how clear their
contributions are to the community, and that by extension such a person
could never become a PMC member.

If my reading is correct (and the sample size *is* still quite small, and
only includes one PMC member), I see that as a serious problem.

How do the other PMC members and community members see this?

Best Regards,
Myrle

Re: Recognizing non-code contributions

Posted by Sean Owen <sr...@apache.org>.
On Tue, Aug 6, 2019 at 1:14 AM Myrle Krantz <my...@apache.org> wrote:
> If someone makes a commit who you are not expecting to make a commit, or in an area you weren't expecting changes in, you'll notice that, right?

Not counterarguments, but just more color on the hesitation:

- Probably, but it's less obvious on a big project than a small one!
- More likely: person commits in an area they know, and it breaks
something elsewhere unexpectedly. Tests can catch most but not all of
this. That's a risk everywhere though.
- Or: most commits aren't _authored_ by committers here (I think?) but
_merged_ by committers. It's still possible to watch for this, but
harder.
- It's harder to retroactively review commits and revert if needed,
not impossible

Honestly I think most of Spark is run with the attitude you describe.
It's the core and SQL modules that generate the worry, because
correctness and semantics are much more sensitive there.

I know at one time we tried the notion of 'maintainers' for areas of
the code, where it was strongly suggested that a change to module X be
reviewed by one of a few experts on that part of the code. It was
never really enforced and so dropped. I recall it was a little
controversial at the time: why are you trying to create committers
with more/less power over the code? Yet I think that's the substance
of the 'what harm can it do?' argument: just make sure committers
stick to their appropriate area and then there's really no worry.


> In order to do that, you'd need to create this kind of in-between status Apache-wide.  I would be very much opposed to doing that for a couple of reasons:
> * It adds complexity for infra and events with no clear benefits to the projects.
> * The risk of creating a second-class status at the ASF is just too high for my comfort.

This is a good practical argument. I'd still push back on the idea
that there is no benefit (cf. supra), but it may not be worth the
headache or simple admin overhead.
I don't buy the second-class citizen argument so much. We already have
a board, PMC, committer statuses. It's just that we're used to these
tiers.
I don't think you're being dogmatic about it, but, some are, some of
the same people railing against squashing new ideas because they're
different!


> Documents are IP.  You're better off if you have an ICLA for that stuff, regardless of where it lands in your project content.  And the most natural point in time to request an ICLA is at committer invitation.

Side point: ICLAs are always nice-to-have but not required, no? most
contributions don't come from those with an ICLA nor would we want to
block contributions on signing one. It's just orthogonal. I take your
point that a regular contributor probably should sign an ICLA, and a
regular contributor should be a committer, just not that they should
be a committer because they should sign an ICLA.


You can tell there's a range of opinions here. I'm probably less
'conservative' about adding committers than most on the PMC, right or
wrong, but more conservative than some at the ASF. I think there's
room to inch towards the middle ground here and this is good
discussion informing the thinking.

---------------------------------------------------------------------
To unsubscribe e-mail: dev-unsubscribe@spark.apache.org


Re: Recognizing non-code contributions

Posted by Jungtaek Lim <ka...@gmail.com>.
My 2 cents as just one of contributors of Apache Spark project.

The thing is, what's the merit for both contributors and PMC members on
granting committership on non-code contributors. I'd rather say someone is
a good candidate to be invited as a committer to co-maintain a part of code
repository if non-code contributions (like documentation) have been
happening on code repository. Assuming we're granting committership to
major contributors on documentation, they would maintain the doc area only
unless they're having confident on the area what they are reviewing. In
many cases, major contributions on documentation often requires major
"technical aspect" of understanding the project (I guess we're not saying
about fixing typos) which would also represent the knowledge on that area.

On the other side, if we are talking about non-code contributors who
contributes "outside" of repository, I'd say there's less (even no) merits
to grant committership. In such case, granting write privilege doesn't help
these contributors to make their contributions easier. No merits on PMC
members as well. For me, the origin meaning of "committership" is just a
"write privilege on repository". While there're more role and
responsibility as well as more merits on committership in ASF, I'd rather
think again what's the real value if the reason of granting committership
doesn't apply to the origin meaning.

If we would like to use "committership" as a recognition on major
contributions for the project in any way, I'd love to see some other
approach (like VIP? actually not sure what it meant in previous mail) to do
so. Let's focus on origin meaning of "committership", and not couple with
providing apache email address or giving chance to get various merits what
ASF committers have been enjoying. I hope there's other way to provide
these merits while we don't grant "unnecessary" privilege.

-Jungtaek Lim (HeartSaVioR)


On Tue, Aug 6, 2019 at 10:08 PM Hyukjin Kwon <gu...@gmail.com> wrote:

> I usually make such judgement about commit bit based upon community
> activity in coding and reviewing.
> If somebody has no activity about those commit bits, I would have no way
> to know about this guy,
> Simply I can't make a judgement about coding activity based upon
> non-coding activity.
>
> Those bugs and commit stuff are pretty critical in this project as I
> described. I would rather try to decrease such
> possibility, not increase it even when such "commit bit" is unnecessary.
>
> We have found and discussed nicer other ways to recognise them, for
> instance, listing them in somewhere else in Spark website.
> Once they are in that list, I suspect it's easier and closer to the
> committership to, say, get an Apache email if it matters.
>
> Shall we avoid such possibilities at all and go for such other safer ways?
> I think you also accept commit bit is unnecessary in this case.
> So, we don't unnecessarily give it to them, which is anyhow critical in
> this project.
>
> > Based on this argumentation you will never invite any committers or even
> merge any pull requests.
> BTW, how did you reach that conclusion? I want somebody who can review PRs
> and fix such bugs, rather than who has more possibility to make such
> mistakes.
>
>
> 2019년 8월 6일 (화) 오후 7:26, Myrle Krantz <my...@apache.org>님이 작성:
>
>> Hey Hyukjin,
>>
>> Apologies for sending this to you twice.  : o)
>>
>> On Tue, Aug 6, 2019 at 9:55 AM Hyukjin Kwon <gu...@gmail.com> wrote:
>>
>>> Myrle,
>>>
>>> > We need to balance two sets of risks here.  But in the case of access
>>> to our software artifacts, the risk is very small, and already has
>>> *multiple* mitigating factors, from the fact that all changes are tracked
>>> to an individual, to the fact that there are notifications sent when
>>> changes are made, (and I'm going to stop listing the benefits of a modern
>>> source control system here, because I know you are aware of them), on
>>> through the fact that you have automated tests, and continuing through the
>>> fact that there is a release process during which artifacts get checked
>>> again.
>>> > If someone makes a commit who you are not expecting to make a commit,
>>> or in an area you weren't expecting changes in, you'll notice that, right?
>>> > What you're talking about here is your security model for your source
>>> repository.  But restricting access isn't really the right security model
>>> for an open source project.
>>>
>>> I don't quite get the argument about commit bit. I _strongly_ disagree
>>> about "the risk is very small,".
>>> Not all of committers track all the changes. There are so many changes
>>> in the upstream and it's already overhead to check all.
>>> Do you know how many bugs Spark faces due to such lack of reviews that
>>> entirely blocks the release sometimes, and how much it takes time to fix up
>>> such commits?
>>> We need expertise and familiarity to Spark.
>>>
>>
>> Let's unroll that a bit.  Say that you invite a non-coding contributor to
>> be a committer.  To make an inappropriate commit two things would have to
>> happen: this person would have to decide to make the commit, and this
>> person would have to set up access to the git repository, either by
>> enabling gitbox integration, or accessing the apache git repository
>> directly.  Before you invite them you make an estimation of the probability
>> that they would do the first: that is decide to make an inappropriate
>> commit.  You decide that that is fairly unlikely.  But for a non-coding
>> contributor the chances of them actually going through the mechanics of
>> making a commit is even more unlikely.  I think we can safely assume that
>> the chance of someone who you've determined is committed to the community
>> and knows their limits of doing this is simply 00.00%.
>>
>> That leaves the question of what the chance is that this person will leak
>> their credentials to a malicious third party intent on introducing bugs
>> into Spark code.  Do you believe there are such malicious third parties?
>> How many attacks have there been on Spark committer credentials?  I believe
>> the likelihood of this happening is 00.00% (but I am willing to be swayed
>> by evidence otherwise -- should probably be discussed on the private@
>> list though if it's out there.: o).
>>
>> But let's say I'm wrong about both of those probabilities.  Let's say the
>> combined probability of one of those two things happening is actually
>> 0.01%.  This is where the advantages of modern source control and tests
>> come in.  Even if there's only a 50% chance that watching commits will
>> catch the error, and only a further 50% chance that tests will catch the
>> error, and only a further 50% chance that the error will be caught in
>> release testing, those chances multiply out at 00.00125%.
>>
>> Based on those guestimates the risk is somewhere between 00.00% and
>> 00.00125%.  The risk is very small.  You take bigger risks every day in
>> order to move your project forward.
>>
>>
>>> It virtually means we will add some more overhead to audit each commit,
>>> even for committers'. Why should we bother add such overhead to harm the
>>> project?
>>> To me, this is the most important fact. I don't think we should just
>>> count the number of positive and negative ones.
>>>
>>
>> Based on this argumentation you will never invite any committers or even
>> merge any pull requests.
>>
>> But you do invite committers and you do merge pull requests because it's
>> good for your project.  Because the risk of doing nothing is greater.
>>
>>
>>> For other reasons, we can just add or discuss about the "this kind of
>>> in-between status Apache-wide", which is a bigger scope than here. You can
>>> ask it to ASF and discuss further.
>>>
>>
>> I can say with considerable confidence: There will be no "in-between"
>> status Apache-wide.  But if you disagree, and want to start a discussion to
>> suggest that, dev@community.apache.org is a good place to go with it.
>>
>> Best Regards,
>> Myrle
>>
>>>

-- 
Name : Jungtaek Lim
Blog : http://medium.com/@heartsavior
Twitter : http://twitter.com/heartsavior
LinkedIn : http://www.linkedin.com/in/heartsavior

Re: Recognizing non-code contributions

Posted by Hyukjin Kwon <gu...@gmail.com>.
I usually make such judgement about commit bit based upon community
activity in coding and reviewing.
If somebody has no activity about those commit bits, I would have no way to
know about this guy,
Simply I can't make a judgement about coding activity based upon non-coding
activity.

Those bugs and commit stuff are pretty critical in this project as I
described. I would rather try to decrease such
possibility, not increase it even when such "commit bit" is unnecessary.

We have found and discussed nicer other ways to recognise them, for
instance, listing them in somewhere else in Spark website.
Once they are in that list, I suspect it's easier and closer to the
committership to, say, get an Apache email if it matters.

Shall we avoid such possibilities at all and go for such other safer ways?
I think you also accept commit bit is unnecessary in this case.
So, we don't unnecessarily give it to them, which is anyhow critical in
this project.

> Based on this argumentation you will never invite any committers or even
merge any pull requests.
BTW, how did you reach that conclusion? I want somebody who can review PRs
and fix such bugs, rather than who has more possibility to make such
mistakes.


2019년 8월 6일 (화) 오후 7:26, Myrle Krantz <my...@apache.org>님이 작성:

> Hey Hyukjin,
>
> Apologies for sending this to you twice.  : o)
>
> On Tue, Aug 6, 2019 at 9:55 AM Hyukjin Kwon <gu...@gmail.com> wrote:
>
>> Myrle,
>>
>> > We need to balance two sets of risks here.  But in the case of access
>> to our software artifacts, the risk is very small, and already has
>> *multiple* mitigating factors, from the fact that all changes are tracked
>> to an individual, to the fact that there are notifications sent when
>> changes are made, (and I'm going to stop listing the benefits of a modern
>> source control system here, because I know you are aware of them), on
>> through the fact that you have automated tests, and continuing through the
>> fact that there is a release process during which artifacts get checked
>> again.
>> > If someone makes a commit who you are not expecting to make a commit,
>> or in an area you weren't expecting changes in, you'll notice that, right?
>> > What you're talking about here is your security model for your source
>> repository.  But restricting access isn't really the right security model
>> for an open source project.
>>
>> I don't quite get the argument about commit bit. I _strongly_ disagree
>> about "the risk is very small,".
>> Not all of committers track all the changes. There are so many changes in
>> the upstream and it's already overhead to check all.
>> Do you know how many bugs Spark faces due to such lack of reviews that
>> entirely blocks the release sometimes, and how much it takes time to fix up
>> such commits?
>> We need expertise and familiarity to Spark.
>>
>
> Let's unroll that a bit.  Say that you invite a non-coding contributor to
> be a committer.  To make an inappropriate commit two things would have to
> happen: this person would have to decide to make the commit, and this
> person would have to set up access to the git repository, either by
> enabling gitbox integration, or accessing the apache git repository
> directly.  Before you invite them you make an estimation of the probability
> that they would do the first: that is decide to make an inappropriate
> commit.  You decide that that is fairly unlikely.  But for a non-coding
> contributor the chances of them actually going through the mechanics of
> making a commit is even more unlikely.  I think we can safely assume that
> the chance of someone who you've determined is committed to the community
> and knows their limits of doing this is simply 00.00%.
>
> That leaves the question of what the chance is that this person will leak
> their credentials to a malicious third party intent on introducing bugs
> into Spark code.  Do you believe there are such malicious third parties?
> How many attacks have there been on Spark committer credentials?  I believe
> the likelihood of this happening is 00.00% (but I am willing to be swayed
> by evidence otherwise -- should probably be discussed on the private@
> list though if it's out there.: o).
>
> But let's say I'm wrong about both of those probabilities.  Let's say the
> combined probability of one of those two things happening is actually
> 0.01%.  This is where the advantages of modern source control and tests
> come in.  Even if there's only a 50% chance that watching commits will
> catch the error, and only a further 50% chance that tests will catch the
> error, and only a further 50% chance that the error will be caught in
> release testing, those chances multiply out at 00.00125%.
>
> Based on those guestimates the risk is somewhere between 00.00% and
> 00.00125%.  The risk is very small.  You take bigger risks every day in
> order to move your project forward.
>
>
>> It virtually means we will add some more overhead to audit each commit,
>> even for committers'. Why should we bother add such overhead to harm the
>> project?
>> To me, this is the most important fact. I don't think we should just
>> count the number of positive and negative ones.
>>
>
> Based on this argumentation you will never invite any committers or even
> merge any pull requests.
>
> But you do invite committers and you do merge pull requests because it's
> good for your project.  Because the risk of doing nothing is greater.
>
>
>> For other reasons, we can just add or discuss about the "this kind of
>> in-between status Apache-wide", which is a bigger scope than here. You can
>> ask it to ASF and discuss further.
>>
>
> I can say with considerable confidence: There will be no "in-between"
> status Apache-wide.  But if you disagree, and want to start a discussion to
> suggest that, dev@community.apache.org is a good place to go with it.
>
> Best Regards,
> Myrle
>
>>

Re: Recognizing non-code contributions

Posted by Myrle Krantz <my...@apache.org>.
Hey Hyukjin,

Apologies for sending this to you twice.  : o)

On Tue, Aug 6, 2019 at 9:55 AM Hyukjin Kwon <gu...@gmail.com> wrote:

> Myrle,
>
> > We need to balance two sets of risks here.  But in the case of access to
> our software artifacts, the risk is very small, and already has *multiple*
> mitigating factors, from the fact that all changes are tracked to an
> individual, to the fact that there are notifications sent when changes are
> made, (and I'm going to stop listing the benefits of a modern source
> control system here, because I know you are aware of them), on through the
> fact that you have automated tests, and continuing through the fact that
> there is a release process during which artifacts get checked again.
> > If someone makes a commit who you are not expecting to make a commit, or
> in an area you weren't expecting changes in, you'll notice that, right?
> > What you're talking about here is your security model for your source
> repository.  But restricting access isn't really the right security model
> for an open source project.
>
> I don't quite get the argument about commit bit. I _strongly_ disagree
> about "the risk is very small,".
> Not all of committers track all the changes. There are so many changes in
> the upstream and it's already overhead to check all.
> Do you know how many bugs Spark faces due to such lack of reviews that
> entirely blocks the release sometimes, and how much it takes time to fix up
> such commits?
> We need expertise and familiarity to Spark.
>

Let's unroll that a bit.  Say that you invite a non-coding contributor to
be a committer.  To make an inappropriate commit two things would have to
happen: this person would have to decide to make the commit, and this
person would have to set up access to the git repository, either by
enabling gitbox integration, or accessing the apache git repository
directly.  Before you invite them you make an estimation of the probability
that they would do the first: that is decide to make an inappropriate
commit.  You decide that that is fairly unlikely.  But for a non-coding
contributor the chances of them actually going through the mechanics of
making a commit is even more unlikely.  I think we can safely assume that
the chance of someone who you've determined is committed to the community
and knows their limits of doing this is simply 00.00%.

That leaves the question of what the chance is that this person will leak
their credentials to a malicious third party intent on introducing bugs
into Spark code.  Do you believe there are such malicious third parties?
How many attacks have there been on Spark committer credentials?  I believe
the likelihood of this happening is 00.00% (but I am willing to be swayed
by evidence otherwise -- should probably be discussed on the private@ list
though if it's out there.: o).

But let's say I'm wrong about both of those probabilities.  Let's say the
combined probability of one of those two things happening is actually
0.01%.  This is where the advantages of modern source control and tests
come in.  Even if there's only a 50% chance that watching commits will
catch the error, and only a further 50% chance that tests will catch the
error, and only a further 50% chance that the error will be caught in
release testing, those chances multiply out at 00.00125%.

Based on those guestimates the risk is somewhere between 00.00% and
00.00125%.  The risk is very small.  You take bigger risks every day in
order to move your project forward.


> It virtually means we will add some more overhead to audit each commit,
> even for committers'. Why should we bother add such overhead to harm the
> project?
> To me, this is the most important fact. I don't think we should just count
> the number of positive and negative ones.
>

Based on this argumentation you will never invite any committers or even
merge any pull requests.

But you do invite committers and you do merge pull requests because it's
good for your project.  Because the risk of doing nothing is greater.


> For other reasons, we can just add or discuss about the "this kind of
> in-between status Apache-wide", which is a bigger scope than here. You can
> ask it to ASF and discuss further.
>

I can say with considerable confidence: There will be no "in-between"
status Apache-wide.  But if you disagree, and want to start a discussion to
suggest that, dev@community.apache.org is a good place to go with it.

Best Regards,
Myrle

>

Re: Recognizing non-code contributions

Posted by Hyukjin Kwon <gu...@gmail.com>.
So, here's my thought:

1. Back to the original point, for recognition of such people, I think we
can simply list up such people in Spark Website somewhere. For instance,

  Person A: Spark Book
  Person B: Meetup leader

I don't know if ASF allows this. Someone needs to check it.


2. If we need the in-between status officially (e.g. Apache email or
something), it should be asked and discussed in ASF, not in a single
project here.


2019년 8월 6일 (화) 오후 4:55, Hyukjin Kwon <gu...@gmail.com>님이 작성:

> Myrle,
>
> > We need to balance two sets of risks here.  But in the case of access to
> our software artifacts, the risk is very small, and already has *multiple*
> mitigating factors, from the fact that all changes are tracked to an
> individual, to the fact that there are notifications sent when changes are
> made, (and I'm going to stop listing the benefits of a modern source
> control system here, because I know you are aware of them), on through the
> fact that you have automated tests, and continuing through the fact that
> there is a release process during which artifacts get checked again.
> > If someone makes a commit who you are not expecting to make a commit, or
> in an area you weren't expecting changes in, you'll notice that, right?
> > What you're talking about here is your security model for your source
> repository.  But restricting access isn't really the right security model
> for an open source project.
>
> I don't quite get the argument about commit bit. I _strongly_ disagree
> about "the risk is very small,".
> Not all of committers track all the changes. There are so many changes in
> the upstream and it's already overhead to check all.
> Do you know how many bugs Spark faces due to such lack of reviews that
> entirely blocks the release sometimes, and how much it takes time to fix up
> such commits?
> We need expertise and familiarity to Spark.
>
> It virtually means we will add some more overhead to audit each commit,
> even for committers'. Why should we bother add such overhead to harm the
> project?
> To me, this is the most important fact. I don't think we should just count
> the number of positive and negative ones.
>
> For other reasons, we can just add or discuss about the "this kind of
> in-between status Apache-wide", which is a bigger scope than here. You can
> ask it to ASF and discuss further.
>
>
> 2019년 8월 6일 (화) 오후 3:14, Myrle Krantz <my...@apache.org>님이 작성:
>
>> Hey Sean,
>>
>> Even though we are discussing our differences, on the whole I don't think
>> we're that far apart in our positions.  Still the differences are where the
>> conversation is actually interesting, so here goes:
>>
>> On Mon, Aug 5, 2019 at 3:55 PM Sean Owen <sr...@apache.org> wrote:
>>
>>> On Mon, Aug 5, 2019 at 3:50 AM Myrle Krantz <my...@apache.org> wrote:
>>> > So... events coordinators?  I'd still make them committers.  I guess
>>> I'm still struggling to understand what problem making people VIP's without
>>> giving them committership is trying to solve.
>>>
>>> We may just agree to disagree, which is fine, but I think the argument
>>> is clear enough: such a person has zero need for the commit bit.
>>> Turning it around, what are we trying to accomplish by giving said
>>> person a commit bit? I know people say there's no harm, but I think
>>> there is at least _some_ downside. We're widening access to change
>>> software artifacts, the main thing that we put ASF process and checks
>>> around for liability reasons. I know the point is trust, and said
>>> person is likely to understand to never use the commit bit, but it
>>> brings us back to the same place. I don't wish to convince anyone else
>>> of my stance, though I do find it more logical, just that it's
>>> reasonable within The Apache Way.
>>>
>>
>> We need to balance two sets of risks here.  But in the case of access to
>> our software artifacts, the risk is very small, and already has *multiple*
>> mitigating factors, from the fact that all changes are tracked to an
>> individual, to the fact that there are notifications sent when changes are
>> made, (and I'm going to stop listing the benefits of a modern source
>> control system here, because I know you are aware of them), on through the
>> fact that you have automated tests, and continuing through the fact that
>> there is a release process during which artifacts get checked again.
>>
>> If someone makes a commit who you are not expecting to make a commit, or
>> in an area you weren't expecting changes in, you'll notice that, right?
>>
>> What you're talking about here is your security model for your source
>> repository.  But restricting access isn't really the right security model
>> for an open source project.
>>
>>
>>> > It also just occurred to me this morning: There are actually other
>>> privileges which go along with the "commit-bit" other than the ability to
>>> commit at will to the project's repos: people who are committers get an
>>> Apache e-mail address, and they get discounted entry to ApacheCon.  People
>>> who are committers also get added to our committers mailing list, and are
>>> thus a little easier to integrate into our foundation-wide efforts.
>>> >
>>> > To apply this to the example above, the Apache e-mail address can make
>>> it a tad easier for an event coordinator to conduct official business for a
>>> project.
>>>
>>> Great points. Again if I'm making it up? a "VIP" should get an Apache
>>> email address and discounts. Sure, why not put them on a committers@
>>> list too for visibility.
>>>
>>
>> In order to do that, you'd need to create this kind of in-between status
>> Apache-wide.  I would be very much opposed to doing that for a couple of
>> reasons:
>> * It adds complexity for infra and events with no clear benefits to the
>> projects.
>> * The risk of creating a second-class status at the ASF is just too high
>> for my comfort.
>>
>>
>>> > But that's not what I'm arguing.  I'm arguing that you can safely give
>>> commit access to anyone for whom you're reasonably certain they will do no
>>> harm.  And your certainty of that can actually be much higher with
>>> non-coding committers.  So if someone suggests a non-coding contributor for
>>> committer, it should be fairly easy to say "yes".  Especially if you're on
>>> a project like Spark where PMC ⊂ committers.
>>>
>>> (Just to again be clear, we aren't talking only about 'non-coding'
>>> committers. If it's shorthand for 'not contributing to docs/code in
>>> the repo', we're on the same page.)
>>>
>>
>> Ack.  I actually think of documentation which goes into the repo as code,
>> so I do think we're on the same page here.
>>
>>
>>> The case that started this is a corner case. The more interesting case
>>> is, in fact, a docs-only contributor. That hasn't quite come up -- we
>>> had a case of build- and config-only committer though. It's worth
>>> keeping these arguments in mind for this more ambiguous hypothetical.
>>> No objections in theory to making said person a committer, but in
>>> practice it may be a harder case, possibly unnecessarily hard.
>>>
>>
>> Documents are IP.  You're better off if you have an ICLA for that stuff,
>> regardless of where it lands in your project content.  And the most natural
>> point in time to request an ICLA is at committer invitation.
>>
>>
>>> > For solving the early and easy recognition problem, I've heard about
>>> people just saying "thank you" or making a "contributor of the month"
>>> honor.  That kind of recognition doesn't necessarily have to be in the form
>>> of a status.
>>>
>>> Sure, we do some of that on PRs, but probably need to do more. There
>>> are some regular contributors doing good work, and I hope they feel
>>> recognized by the fact they get more attention because of their track
>>> record, but being explicit goes a long way.
>>>
>>
>> As they say: positive feedback to negative feedback ratio should be 7:1.
>> There are other parts of the foundation where we need to place more
>> emphasis on positive feedback too.
>>
>> Best,
>> Myrle
>>
>

Re: Recognizing non-code contributions

Posted by Hyukjin Kwon <gu...@gmail.com>.
Myrle,

> We need to balance two sets of risks here.  But in the case of access to
our software artifacts, the risk is very small, and already has *multiple*
mitigating factors, from the fact that all changes are tracked to an
individual, to the fact that there are notifications sent when changes are
made, (and I'm going to stop listing the benefits of a modern source
control system here, because I know you are aware of them), on through the
fact that you have automated tests, and continuing through the fact that
there is a release process during which artifacts get checked again.
> If someone makes a commit who you are not expecting to make a commit, or
in an area you weren't expecting changes in, you'll notice that, right?
> What you're talking about here is your security model for your source
repository.  But restricting access isn't really the right security model
for an open source project.

I don't quite get the argument about commit bit. I _strongly_ disagree
about "the risk is very small,".
Not all of committers track all the changes. There are so many changes in
the upstream and it's already overhead to check all.
Do you know how many bugs Spark faces due to such lack of reviews that
entirely blocks the release sometimes, and how much it takes time to fix up
such commits?
We need expertise and familiarity to Spark.

It virtually means we will add some more overhead to audit each commit,
even for committers'. Why should we bother add such overhead to harm the
project?
To me, this is the most important fact. I don't think we should just count
the number of positive and negative ones.

For other reasons, we can just add or discuss about the "this kind of
in-between status Apache-wide", which is a bigger scope than here. You can
ask it to ASF and discuss further.


2019년 8월 6일 (화) 오후 3:14, Myrle Krantz <my...@apache.org>님이 작성:

> Hey Sean,
>
> Even though we are discussing our differences, on the whole I don't think
> we're that far apart in our positions.  Still the differences are where the
> conversation is actually interesting, so here goes:
>
> On Mon, Aug 5, 2019 at 3:55 PM Sean Owen <sr...@apache.org> wrote:
>
>> On Mon, Aug 5, 2019 at 3:50 AM Myrle Krantz <my...@apache.org> wrote:
>> > So... events coordinators?  I'd still make them committers.  I guess
>> I'm still struggling to understand what problem making people VIP's without
>> giving them committership is trying to solve.
>>
>> We may just agree to disagree, which is fine, but I think the argument
>> is clear enough: such a person has zero need for the commit bit.
>> Turning it around, what are we trying to accomplish by giving said
>> person a commit bit? I know people say there's no harm, but I think
>> there is at least _some_ downside. We're widening access to change
>> software artifacts, the main thing that we put ASF process and checks
>> around for liability reasons. I know the point is trust, and said
>> person is likely to understand to never use the commit bit, but it
>> brings us back to the same place. I don't wish to convince anyone else
>> of my stance, though I do find it more logical, just that it's
>> reasonable within The Apache Way.
>>
>
> We need to balance two sets of risks here.  But in the case of access to
> our software artifacts, the risk is very small, and already has *multiple*
> mitigating factors, from the fact that all changes are tracked to an
> individual, to the fact that there are notifications sent when changes are
> made, (and I'm going to stop listing the benefits of a modern source
> control system here, because I know you are aware of them), on through the
> fact that you have automated tests, and continuing through the fact that
> there is a release process during which artifacts get checked again.
>
> If someone makes a commit who you are not expecting to make a commit, or
> in an area you weren't expecting changes in, you'll notice that, right?
>
> What you're talking about here is your security model for your source
> repository.  But restricting access isn't really the right security model
> for an open source project.
>
>
>> > It also just occurred to me this morning: There are actually other
>> privileges which go along with the "commit-bit" other than the ability to
>> commit at will to the project's repos: people who are committers get an
>> Apache e-mail address, and they get discounted entry to ApacheCon.  People
>> who are committers also get added to our committers mailing list, and are
>> thus a little easier to integrate into our foundation-wide efforts.
>> >
>> > To apply this to the example above, the Apache e-mail address can make
>> it a tad easier for an event coordinator to conduct official business for a
>> project.
>>
>> Great points. Again if I'm making it up? a "VIP" should get an Apache
>> email address and discounts. Sure, why not put them on a committers@
>> list too for visibility.
>>
>
> In order to do that, you'd need to create this kind of in-between status
> Apache-wide.  I would be very much opposed to doing that for a couple of
> reasons:
> * It adds complexity for infra and events with no clear benefits to the
> projects.
> * The risk of creating a second-class status at the ASF is just too high
> for my comfort.
>
>
>> > But that's not what I'm arguing.  I'm arguing that you can safely give
>> commit access to anyone for whom you're reasonably certain they will do no
>> harm.  And your certainty of that can actually be much higher with
>> non-coding committers.  So if someone suggests a non-coding contributor for
>> committer, it should be fairly easy to say "yes".  Especially if you're on
>> a project like Spark where PMC ⊂ committers.
>>
>> (Just to again be clear, we aren't talking only about 'non-coding'
>> committers. If it's shorthand for 'not contributing to docs/code in
>> the repo', we're on the same page.)
>>
>
> Ack.  I actually think of documentation which goes into the repo as code,
> so I do think we're on the same page here.
>
>
>> The case that started this is a corner case. The more interesting case
>> is, in fact, a docs-only contributor. That hasn't quite come up -- we
>> had a case of build- and config-only committer though. It's worth
>> keeping these arguments in mind for this more ambiguous hypothetical.
>> No objections in theory to making said person a committer, but in
>> practice it may be a harder case, possibly unnecessarily hard.
>>
>
> Documents are IP.  You're better off if you have an ICLA for that stuff,
> regardless of where it lands in your project content.  And the most natural
> point in time to request an ICLA is at committer invitation.
>
>
>> > For solving the early and easy recognition problem, I've heard about
>> people just saying "thank you" or making a "contributor of the month"
>> honor.  That kind of recognition doesn't necessarily have to be in the form
>> of a status.
>>
>> Sure, we do some of that on PRs, but probably need to do more. There
>> are some regular contributors doing good work, and I hope they feel
>> recognized by the fact they get more attention because of their track
>> record, but being explicit goes a long way.
>>
>
> As they say: positive feedback to negative feedback ratio should be 7:1.
> There are other parts of the foundation where we need to place more
> emphasis on positive feedback too.
>
> Best,
> Myrle
>

Re: Recognizing non-code contributions

Posted by Myrle Krantz <my...@apache.org>.
Hey Sean,

Even though we are discussing our differences, on the whole I don't think
we're that far apart in our positions.  Still the differences are where the
conversation is actually interesting, so here goes:

On Mon, Aug 5, 2019 at 3:55 PM Sean Owen <sr...@apache.org> wrote:

> On Mon, Aug 5, 2019 at 3:50 AM Myrle Krantz <my...@apache.org> wrote:
> > So... events coordinators?  I'd still make them committers.  I guess I'm
> still struggling to understand what problem making people VIP's without
> giving them committership is trying to solve.
>
> We may just agree to disagree, which is fine, but I think the argument
> is clear enough: such a person has zero need for the commit bit.
> Turning it around, what are we trying to accomplish by giving said
> person a commit bit? I know people say there's no harm, but I think
> there is at least _some_ downside. We're widening access to change
> software artifacts, the main thing that we put ASF process and checks
> around for liability reasons. I know the point is trust, and said
> person is likely to understand to never use the commit bit, but it
> brings us back to the same place. I don't wish to convince anyone else
> of my stance, though I do find it more logical, just that it's
> reasonable within The Apache Way.
>

We need to balance two sets of risks here.  But in the case of access to
our software artifacts, the risk is very small, and already has *multiple*
mitigating factors, from the fact that all changes are tracked to an
individual, to the fact that there are notifications sent when changes are
made, (and I'm going to stop listing the benefits of a modern source
control system here, because I know you are aware of them), on through the
fact that you have automated tests, and continuing through the fact that
there is a release process during which artifacts get checked again.

If someone makes a commit who you are not expecting to make a commit, or in
an area you weren't expecting changes in, you'll notice that, right?

What you're talking about here is your security model for your source
repository.  But restricting access isn't really the right security model
for an open source project.


> > It also just occurred to me this morning: There are actually other
> privileges which go along with the "commit-bit" other than the ability to
> commit at will to the project's repos: people who are committers get an
> Apache e-mail address, and they get discounted entry to ApacheCon.  People
> who are committers also get added to our committers mailing list, and are
> thus a little easier to integrate into our foundation-wide efforts.
> >
> > To apply this to the example above, the Apache e-mail address can make
> it a tad easier for an event coordinator to conduct official business for a
> project.
>
> Great points. Again if I'm making it up? a "VIP" should get an Apache
> email address and discounts. Sure, why not put them on a committers@
> list too for visibility.
>

In order to do that, you'd need to create this kind of in-between status
Apache-wide.  I would be very much opposed to doing that for a couple of
reasons:
* It adds complexity for infra and events with no clear benefits to the
projects.
* The risk of creating a second-class status at the ASF is just too high
for my comfort.


> > But that's not what I'm arguing.  I'm arguing that you can safely give
> commit access to anyone for whom you're reasonably certain they will do no
> harm.  And your certainty of that can actually be much higher with
> non-coding committers.  So if someone suggests a non-coding contributor for
> committer, it should be fairly easy to say "yes".  Especially if you're on
> a project like Spark where PMC ⊂ committers.
>
> (Just to again be clear, we aren't talking only about 'non-coding'
> committers. If it's shorthand for 'not contributing to docs/code in
> the repo', we're on the same page.)
>

Ack.  I actually think of documentation which goes into the repo as code,
so I do think we're on the same page here.


> The case that started this is a corner case. The more interesting case
> is, in fact, a docs-only contributor. That hasn't quite come up -- we
> had a case of build- and config-only committer though. It's worth
> keeping these arguments in mind for this more ambiguous hypothetical.
> No objections in theory to making said person a committer, but in
> practice it may be a harder case, possibly unnecessarily hard.
>

Documents are IP.  You're better off if you have an ICLA for that stuff,
regardless of where it lands in your project content.  And the most natural
point in time to request an ICLA is at committer invitation.


> > For solving the early and easy recognition problem, I've heard about
> people just saying "thank you" or making a "contributor of the month"
> honor.  That kind of recognition doesn't necessarily have to be in the form
> of a status.
>
> Sure, we do some of that on PRs, but probably need to do more. There
> are some regular contributors doing good work, and I hope they feel
> recognized by the fact they get more attention because of their track
> record, but being explicit goes a long way.
>

As they say: positive feedback to negative feedback ratio should be 7:1.
There are other parts of the foundation where we need to place more
emphasis on positive feedback too.

Best,
Myrle

Re: Recognizing non-code contributions

Posted by Sean Owen <sr...@apache.org>.
On Mon, Aug 5, 2019 at 3:50 AM Myrle Krantz <my...@apache.org> wrote:
> So... events coordinators?  I'd still make them committers.  I guess I'm still struggling to understand what problem making people VIP's without giving them committership is trying to solve.

We may just agree to disagree, which is fine, but I think the argument
is clear enough: such a person has zero need for the commit bit.
Turning it around, what are we trying to accomplish by giving said
person a commit bit? I know people say there's no harm, but I think
there is at least _some_ downside. We're widening access to change
software artifacts, the main thing that we put ASF process and checks
around for liability reasons. I know the point is trust, and said
person is likely to understand to never use the commit bit, but it
brings us back to the same place. I don't wish to convince anyone else
of my stance, though I do find it more logical, just that it's
reasonable within The Apache Way.


> It also just occurred to me this morning: There are actually other privileges which go along with the "commit-bit" other than the ability to commit at will to the project's repos: people who are committers get an Apache e-mail address, and they get discounted entry to ApacheCon.  People who are committers also get added to our committers mailing list, and are thus a little easier to integrate into our foundation-wide efforts.
>
> To apply this to the example above, the Apache e-mail address can make it a tad easier for an event coordinator to conduct official business for a project.

Great points. Again if I'm making it up? a "VIP" should get an Apache
email address and discounts. Sure, why not put them on a committers@
list too for visibility.


> But that's not what I'm arguing.  I'm arguing that you can safely give commit access to anyone for whom you're reasonably certain they will do no harm.  And your certainty of that can actually be much higher with non-coding committers.  So if someone suggests a non-coding contributor for committer, it should be fairly easy to say "yes".  Especially if you're on a project like Spark where PMC ⊂ committers.

(Just to again be clear, we aren't talking only about 'non-coding'
committers. If it's shorthand for 'not contributing to docs/code in
the repo', we're on the same page.)
The case that started this is a corner case. The more interesting case
is, in fact, a docs-only contributor. That hasn't quite come up -- we
had a case of build- and config-only committer though. It's worth
keeping these arguments in mind for this more ambiguous hypothetical.
No objections in theory to making said person a committer, but in
practice it may be a harder case, possibly unnecessarily hard.

> For solving the early and easy recognition problem, I've heard about people just saying "thank you" or making a "contributor of the month" honor.  That kind of recognition doesn't necessarily have to be in the form of a status.

Sure, we do some of that on PRs, but probably need to do more. There
are some regular contributors doing good work, and I hope they feel
recognized by the fact they get more attention because of their track
record, but being explicit goes a long way.

---------------------------------------------------------------------
To unsubscribe e-mail: dev-unsubscribe@spark.apache.org


Re: Recognizing non-code contributions

Posted by Myrle Krantz <my...@apache.org>.
On Sun, Aug 4, 2019 at 10:06 PM Sean Owen <sr...@apache.org> wrote:

> Oops, I also failed to copy dev@
>
> On Sun, Aug 4, 2019 at 3:06 PM Sean Owen <sr...@apache.org> wrote:
> >
> > On Sun, Aug 4, 2019 at 1:54 PM Myrle Krantz <my...@apache.org> wrote:
> > >> No, I think the position here was docs-only contributors _could_ be
> > >> committers. The "only coders can be committers" idea was I think a
> > >> misreading and accidental strawman from the members@ thread.
> > >
> > > Can you be specific?  This status us supposed to serve as a form of
> recognition, right?  For who or what contributions?
> >
> > This "VIP" status is purely theoretical, but if I were making it up?
> > it would be for anyone, and any 'valuable' contribution to the
> > community of any kind.
> > Here I'm merely saying that nobody here suggested docs-only
> > contributors could never be committers. That seemed to be what the
> > members@ thread was taking on, as if the argument was that docs-only
> > contributors are at best these "VIPs". I want to make sure this isn't
> > predicated on a misunderstanding.
>

So... events coordinators?  I'd still make them committers.  I guess I'm
still struggling to understand what problem making people VIP's without
giving them committership is trying to solve.

It also just occurred to me this morning: There are actually other
privileges which go along with the "commit-bit" other than the ability to
commit at will to the project's repos: people who are committers get an
Apache e-mail address, and they get discounted entry to ApacheCon.  People
who are committers also get added to our committers mailing list, and are
thus a little easier to integrate into our foundation-wide efforts.

To apply this to the example above, the Apache e-mail address can make it a
tad easier for an event coordinator to conduct official business for a
project.

> > We have a saying at Apache: merit doesn't expire.  This is especially
> important for volunteer-based project work.  People's availability varies.
> If the barrier to re-entry is low, you're more likely to get contributions
> from these people.  Contributions that can have a particularly high value
> because they are informed by a longer history of the project.
> >
> > I know, I'm also here at Apache :) I guess I'm just showing we have
> > already taken a broad view of merit, consistent with Apache, when
> > there are even reasonable arguments about defining committer merit
> > more narrowly. This responds to the general notion I hear, that the
> > PMC here is somehow out of line and needs reeducation.
>

I don't think the PMC is "out of line".  I advocate for ease of access to
other projects too.  (For example:
https://twitter.com/myrleKrantz/status/1110475989941932038).  I do it
because I think it will be helpful to the projects.  But it absolutely is
true that every project decides for themselves what obstacles to access
they want to put in to place.  As long as they are being equitable while
doing that, the projects are within their rights.


> > > But the real question which I feel needs an answer here is: what's the
> harm you fear?  With modern source control, surely any harm is reversible?
> If you're worried about people doing things outside their "abilities", then
> maybe the real question you need to ask when choosing committers is just:
> do they know to ask for help first in fragile parts of the code?  You may
> find that someone who is doesn't see themselves as a coder is *less* likely
> to break code than other committers.
> >
> > This is a vital point. The argument is, what are you afraid of? how
> > much harm can a committer do? The logical extreme is to give everyone
> > commit access, which nobody appears to advocate. But where's the line?
> > What 'merit' are we talking about when we say people earn merit and
> > that's rewarded with committership? You said it: good judgment above
> > all. We have a committer that only run builds and config and doesn't
> > deal with code at all, and we agree that's good.
>

There are some people who will advocate that everyone should receive commit
access.  SVN is an example of this, and they had great success with that
approach.

But that's not what I'm arguing.  I'm arguing that you can safely give
commit access to anyone for whom you're reasonably certain they will do no
harm.  And your certainty of that can actually be much higher with
non-coding committers.  So if someone suggests a non-coding contributor for
committer, it should be fairly easy to say "yes".  Especially if you're on
a project like Spark where PMC ⊂ committers.


> > That is I'm not sure we (speaking broadly for Spark) disagree
> > substantively. Again the premise here was: somebody who hasn't and
> > doesn't want to touch anything about the project or ASF. I'm surprised
> > if there's an argument for making said person a committer; I thought
> > this was the easiest of cases. Still, I am quite sure ASF projects
> > have done this (and I disagreed). It's not crazy. But it's at least as
> > un-crazy to decide said person should not be a committer. Hence
> > surprise that this is prompting calls for reeducation.
>

Some of the rhetoric on members@ was overly harsh.  You're certainly not
crazy. : o)


> > > If the only question is "commitment" to the project, then you
> shouldn't need to worry about people breaking things who have a commit
> bit.  People who are committed to the project won't break things on
> purpose, and they'll quickly learn to not break things on access too.
> That's part of the definition of commitment.
> >
> > I still find this wordplay more than anything. Being a committer and
> > earning merit doesn't reduce to merely being 'committed'. If you
> > define 'committed' as 'won't break things on purpose, etc' then it's
> > circular. OK, how do we decide what constitutes evidence of committed,
> > etc. But at the least I'd suggest that both are a valid POV within
> > "The Apache Way" which we can discuss.
>

Some of that circularity is necessary.  When you give people added karma
for a project, you are guessing what they are going to do with it.  You
can't know claim that someone you give committership is actually going to
merge PRs until they do it.  You have to try and see.  So the circularity
is there for all of your potential committers.


> > > We really shouldn't treat committer status like a reward.  It's just a
> permission: permission to participate more fully in the project.  That
> permission includes, but is not limited to the commit bit.  We should be
> thanking people who accept those permissions because it means they intend
> to continue contributing to the project.
> >
> > You say it shouldn't be a reward, and I agree, although that ship has
> > sailed. But you say it's more than getting commit access, it's being
> > publicly honored and thanked and shown trust, which sounds like a
> > reward? I honestly think we agree in most cases, as I often say that
> > we should make X a committer to enfranchise and further encourage a
> > pattern of contribution backed by good judgment. I accept it
> > inevitably functions like recognition and that's useful, so, roll with
> > it.
>

There is a misunderstanding here, based on a clumsy choice of words on my
part.  Let me try to rephrase.  Committership is not a reward that the PMC
gives to the individual contributor.  Committership is a reward that the
individual contributor gives to the PMC.


> > The answer may be that "VIP" is a bad idea, and I'm not pushing for
> > it. I tabled it as something that might further this very same end:
> > early and easy official recognition, to show trust and encourage
> > contribution. We're really debating how high/low the committer bar
> > should be, and if it's 'low', this idea does very little.
>

For solving the early and easy recognition problem, I've heard about people
just saying "thank you" or making a "contributor of the month" honor.  That
kind of recognition doesn't necessarily have to be in the form of a status.

But again, committership is more than a commit bit.  As an example, if you
can make it easier for someone to attend ApacheCon who is doing good things
for your project and is certain not to harm it, why wouldn't you do that?


> > > I've seen this happen.  Especially when merges start slowing down,
> people either stop contributing altogether or just work from their own
> fork.  If you have enough "non-meriging" committers in your volunteer
> "pipeline" you have a better chance of people stepping up and becoming
> "merging" committers.  And at very least you remove a roadblock to
> participation when you make someone a committer.  Otherwise, you could
> accidentally doom your project to a lifespan equal to the volunteer burnout
> cycle of your one most dedicated volunteer.  Or accidentally couple it too
> tightly to the economic interests of that person's employer.  Either way,
> you'll shorten your project's lifespan unnecessarily.  And that would be a
> shame, because Spark is a really awesome project.
> >
> > On this I entirely agree. It's a different debate, but one we've had a
> > lot over time: while there are possibly risks in adding someone as a
> > committer, there are risks in not doing so too. You might find others
> > on the PMC diverge more from your position, and I don't think it's a
> > secret that I think the PMC has been a little overly conservative. The
> > debate is reasonable (and visible on private@), and in-bounds w.r.t.
> > The Apache Way. It needs to evolve, but, is not broken. I actually
> > think members@ would do well to hear more in-bounds but diverging
> > opinions, from good-faith and smart people from a different project.
> > There are real, subtle questions here. A few people have gotten pretty
> > dismissive of the learned experience of some large, successful ASF
> > projects.
>

I agree here.  And I'm open to suggestions on how to help this conversation
move.

Best Regards,
Myrle

Re: Recognizing non-code contributions

Posted by Sean Owen <sr...@apache.org>.
Oops, I also failed to copy dev@

On Sun, Aug 4, 2019 at 3:06 PM Sean Owen <sr...@apache.org> wrote:
>
> On Sun, Aug 4, 2019 at 1:54 PM Myrle Krantz <my...@apache.org> wrote:
> >> No, I think the position here was docs-only contributors _could_ be
> >> committers. The "only coders can be committers" idea was I think a
> >> misreading and accidental strawman from the members@ thread.
> >
> > Can you be specific?  This status us supposed to serve as a form of recognition, right?  For who or what contributions?
>
> This "VIP" status is purely theoretical, but if I were making it up?
> it would be for anyone, and any 'valuable' contribution to the
> community of any kind.
> Here I'm merely saying that nobody here suggested docs-only
> contributors could never be committers. That seemed to be what the
> members@ thread was taking on, as if the argument was that docs-only
> contributors are at best these "VIPs". I want to make sure this isn't
> predicated on a misunderstanding.
>
>
> > We have a saying at Apache: merit doesn't expire.  This is especially important for volunteer-based project work.  People's availability varies.  If the barrier to re-entry is low, you're more likely to get contributions from these people.  Contributions that can have a particularly high value because they are informed by a longer history of the project.
>
> I know, I'm also here at Apache :) I guess I'm just showing we have
> already taken a broad view of merit, consistent with Apache, when
> there are even reasonable arguments about defining committer merit
> more narrowly. This responds to the general notion I hear, that the
> PMC here is somehow out of line and needs reeducation.
>
>
> > But the real question which I feel needs an answer here is: what's the harm you fear?  With modern source control, surely any harm is reversible?  If you're worried about people doing things outside their "abilities", then maybe the real question you need to ask when choosing committers is just: do they know to ask for help first in fragile parts of the code?  You may find that someone who is doesn't see themselves as a coder is *less* likely to break code than other committers.
>
> This is a vital point. The argument is, what are you afraid of? how
> much harm can a committer do? The logical extreme is to give everyone
> commit access, which nobody appears to advocate. But where's the line?
> What 'merit' are we talking about when we say people earn merit and
> that's rewarded with committership? You said it: good judgment above
> all. We have a committer that only run builds and config and doesn't
> deal with code at all, and we agree that's good.
>
> That is I'm not sure we (speaking broadly for Spark) disagree
> substantively. Again the premise here was: somebody who hasn't and
> doesn't want to touch anything about the project or ASF. I'm surprised
> if there's an argument for making said person a committer; I thought
> this was the easiest of cases. Still, I am quite sure ASF projects
> have done this (and I disagreed). It's not crazy. But it's at least as
> un-crazy to decide said person should not be a committer. Hence
> surprise that this is prompting calls for reeducation.
>
>
> > If the only question is "commitment" to the project, then you shouldn't need to worry about people breaking things who have a commit bit.  People who are committed to the project won't break things on purpose, and they'll quickly learn to not break things on access too.  That's part of the definition of commitment.
>
> I still find this wordplay more than anything. Being a committer and
> earning merit doesn't reduce to merely being 'committed'. If you
> define 'committed' as 'won't break things on purpose, etc' then it's
> circular. OK, how do we decide what constitutes evidence of committed,
> etc. But at the least I'd suggest that both are a valid POV within
> "The Apache Way" which we can discuss.
>
>
> > We really shouldn't treat committer status like a reward.  It's just a permission: permission to participate more fully in the project.  That permission includes, but is not limited to the commit bit.  We should be thanking people who accept those permissions because it means they intend to continue contributing to the project.
>
> You say it shouldn't be a reward, and I agree, although that ship has
> sailed. But you say it's more than getting commit access, it's being
> publicly honored and thanked and shown trust, which sounds like a
> reward? I honestly think we agree in most cases, as I often say that
> we should make X a committer to enfranchise and further encourage a
> pattern of contribution backed by good judgment. I accept it
> inevitably functions like recognition and that's useful, so, roll with
> it.
>
> The answer may be that "VIP" is a bad idea, and I'm not pushing for
> it. I tabled it as something that might further this very same end:
> early and easy official recognition, to show trust and encourage
> contribution. We're really debating how high/low the committer bar
> should be, and if it's 'low', this idea does very little.
>
>
> > I've seen this happen.  Especially when merges start slowing down, people either stop contributing altogether or just work from their own fork.  If you have enough "non-meriging" committers in your volunteer "pipeline" you have a better chance of people stepping up and becoming "merging" committers.  And at very least you remove a roadblock to participation when you make someone a committer.  Otherwise, you could accidentally doom your project to a lifespan equal to the volunteer burnout cycle of your one most dedicated volunteer.  Or accidentally couple it too tightly to the economic interests of that person's employer.  Either way, you'll shorten your project's lifespan unnecessarily.  And that would be a shame, because Spark is a really awesome project.
>
> On this I entirely agree. It's a different debate, but one we've had a
> lot over time: while there are possibly risks in adding someone as a
> committer, there are risks in not doing so too. You might find others
> on the PMC diverge more from your position, and I don't think it's a
> secret that I think the PMC has been a little overly conservative. The
> debate is reasonable (and visible on private@), and in-bounds w.r.t.
> The Apache Way. It needs to evolve, but, is not broken. I actually
> think members@ would do well to hear more in-bounds but diverging
> opinions, from good-faith and smart people from a different project.
> There are real, subtle questions here. A few people have gotten pretty
> dismissive of the learned experience of some large, successful ASF
> projects.

---------------------------------------------------------------------
To unsubscribe e-mail: dev-unsubscribe@spark.apache.org


Re: Recognizing non-code contributions

Posted by Sean Owen <sr...@apache.org>.
On Sun, Aug 4, 2019 at 11:21 AM Myrle Krantz <my...@apache.org> wrote:
> Let me make a guess at what you are trying to accomplish with it.  Correct me please if I'm wrong:
> * You want to encourage contributions that aren't just code contributions.  You recognize for example that good documentation is critical to bringing new contributors and committers to your project.  You recognize that other non-code contributions are also important to your project's success.

No, I think the position here was docs-only contributors _could_ be
committers. The "only coders can be committers" idea was I think a
misreading and accidental strawman from the members@ thread.

> * You want to keep access to the code repositories "tight", ie minimized to people who you view as likely to be actively using that access for the goals of the project.  In particular, someone who writes tutorials, may not even want access to the spark code repositories.

Yes, but in the narrow sense that someone with zero interaction with
the repo doesn't seem to need any repo access. That may actually be
uncontroversial?

There are tougher question here which we've always grappled with:
- What about someone who's stopped contributing but did some good work
in the past? they probably won't use the commit bit or contribute.
Generally I think we've recognized 'past-only' merit with the commit
bit.
- What about someone who's made a lot of code contributions but
doesn't review changes? have they demonstrated the skill that we need
people to deploy when they have the commit bit, which is knowing when
_others_ changes are OK to merge? Again I think we've only distantly
considered that, and there are committers who are almost entirely
contributors and not reviewers

... though I don't think we were questioning that here. (Maybe we should)


> Other arguments I've heard for this (which you may or may not share)
> * Fear of "diluting" the social value of committership in your project.

I think the goal is to tease apart being a committer as having a
particular important power in the repo, and being a committer as
recognition and valuable social status. I don't think there's a worry
that too many committers dilute the 'brand'. On the contrary, I'd say
we _don't_ want mere recognition to be a factor in handing out this
access? Recognition ought to be more freely available, without
debating the effects of giving commit access.


> And here are some possible effects of creating a second category of recognition for non-code contributions which does not include commit access:
> * things that aren't rewarded with committership are somehow viewed as being of lesser value to the project.  (It's already true in the industry that documentation writers, marketers, and event-organizers are sometime looked down on by programmers.)

Agree, though, again, I don't think anyone is saying docs-only
contributors can't be committers here.
Really I just think it's unfortunate how much the commit bit is
considered a 'reward' - maybe that's the problem.

You hit on one of my concerns. In theory some "VIP" status is better
than nothing, but if it's viewed as second-class status more than
anything, it backfires. That is, does it become perceived as a badge
of "someone the PMC has definitely decided won't be a committer,
because they decided to hand out VIP status instead"? It could be
worse than having no status, which indicates nothing about a PMC
decision.


> * People doing these kinds of work may view the default location to do that work as being somewhere other than the spark code repositories, even if the code repository would be a better choice.  (For example, if a tutorial is kept in the code repository, coding examples will automatically show compile errors when there are breaking changes in an API.  But if documentation writers are told, implicitly, not to work in code repositories, the possibility of keeping code examples in the repository may never be considered.)

Spark should have docs, quickstarts, examples. It already does in the
repo and people can and do contribute to them. So, I would agree this
is a risk if the project wasn't welcoming examples, docs, etc. But
that's not the position here nor reality -- I merge doc changes all
the time. Therefore I don't perceive this as related to whether or not
there's a VIP status. It would be a problem with not caring about
docs, full stop.


> * People who want to work on spark may be more likely to chose to fork your repository rather than work with it directly.
> * People who start off in other areas but have minor contributions to make to your code may never make those contributions (for example someone who does manual QA may never make that first step to automate manual work, or improve existing automated tests.  Documentation writers may never bother to create that pull request on your grammar errors for your inline documentation, so inline documentation may become a less-maintained second class citizen.)

Like, if not given the commit bit, they'll work only outside the repo?
I am not sure this is a risk? we're already talking about people who
contribute to Spark without being a committer. Would they stop if we
gave them VIP status but not the commit bit? (the risk there is as
above I think)

Or is the idea that if one doesn't have the rights to merge one's own
changes rapidly, they'll stop? That's better solved by ensuring that
good patches get reviewed and merged quickly, which is a big, separate
problem.

---------------------------------------------------------------------
To unsubscribe e-mail: dev-unsubscribe@spark.apache.org


Re: Recognizing non-code contributions

Posted by Myrle Krantz <my...@apache.org>.
Hey Sean,

On 2019/08/02 13:20:56, Sean Owen <sr...@apache.org> wrote: 
> Yes, there's an interesting idea that came up on members@: should
> there be a status in Spark that doesn't include the commit bit or
> additional 'rights', but is formally recognized by the PMC? An MVP,
> VIP, Knight of the Apache Foo project. I don't think any other project
> does this, but don't think it's _prohibited_. This could recognize
> long-time contributors of any kind (code, docs, external) for whom
> it's not yet right to make a committer. The point would be
> recognition, because there's really nothing short of that the project
> can formally do. I personally am not sure it adds enough to justify
> the process, and may wade too deeply into controversies about whether
> this is just extra gatekeeping vs something helpful.

Before adding an MVP status or the like, we should consider what kinds of effects you are trying to accomplish with it, and what other kinds of effects it might have. 

Let me make a guess at what you are trying to accomplish with it.  Correct me please if I'm wrong:
* You want to encourage contributions that aren't just code contributions.  You recognize for example that good documentation is critical to bringing new contributors and committers to your project.  You recognize that other non-code contributions are also important to your project's success.
* You want to keep access to the code repositories "tight", ie minimized to people who you view as likely to be actively using that access for the goals of the project.  In particular, someone who writes tutorials, may not even want access to the spark code repositories.

If I missed something important, please let me know. 

Other arguments I've heard for this (which you may or may not share)
* Fear of "diluting" the social value of committership in your project.

And here are some possible effects of creating a second category of recognition for non-code contributions which does not include commit access:
* things that aren't rewarded with committership are somehow viewed as being of lesser value to the project.  (It's already true in the industry that documentation writers, marketers, and event-organizers are sometime looked down on by programmers.)
* People doing these kinds of work may view the default location to do that work as being somewhere other than the spark code repositories, even if the code repository would be a better choice.  (For example, if a tutorial is kept in the code repository, coding examples will automatically show compile errors when there are breaking changes in an API.  But if documentation writers are told, implicitly, not to work in code repositories, the possibility of keeping code examples in the repository may never be considered.)
* People who want to work on spark may be more likely to chose to fork your repository rather than work with it directly.
* People who start off in other areas but have minor contributions to make to your code may never make those contributions (for example someone who does manual QA may never make that first step to automate manual work, or improve existing automated tests.  Documentation writers may never bother to create that pull request on your grammar errors for your inline documentation, so inline documentation may become a less-maintained second class citizen.)

Many of these effects are just variants of "Conway's law".  For those who haven't heard of it: 

"organizations which design systems ... are constrained to produce designs which are copies of the communication structures of these organizations." — M. Conway, 1967
https://en.wikipedia.org/wiki/Conway%27s_law

Best Regards,
Myrle

---------------------------------------------------------------------
To unsubscribe e-mail: dev-unsubscribe@spark.apache.org


Re: Recognizing non-code contributions

Posted by Sean Owen <sr...@apache.org>.
Yes, there's an interesting idea that came up on members@: should
there be a status in Spark that doesn't include the commit bit or
additional 'rights', but is formally recognized by the PMC? An MVP,
VIP, Knight of the Apache Foo project. I don't think any other project
does this, but don't think it's _prohibited_. This could recognize
long-time contributors of any kind (code, docs, external) for whom
it's not yet right to make a committer. The point would be
recognition, because there's really nothing short of that the project
can formally do. I personally am not sure it adds enough to justify
the process, and may wade too deeply into controversies about whether
this is just extra gatekeeping vs something helpful.

On Thu, Aug 1, 2019 at 11:09 PM Sean Owen <sr...@apache.org> wrote:
>
> (Let's move this thread to dev@ now as it is a general and important
> community question. This was requested on members@)
>
> On Thu, Aug 1, 2019 at 10:20 PM Matei Zaharia <ma...@gmail.com> wrote:
> >
> > Our text on becoming a committer already says that we want committers who focus on our docs: https://spark.apache.org/committers.html. Just working on docs / books outside the project doesn’t make as much sense IMO.
> >
> > Matei
> >
> > On Aug 1, 2019, at 8:10 PM, Reynold Xin <rx...@databricks.com> wrote:
> >
> > Not sure if ASF allows this but we can start some sort of “MVP” program recognizing non-code contributors...

---------------------------------------------------------------------
To unsubscribe e-mail: dev-unsubscribe@spark.apache.org