You are viewing a plain text version of this content. The canonical link for it is here.

Posted to dev@spark.apache.org by Cody Koeninger <co...@koeninger.org> on 2017/03/07 15:15:38 UTC

Re: Spark Improvement Proposals

Another week, another ping.  Anyone on the PMC willing to call a vote on
this?

On Mon, Feb 27, 2017 at 3:08 PM, Ryan Blue <rb...@netflix.com> wrote:

> I'd like to see more discussion on the issues I raised. I don't think
> there was a response for why voting is limited to PMC members.
>
> Tim was kind enough to reply with his rationale for a shepherd, but I
> don't think that it justifies failing proposals. I think it boiled down to
> "shepherds can be helpful", which isn't a good reason to require them in my
> opinion. Sam also had some good comments on this and I think that there's
> more to talk about.
>
> That said, I'd rather not have this proposal fail because we're tired of
> talking about it. If most people are okay with it as it stands and want a
> vote, I'm fine testing this out and fixing it later.
>
> rb
>
> On Fri, Feb 24, 2017 at 8:28 PM, Joseph Bradley <jo...@databricks.com>
> wrote:
>
>> The current draft LGTM.  I agree some of the various concerns may need to
>> be addressed in the future, depending on how SPIPs progress in practice.
>> If others agree, let's put it to a vote and revisit the proposal in a few
>> months.
>> Joseph
>>
>> On Fri, Feb 24, 2017 at 5:35 AM, Cody Koeninger <co...@koeninger.org>
>> wrote:
>>
>>> It's been a week since any further discussion.
>>>
>>> Do PMC members think the current draft is OK to vote on?
>>>
>>> On Fri, Feb 17, 2017 at 10:41 PM, vaquar khan <va...@gmail.com>
>>> wrote:
>>> > I like document and happy to see SPIP draft version however i feel
>>> shepherd
>>> > role is again hurdle in process improvement ,It's like everything
>>> depends
>>> > only on shepherd .
>>> >
>>> > Also want to add point that SPIP  should be time bound with define SLA
>>> else
>>> > will defeats purpose.
>>> >
>>> >
>>> > Regards,
>>> > Vaquar khan
>>> >
>>> > On Thu, Feb 16, 2017 at 3:26 PM, Ryan Blue <rb...@netflix.com.invalid>
>>> > wrote:
>>> >>
>>> >> > [The shepherd] can advise on technical and procedural
>>> considerations for
>>> >> > people outside the community
>>> >>
>>> >> The sentiment is good, but this doesn't justify requiring a shepherd
>>> for a
>>> >> proposal. There are plenty of people that wouldn't need this, would
>>> get
>>> >> feedback during discussion, or would ask a committer or PMC member if
>>> it
>>> >> weren't a formal requirement.
>>> >>
>>> >> > if no one is willing to be a shepherd, the proposed idea is
>>> probably not
>>> >> > going to receive much traction in the first place.
>>> >>
>>> >> This also doesn't sound like a reason for needing a shepherd. Saying
>>> that
>>> >> a shepherd probably won't hurt the process doesn't give me an idea of
>>> why a
>>> >> shepherd should be required in the first place.
>>> >>
>>> >> What was the motivation for adding a shepherd originally? It may not
>>> be
>>> >> bad and it could be helpful, but neither of those makes me think that
>>> they
>>> >> should be required or else the proposal fails.
>>> >>
>>> >> rb
>>> >>
>>> >> On Thu, Feb 16, 2017 at 12:23 PM, Tim Hunter <
>>> timhunter@databricks.com>
>>> >> wrote:
>>> >>>
>>> >>> The doc looks good to me.
>>> >>>
>>> >>> Ryan, the role of the shepherd is to make sure that someone
>>> >>> knowledgeable with Spark processes is involved: this person can
>>> advise
>>> >>> on technical and procedural considerations for people outside the
>>> >>> community. Also, if no one is willing to be a shepherd, the proposed
>>> >>> idea is probably not going to receive much traction in the first
>>> >>> place.
>>> >>>
>>> >>> Tim
>>> >>>
>>> >>> On Thu, Feb 16, 2017 at 9:17 AM, Cody Koeninger <co...@koeninger.org>
>>> >>> wrote:
>>> >>> > Reynold, thanks, LGTM.
>>> >>> >
>>> >>> > Sean, great concerns.  I agree that behavior is largely cultural
>>> and
>>> >>> > writing down a process won't necessarily solve any problems one
>>> way or
>>> >>> > the other.  But one outwardly visible change I'm hoping for out of
>>> >>> > this a way for people who have a stake in Spark, but can't follow
>>> >>> > jiras closely, to go to the Spark website, see the list of proposed
>>> >>> > major changes, contribute discussion on issues that are relevant to
>>> >>> > their needs, and see a clear direction once a vote has passed.  We
>>> >>> > don't have that now.
>>> >>> >
>>> >>> > Ryan, realistically speaking any PMC member can and will stop any
>>> >>> > changes they don't like anyway, so might as well be up front about
>>> the
>>> >>> > reality of the situation.
>>> >>> >
>>> >>> > On Thu, Feb 16, 2017 at 10:43 AM, Sean Owen <so...@cloudera.com>
>>> wrote:
>>> >>> >> The text seems fine to me. Really, this is not describing a
>>> >>> >> fundamentally
>>> >>> >> new process, which is good. We've always had JIRAs, we've always
>>> been
>>> >>> >> able
>>> >>> >> to call a VOTE for a big question. This just writes down a
>>> sensible
>>> >>> >> set of
>>> >>> >> guidelines for putting those two together when a major change is
>>> >>> >> proposed. I
>>> >>> >> look forward to turning some big JIRAs into a request for a SPIP.
>>> >>> >>
>>> >>> >> My only hesitation is that this seems to be perceived by some as
>>> a new
>>> >>> >> or
>>> >>> >> different thing, that is supposed to solve some problems that
>>> aren't
>>> >>> >> otherwise solvable. I see mentioned problems like: clear process
>>> for
>>> >>> >> managing work, public communication, more committers, some sort of
>>> >>> >> binding
>>> >>> >> outcome and deadline.
>>> >>> >>
>>> >>> >> If SPIP is supposed to be a way to make people design in public
>>> and a
>>> >>> >> way to
>>> >>> >> force attention to a particular change, then, this doesn't do
>>> that by
>>> >>> >> itself. Therefore I don't want to let a detailed discussion of
>>> SPIP
>>> >>> >> detract
>>> >>> >> from the discussion about doing what SPIP implies. It's just a
>>> process
>>> >>> >> document.
>>> >>> >>
>>> >>> >> Still, a fine step IMHO.
>>> >>> >>
>>> >>> >> On Thu, Feb 16, 2017 at 4:22 PM Reynold Xin <rx...@databricks.com>
>>> >>> >> wrote:
>>> >>> >>>
>>> >>> >>> Updated. Any feedback from other community members?
>>> >>> >>>
>>> >>> >>>
>>> >>> >>> On Wed, Feb 15, 2017 at 2:53 AM, Cody Koeninger <
>>> cody@koeninger.org>
>>> >>> >>> wrote:
>>> >>> >>>>
>>> >>> >>>> Thanks for doing that.
>>> >>> >>>>
>>> >>> >>>> Given that there are at least 4 different Apache voting
>>> processes,
>>> >>> >>>> "typical Apache vote process" isn't meaningful to me.
>>> >>> >>>>
>>> >>> >>>> I think the intention is that in order to pass, it needs at
>>> least 3
>>> >>> >>>> +1
>>> >>> >>>> votes from PMC members *and no -1 votes from PMC members*.  But
>>> the
>>> >>> >>>> document
>>> >>> >>>> doesn't explicitly say that second part.
>>> >>> >>>>
>>> >>> >>>> There's also no mention of the duration a vote should remain
>>> open.
>>> >>> >>>> There's a mention of a month for finding a shepherd, but that's
>>> >>> >>>> different.
>>> >>> >>>>
>>> >>> >>>> Other than that, LGTM.
>>> >>> >>>>
>>> >>> >>>> On Mon, Feb 13, 2017 at 9:02 AM, Reynold Xin <
>>> rxin@databricks.com>
>>> >>> >>>> wrote:
>>> >>> >>>>>
>>> >>> >>>>> Here's a new draft that incorporated most of the feedback:
>>> >>> >>>>>
>>> >>> >>>>> https://docs.google.com/document/d/1-Zdi_W-wtuxS9hTK0P9qb2x-
>>> nRanvXmnZ7SUi4qMljg/edit#
>>> >>> >>>>>
>>> >>> >>>>> I added a specific role for SPIP Author and another one for
>>> SPIP
>>> >>> >>>>> Shepherd.
>>> >>> >>>>>
>>> >>> >>>>> On Sat, Feb 11, 2017 at 6:13 PM, Xiao Li <gatorsmile@gmail.com
>>> >
>>> >>> >>>>> wrote:
>>> >>> >>>>>>
>>> >>> >>>>>> During the summit, I also had a lot of discussions over
>>> similar
>>> >>> >>>>>> topics
>>> >>> >>>>>> with multiple Committers and active users. I heard many
>>> fantastic
>>> >>> >>>>>> ideas. I
>>> >>> >>>>>> believe Spark improvement proposals are good channels to
>>> collect
>>> >>> >>>>>> the
>>> >>> >>>>>> requirements/designs.
>>> >>> >>>>>>
>>> >>> >>>>>>
>>> >>> >>>>>> IMO, we also need to consider the priority when working on
>>> these
>>> >>> >>>>>> items.
>>> >>> >>>>>> Even if the proposal is accepted, it does not mean it will be
>>> >>> >>>>>> implemented
>>> >>> >>>>>> and merged immediately. It is not a FIFO queue.
>>> >>> >>>>>>
>>> >>> >>>>>>
>>> >>> >>>>>> Even if some PRs are merged, sometimes, we still have to
>>> revert
>>> >>> >>>>>> them
>>> >>> >>>>>> back, if the design and implementation are not reviewed
>>> carefully.
>>> >>> >>>>>> We have
>>> >>> >>>>>> to ensure our quality. Spark is not an application software.
>>> It is
>>> >>> >>>>>> an
>>> >>> >>>>>> infrastructure software that is being used by many many
>>> companies.
>>> >>> >>>>>> We have
>>> >>> >>>>>> to be very careful in the design and implementation,
>>> especially
>>> >>> >>>>>> adding/changing the external APIs.
>>> >>> >>>>>>
>>> >>> >>>>>>
>>> >>> >>>>>> When I developed the Mainframe infrastructure/middleware
>>> software
>>> >>> >>>>>> in
>>> >>> >>>>>> the past 6 years, I were involved in the discussions with
>>> >>> >>>>>> external/internal
>>> >>> >>>>>> customers. The to-do feature list was always above 100.
>>> Sometimes,
>>> >>> >>>>>> the
>>> >>> >>>>>> customers are feeling frustrated when we are unable to deliver
>>> >>> >>>>>> them on time
>>> >>> >>>>>> due to the resource limits and others. Even if they paid us
>>> >>> >>>>>> billions, we
>>> >>> >>>>>> still need to do it phase by phase or sometimes they have to
>>> >>> >>>>>> accept the
>>> >>> >>>>>> workarounds. That is the reality everyone has to face, I
>>> think.
>>> >>> >>>>>>
>>> >>> >>>>>>
>>> >>> >>>>>> Thanks,
>>> >>> >>>>>>
>>> >>> >>>>>>
>>> >>> >>>>>> Xiao Li
>>> >>> >>>>>>>
>>> >>> >>>>>>>
>>> >>> >>
>>> >>> >
>>> >>> > ------------------------------------------------------------
>>> ---------
>>> >>> > To unsubscribe e-mail: dev-unsubscribe@spark.apache.org
>>> >>> >
>>> >>>
>>> >>> ------------------------------------------------------------
>>> ---------
>>> >>> To unsubscribe e-mail: dev-unsubscribe@spark.apache.org
>>> >>>
>>> >>
>>> >>
>>> >>
>>> >> --
>>> >> Ryan Blue
>>> >> Software Engineer
>>> >> Netflix
>>> >
>>> >
>>> >
>>> >
>>> > --
>>> > Regards,
>>> > Vaquar Khan
>>> > +1 -224-436-0783
>>> >
>>> > IT Architect / Lead Consultant
>>> > Greater Chicago
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe e-mail: dev-unsubscribe@spark.apache.org
>>>
>>>
>>
>>
>> --
>>
>> Joseph Bradley
>>
>> Software Engineer - Machine Learning
>>
>> Databricks, Inc.
>>
>> [image: http://databricks.com] <http://databricks.com/>
>>
>
>
>
> --
> Ryan Blue
> Software Engineer
> Netflix
>

Re: Spark Improvement Proposals

Posted by vaquar khan <va...@gmail.com>.

Many of us have issue with "shepherd role " , i think we should go with
vote.

Regards,
Vaquar khan

On Thu, Mar 9, 2017 at 11:00 AM, Reynold Xin <rx...@databricks.com> wrote:

> I'm fine without a vote. (are we voting on wether we need a vote?)
>
>
> On Thu, Mar 9, 2017 at 8:55 AM, Sean Owen <so...@cloudera.com> wrote:
>
>> I think a VOTE is over-thinking it, and is rarely used, but, can't hurt.
>> Nah, anyone can call a vote. This really isn't that formal. We just want to
>> declare and document consensus.
>>
>> I think SPIP is just a remix of existing process anyway, and don't think
>> it will actually do much anyway, which is why I am sanguine about the whole
>> thing.
>>
>> To bring this to a conclusion, I will just put the contents of the doc in
>> an email tomorrow for a VOTE. Raise any objections now.
>>
>> On Thu, Mar 9, 2017 at 3:39 PM Cody Koeninger <co...@koeninger.org> wrote:
>>
>>> I started this idea as a fork with a merge-able change to docs.
>>> Reynold moved it to his google doc, and has suggested during this
>>> email thread that a vote should occur.
>>> If a vote needs to occur, I can't see anything on
>>> http://apache.org/foundation/voting.html suggesting that I can call
>>> for a vote, which is why I'm asking PMC members to do it since they're
>>> the ones who would vote anyway.
>>> Now Sean is saying this is a code/doc change that can just be reviewed
>>> and merged as usual...which is what I tried to do to begin with.
>>>
>>> The fact that you haven't agreed on a process to agree on your process
>>> is, I think, an indication that the process really does need
>>> improvement ;)
>>>
>>>
>


-- 
Regards,
Vaquar Khan
+1 -224-436-0783

IT Architect / Lead Consultant
Greater Chicago

Re: Spark Improvement Proposals

Posted by Koert Kuipers <ko...@tresata.com>.

gonna end up with a stackoverflow on recursive votes here

On Thu, Mar 9, 2017 at 1:17 PM, Mark Hamstra <ma...@clearstorydata.com>
wrote:

> -0 on voting on whether we need a vote.
>
> On Thu, Mar 9, 2017 at 9:00 AM, Reynold Xin <rx...@databricks.com> wrote:
>
>> I'm fine without a vote. (are we voting on wether we need a vote?)
>>
>>
>> On Thu, Mar 9, 2017 at 8:55 AM, Sean Owen <so...@cloudera.com> wrote:
>>
>>> I think a VOTE is over-thinking it, and is rarely used, but, can't hurt.
>>> Nah, anyone can call a vote. This really isn't that formal. We just want to
>>> declare and document consensus.
>>>
>>> I think SPIP is just a remix of existing process anyway, and don't think
>>> it will actually do much anyway, which is why I am sanguine about the whole
>>> thing.
>>>
>>> To bring this to a conclusion, I will just put the contents of the doc
>>> in an email tomorrow for a VOTE. Raise any objections now.
>>>
>>> On Thu, Mar 9, 2017 at 3:39 PM Cody Koeninger <co...@koeninger.org>
>>> wrote:
>>>
>>>> I started this idea as a fork with a merge-able change to docs.
>>>> Reynold moved it to his google doc, and has suggested during this
>>>> email thread that a vote should occur.
>>>> If a vote needs to occur, I can't see anything on
>>>> http://apache.org/foundation/voting.html suggesting that I can call
>>>> for a vote, which is why I'm asking PMC members to do it since they're
>>>> the ones who would vote anyway.
>>>> Now Sean is saying this is a code/doc change that can just be reviewed
>>>> and merged as usual...which is what I tried to do to begin with.
>>>>
>>>> The fact that you haven't agreed on a process to agree on your process
>>>> is, I think, an indication that the process really does need
>>>> improvement ;)
>>>>
>>>>
>>
>

Re: Spark Improvement Proposals

Posted by Mark Hamstra <ma...@clearstorydata.com>.

-0 on voting on whether we need a vote.

On Thu, Mar 9, 2017 at 9:00 AM, Reynold Xin <rx...@databricks.com> wrote:

> I'm fine without a vote. (are we voting on wether we need a vote?)
>
>
> On Thu, Mar 9, 2017 at 8:55 AM, Sean Owen <so...@cloudera.com> wrote:
>
>> I think a VOTE is over-thinking it, and is rarely used, but, can't hurt.
>> Nah, anyone can call a vote. This really isn't that formal. We just want to
>> declare and document consensus.
>>
>> I think SPIP is just a remix of existing process anyway, and don't think
>> it will actually do much anyway, which is why I am sanguine about the whole
>> thing.
>>
>> To bring this to a conclusion, I will just put the contents of the doc in
>> an email tomorrow for a VOTE. Raise any objections now.
>>
>> On Thu, Mar 9, 2017 at 3:39 PM Cody Koeninger <co...@koeninger.org> wrote:
>>
>>> I started this idea as a fork with a merge-able change to docs.
>>> Reynold moved it to his google doc, and has suggested during this
>>> email thread that a vote should occur.
>>> If a vote needs to occur, I can't see anything on
>>> http://apache.org/foundation/voting.html suggesting that I can call
>>> for a vote, which is why I'm asking PMC members to do it since they're
>>> the ones who would vote anyway.
>>> Now Sean is saying this is a code/doc change that can just be reviewed
>>> and merged as usual...which is what I tried to do to begin with.
>>>
>>> The fact that you haven't agreed on a process to agree on your process
>>> is, I think, an indication that the process really does need
>>> improvement ;)
>>>
>>>
>

Re: Spark Improvement Proposals

Posted by Reynold Xin <rx...@databricks.com>.

I'm fine without a vote. (are we voting on wether we need a vote?)


On Thu, Mar 9, 2017 at 8:55 AM, Sean Owen <so...@cloudera.com> wrote:

> I think a VOTE is over-thinking it, and is rarely used, but, can't hurt.
> Nah, anyone can call a vote. This really isn't that formal. We just want to
> declare and document consensus.
>
> I think SPIP is just a remix of existing process anyway, and don't think
> it will actually do much anyway, which is why I am sanguine about the whole
> thing.
>
> To bring this to a conclusion, I will just put the contents of the doc in
> an email tomorrow for a VOTE. Raise any objections now.
>
> On Thu, Mar 9, 2017 at 3:39 PM Cody Koeninger <co...@koeninger.org> wrote:
>
>> I started this idea as a fork with a merge-able change to docs.
>> Reynold moved it to his google doc, and has suggested during this
>> email thread that a vote should occur.
>> If a vote needs to occur, I can't see anything on
>> http://apache.org/foundation/voting.html suggesting that I can call
>> for a vote, which is why I'm asking PMC members to do it since they're
>> the ones who would vote anyway.
>> Now Sean is saying this is a code/doc change that can just be reviewed
>> and merged as usual...which is what I tried to do to begin with.
>>
>> The fact that you haven't agreed on a process to agree on your process
>> is, I think, an indication that the process really does need
>> improvement ;)
>>
>>

Re: Spark Improvement Proposals

Posted by Sean Owen <so...@cloudera.com>.

Responding to your request for a vote, I meant that this isn't required per
se and the consensus here was not to vote on it. Hence the jokes about
meta-voting protocol. In that sense nothing new happened process-wise,
nothing against ASF norms, if that's your concern.

I think it's just an agreed convention now, that we will VOTE, as normal,
on particular types of changes that we call SPIPs. I mean it's no new
process in the ASF sense because VOTEs are an existing mechanic. I
personally view it as, simply, additional guidance about how to manage huge
JIRAs in a way that makes them stand a chance of moving forward. I suppose
we could VOTE about any JIRA if we wanted. They all proceed via lazy
consensus at the moment.

Practically -- I heard support for codifying this process and no objections
to the final form. This was bouncing around in process purgatory, when no
particular new process was called for.

It takes effect immediately, implicitly, like anything else I guess, like
amendments to code style guidelines. Please uses SPIPs to propose big
changes from here.

As to finding it hard to pick out of the noise, sure, I sympathize. Many
big things happen without a VOTE tag though. It does take a time investment
to triage these email lists. I don't know that this by itself means a VOTE
should have happened.

On Mon, Mar 13, 2017 at 6:15 PM Tom Graves <tg...@yahoo.com> wrote:

> Another thing I think you should send out is when exactly does this take
> affect.  Is it any major new feature without a pull request?   Is it
> anything major starting with the 2.3 release?
>
> Tom
>
>
> On Monday, March 13, 2017 1:08 PM, Tom Graves <tg...@yahoo.com.INVALID>
> wrote:
>
>
> I'm not sure how you can say its not a new process.  If that is the case
> why do we need a page documenting it?
> As a developer if I want to put up a major improvement I have to now
> follow the SPIP whereas before I didn't, that certain seems like a new
> process.  As a PMC member I now have the ability to vote on these SPIPs,
> that seems like something new again.
>
> There are  apache bylaws and then there are project specific bylaws.  As
> far as I know Spark doesn't document any of its project specific bylaws so
> I guess this isn't officially a change to them, but it was implicit before
> that you didn't need any review for major improvements before, now you need
> an explicit vote for them to be approved.  Certainly seems to fall under
> the "Procedural" section in the voting link you sent.
>
> I understand this was under discussion for a while and you have asked for
> peoples feedback multiple times.  But sometimes long threads are easy to
> ignore.  That is why personally I like to see things labelled [VOTE],
> [ANNOUNCE], [DISCUSS] when it gets close to finalizing on something like
> this.
>
> I don't really want to draw this out or argue anymore about it, if I
> really wanted a vote I guess I would -1 the change. I'm not going to do
> that.
> I would at least like to see an announcement go out about it.  The last
> thing I saw you say was you were going to call a vote.  A few people chimed
> in with their thoughts on that vote, but nothing was said after that.
>
> Tom
>
>
>
> On Monday, March 13, 2017 12:36 PM, Sean Owen <so...@cloudera.com> wrote:
>
>
> It's not a new process, in that it doesn't entail anything not already in
> http://apache.org/foundation/voting.html . We're just deciding to call a
> VOTE for this type of code modification.
>
> To your point -- yes, it's been around a long time with no further
> comment, and I called several times for more input. That's pretty strong
> lazy consensus of the form we use every day.
>
> On Mon, Mar 13, 2017 at 5:30 PM Tom Graves <tg...@yahoo.com> wrote:
>
> It seems like if you are adding responsibilities you should do a vote.
> SPIP'S require votes from PMC members so you are now putting more
> responsibility on them. It feels like we should have an official vote to
> make sure they (PMC members) agree with that and to make sure everyone pays
> attention to it.  That thread has been there for a while just as discussion
> and now all of a sudden its implemented without even an announcement being
> sent out about it.
>
> Tom
>
>
>
>
>
>

Re: Spark Improvement Proposals

Posted by Tom Graves <tg...@yahoo.com.INVALID>.

Another thing I think you should send out is when exactly does this take affect. Is it any major new feature without a pull request? Is it anything major starting with the 2.3 release?
Tom

On Monday, March 13, 2017 1:08 PM, Tom Graves <tg...@yahoo.com.INVALID> wrote:

I'm not sure how you can say its not a new process. If that is the case why do we need a page documenting it?
As a developer if I want to put up a major improvement I have to now follow the SPIP whereas before I didn't, that certain seems like a new process. As a PMC member I now have the ability to vote on these SPIPs, that seems like something new again.
There are apache bylaws and then there are project specific bylaws. As far as I know Spark doesn't document any of its project specific bylaws so I guess this isn't officially a change to them, but it was implicit before that you didn't need any review for major improvements before, now you need an explicit vote for them to be approved. Certainly seems to fall under the "Procedural" section in the voting link you sent.
I understand this was under discussion for a while and you have asked for peoples feedback multiple times. But sometimes long threads are easy to ignore. That is why personally I like to see things labelled [VOTE], [ANNOUNCE], [DISCUSS] when it gets close to finalizing on something like this.
I don't really want to draw this out or argue anymore about it, if I really wanted a vote I guess I would -1 the change. I'm not going to do that. I would at least like to see an announcement go out about it. The last thing I saw you say was you were going to call a vote. A few people chimed in with their thoughts on that vote, but nothing was said after that.
Tom

On Monday, March 13, 2017 12:36 PM, Sean Owen <so...@cloudera.com> wrote:

It's not a new process, in that it doesn't entail anything not already in http://apache.org/foundation/voting.html . We're just deciding to call a VOTE for this type of code modification.
To your point -- yes, it's been around a long time with no further comment, and I called several times for more input. That's pretty strong lazy consensus of the form we use every day.

On Mon, Mar 13, 2017 at 5:30 PM Tom Graves <tg...@yahoo.com> wrote:

It seems like if you are adding responsibilities you should do a vote. SPIP'S require votes from PMC members so you are now putting more responsibility on them. It feels like we should have an official vote to make sure they (PMC members) agree with that and to make sure everyone pays attention to it. That thread has been there for a while just as discussion and now all of a sudden its implemented without even an announcement being sent out about it.
Tom

Re: Spark Improvement Proposals

Posted by Tom Graves <tg...@yahoo.com.INVALID>.

On Monday, March 13, 2017 12:36 PM, Sean Owen <so...@cloudera.com> wrote:

On Mon, Mar 13, 2017 at 5:30 PM Tom Graves <tg...@yahoo.com> wrote:

Re: Spark Improvement Proposals

Posted by Sean Owen <so...@cloudera.com>.

It's not a new process, in that it doesn't entail anything not already in
http://apache.org/foundation/voting.html . We're just deciding to call a
VOTE for this type of code modification.

To your point -- yes, it's been around a long time with no further comment,
and I called several times for more input. That's pretty strong lazy
consensus of the form we use every day.

On Mon, Mar 13, 2017 at 5:30 PM Tom Graves <tg...@yahoo.com> wrote:

> It seems like if you are adding responsibilities you should do a vote.
> SPIP'S require votes from PMC members so you are now putting more
> responsibility on them. It feels like we should have an official vote to
> make sure they (PMC members) agree with that and to make sure everyone pays
> attention to it.  That thread has been there for a while just as discussion
> and now all of a sudden its implemented without even an announcement being
> sent out about it.
>
> Tom
>
>

Re: Spark Improvement Proposals

Posted by Tom Graves <tg...@yahoo.com.INVALID>.

On Monday, March 13, 2017 11:37 AM, Sean Owen <so...@cloudera.com> wrote:

This ended up proceeding as a normal doc change, instead of precipitating a meta-vote.However, the text that's on the web site now can certainly be further amended if anyone wants to propose a change from here.
On Mon, Mar 13, 2017 at 1:50 PM Tom Graves <tg...@yahoo.com> wrote:

I think a vote here would be good. I think most of the discussion was done by 4 or 5 people and its a long thread. If nothing else it summarizes everything and gets people attention to the change.
Tom

On Thursday, March 9, 2017 10:55 AM, Sean Owen <so...@cloudera.com> wrote:

I think a VOTE is over-thinking it, and is rarely used, but, can't hurt. Nah, anyone can call a vote. This really isn't that formal. We just want to declare and document consensus.
I think SPIP is just a remix of existing process anyway, and don't think it will actually do much anyway, which is why I am sanguine about the whole thing.
To bring this to a conclusion, I will just put the contents of the doc in an email tomorrow for a VOTE. Raise any objections now.
On Thu, Mar 9, 2017 at 3:39 PM Cody Koeninger <co...@koeninger.org> wrote:

I started this idea as a fork with a merge-able change to docs.
Reynold moved it to his google doc, and has suggested during this
email thread that a vote should occur.
If a vote needs to occur, I can't see anything on
http://apache.org/foundation/voting.html suggesting that I can call
for a vote, which is why I'm asking PMC members to do it since they're
the ones who would vote anyway.
Now Sean is saying this is a code/doc change that can just be reviewed
and merged as usual...which is what I tried to do to begin with.

The fact that you haven't agreed on a process to agree on your process
is, I think, an indication that the process really does need
improvement ;)

Re: Spark Improvement Proposals

Posted by Sean Owen <so...@cloudera.com>.

This ended up proceeding as a normal doc change, instead of precipitating a
meta-vote.
However, the text that's on the web site now can certainly be further
amended if anyone wants to propose a change from here.

On Mon, Mar 13, 2017 at 1:50 PM Tom Graves <tg...@yahoo.com> wrote:

> I think a vote here would be good. I think most of the discussion was done
> by 4 or 5 people and its a long thread.  If nothing else it summarizes
> everything and gets people attention to the change.
>
> Tom
>
>
> On Thursday, March 9, 2017 10:55 AM, Sean Owen <so...@cloudera.com> wrote:
>
>
> I think a VOTE is over-thinking it, and is rarely used, but, can't hurt.
> Nah, anyone can call a vote. This really isn't that formal. We just want to
> declare and document consensus.
>
> I think SPIP is just a remix of existing process anyway, and don't think
> it will actually do much anyway, which is why I am sanguine about the whole
> thing.
>
> To bring this to a conclusion, I will just put the contents of the doc in
> an email tomorrow for a VOTE. Raise any objections now.
>
> On Thu, Mar 9, 2017 at 3:39 PM Cody Koeninger <co...@koeninger.org> wrote:
>
> I started this idea as a fork with a merge-able change to docs.
> Reynold moved it to his google doc, and has suggested during this
> email thread that a vote should occur.
> If a vote needs to occur, I can't see anything on
> http://apache.org/foundation/voting.html suggesting that I can call
> for a vote, which is why I'm asking PMC members to do it since they're
> the ones who would vote anyway.
> Now Sean is saying this is a code/doc change that can just be reviewed
> and merged as usual...which is what I tried to do to begin with.
>
> The fact that you haven't agreed on a process to agree on your process
> is, I think, an indication that the process really does need
> improvement ;)
>
>
>
>

Re: Spark Improvement Proposals

Posted by Tom Graves <tg...@yahoo.com.INVALID>.

On Thursday, March 9, 2017 10:55 AM, Sean Owen <so...@cloudera.com> wrote:

The fact that you haven't agreed on a process to agree on your process
is, I think, an indication that the process really does need
improvement ;)

Re: Spark Improvement Proposals

Posted by Cody Koeninger <co...@koeninger.org>.

Can someone with filter share permissions can make a filter for open
SPIP and one for closed SPIP and share it?

e.g.

project = SPARK AND status in (Open, Reopened, "In Progress") AND
labels=SPIP ORDER BY createdDate DESC

and another with the status closed equivalent

I just made an open ticket with the SPIP label show it should show up

On Fri, Mar 10, 2017 at 11:19 AM, Reynold Xin <rx...@databricks.com> wrote:
> We can just start using spip label and link to it.
>
>
>
> On Fri, Mar 10, 2017 at 9:18 AM, Cody Koeninger <co...@koeninger.org> wrote:
>>
>> So to be clear, if I translate that google doc to markup and submit a
>> PR, you will merge it?
>>
>> If we're just using "spip" label, that's probably fine, but we still
>> need shared filters for open and closed SPIPs so the page can link to
>> them.
>>
>> I do not believe I have jira permissions to share filters, I just
>> attempted to edit one of mine and do not see an add shares field.
>>
>> On Fri, Mar 10, 2017 at 10:54 AM, Sean Owen <so...@cloudera.com> wrote:
>> > Sure, that seems OK to me. I can merge anything like that.
>> > I think anyone can make a new label in JIRA; I don't know if even the
>> > admins
>> > can make a new issue type unfortunately. We may just have to mention a
>> > convention involving title and label or something.
>> >
>> > On Fri, Mar 10, 2017 at 4:52 PM Cody Koeninger <co...@koeninger.org>
>> > wrote:
>> >>
>> >> I think it ought to be its own page, linked from the more / community
>> >> menu dropdowns.
>> >>
>> >> We also need the jira tag, and for the page to clearly link to filters
>> >> that show proposed / completed SPIPs
>> >>
>> >> On Fri, Mar 10, 2017 at 3:39 AM, Sean Owen <so...@cloudera.com> wrote:
>> >> > Alrighty, if nobody is objecting, and nobody calls for a VOTE, then,
>> >> > let's
>> >> > say this document is the SPIP 1.0 process.
>> >> >
>> >> > I think the next step is just to translate the text to some suitable
>> >> > location. I suggest adding it to
>> >> > https://github.com/apache/spark-website/blob/asf-site/contributing.md
>> >> >
>> >> > On Thu, Mar 9, 2017 at 4:55 PM Sean Owen <so...@cloudera.com> wrote:
>> >> >>
>> >> >> I think a VOTE is over-thinking it, and is rarely used, but, can't
>> >> >> hurt.
>> >> >> Nah, anyone can call a vote. This really isn't that formal. We just
>> >> >> want to
>> >> >> declare and document consensus.
>> >> >>
>> >> >> I think SPIP is just a remix of existing process anyway, and don't
>> >> >> think
>> >> >> it will actually do much anyway, which is why I am sanguine about
>> >> >> the
>> >> >> whole
>> >> >> thing.
>> >> >>
>> >> >> To bring this to a conclusion, I will just put the contents of the
>> >> >> doc
>> >> >> in
>> >> >> an email tomorrow for a VOTE. Raise any objections now.
>>
>> ---------------------------------------------------------------------
>> To unsubscribe e-mail: dev-unsubscribe@spark.apache.org
>>
>

---------------------------------------------------------------------
To unsubscribe e-mail: dev-unsubscribe@spark.apache.org

Re: Spark Improvement Proposals

Posted by Reynold Xin <rx...@databricks.com>.

We can just start using spip label and link to it.



On Fri, Mar 10, 2017 at 9:18 AM, Cody Koeninger <co...@koeninger.org> wrote:

> So to be clear, if I translate that google doc to markup and submit a
> PR, you will merge it?
>
> If we're just using "spip" label, that's probably fine, but we still
> need shared filters for open and closed SPIPs so the page can link to
> them.
>
> I do not believe I have jira permissions to share filters, I just
> attempted to edit one of mine and do not see an add shares field.
>
> On Fri, Mar 10, 2017 at 10:54 AM, Sean Owen <so...@cloudera.com> wrote:
> > Sure, that seems OK to me. I can merge anything like that.
> > I think anyone can make a new label in JIRA; I don't know if even the
> admins
> > can make a new issue type unfortunately. We may just have to mention a
> > convention involving title and label or something.
> >
> > On Fri, Mar 10, 2017 at 4:52 PM Cody Koeninger <co...@koeninger.org>
> wrote:
> >>
> >> I think it ought to be its own page, linked from the more / community
> >> menu dropdowns.
> >>
> >> We also need the jira tag, and for the page to clearly link to filters
> >> that show proposed / completed SPIPs
> >>
> >> On Fri, Mar 10, 2017 at 3:39 AM, Sean Owen <so...@cloudera.com> wrote:
> >> > Alrighty, if nobody is objecting, and nobody calls for a VOTE, then,
> >> > let's
> >> > say this document is the SPIP 1.0 process.
> >> >
> >> > I think the next step is just to translate the text to some suitable
> >> > location. I suggest adding it to
> >> > https://github.com/apache/spark-website/blob/asf-site/contributing.md
> >> >
> >> > On Thu, Mar 9, 2017 at 4:55 PM Sean Owen <so...@cloudera.com> wrote:
> >> >>
> >> >> I think a VOTE is over-thinking it, and is rarely used, but, can't
> >> >> hurt.
> >> >> Nah, anyone can call a vote. This really isn't that formal. We just
> >> >> want to
> >> >> declare and document consensus.
> >> >>
> >> >> I think SPIP is just a remix of existing process anyway, and don't
> >> >> think
> >> >> it will actually do much anyway, which is why I am sanguine about the
> >> >> whole
> >> >> thing.
> >> >>
> >> >> To bring this to a conclusion, I will just put the contents of the
> doc
> >> >> in
> >> >> an email tomorrow for a VOTE. Raise any objections now.
>
> ---------------------------------------------------------------------
> To unsubscribe e-mail: dev-unsubscribe@spark.apache.org
>
>

Re: Spark Improvement Proposals

Posted by Cody Koeninger <co...@koeninger.org>.

So to be clear, if I translate that google doc to markup and submit a
PR, you will merge it?

If we're just using "spip" label, that's probably fine, but we still
need shared filters for open and closed SPIPs so the page can link to
them.

I do not believe I have jira permissions to share filters, I just
attempted to edit one of mine and do not see an add shares field.

On Fri, Mar 10, 2017 at 10:54 AM, Sean Owen <so...@cloudera.com> wrote:
> Sure, that seems OK to me. I can merge anything like that.
> I think anyone can make a new label in JIRA; I don't know if even the admins
> can make a new issue type unfortunately. We may just have to mention a
> convention involving title and label or something.
>
> On Fri, Mar 10, 2017 at 4:52 PM Cody Koeninger <co...@koeninger.org> wrote:
>>
>> I think it ought to be its own page, linked from the more / community
>> menu dropdowns.
>>
>> We also need the jira tag, and for the page to clearly link to filters
>> that show proposed / completed SPIPs
>>
>> On Fri, Mar 10, 2017 at 3:39 AM, Sean Owen <so...@cloudera.com> wrote:
>> > Alrighty, if nobody is objecting, and nobody calls for a VOTE, then,
>> > let's
>> > say this document is the SPIP 1.0 process.
>> >
>> > I think the next step is just to translate the text to some suitable
>> > location. I suggest adding it to
>> > https://github.com/apache/spark-website/blob/asf-site/contributing.md
>> >
>> > On Thu, Mar 9, 2017 at 4:55 PM Sean Owen <so...@cloudera.com> wrote:
>> >>
>> >> I think a VOTE is over-thinking it, and is rarely used, but, can't
>> >> hurt.
>> >> Nah, anyone can call a vote. This really isn't that formal. We just
>> >> want to
>> >> declare and document consensus.
>> >>
>> >> I think SPIP is just a remix of existing process anyway, and don't
>> >> think
>> >> it will actually do much anyway, which is why I am sanguine about the
>> >> whole
>> >> thing.
>> >>
>> >> To bring this to a conclusion, I will just put the contents of the doc
>> >> in
>> >> an email tomorrow for a VOTE. Raise any objections now.

---------------------------------------------------------------------
To unsubscribe e-mail: dev-unsubscribe@spark.apache.org

Re: Spark Improvement Proposals

Posted by Cody Koeninger <co...@koeninger.org>.

I think it ought to be its own page, linked from the more / community
menu dropdowns.

We also need the jira tag, and for the page to clearly link to filters
that show proposed / completed SPIPs

On Fri, Mar 10, 2017 at 3:39 AM, Sean Owen <so...@cloudera.com> wrote:
> Alrighty, if nobody is objecting, and nobody calls for a VOTE, then, let's
> say this document is the SPIP 1.0 process.
>
> I think the next step is just to translate the text to some suitable
> location. I suggest adding it to
> https://github.com/apache/spark-website/blob/asf-site/contributing.md
>
> On Thu, Mar 9, 2017 at 4:55 PM Sean Owen <so...@cloudera.com> wrote:
>>
>> I think a VOTE is over-thinking it, and is rarely used, but, can't hurt.
>> Nah, anyone can call a vote. This really isn't that formal. We just want to
>> declare and document consensus.
>>
>> I think SPIP is just a remix of existing process anyway, and don't think
>> it will actually do much anyway, which is why I am sanguine about the whole
>> thing.
>>
>> To bring this to a conclusion, I will just put the contents of the doc in
>> an email tomorrow for a VOTE. Raise any objections now.

---------------------------------------------------------------------
To unsubscribe e-mail: dev-unsubscribe@spark.apache.org

Re: Spark Improvement Proposals

Posted by Sean Owen <so...@cloudera.com>.

Alrighty, if nobody is objecting, and nobody calls for a VOTE, then, let's
say this document is the SPIP 1.0 process.

I think the next step is just to translate the text to some suitable
location. I suggest adding it to
https://github.com/apache/spark-website/blob/asf-site/contributing.md

On Thu, Mar 9, 2017 at 4:55 PM Sean Owen <so...@cloudera.com> wrote:

> I think a VOTE is over-thinking it, and is rarely used, but, can't hurt.
> Nah, anyone can call a vote. This really isn't that formal. We just want to
> declare and document consensus.
>
> I think SPIP is just a remix of existing process anyway, and don't think
> it will actually do much anyway, which is why I am sanguine about the whole
> thing.
>
> To bring this to a conclusion, I will just put the contents of the doc in
> an email tomorrow for a VOTE. Raise any objections now.
>

Re: Spark Improvement Proposals

Posted by Sean Owen <so...@cloudera.com>.

I think a VOTE is over-thinking it, and is rarely used, but, can't hurt.
Nah, anyone can call a vote. This really isn't that formal. We just want to
declare and document consensus.

I think SPIP is just a remix of existing process anyway, and don't think it
will actually do much anyway, which is why I am sanguine about the whole
thing.

To bring this to a conclusion, I will just put the contents of the doc in
an email tomorrow for a VOTE. Raise any objections now.

On Thu, Mar 9, 2017 at 3:39 PM Cody Koeninger <co...@koeninger.org> wrote:

> I started this idea as a fork with a merge-able change to docs.
> Reynold moved it to his google doc, and has suggested during this
> email thread that a vote should occur.
> If a vote needs to occur, I can't see anything on
> http://apache.org/foundation/voting.html suggesting that I can call
> for a vote, which is why I'm asking PMC members to do it since they're
> the ones who would vote anyway.
> Now Sean is saying this is a code/doc change that can just be reviewed
> and merged as usual...which is what I tried to do to begin with.
>
> The fact that you haven't agreed on a process to agree on your process
> is, I think, an indication that the process really does need
> improvement ;)
>
>

Re: Spark Improvement Proposals

Posted by Cody Koeninger <co...@koeninger.org>.

I started this idea as a fork with a merge-able change to docs.
Reynold moved it to his google doc, and has suggested during this
email thread that a vote should occur.
If a vote needs to occur, I can't see anything on
http://apache.org/foundation/voting.html suggesting that I can call
for a vote, which is why I'm asking PMC members to do it since they're
the ones who would vote anyway.
Now Sean is saying this is a code/doc change that can just be reviewed
and merged as usual...which is what I tried to do to begin with.

The fact that you haven't agreed on a process to agree on your process
is, I think, an indication that the process really does need
improvement ;)

On Tue, Mar 7, 2017 at 11:05 AM, Sean Owen <so...@cloudera.com> wrote:
> Do we need a VOTE? heck I think anyone can call one, anyway.
>
> Pre-flight vote check: anyone have objections to the text as-is?
> See
> https://docs.google.com/document/d/1-Zdi_W-wtuxS9hTK0P9qb2x-nRanvXmnZ7SUi4qMljg/edit#
>
> If so let's hash out specific suggest changes.
>
> If not, then I think the next step is to probably update the
> github.com/apache/spark-website repo with the text here. That's a code/doc
> change we can just review and merge as usual.
>
> On Tue, Mar 7, 2017 at 3:15 PM Cody Koeninger <co...@koeninger.org> wrote:
>>
>> Another week, another ping.  Anyone on the PMC willing to call a vote on
>> this?

---------------------------------------------------------------------
To unsubscribe e-mail: dev-unsubscribe@spark.apache.org

Re: Spark Improvement Proposals

Posted by Sean Owen <so...@cloudera.com>.

Do we need a VOTE? heck I think anyone can call one, anyway.

Pre-flight vote check: anyone have objections to the text as-is?
See
https://docs.google.com/document/d/1-Zdi_W-wtuxS9hTK0P9qb2x-nRanvXmnZ7SUi4qMljg/edit#

If so let's hash out specific suggest changes.

If not, then I think the next step is to probably update the
github.com/apache/spark-website repo with the text here. That's a code/doc
change we can just review and merge as usual.

On Tue, Mar 7, 2017 at 3:15 PM Cody Koeninger <co...@koeninger.org> wrote:

> Another week, another ping.  Anyone on the PMC willing to call a vote on
> this?
>