Fwd: [jira] [Commented] (SPARK-6889) Streamline contribution process with update to Contribution wiki, JIRA rules

[Sean Owen <so...@cloudera.com>]

Bringing a discussion to dev@. I think the general questions on the table are:

- Should more changes be rejected? What are the pros/cons of that?
- If no, how do you think about the very large backlog of PRs and JIRAs?
- What should be rejected and why?
- How much support is there for proactively cleaning house now? What
would you close and why?
- What steps can be taken to prevent people from wasting time on JIRAs
/ PRs that will be rejected?
- What if anything does this tell us about the patterns of project
planning to date and what can we learn?

This overlaps with other discussion on SPARK-6889 but per Nicholas
wanted to surface this

---------- Forwarded message ----------
From: Nicholas Chammas (JIRA) <ji...@apache.org>
Date: Tue, Apr 14, 2015 at 3:38 PM
Subject: [jira] [Commented] (SPARK-6889) Streamline contribution
process with update to Contribution wiki, JIRA rules
To: issues@spark.apache.org


Nicholas Chammas commented on SPARK-6889:
-----------------------------------------

{quote}
I also agree that most projects don't say "no" enough and it's
actually bad for everyone. Yes, one goal was to also set more
expectation that lots of changes are rejected. If there is widespread
agreement, I'd also like firmer language in the guide. As you say it
is also a matter of taste and culture, but, I'd personally favor a lot
more "no".
{quote}

Regarding this point about culture, should we have some kind of
discussion on the dev list to nudge people in the right direction?

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@spark.apache.org
For additional commands, e-mail: dev-help@spark.apache.org

Re: [jira] [Commented] (SPARK-6889) Streamline contribution process with update to Contribution wiki, JIRA rules

[Imran Rashid <ir...@cloudera.com>]

These are great questions -- I dunno the answer to most of them, but I'll
try to at least give my take on "What should be rejected and why?"

For new features, I'm often really confused by our guidelines on what to
include and what to exclude.  Maybe we should ask that all new features
make it clear why they should *not* just be a separate package.

Bug fixes are also a little tricky.  On the one hand, its hard to say no to
them -- everyone wants all the bugs fixed.  But I think its actually a lot
harder for someone that isn't experienced with spark to fix a bug in a
clean way, when they don't know the code base.  Often the proposed fixes
are just kludges tacked on somewhere rather than addressing the real
problem.  It might help to clearly say that the most useful thing they can
do is submit bug reports with simple steps to reproduce, or even better to
submit a failing test case.  Of course submitting a patch is great too, but
we could be clear that patches would only be accepted if they fit in the
long-term design for spark.

I really feel that saying "no" more directly would be very helpful.
Actually I think one of the most discouraging things we can do is give a
"soft no" -- say "oh that sounds interesting", but then let the PR languish.

thanks for pushing on this Sean, really useful to have this discussion.

On Tue, Apr 14, 2015 at 10:02 AM, Sean Owen <so...@cloudera.com> wrote:

> Bringing a discussion to dev@. I think the general questions on the table
> are:
>
> - Should more changes be rejected? What are the pros/cons of that?
> - If no, how do you think about the very large backlog of PRs and JIRAs?
> - What should be rejected and why?
> - How much support is there for proactively cleaning house now? What
> would you close and why?
> - What steps can be taken to prevent people from wasting time on JIRAs
> / PRs that will be rejected?
> - What if anything does this tell us about the patterns of project
> planning to date and what can we learn?
>
> This overlaps with other discussion on SPARK-6889 but per Nicholas
> wanted to surface this
>
> ---------- Forwarded message ----------
> From: Nicholas Chammas (JIRA) <ji...@apache.org>
> Date: Tue, Apr 14, 2015 at 3:38 PM
> Subject: [jira] [Commented] (SPARK-6889) Streamline contribution
> process with update to Contribution wiki, JIRA rules
> To: issues@spark.apache.org
>
>
> Nicholas Chammas commented on SPARK-6889:
> -----------------------------------------
>
> {quote}
> I also agree that most projects don't say "no" enough and it's
> actually bad for everyone. Yes, one goal was to also set more
> expectation that lots of changes are rejected. If there is widespread
> agreement, I'd also like firmer language in the guide. As you say it
> is also a matter of taste and culture, but, I'd personally favor a lot
> more "no".
> {quote}
>
> Regarding this point about culture, should we have some kind of
> discussion on the dev list to nudge people in the right direction?
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@spark.apache.org
> For additional commands, e-mail: dev-help@spark.apache.org
>
>

Re: Fwd: [jira] [Commented] (SPARK-6889) Streamline contribution process with update to Contribution wiki, JIRA rules

[Nicholas Chammas <ni...@gmail.com>]

That's a great point, Burak. I think we are still figuring out when and how
to redirect people when their work doesn't fit the main Spark repo, and
Spark Packages should be a good destination in some cases.

Btw, my comment on JIRA (which Sean referenced) regarding rejecting more
patches is here
<https://issues.apache.org/jira/browse/SPARK-6889?focusedCommentId=14493394&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14493394>
.

Nick

On Tue, Apr 14, 2015 at 11:39 AM Burak Yavuz <br...@gmail.com> wrote:

> Hi Sean and fellow devs,
> I also wanted to chime in and remind people of <spark-packages.org>. Just
> because the work of someone doesn't fit into the broader scope of things,
> devs should be encouraged to showcase their hard work in Spark Packages.
> We have been working hard to make it easier for devs to share their work
> and users to access it in Spark Packages. There are some great datasource
> connectors, ml algorithms, streaming connectors in there already. I would
> urge devs and users to check it out!
>
> Best,
> Burak
> On Apr 14, 2015 8:06 AM, "Sean Owen" <so...@cloudera.com> wrote:
>
> > Bringing a discussion to dev@. I think the general questions on the
> table
> > are:
> >
> > - Should more changes be rejected? What are the pros/cons of that?
> > - If no, how do you think about the very large backlog of PRs and JIRAs?
> > - What should be rejected and why?
> > - How much support is there for proactively cleaning house now? What
> > would you close and why?
> > - What steps can be taken to prevent people from wasting time on JIRAs
> > / PRs that will be rejected?
> > - What if anything does this tell us about the patterns of project
> > planning to date and what can we learn?
> >
> > This overlaps with other discussion on SPARK-6889 but per Nicholas
> > wanted to surface this
> >
> > ---------- Forwarded message ----------
> > From: Nicholas Chammas (JIRA) <ji...@apache.org>
> > Date: Tue, Apr 14, 2015 at 3:38 PM
> > Subject: [jira] [Commented] (SPARK-6889) Streamline contribution
> > process with update to Contribution wiki, JIRA rules
> > To: issues@spark.apache.org
> >
> >
> > Nicholas Chammas commented on SPARK-6889:
> > -----------------------------------------
> >
> > {quote}
> > I also agree that most projects don't say "no" enough and it's
> > actually bad for everyone. Yes, one goal was to also set more
> > expectation that lots of changes are rejected. If there is widespread
> > agreement, I'd also like firmer language in the guide. As you say it
> > is also a matter of taste and culture, but, I'd personally favor a lot
> > more "no".
> > {quote}
> >
> > Regarding this point about culture, should we have some kind of
> > discussion on the dev list to nudge people in the right direction?
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: dev-unsubscribe@spark.apache.org
> > For additional commands, e-mail: dev-help@spark.apache.org
> >
> >
>

Re: Fwd: [jira] [Commented] (SPARK-6889) Streamline contribution process with update to Contribution wiki, JIRA rules

[Burak Yavuz <br...@gmail.com>]

Hi Sean and fellow devs,
I also wanted to chime in and remind people of <spark-packages.org>. Just
because the work of someone doesn't fit into the broader scope of things,
devs should be encouraged to showcase their hard work in Spark Packages.
We have been working hard to make it easier for devs to share their work
and users to access it in Spark Packages. There are some great datasource
connectors, ml algorithms, streaming connectors in there already. I would
urge devs and users to check it out!

Best,
Burak
On Apr 14, 2015 8:06 AM, "Sean Owen" <so...@cloudera.com> wrote:

> Bringing a discussion to dev@. I think the general questions on the table
> are:
>
> - Should more changes be rejected? What are the pros/cons of that?
> - If no, how do you think about the very large backlog of PRs and JIRAs?
> - What should be rejected and why?
> - How much support is there for proactively cleaning house now? What
> would you close and why?
> - What steps can be taken to prevent people from wasting time on JIRAs
> / PRs that will be rejected?
> - What if anything does this tell us about the patterns of project
> planning to date and what can we learn?
>
> This overlaps with other discussion on SPARK-6889 but per Nicholas
> wanted to surface this
>
> ---------- Forwarded message ----------
> From: Nicholas Chammas (JIRA) <ji...@apache.org>
> Date: Tue, Apr 14, 2015 at 3:38 PM
> Subject: [jira] [Commented] (SPARK-6889) Streamline contribution
> process with update to Contribution wiki, JIRA rules
> To: issues@spark.apache.org
>
>
> Nicholas Chammas commented on SPARK-6889:
> -----------------------------------------
>
> {quote}
> I also agree that most projects don't say "no" enough and it's
> actually bad for everyone. Yes, one goal was to also set more
> expectation that lots of changes are rejected. If there is widespread
> agreement, I'd also like firmer language in the guide. As you say it
> is also a matter of taste and culture, but, I'd personally favor a lot
> more "no".
> {quote}
>
> Regarding this point about culture, should we have some kind of
> discussion on the dev list to nudge people in the right direction?
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@spark.apache.org
> For additional commands, e-mail: dev-help@spark.apache.org
>
>