You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@spark.apache.org by pzecevic <pe...@gmail.com> on 2015/01/17 12:24:50 UTC
Re: Discourse: A proposed alternative to the Spark User list
Hi, guys!
I'm reviving this old question from Nick Chammas with a new proposal: what
do you think about creating a separate Stack Exchange 'Apache Spark' site
(like 'philosophy' and 'English' etc.)?
I'm not sure what would be the best way to deal with user and dev lists,
though - to merge them into one or create two separate sites...
And I don't know it it's at all possible to migrate current lists to stack
exchange, but I believe it would be an improvement over the current
situation. People are used to stack exchange, it's easy to use and search,
topics (Spark SQL, Streaming, Graphx) could be marked with tags for easy
filtering, code formatting is super easy etc.
What do you all think?
Nick Chammas wrote
> When people have questions about Spark, there are 2 main places (as far as
> I can tell) where they ask them:
>
> - Stack Overflow, under the apache-spark tag
> <http://stackoverflow.com/questions/tagged/apache-spark>
> - This mailing list
>
> The mailing list is valuable as an independent place for discussion that
> is
> part of the Spark project itself. Furthermore, it allows for a broader
> range of discussions than would be allowed on Stack Overflow
> <http://stackoverflow.com/help/dont-ask>.
>
> As the Spark project has grown in popularity, I see that a few problems
> have emerged with this mailing list:
>
> - It’s hard to follow topics (e.g. Streaming vs. SQL) that you’re
> interested in, and it’s hard to know when someone has mentioned you
> specifically.
> - It’s hard to search for existing threads and link information across
> disparate threads.
> - It’s hard to format code and log snippets nicely, and by extension,
> hard to read other people’s posts with this kind of information.
>
> There are existing solutions to all these (and other) problems based
> around
> straight-up discipline or client-side tooling, which users have to conjure
> up for themselves.
>
> I’d like us as a community to consider using Discourse
> <http://www.discourse.org/> as an alternative to, or overlay on top
> of,
> this mailing list, that provides better out-of-the-box solutions to these
> problems.
>
> Discourse is a modern discussion platform built by some of the same people
> who created Stack Overflow. It has many neat features
> <http://v1.discourse.org/about/> that I believe this community would
> benefit from.
>
> For example:
>
> - When a user starts typing up a new post, they get a panel *showing
> existing conversations that look similar*, just like on Stack Overflow.
> - It’s easy to search for posts and link between them.
> - *Markdown support* is built-in to composer.
> - You can *specifically mention people* and they will be notified.
> - Posts can be categorized (e.g. Streaming, SQL, etc.).
> - There is a built-in option for mailing list support which forwards
> all
> activity on the forum to a user’s email address and which allows for
> creation of new posts via email.
>
> What do you think of Discourse as an alternative, more manageable way to
> discus Spark?
>
> There are a few options we can consider:
>
> 1. Work with the ASF as well as the Discourse team to allow Discourse
> to
> act as an overlay on top of this mailing list
>
> <https://meta.discourse.org/t/discourse-as-a-front-end-for-existing-asf-mailing-lists/23167?u=nicholaschammas>,
> allowing people to continue to use the mailing list as-is if they want.
> (This is the toughest but perhaps most attractive option.)
> 2. Create a new Discourse forum for Spark that is not bound to this
> user
> list. This is relatively easy but will effectively fork the community
> on
> this list. (We cannot shut down this mailing in favor of one managed by
> Discourse.)
> 3. Don’t use Discourse. Just encourage people on this list to post
> instead on Stack Overflow whenever possible.
> 4. Something else.
>
> What does everyone think?
>
> Nick
>
--
View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/Discourse-A-proposed-alternative-to-the-Spark-User-list-tp20851p21203.html
Sent from the Apache Spark User List mailing list archive at Nabble.com.
---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@spark.apache.org
For additional commands, e-mail: user-help@spark.apache.org
Re: Discourse: A proposed alternative to the Spark User list
Posted by Nicholas Chammas <ni...@gmail.com>.
I agree with Sean that a Spark-specific Stack Exchange likely won't help
and almost certainly won't make it out of Area 51. The idea certainly
sounds nice from our perspective as Spark users, but it doesn't mesh with
the structure of Stack Exchange or the criteria for creating new sites.
On Thu Jan 22 2015 at 1:23:14 PM Sean Owen <so...@cloudera.com> wrote:
> FWIW I am a moderator for datascience.stackexchange.com, and even that
> hasn't really achieved the critical mass that SE sites are supposed
> to: http://area51.stackexchange.com/proposals/55053/data-science
>
> I think a Spark site would have a lot less traffic. One annoyance is
> that people can't figure out when to post on SO vs Data Science vs
> Cross Validated. A Spark site would have the same problem,
> fragmentation and cross posting with SO. I don't think this would be
> accepted as a StackExchange site and don't think it helps.
>
> On Thu, Jan 22, 2015 at 6:16 PM, pierred <pi...@demartines.com> wrote:
> >
> > A dedicated stackexchange site for Apache Spark sounds to me like the
> > logical solution. Less trolling, more enthusiasm, and with the
> > participation of the people on this list, I think it would very quickly
> > become the reference for many technical questions, as well as a great
> > vehicle to promote the awesomeness of Spark.
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: user-unsubscribe@spark.apache.org
> For additional commands, e-mail: user-help@spark.apache.org
>
>
Re: Discourse: A proposed alternative to the Spark User list
Posted by Marcelo Vanzin <va...@cloudera.com>.
On Thu, Jan 22, 2015 at 10:21 AM, Sean Owen <so...@cloudera.com> wrote:
> I think a Spark site would have a lot less traffic. One annoyance is
> that people can't figure out when to post on SO vs Data Science vs
> Cross Validated.
Another is that a lot of the discussions we see on the Spark users
list would be closed very quickly at Stack Overflow. Long and abstract
discussions are generally a good recipe to get your question closed.
Which is an argument for why Discourse would be more appropriate, I
guess.
Finally, maybe I'm showing my age, but I really dislike having to
follow lots of different places. What would happen is that,
personally, I'd end up either ignoring any new discussion forum, or
just treating it like a mailing list and doing everything by e-mail.
Now get off my lawn.
--
Marcelo
---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@spark.apache.org
For additional commands, e-mail: user-help@spark.apache.org
Re: Discourse: A proposed alternative to the Spark User list
Posted by Sean Owen <so...@cloudera.com>.
FWIW I am a moderator for datascience.stackexchange.com, and even that
hasn't really achieved the critical mass that SE sites are supposed
to: http://area51.stackexchange.com/proposals/55053/data-science
I think a Spark site would have a lot less traffic. One annoyance is
that people can't figure out when to post on SO vs Data Science vs
Cross Validated. A Spark site would have the same problem,
fragmentation and cross posting with SO. I don't think this would be
accepted as a StackExchange site and don't think it helps.
On Thu, Jan 22, 2015 at 6:16 PM, pierred <pi...@demartines.com> wrote:
>
> A dedicated stackexchange site for Apache Spark sounds to me like the
> logical solution. Less trolling, more enthusiasm, and with the
> participation of the people on this list, I think it would very quickly
> become the reference for many technical questions, as well as a great
> vehicle to promote the awesomeness of Spark.
---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@spark.apache.org
For additional commands, e-mail: user-help@spark.apache.org
Re: Discourse: A proposed alternative to the Spark User list
Posted by pierred <pi...@demartines.com>.
Love it!
There is a reason why SO is so effective and popular. Search is excellent,
you can quickly find very thoughtful answers about sometimes thorny
problems, and it is easy to contribute, format code, etc. Perhaps the most
useful feature is that the best answers naturally bubble up to the top, so
these are the ones you see first.
One annoyance is the "troll" phenomenon, see e.g.
http://michael.richter.name/blogs/why-i-no-longer-contribute-to-stackoverflow
(that also mentions other pet peeves about SO). That phenomenon is, IMHO,
most prevalent on the stackoverflow itself, perhaps less so on other
stackexchange sites.
At the same time, I do appreciate the pressure to provide well-written,
concise, and "for the posterity" questions and answers. That peer pressure
is what, to a good extent, makes the material on SO so valuable and useful.
It is probably a tricky balance to strike.
A dedicated stackexchange site for Apache Spark sounds to me like the
logical solution. Less trolling, more enthusiasm, and with the
participation of the people on this list, I think it would very quickly
become the reference for many technical questions, as well as a great
vehicle to promote the awesomeness of Spark.
--
View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/Discourse-A-proposed-alternative-to-the-Spark-User-list-tp20851p21321.html
Sent from the Apache Spark User List mailing list archive at Nabble.com.
---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@spark.apache.org
For additional commands, e-mail: user-help@spark.apache.org
Re: Discourse: A proposed alternative to the Spark User list
Posted by Nicholas Chammas <ni...@gmail.com>.
The Stack Exchange community will not support creating a whole new site
just for Spark (otherwise you’d see dedicated sites for much larger topics
like “Python”). Their tagging system works well enough to separate
questions about different topics, and the apache-spark
<http://stackoverflow.com/questions/tagged/apache-spark> tag on Stack
Overflow is already doing pretty well.
The ASF as well as this community will also not support any migration of
the mailing list to another system due to ASF rules
<http://www.apache.org/foundation/how-it-works.html> and community
fragmentation.
Realistically, the only options available to us that I see are options 1
and 3 from my original email (which can be used together).
Option 3: Change the culture around the user list. Encourage people to use
Stack Overflow whenever possible, and this list only when their question
doesn’t fit SO’s strict rules.
Option 1: Work with the ASF and the Discourse teams to allow Discourse to
be deployed as an overlay on top of this existing mailing list. (e.g. Like
a new UI on top of an old database.)
The goal of both changes would be to make the user list more usable.
Nick
On 2015년 1월 17일 (토) at 오전 8:51 Andrew Ash <an...@andrewash.com> wrote:
People can continue using the stack exchange sites as is with no additional
> work from the Spark team. I would not support migrating our mailing lists
> yet again to another system like Discourse because I fear fragmentation of
> the community between the many sites.
>
> On Sat, Jan 17, 2015 at 6:24 AM, pzecevic <pe...@gmail.com> wrote:
>
>> Hi, guys!
>>
>> I'm reviving this old question from Nick Chammas with a new proposal: what
>> do you think about creating a separate Stack Exchange 'Apache Spark' site
>> (like 'philosophy' and 'English' etc.)?
>>
>> I'm not sure what would be the best way to deal with user and dev lists,
>> though - to merge them into one or create two separate sites...
>>
>> And I don't know it it's at all possible to migrate current lists to stack
>> exchange, but I believe it would be an improvement over the current
>> situation. People are used to stack exchange, it's easy to use and search,
>> topics (Spark SQL, Streaming, Graphx) could be marked with tags for easy
>> filtering, code formatting is super easy etc.
>>
>> What do you all think?
>>
>>
>>
>> Nick Chammas wrote
>> > When people have questions about Spark, there are 2 main places (as far
>> as
>> > I can tell) where they ask them:
>> >
>> > - Stack Overflow, under the apache-spark tag
>> > <http://stackoverflow.com/questions/tagged/apache-spark>
>> > - This mailing list
>> >
>> > The mailing list is valuable as an independent place for discussion that
>> > is
>> > part of the Spark project itself. Furthermore, it allows for a broader
>> > range of discussions than would be allowed on Stack Overflow
>> > <http://stackoverflow.com/help/dont-ask>.
>> >
>> > As the Spark project has grown in popularity, I see that a few problems
>> > have emerged with this mailing list:
>> >
>> > - It’s hard to follow topics (e.g. Streaming vs. SQL) that you’re
>> > interested in, and it’s hard to know when someone has mentioned you
>> > specifically.
>> > - It’s hard to search for existing threads and link information
>> across
>> > disparate threads.
>> > - It’s hard to format code and log snippets nicely, and by extension,
>> > hard to read other people’s posts with this kind of information.
>> >
>> > There are existing solutions to all these (and other) problems based
>> > around
>> > straight-up discipline or client-side tooling, which users have to
>> conjure
>> > up for themselves.
>> >
>> > I’d like us as a community to consider using Discourse
>> > <http://www.discourse.org/> as an alternative to, or overlay on
>> top
>> > of,
>> > this mailing list, that provides better out-of-the-box solutions to
>> these
>> > problems.
>> >
>> > Discourse is a modern discussion platform built by some of the same
>> people
>> > who created Stack Overflow. It has many neat features
>> > <http://v1.discourse.org/about/> that I believe this community
>> would
>> > benefit from.
>> >
>> > For example:
>> >
>> > - When a user starts typing up a new post, they get a panel *showing
>> > existing conversations that look similar*, just like on Stack
>> Overflow.
>> > - It’s easy to search for posts and link between them.
>> > - *Markdown support* is built-in to composer.
>> > - You can *specifically mention people* and they will be notified.
>> > - Posts can be categorized (e.g. Streaming, SQL, etc.).
>> > - There is a built-in option for mailing list support which forwards
>> > all
>> > activity on the forum to a user’s email address and which allows for
>> > creation of new posts via email.
>> >
>> > What do you think of Discourse as an alternative, more manageable way to
>> > discus Spark?
>> >
>> > There are a few options we can consider:
>> >
>> > 1. Work with the ASF as well as the Discourse team to allow Discourse
>> > to
>> > act as an overlay on top of this mailing list
>> >
>> > <https://meta.discourse.org/t/discourse-as-a-front-
>> end-for-existing-asf-mailing-lists/23167?u=nicholaschammas>,
>> > allowing people to continue to use the mailing list as-is if they
>> want.
>> > (This is the toughest but perhaps most attractive option.)
>> > 2. Create a new Discourse forum for Spark that is not bound to this
>> > user
>> > list. This is relatively easy but will effectively fork the community
>> > on
>> > this list. (We cannot shut down this mailing in favor of one managed
>> by
>> > Discourse.)
>> > 3. Don’t use Discourse. Just encourage people on this list to post
>> > instead on Stack Overflow whenever possible.
>> > 4. Something else.
>> >
>> > What does everyone think?
>> >
>> > Nick
>> >
>>
>>
>>
>>
>>
>> --
>> View this message in context: http://apache-spark-user-list.
>> 1001560.n3.nabble.com/Discourse-A-proposed-alternative-to-the-Spark-User-
>> list-tp20851p21203.html
>> Sent from the Apache Spark User List mailing list archive at Nabble.com.
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: user-unsubscribe@spark.apache.org
>> For additional commands, e-mail: user-help@spark.apache.org
>>
>>
>
Re: Discourse: A proposed alternative to the Spark User list
Posted by Andrew Ash <an...@andrewash.com>.
People can continue using the stack exchange sites as is with no additional
work from the Spark team. I would not support migrating our mailing lists
yet again to another system like Discourse because I fear fragmentation of
the community between the many sites.
On Sat, Jan 17, 2015 at 6:24 AM, pzecevic <pe...@gmail.com> wrote:
> Hi, guys!
>
> I'm reviving this old question from Nick Chammas with a new proposal: what
> do you think about creating a separate Stack Exchange 'Apache Spark' site
> (like 'philosophy' and 'English' etc.)?
>
> I'm not sure what would be the best way to deal with user and dev lists,
> though - to merge them into one or create two separate sites...
>
> And I don't know it it's at all possible to migrate current lists to stack
> exchange, but I believe it would be an improvement over the current
> situation. People are used to stack exchange, it's easy to use and search,
> topics (Spark SQL, Streaming, Graphx) could be marked with tags for easy
> filtering, code formatting is super easy etc.
>
> What do you all think?
>
>
>
> Nick Chammas wrote
> > When people have questions about Spark, there are 2 main places (as far
> as
> > I can tell) where they ask them:
> >
> > - Stack Overflow, under the apache-spark tag
> > <http://stackoverflow.com/questions/tagged/apache-spark>
> > - This mailing list
> >
> > The mailing list is valuable as an independent place for discussion that
> > is
> > part of the Spark project itself. Furthermore, it allows for a broader
> > range of discussions than would be allowed on Stack Overflow
> > <http://stackoverflow.com/help/dont-ask>.
> >
> > As the Spark project has grown in popularity, I see that a few problems
> > have emerged with this mailing list:
> >
> > - It’s hard to follow topics (e.g. Streaming vs. SQL) that you’re
> > interested in, and it’s hard to know when someone has mentioned you
> > specifically.
> > - It’s hard to search for existing threads and link information across
> > disparate threads.
> > - It’s hard to format code and log snippets nicely, and by extension,
> > hard to read other people’s posts with this kind of information.
> >
> > There are existing solutions to all these (and other) problems based
> > around
> > straight-up discipline or client-side tooling, which users have to
> conjure
> > up for themselves.
> >
> > I’d like us as a community to consider using Discourse
> > <http://www.discourse.org/> as an alternative to, or overlay on
> top
> > of,
> > this mailing list, that provides better out-of-the-box solutions to these
> > problems.
> >
> > Discourse is a modern discussion platform built by some of the same
> people
> > who created Stack Overflow. It has many neat features
> > <http://v1.discourse.org/about/> that I believe this community
> would
> > benefit from.
> >
> > For example:
> >
> > - When a user starts typing up a new post, they get a panel *showing
> > existing conversations that look similar*, just like on Stack
> Overflow.
> > - It’s easy to search for posts and link between them.
> > - *Markdown support* is built-in to composer.
> > - You can *specifically mention people* and they will be notified.
> > - Posts can be categorized (e.g. Streaming, SQL, etc.).
> > - There is a built-in option for mailing list support which forwards
> > all
> > activity on the forum to a user’s email address and which allows for
> > creation of new posts via email.
> >
> > What do you think of Discourse as an alternative, more manageable way to
> > discus Spark?
> >
> > There are a few options we can consider:
> >
> > 1. Work with the ASF as well as the Discourse team to allow Discourse
> > to
> > act as an overlay on top of this mailing list
> >
> > <
> https://meta.discourse.org/t/discourse-as-a-front-end-for-existing-asf-mailing-lists/23167?u=nicholaschammas>
> ;,
> > allowing people to continue to use the mailing list as-is if they
> want.
> > (This is the toughest but perhaps most attractive option.)
> > 2. Create a new Discourse forum for Spark that is not bound to this
> > user
> > list. This is relatively easy but will effectively fork the community
> > on
> > this list. (We cannot shut down this mailing in favor of one managed
> by
> > Discourse.)
> > 3. Don’t use Discourse. Just encourage people on this list to post
> > instead on Stack Overflow whenever possible.
> > 4. Something else.
> >
> > What does everyone think?
> >
> > Nick
> >
>
>
>
>
>
> --
> View this message in context:
> http://apache-spark-user-list.1001560.n3.nabble.com/Discourse-A-proposed-alternative-to-the-Spark-User-list-tp20851p21203.html
> Sent from the Apache Spark User List mailing list archive at Nabble.com.
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: user-unsubscribe@spark.apache.org
> For additional commands, e-mail: user-help@spark.apache.org
>
>