Re: [discuss] replacing SPIP template with Heilmeier's Catechism?

[Reynold Xin <rx...@databricks.com>]

I incorporated the feedbacks here and updated the SPIP page:
https://github.com/apache/spark-website/pull/156

The new version is live now:
https://spark.apache.org/improvement-proposals.html


On Fri, Aug 31, 2018 at 4:35 PM Ryan Blue <rb...@netflix.com> wrote:

> +1
>
> I think this is a great suggestion. I agree a bit with Sean, but I think
> it is really about mapping these questions into some of the existing
> structure. These are a great way to think about projects, but they're
> general and it would help to rephrase them for a software project, like
> Matei's comment on considering cost. Similarly, we might rephrase
> objectives to be goals/non-goals and add something to highlight that we
> expect absolutely no Jargon. A design sketch is needed to argue how long it
> will take, what is new, and why it would be successful; adding these
> questions will help people understand how to go from that design sketch to
> an argument for that design. I think these will guide people to write
> proposals that is persuasive and well-formed.
>
> rb
>
> On Fri, Aug 31, 2018 at 4:17 PM Jules Damji <dm...@comcast.net> wrote:
>
>> +1
>>
>> One could argue that the litany of the questions are really a
>> double-click on the essence: why, what, how. The three interrogatives ought
>> to be the essence and distillation of any proposal or technical exposition.
>>
>> Cheers
>> Jules
>>
>> Sent from my iPhone
>> Pardon the dumb thumb typos :)
>>
>> On Aug 31, 2018, at 11:23 AM, Reynold Xin <rx...@databricks.com> wrote:
>>
>> I helped craft the current SPIP template
>> <https://spark.apache.org/improvement-proposals.html> last year. I was
>> recently (re-)introduced to the Heilmeier Catechism, a set of questions
>> DARPA developed to evaluate proposals. The set of questions are:
>>
>> - What are you trying to do? Articulate your objectives using absolutely
>> no jargon.
>> - How is it done today, and what are the limits of current practice?
>> - What is new in your approach and why do you think it will be successful?
>> - Who cares? If you are successful, what difference will it make?
>> - What are the risks?
>> - How much will it cost?
>> - How long will it take?
>> - What are the mid-term and final “exams” to check for success?
>>
>> When I read the above list, it resonates really well because they are
>> almost always the same set of questions I ask myself and others before I
>> decide whether something is worth doing. In some ways, our SPIP template
>> tries to capture some of these (e.g. target persona), but are not as
>> explicit and well articulated.
>>
>> What do people think about replacing the current SPIP template with the
>> above?
>>
>> At a high level, I think the Heilmeier's Catechism emphasizes less about
>> the "how", and more the "why" and "what", which is what I'd argue SPIPs
>> should be about. The hows should be left in design docs for larger projects.
>>
>>
>>
>
> --
> Ryan Blue
> Software Engineer
> Netflix
>