Re: [DISCUSS] Change PIP template

[Asaf Mesika <as...@gmail.com>]

Great news: the PR containing the new template of the PIP has been merged.

All voters - please check to see the template is used.

The most important bit in the template is its header - please mind that
when reviewing PIPs. I'm quoting it here:

RULES
> * Never place a link to an external site like Google Doc. The proposal
> should be in this issue entirely.
> * Use a spelling and grammar checker tools if available for you (there are
> plenty of free ones)


PROPOSAL HEALTH CHECK
> I can read the design document and understand the problem statement and
> what you plan to change *without* resorting to a couple of hours of code
> reading just to start having a high level understanding of the change.


Thanks!

On Wed, Mar 29, 2023 at 5:14 PM Asaf Mesika <as...@gmail.com> wrote:

> Bo, I need a review of the PR
> <https://github.com/apache/pulsar/pull/19832> :)
>
> On Wed, Mar 29, 2023 at 4:35 PM 丛搏 <bo...@apache.org> wrote:
>
>> +1
>>
>> Good discussion!
>>
>> Thanks,
>> Bo
>>
>> Asaf Mesika <as...@gmail.com> 于2023年3月29日周三 20:11写道：
>> >
>> > So far only 1 PMC member reviewed it.
>> > Any other PMC member would like to review the new template for PIP?
>> >
>> > On Wed, Mar 22, 2023 at 1:10 PM Asaf Mesika <as...@gmail.com>
>> wrote:
>> >
>> > > Any other PMC member can take a look at the new template PR
>> > > <https://github.com/apache/pulsar/pull/19832>?
>> > > Ideally I would like to have 2-3 PMC member approval for this.
>> > >
>> > >
>> > > On 17 Mar 2023, at 18:23, Michael Marshall <mm...@apache.org>
>> wrote:
>> > >
>> > > Thanks for this initiative, Asaf.
>> > >
>> > > As part of this process, I would like for us to add a security and a
>> > > multi-tenancy section to the PIP template.
>> > >
>> > > As you suggest, the template conveys what the community values, and
>> > > these two sections must always be considered when changing Pulsar in
>> > > fundamental ways.
>> > >
>> > > (Thanks for already adding the security section to your template!)
>> > >
>> > > Thanks,
>> > > Michael
>> > >
>> > > On Thu, Mar 16, 2023 at 2:58 AM Asaf Mesika <as...@gmail.com>
>> wrote:
>> > >
>> > >
>> > > Here's the PR to remove the form and add a new issue template in
>> Markdown
>> > > containing the suggested structure and description for each section.
>> > >
>> > > https://github.com/apache/pulsar/pull/19832
>> > >
>> > >
>> > > On Wed, Mar 1, 2023 at 3:43 PM Elliot West
>> > > <el...@streamnative.io.invalid> wrote:
>> > >
>> > > +1 Asaf
>> > >
>> > > I'd also suggest that we encourage the submission of relevant
>> diagrams.
>> > > This is trivial to do with the GitHub markdown editor, but I suspect
>> is
>> > > often neglected because users do not know the feature exists.
>> > >
>> > > On Wed, 1 Mar 2023 at 13:22, Asaf Mesika <as...@gmail.com>
>> wrote:
>> > >
>> > > Ok.
>> > >
>> > > I'll draft a PR and link it here when I'm done. Thanks!
>> > >
>> > > On Tue, Feb 28, 2023 at 7:08 AM PengHui Li <pe...@apache.org>
>> wrote:
>> > >
>> > > +1
>> > >
>> > > Penghui
>> > >
>> > > On Mon, Feb 27, 2023 at 9:24 PM Asaf Mesika <as...@gmail.com>
>> > >
>> > > wrote:
>> > >
>> > >
>> > > Mails don't support things like markdown diagrams or images and are
>> > > generally less easy to read.
>> > > My proposal includes a required section called Links in which you
>> > >
>> > > need
>> > >
>> > > to
>> > >
>> > > fill in the discussion thread in DEV mailing list and vote thread.
>> > >
>> > >
>> > > On Mon, Feb 27, 2023 at 3:08 PM Girish Sharma <
>> > >
>> > > scrapmachines@gmail.com
>> > >
>> > >
>> > > wrote:
>> > >
>> > > Hi Asaf,
>> > > I was referring to the PIP process, as a whole, as explained in
>> > > https://github.com/apache/pulsar/blob/master/wiki/proposals/PIP.md
>> > > Someone looking at GitHub ticket would find and almost empty PIP GH
>> > >
>> > > issue
>> > >
>> > > while the same PIP has had many discussions over here in the ML.
>> > > There is scope of improvement in the process where we either remove
>> > >
>> > > the
>> > >
>> > > first step to create the PIP over at GitHub and directly present
>> > >
>> > > the
>> > >
>> > > PIP
>> > >
>> > > in
>> > >
>> > > the first mail of the thread here, or we do all discussions in GH.
>> > > Both the ML and GH are searchable and linkable for tracking
>> > >
>> > > purposes.
>> > >
>> > >
>> > > Regards
>> > >
>> > > On Mon, Feb 27, 2023 at 6:23 PM Asaf Mesika <asaf.mesika@gmail.com
>> > >
>> > >
>> > > wrote:
>> > >
>> > >
>> > > On Sun, Feb 26, 2023 at 2:49 PM Girish Sharma <
>> > >
>> > > scrapmachines@gmail.com
>> > >
>> > >
>> > > wrote:
>> > >
>> > > Good proposal Asaf.
>> > > I've also wondered why the PIP creation and discussion process
>> > >
>> > > is
>> > >
>> > > so
>> > >
>> > > separated. The PIP discussion and voting starts off as a GitHub
>> > >
>> > > issue,
>> > >
>> > > but
>> > >
>> > > all of its discussion happens here on the mailing list. Is
>> > >
>> > > there
>> > >
>> > > scope
>> > >
>> > > of
>> > >
>> > > improvement in that process as well?
>> > >
>> > >
>> > > Not sure I follow. Can you outline the problem exactly?
>> > >
>> > >
>> > >
>> > > Regards
>> > >
>> > > On Sun, Feb 26, 2023 at 6:16 PM tison <ti...@apache.org>
>> > >
>> > > wrote:
>> > >
>> > >
>> > > Hi Asaf,
>> > >
>> > > I agree that, generally, a PIP is written as a whole and
>> > >
>> > > paste
>> > >
>> > > as
>> > >
>> > > the
>> > >
>> > > body.
>> > >
>> > > So +1 for your proposal.
>> > >
>> > > Additionally, I'm thinking of moving the doc of procedure
>> > >
>> > > (wiki/PIP.md)
>> > >
>> > > to
>> > >
>> > > the contributions guide and use the new markdown template to
>> > >
>> > > supersede
>> > >
>> > > the
>> > >
>> > > wiki/PIP-template.md. Then we don't need to hold the wiki
>> > >
>> > > folder.
>> > >
>> > >
>> > > It can be an extended version to your proposal, so let's keep
>> > >
>> > > on
>> > >
>> > > your
>> > >
>> > > proposal in this thread. Just for your reference.
>> > >
>> > > Best,
>> > > tison.
>> > >
>> > >
>> > > Asaf Mesika <as...@gmail.com> 于2023年2月26日周日 19:18写道：
>> > >
>> > > Hi,
>> > >
>> > > I would like to suggest two changes I'd like to make to the
>> > >
>> > > PIP
>> > >
>> > > design
>> > >
>> > > template:
>> > > 1. Remove the form - just have a markdown template fill the
>> > >
>> > > issue
>> > >
>> > > body
>> > >
>> > > as
>> > >
>> > > it is created.
>> > > 2. Change the PIP template structure
>> > >
>> > > == Removing the form
>> > >
>> > > Today, when you want to submit a PIP, you are required to
>> > >
>> > > fill
>> > >
>> > > out
>> > >
>> > > a
>> > >
>> > > form
>> > >
>> > > with boxes composed of 3-4 lines length.
>> > > It's not good because:
>> > > * It broadcasts to the author: we want a very small PIP,
>> > >
>> > > something
>> > >
>> > > that
>> > >
>> > > fits those small boxes.
>> > > * It makes the PIP look like a bug, where you fill out
>> > >
>> > > fields.
>> > >
>> > > * It doesn't allow having H2 headings, only H1 headings,
>> > >
>> > > thus
>> > >
>> > > limiting
>> > >
>> > > the
>> > >
>> > > structure.
>> > >
>> > > A PIP is a design essentially, something 1-3 pages long.
>> > >
>> > > Thus,
>> > >
>> > > people take the time to write it down. Preferably, they
>> > >
>> > > copy
>> > >
>> > > paste
>> > >
>> > > the
>> > >
>> > > body
>> > >
>> > > of the PIP issue, and use it to fill in sections.
>> > >
>> > > My suggestion is to define an issue template using only
>> > >
>> > > markdown,
>> > >
>> > > without a
>> > >
>> > > form.
>> > >
>> > > == Changing PIP Structure
>> > >
>> > > Today the structure of the PIP doc (pasted below), is
>> > >
>> > > missing a
>> > >
>> > > section
>> > >
>> > > and
>> > >
>> > > generally aims to jump directly into API changes / code /
>> > >
>> > > implementation.
>> > >
>> > > This results in lots of back and forth emails in an attempt
>> > >
>> > > to
>> > >
>> > > get
>> > >
>> > > the
>> > >
>> > > following essentials:
>> > > * All required background knowledge to understand the
>> > >
>> > > proposal
>> > >
>> > > * A high level overview of the proposed solution
>> > > * Understanding how this proposal will be monitored
>> > > * What steps exactly I need to take if I revert to the
>> > >
>> > > previous
>> > >
>> > > version.
>> > >
>> > >
>> > > The structure I propose below aims to reduce that friction
>> > >
>> > > and
>> > >
>> > > get
>> > >
>> > > all
>> > >
>> > > PIP
>> > >
>> > > aligned to provide that information.
>> > >
>> > > === Today's structure
>> > >
>> > > # Motivation
>> > > * "Explain why this change is needed, what benefits it
>> > >
>> > > would
>> > >
>> > > bring
>> > >
>> > > to
>> > >
>> > > Apache Pulsar and what problem it's trying to solve."
>> > > # Goal
>> > > * "Define the scope of this proposal. Given the motivation
>> > >
>> > > stated
>> > >
>> > > above,
>> > >
>> > > what are the problems that this proposal is addressing and
>> > >
>> > > what
>> > >
>> > > other
>> > >
>> > > items
>> > >
>> > > will be considering out of scope, perhaps to be left to a
>> > >
>> > > different
>> > >
>> > > PIP."
>> > >
>> > > # API Changes
>> > > * "Illustrate all the proposed changes to the API or wire
>> > >
>> > > protocol,
>> > >
>> > > with
>> > >
>> > > examples of all the newly added classes/methods, including
>> > >
>> > > Javadoc"
>> > >
>> > > # Implementation
>> > > * "This should be a detailed description of all the changes
>> > >
>> > > that
>> > >
>> > > are
>> > >
>> > > expected to be made. It should be detailed enough that any
>> > >
>> > > developer
>> > >
>> > > that
>> > >
>> > > is familiar with Pulsar internals would be able to
>> > >
>> > > understand
>> > >
>> > > all
>> > >
>> > > the
>> > >
>> > > parts
>> > >
>> > > of the code changes for this proposal."
>> > > * "This should also serve as documentation for any person
>> > >
>> > > that
>> > >
>> > > is
>> > >
>> > > trying
>> > >
>> > > to
>> > >
>> > > understand or debug the behavior of a certain feature."
>> > > # Alernatives
>> > > * "If there are alternatives that were already considered
>> > >
>> > > by
>> > >
>> > > the
>> > >
>> > > authors
>> > >
>> > > or, after the discussion, by the community, and were
>> > >
>> > > rejected,
>> > >
>> > > please
>> > >
>> > > list
>> > >
>> > > them here along with the reason why they were rejected"
>> > > # Anything else?
>> > >
>> > >
>> > > === My suggestion
>> > >
>> > > # Motivation and Background information
>> > > * Give a high level explanation on all concepts you will be
>> > >
>> > > using
>> > >
>> > > throughout this document. For example, if you want to talk
>> > >
>> > > about
>> > >
>> > > Persistent
>> > >
>> > > Subscriptions, explain briefly (1 paragraph) what this is.
>> > >
>> > > If
>> > >
>> > > you're
>> > >
>> > > going
>> > >
>> > > to talk about Transaction Buffer, explain briefly what this
>> > >
>> > > is.
>> > >
>> > > If
>> > >
>> > > you're
>> > >
>> > > going to change something specific, that goes into a bit
>> > >
>> > > more
>> > >
>> > > detail
>> > >
>> > > about
>> > >
>> > > it and how it works. The Litmus test: I can read the design
>> > >
>> > > document
>> > >
>> > > and
>> > >
>> > > understand the problem statement and what you plan to
>> > >
>> > > change
>> > >
>> > > *without*
>> > >
>> > > resorting to a couple of hours of code reading just to
>> > >
>> > > start
>> > >
>> > > having a
>> > >
>> > > high
>> > >
>> > > level understanding of the change.
>> > > * Provide links where possible if a person wants to dig
>> > >
>> > > deeper
>> > >
>> > > into
>> > >
>> > > the
>> > >
>> > > background information.
>> > > * Explain what is the problem you're trying to solve -
>> > >
>> > > current
>> > >
>> > > situation.
>> > >
>> > > * This section is the "Why" of your proposal.
>> > >
>> > > # Goals
>> > > ## Scope
>> > > * Describe the goals of your proposal, and why it benefits
>> > >
>> > > Apache
>> > >
>> > > Pulsar
>> > >
>> > > ## Out of Scope
>> > > * Describe what you have decided to keep out of scope,
>> > >
>> > > perhaps
>> > >
>> > > left
>> > >
>> > > for a
>> > >
>> > > different PIP/s.
>> > >
>> > > # High-level Design
>> > > * Describe in high level, end-to-end, the solution. This
>> > >
>> > > should
>> > >
>> > > be
>> > >
>> > > a
>> > >
>> > > few
>> > >
>> > > paragraphs long as a guideline.
>> > > * Reading this would allow me to understand the solution
>> > >
>> > > from a
>> > >
>> > > bird's
>> > >
>> > > eye
>> > >
>> > > view, end to end.
>> > > * DON'T put all the design in a Google Doc and share the
>> > >
>> > > link
>> > >
>> > > here,
>> > >
>> > > as
>> > >
>> > > it
>> > >
>> > > won't last the test of time.
>> > >
>> > > # Detailed Design
>> > > * Describe in detail what you plan to do to achieve your
>> > >
>> > > high
>> > >
>> > > level
>> > >
>> > > design
>> > >
>> > > * It should include the following if applicable:
>> > >  * REST API Changes
>> > >  * Protocol Changes
>> > >
>> > > # Monitoring
>> > > * Describe exactly what you will add to Pulsar allowing you
>> > >
>> > > to
>> > >
>> > > monitor/observe this proposal?
>> > >  * If those are metrics, provide the names, description,
>> > >
>> > > labels
>> > >
>> > > and
>> > >
>> > > units
>> > >
>> > >  * Explain what constitutes abnormal that I should pay
>> > >
>> > > attention
>> > >
>> > > to
>> > >
>> > >
>> > > # Backward Compatibility
>> > > * Describe exact instructions if someone needs to revert
>> > >
>> > > from a
>> > >
>> > > version
>> > >
>> > > containing it to a previous version
>> > >
>> > > # Alternatives
>> > > * Describe alternative design decisions and why you have
>> > >
>> > > not
>> > >
>> > > opted
>> > >
>> > > for
>> > >
>> > > them
>> > >
>> > >
>> > > # General notes
>> > > * Any general notes you wish to make
>> > >
>> > > # Links (Updated afterwards)
>> > > * Mailing List discussion thread:
>> > > * Mailing List voting thread:
>> > >
>> > > ==
>> > > Would love to hear what you think about it, before opening
>> > >
>> > > a
>> > >
>> > > PR
>> > >
>> > > about
>> > >
>> > > this.
>> > >
>> > >
>> > >
>> > >
>> > >
>> > > --
>> > > Girish Sharma
>> > >
>> > >
>> > >
>> > >
>> > > --
>> > > Girish Sharma
>> > >
>> > >
>> > >
>> > >
>> > >
>> > >
>> > > --
>> > >
>> > > Elliot West
>> > >
>> > > Senior Platform Engineer
>> > >
>> > > elliot.west@streamnative.io
>> > >
>> > > streamnative.io
>> > >
>> > > <https://github.com/streamnative>
>> > > <https://www.linkedin.com/company/streamnative>
>> > > <https://twitter.com/streamnativeio>
>> > >
>> > >
>> > >
>>
>