Re: [DISCUSS] Flink release retro

[Qingsheng Ren <re...@apache.org>]

Thanks for starting the discussion Matthias!

I think having a retro after a release cycle would be quite helpful to
standardizing the procedure of the release, and also could avoid new
release managers getting stuck on the same issue that happened before. I
prefer the second option that RMs could open a discussion thread in ML at
the end of the release to collect feedback about the last release cycle and
add them to the release wiki page, which would be quite handy for further
RMs.

Best,
Qingsheng
Ververica (Alibaba)

On Mon, Oct 31, 2022 at 11:02 PM Matthias Pohl
<ma...@aiven.io.invalid> wrote:

> Hi everyone,
> I want to bring up the idea of having a retrospective on the release from
> the release manager's perspective. The idea would be to collect feedback on
> what went well and what could be improved for a specific minor release. So
> far, I didn't find anything on that topic. Does the community find this
> useful? Or was this already done but not helpful?
>
> I see three options here:
> 1. Having an actual meeting where issues can be discussed and/or
> experiences can be shared between the release managers of the previous
> release and the release managers of the next minor release. Of course, this
> could be open to other contributors as well. A summary could be provided in
> the Flink wiki (the Flink release's wiki article).
> 2. The release manager(s) provide a summary on the Flink release's wiki
> article as part of the release process.
> 3. Leave the process as is without any additional retrospective but focus
> on improving the documentation if issues arose during the release.
>
> That might help people who consider contributing to the community through
> supporting the release efforts. Additionally, it might help in
> understanding what went wrong in past releases retroactively (e.g. the
> longer release cycle for 1.15).
>
> I'm curious about opinion's on that topic.
>
> Best,
> Matthias
>

Re: [DISCUSS] Flink release retro

[Xingbo Huang <hx...@gmail.com>]

Thanks for starting the discussion, Matthias!

When I was the 1.16 release manager, I checked the dev mailing list to see
what the release managers of the previous two versions did throughout the
release cycle. In release 1.16, after each release sync meeting, we would
send a summary email, which can help capture how the release cycle has
progressed. So we may not really need a dedicated retrospective meeting.
But I think it's a good idea to have a wiki to summarize and review what
can be improved after we finish a release. For example, the release cycle
for 1.16 was also longer than expected. It took almost three months from
the feature freeze to our final official release, a month longer than
expected.

Best,
Xingbo

Jing Ge <ji...@ververica.com> 于2022年11月3日周四 04:56写道：

> Hi all,
>
> I figure it is a good idea and +1 for the async retro. More developers will
> learn from what the release process looks like, which will give them
> context to engage in future releases. It would be great if the conversation
> could somehow follow the traditional retro pattern, e.g. tagged with
> "Liked, learned, Lacked, and Longed for".
>
> Best regards,
> Jing
>
> On Wed, Nov 2, 2022 at 11:21 AM Martijn Visser <ma...@apache.org>
> wrote:
>
> > Hi Matthias,
> >
> > I think it's a good idea to capture how this release cycle has
> progressed.
> > I'm not sure that a classical "retrospective" is the best solution, since
> > it would require multiple people in different timezones to attend a
> virtual
> > meeting.
> >
> > So I would +1 an async retrospective, which could be the questions that
> you
> > would normally ask during a retrospective yet but now via a
> questionnaire.
> > It probably makes sense to have a proposal of the questions that can be
> > asked, discuss them and then sent them out.
> >
> > WDYT?
> >
> > Thanks,
> >
> > Martijn
> >
> > On Wed, Nov 2, 2022 at 9:42 AM Qingsheng Ren <re...@apache.org> wrote:
> >
> > > Thanks for starting the discussion Matthias!
> > >
> > > I think having a retro after a release cycle would be quite helpful to
> > > standardizing the procedure of the release, and also could avoid new
> > > release managers getting stuck on the same issue that happened before.
> I
> > > prefer the second option that RMs could open a discussion thread in ML
> at
> > > the end of the release to collect feedback about the last release cycle
> > and
> > > add them to the release wiki page, which would be quite handy for
> further
> > > RMs.
> > >
> > > Best,
> > > Qingsheng
> > > Ververica (Alibaba)
> > >
> > > On Mon, Oct 31, 2022 at 11:02 PM Matthias Pohl
> > > <ma...@aiven.io.invalid> wrote:
> > >
> > > > Hi everyone,
> > > > I want to bring up the idea of having a retrospective on the release
> > from
> > > > the release manager's perspective. The idea would be to collect
> > feedback
> > > on
> > > > what went well and what could be improved for a specific minor
> release.
> > > So
> > > > far, I didn't find anything on that topic. Does the community find
> this
> > > > useful? Or was this already done but not helpful?
> > > >
> > > > I see three options here:
> > > > 1. Having an actual meeting where issues can be discussed and/or
> > > > experiences can be shared between the release managers of the
> previous
> > > > release and the release managers of the next minor release. Of
> course,
> > > this
> > > > could be open to other contributors as well. A summary could be
> > provided
> > > in
> > > > the Flink wiki (the Flink release's wiki article).
> > > > 2. The release manager(s) provide a summary on the Flink release's
> wiki
> > > > article as part of the release process.
> > > > 3. Leave the process as is without any additional retrospective but
> > focus
> > > > on improving the documentation if issues arose during the release.
> > > >
> > > > That might help people who consider contributing to the community
> > through
> > > > supporting the release efforts. Additionally, it might help in
> > > > understanding what went wrong in past releases retroactively (e.g.
> the
> > > > longer release cycle for 1.15).
> > > >
> > > > I'm curious about opinion's on that topic.
> > > >
> > > > Best,
> > > > Matthias
> > > >
> > >
> >
>

Re: [DISCUSS] Flink release retro

[Jing Ge <ji...@ververica.com>]

Hi all,

I figure it is a good idea and +1 for the async retro. More developers will
learn from what the release process looks like, which will give them
context to engage in future releases. It would be great if the conversation
could somehow follow the traditional retro pattern, e.g. tagged with
"Liked, learned, Lacked, and Longed for".

Best regards,
Jing

On Wed, Nov 2, 2022 at 11:21 AM Martijn Visser <ma...@apache.org>
wrote:

> Hi Matthias,
>
> I think it's a good idea to capture how this release cycle has progressed.
> I'm not sure that a classical "retrospective" is the best solution, since
> it would require multiple people in different timezones to attend a virtual
> meeting.
>
> So I would +1 an async retrospective, which could be the questions that you
> would normally ask during a retrospective yet but now via a questionnaire.
> It probably makes sense to have a proposal of the questions that can be
> asked, discuss them and then sent them out.
>
> WDYT?
>
> Thanks,
>
> Martijn
>
> On Wed, Nov 2, 2022 at 9:42 AM Qingsheng Ren <re...@apache.org> wrote:
>
> > Thanks for starting the discussion Matthias!
> >
> > I think having a retro after a release cycle would be quite helpful to
> > standardizing the procedure of the release, and also could avoid new
> > release managers getting stuck on the same issue that happened before. I
> > prefer the second option that RMs could open a discussion thread in ML at
> > the end of the release to collect feedback about the last release cycle
> and
> > add them to the release wiki page, which would be quite handy for further
> > RMs.
> >
> > Best,
> > Qingsheng
> > Ververica (Alibaba)
> >
> > On Mon, Oct 31, 2022 at 11:02 PM Matthias Pohl
> > <ma...@aiven.io.invalid> wrote:
> >
> > > Hi everyone,
> > > I want to bring up the idea of having a retrospective on the release
> from
> > > the release manager's perspective. The idea would be to collect
> feedback
> > on
> > > what went well and what could be improved for a specific minor release.
> > So
> > > far, I didn't find anything on that topic. Does the community find this
> > > useful? Or was this already done but not helpful?
> > >
> > > I see three options here:
> > > 1. Having an actual meeting where issues can be discussed and/or
> > > experiences can be shared between the release managers of the previous
> > > release and the release managers of the next minor release. Of course,
> > this
> > > could be open to other contributors as well. A summary could be
> provided
> > in
> > > the Flink wiki (the Flink release's wiki article).
> > > 2. The release manager(s) provide a summary on the Flink release's wiki
> > > article as part of the release process.
> > > 3. Leave the process as is without any additional retrospective but
> focus
> > > on improving the documentation if issues arose during the release.
> > >
> > > That might help people who consider contributing to the community
> through
> > > supporting the release efforts. Additionally, it might help in
> > > understanding what went wrong in past releases retroactively (e.g. the
> > > longer release cycle for 1.15).
> > >
> > > I'm curious about opinion's on that topic.
> > >
> > > Best,
> > > Matthias
> > >
> >
>

Re: [DISCUSS] Flink release retro

[Martijn Visser <ma...@apache.org>]

Hi Matthias,

I think it's a good idea to capture how this release cycle has progressed.
I'm not sure that a classical "retrospective" is the best solution, since
it would require multiple people in different timezones to attend a virtual
meeting.

So I would +1 an async retrospective, which could be the questions that you
would normally ask during a retrospective yet but now via a questionnaire.
It probably makes sense to have a proposal of the questions that can be
asked, discuss them and then sent them out.

WDYT?

Thanks,

Martijn

On Wed, Nov 2, 2022 at 9:42 AM Qingsheng Ren <re...@apache.org> wrote:

> Thanks for starting the discussion Matthias!
>
> I think having a retro after a release cycle would be quite helpful to
> standardizing the procedure of the release, and also could avoid new
> release managers getting stuck on the same issue that happened before. I
> prefer the second option that RMs could open a discussion thread in ML at
> the end of the release to collect feedback about the last release cycle and
> add them to the release wiki page, which would be quite handy for further
> RMs.
>
> Best,
> Qingsheng
> Ververica (Alibaba)
>
> On Mon, Oct 31, 2022 at 11:02 PM Matthias Pohl
> <ma...@aiven.io.invalid> wrote:
>
> > Hi everyone,
> > I want to bring up the idea of having a retrospective on the release from
> > the release manager's perspective. The idea would be to collect feedback
> on
> > what went well and what could be improved for a specific minor release.
> So
> > far, I didn't find anything on that topic. Does the community find this
> > useful? Or was this already done but not helpful?
> >
> > I see three options here:
> > 1. Having an actual meeting where issues can be discussed and/or
> > experiences can be shared between the release managers of the previous
> > release and the release managers of the next minor release. Of course,
> this
> > could be open to other contributors as well. A summary could be provided
> in
> > the Flink wiki (the Flink release's wiki article).
> > 2. The release manager(s) provide a summary on the Flink release's wiki
> > article as part of the release process.
> > 3. Leave the process as is without any additional retrospective but focus
> > on improving the documentation if issues arose during the release.
> >
> > That might help people who consider contributing to the community through
> > supporting the release efforts. Additionally, it might help in
> > understanding what went wrong in past releases retroactively (e.g. the
> > longer release cycle for 1.15).
> >
> > I'm curious about opinion's on that topic.
> >
> > Best,
> > Matthias
> >
>

Re: [DISCUSS] Flink release retro

[Matthias Pohl <ma...@aiven.io.INVALID>]

I updated the Flink release wiki article [1] as discussed in my previous
email. An explicit request for feedback on the release process will be part
of the release announcement from now onwards.
Additionally, I added a subsection in wrapping up the release where we
suggest creating a summary per release. This summary can include feedback
from contributors but also general issues that popped up during the release
(e.g. the release process taking longer than anticipated and reasons for
it).

@Xingbo Could you take a moment to add a summary section to the 1.16
release wiki page [2]? ...since the memory of that might still be kind of
fresh.

Matthias

[1]
https://cwiki.apache.org/confluence/display/FLINK/Creating+a+Flink+Release
[2] https://cwiki.apache.org/confluence/display/FLINK/1.16+Release

On Mon, Nov 7, 2022 at 7:16 PM Matthias Pohl <ma...@aiven.io> wrote:

> Thanks for all the insights on how it was done in the past and the pro's
> and con's of the different approaches. ...also being reminded on how we did
> it for 1.14 and the positive feedback we got from it was helpful.
>
> I like the idea of an async survey. I also think that utilizing the
> mailing list for this sounds reasonable and doesn't add too much effort.
> Encouraging developers to reply to the release announcement in the ML might
> be a good place to not only congratulate but also give feedback on what
> could be improved.
>
> We could try that out and see (as a follow-up step) whether it makes sense
> to formalize it even more by providing a questionnaire form as suggested by
> Martijn. I would volunteer in updating the announcement template in the
> wiki [1] adding a sentence explicitly asking for feedback on the release.
> Additionally, the 1.17 release managers are planning to update the release
> management documentation [2] to give better guidance on what is expected
> from a release manager. That documentation should also include collecting
> feedback retroactively on a release. WDYT?
>
> Matthias
>
> [1]
> https://cwiki.apache.org/confluence/display/FLINK/Creating+a+Flink+Release
> [2]
> https://cwiki.apache.org/confluence/display/FLINK/Flink+Release+Management
>
> On Thu, Nov 3, 2022 at 11:08 AM Johannes Moser <jo...@moser.wtf> wrote:
>
>> Hi,
>>
>> We did something like this for the 1.14 release [1].
>> We did that in an async survey-style way approaching key contributors
>> directly. I also think the output was quite nice and we got positive
>> feedback for what we published.
>> I think Robert mentioned they did something similar in the past, with a
>> form-based survey with little feedback.
>>
>> Just as an input.
>>
>> I still think "Flink Backward" is a great name for a retro. :)
>>
>> Best,
>> Joe
>>
>>
>> [1] https://flink.apache.org/2021/11/03/flink-backward.html
>>
>> On Wed, Nov 2, 2022 at 9:42 AM Qingsheng Ren <re...@apache.org> wrote:
>>
>> > Thanks for starting the discussion Matthias!
>> >
>> > I think having a retro after a release cycle would be quite helpful to
>> > standardizing the procedure of the release, and also could avoid new
>> > release managers getting stuck on the same issue that happened before. I
>> > prefer the second option that RMs could open a discussion thread in ML
>> at
>> > the end of the release to collect feedback about the last release cycle
>> and
>> > add them to the release wiki page, which would be quite handy for
>> further
>> > RMs.
>> >
>> > Best,
>> > Qingsheng
>> > Ververica (Alibaba)
>> >
>> > On Mon, Oct 31, 2022 at 11:02 PM Matthias Pohl
>> > <ma...@aiven.io.invalid> wrote:
>> >
>> > > Hi everyone,
>> > > I want to bring up the idea of having a retrospective on the release
>> from
>> > > the release manager's perspective. The idea would be to collect
>> feedback
>> > on
>> > > what went well and what could be improved for a specific minor
>> release.
>> > So
>> > > far, I didn't find anything on that topic. Does the community find
>> this
>> > > useful? Or was this already done but not helpful?
>> > >
>> > > I see three options here:
>> > > 1. Having an actual meeting where issues can be discussed and/or
>> > > experiences can be shared between the release managers of the previous
>> > > release and the release managers of the next minor release. Of course,
>> > this
>> > > could be open to other contributors as well. A summary could be
>> provided
>> > in
>> > > the Flink wiki (the Flink release's wiki article).
>> > > 2. The release manager(s) provide a summary on the Flink release's
>> wiki
>> > > article as part of the release process.
>> > > 3. Leave the process as is without any additional retrospective but
>> focus
>> > > on improving the documentation if issues arose during the release.
>> > >
>> > > That might help people who consider contributing to the community
>> through
>> > > supporting the release efforts. Additionally, it might help in
>> > > understanding what went wrong in past releases retroactively (e.g. the
>> > > longer release cycle for 1.15).
>> > >
>> > > I'm curious about opinion's on that topic.
>> > >
>> > > Best,
>> > > Matthias
>> > >
>> >
>>
>

Re: [DISCUSS] Flink release retro

[Matthias Pohl <ma...@aiven.io.INVALID>]

Thanks for all the insights on how it was done in the past and the pro's
and con's of the different approaches. ...also being reminded on how we did
it for 1.14 and the positive feedback we got from it was helpful.

I like the idea of an async survey. I also think that utilizing the mailing
list for this sounds reasonable and doesn't add too much effort.
Encouraging developers to reply to the release announcement in the ML might
be a good place to not only congratulate but also give feedback on what
could be improved.

We could try that out and see (as a follow-up step) whether it makes sense
to formalize it even more by providing a questionnaire form as suggested by
Martijn. I would volunteer in updating the announcement template in the
wiki [1] adding a sentence explicitly asking for feedback on the release.
Additionally, the 1.17 release managers are planning to update the release
management documentation [2] to give better guidance on what is expected
from a release manager. That documentation should also include collecting
feedback retroactively on a release. WDYT?

Matthias

[1]
https://cwiki.apache.org/confluence/display/FLINK/Creating+a+Flink+Release
[2]
https://cwiki.apache.org/confluence/display/FLINK/Flink+Release+Management

On Thu, Nov 3, 2022 at 11:08 AM Johannes Moser <jo...@moser.wtf> wrote:

> Hi,
>
> We did something like this for the 1.14 release [1].
> We did that in an async survey-style way approaching key contributors
> directly. I also think the output was quite nice and we got positive
> feedback for what we published.
> I think Robert mentioned they did something similar in the past, with a
> form-based survey with little feedback.
>
> Just as an input.
>
> I still think "Flink Backward" is a great name for a retro. :)
>
> Best,
> Joe
>
>
> [1] https://flink.apache.org/2021/11/03/flink-backward.html
>
> On Wed, Nov 2, 2022 at 9:42 AM Qingsheng Ren <re...@apache.org> wrote:
>
> > Thanks for starting the discussion Matthias!
> >
> > I think having a retro after a release cycle would be quite helpful to
> > standardizing the procedure of the release, and also could avoid new
> > release managers getting stuck on the same issue that happened before. I
> > prefer the second option that RMs could open a discussion thread in ML at
> > the end of the release to collect feedback about the last release cycle
> and
> > add them to the release wiki page, which would be quite handy for further
> > RMs.
> >
> > Best,
> > Qingsheng
> > Ververica (Alibaba)
> >
> > On Mon, Oct 31, 2022 at 11:02 PM Matthias Pohl
> > <ma...@aiven.io.invalid> wrote:
> >
> > > Hi everyone,
> > > I want to bring up the idea of having a retrospective on the release
> from
> > > the release manager's perspective. The idea would be to collect
> feedback
> > on
> > > what went well and what could be improved for a specific minor release.
> > So
> > > far, I didn't find anything on that topic. Does the community find this
> > > useful? Or was this already done but not helpful?
> > >
> > > I see three options here:
> > > 1. Having an actual meeting where issues can be discussed and/or
> > > experiences can be shared between the release managers of the previous
> > > release and the release managers of the next minor release. Of course,
> > this
> > > could be open to other contributors as well. A summary could be
> provided
> > in
> > > the Flink wiki (the Flink release's wiki article).
> > > 2. The release manager(s) provide a summary on the Flink release's wiki
> > > article as part of the release process.
> > > 3. Leave the process as is without any additional retrospective but
> focus
> > > on improving the documentation if issues arose during the release.
> > >
> > > That might help people who consider contributing to the community
> through
> > > supporting the release efforts. Additionally, it might help in
> > > understanding what went wrong in past releases retroactively (e.g. the
> > > longer release cycle for 1.15).
> > >
> > > I'm curious about opinion's on that topic.
> > >
> > > Best,
> > > Matthias
> > >
> >
>

Re: [DISCUSS] Flink release retro

[Johannes Moser <jo...@moser.wtf>]

Hi,

We did something like this for the 1.14 release [1].
We did that in an async survey-style way approaching key contributors
directly. I also think the output was quite nice and we got positive
feedback for what we published.
I think Robert mentioned they did something similar in the past, with a
form-based survey with little feedback.

Just as an input.

I still think "Flink Backward" is a great name for a retro. :)

Best,
Joe


[1] https://flink.apache.org/2021/11/03/flink-backward.html

On Wed, Nov 2, 2022 at 9:42 AM Qingsheng Ren <re...@apache.org> wrote:

> Thanks for starting the discussion Matthias!
>
> I think having a retro after a release cycle would be quite helpful to
> standardizing the procedure of the release, and also could avoid new
> release managers getting stuck on the same issue that happened before. I
> prefer the second option that RMs could open a discussion thread in ML at
> the end of the release to collect feedback about the last release cycle and
> add them to the release wiki page, which would be quite handy for further
> RMs.
>
> Best,
> Qingsheng
> Ververica (Alibaba)
>
> On Mon, Oct 31, 2022 at 11:02 PM Matthias Pohl
> <ma...@aiven.io.invalid> wrote:
>
> > Hi everyone,
> > I want to bring up the idea of having a retrospective on the release from
> > the release manager's perspective. The idea would be to collect feedback
> on
> > what went well and what could be improved for a specific minor release.
> So
> > far, I didn't find anything on that topic. Does the community find this
> > useful? Or was this already done but not helpful?
> >
> > I see three options here:
> > 1. Having an actual meeting where issues can be discussed and/or
> > experiences can be shared between the release managers of the previous
> > release and the release managers of the next minor release. Of course,
> this
> > could be open to other contributors as well. A summary could be provided
> in
> > the Flink wiki (the Flink release's wiki article).
> > 2. The release manager(s) provide a summary on the Flink release's wiki
> > article as part of the release process.
> > 3. Leave the process as is without any additional retrospective but focus
> > on improving the documentation if issues arose during the release.
> >
> > That might help people who consider contributing to the community through
> > supporting the release efforts. Additionally, it might help in
> > understanding what went wrong in past releases retroactively (e.g. the
> > longer release cycle for 1.15).
> >
> > I'm curious about opinion's on that topic.
> >
> > Best,
> > Matthias
> >
>