You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@beam.apache.org by Rafael Fernandez <rf...@google.com> on 2018/06/14 20:33:13 UTC
[DISCUSS] Releasing Beam in the presence of emergencies
Hi Beam devs,
Emergencies can and will happen. As Apache Beam adoption continues to grow,
the user community will naturally expect the Beam developer community to
react to critical issues, such as security vulnerabilities in our
dependencies. I want to make sure the dev community is in agreement that we
follow the ASF Vulnerability Handling processes [1] for such occurrences.
In addition, I'd like to discuss cases in which data correctness/loss may
warrant an expedited release (i.e., we did not wait 72 hours), as we did in
2.1.1 [2]. Concretely:
1.
Do we need to add anything to our project website so the user community
knows how we react to such issues?
2.
Should we have an entry in the contributor guide to address critical
point releases, so we eliminate any guesswork in the event of an emergency?
(Example text [3])
Thanks,
r
[1]
*https://apache.org/security/committers.html#vulnerability-handling
<https://apache.org/security/committers.html#vulnerability-handling>*
[2] https://lists.apache.org/list.html?dev@beam.apache.org:lte=40M:2.1.1
*[3] Example text for the contributor guideline:What requires a critical
point release? - A data loss bug- A data corruption bug- A processing
correctness bug- For security vulnerabilities, please follow
https://apache.org/security/committers.html#vulnerability-handling
<https://apache.org/security/committers.html#vulnerability-handling> .What
do we do a critical point release on?Our first priority is to stop the
bleeding. We ought to prioritize a point release for the latest Beam
version, based on the release branch, that cherrypicks only the intended
fix. - We've done it before! Remember 2.1.1
<https://lists.apache.org/list.html?dev@beam.apache.org:lte=40M:2.1.1>? -
Since this is a critical release, we can relax our usual 72 hour voting
policy. It worked well for 2.1.1, we should make it repeatable: Propose,
have PMC folks do due diligence on the request, and sign off. Since this is
critical, we may want to have more than one person working on the release.-
Once we get it out, the community can discuss which previous releases would
benefit from a potential point release. Who proposes a critical point
release?Any member of the community. 3 PMC +1 votes are sufficient to get
the process rolling.*
Re: [DISCUSS] Releasing Beam in the presence of emergencies
Posted by Ahmet Altay <al...@google.com>.
Thank you Rafael.
I think it is a good idea to include our commitment, including concrete
steps on our website. This would make it easier for enterprise users to
choose Beam. Even though this is already partially Apache policy and there
is precedence in our project with 2.1.1 release; increasing the visibility
for users would be a positive addition.
I would suggest changing the link in [2] to (
https://lists.apache.org/list.html?dev@beam.apache.org:gte=1d:2.1.1) so
that link does not expire after some time.
Ahmet
On Thu, Jun 14, 2018 at 1:33 PM, Rafael Fernandez <rf...@google.com>
wrote:
> Hi Beam devs,
>
> Emergencies can and will happen. As Apache Beam adoption continues to
> grow, the user community will naturally expect the Beam developer community
> to react to critical issues, such as security vulnerabilities in our
> dependencies. I want to make sure the dev community is in agreement that we
> follow the ASF Vulnerability Handling processes [1] for such occurrences.
>
> In addition, I'd like to discuss cases in which data correctness/loss may
> warrant an expedited release (i.e., we did not wait 72 hours), as we did in
> 2.1.1 [2]. Concretely:
>
>
> 1.
>
> Do we need to add anything to our project website so the user
> community knows how we react to such issues?
> 2.
>
> Should we have an entry in the contributor guide to address critical
> point releases, so we eliminate any guesswork in the event of an emergency?
> (Example text [3])
>
>
> Thanks,
>
> r
>
>
> [1]
> *https://apache.org/security/committers.html#vulnerability-handling
> <https://apache.org/security/committers.html#vulnerability-handling>*
>
> [2] https://lists.apache.org/list.html?dev@beam.apache.org:lte=40M:2.1.1
>
>
>
>
>
>
>
>
> *[3] Example text for the contributor guideline:What requires a critical
> point release? - A data loss bug- A data corruption bug- A processing
> correctness bug- For security vulnerabilities, please follow
> https://apache.org/security/committers.html#vulnerability-handling
> <https://apache.org/security/committers.html#vulnerability-handling> .What
> do we do a critical point release on?Our first priority is to stop the
> bleeding. We ought to prioritize a point release for the latest Beam
> version, based on the release branch, that cherrypicks only the intended
> fix. - We've done it before! Remember 2.1.1
> <https://lists.apache.org/list.html?dev@beam.apache.org:lte=40M:2.1.1>? -
> Since this is a critical release, we can relax our usual 72 hour voting
> policy. It worked well for 2.1.1, we should make it repeatable: Propose,
> have PMC folks do due diligence on the request, and sign off. Since this is
> critical, we may want to have more than one person working on the release.-
> Once we get it out, the community can discuss which previous releases would
> benefit from a potential point release. Who proposes a critical point
> release?Any member of the community. 3 PMC +1 votes are sufficient to get
> the process rolling.*
>
>
>
>
Re: [DISCUSS] Releasing Beam in the presence of emergencies
Posted by Rafael Fernandez <rf...@google.com>.
Thanks all for the discussion, what I gather is:
(1) Stating the obvious: our release practices and mechanisms match what I
suggested we wrote down in a page. (That's great!). I heard no objections
to affirming our adherence to ASF vulnerability policy.
(2) There are details to be discussed further - specifically, which
releases to fix (which suggests discussion on what a "current" release
could be), and details on who the release manager could be (in the presence
of multiple releases)
So let me propose the following: I'll send a PR to the website for (1), and
for (2), let's organize the discussion along the various axis -- I liked
how Chamikara broke down the space, so we should be able to work along
those lines.
Thanks,
r
On Sun, Jun 17, 2018 at 1:14 AM Romain Manni-Bucau <rm...@gmail.com>
wrote:
> Hi guys
>
> If you take back this thread [1], then beam will only support two branches
> and no others.
>
> On the shortcut release cycle it is not really desired at apache and not
> the normal process so the 3 days vote is required but not a blocker IMHO.
> If really an issue the merge can be done quickly on master and you can
> release on your own nexus in 1h. The side note for the asf release manager
> here will be to emphase the diff to let people review it faster. Less
> change there is, easier it is to vote.
>
This is a good point to discuss. If we simply state this as valid for any
release, the mechanics are simpler. I suggested breaking u p the
discussions along various axis so we can make progress on each concern --
please bring this up on release mechanics.
>
> My question on that process will be for dataflow guys wich often request a
> few more days. Is it ok because dataflow doesnt use multiple versions or
> does it need some work on that side to support a faster support of new beam
> versions?
>
I'm not sure what part of the cycle you are referring to - I have seen RCs
taking longer to validate because we discover issues in the service. If
that's the case, I'm happy to share that the team is working on increasing
test coverage in Beam itself so we minimize this class of surprises.
Thanks Romain!
>
> [1]
> https://lists.apache.org/thread.html/1f6941f112d080de3633d11c574fb67bce5f6f81679ec3bdeaf807f0@
> <dev.beam.apache.org>
>
> Le dim. 17 juin 2018 01:08, Ravi Prakash <ra...@apache.org> a écrit :
>
>> Hi Beam-devs!
>>
>> There is also the consideration of which releases to fix. I am familiar
>> with Apache Hadoop and there may be a "stable", "beta" and "alpha" release.
>> Usually security fixes are ported to one or more of those branches.
>>
>> Cheers
>> Ravi
>>
>> On Sat, Jun 16, 2018 at 2:43 PM, Kenneth Knowles <kl...@google.com> wrote:
>>
>>> I like the idea of explicitly affirming our use of the ASF Vulnerability
>>> Policy as well as listing other classes of bugs that are particularly
>>> critical for a project like Beam. I think the most valuable thing is having
>>> a clear and concise policy for users to understand at a glance. Then we can
>>> have a verbose walkthrough for someone implementing the policy.
>>>
>>> Kenn
>>>
>>> On Sat, Jun 16, 2018 at 9:54 AM Chamikara Jayalath <ch...@google.com>
>>> wrote:
>>>
>>>> While performing a patch release is the correct approach for a critical
>>>> fixes I think there are still several points that we might want to discuss
>>>> and formalize/document if needed.
>>>>
>>>> (1) What if there's an ongoing major/minor version release ?
>>>> If think patch releases should be independent of any ongoing
>>>> major/minor versions releases and should be handled by a separate release
>>>> manager who can commit the the time to cut the release immediately.
>>>>
>>>> (2) What changes should go in ?
>>>> Only the fix that addresses the critical issue should go into the patch
>>>> release. Any other fixes should be held till the next minor version release.
>>>>
>>>> (3) What to do with the faulty release ?
>>>> We should clearly document the issue with the faulty (non-patched)
>>>> release in https://beam.apache.org/get-started/downloads/. This can be
>>>> complemented by other announcements (to user list, twitter, for example)
>>>> and runner specific solutions (rejecting jobs, warnings for running jobs).
>>>>
>>>> (4) Voting period and validation.
>>>> We are performing the patch on top of an already released branch. So
>>>> can we expedite the release validation and voting process ?
>>>>
>>>> I'm sure there are other points.
>>>>
>>>> Thanks,
>>>> Cham
>>>>
>>>> On Thu, Jun 14, 2018 at 10:29 PM Jean-Baptiste Onofré <jb...@nanthrax.net>
>>>> wrote:
>>>>
>>>>> Hi Rafael,
>>>>>
>>>>> It's a good point but I don't see nothing more to do on our side: if a
>>>>> emergency issue is detected, then we have to address it and release a
>>>>> fix release (x.y.z where z is the specific release fixing the issue).
>>>>> The commitment is a best effort as in all community: if an emergency
>>>>> issue is detected, qualified and accepted, then we do our best to
>>>>> provide a fix and do the fix release.
>>>>>
>>>>> So, for me, it's already handled.
>>>>>
>>>>> By the way, just a quick reminder in term of release:
>>>>>
>>>>> - now that gradle release seems ok, we resume our release cycle every ~
>>>>> 6 weeks
>>>>> - we can cut release anytime if required, especially to address
>>>>> emergency issues.
>>>>>
>>>>> Regards
>>>>> JB
>>>>>
>>>>> On 14/06/2018 22:33, Rafael Fernandez wrote:
>>>>> > Hi Beam devs,
>>>>> >
>>>>> > Emergencies can and will happen. As Apache Beam adoption continues to
>>>>> > grow, the user community will naturally expect the Beam developer
>>>>> > community to react to critical issues, such as security
>>>>> vulnerabilities
>>>>> > in our dependencies. I want to make sure the dev community is in
>>>>> > agreement that we follow the ASF Vulnerability Handling processes [1]
>>>>> > for such occurrences.
>>>>> >
>>>>> >
>>>>> > In addition, I'd like to discuss cases in which data correctness/loss
>>>>> > may warrant an expedited release (i.e., we did not wait 72 hours),
>>>>> as we
>>>>> > did in 2.1.1 [2]. Concretely:
>>>>> >
>>>>> >
>>>>> > 1.
>>>>> >
>>>>> > Do we need to add anything to our project website so the user
>>>>> > community knows how we react to such issues?
>>>>> >
>>>>> > 2.
>>>>> >
>>>>> > Should we have an entry in the contributor guide to address
>>>>> critical
>>>>> > point releases, so we eliminate any guesswork in the event of an
>>>>> > emergency? (Example text [3])
>>>>> >
>>>>> >
>>>>> > Thanks,
>>>>> >
>>>>> > r
>>>>> >
>>>>> >
>>>>> > [1]
>>>>> >
>>>>> > _https://apache.org/security/committers.html#vulnerability-handling_
>>>>> >
>>>>> > [2]
>>>>> https://lists.apache.org/list.html?dev@beam.apache.org:lte=40M:2.1.1
>>>>> >
>>>>> > *
>>>>> >
>>>>> > [3] Example text for the contributor guideline:
>>>>> >
>>>>> >
>>>>> > What requires a critical point release?
>>>>> >
>>>>> > *
>>>>> >
>>>>> > A data loss bug
>>>>> >
>>>>> > *
>>>>> >
>>>>> > A data corruption bug
>>>>> >
>>>>> > *
>>>>> >
>>>>> > A processing correctness bug
>>>>> >
>>>>> > *
>>>>> >
>>>>> > For security vulnerabilities, please follow
>>>>> >
>>>>> https://apache.org/security/committers.html#vulnerability-handling .
>>>>> >
>>>>> >
>>>>> > What do we do a critical point release on?
>>>>> >
>>>>> > Our first priority is to stop the bleeding. We ought to prioritize a
>>>>> > point release for the latest Beam version, based on the release
>>>>> branch,
>>>>> > that cherrypicks only the intended fix.
>>>>> >
>>>>> > *
>>>>> >
>>>>> > We've done it before! Remember 2.1.1
>>>>> > <
>>>>> https://lists.apache.org/list.html?dev@beam.apache.org:lte=40M:2.1.1>?
>>>>> >
>>>>> > o
>>>>> >
>>>>> > Since this is a critical release, we can relax our usual 72
>>>>> hour
>>>>> > voting policy. It worked well for 2.1.1, we should make it
>>>>> > repeatable: Propose, have PMC folks do due diligence on the
>>>>> > request, and sign off. Since this is critical, we may want to
>>>>> > have more than one person working on the release.
>>>>> >
>>>>> > *
>>>>> >
>>>>> > Once we get it out, the community can discuss which previous
>>>>> > releases would benefit from a potential point release.
>>>>> >
>>>>> >
>>>>> > Who proposes a critical point release?
>>>>> >
>>>>> > Any member of the community. 3 PMC +1 votes are sufficient to get the
>>>>> > process rolling.
>>>>> >
>>>>> > *
>>>>> >
>>>>> >
>>>>> >
>>>>>
>>>>> --
>>>>> Jean-Baptiste Onofré
>>>>> jbonofre@apache.org
>>>>> http://blog.nanthrax.net
>>>>> Talend - http://www.talend.com
>>>>>
>>>>
>>
Re: [DISCUSS] Releasing Beam in the presence of emergencies
Posted by Romain Manni-Bucau <rm...@gmail.com>.
Hi guys
If you take back this thread [1], then beam will only support two branches
and no others.
On the shortcut release cycle it is not really desired at apache and not
the normal process so the 3 days vote is required but not a blocker IMHO.
If really an issue the merge can be done quickly on master and you can
release on your own nexus in 1h. The side note for the asf release manager
here will be to emphase the diff to let people review it faster. Less
change there is, easier it is to vote.
My question on that process will be for dataflow guys wich often request a
few more days. Is it ok because dataflow doesnt use multiple versions or
does it need some work on that side to support a faster support of new beam
versions?
[1]
https://lists.apache.org/thread.html/1f6941f112d080de3633d11c574fb67bce5f6f81679ec3bdeaf807f0@
<dev.beam.apache.org>
Le dim. 17 juin 2018 01:08, Ravi Prakash <ra...@apache.org> a écrit :
> Hi Beam-devs!
>
> There is also the consideration of which releases to fix. I am familiar
> with Apache Hadoop and there may be a "stable", "beta" and "alpha" release.
> Usually security fixes are ported to one or more of those branches.
>
> Cheers
> Ravi
>
> On Sat, Jun 16, 2018 at 2:43 PM, Kenneth Knowles <kl...@google.com> wrote:
>
>> I like the idea of explicitly affirming our use of the ASF Vulnerability
>> Policy as well as listing other classes of bugs that are particularly
>> critical for a project like Beam. I think the most valuable thing is having
>> a clear and concise policy for users to understand at a glance. Then we can
>> have a verbose walkthrough for someone implementing the policy.
>>
>> Kenn
>>
>> On Sat, Jun 16, 2018 at 9:54 AM Chamikara Jayalath <ch...@google.com>
>> wrote:
>>
>>> While performing a patch release is the correct approach for a critical
>>> fixes I think there are still several points that we might want to discuss
>>> and formalize/document if needed.
>>>
>>> (1) What if there's an ongoing major/minor version release ?
>>> If think patch releases should be independent of any ongoing major/minor
>>> versions releases and should be handled by a separate release manager who
>>> can commit the the time to cut the release immediately.
>>>
>>> (2) What changes should go in ?
>>> Only the fix that addresses the critical issue should go into the patch
>>> release. Any other fixes should be held till the next minor version release.
>>>
>>> (3) What to do with the faulty release ?
>>> We should clearly document the issue with the faulty (non-patched)
>>> release in https://beam.apache.org/get-started/downloads/. This can be
>>> complemented by other announcements (to user list, twitter, for example)
>>> and runner specific solutions (rejecting jobs, warnings for running jobs).
>>>
>>> (4) Voting period and validation.
>>> We are performing the patch on top of an already released branch. So can
>>> we expedite the release validation and voting process ?
>>>
>>> I'm sure there are other points.
>>>
>>> Thanks,
>>> Cham
>>>
>>> On Thu, Jun 14, 2018 at 10:29 PM Jean-Baptiste Onofré <jb...@nanthrax.net>
>>> wrote:
>>>
>>>> Hi Rafael,
>>>>
>>>> It's a good point but I don't see nothing more to do on our side: if a
>>>> emergency issue is detected, then we have to address it and release a
>>>> fix release (x.y.z where z is the specific release fixing the issue).
>>>> The commitment is a best effort as in all community: if an emergency
>>>> issue is detected, qualified and accepted, then we do our best to
>>>> provide a fix and do the fix release.
>>>>
>>>> So, for me, it's already handled.
>>>>
>>>> By the way, just a quick reminder in term of release:
>>>>
>>>> - now that gradle release seems ok, we resume our release cycle every ~
>>>> 6 weeks
>>>> - we can cut release anytime if required, especially to address
>>>> emergency issues.
>>>>
>>>> Regards
>>>> JB
>>>>
>>>> On 14/06/2018 22:33, Rafael Fernandez wrote:
>>>> > Hi Beam devs,
>>>> >
>>>> > Emergencies can and will happen. As Apache Beam adoption continues to
>>>> > grow, the user community will naturally expect the Beam developer
>>>> > community to react to critical issues, such as security
>>>> vulnerabilities
>>>> > in our dependencies. I want to make sure the dev community is in
>>>> > agreement that we follow the ASF Vulnerability Handling processes [1]
>>>> > for such occurrences.
>>>> >
>>>> >
>>>> > In addition, I'd like to discuss cases in which data correctness/loss
>>>> > may warrant an expedited release (i.e., we did not wait 72 hours), as
>>>> we
>>>> > did in 2.1.1 [2]. Concretely:
>>>> >
>>>> >
>>>> > 1.
>>>> >
>>>> > Do we need to add anything to our project website so the user
>>>> > community knows how we react to such issues?
>>>> >
>>>> > 2.
>>>> >
>>>> > Should we have an entry in the contributor guide to address
>>>> critical
>>>> > point releases, so we eliminate any guesswork in the event of an
>>>> > emergency? (Example text [3])
>>>> >
>>>> >
>>>> > Thanks,
>>>> >
>>>> > r
>>>> >
>>>> >
>>>> > [1]
>>>> >
>>>> > _https://apache.org/security/committers.html#vulnerability-handling_
>>>> >
>>>> > [2]
>>>> https://lists.apache.org/list.html?dev@beam.apache.org:lte=40M:2.1.1
>>>> >
>>>> > *
>>>> >
>>>> > [3] Example text for the contributor guideline:
>>>> >
>>>> >
>>>> > What requires a critical point release?
>>>> >
>>>> > *
>>>> >
>>>> > A data loss bug
>>>> >
>>>> > *
>>>> >
>>>> > A data corruption bug
>>>> >
>>>> > *
>>>> >
>>>> > A processing correctness bug
>>>> >
>>>> > *
>>>> >
>>>> > For security vulnerabilities, please follow
>>>> >
>>>> https://apache.org/security/committers.html#vulnerability-handling .
>>>> >
>>>> >
>>>> > What do we do a critical point release on?
>>>> >
>>>> > Our first priority is to stop the bleeding. We ought to prioritize a
>>>> > point release for the latest Beam version, based on the release
>>>> branch,
>>>> > that cherrypicks only the intended fix.
>>>> >
>>>> > *
>>>> >
>>>> > We've done it before! Remember 2.1.1
>>>> > <
>>>> https://lists.apache.org/list.html?dev@beam.apache.org:lte=40M:2.1.1>?
>>>> >
>>>> > o
>>>> >
>>>> > Since this is a critical release, we can relax our usual 72
>>>> hour
>>>> > voting policy. It worked well for 2.1.1, we should make it
>>>> > repeatable: Propose, have PMC folks do due diligence on the
>>>> > request, and sign off. Since this is critical, we may want to
>>>> > have more than one person working on the release.
>>>> >
>>>> > *
>>>> >
>>>> > Once we get it out, the community can discuss which previous
>>>> > releases would benefit from a potential point release.
>>>> >
>>>> >
>>>> > Who proposes a critical point release?
>>>> >
>>>> > Any member of the community. 3 PMC +1 votes are sufficient to get the
>>>> > process rolling.
>>>> >
>>>> > *
>>>> >
>>>> >
>>>> >
>>>>
>>>> --
>>>> Jean-Baptiste Onofré
>>>> jbonofre@apache.org
>>>> http://blog.nanthrax.net
>>>> Talend - http://www.talend.com
>>>>
>>>
>
Re: [DISCUSS] Releasing Beam in the presence of emergencies
Posted by Ravi Prakash <ra...@apache.org>.
Hi Beam-devs!
There is also the consideration of which releases to fix. I am familiar
with Apache Hadoop and there may be a "stable", "beta" and "alpha" release.
Usually security fixes are ported to one or more of those branches.
Cheers
Ravi
On Sat, Jun 16, 2018 at 2:43 PM, Kenneth Knowles <kl...@google.com> wrote:
> I like the idea of explicitly affirming our use of the ASF Vulnerability
> Policy as well as listing other classes of bugs that are particularly
> critical for a project like Beam. I think the most valuable thing is having
> a clear and concise policy for users to understand at a glance. Then we can
> have a verbose walkthrough for someone implementing the policy.
>
> Kenn
>
> On Sat, Jun 16, 2018 at 9:54 AM Chamikara Jayalath <ch...@google.com>
> wrote:
>
>> While performing a patch release is the correct approach for a critical
>> fixes I think there are still several points that we might want to discuss
>> and formalize/document if needed.
>>
>> (1) What if there's an ongoing major/minor version release ?
>> If think patch releases should be independent of any ongoing major/minor
>> versions releases and should be handled by a separate release manager who
>> can commit the the time to cut the release immediately.
>>
>> (2) What changes should go in ?
>> Only the fix that addresses the critical issue should go into the patch
>> release. Any other fixes should be held till the next minor version release.
>>
>> (3) What to do with the faulty release ?
>> We should clearly document the issue with the faulty (non-patched)
>> release in https://beam.apache.org/get-started/downloads/. This can be
>> complemented by other announcements (to user list, twitter, for example)
>> and runner specific solutions (rejecting jobs, warnings for running jobs).
>>
>> (4) Voting period and validation.
>> We are performing the patch on top of an already released branch. So can
>> we expedite the release validation and voting process ?
>>
>> I'm sure there are other points.
>>
>> Thanks,
>> Cham
>>
>> On Thu, Jun 14, 2018 at 10:29 PM Jean-Baptiste Onofré <jb...@nanthrax.net>
>> wrote:
>>
>>> Hi Rafael,
>>>
>>> It's a good point but I don't see nothing more to do on our side: if a
>>> emergency issue is detected, then we have to address it and release a
>>> fix release (x.y.z where z is the specific release fixing the issue).
>>> The commitment is a best effort as in all community: if an emergency
>>> issue is detected, qualified and accepted, then we do our best to
>>> provide a fix and do the fix release.
>>>
>>> So, for me, it's already handled.
>>>
>>> By the way, just a quick reminder in term of release:
>>>
>>> - now that gradle release seems ok, we resume our release cycle every ~
>>> 6 weeks
>>> - we can cut release anytime if required, especially to address
>>> emergency issues.
>>>
>>> Regards
>>> JB
>>>
>>> On 14/06/2018 22:33, Rafael Fernandez wrote:
>>> > Hi Beam devs,
>>> >
>>> > Emergencies can and will happen. As Apache Beam adoption continues to
>>> > grow, the user community will naturally expect the Beam developer
>>> > community to react to critical issues, such as security vulnerabilities
>>> > in our dependencies. I want to make sure the dev community is in
>>> > agreement that we follow the ASF Vulnerability Handling processes [1]
>>> > for such occurrences.
>>> >
>>> >
>>> > In addition, I'd like to discuss cases in which data correctness/loss
>>> > may warrant an expedited release (i.e., we did not wait 72 hours), as
>>> we
>>> > did in 2.1.1 [2]. Concretely:
>>> >
>>> >
>>> > 1.
>>> >
>>> > Do we need to add anything to our project website so the user
>>> > community knows how we react to such issues?
>>> >
>>> > 2.
>>> >
>>> > Should we have an entry in the contributor guide to address
>>> critical
>>> > point releases, so we eliminate any guesswork in the event of an
>>> > emergency? (Example text [3])
>>> >
>>> >
>>> > Thanks,
>>> >
>>> > r
>>> >
>>> >
>>> > [1]
>>> >
>>> > _https://apache.org/security/committers.html#vulnerability-handling_
>>> >
>>> > [2] https://lists.apache.org/list.html?dev@beam.apache.org:lte=
>>> 40M:2.1.1
>>> >
>>> > *
>>> >
>>> > [3] Example text for the contributor guideline:
>>> >
>>> >
>>> > What requires a critical point release?
>>> >
>>> > *
>>> >
>>> > A data loss bug
>>> >
>>> > *
>>> >
>>> > A data corruption bug
>>> >
>>> > *
>>> >
>>> > A processing correctness bug
>>> >
>>> > *
>>> >
>>> > For security vulnerabilities, please follow
>>> > https://apache.org/security/committers.html#vulnerability-handling
>>> .
>>> >
>>> >
>>> > What do we do a critical point release on?
>>> >
>>> > Our first priority is to stop the bleeding. We ought to prioritize a
>>> > point release for the latest Beam version, based on the release branch,
>>> > that cherrypicks only the intended fix.
>>> >
>>> > *
>>> >
>>> > We've done it before! Remember 2.1.1
>>> > <https://lists.apache.org/list.html?dev@beam.apache.org:
>>> lte=40M:2.1.1>?
>>> >
>>> > o
>>> >
>>> > Since this is a critical release, we can relax our usual 72
>>> hour
>>> > voting policy. It worked well for 2.1.1, we should make it
>>> > repeatable: Propose, have PMC folks do due diligence on the
>>> > request, and sign off. Since this is critical, we may want to
>>> > have more than one person working on the release.
>>> >
>>> > *
>>> >
>>> > Once we get it out, the community can discuss which previous
>>> > releases would benefit from a potential point release.
>>> >
>>> >
>>> > Who proposes a critical point release?
>>> >
>>> > Any member of the community. 3 PMC +1 votes are sufficient to get the
>>> > process rolling.
>>> >
>>> > *
>>> >
>>> >
>>> >
>>>
>>> --
>>> Jean-Baptiste Onofré
>>> jbonofre@apache.org
>>> http://blog.nanthrax.net
>>> Talend - http://www.talend.com
>>>
>>
Re: [DISCUSS] Releasing Beam in the presence of emergencies
Posted by Kenneth Knowles <kl...@google.com>.
I like the idea of explicitly affirming our use of the ASF Vulnerability
Policy as well as listing other classes of bugs that are particularly
critical for a project like Beam. I think the most valuable thing is having
a clear and concise policy for users to understand at a glance. Then we can
have a verbose walkthrough for someone implementing the policy.
Kenn
On Sat, Jun 16, 2018 at 9:54 AM Chamikara Jayalath <ch...@google.com>
wrote:
> While performing a patch release is the correct approach for a critical
> fixes I think there are still several points that we might want to discuss
> and formalize/document if needed.
>
> (1) What if there's an ongoing major/minor version release ?
> If think patch releases should be independent of any ongoing major/minor
> versions releases and should be handled by a separate release manager who
> can commit the the time to cut the release immediately.
>
> (2) What changes should go in ?
> Only the fix that addresses the critical issue should go into the patch
> release. Any other fixes should be held till the next minor version release.
>
> (3) What to do with the faulty release ?
> We should clearly document the issue with the faulty (non-patched) release
> in https://beam.apache.org/get-started/downloads/. This can be
> complemented by other announcements (to user list, twitter, for example)
> and runner specific solutions (rejecting jobs, warnings for running jobs).
>
> (4) Voting period and validation.
> We are performing the patch on top of an already released branch. So can
> we expedite the release validation and voting process ?
>
> I'm sure there are other points.
>
> Thanks,
> Cham
>
> On Thu, Jun 14, 2018 at 10:29 PM Jean-Baptiste Onofré <jb...@nanthrax.net>
> wrote:
>
>> Hi Rafael,
>>
>> It's a good point but I don't see nothing more to do on our side: if a
>> emergency issue is detected, then we have to address it and release a
>> fix release (x.y.z where z is the specific release fixing the issue).
>> The commitment is a best effort as in all community: if an emergency
>> issue is detected, qualified and accepted, then we do our best to
>> provide a fix and do the fix release.
>>
>> So, for me, it's already handled.
>>
>> By the way, just a quick reminder in term of release:
>>
>> - now that gradle release seems ok, we resume our release cycle every ~
>> 6 weeks
>> - we can cut release anytime if required, especially to address
>> emergency issues.
>>
>> Regards
>> JB
>>
>> On 14/06/2018 22:33, Rafael Fernandez wrote:
>> > Hi Beam devs,
>> >
>> > Emergencies can and will happen. As Apache Beam adoption continues to
>> > grow, the user community will naturally expect the Beam developer
>> > community to react to critical issues, such as security vulnerabilities
>> > in our dependencies. I want to make sure the dev community is in
>> > agreement that we follow the ASF Vulnerability Handling processes [1]
>> > for such occurrences.
>> >
>> >
>> > In addition, I'd like to discuss cases in which data correctness/loss
>> > may warrant an expedited release (i.e., we did not wait 72 hours), as we
>> > did in 2.1.1 [2]. Concretely:
>> >
>> >
>> > 1.
>> >
>> > Do we need to add anything to our project website so the user
>> > community knows how we react to such issues?
>> >
>> > 2.
>> >
>> > Should we have an entry in the contributor guide to address critical
>> > point releases, so we eliminate any guesswork in the event of an
>> > emergency? (Example text [3])
>> >
>> >
>> > Thanks,
>> >
>> > r
>> >
>> >
>> > [1]
>> >
>> > _https://apache.org/security/committers.html#vulnerability-handling_
>> >
>> > [2]
>> https://lists.apache.org/list.html?dev@beam.apache.org:lte=40M:2.1.1
>> >
>> > *
>> >
>> > [3] Example text for the contributor guideline:
>> >
>> >
>> > What requires a critical point release?
>> >
>> > *
>> >
>> > A data loss bug
>> >
>> > *
>> >
>> > A data corruption bug
>> >
>> > *
>> >
>> > A processing correctness bug
>> >
>> > *
>> >
>> > For security vulnerabilities, please follow
>> > https://apache.org/security/committers.html#vulnerability-handling
>> .
>> >
>> >
>> > What do we do a critical point release on?
>> >
>> > Our first priority is to stop the bleeding. We ought to prioritize a
>> > point release for the latest Beam version, based on the release branch,
>> > that cherrypicks only the intended fix.
>> >
>> > *
>> >
>> > We've done it before! Remember 2.1.1
>> > <
>> https://lists.apache.org/list.html?dev@beam.apache.org:lte=40M:2.1.1>?
>> >
>> > o
>> >
>> > Since this is a critical release, we can relax our usual 72 hour
>> > voting policy. It worked well for 2.1.1, we should make it
>> > repeatable: Propose, have PMC folks do due diligence on the
>> > request, and sign off. Since this is critical, we may want to
>> > have more than one person working on the release.
>> >
>> > *
>> >
>> > Once we get it out, the community can discuss which previous
>> > releases would benefit from a potential point release.
>> >
>> >
>> > Who proposes a critical point release?
>> >
>> > Any member of the community. 3 PMC +1 votes are sufficient to get the
>> > process rolling.
>> >
>> > *
>> >
>> >
>> >
>>
>> --
>> Jean-Baptiste Onofré
>> jbonofre@apache.org
>> http://blog.nanthrax.net
>> Talend - http://www.talend.com
>>
>
Re: [DISCUSS] Releasing Beam in the presence of emergencies
Posted by Chamikara Jayalath <ch...@google.com>.
While performing a patch release is the correct approach for a critical
fixes I think there are still several points that we might want to discuss
and formalize/document if needed.
(1) What if there's an ongoing major/minor version release ?
If think patch releases should be independent of any ongoing major/minor
versions releases and should be handled by a separate release manager who
can commit the the time to cut the release immediately.
(2) What changes should go in ?
Only the fix that addresses the critical issue should go into the patch
release. Any other fixes should be held till the next minor version release.
(3) What to do with the faulty release ?
We should clearly document the issue with the faulty (non-patched) release
in https://beam.apache.org/get-started/downloads/. This can be complemented
by other announcements (to user list, twitter, for example) and runner
specific solutions (rejecting jobs, warnings for running jobs).
(4) Voting period and validation.
We are performing the patch on top of an already released branch. So can we
expedite the release validation and voting process ?
I'm sure there are other points.
Thanks,
Cham
On Thu, Jun 14, 2018 at 10:29 PM Jean-Baptiste Onofré <jb...@nanthrax.net>
wrote:
> Hi Rafael,
>
> It's a good point but I don't see nothing more to do on our side: if a
> emergency issue is detected, then we have to address it and release a
> fix release (x.y.z where z is the specific release fixing the issue).
> The commitment is a best effort as in all community: if an emergency
> issue is detected, qualified and accepted, then we do our best to
> provide a fix and do the fix release.
>
> So, for me, it's already handled.
>
> By the way, just a quick reminder in term of release:
>
> - now that gradle release seems ok, we resume our release cycle every ~
> 6 weeks
> - we can cut release anytime if required, especially to address
> emergency issues.
>
> Regards
> JB
>
> On 14/06/2018 22:33, Rafael Fernandez wrote:
> > Hi Beam devs,
> >
> > Emergencies can and will happen. As Apache Beam adoption continues to
> > grow, the user community will naturally expect the Beam developer
> > community to react to critical issues, such as security vulnerabilities
> > in our dependencies. I want to make sure the dev community is in
> > agreement that we follow the ASF Vulnerability Handling processes [1]
> > for such occurrences.
> >
> >
> > In addition, I'd like to discuss cases in which data correctness/loss
> > may warrant an expedited release (i.e., we did not wait 72 hours), as we
> > did in 2.1.1 [2]. Concretely:
> >
> >
> > 1.
> >
> > Do we need to add anything to our project website so the user
> > community knows how we react to such issues?
> >
> > 2.
> >
> > Should we have an entry in the contributor guide to address critical
> > point releases, so we eliminate any guesswork in the event of an
> > emergency? (Example text [3])
> >
> >
> > Thanks,
> >
> > r
> >
> >
> > [1]
> >
> > _https://apache.org/security/committers.html#vulnerability-handling_
> >
> > [2] https://lists.apache.org/list.html?dev@beam.apache.org:lte=40M:2.1.1
> >
> > *
> >
> > [3] Example text for the contributor guideline:
> >
> >
> > What requires a critical point release?
> >
> > *
> >
> > A data loss bug
> >
> > *
> >
> > A data corruption bug
> >
> > *
> >
> > A processing correctness bug
> >
> > *
> >
> > For security vulnerabilities, please follow
> > https://apache.org/security/committers.html#vulnerability-handling .
> >
> >
> > What do we do a critical point release on?
> >
> > Our first priority is to stop the bleeding. We ought to prioritize a
> > point release for the latest Beam version, based on the release branch,
> > that cherrypicks only the intended fix.
> >
> > *
> >
> > We've done it before! Remember 2.1.1
> > <
> https://lists.apache.org/list.html?dev@beam.apache.org:lte=40M:2.1.1>?
> >
> > o
> >
> > Since this is a critical release, we can relax our usual 72 hour
> > voting policy. It worked well for 2.1.1, we should make it
> > repeatable: Propose, have PMC folks do due diligence on the
> > request, and sign off. Since this is critical, we may want to
> > have more than one person working on the release.
> >
> > *
> >
> > Once we get it out, the community can discuss which previous
> > releases would benefit from a potential point release.
> >
> >
> > Who proposes a critical point release?
> >
> > Any member of the community. 3 PMC +1 votes are sufficient to get the
> > process rolling.
> >
> > *
> >
> >
> >
>
> --
> Jean-Baptiste Onofré
> jbonofre@apache.org
> http://blog.nanthrax.net
> Talend - http://www.talend.com
>
Re: [DISCUSS] Releasing Beam in the presence of emergencies
Posted by Jean-Baptiste Onofré <jb...@nanthrax.net>.
Hi Rafael,
It's a good point but I don't see nothing more to do on our side: if a
emergency issue is detected, then we have to address it and release a
fix release (x.y.z where z is the specific release fixing the issue).
The commitment is a best effort as in all community: if an emergency
issue is detected, qualified and accepted, then we do our best to
provide a fix and do the fix release.
So, for me, it's already handled.
By the way, just a quick reminder in term of release:
- now that gradle release seems ok, we resume our release cycle every ~
6 weeks
- we can cut release anytime if required, especially to address
emergency issues.
Regards
JB
On 14/06/2018 22:33, Rafael Fernandez wrote:
> Hi Beam devs,
>
> Emergencies can and will happen. As Apache Beam adoption continues to
> grow, the user community will naturally expect the Beam developer
> community to react to critical issues, such as security vulnerabilities
> in our dependencies. I want to make sure the dev community is in
> agreement that we follow the ASF Vulnerability Handling processes [1]
> for such occurrences.
>
>
> In addition, I'd like to discuss cases in which data correctness/loss
> may warrant an expedited release (i.e., we did not wait 72 hours), as we
> did in 2.1.1 [2]. Concretely:
>
>
> 1.
>
> Do we need to add anything to our project website so the user
> community knows how we react to such issues?
>
> 2.
>
> Should we have an entry in the contributor guide to address critical
> point releases, so we eliminate any guesswork in the event of an
> emergency? (Example text [3])
>
>
> Thanks,
>
> r
>
>
> [1]
>
> _https://apache.org/security/committers.html#vulnerability-handling_
>
> [2] https://lists.apache.org/list.html?dev@beam.apache.org:lte=40M:2.1.1
>
> *
>
> [3] Example text for the contributor guideline:
>
>
> What requires a critical point release?
>
> *
>
> A data loss bug
>
> *
>
> A data corruption bug
>
> *
>
> A processing correctness bug
>
> *
>
> For security vulnerabilities, please follow
> https://apache.org/security/committers.html#vulnerability-handling .
>
>
> What do we do a critical point release on?
>
> Our first priority is to stop the bleeding. We ought to prioritize a
> point release for the latest Beam version, based on the release branch,
> that cherrypicks only the intended fix.
>
> *
>
> We've done it before! Remember 2.1.1
> <https://lists.apache.org/list.html?dev@beam.apache.org:lte=40M:2.1.1>?
>
> o
>
> Since this is a critical release, we can relax our usual 72 hour
> voting policy. It worked well for 2.1.1, we should make it
> repeatable: Propose, have PMC folks do due diligence on the
> request, and sign off. Since this is critical, we may want to
> have more than one person working on the release.
>
> *
>
> Once we get it out, the community can discuss which previous
> releases would benefit from a potential point release.
>
>
> Who proposes a critical point release?
>
> Any member of the community. 3 PMC +1 votes are sufficient to get the
> process rolling.
>
> *
>
>
>
--
Jean-Baptiste Onofré
jbonofre@apache.org
http://blog.nanthrax.net
Talend - http://www.talend.com