Re: Who can review security issues?

[Hen <ba...@apache.org>]

How did this go Jarek?

It does feel that we need something like this as currently there is no real
path over time for a researcher to contribute and become the one who
reviews reports from researchers.

On Wed, May 3, 2023 at 2:22 AM Jarek Potiuk <ja...@potiuk.com> wrote:

> This is a topic I was actively researching recently and I have some work in
> progress that might bring some improvements to the process for the ASF and
> for projects that - similarly as Airflow - struggle with handling an
> increasing number of security issues raised to us.
>
> TL;DR; We have an idea in Apache Airflow how to improve it that we are
> discussing in our PMC to also allow selected (and approved by PMC members)
> non-committers to be on the security list.
>
> This had some initial positive feedback among a few PMC members of ours and
> I am in a process of getting it reviewed, approved on the PMC/committer
> level and I am planning to lead implementation of it.
>
> I also discussed it with Mark and Arnout privately so far, and it looks
> like they are generally ok with it - where we might allow selected
> contributors who are not committers (yet) to be subscribed there under the
> same conditions of not disclosing security issues to others (And ICLA
> signed with the Foundation).
>
> We have a PR in progress to our security policy  describing it in more
> details https://github.com/apache/airflow/pull/30960/files - and so far we
> discuss it in private@ but soon we are going to present the proposal to
> the
> devlist.
>
> More context:
>
> The main point is that there are people who are not yet ready to be a
> committer - yet -  but there are various reasons why they might be helpful
> in solving security issues:
>
> - they have incentives or are security experts in areas we need to assess
> and fix issues
> - they can bring valuable insights from outside and specifically when they
> exclusively focus on security
> - they are paid to do general security work (often by those who are
> stakeholders in the project)
>
> The condition is that those people are also known to PMC members and they
> can recommend adding such a person in a vote.
>
> The idea is tha PMC still has full control over who is in the group (with
> voting) and that there is an expectation that such people will be actively
> participating, not only listening (lack of activity is going to be a
> condition to stay in the group). Active participation in such a group in
> our vision is - as I see it - a clear way to committership (or becoming
> member if you are already a committer)
>
> Limiting the group to existing  committers, I think, closes the way for
> outside security researchers and experts to become comitters in the
> first place. It's a bit of a chicken-egg problem. So far, In order to
> contribute (other than find and fix) to the security of the project, you
> need to be a PMC (or a committer at least if you have security@ group),
> but
> in order to become a committer (if you focus on security), you need to
> contribute to the security of the project. But contribution to security is
> also assessment of an impact, discussing severity, discussing ways how the
> issue can be solved, and most importantly - sometimes explaining to other
> participants of the discussion some intrinsic details of security issues
> raised because the way how the issue can be exploited is not-at-all
> obvious. All the 'discussion" part is currently closed to non-committers
> (who often are not security experts).
>
> This also comes from my personal experience with handling many security
> issues reported to us by security researchers:
>
> There are a number of people who are **just** focusing on a security work
> and they would not contribute in any other way, but the security area is
> important (and will be even more important with upcoming legislation
> changes)  IMHO when you limit security discussion to PMC and committers
> only, you are not opening up to such people. I actually have been
> approached by some of the security researchers who reported security issues
> and they wanted to know how they can become more active and help the
> project - so far I pointed them out to our committership rules and told
> them, they have to become committers first to be more involved on the
> security side - and so far this is as far as they got.
>
> J,.
>
>

Re: Who can review security issues?

[Jarek Potiuk <ja...@potiuk.com>]

Our talk about the learnings (with Arnout) has **just** been approved
for the Community track of the Community over Code NA in Halifax in
October, so we will share learnings then if you (or anyone else
interested) will be there :). I will try to make sure it is also
recorded.

But yes. I think we have mostly positive (also some negative or not
yet concluded) results and findings from following the process.
Arnout will likely have his own perspective as well.

My Good key points so far:

* since May we slashed more than half of our outstanding issues (20 to
less than 10). We are on a good path to get to 0 by the time of the
talk :) Most of the "really really really old" issues have been
addressed. We triage and discuss new issues rather quickly after they
come now - when we know that we will get a "live" discussion happening
on them and people who are involved.

* as result of the discussions of everyone involved -we not only fixed
the old issues but worked out and implemented a strategic
communication changes
(https://airflow.apache.org/docs/apache-airflow/stable/security/index.html#airflow-security-model-user-types),
code changes  and clarifications in our security model to address a
big class of security issues (also future ones). I think also we build
a good "shared" knowledge of what is and what is not a security issue
and how to handle them better.

* the tooling and clarity about the process of addressing issues is
absolutely a must - without it we would not be able to have "as
frictionless as possible" experience and be able to use the time of
all the people involved effectively - knowing what to do.

* I think key part of the process was (also not originally
anticipated) - effective involvement of release managers in the
process at the level of "mechanics of release" - the release managers
are mainly observing and following discussions and introduce some
improvements in the process that will let them follow the release
process pretty much "automatically" without having to make decisions
where they are not

* I think having an active "outside" researcher on the team who can
provide their perspective was a brilliant idea. I think the comments
of Kei - the security researcher we invited - helped us a LOT on our
decisions and we learned a ton of things. Many times we changed our
collective opinion on an issue listening to Kei. I think also being
"outside" helps to see things with the eyes of those who are not as
much vested in the project as we are and also see a lot of other
projects and "best practices" out there. This was something I had not
anticipated originally, but this was extremely valuable input IMHO.

My So-So/Bad/Jury Still Out points so far:

* it still needed a lot of time, dedication, persistence and effort of
someone leading it (it was me in this case) to make it happen - and
while we already achieve the point where discussions are happening
when I am on holidays (YAY), I think it required dedication and drive
of someone who had "let's get it working" as top-priority and time to
do so. Not sure if anything can be done here, or maybe it is part of
my personality and possibly some bias from my side (and maybe a little
too much expectations). But at least we have a clear process and we
have gone quite some times through it and I hope once we get into the
stage where we have no backlog ("no broken windows policy") this might
become self-sustainable and far less effort will be needed in
"driving" the process.

* when it comes to the engagement of people from security-related
teams of stakeholders (people who are not committers), the jury is
still out IMHO. It worked in some cases where people were motivated on
their own and saw it as part of their "commttership" journey (I am
about to discuss one of them to become a committer), but I am so far
observing how the engagement of those who promised it looks like, and
in a number of cases it is far less than I would have expected. I plan
to do a kind of retrospective/start the process of  "are you sure you
want to be here - you have not contributed much or nothing at all"
when I have more data points  -  so you need to be rather careful who
you invite from outside.

J

On Mon, Jul 31, 2023 at 3:09 AM Hen <ba...@apache.org> wrote:
>
> How did this go Jarek?
>
> It does feel that we need something like this as currently there is no real
> path over time for a researcher to contribute and become the one who
> reviews reports from researchers.
>
> On Wed, May 3, 2023 at 2:22 AM Jarek Potiuk <ja...@potiuk.com> wrote:
>
> > This is a topic I was actively researching recently and I have some work in
> > progress that might bring some improvements to the process for the ASF and
> > for projects that - similarly as Airflow - struggle with handling an
> > increasing number of security issues raised to us.
> >
> > TL;DR; We have an idea in Apache Airflow how to improve it that we are
> > discussing in our PMC to also allow selected (and approved by PMC members)
> > non-committers to be on the security list.
> >
> > This had some initial positive feedback among a few PMC members of ours and
> > I am in a process of getting it reviewed, approved on the PMC/committer
> > level and I am planning to lead implementation of it.
> >
> > I also discussed it with Mark and Arnout privately so far, and it looks
> > like they are generally ok with it - where we might allow selected
> > contributors who are not committers (yet) to be subscribed there under the
> > same conditions of not disclosing security issues to others (And ICLA
> > signed with the Foundation).
> >
> > We have a PR in progress to our security policy  describing it in more
> > details https://github.com/apache/airflow/pull/30960/files - and so far we
> > discuss it in private@ but soon we are going to present the proposal to
> > the
> > devlist.
> >
> > More context:
> >
> > The main point is that there are people who are not yet ready to be a
> > committer - yet -  but there are various reasons why they might be helpful
> > in solving security issues:
> >
> > - they have incentives or are security experts in areas we need to assess
> > and fix issues
> > - they can bring valuable insights from outside and specifically when they
> > exclusively focus on security
> > - they are paid to do general security work (often by those who are
> > stakeholders in the project)
> >
> > The condition is that those people are also known to PMC members and they
> > can recommend adding such a person in a vote.
> >
> > The idea is tha PMC still has full control over who is in the group (with
> > voting) and that there is an expectation that such people will be actively
> > participating, not only listening (lack of activity is going to be a
> > condition to stay in the group). Active participation in such a group in
> > our vision is - as I see it - a clear way to committership (or becoming
> > member if you are already a committer)
> >
> > Limiting the group to existing  committers, I think, closes the way for
> > outside security researchers and experts to become comitters in the
> > first place. It's a bit of a chicken-egg problem. So far, In order to
> > contribute (other than find and fix) to the security of the project, you
> > need to be a PMC (or a committer at least if you have security@ group),
> > but
> > in order to become a committer (if you focus on security), you need to
> > contribute to the security of the project. But contribution to security is
> > also assessment of an impact, discussing severity, discussing ways how the
> > issue can be solved, and most importantly - sometimes explaining to other
> > participants of the discussion some intrinsic details of security issues
> > raised because the way how the issue can be exploited is not-at-all
> > obvious. All the 'discussion" part is currently closed to non-committers
> > (who often are not security experts).
> >
> > This also comes from my personal experience with handling many security
> > issues reported to us by security researchers:
> >
> > There are a number of people who are **just** focusing on a security work
> > and they would not contribute in any other way, but the security area is
> > important (and will be even more important with upcoming legislation
> > changes)  IMHO when you limit security discussion to PMC and committers
> > only, you are not opening up to such people. I actually have been
> > approached by some of the security researchers who reported security issues
> > and they wanted to know how they can become more active and help the
> > project - so far I pointed them out to our committership rules and told
> > them, they have to become committers first to be more involved on the
> > security side - and so far this is as far as they got.
> >
> > J,.
> >
> >

---------------------------------------------------------------------
To unsubscribe, e-mail: security-discuss-unsubscribe@community.apache.org
For additional commands, e-mail: security-discuss-help@community.apache.org