You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@airflow.apache.org by Jarek Potiuk <ja...@potiuk.com> on 2023/05/09 06:31:25 UTC
[PROPOSAL] Solidifying and improving security handling process for Airflow
Hello everyone,
I have a proposal of how to improve and streamline our security issue
handling process.
We have been discussing it for some time in the private@airflow.apache.org
- since this is about security and by default those discussion are in
provate@, and we we also consulted it with security@apache.org and I have
the proposal that is described in this PR that I wanted to hear the
community opinion of:
https://github.com/apache/airflow/pull/30960
More context and some (presumably :D ) FAQs below:
* Why are we doing this now?
We recently started to receive a growing number of security related reports
(by security researchers). So far the security discussion and fixing has
happened between PMC members only, but we seem to need a bit more community
help on that.
* What problem do we want to solve?
This is a relatively small group, and there are some PMC members that are
not too active (naturally), as well sometimes we lack the expertise or time
to be able to properly focus, timely diagnose and promptly enough fix such
requests. And we think we should do better.
* What's the assumption we have here?
There is - conceptually - no problem to invite others to help us with
diagnosing and handling the security issues, especially that there are
people who specialize in those and we could invite such people on the base
of trust. There are stakeholders in Airflow that have their own security
teams already looking at Airflow Security and those people are security
experts, they are trusted but at the same time, they do not focus
exclusively on Airflow. There are also security experts who reported issues
in the past and they expressed their interest in helping in handling the
security issues as well.
* Why does the current PMC-only approach work in a suboptimal way?
We sometimes lack time or expertise to go deeply into security issues, It
would be difficult for people who are security experts mostly or
committers who would like to become PMC members and would like to help via
being more active in handling security issues, to get to the PMC before
they would be able to help. This is mostly a chicken-egg problem - someone
who could raise to be a committer or PMC by mostly focusing on the security
aspect of Airflow is a super valuable community member, but they can't help
currently if they are not PMC members. And they can't become a PMC member -
because they don't even know about those security issues they could help
with.
* What's the proposal?
We propose that we change the process by creating a dedicated, smaller and
focused security@airflow.apache.org team and by allowing for this group to
contain not only selected PMC members but also selected committers and
possibly even people who are not yet committers but who we have already an
interest in airflow, who PMC members will approve to join such a team on a
base of trust and expectation that they will be actively helping with
diagnosing and fixing the issues.
This is going to be entirely at the discretion of the PMC to approve such
people from outside of the PMC. This group will also always have release
managers so that they are aware of the security issues being solved and can
include them in announcements when we release new software
* What's in it for those who join the team?
Active participation in such a group will bring glory and fame (of course
:) ). As the nature and of the discussions and amount of contributions are
private so we are going to get the people involved credited as remediation
contributors in the announced CVEs. Of course also active participation is
a good way to quicken the path to become a committer and PMC member (see
the PR).
But there is expectation that the people in the group will be actively
helping in diagnosing and solving the issues and we will keep an eye on
that. That group will be small and focused and "just listening" right to
what happens there is reserved only for release managers whose job will be
to make sure all the fixed problems are announced,
Also - what's more important - there is an expectation of secrecy. Security
issues that are not yet announced should not be discussed in the open.
Security issues for which solutions are not yet public in the form of PR
should not be even hinted at to anyone - including employees of those
people. This is a great responsibility to bear.
And I personally (and here is my personal take) - find it really great to
be able to work on such security issues. They are often a bit
brain-teasers: one - to understand what the issue really is and how it can
be exploited, followed by a team discussion on how severe they are,
followed by finding a way to solve them to keep maximum backwards
compatibility possible (but sometimes necessarily breaking it). Since there
will be other security team members that you can bounce your ideas and
understanding of the issue off, this also allows you to learn about new
aspects of the security and get to know Airflow better in parts you had no
chance to look at. Fixing security issues when they are diagnosed and
discussed is usually fast and simple - often one-liner PRs. rarely
implementing bigger features. Which has the nice property of feeling
accomplishment quickly - as opposed to working on something bigger for
weeks.
Also - there is sometimes an opportunity to help others (for example Bolke
recently worked with FAB team and implemented Rate Limiting:
https://github.com/dpgaspar/Flask-AppBuilder/pull/1976 which I have
integrated in 2.6.0 : https://github.com/apache/airflow/pull/29766 so that
we could fix a long-standing issue with possibility of cracking
user's passwords by brute-force. So we also have a chance to collaborate
with others and help them at the same time as helping us - this has been
announced here https://nvd.nist.gov/vuln/detail/CVE-2023-29005 so it's not
our CVE, but we have been affected by it and it was actually us to push on
fixing it (which reminds me to ask Daniel from FAB to putt Bolke as
remediation developer in the CVE :D ).
* What are the next steps?
The idea is described in detail in the PR - feel free to comment there.
Essentially - if the idea gets generally positive feedback, we will create
such a team and we will either invite or accept a selected few people -
interested PMC members, but also committers and people who are not (yet)
committers but who we know have an interest in improving Airflow's
security. This group will start with invited people, but if you are
interested to join such a group and understand the obligations it brings -
feel free to write to private@airflow.apache.org and tell us why :). Then
we establish the group, merge the PR and the first thing to do will be to
agree on ways of collaboration in an efficient way using tools we have at
our disposal (and there are some good ideas and tools already). We have
some open issues raised to us that the group can start working on right
away, so it won't be boring.
J.
Re: [PROPOSAL] Solidifying and improving security handling process for Airflow
Posted by "Ferruzzi, Dennis" <fe...@amazon.com.INVALID>.
Neat. I like the idea of expanding the membership to folks who have expertise in such things. Seems to make sense to me, we can't all be experts in everything.
non-binding +1?
- ferruzzi
________________________________
From: Jarek Potiuk <ja...@potiuk.com>
Sent: Monday, May 8, 2023 11:31 PM
To: dev@airflow.apache.org
Cc: security
Subject: [EXTERNAL] [PROPOSAL] Solidifying and improving security handling process for Airflow
CAUTION: This email originated from outside of the organization. Do not click links or open attachments unless you can confirm the sender and know the content is safe.
Hello everyone,
I have a proposal of how to improve and streamline our security issue
handling process.
We have been discussing it for some time in the private@airflow.apache.org
- since this is about security and by default those discussion are in
provate@, and we we also consulted it with security@apache.org and I have
the proposal that is described in this PR that I wanted to hear the
community opinion of:
https://github.com/apache/airflow/pull/30960
More context and some (presumably :D ) FAQs below:
* Why are we doing this now?
We recently started to receive a growing number of security related reports
(by security researchers). So far the security discussion and fixing has
happened between PMC members only, but we seem to need a bit more community
help on that.
* What problem do we want to solve?
This is a relatively small group, and there are some PMC members that are
not too active (naturally), as well sometimes we lack the expertise or time
to be able to properly focus, timely diagnose and promptly enough fix such
requests. And we think we should do better.
* What's the assumption we have here?
There is - conceptually - no problem to invite others to help us with
diagnosing and handling the security issues, especially that there are
people who specialize in those and we could invite such people on the base
of trust. There are stakeholders in Airflow that have their own security
teams already looking at Airflow Security and those people are security
experts, they are trusted but at the same time, they do not focus
exclusively on Airflow. There are also security experts who reported issues
in the past and they expressed their interest in helping in handling the
security issues as well.
* Why does the current PMC-only approach work in a suboptimal way?
We sometimes lack time or expertise to go deeply into security issues, It
would be difficult for people who are security experts mostly or
committers who would like to become PMC members and would like to help via
being more active in handling security issues, to get to the PMC before
they would be able to help. This is mostly a chicken-egg problem - someone
who could raise to be a committer or PMC by mostly focusing on the security
aspect of Airflow is a super valuable community member, but they can't help
currently if they are not PMC members. And they can't become a PMC member -
because they don't even know about those security issues they could help
with.
* What's the proposal?
We propose that we change the process by creating a dedicated, smaller and
focused security@airflow.apache.org team and by allowing for this group to
contain not only selected PMC members but also selected committers and
possibly even people who are not yet committers but who we have already an
interest in airflow, who PMC members will approve to join such a team on a
base of trust and expectation that they will be actively helping with
diagnosing and fixing the issues.
This is going to be entirely at the discretion of the PMC to approve such
people from outside of the PMC. This group will also always have release
managers so that they are aware of the security issues being solved and can
include them in announcements when we release new software
* What's in it for those who join the team?
Active participation in such a group will bring glory and fame (of course
:) ). As the nature and of the discussions and amount of contributions are
private so we are going to get the people involved credited as remediation
contributors in the announced CVEs. Of course also active participation is
a good way to quicken the path to become a committer and PMC member (see
the PR).
But there is expectation that the people in the group will be actively
helping in diagnosing and solving the issues and we will keep an eye on
that. That group will be small and focused and "just listening" right to
what happens there is reserved only for release managers whose job will be
to make sure all the fixed problems are announced,
Also - what's more important - there is an expectation of secrecy. Security
issues that are not yet announced should not be discussed in the open.
Security issues for which solutions are not yet public in the form of PR
should not be even hinted at to anyone - including employees of those
people. This is a great responsibility to bear.
And I personally (and here is my personal take) - find it really great to
be able to work on such security issues. They are often a bit
brain-teasers: one - to understand what the issue really is and how it can
be exploited, followed by a team discussion on how severe they are,
followed by finding a way to solve them to keep maximum backwards
compatibility possible (but sometimes necessarily breaking it). Since there
will be other security team members that you can bounce your ideas and
understanding of the issue off, this also allows you to learn about new
aspects of the security and get to know Airflow better in parts you had no
chance to look at. Fixing security issues when they are diagnosed and
discussed is usually fast and simple - often one-liner PRs. rarely
implementing bigger features. Which has the nice property of feeling
accomplishment quickly - as opposed to working on something bigger for
weeks.
Also - there is sometimes an opportunity to help others (for example Bolke
recently worked with FAB team and implemented Rate Limiting:
https://github.com/dpgaspar/Flask-AppBuilder/pull/1976 which I have
integrated in 2.6.0 : https://github.com/apache/airflow/pull/29766 so that
we could fix a long-standing issue with possibility of cracking
user's passwords by brute-force. So we also have a chance to collaborate
with others and help them at the same time as helping us - this has been
announced here https://nvd.nist.gov/vuln/detail/CVE-2023-29005 so it's not
our CVE, but we have been affected by it and it was actually us to push on
fixing it (which reminds me to ask Daniel from FAB to putt Bolke as
remediation developer in the CVE :D ).
* What are the next steps?
The idea is described in detail in the PR - feel free to comment there.
Essentially - if the idea gets generally positive feedback, we will create
such a team and we will either invite or accept a selected few people -
interested PMC members, but also committers and people who are not (yet)
committers but who we know have an interest in improving Airflow's
security. This group will start with invited people, but if you are
interested to join such a group and understand the obligations it brings -
feel free to write to private@airflow.apache.org and tell us why :). Then
we establish the group, merge the PR and the first thing to do will be to
agree on ways of collaboration in an efficient way using tools we have at
our disposal (and there are some good ideas and tools already). We have
some open issues raised to us that the group can start working on right
away, so it won't be boring.
J.