You are viewing a plain text version of this content. The canonical link for it is here.
Posted to security-discuss@community.apache.org by Jonathan Leitschuh <jo...@gmail.com> on 2022/02/04 19:48:55 UTC
Security Researchers and The ASF Vulnerability Disclosure Processes
To the Apache Software Foundation Security Discussion Group,
Hello, my name is Jonathan Leitschuh. I’m an OSS Security Researcher and
recipient of the first-ever Dan Kaminsky Fellowship
<https://www.humansecurity.com/blog/our-first-dan-kaminsky-fellow>. I’m
reaching out today to discuss some issues I’ve had in the past with the ASF
regarding vulnerability disclosure and coordinating that disclosure.
First off, I want to communicate that I clearly understand that in OSS,
nobody is owed anything, including when engaging with a volunteer run
foundation like the ASF. All software is provided as-is and OSS maintainers
are under no obligation to fix any bug or vulnerability identified by the
wilder OSS community.
In that same light, for a security researcher, the easiest and most
straightforward way to get a vulnerability resolved is to publicly disclose
the vulnerability via full-disclosure. However, doing so allows attackers
time to leverage these vulnerabilities to exploit end-users. In general, I
prefer to avoid this option if possible and would rather coordinate
disclosure with the maintainers.
Mark Thomas recently communicated to me:
> As a volunteer org providing stewardship for 350+ projects and 200+
million lines of code we do not have the resources to implement a bespoke
disclosure process for each reporter.
On my side, I’m looking for vulnerabilities that are endemic to the OSS
ecosystem. As a part of the Dan Kaminsky Fellowship, my mission for the
next year is to work to eliminate entire classes of security
vulnerabilities from OSS leveraging CodeQL and bulk PR generation.
Since I’m trying to work on these endemic problems, I’m in an inverted
situation at a similar scale. I’m looking at hundreds or thousands of
vulnerabilities with a smattering of disclosure channels per organization,
between email, TalkTalk, Google Groups, Hacker1, BugCrowd, Google’s
Disclosure Process via Monorail, ect... keeping track of all these
disclosure channels and the various disclosure states is overwhelming. As a
result of my security research, I’m usually looking at hundreds or
thousands of impacted OSS projects and I have to be selective about the
projects I disclose to.
As a result of my experience, not only with the ASF, but other
organizations and OSS projects, I’ve decided to change my disclosure
process to centrally locate my vulnerability disclosure. It is my intention
to establish automation around disclosure of vulnerabilities to
consistently enforce my 90-day disclosure timeline. These new policies can
be found here:
https://github.com/JLLeitschuh/security-research/blob/main/README.md
These new policies I’ve established have already caused some conflict. I
hope that this email helps explain that my new policies and practices are
coming from a place of frustration with the current ASF process, not that
I’m trying to intentionally upset anyone. I don’t hold any grudges against
anyone specifically at the ASF, but I have struggled enough with the ASF
process that I’ve thrown up my hands and decided to forge my own disclosure
path.
I’ve been told that the ASF has a “if it didn’t happen in email, it didn’t
happen” culture, which I understand and respect. I offer the ASF the
following suggestion: set up a dummy GitHub account that gets notifications
sent to security@apache.org. That way, The ASF can get a nice searchable
papertrail which will work with your existing infrastructure.
Here are the core set of issues I’ve experienced surrounding disclosure
regarding the ASF:
-
Emails being sinkholed without any response for months
-
Lack of collaboration/communication with the original reporter
-
Not verifying fixes are sufficient with the original reporter
-
Lack of collaboration with the original reporter with respect to the
vulnerability disclosure contents & CVE body
-
Disclosures lack important information like the signatures of methods
that are impacted or REST endpoints that are vulnerable.
-
Disclosures lack links to the commits that actually resolve the issue.
-
Vulnerabilities not being handled within the previously communicated
disclosure timeline.
-
The ASF security team siding with the ASF project maintainers over the
researcher regarding vulnerability presence/impact when a vulnerability is
clearly present.
The ASF has an established Vulnerability Handling Process:
https://www.apache.org/security/committers.html
However, looking through my email today I very quickly found examples of
cases where the ASF had not followed this process. In particular:
-
CVE-2020-35451: Apache Oozie - Skipped steps 12 & 13
-
CVE-2021-36151: Apache Goblin - Skipped steps 5, 6, 7, 12, & 13
In this case, the vulnerability wasn’t ever even confirmed by anyone.
I did receive an apology from the ASF security team regarding these issues,
which I greatly appreciate, however they are good examples of the problems
I’ve experienced with the ASF.
Net result from my previous interaction with the ASF: I feel disrespected
and ignored and feel like my contributions aren't valued. This has led me
to a feeling of resentment towards the ASF and an overall feeling of
distrust towards the established ASF vulnerability handling process.
The point of this email is to explain my perspective and feelings of
frustration I’ve experienced with the ASF. Hopefully through this dialogue
a constructive outcome can be achieved.
Respectfully,
Jonathan Leitschuh
Dan Kaminsky Fellowship
Re: Security Researchers and The ASF Vulnerability Disclosure Processes
Posted by Hen <ba...@apache.org>.
Updating after chatting with Jonathan. I was misunderstanding "written for
public consumption" and thinking his step 1 was in public :)
On Mon, Mar 14, 2022 at 11:57 AM Hen <ba...@apache.org> wrote:
>
> Apologies for my subsequent cluelessness/over simplification.
>
> If I understand the thread correctly, the ASF has not been hitting the bar
> on every security issue, and Jonathan is going to move to a low-detail-zero
> day process followed 90 days later by the full details (though presumably
> that is due to projects in general rather than just the ASF's influence).
> Sounds like it's going to be very painful for all involved, but I
> understand the frustration leading to it.
>
> Getting 200 semi-independent groups to follow the same process will always
> be challenging; but failing to do so also has repercussions. It feels like
> the only solution is more investment in managing each security incident
> (with huge kudos to the folk who do this today), possibly paid.
>
> I know we have the process for a security incident defined from a
> project's point of view. If we assume that someone central is managing each
> item, can we define the process from the central-management point of view
> to help scale the current folk more by making it a more clearly defined
> role?
>
> Hen
>
> On Fri, Feb 4, 2022 at 2:26 PM Mark J Cox <mj...@apache.org> wrote:
>
>> On Fri, Feb 4, 2022 at 7:49 PM Jonathan Leitschuh <
>> jonathan.leitschuh@gmail.com> wrote:
>>
>> > To the Apache Software Foundation Security Discussion Group,
>> >
>> > Hello, my name is Jonathan Leitschuh. I’m an OSS Security Researcher and
>> > recipient of the first-ever Dan Kaminsky Fellowship
>> > <https://www.humansecurity.com/blog/our-first-dan-kaminsky-fellow>. I’m
>> > reaching out today to discuss some issues I’ve had in the past with the
>> ASF
>> > regarding vulnerability disclosure and coordinating that disclosure.
>> >
>> > First off, I want to communicate that I clearly understand that in OSS,
>> > nobody is owed anything, including when engaging with a volunteer run
>> > foundation like the ASF. All software is provided as-is and OSS
>> maintainers
>> > are under no obligation to fix any bug or vulnerability identified by
>> the
>> > wilder OSS community.
>> >
>> > In that same light, for a security researcher, the easiest and most
>> > straightforward way to get a vulnerability resolved is to publicly
>> disclose
>> > the vulnerability via full-disclosure. However, doing so allows
>> attackers
>> > time to leverage these vulnerabilities to exploit end-users. In
>> general, I
>> > prefer to avoid this option if possible and would rather coordinate
>> > disclosure with the maintainers.
>> >
>> > Mark Thomas recently communicated to me:
>> >
>> > > As a volunteer org providing stewardship for 350+ projects and 200+
>> > million lines of code we do not have the resources to implement a
>> bespoke
>> > disclosure process for each reporter.
>> >
>> > On my side, I’m looking for vulnerabilities that are endemic to the OSS
>> > ecosystem. As a part of the Dan Kaminsky Fellowship, my mission for the
>> > next year is to work to eliminate entire classes of security
>> > vulnerabilities from OSS leveraging CodeQL and bulk PR generation.
>> >
>> > Since I’m trying to work on these endemic problems, I’m in an inverted
>> > situation at a similar scale. I’m looking at hundreds or thousands of
>> > vulnerabilities with a smattering of disclosure channels per
>> organization,
>> > between email, TalkTalk, Google Groups, Hacker1, BugCrowd, Google’s
>> > Disclosure Process via Monorail, ect... keeping track of all these
>> > disclosure channels and the various disclosure states is overwhelming.
>> As a
>> > result of my security research, I’m usually looking at hundreds or
>> > thousands of impacted OSS projects and I have to be selective about the
>> > projects I disclose to.
>> >
>> > As a result of my experience, not only with the ASF, but other
>> > organizations and OSS projects, I’ve decided to change my disclosure
>> > process to centrally locate my vulnerability disclosure. It is my
>> intention
>> > to establish automation around disclosure of vulnerabilities to
>> > consistently enforce my 90-day disclosure timeline. These new policies
>> can
>> > be found here:
>> >
>> > https://github.com/JLLeitschuh/security-research/blob/main/README.md
>> >
>> > These new policies I’ve established have already caused some conflict. I
>> > hope that this email helps explain that my new policies and practices
>> are
>> > coming from a place of frustration with the current ASF process, not
>> that
>> > I’m trying to intentionally upset anyone. I don’t hold any grudges
>> against
>> > anyone specifically at the ASF, but I have struggled enough with the ASF
>> > process that I’ve thrown up my hands and decided to forge my own
>> disclosure
>> > path.
>> >
>> > I’ve been told that the ASF has a “if it didn’t happen in email, it
>> didn’t
>> > happen” culture, which I understand and respect. I offer the ASF the
>> > following suggestion: set up a dummy GitHub account that gets
>> notifications
>> > sent to security@apache.org. That way, The ASF can get a nice
>> searchable
>> > papertrail which will work with your existing infrastructure.
>> >
>> > Here are the core set of issues I’ve experienced surrounding disclosure
>> > regarding the ASF:
>> >
>> >
>> > -
>> >
>> > Emails being sinkholed without any response for months
>> > -
>> >
>> > Lack of collaboration/communication with the original reporter
>> > -
>> >
>> > Not verifying fixes are sufficient with the original reporter
>> > -
>> >
>> > Lack of collaboration with the original reporter with respect to the
>> > vulnerability disclosure contents & CVE body
>> > -
>> >
>> > Disclosures lack important information like the signatures of methods
>> > that are impacted or REST endpoints that are vulnerable.
>> > -
>> >
>> > Disclosures lack links to the commits that actually resolve the
>> issue.
>> > -
>> >
>> > Vulnerabilities not being handled within the previously communicated
>> > disclosure timeline.
>> > -
>> >
>> > The ASF security team siding with the ASF project maintainers over
>> the
>> > researcher regarding vulnerability presence/impact when a
>> vulnerability
>> > is
>> > clearly present.
>> >
>> >
>> >
>> > The ASF has an established Vulnerability Handling Process:
>> > https://www.apache.org/security/committers.html
>> >
>> > However, looking through my email today I very quickly found examples of
>> > cases where the ASF had not followed this process. In particular:
>> >
>> > -
>> >
>> > CVE-2020-35451: Apache Oozie - Skipped steps 12 & 13
>> > -
>> >
>> > CVE-2021-36151: Apache Goblin - Skipped steps 5, 6, 7, 12, & 13
>> >
>> > In this case, the vulnerability wasn’t ever even confirmed by anyone.
>> >
>> > I did receive an apology from the ASF security team regarding these
>> issues,
>> > which I greatly appreciate, however they are good examples of the
>> problems
>> > I’ve experienced with the ASF.
>> >
>> > Net result from my previous interaction with the ASF: I feel
>> disrespected
>> > and ignored and feel like my contributions aren't valued. This has led
>> me
>> > to a feeling of resentment towards the ASF and an overall feeling of
>> > distrust towards the established ASF vulnerability handling process.
>> >
>> > The point of this email is to explain my perspective and feelings of
>> > frustration I’ve experienced with the ASF. Hopefully through this
>> dialogue
>> > a constructive outcome can be achieved.
>> >
>> > Respectfully,
>> >
>> > Jonathan Leitschuh
>> >
>> > Dan Kaminsky Fellowship
>> >
>>
>> Hi Jonathan;
>>
>> We certainly have appreciated your reports over the years which have led
>> to
>> a number of security updates across our projects. One of the reasons we
>> set up this public list was so that we had a place we could work on
>> process
>> and policy concerns in a public forum not just limited to the private
>> security teams.
>>
>> You called out a few examples of concerns in issues you’ve submitted; one
>> of which was prior to a change to our process last year specifically so
>> that issues sent to security@apache.org get a confirmation now from the
>> security team rather than from the project team, and so ensuring that the
>> security team can be the point of contact if there is any break-down in
>> communication. We do deal with over 400 reports a year from researchers
>> and your experience isn’t what we strive to accomplish. We’ve already
>> passed on your comments to the projects mentioned. In general, where you
>> have concerns with the handling of any issue, the best way to raise the
>> concern is with a follow-up message on the email thread for that report,
>> or
>> (after an issue is public) on the appropriate dev list so it reaches
>> everyone involved in the project development (Our projects don't all
>> monitor this list).
>>
>> An issue that affects a wide range of projects is always tricky to
>> co-ordinate, where we have to bring many different project teams all with
>> their own management, release cycles, and communities into the embargoed
>> discussion, that’s pretty hard to automate and requires a lot of manual
>> effort, so for these types of multiple-affected-project issues we can
>> certainly look at ways that will mutually work for both researchers and
>> the
>> ASF projects without creating exceptions or a separate process for every
>> case.
>>
>> Regards, Mark
>>
>
Re: Security Researchers and The ASF Vulnerability Disclosure Processes
Posted by Mark J Cox <mj...@apache.org>.
To respond to your later point:
On Mon, Mar 14, 2022 at 6:59 PM Hen <ba...@apache.org> wrote:
> I know we have the process for a security incident defined from a project's
> point of view. If we assume that someone central is managing each item, can
> we define the process from the central-management point of view to help
> scale the current folk more by making it a more clearly defined role?
We do have a defined internal workflow process for the folks who work on
ASF Security Handling and it's based very much around a "Getting Things
Done" (David Allen) perspective where we're clear what the "next action" is
on every issue we're tracking. (The doc is internal just because it's
pretty much all about the mechanics of using our internal tools and
mailboxes to support the workflow). That means we can swap between
security handlers and it's obvious what the next state is and what needs an
action. We do refine that as we learn from issues and from looking at the
yearly reports we write. We've also recently had more ASF members join the
team and help out handling issues, which has freed up time to spend going
through the open issues on a regular basis ("Weekly review") and helping
projects respond to them. I'd personally say that the biggest room for
improvement is that we tend to concentrate more on critical and important
issues, so things that are pretty low risk and low severity in projects
that are busy or short of resources can tend to linger around for too
long. It is often the lowest risk issues that require the most work!
There's a good list of some things we want to do better called out in the
Brainstorm doc.
Mark
Re: Security Researchers and The ASF Vulnerability Disclosure Processes
Posted by Hen <ba...@apache.org>.
Apologies for my subsequent cluelessness/over simplification.
If I understand the thread correctly, the ASF has not been hitting the bar
on every security issue, and Jonathan is going to move to a low-detail-zero
day process followed 90 days later by the full details (though presumably
that is due to projects in general rather than just the ASF's influence).
Sounds like it's going to be very painful for all involved, but I
understand the frustration leading to it.
Getting 200 semi-independent groups to follow the same process will always
be challenging; but failing to do so also has repercussions. It feels like
the only solution is more investment in managing each security incident
(with huge kudos to the folk who do this today), possibly paid.
I know we have the process for a security incident defined from a project's
point of view. If we assume that someone central is managing each item, can
we define the process from the central-management point of view to help
scale the current folk more by making it a more clearly defined role?
Hen
On Fri, Feb 4, 2022 at 2:26 PM Mark J Cox <mj...@apache.org> wrote:
> On Fri, Feb 4, 2022 at 7:49 PM Jonathan Leitschuh <
> jonathan.leitschuh@gmail.com> wrote:
>
> > To the Apache Software Foundation Security Discussion Group,
> >
> > Hello, my name is Jonathan Leitschuh. I’m an OSS Security Researcher and
> > recipient of the first-ever Dan Kaminsky Fellowship
> > <https://www.humansecurity.com/blog/our-first-dan-kaminsky-fellow>. I’m
> > reaching out today to discuss some issues I’ve had in the past with the
> ASF
> > regarding vulnerability disclosure and coordinating that disclosure.
> >
> > First off, I want to communicate that I clearly understand that in OSS,
> > nobody is owed anything, including when engaging with a volunteer run
> > foundation like the ASF. All software is provided as-is and OSS
> maintainers
> > are under no obligation to fix any bug or vulnerability identified by the
> > wilder OSS community.
> >
> > In that same light, for a security researcher, the easiest and most
> > straightforward way to get a vulnerability resolved is to publicly
> disclose
> > the vulnerability via full-disclosure. However, doing so allows attackers
> > time to leverage these vulnerabilities to exploit end-users. In general,
> I
> > prefer to avoid this option if possible and would rather coordinate
> > disclosure with the maintainers.
> >
> > Mark Thomas recently communicated to me:
> >
> > > As a volunteer org providing stewardship for 350+ projects and 200+
> > million lines of code we do not have the resources to implement a bespoke
> > disclosure process for each reporter.
> >
> > On my side, I’m looking for vulnerabilities that are endemic to the OSS
> > ecosystem. As a part of the Dan Kaminsky Fellowship, my mission for the
> > next year is to work to eliminate entire classes of security
> > vulnerabilities from OSS leveraging CodeQL and bulk PR generation.
> >
> > Since I’m trying to work on these endemic problems, I’m in an inverted
> > situation at a similar scale. I’m looking at hundreds or thousands of
> > vulnerabilities with a smattering of disclosure channels per
> organization,
> > between email, TalkTalk, Google Groups, Hacker1, BugCrowd, Google’s
> > Disclosure Process via Monorail, ect... keeping track of all these
> > disclosure channels and the various disclosure states is overwhelming.
> As a
> > result of my security research, I’m usually looking at hundreds or
> > thousands of impacted OSS projects and I have to be selective about the
> > projects I disclose to.
> >
> > As a result of my experience, not only with the ASF, but other
> > organizations and OSS projects, I’ve decided to change my disclosure
> > process to centrally locate my vulnerability disclosure. It is my
> intention
> > to establish automation around disclosure of vulnerabilities to
> > consistently enforce my 90-day disclosure timeline. These new policies
> can
> > be found here:
> >
> > https://github.com/JLLeitschuh/security-research/blob/main/README.md
> >
> > These new policies I’ve established have already caused some conflict. I
> > hope that this email helps explain that my new policies and practices are
> > coming from a place of frustration with the current ASF process, not that
> > I’m trying to intentionally upset anyone. I don’t hold any grudges
> against
> > anyone specifically at the ASF, but I have struggled enough with the ASF
> > process that I’ve thrown up my hands and decided to forge my own
> disclosure
> > path.
> >
> > I’ve been told that the ASF has a “if it didn’t happen in email, it
> didn’t
> > happen” culture, which I understand and respect. I offer the ASF the
> > following suggestion: set up a dummy GitHub account that gets
> notifications
> > sent to security@apache.org. That way, The ASF can get a nice searchable
> > papertrail which will work with your existing infrastructure.
> >
> > Here are the core set of issues I’ve experienced surrounding disclosure
> > regarding the ASF:
> >
> >
> > -
> >
> > Emails being sinkholed without any response for months
> > -
> >
> > Lack of collaboration/communication with the original reporter
> > -
> >
> > Not verifying fixes are sufficient with the original reporter
> > -
> >
> > Lack of collaboration with the original reporter with respect to the
> > vulnerability disclosure contents & CVE body
> > -
> >
> > Disclosures lack important information like the signatures of methods
> > that are impacted or REST endpoints that are vulnerable.
> > -
> >
> > Disclosures lack links to the commits that actually resolve the issue.
> > -
> >
> > Vulnerabilities not being handled within the previously communicated
> > disclosure timeline.
> > -
> >
> > The ASF security team siding with the ASF project maintainers over the
> > researcher regarding vulnerability presence/impact when a
> vulnerability
> > is
> > clearly present.
> >
> >
> >
> > The ASF has an established Vulnerability Handling Process:
> > https://www.apache.org/security/committers.html
> >
> > However, looking through my email today I very quickly found examples of
> > cases where the ASF had not followed this process. In particular:
> >
> > -
> >
> > CVE-2020-35451: Apache Oozie - Skipped steps 12 & 13
> > -
> >
> > CVE-2021-36151: Apache Goblin - Skipped steps 5, 6, 7, 12, & 13
> >
> > In this case, the vulnerability wasn’t ever even confirmed by anyone.
> >
> > I did receive an apology from the ASF security team regarding these
> issues,
> > which I greatly appreciate, however they are good examples of the
> problems
> > I’ve experienced with the ASF.
> >
> > Net result from my previous interaction with the ASF: I feel disrespected
> > and ignored and feel like my contributions aren't valued. This has led me
> > to a feeling of resentment towards the ASF and an overall feeling of
> > distrust towards the established ASF vulnerability handling process.
> >
> > The point of this email is to explain my perspective and feelings of
> > frustration I’ve experienced with the ASF. Hopefully through this
> dialogue
> > a constructive outcome can be achieved.
> >
> > Respectfully,
> >
> > Jonathan Leitschuh
> >
> > Dan Kaminsky Fellowship
> >
>
> Hi Jonathan;
>
> We certainly have appreciated your reports over the years which have led to
> a number of security updates across our projects. One of the reasons we
> set up this public list was so that we had a place we could work on process
> and policy concerns in a public forum not just limited to the private
> security teams.
>
> You called out a few examples of concerns in issues you’ve submitted; one
> of which was prior to a change to our process last year specifically so
> that issues sent to security@apache.org get a confirmation now from the
> security team rather than from the project team, and so ensuring that the
> security team can be the point of contact if there is any break-down in
> communication. We do deal with over 400 reports a year from researchers
> and your experience isn’t what we strive to accomplish. We’ve already
> passed on your comments to the projects mentioned. In general, where you
> have concerns with the handling of any issue, the best way to raise the
> concern is with a follow-up message on the email thread for that report, or
> (after an issue is public) on the appropriate dev list so it reaches
> everyone involved in the project development (Our projects don't all
> monitor this list).
>
> An issue that affects a wide range of projects is always tricky to
> co-ordinate, where we have to bring many different project teams all with
> their own management, release cycles, and communities into the embargoed
> discussion, that’s pretty hard to automate and requires a lot of manual
> effort, so for these types of multiple-affected-project issues we can
> certainly look at ways that will mutually work for both researchers and the
> ASF projects without creating exceptions or a separate process for every
> case.
>
> Regards, Mark
>
Re: Security Researchers and The ASF Vulnerability Disclosure Processes
Posted by Mark J Cox <mj...@apache.org>.
On Fri, Feb 4, 2022 at 7:49 PM Jonathan Leitschuh <
jonathan.leitschuh@gmail.com> wrote:
> To the Apache Software Foundation Security Discussion Group,
>
> Hello, my name is Jonathan Leitschuh. I’m an OSS Security Researcher and
> recipient of the first-ever Dan Kaminsky Fellowship
> <https://www.humansecurity.com/blog/our-first-dan-kaminsky-fellow>. I’m
> reaching out today to discuss some issues I’ve had in the past with the ASF
> regarding vulnerability disclosure and coordinating that disclosure.
>
> First off, I want to communicate that I clearly understand that in OSS,
> nobody is owed anything, including when engaging with a volunteer run
> foundation like the ASF. All software is provided as-is and OSS maintainers
> are under no obligation to fix any bug or vulnerability identified by the
> wilder OSS community.
>
> In that same light, for a security researcher, the easiest and most
> straightforward way to get a vulnerability resolved is to publicly disclose
> the vulnerability via full-disclosure. However, doing so allows attackers
> time to leverage these vulnerabilities to exploit end-users. In general, I
> prefer to avoid this option if possible and would rather coordinate
> disclosure with the maintainers.
>
> Mark Thomas recently communicated to me:
>
> > As a volunteer org providing stewardship for 350+ projects and 200+
> million lines of code we do not have the resources to implement a bespoke
> disclosure process for each reporter.
>
> On my side, I’m looking for vulnerabilities that are endemic to the OSS
> ecosystem. As a part of the Dan Kaminsky Fellowship, my mission for the
> next year is to work to eliminate entire classes of security
> vulnerabilities from OSS leveraging CodeQL and bulk PR generation.
>
> Since I’m trying to work on these endemic problems, I’m in an inverted
> situation at a similar scale. I’m looking at hundreds or thousands of
> vulnerabilities with a smattering of disclosure channels per organization,
> between email, TalkTalk, Google Groups, Hacker1, BugCrowd, Google’s
> Disclosure Process via Monorail, ect... keeping track of all these
> disclosure channels and the various disclosure states is overwhelming. As a
> result of my security research, I’m usually looking at hundreds or
> thousands of impacted OSS projects and I have to be selective about the
> projects I disclose to.
>
> As a result of my experience, not only with the ASF, but other
> organizations and OSS projects, I’ve decided to change my disclosure
> process to centrally locate my vulnerability disclosure. It is my intention
> to establish automation around disclosure of vulnerabilities to
> consistently enforce my 90-day disclosure timeline. These new policies can
> be found here:
>
> https://github.com/JLLeitschuh/security-research/blob/main/README.md
>
> These new policies I’ve established have already caused some conflict. I
> hope that this email helps explain that my new policies and practices are
> coming from a place of frustration with the current ASF process, not that
> I’m trying to intentionally upset anyone. I don’t hold any grudges against
> anyone specifically at the ASF, but I have struggled enough with the ASF
> process that I’ve thrown up my hands and decided to forge my own disclosure
> path.
>
> I’ve been told that the ASF has a “if it didn’t happen in email, it didn’t
> happen” culture, which I understand and respect. I offer the ASF the
> following suggestion: set up a dummy GitHub account that gets notifications
> sent to security@apache.org. That way, The ASF can get a nice searchable
> papertrail which will work with your existing infrastructure.
>
> Here are the core set of issues I’ve experienced surrounding disclosure
> regarding the ASF:
>
>
> -
>
> Emails being sinkholed without any response for months
> -
>
> Lack of collaboration/communication with the original reporter
> -
>
> Not verifying fixes are sufficient with the original reporter
> -
>
> Lack of collaboration with the original reporter with respect to the
> vulnerability disclosure contents & CVE body
> -
>
> Disclosures lack important information like the signatures of methods
> that are impacted or REST endpoints that are vulnerable.
> -
>
> Disclosures lack links to the commits that actually resolve the issue.
> -
>
> Vulnerabilities not being handled within the previously communicated
> disclosure timeline.
> -
>
> The ASF security team siding with the ASF project maintainers over the
> researcher regarding vulnerability presence/impact when a vulnerability
> is
> clearly present.
>
>
>
> The ASF has an established Vulnerability Handling Process:
> https://www.apache.org/security/committers.html
>
> However, looking through my email today I very quickly found examples of
> cases where the ASF had not followed this process. In particular:
>
> -
>
> CVE-2020-35451: Apache Oozie - Skipped steps 12 & 13
> -
>
> CVE-2021-36151: Apache Goblin - Skipped steps 5, 6, 7, 12, & 13
>
> In this case, the vulnerability wasn’t ever even confirmed by anyone.
>
> I did receive an apology from the ASF security team regarding these issues,
> which I greatly appreciate, however they are good examples of the problems
> I’ve experienced with the ASF.
>
> Net result from my previous interaction with the ASF: I feel disrespected
> and ignored and feel like my contributions aren't valued. This has led me
> to a feeling of resentment towards the ASF and an overall feeling of
> distrust towards the established ASF vulnerability handling process.
>
> The point of this email is to explain my perspective and feelings of
> frustration I’ve experienced with the ASF. Hopefully through this dialogue
> a constructive outcome can be achieved.
>
> Respectfully,
>
> Jonathan Leitschuh
>
> Dan Kaminsky Fellowship
>
Hi Jonathan;
We certainly have appreciated your reports over the years which have led to
a number of security updates across our projects. One of the reasons we
set up this public list was so that we had a place we could work on process
and policy concerns in a public forum not just limited to the private
security teams.
You called out a few examples of concerns in issues you’ve submitted; one
of which was prior to a change to our process last year specifically so
that issues sent to security@apache.org get a confirmation now from the
security team rather than from the project team, and so ensuring that the
security team can be the point of contact if there is any break-down in
communication. We do deal with over 400 reports a year from researchers
and your experience isn’t what we strive to accomplish. We’ve already
passed on your comments to the projects mentioned. In general, where you
have concerns with the handling of any issue, the best way to raise the
concern is with a follow-up message on the email thread for that report, or
(after an issue is public) on the appropriate dev list so it reaches
everyone involved in the project development (Our projects don't all
monitor this list).
An issue that affects a wide range of projects is always tricky to
co-ordinate, where we have to bring many different project teams all with
their own management, release cycles, and communities into the embargoed
discussion, that’s pretty hard to automate and requires a lot of manual
effort, so for these types of multiple-affected-project issues we can
certainly look at ways that will mutually work for both researchers and the
ASF projects without creating exceptions or a separate process for every
case.
Regards, Mark