You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-dev@hadoop.apache.org by Jon Jarboe <jj...@coverity.com> on 2013/08/26 17:21:52 UTC
FW: Coverity Scan (MAPREDUCE-5032)
I've been working with DataStax on their use of Coverity with Cassandra, and decided to give the Hadoop 1.2.1 source tarball a run through our analyzer. I found some interesting issues, and noticed that some of them are integer overflow defects that align with the open MAPREDUCE-5032 issue. Other issues range from concurrency problems to cross-site scripting to resource leaks, but I haven't tried to match those up to existing JIRA issues.
Email is not the best forum for investigating these issues, so I'd be happy to post them on Coverity's Scan server for your review. If you're not familiar with Coverity Scan, it is our free cloud-based service for OSS projects (https://scan.coverity.com). I realize that false positives can be a concern, and I'd like to point out that Coverity is specifically designed to minimize false positives.
If somebody is interested in looking through the results, please let me know. To get an initial analysis into Scan, please let me know whether the 1.2.1 source is a good place to start. I can analyze a different rev/branch if that's more interesting. If you see value, we can always set up additional branches.
Best regards, and thanks for your time.
Jon Jarboe | Senior Technical Manager
Coverity | 185 Berry Street | Suite 6500, Lobby 3 | San Francisco, CA 94107
O: +1 214-531-3496 | M: +1 214-531-3496 | E: jjarboe@coverity.com<ma...@coverity.com>
Web: www.coverity.com<http://www.coverity.com> | Twitter: @Coverity
The Leader in Development Testing
Re: FW: Coverity Scan (MAPREDUCE-5032)
Posted by Roman Shaposhnik <rv...@apache.org>.
On Mon, Aug 26, 2013 at 8:21 AM, Jon Jarboe <jj...@coverity.com> wrote:
> I've been working with DataStax on their use of Coverity with Cassandra, and decided to give the Hadoop 1.2.1 source tarball a run through our analyzer. I found some interesting issues, and noticed that some of them are integer overflow defects that align with the open MAPREDUCE-5032 issue. Other issues range from concurrency problems to cross-site scripting to resource leaks, but I haven't tried to match those up to existing JIRA issues.
Any chance you can run the same analysis on a recently
released 2.1.0-beta?
I think it would be a huge help in finding out issue there
early before 2.1.[12]-beta get released.
Let me know if I can help with the analysis somehow.
Thanks,
Roman.
RE: Coverity Scan (MAPREDUCE-5032)
Posted by Jon Jarboe <jj...@coverity.com>.
Thanks, Roman, that's definitely still true. The web interface provides all sorts of cross-referencing, code browsing, defect history, and other capabilities that are lost in a simple report.
For what it's worth, there are multiple unrelated issues so it probably doesn't make sense to throw them into a single JIRA anyway.
Jon
(214) 531-3496
> -----Original Message-----
> From: shaposhnik@gmail.com [mailto:shaposhnik@gmail.com] On Behalf Of
> Roman Shaposhnik
> Sent: Monday, August 26, 2013 12:50 PM
> To: common-dev@hadoop.apache.org
> Subject: Re: Coverity Scan (MAPREDUCE-5032)
>
> On Mon, Aug 26, 2013 at 10:43 AM, Vinod Kumar Vavilapalli
> <vi...@apache.org> wrote:
> >
> > Can you file a JIRA and attach the report there? That is the best way to
> move this forward.
>
> Last time I was involved in a Coverity scan was when they scanned another
> project I'm committer on (FFmpeg). The lesson there was that the value you
> get out of browsing on their site https://scan.coverity.com is immeasurably
> higher than from any static report that can be attached to a JIRA.
>
> Also, at least in FFmpeg's case, Coverity identified a few things that could've
> been used as potential exploits so it made perfect sense to have a white-list
> of project members who could get access to the initial report instead of going
> all public with it to begin with (which would happen if it just gets attached to
> a JIRA in its entirety).
>
> Just my 2c worth of working with them in the past.
>
> Thanks,
> Roman.
Re: Coverity Scan (MAPREDUCE-5032)
Posted by Arun C Murthy <ac...@hortonworks.com>.
Agree with Aaron. Let's move this discussion to security@. Thanks.
On Sep 30, 2013, at 5:57 PM, Aaron T. Myers <at...@apache.org> wrote:
> I strongly recommend that we take this conversation over to the
> (committers-only) security@hadoop.apache.org mailing list. In general we
> try to follow the Apache recommendations when it comes to addressing
> security issues, which involves not publicly disclosing the vulnerability
> until there are released version(s) with the issue(s) addressed.
>
> Best,
> Aaron
>
>
> On Mon, Aug 26, 2013 at 8:24 PM, Jon Jarboe <jj...@coverity.com> wrote:
>
>> Thanks for the interest. I'm in the process of building the 2.1.0 beta as
>> suggested by Roman.
>>
>> Jon
>> (214) 531-3496
>>
>>
>>> -----Original Message-----
>>> From: Ottenheimer, Davi [mailto:Davi.Ottenheimer@emc.com]
>>> Sent: Monday, August 26, 2013 1:11 PM
>>> To: common-dev@hadoop.apache.org
>>> Subject: RE: Coverity Scan (MAPREDUCE-5032)
>>>
>>> Perhaps open the JIRA with only a reference/link to the Coverity report,
>> and
>>> limit access to only those working on the issues.
>>>
>>> Full disclosure, update the JIRA, after fix.
>>>
>>> --
>>> Davi Ottenheimer
>>> Senior Director of Trust
>>> EMC Corporation
>>> davi.ottenheimer@emc.com | @daviottenheimer | +1-415-271-6259
>>> blog: http://www.flyingpenguin.com/
>>>
>>>
>>>> -----Original Message-----
>>>> From: shaposhnik@gmail.com [mailto:shaposhnik@gmail.com] On Behalf
>>> Of
>>>> Roman Shaposhnik
>>>> Sent: Monday, August 26, 2013 10:50 AM
>>>> To: common-dev@hadoop.apache.org
>>>> Subject: Re: Coverity Scan (MAPREDUCE-5032)
>>>>
>>>> On Mon, Aug 26, 2013 at 10:43 AM, Vinod Kumar Vavilapalli
>>>> <vi...@apache.org> wrote:
>>>>>
>>>>> Can you file a JIRA and attach the report there? That is the best
>>>>> way to
>>>> move this forward.
>>>>
>>>> Last time I was involved in a Coverity scan was when they scanned
>>>> another project I'm committer on (FFmpeg). The lesson there was that
>>>> the value you get out of browsing on their site
>>>> https://scan.coverity.com is immeasurably higher than from any static
>>> report that can be attached to a JIRA.
>>>>
>>>> Also, at least in FFmpeg's case, Coverity identified a few things that
>>>> could've been used as potential exploits so it made perfect sense to
>>>> have a white-list of project members who could get access to the
>>>> initial report instead of going all public with it to begin with
>>>> (which would happen if it just gets attached to a JIRA in its
>> entirety).
>>>>
>>>> Just my 2c worth of working with them in the past.
>>>>
>>>> Thanks,
>>>> Roman.
>>>
>>
>>
>>
--
Arun C. Murthy
Hortonworks Inc.
http://hortonworks.com/
--
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity to
which it is addressed and may contain information that is confidential,
privileged and exempt from disclosure under applicable law. If the reader
of this message is not the intended recipient, you are hereby notified that
any printing, copying, dissemination, distribution, disclosure or
forwarding of this communication is strictly prohibited. If you have
received this communication in error, please contact the sender immediately
and delete it from your system. Thank You.
Re: Coverity Scan (MAPREDUCE-5032)
Posted by "Aaron T. Myers" <at...@apache.org>.
I strongly recommend that we take this conversation over to the
(committers-only) security@hadoop.apache.org mailing list. In general we
try to follow the Apache recommendations when it comes to addressing
security issues, which involves not publicly disclosing the vulnerability
until there are released version(s) with the issue(s) addressed.
Best,
Aaron
On Mon, Aug 26, 2013 at 8:24 PM, Jon Jarboe <jj...@coverity.com> wrote:
> Thanks for the interest. I'm in the process of building the 2.1.0 beta as
> suggested by Roman.
>
> Jon
> (214) 531-3496
>
>
> > -----Original Message-----
> > From: Ottenheimer, Davi [mailto:Davi.Ottenheimer@emc.com]
> > Sent: Monday, August 26, 2013 1:11 PM
> > To: common-dev@hadoop.apache.org
> > Subject: RE: Coverity Scan (MAPREDUCE-5032)
> >
> > Perhaps open the JIRA with only a reference/link to the Coverity report,
> and
> > limit access to only those working on the issues.
> >
> > Full disclosure, update the JIRA, after fix.
> >
> > --
> > Davi Ottenheimer
> > Senior Director of Trust
> > EMC Corporation
> > davi.ottenheimer@emc.com | @daviottenheimer | +1-415-271-6259
> > blog: http://www.flyingpenguin.com/
> >
> >
> > > -----Original Message-----
> > > From: shaposhnik@gmail.com [mailto:shaposhnik@gmail.com] On Behalf
> > Of
> > > Roman Shaposhnik
> > > Sent: Monday, August 26, 2013 10:50 AM
> > > To: common-dev@hadoop.apache.org
> > > Subject: Re: Coverity Scan (MAPREDUCE-5032)
> > >
> > > On Mon, Aug 26, 2013 at 10:43 AM, Vinod Kumar Vavilapalli
> > > <vi...@apache.org> wrote:
> > > >
> > > > Can you file a JIRA and attach the report there? That is the best
> > > > way to
> > > move this forward.
> > >
> > > Last time I was involved in a Coverity scan was when they scanned
> > > another project I'm committer on (FFmpeg). The lesson there was that
> > > the value you get out of browsing on their site
> > > https://scan.coverity.com is immeasurably higher than from any static
> > report that can be attached to a JIRA.
> > >
> > > Also, at least in FFmpeg's case, Coverity identified a few things that
> > > could've been used as potential exploits so it made perfect sense to
> > > have a white-list of project members who could get access to the
> > > initial report instead of going all public with it to begin with
> > > (which would happen if it just gets attached to a JIRA in its
> entirety).
> > >
> > > Just my 2c worth of working with them in the past.
> > >
> > > Thanks,
> > > Roman.
> >
>
>
>
RE: Coverity Scan (MAPREDUCE-5032)
Posted by Jon Jarboe <jj...@coverity.com>.
Thanks for the interest. I'm in the process of building the 2.1.0 beta as suggested by Roman.
Jon
(214) 531-3496
> -----Original Message-----
> From: Ottenheimer, Davi [mailto:Davi.Ottenheimer@emc.com]
> Sent: Monday, August 26, 2013 1:11 PM
> To: common-dev@hadoop.apache.org
> Subject: RE: Coverity Scan (MAPREDUCE-5032)
>
> Perhaps open the JIRA with only a reference/link to the Coverity report, and
> limit access to only those working on the issues.
>
> Full disclosure, update the JIRA, after fix.
>
> --
> Davi Ottenheimer
> Senior Director of Trust
> EMC Corporation
> davi.ottenheimer@emc.com | @daviottenheimer | +1-415-271-6259
> blog: http://www.flyingpenguin.com/
>
>
> > -----Original Message-----
> > From: shaposhnik@gmail.com [mailto:shaposhnik@gmail.com] On Behalf
> Of
> > Roman Shaposhnik
> > Sent: Monday, August 26, 2013 10:50 AM
> > To: common-dev@hadoop.apache.org
> > Subject: Re: Coverity Scan (MAPREDUCE-5032)
> >
> > On Mon, Aug 26, 2013 at 10:43 AM, Vinod Kumar Vavilapalli
> > <vi...@apache.org> wrote:
> > >
> > > Can you file a JIRA and attach the report there? That is the best
> > > way to
> > move this forward.
> >
> > Last time I was involved in a Coverity scan was when they scanned
> > another project I'm committer on (FFmpeg). The lesson there was that
> > the value you get out of browsing on their site
> > https://scan.coverity.com is immeasurably higher than from any static
> report that can be attached to a JIRA.
> >
> > Also, at least in FFmpeg's case, Coverity identified a few things that
> > could've been used as potential exploits so it made perfect sense to
> > have a white-list of project members who could get access to the
> > initial report instead of going all public with it to begin with
> > (which would happen if it just gets attached to a JIRA in its entirety).
> >
> > Just my 2c worth of working with them in the past.
> >
> > Thanks,
> > Roman.
>
RE: Coverity Scan (MAPREDUCE-5032)
Posted by "Ottenheimer, Davi" <Da...@emc.com>.
Perhaps open the JIRA with only a reference/link to the Coverity report, and limit access to only those working on the issues.
Full disclosure, update the JIRA, after fix.
--
Davi Ottenheimer
Senior Director of Trust
EMC Corporation
davi.ottenheimer@emc.com | @daviottenheimer | +1-415-271-6259
blog: http://www.flyingpenguin.com/
> -----Original Message-----
> From: shaposhnik@gmail.com [mailto:shaposhnik@gmail.com] On Behalf Of
> Roman Shaposhnik
> Sent: Monday, August 26, 2013 10:50 AM
> To: common-dev@hadoop.apache.org
> Subject: Re: Coverity Scan (MAPREDUCE-5032)
>
> On Mon, Aug 26, 2013 at 10:43 AM, Vinod Kumar Vavilapalli
> <vi...@apache.org> wrote:
> >
> > Can you file a JIRA and attach the report there? That is the best way to
> move this forward.
>
> Last time I was involved in a Coverity scan was when they scanned another
> project I'm committer on (FFmpeg). The lesson there was that the value you
> get out of browsing on their site https://scan.coverity.com is immeasurably
> higher than from any static report that can be attached to a JIRA.
>
> Also, at least in FFmpeg's case, Coverity identified a few things that could've
> been used as potential exploits so it made perfect sense to have a white-list
> of project members who could get access to the initial report instead of going
> all public with it to begin with (which would happen if it just gets attached to
> a JIRA in its entirety).
>
> Just my 2c worth of working with them in the past.
>
> Thanks,
> Roman.
Re: Coverity Scan (MAPREDUCE-5032)
Posted by Roman Shaposhnik <rv...@apache.org>.
On Mon, Aug 26, 2013 at 10:43 AM, Vinod Kumar Vavilapalli
<vi...@apache.org> wrote:
>
> Can you file a JIRA and attach the report there? That is the best way to move this forward.
Last time I was involved in a Coverity scan was when they scanned another
project I'm committer on (FFmpeg). The lesson there was that the value
you get out of browsing on their site https://scan.coverity.com is immeasurably
higher than from any static report that can be attached to a JIRA.
Also, at least in FFmpeg's case, Coverity identified a few things that
could've been used as potential exploits so it made perfect sense
to have a white-list of project members who could get access to
the initial report instead of going all public with it to begin with (which
would happen if it just gets attached to a JIRA in its entirety).
Just my 2c worth of working with them in the past.
Thanks,
Roman.
Re: Coverity Scan (MAPREDUCE-5032)
Posted by Vinod Kumar Vavilapalli <vi...@apache.org>.
Can you file a JIRA and attach the report there? That is the best way to move this forward.
Thanks,
+Vinod Kumar Vavilapalli
On Aug 26, 2013, at 8:21 AM, Jon Jarboe wrote:
> I've been working with DataStax on their use of Coverity with Cassandra, and decided to give the Hadoop 1.2.1 source tarball a run through our analyzer. I found some interesting issues, and noticed that some of them are integer overflow defects that align with the open MAPREDUCE-5032 issue. Other issues range from concurrency problems to cross-site scripting to resource leaks, but I haven't tried to match those up to existing JIRA issues.
>
> Email is not the best forum for investigating these issues, so I'd be happy to post them on Coverity's Scan server for your review. If you're not familiar with Coverity Scan, it is our free cloud-based service for OSS projects (https://scan.coverity.com). I realize that false positives can be a concern, and I'd like to point out that Coverity is specifically designed to minimize false positives.
>
> If somebody is interested in looking through the results, please let me know. To get an initial analysis into Scan, please let me know whether the 1.2.1 source is a good place to start. I can analyze a different rev/branch if that's more interesting. If you see value, we can always set up additional branches.
>
> Best regards, and thanks for your time.
>
> Jon Jarboe | Senior Technical Manager
> Coverity | 185 Berry Street | Suite 6500, Lobby 3 | San Francisco, CA 94107
> O: +1 214-531-3496 | M: +1 214-531-3496 | E: jjarboe@coverity.com<ma...@coverity.com>
> Web: www.coverity.com<http://www.coverity.com> | Twitter: @Coverity
>
> The Leader in Development Testing
>
--
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity to
which it is addressed and may contain information that is confidential,
privileged and exempt from disclosure under applicable law. If the reader
of this message is not the intended recipient, you are hereby notified that
any printing, copying, dissemination, distribution, disclosure or
forwarding of this communication is strictly prohibited. If you have
received this communication in error, please contact the sender immediately
and delete it from your system. Thank You.
Fwd: FW: Coverity Scan (MAPREDUCE-5032)
Posted by Ted Yu <yu...@gmail.com>.
FYI
---------- Forwarded message ----------
From: Jon Jarboe <jj...@coverity.com>
Date: Mon, Aug 26, 2013 at 8:21 AM
Subject: FW: Coverity Scan (MAPREDUCE-5032)
To: "common-dev@hadoop.apache.org" <co...@hadoop.apache.org>
I've been working with DataStax on their use of Coverity with Cassandra,
and decided to give the Hadoop 1.2.1 source tarball a run through our
analyzer. I found some interesting issues, and noticed that some of them
are integer overflow defects that align with the open MAPREDUCE-5032 issue.
Other issues range from concurrency problems to cross-site scripting to
resource leaks, but I haven't tried to match those up to existing JIRA
issues.
Email is not the best forum for investigating these issues, so I'd be happy
to post them on Coverity's Scan server for your review. If you're not
familiar with Coverity Scan, it is our free cloud-based service for OSS
projects (https://scan.coverity.com). I realize that false positives can
be a concern, and I'd like to point out that Coverity is specifically
designed to minimize false positives.
If somebody is interested in looking through the results, please let me
know. To get an initial analysis into Scan, please let me know whether the
1.2.1 source is a good place to start. I can analyze a different
rev/branch if that's more interesting. If you see value, we can always set
up additional branches.
Best regards, and thanks for your time.
Jon Jarboe | Senior Technical Manager
Coverity | 185 Berry Street | Suite 6500, Lobby 3 | San Francisco, CA 94107
O: +1 214-531-3496 | M: +1 214-531-3496 | E: jjarboe@coverity.com<mailto:
jjarboe@coverity.com>
Web: www.coverity.com<http://www.coverity.com> | Twitter: @Coverity
The Leader in Development Testing