You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-dev@hadoop.apache.org by Konstantin Boudnik <co...@yahoo-inc.com> on 2010/05/06 03:50:53 UTC
[DISCUSSION]: Integrating SureLogic into Hadoop
Hello.
As some of you might know we have the license for great concurrency analysis
software from SureLogic.
SureLogic engineers gave some run for HDFS, MR, and Zookeeper code at the end
of last year. The tools seem very promising and supposedly bring us the
value (linked from the page below).
Please read this Wiki page to get more information about the tool and to
get some understanding how it works
http://wiki.apache.org/hadoop/HowToUseConcurrencyAnalysisTools
Here's the additional information on how to install/upgrade to the latest
release: http://surelogic.com/static/eclipse/install.html
There's also a couple of JIRAs w/ patches (HDFS-801, MAPREDUCE-1259) to
integrate annotations into our current source code base. The annotations are
represented as a jar file with some interface classes. They are redistributed
under Apache license. The retention policy of annotations are 'compile time',
i.e. this jar isn't required for the runtime of Hadoop.
It'd be great to hear community thoughts on this so we can make some decision
about this toolset.
Please comment. Thanks
Cos
Re: [DISCUSSION]: Integrating SureLogic into Hadoop
Posted by Luke <vi...@gmail.com>.
On a related note, any objections to use
http://code.google.com/p/thread-weaver/ in our unit tests?
__Luke
On Fri, May 7, 2010 at 4:35 PM, Todd Lipcon <to...@cloudera.com> wrote:
> Hi Cos,
>
> This looks great, and I'm excited to have more ways of finding these tricky
> bugs. Are there any examples of bugs found already by these techniques?
>
> The one concern I have about the proposal is with this:
>> SureLogic analysis is going to be included to the test-patch process. This
> said new patches are required not to raise SureLogic warnings level (similar
> to the requirements about FindBugs or javac).
>
> This is slightly worrisome since the SureLogic license is only available to
> committers. For non-committers, I think this may prove to be difficult since
> they won't have any local means of checking for warnings and verifying that
> they've fixed them.
>
> Thanks
> -Todd
>
>
> 2010/5/5 Konstantin Boudnik <co...@yahoo-inc.com>
>
>> Hello.
>>
>> As some of you might know we have the license for great concurrency
>> analysis
>> software from SureLogic.
>>
>> SureLogic engineers gave some run for HDFS, MR, and Zookeeper code at the
>> end
>> of last year. The tools seem very promising and supposedly bring us the
>> value (linked from the page below).
>>
>> Please read this Wiki page to get more information about the tool and to
>> get some understanding how it works
>> http://wiki.apache.org/hadoop/HowToUseConcurrencyAnalysisTools
>>
>> Here's the additional information on how to install/upgrade to the latest
>> release: http://surelogic.com/static/eclipse/install.html
>>
>> There's also a couple of JIRAs w/ patches (HDFS-801, MAPREDUCE-1259) to
>> integrate annotations into our current source code base. The annotations
>> are
>> represented as a jar file with some interface classes. They are
>> redistributed
>> under Apache license. The retention policy of annotations are 'compile
>> time',
>> i.e. this jar isn't required for the runtime of Hadoop.
>>
>> It'd be great to hear community thoughts on this so we can make some
>> decision
>> about this toolset.
>>
>> Please comment. Thanks
>> Cos
>>
>
>
>
> --
> Todd Lipcon
> Software Engineer, Cloudera
>
Re: [DISCUSSION]: Integrating SureLogic into Hadoop
Posted by Konstantin Boudnik <co...@yahoo-inc.com>.
Here's the next step then:
- I am going to update HowToUseConcurrencyAnalysisTools wiki with the
details on how contributors can get the license
- it seems we are ready to add up promisses.jar to the list of our
dependencies (patches are already available for HDFS and MR)
Shall we have a vote for this or something?
On Fri, May 14, 2010 at 01:23PM, Todd Lipcon wrote:
> I agree, thanks for looking into it, and this sounds entirely reasonable. I
> imagine a great number of the contributors are committers on some apache
> project or another (if not Hadoop itself) so we'll only need to make special
> exemptions occasionally.
>
> -Todd
>
> On Fri, May 14, 2010 at 1:19 PM, Konstantin Shvachko <sh...@yahoo-inc.com>wrote:
>
> > This is good news!
> > I found SureLogic stack useful for finding bugs.
> > It was especially helpful in detecting synchronization issues.
> > Good that the licensing issues are cleared out.
> > Thanks, Cos.
> > --Konstantin
> >
> >
> >
> > On 5/14/2010 12:59 PM, Konstantin Boudnik wrote:
> >
> >> Here's an update from SureLogic on the licensing of the software to the
> >> broader contributors community.
> >>
> >> 1) For now we should be able to use 'committers' license (the one we
> >> already have)
> >> to share with contributors on per case basis (a contributor needs to
> >> ask for it and after
> >> a consideration by a designated committer it can be send our to the
> >> requester)
> >> 2) in the future i.e. past 10/31/2010 when the current license will expire
> >> and
> >> be updated, we'll provide a POC so that our contributors can ask for a
> >> license to be given to them (similar to #1 above).
> >>
> >> Right now, SureLogic works on license auto-update feature where new
> >> license
> >> will be automatically retrieved by the tools upon the expiration of an old
> >> one. Thus #2) will be moving to that "self-renewing" license mechanism to
> >> avoid having to pass out new licenses each year.
> >>
> >> So, I think this all seems very reasonable and we can effectively close
> >> the
> >> 'license' issue.
> >>
> >> On the other hand, I got the word that CLI interfaces for their tools are
> >> in
> >> the testing phase right now, so we should be able to try them out shortly.
> >>
> >> Do we have any other concerns at this point?
> >>
> >> Cos
> >>
> >> On Mon, May 10, 2010 at 04:37PM, Konstantin Boudnik wrote:
> >>
> >>> This is very valid concern Todd. I am talking with SureLogic at the
> >>> moment to
> >>> find out if they will be willing to provide the license to all Hadoop
> >>> contributors.
> >>>
> >>> On the other hand not including such type of validation into standard
> >>> patch
> >>> validation process poses a danger of code degradation as time passes.
> >>> E.g. we
> >>> can add SureLogic validation into secondary build but then Hadoop
> >>> committers
> >>> will have to make sure that no new SureLogic's warnings were added in the
> >>> last week or so and fix them immediately or at least open a JIRA to track
> >>> such
> >>> issues. Certainly there some other ways.
> >>>
> >>> I believe we might need to make an effort here to help SureLogic to limit
> >>> the
> >>> distribution of the license by Hadoop contributors crowd. I am not sure
> >>> if this is
> >>> Ok to ask contributors to fill some kind of legal form where they agree
> >>> not to
> >>> use the copy of the license for any projects not hosted by ASF or
> >>> something?
> >>> Similar to what we fill for our the contributions to ASF? But first let's
> >>> hear
> >>> SureLogic take on the license ;)
> >>>
> >>> Cos
> >>>
> >>> On Fri, May 07, 2010 at 04:35PM, Todd Lipcon wrote:
> >>>
> >>>> Hi Cos,
> >>>>
> >>>> This looks great, and I'm excited to have more ways of finding these
> >>>> tricky
> >>>> bugs. Are there any examples of bugs found already by these techniques?
> >>>>
> >>>> The one concern I have about the proposal is with this:
> >>>>
> >>>>> SureLogic analysis is going to be included to the test-patch process.
> >>>>> This
> >>>>>
> >>>> said new patches are required not to raise SureLogic warnings level
> >>>> (similar
> >>>> to the requirements about FindBugs or javac).
> >>>>
> >>>> This is slightly worrisome since the SureLogic license is only available
> >>>> to
> >>>> committers. For non-committers, I think this may prove to be difficult
> >>>> since
> >>>> they won't have any local means of checking for warnings and verifying
> >>>> that
> >>>> they've fixed them.
> >>>>
> >>>> Thanks
> >>>> -Todd
> >>>>
> >>>>
> >>>> 2010/5/5 Konstantin Boudnik<co...@yahoo-inc.com>
> >>>>
> >>>> Hello.
> >>>>>
> >>>>> As some of you might know we have the license for great concurrency
> >>>>> analysis
> >>>>> software from SureLogic.
> >>>>>
> >>>>> SureLogic engineers gave some run for HDFS, MR, and Zookeeper code at
> >>>>> the
> >>>>> end
> >>>>> of last year. The tools seem very promising and supposedly bring us the
> >>>>> value (linked from the page below).
> >>>>>
> >>>>> Please read this Wiki page to get more information about the tool and
> >>>>> to
> >>>>> get some understanding how it works
> >>>>> http://wiki.apache.org/hadoop/HowToUseConcurrencyAnalysisTools
> >>>>>
> >>>>> Here's the additional information on how to install/upgrade to the
> >>>>> latest
> >>>>> release: http://surelogic.com/static/eclipse/install.html
> >>>>>
> >>>>> There's also a couple of JIRAs w/ patches (HDFS-801, MAPREDUCE-1259) to
> >>>>> integrate annotations into our current source code base. The
> >>>>> annotations
> >>>>> are
> >>>>> represented as a jar file with some interface classes. They are
> >>>>> redistributed
> >>>>> under Apache license. The retention policy of annotations are 'compile
> >>>>> time',
> >>>>> i.e. this jar isn't required for the runtime of Hadoop.
> >>>>>
> >>>>> It'd be great to hear community thoughts on this so we can make some
> >>>>> decision
> >>>>> about this toolset.
> >>>>>
> >>>>> Please comment. Thanks
> >>>>> Cos
> >>>>>
> >>>>>
> >>>>
> >>>>
> >>>> --
> >>>> Todd Lipcon
> >>>> Software Engineer, Cloudera
> >>>>
> >>>
> >
>
>
> --
> Todd Lipcon
> Software Engineer, Cloudera
Re: [DISCUSSION]: Integrating SureLogic into Hadoop
Posted by Todd Lipcon <to...@cloudera.com>.
I agree, thanks for looking into it, and this sounds entirely reasonable. I
imagine a great number of the contributors are committers on some apache
project or another (if not Hadoop itself) so we'll only need to make special
exemptions occasionally.
-Todd
On Fri, May 14, 2010 at 1:19 PM, Konstantin Shvachko <sh...@yahoo-inc.com>wrote:
> This is good news!
> I found SureLogic stack useful for finding bugs.
> It was especially helpful in detecting synchronization issues.
> Good that the licensing issues are cleared out.
> Thanks, Cos.
> --Konstantin
>
>
>
> On 5/14/2010 12:59 PM, Konstantin Boudnik wrote:
>
>> Here's an update from SureLogic on the licensing of the software to the
>> broader contributors community.
>>
>> 1) For now we should be able to use 'committers' license (the one we
>> already have)
>> to share with contributors on per case basis (a contributor needs to
>> ask for it and after
>> a consideration by a designated committer it can be send our to the
>> requester)
>> 2) in the future i.e. past 10/31/2010 when the current license will expire
>> and
>> be updated, we'll provide a POC so that our contributors can ask for a
>> license to be given to them (similar to #1 above).
>>
>> Right now, SureLogic works on license auto-update feature where new
>> license
>> will be automatically retrieved by the tools upon the expiration of an old
>> one. Thus #2) will be moving to that "self-renewing" license mechanism to
>> avoid having to pass out new licenses each year.
>>
>> So, I think this all seems very reasonable and we can effectively close
>> the
>> 'license' issue.
>>
>> On the other hand, I got the word that CLI interfaces for their tools are
>> in
>> the testing phase right now, so we should be able to try them out shortly.
>>
>> Do we have any other concerns at this point?
>>
>> Cos
>>
>> On Mon, May 10, 2010 at 04:37PM, Konstantin Boudnik wrote:
>>
>>> This is very valid concern Todd. I am talking with SureLogic at the
>>> moment to
>>> find out if they will be willing to provide the license to all Hadoop
>>> contributors.
>>>
>>> On the other hand not including such type of validation into standard
>>> patch
>>> validation process poses a danger of code degradation as time passes.
>>> E.g. we
>>> can add SureLogic validation into secondary build but then Hadoop
>>> committers
>>> will have to make sure that no new SureLogic's warnings were added in the
>>> last week or so and fix them immediately or at least open a JIRA to track
>>> such
>>> issues. Certainly there some other ways.
>>>
>>> I believe we might need to make an effort here to help SureLogic to limit
>>> the
>>> distribution of the license by Hadoop contributors crowd. I am not sure
>>> if this is
>>> Ok to ask contributors to fill some kind of legal form where they agree
>>> not to
>>> use the copy of the license for any projects not hosted by ASF or
>>> something?
>>> Similar to what we fill for our the contributions to ASF? But first let's
>>> hear
>>> SureLogic take on the license ;)
>>>
>>> Cos
>>>
>>> On Fri, May 07, 2010 at 04:35PM, Todd Lipcon wrote:
>>>
>>>> Hi Cos,
>>>>
>>>> This looks great, and I'm excited to have more ways of finding these
>>>> tricky
>>>> bugs. Are there any examples of bugs found already by these techniques?
>>>>
>>>> The one concern I have about the proposal is with this:
>>>>
>>>>> SureLogic analysis is going to be included to the test-patch process.
>>>>> This
>>>>>
>>>> said new patches are required not to raise SureLogic warnings level
>>>> (similar
>>>> to the requirements about FindBugs or javac).
>>>>
>>>> This is slightly worrisome since the SureLogic license is only available
>>>> to
>>>> committers. For non-committers, I think this may prove to be difficult
>>>> since
>>>> they won't have any local means of checking for warnings and verifying
>>>> that
>>>> they've fixed them.
>>>>
>>>> Thanks
>>>> -Todd
>>>>
>>>>
>>>> 2010/5/5 Konstantin Boudnik<co...@yahoo-inc.com>
>>>>
>>>> Hello.
>>>>>
>>>>> As some of you might know we have the license for great concurrency
>>>>> analysis
>>>>> software from SureLogic.
>>>>>
>>>>> SureLogic engineers gave some run for HDFS, MR, and Zookeeper code at
>>>>> the
>>>>> end
>>>>> of last year. The tools seem very promising and supposedly bring us the
>>>>> value (linked from the page below).
>>>>>
>>>>> Please read this Wiki page to get more information about the tool and
>>>>> to
>>>>> get some understanding how it works
>>>>> http://wiki.apache.org/hadoop/HowToUseConcurrencyAnalysisTools
>>>>>
>>>>> Here's the additional information on how to install/upgrade to the
>>>>> latest
>>>>> release: http://surelogic.com/static/eclipse/install.html
>>>>>
>>>>> There's also a couple of JIRAs w/ patches (HDFS-801, MAPREDUCE-1259) to
>>>>> integrate annotations into our current source code base. The
>>>>> annotations
>>>>> are
>>>>> represented as a jar file with some interface classes. They are
>>>>> redistributed
>>>>> under Apache license. The retention policy of annotations are 'compile
>>>>> time',
>>>>> i.e. this jar isn't required for the runtime of Hadoop.
>>>>>
>>>>> It'd be great to hear community thoughts on this so we can make some
>>>>> decision
>>>>> about this toolset.
>>>>>
>>>>> Please comment. Thanks
>>>>> Cos
>>>>>
>>>>>
>>>>
>>>>
>>>> --
>>>> Todd Lipcon
>>>> Software Engineer, Cloudera
>>>>
>>>
>
--
Todd Lipcon
Software Engineer, Cloudera
Re: [DISCUSSION]: Integrating SureLogic into Hadoop
Posted by Konstantin Shvachko <sh...@yahoo-inc.com>.
This is good news!
I found SureLogic stack useful for finding bugs.
It was especially helpful in detecting synchronization issues.
Good that the licensing issues are cleared out.
Thanks, Cos.
--Konstantin
On 5/14/2010 12:59 PM, Konstantin Boudnik wrote:
> Here's an update from SureLogic on the licensing of the software to the
> broader contributors community.
>
> 1) For now we should be able to use 'committers' license (the one we already have)
> to share with contributors on per case basis (a contributor needs to ask for it and after
> a consideration by a designated committer it can be send our to the
> requester)
> 2) in the future i.e. past 10/31/2010 when the current license will expire and
> be updated, we'll provide a POC so that our contributors can ask for a
> license to be given to them (similar to #1 above).
>
> Right now, SureLogic works on license auto-update feature where new license
> will be automatically retrieved by the tools upon the expiration of an old
> one. Thus #2) will be moving to that "self-renewing" license mechanism to
> avoid having to pass out new licenses each year.
>
> So, I think this all seems very reasonable and we can effectively close the
> 'license' issue.
>
> On the other hand, I got the word that CLI interfaces for their tools are in
> the testing phase right now, so we should be able to try them out shortly.
>
> Do we have any other concerns at this point?
>
> Cos
>
> On Mon, May 10, 2010 at 04:37PM, Konstantin Boudnik wrote:
>> This is very valid concern Todd. I am talking with SureLogic at the moment to
>> find out if they will be willing to provide the license to all Hadoop
>> contributors.
>>
>> On the other hand not including such type of validation into standard patch
>> validation process poses a danger of code degradation as time passes. E.g. we
>> can add SureLogic validation into secondary build but then Hadoop committers
>> will have to make sure that no new SureLogic's warnings were added in the
>> last week or so and fix them immediately or at least open a JIRA to track such
>> issues. Certainly there some other ways.
>>
>> I believe we might need to make an effort here to help SureLogic to limit the
>> distribution of the license by Hadoop contributors crowd. I am not sure if this is
>> Ok to ask contributors to fill some kind of legal form where they agree not to
>> use the copy of the license for any projects not hosted by ASF or something?
>> Similar to what we fill for our the contributions to ASF? But first let's hear
>> SureLogic take on the license ;)
>>
>> Cos
>>
>> On Fri, May 07, 2010 at 04:35PM, Todd Lipcon wrote:
>>> Hi Cos,
>>>
>>> This looks great, and I'm excited to have more ways of finding these tricky
>>> bugs. Are there any examples of bugs found already by these techniques?
>>>
>>> The one concern I have about the proposal is with this:
>>>> SureLogic analysis is going to be included to the test-patch process. This
>>> said new patches are required not to raise SureLogic warnings level (similar
>>> to the requirements about FindBugs or javac).
>>>
>>> This is slightly worrisome since the SureLogic license is only available to
>>> committers. For non-committers, I think this may prove to be difficult since
>>> they won't have any local means of checking for warnings and verifying that
>>> they've fixed them.
>>>
>>> Thanks
>>> -Todd
>>>
>>>
>>> 2010/5/5 Konstantin Boudnik<co...@yahoo-inc.com>
>>>
>>>> Hello.
>>>>
>>>> As some of you might know we have the license for great concurrency
>>>> analysis
>>>> software from SureLogic.
>>>>
>>>> SureLogic engineers gave some run for HDFS, MR, and Zookeeper code at the
>>>> end
>>>> of last year. The tools seem very promising and supposedly bring us the
>>>> value (linked from the page below).
>>>>
>>>> Please read this Wiki page to get more information about the tool and to
>>>> get some understanding how it works
>>>> http://wiki.apache.org/hadoop/HowToUseConcurrencyAnalysisTools
>>>>
>>>> Here's the additional information on how to install/upgrade to the latest
>>>> release: http://surelogic.com/static/eclipse/install.html
>>>>
>>>> There's also a couple of JIRAs w/ patches (HDFS-801, MAPREDUCE-1259) to
>>>> integrate annotations into our current source code base. The annotations
>>>> are
>>>> represented as a jar file with some interface classes. They are
>>>> redistributed
>>>> under Apache license. The retention policy of annotations are 'compile
>>>> time',
>>>> i.e. this jar isn't required for the runtime of Hadoop.
>>>>
>>>> It'd be great to hear community thoughts on this so we can make some
>>>> decision
>>>> about this toolset.
>>>>
>>>> Please comment. Thanks
>>>> Cos
>>>>
>>>
>>>
>>>
>>> --
>>> Todd Lipcon
>>> Software Engineer, Cloudera
Re: [DISCUSSION]: Integrating SureLogic into Hadoop
Posted by Konstantin Boudnik <co...@yahoo-inc.com>.
Here's an update from SureLogic on the licensing of the software to the
broader contributors community.
1) For now we should be able to use 'committers' license (the one we already have)
to share with contributors on per case basis (a contributor needs to ask for it and after
a consideration by a designated committer it can be send our to the
requester)
2) in the future i.e. past 10/31/2010 when the current license will expire and
be updated, we'll provide a POC so that our contributors can ask for a
license to be given to them (similar to #1 above).
Right now, SureLogic works on license auto-update feature where new license
will be automatically retrieved by the tools upon the expiration of an old
one. Thus #2) will be moving to that "self-renewing" license mechanism to
avoid having to pass out new licenses each year.
So, I think this all seems very reasonable and we can effectively close the
'license' issue.
On the other hand, I got the word that CLI interfaces for their tools are in
the testing phase right now, so we should be able to try them out shortly.
Do we have any other concerns at this point?
Cos
On Mon, May 10, 2010 at 04:37PM, Konstantin Boudnik wrote:
> This is very valid concern Todd. I am talking with SureLogic at the moment to
> find out if they will be willing to provide the license to all Hadoop
> contributors.
>
> On the other hand not including such type of validation into standard patch
> validation process poses a danger of code degradation as time passes. E.g. we
> can add SureLogic validation into secondary build but then Hadoop committers
> will have to make sure that no new SureLogic's warnings were added in the
> last week or so and fix them immediately or at least open a JIRA to track such
> issues. Certainly there some other ways.
>
> I believe we might need to make an effort here to help SureLogic to limit the
> distribution of the license by Hadoop contributors crowd. I am not sure if this is
> Ok to ask contributors to fill some kind of legal form where they agree not to
> use the copy of the license for any projects not hosted by ASF or something?
> Similar to what we fill for our the contributions to ASF? But first let's hear
> SureLogic take on the license ;)
>
> Cos
>
> On Fri, May 07, 2010 at 04:35PM, Todd Lipcon wrote:
> > Hi Cos,
> >
> > This looks great, and I'm excited to have more ways of finding these tricky
> > bugs. Are there any examples of bugs found already by these techniques?
> >
> > The one concern I have about the proposal is with this:
> > > SureLogic analysis is going to be included to the test-patch process. This
> > said new patches are required not to raise SureLogic warnings level (similar
> > to the requirements about FindBugs or javac).
> >
> > This is slightly worrisome since the SureLogic license is only available to
> > committers. For non-committers, I think this may prove to be difficult since
> > they won't have any local means of checking for warnings and verifying that
> > they've fixed them.
> >
> > Thanks
> > -Todd
> >
> >
> > 2010/5/5 Konstantin Boudnik <co...@yahoo-inc.com>
> >
> > > Hello.
> > >
> > > As some of you might know we have the license for great concurrency
> > > analysis
> > > software from SureLogic.
> > >
> > > SureLogic engineers gave some run for HDFS, MR, and Zookeeper code at the
> > > end
> > > of last year. The tools seem very promising and supposedly bring us the
> > > value (linked from the page below).
> > >
> > > Please read this Wiki page to get more information about the tool and to
> > > get some understanding how it works
> > > http://wiki.apache.org/hadoop/HowToUseConcurrencyAnalysisTools
> > >
> > > Here's the additional information on how to install/upgrade to the latest
> > > release: http://surelogic.com/static/eclipse/install.html
> > >
> > > There's also a couple of JIRAs w/ patches (HDFS-801, MAPREDUCE-1259) to
> > > integrate annotations into our current source code base. The annotations
> > > are
> > > represented as a jar file with some interface classes. They are
> > > redistributed
> > > under Apache license. The retention policy of annotations are 'compile
> > > time',
> > > i.e. this jar isn't required for the runtime of Hadoop.
> > >
> > > It'd be great to hear community thoughts on this so we can make some
> > > decision
> > > about this toolset.
> > >
> > > Please comment. Thanks
> > > Cos
> > >
> >
> >
> >
> > --
> > Todd Lipcon
> > Software Engineer, Cloudera
Re: [DISCUSSION]: Integrating SureLogic into Hadoop
Posted by Todd Lipcon <to...@cloudera.com>.
Great, thanks for checking with SureLogic, Cos! I absolutely agree
that hudson QAbot tests are critical for keeping the number of
warnings low (or 0!)
-Todd
2010/5/10 Konstantin Boudnik <co...@yahoo-inc.com>:
> This is very valid concern Todd. I am talking with SureLogic at the moment to
> find out if they will be willing to provide the license to all Hadoop
> contributors.
>
> On the other hand not including such type of validation into standard patch
> validation process poses a danger of code degradation as time passes. E.g. we
> can add SureLogic validation into secondary build but then Hadoop committers
> will have to make sure that no new SureLogic's warnings were added in the
> last week or so and fix them immediately or at least open a JIRA to track such
> issues. Certainly there some other ways.
>
> I believe we might need to make an effort here to help SureLogic to limit the
> distribution of the license by Hadoop contributors crowd. I am not sure if this is
> Ok to ask contributors to fill some kind of legal form where they agree not to
> use the copy of the license for any projects not hosted by ASF or something?
> Similar to what we fill for our the contributions to ASF? But first let's hear
> SureLogic take on the license ;)
>
> Cos
>
> On Fri, May 07, 2010 at 04:35PM, Todd Lipcon wrote:
>> Hi Cos,
>>
>> This looks great, and I'm excited to have more ways of finding these tricky
>> bugs. Are there any examples of bugs found already by these techniques?
>>
>> The one concern I have about the proposal is with this:
>> > SureLogic analysis is going to be included to the test-patch process. This
>> said new patches are required not to raise SureLogic warnings level (similar
>> to the requirements about FindBugs or javac).
>>
>> This is slightly worrisome since the SureLogic license is only available to
>> committers. For non-committers, I think this may prove to be difficult since
>> they won't have any local means of checking for warnings and verifying that
>> they've fixed them.
>>
>> Thanks
>> -Todd
>>
>>
>> 2010/5/5 Konstantin Boudnik <co...@yahoo-inc.com>
>>
>> > Hello.
>> >
>> > As some of you might know we have the license for great concurrency
>> > analysis
>> > software from SureLogic.
>> >
>> > SureLogic engineers gave some run for HDFS, MR, and Zookeeper code at the
>> > end
>> > of last year. The tools seem very promising and supposedly bring us the
>> > value (linked from the page below).
>> >
>> > Please read this Wiki page to get more information about the tool and to
>> > get some understanding how it works
>> > http://wiki.apache.org/hadoop/HowToUseConcurrencyAnalysisTools
>> >
>> > Here's the additional information on how to install/upgrade to the latest
>> > release: http://surelogic.com/static/eclipse/install.html
>> >
>> > There's also a couple of JIRAs w/ patches (HDFS-801, MAPREDUCE-1259) to
>> > integrate annotations into our current source code base. The annotations
>> > are
>> > represented as a jar file with some interface classes. They are
>> > redistributed
>> > under Apache license. The retention policy of annotations are 'compile
>> > time',
>> > i.e. this jar isn't required for the runtime of Hadoop.
>> >
>> > It'd be great to hear community thoughts on this so we can make some
>> > decision
>> > about this toolset.
>> >
>> > Please comment. Thanks
>> > Cos
>> >
>>
>>
>>
>> --
>> Todd Lipcon
>> Software Engineer, Cloudera
>
--
Todd Lipcon
Software Engineer, Cloudera
Re: [DISCUSSION]: Integrating SureLogic into Hadoop
Posted by Konstantin Boudnik <co...@yahoo-inc.com>.
This is very valid concern Todd. I am talking with SureLogic at the moment to
find out if they will be willing to provide the license to all Hadoop
contributors.
On the other hand not including such type of validation into standard patch
validation process poses a danger of code degradation as time passes. E.g. we
can add SureLogic validation into secondary build but then Hadoop committers
will have to make sure that no new SureLogic's warnings were added in the
last week or so and fix them immediately or at least open a JIRA to track such
issues. Certainly there some other ways.
I believe we might need to make an effort here to help SureLogic to limit the
distribution of the license by Hadoop contributors crowd. I am not sure if this is
Ok to ask contributors to fill some kind of legal form where they agree not to
use the copy of the license for any projects not hosted by ASF or something?
Similar to what we fill for our the contributions to ASF? But first let's hear
SureLogic take on the license ;)
Cos
On Fri, May 07, 2010 at 04:35PM, Todd Lipcon wrote:
> Hi Cos,
>
> This looks great, and I'm excited to have more ways of finding these tricky
> bugs. Are there any examples of bugs found already by these techniques?
>
> The one concern I have about the proposal is with this:
> > SureLogic analysis is going to be included to the test-patch process. This
> said new patches are required not to raise SureLogic warnings level (similar
> to the requirements about FindBugs or javac).
>
> This is slightly worrisome since the SureLogic license is only available to
> committers. For non-committers, I think this may prove to be difficult since
> they won't have any local means of checking for warnings and verifying that
> they've fixed them.
>
> Thanks
> -Todd
>
>
> 2010/5/5 Konstantin Boudnik <co...@yahoo-inc.com>
>
> > Hello.
> >
> > As some of you might know we have the license for great concurrency
> > analysis
> > software from SureLogic.
> >
> > SureLogic engineers gave some run for HDFS, MR, and Zookeeper code at the
> > end
> > of last year. The tools seem very promising and supposedly bring us the
> > value (linked from the page below).
> >
> > Please read this Wiki page to get more information about the tool and to
> > get some understanding how it works
> > http://wiki.apache.org/hadoop/HowToUseConcurrencyAnalysisTools
> >
> > Here's the additional information on how to install/upgrade to the latest
> > release: http://surelogic.com/static/eclipse/install.html
> >
> > There's also a couple of JIRAs w/ patches (HDFS-801, MAPREDUCE-1259) to
> > integrate annotations into our current source code base. The annotations
> > are
> > represented as a jar file with some interface classes. They are
> > redistributed
> > under Apache license. The retention policy of annotations are 'compile
> > time',
> > i.e. this jar isn't required for the runtime of Hadoop.
> >
> > It'd be great to hear community thoughts on this so we can make some
> > decision
> > about this toolset.
> >
> > Please comment. Thanks
> > Cos
> >
>
>
>
> --
> Todd Lipcon
> Software Engineer, Cloudera
Re: [DISCUSSION]: Integrating SureLogic into Hadoop
Posted by Todd Lipcon <to...@cloudera.com>.
Hi Cos,
This looks great, and I'm excited to have more ways of finding these tricky
bugs. Are there any examples of bugs found already by these techniques?
The one concern I have about the proposal is with this:
> SureLogic analysis is going to be included to the test-patch process. This
said new patches are required not to raise SureLogic warnings level (similar
to the requirements about FindBugs or javac).
This is slightly worrisome since the SureLogic license is only available to
committers. For non-committers, I think this may prove to be difficult since
they won't have any local means of checking for warnings and verifying that
they've fixed them.
Thanks
-Todd
2010/5/5 Konstantin Boudnik <co...@yahoo-inc.com>
> Hello.
>
> As some of you might know we have the license for great concurrency
> analysis
> software from SureLogic.
>
> SureLogic engineers gave some run for HDFS, MR, and Zookeeper code at the
> end
> of last year. The tools seem very promising and supposedly bring us the
> value (linked from the page below).
>
> Please read this Wiki page to get more information about the tool and to
> get some understanding how it works
> http://wiki.apache.org/hadoop/HowToUseConcurrencyAnalysisTools
>
> Here's the additional information on how to install/upgrade to the latest
> release: http://surelogic.com/static/eclipse/install.html
>
> There's also a couple of JIRAs w/ patches (HDFS-801, MAPREDUCE-1259) to
> integrate annotations into our current source code base. The annotations
> are
> represented as a jar file with some interface classes. They are
> redistributed
> under Apache license. The retention policy of annotations are 'compile
> time',
> i.e. this jar isn't required for the runtime of Hadoop.
>
> It'd be great to hear community thoughts on this so we can make some
> decision
> about this toolset.
>
> Please comment. Thanks
> Cos
>
--
Todd Lipcon
Software Engineer, Cloudera